A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(render): Voxel GI lights from a world-space cache of what the camera has seen, run as compute passes
yhh54ac0a5af034
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 3b2737e42..2c79cedc1 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -17,6 +17,7 @@ published separately; it ships inside the editor.### Added- **Contact shadows and screen-space bounce light.** Contact Shadows and Screen Space GI effects add small sharp shadows at contacts and one bounce of colour between surfaces, and the weather softens the sun's shadows. `contactShadows` marches a short ray toward the scene's sun in the depth buffer; `ssgi` gathers eight rays a pixel over the G-buffer's normals at half size into a float target and averages the neighbours on the same surface before laying the light, times the receiver's colour, over the picture. The Weather component raises the sun's `sourceAngle` and lowers its `shadowStrength` as the sky clouds over.+- **Voxel GI.** Voxel GI adds diffuse bounce light from a world-space voxel cache the camera remembers, on WebGPU. The cache is a voxel grid around the camera that each frame takes in the lit picture (the radiance and facing of the surfaces it sees) and fades what it held; light is gathered by marching cosine-weighted rays through it, so a wall that has left the screen still tints the floor beside it and a moving light lights its surroundings, which Screen Space GI cannot do. It holds only what the camera has seen, at the voxel's size, with one diffuse bounce: it is not a trace of the scene's geometry, and not Lumen. A device with no compute stage goes without it.- **Compute effects.** A post-process effect can be a WGSL compute shader over the picture, run on WebGPU and left out on WebGL2, and Sharpen is the first. An effect registers with `compute` instead of `factory`; it is handed the picture so far, a storage texture to write, its parameters, the size and the scene's depth. Sharpen pushes each pixel away from its neighbours and holds it inside their range, so an edge steepens without a halo. A compute effect refines a picture that is already whole, so a device with no compute stage simply goes without it.- **Sky.** The Sky Atmosphere effect scatters the sun's light through an atmosphere and draws volumetric clouds that the sun lights and the wind moves. The dome is single scattering through a planet's atmosphere (Rayleigh and Mie), so a sunset's reds and the afterglow come from the same calculation as noon's blue; the clouds are a slab marched against a noise density, lit through what lies between them and the sun, and a Weather's overcast raises their coverage.- **Time of day.** A TimeOfDay component moves the sun and the moon across the sky, and a Sky Atmosphere effect draws the day, the dusk and the night. The sun is placed from the hour, the latitude and the season; the scene's directional light is aimed at it by day and at the moon by night, scaled and tinted by a day's curve, and the ambient light, baked reflections, fog and exposure follow. Sky Atmosphere draws the dome, the sun, a dusk glow, the moon and stars where there is no surface, and Weather greys it as it clouds over. A directional light that follows a panorama's sun can now be aimed by its own Transform (`aimFollowedSun`), which is what lets a panorama photographed with the sun in one place be lit by a sun that moves.diff --git a/cmake/ESEngineSources.cmake b/cmake/ESEngineSources.cmakeindex 18ba6e606..c53586d52 100644--- a/cmake/ESEngineSources.cmake+++ b/cmake/ESEngineSources.cmake@@ -103,6 +103,7 @@ endif()if(ES_ENABLE_POSTPROCESS)list(APPEND ESENGINE_SOURCES${ESENGINE_ROOT}/src/esengine/renderer/frame/PostProcessPipeline.cpp+ ${ESENGINE_ROOT}/src/esengine/renderer/frame/VoxelGI.cpp)endif()diff --git a/desktop b/desktopindex b0b3e718c..27b816fa0 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit b0b3e718c4b901401334eb303b4675613726be49+Subproject commit 27b816fa07eebee018a3cedfa1f5bb5e35f27f79diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdxindex 130d5253d..09bf70bad 100644--- a/docs/astro/src/content/docs/graphics/post-processing.mdx+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx@@ -25,6 +25,7 @@ several can stack in one frame at no extra target cost beyond the passes.| `distanceFog` | Linear fog by distance from the camera | `near`, `far`, `intensity`, colour || `heightFog` | Exponential height fog with the sun's glow and a hazy horizon | `density`, `heightFalloff`, `baseHeight`, `sunAzimuth`, `sunElevation`, `skyFog`, colours || `skyAtmosphere` | An atmosphere that scatters the sun's light, with sun, moon, stars and volumetric clouds, drawn where there is no surface | `intensity`, `haze`, `altitude`, `clouds`, `cloudBase`, `cloudThickness`, `windSpeed`, `overcast`, `sunAzimuth`, `sunElevation` |+| `voxelGI` | Diffuse global illumination from a world-space voxel cache the camera remembers (WebGPU only) | `intensity`, `rays`, `voxelSize`, `range`, `maxBounce`, `decay`, `blend`, `divisor` || `sharpen` | Sharpening that cannot make a halo (a compute effect: WebGPU only) | `intensity`, `limit` || `lightShafts` | Sun rays through whatever stands in front of it | `intensity`, `sunAzimuth`, `sunElevation`, `decay`, `reach` || `contactShadows` | The small sharp shadows at a contact a shadow map is too coarse to hold | `intensity`, `length`, `thickness` |@@ -148,6 +149,28 @@ a second output (`layout(location = 1) out vec4`, or a struct return in WGSL) anOn WebGL2 and WebGPU alike the attachment is multisampled with the scene and resolved with it. A blended draw never writesit — a normal averaged with the one behind it is neither's — and the scene's own background clears it to 0.+### Voxel GI (WebGPU)++`voxelGI` is diffuse global illumination from a **world-space voxel cache**. A grid of voxels (128 × 64 × 128 around the camera, `voxelSize`+world units each) remembers what the camera has seen: every frame the lit picture is written into the voxels its visible surfaces+stand in — their average radiance and the way they face — and what was there before fades by `decay`. Light is then gathered the way a+path tracer would, but through the grid: from each visible surface a few cosine-weighted rays (`rays`) are marched cell by cell, and+where one meets a voxel that faces it, that voxel's radiance is the light it brings. The gather runs at a fraction of the picture's size+(`divisor`), is blended over frames (`blend`) and is upsampled with the depth, so it does not bleed across an edge.++Because the cache is the world's and not the screen's, a bright wall that has left the picture still tints the floor beside it, a surface lit+only by another is lit, and a moving light lights its surroundings — the three things the screen-space bounce (`ssgi`) cannot do.++It is **not a trace of the scene's geometry**, and it says so:++- the cache holds the surfaces the camera has *seen*, at the voxel's size — a surface it has not seen yet sends no light, and a thin one is as thick as a cell;+- one bounce, diffuse only (no glossy transport), and the picture it is injected from carries last frame's light, so a bright surface feeds itself a little;+- it needs a **compute stage**: WebGPU. On WebGL2 the effect is left out and the picture goes through as it was — use `ssgi` there;+- it costs a grid of memory (about 32 MB), four compute passes a frame and one cache per camera.++It is not Unreal's Lumen: there is no distance-field or hardware ray tracing and no surface cache of the scene's meshes, which is what lets Lumen light what+has never been on screen. Voxel GI is the version of the idea that needs only the picture.+### Compute effects (WebGPU)An effect can be a **compute shader** instead of a draw: WGSL run over the whole picture, which is how an effect that readsdiff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdxindex 6a148c65a..f15db2313 100644--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx@@ -24,6 +24,7 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess| `distanceFog` | 按到相机的距离做线性雾 | `near`、`far`、`intensity`、颜色 || `heightFog` | 指数高度雾,带太阳辉光和朦胧的地平线 | `density`、`heightFalloff`、`baseHeight`、`sunAzimuth`、`sunElevation`、`skyFog`、颜色 || `skyAtmosphere` | 散射太阳光的大气,带太阳、月亮、星星和体积云,画在没有表面的地方 | `intensity`、`haze`、`altitude`、`clouds`、`cloudBase`、`cloudThickness`、`windSpeed`、`overcast`、`sunAzimuth`、`sunElevation` |+| `voxelGI` | 基于相机记住的世界空间体素缓存的漫反射全局光照(仅 WebGPU) | `intensity`、`rays`、`voxelSize`、`range`、`maxBounce`、`decay`、`blend`、`divisor` || `sharpen` | 不会出光晕的锐化(计算特效:仅 WebGPU) | `intensity`、`limit` || `lightShafts` | 太阳光穿过前方遮挡物形成的光束 | `intensity`、`sunAzimuth`、`sunElevation`、`decay`、`reach` || `contactShadows` | 阴影贴图太粗放不下的接触处的小而锐利的阴影 | `intensity`、`length`、`thickness` |@@ -115,6 +116,24 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess在 WebGL2 和 WebGPU 上,附件都与场景一起多重采样、一起解析。混合绘制从不写它——与背后那个平均过的法线谁也不是——场景自己的背景把它清成 0。+### 体素 GI(WebGPU)++`voxelGI` 是基于**世界空间体素缓存**的漫反射全局光照。一个围绕相机的体素网格(128 × 64 × 128,每个 `voxelSize` 世界单位)记住相机见过的东西:+每一帧,被照亮的画面被写进它可见表面所在的体素——它们的平均辐射度和朝向——原有的内容按 `decay` 衰减。然后像路径追踪那样收集光,只是穿过网格:+从每个可见表面发出几条余弦加权的光线(`rays`),逐格前进,遇到一个朝向它的体素,那个体素的辐射度就是它带来的光。收集在画面尺寸的一部分上进行+(`divisor`),跨帧混合(`blend`),并用深度上采样,所以不会越过边缘渗色。++因为缓存属于世界而不是屏幕,一面已经离开画面的亮墙仍会给旁边的地面染色,只被另一个表面照亮的表面也能被照亮,移动的光会照亮周围——这三件事屏幕空间的弹射(`ssgi`)都做不到。++它**不是对场景几何体的追踪**,并且如实说明:++- 缓存里只有相机*见过*的表面,精度是体素大小——没见过的表面不发光,薄的东西和一格一样厚;+- 一次弹射,只有漫反射(没有光泽传输),注入用的画面带着上一帧的光,所以很亮的表面会稍微自己给自己供光;+- 它需要**计算阶段**:WebGPU。在 WebGL2 上该特效会被拿掉,画面原样通过——那里请用 `ssgi`;+- 它要一块网格内存(约 32 MB)、每帧四个计算 pass、每个相机一份缓存。++它不是虚幻的 Lumen:没有距离场或硬件光追,也没有场景网格的表面缓存——这正是 Lumen 能照亮从未出现在屏幕上的东西的原因。体素 GI 是这个思路中只需要画面的那一版。+### 计算特效(WebGPU)一个特效可以是**计算着色器**而不是绘制:在整张画面上运行的 WGSL,适合需要读取很多相邻像素、或要保留自己结果的特效,不必每一步都开一个渲染 pass。diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.mdindex 797cbb9ef..e7911ab17 100644--- a/sdk/etc/index.api.md+++ b/sdk/etc/index.api.md@@ -3581,6 +3581,7 @@ compute: ComputeEffectSource | undefinedfactory: (() => ShaderHandle) | undefinedlabel: stringmultiPass: EffectSubPass[] | undefined+native: string | undefinedtextures: EffectTextureDef[] | undefinedtype: stringuniforms: EffectUniformDef[]@@ -6657,6 +6658,7 @@ static prototype: PostProcessPlugin## PostProcessStack — class @experimental```addComputePass: (name: string, compute: ComputePassSource) => PostProcessStack+addNativePass: (name: string, kind: string) => PostProcessStackaddPass: (name: string, shader: ShaderHandle, scale?: number, continuesEffect?: boolean) => PostProcessStackclearDirty: () => voidclearPasses: () => PostProcessStackdiff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.mdindex 7912e7e38..720f90d67 100644--- a/sdk/etc/index.native.api.md+++ b/sdk/etc/index.native.api.md@@ -3589,6 +3589,7 @@ compute: ComputeEffectSource | undefinedfactory: (() => ShaderHandle) | undefinedlabel: stringmultiPass: EffectSubPass[] | undefined+native: string | undefinedtextures: EffectTextureDef[] | undefinedtype: stringuniforms: EffectUniformDef[]@@ -6860,6 +6861,7 @@ static prototype: PostProcessPlugin## PostProcessStack — class @experimental```addComputePass: (name: string, compute: ComputePassSource) => PostProcessStack+addNativePass: (name: string, kind: string) => PostProcessStackaddPass: (name: string, shader: ShaderHandle, scale?: number, continuesEffect?: boolean) => PostProcessStackclearDirty: () => voidclearPasses: () => PostProcessStackdiff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.mdindex 9681a7345..6cfb61956 100644--- a/sdk/etc/index.node.api.md+++ b/sdk/etc/index.node.api.md@@ -3576,6 +3576,7 @@ compute: ComputeEffectSource | undefinedfactory: (() => ShaderHandle) | undefinedlabel: stringmultiPass: EffectSubPass[] | undefined+native: string | undefinedtextures: EffectTextureDef[] | undefinedtype: stringuniforms: EffectUniformDef[]@@ -6652,6 +6653,7 @@ static prototype: PostProcessPlugin## PostProcessStack — class @experimental```addComputePass: (name: string, compute: ComputePassSource) => PostProcessStack+addNativePass: (name: string, kind: string) => PostProcessStackaddPass: (name: string, shader: ShaderHandle, scale?: number, continuesEffect?: boolean) => PostProcessStackclearDirty: () => voidclearPasses: () => PostProcessStackdiff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.mdindex 079a4fe30..b73f8f67a 100644--- a/sdk/etc/minigame.api.md+++ b/sdk/etc/minigame.api.md@@ -3576,6 +3576,7 @@ compute: ComputeEffectSource | undefinedfactory: (() => ShaderHandle) | undefinedlabel: stringmultiPass: EffectSubPass[] | undefined+native: string | undefinedtextures: EffectTextureDef[] | undefinedtype: stringuniforms: EffectUniformDef[]@@ -6945,6 +6946,7 @@ static prototype: PostProcessPlugin## PostProcessStack — class @experimental```addComputePass: (name: string, compute: ComputePassSource) => PostProcessStack+addNativePass: (name: string, kind: string) => PostProcessStackaddPass: (name: string, shader: ShaderHandle, scale?: number, continuesEffect?: boolean) => PostProcessStackclearDirty: () => voidclearPasses: () => PostProcessStackdiff --git a/sdk/etc/wasm.api.md b/sdk/etc/wasm.api.mdindex e742f7621..51e5a9c69 100644--- a/sdk/etc/wasm.api.md+++ b/sdk/etc/wasm.api.md@@ -383,6 +383,7 @@ particle_set_speed_lut: ((entity: number, ptr: number, count: number) => void) |particle_stop: ((registry: CppRegistry, entity: number) => void) | undefinedparticle_update: ((registry: CppRegistry, dt: number) => void) | undefinedpostprocess_addComputePass: ((name: string, wgsl: string, params: string, workgroupX: number, workgroupY: number) => number) | undefined+postprocess_addNativePass: ((name: string, kind: string) => number) | undefinedpostprocess_addPass: (name: string, shaderHandle: number) => numberpostprocess_addScreenPass: (name: string, shaderHandle: number) => numberpostprocess_begin: () => voiddiff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.mdindex 894753c0e..43aa59e78 100644--- a/sdk/etc/wechat.api.md+++ b/sdk/etc/wechat.api.md@@ -3576,6 +3576,7 @@ compute: ComputeEffectSource | undefinedfactory: (() => ShaderHandle) | undefinedlabel: stringmultiPass: EffectSubPass[] | undefined+native: string | undefinedtextures: EffectTextureDef[] | undefinedtype: stringuniforms: EffectUniformDef[]@@ -6659,6 +6660,7 @@ static prototype: PostProcessPlugin## PostProcessStack — class @experimental```addComputePass: (name: string, compute: ComputePassSource) => PostProcessStack+addNativePass: (name: string, kind: string) => PostProcessStackaddPass: (name: string, shader: ShaderHandle, scale?: number, continuesEffect?: boolean) => PostProcessStackclearDirty: () => voidclearPasses: () => PostProcessStackdiff --git a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts b/sdk/src/ecs/bridge/nativeEngineApi.generated.tsindex ec2d406ea..e32779213 100644--- a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts+++ b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts@@ -100,6 +100,7 @@ export interface NativeEngineApi {particle_stop?(registry: unknown, entity: number): void;particle_update?(registry: unknown, dt: number): void;postprocess_addComputePass?(name: string, wgsl: string, params: string, workgroupX: number, workgroupY: number): number;+ postprocess_addNativePass?(name: string, kind: string): number;postprocess_addPass?(name: string, shaderHandle: number): number;postprocess_addScreenPass?(name: string, shaderHandle: number): number;postprocess_begin?(): void;@@ -415,6 +416,7 @@ export function createNativeEngineApi(bind('particle_stop', 'es_particle_stop', true);bind('particle_update', 'es_particle_update', true);bind('postprocess_addComputePass', 'es_postprocess_addComputePass', false);+ bind('postprocess_addNativePass', 'es_postprocess_addNativePass', false);bind('postprocess_addPass', 'es_postprocess_addPass', false);bind('postprocess_addScreenPass', 'es_postprocess_addScreenPass', false);bind('postprocess_begin', 'es_postprocess_begin', false);diff --git a/sdk/src/postprocess/PostProcessAPI.ts b/sdk/src/postprocess/PostProcessAPI.tsindex a85af9876..e924ea463 100644--- a/sdk/src/postprocess/PostProcessAPI.ts+++ b/sdk/src/postprocess/PostProcessAPI.ts@@ -30,7 +30,7 @@ type PostProcessCore = Required<Pick<NonNullable<EngineApi>,| 'postprocess_clearScreenPasses' | 'postprocess_setScreenUniformFloat'| 'postprocess_setScreenUniformVec4'>>// A core without a compute stage (or built before there was one) does not answer this one.- & Pick<NonNullable<EngineApi>, 'postprocess_addComputePass' | 'postprocess_supportsCompute'>;+ & Pick<NonNullable<EngineApi>, 'postprocess_addComputePass' | 'postprocess_supportsCompute' | 'postprocess_addNativePass'>;/** Guarded view of the core: after a wasm abort a call throws instead of reaching* a dead module; a native host's bindings never abort. */@@ -120,7 +120,10 @@ export function syncStackToWasm(stack: PostProcessStack, force = false,for (const pass of stack.passes) {if (!pass.enabled || !allow(pass.name)) continue;try {- if (pass.compute) {+ if (pass.native) {+ if (!m.postprocess_addNativePass) continue;+ m.postprocess_addNativePass(pass.name, pass.native);+ } else if (pass.compute) {// No compute on this core (WebGL2): the pass is left out and the picture goes through as it was.if (!m.postprocess_addComputePass) continue;m.postprocess_addComputePass(pass.name, pass.compute.source, pass.compute.params.join(','),diff --git a/sdk/src/postprocess/PostProcessStack.ts b/sdk/src/postprocess/PostProcessStack.tsindex 201112978..6616c9893 100644--- a/sdk/src/postprocess/PostProcessStack.ts+++ b/sdk/src/postprocess/PostProcessStack.ts@@ -22,6 +22,8 @@ export interface PassConfig {continuesEffect: boolean;/** Set for a compute pass: it has no `shader`, and the engine dispatches this WGSL over the picture instead. */compute?: ComputePassSource;+ /** Set for a pass the engine implements itself ("voxelGI"): it has no `shader`, and the engine declares its own passes. */+ native?: string;}/** What a compute pass runs: the WGSL, the names of its f32 params in the order its params block lists them, and its workgroup size. */@@ -103,6 +105,23 @@ export class PostProcessStack {return this;}+ /** Adds a pass the engine implements itself, by kind. A device with no compute stage leaves it out of the chain. */+ addNativePass(name: string, kind: string): this {+ this.passes_.push({+ name,+ shader: 0 as ShaderHandle,+ enabled: true,+ floatUniforms: new Map(),+ vec4Uniforms: new Map(),+ textureUniforms: new Map(),+ scale: 1,+ continuesEffect: false,+ native: kind,+ });+ this.dirty_ = true;+ return this;+ }+removePass(name: string): this {const idx = this.passes_.findIndex(p => p.name === name);if (idx !== -1) {diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.tsindex 7e43a9c21..bf9b90f75 100644--- a/sdk/src/postprocess/effects.ts+++ b/sdk/src/postprocess/effects.ts@@ -64,6 +64,8 @@ export interface EffectDef {factory?: () => ShaderHandle;/** Set instead of `factory` for an effect that runs as a compute shader. */compute?: ComputeEffectSource;+ /** Set instead of `factory` for an effect the engine implements itself, in compute passes of its own ("voxelGI"). */+ native?: string;uniforms: EffectUniformDef[];/** Texture params, carried on the volume as serialized refs (`textures`). */textures?: EffectTextureDef[];@@ -287,6 +289,37 @@ registerEffect({],});+// Diffuse global illumination from a world-space voxel cache the frame seeds and the camera remembers: light a surface+// that has left the screen still gives is kept. Compute passes the engine declares itself, so WebGPU only; on WebGL2+// the picture goes through as it was (use Screen Space GI there).+registerEffect({+ type: 'voxelGI',+ label: 'Voxel GI',+ native: 'voxelGI',+ uniforms: [+ // How much of the gathered light is added over the picture.+ { name: 'u_intensity', label: 'Intensity', min: 0, max: 4, step: 0.01, defaultValue: 1, neutralValue: 0 },+ // Rays per surface point: more is smoother and costs in proportion.+ { name: 'u_rays', label: 'Rays', min: 1, max: 32, step: 1, defaultValue: 8 },+ // The world size of a voxel: the cache is 128 x 64 x 128 of them around the camera, so this sets how far it reaches and how finely.+ { name: 'u_voxelSize', label: 'Voxel Size', min: 4, max: 400, step: 1, defaultValue: 40 },+ // How far a ray looks, in voxels.+ { name: 'u_range', label: 'Range', min: 4, max: 90, step: 1, defaultValue: 48 },+ // Brighter than this a surface does not give out, so one lamp does not light the street from a single pixel.+ { name: 'u_maxBounce', label: 'Max Bounce', min: 0.5, max: 16, step: 0.5, defaultValue: 3 },+ // What the cache keeps each frame of what it held: lower forgets sooner (a moving light follows faster), higher is steadier.+ { name: 'u_decay', label: 'Memory', min: 0, max: 0.99, step: 0.01, defaultValue: 0.93 },+ // How far off its own voxel a ray starts, in voxels, so a surface is not lit by itself.+ { name: 'u_bias', label: 'Start Bias', min: 0, max: 4, step: 0.1, defaultValue: 1.2 },+ // The gather runs at 1 / this of the picture's size along each axis.+ { name: 'u_divisor', label: 'Resolution Divisor', min: 1, max: 4, step: 1, defaultValue: 2 },+ // How much of each frame's gather goes into the picture; lower is smoother and slower to follow a change.+ { name: 'u_blend', label: 'Temporal Blend', min: 0.02, max: 1, step: 0.01, defaultValue: 0.2 },+ // 1 shows the gathered light alone.+ { name: 'u_debug', label: 'Show GI Only', min: 0, max: 1, step: 1, defaultValue: 0 },+ ],+});+// A compute effect: only a device with a compute stage (WebGPU) runs it; on WebGL2 the picture goes through as it was.registerEffect({type: 'sharpen',diff --git a/sdk/src/postprocess/volumeSystem.ts b/sdk/src/postprocess/volumeSystem.tsindex cb7da4ffb..fbf132dfd 100644--- a/sdk/src/postprocess/volumeSystem.ts+++ b/sdk/src/postprocess/volumeSystem.ts@@ -175,6 +175,9 @@ function applyBlendedEffects(applyTextures(stack, subPass.name, effectData.textures);applyColors(stack, subPass.name, effectData.colors);}+ } else if (def.native) {+ stack.addNativePass(effectType, def.native);+ for (const u of def.uniforms) stack.setUniform(effectType, u.name, effectData.uniforms.get(u.name) ?? u.defaultValue);} else if (def.compute) {// The params block is the effect's uniforms in their listed order, each at the volume's value or the effect's own default.stack.addComputePass(effectType, {diff --git a/sdk/src/wasm.ts b/sdk/src/wasm.tsindex c54b6c923..ff739d241 100644--- a/sdk/src/wasm.ts+++ b/sdk/src/wasm.ts@@ -524,6 +524,7 @@ export interface ESEngineModule {postprocess_addPass(name: string, shaderHandle: number): number;postprocess_addComputePass?(name: string, wgsl: string, params: string, workgroupX: number, workgroupY: number): number;postprocess_supportsCompute?(): boolean;+ postprocess_addNativePass?(name: string, kind: string): number;postprocess_setPassScale(name: string, scale: number): void;postprocess_setPassContinuesEffect(name: string): void;postprocess_setMsaaSamples(samples: number): void;diff --git a/sdk/tests/compute-effect.test.ts b/sdk/tests/compute-effect.test.tsindex 7a52fc2de..99ff97d77 100644--- a/sdk/tests/compute-effect.test.ts+++ b/sdk/tests/compute-effect.test.ts@@ -55,9 +55,9 @@ describe('compute effects', () => {expect(def!.compute!.source).toMatch(/struct Params \{ intensity : f32, limit : f32/);});- it('keeps every effect either a draw or a compute shader, never neither', () => {+ it('keeps every effect either a draw, a compute shader or a native pass, never none', () => {for (const def of getAllEffectDefs()) {- const kinds = (def.factory ? 1 : 0) + (def.compute ? 1 : 0) + (def.multiPass ? 1 : 0);+ const kinds = (def.factory ? 1 : 0) + (def.compute ? 1 : 0) + (def.native ? 1 : 0) + (def.multiPass ? 1 : 0);expect(kinds, `${def.type} must say how it runs`).toBeGreaterThanOrEqual(1);}});@@ -96,3 +96,42 @@ describe('compute effects', () => {expect(getEffectDef('testCompute')!.compute!.workgroupSize).toEqual([8, 8]);});});++describe('native effects', () => {+ it('registers Voxel GI as an effect the engine implements itself', () => {+ const def = getEffectDef('voxelGI');+ expect(def).toBeDefined();+ expect(def!.native).toBe('voxelGI');+ expect(def!.factory).toBeUndefined();+ expect(def!.compute).toBeUndefined();+ // The native side reads these by name, with these defaults when one is not set.+ expect(def!.uniforms.map((u) => u.name)).toEqual([+ 'u_intensity', 'u_rays', 'u_voxelSize', 'u_range', 'u_maxBounce', 'u_decay', 'u_bias', 'u_divisor', 'u_blend', 'u_debug',+ ]);+ const defaults = Object.fromEntries(def!.uniforms.map((u) => [u.name, u.defaultValue]));+ expect(defaults.u_rays).toBe(8);+ expect(defaults.u_divisor).toBe(2);+ });++ it('pushes a native pass by its kind, and leaves it out on a core that cannot', () => {+ const pp = new PostProcessAPI();+ const withNative = { ...mockCore(true), postprocess_addNativePass: vi.fn().mockReturnValue(0) };+ initPostProcessAPI(withNative as unknown as ESEngineModule);+ const stack = new PostProcessStack(pp.state);+ stack.addNativePass('voxelGI', 'voxelGI');+ stack.setUniform('voxelGI', 'u_rays', 12);+ syncStackToWasm(stack, true);+ expect(withNative.postprocess_addNativePass).toHaveBeenCalledWith('voxelGI', 'voxelGI');+ expect(withNative.postprocess_addPass).not.toHaveBeenCalled();+ expect(withNative.postprocess_setUniformFloat).toHaveBeenCalledWith('voxelGI', 'u_rays', 12);+ shutdownPostProcessAPI();++ const without = mockCore(true);+ initPostProcessAPI(without as unknown as ESEngineModule);+ const other = new PostProcessStack(pp.state);+ other.addNativePass('voxelGI', 'voxelGI');+ expect(() => syncStackToWasm(other, true)).not.toThrow();+ expect(without.postprocess_addPass).not.toHaveBeenCalled();+ shutdownPostProcessAPI();+ });+});diff --git a/src/esengine/bindings/PostProcessBindings.cpp b/src/esengine/bindings/PostProcessBindings.cppindex 51b09df60..0af4ff980 100644--- a/src/esengine/bindings/PostProcessBindings.cpp+++ b/src/esengine/bindings/PostProcessBindings.cpp@@ -81,6 +81,11 @@ u32 postprocess_addComputePass(const std::string& name, const std::string& wgsl,return g_postProcessPipeline->addComputePass(name, wgsl, names, workgroupX, workgroupY);}+u32 postprocess_addNativePass(const std::string& name, const std::string& kind) {+ if (!g_postProcessPipeline) return 0;+ return g_postProcessPipeline->addNativePass(name, kind);+}+bool postprocess_supportsCompute() {return g_postProcessPipeline ? g_postProcessPipeline->supportsCompute() : (g_device && g_device->supportsCompute());}diff --git a/src/esengine/bindings/PostProcessBindings.hpp b/src/esengine/bindings/PostProcessBindings.hppindex 3c9dd65f7..8513012e0 100644--- a/src/esengine/bindings/PostProcessBindings.hpp+++ b/src/esengine/bindings/PostProcessBindings.hpp@@ -21,6 +21,11 @@ u32 postprocess_addComputePass(const std::string& name, const std::string& wgsl,u32 workgroupX, u32 workgroupY);/** @brief Whether compute passes run on this device: WebGPU yes, WebGL2 no. */bool postprocess_supportsCompute();+/**+ * @brief Adds a pass the engine implements itself ("voxelGI"); its parameters are set by name like any pass's.+ * @details On a device with no compute stage the pass is left out of the chain and the picture goes through as it was.+ */+u32 postprocess_addNativePass(const std::string& name, const std::string& kind);/** The project's multisampling request for the scene target; 1 = off. Clamped* by what the device supports — the device answers capability, not policy. */void postprocess_setMsaaSamples(u32 samples);diff --git a/src/esengine/bindings/WebSDKEntry.cpp b/src/esengine/bindings/WebSDKEntry.cppindex 11e71d273..55b0a696a 100644--- a/src/esengine/bindings/WebSDKEntry.cpp+++ b/src/esengine/bindings/WebSDKEntry.cpp@@ -598,6 +598,7 @@ EMSCRIPTEN_BINDINGS(esengine_renderer) {emscripten::function("postprocess_addPass", &esengine::postprocess_addPass);emscripten::function("postprocess_addComputePass", &esengine::postprocess_addComputePass);emscripten::function("postprocess_supportsCompute", &esengine::postprocess_supportsCompute);+ emscripten::function("postprocess_addNativePass", &esengine::postprocess_addNativePass);emscripten::function("postprocess_setPassScale", &esengine::postprocess_setPassScale);emscripten::function("postprocess_setPassContinuesEffect", &esengine::postprocess_setPassContinuesEffect);emscripten::function("postprocess_setMsaaSamples", &esengine::postprocess_setMsaaSamples);diff --git a/src/esengine/renderer/frame/PostProcessPipeline.cpp b/src/esengine/renderer/frame/PostProcessPipeline.cppindex 26797b353..a22b405e9 100644--- a/src/esengine/renderer/frame/PostProcessPipeline.cpp+++ b/src/esengine/renderer/frame/PostProcessPipeline.cpp@@ -13,6 +13,7 @@#include "./PostProcessPipeline.hpp"#include "./RenderContext.hpp"+#include "./VoxelGI.hpp"#include "../rhi/GfxDevice.hpp"#include "../store/MaterialConstants.hpp"#include "../store/LightConstants.hpp"@@ -180,6 +181,7 @@ void PostProcessPipeline::shutdown() {for (auto& pass : screenPasses_) releasePassResources(pass);for (auto& [source, program] : compute_cache_) device_.deleteCompute(program);compute_cache_.clear();+ voxel_gis_.clear();if (compute_view_ubo_ != BufferHandle::Invalid) {device_.deleteBuffer(compute_view_ubo_);compute_view_ubo_ = BufferHandle::Invalid;@@ -260,9 +262,19 @@ u32 PostProcessPipeline::addPass(const std::string& name, resource::ShaderHandlebool PostProcessPipeline::supportsCompute() const { return device_.supportsCompute(); }bool PostProcessPipeline::runnable(const PostProcessPass& pass) const {+ if (!pass.nativeKind.empty()) return pass.enabled && device_.supportsCompute();return pass.enabled && (!pass.isCompute || (device_.supportsCompute() && pass.compute != ComputeHandle::Invalid));}+u32 PostProcessPipeline::addNativePass(const std::string& name, const std::string& kind) {+ PostProcessPass pass;+ pass.name = name;+ pass.enabled = true;+ pass.nativeKind = kind;+ passes_.push_back(pass);+ return static_cast<u32>(passes_.size() - 1);+}+u32 PostProcessPipeline::addComputePass(const std::string& name, const std::string& wgsl,const std::vector<std::string>& params, u32 workgroupX, u32 workgroupY) {PostProcessPass pass;@@ -345,6 +357,11 @@ bool PostProcessPipeline::passesWantGBuffer() {if (bypass_) return false;for (const auto& pass : passes_) {if (!pass.enabled) continue;+ // The voxel GI reads the surface's normal, whatever the effect's shader (it has none).+ if (!pass.nativeKind.empty()) {+ if (device_.supportsCompute()) return true;+ continue;+ }const MaterialUniformLayout* layout = context_.materials().layoutFor(pass.shader);if (layout && layout->usesGBuffer) return true;}@@ -999,6 +1016,38 @@ void PostProcessPipeline::runChain(std::vector<PostProcessPass>& passes, rg::Resrg::TargetDesc writeDesc = (i == lastEnabled) ? blitDesc : desc;if (i != lastEnabled) writeDesc.scale = pass.scale;if (passIsHighPrecision(pass)) writeDesc.format = GfxPixelFormat::RGBA16F;+ if (!pass.nativeKind.empty()) {+ // A pass the engine implements itself, in compute passes of its own that it declares onto the graph.+ if (pass.nativeKind == "voxelGI" && &passes == &passes_ && (scene_needs_depth_ || scene_gbuffer_) && scene_gbuffer_) {+ const auto get = [&pass](const char* name, f32 fallback) {+ const auto it = pass.floatUniforms.find(name);+ return it != pass.floatUniforms.end() ? it->second : fallback;+ };+ VoxelGIParams gi;+ gi.intensity = get("u_intensity", gi.intensity);+ gi.rays = static_cast<u32>(std::max(get("u_rays", static_cast<f32>(gi.rays)), 1.0f));+ gi.voxelSize = get("u_voxelSize", gi.voxelSize);+ gi.range = get("u_range", gi.range);+ gi.maxBounce = get("u_maxBounce", gi.maxBounce);+ gi.decay = get("u_decay", gi.decay);+ gi.bias = get("u_bias", gi.bias);+ gi.divisor = static_cast<u32>(std::max(get("u_divisor", static_cast<f32>(gi.divisor)), 1.0f));+ gi.debug = get("u_debug", gi.debug);+ gi.blend = get("u_blend", gi.blend);+ auto& cache = voxel_gis_[temporalKey_];+ if (!cache) {+ // A handful of cameras at most have a cache each; one past that evicts another's, which rebuilds when its camera returns.+ if (voxel_gis_.size() > 4) {+ for (auto it = voxel_gis_.begin(); it != voxel_gis_.end(); ++it) {+ if (it->first != temporalKey_) { voxel_gis_.erase(it); break; }+ }+ }+ voxel_gis_[temporalKey_] = makeUnique<VoxelGI>(device_, context_);+ }+ input = voxel_gis_[temporalKey_]->declare(*graph_, input, scene, gi, width_, height_);+ }+ continue;+ }if (pass.isCompute) {// A compute pass writes a storage texture, which is a half float whatever the chain keeps between passes (the// shader says its format), and it reads the scene's depth, so the scene has to outlive it.diff --git a/src/esengine/renderer/frame/PostProcessPipeline.hpp b/src/esengine/renderer/frame/PostProcessPipeline.hppindex 007d721d0..84c5564ce 100644--- a/src/esengine/renderer/frame/PostProcessPipeline.hpp+++ b/src/esengine/renderer/frame/PostProcessPipeline.hpp@@ -23,12 +23,14 @@#include <string>#include <vector>#include <unordered_map>+#include <memory>namespace esengine {class GfxDevice;class RenderContext;class Shader;+class VoxelGI;namespace resource {class ResourceManager;@@ -87,6 +89,8 @@ struct PostProcessPass {/// order, and the workgroup the shader declares. Invalid for every pass that draws.ComputeHandle compute = ComputeHandle::Invalid;bool isCompute = false;+ /// A pass the engine implements itself, in compute passes of its own (see VoxelGI): its name, empty for every other pass.+ std::string nativeKind;std::vector<std::string> computeParams;u32 workgroupX = 8;u32 workgroupY = 8;@@ -167,6 +171,11 @@ public:*/u32 addComputePass(const std::string& name, const std::string& wgsl, const std::vector<std::string>& params,u32 workgroupX, u32 workgroupY);+ /**+ * @brief Adds a pass the engine implements itself: "voxelGI" (see VoxelGI). Its parameters are set like any pass's, by name.+ * @details Needs a device with a compute stage; elsewhere the pass is left out of the chain.+ */+ u32 addNativePass(const std::string& name, const std::string& kind);/// Whether passes added with addComputePass run on this device.bool supportsCompute() const;/// The project's multisampling request; 1 = off. Clamped by the device.@@ -526,6 +535,9 @@ private:/// Compute programs by source, kept while the pipeline lives: the passes are pushed again every camera, and a/// program is a compiled pipeline, not something to build per frame.std::unordered_map<std::string, ComputeHandle> compute_cache_;+ /// The voxel GI's cache lives here, across frames: what it has seen is the point of it. One per camera, since a cache is a+ /// window around the eye and two eyes would take turns tearing each other's down.+ std::unordered_map<u64, Unique<VoxelGI>> voxel_gis_;/// The size block a compute pass reads, one for all of them (each dispatch writes it just before it runs).BufferHandle compute_view_ubo_ = BufferHandle::Invalid;/// Declares one chain onto an already-begun graph and runs it, ending in thediff --git a/src/esengine/renderer/frame/RenderContext.hpp b/src/esengine/renderer/frame/RenderContext.hppindex 21fcc163b..fe96af017 100644--- a/src/esengine/renderer/frame/RenderContext.hpp+++ b/src/esengine/renderer/frame/RenderContext.hpp@@ -176,6 +176,9 @@ public:*/void useFrameConstants(BufferHandle ubo, const glm::mat4& viewProjection);+ /** @brief The buffer the camera's FrameConstants live in, for a compute pass that reads them (it is bound at group 0). */+ BufferHandle frameUbo() const { return frameUbo_; }+/** @brief Bytes a buffer passed to useFrameConstants must hold. */static u32 frameConstantsSize();diff --git a/src/esengine/renderer/frame/VoxelGI.cpp b/src/esengine/renderer/frame/VoxelGI.cppnew file mode 100644index 000000000..11c198558--- /dev/null+++ b/src/esengine/renderer/frame/VoxelGI.cpp@@ -0,0 +1,542 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+#include "./VoxelGI.hpp"+#include "./FrameConstants.hpp"+#include "./RenderContext.hpp"+#include "../rhi/GfxDevice.hpp"+#include "../../core/Log.hpp"++#include <algorithm>+#include <cmath>+#include <vector>++namespace esengine {++namespace {++/// Eight u32 per voxel: radiance r, g, b (fixed point), a sample count, the facing x, y, z (fixed point), a spare.+constexpr u32 kVoxelWords = 8;++/// What every kernel sees at group 0: the frame, the cache's parameters and the cache itself.+const char* kCommon = R"WGSL(+struct Frame { projection : mat4x4f, camera : vec4f, inverseViewProjection : mat4x4f };+struct GI {+ origin : vec4i, // xyz: the window's first cell; w: the frame+ dims : vec4u, // xyz: voxels per axis; w: rays per point+ a : vec4f, // x: voxel size, y: what the grid keeps, z: the brightest a surface gives, w: intensity+ b : vec4f, // x: how far a ray looks (cells), y: where it starts (cells), z: unused, w: debug+ view : vec4f, // x: the gather's divisor, y: 1 when the last gather is a history to blend with, z: how much of this one goes in+ prevViewProjection : mat4x4f,+};+@group(0) @binding(0) var<uniform> frame : Frame;+@group(0) @binding(1) var<uniform> gi : GI;+@group(0) @binding(2) var<storage, read_write> grid : array<atomic<u32>>;+@group(0) @binding(3) var<storage, read_write> tags : array<u32>;++const WORDS : u32 = 8u;+const FIXED : f32 = 1024.0;+const FACING : f32 = 256.0;++fn hash3(c : vec3i) -> u32 {+ var h = (u32(c.x) * 73856093u) ^ (u32(c.y) * 19349663u) ^ (u32(c.z) * 83492791u);+ h = (h ^ (h >> 16u)) * 0x45d9f3bu;+ h = h ^ (h >> 16u);+ return h | 1u;+}+fn wrapIndex(cell : vec3i) -> u32 {+ let d = vec3i(gi.dims.xyz);+ let w = ((cell % d) + d) % d;+ return u32(w.x) + u32(w.y) * gi.dims.x + u32(w.z) * gi.dims.x * gi.dims.y;+}+fn inWindow(cell : vec3i) -> bool {+ let o = gi.origin.xyz;+ let d = vec3i(gi.dims.xyz);+ return all(cell >= o) && all(cell < o + d);+}+)WGSL";++/// What the depth, the G-buffer and the frame say about a pixel, shared by the kernels that read them.+const char* kSurface = R"WGSL(+fn worldFromDepth(uv : vec2f, depth : f32) -> vec3f {+ let clip = vec4f(uv.x * 2.0 - 1.0, 1.0 - uv.y * 2.0, depth, 1.0);+ let world = frame.inverseViewProjection * clip;+ return world.xyz / world.w;+}+fn decodeNormal(g : vec4f) -> vec3f {+ let f = g.rg * 2.0 - 1.0;+ var n = vec3f(f.x, f.y, 1.0 - abs(f.x) - abs(f.y));+ let k = clamp(-n.z, 0.0, 1.0);+ n = vec3f(n.x + select(k, -k, n.x >= 0.0), n.y + select(k, -k, n.y >= 0.0), n.z);+ return normalize(n);+}+fn viewDirection(p : vec3f) -> vec3f {+ if (frame.camera.w > 0.5) { return normalize(frame.camera.xyz - p); }+ return frame.camera.xyz;+}+)WGSL";++/// Ages the grid, and hands cells that came into the window since last frame back empty.+const char* kMaintain = R"WGSL(+@compute @workgroup_size(64)+fn main(@builtin(global_invocation_id) id : vec3u) {+ let total = gi.dims.x * gi.dims.y * gi.dims.z;+ let i = id.x;+ if (i >= total) { return; }+ let g = vec3i(i32(i % gi.dims.x), i32((i / gi.dims.x) % gi.dims.y), i32(i / (gi.dims.x * gi.dims.y)));+ let d = vec3i(gi.dims.xyz);+ // The cell this slot stands for in the window as it is now: toroidal, so the window slides over the same memory.+ let rel = ((g - gi.origin.xyz) % d + d) % d;+ let cell = gi.origin.xyz + rel;+ let tag = hash3(cell);+ let base = i * WORDS;+ if (tags[i] != tag) {+ tags[i] = tag;+ for (var k = 0u; k < WORDS; k = k + 1u) { atomicStore(&grid[base + k], 0u); }+ } else {+ for (var k = 0u; k < 7u; k = k + 1u) {+ atomicStore(&grid[base + k], u32(f32(atomicLoad(&grid[base + k])) * gi.a.y));+ }+ }+}+)WGSL";++/// The lit picture into the voxels its visible surfaces stand in.+const char* kInject = R"WGSL(+@group(0) @binding(4) var scene : texture_2d<f32>;+@group(0) @binding(5) var depthTex : texture_depth_2d;+@group(0) @binding(6) var gbuffer : texture_2d<f32>;++@compute @workgroup_size(8, 8)+fn main(@builtin(global_invocation_id) id : vec3u) {+ let size = vec2i(textureDimensions(depthTex));+ let t = vec2i(id.xy) * 2;+ if (t.x >= size.x || t.y >= size.y) { return; }+ let depth = textureLoad(depthTex, t, 0);+ if (depth >= 1.0) { return; }+ let g = textureLoad(gbuffer, t, 0);+ if (g.a <= 0.7) { return; }+ var n = decodeNormal(g);+ let p = worldFromDepth((vec2f(t) + 0.5) / vec2f(size), depth);+ if (dot(n, viewDirection(p)) < 0.0) { n = -n; }+ let cell = vec3i(floor(p / gi.a.x));+ if (!inWindow(cell)) { return; }+ let base = wrapIndex(cell) * WORDS;+ let c = clamp(textureLoad(scene, t, 0).rgb, vec3f(0.0), vec3f(gi.a.z));+ atomicAdd(&grid[base], u32(c.r * FIXED));+ atomicAdd(&grid[base + 1u], u32(c.g * FIXED));+ atomicAdd(&grid[base + 2u], u32(c.b * FIXED));+ atomicAdd(&grid[base + 3u], 1u);+ let f = n * 0.5 + 0.5;+ atomicAdd(&grid[base + 4u], u32(f.x * FACING));+ atomicAdd(&grid[base + 5u], u32(f.y * FACING));+ atomicAdd(&grid[base + 6u], u32(f.z * FACING));+}+)WGSL";++/// Rays from each visible surface through the grid, at a fraction of the picture's size.+const char* kGather = R"WGSL(+@group(0) @binding(5) var depthTex : texture_depth_2d;+@group(0) @binding(6) var gbuffer : texture_2d<f32>;+@group(0) @binding(7) var outTex : texture_storage_2d<rgba16float, write>;+@group(0) @binding(9) var histTex : texture_2d<f32>;++fn ign(p : vec2f) -> f32 {+ return fract(52.9829189 * fract(dot(p, vec2f(0.06711056, 0.00583715))));+}+fn cosineDirection(n : vec3f, u : f32, v : f32) -> vec3f {+ let r = sqrt(u);+ let phi = 6.2831853 * v;+ var up = vec3f(1.0, 0.0, 0.0);+ if (abs(n.y) < 0.99) { up = vec3f(0.0, 1.0, 0.0); }+ let t = normalize(cross(up, n));+ let b = cross(n, t);+ return normalize(t * (r * cos(phi)) + b * (r * sin(phi)) + n * sqrt(max(1.0 - u, 0.0)));+}++/// The light a ray from @p origin along @p dir brings: that of the first voxel it meets that faces it.+fn trace(origin : vec3f, dir : vec3f) -> vec3f {+ let vs = gi.a.x;+ let pos = origin / vs;+ var cell = vec3i(floor(pos));+ let stepv = vec3i(select(vec3f(-1.0), vec3f(1.0), dir >= vec3f(0.0)));+ let safe = select(dir, vec3f(1e-6), abs(dir) < vec3f(1e-6));+ let delta = abs(vec3f(1.0) / safe);+ let next = select(vec3f(cell), vec3f(cell + vec3i(1)), dir >= vec3f(0.0));+ var tMax = (next - pos) / safe;+ let range = gi.b.x;+ var t = 0.0;+ for (var i = 0; i < 96; i = i + 1) {+ if (t > range) { break; }+ // Step into the neighbouring cell along the axis whose boundary comes first.+ if (tMax.x < tMax.y && tMax.x < tMax.z) {+ cell.x = cell.x + stepv.x; t = tMax.x; tMax.x = tMax.x + delta.x;+ } else if (tMax.y < tMax.z) {+ cell.y = cell.y + stepv.y; t = tMax.y; tMax.y = tMax.y + delta.y;+ } else {+ cell.z = cell.z + stepv.z; t = tMax.z; tMax.z = tMax.z + delta.z;+ }+ if (!inWindow(cell)) { break; }+ let idx = wrapIndex(cell);+ if (tags[idx] != hash3(cell)) { continue; }+ let base = idx * WORDS;+ let count = atomicLoad(&grid[base + 3u]);+ if (count < 2u) { continue; }+ let inv = 1.0 / f32(count);+ let facing = (vec3f(f32(atomicLoad(&grid[base + 4u])), f32(atomicLoad(&grid[base + 5u])), f32(atomicLoad(&grid[base + 6u]))) * inv / FACING) * 2.0 - 1.0;+ // A surface is met from the side it faces. One facing away is behind the ray, or the ray skims along it.+ if (dot(normalize(facing), dir) > -0.15) { continue; }+ let radiance = vec3f(f32(atomicLoad(&grid[base])), f32(atomicLoad(&grid[base + 1u])), f32(atomicLoad(&grid[base + 2u]))) * inv / FIXED;+ return radiance * clamp(1.0 - t / range, 0.0, 1.0);+ }+ return vec3f(0.0);+}++@compute @workgroup_size(8, 8)+fn main(@builtin(global_invocation_id) id : vec3u) {+ let lowSize = vec2i(textureDimensions(outTex));+ let p = vec2i(id.xy);+ if (p.x >= lowSize.x || p.y >= lowSize.y) { return; }+ let size = vec2i(textureDimensions(depthTex));+ let div = i32(gi.view.x);+ let t = min(p * div + vec2i(i32(gi.origin.w) % div, (i32(gi.origin.w) / div) % div), size - vec2i(1));+ let depth = textureLoad(depthTex, t, 0);+ let g = textureLoad(gbuffer, t, 0);+ if (depth >= 1.0 || g.a <= 0.7) {+ textureStore(outTex, p, vec4f(0.0));+ return;+ }+ var n = decodeNormal(g);+ let world = worldFromDepth((vec2f(t) + 0.5) / vec2f(size), depth);+ if (dot(n, viewDirection(world)) < 0.0) { n = -n; }+ let origin = world + n * (gi.a.x * gi.b.y);++ let rays = i32(gi.dims.w);+ let r1 = ign(vec2f(p) + vec2f(0.0, f32(gi.origin.w % 16)));+ let r2 = ign(vec2f(p.yx) + vec2f(f32(gi.origin.w % 16) * 5.588238, 0.0));+ var sum = vec3f(0.0);+ for (var k = 0; k < rays; k = k + 1) {+ let u = fract((f32(k) + 0.5) / f32(rays) + r1);+ let v = fract(f32(k) * 0.61803399 + r2);+ sum = sum + trace(origin, cosineDirection(n, u, v));+ }+ var light = sum / f32(rays);+ // What the last frames agreed on, found where this surface was in the last frame's picture.+ if (gi.view.y > 0.5) {+ let prevClip = gi.prevViewProjection * vec4f(world, 1.0);+ if (prevClip.w > 1e-4) {+ let ndc = prevClip.xy / prevClip.w;+ let uv = vec2f(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5);+ if (uv.x >= 0.0 && uv.x <= 1.0 && uv.y >= 0.0 && uv.y <= 1.0) {+ let q = clamp(vec2i(uv * vec2f(lowSize)), vec2i(0), lowSize - vec2i(1));+ light = mix(textureLoad(histTex, q, 0).rgb, light, gi.view.z);+ }+ }+ }+ textureStore(outTex, p, vec4f(light, 1.0));+}+)WGSL";++/// The gathered light over the picture, upsampled with the depth so it does not bleed across an edge.+const char* kApply = R"WGSL(+@group(0) @binding(4) var scene : texture_2d<f32>;+@group(0) @binding(5) var depthTex : texture_depth_2d;+@group(0) @binding(7) var outTex : texture_storage_2d<rgba16float, write>;+@group(0) @binding(8) var giTex : texture_2d<f32>;++@compute @workgroup_size(8, 8)+fn main(@builtin(global_invocation_id) id : vec3u) {+ let size = vec2i(textureDimensions(depthTex));+ let t = vec2i(id.xy);+ if (t.x >= size.x || t.y >= size.y) { return; }+ let src = textureLoad(scene, t, 0);+ let lowSize = vec2i(textureDimensions(giTex));+ let div = i32(gi.view.x);+ let depth = textureLoad(depthTex, t, 0);+ var light = vec3f(0.0);+ if (depth < 1.0) {+ // A depth-weighted blend of the low-resolution texels around the pixel: the ones on the same surface count, the ones+ // across an edge do not. The window is wider than the bilinear four, which is what smooths the rays' noise.+ let centre = vec2i(floor((vec2f(t) + 0.5) / f32(div) - 0.5));+ var total = 0.0;+ var acc = vec3f(0.0);+ for (var dy = -1; dy <= 2; dy = dy + 1) {+ for (var dx = -1; dx <= 2; dx = dx + 1) {+ let q = clamp(centre + vec2i(dx, dy), vec2i(0), lowSize - vec2i(1));+ let qd = textureLoad(depthTex, min(q * div, size - vec2i(1)), 0);+ if (qd >= 1.0) { continue; }+ let near = abs(vec2f(t) / f32(div) - 0.5 - vec2f(q));+ let spatial = max(1.0 - 0.4 * (near.x + near.y), 0.05);+ // Depth is nonlinear: it is the ratio of what is left to the far plane that says how far apart two surfaces are.+ let w = spatial * exp(-abs(log((1.0 - qd + 1e-6) / (1.0 - depth + 1e-6))) * 24.0);+ acc = acc + textureLoad(giTex, q, 0).rgb * w;+ total = total + w;+ }+ }+ if (total > 1e-4) { light = acc / total; }+ }+ // What the gathered light is lit onto wears the surface's own colour (its chroma, kept below white).+ let lum = max(dot(src.rgb, vec3f(0.2126, 0.7152, 0.0722)), 0.02);+ let albedo = mix(vec3f(0.55), clamp(src.rgb / lum, vec3f(0.0), vec3f(1.6)) * 0.55, 0.7);+ var result = src.rgb + albedo * light * gi.a.w;+ if (gi.b.w > 0.5) { result = light * gi.a.w; }+ textureStore(outTex, t, vec4f(result, src.a));+}+)WGSL";++std::string join(const char* a, const char* b, const char* c = "") {+ return std::string(a) + b + c;+}++struct ParamsBlock {+ i32 origin[4];+ u32 dims[4];+ f32 a[4];+ f32 b[4];+ f32 view[4];+ f32 prevViewProjection[16];+};+static_assert(sizeof(ParamsBlock) == 144, "the GI block is five vec4s and a matrix");++} // namespace++VoxelGI::VoxelGI(GfxDevice& device, RenderContext& context) : device_(device), context_(context) {}++VoxelGI::~VoxelGI() {+ release();+}++void VoxelGI::release() {+ for (BufferHandle* b : {&grid_, &tags_, ¶ms_}) {+ if (*b != BufferHandle::Invalid) device_.deleteBuffer(*b);+ *b = BufferHandle::Invalid;+ }+ for (TextureHandle& t : history_) {+ if (t != TextureHandle::Invalid) device_.deleteTexture(t);+ t = TextureHandle::Invalid;+ }+ historyWidth_ = historyHeight_ = 0;+ historyValid_ = false;+ for (ComputeHandle* c : {&maintain_, &inject_, &gather_, &apply_}) {+ if (*c != ComputeHandle::Invalid) device_.deleteCompute(*c);+ *c = ComputeHandle::Invalid;+ }+ failed_ = false;+}++bool VoxelGI::ensure() {+ if (failed_) return false;+ if (!device_.supportsCompute()) return false;+ const u32 voxels = kDimsX * kDimsY * kDimsZ;+ if (grid_ == BufferHandle::Invalid) {+ grid_ = device_.createBuffer({GfxBufferUsage::Storage, voxels * kVoxelWords * 4u, false}, GfxContent::transient(), nullptr);+ tags_ = device_.createBuffer({GfxBufferUsage::Storage, voxels * 4u, false}, GfxContent::transient(), nullptr);+ params_ = device_.createBuffer({GfxBufferUsage::Uniform, sizeof(ParamsBlock), true}, GfxContent::transient(), nullptr);+ }+ if (grid_ == BufferHandle::Invalid || tags_ == BufferHandle::Invalid || params_ == BufferHandle::Invalid) {+ ES_LOG_ERROR("VoxelGI: could not allocate the voxel cache");+ failed_ = true;+ return false;+ }+ const auto make = [&](ComputeHandle& handle, const std::string& source, const char* name) {+ if (handle != ComputeHandle::Invalid) return true;+ std::string log;+ handle = device_.createCompute(GfxComputeSource{GfxShaderLanguage::WGSL, source.c_str(), "main"}, &log);+ if (handle == ComputeHandle::Invalid) {+ ES_LOG_ERROR("VoxelGI: the {} kernel did not compile: {}", name, log);+ failed_ = true;+ return false;+ }+ return true;+ };+ return make(maintain_, join(kCommon, kMaintain), "maintain") && make(inject_, join(kCommon, kSurface, kInject), "inject")+ && make(gather_, join(kCommon, kSurface, kGather), "gather") && make(apply_, join(kCommon, kApply), "apply");+}++bool VoxelGI::ensureHistory(u32 width, u32 height) {+ if (history_[0] != TextureHandle::Invalid && historyWidth_ == width && historyHeight_ == height) return true;+ for (TextureHandle& t : history_) {+ if (t != TextureHandle::Invalid) device_.deleteTexture(t);+ t = TextureHandle::Invalid;+ }+ historyValid_ = false;+ TextureDesc desc;+ desc.width = width;+ desc.height = height;+ desc.format = GfxPixelFormat::RGBA16F;+ desc.minFilter = desc.magFilter = TextureFilter::Nearest;+ desc.storage = true;+ for (TextureHandle& t : history_) t = device_.createTexture(desc, GfxContent::transient(), nullptr);+ if (history_[0] == TextureHandle::Invalid || history_[1] == TextureHandle::Invalid) {+ ES_LOG_ERROR("VoxelGI: could not allocate the gather history");+ return false;+ }+ historyWidth_ = width;+ historyHeight_ = height;+ return true;+}++void VoxelGI::writeParams(const VoxelGIParams& p) {+ // The window is centred on the eye, snapped to whole cells so the grid does not swim as the camera moves.+ const glm::vec4 eye = cameraFromViewProjection(context_.viewProjection());+ const f32 vs = std::max(p.voxelSize, 0.001f);+ ParamsBlock block{};+ block.origin[0] = static_cast<i32>(std::floor(eye.x / vs)) - static_cast<i32>(kDimsX / 2);+ block.origin[1] = static_cast<i32>(std::floor(eye.y / vs)) - static_cast<i32>(kDimsY / 2);+ block.origin[2] = static_cast<i32>(std::floor(eye.z / vs)) - static_cast<i32>(kDimsZ / 2);+ block.origin[3] = static_cast<i32>(frame_);+ block.dims[0] = kDimsX;+ block.dims[1] = kDimsY;+ block.dims[2] = kDimsZ;+ block.dims[3] = std::clamp(p.rays, 1u, 64u);+ block.a[0] = vs;+ block.a[1] = std::clamp(p.decay, 0.0f, 0.999f);+ block.a[2] = std::max(p.maxBounce, 0.01f);+ block.a[3] = p.intensity;+ block.b[0] = std::clamp(p.range, 1.0f, 90.0f);+ block.b[1] = std::max(p.bias, 0.0f);+ block.b[3] = p.debug;+ block.view[0] = static_cast<f32>(std::clamp(p.divisor, 1u, 4u));+ // A change of voxel size makes every voxel a different place, so what was gathered under the old one is not history.+ if (previousVoxelSize_ != vs) historyValid_ = false;+ previousVoxelSize_ = vs;+ block.view[1] = historyValid_ ? 1.0f : 0.0f;+ block.view[2] = std::clamp(p.blend, 0.01f, 1.0f);+ std::copy(previousViewProjection_, previousViewProjection_ + 16, block.prevViewProjection);+ const glm::mat4& vp = context_.viewProjection();+ for (int c = 0; c < 4; ++c) for (int r = 0; r < 4; ++r) previousViewProjection_[c * 4 + r] = vp[c][r];+ device_.updateBuffer(params_, 0, &block, sizeof(block));+ origin_[0] = block.origin[0];+ origin_[1] = block.origin[1];+ origin_[2] = block.origin[2];+ ++frame_;+}++rg::ResourceId VoxelGI::declare(rg::RenderGraph& graph, rg::ResourceId input, rg::ResourceId scene, const VoxelGIParams& params,+ u32 width, u32 height) {+ if (!ensure() || input == rg::kNoResource || scene == rg::kNoResource) return input;++ // The passes are tied together by what each writes, since the graph only runs what the image reaches. The first two write a+ // single texel nobody samples: their work is in the grid, and the texel is what says "this ran before that".+ rg::TargetDesc token;+ token.width = 1;+ token.height = 1;+ token.format = GfxPixelFormat::RGBA8;+ token.storage = true;+ token.linearFilter = false;++ const u32 divisor = std::clamp(params.divisor, 1u, 4u);+ const VoxelGIParams p = params;+ // The size the graph gives a target at 1 / divisor: the history has to be exactly that.+ const f32 scale = 1.0f / static_cast<f32>(divisor);+ const u32 lowWidth = std::max(1u, static_cast<u32>(static_cast<f32>(width) * scale));+ const u32 lowHeight = std::max(1u, static_cast<u32>(static_cast<f32>(height) * scale));+ if (!ensureHistory(lowWidth, lowHeight)) return input;+ // This frame writes one of the two and reads the other.+ const u32 cur = frame_ & 1u;++ const rg::ResourceId maintainDone = graph.createTarget(token);+ {+ rg::PassDesc pass;+ pass.name = "gi_maintain";+ pass.compute = true;+ pass.write = maintainDone;+ pass.dependencies = {scene};+ pass.execute = [this, p](const rg::PassContext&) {+ writeParams(p);+ const GfxComputeBinding bindings[4] = {+ GfxComputeBinding::uniform(0, context_.frameUbo()),+ GfxComputeBinding::uniform(1, params_),+ GfxComputeBinding::storage(2, grid_),+ GfxComputeBinding::storage(3, tags_),+ };+ device_.dispatchCompute(maintain_, bindings, 4, (kDimsX * kDimsY * kDimsZ + 63) / 64);+ };+ graph.addPass(std::move(pass));+ }++ const rg::ResourceId injectDone = graph.createTarget(token);+ {+ rg::PassDesc pass;+ pass.name = "gi_inject";+ pass.compute = true;+ pass.reads = {input};+ pass.dependencies = {scene, maintainDone};+ pass.write = injectDone;+ pass.execute = [this, &graph, scene, width, height](const rg::PassContext& ctx) {+ const TextureHandle depth = graph.depthTextureOf(scene);+ const TextureHandle gbuffer = graph.gbufferTextureOf(scene);+ if (depth == TextureHandle::Invalid || gbuffer == TextureHandle::Invalid) return;+ const GfxComputeBinding bindings[7] = {+ GfxComputeBinding::uniform(0, context_.frameUbo()),+ GfxComputeBinding::uniform(1, params_),+ GfxComputeBinding::storage(2, grid_),+ GfxComputeBinding::storage(3, tags_),+ GfxComputeBinding::sampled(4, ctx.input(0)),+ GfxComputeBinding::depth(5, depth),+ GfxComputeBinding::sampled(6, gbuffer),+ };+ device_.dispatchCompute(inject_, bindings, 7, (width / 2 + 7) / 8, (height / 2 + 7) / 8);+ };+ graph.addPass(std::move(pass));+ }++ const rg::ResourceId gathered = graph.createTarget(token);+ {+ rg::PassDesc pass;+ pass.name = "gi_gather";+ pass.compute = true;+ pass.dependencies = {scene, injectDone};+ pass.write = gathered;+ pass.execute = [this, &graph, scene, cur](const rg::PassContext&) {+ const TextureHandle depth = graph.depthTextureOf(scene);+ const TextureHandle gbuffer = graph.gbufferTextureOf(scene);+ if (depth == TextureHandle::Invalid || gbuffer == TextureHandle::Invalid) return;+ const GfxComputeBinding bindings[8] = {+ GfxComputeBinding::uniform(0, context_.frameUbo()),+ GfxComputeBinding::uniform(1, params_),+ GfxComputeBinding::storage(2, grid_),+ GfxComputeBinding::storage(3, tags_),+ GfxComputeBinding::depth(5, depth),+ GfxComputeBinding::sampled(6, gbuffer),+ GfxComputeBinding::storageTexture(7, history_[cur]),+ GfxComputeBinding::sampled(9, history_[cur ^ 1u]),+ };+ device_.dispatchCompute(gather_, bindings, 8, (historyWidth_ + 7) / 8, (historyHeight_ + 7) / 8);+ historyValid_ = true;+ };+ graph.addPass(std::move(pass));+ }++ rg::TargetDesc full;+ full.format = GfxPixelFormat::RGBA16F;+ full.storage = true;+ full.linearFilter = true;+ const rg::ResourceId result = graph.createTarget(full);+ {+ rg::PassDesc pass;+ pass.name = "gi_apply";+ pass.compute = true;+ pass.reads = {input};+ pass.dependencies = {scene, gathered};+ pass.write = result;+ pass.execute = [this, &graph, scene, cur](const rg::PassContext& ctx) {+ const TextureHandle depth = graph.depthTextureOf(scene);+ if (depth == TextureHandle::Invalid || ctx.output == TextureHandle::Invalid) return;+ const GfxComputeBinding bindings[8] = {+ GfxComputeBinding::uniform(0, context_.frameUbo()),+ GfxComputeBinding::uniform(1, params_),+ GfxComputeBinding::storage(2, grid_),+ GfxComputeBinding::storage(3, tags_),+ GfxComputeBinding::sampled(4, ctx.input(0)),+ GfxComputeBinding::depth(5, depth),+ GfxComputeBinding::storageTexture(7, ctx.output),+ GfxComputeBinding::sampled(8, history_[cur]),+ };+ device_.dispatchCompute(apply_, bindings, 8, (ctx.width + 7) / 8, (ctx.height + 7) / 8);+ };+ graph.addPass(std::move(pass));+ }+ return result;+}++} // namespace esenginediff --git a/src/esengine/renderer/frame/VoxelGI.hpp b/src/esengine/renderer/frame/VoxelGI.hppnew file mode 100644index 000000000..308935302--- /dev/null+++ b/src/esengine/renderer/frame/VoxelGI.hpp@@ -0,0 +1,115 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file VoxelGI.hpp+ * @brief Diffuse global illumination from a world-space voxel cache, run as compute passes (WebGPU).+ *+ * @details What a frame can see of the scene, remembered in the world. A grid of voxels follows the camera (addressed+ * toroidally, so it does not move with it): every frame the lit picture is injected into the voxels its visible+ * surfaces stand in — their average radiance and the way they face — and what was there before fades. Light then+ * is gathered the way a path tracer would, but through the grid: from each visible surface a few cosine-weighted+ * rays are marched cell by cell (a DDA walk), and where one meets a voxel that faces it, that voxel's radiance is+ * the light it brings. Because the grid keeps what it has seen, a red wall that has left the screen still tints the+ * floor beside it, and a surface lit only by another is lit — which a screen-space pass cannot do.+ *+ * It is not a trace of the scene's geometry: the grid holds the surfaces the camera has seen, at the voxel's size,+ * so a surface it has not seen sends no light and a thin one is as thick as a cell. That is the price of needing no+ * mesh voxelization, and the reason the result is called what it is.+ *+ * Four compute passes per frame, declared into the frame's render graph: maintain (age the grid, hand cells that+ * moved into the window back as empty), inject (the picture into the grid), gather (rays through the grid, at a+ * fraction of the picture's size) and apply (the gathered light over the picture, upsampled with the depth).+ * Needs a device with a compute stage; elsewhere the pass is left out of the chain.+ */+#pragma once++#include "../../core/Types.hpp"+#include "../graph/RenderGraph.hpp"+#include "../rhi/GfxEnums.hpp"++#include <string>+#include <unordered_map>++namespace esengine {++class GfxDevice;+class RenderContext;++/** What the effect is asked for, from the pass's uniforms. */+struct VoxelGIParams {+ /// How much of the gathered light is added over the picture.+ f32 intensity = 1.0f;+ /// Rays per surface point.+ u32 rays = 8;+ /// World size of one voxel.+ f32 voxelSize = 40.0f;+ /// How far a ray looks, in voxels.+ f32 range = 48.0f;+ /// The brightest a surface is taken to give out, so one lamp does not light the street from a single pixel.+ f32 maxBounce = 3.0f;+ /// What the grid keeps each frame of what it held: 0 forgets at once, 1 never.+ f32 decay = 0.93f;+ /// How far off its own voxel a ray starts, in voxels, so a surface is not lit by itself.+ f32 bias = 1.2f;+ /// The gather runs at 1 / this of the picture's size per axis.+ u32 divisor = 2;+ /// 1 shows the gathered light alone.+ f32 debug = 0.0f;+ /// How much of each frame's gather goes into the picture the next frame builds on (the rest is what the last frames agreed on).+ f32 blend = 0.2f;+};++class VoxelGI {+public:+ VoxelGI(GfxDevice& device, RenderContext& context);+ ~VoxelGI();++ VoxelGI(const VoxelGI&) = delete;+ VoxelGI& operator=(const VoxelGI&) = delete;++ /**+ * @brief Declares the four passes onto @p graph, reading @p input and the scene's depth and G-buffer.+ * @param scene The scene's resource, kept alive through the passes (they read its depth and G-buffer).+ * @return The resource holding @p input with the gathered light added; @p input itself when there is nothing to do.+ */+ rg::ResourceId declare(rg::RenderGraph& graph, rg::ResourceId input, rg::ResourceId scene, const VoxelGIParams& params,+ u32 width, u32 height);++ /** Frees the device objects (the grid, the programs). */+ void release();++ /** The grid's size in voxels per axis: the window the cache covers is this many voxels across. */+ static constexpr u32 kDimsX = 128;+ static constexpr u32 kDimsY = 64;+ static constexpr u32 kDimsZ = 128;++private:+ struct Programs;+ bool ensure();+ /// The two pictures the gathers accumulate into, made again when their size changes.+ bool ensureHistory(u32 width, u32 height);+ void writeParams(const VoxelGIParams& p);++ GfxDevice& device_;+ RenderContext& context_;+ BufferHandle grid_ = BufferHandle::Invalid;+ BufferHandle tags_ = BufferHandle::Invalid;+ BufferHandle params_ = BufferHandle::Invalid;+ ComputeHandle maintain_ = ComputeHandle::Invalid;+ ComputeHandle inject_ = ComputeHandle::Invalid;+ ComputeHandle gather_ = ComputeHandle::Invalid;+ ComputeHandle apply_ = ComputeHandle::Invalid;+ bool failed_ = false;+ u32 frame_ = 0;+ TextureHandle history_[2] = {TextureHandle::Invalid, TextureHandle::Invalid};+ u32 historyWidth_ = 0;+ u32 historyHeight_ = 0;+ /// Whether the picture the other one holds is a gather of the view the last frame saw, to blend with.+ bool historyValid_ = false;+ f32 previousViewProjection_[16] = {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1};+ f32 previousVoxelSize_ = 0.0f;+ /// The window's origin, in cells, as of the frame being drawn.+ i32 origin_[3] = {0, 0, 0};+};++} // namespace esenginediff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex af12d5df7..fa6b604e0 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -2493,6 +2493,8 @@ export const NOTED = {{"note":"Skinned meshes are followed by their bones, from the palette they were last drawn with."},"feat(render): fog follows the scene's sun and can walk the air against its shadow map":{"note":"Height Fog and Light Shafts follow the scene's own sun, and Height Fog can carry the sun's shadows as volumetric light."},+ "feat(render): Voxel GI lights from a world-space cache of what the camera has seen, run as compute passes":+ {"note":"Voxel GI adds diffuse bounce light from a world-space voxel cache the camera remembers, on WebGPU."},"feat(render): post-process effects can be compute shaders, and Sharpen is one":{"note":"A post-process effect can be a WGSL compute shader over the picture, run on WebGPU and left out on WebGL2, and Sharpen is the first."},"feat(render): the WebGPU device runs compute shaders, and the render graph has compute passes":