Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): probe volumes light each pixel

yhh98eefc186bb9
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 584ade93a..78e2046e4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -25,6 +25,7 @@ published separately; it ships inside the editor.
- **The sky lights the scene.** Sky Atmosphere's **Light the Scene** makes the sky the scene's environment: every Ambient light that names no environment of its own lights with it, its diffuse light from the dome and the clouds and its reflection on metal and wet ground, scaled by the light's colour and intensity as an imported one is. The sky is worked out again as it changes (the sun moving, a weather coming in, clouds drifting), at most once a second; with clouds that takes about 20 ms on a desktop. The sun's disc is left out, since the directional light the sky follows is its direct light. Off by default, so a scene already lit by hand looks as it did.
- **Rooms stay dark under a sky that changes (sky occlusion).** With **Sky occlusion** on in the Lighting settings (`BakedLighting.skyOcclusion`), a probe bake keeps how much of the sky each probe sees instead of the sky itself: the probes hold the lights and their bounces, and the sky the frame has now (a live Sky Atmosphere, a weather, an environment) is added through what each probe sees of it, to the diffuse light and to the sky's reflection. A ball inside a roofed room measured 119 where the baked sky gave it 150, and a ball outside kept its light. A `.esprobes` baked this way carries `skyVisibility`; one without it lights as before. Probes are read per object, as Unity's light probes are: one large mesh spanning indoors and out takes one value.
+- **A probe volume lights each pixel, not each object.** A lit mesh inside a `LightProbeVolume` reads the baked grid where each of its pixels stands, so one large mesh that spans a dark room and the open yard outside it (a merged building, an imported level) is dark inside and lit outside, as in Unity's Adaptive Probe Volumes; before, it took the one value at its origin and was lit the same end to end. The grids ride one half-float atlas the frame builds when a bake changes. A mesh with lightmap UVs that has no bake still reads its volume per object, as before.
- **Defaults follow the project's units.** Every default that is a length, a speed or an acceleration — a character's capsule, a first-person walk and jump, a view model's near plane, a nav volume's cells, a perceiver's range, a particle's size and speed — is written at 100 units a metre and scaled by the project's units per metre (Project Settings → Physics → 3D). A centimetre project gets exactly the defaults it had; a metre project (an imported glTF scene) now gets metres instead of values 100 times too big. A value a scene states is never scaled. A component declares its own with `worldUnitFields`. Post-process effects follow too: a length a volume leaves unset (rain and snow shelter range and puddle size, SSR distance and thickness, GTAO / SSAO / SSGI radius, contact-shadow length, depth-of-field focus, voxel GI cell, height-fog start and range) takes its default in the project's units, and fog density and height falloff — rates per length — by the inverse; the inspector offers ranges in those units and writes an added effect's defaults in them. A metre project's rain used to shelter within 3 km.
- **`Particle.emitAt(emitter, count, position, { normal, velocity })` spawns a burst where something happened, now** — an impact's sparks, a muzzle's flash — from the emitter's own settings, its shape turned onto the normal, without moving the emitter or replaying it (Unity's `ParticleSystem.Emit`). CPU-simulated emitters.
- **A first-person player has its own jump feel.** `FirstPersonController.gravityScale` (the player's gravity as a share of the world's), `coyoteTime` (a jump just after walking off an edge still takes off) and `jumpBuffer` (a jump pressed just before landing takes off as it lands).
diff --git a/fixtures/scenes/probe-per-pixel.esscene b/fixtures/scenes/probe-per-pixel.esscene
new file mode 100644
index 000000000..bcc0e81ef
--- /dev/null
+++ b/fixtures/scenes/probe-per-pixel.esscene
@@ -0,0 +1,104 @@
+{
+ "version": 4,
+ "name": "ProbePerPixel",
+ "entities": [
+ {
+ "id": 0,
+ "name": "Camera",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 10
+ }
+ }
+ },
+ {
+ "type": "Camera",
+ "data": {
+ "projectionType": 1,
+ "orthoSize": 300,
+ "isActive": true,
+ "priority": 0
+ }
+ }
+ ]
+ },
+ {
+ "id": 1,
+ "name": "Probes",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ }
+ }
+ },
+ {
+ "type": "LightProbeVolume",
+ "data": {
+ "probes": "/scenes/probe-ends.esprobes",
+ "halfExtents": {
+ "x": 120,
+ "y": 400,
+ "z": 400
+ },
+ "enabled": true
+ }
+ }
+ ]
+ },
+ {
+ "id": 3,
+ "name": "Wide",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ },
+ "scale": {
+ "x": 0.6,
+ "y": 0.2,
+ "z": 1
+ }
+ }
+ },
+ {
+ "type": "MeshRenderer",
+ "data": {
+ "color": {
+ "r": 1,
+ "g": 1,
+ "b": 1,
+ "a": 1
+ },
+ "layer": 0,
+ "enabled": true,
+ "lit": true,
+ "mesh": "/scenes/lit-triangles.esmesh"
+ }
+ }
+ ]
+ }
+ ]
+}
diff --git a/fixtures/scenes/probe-per-pixel.textures.json b/fixtures/scenes/probe-per-pixel.textures.json
new file mode 100644
index 000000000..0967ef424
--- /dev/null
+++ b/fixtures/scenes/probe-per-pixel.textures.json
@@ -0,0 +1 @@
+{}
diff --git a/sdk/src/render/builtinShaders.ts b/sdk/src/render/builtinShaders.ts
index e786ee4f6..a49954884 100644
--- a/sdk/src/render/builtinShaders.ts
+++ b/sdk/src/render/builtinShaders.ts
@@ -967,6 +967,7 @@ void main() {
if (gl > glossLevel) { glossLevel = gl; glossTint = g / max(gl, 1e-4); }
}
float lightAmount = clamp(max(lit.r, max(lit.g, lit.b)), 0.0, 1.0);
+ g_probePos = P;
vec3 ambient = indirectIrradiance(N) * ao;
// The shade colour fills in where the steps leave off, so the cut between lit and shaded is a change of colour and not only of brightness.
vec3 colour = base.rgb * (ambient + lit + u_shadeColor.rgb * (1.0 - lightAmount));
@@ -1044,6 +1045,7 @@ fn packNormal(n : vec3f) -> vec2f {
if (gl > glossLevel) { glossLevel = gl; glossTint = term.gloss / max(gl, 1e-4); }
}
let lightAmount = clamp(max(lit.r, max(lit.g, lit.b)), 0.0, 1.0);
+ g_probePos = P;
let ambient = indirectIrradiance(N) * ao;
var colour = base.rgb * (ambient + lit + mc.u_shadeColor.rgb * (1.0 - lightAmount));
let spec = smoothstep(mc.u_specThreshold - mc.u_specSoftness, mc.u_specThreshold + mc.u_specSoftness, glossLevel);
diff --git a/src/esengine/bindings/RendererBindings.cpp b/src/esengine/bindings/RendererBindings.cpp
index c25b4272d..d38986a9e 100644
--- a/src/esengine/bindings/RendererBindings.cpp
+++ b/src/esengine/bindings/RendererBindings.cpp
@@ -693,6 +693,7 @@ void probe_volume_setSkyVisibility(u32 volumeHandle, uintptr_t visibilityPtr) {
const f32* v = boundarySpan<f32>(visibilityPtr, want, "probe_volume_setSkyVisibility.visibility");
if (!v) return;
grid->skyVisibility.assign(v, v + want);
+ grid->revision = ProbeVolume::nextRevision();
}
void probe_volume_release(u32 volumeHandle) {
diff --git a/src/esengine/renderer/draw/BatchBuilder.cpp b/src/esengine/renderer/draw/BatchBuilder.cpp
index 562fd91ca..c6b5ab42b 100644
--- a/src/esengine/renderer/draw/BatchBuilder.cpp
+++ b/src/esengine/renderer/draw/BatchBuilder.cpp
@@ -175,6 +175,10 @@ void pushBatchDraw(DrawList& drawList, const ClipState& clips,
for (u8 slot = cmd.texture_count; slot < 5; ++slot) cmd.texture_ids[slot] = key.textureId;
cmd.texture_ids[5] = key.lightmapTextureId;
cmd.texture_count = 6;
+ } else if (key.probeGridTextureId != 0 && cmd.texture_count >= 1 && key.layoutId != LayoutId::Batch) {
+ for (u8 slot = cmd.texture_count; slot < 5; ++slot) cmd.texture_ids[slot] = key.textureId;
+ cmd.texture_ids[5] = key.probeGridTextureId;
+ cmd.texture_count = 6;
}
cmd.entity = key.entity;
cmd.type = key.type;
diff --git a/src/esengine/renderer/draw/BatchBuilder.hpp b/src/esengine/renderer/draw/BatchBuilder.hpp
index 0b5230a19..6a5fc136e 100644
--- a/src/esengine/renderer/draw/BatchBuilder.hpp
+++ b/src/esengine/renderer/draw/BatchBuilder.hpp
@@ -156,6 +156,9 @@ struct BatchDrawKey {
/// Part of the key and not only of the command: two objects sharing a mesh but
/// baked into different atlases are two draws.
u32 lightmapTextureId = 0;
+ /// The frame's probe atlas, for a draw that reads its probe volume per pixel. On the lightmap's unit,
+ /// which such a draw never needs: a draw reading a bake asks no volume for its light.
+ u32 probeGridTextureId = 0;
};
/// Check before formatting caller-owned vertices, using the same group layer,
diff --git a/src/esengine/renderer/frame/RenderFrame.cpp b/src/esengine/renderer/frame/RenderFrame.cpp
index 75853eead..1e1c107a6 100644
--- a/src/esengine/renderer/frame/RenderFrame.cpp
+++ b/src/esengine/renderer/frame/RenderFrame.cpp
@@ -1539,6 +1539,14 @@ void RenderFrame::collectProbes(ecs::Registry& registry) {
const glm::vec3 half = glm::abs(volume.halfExtents);
probes.add(centre - half, centre + half, grid);
}
+
+ // 2048 is the side WebGL2 guarantees; a grid past it keeps the per-object read rather than failing.
+ if (probes.layoutAtlas(2048) && probes.atlasWidth() > 0) {
+ probes.fillAtlas(probe_atlas_texels_);
+ probe_atlas_ = Texture::createRaw(device_, GfxContent::retained(), probes.atlasWidth(), probes.atlasHeight(),
+ probe_atlas_texels_.data(), TextureFormat::RGBA16F, false);
+ }
+ probes.setAtlasTexture(probes.atlasWidth() > 0 && probe_atlas_ ? probe_atlas_->getId() : 0);
}
/**
diff --git a/src/esengine/renderer/frame/RenderFrame.hpp b/src/esengine/renderer/frame/RenderFrame.hpp
index ded6b8db8..2b3bae144 100644
--- a/src/esengine/renderer/frame/RenderFrame.hpp
+++ b/src/esengine/renderer/frame/RenderFrame.hpp
@@ -24,6 +24,7 @@
#endif
#include "./FrameCapture.hpp"
#include "../rhi/GfxDevice.hpp"
+#include "../rhi/Texture.hpp"
#include "../rhi/TransientBufferPool.hpp"
#include "../draw/DrawList.hpp"
#include "../draw/RetainedText.hpp"
@@ -786,6 +787,10 @@ private:
};
std::unordered_map<u32, PlanarCapture> planar_captures_;
TransientBufferPool planar_pool_{device_};
+ /// The probe volumes' grids, packed for the per-pixel read (ProbeStore::layoutAtlas). Remade only when a grid
+ /// or the set of grids changes, so a still scene uploads it once.
+ Unique<Texture> probe_atlas_;
+ std::vector<u16> probe_atlas_texels_;
DrawList planar_list_;
u64 planar_frame_ = 0;
f32 planar_elapsed_ = 0;
diff --git a/src/esengine/renderer/plugins/MeshPlugin.cpp b/src/esengine/renderer/plugins/MeshPlugin.cpp
index 8ea1a9718..899a0b3c5 100644
--- a/src/esengine/renderer/plugins/MeshPlugin.cpp
+++ b/src/esengine/renderer/plugins/MeshPlugin.cpp
@@ -1108,8 +1108,12 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
// like anything else. At the ENTITY's position: bones place a skin.
if (!depthPass) {
ProbeConstants probe;
+ // Per pixel where the program can read the atlas: a mesh with lightmap UVs is compiled with
+ // the bake's sampler on that unit, so it keeps the one value its position reads.
+ const ProbeStore& store = ctx.render_context.probes();
const bool volume = key.lightmapTextureId == 0
- && ctx.render_context.probes().sample(position, probe);
+ && store.sample(position, probe, !resident->hasLightmapUV);
+ if (volume && probe.irradiance[0].w > 1.5f) key.probeGridTextureId = store.atlasTexture();
const u32 slot = ctx.render_context.reflections().slotAt(position);
probe.reflection.x = static_cast<f32>(slot);
if (volume || slot != 0) key.probeIndex = draw_list.addProbe(probe);
diff --git a/src/esengine/renderer/rhi/Shader.cpp b/src/esengine/renderer/rhi/Shader.cpp
index b19d563da..508abf414 100644
--- a/src/esengine/renderer/rhi/Shader.cpp
+++ b/src/esengine/renderer/rhi/Shader.cpp
@@ -311,6 +311,7 @@ void Shader::finishLink() {
{SHADOW_2D_SAMPLER, SHADOW_2D_TEXTURE_UNIT},
{SHAPE_2D_SAMPLER, SHAPE_2D_TEXTURE_UNIT},
{LIGHTMAP_SAMPLER, LIGHTMAP_TEXTURE_UNIT},
+ {PROBE_GRID_SAMPLER, LIGHTMAP_TEXTURE_UNIT},
{MORPH_DELTA_SAMPLER, MORPH_DELTA_TEXTURE_UNIT},
{INSTANCE_DATA_SAMPLER, MESH_INSTANCE_TEXTURE_UNIT},
};
diff --git a/src/esengine/renderer/store/EngineTextureUnits.hpp b/src/esengine/renderer/store/EngineTextureUnits.hpp
index b68dd084a..887721dd8 100644
--- a/src/esengine/renderer/store/EngineTextureUnits.hpp
+++ b/src/esengine/renderer/store/EngineTextureUnits.hpp
@@ -43,6 +43,13 @@ inline constexpr u32 MORPH_DELTA_TEXTURE_UNIT = 4;
inline constexpr const char* LIGHTMAP_SAMPLER = "u_lightmap";
inline constexpr u32 LIGHTMAP_TEXTURE_UNIT = 5;
+/**
+ * @brief The frame's probe atlas, read per pixel, on the bake's unit.
+ * @details A draw that reads a bake asks no volume for its light, so the two never meet in one draw; a mesh
+ * variant compiled with the bake's sampler declares no atlas at all and reads its volume per object.
+ */
+inline constexpr const char* PROBE_GRID_SAMPLER = "u_probeGrid";
+
/**
* @brief The 2D light-shape mask, and the unit below the shadow mask's.
* @details The same target shape carrying the other half of "how much of this light is
diff --git a/src/esengine/renderer/store/ProbeStore.hpp b/src/esengine/renderer/store/ProbeStore.hpp
index dc510b370..a4f5c812d 100644
--- a/src/esengine/renderer/store/ProbeStore.hpp
+++ b/src/esengine/renderer/store/ProbeStore.hpp
@@ -8,13 +8,21 @@
* one reads. Filled during collection and read during collection — a
* volume is borrowed, not copied, so nothing here outlives the frame
* that gathered it.
+ *
+ * Two ways a draw reads one. Per object: the nine coefficients where the
+ * draw stands, read here on the CPU — one value for the whole mesh. Per
+ * pixel: the grid itself, packed into the frame's probe atlas, and the
+ * draw's run says where in it — so one merged mesh spanning a shed and
+ * the yard outside is dark under the roof and lit in the open.
*/
#pragma once
+#include "../../core/Half.hpp"
#include "../../core/Types.hpp"
#include "../../resource/ProbeVolume.hpp"
#include "./ProbeConstants.hpp"
+#include <algorithm>
#include <cmath>
#include <glm/glm.hpp>
#include <vector>
@@ -29,6 +37,9 @@ public:
glm::vec3 min{0.0f};
glm::vec3 max{0.0f};
const ProbeVolume* grid = nullptr;
+ /// Where its grid sits in the atlas: the texel its coefficient 0, probe (0,0,0) is at. -1 = not placed.
+ i32 atlasX = -1;
+ i32 atlasY = -1;
};
void clear() { volumes_.clear(); }
@@ -51,7 +62,7 @@ public:
* volumes overlap the SMALLER wins: a room inside a hall was baked to
* say something the hall's spacing could not.
*/
- bool sample(const glm::vec3& p, ProbeConstants& out) const {
+ bool sample(const glm::vec3& p, ProbeConstants& out, bool perPixel = false) const {
const Volume* best = nullptr;
f32 bestSize = 0.0f;
for (const Volume& v : volumes_) {
@@ -64,6 +75,10 @@ public:
if (!best) return false;
const ProbeVolume& g = *best->grid;
+ if (perPixel && best->atlasX >= 0 && atlas_texture_ != 0) {
+ writeGridRun(*best, out);
+ return true;
+ }
i32 low[3] = {0, 0, 0};
i32 high[3] = {0, 0, 0};
f32 frac[3] = {0.0f, 0.0f, 0.0f};
@@ -111,8 +126,116 @@ public:
return true;
}
+ /**
+ * @brief Places every volume's grid in the atlas. True when the atlas's texels changed and want uploading.
+ *
+ * @details A grid is a block nine coefficient tiles wide (each resX x resY, one per coefficient) and resZ
+ * slices tall, the blocks stacked down the atlas. Within a tile the hardware filters x and y and never
+ * across a tile's edge (a read sits between the first and last probe's texel centres), and the shader
+ * blends the two slices around its z. A grid that would not fit @p maxSize keeps the per-object read.
+ * Two volumes reading the same grid share its block.
+ */
+ bool layoutAtlas(u32 maxSize) {
+ std::vector<u64> key;
+ key.reserve(volumes_.size());
+ u32 width = 0;
+ u32 height = 0;
+ for (usize i = 0; i < volumes_.size(); ++i) {
+ Volume& v = volumes_[i];
+ v.atlasX = -1;
+ v.atlasY = -1;
+ for (usize j = 0; j < i; ++j) {
+ if (volumes_[j].grid == v.grid) { v.atlasX = volumes_[j].atlasX; v.atlasY = volumes_[j].atlasY; break; }
+ }
+ if (v.atlasX >= 0) continue;
+ const glm::ivec3 r = v.grid->resolution;
+ const u32 w = 9u * static_cast<u32>(r.x);
+ const u32 h = static_cast<u32>(r.y) * static_cast<u32>(r.z);
+ if (w > maxSize || height + h > maxSize) continue;
+ v.atlasX = 0;
+ v.atlasY = static_cast<i32>(height);
+ height += h;
+ width = std::max(width, w);
+ key.push_back(v.grid->revision);
+ key.push_back(static_cast<u64>(v.atlasY));
+ }
+ const bool changed = key != atlas_key_ || width != atlas_width_ || height != atlas_height_;
+ atlas_key_ = std::move(key);
+ atlas_width_ = width;
+ atlas_height_ = height;
+ return changed;
+ }
+
+ u32 atlasWidth() const { return atlas_width_; }
+ u32 atlasHeight() const { return atlas_height_; }
+
+ /** @brief The atlas's texels, RGBA half floats: a coefficient's rgb, and the sky's visibility in a (0 for a grid
+ * baked without it). Laid out as layoutAtlas placed them. */
+ void fillAtlas(std::vector<u16>& out) const {
+ out.assign(static_cast<usize>(atlas_width_) * atlas_height_ * 4, 0);
+ for (usize i = 0; i < volumes_.size(); ++i) {
+ const Volume& v = volumes_[i];
+ if (v.atlasX < 0) continue;
+ bool repeat = false;
+ for (usize j = 0; j < i; ++j) repeat = repeat || volumes_[j].grid == v.grid;
+ if (repeat) continue;
+ const ProbeVolume& g = *v.grid;
+ const i32 rx = g.resolution.x;
+ const i32 ry = g.resolution.y;
+ const i32 rz = g.resolution.z;
+ const bool sky = g.hasSkyVisibility();
+ for (i32 z = 0; z < rz; ++z) {
+ for (i32 y = 0; y < ry; ++y) {
+ for (i32 x = 0; x < rx; ++x) {
+ const usize probe = static_cast<usize>(x + y * rx + z * rx * ry) * 9;
+ const usize row = static_cast<usize>(v.atlasY + z * ry + y);
+ for (i32 c = 0; c < 9; ++c) {
+ const usize col = static_cast<usize>(v.atlasX + c * rx + x);
+ u16* t = &out[(row * atlas_width_ + col) * 4];
+ const glm::vec3& sh = g.irradiance[probe + static_cast<usize>(c)];
+ t[0] = packHalf(sh.x);
+ t[1] = packHalf(sh.y);
+ t[2] = packHalf(sh.z);
+ t[3] = packHalf(sky ? g.skyVisibility[probe + static_cast<usize>(c)] : 0.0f);
+ }
+ }
+ }
+ }
+ }
+ }
+
+ /** @brief The atlas texture this frame's per-pixel runs point into; 0 = none, and every draw reads per object. */
+ void setAtlasTexture(u32 id) { atlas_texture_ = id; }
+ u32 atlasTexture() const { return atlas_texture_; }
+
private:
+ /**
+ * @brief A run that says where the grid is rather than what it holds; the shaders' probeGridIrradiance reads it.
+ * @details [0].w = 2 marks it. [1] = the box's min and resX, [2] = probes per world unit along each axis (resolution
+ * less one over the box's size; 0 on a flat axis) and resY, [3] = the block's first texel and the atlas's
+ * inverse size, [4].x = resZ. The reflection column and the sky flag stay where a per-object run has them.
+ */
+ void writeGridRun(const Volume& v, ProbeConstants& out) const {
+ const ProbeVolume& g = *v.grid;
+ const glm::vec3 e = v.max - v.min;
+ const glm::vec3 r = glm::vec3(g.resolution);
+ out = ProbeConstants{};
+ out.irradiance[0].w = 2.0f;
+ out.irradiance[1] = glm::vec4(v.min, r.x);
+ out.irradiance[2] = glm::vec4(r.x > 1.0f ? (r.x - 1.0f) / e.x : 0.0f,
+ r.y > 1.0f ? (r.y - 1.0f) / e.y : 0.0f,
+ r.z > 1.0f ? (r.z - 1.0f) / e.z : 0.0f, r.y);
+ out.irradiance[3] = glm::vec4(static_cast<f32>(v.atlasX), static_cast<f32>(v.atlasY),
+ 1.0f / static_cast<f32>(atlas_width_), 1.0f / static_cast<f32>(atlas_height_));
+ out.irradiance[4] = glm::vec4(r.z, 0.0f, 0.0f, 0.0f);
+ if (g.hasSkyVisibility()) out.reflection.z = 1.0f;
+ }
+
std::vector<Volume> volumes_;
+ std::vector<u64> atlas_key_;
+ u32 atlas_width_ = 0;
+ u32 atlas_height_ = 0;
+ u32 atlas_texture_ = 0;
};
} // namespace esengine
diff --git a/src/esengine/resource/ProbeVolume.hpp b/src/esengine/resource/ProbeVolume.hpp
index bbcec4bf7..9bea69b4e 100644
--- a/src/esengine/resource/ProbeVolume.hpp
+++ b/src/esengine/resource/ProbeVolume.hpp
@@ -9,6 +9,7 @@
#include "../core/Types.hpp"
#include "../math/Math.hpp"
+#include <atomic>
#include <vector>
namespace esengine {
@@ -37,6 +38,16 @@ public:
* holds the lights and bounces alone and the frame's sky is added through this. */
std::vector<f32> skyVisibility;
+ /** @brief Changes whenever the coefficients do (a new grid, its sky visibility set): what the frame's probe
+ * atlas keys on, so it uploads a grid once and again only when the grid itself changed. Unique across
+ * grids, so a released grid whose memory a new one reuses is never mistaken for it. */
+ u64 revision = 0;
+
+ static u64 nextRevision() {
+ static std::atomic<u64> next{0};
+ return ++next;
+ }
+
bool hasSkyVisibility() const {
return !skyVisibility.empty() && skyVisibility.size() == static_cast<usize>(probeCount()) * 9;
}
diff --git a/src/esengine/resource/ResourceManager.cpp b/src/esengine/resource/ResourceManager.cpp
index 528ad49cf..10089796f 100644
--- a/src/esengine/resource/ResourceManager.cpp
+++ b/src/esengine/resource/ResourceManager.cpp
@@ -899,6 +899,7 @@ ProbeVolumeHandle ResourceManager::createProbeVolume(const glm::ivec3& resolutio
for (usize i = 0; i < volume->irradiance.size(); ++i) {
volume->irradiance[i] = {irradiance[i * 3], irradiance[i * 3 + 1], irradiance[i * 3 + 2]};
}
+ volume->revision = ProbeVolume::nextRevision();
return probeVolumes_.add(std::move(volume));
}
diff --git a/src/esengine/resource/ShaderParser.cpp b/src/esengine/resource/ShaderParser.cpp
index 1397443dc..66d27da8f 100644
--- a/src/esengine/resource/ShaderParser.cpp
+++ b/src/esengine/resource/ShaderParser.cpp
@@ -1466,6 +1466,8 @@ var<private> g_hairShadow : f32 = 0.0;
var<private> g_shadowHard : f32 = 0.0;
// The GLSL twin's v_lightmap, handed over by the entry point like the slot above.
var<private> g_lightmap : vec3f = vec3f(0.0);
+// Where the surface being lit stands, for a volume read per pixel: set by whatever lights it (the BRDF, envRadiance).
+var<private> g_probePos : vec3f = vec3f(0.0);
// The mesh's own tangent frame, handed over by the entry point like the slot above; zero where the shader never sets it, which
// is what sends perturbNormal back to deriving a frame per pixel.
var<private> g_worldTangent : vec4f = vec4f(0.0);
@@ -1949,10 +1951,36 @@ fn bakedIrradiance() -> vec3f {
return c * c * 8.0;
}
#endif
+#ifndef MESH_LIGHTMAP
+// See the GLSL twin: the grid itself, at P, through the frame's probe atlas.
+fn probeGridIrradiance(N : vec3f, P : vec3f) -> vec4f {
+ let base = clamp(i32(g_probeSlot), 0, 99) * 10;
+ let a = pc.u_probeIrradiance[base + 1];
+ let b = pc.u_probeIrradiance[base + 2];
+ let c = pc.u_probeIrradiance[base + 3];
+ let last = vec3f(a.w, b.w, pc.u_probeIrradiance[base + 4].x) - 1.0;
+ let g = clamp((P - a.xyz) * b.xyz, vec3f(0.0), last);
+ let z0 = min(floor(g.z), max(last.z - 1.0, 0.0));
+ let z1 = min(z0 + 1.0, last.z);
+ let fz = clamp(g.z - z0, 0.0, 1.0);
+ let inTile = c.xy + g.xy + 0.5;
+ var basis = array<f32, 9>(0.282095, 0.488603 * N.y, 0.488603 * N.z, 0.488603 * N.x,
+ 1.092548 * N.x * N.y, 1.092548 * N.y * N.z, 0.315392 * (3.0 * N.z * N.z - 1.0),
+ 1.092548 * N.x * N.z, 0.546274 * (N.x * N.x - N.y * N.y));
+ var sum = vec4f(0.0);
+ for (var i = 0; i < 9; i = i + 1) {
+ let tile = inTile + vec2f(f32(i) * a.w, 0.0);
+ let lo = textureSampleLevel(t5, s5, (tile + vec2f(0.0, z0 * b.w)) * c.zw, 0.0);
+ let hi = textureSampleLevel(t5, s5, (tile + vec2f(0.0, z1 * b.w)) * c.zw, 0.0);
+ sum += mix(lo, hi, fz) * basis[i];
+ }
+ return sum;
+}
+#endif
// See the GLSL twin: how much of the sky a probe sees toward d, 1 where nothing was baked of it.
fn skyOcclusion(d : vec3f) -> f32 {
let base = clamp(i32(g_probeSlot), 0, 99) * 10;
- let v = pc.u_probeIrradiance[base + 9].y * 0.282095
+ var v = pc.u_probeIrradiance[base + 9].y * 0.282095
+ pc.u_probeIrradiance[base + 1].w * (0.488603 * d.y)
+ pc.u_probeIrradiance[base + 2].w * (0.488603 * d.z)
+ pc.u_probeIrradiance[base + 3].w * (0.488603 * d.x)
@@ -1961,13 +1989,24 @@ fn skyOcclusion(d : vec3f) -> f32 {
+ pc.u_probeIrradiance[base + 6].w * (0.315392 * (3.0 * d.z * d.z - 1.0))
+ pc.u_probeIrradiance[base + 7].w * (1.092548 * d.x * d.z)
+ pc.u_probeIrradiance[base + 8].w * (0.546274 * (d.x * d.x - d.y * d.y));
+#ifndef MESH_LIGHTMAP
+ if (pc.u_probeIrradiance[base].w > 1.5) { v = probeGridIrradiance(d, g_probePos).a; }
+#endif
return mix(1.0, clamp(v, 0.0, 1.0), step(0.5, pc.u_probeIrradiance[base + 9].z));
}
// See the GLSL twin: one exit, for the shape the two share.
fn indirectIrradiance(N : vec3f) -> vec3f {
let base = clamp(i32(g_probeSlot), 0, 99) * 10;
+ let mode = pc.u_probeIrradiance[base].w;
var result : vec3f;
- if (pc.u_probeIrradiance[base].w > 0.5) {
+#ifndef MESH_LIGHTMAP
+ if (mode > 1.5) {
+ let grid = probeGridIrradiance(N, g_probePos);
+ let sky = step(0.5, pc.u_probeIrradiance[base + 9].z);
+ result = max(grid.rgb, vec3f(0.0)) * lc.u_bakedTint.rgb + envIrradiance(N) * (clamp(grid.a, 0.0, 1.0) * sky);
+ } else
+#endif
+ if (mode > 0.5) {
var sh : array<vec4f, 9>;
for (var i = 0; i < 9; i = i + 1) { sh[i] = pc.u_probeIrradiance[base + i]; }
let sky = step(0.5, pc.u_probeIrradiance[base + 9].z);
@@ -2038,6 +2077,7 @@ fn reflectionDirection(R : vec3f, P : vec3f, column : f32) -> vec3f {
return (P + R * t) - (lo + hi) * 0.5;
}
fn envRadiance(R : vec3f, roughness : f32, column : f32, P : vec3f) -> vec3f {
+ g_probePos = P;
#ifdef ES_ENV_MAP
if (planar.planarParams.x > 0.5) {
let clip = planar.planarProjection * vec4f(P, 1.0);
@@ -2146,6 +2186,7 @@ fn filmFresnel(F : vec3f, cosTheta : f32, film : vec4f) -> vec3f {
}
fn applyLightingSurfaceBRDF(diffuse : vec3f, F0 : vec3f, N : vec3f, worldPos : vec3f, V : vec3f,
roughness : f32, specular : f32, ao : f32, ext : SurfaceExt) -> vec3f {
+ g_probePos = worldPos;
let a = max(roughness * roughness, 1e-3);
let NdotV = max(dot(N, V), 1e-4);
var lit = indirectIrradiance(N) * ao;
@@ -2621,8 +2662,10 @@ ShaderParser::AssembledStage assembleWGSLStage(const ParsedShader& parsed,
if (stage == ShaderStage::Vertex && bodyIt->second.find("esInstance") != std::string::npos) {
inject(instanceRecordWGSL());
}
- // Last, so it sees every declaration made above as well as the body's own.
- inject(wgslReachedTextureDecls(assembled.str() + bodyIt->second,
+ // Last, so it sees every declaration made above as well as the body's own. Read with this variant's features
+ // applied: a unit declared behind a feature that is off is not declared (the mesh twin's t5 is the bake's, under
+ // MESH_LIGHTMAP), and one reached only behind such a feature is not reached.
+ inject(wgslReachedTextureDecls(preprocessConditionals(assembled.str() + bodyIt->second, features),
parsed.domain == "PostProcess"));
assembled << bodyText;
@@ -2862,6 +2905,12 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
" highp mat4 u_planarProjection; highp vec4 u_planarParams;\n"
"};\n"
"flat in highp float v_probeSlot;\n"
+ // The frame's probe atlas, for a volume read per pixel; a variant reading a bake has that sampler on this unit.
+ "#ifndef MESH_LIGHTMAP\n"
+ "uniform highp sampler2D u_probeGrid;\n"
+ "#endif\n"
+ // Where the surface being lit stands, for that read: set by whatever lights it (the BRDF, envRadiance).
+ "highp vec3 g_probePos = vec3(0.0);\n"
// The shadow map rides the draw's third texture slot, behind the feature the
// MESH vertex sources set: the batch stream owns 0..7 as a per-vertex merge
// product, so a sampler pinned to slot 2 there would read someone's sprite.
@@ -3365,6 +3414,41 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
// or a draw carrying no run at all — is the environment, which is what every
// surface reflected before there were probes.
"highp float reflectionColumn() { return u_probeIrradiance[probeRun() + 9].x; }\n"
+ // A run marked 2 says where the grid is, not what it holds (ProbeStore::writeGridRun): its nine
+ // coefficient tiles in the frame's atlas, read here at P. The hardware blends x and y inside a tile; the
+ // two slices around P's z are blended here. rgb is the irradiance toward N, a the sky's visibility.
+ "#ifndef MESH_LIGHTMAP\n"
+ "highp vec4 probeGridIrradiance(in highp vec3 N, in highp vec3 P) {\n"
+ " highp int base = probeRun();\n"
+ " highp vec4 a = u_probeIrradiance[base + 1];\n"
+ " highp vec4 b = u_probeIrradiance[base + 2];\n"
+ " highp vec4 c = u_probeIrradiance[base + 3];\n"
+ " highp vec3 last = vec3(a.w, b.w, u_probeIrradiance[base + 4].x) - 1.0;\n"
+ " highp vec3 g = clamp((P - a.xyz) * b.xyz, vec3(0.0), last);\n"
+ " highp float z0 = min(floor(g.z), max(last.z - 1.0, 0.0));\n"
+ " highp float z1 = min(z0 + 1.0, last.z);\n"
+ " highp float fz = clamp(g.z - z0, 0.0, 1.0);\n"
+ " highp vec2 inTile = c.xy + g.xy + 0.5;\n"
+ " highp float basis[9];\n"
+ " basis[0] = 0.282095;\n"
+ " basis[1] = 0.488603 * N.y;\n"
+ " basis[2] = 0.488603 * N.z;\n"
+ " basis[3] = 0.488603 * N.x;\n"
+ " basis[4] = 1.092548 * N.x * N.y;\n"
+ " basis[5] = 1.092548 * N.y * N.z;\n"
+ " basis[6] = 0.315392 * (3.0 * N.z * N.z - 1.0);\n"
+ " basis[7] = 1.092548 * N.x * N.z;\n"
+ " basis[8] = 0.546274 * (N.x * N.x - N.y * N.y);\n"
+ " highp vec4 sum = vec4(0.0);\n"
+ " for (int i = 0; i < 9; i++) {\n"
+ " highp vec2 tile = inTile + vec2(float(i) * a.w, 0.0);\n"
+ " highp vec4 lo = textureLod(u_probeGrid, (tile + vec2(0.0, z0 * b.w)) * c.zw, 0.0);\n"
+ " highp vec4 hi = textureLod(u_probeGrid, (tile + vec2(0.0, z1 * b.w)) * c.zw, 0.0);\n"
+ " sum += mix(lo, hi, fz) * basis[i];\n"
+ " }\n"
+ " return sum;\n"
+ "}\n"
+ "#endif\n"
// A bake outranks a volume for the same reason a volume outranks the sky:
// it holds every light and bounce that reached this very point.
// How much of the sky the probe this instance stands among sees toward d: nine coefficients of
@@ -3381,6 +3465,9 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
" + u_probeIrradiance[base + 6].w * (0.315392 * (3.0 * d.z * d.z - 1.0))\n"
" + u_probeIrradiance[base + 7].w * (1.092548 * d.x * d.z)\n"
" + u_probeIrradiance[base + 8].w * (0.546274 * (d.x * d.x - d.y * d.y));\n"
+ "#ifndef MESH_LIGHTMAP\n"
+ " if (u_probeIrradiance[base].w > 1.5) v = probeGridIrradiance(d, g_probePos).a;\n"
+ "#endif\n"
" return mix(1.0, clamp(v, 0.0, 1.0), step(0.5, u_probeIrradiance[base + 9].z));\n"
"}\n"
// A grid baked with sky occlusion holds the lights and their bounces WITHOUT the sky; the sky is
@@ -3392,8 +3479,16 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
// its stack and killed the GPU process (0xC00000FD in D3DCompiler_47) on the lightmapped lit variant.
"highp vec3 indirectIrradiance(in highp vec3 N) {\n"
" highp int base = probeRun();\n"
+ " highp float mode = u_probeIrradiance[base].w;\n"
" highp vec3 result;\n"
- " if (u_probeIrradiance[base].w > 0.5) {\n"
+ "#ifndef MESH_LIGHTMAP\n"
+ " if (mode > 1.5) {\n"
+ " highp vec4 grid = probeGridIrradiance(N, g_probePos);\n"
+ " highp float sky = step(0.5, u_probeIrradiance[base + 9].z);\n"
+ " result = max(grid.rgb, vec3(0.0)) * u_bakedTint.rgb + envIrradiance(N) * (clamp(grid.a, 0.0, 1.0) * sky);\n"
+ " } else\n"
+ "#endif\n"
+ " if (mode > 0.5) {\n"
" highp vec4 sh[9];\n"
" for (int i = 0; i < 9; i++) sh[i] = u_probeIrradiance[base + i];\n"
" highp float sky = step(0.5, u_probeIrradiance[base + 9].z);\n"
@@ -3471,6 +3566,7 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
// this is the flat ambient term, the same way the diffuse half is.
"highp vec3 envRadiance(in highp vec3 R, in highp float roughness,\n"
" in highp float column, in highp vec3 P) {\n"
+ " g_probePos = P;\n"
"#ifdef ES_ENV_MAP\n"
" if (u_planarParams.x > 0.5) {\n"
" highp vec4 clip = u_planarProjection * vec4(P, 1.0);\n"
@@ -3625,6 +3721,7 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
" in highp vec3 worldPos, in highp vec3 V,\n"
" in highp float roughness, in highp float specular,\n"
" in highp float ao, in SurfaceExt ext) {\n"
+ " g_probePos = worldPos;\n"
// glTF's specularFactor scales the whole lobe, so 0 removes it rather than
// leaving Schlick's grazing-angle rim behind.
" highp float a = max(roughness * roughness, 1e-3);\n"
diff --git a/tests/renderer/test_probe_store.cpp b/tests/renderer/test_probe_store.cpp
index 5f57023bb..2e51ad1c3 100644
--- a/tests/renderer/test_probe_store.cpp
+++ b/tests/renderer/test_probe_store.cpp
@@ -119,11 +119,73 @@ static void testMalformedGridsRefused() {
CHECK(store.count() == 1, "the one that can be read is");
}
+/** The atlas a per-pixel read samples: each grid a block of nine coefficient tiles,
+ * slices stacked down; a coefficient's texel is where the shader will look for it,
+ * and its alpha is the sky the probe sees. A grid two volumes share is one block. */
+static void testAtlasLayout() {
+ ProbeVolume a = ramp({2, 1, 3}, {1.0f, 0.0f, 0.0f}, {0.0f, 0.0f, 1.0f});
+ a.skyVisibility.assign(a.irradiance.size(), 0.25f);
+ a.revision = 1;
+ ProbeVolume b = ramp({3, 2, 1}, {0.0f, 1.0f, 0.0f}, {0.0f, 1.0f, 0.0f});
+ b.revision = 2;
+ ProbeStore store;
+ store.add({-1.0f, -1.0f, -1.0f}, {1.0f, 1.0f, 1.0f}, &a);
+ store.add({5.0f, 5.0f, 5.0f}, {6.0f, 6.0f, 6.0f}, &b);
+ store.add({-9.0f, -9.0f, -9.0f}, {-8.0f, -8.0f, -8.0f}, &a);
+
+ CHECK(store.layoutAtlas(2048), "a first layout wants an upload");
+ CHECK(store.atlasWidth() == 27, "the widest block is nine tiles of three");
+ CHECK(store.atlasHeight() == 3 + 2, "two blocks, the shared grid placed once");
+ CHECK(!store.layoutAtlas(2048), "the same grids again want nothing uploaded");
+ b.revision = 3;
+ CHECK(store.layoutAtlas(2048), "a grid that changed wants the atlas again");
+
+ std::vector<u16> texels;
+ store.fillAtlas(texels);
+ CHECK(texels.size() == 27u * 5u * 4u, "four halves a texel");
+ // a's last probe (x 1, z 2) is its ramp's end, blue; its coefficient 0 is tile 0, row z*resY + y = 2.
+ const usize last = (2u * 27u + 1u) * 4u;
+ CHECK(texels[last + 2] == packHalf(1.0f) && texels[last + 0] == packHalf(0.0f), "a coefficient sits where the shader reads it");
+ CHECK(texels[last + 3] == packHalf(0.25f), "the sky's visibility rides alpha");
+ // b sits below a: row 3, its constant green.
+ CHECK(texels[(3u * 27u) * 4u + 1] == packHalf(1.0f), "the second block starts under the first");
+ CHECK(texels[(3u * 27u) * 4u + 3] == packHalf(0.0f), "a grid without sky occlusion leaves alpha 0");
+}
+
+/** Asked per pixel, a volume in the atlas answers where its grid is rather than what
+ * it holds; asked per object, or with no atlas texture, it answers as it always did. */
+static void testGridRun() {
+ ProbeVolume v = ramp({3, 2, 1}, {1.0f, 0.0f, 0.0f}, {0.0f, 0.0f, 1.0f});
+ v.revision = 7;
+ ProbeStore store;
+ store.add({-4.0f, 0.0f, -1.0f}, {4.0f, 2.0f, 1.0f}, &v);
+ store.layoutAtlas(2048);
+
+ ProbeConstants out{};
+ CHECK(store.sample({0.0f, 1.0f, 0.0f}, out, true), "a point in the box is held");
+ CHECK(near(out.irradiance[0].w, 1.0f), "with no atlas texture the read stays per object");
+
+ store.setAtlasTexture(42);
+ CHECK(store.sample({0.0f, 1.0f, 0.0f}, out, true), "and with one");
+ CHECK(near(out.irradiance[0].w, 2.0f), "the run is marked as a grid");
+ CHECK(near(out.irradiance[1].x, -4.0f) && near(out.irradiance[1].w, 3.0f), "the box's min and resX");
+ CHECK(near(out.irradiance[2].x, 2.0f / 8.0f) && near(out.irradiance[2].y, 1.0f / 2.0f), "probes per unit along x and y");
+ CHECK(near(out.irradiance[2].z, 0.0f), "a flat axis reads one slice");
+ CHECK(near(out.irradiance[3].z, 1.0f / 27.0f) && near(out.irradiance[3].w, 1.0f / 2.0f), "the atlas's inverse size");
+ CHECK(near(out.irradiance[4].x, 1.0f), "resZ");
+
+ ProbeConstants object{};
+ CHECK(store.sample({0.0f, 1.0f, 0.0f}, object, false) && near(object.irradiance[0].w, 1.0f),
+ "a draw that cannot read the atlas still gets its nine");
+}
+
int main() {
testInsideAndOutside();
testTrilinearOnEachAxis();
testSmallestVolumeWins();
testMalformedGridsRefused();
+ testAtlasLayout();
+ testGridRun();
std::printf(g_failures ? "\n%d failure(s)\n" : "\nall probe store claims hold\n",
g_failures);
return g_failures ? 1 : 0;
diff --git a/tools/gen-shader-twins.mjs b/tools/gen-shader-twins.mjs
index e5b233d0c..9dbc5df1c 100644
--- a/tools/gen-shader-twins.mjs
+++ b/tools/gen-shader-twins.mjs
@@ -90,6 +90,8 @@ const ENGINE_SAMPLERS = [
{ name: 'u_shadow2D', unit: 7 },
{ name: 'u_lightShape2D', unit: 6 },
{ name: 'u_lightmap', unit: 5 },
+ // The probe atlas a volume read per pixel samples, on the bake's unit: a variant declares one or the other (EngineTextureUnits.hpp).
+ { name: 'u_probeGrid', unit: 5 },
// The two the VERTEX stage reads, pinned in MorphConstants.hpp and
// InstanceConstants.hpp. The record sits on the TOP unit, above every unit a
// material's own textures are handed (see GfxEnums MESH_INSTANCE_TEXTURE_UNIT).
diff --git a/tools/renderScenes.mjs b/tools/renderScenes.mjs
index d3ad34a0e..415d37695 100644
--- a/tools/renderScenes.mjs
+++ b/tools/renderScenes.mjs
@@ -278,6 +278,10 @@ export const SCENES = [
// A baked surface and a probe-lit one inside ONE volume. The bake already holds
// every bounce that reached it, so the atlas half must stay red: a draw taking
// both would come out purple, and only a frame with both in it can say so.
+ // ONE mesh across the same box. Read per object it would be the purple its centre stands in, end to end; read per
+ // pixel (the grid through the frame's probe atlas) its left end is red and its right blue, as a merged building
+ // spanning a shed and the yard is dark under the roof and lit outside.
+ { id: "probe-per-pixel", tier: "pr", webgpu: true, env: { "ESTELLA_VERIFY_SCENE": "/scenes/probe-per-pixel.esscene", "ESTELLA_VERIFY_MANIFEST": "/scenes/probe-per-pixel.textures.json", "ESTELLA_VERIFY_W": "256", "ESTELLA_VERIFY_H": "256", "ESTELLA_VERIFY_STEPS": "2", "ESTELLA_VERIFY_EXPECT": "[{\"x\":0.36,\"y\":0.5,\"rgb\":[213,0,38],\"tol\":20},{\"x\":0.625,\"y\":0.5,\"rgb\":[43,0,193],\"tol\":20}]" } },
{ id: "probe-lightmap", tier: "pr", webgpu: true, env: { "ESTELLA_VERIFY_SCENE": "/scenes/probe-lightmap.esscene", "ESTELLA_VERIFY_MANIFEST": "/scenes/probe-lightmap.textures.json", "ESTELLA_VERIFY_W": "256", "ESTELLA_VERIFY_H": "256", "ESTELLA_VERIFY_STEPS": "2", "ESTELLA_VERIFY_EXPECT": "[{\"x\":0.25,\"y\":0.5667,\"rgb\":[255,0,0],\"tol\":30},{\"x\":0.85,\"y\":0.5,\"rgb\":[0,0,255],\"tol\":30}]" } },
// A clip driving the weight, which is what an imported `weights` channel
// becomes. The shape has to cross the boundary as a LIST — a weight written