Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): sky occlusion — rooms stay dark under a sky that changes

yhh673d6737ffbb
diff --git a/CHANGELOG.md b/CHANGELOG.md
index de74dc497..0fa8d48a3 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -24,9 +24,11 @@ published separately; it ships inside the editor.
- **A material reads its maps through Tiling, Offset and Rotation.** The Model shader's `u_uvTiling`, `u_uvOffset` and `u_uvRotation` move where every map is read (colour, normal, occlusion, metal-roughness, emission), as Unity's Tiling and Offset do; Ground Blend takes them too. A glTF's `KHR_texture_transform` is imported into them, turned into the engine's V-up uv, where before it was dropped with a warning and a tiled texture set was stretched once over each mesh: a three.js port of a container port lost 224 of them. Maps of one material that disagree are all read through the first map's, and the import says so. A cut-out's shadow and depth prepass read through the same transform, so a tiled fence casts the shadow of the fence drawn.
- **The sky lights the scene.** Sky Atmosphere's **Light the Scene** makes the sky the scene's environment: every Ambient light that names no environment of its own lights with it, its diffuse light from the dome and the clouds and its reflection on metal and wet ground, scaled by the light's colour and intensity as an imported one is. The sky is worked out again as it changes (the sun moving, a weather coming in, clouds drifting), at most once a second; with clouds that takes about 20 ms on a desktop. The sun's disc is left out, since the directional light the sky follows is its direct light. Off by default, so a scene already lit by hand looks as it did.
+- **Rooms stay dark under a sky that changes (sky occlusion).** With **Sky occlusion** on in the Lighting settings (`BakedLighting.skyOcclusion`), a probe bake keeps how much of the sky each probe sees instead of the sky itself: the probes hold the lights and their bounces, and the sky the frame has now (a live Sky Atmosphere, a weather, an environment) is added through what each probe sees of it, to the diffuse light and to the sky's reflection. A ball inside a roofed room measured 119 where the baked sky gave it 150, and a ball outside kept its light. A `.esprobes` baked this way carries `skyVisibility`; one without it lights as before. Probes are read per object, as Unity's light probes are: one large mesh spanning indoors and out takes one value.
### Fixed
+- **Geometry with no lightmap UV set still stands in a bake's light.** It was left out of the bake altogether, so it cast no shadow on the baked surfaces and the probes saw straight through it. It now shadows them and hides the sky from the probes, while receiving no baked light itself (the import still says to generate lightmap UVs for that).
- **An opaque surface no longer lets the sky through where its map's alpha is below 1.** An unblended draw wrote its base colour's alpha into the frame, and Sky Atmosphere reads a pixel's alpha as how much of it geometry covers, so wherever a texture used its alpha for something else (glTF's OPAQUE ignores it) the clouds showed through the wall. An opaque Model surface now writes 1, except a cut-out's coverage and the subsurface mark.
- **The sky no longer shows through everything when water or glass and a blended surface are both in view.** The frame keeps the opaque scene aside and puts it back around the blended draws; it was put back through the copy made for refraction, which marks its alpha 0.5, so every pixel came back half covered and the sky showed through half of the picture.
- **Ground under a roof stays dry in the reflections too.** Screen Space Reflections' Wetness and Puddles read the rain's map of what stands over each place, as the rain itself does, so a floor indoors or under an awning is no longer a wet mirror. The map is drawn only for an effect that states a reach (Rain, Snow Cover); reflections borrow it, with its reach and bias, and cost no second picture of the scene on their own.
diff --git a/desktop b/desktop
index d7ee98ac6..860430181 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit d7ee98ac65a3bc308fb31559f12d3cd59419bbb0
+Subproject commit 860430181fc15a6abca144d49e087d213c5e47ba
diff --git a/docs/astro/src/data/apiStability.generated.json b/docs/astro/src/data/apiStability.generated.json
index 7342ffff4..d939afdec 100644
--- a/docs/astro/src/data/apiStability.generated.json
+++ b/docs/astro/src/data/apiStability.generated.json
@@ -3,10 +3,10 @@
"counts": {
"public": 115,
"beta": 178,
- "experimental": 3025,
+ "experimental": 3034,
"deprecated": 3
},
- "total": 3318,
+ "total": 3327,
"subsystems": [
{
"id": "mobile-controls",
diff --git a/docs/astro/src/data/components.generated.json b/docs/astro/src/data/components.generated.json
index 5e4ba42b7..58fe6621a 100644
--- a/docs/astro/src/data/components.generated.json
+++ b/docs/astro/src/data/components.generated.json
@@ -225,6 +225,12 @@
"doc": "Directions each probe gathers, over the whole sphere.",
"range": "≥ 1"
},
+ {
+ "key": "skyOcclusion",
+ "type": "boolean",
+ "default": "false",
+ "doc": "Probes keep how much sky they see, not the sky itself: the sky in play (a live Sky Atmosphere, a weather) lights them through it, so rooms stay dark."
+ },
{
"key": "bakedFrom",
"type": "string",
diff --git a/fixtures/scenes/lit-builtin-lightmap-uv.esscene b/fixtures/scenes/lit-builtin-lightmap-uv.esscene
new file mode 100644
index 000000000..bdd2c6297
--- /dev/null
+++ b/fixtures/scenes/lit-builtin-lightmap-uv.esscene
@@ -0,0 +1,120 @@
+{
+ "version": 4,
+ "name": "LitBuiltinLightmapUv",
+ "entities": [
+ {
+ "id": 0,
+ "name": "Camera",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 260,
+ "z": 1050
+ },
+ "rotation": {
+ "w": 0.9914,
+ "x": -0.1305,
+ "y": 0,
+ "z": 0
+ }
+ }
+ },
+ {
+ "type": "Camera",
+ "data": {
+ "projectionType": 0,
+ "fov": 50,
+ "nearPlane": 10,
+ "farPlane": 20000,
+ "isActive": true,
+ "priority": 0
+ }
+ }
+ ]
+ },
+ {
+ "id": 1,
+ "name": "Sun",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ },
+ "rotation": {
+ "w": 0.8339,
+ "x": -0.5209,
+ "y": 0.1823,
+ "z": 0
+ }
+ }
+ },
+ {
+ "type": "Light",
+ "data": {
+ "type": 1,
+ "intensity": 1.2,
+ "color": {
+ "r": 1,
+ "g": 0.96,
+ "b": 0.9,
+ "a": 1
+ },
+ "meshShadows": false,
+ "enabled": true
+ }
+ }
+ ]
+ },
+ {
+ "id": 3,
+ "name": "Plane",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ },
+ "scale": {
+ "x": 12,
+ "y": 1,
+ "z": 12
+ }
+ }
+ },
+ {
+ "type": "MeshRenderer",
+ "data": {
+ "mesh": "builtin:plane",
+ "lit": true,
+ "opaque": true,
+ "color": {
+ "r": 0.45,
+ "g": 0.47,
+ "b": 0.45,
+ "a": 1
+ }
+ }
+ }
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/fixtures/scenes/probe-sky.esprobes b/fixtures/scenes/probe-sky.esprobes
new file mode 100644
index 000000000..1e83f47c0
--- /dev/null
+++ b/fixtures/scenes/probe-sky.esprobes
@@ -0,0 +1,6 @@
+{
+ "version": 1,
+ "resolution": [1,1,1],
+ "irradiance": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
+ "skyVisibility": [0.708981,0,0,0,0,0,0,0,0]
+}
diff --git a/fixtures/scenes/probe-sky.esscene b/fixtures/scenes/probe-sky.esscene
new file mode 100644
index 000000000..2b6dadc9b
--- /dev/null
+++ b/fixtures/scenes/probe-sky.esscene
@@ -0,0 +1,137 @@
+{
+ "version": 4,
+ "name": "ProbeSky",
+ "entities": [
+ {
+ "id": 0,
+ "name": "Camera",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 10
+ }
+ }
+ },
+ {
+ "type": "Camera",
+ "data": {
+ "projectionType": 1,
+ "orthoSize": 300,
+ "isActive": true,
+ "priority": 0
+ }
+ }
+ ]
+ },
+ {
+ "id": 1,
+ "name": "Probes",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ }
+ }
+ },
+ {
+ "type": "LightProbeVolume",
+ "data": {
+ "probes": "/scenes/probe-sky.esprobes",
+ "halfExtents": {
+ "x": 400,
+ "y": 400,
+ "z": 400
+ },
+ "enabled": true
+ }
+ }
+ ]
+ },
+ {
+ "id": 2,
+ "name": "LitTriangles",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ },
+ "scale": {
+ "x": 1,
+ "y": 1,
+ "z": 1
+ }
+ }
+ },
+ {
+ "type": "MeshRenderer",
+ "data": {
+ "color": {
+ "r": 1,
+ "g": 1,
+ "b": 1,
+ "a": 1
+ },
+ "layer": 0,
+ "enabled": true,
+ "lit": true,
+ "mesh": "/scenes/lit-triangles.esmesh"
+ }
+ }
+ ]
+ },
+ {
+ "id": 3,
+ "name": "Sky",
+ "parent": null,
+ "children": [],
+ "visible": true,
+ "components": [
+ {
+ "type": "Transform",
+ "data": {
+ "position": {
+ "x": 0,
+ "y": 0,
+ "z": 0
+ }
+ }
+ },
+ {
+ "type": "Light",
+ "data": {
+ "type": 2,
+ "color": {
+ "r": 1,
+ "g": 1,
+ "b": 1,
+ "a": 1
+ },
+ "intensity": 1,
+ "enabled": true
+ }
+ }
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/fixtures/scenes/probe-sky.textures.json b/fixtures/scenes/probe-sky.textures.json
new file mode 100644
index 000000000..0967ef424
--- /dev/null
+++ b/fixtures/scenes/probe-sky.textures.json
@@ -0,0 +1 @@
+{}
diff --git a/pipeline/bin/estella.mjs b/pipeline/bin/estella.mjs
index 8492f9ce6..b5f33f0d7 100644
--- a/pipeline/bin/estella.mjs
+++ b/pipeline/bin/estella.mjs
@@ -275,6 +275,7 @@ async function bakeScene(baker, meta, sceneFile, check) {
probeSamples: declared.probeSamples,
};
for (const k of Object.keys(options)) if (typeof options[k] !== 'number') delete options[k];
+ if (declared.skyOcclusion === true) options.skyOcclusion = true;
// Where an asset ref resolves to. Not relative to the SCENE: a ref is what the
// project's asset browser shows, and the editor resolves it against the root.
diff --git a/pipeline/src/assets/lightmapBake.ts b/pipeline/src/assets/lightmapBake.ts
index 29c8a9416..acc52ba1a 100644
--- a/pipeline/src/assets/lightmapBake.ts
+++ b/pipeline/src/assets/lightmapBake.ts
@@ -80,6 +80,9 @@ export interface ProbeVolumeDocument {
version: number;
resolution: [number, number, number];
irradiance: number[];
+ /** Baked with sky occlusion: nine coefficients per probe of how much of the sky each sees, and the
+ * irradiance holds the lights alone. Absent otherwise (the irradiance holds the sky as it was). */
+ skyVisibility?: number[];
}
export interface SceneBakeInput {
@@ -150,7 +153,9 @@ const round6 = (v: number): number => Number(v.toPrecision(6));
export function probeDocumentText(doc: ProbeVolumeDocument): string {
return `{\n "version": ${doc.version},\n`
+ ` "resolution": ${JSON.stringify(doc.resolution)},\n`
- + ` "irradiance": ${JSON.stringify(doc.irradiance)}\n}\n`;
+ + ` "irradiance": ${JSON.stringify(doc.irradiance)}`
+ + (doc.skyVisibility ? `,\n "skyVisibility": ${JSON.stringify(doc.skyVisibility)}` : '')
+ + `\n}\n`;
}
/** Stock geometry as the runtime draws it: the template carries its lightmap UV set, so the bake
@@ -369,6 +374,7 @@ function* sceneBakeSteps(input: SceneBakeInput, averages = new Map<string, Textu
const { lights, environment } = resolveEnvironmentSun(input.lights, input.environment,
input.options?.ambient ?? [0, 0, 0], warnings);
const surfaces: BakeSurface[] = [];
+ const occluders: BakeSurface[] = [];
const slot: number[] = [];
let moving = 0;
// Stock geometry is rebuilt from code every run, so its lightmap UVs are
@@ -395,8 +401,11 @@ function* sceneBakeSteps(input: SceneBakeInput, averages = new Map<string, Textu
continue;
}
if (!mesh.channels.some((c) => c.semantic === MeshChannel.TexCoord1)) {
- warnings.push(`${s.label}: no second UV set, so it receives no baked light —`
- + ' turn on Generate Lightmap UVs in its model\'s import settings and reimport');
+ warnings.push(`${s.label}: no second UV set, so it receives no baked light (it still shadows the bake`
+ + ' and hides the sky from the probes) — turn on Generate Lightmap UVs in its model\'s import'
+ + ' settings and reimport');
+ occluders.push({ mesh, transform: s.transform, label: s.label, twoSided: s.twoSided,
+ coverage: albedoOf(s, cutoffOf(s.material, cutoffs), averages, warnings).coverage });
continue;
}
slot.push(i);
@@ -449,10 +458,11 @@ function* sceneBakeSteps(input: SceneBakeInput, averages = new Map<string, Textu
for (const w of packed.warnings) warnings.push(w);
return packed;
};
- const asDocument = (grid: ProbeGrid, sh: Float32Array): ProbeVolumeDocument => ({
+ const asDocument = (grid: ProbeGrid, sh: Float32Array, sky?: Float32Array): ProbeVolumeDocument => ({
version: 1,
resolution: [grid.resolution[0], grid.resolution[1], grid.resolution[2]],
irradiance: Array.from(sh, round6),
+ ...(sky ? { skyVisibility: Array.from(sky, round6) } : {}),
});
if (surfaces.length === 0) {
@@ -460,8 +470,8 @@ function* sceneBakeSteps(input: SceneBakeInput, averages = new Map<string, Textu
// The volumes are still solved: a scene whose only light is ambient has
// nothing to bake into an atlas and still has somewhere to stand.
const empty = yield* bakeLightmapSteps([], lights,
- { ...input.options, probeGrids: grids, ...reflectionOptions });
- gridSlot.forEach((at, k) => { probes[at] = asDocument(grids[k], empty.probes[k]); });
+ { ...input.options, probeGrids: grids, occluders, ...reflectionOptions });
+ gridSlot.forEach((at, k) => { probes[at] = asDocument(grids[k], empty.probes[k], empty.probeSkyVisibility[k]); });
return {
atlasBytes: encodeRgbaPng(1, 1, new Uint8Array([0, 0, 0, 255])),
scaleOffset: input.surfaces.map(() => null),
@@ -472,7 +482,7 @@ function* sceneBakeSteps(input: SceneBakeInput, averages = new Map<string, Textu
}
const result = yield* bakeLightmapSteps(surfaces, lights,
- { ...input.options, probeGrids: grids, ...reflectionOptions });
+ { ...input.options, probeGrids: grids, occluders, ...reflectionOptions });
result.shared.forEach((share, k) => {
if (share > 0.25) {
warnings.push(`${surfaces[k]!.label ?? `surface ${k}`}: its lightmap UVs lay`
@@ -484,7 +494,7 @@ function* sceneBakeSteps(input: SceneBakeInput, averages = new Map<string, Textu
const scaleOffset: Array<[number, number, number, number] | null> =
input.surfaces.map(() => null);
slot.forEach((at, k) => { scaleOffset[at] = result.scaleOffset[k]; });
- gridSlot.forEach((at, k) => { probes[at] = asDocument(grids[k], result.probes[k]); });
+ gridSlot.forEach((at, k) => { probes[at] = asDocument(grids[k], result.probes[k], result.probeSkyVisibility[k]); });
return {
atlasBytes: encodeRgbaPng(result.size, result.size, lightmapImage(result.pixels, result.size)),
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 0b58b5fe9..4b49b4f94 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2137 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2149 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -1424,7 +1424,7 @@ type: number | undefined
## BAKE_DEFAULTS — const @experimental
```
-{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
+{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; skyOcclusion: boolean; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
```
## BONE_LIMITS — const @experimental
@@ -1517,6 +1517,7 @@ ambient: readonly [number, number, number] | undefined
atlasSize: number | undefined
bounces: number | undefined
dilate: number | undefined
+occluders: readonly BakeSurface[] | undefined
probeGrids: readonly ProbeGrid[] | undefined
probeSamples: number | undefined
reflectionProbes: readonly (readonly [number, number, number])[] | undefined
@@ -1524,6 +1525,7 @@ reflectionSky: SkyRadiance | undefined
reflectionWidth: number | undefined
samples: number | undefined
sky: SkySpec | SkyRadiance | undefined
+skyOcclusion: boolean | undefined
texelsPerUnit: number | undefined
```
@@ -1537,6 +1539,7 @@ progress: number
```
lumels: number
pixels: Uint8Array<ArrayBufferLike>
+probeSkyVisibility: Float32Array<ArrayBufferLike>[]
probes: Float32Array<ArrayBufferLike>[]
reflections: CapturedPanorama[]
scaleOffset: [number, number, number, number][]
@@ -1584,6 +1587,7 @@ bakedFrom: string
bounces: number
probeSamples: number
samples: number
+skyOcclusion: boolean
texelsPerUnit: number
```
@@ -5054,6 +5058,7 @@ hand: number | null
joints: number
missing: string[]
pose: string
+touching: string[]
weight: number
```
@@ -5504,7 +5509,10 @@ static prototype: InputRouter
## InputState — class @beta
```
+@internal admitTouch: (id: number) => boolean
@internal cancelVirtualSources: () => void
+@internal forgetTouches: () => void
+@internal touchIgnored: (id: number, retiring?: boolean) => boolean
beginFixedStep: () => void
clearFrameState: () => void
clearVirtual: () => void
@@ -5536,12 +5544,10 @@ isKeyReleased: (key: string) => boolean
isMouseButtonDown: (button: number) => boolean
isMouseButtonPressed: (button: number) => boolean
isMouseButtonReleased: (button: number) => boolean
+isMultiTouchEnabled: () => boolean
isPointerOverUI: () => boolean
isTouchActive: (id: number) => boolean
-@internal admitTouch: (id: number) => boolean
keysDown: Set<string>
-@internal forgetTouches: () => void
-@internal touchIgnored: (id: number, retiring?: boolean) => boolean
keysPressed: Set<string>
keysPressedFixed: Set<string>
keysReleased: Set<string>
@@ -5564,6 +5570,7 @@ releaseGamepad: (index?: number) => void
removeVirtualSource: (owner: object) => void
scrollDeltaX: number
scrollDeltaY: number
+setMultiTouch: (enabled: boolean) => void
setVirtual: (id: string, x: number, y?: number) => void
setVirtualSource: (owner: object, id: string, x: number, y?: number) => void
touchAvailable: boolean
@@ -5573,7 +5580,6 @@ touchesStarted: Map<number, TouchPoint>
updateGamepads: (snapshots: GamepadSnapshot[]) => void
static new (): InputState
static prototype: InputState
-isMultiTouchEnabled: () => boolean
```
## InstantiatePrefabOptions — interface @experimental
@@ -5599,7 +5605,6 @@ BuiltinComponentDef<InteractableData>
## InteractableData — interface @experimental
```
blockRaycast: boolean
-setMultiTouch: (enabled: boolean) => void
enabled: boolean
raycastTarget: boolean
```
@@ -6843,6 +6848,13 @@ static new (atlasSize: number): PageTable
static prototype: PageTable
```
+## Panorama — interface @experimental
+```
+height: number
+rgb: Float32Array<ArrayBufferLike>
+width: number
+```
+
## Parent — const @public
```
BuiltinComponentDef<ParentData>
@@ -9218,6 +9230,7 @@ audioCacheBudget: number | undefined
defaultFontFamily: string | undefined
maxDeltaTime: number | undefined
maxFixedSteps: number | undefined
+multiTouch: boolean | undefined
retainedBudget: number | undefined
sceneTransitionColor: string | undefined
sceneTransitionDuration: number | undefined
@@ -9260,7 +9273,6 @@ uiTheme: "light" | "dark" | undefined
uiThemeOverrides: ThemeOverrides | undefined
worlds: WorldManifest[] | undefined
```
-multiTouch: boolean | undefined
## RuntimeOnly — const @public
```
@@ -14510,6 +14522,11 @@ AssetPlugin
(atlas: CookedAtlasInfo, pagePath: string): Pick<CatalogEntry, "atlas" | "frame" | "uv">
```
+## atlasLayout — function @experimental
+```
+(faceSize: number, mipCount: number): { width: number; height: number; offsets: number[]; }
+```
+
## attachTooltip — function @beta
```
(world: World, target: Entity, options?: Partial<UITooltipData>): void
@@ -15230,6 +15247,11 @@ Plugin
(bytes: Uint8Array): MeshData
```
+## decodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, a: number, maxRange: number): [number, number, number]
+```
+
## decomposePolygon2D — function @experimental
```
(vertices: readonly Vec2[]): PolygonDecomposition
@@ -15520,6 +15542,11 @@ DragPlugin
(data: MeshData): Uint8Array
```
+## encodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, maxRange: number, out: Uint8Array, at: number): void
+```
+
## engineWebGPUFeatures — function @experimental
```
(adapter: WebGPUAdapterLike): string[]
@@ -16580,6 +16607,16 @@ MeshRendererPlugin
(rings: number, cellSize: number): number
```
+## octDecode — function @experimental
+```
+(u: number, v: number): [number, number, number]
+```
+
+## octEncode — function @experimental
+```
+(dx: number, dy: number, dz: number): [number, number]
+```
+
## onCameraCommitChanged — function @beta
```
(fn: CameraCommitListener): () => void
@@ -16965,6 +17002,11 @@ PostProcessPlugin
PrefabsPlugin
```
+## prefilterOctahedral — function @experimental
+```
+(env: Panorama, faceSize: number, mipCount: number, maxRange: number): { width: number; height: number; rgba: Uint8Array; }
+```
+
## preloadGlyphs — function @experimental
```
(app: App, entity: Entity, text: string, options?: GlyphPreloadOptions): GlyphPreload
@@ -16995,6 +17037,11 @@ PrefabsPlugin
(p: ProceduralParams): Heightfield
```
+## projectIrradianceSH — function @experimental
+```
+(env: Panorama): Float32Array
+```
+
## projectSideModuleIds — function @experimental
```
(): string[]
@@ -17400,6 +17447,11 @@ SafeAreaPlugin
(keys: readonly Keyframe[], at: number): number
```
+## samplePanorama — function @experimental
+```
+(env: Panorama, dx: number, dy: number, dz: number, out: Float32Array): void
+```
+
## sampleRootPlacement — function @experimental
```
(asset: TimelineAsset, time: number, out: RootPlacement): boolean
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index f78c5d966..51adc7d68 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.native
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 171 beta · 2157 experimental · 3 deprecated
+Symbols: 115 public · 171 beta · 2169 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
@@ -1424,7 +1424,7 @@ type: number | undefined
## BAKE_DEFAULTS — const @experimental
```
-{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
+{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; skyOcclusion: boolean; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
```
## BONE_LIMITS — const @experimental
@@ -1517,6 +1517,7 @@ ambient: readonly [number, number, number] | undefined
atlasSize: number | undefined
bounces: number | undefined
dilate: number | undefined
+occluders: readonly BakeSurface[] | undefined
probeGrids: readonly ProbeGrid[] | undefined
probeSamples: number | undefined
reflectionProbes: readonly (readonly [number, number, number])[] | undefined
@@ -1524,6 +1525,7 @@ reflectionSky: SkyRadiance | undefined
reflectionWidth: number | undefined
samples: number | undefined
sky: SkySpec | SkyRadiance | undefined
+skyOcclusion: boolean | undefined
texelsPerUnit: number | undefined
```
@@ -1537,6 +1539,7 @@ progress: number
```
lumels: number
pixels: Uint8Array<ArrayBufferLike>
+probeSkyVisibility: Float32Array<ArrayBufferLike>[]
probes: Float32Array<ArrayBufferLike>[]
reflections: CapturedPanorama[]
scaleOffset: [number, number, number, number][]
@@ -1584,6 +1587,7 @@ bakedFrom: string
bounces: number
probeSamples: number
samples: number
+skyOcclusion: boolean
texelsPerUnit: number
```
@@ -5062,6 +5066,7 @@ hand: number | null
joints: number
missing: string[]
pose: string
+touching: string[]
weight: number
```
@@ -5512,7 +5517,10 @@ static prototype: InputRouter
## InputState — class @beta
```
+@internal admitTouch: (id: number) => boolean
@internal cancelVirtualSources: () => void
+@internal forgetTouches: () => void
+@internal touchIgnored: (id: number, retiring?: boolean) => boolean
beginFixedStep: () => void
clearFrameState: () => void
clearVirtual: () => void
@@ -5544,12 +5552,10 @@ isKeyReleased: (key: string) => boolean
isMouseButtonDown: (button: number) => boolean
isMouseButtonPressed: (button: number) => boolean
isMouseButtonReleased: (button: number) => boolean
+isMultiTouchEnabled: () => boolean
isPointerOverUI: () => boolean
isTouchActive: (id: number) => boolean
-@internal admitTouch: (id: number) => boolean
keysDown: Set<string>
-@internal forgetTouches: () => void
-@internal touchIgnored: (id: number, retiring?: boolean) => boolean
keysPressed: Set<string>
keysPressedFixed: Set<string>
keysReleased: Set<string>
@@ -5572,6 +5578,7 @@ releaseGamepad: (index?: number) => void
removeVirtualSource: (owner: object) => void
scrollDeltaX: number
scrollDeltaY: number
+setMultiTouch: (enabled: boolean) => void
setVirtual: (id: string, x: number, y?: number) => void
setVirtualSource: (owner: object, id: string, x: number, y?: number) => void
touchAvailable: boolean
@@ -5581,7 +5588,6 @@ touchesStarted: Map<number, TouchPoint>
updateGamepads: (snapshots: GamepadSnapshot[]) => void
static new (): InputState
static prototype: InputState
-isMultiTouchEnabled: () => boolean
```
## InstantiatePrefabOptions — interface @experimental
@@ -5607,7 +5613,6 @@ BuiltinComponentDef<InteractableData>
## InteractableData — interface @experimental
```
blockRaycast: boolean
-setMultiTouch: (enabled: boolean) => void
enabled: boolean
raycastTarget: boolean
```
@@ -7049,6 +7054,13 @@ static new (atlasSize: number): PageTable
static prototype: PageTable
```
+## Panorama — interface @experimental
+```
+height: number
+rgb: Float32Array<ArrayBufferLike>
+width: number
+```
+
## Parent — const @public
```
BuiltinComponentDef<ParentData>
@@ -9439,6 +9451,7 @@ audioCacheBudget: number | undefined
defaultFontFamily: string | undefined
maxDeltaTime: number | undefined
maxFixedSteps: number | undefined
+multiTouch: boolean | undefined
retainedBudget: number | undefined
sceneTransitionColor: string | undefined
sceneTransitionDuration: number | undefined
@@ -9481,7 +9494,6 @@ uiTheme: "light" | "dark" | undefined
uiThemeOverrides: ThemeOverrides | undefined
worlds: WorldManifest[] | undefined
```
-multiTouch: boolean | undefined
## RuntimeOnly — const @public
```
@@ -14746,6 +14758,11 @@ AssetPlugin
(atlas: CookedAtlasInfo, pagePath: string): Pick<CatalogEntry, "atlas" | "frame" | "uv">
```
+## atlasLayout — function @experimental
+```
+(faceSize: number, mipCount: number): { width: number; height: number; offsets: number[]; }
+```
+
## attachTooltip — function @beta
```
(world: World, target: Entity, options?: Partial<UITooltipData>): void
@@ -15491,6 +15508,11 @@ Plugin
(bytes: Uint8Array): MeshData
```
+## decodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, a: number, maxRange: number): [number, number, number]
+```
+
## decomposePolygon2D — function @experimental
```
(vertices: readonly Vec2[]): PolygonDecomposition
@@ -15781,6 +15803,11 @@ DragPlugin
(data: MeshData): Uint8Array
```
+## encodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, maxRange: number, out: Uint8Array, at: number): void
+```
+
## engineWebGPUFeatures — function @experimental
```
(adapter: WebGPUAdapterLike): string[]
@@ -16851,6 +16878,16 @@ MeshRendererPlugin
(rings: number, cellSize: number): number
```
+## octDecode — function @experimental
+```
+(u: number, v: number): [number, number, number]
+```
+
+## octEncode — function @experimental
+```
+(dx: number, dy: number, dz: number): [number, number]
+```
+
## onCameraCommitChanged — function @beta
```
(fn: CameraCommitListener): () => void
@@ -17236,6 +17273,11 @@ PostProcessPlugin
PrefabsPlugin
```
+## prefilterOctahedral — function @experimental
+```
+(env: Panorama, faceSize: number, mipCount: number, maxRange: number): { width: number; height: number; rgba: Uint8Array; }
+```
+
## preloadGlyphs — function @experimental
```
(app: App, entity: Entity, text: string, options?: GlyphPreloadOptions): GlyphPreload
@@ -17266,6 +17308,11 @@ PrefabsPlugin
(p: ProceduralParams): Heightfield
```
+## projectIrradianceSH — function @experimental
+```
+(env: Panorama): Float32Array
+```
+
## projectSideModuleIds — function @experimental
```
(): string[]
@@ -17671,6 +17718,11 @@ SafeAreaPlugin
(keys: readonly Keyframe[], at: number): number
```
+## samplePanorama — function @experimental
+```
+(env: Panorama, dx: number, dy: number, dz: number, out: Float32Array): void
+```
+
## sampleRootPlacement — function @experimental
```
(asset: TimelineAsset, time: number, out: RootPlacement): boolean
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index 711d0bfda..ddcc1fe19 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/index.node
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 168 beta · 2136 experimental · 3 deprecated
+Symbols: 115 public · 168 beta · 2148 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -1419,7 +1419,7 @@ type: number | undefined
## BAKE_DEFAULTS — const @experimental
```
-{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
+{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; skyOcclusion: boolean; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
```
## BONE_LIMITS — const @experimental
@@ -1512,6 +1512,7 @@ ambient: readonly [number, number, number] | undefined
atlasSize: number | undefined
bounces: number | undefined
dilate: number | undefined
+occluders: readonly BakeSurface[] | undefined
probeGrids: readonly ProbeGrid[] | undefined
probeSamples: number | undefined
reflectionProbes: readonly (readonly [number, number, number])[] | undefined
@@ -1519,6 +1520,7 @@ reflectionSky: SkyRadiance | undefined
reflectionWidth: number | undefined
samples: number | undefined
sky: SkySpec | SkyRadiance | undefined
+skyOcclusion: boolean | undefined
texelsPerUnit: number | undefined
```
@@ -1532,6 +1534,7 @@ progress: number
```
lumels: number
pixels: Uint8Array<ArrayBufferLike>
+probeSkyVisibility: Float32Array<ArrayBufferLike>[]
probes: Float32Array<ArrayBufferLike>[]
reflections: CapturedPanorama[]
scaleOffset: [number, number, number, number][]
@@ -1579,6 +1582,7 @@ bakedFrom: string
bounces: number
probeSamples: number
samples: number
+skyOcclusion: boolean
texelsPerUnit: number
```
@@ -5049,6 +5053,7 @@ hand: number | null
joints: number
missing: string[]
pose: string
+touching: string[]
weight: number
```
@@ -5499,7 +5504,10 @@ static prototype: InputRouter
## InputState — class @beta
```
+@internal admitTouch: (id: number) => boolean
@internal cancelVirtualSources: () => void
+@internal forgetTouches: () => void
+@internal touchIgnored: (id: number, retiring?: boolean) => boolean
beginFixedStep: () => void
clearFrameState: () => void
clearVirtual: () => void
@@ -5531,12 +5539,10 @@ isKeyReleased: (key: string) => boolean
isMouseButtonDown: (button: number) => boolean
isMouseButtonPressed: (button: number) => boolean
isMouseButtonReleased: (button: number) => boolean
+isMultiTouchEnabled: () => boolean
isPointerOverUI: () => boolean
isTouchActive: (id: number) => boolean
-@internal admitTouch: (id: number) => boolean
keysDown: Set<string>
-@internal forgetTouches: () => void
-@internal touchIgnored: (id: number, retiring?: boolean) => boolean
keysPressed: Set<string>
keysPressedFixed: Set<string>
keysReleased: Set<string>
@@ -5559,6 +5565,7 @@ releaseGamepad: (index?: number) => void
removeVirtualSource: (owner: object) => void
scrollDeltaX: number
scrollDeltaY: number
+setMultiTouch: (enabled: boolean) => void
setVirtual: (id: string, x: number, y?: number) => void
setVirtualSource: (owner: object, id: string, x: number, y?: number) => void
touchAvailable: boolean
@@ -5568,7 +5575,6 @@ touchesStarted: Map<number, TouchPoint>
updateGamepads: (snapshots: GamepadSnapshot[]) => void
static new (): InputState
static prototype: InputState
-isMultiTouchEnabled: () => boolean
```
## InstantiatePrefabOptions — interface @experimental
@@ -5594,7 +5600,6 @@ BuiltinComponentDef<InteractableData>
## InteractableData — interface @experimental
```
blockRaycast: boolean
-setMultiTouch: (enabled: boolean) => void
enabled: boolean
raycastTarget: boolean
```
@@ -6838,6 +6843,13 @@ static new (atlasSize: number): PageTable
static prototype: PageTable
```
+## Panorama — interface @experimental
+```
+height: number
+rgb: Float32Array<ArrayBufferLike>
+width: number
+```
+
## Parent — const @public
```
BuiltinComponentDef<ParentData>
@@ -9213,6 +9225,7 @@ audioCacheBudget: number | undefined
defaultFontFamily: string | undefined
maxDeltaTime: number | undefined
maxFixedSteps: number | undefined
+multiTouch: boolean | undefined
retainedBudget: number | undefined
sceneTransitionColor: string | undefined
sceneTransitionDuration: number | undefined
@@ -9255,7 +9268,6 @@ uiTheme: "light" | "dark" | undefined
uiThemeOverrides: ThemeOverrides | undefined
worlds: WorldManifest[] | undefined
```
-multiTouch: boolean | undefined
## RuntimeOnly — const @public
```
@@ -14505,6 +14517,11 @@ AssetPlugin
(atlas: CookedAtlasInfo, pagePath: string): Pick<CatalogEntry, "atlas" | "frame" | "uv">
```
+## atlasLayout — function @experimental
+```
+(faceSize: number, mipCount: number): { width: number; height: number; offsets: number[]; }
+```
+
## attachTooltip — function @beta
```
(world: World, target: Entity, options?: Partial<UITooltipData>): void
@@ -15225,6 +15242,11 @@ Plugin
(bytes: Uint8Array): MeshData
```
+## decodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, a: number, maxRange: number): [number, number, number]
+```
+
## decomposePolygon2D — function @experimental
```
(vertices: readonly Vec2[]): PolygonDecomposition
@@ -15515,6 +15537,11 @@ DragPlugin
(data: MeshData): Uint8Array
```
+## encodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, maxRange: number, out: Uint8Array, at: number): void
+```
+
## engineWebGPUFeatures — function @experimental
```
(adapter: WebGPUAdapterLike): string[]
@@ -16585,6 +16612,16 @@ NodePlatformAdapter
(rings: number, cellSize: number): number
```
+## octDecode — function @experimental
+```
+(u: number, v: number): [number, number, number]
+```
+
+## octEncode — function @experimental
+```
+(dx: number, dy: number, dz: number): [number, number]
+```
+
## onCameraCommitChanged — function @beta
```
(fn: CameraCommitListener): () => void
@@ -16970,6 +17007,11 @@ PostProcessPlugin
PrefabsPlugin
```
+## prefilterOctahedral — function @experimental
+```
+(env: Panorama, faceSize: number, mipCount: number, maxRange: number): { width: number; height: number; rgba: Uint8Array; }
+```
+
## preloadGlyphs — function @experimental
```
(app: App, entity: Entity, text: string, options?: GlyphPreloadOptions): GlyphPreload
@@ -17000,6 +17042,11 @@ PrefabsPlugin
(p: ProceduralParams): Heightfield
```
+## projectIrradianceSH — function @experimental
+```
+(env: Panorama): Float32Array
+```
+
## projectSideModuleIds — function @experimental
```
(): string[]
@@ -17405,6 +17452,11 @@ SafeAreaPlugin
(keys: readonly Keyframe[], at: number): number
```
+## samplePanorama — function @experimental
+```
+(env: Panorama, dx: number, dy: number, dz: number, out: Float32Array): void
+```
+
## sampleRootPlacement — function @experimental
```
(asset: TimelineAsset, time: number, out: RootPlacement): boolean
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index dbb61de73..7f3bafe1e 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/minigame
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2158 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2170 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -1419,7 +1419,7 @@ type: number | undefined
## BAKE_DEFAULTS — const @experimental
```
-{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
+{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; skyOcclusion: boolean; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
```
## BONE_LIMITS — const @experimental
@@ -1512,6 +1512,7 @@ ambient: readonly [number, number, number] | undefined
atlasSize: number | undefined
bounces: number | undefined
dilate: number | undefined
+occluders: readonly BakeSurface[] | undefined
probeGrids: readonly ProbeGrid[] | undefined
probeSamples: number | undefined
reflectionProbes: readonly (readonly [number, number, number])[] | undefined
@@ -1519,6 +1520,7 @@ reflectionSky: SkyRadiance | undefined
reflectionWidth: number | undefined
samples: number | undefined
sky: SkySpec | SkyRadiance | undefined
+skyOcclusion: boolean | undefined
texelsPerUnit: number | undefined
```
@@ -1532,6 +1534,7 @@ progress: number
```
lumels: number
pixels: Uint8Array<ArrayBufferLike>
+probeSkyVisibility: Float32Array<ArrayBufferLike>[]
probes: Float32Array<ArrayBufferLike>[]
reflections: CapturedPanorama[]
scaleOffset: [number, number, number, number][]
@@ -1579,6 +1582,7 @@ bakedFrom: string
bounces: number
probeSamples: number
samples: number
+skyOcclusion: boolean
texelsPerUnit: number
```
@@ -5049,6 +5053,7 @@ hand: number | null
joints: number
missing: string[]
pose: string
+touching: string[]
weight: number
```
@@ -5499,7 +5504,10 @@ static prototype: InputRouter
## InputState — class @beta
```
+@internal admitTouch: (id: number) => boolean
@internal cancelVirtualSources: () => void
+@internal forgetTouches: () => void
+@internal touchIgnored: (id: number, retiring?: boolean) => boolean
beginFixedStep: () => void
clearFrameState: () => void
clearVirtual: () => void
@@ -5531,12 +5539,10 @@ isKeyReleased: (key: string) => boolean
isMouseButtonDown: (button: number) => boolean
isMouseButtonPressed: (button: number) => boolean
isMouseButtonReleased: (button: number) => boolean
+isMultiTouchEnabled: () => boolean
isPointerOverUI: () => boolean
isTouchActive: (id: number) => boolean
-@internal admitTouch: (id: number) => boolean
keysDown: Set<string>
-@internal forgetTouches: () => void
-@internal touchIgnored: (id: number, retiring?: boolean) => boolean
keysPressed: Set<string>
keysPressedFixed: Set<string>
keysReleased: Set<string>
@@ -5559,6 +5565,7 @@ releaseGamepad: (index?: number) => void
removeVirtualSource: (owner: object) => void
scrollDeltaX: number
scrollDeltaY: number
+setMultiTouch: (enabled: boolean) => void
setVirtual: (id: string, x: number, y?: number) => void
setVirtualSource: (owner: object, id: string, x: number, y?: number) => void
touchAvailable: boolean
@@ -5568,7 +5575,6 @@ touchesStarted: Map<number, TouchPoint>
updateGamepads: (snapshots: GamepadSnapshot[]) => void
static new (): InputState
static prototype: InputState
-isMultiTouchEnabled: () => boolean
```
## InstantiatePrefabOptions — interface @experimental
@@ -5594,7 +5600,6 @@ BuiltinComponentDef<InteractableData>
## InteractableData — interface @experimental
```
blockRaycast: boolean
-setMultiTouch: (enabled: boolean) => void
enabled: boolean
raycastTarget: boolean
```
@@ -7134,6 +7139,13 @@ static new (atlasSize: number): PageTable
static prototype: PageTable
```
+## Panorama — interface @experimental
+```
+height: number
+rgb: Float32Array<ArrayBufferLike>
+width: number
+```
+
## Parent — const @public
```
BuiltinComponentDef<ParentData>
@@ -9509,6 +9521,7 @@ audioCacheBudget: number | undefined
defaultFontFamily: string | undefined
maxDeltaTime: number | undefined
maxFixedSteps: number | undefined
+multiTouch: boolean | undefined
retainedBudget: number | undefined
sceneTransitionColor: string | undefined
sceneTransitionDuration: number | undefined
@@ -9551,7 +9564,6 @@ uiTheme: "light" | "dark" | undefined
uiThemeOverrides: ThemeOverrides | undefined
worlds: WorldManifest[] | undefined
```
-multiTouch: boolean | undefined
## RuntimeOnly — const @public
```
@@ -14807,6 +14819,11 @@ AssetPlugin
(atlas: CookedAtlasInfo, pagePath: string): Pick<CatalogEntry, "atlas" | "frame" | "uv">
```
+## atlasLayout — function @experimental
+```
+(faceSize: number, mipCount: number): { width: number; height: number; offsets: number[]; }
+```
+
## attachTooltip — function @beta
```
(world: World, target: Entity, options?: Partial<UITooltipData>): void
@@ -15532,6 +15549,11 @@ Plugin
(bytes: Uint8Array): MeshData
```
+## decodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, a: number, maxRange: number): [number, number, number]
+```
+
## decomposePolygon2D — function @experimental
```
(vertices: readonly Vec2[]): PolygonDecomposition
@@ -15822,6 +15844,11 @@ DragPlugin
(data: MeshData): Uint8Array
```
+## encodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, maxRange: number, out: Uint8Array, at: number): void
+```
+
## engineWebGPUFeatures — function @experimental
```
(adapter: WebGPUAdapterLike): string[]
@@ -16892,6 +16919,16 @@ MeshRendererPlugin
(rings: number, cellSize: number): number
```
+## octDecode — function @experimental
+```
+(u: number, v: number): [number, number, number]
+```
+
+## octEncode — function @experimental
+```
+(dx: number, dy: number, dz: number): [number, number]
+```
+
## onCameraCommitChanged — function @beta
```
(fn: CameraCommitListener): () => void
@@ -17277,6 +17314,11 @@ PostProcessPlugin
PrefabsPlugin
```
+## prefilterOctahedral — function @experimental
+```
+(env: Panorama, faceSize: number, mipCount: number, maxRange: number): { width: number; height: number; rgba: Uint8Array; }
+```
+
## preloadGlyphs — function @experimental
```
(app: App, entity: Entity, text: string, options?: GlyphPreloadOptions): GlyphPreload
@@ -17307,6 +17349,11 @@ PrefabsPlugin
(p: ProceduralParams): Heightfield
```
+## projectIrradianceSH — function @experimental
+```
+(env: Panorama): Float32Array
+```
+
## projectSideModuleIds — function @experimental
```
(): string[]
@@ -17712,6 +17759,11 @@ SafeAreaPlugin
(keys: readonly Keyframe[], at: number): number
```
+## samplePanorama — function @experimental
+```
+(env: Panorama, dx: number, dy: number, dz: number, out: Float32Array): void
+```
+
## sampleRootPlacement — function @experimental
```
(asset: TimelineAsset, time: number, out: RootPlacement): boolean
diff --git a/sdk/etc/wasm.api.md b/sdk/etc/wasm.api.md
index 864bfa7e9..d570e3170 100644
--- a/sdk/etc/wasm.api.md
+++ b/sdk/etc/wasm.api.md
@@ -454,6 +454,7 @@ postprocess_shutdown: () => void
postprocess_supportsCompute: (() => boolean) | undefined
probe_volume_create: ((resX: number, resY: number, resZ: number, shPtr: number) => number) | undefined
probe_volume_release: ((volumeHandle: number) => void) | undefined
+probe_volume_setSkyVisibility: ((volumeHandle: number, visibilityPtr: number) => void) | undefined
provideReplacementDevice: (() => boolean) | undefined
recoverDevice: (() => boolean) | undefined
registryLayoutEpoch: (registry: CppRegistry) => number
@@ -555,6 +556,7 @@ renderer_setPlanarReflectionQuality: ((scale: number, maxSize: number, updatesPe
renderer_setQualityBudgets: ((atlasSize: number, cellSize: number, cascades: number, particleLimit: number) => void) | undefined
renderer_setShadow2DSoftSamples: ((samples: number) => void) | undefined
renderer_setShadowDistance: ((distance: number) => void) | undefined
+renderer_setSkyEnvironment: ((environmentHandle: number) => void) | undefined
renderer_setStage: (stage: number) => void
renderer_setTextureParams: (textureId: number, minFilter: number, magFilter: number, wrapS: number, wrapT: number) => void
renderer_setTreeSortLayers: ((mask: number) => void) | undefined
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index fd29c5198..26864ab98 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -1,7 +1,7 @@
# API surface — esengine/wechat
<!-- Generated by tools/api-surface.mjs — do not edit. Run --update to accept changes. -->
-Symbols: 115 public · 166 beta · 2149 experimental · 3 deprecated
+Symbols: 115 public · 166 beta · 2161 experimental · 3 deprecated
## ANCHOR_AXES — const @experimental
```
@@ -1419,7 +1419,7 @@ type: number | undefined
## BAKE_DEFAULTS — const @experimental
```
-{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
+{ atlasSize: number; texelsPerUnit: number; bounces: number; samples: number; ambient: readonly [number, number, number]; dilate: number; probeGrids: readonly ProbeGrid[]; probeSamples: number; skyOcclusion: boolean; reflectionProbes: readonly (readonly [number, number, number])[]; reflectionWidth: number; }
```
## BONE_LIMITS — const @experimental
@@ -1512,6 +1512,7 @@ ambient: readonly [number, number, number] | undefined
atlasSize: number | undefined
bounces: number | undefined
dilate: number | undefined
+occluders: readonly BakeSurface[] | undefined
probeGrids: readonly ProbeGrid[] | undefined
probeSamples: number | undefined
reflectionProbes: readonly (readonly [number, number, number])[] | undefined
@@ -1519,6 +1520,7 @@ reflectionSky: SkyRadiance | undefined
reflectionWidth: number | undefined
samples: number | undefined
sky: SkySpec | SkyRadiance | undefined
+skyOcclusion: boolean | undefined
texelsPerUnit: number | undefined
```
@@ -1532,6 +1534,7 @@ progress: number
```
lumels: number
pixels: Uint8Array<ArrayBufferLike>
+probeSkyVisibility: Float32Array<ArrayBufferLike>[]
probes: Float32Array<ArrayBufferLike>[]
reflections: CapturedPanorama[]
scaleOffset: [number, number, number, number][]
@@ -1579,6 +1582,7 @@ bakedFrom: string
bounces: number
probeSamples: number
samples: number
+skyOcclusion: boolean
texelsPerUnit: number
```
@@ -5049,6 +5053,7 @@ hand: number | null
joints: number
missing: string[]
pose: string
+touching: string[]
weight: number
```
@@ -5506,7 +5511,10 @@ static prototype: InputRouter
## InputState — class @beta
```
+@internal admitTouch: (id: number) => boolean
@internal cancelVirtualSources: () => void
+@internal forgetTouches: () => void
+@internal touchIgnored: (id: number, retiring?: boolean) => boolean
beginFixedStep: () => void
clearFrameState: () => void
clearVirtual: () => void
@@ -5538,12 +5546,10 @@ isKeyReleased: (key: string) => boolean
isMouseButtonDown: (button: number) => boolean
isMouseButtonPressed: (button: number) => boolean
isMouseButtonReleased: (button: number) => boolean
+isMultiTouchEnabled: () => boolean
isPointerOverUI: () => boolean
isTouchActive: (id: number) => boolean
-@internal admitTouch: (id: number) => boolean
keysDown: Set<string>
-@internal forgetTouches: () => void
-@internal touchIgnored: (id: number, retiring?: boolean) => boolean
keysPressed: Set<string>
keysPressedFixed: Set<string>
keysReleased: Set<string>
@@ -5566,6 +5572,7 @@ releaseGamepad: (index?: number) => void
removeVirtualSource: (owner: object) => void
scrollDeltaX: number
scrollDeltaY: number
+setMultiTouch: (enabled: boolean) => void
setVirtual: (id: string, x: number, y?: number) => void
setVirtualSource: (owner: object, id: string, x: number, y?: number) => void
touchAvailable: boolean
@@ -5575,7 +5582,6 @@ touchesStarted: Map<number, TouchPoint>
updateGamepads: (snapshots: GamepadSnapshot[]) => void
static new (): InputState
static prototype: InputState
-isMultiTouchEnabled: () => boolean
```
## InstantiatePrefabOptions — interface @experimental
@@ -5601,7 +5607,6 @@ BuiltinComponentDef<InteractableData>
## InteractableData — interface @experimental
```
blockRaycast: boolean
-setMultiTouch: (enabled: boolean) => void
enabled: boolean
raycastTarget: boolean
```
@@ -6845,6 +6850,13 @@ static new (atlasSize: number): PageTable
static prototype: PageTable
```
+## Panorama — interface @experimental
+```
+height: number
+rgb: Float32Array<ArrayBufferLike>
+width: number
+```
+
## Parent — const @public
```
BuiltinComponentDef<ParentData>
@@ -9220,6 +9232,7 @@ audioCacheBudget: number | undefined
defaultFontFamily: string | undefined
maxDeltaTime: number | undefined
maxFixedSteps: number | undefined
+multiTouch: boolean | undefined
retainedBudget: number | undefined
sceneTransitionColor: string | undefined
sceneTransitionDuration: number | undefined
@@ -9262,7 +9275,6 @@ uiTheme: "light" | "dark" | undefined
uiThemeOverrides: ThemeOverrides | undefined
worlds: WorldManifest[] | undefined
```
-multiTouch: boolean | undefined
## RuntimeOnly — const @public
```
@@ -14538,6 +14550,11 @@ AssetPlugin
(atlas: CookedAtlasInfo, pagePath: string): Pick<CatalogEntry, "atlas" | "frame" | "uv">
```
+## atlasLayout — function @experimental
+```
+(faceSize: number, mipCount: number): { width: number; height: number; offsets: number[]; }
+```
+
## attachTooltip — function @beta
```
(world: World, target: Entity, options?: Partial<UITooltipData>): void
@@ -15258,6 +15275,11 @@ Plugin
(bytes: Uint8Array): MeshData
```
+## decodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, a: number, maxRange: number): [number, number, number]
+```
+
## decomposePolygon2D — function @experimental
```
(vertices: readonly Vec2[]): PolygonDecomposition
@@ -15548,6 +15570,11 @@ DragPlugin
(data: MeshData): Uint8Array
```
+## encodeRgbm — function @experimental
+```
+(r: number, g: number, b: number, maxRange: number, out: Uint8Array, at: number): void
+```
+
## engineWebGPUFeatures — function @experimental
```
(adapter: WebGPUAdapterLike): string[]
@@ -16618,6 +16645,16 @@ MeshRendererPlugin
(rings: number, cellSize: number): number
```
+## octDecode — function @experimental
+```
+(u: number, v: number): [number, number, number]
+```
+
+## octEncode — function @experimental
+```
+(dx: number, dy: number, dz: number): [number, number]
+```
+
## onCameraCommitChanged — function @beta
```
(fn: CameraCommitListener): () => void
@@ -17003,6 +17040,11 @@ PostProcessPlugin
PrefabsPlugin
```
+## prefilterOctahedral — function @experimental
+```
+(env: Panorama, faceSize: number, mipCount: number, maxRange: number): { width: number; height: number; rgba: Uint8Array; }
+```
+
## preloadGlyphs — function @experimental
```
(app: App, entity: Entity, text: string, options?: GlyphPreloadOptions): GlyphPreload
@@ -17033,6 +17075,11 @@ PrefabsPlugin
(p: ProceduralParams): Heightfield
```
+## projectIrradianceSH — function @experimental
+```
+(env: Panorama): Float32Array
+```
+
## projectSideModuleIds — function @experimental
```
(): string[]
@@ -17438,6 +17485,11 @@ SafeAreaPlugin
(keys: readonly Keyframe[], at: number): number
```
+## samplePanorama — function @experimental
+```
+(env: Panorama, dx: number, dy: number, dz: number, out: Float32Array): void
+```
+
## sampleRootPlacement — function @experimental
```
(asset: TimelineAsset, time: number, out: RootPlacement): boolean
diff --git a/sdk/src/asset/loaders/ProbeVolumeAssetLoader.ts b/sdk/src/asset/loaders/ProbeVolumeAssetLoader.ts
index b32c0be68..401e1d79d 100644
--- a/sdk/src/asset/loaders/ProbeVolumeAssetLoader.ts
+++ b/sdk/src/asset/loaders/ProbeVolumeAssetLoader.ts
@@ -12,6 +12,9 @@ interface ProbeVolumeAssetData {
resolution: [number, number, number];
/** Nine RGB coefficients per probe, x fastest then y then z. */
irradiance: number[];
+ /** With sky occlusion: nine coefficients per probe of how much of the sky each sees; the irradiance
+ * then holds the lights alone, and the frame's sky is added through these. */
+ skyVisibility?: number[];
}
/** A baked probe grid, named by the handle a LightProbeVolume references. */
@@ -60,6 +63,22 @@ export class ProbeVolumeAssetLoader implements AssetLoader<ProbeVolumeResult> {
if (!handle) {
throw new Error(`the engine rejected the probe volume in ${path}`);
}
+ if (data.skyVisibility !== undefined) {
+ const visibility = data.skyVisibility;
+ const count = res[0] * res[1] * res[2] * 9;
+ if (!Array.isArray(visibility) || visibility.length !== count) {
+ throw new Error(`${path}: ${res[0]}x${res[1]}x${res[2]} probes want ${count} sky visibility `
+ + `coefficients, got ${Array.isArray(visibility) ? visibility.length : 0}`);
+ }
+ if (!m.probe_volume_setSkyVisibility) {
+ throw new Error(`${path} is baked with sky occlusion, which this engine build cannot read`);
+ }
+ withScratch(m, (alloc) => {
+ const ptr = alloc(count * 4);
+ m.HEAPF32.set(Float32Array.from(visibility), ptr >> 2);
+ m.probe_volume_setSkyVisibility!(handle, ptr);
+ });
+ }
return { handle };
}
diff --git a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
index 713192e24..f01ee52f7 100644
--- a/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
+++ b/sdk/src/ecs/bridge/nativeEngineApi.generated.ts
@@ -137,6 +137,7 @@ export interface NativeEngineApi {
postprocess_supportsCompute?(): boolean;
probe_volume_create?(resX: number, resY: number, resZ: number, shPtr: number): number;
probe_volume_release?(volumeHandle: number): void;
+ probe_volume_setSkyVisibility?(volumeHandle: number, visibilityPtr: number): void;
registry_batchSyncPhysicsTransforms?(registry: unknown, bufferPtr: number, count: number, ppu: number): void;
registry_componentMutationRevision?(registry: unknown, component: number, topology: boolean): number;
registry_entityChangeResetEpoch?(registry: unknown): number;
@@ -480,6 +481,7 @@ export function createNativeEngineApi(
bind('postprocess_supportsCompute', 'es_postprocess_supportsCompute', false);
bind('probe_volume_create', 'es_probe_volume_create', false);
bind('probe_volume_release', 'es_probe_volume_release', false);
+ bind('probe_volume_setSkyVisibility', 'es_probe_volume_setSkyVisibility', false);
bind('registry_batchSyncPhysicsTransforms', 'es_registry_batchSyncPhysicsTransforms', true);
bind('registry_componentMutationRevision', 'es_registry_componentMutationRevision', true);
bind('registry_entityChangeResetEpoch', 'es_registry_entityChangeResetEpoch', true);
diff --git a/sdk/src/lightmap/bake.ts b/sdk/src/lightmap/bake.ts
index 021f31393..0ab0d2fbf 100644
--- a/sdk/src/lightmap/bake.ts
+++ b/sdk/src/lightmap/bake.ts
@@ -48,6 +48,13 @@ export interface BakeOptions {
* a bake cannot hold still. Solved after the surfaces, because a probe
* gathers what they ended up giving off. */
probeGrids?: readonly ProbeGrid[];
+ /** Bake the probes WITHOUT the sky, and how much of it each sees: the frame then lights them with
+ * the sky it has (a live Sky Atmosphere, a weather) seen through that, so a room stays dark under
+ * a sky that changes. Off, the probes hold the sky as it was at the bake. */
+ skyOcclusion?: boolean;
+ /** What stands in the light and holds none of it: geometry with no lightmap UV set (lit by probes when the
+ * game runs). It shadows the surfaces and hides the sky from the probes; a ray that meets it finds it unlit. */
+ occluders?: readonly BakeSurface[];
/** Directions each probe gathers. Over the whole sphere, so this buys less
* per ray than a lumel's hemisphere does. */
probeSamples?: number;
@@ -77,6 +84,9 @@ export interface BakeResult {
/** Nine RGB coefficients per probe, one array per requested grid, in grid
* order with x varying fastest — what a `.esprobes` carries. */
probes: Float32Array[];
+ /** With {@link BakeOptions.skyOcclusion}: nine coefficients per probe, one array per grid, of how
+ * much of the sky each sees. Empty without it. */
+ probeSkyVisibility: Float32Array[];
/** One captured sphere per {@link BakeOptions.reflectionProbes}, in the order
* they were given, for the caller to prefilter into an atlas column. */
reflections: CapturedPanorama[];
@@ -97,6 +107,7 @@ export const BAKE_DEFAULTS = {
dilate: 2,
probeGrids: [] as readonly ProbeGrid[],
probeSamples: 128,
+ skyOcclusion: false,
reflectionProbes: [] as readonly (readonly [number, number, number])[],
reflectionWidth: 64,
};
@@ -104,9 +115,15 @@ export const BAKE_DEFAULTS = {
const chan = (m: MeshData, s: number): MeshChannelDesc | undefined =>
m.channels.find((c) => c.semantic === s);
-/** Every surface's triangles in world space, plus what a ray hit has to know. */
-function collect(surfaces: readonly BakeSurface[], patches: readonly SurfacePatch[],
- size: number, texelsPerUnit: number): { soup: TriangleSoup; lookup: HitLookup } {
+/** Every surface's triangles in world space, plus what a ray hit has to know. Occluders come last, with no
+ * place in the atlas: a ray that meets one finds something there, unlit. */
+function collect(lit: readonly BakeSurface[], litPatches: readonly SurfacePatch[],
+ size: number, texelsPerUnit: number,
+ occluders: readonly BakeSurface[] = []): { soup: TriangleSoup; lookup: HitLookup } {
+ const surfaces = occluders.length ? [...lit, ...occluders] : lit;
+ const patches: readonly SurfacePatch[] = occluders.length
+ ? [...litPatches, ...occluders.map((): SurfacePatch => ({ side: 0, x: -1, y: -1, rotated: false, scaleOffset: [0, 0, 0, 0] }))]
+ : litPatches;
let total = 0;
for (const s of surfaces) total += Math.floor(s.mesh.indices.length / 3);
const positions = new Float32Array(total * 9);
@@ -321,7 +338,7 @@ export function* bakeLightmapSteps(surfaces: readonly BakeSurface[], lights: rea
const patches = layoutAtlas(surfaces, size, opts.texelsPerUnit);
if (!patches) throw new Error(refusal(surfaces, size, opts.texelsPerUnit));
const lumels = rasterizeLumels(surfaces, patches, size);
- const { soup, lookup } = collect(surfaces, patches, size, opts.texelsPerUnit);
+ const { soup, lookup } = collect(surfaces, patches, size, opts.texelsPerUnit, options.occluders ?? []);
const skyOf = options.sky ?? { kind: 'flat', rgb: opts.ambient };
const scene: BakeScene = { lumels, tris: soup, lookup, lights, sky: skyOf,
atlasSize: size, samples: opts.samples };
@@ -368,8 +385,13 @@ export function* bakeLightmapSteps(surfaces: readonly BakeSurface[], lights: rea
// surfaces ended up giving off, and before it is quantised to eight bits.
let built: Bvh | null = null;
const bvh = (): Bvh => (built ??= new Bvh(soup));
- const probes = opts.probeGrids.map((grid) =>
- solveProbes(grid, bvh(), lookup, radiance, size, opts.probeSamples, sky));
+ const probeSkyVisibility: Float32Array[] = [];
+ const probes = opts.probeGrids.map((grid) => {
+ const visibility = opts.skyOcclusion
+ ? new Float32Array(Math.max(0, grid.resolution[0] * grid.resolution[1] * grid.resolution[2]) * 9) : undefined;
+ if (visibility) probeSkyVisibility.push(visibility);
+ return solveProbes(grid, bvh(), lookup, radiance, size, opts.probeSamples, sky, visibility);
+ });
// Captured from the same field, after the same bounces: a reflection of a
// wall and the light that wall casts are then the same number.
@@ -385,6 +407,7 @@ export function* bakeLightmapSteps(surfaces: readonly BakeSurface[], lights: rea
lumels: lumels.count,
shared: lumels.shared,
probes,
+ probeSkyVisibility,
reflections,
};
}
diff --git a/sdk/src/lightmap/components.ts b/sdk/src/lightmap/components.ts
index 332e0dc76..64e503ac7 100644
--- a/sdk/src/lightmap/components.ts
+++ b/sdk/src/lightmap/components.ts
@@ -28,6 +28,9 @@ export interface BakedLightingData {
/** Directions each probe gathers — over the whole sphere, so this buys less
* per ray than a lumel's hemisphere does. */
probeSamples: number;
+ /** Bake the probes without the sky and with how much of it each sees, so the sky the frame has
+ * (a live Sky Atmosphere, a weather) lights them through it: a room stays dark under any sky. */
+ skyOcclusion: boolean;
/** What the last bake was OF. Empty until one has run; different from what
* the scene fingerprints to now means the light is out of date. The baker
* writes it — not `readonly`, which means "the engine computes this and the
@@ -52,6 +55,7 @@ export const BakedLighting: ComponentDef<BakedLightingData> = defineComponent<Ba
bounces: BAKE_DEFAULTS.bounces,
samples: BAKE_DEFAULTS.samples,
probeSamples: BAKE_DEFAULTS.probeSamples,
+ skyOcclusion: BAKE_DEFAULTS.skyOcclusion,
bakedFrom: '',
},
{
@@ -62,6 +66,7 @@ export const BakedLighting: ComponentDef<BakedLightingData> = defineComponent<Ba
bounces: { min: 0, max: 16, tooltip: 'How many times light may reflect. 0 = direct only. An enclosed room of light walls needs 8 or more, or it bakes too dark.' },
samples: { min: 1, tooltip: 'Rays each lumel gathers per bounce.' },
probeSamples: { min: 1, tooltip: 'Directions each probe gathers, over the whole sphere.' },
+ skyOcclusion: { tooltip: 'Probes keep how much sky they see, not the sky itself: the sky in play (a live Sky Atmosphere, a weather) lights them through it, so rooms stay dark.' },
bakedFrom: { advanced: true, tooltip: 'Fingerprint of the inputs the last bake read.' },
},
},
diff --git a/sdk/src/lightmap/fingerprint.ts b/sdk/src/lightmap/fingerprint.ts
index 439d3783e..5800b6eb9 100644
--- a/sdk/src/lightmap/fingerprint.ts
+++ b/sdk/src/lightmap/fingerprint.ts
@@ -87,7 +87,9 @@ export function bakeFingerprint(inputs: BakeInputs): string {
for (const r of inputs.reflections ?? []) lines.push(`r|${nums(r)}`);
lines.sort();
lines.push(`o|${opts.atlasSize}|${opts.texelsPerUnit}|${opts.bounces}|${opts.samples}`
- + `|${opts.probeSamples}|${opts.dilate}|${nums(inputs.ambient ?? opts.ambient)}`);
+ + `|${opts.probeSamples}|${opts.dilate}|${nums(inputs.ambient ?? opts.ambient)}`
+ // Only where it is on: a scene that never asked for it keeps the fingerprint its last bake wrote.
+ + (opts.skyOcclusion ? '|skyOcclusion' : ''));
const text = lines.join('\n');
const hex = (v: number): string => v.toString(16).padStart(8, '0');
return hex(fnv1a(text, 0x811c9dc5)) + hex(fnv1a(text, 0x9e3779b9));
diff --git a/sdk/src/lightmap/probes.ts b/sdk/src/lightmap/probes.ts
index 912b60c61..42f45fbb5 100644
--- a/sdk/src/lightmap/probes.ts
+++ b/sdk/src/lightmap/probes.ts
@@ -13,7 +13,7 @@
import type { Bvh } from './bvh';
import { rayRadiance, type HitLookup } from './solve';
import type { SkyRadiance } from './reflection';
-import { shBasis, convolveCosine } from './sh';
+import { shBasis, convolveCosine, convolveCosineScalar } from './sh';
/** Where one grid of probes sits, and how many. */
export interface ProbeGrid {
@@ -61,10 +61,16 @@ export function probeAt(min: number, max: number, i: number, n: number): number
* `atlas` is the solved light field indexed by texel — what a surface gives off,
* before its own albedo. A ray that hits geometry with no place in the atlas
* returns black rather than the sky: something is there, and it is unlit.
+ *
+ * With `skyVisibility` (nine floats per probe, zeroed), the sky is NOT gathered: a
+ * ray that escapes adds to how much of the sky the probe sees instead, convolved
+ * the same way, so the frame can light the probe with whatever sky it has then
+ * (sky occlusion). What the sky gave the surfaces the probe sees, their bounce, is
+ * still in the atlas and so still in the irradiance.
*/
export function solveProbes(grid: ProbeGrid, bvh: Bvh, lookup: HitLookup,
atlas: Float32Array, atlasSize: number, samples: number,
- sky: SkyRadiance): Float32Array {
+ sky: SkyRadiance, skyVisibility?: Float32Array): Float32Array {
const [nx, ny, nz] = grid.resolution;
const count = Math.max(0, nx) * Math.max(0, ny) * Math.max(0, nz);
const out = new Float32Array(count * 27);
@@ -83,14 +89,19 @@ export function solveProbes(grid: ProbeGrid, bvh: Bvh, lookup: HitLookup,
for (let x = 0; x < nx; x++) {
const px = probeAt(grid.min[0], grid.max[0], x, nx);
const at = (x + y * nx + z * nx * ny) * 27;
+ const vAt = (x + y * nx + z * nx * ny) * 9;
for (let s = 0; s < samples; s++) {
const dx = dirs[s * 3]!, dy = dirs[s * 3 + 1]!, dz = dirs[s * 3 + 2]!;
+ shBasis(dx, dy, dz, basis);
if (!rayRadiance(bvh, lookup, atlas, atlasSize, px, py, pz, dx, dy, dz,
seen, 0, at * samples + s)) {
+ if (skyVisibility) {
+ for (let i = 0; i < 9; i++) skyVisibility[vAt + i]! += basis[i]! * weight;
+ continue;
+ }
sky(dx, dy, dz, seen, 0);
}
const r = seen[0]!, g = seen[1]!, b = seen[2]!;
- shBasis(dx, dy, dz, basis);
for (let i = 0; i < 9; i++) {
const w = basis[i]! * weight;
out[at + i * 3]! += r * w;
@@ -99,6 +110,7 @@ export function solveProbes(grid: ProbeGrid, bvh: Bvh, lookup: HitLookup,
}
}
convolveCosine(out, at);
+ if (skyVisibility) convolveCosineScalar(skyVisibility, vAt);
}
}
}
diff --git a/sdk/src/lightmap/sh.ts b/sdk/src/lightmap/sh.ts
index 84ea7599e..cf68eb412 100644
--- a/sdk/src/lightmap/sh.ts
+++ b/sdk/src/lightmap/sh.ts
@@ -45,6 +45,11 @@ export function convolveCosine(coefficients: Float32Array, at = 0): void {
}
}
+/** {@link convolveCosine} for nine scalar coefficients (a visibility, not a colour), in place. */
+export function convolveCosineScalar(coefficients: Float32Array, at = 0): void {
+ for (let i = 0; i < 9; i++) coefficients[at + i]! *= SH_COSINE_BAND[i];
+}
+
/** Evaluate nine RGB coefficients at `d` — the shader's reconstruction, in
* TypeScript, so a test can state what a pixel should be. */
export function evalIrradianceSH(coefficients: ArrayLike<number>,
diff --git a/sdk/src/wasm.ts b/sdk/src/wasm.ts
index 128a56a34..127d9d2da 100644
--- a/sdk/src/wasm.ts
+++ b/sdk/src/wasm.ts
@@ -367,6 +367,8 @@ export interface ESEngineModule {
probe_volume_create?(resX: number, resY: number, resZ: number, shPtr: number): number;
/** Release a probe volume. */
probe_volume_release?(volumeHandle: number): void;
+ /** Nine floats per probe of how much of the sky each sees (a bake with sky occlusion). */
+ probe_volume_setSkyVisibility?(volumeHandle: number, visibilityPtr: number): void;
/** The same for every MeshRenderer in the world; returns how many were frozen. */
meshRenderer_makeAllResident?(registry: CppRegistry): number;
/**
diff --git a/sdk/tests/lightmap-probes.test.ts b/sdk/tests/lightmap-probes.test.ts
index d30cefefd..2853eb937 100644
--- a/sdk/tests/lightmap-probes.test.ts
+++ b/sdk/tests/lightmap-probes.test.ts
@@ -51,6 +51,54 @@ const baked = (mesh: MeshData): MeshData => unwrapLightmapUV(mesh).mesh;
/** One probe at the centre of a box that holds nothing. */
const ONE: ProbeGrid = { min: [-50, -50, -50], max: [50, 50, 50], resolution: [1, 1, 1] };
+/** Nine scalar coefficients at `d`, the shaders' skyOcclusion without its clamp. */
+function visibilityAt(v: Float32Array, d: readonly number[], at = 0): number {
+ const [r] = evalIrradianceSH(Float32Array.from([...v.slice(at, at + 9)].flatMap((c) => [c, c, c])), d[0]!, d[1]!, d[2]!);
+ return r;
+}
+
+describe('a probe grid baked with sky occlusion', () => {
+ it('holds no sky, and sees all of it where nothing stands', () => {
+ const result = bakeLightmap([], [], {
+ ambient: [0.25, 0.5, 0.75], probeGrids: [ONE], probeSamples: 256, skyOcclusion: true,
+ });
+ const sh = result.probes[0]!;
+ const sky = result.probeSkyVisibility[0]!;
+ expect(sky.length).toBe(9);
+ for (const d of [[0, 1, 0], [1, 0, 0], [0, -1, 0]]) {
+ expect(evalIrradianceSH(sh, d[0]!, d[1]!, d[2]!).every((c) => Math.abs(c) < 1e-6)).toBe(true);
+ expect(visibilityAt(sky, d)).toBeCloseTo(1, 1);
+ }
+ });
+
+ it('sees less sky toward a wall than away from it', () => {
+ const wall: BakeSurface = { mesh: baked(facingBack(400)), transform: translated(20, 0, 0), albedo: [0.5, 0.5, 0.5] };
+ const result = bakeLightmap([wall], [], {
+ ambient: [1, 1, 1], probeGrids: [ONE], probeSamples: 512, skyOcclusion: true, bounces: 0,
+ });
+ const sky = result.probeSkyVisibility[0]!;
+ expect(visibilityAt(sky, [1, 0, 0])).toBeLessThan(0.4);
+ expect(visibilityAt(sky, [-1, 0, 0])).toBeGreaterThan(0.85);
+ });
+
+ it('is hidden by geometry that holds no baked light of its own (an occluder)', () => {
+ // The same wall with no lightmap UV set: it receives nothing, and still stands between the probe and the sky.
+ const wall: BakeSurface = { mesh: facingBack(400), transform: translated(20, 0, 0), albedo: [0.5, 0.5, 0.5] };
+ const result = bakeLightmap([], [], {
+ ambient: [1, 1, 1], probeGrids: [ONE], probeSamples: 512, skyOcclusion: true, occluders: [wall],
+ });
+ const sky = result.probeSkyVisibility[0]!;
+ expect(visibilityAt(sky, [1, 0, 0])).toBeLessThan(0.4);
+ expect(visibilityAt(sky, [-1, 0, 0])).toBeGreaterThan(0.85);
+ });
+
+ it('is not baked unless asked for', () => {
+ const result = bakeLightmap([], [], { ambient: [1, 1, 1], probeGrids: [ONE], probeSamples: 64 });
+ expect(result.probeSkyVisibility).toEqual([]);
+ expect(evalIrradianceSH(result.probes[0]!, 0, 1, 0)[0]).toBeCloseTo(1, 1);
+ });
+});
+
describe('a probe grid', () => {
it('reads back an ambient field as itself', () => {
// The whole SH convention in one claim: project, convolve, over pi. A
diff --git a/src/esengine/bindings/RendererBindings.hpp b/src/esengine/bindings/RendererBindings.hpp
index 2fce2971e..e6a1ce451 100644
--- a/src/esengine/bindings/RendererBindings.hpp
+++ b/src/esengine/bindings/RendererBindings.hpp
@@ -177,6 +177,11 @@ u32 probe_volume_create(i32 resX, i32 resY, i32 resZ, uintptr_t shPtr);
/** @brief Releases a probe volume. */
void probe_volume_release(u32 volumeHandle);
+
+/** @brief Gives a probe grid how much of the sky each probe sees: nine floats per probe, in grid order
+ * (a bake with sky occlusion). The grid's irradiance then holds the lights alone, and the
+ * frame's sky is added through this. */
+void probe_volume_setSkyVisibility(u32 volumeHandle, uintptr_t visibilityPtr);
/** @brief Freezes every MeshRenderer in the world; returns how many were frozen. */
u32 meshRenderer_makeAllResident(ecs::Registry& registry);
/** @brief Points every MeshRenderer at one resident mesh; returns how many. */
diff --git a/src/esengine/bindings/WebSDKEntry.cpp b/src/esengine/bindings/WebSDKEntry.cpp
index 31a2400ba..23f00ec19 100644
--- a/src/esengine/bindings/WebSDKEntry.cpp
+++ b/src/esengine/bindings/WebSDKEntry.cpp
@@ -551,6 +551,7 @@ EMSCRIPTEN_BINDINGS(esengine_renderer) {
emscripten::function("renderer_setSkyEnvironment", &esengine::renderer_setSkyEnvironment);
emscripten::function("probe_volume_create", &esengine::probe_volume_create);
emscripten::function("probe_volume_release", &esengine::probe_volume_release);
+ emscripten::function("probe_volume_setSkyVisibility", &esengine::probe_volume_setSkyVisibility);
emscripten::function("meshRenderer_localBounds", &esengine::meshRenderer_localBounds);
emscripten::function("meshRenderer_makeAllResident", &esengine::meshRenderer_makeAllResident);
emscripten::function("meshRenderer_setMeshAll", &esengine::meshRenderer_setMeshAll);
diff --git a/src/esengine/renderer/store/ProbeConstants.hpp b/src/esengine/renderer/store/ProbeConstants.hpp
index dc91fd9d3..efa949039 100644
--- a/src/esengine/renderer/store/ProbeConstants.hpp
+++ b/src/esengine/renderer/store/ProbeConstants.hpp
@@ -49,7 +49,9 @@ struct ProbeConstants {
glm::vec4 irradiance[9]{};
/// x = which column of the frame's reflection atlas this instance reflects.
/// 0 is the environment, so an instance inside no probe needs nothing written
- /// here and a zeroed run is already right.
+ /// here and a zeroed run is already right. y = the first coefficient of the sky's
+ /// visibility (the other eight ride `irradiance[1..8].w`), z = 1 when there is one
+ /// (a grid baked with sky occlusion): the shaders' skyOcclusion().
glm::vec4 reflection{0.0f};
};
diff --git a/src/esengine/renderer/store/ProbeStore.hpp b/src/esengine/renderer/store/ProbeStore.hpp
index 79f7b1254..dc510b370 100644
--- a/src/esengine/renderer/store/ProbeStore.hpp
+++ b/src/esengine/renderer/store/ProbeStore.hpp
@@ -83,6 +83,7 @@ public:
const i32 strideY = g.resolution.x;
const i32 strideZ = g.resolution.x * g.resolution.y;
+ const bool sky = g.hasSkyVisibility();
out = ProbeConstants{};
for (u32 corner = 0; corner < 8; ++corner) {
const f32 w = ((corner & 1u) ? frac[0] : 1.0f - frac[0])
@@ -99,8 +100,14 @@ public:
out.irradiance[c].y += sh.y * w;
out.irradiance[c].z += sh.z * w;
}
+ // The sky's visibility rides the spare channels: [1..8].w and reflection.y for the first.
+ if (sky) {
+ out.reflection.y += g.skyVisibility[base] * w;
+ for (u32 c = 1; c < 9; ++c) out.irradiance[c].w += g.skyVisibility[base + c] * w;
+ }
}
out.irradiance[0].w = 1.0f;
+ if (sky) out.reflection.z = 1.0f;
return true;
}
diff --git a/src/esengine/resource/ProbeVolume.hpp b/src/esengine/resource/ProbeVolume.hpp
index bbfd89661..bbcec4bf7 100644
--- a/src/esengine/resource/ProbeVolume.hpp
+++ b/src/esengine/resource/ProbeVolume.hpp
@@ -31,6 +31,16 @@ public:
/** @brief Nine RGB coefficients per probe, in grid order. */
std::vector<glm::vec3> irradiance;
+ /** @brief How much of the sky each probe sees: nine coefficients per probe, cosine-convolved like the
+ * irradiance (1 everywhere is open sky), in grid order. Empty for a grid baked without sky
+ * occlusion, whose irradiance then holds the sky as it was at the bake; with it, the irradiance
+ * holds the lights and bounces alone and the frame's sky is added through this. */
+ std::vector<f32> skyVisibility;
+
+ bool hasSkyVisibility() const {
+ return !skyVisibility.empty() && skyVisibility.size() == static_cast<usize>(probeCount()) * 9;
+ }
+
u32 probeCount() const {
if (resolution.x <= 0 || resolution.y <= 0 || resolution.z <= 0) return 0;
return static_cast<u32>(resolution.x) * static_cast<u32>(resolution.y)
diff --git a/src/esengine/resource/ShaderParser.cpp b/src/esengine/resource/ShaderParser.cpp
index ff67cf744..1397443dc 100644
--- a/src/esengine/resource/ShaderParser.cpp
+++ b/src/esengine/resource/ShaderParser.cpp
@@ -1949,17 +1949,36 @@ fn bakedIrradiance() -> vec3f {
return c * c * 8.0;
}
#endif
+// See the GLSL twin: how much of the sky a probe sees toward d, 1 where nothing was baked of it.
+fn skyOcclusion(d : vec3f) -> f32 {
+ let base = clamp(i32(g_probeSlot), 0, 99) * 10;
+ let v = pc.u_probeIrradiance[base + 9].y * 0.282095
+ + pc.u_probeIrradiance[base + 1].w * (0.488603 * d.y)
+ + pc.u_probeIrradiance[base + 2].w * (0.488603 * d.z)
+ + pc.u_probeIrradiance[base + 3].w * (0.488603 * d.x)
+ + pc.u_probeIrradiance[base + 4].w * (1.092548 * d.x * d.y)
+ + pc.u_probeIrradiance[base + 5].w * (1.092548 * d.y * d.z)
+ + pc.u_probeIrradiance[base + 6].w * (0.315392 * (3.0 * d.z * d.z - 1.0))
+ + pc.u_probeIrradiance[base + 7].w * (1.092548 * d.x * d.z)
+ + pc.u_probeIrradiance[base + 8].w * (0.546274 * (d.x * d.x - d.y * d.y));
+ return mix(1.0, clamp(v, 0.0, 1.0), step(0.5, pc.u_probeIrradiance[base + 9].z));
+}
+// See the GLSL twin: one exit, for the shape the two share.
fn indirectIrradiance(N : vec3f) -> vec3f {
-#ifdef MESH_LIGHTMAP
- if (g_lightmap.z > 0.5) { return bakedIrradiance() * lc.u_bakedTint.rgb; }
-#endif
let base = clamp(i32(g_probeSlot), 0, 99) * 10;
+ var result : vec3f;
if (pc.u_probeIrradiance[base].w > 0.5) {
var sh : array<vec4f, 9>;
for (var i = 0; i < 9; i = i + 1) { sh[i] = pc.u_probeIrradiance[base + i]; }
- return max(shIrradiance(sh, N), vec3f(0.0)) * lc.u_bakedTint.rgb;
+ let sky = step(0.5, pc.u_probeIrradiance[base + 9].z);
+ result = max(shIrradiance(sh, N), vec3f(0.0)) * lc.u_bakedTint.rgb + envIrradiance(N) * (skyOcclusion(N) * sky);
+ } else {
+ result = envIrradiance(N);
}
- return envIrradiance(N);
+#ifdef MESH_LIGHTMAP
+ if (g_lightmap.z > 0.5) { result = bakedIrradiance() * lc.u_bakedTint.rgb; }
+#endif
+ return result;
}
// Which column of the atlas this instance reflects, from the run its own index
// names. Zero — an instance inside no probe, or a draw that carries no run at all
@@ -2050,7 +2069,7 @@ fn envRadiance(R : vec3f, roughness : f32, column : f32, P : vec3f) -> vec3f {
let grey = dot(baked, vec3f(0.2126, 0.7152, 0.0722));
return mix(vec3f(grey), baked, lc.u_reflParams.z) * lc.u_reflParams.y;
}
- return lc.u_ambient.rgb + mix(a, b, lod - lo) * lc.u_envTint.rgb;
+ return (lc.u_ambient.rgb + mix(a, b, lod - lo) * lc.u_envTint.rgb) * skyOcclusion(R);
}
// Which definition of "where is the light" this surface is shaded with. One with a
// real third coordinate measures against the light's own; a sprite has no depth to
@@ -3348,17 +3367,44 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
"highp float reflectionColumn() { return u_probeIrradiance[probeRun() + 9].x; }\n"
// A bake outranks a volume for the same reason a volume outranks the sky:
// it holds every light and bounce that reached this very point.
+ // How much of the sky the probe this instance stands among sees toward d: nine coefficients of
+ // visibility, cosine-convolved like the irradiance, packed in the run's spare channels (the w of
+ // [1..8], the y of [9]; [9].z says there are any). 1 where none were baked: the sky is open.
+ "highp float skyOcclusion(in highp vec3 d) {\n"
+ " highp int base = probeRun();\n"
+ " highp float v = u_probeIrradiance[base + 9].y * 0.282095\n"
+ " + u_probeIrradiance[base + 1].w * (0.488603 * d.y)\n"
+ " + u_probeIrradiance[base + 2].w * (0.488603 * d.z)\n"
+ " + u_probeIrradiance[base + 3].w * (0.488603 * d.x)\n"
+ " + u_probeIrradiance[base + 4].w * (1.092548 * d.x * d.y)\n"
+ " + u_probeIrradiance[base + 5].w * (1.092548 * d.y * d.z)\n"
+ " + u_probeIrradiance[base + 6].w * (0.315392 * (3.0 * d.z * d.z - 1.0))\n"
+ " + u_probeIrradiance[base + 7].w * (1.092548 * d.x * d.z)\n"
+ " + u_probeIrradiance[base + 8].w * (0.546274 * (d.x * d.x - d.y * d.y));\n"
+ " return mix(1.0, clamp(v, 0.0, 1.0), step(0.5, u_probeIrradiance[base + 9].z));\n"
+ "}\n"
+ // A grid baked with sky occlusion holds the lights and their bounces WITHOUT the sky; the sky is
+ // the frame's own (an environment, a live Sky Atmosphere) seen through what the probe sees of it,
+ // so a room stays dark under a sky that changes.
+ // ONE exit and no early return, here and in skyOcclusion: HLSL inlines every function, and D3D's compiler
+ // structures the control flow of what it inlined recursively. Returns from inside branches of the
+ // functions the light loop and the reflection inline (this one, skyOcclusion, envRadiance) took it past
+ // its stack and killed the GPU process (0xC00000FD in D3DCompiler_47) on the lightmapped lit variant.
"highp vec3 indirectIrradiance(in highp vec3 N) {\n"
- "#ifdef MESH_LIGHTMAP\n"
- " if (v_lightmap.z > 0.5) return bakedIrradiance() * u_bakedTint.rgb;\n"
- "#endif\n"
" highp int base = probeRun();\n"
+ " highp vec3 result;\n"
" if (u_probeIrradiance[base].w > 0.5) {\n"
" highp vec4 sh[9];\n"
" for (int i = 0; i < 9; i++) sh[i] = u_probeIrradiance[base + i];\n"
- " return max(shIrradiance(sh, N), vec3(0.0)) * u_bakedTint.rgb;\n"
+ " highp float sky = step(0.5, u_probeIrradiance[base + 9].z);\n"
+ " result = max(shIrradiance(sh, N), vec3(0.0)) * u_bakedTint.rgb + envIrradiance(N) * (skyOcclusion(N) * sky);\n"
+ " } else {\n"
+ " result = envIrradiance(N);\n"
" }\n"
- " return envIrradiance(N);\n"
+ "#ifdef MESH_LIGHTMAP\n"
+ " if (v_lightmap.z > 0.5) result = bakedIrradiance() * u_bakedTint.rgb;\n"
+ "#endif\n"
+ " return result;\n"
"}\n"
// Direction -> the octahedral unit square, +Y at the centre. The importer's
// octEncode, in the shading language; the two must agree texel for texel.
@@ -3454,7 +3500,8 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p
" highp float grey = dot(baked, vec3(0.2126, 0.7152, 0.0722));\n"
" return mix(vec3(grey), baked, u_reflParams.z) * u_reflParams.y;\n"
" }\n"
- " return u_ambient.rgb + mix(a, b, lod - lo) * u_envTint.rgb;\n"
+ // The sky's own column, as much of it as the probe sees (1 without a sky-occlusion bake).
+ " return (u_ambient.rgb + mix(a, b, lod - lo) * u_envTint.rgb) * skyOcclusion(R);\n"
"}\n"
// Where a positional light is, from here, and how far. Real geometry measures
// against the light's own world height; a sprite has no depth of its own and
diff --git a/tools/renderScenes.mjs b/tools/renderScenes.mjs
index e66bf7e19..5181e89c5 100644
--- a/tools/renderScenes.mjs
+++ b/tools/renderScenes.mjs
@@ -266,6 +266,10 @@ export const SCENES = [
// The nine coefficients carry a DIRECTION, not just an amount. The fixture's
// two triangles face +Z and +X, and the grid is grey with red along +X — so a
// volume flattened to its constant term paints these two the same grey.
+ // A grid baked with sky occlusion: no light of its own, a fifth of the sky seen everywhere, under a plain white Ambient light. The sky
+ // is the frame's, seen through the grid: a fifth of white, 54. Read as a grid of the old kind (the sky left out) it is black, 3.
+ { id: "lit-builtin-lightmap-uv", tier: "pr", webgpu: true, env: { "ESTELLA_VERIFY_COLORSPACE": "linear", "ESTELLA_VERIFY_SCENE": "/scenes/lit-builtin-lightmap-uv.esscene", "ESTELLA_VERIFY_W": "256", "ESTELLA_VERIFY_H": "256", "ESTELLA_VERIFY_STEPS": "4", "ESTELLA_VERIFY_EXPECT": "[{\"x\":0.5,\"y\":0.75,\"rgb\":[183,179,164],\"tol\":20}]" } },
+ { id: "probe-sky", tier: "pr", webgpu: true, env: { "ESTELLA_VERIFY_SCENE": "/scenes/probe-sky.esscene", "ESTELLA_VERIFY_MANIFEST": "/scenes/probe-sky.textures.json", "ESTELLA_VERIFY_W": "256", "ESTELLA_VERIFY_H": "256", "ESTELLA_VERIFY_STEPS": "2", "ESTELLA_VERIFY_EXPECT": "[{\"x\":0.3,\"y\":0.556,\"rgb\":[54,54,54],\"tol\":10}]" } },
{ id: "probe-facing", tier: "pr", webgpu: true, env: { "ESTELLA_VERIFY_SCENE": "/scenes/probe-facing.esscene", "ESTELLA_VERIFY_MANIFEST": "/scenes/probe-facing.textures.json", "ESTELLA_VERIFY_W": "256", "ESTELLA_VERIFY_H": "256", "ESTELLA_VERIFY_STEPS": "2", "ESTELLA_VERIFY_EXPECT": "[{\"x\":0.3,\"y\":0.556,\"rgb\":[128,128,128],\"tol\":30},{\"x\":0.7,\"y\":0.556,\"rgb\":[255,128,128],\"tol\":30}]" } },
// Red at one end of the box, blue at the other. The object in the MIDDLE is the
// claim: nearest-probe would paint it one of the two, and only a read that