Skip to content

Code-first video & image, rendered natively on WebGPU.

feat(3d): volumetric fog, rect lights, Substrate materials and HDRI/EXR IBL

okanasl8b47a79558a8
diff --git a/nodes/node-compositor/SKILL.md b/nodes/node-compositor/SKILL.md
index e6302c9c..5ece9dc1 100644
--- a/nodes/node-compositor/SKILL.md
+++ b/nodes/node-compositor/SKILL.md
@@ -171,7 +171,7 @@ Layout semantics (HTML-like):
- **`tracks`** (array): Up to 24 animation tracks.
Each track represents animatable property modifications:
- **`id`** (string, required): Unique identifier for the track.
- - **`prop`** (string, enum, required): `x`, `y`, `scale`, `rotation`, `opacity`, `fill`, `color`, `letterSpacing`, `width`, `height`, `volume`, `hidden`, `muted`, `fontSize`, `text`, `trimStart`, `trimEnd`, `trimOffset`, `strokeWidth`, `strokeDashOffset`, `cornerRadius`, `starInnerRadiusRatio`, `fillColor`, `strokeColor`, `rangeStart`, `rangeEnd`, `offset`, `animatorRangeStart`, `animatorRangeEnd`, `animatorOffset`, `rotateX`, `rotateY`, `rotateZ`, `perspective`, `translateZ`, `perspectiveOriginX`, `perspectiveOriginY`, `progress`, `drawProgress`, `chartProgress`.
+ - **`prop`** (string, enum, required): `x`, `y`, `scale`, `rotation`, `opacity`, `fill`, `color`, `letterSpacing`, `width`, `height`, `volume`, `hidden`, `muted`, `fontSize`, `text`, `trimStart`, `trimEnd`, `trimOffset`, `strokeWidth`, `strokeDashOffset`, `cornerRadius`, `starInnerRadiusRatio`, `fillColor`, `strokeColor`, `rangeStart`, `rangeEnd`, `offset`, `animatorRangeStart`, `animatorRangeEnd`, `animatorOffset`, `rotateX`, `rotateY`, `rotateZ`, `perspective`, `translateZ`, `perspectiveOriginX`, `perspectiveOriginY`, `progress`, `drawProgress`, `chartProgress`, `volumetricFogDensity`, `volumetricFogAnisotropy`, `volumetricFogNoiseStrength`, `emissive`, `emissiveIntensity`, `clearcoat`, `clearcoatRoughness`, `anisotropy`, `anisotropyRotation`, `subsurface`, `subsurfaceColor`, `sheen`, `sheenColor`, `sheenRoughness`, `iridescence`, `iridescenceThickness`, `iridescenceIor`, `triplanarScale`, `triplanarSharpness`.
- **`source`** (object, optional, default `{ type: "keyframe" }`): Driver evaluating this track.
- `{ type: "keyframe" }`: Keyframe sequence (requires `keyframes`).
- `{ type: "signal", inputHandleId: string, multiplier?: number, offset?: number, smoothingWindowFrames?: number, signalMode?: "continuous" | "accumulate", threshold?: number, debounceFrames?: number, colorMode?: "interpolate" | "hueRotate" | "threshold", colorA?: string, colorB?: string, colorThreshold?: number }`: Samples a connected `Signal` variable input handle (e.g. from `AudioSignalExtractor` or `SignalMath`). In `"continuous"` mode, maps amplitude or continuous values directly. In `"accumulate"` mode, acts as a discrete event integrator that detects rising-edge acoustic transients/peaks exceeding `threshold` (with `debounceFrames` guard) and increments an integer step count on each peak — ideal for audio-reactive typography where each keystroke/drum hit reveals letters sequentially. Also supports dynamic color properties (`fill`, `color`) via linear interpolation, hue rotation, or binary threshold switching. Aliases `handleId` (for `inputHandleId`), `amplitude` (for `multiplier`), and `smoothing` (for `smoothingWindowFrames`) are also accepted. For procedural drivers like `signal`, `keyframes: []` can be empty.
@@ -211,7 +211,11 @@ Each track represents animatable property modifications:
- **Picture-in-Picture:** a `flex` row with two `media` nodes (`fit: "cover"`, each `grow: 1`).
- **Audio-Reactive 3D Models:** a `kind: "model3d"` node loading an OBJ/FBX/GLTF/STL/PLY mesh with `audioDeform: { audioTrackId, mode: "normal_extrusion" | "radial_pulse" | "harmonic_wave" | "twist" | "ripple", amplitudeMultiplier, damping }`, deformed directly in VRAM via WebGPU compute shaders driven by audio frequency spectra and transient energies. `twist` turns each slice about Y by its height (harder on the bass); `ripple` runs concentric waves across XY, struck by the drums. `audioTrackId` may be an audio layer's id or an audio file path.
- **Several 3D scenes:** a `box`/`flex`/`block` with a numeric `width` and `height` that holds a `camera` and the layers it films renders as its own scene (one holding only a camera stays a shot in a camera cut list): a nested composition with only its own camera, lights and environment, on its own clock (children start from the box's frame 0), placed in its parent like a media layer. Transitions, opacity and mattes on the box then blend finished pictures. Without that, every 3D layer shares one world and the active camera (the latest to start).
-- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; directional and spot lights can each cast shadows, up to the device texture-array layer limit.
+- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; `directional`, `spot`, `rect` (1 layer each), and `point` lights (6 cube-face layers) can each cast shadows, up to the device texture-array layer limit.
+- **Rect Area Lights (`lightType: "rect"`):** rectangular softbox/panel area lights (`width`, `height`, `twoSided`, `rotation`, `x/y/z`, `targetX/Y/Z`) use solid-angle subtended diffuse irradiance and Karis representative-point GGX specular shading, and can cast perspective frustum shadows (`castShadow: true`).
+- **Volumetric Fog & Light Shafts (`volumetricFog` in `environment`):** `kind: "light"` with `lightType: "environment"` and `volumetricFog: true | { density, anisotropy, heightFalloff, baseHeight, color, lightIntensity, ambientIntensity, maxDistance, steps, noise: { scale, strength, speed, octaves } }` runs half-resolution raymarching with Interleaved Gradient Noise jitter, Henyey-Greenstein phase scattering, 3D FBM wind-animated density noise, and multi-light shadow-map sampling (carving god rays behind shadow casters), followed by depth-aware bilateral upsampling. `volumetricFogDensity`, `volumetricFogAnisotropy`, and `volumetricFogNoiseStrength` can be animated or driven by `Signal`s.
+- **Substrate multi-lobe materials:** physical-shading surfaces (any scene with an `environment` light) on `model3d` nodes, 3D cards and 3D text accept extra lobes on top of `roughness`/`metallic`: `clearcoat` 0..1 + `clearcoatRoughness` (default 0.08; IOR 1.5 GGX varnish that attenuates the base), `anisotropy` -1..1 + `anisotropyRotation` degrees (tangent-space anisotropic GGX: brushed metal, satin), `subsurface` 0..1 + `subsurfaceColor` (default `#ff6b4a`; wrap diffuse, terminator bleed and back-lit transmission), `sheen` 0..1 + `sheenColor` + `sheenRoughness` (default 0.5; Charlie cloth/velvet lobe), `iridescence` 0..1 or `{ intensity, thickness, ior }` + `iridescenceThickness` nm (100..1200, default 400) + `iridescenceIor` (default 1.3; RGB thin-film interference), and `triplanar: true | { scale, sharpness }` + `triplanarScale` (repeats per 256 world px) + `triplanarSharpness` (1..16, default 4) for world-space UV-less texture projection. All numeric lobe props are animatable.
+- **HDRI environments:** an environment light's `hdri` takes an equirectangular panorama: a Radiance `.hdr` (RGBE) or OpenEXR `.exr` file path (resolved from the working directory) or the file's bytes (told apart by magic bytes); it then replaces the procedural preset as the image-based light for reflections and ambient, honoring `intensity`, `rotation` and `exposure`. On first use per device a compute pass prefilters it into a GGX radiance cubemap (faces up to 256², one mip per roughness step, split-sum importance sampling) for reflections, and the CPU projects it onto L2 spherical harmonics for diffuse irradiance; both are cached (up to 8 panoramas per device, least recently used destroyed). `.exr` reads single-part scanline or tiled files (level 0 of mip/ripmaps) with NONE, RLE, ZIPS, ZIP, PIZ or PXR24 compression and HALF/FLOAT/UINT `R`,`G`,`B` (or `Y`) channels; lossy B44/B44A/DWAA/DWAB/HTJ2K, deep and multi-part files fail with an error naming the cause.
- **Art-directed 3D lighting:** `material: "toon"` on meshes, flat 3D cards and native 3D text uses the shared direct lights, colors, point/spot attenuation, cast shadows, environment fog and tone mapping. Optional `toon: { shadowColor, lightColor, threshold, softness, diffuse }` controls the shadow/lit albedo multipliers, terminator cosine/width and Lambert modelling (defaults: model shade color or `#80808c`, white, 0.075, 0.25, 0). Lit/physical materials retain their own shading.
- **Geometry outlines:** an environment light's optional `outlines: true | { color, width, referenceHeight, creaseAngle, depthThreshold, opacity }` runs a depth/normal ink pass before camera depth blur and bloom (defaults: `#16131c`, 1.5 px at height 1080, 45°, 0.01, 1). It reuses the linear-depth attachment's spare channels for octahedral surface normals and outline participation. Inverse-depth curvature avoids false contours on perspective planes; normal discontinuities find creases without inking texture/shadow/triangulation seams. `outline: false` on a layer/model opts out while retaining depth occlusion; `creaseAngle: 180` disables creases. Ink fades into the environment fog. 3D `anchorX/Y` pivots remain fixed under scale/rotation and leave identity placement unchanged.
With MSAA enabled, retain original scene color and geometry samples and shade ink per sample before resolving color. A single-sample ink pass after the scene resolve loses subpixel coverage between outlined surfaces. Lens blur uses a separate nearest-sample geometry resolve. The scene's geometry is still drawn once.
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 37d3a0a1..9285590e 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -568,6 +568,21 @@ function surfaceProps(
castShadow?: boolean;
toon?: CompositionNodeBase["toon"];
outline?: boolean;
+ clearcoat?: number;
+ clearcoatRoughness?: number;
+ anisotropy?: number;
+ anisotropyRotation?: number;
+ subsurface?: number;
+ subsurfaceColor?: string;
+ sheen?: number;
+ sheenColor?: string;
+ sheenRoughness?: number;
+ iridescence?: CompositionNodeBase["iridescence"];
+ iridescenceThickness?: number;
+ iridescenceIor?: number;
+ triplanar?: CompositionNodeBase["triplanar"];
+ triplanarScale?: number;
+ triplanarSharpness?: number;
} {
// glTF emissiveFactor is linear; draws take display (sRGB) colors.
const toSrgb = (v: number) =>
@@ -594,6 +609,44 @@ function surfaceProps(
if (lop.castShadow !== undefined) out.castShadow = lop.castShadow;
out.toon = target?.toon ?? lop.toon;
out.outline = target?.outline ?? lop.outline;
+ const clearcoat = target?.clearcoat ?? lop.clearcoat;
+ if (clearcoat !== undefined) out.clearcoat = clearcoat;
+ const clearcoatRoughness =
+ target?.clearcoatRoughness ?? lop.clearcoatRoughness;
+ if (clearcoatRoughness !== undefined)
+ out.clearcoatRoughness = clearcoatRoughness;
+ const anisotropy = target?.anisotropy ?? lop.anisotropy;
+ if (anisotropy !== undefined) out.anisotropy = anisotropy;
+ const anisotropyRotation =
+ target?.anisotropyRotation ?? lop.anisotropyRotation;
+ if (anisotropyRotation !== undefined)
+ out.anisotropyRotation = anisotropyRotation;
+ const subsurface = target?.subsurface ?? lop.subsurface;
+ if (subsurface !== undefined) out.subsurface = subsurface;
+ const subsurfaceColor = target?.subsurfaceColor ?? lop.subsurfaceColor;
+ if (subsurfaceColor !== undefined) out.subsurfaceColor = subsurfaceColor;
+ const sheen = target?.sheen ?? lop.sheen;
+ if (sheen !== undefined) out.sheen = sheen;
+ const sheenColor = target?.sheenColor ?? lop.sheenColor;
+ if (sheenColor !== undefined) out.sheenColor = sheenColor;
+ const sheenRoughness = target?.sheenRoughness ?? lop.sheenRoughness;
+ if (sheenRoughness !== undefined) out.sheenRoughness = sheenRoughness;
+ const iridescence = target?.iridescence ?? lop.iridescence;
+ if (iridescence !== undefined) out.iridescence = iridescence;
+ const iridescenceThickness =
+ target?.iridescenceThickness ?? lop.iridescenceThickness;
+ if (iridescenceThickness !== undefined)
+ out.iridescenceThickness = iridescenceThickness;
+ const iridescenceIor = target?.iridescenceIor ?? lop.iridescenceIor;
+ if (iridescenceIor !== undefined) out.iridescenceIor = iridescenceIor;
+ const triplanar = target?.triplanar ?? lop.triplanar;
+ if (triplanar !== undefined) out.triplanar = triplanar;
+ const triplanarScale = target?.triplanarScale ?? lop.triplanarScale;
+ if (triplanarScale !== undefined) out.triplanarScale = triplanarScale;
+ const triplanarSharpness =
+ target?.triplanarSharpness ?? lop.triplanarSharpness;
+ if (triplanarSharpness !== undefined)
+ out.triplanarSharpness = triplanarSharpness;
return out;
}
@@ -1368,12 +1421,41 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
| "directional"
| "point"
| "spot"
+ | "rect"
| "environment";
if (lightType === "environment") {
// The scene's world: the last visible one wins.
const intensity = target?.intensity ?? lop.intensity;
+ const vfDensity =
+ target?.volumetricFogDensity ?? lop.volumetricFogDensity;
+ const vfAnisotropy =
+ target?.volumetricFogAnisotropy ?? lop.volumetricFogAnisotropy;
+ const vfNoiseStrength =
+ target?.volumetricFogNoiseStrength ??
+ lop.volumetricFogNoiseStrength;
+ const rawVf = lop.volumetricFog;
+ const baseVf =
+ rawVf === true
+ ? {}
+ : rawVf && typeof rawVf === "object"
+ ? { ...rawVf }
+ : undefined;
+ const volumetricFog = baseVf
+ ? {
+ ...baseVf,
+ ...(vfDensity !== undefined && { density: vfDensity }),
+ ...(vfAnisotropy !== undefined && { anisotropy: vfAnisotropy }),
+ ...(vfNoiseStrength !== undefined && {
+ noise: {
+ ...(typeof baseVf.noise === "object" ? baseVf.noise : {}),
+ strength: vfNoiseStrength,
+ },
+ }),
+ }
+ : undefined;
sceneEnvironment = {
preset: lop.preset,
+ hdri: lop.hdri,
top: lop.top,
horizon: lop.horizon,
bottom: lop.bottom,
@@ -1384,6 +1466,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
fog: lop.fog,
floor: lop.floor,
outlines: lop.outlines,
+ ...(volumetricFog && { volumetricFog }),
...(intensity !== undefined && { intensity }),
};
sceneBloom =
@@ -1407,6 +1490,16 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const angle = target?.angle ?? lop.angle ?? 45;
const penumbra = target?.penumbra ?? lop.penumbra ?? 0.2;
const decay = target?.decay ?? lop.decay ?? 1;
+ const width =
+ typeof (target?.width ?? lop.width) === "number"
+ ? ((target?.width ?? lop.width) as number)
+ : undefined;
+ const height =
+ typeof (target?.height ?? lop.height) === "number"
+ ? ((target?.height ?? lop.height) as number)
+ : undefined;
+ const twoSided = Boolean(target?.twoSided ?? lop.twoSided);
+ const rotation = target?.rotation ?? lop.rotation ?? 0;
sceneLights.push({
type: lightType,
@@ -1414,10 +1507,15 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
intensity,
position: [x, y, z],
target: [targetX, targetY, targetZ],
+ ...(Array.isArray(lop.direction) && { direction: lop.direction }),
radius,
angle,
penumbra,
decay,
+ ...(width !== undefined && { width }),
+ ...(height !== undefined && { height }),
+ twoSided,
+ rotation,
castShadow: Boolean(lop.castShadow),
shadowSoftness: target?.shadowSoftness ?? lop.shadowSoftness,
shadowDarkness: target?.shadowDarkness ?? lop.shadowDarkness,
@@ -1527,7 +1625,9 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const outlineSetting = sceneEnvironment?.outlines;
const outlines = outlineSetting === true ? {} : outlineSetting || null;
- const geometryEnabled = dofEnabled || Boolean(outlines);
+ const volumetricFogEnabled = Boolean(sceneEnvironment?.volumetricFog);
+ const geometryEnabled =
+ dofEnabled || Boolean(outlines) || volumetricFogEnabled;
let linearDepthTex: GPUTexture | undefined;
let linearDepthView: GPUTextureView | undefined;
if (geometryEnabled) {
@@ -1545,12 +1645,16 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const shadowLights = sceneLights
.filter((light) => (light.lightType ?? light.type) !== "ambient")
.map((light, index) => ({ light, index }))
- .filter(
- ({ light }) =>
+ .filter(({ light }) => {
+ const k = light.lightType ?? light.type;
+ return (
light.castShadow &&
- ((light.lightType ?? light.type) === "directional" ||
- (light.lightType ?? light.type) === "spot"),
- );
+ (k === "directional" ||
+ k === "spot" ||
+ k === "rect" ||
+ k === "point")
+ );
+ });
if (shadowLights.length > 0) {
const casters: ShadowCaster3D[] = [];
const receivers: ReturnType<typeof r3d.worldBounds>[] = [];
@@ -1580,7 +1684,19 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
}
}
if (casters.length > 0) {
+ const totalShadowLayers = shadowLights.reduce(
+ (sum, { light }) =>
+ sum + ((light.lightType ?? light.type) === "point" ? 6 : 1),
+ 0,
+ );
+ const allPointShadows = shadowLights.every(
+ ({ light }) => (light.lightType ?? light.type) === "point",
+ );
+ const resolvedShadowMapSize =
+ shadowMapSize ?? (allPointShadows ? 1024 : 2048);
+ let currentLayer = 0;
for (const { light, index } of shadowLights) {
+ const needed = (light.lightType ?? light.type) === "point" ? 6 : 1;
const result = r3d.renderShadowMap(
encoder,
light,
@@ -1588,12 +1704,15 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
casters,
receivers,
{
- mapSize: shadowMapSize,
- layer: shadow.length,
- layerCount: shadowLights.length,
+ mapSize: resolvedShadowMapSize,
+ layer: currentLayer,
+ layerCount: totalShadowLayers,
},
);
- if (result) shadow.push(result);
+ if (result) {
+ shadow.push(result);
+ currentLayer += result.layerCount ?? needed;
+ }
}
}
}
@@ -1821,7 +1940,9 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
let final3dTex = scene3dTex;
if (outlines && linearDepthTex) {
- if (dofEnabled) r3d.resolveGeometry(encoder, linearDepthTex);
+ if (dofEnabled || volumetricFogEnabled) {
+ r3d.resolveGeometry(encoder, linearDepthTex);
+ }
const outlinedTex = ctx.renderer.getTemporaryTexture(
nativeWidth,
nativeHeight,
@@ -1838,6 +1959,32 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
);
final3dTex = outlinedTex;
}
+ if (volumetricFogEnabled && sceneEnvironment && linearDepthTex) {
+ const vfogTex = ctx.renderer.getTemporaryTexture(
+ nativeWidth,
+ nativeHeight,
+ [
+ curTex,
+ targetTexture,
+ scene3dTex,
+ final3dTex,
+ linearDepthTex,
+ ...curExcludes,
+ ],
+ );
+ r3d.applyVolumetricFog(
+ encoder,
+ final3dTex,
+ linearDepthTex,
+ vfogTex.createView(),
+ cameraBindGroup,
+ lightsBindGroup,
+ sceneEnvironment,
+ shiftedFrame / Math.max(1, fps ?? 24),
+ shiftedFrame,
+ );
+ final3dTex = vfogTex;
+ }
if (dofEnabled && linearDepthTex) {
const dofResultTex = ctx.renderer.getTemporaryTexture(
nativeWidth,
diff --git a/nodes/node-compositor/src/shared/compiler.ts b/nodes/node-compositor/src/shared/compiler.ts
index 98031ec1..c2f0ca0b 100644
--- a/nodes/node-compositor/src/shared/compiler.ts
+++ b/nodes/node-compositor/src/shared/compiler.ts
@@ -60,6 +60,23 @@ export interface LayerStub {
specularIntensity?: number;
ambientIntensity?: number;
metallic?: number;
+ emissive?: string;
+ emissiveIntensity?: number;
+ clearcoat?: number;
+ clearcoatRoughness?: number;
+ anisotropy?: number;
+ anisotropyRotation?: number;
+ subsurface?: number;
+ subsurfaceColor?: string;
+ sheen?: number;
+ sheenColor?: string;
+ sheenRoughness?: number;
+ iridescence?: number | Record<string, unknown>;
+ iridescenceThickness?: number;
+ iridescenceIor?: number;
+ triplanar?: boolean | Record<string, unknown>;
+ triplanarScale?: number;
+ triplanarSharpness?: number;
// 3D Camera Props
cameraX?: number;
cameraY?: number;
@@ -90,11 +107,16 @@ export interface LayerStub {
scaleY?: number;
// 3D Light Props
lightType?: string;
+ color?: string;
intensity?: number;
radius?: number;
angle?: number;
penumbra?: number;
decay?: number;
+ exposure?: number;
+ volumetricFogDensity?: number;
+ volumetricFogAnisotropy?: number;
+ volumetricFogNoiseStrength?: number;
[key: string]: unknown;
}
@@ -728,12 +750,36 @@ function hashNodes(
specularIntensity: lop.specularIntensity,
ambientIntensity: lop.ambientIntensity,
metallic: lop.metallic,
+ emissive: lop.emissive,
+ emissiveIntensity: lop.emissiveIntensity,
+ clearcoat: lop.clearcoat,
+ clearcoatRoughness: lop.clearcoatRoughness,
+ anisotropy: lop.anisotropy,
+ anisotropyRotation: lop.anisotropyRotation,
+ subsurface: lop.subsurface,
+ subsurfaceColor: lop.subsurfaceColor,
+ sheen: lop.sheen,
+ sheenColor: lop.sheenColor,
+ sheenRoughness: lop.sheenRoughness,
+ iridescence: lop.iridescence,
+ iridescenceThickness: lop.iridescenceThickness,
+ iridescenceIor: lop.iridescenceIor,
+ triplanar: lop.triplanar,
+ triplanarScale: lop.triplanarScale,
+ triplanarSharpness: lop.triplanarSharpness,
// 3D Light Props
lightType: lop.lightType,
+ color: lop.color,
intensity: lop.intensity,
radius: lop.radius,
angle: lop.angle,
penumbra: lop.penumbra,
+ exposure: lop.exposure,
+ hdri: typeof lop.hdri === "string" ? lop.hdri : Boolean(lop.hdri),
+ volumetricFog: lop.volumetricFog,
+ volumetricFogDensity: lop.volumetricFogDensity,
+ volumetricFogAnisotropy: lop.volumetricFogAnisotropy,
+ volumetricFogNoiseStrength: lop.volumetricFogNoiseStrength,
// 3D Model Props
animationProgress: lop.animationProgress,
animationTime: lop.animationTime,
@@ -869,7 +915,7 @@ export function compileTimeline(
: false),
z: lop.z ?? (lop.kind === "camera" ? (lop.z ?? -1500) : 0),
scaleZ: lop.scaleZ ?? 1,
- twoSided: lop.twoSided ?? true,
+ twoSided: lop.twoSided ?? lop.kind !== "light",
// 3D Material Props
material: lop.material,
shininess: lop.shininess,
@@ -877,6 +923,23 @@ export function compileTimeline(
specularIntensity: lop.specularIntensity,
ambientIntensity: lop.ambientIntensity,
metallic: lop.metallic,
+ emissive: lop.emissive,
+ emissiveIntensity: lop.emissiveIntensity,
+ clearcoat: lop.clearcoat,
+ clearcoatRoughness: lop.clearcoatRoughness,
+ anisotropy: lop.anisotropy,
+ anisotropyRotation: lop.anisotropyRotation,
+ subsurface: lop.subsurface,
+ subsurfaceColor: lop.subsurfaceColor,
+ sheen: lop.sheen,
+ sheenColor: lop.sheenColor,
+ sheenRoughness: lop.sheenRoughness,
+ iridescence: lop.iridescence,
+ iridescenceThickness: lop.iridescenceThickness,
+ iridescenceIor: lop.iridescenceIor,
+ triplanar: lop.triplanar,
+ triplanarScale: lop.triplanarScale,
+ triplanarSharpness: lop.triplanarSharpness,
// 3D Model Props
animationProgress: lop.animationProgress ?? 0,
animationTime: lop.animationTime ?? 0,
@@ -911,11 +974,16 @@ export function compileTimeline(
orbitElevation: lop.orbitElevation,
// 3D Light Props
lightType: lop.lightType,
+ color: lop.color,
intensity: lop.intensity ?? (lop.kind === "light" ? 1 : undefined),
radius: lop.radius,
angle: lop.angle,
penumbra: lop.penumbra,
decay: lop.decay,
+ exposure: lop.exposure,
+ volumetricFogDensity: lop.volumetricFogDensity,
+ volumetricFogAnisotropy: lop.volumetricFogAnisotropy,
+ volumetricFogNoiseStrength: lop.volumetricFogNoiseStrength,
};
for (const track of lop.animation?.tracks ?? []) {
diff --git a/nodes/node-compositor/src/shared/processor.ts b/nodes/node-compositor/src/shared/processor.ts
index e546f302..49bcced5 100644
--- a/nodes/node-compositor/src/shared/processor.ts
+++ b/nodes/node-compositor/src/shared/processor.ts
@@ -97,7 +97,10 @@ export function processCompositor(
isVideoMode,
fps,
...(durationMs !== undefined && { durationMs }),
- resolveMedia: (inputHandleId) => mediaByHandle.get(inputHandleId),
+ resolveMedia: (inputHandleId) =>
+ inputHandleId === undefined
+ ? undefined
+ : mediaByHandle.get(inputHandleId),
signals: signalsObj,
});
}
diff --git a/packages/compositions/src/program/animation.ts b/packages/compositions/src/program/animation.ts
index 1165b203..909f076f 100644
--- a/packages/compositions/src/program/animation.ts
+++ b/packages/compositions/src/program/animation.ts
@@ -8,6 +8,34 @@ import type {
TrackSource,
} from "./schema.js";
+/** Common 3D Euler rotation aliases accepted alongside `rotateX` / `rotateY` / `rotateZ`. */
+export type AnimatablePropAlias = "rotationX" | "rotationY" | "rotationZ";
+
+export type CanonicalAnimatableProp<
+ P extends AnimatableProp | AnimatablePropAlias,
+> = P extends "rotationX"
+ ? "rotateX"
+ : P extends "rotationY"
+ ? "rotateY"
+ : P extends "rotationZ"
+ ? "rotateZ"
+ : P;
+
+const PROP_ALIASES: Record<AnimatablePropAlias, AnimatableProp> = {
+ rotationX: "rotateX",
+ rotationY: "rotateY",
+ rotationZ: "rotateZ",
+};
+
+export function normalizeAnimatableProp(
+ prop: AnimatableProp | AnimatablePropAlias,
+): AnimatableProp {
+ return (
+ (PROP_ALIASES as Record<string, AnimatableProp | undefined>)[prop] ??
+ (prop as AnimatableProp)
+ );
+}
+
/**
* One keyframe as a tuple: `[frame, value]` or `[frame, value, ease]`. The
* ease shapes the motion arriving at this keyframe from the one before.
@@ -321,38 +349,63 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
return this;
}
- public static fromTo<P extends AnimatableProp = AnimatableProp>(
+ public static fromTo<
+ P extends AnimatableProp | AnimatablePropAlias = AnimatableProp,
+ >(
prop: P,
fromValue: number | string | boolean,
toValue: number | string | boolean,
options: FromToOptions = {},
- ): LayerAnimation<P> {
- return new LayerAnimation<P>().fromTo(prop, fromValue, toValue, options);
+ ): LayerAnimation<CanonicalAnimatableProp<P>> {
+ return new LayerAnimation<CanonicalAnimatableProp<P>>().fromTo(
+ prop,
+ fromValue,
+ toValue,
+ options,
+ );
}
/** A track from a list of `[frame, value, ease?]` keyframes; see {@link LayerAnimation.keys}. */
- public static keys<P extends AnimatableProp = AnimatableProp>(
+ public static keys<
+ P extends AnimatableProp | AnimatablePropAlias = AnimatableProp,
+ >(
prop: P,
keyframes: readonly KeyframeTuple[],
options?: TrackKeyOptions,
- ): LayerAnimation<P> {
- return new LayerAnimation<P>().keys(prop, keyframes, options);
+ ): LayerAnimation<CanonicalAnimatableProp<P>> {
+ return new LayerAnimation<CanonicalAnimatableProp<P>>().keys(
+ prop,
+ keyframes,
+ options,
+ );
}
- public static signal<P extends AnimatableProp = AnimatableProp>(
+ public static signal<
+ P extends AnimatableProp | AnimatablePropAlias = AnimatableProp,
+ >(
prop: P,
signalOrHandleId: unknown,
options: SignalTrackOptions = {},
- ): LayerAnimation<P> {
- return new LayerAnimation<P>().signal(prop, signalOrHandleId, options);
+ ): LayerAnimation<CanonicalAnimatableProp<P>> {
+ return new LayerAnimation<CanonicalAnimatableProp<P>>().signal(
+ prop,
+ signalOrHandleId,
+ options,
+ );
}
- public static colorSignal<P extends AnimatableProp = AnimatableProp>(
+ public static colorSignal<
+ P extends AnimatableProp | AnimatablePropAlias = AnimatableProp,
+ >(
prop: P,
signalOrHandleId: unknown,
options: ColorSignalOptions,
- ): LayerAnimation<P> {
- return new LayerAnimation<P>().colorSignal(prop, signalOrHandleId, options);
+ ): LayerAnimation<CanonicalAnimatableProp<P>> {
+ return new LayerAnimation<CanonicalAnimatableProp<P>>().colorSignal(
+ prop,
+ signalOrHandleId,
+ options,
+ );
}
public static kineticSweep(
@@ -396,14 +449,14 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
}
public static wiggle(
- prop: AnimatableProp,
+ prop: AnimatableProp | AnimatablePropAlias,
options: WiggleOptions = {},
): LayerAnimation {
return new LayerAnimation().wiggle(prop, options);
}
public static spring(
- prop: AnimatableProp,
+ prop: AnimatableProp | AnimatablePropAlias,
options: SpringOptions = {},
): LayerAnimation {
return new LayerAnimation().spring(prop, options);
@@ -454,6 +507,30 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
return new LayerAnimation().fromTo("rotateZ", fromAngle, toAngle, options);
}
+ public static rotationX(
+ fromAngle: number,
+ toAngle: number,
+ options: FromToOptions = {},
+ ): LayerAnimation {
+ return new LayerAnimation().fromTo("rotateX", fromAngle, toAngle, options);
+ }
+
+ public static rotationY(
+ fromAngle: number,
+ toAngle: number,
+ options: FromToOptions = {},
+ ): LayerAnimation {
+ return new LayerAnimation().fromTo("rotateY", fromAngle, toAngle, options);
+ }
+
+ public static rotationZ(
+ fromAngle: number,
+ toAngle: number,
+ options: FromToOptions = {},
+ ): LayerAnimation {
+ return new LayerAnimation().fromTo("rotateZ", fromAngle, toAngle, options);
+ }
+
public static rotation(
fromAngle: number,
toAngle: number,
@@ -496,10 +573,11 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* - `-1`: loops indefinitely.
* - `N`: repeats N times after the initial iteration (total N + 1 iterations).
*/
- public repeat(count = -1, prop?: AnimatableProp): this {
+ public repeat(count = -1, prop?: AnimatableProp | AnimatablePropAlias): this {
+ const canonical = prop ? normalizeAnimatableProp(prop) : undefined;
this.defaultRepeat = count;
for (const track of this.tracks) {
- if (!prop || track.prop === prop) {
+ if (!canonical || track.prop === canonical) {
track.repeat = count;
}
}
@@ -509,10 +587,14 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
/**
* Enables or disables yoyo (alternate playback direction on each repeat iteration).
*/
- public yoyo(enable = true, prop?: AnimatableProp): this {
+ public yoyo(
+ enable = true,
+ prop?: AnimatableProp | AnimatablePropAlias,
+ ): this {
+ const canonical = prop ? normalizeAnimatableProp(prop) : undefined;
this.defaultYoyo = enable;
for (const track of this.tracks) {
- if (!prop || track.prop === prop) {
+ if (!canonical || track.prop === canonical) {
track.yoyo = enable;
}
}
@@ -523,7 +605,10 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* Shorthand to loop keyframed animation continuously (infinite repeat).
* If `options.yoyo` is true, oscillates back and forth.
*/
- public loop(options: { yoyo?: boolean } = {}, prop?: AnimatableProp): this {
+ public loop(
+ options: { yoyo?: boolean } = {},
+ prop?: AnimatableProp | AnimatablePropAlias,
+ ): this {
this.repeat(-1, prop);
if (options.yoyo) {
this.yoyo(true, prop);
@@ -535,11 +620,12 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* Creates a keyframed track animating from fromValue to toValue.
*/
public fromTo(
- prop: AnimatableProp,
+ rawProp: AnimatableProp | AnimatablePropAlias,
fromValue: number | string | boolean,
toValue: number | string | boolean,
options: FromToOptions = {},
): this {
+ const prop = normalizeAnimatableProp(rawProp);
const start = options.from ?? options.start ?? options.fromFrame ?? 0;
const end =
options.to ??
@@ -610,11 +696,12 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* Appends a keyframe to an existing track for prop, or creates a new track.
*/
public keyframe(
- prop: AnimatableProp,
+ rawProp: AnimatableProp | AnimatablePropAlias,
frame: number,
value: number | string | boolean,
ease?: string | EaseRef,
): this {
+ const prop = normalizeAnimatableProp(rawProp);
let track = this.tracks.find(
(t) => t.prop === prop && (!t.source || t.source.type === "keyframe"),
);
@@ -653,10 +740,11 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* Frames are rounded to whole frames, so beat math can be passed straight in.
*/
public keys(
- prop: AnimatableProp,
+ rawProp: AnimatableProp | AnimatablePropAlias,
keyframes: readonly KeyframeTuple[],
options?: TrackKeyOptions,
): this {
+ const prop = normalizeAnimatableProp(rawProp);
for (const [frame, value, ease] of keyframes) {
this.keyframe(prop, Math.round(frame), value, ease);
}
@@ -854,6 +942,33 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
return this.fromTo("rotateZ", fromAngle, toAngle, options);
}
+ /** Alias for {@link rotateX}. */
+ public rotationX(
+ fromAngle: number,
+ toAngle: number,
+ options: FromToOptions = {},
+ ): this {
+ return this.fromTo("rotateX", fromAngle, toAngle, options);
+ }
+
+ /** Alias for {@link rotateY}. */
+ public rotationY(
+ fromAngle: number,
+ toAngle: number,
+ options: FromToOptions = {},
+ ): this {
+ return this.fromTo("rotateY", fromAngle, toAngle, options);
+ }
+
+ /** Alias for {@link rotateZ}. */
+ public rotationZ(
+ fromAngle: number,
+ toAngle: number,
+ options: FromToOptions = {},
+ ): this {
+ return this.fromTo("rotateZ", fromAngle, toAngle, options);
+ }
+
/**
* Animate scale along the X axis.
*/
@@ -953,10 +1068,11 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* registered with `comp.addSignal(name, signal)`.
*/
public signal(
- prop: AnimatableProp,
+ rawProp: AnimatableProp | AnimatablePropAlias,
signalOrHandleId: unknown,
options: SignalTrackOptions = {},
): this {
+ const prop = normalizeAnimatableProp(rawProp);
const { handleId, inline } = resolveSignalArg(signalOrHandleId);
const id = `${prop}_sig_${handleId}_${this.tracks.length}`;
@@ -989,10 +1105,11 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
* registered name, as in {@link LayerAnimation.signal}).
*/
public colorSignal(
- prop: AnimatableProp,
+ rawProp: AnimatableProp | AnimatablePropAlias,
signalOrHandleId: unknown,
options: ColorSignalOptions,
): this {
+ const prop = normalizeAnimatableProp(rawProp);
const { handleId, inline } = resolveSignalArg(signalOrHandleId);
const id = `${prop}_colorSig_${handleId}_${this.tracks.length}`;
@@ -1022,7 +1139,11 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
/**
* Adds organic Perlin noise wiggling to a property.
*/
- public wiggle(prop: AnimatableProp, options: WiggleOptions = {}): this {
+ public wiggle(
+ rawProp: AnimatableProp | AnimatablePropAlias,
+ options: WiggleOptions = {},
+ ): this {
+ const prop = normalizeAnimatableProp(rawProp);
const id = `${prop}_wiggle_${this.tracks.length}`;
const source: TrackSource = {
type: "wiggle",
@@ -1044,7 +1165,11 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>
/**
* Adds physical harmonic spring overshoot behavior to a property transition.
*/
- public spring(prop: AnimatableProp, options: SpringOptions = {}): this {
+ public spring(
+ rawProp: AnimatableProp | AnimatablePropAlias,
+ options: SpringOptions = {},
+ ): this {
+ const prop = normalizeAnimatableProp(rawProp);
const id = `${prop}_spring_${this.tracks.length}`;
const source: TrackSource = {
type: "springOvershoot",
diff --git a/packages/compositions/src/program/evaluator.ts b/packages/compositions/src/program/evaluator.ts
index cc3ba07a..e0a4d3bf 100644
--- a/packages/compositions/src/program/evaluator.ts
+++ b/packages/compositions/src/program/evaluator.ts
@@ -633,6 +633,9 @@ const COLOR_PROPS = new Set([
"strokeColor",
"textBackgroundColor",
"color",
+ "emissive",
+ "subsurfaceColor",
+ "sheenColor",
]);
function computeRepeatFrame(
diff --git a/packages/compositions/src/program/lint.test.ts b/packages/compositions/src/program/lint.test.ts
index 5fdea5b5..714278c7 100644
--- a/packages/compositions/src/program/lint.test.ts
+++ b/packages/compositions/src/program/lint.test.ts
@@ -103,7 +103,7 @@ describe("lintComposition", () => {
it("flags lights and materials that do nothing", () => {
const c = codes([
- { id: "p", kind: "light", lightType: "point", castShadow: true },
+ { id: "a", kind: "light", lightType: "ambient", castShadow: true },
{ id: "m", kind: "model3d", metallic: 1 },
]);
expect(c).toContain("shadow_unsupported_light");
@@ -111,6 +111,8 @@ describe("lintComposition", () => {
expect(
codes([
{ id: "e", kind: "light", lightType: "environment" },
+ { id: "p", kind: "light", lightType: "point", castShadow: true },
+ { id: "r", kind: "light", lightType: "rect", castShadow: true },
{ id: "m", kind: "model3d", metallic: 1 },
]),
).toEqual([]);
diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.ts
index 957b4040..2ee25293 100644
--- a/packages/compositions/src/program/lint.ts
+++ b/packages/compositions/src/program/lint.ts
@@ -168,11 +168,16 @@ export function lintComposition(
if (type === "environment") hasEnvironment = true;
if (type === "ambient") ambientLights.push(id);
if (node.castShadow) {
- if (type !== "directional" && type !== "spot")
+ if (
+ type !== "directional" &&
+ type !== "spot" &&
+ type !== "point" &&
+ type !== "rect"
+ )
push(
"warning",
"shadow_unsupported_light",
- `'${id}' is a ${String(type)} light: only directional and spot lights cast shadows.`,
+ `'${id}' is a ${String(type)} light: only directional, spot, point, and rect lights cast shadows.`,
id,
);
}
diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.ts
index 6f061b55..068d8490 100644
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@ -196,7 +196,24 @@ export const AnimatablePropSchema = z.enum([
// Physically based surfaces & world
"emissive",
"emissiveIntensity",
+ "clearcoat",
+ "clearcoatRoughness",
+ "anisotropy",
+ "anisotropyRotation",
+ "subsurface",
+ "subsurfaceColor",
+ "sheen",
+ "sheenColor",
+ "sheenRoughness",
+ "iridescence",
+ "iridescenceThickness",
+ "iridescenceIor",
+ "triplanarScale",
+ "triplanarSharpness",
"exposure",
+ "volumetricFogDensity",
+ "volumetricFogAnisotropy",
+ "volumetricFogNoiseStrength",
]);
export type AnimatableProp = z.infer<typeof AnimatablePropSchema>;
@@ -222,15 +239,7 @@ export type SpatialAnimatableProp =
| "translateZ"
| "motionBlurShutter";
-export type BoxAnimatableProp =
- | SpatialAnimatableProp
- | "gap"
- | "padding"
- | "borderRadius"
- | "fill"
- | "color"
- | "borderColor"
- | "borderWidth"
+export type MaterialAnimatableProp =
| "shininess"
| "roughness"
| "specularIntensity"
@@ -242,9 +251,34 @@ export type BoxAnimatableProp =
| "fresnelPower"
| "emissive"
| "emissiveIntensity"
+ | "clearcoat"
+ | "clearcoatRoughness"
+ | "anisotropy"
+ | "anisotropyRotation"
+ | "subsurface"
+ | "subsurfaceColor"
+ | "sheen"
+ | "sheenColor"
+ | "sheenRoughness"
+ | "iridescence"
+ | "iridescenceThickness"
+ | "iridescenceIor"
+ | "triplanarScale"
+ | "triplanarSharpness"
| "envIntensity"
| "exposure";
+export type BoxAnimatableProp =
+ | SpatialAnimatableProp
+ | MaterialAnimatableProp
+ | "gap"
+ | "padding"
+ | "borderRadius"
+ | "fill"
+ | "color"
+ | "borderColor"
+ | "borderWidth";
+
export type TextAnimatableProp =
| SpatialAnimatableProp
| "text"
@@ -304,15 +338,24 @@ export type CameraAnimatableProp =
export type LightAnimatableProp =
| SpatialAnimatableProp
+ | "targetX"
+ | "targetY"
+ | "targetZ"
| "intensity"
| "angle"
| "radius"
| "penumbra"
| "decay"
- | "color";
+ | "color"
+ | "exposure"
+ | "volumetricFogDensity"
+ | "volumetricFogAnisotropy"
+ | "volumetricFogNoiseStrength";
export type Model3DAnimatableProp =
| SpatialAnimatableProp
+ | MaterialAnimatableProp
+ | "color"
| "animationProgress"
| "animationTime"
| "animationSpeed"
@@ -711,6 +754,23 @@ export const GeometryOutlineOptionsSchema: z.ZodType<GeometryOutlineOptions> = z
})
.strict();
+export const IridescenceConfigSchema = z
+ .object({
+ intensity: z.number().min(0).max(1).optional(),
+ thickness: z.number().min(100).max(1200).optional(),
+ ior: z.number().min(1.05).max(2.5).optional(),
+ })
+ .strict();
+export type IridescenceConfig = z.infer<typeof IridescenceConfigSchema>;
+
+export const TriplanarConfigSchema = z
+ .object({
+ scale: z.number().min(0).optional(),
+ sharpness: z.number().min(1).max(16).optional(),
+ })
+ .strict();
+export type TriplanarConfig = z.infer<typeof TriplanarConfigSchema>;
+
const BaseNodeFields = {
id: z.string().min(1),
/** Graph binding — which connected input this node renders (for text/media nodes). */
@@ -803,6 +863,38 @@ const BaseNodeFields = {
emissive: ColorSchema.optional(),
/** Emissive multiplier; above 1 glows past white (feeds bloom). Default 1. */
emissiveIntensity: z.number().min(0).optional(),
+ /** Secondary dielectric clearcoat/varnish lobe 0..1. */
+ clearcoat: z.number().min(0).max(1).optional(),
+ /** Roughness of the clearcoat layer 0..1 (default 0.08). */
+ clearcoatRoughness: z.number().min(0).max(1).optional(),
+ /** Anisotropic GGX highlight stretch -1..1. */
+ anisotropy: z.number().min(-1).max(1).optional(),
+ /** Brush tangent rotation in degrees around the surface normal. */
+ anisotropyRotation: z.number().optional(),
+ /** Subsurface wrap diffuse and translucent back-scattering 0..1. */
+ subsurface: z.number().min(0).max(1).optional(),
+ /** Subsurface scatter & transmission tint color. */
+ subsurfaceColor: ColorSchema.optional(),
+ /** Charlie cloth / velvet grazing retro-reflection lobe 0..1. */
+ sheen: z.number().min(0).max(1).optional(),
+ /** Cloth sheen tint color. */
+ sheenColor: ColorSchema.optional(),
+ /** Cloth sheen microfiber roughness 0.05..1 (default 0.5). */
+ sheenRoughness: z.number().min(0.05).max(1).optional(),
+ /** Thin-film spectral iridescence 0..1 or `{ intensity, thickness, ior }`. */
+ iridescence: z
+ .union([z.number().min(0).max(1), IridescenceConfigSchema])
+ .optional(),
+ /** Thin-film thickness in nanometers 100..1200 (default 400). */
+ iridescenceThickness: z.number().min(100).max(1200).optional(),
+ /** Thin-film refractive index 1.05..2.5 (default 1.3). */
+ iridescenceIor: z.number().min(1.05).max(2.5).optional(),
+ /** Seamless world-space triplanar texture projection. */
+ triplanar: z.union([z.boolean(), TriplanarConfigSchema]).optional(),
+ /** World-space triplanar tiling scale. */
+ triplanarScale: z.number().min(0).optional(),
+ /** World-space triplanar axis blend sharpness 1..16 (default 4). */
+ triplanarSharpness: z.number().min(1).max(16).optional(),
/** Falls into a `castShadow` light's shadow map. Meshes default on, flat layers off. */
castShadow: z.boolean().optional(),
/** Art-directed toon lighting, including cast shadows and colored lights. */
@@ -898,6 +990,21 @@ export interface CompositionNodeBase {
fresnelPower?: number;
emissive?: string;
emissiveIntensity?: number;
+ clearcoat?: number;
+ clearcoatRoughness?: number;
+ anisotropy?: number;
+ anisotropyRotation?: number;
+ subsurface?: number;
+ subsurfaceColor?: string;
+ sheen?: number;
+ sheenColor?: string;
+ sheenRoughness?: number;
+ iridescence?: number | IridescenceConfig;
+ iridescenceThickness?: number;
+ iridescenceIor?: number;
+ triplanar?: boolean | TriplanarConfig;
+ triplanarScale?: number;
+ triplanarSharpness?: number;
castShadow?: boolean;
toon?: ToonShading;
outline?: boolean;
@@ -1261,7 +1368,8 @@ export const CameraLensSchema = z
})
.strict();
-export type CameraLens = z.infer<typeof CameraLensSchema>;
+/** Camera lens as authored: fields with schema defaults may be left out. */
+export type CameraLens = z.input<typeof CameraLensSchema>;
export const CameraDoFSchema = z
.object({
@@ -1287,7 +1395,7 @@ export const CameraShakeSchema = z
})
.strict();
-export type CameraShake = z.infer<typeof CameraShakeSchema>;
+export type CameraShake = z.input<typeof CameraShakeSchema>;
export interface CameraNode {
id: string;
@@ -1338,12 +1446,14 @@ export type LightType =
| "directional"
| "point"
| "spot"
+ | "rect"
| "environment";
export const LightTypeSchema = z.enum([
"ambient",
"directional",
"point",
"spot",
+ "rect",
/** Scene world: image-based light, tone mapping, exposure, fog. Turns on physical shading. */
"environment",
]);
@@ -1351,8 +1461,10 @@ export const LightTypeSchema = z.enum([
export const EnvironmentPresetSchema = z.enum([
"studio",
"soft",
+ "warm",
"sunset",
"night",
+ "neon",
"neutral",
]);
export type EnvironmentPreset = z.infer<typeof EnvironmentPresetSchema>;
@@ -1403,6 +1515,52 @@ export const EnvironmentFloorSchema = z
export type EnvironmentFloor = z.infer<typeof EnvironmentFloorSchema>;
+export const EnvironmentVolumetricFogNoiseSchema = z
+ .object({
+ /** World-space noise frequency (1 / period in px). Default 0.003. */
+ scale: z.number().min(0.0001).optional(),
+ /** Density modulation depth 0..1. Default 0.45. */
+ strength: z.number().min(0).max(1).optional(),
+ /** Wind drift velocity [vx, vy, vz] in world px/sec. Default [20, -8, 12]. */
+ speed: z.tuple([z.number(), z.number(), z.number()]).optional(),
+ /** FBM octaves 1..3. Default 2. */
+ octaves: z.number().int().min(1).max(3).optional(),
+ })
+ .strict();
+export type EnvironmentVolumetricFogNoise = z.infer<
+ typeof EnvironmentVolumetricFogNoiseSchema
+>;
+
+export const EnvironmentVolumetricFogSchema = z
+ .object({
+ /** Base extinction/scattering density (typical range 0.002..0.15, default 0.02). */
+ density: z.number().min(0).optional(),
+ /** In-scattering albedo tint (default: environment horizon color). */
+ color: ColorSchema.optional(),
+ /** Henyey-Greenstein forward/back scatter asymmetry g in (-0.95..0.95). Default 0.55. */
+ anisotropy: z.number().min(-0.95).max(0.95).optional(),
+ /** Exponential height falloff rate (0 = uniform, >0 = ground mist). Default 0.0015. */
+ heightFalloff: z.number().min(0).optional(),
+ /** Reference world Y height for ground fog (px, y down). Default 540. */
+ baseHeight: z.number().optional(),
+ /** Raymarch step count per half-res pixel (8..64, default 28). */
+ steps: z.number().int().min(8).max(64).optional(),
+ /** Maximum raymarch distance from camera in world px. Default 2800. */
+ maxDistance: z.number().min(100).optional(),
+ /** Multiplier on direct-light in-scattering / god rays. Default 1.5. */
+ lightIntensity: z.number().min(0).optional(),
+ /** Multiplier on ambient / sky in-scattering. Default 0.35. */
+ ambientIntensity: z.number().min(0).optional(),
+ /** Optional 3D procedural FBM turbulence in the fog medium. */
+ noise: z
+ .union([z.boolean(), EnvironmentVolumetricFogNoiseSchema])
+ .optional(),
+ })
+ .strict();
+export type EnvironmentVolumetricFog = z.infer<
+ typeof EnvironmentVolumetricFogSchema
+>;
+
export interface LightNode {
id: string;
kind: "light";
@@ -1420,16 +1578,24 @@ export interface LightNode {
decay?: number;
angle?: number;
penumbra?: number;
- /** Directional / spot: render a shadow map. */
+ /** Area light width in world px (default 240, rect lights only). */
+ width?: number;
+ /** Area light height in world px (default 140, rect lights only). */
+ height?: number;
+ /** Emit from both faces of a rect light (default false). */
+ twoSided?: boolean;
+ /** Directional / spot / point / rect: render a shadow map. */
castShadow?: boolean;
/** Shadow edge blur in shadow-map texels. Default 2.5. */
shadowSoftness?: number;
/** 0..1, how dark shadows get. Default 1. */
shadowDarkness?: number;
- /** Shadow map resolution (texels per side). Default 2048. */
+ /** Shadow map resolution (texels per side). Default 2048 (1024 for point). */
shadowMapSize?: number;
// environment
preset?: EnvironmentPreset;
+ /** Equirectangular panorama for real HDRI IBL: a `.hdr` (Radiance RGBE) or `.exr` (OpenEXR) path, or the file's bytes. */
+ hdri?: string | Uint8Array;
top?: string;
horizon?: string;
bottom?: string;
@@ -1439,6 +1605,14 @@ export interface LightNode {
toneMapping?: ToneMapping;
fog?: EnvironmentFog;
floor?: EnvironmentFloor;
+ /** Froxel-style volumetric participating media & shadowed light shafts (god rays). */
+ volumetricFog?: boolean | EnvironmentVolumetricFog;
+ /** Animatable / signal-driven override for volumetricFog.density. */
+ volumetricFogDensity?: number;
+ /** Animatable / signal-driven override for volumetricFog.anisotropy. */
+ volumetricFogAnisotropy?: number;
+ /** Animatable / signal-driven override for volumetricFog.noise.strength. */
+ volumetricFogNoiseStrength?: number;
/** Optional depth/normal outlines, before depth of field and bloom. */
outlines?: boolean | GeometryOutlineOptions;
/** Glow around emissive surfaces and hot highlights: `true` or settings. */
@@ -1466,11 +1640,17 @@ export const LightNodeSchema: z.ZodType<LightNode> = z
decay: z.number().min(0).default(2.0),
angle: z.number().min(0).max(180).default(45),
penumbra: z.number().min(0).max(1).default(0.2),
+ width: z.number().min(0).optional(),
+ height: z.number().min(0).optional(),
+ twoSided: z.boolean().optional(),
castShadow: z.boolean().optional(),
shadowSoftness: z.number().min(0).max(64).optional(),
shadowDarkness: z.number().min(0).max(1).optional(),
shadowMapSize: z.number().int().min(256).max(8192).optional(),
preset: EnvironmentPresetSchema.optional(),
+ hdri: z
+ .union([z.string(), z.custom<Uint8Array>((v) => v instanceof Uint8Array)])
+ .optional(),
top: ColorSchema.optional(),
horizon: ColorSchema.optional(),
bottom: ColorSchema.optional(),
@@ -1480,6 +1660,12 @@ export const LightNodeSchema: z.ZodType<LightNode> = z
toneMapping: ToneMappingSchema.optional(),
fog: EnvironmentFogSchema.optional(),
floor: EnvironmentFloorSchema.optional(),
+ volumetricFog: z
+ .union([z.boolean(), EnvironmentVolumetricFogSchema])
+ .optional(),
+ volumetricFogDensity: z.number().min(0).optional(),
+ volumetricFogAnisotropy: z.number().min(-0.95).max(0.95).optional(),
+ volumetricFogNoiseStrength: z.number().min(0).max(1).optional(),
outlines: z.union([z.boolean(), GeometryOutlineOptionsSchema]).optional(),
bloom: z.union([z.boolean(), BloomConfigSchema]).optional(),
startFrame: z.number().int().min(0).default(0),
@@ -1525,6 +1711,26 @@ export interface Model3DNode {
specularIntensity?: number;
ambientIntensity?: number;
metallic?: number;
+ emissive?: string;
+ emissiveIntensity?: number;
+ clearcoat?: number;
+ clearcoatRoughness?: number;
+ anisotropy?: number;
+ anisotropyRotation?: number;
+ subsurface?: number;
+ subsurfaceColor?: string;
+ sheen?: number;
+ sheenColor?: string;
+ sheenRoughness?: number;
+ iridescence?: number | IridescenceConfig;
+ iridescenceThickness?: number;
+ iridescenceIor?: number;
+ triplanar?: boolean | TriplanarConfig;
+ triplanarScale?: number;
+ triplanarSharpness?: number;
+ toon?: ToonShading;
+ outline?: boolean;
+ castShadow?: boolean;
color?: string;
colorTint?: string;
opacity?: number;
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 9852e646..5628a8db 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -164,9 +164,12 @@ import {
collectNodeOps,
compositorToProgram,
type EnvironmentPreset,
+ type EnvironmentVolumetricFog,
+ type EnvironmentVolumetricFogNoise,
type FlexNode,
LayerAnimation,
type LayoutNode,
+ type LightAnimatableProp,
type LightNode,
lintComposition,
type MatteMode,
@@ -538,6 +541,10 @@ export {
EnvironmentFogSchema,
type EnvironmentPreset,
EnvironmentPresetSchema,
+ type EnvironmentVolumetricFog,
+ type EnvironmentVolumetricFogNoise,
+ EnvironmentVolumetricFogNoiseSchema,
+ EnvironmentVolumetricFogSchema,
type ExpressionEvaluationContext,
type ExpressionSelector,
ExpressionSelectorSchema,
@@ -681,6 +688,7 @@ export * from "@framefields/core";
export * from "@framefields/renderers";
export type {
AnimatablePropsFor,
+ LightAnimatableProp,
TextAnimatableProp,
ShapeAnimatableProp,
MediaAnimatableProp,
@@ -727,6 +735,7 @@ export {
SSAOPipeline,
shaderStore,
textureCache,
+ VolumetricFogPipeline,
} from "@framefields/webgpu-renderers";
export {
type ChartAnimationOptions,
@@ -1079,7 +1088,7 @@ export class Media {
}
/**
- * Renders a single frame from this media pipeline to a PNG Buffer.
+ * Renders a single frame from this media pipeline to an image or raw RGBA Buffer.
* Can render either by timestamp in milliseconds (`atMs`) or by frame index (`frame`).
*/
public async renderFrame(
@@ -1087,6 +1096,8 @@ export class Media {
atMs?: number;
frame?: number;
fps?: number;
+ renderId?: string;
+ format?: "png" | "rgba" | "raw" | "jpeg" | "jpg" | "webp";
renderer?: HeadlessMediaRenderer;
} = {},
): Promise<Buffer> {
@@ -2300,17 +2311,25 @@ export class Composition {
}
/**
- * Renders a single frame from the composition to a PNG Buffer.
+ * Renders a single frame from the composition to an image or raw RGBA Buffer.
* Can render either by timestamp in milliseconds (`atMs`) or by frame index (`frame`).
*/
public async renderFrame(
- options: {
- atMs?: number;
- fps?: number;
- frame?: number;
- renderer?: HeadlessMediaRenderer;
- } = {},
+ frameOrOptions:
+ | number
+ | {
+ atMs?: number;
+ fps?: number;
+ frame?: number;
+ renderId?: string;
+ format?: "png" | "rgba" | "raw" | "jpeg" | "jpg" | "webp";
+ renderer?: HeadlessMediaRenderer;
+ } = {},
): Promise<Buffer> {
+ const options =
+ typeof frameOrOptions === "number"
+ ? { frame: frameOrOptions }
+ : frameOrOptions;
const { frame } = resolveFrameRequest(options, this.fps);
if (options.fps !== undefined && options.fps !== this.fps)
throw new Error(
@@ -2344,6 +2363,8 @@ export class Composition {
return renderer.renderFrame(vm, {
fps: this.fps,
frame,
+ renderId: options.renderId,
+ format: options.format,
});
}
@@ -2543,6 +2564,19 @@ function withAnimation<
TCap extends NodeCapability = "visual",
>(node: T): AnimatableNode<T, TCap> {
const target = node as AnimatableNode<T, TCap>;
+ const raw = target as unknown as Record<string, unknown>;
+ if (raw.rotationX !== undefined) {
+ if (raw.rotateX === undefined) raw.rotateX = raw.rotationX;
+ delete raw.rotationX;
+ }
+ if (raw.rotationY !== undefined) {
+ if (raw.rotateY === undefined) raw.rotateY = raw.rotationY;
+ delete raw.rotationY;
+ }
+ if (raw.rotationZ !== undefined) {
+ if (raw.rotateZ === undefined) raw.rotateZ = raw.rotationZ;
+ delete raw.rotationZ;
+ }
// Animation builders are structurally valid inputs, but carry enumerable
// authoring defaults that are not fields of the strict program schema.
if (target.animation instanceof LayerAnimation)
@@ -2838,9 +2872,52 @@ function withAnimation<
configurable: true,
});
+ bindMaterialSignals(target);
return target;
}
+/**
+ * Numeric surface props a live `Signal` may drive directly (`clearcoat: sig`):
+ * the value written to the node is the static fallback, the signal becomes a
+ * draw-time signal track. `iridescence` counts only when it is the number form.
+ */
+const MATERIAL_SIGNAL_PROPS: ReadonlyArray<{
+ prop: AnimatableProp;
+ fallback: number;
+}> = [
+ { prop: "roughness", fallback: 0.5 },
+ { prop: "metallic", fallback: 0 },
+ { prop: "emissiveIntensity", fallback: 1 },
+ { prop: "clearcoat", fallback: 0 },
+ { prop: "clearcoatRoughness", fallback: 0.08 },
+ { prop: "anisotropy", fallback: 0 },
+ { prop: "anisotropyRotation", fallback: 0 },
+ { prop: "subsurface", fallback: 0 },
+ { prop: "sheen", fallback: 0 },
+ { prop: "sheenRoughness", fallback: 0.5 },
+ { prop: "iridescence", fallback: 0 },
+ { prop: "iridescenceThickness", fallback: 400 },
+ { prop: "iridescenceIor", fallback: 1.3 },
+ { prop: "triplanarScale", fallback: 1 },
+ { prop: "triplanarSharpness", fallback: 4 },
+];
+
+/** Moves signal-valued material props of a node onto signal tracks. */
+function bindMaterialSignals<T extends LayoutNode, TCap extends NodeCapability>(
+ target: AnimatableNode<T, TCap>,
+): void {
+ const raw = target as unknown as Record<string, unknown>;
+ let anim: LayerAnimation<AnimatableProp> | undefined;
+ for (const { prop, fallback } of MATERIAL_SIGNAL_PROPS) {
+ const val = raw[prop];
+ if (!isSignalLike(val)) continue;
+ anim ??= LayerAnimation.create<AnimatableProp>();
+ anim.signal(prop, val);
+ raw[prop] = fallback;
+ }
+ if (anim) target.animate(anim);
+}
+
export const Layer = {
video: (
src: string,
@@ -3336,29 +3413,62 @@ export const Layer = {
props: Partial<BoxNode> = {},
): AnimatableNode<BoxNode> =>
buildChart(options, props, FontManager.getAll()[0]?.family ?? "Inter"),
- camera: (options: Partial<CameraNode> = {}): AnimatableNode<CameraNode> =>
- withAnimation<CameraNode>({
- id: options.id ?? autoId("camera"),
+ camera: (options: CameraOptions = {}): AnimatableNode<CameraNode> => {
+ const {
+ fov,
+ focalLength,
+ sensorWidth,
+ near,
+ far,
+ zoom,
+ lens: existingLens,
+ ...rest
+ } = options;
+ const hasTopLens =
+ fov !== undefined ||
+ focalLength !== undefined ||
+ sensorWidth !== undefined ||
+ near !== undefined ||
+ far !== undefined ||
+ zoom !== undefined;
+ const mergedLens =
+ hasTopLens || existingLens !== undefined
+ ? {
+ ...existingLens,
+ ...(fov !== undefined && { fov }),
+ ...(focalLength !== undefined && { focalLength }),
+ ...(sensorWidth !== undefined && { sensorWidth }),
+ ...(near !== undefined && { near }),
+ ...(far !== undefined && { far }),
+ ...(zoom !== undefined && { zoom }),
+ }
+ : undefined;
+ return withAnimation<CameraNode>({
+ id: rest.id ?? autoId("camera"),
kind: "camera",
- mode: options.mode ?? "lookAt",
- pitch: options.pitch ?? 0,
- yaw: options.yaw ?? 0,
- roll: options.roll ?? 0,
- ...options,
- }),
- light: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>
- withAnimation<LightNode>({
- id: options.id ?? autoId("light"),
- kind: "light",
- lightType: options.lightType ?? "point",
- color: options.color ?? "#ffffff",
- intensity: options.intensity ?? 1.0,
- ...options,
- }),
+ mode: rest.mode ?? "lookAt",
+ pitch: rest.pitch ?? 0,
+ yaw: rest.yaw ?? 0,
+ roll: rest.roll ?? 0,
+ ...rest,
+ ...(mergedLens !== undefined && { lens: mergedLens }),
+ });
+ },
+ light: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ buildLightNode(
+ {
+ id: options.id ?? autoId("light"),
+ kind: "light",
+ lightType: options.lightType ?? "point",
+ color: options.color ?? "#ffffff",
+ intensity: 1.0,
+ },
+ options as Record<string, unknown>,
+ ),
ambientLight: (
- color: string | Partial<LightNode> = "#ffffff",
- intensity: number = 0.5,
- options: Partial<LightNode> = {},
+ color: string | LightOptions = "#ffffff",
+ intensity: SignalOrNumber = 0.5,
+ options: LightOptions = {},
): AnimatableNode<LightNode> => Light.ambient(color, intensity, options),
/**
* Scenes back to back with transitions between them; see `Transition`.
@@ -3370,18 +3480,22 @@ export const Layer = {
entries: SequenceEntry[],
options?: SceneSequenceOptions,
): SequenceNode => sequence(entries, options),
- /** See {@link Light.environment}: `Layer.environment("studio", { exposure: 0.3 })`. */
+ /** See {@link Light.environment}: `Layer.environment("studio", { exposure: 0.3 })` or `Layer.environment("sky.hdr")`. */
environment: (
- presetOrOptions: EnvironmentPreset | EnvironmentOptions = "studio",
+ presetOrOptions:
+ | EnvironmentPreset
+ | HdriPath
+ | EnvironmentOptions = "studio",
options: EnvironmentOptions = {},
): AnimatableNode<LightNode> => Light.environment(presetOrOptions, options),
- directionalLight: (
- options: Partial<LightNode> = {},
- ): AnimatableNode<LightNode> => Light.directional(options),
- pointLight: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>
+ directionalLight: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ Light.directional(options),
+ pointLight: (options: LightOptions = {}): AnimatableNode<LightNode> =>
Light.point(options),
- spotLight: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>
+ spotLight: (options: LightOptions = {}): AnimatableNode<LightNode> =>
Light.spot(options),
+ rectLight: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ Light.rect(options),
cube: (options: Cube3DOptions): AnimatableNode<BoxNode> =>
Layer3D.cube(options),
carousel3d: (options: Carousel3DOptions): AnimatableNode<BoxNode> =>
@@ -3390,7 +3504,7 @@ export const Layer = {
Layer3D.plane(options),
prism3d: (options: Prism3DOptions = {}): SolidShape3DNode =>
Layer3D.prism(options),
- model: (options: Partial<Model3DNode> = {}): AnimatableNode<Model3DNode> =>
+ model: (options: Model3DOptions = {}): AnimatableNode<Model3DNode> =>
withAnimation({
id: options.id ?? autoId("model3d"),
kind: "model3d",
@@ -3408,7 +3522,7 @@ export const Layer = {
}),
obj: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3417,7 +3531,7 @@ export const Layer = {
}),
fbx: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3426,7 +3540,7 @@ export const Layer = {
}),
gltf: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3435,7 +3549,7 @@ export const Layer = {
}),
glb: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3444,7 +3558,7 @@ export const Layer = {
}),
stl: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3453,7 +3567,7 @@ export const Layer = {
}),
ply: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3462,7 +3576,7 @@ export const Layer = {
}),
vox: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3471,7 +3585,7 @@ export const Layer = {
}),
threeDS: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3480,7 +3594,7 @@ export const Layer = {
}),
off: (
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> =>
Layer.model({
src,
@@ -3489,115 +3603,364 @@ export const Layer = {
}),
};
+export interface CameraOptions extends Partial<CameraNode> {
+ /** Top-level shorthand for `lens.fov` (vertical field of view in degrees). */
+ fov?: number;
+ /** Top-level shorthand for `lens.focalLength` (mm). */
+ focalLength?: number;
+ /** Top-level shorthand for `lens.sensorWidth` (mm). */
+ sensorWidth?: number;
+ /** Top-level shorthand for `lens.near` clipping plane. */
+ near?: number;
+ /** Top-level shorthand for `lens.far` clipping plane. */
+ far?: number;
+ /** Top-level shorthand for `lens.zoom` multiplier. */
+ zoom?: number;
+}
+
+export interface Model3DOptions extends Partial<Model3DNode> {
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
+}
+
+export type SignalOrNumber =
+ | number
+ | ProgrammaticSignal
+ | Signal<number>
+ | { get: (ctx: FrameContext) => number };
+
+export interface VolumetricFogNoiseOptions
+ extends Omit<EnvironmentVolumetricFogNoise, "strength"> {
+ strength?: SignalOrNumber;
+}
+
+export interface VolumetricFogOptions
+ extends Omit<EnvironmentVolumetricFog, "density" | "anisotropy" | "noise"> {
+ density?: SignalOrNumber;
+ anisotropy?: SignalOrNumber;
+ noise?: boolean | VolumetricFogNoiseOptions;
+}
+
+/** An equirectangular panorama path (Radiance `.hdr` or OpenEXR `.exr`), usable in place of a preset name. */
+export type HdriPath =
+ | `${string}.hdr`
+ | `${string}.HDR`
+ | `${string}.exr`
+ | `${string}.EXR`;
+
+function isHdriPath(value: string): value is HdriPath {
+ return /\.(hdr|exr)$/i.test(value);
+}
+
/** World settings for `Layer.environment` (see `LightNode` for every field). */
-export type EnvironmentOptions = Partial<
- Pick<
- LightNode,
- | "id"
- | "preset"
- | "top"
- | "horizon"
- | "bottom"
+export interface EnvironmentOptions
+ extends Partial<
+ Pick<
+ LightNode,
+ | "id"
+ | "preset"
+ | "hdri"
+ | "color"
+ | "top"
+ | "horizon"
+ | "bottom"
+ | "softboxes"
+ | "toneMapping"
+ | "fog"
+ | "floor"
+ | "outlines"
+ | "bloom"
+ | "startFrame"
+ | "durationFrames"
+ >
+ > {
+ intensity?: SignalOrNumber;
+ rotation?: SignalOrNumber;
+ exposure?: SignalOrNumber;
+ volumetricFog?: boolean | VolumetricFogOptions;
+ volumetricFogDensity?: SignalOrNumber;
+ volumetricFogAnisotropy?: SignalOrNumber;
+ volumetricFogNoiseStrength?: SignalOrNumber;
+}
+
+/** Options for `Light.*` and `Layer.*Light` (numeric fields also accept live `Signal`s). */
+export interface LightOptions
+ extends Omit<
+ Partial<LightNode>,
| "intensity"
- | "softboxes"
+ | "x"
+ | "y"
+ | "z"
+ | "targetX"
+ | "targetY"
+ | "targetZ"
+ | "radius"
+ | "angle"
+ | "penumbra"
+ | "decay"
+ | "width"
+ | "height"
| "rotation"
| "exposure"
- | "toneMapping"
- | "fog"
- | "floor"
- | "outlines"
- | "bloom"
- | "startFrame"
- | "durationFrames"
- >
->;
+ | "volumetricFog"
+ | "volumetricFogDensity"
+ | "volumetricFogAnisotropy"
+ | "volumetricFogNoiseStrength"
+ > {
+ intensity?: SignalOrNumber;
+ x?: SignalOrNumber;
+ y?: SignalOrNumber;
+ z?: SignalOrNumber;
+ targetX?: SignalOrNumber;
+ targetY?: SignalOrNumber;
+ targetZ?: SignalOrNumber;
+ radius?: SignalOrNumber;
+ angle?: SignalOrNumber;
+ penumbra?: SignalOrNumber;
+ decay?: SignalOrNumber;
+ width?: SignalOrNumber;
+ height?: SignalOrNumber;
+ rotation?: SignalOrNumber;
+ exposure?: SignalOrNumber;
+ volumetricFog?: boolean | VolumetricFogOptions;
+ volumetricFogDensity?: SignalOrNumber;
+ volumetricFogAnisotropy?: SignalOrNumber;
+ volumetricFogNoiseStrength?: SignalOrNumber;
+}
+
+const LIGHT_SIGNAL_PROPS: ReadonlyArray<{
+ prop: LightAnimatableProp;
+ fallback: number;
+}> = [
+ { prop: "intensity", fallback: 1.0 },
+ { prop: "x", fallback: 0 },
+ { prop: "y", fallback: 0 },
+ { prop: "z", fallback: 0 },
+ { prop: "targetX", fallback: 0 },
+ { prop: "targetY", fallback: 0 },
+ { prop: "targetZ", fallback: 0 },
+ { prop: "radius", fallback: 1000 },
+ { prop: "angle", fallback: 45 },
+ { prop: "penumbra", fallback: 0.2 },
+ { prop: "decay", fallback: 1.0 },
+ { prop: "width", fallback: 240 },
+ { prop: "height", fallback: 140 },
+ { prop: "rotation", fallback: 0 },
+ { prop: "exposure", fallback: 0 },
+ { prop: "volumetricFogDensity", fallback: 0.02 },
+ { prop: "volumetricFogAnisotropy", fallback: 0.55 },
+ { prop: "volumetricFogNoiseStrength", fallback: 0.45 },
+];
+
+function isSignalLike(val: unknown): boolean {
+ if (!val || typeof val !== "object") return false;
+ const obj = val as {
+ get?: unknown;
+ inputHandleId?: unknown;
+ gpuBinding?: unknown;
+ };
+ return (
+ typeof obj.get === "function" ||
+ typeof obj.inputHandleId === "string" ||
+ typeof obj.gpuBinding === "object"
+ );
+}
+
+function buildLightNode(
+ defaults: LightNode,
+ rawOpts: Record<string, unknown>,
+): AnimatableNode<LightNode> {
+ const cleaned: Record<string, unknown> = { ...defaults, ...rawOpts };
+ const signalBindings: Array<[LightAnimatableProp, unknown]> = [];
+
+ for (const { prop, fallback } of LIGHT_SIGNAL_PROPS) {
+ const val = cleaned[prop];
+ if (isSignalLike(val)) {
+ signalBindings.push([prop, val]);
+ const defVal = (defaults as unknown as Record<string, unknown>)[prop];
+ cleaned[prop] = typeof defVal === "number" ? defVal : fallback;
+ }
+ }
+
+ const rawVf = cleaned.volumetricFog;
+ if (rawVf && typeof rawVf === "object") {
+ const vfCopy: Record<string, unknown> = {
+ ...(rawVf as Record<string, unknown>),
+ };
+ if (isSignalLike(vfCopy.density)) {
+ signalBindings.push(["volumetricFogDensity", vfCopy.density]);
+ vfCopy.density = 0.02;
+ }
+ if (isSignalLike(vfCopy.anisotropy)) {
+ signalBindings.push(["volumetricFogAnisotropy", vfCopy.anisotropy]);
+ vfCopy.anisotropy = 0.55;
+ }
+ if (vfCopy.noise && typeof vfCopy.noise === "object") {
+ const noiseCopy: Record<string, unknown> = {
+ ...(vfCopy.noise as Record<string, unknown>),
+ };
+ if (isSignalLike(noiseCopy.strength)) {
+ signalBindings.push(["volumetricFogNoiseStrength", noiseCopy.strength]);
+ noiseCopy.strength = 0.45;
+ }
+ vfCopy.noise = noiseCopy;
+ }
+ cleaned.volumetricFog = vfCopy;
+ }
+
+ let node = withAnimation(cleaned as unknown as LightNode);
+ if (signalBindings.length > 0) {
+ const anim = LayerAnimation.create<LightAnimatableProp>();
+ for (const [prop, sig] of signalBindings) {
+ anim.signal(prop, sig as never);
+ }
+ node = node.animate(anim);
+ }
+ return node;
+}
export const Light = {
/**
* The scene's world: a procedural studio environment that lights and
- * reflects off every surface, plus exposure, tone mapping and fog. Adding
- * one switches the scene to physically based shading (GGX, real
- * `roughness` / `metallic`, `emissive` that blooms past white).
+ * reflects off every surface, plus exposure, tone mapping, fog and
+ * volumetric light shafts. Adding one switches the scene to physically
+ * based shading (GGX, real `roughness` / `metallic`, `emissive` that
+ * blooms past white). Pass a `.hdr` or `.exr` path instead of a preset name to light
+ * and reflect from a captured equirectangular panorama:
+ * `Light.environment("assets/hdri/studio.hdr", { intensity: 1.5 })`.
*/
environment: (
- presetOrOptions: EnvironmentPreset | EnvironmentOptions = "studio",
+ presetOrOptions:
+ | EnvironmentPreset
+ | HdriPath
+ | EnvironmentOptions = "studio",
options: EnvironmentOptions = {},
): AnimatableNode<LightNode> => {
- const opts =
- typeof presetOrOptions === "string"
- ? { preset: presetOrOptions, ...options }
- : { ...presetOrOptions, ...options };
- return withAnimation({
- kind: "light",
- lightType: "environment",
- preset: "studio",
- intensity: 1,
- ...opts,
- id: opts.id ?? autoId("environment"),
- });
+ const first: EnvironmentOptions =
+ typeof presetOrOptions !== "string"
+ ? presetOrOptions
+ : isHdriPath(presetOrOptions)
+ ? { hdri: presetOrOptions }
+ : { preset: presetOrOptions };
+ const opts = { ...first, ...options };
+ return buildLightNode(
+ {
+ id: opts.id ?? autoId("environment"),
+ kind: "light",
+ lightType: "environment",
+ preset: "studio",
+ intensity: 1,
+ },
+ opts as Record<string, unknown>,
+ );
},
ambient: (
- color: string | Partial<LightNode> = "#ffffff",
- intensity: number = 0.5,
- options: Partial<LightNode> = {},
- ): AnimatableNode<LightNode> =>
- withAnimation({
- id: options.id ?? autoId("light-ambient"),
- kind: "light",
- lightType: "ambient",
+ color: string | LightOptions = "#ffffff",
+ intensity: SignalOrNumber = 0.5,
+ options: LightOptions = {},
+ ): AnimatableNode<LightNode> => {
+ const merged: LightOptions = {
color: typeof color === "string" ? color : (color.color ?? "#ffffff"),
intensity,
...(typeof color === "object" && color),
...options,
- }),
- directional: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>
- withAnimation({
- id: options.id ?? autoId("light-directional"),
- kind: "light",
- lightType: "directional",
- color: options.color ?? "#ffffff",
- intensity: options.intensity ?? 1.0,
- x: options.x ?? 0,
- y: options.y ?? -1000,
- z: options.z ?? -1000,
- targetX: options.targetX ?? 0,
- targetY: options.targetY ?? 0,
- targetZ: options.targetZ ?? 0,
- ...options,
- }),
- point: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>
- withAnimation({
- id: options.id ?? autoId("light-point"),
- kind: "light",
- lightType: "point",
- color: options.color ?? "#ffffff",
- intensity: options.intensity ?? 1.0,
- x: options.x ?? 0,
- y: options.y ?? 0,
- z: options.z ?? -500,
- radius: options.radius ?? 1000,
- decay: options.decay ?? 1.0,
- ...options,
- }),
- spot: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>
- withAnimation({
- id: options.id ?? autoId("light-spot"),
- kind: "light",
- lightType: "spot",
- color: options.color ?? "#ffffff",
- intensity: options.intensity ?? 1.0,
- x: options.x ?? 0,
- y: options.y ?? -500,
- z: options.z ?? -1000,
- targetX: options.targetX ?? 0,
- targetY: options.targetY ?? 0,
- targetZ: options.targetZ ?? 0,
- angle: options.angle ?? 45,
- penumbra: options.penumbra ?? 0.2,
- radius: options.radius ?? 2000,
- decay: options.decay ?? 1.0,
- ...options,
- }),
+ };
+ return buildLightNode(
+ {
+ id: merged.id ?? autoId("light-ambient"),
+ kind: "light",
+ lightType: "ambient",
+ color: "#ffffff",
+ intensity: 0.5,
+ },
+ merged as Record<string, unknown>,
+ );
+ },
+ directional: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ buildLightNode(
+ {
+ id: options.id ?? autoId("light-directional"),
+ kind: "light",
+ lightType: "directional",
+ color: options.color ?? "#ffffff",
+ intensity: 1.0,
+ x: 0,
+ y: -1000,
+ z: -1000,
+ targetX: 0,
+ targetY: 0,
+ targetZ: 0,
+ },
+ options as Record<string, unknown>,
+ ),
+ point: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ buildLightNode(
+ {
+ id: options.id ?? autoId("light-point"),
+ kind: "light",
+ lightType: "point",
+ color: options.color ?? "#ffffff",
+ intensity: 1.0,
+ x: 0,
+ y: 0,
+ z: -500,
+ radius: 1000,
+ decay: 1.0,
+ },
+ options as Record<string, unknown>,
+ ),
+ spot: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ buildLightNode(
+ {
+ id: options.id ?? autoId("light-spot"),
+ kind: "light",
+ lightType: "spot",
+ color: options.color ?? "#ffffff",
+ intensity: 1.0,
+ x: 0,
+ y: -500,
+ z: -1000,
+ targetX: 0,
+ targetY: 0,
+ targetZ: 0,
+ angle: 45,
+ penumbra: 0.2,
+ radius: 2000,
+ decay: 1.0,
+ },
+ options as Record<string, unknown>,
+ ),
+ /**
+ * A rectangular area light (studio softbox / LED panel / neon window)
+ * with solid-angle diffuse illumination and representative-point GGX
+ * specular reflections.
+ */
+ rect: (options: LightOptions = {}): AnimatableNode<LightNode> =>
+ buildLightNode(
+ {
+ id: options.id ?? autoId("light-rect"),
+ kind: "light",
+ lightType: "rect",
+ color: options.color ?? "#ffffff",
+ intensity: 1.0,
+ x: 0,
+ y: -300,
+ z: -800,
+ targetX: 0,
+ targetY: 0,
+ targetZ: 0,
+ width: 240,
+ height: 140,
+ twoSided: false,
+ radius: 2000,
+ decay: 1.5,
+ },
+ options as Record<string, unknown>,
+ ),
};
export {
diff --git a/packages/framefields/src/lighting.ts b/packages/framefields/src/lighting.ts
index 66d97ddf..bff425dd 100644
--- a/packages/framefields/src/lighting.ts
+++ b/packages/framefields/src/lighting.ts
@@ -28,6 +28,8 @@ export interface LightingRigOptions {
shadowSoftness?: number;
/** Fog fading into the background, e.g. `{ color: BG, start: 2500, end: 6000 }`. */
fog?: EnvironmentOptions["fog"];
+ /** Volumetric fog & shadowed light shafts (god rays): `true` or settings. */
+ volumetricFog?: EnvironmentOptions["volumetricFog"];
/** Glow around emissive surfaces and highlights: `true` or settings. */
bloom?: EnvironmentOptions["bloom"];
/** Prefix for the rig's node ids. */
@@ -121,6 +123,7 @@ function rig(
exposure: options.exposure,
toneMapping: options.toneMapping,
fog: options.fog,
+ volumetricFog: options.volumetricFog,
bloom: options.bloom,
}),
Light.directional({
diff --git a/packages/framefields/src/shapes3d.ts b/packages/framefields/src/shapes3d.ts
index f886bbf1..43a71718 100644
--- a/packages/framefields/src/shapes3d.ts
+++ b/packages/framefields/src/shapes3d.ts
@@ -1,9 +1,11 @@
import type {
BoxNode,
+ IridescenceConfig,
LayoutNode,
MaterialType,
Model3DNode,
ToonShading,
+ TriplanarConfig,
} from "@framefields/compositions/program";
import {
type CapsuleGeometryOptions,
@@ -28,7 +30,12 @@ import {
type TubePathSpec,
} from "@framefields/webgpu-renderers";
import { autoId } from "./ids.js";
-import { type AnimatableNode, Layer } from "./index.js";
+import {
+ type AnimatableNode,
+ Layer,
+ type Model3DOptions,
+ type SignalOrNumber,
+} from "./index.js";
import type { PhysicsBody3D } from "./physics/world3d.js";
import { createSwarm3D, type Swarm3DOptions } from "./swarm3d.js";
@@ -44,6 +51,37 @@ export interface Surface3DOptions {
emissiveIntensity?: number;
/** Falls into a `castShadow` light's shadow map (meshes default on, flat layers off). */
castShadow?: boolean;
+ // Substrate lobes: physical shading only (needs `Layer.environment`).
+ /** Clear varnish layer over the base 0..1 (car paint, lacquer, wet coats). */
+ clearcoat?: SignalOrNumber;
+ /** Roughness of the clearcoat layer 0..1. Default 0.08. */
+ clearcoatRoughness?: SignalOrNumber;
+ /** Stretched highlight -1..1 (brushed metal, satin, hair). 0 = round. */
+ anisotropy?: SignalOrNumber;
+ /** Brush direction in degrees around the surface normal. Default 0. */
+ anisotropyRotation?: SignalOrNumber;
+ /** Light bleeding through the surface 0..1 (wax, jade, skin, leaves). */
+ subsurface?: SignalOrNumber;
+ /** Color of the scattered light (CSS color). Default "#ff6b4a". */
+ subsurfaceColor?: string;
+ /** Velvet / cloth rim glow at grazing angles 0..1. */
+ sheen?: SignalOrNumber;
+ /** Sheen tint (CSS color). Default "#ffffff". */
+ sheenColor?: string;
+ /** Sheen fiber roughness 0.05..1. Default 0.5. */
+ sheenRoughness?: SignalOrNumber;
+ /** Thin-film rainbow (soap, oil, beetle shells): 0..1 or `{ intensity, thickness, ior }`. */
+ iridescence?: SignalOrNumber | IridescenceConfig;
+ /** Film thickness in nanometers 100..1200. Default 400. */
+ iridescenceThickness?: SignalOrNumber;
+ /** Film refractive index 1.05..2.5. Default 1.3. */
+ iridescenceIor?: SignalOrNumber;
+ /** Project textures in world space instead of UVs: `true` or `{ scale, sharpness }`. */
+ triplanar?: boolean | TriplanarConfig;
+ /** Triplanar tiling: repeats per 256 world px (> 0 turns triplanar on). */
+ triplanarScale?: SignalOrNumber;
+ /** Triplanar axis blend sharpness 1..16. Default 4. */
+ triplanarSharpness?: SignalOrNumber;
}
const MATERIAL_KEYS = [
@@ -63,6 +101,21 @@ const MATERIAL_KEYS = [
"emissive",
"emissiveIntensity",
"castShadow",
+ "clearcoat",
+ "clearcoatRoughness",
+ "anisotropy",
+ "anisotropyRotation",
+ "subsurface",
+ "subsurfaceColor",
+ "sheen",
+ "sheenColor",
+ "sheenRoughness",
+ "iridescence",
+ "iridescenceThickness",
+ "iridescenceIor",
+ "triplanar",
+ "triplanarScale",
+ "triplanarSharpness",
] as const;
/** The material and surface options that were actually given. */
@@ -106,6 +159,12 @@ export interface Cube3DOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
faces?: {
front?: CubeFaceInput;
back?: CubeFaceInput;
@@ -139,6 +198,12 @@ export interface Carousel3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
faceInward?: boolean;
itemWidth?: number;
itemHeight?: number;
@@ -154,8 +219,12 @@ export interface Prism3DOptions extends Surface3DOptions {
radius?: number;
/** Optional top (-y) radius; defaults to `radius`. */
radiusTop?: number;
+ /** Alias for `radiusTop`. */
+ topRadius?: number;
/** Optional bottom (+y) radius; defaults to `radius`. */
radiusBottom?: number;
+ /** Alias for `radiusBottom`. */
+ bottomRadius?: number;
/** Height along Y (default 100). */
height?: number;
x?: number;
@@ -164,6 +233,12 @@ export interface Prism3DOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
scale?: number;
scaleX?: number;
scaleY?: number;
@@ -205,6 +280,8 @@ export interface Pyramid3DOptions extends Surface3DOptions {
radius?: number;
/** Optional top (-y) truncation radius for a frustum pyramid (default 0 = sharp apex). */
radiusTop?: number;
+ /** Alias for `radiusTop`. */
+ topRadius?: number;
/** Height from base (+y) to apex (-y) (default 100). */
height?: number;
/** Leave the base cap off (default false = solid capped pyramid). */
@@ -217,6 +294,12 @@ export interface Pyramid3DOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
scale?: number;
scaleX?: number;
scaleY?: number;
@@ -251,6 +334,12 @@ export interface Plane3DOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
background?: string;
borderRadius?: number;
twoSided?: boolean;
@@ -283,6 +372,12 @@ export interface Grid3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
opacity?: number;
}
@@ -300,6 +395,12 @@ export interface Text3DOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
scale?: number;
scaleX?: number;
scaleY?: number;
@@ -362,6 +463,12 @@ export interface Primitive3DNodeOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
scale?: number;
scaleX?: number;
scaleY?: number;
@@ -478,6 +585,12 @@ export interface Instances3DOptions extends Surface3DOptions {
rotateX?: number;
rotateY?: number;
rotateZ?: number;
+ /** Alias for `rotateX`. */
+ rotationX?: number;
+ /** Alias for `rotateY`. */
+ rotationY?: number;
+ /** Alias for `rotateZ`. */
+ rotationZ?: number;
scale?: number;
color?: string;
colorTint?: string;
@@ -535,9 +648,9 @@ function primitiveNode(
x: o.x ?? 0,
y: o.y ?? 0,
z: o.z ?? 0,
- rotateX: o.rotateX ?? 0,
- rotateY: o.rotateY ?? 0,
- rotateZ: o.rotateZ ?? 0,
+ rotateX: o.rotateX ?? o.rotationX ?? 0,
+ rotateY: o.rotateY ?? o.rotationY ?? 0,
+ rotateZ: o.rotateZ ?? o.rotationZ ?? 0,
scale: o.scale ?? 1,
scaleX: o.scaleX ?? 1,
scaleY: o.scaleY ?? 1,
@@ -759,9 +872,9 @@ export const Layer3D = {
height: h,
is3D: true,
transformStyle: "preserve-3d",
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
opacity: options.opacity ?? 1,
background: "transparent",
children: faces,
@@ -810,9 +923,9 @@ export const Layer3D = {
height: options.itemHeight ?? 200,
is3D: true,
transformStyle: "preserve-3d",
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
opacity: options.opacity ?? 1,
background: "transparent",
children: transformedItems,
@@ -826,10 +939,12 @@ export const Layer3D = {
* outward flat-shaded facet normals, and full material options.
* If `sideConfigs` are provided, builds a multi-panel preserve-3d container with all material props applied.
*/
- prism(options: Prism3DOptions): Shape3DContainer {
- const n = Math.max(3, Math.round(options.sides));
+ prism(options: Prism3DOptions = {}): SolidShape3DNode {
+ const n = Math.max(3, Math.round(options.sides ?? 6));
+ const radius = options.radius ?? 60;
+ const height = options.height ?? 100;
if (options.sideConfigs && options.sideConfigs.length > 0) {
- const { radius, height, twoSided = true } = options;
+ const { twoSided = true } = options;
const sideWidth = 2 * radius * Math.tan(Math.PI / n);
const angleStep = 360 / n;
const panels: AnimatableNode<BoxNode>[] = [];
@@ -900,25 +1015,28 @@ export const Layer3D = {
height,
is3D: true,
transformStyle: "preserve-3d",
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
opacity: options.opacity ?? 1,
background: "transparent",
children: panels,
});
- return withParts(container, { faces: panels });
+ return withParts(container, {
+ faces: panels,
+ items: panels,
+ }) as unknown as SolidShape3DNode;
}
const modelNode = primitiveNode(
"prism",
{
type: "cylinder",
- radius: options.radius,
- radiusTop: options.radiusTop ?? options.radius,
- radiusBottom: options.radiusBottom ?? options.radius,
- height: options.height,
+ radius,
+ radiusTop: options.radiusTop ?? options.topRadius ?? radius,
+ radiusBottom: options.radiusBottom ?? options.bottomRadius ?? radius,
+ height,
segments: n,
openEnded: options.openEnded ?? false,
flatShaded: options.flatShaded ?? true,
@@ -933,23 +1051,23 @@ export const Layer3D = {
return withParts(modelNode, {
faces: [modelNode],
items: [modelNode],
- }) as unknown as Shape3DContainer;
+ }) as unknown as SolidShape3DNode;
},
/**
* Creates a solid N-sided regular polygonal pyramid (or truncated pyramid when `radiusTop > 0`)
* with a bottom cap and crisp flat-shaded side facets.
*/
- pyramid(options: Pyramid3DOptions): AnimatableNode<Model3DNode> {
+ pyramid(options: Pyramid3DOptions = {}): AnimatableNode<Model3DNode> {
const n = Math.max(3, Math.round(options.sides ?? 4));
return primitiveNode(
"pyramid",
{
type: "cylinder",
radius: options.radius ?? 60,
- radiusTop: options.radiusTop ?? 0,
+ radiusTop: options.radiusTop ?? options.topRadius ?? 0,
radiusBottom: options.radius ?? 60,
- height: options.height,
+ height: options.height ?? 100,
segments: n,
openEnded: options.openEnded ?? false,
flatShaded: options.flatShaded ?? true,
@@ -976,9 +1094,9 @@ export const Layer3D = {
z: options.z ?? 0,
is3D: true,
twoSided: options.twoSided ?? true,
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
background: options.background ?? "#3b82f6",
borderRadius: options.borderRadius ?? 0,
opacity: options.opacity ?? 1,
@@ -1046,9 +1164,9 @@ export const Layer3D = {
z: options.z ?? 0,
is3D: true,
transformStyle: "preserve-3d",
- rotateX: options.rotateX ?? 90,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 90,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
background: "transparent",
opacity: options.opacity ?? 1,
children: lines,
@@ -1074,9 +1192,9 @@ export const Layer3D = {
z: options.z ?? 0,
is3D: true,
twoSided: options.twoSided ?? true,
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
fontSize: options.fontSize ?? 48,
fontFamily: options.fontFamily,
fontWeight: options.fontWeight ?? "bold",
@@ -1166,9 +1284,9 @@ export const Layer3D = {
x: options.x ?? 0,
y: options.y ?? 0,
z: options.z ?? 0,
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
scale: options.scale ?? 1,
scaleX: options.scaleX ?? 1,
scaleY: options.scaleY ?? 1,
@@ -1206,7 +1324,7 @@ export const Layer3D = {
width: options.width,
height: options.height,
depth: options.depth,
- radius: options.radius,
+ radius: options.radius ?? options.cornerRadius ?? options.borderRadius,
segments: options.segments,
taper: options.taper,
},
@@ -1230,8 +1348,8 @@ export const Layer3D = {
{
type: "cylinder",
radius: options.radius,
- radiusTop: options.radiusTop,
- radiusBottom: options.radiusBottom,
+ radiusTop: options.radiusTop ?? options.topRadius,
+ radiusBottom: options.radiusBottom ?? options.bottomRadius,
height: options.height,
segments: options.segments,
openEnded: options.openEnded,
@@ -1253,8 +1371,8 @@ export const Layer3D = {
"torus",
{
type: "torus",
- radius: options.radius,
- tube: options.tube,
+ radius: options.radius ?? options.majorRadius,
+ tube: options.tube ?? options.minorRadius ?? options.tubeRadius,
radialSegments: options.radialSegments,
tubularSegments: options.tubularSegments,
arc: options.arc,
@@ -1507,9 +1625,9 @@ export const Layer3D = {
x: options.x ?? 0,
y: options.y ?? 0,
z: options.z ?? 0,
- rotateX: options.rotateX ?? 0,
- rotateY: options.rotateY ?? 0,
- rotateZ: options.rotateZ ?? 0,
+ rotateX: options.rotateX ?? options.rotationX ?? 0,
+ rotateY: options.rotateY ?? options.rotationY ?? 0,
+ rotateZ: options.rotateZ ?? options.rotationZ ?? 0,
scale: options.scale ?? 1,
color: options.color,
colorTint: options.colorTint,
@@ -1549,77 +1667,56 @@ export const Layer3D = {
/**
* Creates a 3D model node rendered natively with WebGPU mesh shaders.
*/
- model(options: Partial<Model3DNode> = {}): AnimatableNode<Model3DNode> {
+ model(options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.model(options);
},
/**
* Convenience helper to create a 3D Wavefront OBJ model node.
*/
- obj(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ obj(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.obj(src, options);
},
/**
* Convenience helper to create a 3D Autodesk FBX model node with skeletal animation support.
*/
- fbx(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ fbx(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.fbx(src, options);
},
/**
* Convenience helper to create a 3D glTF 2.0 model node.
*/
- gltf(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ gltf(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.gltf(src, options);
},
/**
* Convenience helper to create a 3D GLB binary container model node.
*/
- glb(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ glb(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.glb(src, options);
},
/**
* Convenience helper to create a 3D STL CAD / 3D printing model node.
*/
- stl(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ stl(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.stl(src, options);
},
/**
* Convenience helper to create a 3D Stanford PLY scan model node.
*/
- ply(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ ply(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.ply(src, options);
},
/**
* Convenience helper to create a 3D MagicaVoxel VOX model node.
*/
- vox(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ vox(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.vox(src, options);
},
@@ -1628,7 +1725,7 @@ export const Layer3D = {
*/
threeDS(
src: string,
- options: Partial<Model3DNode> = {},
+ options: Model3DOptions = {},
): AnimatableNode<Model3DNode> {
return Layer.threeDS(src, options);
},
@@ -1636,10 +1733,7 @@ export const Layer3D = {
/**
* Convenience helper to create a 3D Object File Format (OFF) model node.
*/
- off(
- src: string,
- options: Partial<Model3DNode> = {},
- ): AnimatableNode<Model3DNode> {
+ off(src: string, options: Model3DOptions = {}): AnimatableNode<Model3DNode> {
return Layer.off(src, options);
},
};
diff --git a/packages/framefields/src/substrate-materials-render.test.ts b/packages/framefields/src/substrate-materials-render.test.ts
new file mode 100644
index 00000000..bbb05e19
--- /dev/null
+++ b/packages/framefields/src/substrate-materials-render.test.ts
@@ -0,0 +1,387 @@
+import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { encodeHdr, parseExr } from "@framefields/webgpu-renderers";
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ CompositorProgramSchema,
+ Layer,
+ Layer3D,
+ Signal,
+} from "./index.js";
+import type { Surface3DOptions } from "./shapes3d.js";
+import { decodePngPixels } from "./test-utils/pixels.js";
+
+const W = 200;
+const H = 160;
+const CX = W / 2;
+const CY = H / 2;
+const R = 56;
+
+type SphereOptions = Surface3DOptions & {
+ color?: string;
+ roughness?: number;
+ metallic?: number;
+};
+
+interface SceneOptions {
+ sphere: SphereOptions;
+ /** Key light direction, from the light toward the sphere (y down). */
+ keyFrom?: [number, number, number];
+ environment?: Parameters<typeof Layer.environment>;
+}
+
+async function render(o: SceneOptions): Promise<Uint8ClampedArray> {
+ const comp = new Composition({
+ width: W,
+ height: H,
+ fps: 30,
+ durationFrames: 1,
+ backgroundColor: "#000000",
+ });
+ const [lx, ly, lz] = o.keyFrom ?? [-0.6, -1, -0.8];
+ comp.add([
+ Layer.camera({
+ x: CX,
+ y: CY,
+ z: -500,
+ targetX: CX,
+ targetY: CY,
+ targetZ: 0,
+ }),
+ Layer.environment(...(o.environment ?? ["night", { intensity: 0.15 }])),
+ Layer.directionalLight({
+ x: CX + lx * 1000,
+ y: CY + ly * 1000,
+ z: lz * 1000,
+ targetX: CX,
+ targetY: CY,
+ targetZ: 0,
+ intensity: 2.2,
+ color: "#ffffff",
+ }),
+ Layer3D.sphere({
+ x: CX,
+ y: CY,
+ z: 0,
+ radius: R,
+ segments: 64,
+ ...o.sphere,
+ }),
+ ]);
+ return (await decodePngPixels(await comp.renderFrame({ frame: 0 }))).pixels;
+}
+
+function sumDiff(a: Uint8ClampedArray, b: Uint8ClampedArray): number {
+ let total = 0;
+ for (let i = 0; i < a.length; i++)
+ total += Math.abs((a[i] ?? 0) - (b[i] ?? 0));
+ return total;
+}
+
+interface Disc {
+ cx: number;
+ cy: number;
+ r: number;
+}
+
+/** The sphere's on-screen disc: bounding box of every non-background pixel in `images`. */
+function silhouette(...images: Uint8ClampedArray[]): Disc {
+ let x0 = W;
+ let y0 = H;
+ let x1 = -1;
+ let y1 = -1;
+ for (let y = 0; y < H; y++) {
+ for (let x = 0; x < W; x++) {
+ const i = (y * W + x) * 4;
+ const lit = images.some(
+ (px) => (px[i] ?? 0) + (px[i + 1] ?? 0) + (px[i + 2] ?? 0) > 6,
+ );
+ if (!lit) continue;
+ x0 = Math.min(x0, x);
+ y0 = Math.min(y0, y);
+ x1 = Math.max(x1, x);
+ y1 = Math.max(y1, y);
+ }
+ }
+ expect(x1).toBeGreaterThan(x0);
+ return { cx: (x0 + x1 + 1) / 2, cy: (y0 + y1 + 1) / 2, r: (x1 - x0 + 1) / 2 };
+}
+
+/** Mean RGB over pixels whose distance from the disc center lies in [r0, r1) * radius. */
+function ringMean(
+ px: Uint8ClampedArray,
+ disc: Disc,
+ r0: number,
+ r1: number,
+ filter: (dx: number, dy: number) => boolean = () => true,
+): [number, number, number] {
+ const sum = [0, 0, 0];
+ let n = 0;
+ for (let y = 0; y < H; y++) {
+ for (let x = 0; x < W; x++) {
+ const dx = x + 0.5 - disc.cx;
+ const dy = y + 0.5 - disc.cy;
+ const d = Math.hypot(dx, dy) / disc.r;
+ if (d < r0 || d >= r1 || !filter(dx, dy)) continue;
+ const i = (y * W + x) * 4;
+ sum[0] += px[i] ?? 0;
+ sum[1] += px[i + 1] ?? 0;
+ sum[2] += px[i + 2] ?? 0;
+ n++;
+ }
+ }
+ expect(n).toBeGreaterThan(20);
+ return [sum[0] / n, sum[1] / n, sum[2] / n];
+}
+
+function maxLuma(px: Uint8ClampedArray): number {
+ let m = 0;
+ for (let i = 0; i < px.length; i += 4) {
+ m = Math.max(
+ m,
+ 0.2126 * (px[i] ?? 0) +
+ 0.7152 * (px[i + 1] ?? 0) +
+ 0.0722 * (px[i + 2] ?? 0),
+ );
+ }
+ return m;
+}
+
+const luma = ([r, g, b]: [number, number, number]) =>
+ 0.2126 * r + 0.7152 * g + 0.0722 * b;
+
+/** Equirectangular panorama: `sky` above the horizon, `ground` below. */
+function splitPanorama(
+ sky: [number, number, number],
+ ground: [number, number, number],
+) {
+ const w = 64;
+ const h = 32;
+ const px = new Float32Array(w * h * 4);
+ for (let y = 0; y < h; y++) {
+ const c = y < h / 2 ? sky : ground;
+ for (let x = 0; x < w; x++) px.set([...c, 1], (y * w + x) * 4);
+ }
+ return encodeHdr(w, h, px);
+}
+
+describe("Phase 2: Substrate multi-lobe materials", () => {
+ const roughRed: SphereOptions = {
+ color: "#8a1020",
+ roughness: 0.85,
+ metallic: 0,
+ };
+
+ it("clearcoat adds a sharp varnish highlight over a rough base", async () => {
+ const base = await render({ sphere: roughRed });
+ const coated = await render({
+ sphere: { ...roughRed, clearcoat: 1, clearcoatRoughness: 0.04 },
+ });
+ expect(maxLuma(coated)).toBeGreaterThan(maxLuma(base) + 40);
+ expect(sumDiff(base, coated)).toBeGreaterThan(2000);
+ });
+
+ it("anisotropy stretches the highlight along a rotatable brush direction", async () => {
+ const metal: SphereOptions = {
+ color: "#c8ccd4",
+ roughness: 0.35,
+ metallic: 1,
+ };
+ const iso = await render({ sphere: metal });
+ const brushed = await render({ sphere: { ...metal, anisotropy: 0.9 } });
+ const turned = await render({
+ sphere: { ...metal, anisotropy: 0.9, anisotropyRotation: 90 },
+ });
+ expect(sumDiff(iso, brushed)).toBeGreaterThan(15000);
+ expect(sumDiff(brushed, turned)).toBeGreaterThan(15000);
+ });
+
+ it("subsurface bleeds tinted light past the terminator of a back-lit sphere", async () => {
+ const wax: SphereOptions = { color: "#e8e0d0", roughness: 0.6 };
+ const keyFrom: [number, number, number] = [0.9, -0.3, 1];
+ const opaque = await render({ sphere: wax, keyFrom });
+ const sss = await render({
+ sphere: { ...wax, subsurface: 1, subsurfaceColor: "#ff5030" },
+ keyFrom,
+ });
+ // The side facing the camera, away from the light.
+ const shadowSide = (dx: number) => dx < 0;
+ const disc = silhouette(opaque, sss);
+ const before = ringMean(opaque, disc, 0, 0.95, shadowSide);
+ const after = ringMean(sss, disc, 0, 0.95, shadowSide);
+ expect(luma(after)).toBeGreaterThan(luma(before) + 4);
+ // The bleed carries the subsurface tint: red rises more than blue.
+ expect(after[0] - before[0]).toBeGreaterThan(after[2] - before[2]);
+ });
+
+ it("sheen lifts grazing angles more than the center (velvet rim)", async () => {
+ const velvet: SphereOptions = { color: "#2a0a3a", roughness: 0.9 };
+ const plain = await render({ sphere: velvet });
+ const sheen = await render({
+ sphere: {
+ ...velvet,
+ sheen: 1,
+ sheenColor: "#ffd0ff",
+ sheenRoughness: 0.4,
+ },
+ });
+ const disc = silhouette(plain, sheen);
+ const rimGain =
+ luma(ringMean(sheen, disc, 0.75, 0.97)) -
+ luma(ringMean(plain, disc, 0.75, 0.97));
+ const coreGain =
+ luma(ringMean(sheen, disc, 0, 0.4)) - luma(ringMean(plain, disc, 0, 0.4));
+ expect(rimGain).toBeGreaterThan(3);
+ expect(rimGain).toBeGreaterThan(coreGain);
+ });
+
+ it("iridescence shifts reflected color with film thickness", async () => {
+ const pearl: SphereOptions = {
+ color: "#202020",
+ roughness: 0.15,
+ metallic: 0.2,
+ };
+ const env: SceneOptions["environment"] = ["studio", { intensity: 1 }];
+ const plain = await render({ sphere: pearl, environment: env });
+ const thin = await render({
+ sphere: { ...pearl, iridescence: { intensity: 1, thickness: 300 } },
+ environment: env,
+ });
+ const thick = await render({
+ sphere: { ...pearl, iridescence: 1, iridescenceThickness: 700 },
+ environment: env,
+ });
+ expect(sumDiff(plain, thin)).toBeGreaterThan(15000);
+ const disc = silhouette(thin, thick);
+ const a = ringMean(thin, disc, 0, 0.95);
+ const b = ringMean(thick, disc, 0, 0.95);
+ // Different film thickness: different interference color, not just brightness.
+ const hueShift =
+ Math.abs(a[0] - a[2] - (b[0] - b[2])) +
+ Math.abs(a[1] - a[2] - (b[1] - b[2]));
+ expect(hueShift).toBeGreaterThan(3);
+ });
+
+ it("primitives forward every lobe to the node and the spec stays schema-valid", () => {
+ const clearcoat = Signal.fromArray([0, 1]);
+ const sphere = Layer3D.sphere({
+ radius: 40,
+ clearcoat,
+ clearcoatRoughness: 0.1,
+ anisotropy: 0.5,
+ anisotropyRotation: 30,
+ subsurface: 0.4,
+ subsurfaceColor: "#ff8060",
+ sheen: 0.3,
+ sheenColor: "#ffffff",
+ sheenRoughness: 0.6,
+ iridescence: { intensity: 0.8, thickness: 420, ior: 1.4 },
+ triplanar: { scale: 2, sharpness: 6 },
+ });
+ expect(sphere).toMatchObject({
+ clearcoat: 0,
+ clearcoatRoughness: 0.1,
+ anisotropy: 0.5,
+ anisotropyRotation: 30,
+ subsurface: 0.4,
+ subsurfaceColor: "#ff8060",
+ sheen: 0.3,
+ sheenColor: "#ffffff",
+ sheenRoughness: 0.6,
+ iridescence: { intensity: 0.8, thickness: 420, ior: 1.4 },
+ triplanar: { scale: 2, sharpness: 6 },
+ });
+ // The Signal became a track; the node keeps the static fallback.
+ expect(sphere.animation?.tracks.map((t) => t.prop)).toContain("clearcoat");
+
+ const comp = new Composition({
+ width: 64,
+ height: 64,
+ fps: 30,
+ durationFrames: 2,
+ });
+ comp.add([Layer.environment("studio"), sphere]);
+ expect(CompositorProgramSchema.safeParse(comp.toSpec()).success).toBe(true);
+ });
+});
+
+describe("Phase 2: HDRI environments", () => {
+ const chrome: SphereOptions = {
+ color: "#ffffff",
+ roughness: 0.05,
+ metallic: 1,
+ };
+ const top = (dx: number, dy: number) => dy < -Math.abs(dx) * 0.5;
+
+ it("reflects the panorama: swapping sky and ground swaps the sphere's colors", async () => {
+ const red: [number, number, number] = [2, 0.05, 0.05];
+ const blue: [number, number, number] = [0.05, 0.05, 2];
+ const redSky = await render({
+ sphere: chrome,
+ environment: [{ hdri: splitPanorama(red, blue), intensity: 1 }],
+ });
+ const blueSky = await render({
+ sphere: chrome,
+ environment: [{ hdri: splitPanorama(blue, red), intensity: 1 }],
+ });
+ const disc = silhouette(redSky, blueSky);
+ const a = ringMean(redSky, disc, 0.1, 0.9, top);
+ const b = ringMean(blueSky, disc, 0.1, 0.9, top);
+ expect(a[0] - a[2]).toBeGreaterThan(40);
+ expect(b[2] - b[0]).toBeGreaterThan(40);
+ });
+
+ it("takes a .hdr path in place of a preset name, intensity included", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "framefields-hdri-"));
+ const file = join(dir, "warm.hdr") as `${string}.hdr`;
+ writeFileSync(file, splitPanorama([1.6, 1.0, 0.4], [0.3, 0.2, 0.1]));
+
+ const node = Layer.environment(file, { intensity: 1.5 });
+ expect(node).toMatchObject({
+ lightType: "environment",
+ hdri: file,
+ intensity: 1.5,
+ });
+
+ const preset = await render({ sphere: chrome, environment: ["studio"] });
+ const fromPath = await render({ sphere: chrome, environment: [file] });
+ const brighter = await render({
+ sphere: chrome,
+ environment: [file, { intensity: 2 }],
+ });
+ expect(sumDiff(preset, fromPath)).toBeGreaterThan(20000);
+ const disc = silhouette(fromPath, brighter);
+ expect(luma(ringMean(brighter, disc, 0, 0.9))).toBeGreaterThan(
+ luma(ringMean(fromPath, disc, 0, 0.9)) + 5,
+ );
+ });
+
+ it("lights from an OpenEXR panorama exactly as from the same pixels in .hdr", async () => {
+ // A ZIP-compressed half-float EXR written by the reference OpenEXR library.
+ const exr = join(
+ import.meta.dirname,
+ "../../webgpu-renderers/src/loaders/__fixtures__/exr/zip-half.exr",
+ ) as `${string}.exr`;
+ const image = parseExr(new Uint8Array(readFileSync(exr)));
+ const hdr = join(
+ mkdtempSync(join(tmpdir(), "framefields-exr-")),
+ "same.hdr",
+ ) as `${string}.hdr`;
+ writeFileSync(hdr, encodeHdr(image.width, image.height, image.data));
+
+ const fromExr = await render({ sphere: chrome, environment: [exr] });
+ const fromHdr = await render({ sphere: chrome, environment: [hdr] });
+ const preset = await render({ sphere: chrome, environment: ["studio"] });
+ const disc = silhouette(fromExr, fromHdr);
+ const a = ringMean(fromExr, disc, 0, 0.9);
+ const b = ringMean(fromHdr, disc, 0, 0.9);
+ // RGBE keeps 8 mantissa bits: the two agree to within a level or two.
+ for (let c = 0; c < 3; c++)
+ expect(Math.abs((a[c] ?? 0) - (b[c] ?? 0))).toBeLessThan(2);
+ expect(sumDiff(fromExr, preset)).toBeGreaterThan(
+ 20 * sumDiff(fromExr, fromHdr),
+ );
+ });
+});
diff --git a/packages/framefields/src/volumetric-and-rect-light-render.test.ts b/packages/framefields/src/volumetric-and-rect-light-render.test.ts
new file mode 100644
index 00000000..c044db81
--- /dev/null
+++ b/packages/framefields/src/volumetric-and-rect-light-render.test.ts
@@ -0,0 +1,530 @@
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ CompositorProgramSchema,
+ HeadlessMediaRenderer,
+ Layer,
+ Layer3D,
+ LayerAnimation,
+ Signal,
+} from "./index.js";
+import { decodePngPixels } from "./test-utils/pixels.js";
+
+type Proto = Record<string, (...args: unknown[]) => unknown>;
+
+function sumDiff(a: Uint8ClampedArray, b: Uint8ClampedArray): number {
+ let total = 0;
+ for (let i = 0; i < a.length; i++) {
+ total += Math.abs(a[i] - (b[i] ?? 0));
+ }
+ return total;
+}
+
+function lumaAt(
+ pixels: Uint8ClampedArray,
+ width: number,
+ x: number,
+ y: number,
+): number {
+ const idx = (Math.round(y) * width + Math.round(x)) * 4;
+ return (
+ 0.2126 * (pixels[idx] ?? 0) +
+ 0.7152 * (pixels[idx + 1] ?? 0) +
+ 0.0722 * (pixels[idx + 2] ?? 0)
+ );
+}
+
+describe("Phase 1: Volumetric Fog, Light Shafts, Rect Area Lights & Omnidirectional Point Shadows", () => {
+ it("carves shadowed volumetric light shafts (god rays) behind an occluder", async () => {
+ const buildScene = (withVolumetricFog: boolean, withOccluder: boolean) => {
+ const comp = new Composition({
+ width: 320,
+ height: 240,
+ fps: 30,
+ durationFrames: 10,
+ backgroundColor: "#04060c",
+ });
+ comp.add([
+ Layer.camera({
+ x: 160,
+ y: 120,
+ z: -600,
+ targetX: 160,
+ targetY: 120,
+ targetZ: 0,
+ }),
+ Layer.environment("night", {
+ intensity: 0.2,
+ ...(withVolumetricFog && {
+ volumetricFog: {
+ density: 0.055,
+ color: "#8ab8ff",
+ anisotropy: 0.65,
+ lightIntensity: 3.0,
+ ambientIntensity: 0.2,
+ steps: 32,
+ maxDistance: 1800,
+ noise: { scale: 0.004, strength: 0.35, octaves: 2 },
+ },
+ }),
+ }),
+ Layer.spotLight({
+ x: 160,
+ y: -220,
+ z: 100,
+ targetX: 160,
+ targetY: 220,
+ targetZ: 0,
+ angle: 50,
+ penumbra: 0.25,
+ radius: 1800,
+ color: "#ffe4b5",
+ intensity: 4.5,
+ castShadow: true,
+ }),
+ ...(withOccluder
+ ? [
+ Layer3D.roundedBox({
+ x: 160,
+ y: 40,
+ z: 40,
+ width: 110,
+ height: 36,
+ depth: 110,
+ color: "#181c24",
+ roughness: 0.6,
+ castShadow: true,
+ }),
+ ]
+ : []),
+ ]);
+ return comp;
+ };
+
+ const noFog = (
+ await decodePngPixels(
+ await buildScene(false, true).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+ const fogWithOccluder = (
+ await decodePngPixels(
+ await buildScene(true, true).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+ const fogUnoccluded = (
+ await decodePngPixels(
+ await buildScene(true, false).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+
+ // Volumetric fog significantly brightens the participating medium compared to empty air
+ expect(sumDiff(noFog, fogWithOccluder)).toBeGreaterThan(25000);
+ // Shadow caster carves a dark volumetric shadow shaft in the fog below the box
+ expect(sumDiff(fogUnoccluded, fogWithOccluder)).toBeGreaterThan(8000);
+ });
+
+ it("renders Layer.rectLight area light illumination, representative-point specular, and shadows", async () => {
+ const buildRectScene = (castShadow: boolean, intensity: number) => {
+ const comp = new Composition({
+ width: 300,
+ height: 220,
+ fps: 30,
+ durationFrames: 10,
+ backgroundColor: "#050508",
+ });
+ comp.add([
+ Layer.camera({
+ x: 150,
+ y: -120,
+ z: -480,
+ targetX: 150,
+ targetY: 140,
+ targetZ: 0,
+ }),
+ Layer.environment("night", { intensity: 0.1 }),
+ Layer.rectLight({
+ x: 150,
+ y: -160,
+ z: -140,
+ targetX: 150,
+ targetY: 160,
+ targetZ: 0,
+ width: 320,
+ height: 110,
+ color: "#fff2dd",
+ intensity,
+ radius: 1600,
+ castShadow,
+ }),
+ Layer3D.roundedBox({
+ x: 150,
+ y: 170,
+ z: 0,
+ width: 260,
+ height: 18,
+ depth: 260,
+ radius: 0,
+ color: "#d8dce6",
+ roughness: 0.18,
+ metallic: 0.25,
+ }),
+ Layer3D.sphere({
+ x: 150,
+ y: 95,
+ z: -30,
+ radius: 42,
+ color: "#ffffff",
+ roughness: 0.2,
+ castShadow: true,
+ }),
+ ]);
+ return comp;
+ };
+
+ const dark = (
+ await decodePngPixels(
+ await buildRectScene(false, 0).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+ const unshadowed = (
+ await decodePngPixels(
+ await buildRectScene(false, 3.2).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+ const shadowed = (
+ await decodePngPixels(
+ await buildRectScene(true, 3.2).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+
+ expect(sumDiff(dark, unshadowed)).toBeGreaterThan(40000);
+ expect(sumDiff(unshadowed, shadowed)).toBeGreaterThan(3000);
+ });
+
+ it("renders omnidirectional 6-face point light shadows in opposite directions simultaneously", async () => {
+ const buildPointScene = (castShadow: boolean) => {
+ const comp = new Composition({
+ width: 360,
+ height: 220,
+ fps: 30,
+ durationFrames: 10,
+ backgroundColor: "#08080c",
+ });
+ comp.add([
+ Layer.camera({
+ x: 180,
+ y: 110,
+ z: -520,
+ targetX: 180,
+ targetY: 110,
+ targetZ: 0,
+ }),
+ Layer.ambientLight("#ffffff", 0.08),
+ // Central point light between left and right casters/walls
+ Layer.pointLight({
+ x: 180,
+ y: 110,
+ z: 0,
+ color: "#ffffff",
+ intensity: 2.2,
+ radius: 1200,
+ castShadow,
+ }),
+ // Left receiver wall (-X) and left shadow caster
+ Layer3D.roundedBox({
+ x: 30,
+ y: 110,
+ z: 0,
+ width: 16,
+ height: 180,
+ depth: 180,
+ radius: 0,
+ color: "#e0e0e0",
+ castShadow: false,
+ }),
+ Layer3D.sphere({
+ x: 105,
+ y: 110,
+ z: 0,
+ radius: 32,
+ color: "#ff5533",
+ castShadow: true,
+ }),
+ // Right receiver wall (+X) and right shadow caster
+ Layer3D.roundedBox({
+ x: 330,
+ y: 110,
+ z: 0,
+ width: 16,
+ height: 180,
+ depth: 180,
+ radius: 0,
+ color: "#e0e0e0",
+ castShadow: false,
+ }),
+ Layer3D.sphere({
+ x: 255,
+ y: 110,
+ z: 0,
+ radius: 32,
+ color: "#3388ff",
+ castShadow: true,
+ }),
+ ]);
+ return comp;
+ };
+
+ const unshadowed = (
+ await decodePngPixels(
+ await buildPointScene(false).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+ const shadowed = (
+ await decodePngPixels(
+ await buildPointScene(true).renderFrame({ frame: 0 }),
+ )
+ ).pixels;
+
+ expect(sumDiff(unshadowed, shadowed)).toBeGreaterThan(2500);
+ });
+
+ it("integrates live Signals directly in volumetricFog and Layer.rectLight while keeping comp.toSpec() schema-valid", async () => {
+ const fogDensitySignal = Signal.fromArray([0.0, 0.09, 0.0]);
+ const lightIntensitySignal = Signal.fromArray([0.0, 4.0, 0.0]);
+ const rectWidthSignal = Signal.fromArray([80, 360, 80]);
+
+ const comp = new Composition({
+ width: 240,
+ height: 180,
+ fps: 30,
+ durationFrames: 3,
+ backgroundColor: "#04050a",
+ });
+
+ comp.add([
+ Layer.camera({
+ x: 120,
+ y: 90,
+ z: -420,
+ targetX: 120,
+ targetY: 90,
+ targetZ: 0,
+ }),
+ Layer.environment("night", {
+ intensity: 0.1,
+ volumetricFog: {
+ density: fogDensitySignal,
+ color: "#9bc4ff",
+ anisotropy: 0.6,
+ lightIntensity: 2.5,
+ },
+ }),
+ Layer.rectLight({
+ x: 120,
+ y: 20,
+ z: -100,
+ targetX: 120,
+ targetY: 120,
+ targetZ: 0,
+ width: rectWidthSignal,
+ height: 100,
+ intensity: lightIntensitySignal,
+ color: "#ffffff",
+ }),
+ Layer3D.roundedBox({
+ x: 120,
+ y: 135,
+ z: 0,
+ width: 180,
+ height: 20,
+ depth: 180,
+ color: "#cccccc",
+ }),
+ ]);
+
+ // Verify comp.toSpec() is pure JSON and passes strict Zod validation
+ const spec = comp.toSpec();
+ expect(() => CompositorProgramSchema.parse(spec)).not.toThrow();
+
+ const renderer = new HeadlessMediaRenderer();
+ const f0 = await decodePngPixels(
+ await comp.renderFrame({ frame: 0, renderer }),
+ );
+ const f1 = await decodePngPixels(
+ await comp.renderFrame({ frame: 1, renderer }),
+ );
+
+ // Frame 1 has high signal-driven fog density and rect light intensity
+ const centerLumaF0 = lumaAt(f0.pixels, 240, 120, 80);
+ const centerLumaF1 = lumaAt(f1.pixels, 240, 120, 80);
+ expect(centerLumaF1).toBeGreaterThan(centerLumaF0 + 15);
+ });
+
+ it("allocates zero new GPU textures and recycles buffers across 20 frames with volumetric fog, rect light & point shadows", async () => {
+ const densitySig = Signal.fromArray(
+ Array.from({ length: 30 }, (_, i) => 0.02 + 0.02 * Math.sin(i * 0.3)),
+ );
+ const comp = new Composition({
+ width: 200,
+ height: 160,
+ fps: 30,
+ durationFrames: 30,
+ backgroundColor: "#05060a",
+ });
+ comp.add([
+ Layer.camera({
+ x: 100,
+ y: 80,
+ z: -380,
+ targetX: 100,
+ targetY: 80,
+ targetZ: 0,
+ }),
+ Layer.environment("night", {
+ volumetricFog: {
+ density: densitySig,
+ anisotropy: 0.55,
+ steps: 20,
+ noise: { strength: 0.4 },
+ },
+ }),
+ Layer.rectLight({
+ x: 100,
+ y: -120,
+ z: -100,
+ targetX: 100,
+ targetY: 80,
+ targetZ: 0,
+ width: 200,
+ height: 90,
+ intensity: 2.0,
+ castShadow: true,
+ }),
+ Layer.pointLight({
+ x: 60,
+ y: 40,
+ z: -60,
+ intensity: 1.5,
+ radius: 800,
+ castShadow: true,
+ }),
+ Layer3D.sphere({
+ x: 100,
+ y: 80,
+ z: 0,
+ radius: 36,
+ color: "#e8ecf4",
+ castShadow: true,
+ }),
+ ]);
+
+ const renderer = new HeadlessMediaRenderer();
+ // Warm up pools on first 3 frames
+ for (let f = 0; f < 3; f++) {
+ await comp.renderFrame({ frame: f, renderer });
+ }
+
+ const proto = (globalThis as unknown as { GPUDevice: { prototype: Proto } })
+ .GPUDevice.prototype;
+ const origCreateBuffer = proto.createBuffer;
+ const origCreateTexture = proto.createTexture;
+ let buffersCreated = 0;
+ let texturesCreated = 0;
+ proto.createBuffer = function (...args: unknown[]) {
+ buffersCreated++;
+ return origCreateBuffer.apply(this, args);
+ };
+ proto.createTexture = function (...args: unknown[]) {
+ texturesCreated++;
+ return origCreateTexture.apply(this, args);
+ };
+ try {
+ for (let f = 3; f < 23; f++) {
+ await comp.renderFrame({ frame: f, renderer });
+ }
+ } finally {
+ proto.createBuffer = origCreateBuffer;
+ proto.createTexture = origCreateTexture;
+ }
+
+ expect(texturesCreated).toBe(0);
+ expect(buffersCreated).toBe(0);
+ });
+
+ it("supports ergonomic 3D property aliases (topRadius, majorRadius, cornerRadius, rotationX/Y/Z, camera fov, renderFrame format)", async () => {
+ const comp = new Composition({
+ width: 240,
+ height: 160,
+ fps: 30,
+ durationFrames: 30,
+ backgroundColor: "#060810",
+ });
+
+ const cam = Layer.camera({
+ x: 120,
+ y: 80,
+ z: -400,
+ targetX: 120,
+ targetY: 80,
+ targetZ: 0,
+ fov: 42,
+ near: 10,
+ far: 4000,
+ });
+ expect(cam.lens?.fov).toBe(42);
+ expect(cam.lens?.near).toBe(10);
+ expect(cam.lens?.far).toBe(4000);
+
+ const box = Layer3D.roundedBox({
+ x: 60,
+ y: 80,
+ z: 0,
+ width: 50,
+ height: 50,
+ depth: 50,
+ cornerRadius: 8,
+ rotationZ: 15,
+ color: "#4a78c4",
+ });
+ expect(box.rotateZ).toBe(15);
+
+ const cyl = Layer3D.cylinder({
+ x: 120,
+ y: 80,
+ z: 0,
+ topRadius: 16,
+ bottomRadius: 24,
+ height: 60,
+ rotationX: 10,
+ color: "#88a0c8",
+ });
+ expect(cyl.rotateX).toBe(10);
+
+ const torus = Layer3D.torus({
+ x: 180,
+ y: 80,
+ z: 0,
+ majorRadius: 28,
+ minorRadius: 8,
+ rotationY: 25,
+ color: "#f0c868",
+ }).animate(
+ LayerAnimation.keys("rotationY", [
+ [0, 0],
+ [29, 180],
+ ]),
+ );
+ expect(torus.rotateY).toBe(25);
+ expect(torus.animation?.tracks[0]?.prop).toBe("rotateY");
+
+ comp.add([cam, Layer.environment("studio"), box, cyl, torus]);
+
+ // Ensure strict schema validation passes with zero unknown properties
+ expect(() => CompositorProgramSchema.parse(comp.toSpec())).not.toThrow();
+
+ const pngBuf = await comp.renderFrame({ frame: 10, format: "png" });
+ expect(pngBuf[0]).toBe(0x89); // PNG magic byte
+ const rgbaBuf = await comp.renderFrame({ frame: 10, format: "rgba" });
+ expect(rgbaBuf.byteLength).toBe(240 * 160 * 4);
+ });
+});
diff --git a/packages/renderer/src/headless-webgpu-renderer.ts b/packages/renderer/src/headless-webgpu-renderer.ts
index ad7e738e..3b5b0f91 100644
--- a/packages/renderer/src/headless-webgpu-renderer.ts
+++ b/packages/renderer/src/headless-webgpu-renderer.ts
@@ -592,11 +592,11 @@ class NodeSurfacePool {
}
release(device: GPUDevice, surface: NodeSurfaceProvider): void {
- if (this.idle.length >= this.maxIdle) {
+ while (this.idle.length >= this.maxIdle) {
+ const evicted = this.idle.shift();
try {
- surface.destroy();
+ evicted?.surface.destroy();
} catch {}
- return;
}
this.idle.push({ device, surface });
}
@@ -704,8 +704,8 @@ export class HeadlessWebGPURenderer {
fps = 24,
options: {
renderId?: string;
- /** "rgba" skips PNG encoding and returns width*height*4 raw pixels. */
- format?: "png" | "rgba";
+ /** "rgba" / "raw" skips image encoding and returns width*height*4 raw pixels. */
+ format?: "png" | "rgba" | "raw" | "jpeg" | "jpg" | "webp";
} = {},
): Promise<Buffer> {
await HeadlessWebGPURenderer.initialize();
@@ -834,15 +834,19 @@ export class HeadlessWebGPURenderer {
pixels.byteLength,
);
- if (options.format === "rgba") return Buffer.from(pixelsArr);
+ if (options.format === "rgba" || options.format === "raw")
+ return Buffer.from(pixelsArr);
- const pngBuffer = await sharp(Buffer.from(pixelsArr), {
+ const img = sharp(Buffer.from(pixelsArr), {
raw: { width, height, channels: 4 },
- })
- .png()
- .toBuffer();
-
- return pngBuffer;
+ });
+ if (options.format === "jpeg" || options.format === "jpg") {
+ return img.jpeg().toBuffer();
+ }
+ if (options.format === "webp") {
+ return img.webp().toBuffer();
+ }
+ return img.png().toBuffer();
} finally {
try {
clearAllVideoCache();
@@ -884,8 +888,8 @@ export class HeadlessWebGPURenderer {
fps?: number;
/** Reuse compiled state across calls; see `renderImage`. */
renderId?: string;
- /** "rgba" returns raw width*height*4 pixels instead of a PNG. */
- format?: "png" | "rgba";
+ /** "rgba" / "raw" returns raw width*height*4 pixels instead of an encoded image. */
+ format?: "png" | "rgba" | "raw" | "jpeg" | "jpg" | "webp";
} = {},
): Promise<Buffer> {
const vm =
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/README.md b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/README.md
new file mode 100644
index 00000000..eb65f4bb
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/README.md
@@ -0,0 +1,17 @@
+# OpenEXR decoder fixtures
+
+Encoded by independent tools so `exr-loader.test.ts` checks the decoder against
+the reference implementation, not against an encoder of our own.
+
+1. A 23×37 RGB float image whose pixels are `fixtureValue(x, y, c)` in
+ `exr-loader.test.ts` was written as raw `gbrpf32le` and encoded with
+ `ffmpeg -c:v exr -compression none -format float` (`src.exr`).
+2. Each `<compression>-<half|float>.exr` is
+ `exrmetrics --convert -z <compression> --pixelmode <half|float> src.exr -o …`
+ (OpenEXR 3.x reference library).
+3. `tiled-zip-half.exr` is `exrmaketiled -o -z zip -t 16 16 zip-half.exr`;
+ `tiled-mip-piz-float.exr` is `exrmaketiled -m -z piz -t 16 8 piz-float.exr`.
+4. `b44-half.exr` (lossy, unsupported) checks the error message.
+
+37 rows span several ZIP (16-line) and PIZ (32-line) blocks with a partial last
+block; the first five columns are constant runs for RLE.
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/b44-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/b44-half.exr
new file mode 100644
index 00000000..05dce583
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/b44-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/none-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/none-float.exr
new file mode 100644
index 00000000..9d4c55e3
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/none-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/none-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/none-half.exr
new file mode 100644
index 00000000..c01912d9
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/none-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/piz-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/piz-float.exr
new file mode 100644
index 00000000..f57a1411
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/piz-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/piz-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/piz-half.exr
new file mode 100644
index 00000000..ac668306
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/piz-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/pxr24-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/pxr24-float.exr
new file mode 100644
index 00000000..0eb0236e
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/pxr24-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/pxr24-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/pxr24-half.exr
new file mode 100644
index 00000000..cca90586
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/pxr24-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/rle-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/rle-float.exr
new file mode 100644
index 00000000..c9d37361
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/rle-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/rle-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/rle-half.exr
new file mode 100644
index 00000000..e802d731
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/rle-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/tiled-mip-piz-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/tiled-mip-piz-float.exr
new file mode 100644
index 00000000..755f7727
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/tiled-mip-piz-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/tiled-zip-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/tiled-zip-half.exr
new file mode 100644
index 00000000..d3da07f3
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/tiled-zip-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zip-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zip-float.exr
new file mode 100644
index 00000000..83179530
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zip-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zip-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zip-half.exr
new file mode 100644
index 00000000..f9b724e9
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zip-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zips-float.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zips-float.exr
new file mode 100644
index 00000000..c43502ba
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zips-float.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zips-half.exr b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zips-half.exr
new file mode 100644
index 00000000..b7f8355f
Binary files /dev/null and b/packages/webgpu-renderers/src/loaders/__fixtures__/exr/zips-half.exr differ
diff --git a/packages/webgpu-renderers/src/loaders/exr-loader.test.ts b/packages/webgpu-renderers/src/loaders/exr-loader.test.ts
new file mode 100644
index 00000000..141032b5
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/exr-loader.test.ts
@@ -0,0 +1,86 @@
+import { readFileSync } from "node:fs";
+import { join } from "node:path";
+import { describe, expect, it } from "vitest";
+import { isExr, parseExr } from "./exr-loader.js";
+import { decodeHdrSource, encodeHdr } from "./hdr-loader.js";
+
+const FIXTURES = join(import.meta.dirname, "__fixtures__", "exr");
+const W = 23;
+const H = 37;
+
+/** The source image every fixture encodes (see __fixtures__/exr/README.md). */
+function fixtureValue(x: number, y: number, c: number): number {
+ if (x < 5) return [1, 0.5, 0.25][c] ?? 0;
+ const base = 0.01 + ((x * 7 + y * 13 + c * 5) % 29) / 29;
+ return base * 2 ** ((y % 9) - 4);
+}
+
+const fixture = (name: string) =>
+ new Uint8Array(readFileSync(join(FIXTURES, `${name}.exr`)));
+
+function expectImage(name: string, relTolerance: number): void {
+ const img = parseExr(fixture(name));
+ expect(img.width).toBe(W);
+ expect(img.height).toBe(H);
+ let worst = 0;
+ for (let y = 0; y < H; y++) {
+ for (let x = 0; x < W; x++) {
+ for (let c = 0; c < 3; c++) {
+ const want = Math.fround(fixtureValue(x, y, c));
+ const got = img.data[(y * W + x) * 4 + c] ?? Number.NaN;
+ worst = Math.max(worst, Math.abs(got - want) / want);
+ }
+ expect(img.data[(y * W + x) * 4 + 3]).toBe(1);
+ }
+ }
+ expect(worst).toBeLessThanOrEqual(relTolerance);
+}
+
+describe("OpenEXR decoding against reference-encoded files", () => {
+ const HALF = 2 ** -11; // half keeps 11 significant bits
+ for (const codec of ["none", "rle", "zips", "zip", "piz"]) {
+ it(`${codec}: float is exact, half is within half precision`, () => {
+ expectImage(`${codec}-float`, 0);
+ expectImage(`${codec}-half`, HALF);
+ });
+ }
+
+ it("pxr24: half is lossless, float keeps 24 bits", () => {
+ expectImage("pxr24-half", HALF);
+ expectImage("pxr24-float", 2 ** -15);
+ expect(parseExr(fixture("pxr24-half")).data).toEqual(
+ parseExr(fixture("none-half")).data,
+ );
+ });
+
+ it("tiled images decode to the same pixels as their scanline sources", () => {
+ expect(parseExr(fixture("tiled-zip-half")).data).toEqual(
+ parseExr(fixture("zip-half")).data,
+ );
+ // Mipmapped: level 0 only.
+ expect(parseExr(fixture("tiled-mip-piz-float")).data).toEqual(
+ parseExr(fixture("piz-float")).data,
+ );
+ });
+
+ it("names the codec when it is lossy and unsupported", () => {
+ expect(() => parseExr(fixture("b44-half"))).toThrow(
+ /B44 compression is not supported/,
+ );
+ });
+
+ it("rejects truncated files instead of returning partial pixels", () => {
+ const bytes = fixture("piz-half");
+ expect(() => parseExr(bytes.subarray(0, bytes.length - 200))).toThrow();
+ });
+
+ it("is told apart from Radiance .hdr by its magic bytes", () => {
+ expect(isExr(fixture("zip-half"))).toBe(true);
+ const hdr = encodeHdr(2, 1, new Float32Array([1, 2, 3, 1, 4, 5, 6, 1]));
+ expect(isExr(hdr)).toBe(false);
+ expect(decodeHdrSource(join(FIXTURES, "zip-float.exr")).data).toEqual(
+ parseExr(fixture("zip-float")).data,
+ );
+ expect(decodeHdrSource(hdr).width).toBe(2);
+ });
+});
diff --git a/packages/webgpu-renderers/src/loaders/exr-loader.ts b/packages/webgpu-renderers/src/loaders/exr-loader.ts
new file mode 100644
index 00000000..80ad7cef
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/exr-loader.ts
@@ -0,0 +1,704 @@
+/**
+ * OpenEXR decoder for HDRI panoramas, in pure TypeScript.
+ *
+ * Reads single-part scanline and tiled images (level 0 of mipmapped and
+ * ripmapped files) with NONE, RLE, ZIPS, ZIP, PIZ and PXR24 compression and
+ * HALF / FLOAT / UINT channels. The lossy B44 / B44A / DWAA / DWAB / HTJ2K
+ * codecs, deep data and multi-part files are rejected with a clear error.
+ * Block formats follow the OpenEXR reference implementation (ImfZip,
+ * ImfRle, ImfPizCompressor + ImfHuf + ImfWav, ImfPxr24Compressor).
+ */
+import { inflateSync } from "node:zlib";
+import type { DecodedHdrImage } from "./hdr-loader.js";
+
+const MAGIC = 20000630;
+const FLAG_TILED = 0x200;
+const FLAG_DEEP = 0x800;
+const FLAG_MULTIPART = 0x1000;
+
+const PIXEL_UINT = 0;
+const PIXEL_HALF = 1;
+const PIXEL_FLOAT = 2;
+
+const COMPRESSION_NAMES = [
+ "NONE",
+ "RLE",
+ "ZIPS",
+ "ZIP",
+ "PIZ",
+ "PXR24",
+ "B44",
+ "B44A",
+ "DWAA",
+ "DWAB",
+ "HTJ2K256",
+ "HTJ2K32",
+];
+/** Scan lines per chunk, per supported compression. */
+const LINES_PER_BLOCK: Record<number, number> = {
+ 0: 1,
+ 1: 1,
+ 2: 1,
+ 3: 16,
+ 4: 32,
+ 5: 16,
+};
+
+interface ExrChannel {
+ name: string;
+ type: number;
+ /** Bytes per sample in the uncompressed layout. */
+ size: number;
+}
+
+interface ExrHeader {
+ channels: ExrChannel[];
+ compression: number;
+ xMin: number;
+ yMin: number;
+ width: number;
+ height: number;
+ tiled: boolean;
+ tileWidth: number;
+ tileHeight: number;
+ /** Byte offset of the chunk offset table. */
+ tableOffset: number;
+}
+
+/** True when `bytes` start with the OpenEXR magic number. */
+export function isExr(bytes: Uint8Array): boolean {
+ return (
+ bytes.length >= 4 &&
+ bytes[0] === 0x76 &&
+ bytes[1] === 0x2f &&
+ bytes[2] === 0x31 &&
+ bytes[3] === 0x01
+ );
+}
+
+class Reader {
+ readonly view: DataView;
+ offset: number;
+ constructor(
+ readonly bytes: Uint8Array,
+ offset = 0,
+ ) {
+ this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
+ this.offset = offset;
+ }
+ u8(): number {
+ return this.view.getUint8(this.offset++);
+ }
+ u16(): number {
+ const v = this.view.getUint16(this.offset, true);
+ this.offset += 2;
+ return v;
+ }
+ i32(): number {
+ const v = this.view.getInt32(this.offset, true);
+ this.offset += 4;
+ return v;
+ }
+ u32(): number {
+ const v = this.view.getUint32(this.offset, true);
+ this.offset += 4;
+ return v;
+ }
+ u64(): number {
+ const v = this.view.getBigUint64(this.offset, true);
+ this.offset += 8;
+ return Number(v);
+ }
+ cstring(): string {
+ const start = this.offset;
+ while (this.offset < this.bytes.length && this.bytes[this.offset] !== 0)
+ this.offset++;
+ const s = String.fromCharCode(...this.bytes.subarray(start, this.offset));
+ this.offset++;
+ return s;
+ }
+}
+
+function sampleSize(type: number): number {
+ if (type === PIXEL_HALF) return 2;
+ if (type === PIXEL_UINT || type === PIXEL_FLOAT) return 4;
+ throw new Error(`OpenEXR: unknown pixel type ${type}`);
+}
+
+function readChannels(r: Reader, end: number): ExrChannel[] {
+ const channels: ExrChannel[] = [];
+ while (r.offset < end) {
+ const name = r.cstring();
+ if (name === "") break;
+ const type = r.i32();
+ r.offset += 4; // pLinear + reserved
+ const xSampling = r.i32();
+ const ySampling = r.i32();
+ if (xSampling !== 1 || ySampling !== 1)
+ throw new Error(
+ `OpenEXR: subsampled channel '${name}' (${xSampling}×${ySampling}) is not supported`,
+ );
+ channels.push({ name, type, size: sampleSize(type) });
+ }
+ return channels;
+}
+
+function readHeader(bytes: Uint8Array): ExrHeader {
+ const r = new Reader(bytes);
+ if (r.i32() !== MAGIC) throw new Error("Not an OpenEXR file");
+ const version = r.i32();
+ if (version & FLAG_MULTIPART)
+ throw new Error("OpenEXR: multi-part files are not supported");
+ if (version & FLAG_DEEP)
+ throw new Error("OpenEXR: deep images are not supported");
+ const header: ExrHeader = {
+ channels: [],
+ compression: -1,
+ xMin: 0,
+ yMin: 0,
+ width: 0,
+ height: 0,
+ tiled: (version & FLAG_TILED) !== 0,
+ tileWidth: 0,
+ tileHeight: 0,
+ tableOffset: 0,
+ };
+ for (;;) {
+ const name = r.cstring();
+ if (name === "") break;
+ r.cstring(); // attribute type
+ const size = r.i32();
+ const end = r.offset + size;
+ readAttribute(r, name, end, header);
+ r.offset = end;
+ }
+ header.tableOffset = r.offset;
+ return header;
+}
+
+function readAttribute(
+ r: Reader,
+ name: string,
+ end: number,
+ header: ExrHeader,
+): void {
+ if (name === "channels") {
+ header.channels = readChannels(r, end);
+ } else if (name === "compression") {
+ header.compression = r.u8();
+ } else if (name === "dataWindow") {
+ header.xMin = r.i32();
+ header.yMin = r.i32();
+ header.width = r.i32() - header.xMin + 1;
+ header.height = r.i32() - header.yMin + 1;
+ } else if (name === "tiles") {
+ header.tileWidth = r.u32();
+ header.tileHeight = r.u32();
+ }
+}
+
+// ── ZIP / RLE: byte predictor and half-split interleave ──────────────
+
+function unpredictAndInterleave(tmp: Uint8Array): Uint8Array {
+ for (let i = 1; i < tmp.length; i++) {
+ tmp[i] = ((tmp[i - 1] ?? 0) + (tmp[i] ?? 0) - 128) & 0xff;
+ }
+ const out = new Uint8Array(tmp.length);
+ const half = (tmp.length + 1) >> 1;
+ for (let i = 0, t1 = 0, t2 = half; i < out.length; ) {
+ out[i++] = tmp[t1++] ?? 0;
+ if (i < out.length) out[i++] = tmp[t2++] ?? 0;
+ }
+ return out;
+}
+
+function rleDecode(src: Uint8Array, size: number): Uint8Array {
+ const out = new Uint8Array(size);
+ let o = 0;
+ let i = 0;
+ while (i < src.length && o < size) {
+ const code = ((src[i++] ?? 0) << 24) >> 24;
+ if (code < 0) {
+ const n = -code;
+ out.set(src.subarray(i, i + n), o);
+ i += n;
+ o += n;
+ } else {
+ out.fill(src[i++] ?? 0, o, o + code + 1);
+ o += code + 1;
+ }
+ }
+ if (o !== size) throw new Error("OpenEXR: corrupt RLE block");
+ return out;
+}
+
+// ── PXR24: per-channel byte planes of differences ────────────────────
+
+function pxr24Decode(
+ src: Uint8Array,
+ width: number,
+ lines: number,
+ channels: ExrChannel[],
+ size: number,
+): Uint8Array {
+ const tmp = inflateSync(src);
+ const out = new Uint8Array(size);
+ const view = new DataView(out.buffer);
+ let t = 0;
+ let o = 0;
+ for (let y = 0; y < lines; y++) {
+ for (const ch of channels) {
+ const planes =
+ ch.type === PIXEL_HALF ? 2 : ch.type === PIXEL_FLOAT ? 3 : 4;
+ let pixel = 0;
+ for (let x = 0; x < width; x++) {
+ let diff = 0;
+ for (let p = 0; p < planes; p++) {
+ diff = diff * 256 + (tmp[t + p * width + x] ?? 0);
+ }
+ if (ch.type === PIXEL_FLOAT) diff *= 256;
+ pixel = (pixel + diff) % 4294967296;
+ if (ch.type === PIXEL_HALF) {
+ view.setUint16(o, pixel & 0xffff, true);
+ o += 2;
+ } else {
+ view.setUint32(o, pixel, true);
+ o += 4;
+ }
+ }
+ t += planes * width;
+ }
+ }
+ return out;
+}
+
+// ── PIZ: Huffman + LUT + Haar wavelet ────────────────────────────────
+
+const HUF_ENCSIZE = 65537;
+const HUF_DECBITS = 14;
+
+class BitReader {
+ pos = 0;
+ constructor(
+ readonly bytes: Uint8Array,
+ readonly start: number,
+ ) {}
+ /** Next `n` (≤ 17) bits, MSB first, without consuming them; zero past the end. */
+ peek(n: number): number {
+ const b = this.start + (this.pos >>> 3);
+ const v =
+ ((this.bytes[b] ?? 0) << 16) |
+ ((this.bytes[b + 1] ?? 0) << 8) |
+ (this.bytes[b + 2] ?? 0);
+ return (v >>> (24 - (this.pos & 7) - n)) & ((1 << n) - 1);
+ }
+ read(n: number): number {
+ const v = this.peek(n);
+ this.pos += n;
+ return v;
+ }
+}
+
+/** Code lengths for symbols im..iM, packed 6 bits each with zero runs (hufUnpackEncTable). */
+function unpackCodeLengths(
+ bits: BitReader,
+ im: number,
+ iM: number,
+): Uint8Array {
+ const lengths = new Uint8Array(HUF_ENCSIZE);
+ for (let i = im; i <= iM; i++) {
+ const l = bits.read(6);
+ if (l === 63) {
+ const run = bits.read(8) + 6;
+ if (i + run > iM + 1) throw new Error("OpenEXR: corrupt Huffman table");
+ i += run - 1;
+ } else if (l >= 59) {
+ const run = l - 59 + 2;
+ if (i + run > iM + 1) throw new Error("OpenEXR: corrupt Huffman table");
+ i += run - 1;
+ } else {
+ lengths[i] = l;
+ }
+ }
+ return lengths;
+}
+
+interface HuffmanTable {
+ /** Short codes: `symbol << 8 | length` for every HUF_DECBITS-bit prefix, 0 if longer. */
+ short: Int32Array;
+ /** Canonical decoding for long codes: first code and symbols per length. */
+ firstCode: number[];
+ symbols: number[][];
+}
+
+/** Canonical codes as OpenEXR assigns them (hufCanonicalCodeTable). */
+function buildHuffmanTable(lengths: Uint8Array): HuffmanTable {
+ const count = new Array<number>(59).fill(0);
+ for (const l of lengths) count[l] = (count[l] ?? 0) + 1;
+ const firstCode = new Array<number>(59).fill(0);
+ let c = 0;
+ for (let l = 58; l > 0; l--) {
+ firstCode[l] = c;
+ c = Math.floor((c + (count[l] ?? 0)) / 2);
+ }
+ const next = firstCode.slice();
+ const symbols: number[][] = Array.from({ length: 59 }, () => []);
+ const short = new Int32Array(1 << HUF_DECBITS);
+ for (let s = 0; s < lengths.length; s++) {
+ const l = lengths[s] ?? 0;
+ if (l === 0) continue;
+ const code = next[l] ?? 0;
+ next[l] = code + 1;
+ symbols[l]?.push(s);
+ if (l <= HUF_DECBITS) {
+ const shift = HUF_DECBITS - l;
+ const startIdx = code * 2 ** shift;
+ short.fill((s << 8) | l, startIdx, startIdx + 2 ** shift);
+ }
+ }
+ return { short, firstCode, symbols };
+}
+
+function decodeSymbol(
+ bits: BitReader,
+ table: HuffmanTable,
+ nBits: number,
+): number {
+ const entry = table.short[bits.peek(HUF_DECBITS)] ?? 0;
+ const len = entry & 0xff;
+ if (len > 0 && bits.pos + len <= nBits) {
+ bits.pos += len;
+ return entry >>> 8;
+ }
+ // Longer than the table: extend bit by bit until a code of that length matches.
+ let code = 0;
+ for (let l = 1; l <= 58 && bits.pos < nBits; l++) {
+ code = code * 2 + bits.read(1);
+ const offset = code - (table.firstCode[l] ?? 0);
+ const syms = table.symbols[l] ?? [];
+ if (offset >= 0 && offset < syms.length) return syms[offset] ?? 0;
+ }
+ throw new Error("OpenEXR: corrupt Huffman data");
+}
+
+function hufUncompress(src: Uint8Array, start: number, out: Uint16Array): void {
+ const r = new Reader(src, start);
+ const im = r.u32();
+ const iM = r.u32();
+ r.offset += 4; // table length
+ const nBits = r.u32();
+ r.offset += 4;
+ if (im >= HUF_ENCSIZE || iM >= HUF_ENCSIZE)
+ throw new Error("OpenEXR: corrupt Huffman header");
+ const tableBits = new BitReader(src, r.offset);
+ const table = buildHuffmanTable(unpackCodeLengths(tableBits, im, iM));
+ const bits = new BitReader(src, r.offset + Math.ceil(tableBits.pos / 8));
+ let o = 0;
+ while (bits.pos < nBits && o < out.length) {
+ const sym = decodeSymbol(bits, table, nBits);
+ if (sym === iM) {
+ // Run-length code: repeat the previous symbol.
+ const run = bits.read(8);
+ if (o === 0 || o + run > out.length)
+ throw new Error("OpenEXR: corrupt Huffman run");
+ out.fill(out[o - 1] ?? 0, o, o + run);
+ o += run;
+ } else {
+ out[o++] = sym;
+ }
+ }
+ if (o !== out.length) throw new Error("OpenEXR: truncated Huffman data");
+}
+
+function wdec14(l: number, h: number): [number, number] {
+ const ls = (l << 16) >> 16;
+ const hs = (h << 16) >> 16;
+ const ai = ls + (hs & 1) + (hs >> 1);
+ return [ai & 0xffff, (ai - hs) & 0xffff];
+}
+
+function wdec16(l: number, h: number): [number, number] {
+ const b = (l - (h >> 1)) & 0xffff;
+ return [(h + b - 0x8000) & 0xffff, b];
+}
+
+/** Inverse 2D Haar wavelet in place (ImfWav wav2Decode). */
+function wav2Decode(
+ buf: Uint16Array,
+ j: number,
+ nx: number,
+ ox: number,
+ ny: number,
+ oy: number,
+ maxValue: number,
+): void {
+ const dec = maxValue < 1 << 14 ? wdec14 : wdec16;
+ const n = Math.min(nx, ny);
+ let p = 1;
+ while (p <= n) p <<= 1;
+ p >>= 1;
+ let p2 = p;
+ p >>= 1;
+ const at = (i: number) => buf[i + j] ?? 0;
+ const set = (i: number, v: number) => {
+ buf[i + j] = v;
+ };
+ while (p >= 1) {
+ let py = 0;
+ const ey = oy * (ny - p2);
+ const oy1 = oy * p;
+ const oy2 = oy * p2;
+ const ox1 = ox * p;
+ const ox2 = ox * p2;
+ for (; py <= ey; py += oy2) {
+ let px = py;
+ const ex = py + ox * (nx - p2);
+ for (; px <= ex; px += ox2) {
+ const p01 = px + ox1;
+ const p10 = px + oy1;
+ const p11 = p10 + ox1;
+ const [i00, i10] = dec(at(px), at(p10));
+ const [i01, i11] = dec(at(p01), at(p11));
+ const [a, b] = dec(i00, i01);
+ const [c, d] = dec(i10, i11);
+ set(px, a);
+ set(p01, b);
+ set(p10, c);
+ set(p11, d);
+ }
+ if (nx & p) {
+ const p10 = px + oy1;
+ const [a, b] = dec(at(px), at(p10));
+ set(px, a);
+ set(p10, b);
+ }
+ }
+ if (ny & p) {
+ let px = py;
+ const ex = py + ox * (nx - p2);
+ for (; px <= ex; px += ox2) {
+ const p01 = px + ox1;
+ const [a, b] = dec(at(px), at(p01));
+ set(px, a);
+ set(p01, b);
+ }
+ }
+ p2 = p;
+ p >>= 1;
+ }
+}
+
+function pizDecode(
+ src: Uint8Array,
+ width: number,
+ lines: number,
+ channels: ExrChannel[],
+ size: number,
+): Uint8Array {
+ const r = new Reader(src);
+ const minNonZero = r.u16();
+ const maxNonZero = r.u16();
+ if (maxNonZero >= 8192) throw new Error("OpenEXR: corrupt PIZ bitmap");
+ const bitmap = new Uint8Array(8192);
+ if (minNonZero <= maxNonZero) {
+ const n = maxNonZero - minNonZero + 1;
+ bitmap.set(src.subarray(r.offset, r.offset + n), minNonZero);
+ r.offset += n;
+ }
+ // Reverse LUT: dense codes back to the 16-bit values present in the block.
+ const lut = new Uint16Array(65536);
+ let k = 0;
+ for (let i = 0; i < 65536; i++) {
+ if (i === 0 || (bitmap[i >> 3] ?? 0) & (1 << (i & 7))) lut[k++] = i;
+ }
+ const maxValue = k - 1;
+ const length = r.i32();
+ const words = new Uint16Array(size / 2);
+ hufUncompress(src.subarray(0, r.offset + length), r.offset, words);
+ let start = 0;
+ for (const ch of channels) {
+ const perSample = ch.size / 2;
+ for (let j = 0; j < perSample; j++) {
+ wav2Decode(
+ words,
+ start + j,
+ width,
+ perSample,
+ lines,
+ width * perSample,
+ maxValue,
+ );
+ }
+ start += width * lines * perSample;
+ }
+ for (let i = 0; i < words.length; i++) words[i] = lut[words[i] ?? 0] ?? 0;
+ // Channel-planar words back to the line-interleaved layout.
+ const out = new Uint8Array(size);
+ const outWords = new Uint16Array(out.buffer);
+ let o = 0;
+ const cursors: number[] = [];
+ let c0 = 0;
+ for (const ch of channels) {
+ cursors.push(c0);
+ c0 += width * lines * (ch.size / 2);
+ }
+ for (let y = 0; y < lines; y++) {
+ channels.forEach((ch, ci) => {
+ const n = width * (ch.size / 2);
+ const from = cursors[ci] ?? 0;
+ outWords.set(words.subarray(from, from + n), o);
+ cursors[ci] = from + n;
+ o += n;
+ });
+ }
+ return out;
+}
+
+/** One chunk's pixels in the uncompressed layout: per line, per channel, `width` samples. */
+function decompressBlock(
+ header: ExrHeader,
+ data: Uint8Array,
+ width: number,
+ lines: number,
+): Uint8Array {
+ const bytesPerLine = header.channels.reduce((n, c) => n + c.size * width, 0);
+ const size = bytesPerLine * lines;
+ // Every codec stores a chunk raw when compressing would not shrink it.
+ if (data.length >= size) return data.subarray(0, size);
+ switch (header.compression) {
+ case 1:
+ return unpredictAndInterleave(rleDecode(data, size));
+ case 2:
+ case 3:
+ return unpredictAndInterleave(new Uint8Array(inflateSync(data)));
+ case 4:
+ return pizDecode(data, width, lines, header.channels, size);
+ case 5:
+ return pxr24Decode(data, width, lines, header.channels, size);
+ default:
+ throw new Error(
+ `OpenEXR: compression ${header.compression} cannot be decoded`,
+ );
+ }
+}
+
+let halfTable: Float32Array | null = null;
+
+function halfToFloat(): Float32Array {
+ if (halfTable) return halfTable;
+ const t = new Float32Array(65536);
+ for (let h = 0; h < 65536; h++) {
+ const s = h & 0x8000 ? -1 : 1;
+ const e = (h >> 10) & 0x1f;
+ const m = h & 0x3ff;
+ t[h] =
+ e === 0
+ ? s * m * 2 ** -24
+ : e === 31
+ ? m
+ ? Number.NaN
+ : s * Number.POSITIVE_INFINITY
+ : s * (1 + m / 1024) * 2 ** (e - 15);
+ }
+ halfTable = t;
+ return t;
+}
+
+/** Which source channel feeds R, G and B: RGB, or luminance Y for all three. */
+function colorChannels(channels: ExrChannel[]): number[] {
+ const index = (name: string) => channels.findIndex((c) => c.name === name);
+ const rgb = ["R", "G", "B"].map(index);
+ if (rgb.every((i) => i >= 0)) return rgb;
+ const y = index("Y");
+ if (y >= 0) return [y, y, y];
+ throw new Error(
+ `OpenEXR: no R/G/B or Y channels (found ${channels.map((c) => c.name).join(", ")})`,
+ );
+}
+
+/** Writes a decoded block into the RGBA output at (x0, y0) of the data window. */
+function storeBlock(
+ header: ExrHeader,
+ pick: number[],
+ block: Uint8Array,
+ x0: number,
+ y0: number,
+ width: number,
+ lines: number,
+ out: Float32Array,
+): void {
+ const view = new DataView(block.buffer, block.byteOffset, block.byteLength);
+ const half = halfToFloat();
+ const offsets: number[] = [];
+ let lineBytes = 0;
+ for (const c of header.channels) {
+ offsets.push(lineBytes);
+ lineBytes += c.size * width;
+ }
+ for (let y = 0; y < lines; y++) {
+ const row = y0 + y;
+ if (row < 0 || row >= header.height) continue;
+ for (let k = 0; k < 3; k++) {
+ const ci = pick[k] ?? 0;
+ const ch = header.channels[ci];
+ if (!ch) continue;
+ let at = y * lineBytes + (offsets[ci] ?? 0);
+ for (let x = 0; x < width; x++, at += ch.size) {
+ const v =
+ ch.type === PIXEL_HALF
+ ? (half[view.getUint16(at, true)] ?? 0)
+ : ch.type === PIXEL_FLOAT
+ ? view.getFloat32(at, true)
+ : view.getUint32(at, true);
+ out[(row * header.width + x0 + x) * 4 + k] = v;
+ }
+ }
+ }
+}
+
+/** Decodes an OpenEXR panorama into linear RGBA float32 (alpha 1). */
+export function parseExr(input: Uint8Array | ArrayBuffer): DecodedHdrImage {
+ const bytes = input instanceof Uint8Array ? input : new Uint8Array(input);
+ const header = readHeader(bytes);
+ const linesPerBlock = LINES_PER_BLOCK[header.compression];
+ if (linesPerBlock === undefined)
+ throw new Error(
+ `OpenEXR: ${COMPRESSION_NAMES[header.compression] ?? header.compression} compression is not supported (use NONE, RLE, ZIPS, ZIP, PIZ or PXR24)`,
+ );
+ const pick = colorChannels(header.channels);
+ const { width, height } = header;
+ const data = new Float32Array(width * height * 4);
+ for (let i = 3; i < data.length; i += 4) data[i] = 1;
+ const r = new Reader(bytes, header.tableOffset);
+ if (header.tiled) {
+ const tw = header.tileWidth;
+ const th = header.tileHeight;
+ const count = Math.ceil(width / tw) * Math.ceil(height / th);
+ const offsets = Array.from({ length: count }, () => r.u64());
+ for (const offset of offsets) {
+ const c = new Reader(bytes, offset);
+ const dx = c.i32();
+ const dy = c.i32();
+ c.offset += 8; // level x, y: level-0 tiles come first
+ const size = c.i32();
+ const w = Math.min(tw, width - dx * tw);
+ const h = Math.min(th, height - dy * th);
+ const chunk = bytes.subarray(c.offset, c.offset + size);
+ const block = decompressBlock(header, chunk, w, h);
+ storeBlock(header, pick, block, dx * tw, dy * th, w, h, data);
+ }
+ } else {
+ const count = Math.ceil(height / linesPerBlock);
+ const offsets = Array.from({ length: count }, () => r.u64());
+ for (const offset of offsets) {
+ const c = new Reader(bytes, offset);
+ const y = c.i32() - header.yMin;
+ const size = c.i32();
+ const lines = Math.min(linesPerBlock, height - y);
+ const chunk = bytes.subarray(c.offset, c.offset + size);
+ const block = decompressBlock(header, chunk, width, lines);
+ storeBlock(header, pick, block, 0, y, width, lines, data);
+ }
+ }
+ return { width, height, data };
+}
diff --git a/packages/webgpu-renderers/src/loaders/hdr-loader.test.ts b/packages/webgpu-renderers/src/loaders/hdr-loader.test.ts
new file mode 100644
index 00000000..35956663
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/hdr-loader.test.ts
@@ -0,0 +1,172 @@
+import { describe, expect, it } from "vitest";
+import {
+ type DecodedHdrImage,
+ encodeHdr,
+ equirectDirection,
+ evaluateIrradianceSh,
+ float32ToFloat16,
+ parseHdr,
+ projectIrradianceSh,
+} from "./hdr-loader.js";
+
+function gradient(width: number, height: number): Float32Array {
+ const px = new Float32Array(width * height * 4);
+ for (let y = 0; y < height; y++) {
+ for (let x = 0; x < width; x++) {
+ const i = (y * width + x) * 4;
+ px[i] = (x + 1) * 0.37;
+ px[i + 1] = (y + 1) * 1.9;
+ px[i + 2] = 0.002 * (x + y + 1);
+ px[i + 3] = 1;
+ }
+ }
+ return px;
+}
+
+/** Radiance new-style RLE for one scanline: each channel as a single literal run. */
+function rleScanline(rgbe: Uint8Array, width: number): Uint8Array {
+ const out: number[] = [2, 2, width >> 8, width & 0xff];
+ for (let ch = 0; ch < 4; ch++) {
+ // A run of `width` identical bytes for alpha-ish channels, literals for the rest.
+ const values = Array.from({ length: width }, (_, x) => rgbe[x * 4 + ch]);
+ if (values.every((v) => v === values[0])) {
+ out.push(128 + width, values[0] ?? 0);
+ } else {
+ out.push(width, ...values.map((v) => v ?? 0));
+ }
+ }
+ return Uint8Array.from(out);
+}
+
+describe("Radiance .hdr decoding", () => {
+ it("round-trips linear radiance through RGBE within mantissa precision", () => {
+ const w = 16;
+ const h = 8;
+ const src = gradient(w, h);
+ const decoded = parseHdr(encodeHdr(w, h, src));
+ expect(decoded.width).toBe(w);
+ expect(decoded.height).toBe(h);
+ for (let i = 0; i < src.length; i += 4) {
+ const max = Math.max(src[i] ?? 0, src[i + 1] ?? 0, src[i + 2] ?? 0);
+ for (let c = 0; c < 3; c++) {
+ // RGBE shares one exponent: error is bounded by max / 256 per channel.
+ expect(
+ Math.abs((decoded.data[i + c] ?? 0) - (src[i + c] ?? 0)),
+ ).toBeLessThanOrEqual(max / 256 + 1e-6);
+ }
+ expect(decoded.data[i + 3]).toBe(1);
+ }
+ });
+
+ it("decodes new-style RLE scanlines identically to flat ones", () => {
+ const w = 12;
+ const h = 3;
+ const flat = encodeHdr(w, h, gradient(w, h));
+ const header = "#?RADIANCE\nFORMAT=32-bit_rle_rgbe\n\n";
+ const headerEnd =
+ flat.indexOf(0x0a, flat.indexOf(0x0a, header.length) - 1) + 1;
+ const parts: Uint8Array[] = [flat.subarray(0, headerEnd)];
+ for (let y = 0; y < h; y++) {
+ const row = flat.subarray(
+ headerEnd + y * w * 4,
+ headerEnd + (y + 1) * w * 4,
+ );
+ parts.push(rleScanline(row, w));
+ }
+ const rle = new Uint8Array(parts.reduce((n, p) => n + p.length, 0));
+ let o = 0;
+ for (const p of parts) {
+ rle.set(p, o);
+ o += p.length;
+ }
+ expect(rle.length).not.toBe(flat.length);
+ expect(parseHdr(rle).data).toEqual(parseHdr(flat).data);
+ });
+
+ it("rejects files without a Radiance signature or with unsupported orientation", () => {
+ const bytes = (s: string) => Uint8Array.from(s, (ch) => ch.charCodeAt(0));
+ expect(() => parseHdr(bytes("P6\n1 1\n255\n"))).toThrow(/signature/);
+ expect(() => parseHdr(bytes("#?RADIANCE\n\n+Y 1 +X 1\n\0\0\0\0"))).toThrow(
+ /resolution/,
+ );
+ expect(() => parseHdr(bytes("#?RADIANCE\n\n-Y 2 +X 1\n\0\0\0\0"))).toThrow(
+ /EOF/,
+ );
+ });
+
+ it("converts float32 to half-float bits, clamping to the largest finite half", () => {
+ expect(float32ToFloat16(0)).toBe(0);
+ expect(float32ToFloat16(1)).toBe(0x3c00);
+ expect(float32ToFloat16(-2)).toBe(0xc000);
+ expect(float32ToFloat16(0.5)).toBe(0x3800);
+ expect(float32ToFloat16(65504)).toBe(0x7bff);
+ expect(float32ToFloat16(1e9)).toBe(0x7bff);
+ expect(float32ToFloat16(Number.NaN)).toBe(0);
+ // Smallest subnormal half is 2^-24.
+ expect(float32ToFloat16(2 ** -24)).toBe(1);
+ });
+});
+
+function panorama(
+ width: number,
+ height: number,
+ radiance: (d: [number, number, number]) => [number, number, number],
+): DecodedHdrImage {
+ const data = new Float32Array(width * height * 4);
+ for (let y = 0; y < height; y++) {
+ for (let x = 0; x < width; x++) {
+ const d = equirectDirection((x + 0.5) / width, (y + 0.5) / height);
+ data.set([...radiance(d), 1], (y * width + x) * 4);
+ }
+ }
+ return { width, height, data };
+}
+
+describe("Irradiance spherical harmonics", () => {
+ it("maps texel centers to unit directions, u = 0.5 looking down -z, top row up (-y)", () => {
+ const [x, y, z] = equirectDirection(0.5, 0.5);
+ expect([x, y, z].map((v) => +v.toFixed(6))).toEqual([0, -0, -1]);
+ expect(equirectDirection(0.3, 0)[1]).toBeCloseTo(-1);
+ expect(equirectDirection(0.75, 0.5)[0]).toBeCloseTo(1);
+ });
+
+ it("returns a uniform environment's radiance at every normal", () => {
+ const sh = projectIrradianceSh(panorama(128, 64, () => [0.5, 1, 2]));
+ for (const n of [
+ [0, -1, 0],
+ [1, 0, 0],
+ [0, 0.6, -0.8],
+ ] as [number, number, number][]) {
+ const e = evaluateIrradianceSh(sh, n);
+ expect(e[0]).toBeCloseTo(0.5, 2);
+ expect(e[1]).toBeCloseTo(1, 2);
+ expect(e[2]).toBeCloseTo(2, 2);
+ }
+ });
+
+ it("matches the exact cosine-weighted mean of a bright sky over a black ground", () => {
+ // Sky-only lighting gives (1 + cos θ) / 2 from the zenith, linear in cos θ,
+ // which L2 harmonics represent exactly.
+ const sh = projectIrradianceSh(
+ panorama(256, 128, (d) => (d[1] < 0 ? [1, 1, 1] : [0, 0, 0])),
+ );
+ const at = (n: [number, number, number]) => evaluateIrradianceSh(sh, n)[0];
+ expect(at([0, -1, 0])).toBeCloseTo(1, 1);
+ expect(at([0, 1, 0])).toBeCloseTo(0, 1);
+ expect(at([1, 0, 0])).toBeCloseTo(0.5, 2);
+ expect(at([0, -0.5, Math.sqrt(0.75)])).toBeCloseTo(0.75, 1);
+ });
+
+ it("reads wide panoramas at a bounded resolution with the same result", () => {
+ const light = (d: [number, number, number]): [number, number, number] => [
+ Math.max(0, d[0]) * 3,
+ Math.max(0, -d[1]),
+ Math.max(0, d[2]) * 2,
+ ];
+ const small = projectIrradianceSh(panorama(256, 128, light));
+ const wide = projectIrradianceSh(panorama(1024, 512, light));
+ for (let i = 0; i < 36; i++) {
+ expect(wide[i] ?? 0).toBeCloseTo(small[i] ?? 0, 2);
+ }
+ });
+});
diff --git a/packages/webgpu-renderers/src/loaders/hdr-loader.ts b/packages/webgpu-renderers/src/loaders/hdr-loader.ts
new file mode 100644
index 00000000..71b8601f
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/hdr-loader.ts
@@ -0,0 +1,385 @@
+import { readFileSync } from "node:fs";
+import { isExr, parseExr } from "./exr-loader.js";
+
+export interface DecodedHdrImage {
+ width: number;
+ height: number;
+ /** Linear scene-referred RGBA float32 pixels (`width * height * 4`). */
+ data: Float32Array;
+}
+
+function readAsciiLine(
+ bytes: Uint8Array,
+ start: number,
+): { line: string; next: number } {
+ let i = start;
+ while (i < bytes.length && bytes[i] !== 0x0a) {
+ i++;
+ }
+ let end = i;
+ if (end > start && bytes[end - 1] === 0x0d) {
+ end--;
+ }
+ let line = "";
+ for (let k = start; k < end; k++) {
+ line += String.fromCharCode(bytes[k] ?? 0);
+ }
+ return { line, next: Math.min(bytes.length, i + 1) };
+}
+
+function parseHdrHeader(bytes: Uint8Array): {
+ width: number;
+ height: number;
+ offset: number;
+} {
+ let cursor = 0;
+ const first = readAsciiLine(bytes, cursor);
+ cursor = first.next;
+ if (
+ !first.line.startsWith("#?RADIANCE") &&
+ !first.line.startsWith("#?RGBE")
+ ) {
+ throw new Error(
+ "Invalid Radiance .hdr file: missing #?RADIANCE or #?RGBE signature",
+ );
+ }
+
+ while (cursor < bytes.length) {
+ const { line, next } = readAsciiLine(bytes, cursor);
+ cursor = next;
+ if (line.trim() === "") break;
+ }
+
+ const res = readAsciiLine(bytes, cursor);
+ cursor = res.next;
+ const match = res.line.trim().match(/^-Y\s+(\d+)\s+\+X\s+(\d+)$/i);
+ if (!match) {
+ throw new Error(
+ `Unsupported Radiance .hdr resolution string: "${res.line}"`,
+ );
+ }
+ const height = Number.parseInt(match[1] ?? "0", 10);
+ const width = Number.parseInt(match[2] ?? "0", 10);
+ if (width <= 0 || height <= 0) {
+ throw new Error(`Invalid Radiance .hdr dimensions: ${width}x${height}`);
+ }
+ return { width, height, offset: cursor };
+}
+
+function decodeRleScanline(
+ bytes: Uint8Array,
+ startOffset: number,
+ width: number,
+ scanlineRgbe: Uint8Array,
+): number {
+ let cursor = startOffset + 4;
+ for (let ch = 0; ch < 4; ch++) {
+ let x = 0;
+ while (x < width) {
+ if (cursor >= bytes.length) {
+ throw new Error("Truncated RLE scanline in Radiance .hdr file");
+ }
+ const code = bytes[cursor++] ?? 0;
+ if (code > 128) {
+ const count = code - 128;
+ const val = bytes[cursor++] ?? 0;
+ for (let k = 0; k < count && x < width; k++) {
+ scanlineRgbe[x * 4 + ch] = val;
+ x++;
+ }
+ } else {
+ const count = code;
+ for (let k = 0; k < count && x < width; k++) {
+ scanlineRgbe[x * 4 + ch] = bytes[cursor++] ?? 0;
+ x++;
+ }
+ }
+ }
+ }
+ return cursor;
+}
+
+function rgbeToFloatRgba(
+ scanlineRgbe: Uint8Array,
+ width: number,
+ out: Float32Array,
+ rowOffsetFloats: number,
+): void {
+ for (let x = 0; x < width; x++) {
+ const i4 = x * 4;
+ const e = scanlineRgbe[i4 + 3] ?? 0;
+ const dst = rowOffsetFloats + i4;
+ if (e === 0) {
+ out[dst] = 0;
+ out[dst + 1] = 0;
+ out[dst + 2] = 0;
+ out[dst + 3] = 1;
+ } else {
+ const scale = 2 ** (e - 136);
+ out[dst] = ((scanlineRgbe[i4] ?? 0) + 0.5) * scale;
+ out[dst + 1] = ((scanlineRgbe[i4 + 1] ?? 0) + 0.5) * scale;
+ out[dst + 2] = ((scanlineRgbe[i4 + 2] ?? 0) + 0.5) * scale;
+ out[dst + 3] = 1;
+ }
+ }
+}
+
+/**
+ * Decodes a Radiance `.hdr` (RGBE) binary buffer into linear RGBA float32 radiance.
+ */
+export function parseHdr(input: Uint8Array | ArrayBuffer): DecodedHdrImage {
+ const bytes = input instanceof Uint8Array ? input : new Uint8Array(input);
+ const { width, height, offset: startOffset } = parseHdrHeader(bytes);
+ const data = new Float32Array(width * height * 4);
+ const scanlineRgbe = new Uint8Array(width * 4);
+ let cursor = startOffset;
+
+ for (let y = 0; y < height; y++) {
+ if (cursor + 4 > bytes.length) {
+ throw new Error("Unexpected EOF while reading Radiance .hdr scanline");
+ }
+ const b0 = bytes[cursor] ?? 0;
+ const b1 = bytes[cursor + 1] ?? 0;
+ const b2 = bytes[cursor + 2] ?? 0;
+ const b3 = bytes[cursor + 3] ?? 0;
+ const isAdaptiveRle =
+ width >= 8 &&
+ width <= 0x7fff &&
+ b0 === 2 &&
+ b1 === 2 &&
+ ((b2 << 8) | b3) === width;
+
+ if (isAdaptiveRle) {
+ cursor = decodeRleScanline(bytes, cursor, width, scanlineRgbe);
+ } else {
+ const byteLen = width * 4;
+ scanlineRgbe.set(bytes.subarray(cursor, cursor + byteLen));
+ cursor += byteLen;
+ }
+ rgbeToFloatRgba(scanlineRgbe, width, data, y * width * 4);
+ }
+
+ return { width, height, data };
+}
+
+/**
+ * Encodes linear RGB/RGBA float32 pixels into a valid Radiance `.hdr` (RGBE) binary buffer.
+ */
+export function encodeHdr(
+ width: number,
+ height: number,
+ rgbaOrRgb: Float32Array,
+ stride: 3 | 4 = 4,
+): Uint8Array {
+ const header = `#?RADIANCE\nFORMAT=32-bit_rle_rgbe\n\n-Y ${height} +X ${width}\n`;
+ const headerBytes = new Uint8Array(header.length);
+ for (let i = 0; i < header.length; i++) {
+ headerBytes[i] = header.charCodeAt(i);
+ }
+ const pixelBytes = width * height * 4;
+ const out = new Uint8Array(headerBytes.length + pixelBytes);
+ out.set(headerBytes, 0);
+
+ let dst = headerBytes.length;
+ const count = width * height;
+ for (let i = 0; i < count; i++) {
+ const src = i * stride;
+ const r = Math.max(0, rgbaOrRgb[src] ?? 0);
+ const g = Math.max(0, rgbaOrRgb[src + 1] ?? 0);
+ const b = Math.max(0, rgbaOrRgb[src + 2] ?? 0);
+ const maxComp = Math.max(r, g, b);
+ if (maxComp < 1e-32) {
+ out[dst] = 0;
+ out[dst + 1] = 0;
+ out[dst + 2] = 0;
+ out[dst + 3] = 0;
+ } else {
+ const exp = Math.floor(Math.log2(maxComp)) + 1;
+ const scale = 2 ** (8 - exp);
+ out[dst] = Math.min(255, Math.max(0, Math.floor(r * scale)));
+ out[dst + 1] = Math.min(255, Math.max(0, Math.floor(g * scale)));
+ out[dst + 2] = Math.min(255, Math.max(0, Math.floor(b * scale)));
+ out[dst + 3] = Math.min(255, Math.max(0, exp + 128));
+ }
+ dst += 4;
+ }
+ return out;
+}
+
+const f32View = new Float32Array(1);
+const u32View = new Uint32Array(f32View.buffer);
+
+/** Converts a 32-bit float to IEEE 754 16-bit half-float bits, clamping to max finite half (65504). */
+export function float32ToFloat16(val: number): number {
+ const clamped = Math.max(
+ -65504,
+ Math.min(65504, Number.isFinite(val) ? val : 0),
+ );
+ f32View[0] = clamped;
+ const x = u32View[0] ?? 0;
+ const sign = (x >>> 16) & 0x8000;
+ const exp = ((x >>> 23) & 0xff) - 127 + 15;
+ const mant = x & 0x7fffff;
+ if (exp <= 0) {
+ if (exp < -10) return sign;
+ const m = (mant | 0x800000) >>> (1 - exp + 13);
+ return sign | m;
+ }
+ if (exp >= 31) {
+ return sign | 0x7bff;
+ }
+ return sign | (exp << 10) | (mant >>> 13);
+}
+
+/** Converts RGBA float32 pixels to `rgba16float` texel bits. */
+export function floatRgbaToHalf16(src: Float32Array): Uint16Array {
+ const out = new Uint16Array(src.length);
+ for (let i = 0; i < src.length; i++) {
+ out[i] = float32ToFloat16(src[i] ?? 0);
+ }
+ return out;
+}
+
+/** Halves an equirectangular panorama with horizontal wrap-around filtering. */
+export function downsampleEquirect(image: DecodedHdrImage): DecodedHdrImage {
+ const { data: src, width: srcW, height: srcH } = image;
+ const dstW = Math.max(1, srcW >> 1);
+ const dstH = Math.max(1, srcH >> 1);
+ const dst = new Float32Array(dstW * dstH * 4);
+ for (let y = 0; y < dstH; y++) {
+ const sy0 = Math.min(srcH - 1, y * 2);
+ const sy1 = Math.min(srcH - 1, y * 2 + 1);
+ for (let x = 0; x < dstW; x++) {
+ const sx0 = (x * 2) % srcW;
+ const sx1 = (x * 2 + 1) % srcW;
+ const sxPrev = (sx0 + srcW - 1) % srcW;
+ const sxNext = (sx1 + 1) % srcW;
+ const idx00 = (sy0 * srcW + sx0) * 4;
+ const idx10 = (sy0 * srcW + sx1) * 4;
+ const idx01 = (sy1 * srcW + sx0) * 4;
+ const idx11 = (sy1 * srcW + sx1) * 4;
+ const idxL = (sy0 * srcW + sxPrev) * 4;
+ const idxR = (sy1 * srcW + sxNext) * 4;
+ const outIdx = (y * dstW + x) * 4;
+ for (let c = 0; c < 3; c++) {
+ const core =
+ (src[idx00 + c] ?? 0) +
+ (src[idx10 + c] ?? 0) +
+ (src[idx01 + c] ?? 0) +
+ (src[idx11 + c] ?? 0);
+ const wing = (src[idxL + c] ?? 0) + (src[idxR + c] ?? 0);
+ dst[outIdx + c] = core * 0.2 + wing * 0.1;
+ }
+ dst[outIdx + 3] = 1;
+ }
+ }
+ return { width: dstW, height: dstH, data: dst };
+}
+
+/**
+ * Decodes a panorama from a file path or its bytes: OpenEXR (`.exr`) or
+ * Radiance RGBE (`.hdr`), told apart by their magic bytes.
+ */
+export function decodeHdrSource(source: string | Uint8Array): DecodedHdrImage {
+ const bytes =
+ typeof source === "string" ? new Uint8Array(readFileSync(source)) : source;
+ return isExr(bytes) ? parseExr(bytes) : parseHdr(bytes);
+}
+
+/**
+ * World direction of an equirectangular texel center (y points down; u = 0.5
+ * looks down -z). The shaders sample the same mapping.
+ */
+export function equirectDirection(
+ u: number,
+ v: number,
+): [number, number, number] {
+ const theta = v * Math.PI;
+ const a = (u - 0.5) * 2 * Math.PI;
+ const s = Math.sin(theta);
+ return [s * Math.sin(a), -Math.cos(theta), -s * Math.cos(a)];
+}
+
+/** Real L2 spherical-harmonic basis, in the order the shaders evaluate it. */
+export function shBasis(x: number, y: number, z: number): number[] {
+ return [
+ 0.282095,
+ 0.488603 * y,
+ 0.488603 * z,
+ 0.488603 * x,
+ 1.092548 * x * y,
+ 1.092548 * y * z,
+ 0.315392 * (3 * z * z - 1),
+ 1.092548 * x * z,
+ 0.546274 * (x * x - y * y),
+ ];
+}
+
+/** Cosine-lobe convolution per band (Ramamoorthi & Hanrahan), over π. */
+const SH_BAND_SCALE = [
+ 1,
+ 2 / 3,
+ 2 / 3,
+ 2 / 3,
+ 1 / 4,
+ 1 / 4,
+ 1 / 4,
+ 1 / 4,
+ 1 / 4,
+];
+
+/** Widest panorama row the SH projection reads; L2 needs nothing finer. */
+const SH_MAX_WIDTH = 256;
+
+/**
+ * Projects a panorama's diffuse lighting onto 9 L2 spherical harmonics
+ * (36 floats: 9 × rgb + pad). Evaluated at a normal, the result is the
+ * cosine-weighted mean radiance arriving there (irradiance / π), so a
+ * Lambertian surface of albedo `c` reflects `c × result`.
+ */
+export function projectIrradianceSh(image: DecodedHdrImage): Float32Array {
+ let src = image;
+ while (src.width > SH_MAX_WIDTH) src = downsampleEquirect(src);
+ const { width: w, height: h, data } = src;
+ const acc = new Float64Array(27);
+ const dPhiDTheta = ((2 * Math.PI) / w) * (Math.PI / h);
+ for (let y = 0; y < h; y++) {
+ const v = (y + 0.5) / h;
+ const weight = dPhiDTheta * Math.sin(v * Math.PI);
+ for (let x = 0; x < w; x++) {
+ const [dx, dy, dz] = equirectDirection((x + 0.5) / w, v);
+ const basis = shBasis(dx, dy, dz);
+ const i = (y * w + x) * 4;
+ for (let k = 0; k < 9; k++) {
+ const b = (basis[k] ?? 0) * weight;
+ acc[k * 3] += (data[i] ?? 0) * b;
+ acc[k * 3 + 1] += (data[i + 1] ?? 0) * b;
+ acc[k * 3 + 2] += (data[i + 2] ?? 0) * b;
+ }
+ }
+ }
+ const out = new Float32Array(36);
+ for (let k = 0; k < 9; k++) {
+ const scale = SH_BAND_SCALE[k] ?? 0;
+ out[k * 4] = (acc[k * 3] ?? 0) * scale;
+ out[k * 4 + 1] = (acc[k * 3 + 1] ?? 0) * scale;
+ out[k * 4 + 2] = (acc[k * 3 + 2] ?? 0) * scale;
+ }
+ return out;
+}
+
+/** Evaluates `projectIrradianceSh` coefficients at a unit normal. */
+export function evaluateIrradianceSh(
+ sh: Float32Array,
+ n: [number, number, number],
+): [number, number, number] {
+ const basis = shBasis(n[0], n[1], n[2]);
+ const out: [number, number, number] = [0, 0, 0];
+ for (let k = 0; k < 9; k++) {
+ const b = basis[k] ?? 0;
+ out[0] += (sh[k * 4] ?? 0) * b;
+ out[1] += (sh[k * 4 + 1] ?? 0) * b;
+ out[2] += (sh[k * 4 + 2] ?? 0) * b;
+ }
+ return out;
+}
diff --git a/packages/webgpu-renderers/src/loaders/index.ts b/packages/webgpu-renderers/src/loaders/index.ts
index 9b9439cb..bc64b787 100644
--- a/packages/webgpu-renderers/src/loaders/index.ts
+++ b/packages/webgpu-renderers/src/loaders/index.ts
@@ -1,10 +1,12 @@
-export * from "./types.js";
-export * from "./obj-loader.js";
+export * from "./3ds-loader.js";
+export * from "./exr-loader.js";
export * from "./fbx-loader.js";
-export * from "./stl-loader.js";
-export * from "./ply-loader.js";
export * from "./gltf-loader.js";
-export * from "./vox-loader.js";
-export * from "./3ds-loader.js";
+export * from "./hdr-loader.js";
+export * from "./obj-loader.js";
export * from "./off-loader.js";
+export * from "./ply-loader.js";
export * from "./pose.js";
+export * from "./stl-loader.js";
+export * from "./types.js";
+export * from "./vox-loader.js";
diff --git a/packages/webgpu-renderers/src/math3d/environment3d.ts b/packages/webgpu-renderers/src/math3d/environment3d.ts
index 189555a0..59dc6c79 100644
--- a/packages/webgpu-renderers/src/math3d/environment3d.ts
+++ b/packages/webgpu-renderers/src/math3d/environment3d.ts
@@ -16,8 +16,10 @@ export type ToneMapping = "neutral" | "agx" | "aces" | "none";
export type EnvironmentPreset =
| "studio"
| "soft"
+ | "warm"
| "sunset"
| "night"
+ | "neon"
| "neutral";
export interface Environment3DFog {
@@ -46,8 +48,44 @@ export interface Environment3DFloor {
opacity?: number;
}
+export interface Environment3DVolumetricFogNoise {
+ /** World-space noise frequency (1 / period in px). Default 0.003. */
+ scale?: number;
+ /** Density modulation depth 0..1. Default 0.45. */
+ strength?: number;
+ /** Wind drift velocity [vx, vy, vz] in world px/sec. Default [20, -8, 12]. */
+ speed?: [number, number, number];
+ /** FBM octaves 1..3. Default 2. */
+ octaves?: number;
+}
+
+export interface Environment3DVolumetricFog {
+ /** Base extinction/scattering density (typical range 0.002..0.15, default 0.02). */
+ density?: number;
+ /** In-scattering albedo tint (default: environment horizon color). */
+ color?: string;
+ /** Henyey-Greenstein forward/back scatter asymmetry g in (-0.95..0.95). Default 0.55. */
+ anisotropy?: number;
+ /** Exponential height falloff rate (0 = uniform, >0 = ground mist). Default 0.0015. */
+ heightFalloff?: number;
+ /** Reference world Y height for ground fog (px, y down). Default 540. */
+ baseHeight?: number;
+ /** Raymarch step count per half-res pixel (8..64, default 28). */
+ steps?: number;
+ /** Maximum raymarch distance from camera in world px. Default 2800. */
+ maxDistance?: number;
+ /** Multiplier on direct-light in-scattering / god rays. Default 1.5. */
+ lightIntensity?: number;
+ /** Multiplier on ambient / sky in-scattering. Default 0.35. */
+ ambientIntensity?: number;
+ /** Optional 3D procedural FBM turbulence in the fog medium. */
+ noise?: boolean | Environment3DVolumetricFogNoise;
+}
+
export interface Environment3DOptions {
preset?: EnvironmentPreset;
+ /** Equirectangular panorama for real HDRI IBL: a `.hdr` (Radiance RGBE) or `.exr` (OpenEXR) path, or the file's bytes. */
+ hdri?: string | Uint8Array;
/** Overrides the preset's sky (above), horizon and ground colors. */
top?: string;
horizon?: string;
@@ -65,6 +103,8 @@ export interface Environment3DOptions {
fog?: Environment3DFog;
floor?: Environment3DFloor;
outlines?: boolean | GeometryOutlineOptions;
+ /** Froxel-style volumetric participating media & shadowed light shafts (god rays). */
+ volumetricFog?: boolean | Environment3DVolumetricFog;
}
interface PresetDef {
@@ -89,6 +129,13 @@ export const ENVIRONMENT_PRESETS: Record<EnvironmentPreset, PresetDef> = {
bottom: "#77726a",
softboxes: 1.2,
},
+ /** Warm tungsten studio with golden key and amber bounce. */
+ warm: {
+ top: "#f4d7b5",
+ horizon: "#c99768",
+ bottom: "#3d281c",
+ softboxes: 2.6,
+ },
/** Warm horizon, cool sky. */
sunset: {
top: "#4d63a3",
@@ -103,6 +150,13 @@ export const ENVIRONMENT_PRESETS: Record<EnvironmentPreset, PresetDef> = {
bottom: "#05060b",
softboxes: 0.5,
},
+ /** Cyberpunk neon studio with electric cyan top and magenta horizon. */
+ neon: {
+ top: "#26b0ff",
+ horizon: "#d926a9",
+ bottom: "#090414",
+ softboxes: 2.8,
+ },
/** Flat grey: image-based light without studio reflections. */
neutral: {
top: "#9a9a9a",
@@ -126,11 +180,29 @@ export function linearRgb(color: string): [number, number, number] {
return [lin(c[0]), lin(c[1]), lin(c[2])];
}
+export interface ResolvedVolumetricFog {
+ density: number;
+ color: [number, number, number];
+ anisotropy: number;
+ heightFalloff: number;
+ baseHeight: number;
+ steps: number;
+ maxDistance: number;
+ lightIntensity: number;
+ ambientIntensity: number;
+ noiseScale: number;
+ noiseStrength: number;
+ noiseSpeed: [number, number, number];
+ noiseOctaves: number;
+}
+
export interface ResolvedEnvironment {
top: [number, number, number];
horizon: [number, number, number];
bottom: [number, number, number];
softboxes: number;
+ intensity: number;
+ hdri?: string | Uint8Array;
rotationRad: number;
exposureMultiplier: number;
toneMapping: number;
@@ -148,24 +220,62 @@ export interface ResolvedEnvironment {
fadeDistance: number;
opacity: number;
};
+ volumetricFog?: ResolvedVolumetricFog;
+}
+
+export function resolveVolumetricFog(
+ vf: boolean | Environment3DVolumetricFog | undefined,
+ defaultColor: [number, number, number],
+): ResolvedVolumetricFog | undefined {
+ if (!vf) return undefined;
+ const cfg: Environment3DVolumetricFog = vf === true ? {} : vf;
+ const noiseCfg: Environment3DVolumetricFogNoise | undefined =
+ cfg.noise === true
+ ? {}
+ : cfg.noise && typeof cfg.noise === "object"
+ ? cfg.noise
+ : undefined;
+ return {
+ density: Math.max(0, cfg.density ?? 0.02),
+ color: cfg.color ? linearRgb(cfg.color) : defaultColor,
+ anisotropy: Math.max(-0.95, Math.min(0.95, cfg.anisotropy ?? 0.55)),
+ heightFalloff: Math.max(0, cfg.heightFalloff ?? 0.0015),
+ baseHeight: cfg.baseHeight ?? 540,
+ steps: Math.max(8, Math.min(64, Math.round(cfg.steps ?? 28))),
+ maxDistance: Math.max(100, cfg.maxDistance ?? 2800),
+ lightIntensity: Math.max(0, cfg.lightIntensity ?? 1.5),
+ ambientIntensity: Math.max(0, cfg.ambientIntensity ?? 0.35),
+ noiseScale: Math.max(0.0001, noiseCfg?.scale ?? 0.003),
+ noiseStrength: noiseCfg
+ ? Math.max(0, Math.min(1, noiseCfg.strength ?? 0.45))
+ : 0,
+ noiseSpeed: noiseCfg?.speed ?? [20, -8, 12],
+ noiseOctaves: noiseCfg
+ ? Math.max(1, Math.min(3, Math.round(noiseCfg.octaves ?? 2)))
+ : 0,
+ };
}
/** Linear radiance and shader codes for an environment. */
export function resolveEnvironment(
opts: Environment3DOptions,
): ResolvedEnvironment {
- const preset = ENVIRONMENT_PRESETS[opts.preset ?? "studio"];
+ const preset =
+ ENVIRONMENT_PRESETS[opts.preset ?? "studio"] ?? ENVIRONMENT_PRESETS.studio;
const k = opts.intensity ?? 1;
const scale = (c: [number, number, number]): [number, number, number] => [
c[0] * k,
c[1] * k,
c[2] * k,
];
+ const horizonLinear = linearRgb(opts.horizon ?? preset.horizon);
return {
top: scale(linearRgb(opts.top ?? preset.top)),
- horizon: scale(linearRgb(opts.horizon ?? preset.horizon)),
+ horizon: scale(horizonLinear),
bottom: scale(linearRgb(opts.bottom ?? preset.bottom)),
softboxes: (opts.softboxes ?? preset.softboxes) * k,
+ intensity: k,
+ ...(opts.hdri !== undefined && { hdri: opts.hdri }),
rotationRad: ((opts.rotation ?? 0) * Math.PI) / 180,
exposureMultiplier: 2 ** (opts.exposure ?? 0),
toneMapping: TONE_MAPPING_CODE[opts.toneMapping ?? "neutral"],
@@ -190,12 +300,13 @@ export function resolveEnvironment(
opacity: Math.max(0, Math.min(1, opts.floor.opacity ?? 1.0)),
}
: undefined,
+ volumetricFog: resolveVolumetricFog(opts.volumetricFog, horizonLinear),
};
}
/** Where a shadow-casting light's depth map lives, for the lit shaders. */
export interface ShadowUniformInfo {
- /** Light view-projection (world → shadow clip). */
+ /** Light view-projection (world → shadow clip). For omnidirectional point shadows, element [0] stores far plane. */
matrix: ArrayLike<number>;
/** Index of the casting light among the packed (non-ambient) lights. */
lightIndex: number;
@@ -209,4 +320,14 @@ export interface ShadowUniformInfo {
normalOffset: number;
/** 0 = invisible shadows, 1 = fully dark. */
darkness: number;
+ /** Starting layer index in the shadow depth 2D array texture. */
+ baseLayer?: number;
+ /** Number of array layers consumed by this shadow entry (1 for directional/spot/rect, 6 for point). */
+ layerCount?: number;
+ /** Shadow projection mode: 1 = 2D frustum, 2 = 6-face omnidirectional cube. */
+ mode?: 1 | 2;
+ /** Near plane distance in world px (used by mode=2 omnidirectional point shadows). */
+ near?: number;
+ /** Far plane distance in world px (used by mode=2 omnidirectional point shadows). */
+ far?: number;
}
diff --git a/packages/webgpu-renderers/src/math3d/light3d.ts b/packages/webgpu-renderers/src/math3d/light3d.ts
index 882b4fcd..3bef862a 100644
--- a/packages/webgpu-renderers/src/math3d/light3d.ts
+++ b/packages/webgpu-renderers/src/math3d/light3d.ts
@@ -13,7 +13,7 @@ import {
import { type Mat4, Matrix4Math } from "./matrix4.js";
import { type Vec3, Vector3Math } from "./vector3.js";
-export type LightKind = "ambient" | "directional" | "point" | "spot";
+export type LightKind = "ambient" | "directional" | "point" | "spot" | "rect";
export interface Light3DOptions {
id?: string;
@@ -39,7 +39,15 @@ export interface Light3DOptions {
decay?: number; // Attenuation exponent (default 2.0)
angle?: number; // Outer cone half-angle in degrees (default 45)
penumbra?: number; // Softness fraction [0, 1] (default 0.2)
- /** Directional or spot: render a shadow map (the first such light wins). */
+ /** Area light width in world px (default 240, rect lights only). */
+ width?: number;
+ /** Area light height in world px (default 140, rect lights only). */
+ height?: number;
+ /** Emit from both faces of a rect light (default false). */
+ twoSided?: boolean;
+ /** In-plane roll angle in degrees around the light's forward axis (default 0). */
+ rotation?: number;
+ /** Directional, spot, point, or rect: render a shadow map. */
castShadow?: boolean;
/** Shadow edge blur in shadow-map texels (default 2). */
shadowSoftness?: number;
@@ -48,23 +56,36 @@ export interface Light3DOptions {
}
export interface GPULightData {
- /** xyz = position, w = type (0=ambient, 1=directional, 2=point, 3=spot) */
+ /** xyz = position, w = type (0=ambient, 1=directional, 2=point, 3=spot, 4=rect) */
posType: [number, number, number, number];
/** xyz = normalized direction, w = radius */
dirRadius: [number, number, number, number];
/** rgb = linear color, w = intensity */
colorIntensity: [number, number, number, number];
- /** x = cos(outer), y = cos(inner), z = decay, w = 0 */
+ /** x = cos(outer), y = cos(inner), z = decay, w = twoSided (rect) */
spotParams: [number, number, number, number];
+ /** xyz = normalized right axis of rect area light, w = halfWidth */
+ rectRight: [number, number, number, number];
+ /** xyz = normalized up axis of rect area light, w = halfHeight */
+ rectUp: [number, number, number, number];
}
/** Slots retained in the legacy uniform prefix; actual direct lights use storage. */
export const MAX_LIGHTS = 8;
/**
* 16 (ambient) + 16 (params) + 8 * 64 (lights), then the physical-shading
- * block appended after them: 8 vec4 (128) + shadow matrix (64) + floorParams (16).
+ * block appended after them: 8 vec4 (128) + shadow matrix (64) +
+ * floorParams (16) + HDRI irradiance SH (9 vec4, 144).
*/
-export const LIGHT_UNIFORM_SIZE_BYTES = 544 + 128 + 64 + 32;
+export const LIGHT_UNIFORM_SIZE_BYTES = 544 + 128 + 64 + 16 + 144;
+
+/** A prefiltered HDRI as the lights uniform describes it. */
+export interface HdriUniforms {
+ /** Highest mip of the radiance cubemap (roughness 1). */
+ maxMip: number;
+ /** 9 L2 irradiance SH coefficients, one vec4 (rgb + pad) each. */
+ sh: Float32Array;
+}
export interface LightsUniformData {
ambientColor: [number, number, number, number];
@@ -158,6 +179,8 @@ function hslToRgb(h: number, s: number, l: number): [number, number, number] {
return [hue2rgb(p, q, h + 1 / 3), hue2rgb(p, q, h), hue2rgb(p, q, h - 1 / 3)];
}
+export const DIRECT_LIGHT_FLOATS = 48;
+
export const Light3D = {
/**
* Computes the 3x3 normal matrix (stored in 4x4 layout) from a 4x4 Model matrix.
@@ -206,6 +229,7 @@ export const Light3D = {
else if (kind === "directional") typeVal = 1.0;
else if (kind === "point") typeVal = 2.0;
else if (kind === "spot") typeVal = 3.0;
+ else if (kind === "rect") typeVal = 4.0;
let dir: Vec3 = [0, 0, -1];
if (opt.direction) {
@@ -237,11 +261,33 @@ export const Light3D = {
const cosOuter = Math.cos(outerRad);
const cosInner = Math.cos(innerRad);
+ const worldUp: Vec3 = Math.abs(dir[1]) > 0.999 ? [0, 0, 1] : [0, 1, 0];
+ const baseRight = Vector3Math.normalize(Vector3Math.cross(worldUp, dir));
+ const baseUp = Vector3Math.normalize(Vector3Math.cross(dir, baseRight));
+ const rollRad = ((opt.rotation ?? 0) * Math.PI) / 180;
+ const cosR = Math.cos(rollRad);
+ const sinR = Math.sin(rollRad);
+ const right: Vec3 = [
+ baseRight[0] * cosR + baseUp[0] * sinR,
+ baseRight[1] * cosR + baseUp[1] * sinR,
+ baseRight[2] * cosR + baseUp[2] * sinR,
+ ];
+ const up: Vec3 = [
+ -baseRight[0] * sinR + baseUp[0] * cosR,
+ -baseRight[1] * sinR + baseUp[1] * cosR,
+ -baseRight[2] * sinR + baseUp[2] * cosR,
+ ];
+ const halfWidth = Math.max(0.5, (opt.width ?? 240) * 0.5);
+ const halfHeight = Math.max(0.5, (opt.height ?? 140) * 0.5);
+ const twoSidedFlag = opt.twoSided ? 1.0 : 0.0;
+
return {
posType: [posX, posY, posZ, typeVal],
dirRadius: [dir[0], dir[1], dir[2], radius],
colorIntensity: [color[0], color[1], color[2], intensity],
- spotParams: [cosOuter, cosInner, decay, 0.0],
+ spotParams: [cosOuter, cosInner, decay, twoSidedFlag],
+ rectRight: [right[0], right[1], right[2], halfWidth],
+ rectUp: [up[0], up[1], up[2], halfHeight],
};
},
@@ -253,6 +299,7 @@ export const Light3D = {
environment?: Environment3DOptions | null,
shadow?: ShadowUniformInfo | null,
isReflectionPass = false,
+ hdri?: HdriUniforms,
): Float32Array {
const buffer = new Float32Array(LIGHT_UNIFORM_SIZE_BYTES / 4);
Light3D.packPhysicalBlock(
@@ -261,6 +308,7 @@ export const Light3D = {
environment,
shadow,
isReflectionPass,
+ hdri,
);
if ((!lights || lights.length === 0) && !environment) {
@@ -345,28 +393,40 @@ export const Light3D = {
const direct = lights.filter(
(light) => (light.lightType ?? light.type) !== "ambient",
);
- const data = new Float32Array(Math.max(1, direct.length) * 40);
- const shadowLayers = new Map(
- shadows.map((shadow, layer) => [shadow.lightIndex, layer]),
+ const data = new Float32Array(
+ Math.max(1, direct.length) * DIRECT_LIGHT_FLOATS,
+ );
+ const shadowByLightIndex = new Map(
+ shadows.map((shadow, idx) => [shadow.lightIndex, { shadow, idx }]),
);
for (const [i, light] of direct.entries()) {
const packed = Light3D.processLight(light);
- const offset = i * 40;
+ const offset = i * DIRECT_LIGHT_FLOATS;
data.set(packed.posType, offset);
data.set(packed.dirRadius, offset + 4);
data.set(packed.colorIntensity, offset + 8);
data.set(packed.spotParams, offset + 12);
- const layer = shadowLayers.get(i);
- const shadow = layer === undefined ? undefined : shadows[layer];
- if (shadow && layer !== undefined) {
+ const entry = shadowByLightIndex.get(i);
+ if (entry) {
+ const { shadow, idx } = entry;
for (let j = 0; j < 16; j++)
data[offset + 16 + j] = shadow.matrix[j] ?? 0;
- data.set([shadow.softness, shadow.mapSize, layer, 1], offset + 32);
+ const mode = shadow.mode ?? 1;
+ if (mode === 2) {
+ data[offset + 16] = shadow.far ?? packed.dirRadius[3];
+ }
+ const baseLayer = shadow.baseLayer ?? idx;
+ data.set(
+ [shadow.softness, shadow.mapSize, baseLayer, mode],
+ offset + 32,
+ );
data.set(
- [shadow.bias, shadow.normalOffset, shadow.darkness, 0],
+ [shadow.bias, shadow.normalOffset, shadow.darkness, shadow.near ?? 8],
offset + 36,
);
}
+ data.set(packed.rectRight, offset + 40);
+ data.set(packed.rectUp, offset + 44);
}
return data;
},
@@ -377,7 +437,13 @@ export const Light3D = {
for (const l of lights) {
const kind = l.lightType ?? l.type;
if (kind === "ambient") continue;
- if (l.castShadow && (kind === "directional" || kind === "spot")) {
+ if (
+ l.castShadow &&
+ (kind === "directional" ||
+ kind === "spot" ||
+ kind === "point" ||
+ kind === "rect")
+ ) {
return index;
}
index++;
@@ -392,6 +458,7 @@ export const Light3D = {
environment?: Environment3DOptions | null,
shadow?: ShadowUniformInfo | null,
isReflectionPass = false,
+ hdri?: HdriUniforms,
): void {
const base = 8 + MAX_LIGHTS * 16; // first float after the lights array
let ambR = 0;
@@ -412,14 +479,19 @@ export const Light3D = {
buffer[base + 2] = ambB;
buffer[base + 3] = 1; // exposure multiplier
+ let envIntensity = 1.0;
if (environment) {
const env = resolveEnvironment(environment);
+ envIntensity = env.intensity;
+ // 1 procedural; 2 + max mip with a prefiltered HDRI cubemap bound.
+ const envMode = hdri ? 2 + hdri.maxMip : 1;
+ if (hdri) buffer.set(hdri.sh.subarray(0, 36), base + 52);
buffer[6] = 1; // physical shading
buffer[7] = env.toneMapping;
buffer[base + 3] = env.exposureMultiplier;
buffer.set([...env.top, env.rotationRad], base + 4);
buffer.set([...env.horizon, env.softboxes], base + 8);
- buffer.set([...env.bottom, 1], base + 12);
+ buffer.set([...env.bottom, envMode], base + 12);
if (env.fog) {
buffer.set([...env.fog.color, 1], base + 16);
buffer.set([env.fog.start, env.fog.end, env.fog.opacity, 0], base + 20);
@@ -443,12 +515,12 @@ export const Light3D = {
buffer[base + 48] = environment.floor.y;
buffer[base + 49] = environment.floor.fadeDistance ?? 900;
buffer[base + 50] = isReflectionPass ? 1.0 : 0.0;
- buffer[base + 51] = 0.0;
+ buffer[base + 51] = envIntensity;
} else {
buffer[base + 48] = 0.0;
buffer[base + 49] = 0.0;
buffer[base + 50] = 0.0;
- buffer[base + 51] = 0.0;
+ buffer[base + 51] = envIntensity;
}
},
};
diff --git a/packages/webgpu-renderers/src/renderer3d/hdr-environment.test.ts b/packages/webgpu-renderers/src/renderer3d/hdr-environment.test.ts
new file mode 100644
index 00000000..39935a57
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/hdr-environment.test.ts
@@ -0,0 +1,239 @@
+import { mkdtempSync, utimesSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { beforeAll, describe, expect, it } from "vitest";
+import { ensureDevice } from "../device.js";
+import {
+ type DecodedHdrImage,
+ encodeHdr,
+ equirectDirection,
+} from "../loaders/hdr-loader.js";
+import {
+ cubeFaceSize,
+ getOrCreateHdrEnvironment,
+ MAX_HDR_CACHE_ENTRIES,
+} from "./hdr-environment.js";
+
+let device: GPUDevice;
+beforeAll(async () => {
+ device = await ensureDevice();
+ device.addEventListener("uncapturederror", (e) => {
+ throw new Error(`WebGPU: ${(e as GPUUncapturedErrorEvent).error.message}`);
+ });
+});
+
+function panorama(
+ width: number,
+ height: number,
+ radiance: (d: [number, number, number]) => [number, number, number],
+): DecodedHdrImage {
+ const data = new Float32Array(width * height * 4);
+ for (let y = 0; y < height; y++) {
+ for (let x = 0; x < width; x++) {
+ const d = equirectDirection((x + 0.5) / width, (y + 0.5) / height);
+ data.set([...radiance(d), 1], (y * width + x) * 4);
+ }
+ }
+ return { width, height, data };
+}
+
+function halfToFloat(h: number): number {
+ const s = h & 0x8000 ? -1 : 1;
+ const e = (h >> 10) & 0x1f;
+ const m = h & 0x3ff;
+ if (e === 0) return s * m * 2 ** -24;
+ if (e === 31) return m ? Number.NaN : s * Number.POSITIVE_INFINITY;
+ return s * (1 + m / 1024) * 2 ** (e - 15);
+}
+
+/** Reads one mip of all six faces: `[face][y * size + x] = rgb`. */
+async function readMip(
+ cube: GPUTexture,
+ mip: number,
+): Promise<[number, number, number][][]> {
+ const size = Math.max(1, cube.width >> mip);
+ const bytesPerRow = Math.ceil((size * 8) / 256) * 256;
+ const buffer = device.createBuffer({
+ size: bytesPerRow * size * 6,
+ usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ,
+ });
+ device.pushErrorScope("validation");
+ const encoder = device.createCommandEncoder();
+ encoder.copyTextureToBuffer(
+ { texture: cube, mipLevel: mip },
+ { buffer, bytesPerRow, rowsPerImage: size },
+ { width: size, height: size, depthOrArrayLayers: 6 },
+ );
+ device.queue.submit([encoder.finish()]);
+ expect(await device.popErrorScope()).toBeNull();
+ await buffer.mapAsync(GPUMapMode.READ);
+ const half = new Uint16Array(buffer.getMappedRange().slice(0));
+ buffer.destroy();
+ const faces: [number, number, number][][] = [];
+ for (let f = 0; f < 6; f++) {
+ const texels: [number, number, number][] = [];
+ for (let y = 0; y < size; y++) {
+ for (let x = 0; x < size; x++) {
+ const i = (f * size * bytesPerRow + y * bytesPerRow) / 2 + x * 4;
+ texels.push([
+ halfToFloat(half[i] ?? 0),
+ halfToFloat(half[i + 1] ?? 0),
+ halfToFloat(half[i + 2] ?? 0),
+ ]);
+ }
+ }
+ faces.push(texels);
+ }
+ return faces;
+}
+
+/** WebGPU cube face orientation, as the prefilter shader writes it. */
+function faceDirection(
+ face: number,
+ s: number,
+ t: number,
+): [number, number, number] {
+ const d: [number, number, number][] = [
+ [1, -t, -s],
+ [-1, -t, s],
+ [s, 1, t],
+ [s, -1, -t],
+ [s, -t, 1],
+ [-s, -t, -1],
+ ];
+ const [x, y, z] = d[face] ?? [0, 0, 1];
+ const len = Math.hypot(x, y, z);
+ return [x / len, y / len, z / len];
+}
+
+describe("HDR environment cubemap prefilter", () => {
+ it("sizes faces to about a quarter of the panorama width, within 8..256", () => {
+ expect(cubeFaceSize(64)).toBe(16);
+ expect(cubeFaceSize(512)).toBe(128);
+ expect(cubeFaceSize(4096)).toBe(256);
+ expect(cubeFaceSize(8)).toBe(8);
+ });
+
+ it("stores the panorama at mirror roughness in every face's orientation", async () => {
+ // Radiance encodes direction, so each texel shows where it looks.
+ const env = getOrCreateHdrEnvironment(
+ device,
+ panorama(512, 256, (d) => [d[0] + 1, d[1] + 1, d[2] + 1]),
+ );
+ expect(env.faceSize).toBe(128);
+ expect(env.mipLevelCount).toBe(6);
+ const faces = await readMip(env.cube, 0);
+ const size = env.faceSize;
+ for (let f = 0; f < 6; f++) {
+ for (const [x, y] of [
+ [3, 5],
+ [64, 64],
+ [120, 17],
+ ] as const) {
+ const want = faceDirection(
+ f,
+ ((x + 0.5) / size) * 2 - 1,
+ ((y + 0.5) / size) * 2 - 1,
+ );
+ const got = faces[f]?.[y * size + x] ?? [0, 0, 0];
+ for (let c = 0; c < 3; c++) {
+ expect(Math.abs((got[c] ?? 0) - 1 - (want[c] ?? 0))).toBeLessThan(
+ 0.03,
+ );
+ }
+ }
+ }
+ });
+
+ it("keeps a uniform environment uniform at every roughness", async () => {
+ const env = getOrCreateHdrEnvironment(
+ device,
+ panorama(128, 64, () => [0.25, 1.5, 4]),
+ );
+ for (let mip = 0; mip < env.mipLevelCount; mip++) {
+ for (const face of await readMip(env.cube, mip)) {
+ for (const [r, g, b] of face) {
+ expect(r).toBeCloseTo(0.25, 1);
+ expect(g).toBeCloseTo(1.5, 1);
+ expect(b).toBeCloseTo(4, 0);
+ }
+ }
+ }
+ });
+
+ it("spreads a small bright source wider and dimmer as roughness rises", async () => {
+ // A hot disc straight ahead (-z) on a black panorama.
+ const env = getOrCreateHdrEnvironment(
+ device,
+ panorama(512, 256, (d) => (d[2] < -0.995 ? [200, 200, 200] : [0, 0, 0])),
+ );
+ const peaks: number[] = [];
+ const lit: number[] = [];
+ for (let mip = 0; mip < env.mipLevelCount; mip++) {
+ // Face 5 is -z.
+ const face = (await readMip(env.cube, mip))[5] ?? [];
+ peaks.push(Math.max(...face.map((t) => t[0])));
+ lit.push(face.filter((t) => t[0] > 0.05).length / face.length);
+ }
+ for (let mip = 1; mip < peaks.length; mip++) {
+ expect(peaks[mip]).toBeLessThan(peaks[mip - 1] ?? 0);
+ expect(lit[mip]).toBeGreaterThanOrEqual(lit[mip - 1] ?? 0);
+ }
+ expect(lit[peaks.length - 1]).toBeGreaterThan(0.9);
+ });
+
+ it("projects irradiance SH alongside the cubemap", () => {
+ const env = getOrCreateHdrEnvironment(
+ device,
+ panorama(64, 32, () => [1, 2, 3]),
+ );
+ // DC term of a uniform radiance L: L * sqrt(4π) * 1.
+ expect(env.sh[0]).toBeCloseTo(Math.sqrt(4 * Math.PI), 2);
+ expect(env.sh[2]).toBeCloseTo(3 * Math.sqrt(4 * Math.PI), 1);
+ });
+});
+
+describe("HDR environment cache", () => {
+ const variant = (k: number) => {
+ const px = new Float32Array(16 * 8 * 4).fill(1);
+ px[0] = k + 2;
+ return encodeHdr(16, 8, px);
+ };
+
+ it("reuses one environment per content, not per byte array", () => {
+ const a = getOrCreateHdrEnvironment(device, variant(100));
+ expect(getOrCreateHdrEnvironment(device, variant(100))).toBe(a);
+ expect(getOrCreateHdrEnvironment(device, variant(101))).not.toBe(a);
+ });
+
+ it("evicts the least recently used environment past the cap", () => {
+ const sources = Array.from({ length: MAX_HDR_CACHE_ENTRIES + 1 }, (_, k) =>
+ variant(k),
+ );
+ const first = sources.map((s) => getOrCreateHdrEnvironment(device, s));
+ // Everything past the cap evicted the oldest; touching keeps one alive.
+ const touched = getOrCreateHdrEnvironment(device, sources[1] as Uint8Array);
+ expect(touched).toBe(first[1]);
+ getOrCreateHdrEnvironment(device, variant(500));
+ expect(getOrCreateHdrEnvironment(device, sources[1] as Uint8Array)).toBe(
+ first[1],
+ );
+ expect(
+ getOrCreateHdrEnvironment(device, sources[0] as Uint8Array),
+ ).not.toBe(first[0]);
+ });
+
+ it("reloads a file path after the file changes on disk", () => {
+ const dir = mkdtempSync(join(tmpdir(), "hdr-environment-"));
+ const file = join(dir, "env.hdr");
+ writeFileSync(file, variant(900));
+ utimesSync(file, 1000, 1000);
+ const first = getOrCreateHdrEnvironment(device, file);
+ expect(getOrCreateHdrEnvironment(device, file)).toBe(first);
+ writeFileSync(file, encodeHdr(64, 32, new Float32Array(64 * 32 * 4)));
+ utimesSync(file, 2000, 2000);
+ const second = getOrCreateHdrEnvironment(device, file);
+ expect(second).not.toBe(first);
+ expect(second.faceSize).toBe(16);
+ });
+});
diff --git a/packages/webgpu-renderers/src/renderer3d/hdr-environment.ts b/packages/webgpu-renderers/src/renderer3d/hdr-environment.ts
new file mode 100644
index 00000000..37f88fe2
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/hdr-environment.ts
@@ -0,0 +1,376 @@
+import { createHash } from "node:crypto";
+import { statSync } from "node:fs";
+import {
+ type DecodedHdrImage,
+ decodeHdrSource,
+ downsampleEquirect,
+ floatRgbaToHalf16,
+ projectIrradianceSh,
+} from "../loaders/hdr-loader.js";
+
+/** A captured panorama, ready for image-based lighting. */
+export interface HdrEnvironment {
+ /**
+ * GGX-prefiltered radiance cubemap: mip `m` holds the panorama as a surface
+ * of perceptual roughness `m / (mipLevelCount - 1)` reflects it.
+ */
+ cube: GPUTexture;
+ faceSize: number;
+ mipLevelCount: number;
+ /** Diffuse irradiance as 9 L2 SH coefficients (`projectIrradianceSh`). */
+ sh: Float32Array;
+}
+
+/** Most environments kept resident per device; the least recently used is destroyed. */
+export const MAX_HDR_CACHE_ENTRIES = 8;
+/** Largest cube face; finer than this adds memory, not visible detail. */
+const MAX_FACE_SIZE = 256;
+/** Roughest level is 4×4: GGX at roughness 1 has nothing finer to show. */
+const MAX_CUBE_MIPS = 7;
+/** Importance samples per texel for every rough level. */
+const PREFILTER_SAMPLES = 256;
+
+const deviceCaches = new WeakMap<GPUDevice, Map<string, HdrEnvironment>>();
+/** Stable keys for in-memory sources, so the same object never hashes twice. */
+const objectKeys = new WeakMap<object, string>();
+
+export const hdrPrefilterWgsl = /* wgsl */ `
+const PI = 3.14159265359;
+
+struct Params {
+ faceSize : u32,
+ sampleCount : u32,
+ roughness : f32,
+ srcMaxLod : f32,
+ srcTexelSolidAngle : f32,
+ cubeTexelSolidAngle : f32,
+};
+
+@group(0) @binding(0) var src : texture_2d<f32>;
+@group(0) @binding(1) var srcSampler : sampler;
+@group(0) @binding(2) var dst : texture_storage_2d_array<rgba16float, write>;
+@group(0) @binding(3) var<uniform> params : Params;
+
+// WebGPU cube face orientation: s right, t down, both in -1..1.
+fn faceDirection(face : u32, s : f32, t : f32) -> vec3<f32> {
+ switch face {
+ case 0u: { return vec3<f32>(1.0, -t, -s); }
+ case 1u: { return vec3<f32>(-1.0, -t, s); }
+ case 2u: { return vec3<f32>(s, 1.0, t); }
+ case 3u: { return vec3<f32>(s, -1.0, -t); }
+ case 4u: { return vec3<f32>(s, -t, 1.0); }
+ default: { return vec3<f32>(-s, -t, -1.0); }
+ }
+}
+
+// Same mapping as equirectDirection (y down, u = 0.5 looks down -z).
+fn equirectUv(d : vec3<f32>) -> vec2<f32> {
+ let u = fract(atan2(d.x, -d.z) / (2.0 * PI) + 0.5);
+ let v = acos(clamp(-d.y, -1.0, 1.0)) / PI;
+ return vec2<f32>(u, v);
+}
+
+fn sampleSource(d : vec3<f32>, lod : f32) -> vec3<f32> {
+ return textureSampleLevel(src, srcSampler, equirectUv(d), clamp(lod, 0.0, params.srcMaxLod)).rgb;
+}
+
+fn radicalInverse(i : u32) -> f32 {
+ var b = (i << 16u) | (i >> 16u);
+ b = ((b & 0x55555555u) << 1u) | ((b & 0xAAAAAAAAu) >> 1u);
+ b = ((b & 0x33333333u) << 2u) | ((b & 0xCCCCCCCCu) >> 2u);
+ b = ((b & 0x0F0F0F0Fu) << 4u) | ((b & 0xF0F0F0F0u) >> 4u);
+ b = ((b & 0x00FF00FFu) << 8u) | ((b & 0xFF00FF00u) >> 8u);
+ return f32(b) * 2.3283064365386963e-10;
+}
+
+@compute @workgroup_size(8, 8, 1)
+fn main(@builtin(global_invocation_id) gid : vec3<u32>) {
+ let size = params.faceSize;
+ if (gid.x >= size || gid.y >= size) {
+ return;
+ }
+ let st = (vec2<f32>(gid.xy) + 0.5) / f32(size) * 2.0 - 1.0;
+ let N = normalize(faceDirection(gid.z, st.x, st.y));
+ // A mirror level resamples the panorama at the cube's own footprint.
+ let mirrorLod = 0.5 * log2(params.cubeTexelSolidAngle / params.srcTexelSolidAngle);
+ if (params.roughness <= 0.0) {
+ textureStore(dst, gid.xy, gid.z, vec4<f32>(sampleSource(N, mirrorLod), 1.0));
+ return;
+ }
+
+ // Split-sum prefilter (Karis 2013) with view = normal, filtered importance
+ // sampling (Krivanek & Colbert 2008) to keep few samples noise-free.
+ let a = params.roughness * params.roughness;
+ let a2 = a * a;
+ let up = select(vec3<f32>(1.0, 0.0, 0.0), vec3<f32>(0.0, 1.0, 0.0), abs(N.y) < 0.999);
+ let T = normalize(cross(up, N));
+ let B = cross(N, T);
+ var sum = vec3<f32>(0.0);
+ var weight = 0.0;
+ let n = params.sampleCount;
+ for (var i = 0u; i < n; i++) {
+ let xi = vec2<f32>(f32(i) / f32(n), radicalInverse(i));
+ let phi = 2.0 * PI * xi.x;
+ let cosTheta = sqrt((1.0 - xi.y) / (1.0 + (a2 - 1.0) * xi.y));
+ let sinTheta = sqrt(1.0 - cosTheta * cosTheta);
+ let H = normalize(T * (sinTheta * cos(phi)) + B * (sinTheta * sin(phi)) + N * cosTheta);
+ let L = 2.0 * dot(N, H) * H - N;
+ let nDotL = dot(N, L);
+ if (nDotL <= 0.0) {
+ continue;
+ }
+ let nDotH = max(dot(N, H), 0.0);
+ let dDen = nDotH * nDotH * (a2 - 1.0) + 1.0;
+ let D = a2 / (PI * dDen * dDen);
+ // pdf(L) = D * nDotH / (4 vDotH), and vDotH = nDotH with V = N.
+ let pdf = max(D * 0.25, 1e-6);
+ let sampleSolidAngle = 1.0 / (f32(n) * pdf);
+ let lod = 0.5 * log2(sampleSolidAngle / params.srcTexelSolidAngle) + 1.0;
+ sum += sampleSource(L, lod) * nDotL;
+ weight += nDotL;
+ }
+ textureStore(dst, gid.xy, gid.z, vec4<f32>(sum / max(weight, 1e-6), 1.0));
+}
+`;
+
+interface PrefilterResources {
+ pipeline: GPUComputePipeline;
+ layout: GPUBindGroupLayout;
+ sampler: GPUSampler;
+}
+
+const prefilterResources = new WeakMap<GPUDevice, PrefilterResources>();
+
+function getPrefilterResources(device: GPUDevice): PrefilterResources {
+ const cached = prefilterResources.get(device);
+ if (cached) return cached;
+ const layout = device.createBindGroupLayout({
+ label: "HdrPrefilterLayout",
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.COMPUTE,
+ texture: { sampleType: "float" },
+ },
+ {
+ binding: 1,
+ visibility: GPUShaderStage.COMPUTE,
+ sampler: { type: "filtering" },
+ },
+ {
+ binding: 2,
+ visibility: GPUShaderStage.COMPUTE,
+ storageTexture: {
+ access: "write-only",
+ format: "rgba16float",
+ viewDimension: "2d-array",
+ },
+ },
+ {
+ binding: 3,
+ visibility: GPUShaderStage.COMPUTE,
+ buffer: { type: "uniform" },
+ },
+ ],
+ });
+ const pipeline = device.createComputePipeline({
+ label: "HdrPrefilterPipeline",
+ layout: device.createPipelineLayout({ bindGroupLayouts: [layout] }),
+ compute: {
+ module: device.createShaderModule({
+ label: "HdrPrefilterShader",
+ code: hdrPrefilterWgsl,
+ }),
+ entryPoint: "main",
+ },
+ });
+ const sampler = device.createSampler({
+ label: "HdrPrefilterSampler",
+ magFilter: "linear",
+ minFilter: "linear",
+ mipmapFilter: "linear",
+ addressModeU: "repeat",
+ addressModeV: "clamp-to-edge",
+ });
+ const res = { pipeline, layout, sampler };
+ prefilterResources.set(device, res);
+ return res;
+}
+
+/** Uploads the panorama with a full mip chain, the prefilter's sampling source. */
+function uploadEquirect(device: GPUDevice, image: DecodedHdrImage): GPUTexture {
+ const mipLevelCount =
+ Math.floor(Math.log2(Math.max(image.width, image.height))) + 1;
+ const texture = device.createTexture({
+ label: "HdrEquirectSource",
+ size: { width: image.width, height: image.height },
+ mipLevelCount,
+ format: "rgba16float",
+ usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST,
+ });
+ let level = image;
+ for (let mip = 0; mip < mipLevelCount; mip++) {
+ device.queue.writeTexture(
+ { texture, mipLevel: mip },
+ floatRgbaToHalf16(level.data),
+ { bytesPerRow: level.width * 8, rowsPerImage: level.height },
+ { width: level.width, height: level.height },
+ );
+ if (mip + 1 < mipLevelCount) level = downsampleEquirect(level);
+ }
+ return texture;
+}
+
+/** Cube face size for a panorama: about a quarter of its width, a power of two. */
+export function cubeFaceSize(panoramaWidth: number): number {
+ const quarter = Math.max(1, panoramaWidth / 4);
+ return Math.max(
+ 8,
+ Math.min(MAX_FACE_SIZE, 2 ** Math.floor(Math.log2(quarter))),
+ );
+}
+
+/** Prefilters an equirectangular panorama into a GGX radiance cubemap on the GPU. */
+function prefilterCube(
+ device: GPUDevice,
+ image: DecodedHdrImage,
+ label: string,
+): { cube: GPUTexture; faceSize: number; mipLevelCount: number } {
+ const { pipeline, layout, sampler } = getPrefilterResources(device);
+ const source = uploadEquirect(device, image);
+ const faceSize = cubeFaceSize(image.width);
+ const mipLevelCount = Math.min(
+ MAX_CUBE_MIPS,
+ Math.floor(Math.log2(faceSize)) - 1,
+ );
+ const cube = device.createTexture({
+ label: `HdrEnvironmentCube:${label}`,
+ size: { width: faceSize, height: faceSize, depthOrArrayLayers: 6 },
+ mipLevelCount,
+ format: "rgba16float",
+ // COPY_SRC lets tools and tests read the prefiltered levels back.
+ usage:
+ GPUTextureUsage.TEXTURE_BINDING |
+ GPUTextureUsage.STORAGE_BINDING |
+ GPUTextureUsage.COPY_SRC,
+ });
+ const srcTexelSolidAngle = (4 * Math.PI) / (image.width * image.height);
+ const sourceView = source.createView();
+ const buffers: GPUBuffer[] = [];
+ const encoder = device.createCommandEncoder({ label: "HdrPrefilter" });
+ const pass = encoder.beginComputePass({ label: "HdrPrefilterPass" });
+ pass.setPipeline(pipeline);
+ for (let mip = 0; mip < mipLevelCount; mip++) {
+ const size = Math.max(1, faceSize >> mip);
+ const params = new ArrayBuffer(32);
+ const u32 = new Uint32Array(params);
+ const f32 = new Float32Array(params);
+ u32[0] = size;
+ u32[1] = PREFILTER_SAMPLES;
+ f32[2] = mipLevelCount > 1 ? mip / (mipLevelCount - 1) : 0;
+ f32[3] = source.mipLevelCount - 1;
+ f32[4] = srcTexelSolidAngle;
+ f32[5] = (4 * Math.PI) / (6 * size * size);
+ const buffer = device.createBuffer({
+ label: `HdrPrefilterParams${mip}`,
+ size: params.byteLength,
+ usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ });
+ device.queue.writeBuffer(buffer, 0, params);
+ buffers.push(buffer);
+ pass.setBindGroup(
+ 0,
+ device.createBindGroup({
+ layout,
+ entries: [
+ { binding: 0, resource: sourceView },
+ { binding: 1, resource: sampler },
+ {
+ binding: 2,
+ resource: cube.createView({
+ dimension: "2d-array",
+ baseMipLevel: mip,
+ mipLevelCount: 1,
+ }),
+ },
+ { binding: 3, resource: { buffer } },
+ ],
+ }),
+ );
+ const groups = Math.ceil(size / 8);
+ pass.dispatchWorkgroups(groups, groups, 6);
+ }
+ pass.end();
+ device.queue.submit([encoder.finish()]);
+ // Freed once the submitted prefilter has run.
+ source.destroy();
+ for (const b of buffers) b.destroy();
+ return { cube, faceSize, mipLevelCount };
+}
+
+function contentKey(source: Uint8Array | DecodedHdrImage): string {
+ const known = objectKeys.get(source);
+ if (known) return known;
+ const hash = createHash("sha1");
+ if (source instanceof Uint8Array) {
+ hash.update(source);
+ } else {
+ hash.update(`${source.width}x${source.height}`);
+ hash.update(
+ new Uint8Array(
+ source.data.buffer,
+ source.data.byteOffset,
+ source.data.byteLength,
+ ),
+ );
+ }
+ const key = `sha1:${hash.digest("hex")}`;
+ objectKeys.set(source, key);
+ return key;
+}
+
+/**
+ * The prefiltered cubemap and irradiance SH for a `.hdr` path, its bytes or a
+ * decoded panorama, built once per device. A bounded LRU destroys evicted
+ * cubemaps so long renders never leak VRAM; a path keys on its modification
+ * time too, so an edited panorama reloads.
+ */
+export function getOrCreateHdrEnvironment(
+ device: GPUDevice,
+ source: string | Uint8Array | DecodedHdrImage,
+ cacheKey?: string,
+): HdrEnvironment {
+ const key =
+ cacheKey ??
+ (typeof source === "string"
+ ? `${source}@${statSync(source).mtimeMs}`
+ : contentKey(source));
+ let map = deviceCaches.get(device);
+ if (!map) {
+ map = new Map();
+ deviceCaches.set(device, map);
+ }
+ const existing = map.get(key);
+ if (existing) {
+ map.delete(key);
+ map.set(key, existing);
+ return existing;
+ }
+ const image =
+ typeof source === "string" || source instanceof Uint8Array
+ ? decodeHdrSource(source)
+ : source;
+ const entry: HdrEnvironment = {
+ ...prefilterCube(device, image, key),
+ sh: projectIrradianceSh(image),
+ };
+ if (map.size >= MAX_HDR_CACHE_ENTRIES) {
+ const oldest = map.keys().next().value;
+ if (oldest !== undefined) {
+ map.get(oldest)?.cube.destroy();
+ map.delete(oldest);
+ }
+ }
+ map.set(key, entry);
+ return entry;
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/index.ts b/packages/webgpu-renderers/src/renderer3d/index.ts
index 041d021c..6d5c25bc 100644
--- a/packages/webgpu-renderers/src/renderer3d/index.ts
+++ b/packages/webgpu-renderers/src/renderer3d/index.ts
@@ -2,6 +2,7 @@ export * from "./audio-mesh-deform-pipeline.js";
export * from "./bloom-pipeline.js";
export * from "./floor-reflection-pipeline.js";
export * from "./glass-pipeline.js";
+export * from "./hdr-environment.js";
export * from "./matte3d.js";
export * from "./mesh3d-pipeline.js";
export * from "./motion-blur-pipeline.js";
@@ -15,3 +16,4 @@ export * from "./ssao-pipeline.js";
export * from "./surface-params.js";
export * from "./swarm3d-pipeline.js";
export * from "./text-extruder.js";
+export * from "./volumetric-fog-pipeline.js";
diff --git a/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts b/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts
index 3856c2d5..b6531053 100644
--- a/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts
+++ b/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts
@@ -33,6 +33,10 @@ export interface RoundedBoxGeometryOptions extends PrimitiveSurface {
depth: number;
/** Corner and edge radius, clamped to half the smallest side. Default 0 (a sharp box). */
radius?: number;
+ /** Alias for `radius`. */
+ cornerRadius?: number;
+ /** Alias for `radius`. */
+ borderRadius?: number;
/** Segments per rounded quarter (default 4). */
segments?: number;
/**
@@ -53,8 +57,12 @@ export interface CylinderGeometryOptions extends PrimitiveSurface {
type: "cylinder";
/** Radius of the top (-y) cap; defaults to `radius`. */
radiusTop?: number;
+ /** Alias for `radiusTop`. */
+ topRadius?: number;
/** Radius of the bottom (+y) cap; defaults to `radius`. */
radiusBottom?: number;
+ /** Alias for `radiusBottom`. */
+ bottomRadius?: number;
radius?: number;
height: number;
/** Segments around (default 32). */
@@ -69,8 +77,14 @@ export interface TorusGeometryOptions extends PrimitiveSurface {
type: "torus";
/** Major radius from the torus center to the center of the tube (default 60). */
radius?: number;
+ /** Alias for `radius`. */
+ majorRadius?: number;
/** Minor radius of the circular tube cross-section (default 20). */
tube?: number;
+ /** Alias for `tube`. */
+ minorRadius?: number;
+ /** Alias for `tube`. */
+ tubeRadius?: number;
/** Segments around the tube cross-section (default 24). */
radialSegments?: number;
/** Segments around the main ring (default 48). */
@@ -397,7 +411,8 @@ function roundedBox(o: RoundedBoxGeometryOptions, mb: MeshBuilder): void {
const hx = o.width / 2;
const hy = o.height / 2;
const hz = o.depth / 2;
- const r = Math.max(0, Math.min(o.radius ?? 0, hx, hy, hz));
+ const rawR = o.radius ?? o.cornerRadius ?? o.borderRadius ?? 0;
+ const r = Math.max(0, Math.min(rawR, hx, hy, hz));
const seg = Math.max(1, Math.round(o.segments ?? 4));
const taper = o.taper ?? 1;
const ix = hx - r;
@@ -508,8 +523,11 @@ function sphere(o: SphereGeometryOptions, mb: MeshBuilder): void {
function cylinder(o: CylinderGeometryOptions, mb: MeshBuilder): void {
const seg = Math.max(3, Math.round(o.segments ?? 32));
- const rTop = Math.max(0, o.radiusTop ?? o.radius ?? 50);
- const rBottom = Math.max(0, o.radiusBottom ?? o.radius ?? 50);
+ const rTop = Math.max(0, o.radiusTop ?? o.topRadius ?? o.radius ?? 50);
+ const rBottom = Math.max(
+ 0,
+ o.radiusBottom ?? o.bottomRadius ?? o.radius ?? 50,
+ );
const hy = o.height / 2;
if (o.flatShaded) {
@@ -594,8 +612,8 @@ function cylinder(o: CylinderGeometryOptions, mb: MeshBuilder): void {
* camera looking along +Z), with major radius `radius` and tube radius `tube`.
*/
function torus(o: TorusGeometryOptions, mb: MeshBuilder): void {
- const R = Math.max(1e-3, o.radius ?? 60);
- const r = Math.max(1e-3, o.tube ?? 20);
+ const R = Math.max(1e-3, o.radius ?? o.majorRadius ?? 60);
+ const r = Math.max(1e-3, o.tube ?? o.minorRadius ?? o.tubeRadius ?? 20);
const radialSegments = Math.max(3, Math.round(o.radialSegments ?? 24));
const tubularSegments = Math.max(3, Math.round(o.tubularSegments ?? 48));
const arcDeg = o.arc ?? 360;
diff --git a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
index 26dc9c8d..5037c42f 100644
--- a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
@@ -16,7 +16,8 @@ import {
/**
* Group 3 of every lit 3D shader: the lights uniform, then the shadow map and
- * its comparison sampler (a 1×1 cleared map when no light casts shadows).
+ * its comparison sampler (a 1×1 cleared map when no light casts shadows), the
+ * direct lights, and the HDRI radiance cubemap with its sampler.
*/
export function createLightsLayout(
device: GPUDevice,
@@ -45,6 +46,16 @@ export function createLightsLayout(
visibility: GPUShaderStage.FRAGMENT,
buffer: { type: "read-only-storage" },
},
+ {
+ binding: 4,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float", viewDimension: "cube" },
+ },
+ {
+ binding: 5,
+ visibility: GPUShaderStage.FRAGMENT,
+ sampler: { type: "filtering" },
+ },
],
});
}
@@ -96,19 +107,11 @@ export interface ModelUniformData extends SurfaceParams {
metallic?: number;
}
-export type MaterialUniformData = Pick<
- ModelUniformData,
- | "material"
- | "shininess"
- | "roughness"
- | "specularIntensity"
- | "ambientIntensity"
- | "metallic"
- | "emissive"
- | "emissiveIntensity"
- | "toon"
- | "outline"
->;
+export type MaterialUniformData = SurfaceParams &
+ Pick<
+ ModelUniformData,
+ "material" | "shininess" | "specularIntensity" | "ambientIntensity"
+ >;
/** [shininess, specularIntensity, ambientIntensity, materialMode] as the 3D shaders' lighting reads them. */
export function packMaterialParams(
diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
index 3f28422f..7b3fb0ca 100644
--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
@@ -26,15 +26,17 @@ import type {
Mesh3DData,
Model3DData,
} from "../loaders/types.js";
-import type {
- Environment3DFloor,
- Environment3DFog,
- Environment3DOptions,
- ShadowUniformInfo,
+import {
+ type Environment3DFloor,
+ type Environment3DFog,
+ type Environment3DOptions,
+ resolveEnvironment,
+ type ShadowUniformInfo,
} from "../math3d/environment3d.js";
import {
Camera3D,
type Camera3DPose,
+ type HdriUniforms,
LIGHT_UNIFORM_SIZE_BYTES,
Light3D,
type Light3DOptions,
@@ -53,6 +55,7 @@ import { type BloomOptions, BloomPipeline } from "./bloom-pipeline.js";
import { FloorReflectionPipeline } from "./floor-reflection-pipeline.js";
import { GeometryOutlinePipeline } from "./geometry-outline-pipeline.js";
import { GeometryResolvePipeline } from "./geometry-resolve-pipeline.js";
+import { getOrCreateHdrEnvironment } from "./hdr-environment.js";
import { decodeKtx2, isKtx2 } from "./ktx2.js";
import {
type GPUMeshBuffers,
@@ -64,12 +67,14 @@ import { Quad3DPipeline, type Relief3D } from "./quad3d-pipeline.js";
import { SHADOW_DEPTH_FORMAT } from "./shadow-depth.js";
import {
type Aabb,
+ fitPointShadowFrusta,
fitShadowFrustum,
transformAabb,
} from "./shadow-frustum.js";
import { Slug3DPipeline } from "./slug3d-pipeline.js";
import type { SurfaceParams } from "./surface-params.js";
import { Swarm3DPipeline } from "./swarm3d-pipeline.js";
+import { VolumetricFogPipeline } from "./volumetric-fog-pipeline.js";
const ADDRESS_MODES: Record<"repeat" | "clamp" | "mirror", GPUAddressMode> = {
repeat: "repeat",
@@ -303,10 +308,13 @@ export class Renderer3D {
private shadowTextures = new Map<string, GPUTexture>();
private placeholderShadowTexture: GPUTexture | null = null;
private shadowSampler: GPUSampler | null = null;
+ private placeholderEnvTexture: GPUTexture | null = null;
+ private envSampler: GPUSampler | null = null;
private meshBounds = new WeakMap<Mesh3DData, Aabb>();
private bloomPipeline: BloomPipeline | null = null;
private outlinePipeline: GeometryOutlinePipeline | null = null;
private geometryResolvePipeline: GeometryResolvePipeline | null = null;
+ private volumetricFogPipeline: VolumetricFogPipeline | null = null;
private lastCameraBuffer: GPUBuffer | null = null;
/**
@@ -463,6 +471,7 @@ export class Renderer3D {
this.attachmentsUsed.clear();
this.outlinePipeline?.reset();
this.bloomPipeline?.resetPools();
+ this.volumetricFogPipeline?.resetPools();
this.cameraPool.reset();
// The cached camera buffer goes back to the pool and may be handed
// out again (e.g. to a shadow pass's light camera): never reuse it
@@ -692,6 +701,15 @@ export class Renderer3D {
return buffer;
}
+ /** The environment's prefiltered HDRI, as the lights uniform describes it. */
+ private hdriUniforms(
+ environment?: Environment3DOptions | null,
+ ): HdriUniforms | undefined {
+ if (!environment?.hdri) return undefined;
+ const env = getOrCreateHdrEnvironment(this.device, environment.hdri);
+ return { maxMip: env.mipLevelCount - 1, sh: env.sh };
+ }
+
createCameraBindGroup(cameraBuffer: GPUBuffer): GPUBindGroup {
return this.cameraUniformBindGroupCache.getBindGroup(
this.device,
@@ -704,20 +722,24 @@ export class Renderer3D {
lights?: Light3DOptions[],
world?: Scene3DWorld,
): GPUBuffer {
+ const hdri = this.hdriUniforms(world?.environment);
const lightsData = Light3D.packLightsUniforms(
lights ?? [],
world?.environment,
Array.isArray(world?.shadow) ? world.shadow[0] : world?.shadow,
+ false,
+ hdri,
);
return this.lightsPool.getBuffer(this.device, lightsData);
}
- /** Lights uniform + shadow map + comparison sampler (group 3 of every lit shader). */
+ /** Lights uniform + shadow map + comparison sampler + environment map (group 3 of every lit shader). */
private createLightsBindGroup(
lightsBuffer: GPUBuffer,
withShadow: boolean,
lights: Light3DOptions[],
shadows: ShadowUniformInfo[],
+ environment?: Environment3DOptions | null,
): GPUBindGroup {
const data = Light3D.packDirectLights(lights, shadows);
if (data.byteLength > this.device.limits.maxStorageBufferBindingSize)
@@ -740,6 +762,23 @@ export class Renderer3D {
usage:
GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.RENDER_ATTACHMENT,
});
+ this.envSampler ??= this.device.createSampler({
+ label: "HdrEnvironmentSampler",
+ magFilter: "linear",
+ minFilter: "linear",
+ mipmapFilter: "linear",
+ addressModeU: "repeat",
+ addressModeV: "clamp-to-edge",
+ });
+ this.placeholderEnvTexture ??= this.device.createTexture({
+ label: "HdrEnvironmentPlaceholder",
+ size: { width: 1, height: 1, depthOrArrayLayers: 6 },
+ format: "rgba16float",
+ usage: GPUTextureUsage.TEXTURE_BINDING,
+ });
+ const envTex = environment?.hdri
+ ? getOrCreateHdrEnvironment(this.device, environment.hdri).cube
+ : this.placeholderEnvTexture;
const map =
withShadow && this.shadowTexture
? this.shadowTexture
@@ -752,6 +791,8 @@ export class Renderer3D {
{ binding: 1, resource: map.createView({ dimension: "2d-array" }) },
{ binding: 2, resource: this.shadowSampler },
{ binding: 3, resource: { buffer: directBuffer } },
+ { binding: 4, resource: envTex.createView({ dimension: "cube" }) },
+ { binding: 5, resource: this.envSampler },
],
});
}
@@ -793,48 +834,15 @@ export class Renderer3D {
);
}
- /**
- * Renders the shadow map for `light` (a directional or spot light with
- * `castShadow`) from `casters`, fitting its frustum around them and deep
- * enough for `receivers`. Pass the result to `beginPass` as `world.shadow`.
- */
- renderShadowMap(
- encoder: GPUCommandEncoder,
- light: Light3DOptions,
- lightIndex: number,
- casters: ShadowCaster3D[],
- receivers: Aabb[],
- options: ShadowMapOptions3D = {},
- ): ShadowUniformInfo | null {
- const kind = light.lightType ?? light.type;
- if (kind !== "directional" && kind !== "spot") return null;
- const processed = Light3D.processLight(light);
- const position: Vec3 = [
- processed.posType[0],
- processed.posType[1],
- processed.posType[2],
- ];
- const direction: Vec3 = [
- processed.dirRadius[0],
- processed.dirRadius[1],
- processed.dirRadius[2],
- ];
- const mapSize = options.mapSize ?? 2048;
- const layer = options.layer ?? 0;
- const layerCount = options.layerCount ?? 1;
- if (layerCount > this.device.limits.maxTextureArrayLayers)
+ private getOrCreateShadowTexture(
+ mapSize: number,
+ layerCount: number,
+ ): GPUTexture {
+ if (layerCount > this.device.limits.maxTextureArrayLayers) {
throw new Error(
`Shadow maps require ${layerCount} texture layers, exceeding the device limit of ${this.device.limits.maxTextureArrayLayers}`,
);
- const casterBounds = casters.map((c) => this.worldBounds(c));
- const frustum = fitShadowFrustum(
- { type: kind, position, direction, angle: light.angle },
- casterBounds,
- receivers,
- mapSize,
- );
- if (!frustum) return null;
-
+ }
const key = `${mapSize}:${layerCount}`;
let texture = this.shadowTextures.get(key);
if (!texture) {
@@ -852,13 +860,24 @@ export class Renderer3D {
this.shadowTextures.set(key, texture);
}
this.shadowTexture = texture;
+ return texture;
+ }
- // The light's "camera": only viewProj matters to the vertex shaders.
+ private renderShadowLayerPass(
+ encoder: GPUCommandEncoder,
+ texture: GPUTexture,
+ layer: number,
+ mapSize: number,
+ viewProj: Mat4,
+ position: Vec3,
+ direction: Vec3,
+ casters: ShadowCaster3D[],
+ ): void {
const lightCam = new Float32Array(80);
- lightCam.set(frustum.viewProj, 32);
- lightCam.set(frustum.viewProj, 0);
+ lightCam.set(viewProj, 32);
+ lightCam.set(viewProj, 0);
lightCam[5] = 1;
- lightCam.set(frustum.viewProj, 16);
+ lightCam.set(viewProj, 16);
lightCam[64] = position[0];
lightCam[65] = position[1];
lightCam[66] = position[2];
@@ -875,7 +894,7 @@ export class Renderer3D {
label: "ShadowMapPass",
colorAttachments: [],
depthStencilAttachment: {
- view: this.shadowTexture.createView({
+ view: texture.createView({
dimension: "2d",
baseArrayLayer: layer,
arrayLayerCount: 1,
@@ -929,6 +948,122 @@ export class Renderer3D {
}
}
pass.end();
+ }
+
+ /**
+ * Renders the shadow map for `light` (a directional, spot, rect, or point light with
+ * `castShadow`) from `casters`, fitting its frustum around them and deep
+ * enough for `receivers`. Pass the result to `beginPass` as `world.shadow`.
+ */
+ renderShadowMap(
+ encoder: GPUCommandEncoder,
+ light: Light3DOptions,
+ lightIndex: number,
+ casters: ShadowCaster3D[],
+ receivers: Aabb[],
+ options: ShadowMapOptions3D = {},
+ ): ShadowUniformInfo | null {
+ const kind = light.lightType ?? light.type;
+ if (
+ kind !== "directional" &&
+ kind !== "spot" &&
+ kind !== "rect" &&
+ kind !== "point"
+ ) {
+ return null;
+ }
+ const processed = Light3D.processLight(light);
+ const position: Vec3 = [
+ processed.posType[0],
+ processed.posType[1],
+ processed.posType[2],
+ ];
+ const direction: Vec3 = [
+ processed.dirRadius[0],
+ processed.dirRadius[1],
+ processed.dirRadius[2],
+ ];
+ const mapSize =
+ options.mapSize ??
+ (kind === "point" && (options.layerCount ?? 6) === 6 ? 1024 : 2048);
+ const layer = options.layer ?? 0;
+ const casterBounds = casters.map((c) => this.worldBounds(c));
+
+ if (kind === "point") {
+ const totalLayers = Math.max(layer + 6, options.layerCount ?? 6);
+ const frusta = fitPointShadowFrusta(
+ position,
+ processed.dirRadius[3],
+ casterBounds,
+ receivers,
+ mapSize,
+ );
+ if (!frusta) return null;
+ const texture = this.getOrCreateShadowTexture(mapSize, totalLayers);
+ const cubeDirs: Vec3[] = [
+ [1, 0, 0],
+ [-1, 0, 0],
+ [0, 1, 0],
+ [0, -1, 0],
+ [0, 0, 1],
+ [0, 0, -1],
+ ];
+ for (let f = 0; f < 6; f++) {
+ this.renderShadowLayerPass(
+ encoder,
+ texture,
+ layer + f,
+ mapSize,
+ frusta.viewProjs[f],
+ position,
+ cubeDirs[f],
+ casters,
+ );
+ }
+ const mat = new Float32Array(16);
+ mat[0] = frusta.far;
+ return {
+ matrix: mat,
+ lightIndex,
+ mapSize,
+ softness: options.softness ?? light.shadowSoftness ?? 2.5,
+ bias: 0.0012,
+ normalOffset: frusta.texelWorldSize * 1.8,
+ darkness: options.darkness ?? light.shadowDarkness ?? 1,
+ baseLayer: layer,
+ layerCount: 6,
+ mode: 2,
+ near: frusta.near,
+ far: frusta.far,
+ };
+ }
+
+ const totalLayers = Math.max(layer + 1, options.layerCount ?? 1);
+ const frustum = fitShadowFrustum(
+ {
+ type: kind,
+ position,
+ direction,
+ angle: light.angle,
+ radius: processed.dirRadius[3],
+ },
+ casterBounds,
+ receivers,
+ mapSize,
+ );
+ if (!frustum) return null;
+
+ const texture = this.getOrCreateShadowTexture(mapSize, totalLayers);
+ this.renderShadowLayerPass(
+ encoder,
+ texture,
+ layer,
+ mapSize,
+ frustum.viewProj,
+ position,
+ direction,
+ casters,
+ );
return {
matrix: frustum.viewProj,
@@ -938,6 +1073,9 @@ export class Renderer3D {
bias: kind === "directional" ? 0.0006 : 0.00008,
normalOffset: frustum.texelWorldSize * 1.5,
darkness: options.darkness ?? light.shadowDarkness ?? 1,
+ baseLayer: layer,
+ layerCount: 1,
+ mode: 1,
};
}
@@ -978,6 +1116,7 @@ export class Renderer3D {
? world.shadow
: [world.shadow]
: [],
+ world?.environment,
);
const msaa = this.sampleCount > 1;
@@ -1115,11 +1254,13 @@ export class Renderer3D {
const cameraBuffer = this.packCameraUniforms(camera, reflWidth, reflHeight);
const cameraBindGroup = this.createCameraBindGroup(cameraBuffer);
+ const hdri = this.hdriUniforms(world?.environment);
const lightsData = Light3D.packLightsUniforms(
lights ?? [],
world?.environment,
Array.isArray(world?.shadow) ? world.shadow[0] : world?.shadow,
true,
+ hdri,
);
const lightsBuffer = this.lightsPool.getBuffer(this.device, lightsData);
const lightsBindGroup = this.createLightsBindGroup(
@@ -1131,6 +1272,7 @@ export class Renderer3D {
? world.shadow
: [world.shadow]
: [],
+ world?.environment,
);
const msaa = this.sampleCount > 1;
@@ -1342,6 +1484,21 @@ export class Renderer3D {
toon: opts.toon,
outline: opts.outline,
matte: opts.matte,
+ clearcoat: opts.clearcoat,
+ clearcoatRoughness: opts.clearcoatRoughness,
+ anisotropy: opts.anisotropy,
+ anisotropyRotation: opts.anisotropyRotation,
+ subsurface: opts.subsurface,
+ subsurfaceColor: opts.subsurfaceColor,
+ sheen: opts.sheen,
+ sheenColor: opts.sheenColor,
+ sheenRoughness: opts.sheenRoughness,
+ iridescence: opts.iridescence,
+ iridescenceThickness: opts.iridescenceThickness,
+ iridescenceIor: opts.iridescenceIor,
+ triplanar: opts.triplanar,
+ triplanarScale: opts.triplanarScale,
+ triplanarSharpness: opts.triplanarSharpness,
},
useMrt,
this.passWritesHdr,
@@ -1418,6 +1575,21 @@ export class Renderer3D {
outline: opts.outline,
matte: opts.matte,
relief: opts.relief,
+ clearcoat: opts.clearcoat,
+ clearcoatRoughness: opts.clearcoatRoughness,
+ anisotropy: opts.anisotropy,
+ anisotropyRotation: opts.anisotropyRotation,
+ subsurface: opts.subsurface,
+ subsurfaceColor: opts.subsurfaceColor,
+ sheen: opts.sheen,
+ sheenColor: opts.sheenColor,
+ sheenRoughness: opts.sheenRoughness,
+ iridescence: opts.iridescence,
+ iridescenceThickness: opts.iridescenceThickness,
+ iridescenceIor: opts.iridescenceIor,
+ triplanar: opts.triplanar,
+ triplanarScale: opts.triplanarScale,
+ triplanarSharpness: opts.triplanarSharpness,
},
useMrt,
this.passWritesHdr,
@@ -1768,6 +1940,21 @@ export class Renderer3D {
outline: opts.outline,
matte: opts.matte,
maps: opts.maps,
+ clearcoat: opts.clearcoat,
+ clearcoatRoughness: opts.clearcoatRoughness,
+ anisotropy: opts.anisotropy,
+ anisotropyRotation: opts.anisotropyRotation,
+ subsurface: opts.subsurface,
+ subsurfaceColor: opts.subsurfaceColor,
+ sheen: opts.sheen,
+ sheenColor: opts.sheenColor,
+ sheenRoughness: opts.sheenRoughness,
+ iridescence: opts.iridescence,
+ iridescenceThickness: opts.iridescenceThickness,
+ iridescenceIor: opts.iridescenceIor,
+ triplanar: opts.triplanar,
+ triplanarScale: opts.triplanarScale,
+ triplanarSharpness: opts.triplanarSharpness,
isInstanced: Boolean(
opts.instanceBuffer || (opts.instanceCount && opts.instanceCount > 1),
),
@@ -2065,6 +2252,60 @@ export class Renderer3D {
pass.end();
}
+ /**
+ * Raymarches volumetric fog and shadowed light shafts at half-resolution,
+ * then composites over `sourceColorTex` into `destView` using depth-aware
+ * bilateral upsampling.
+ */
+ applyVolumetricFog(
+ encoder: GPUCommandEncoder,
+ sourceColorTex: GPUTexture,
+ linearDepthTex: GPUTexture,
+ destView: GPUTextureView,
+ cameraBindGroup: GPUBindGroup,
+ lightsBindGroup: GPUBindGroup,
+ environment: Environment3DOptions,
+ timeSeconds = 0,
+ frameIndex = 0,
+ ): void {
+ const resolved = resolveEnvironment(environment);
+ if (!resolved.volumetricFog) return;
+ const fullWidth = sourceColorTex.width;
+ const fullHeight = sourceColorTex.height;
+ const halfWidth = Math.max(1, Math.ceil(fullWidth / 2));
+ const halfHeight = Math.max(1, Math.ceil(fullHeight / 2));
+ const halfFogTex = this.attachment(
+ "Renderer3DVolumetricFogHalf",
+ halfWidth,
+ halfHeight,
+ {
+ format: "rgba16float",
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
+ },
+ );
+ this.volumetricFogPipeline ??= new VolumetricFogPipeline(
+ this.device,
+ this.format,
+ );
+ this.volumetricFogPipeline.render({
+ encoder,
+ sourceColorTex,
+ linearDepthTex,
+ halfFogTex,
+ destView,
+ cameraBindGroup,
+ lightsBindGroup,
+ fog: resolved.volumetricFog,
+ fullWidth,
+ fullHeight,
+ halfWidth,
+ halfHeight,
+ timeSeconds,
+ frameIndex,
+ });
+ }
+
/** Uploads raw `InstanceData` float buffer to a GPU storage buffer from the instance pool. */
uploadInstanceData(data: Float32Array): GPUBuffer {
return this.instancePool.getBuffer(this.device, data);
@@ -2102,10 +2343,12 @@ export class Renderer3D {
this.attachments.clear();
this.bloomPipeline?.destroy();
this.outlinePipeline?.destroy();
+ this.volumetricFogPipeline?.destroy();
this.depthTexture?.destroy();
for (const texture of this.shadowTextures.values()) texture.destroy();
this.shadowTextures.clear();
this.placeholderShadowTexture?.destroy();
+ this.placeholderEnvTexture?.destroy();
this.linearDepthTexture?.destroy();
this.msaaColorTexture?.destroy();
this.msaaLinearDepthTexture?.destroy();
diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts
index fba87876..2da89c82 100644
--- a/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts
+++ b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts
@@ -3,6 +3,7 @@ import { Light3D } from "../math3d/light3d.js";
import type { Mat4 } from "../math3d/matrix4.js";
import {
type Aabb,
+ fitPointShadowFrusta,
fitShadowFrustum,
transformAabb,
} from "./shadow-frustum.js";
@@ -100,7 +101,7 @@ describe("Light3D physical block", () => {
expect(agx[8 + 8 * 16 + 3]).toBeCloseTo(2); // exposure +1 stop
});
- it("indexes the shadow light among non-ambient lights", () => {
+ it("indexes the shadow light among non-ambient lights (including point and rect)", () => {
const lights = [
{ type: "ambient" as const },
{ type: "point" as const },
@@ -109,6 +110,57 @@ describe("Light3D physical block", () => {
expect(Light3D.shadowCasterIndex(lights)).toBe(1);
expect(
Light3D.shadowCasterIndex([{ type: "point" as const, castShadow: true }]),
- ).toBe(-1);
+ ).toBe(0);
+ expect(
+ Light3D.shadowCasterIndex([{ type: "rect" as const, castShadow: true }]),
+ ).toBe(0);
+ });
+
+ it("packs rect area light orthonormal basis and half-dimensions into direct lights", () => {
+ const packed = Light3D.packDirectLights([
+ {
+ type: "rect",
+ x: 100,
+ y: 200,
+ z: -300,
+ targetX: 100,
+ targetY: 200,
+ targetZ: 0,
+ width: 400,
+ height: 160,
+ twoSided: true,
+ },
+ ]);
+ expect(packed[3]).toBe(4); // type = 4 (rect)
+ expect(packed[15]).toBe(1); // twoSided = 1
+ expect(packed[43]).toBeCloseTo(200); // halfWidth = 200
+ expect(packed[47]).toBeCloseTo(80); // halfHeight = 80
+ });
+
+ it("fits 6 cube-face frusta for omnidirectional point light shadows", () => {
+ const casterPosX = box(300, 0, 0, 40);
+ const casterNegZ = box(0, 0, -400, 40);
+ const frusta = fitPointShadowFrusta(
+ [0, 0, 0],
+ 1000,
+ [casterPosX, casterNegZ],
+ [casterPosX, casterNegZ],
+ 1024,
+ );
+ expect(frusta).not.toBeNull();
+ if (!frusta) throw new Error("expected frusta");
+ expect(frusta.viewProjs).toHaveLength(6);
+ // Face 0 (+X) sees casterPosX at the center of its frustum
+ const [x0, y0, z0] = project(frusta.viewProjs[0], [300, 0, 0]);
+ expect(Math.abs(x0)).toBeLessThan(1e-5);
+ expect(Math.abs(y0)).toBeLessThan(1e-5);
+ expect(z0).toBeGreaterThan(0);
+ expect(z0).toBeLessThan(1);
+ // Face 5 (-Z) sees casterNegZ at the center of its frustum
+ const [x5, y5, z5] = project(frusta.viewProjs[5], [0, 0, -400]);
+ expect(Math.abs(x5)).toBeLessThan(1e-5);
+ expect(Math.abs(y5)).toBeLessThan(1e-5);
+ expect(z5).toBeGreaterThan(0);
+ expect(z5).toBeLessThan(1);
});
});
diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts
index a22a0674..50e99ab0 100644
--- a/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts
+++ b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts
@@ -12,13 +12,15 @@ export interface Aabb {
}
export interface ShadowLight {
- type: "directional" | "spot";
- /** World position (spot); directional lights only use `direction`. */
+ type: "directional" | "spot" | "rect";
+ /** World position (spot / rect); directional lights only use `direction`. */
position: Vec3;
/** Unit direction the light travels. */
direction: Vec3;
- /** Spot cone half-angle in degrees. */
+ /** Spot cone half-angle in degrees (for rect lights, defaults to 65). */
angle?: number;
+ /** Maximum range of the light (world px). */
+ radius?: number;
}
export interface ShadowFrustum {
@@ -27,6 +29,14 @@ export interface ShadowFrustum {
texelWorldSize: number;
}
+export interface PointShadowFrusta {
+ /** 6 viewProj matrices in order: +X, -X, +Y, -Y, +Z, -Z. */
+ viewProjs: [Mat4, Mat4, Mat4, Mat4, Mat4, Mat4];
+ near: number;
+ far: number;
+ texelWorldSize: number;
+}
+
export function aabbCorners(box: Aabb): Vec3[] {
const { min, max } = box;
return [
@@ -134,23 +144,29 @@ export function fitShadowFrustum(
const eye = light.position;
const view = Matrix4Math.lookAt(eye, Vector3Math.add(eye, dir), upFor(dir));
- let near = Infinity;
- let far = 0;
+ let casterNear = Infinity;
let casterDist = 0;
+ for (const p of casterPts) {
+ const z = transformPoint(view, p)[2];
+ if (z > 0) {
+ casterNear = Math.min(casterNear, z);
+ casterDist = Math.max(casterDist, z);
+ }
+ }
+ let far = casterDist;
for (const p of allPts) {
const z = transformPoint(view, p)[2];
if (z > 0) {
- near = Math.min(near, z);
far = Math.max(far, z);
}
}
- for (const p of casterPts) {
- casterDist = Math.max(casterDist, transformPoint(view, p)[2]);
- }
- if (!Number.isFinite(near) || far <= 0) return null;
- near = Math.max(1, near * 0.9);
- far = far * 1.05 + 1;
- const fovRad = (Math.min(170, 2 * (light.angle ?? 45) + 4) * Math.PI) / 180;
+ if (!Number.isFinite(casterNear) || far <= 0) return null;
+ const near = Math.max(2, casterNear * 0.85);
+ const volumetricReach = Math.min(light.radius ?? 2600, casterDist * 2.5);
+ far = Math.max(far * 1.05 + 1, volumetricReach);
+ const defaultAngle = light.type === "rect" ? 65 : 45;
+ const fovRad =
+ (Math.min(170, 2 * (light.angle ?? defaultAngle) + 4) * Math.PI) / 180;
const proj = Matrix4Math.perspective(fovRad, 1, near, far);
return {
viewProj: Matrix4Math.multiply(proj, view),
@@ -158,3 +174,74 @@ export function fitShadowFrustum(
(2 * Math.tan(fovRad / 2) * Math.max(casterDist, near)) / mapSize,
};
}
+
+const POINT_CUBE_FACES: ReadonlyArray<{ dir: Vec3; up: Vec3 }> = [
+ { dir: [1, 0, 0], up: [0, -1, 0] }, // 0: +X
+ { dir: [-1, 0, 0], up: [0, -1, 0] }, // 1: -X
+ { dir: [0, 1, 0], up: [0, 0, 1] }, // 2: +Y
+ { dir: [0, -1, 0], up: [0, 0, -1] }, // 3: -Y
+ { dir: [0, 0, 1], up: [0, -1, 0] }, // 4: +Z
+ { dir: [0, 0, -1], up: [0, -1, 0] }, // 5: -Z
+];
+
+/**
+ * Builds the 6-face 90-degree perspective frusta (+X, -X, +Y, -Y, +Z, -Z) for
+ * an omnidirectional shadow-casting point light.
+ */
+export function fitPointShadowFrusta(
+ position: Vec3,
+ radius: number,
+ casters: Aabb[],
+ receivers: Aabb[],
+ mapSize: number,
+): PointShadowFrusta | null {
+ if (casters.length === 0) return null;
+ const casterPts = casters.flatMap(aabbCorners);
+ const allPts = [...casterPts, ...receivers.flatMap(aabbCorners)];
+
+ let minDist = Infinity;
+ let maxDist = 0;
+ let maxCasterDist = 0;
+ for (const p of allPts) {
+ const dx = p[0] - position[0];
+ const dy = p[1] - position[1];
+ const dz = p[2] - position[2];
+ const dist = Math.hypot(dx, dy, dz);
+ const maxAxis = Math.max(Math.abs(dx), Math.abs(dy), Math.abs(dz));
+ if (maxAxis > 1e-3) {
+ minDist = Math.min(minDist, maxAxis);
+ }
+ maxDist = Math.max(maxDist, dist);
+ }
+ for (const p of casterPts) {
+ const dx = p[0] - position[0];
+ const dy = p[1] - position[1];
+ const dz = p[2] - position[2];
+ maxCasterDist = Math.max(maxCasterDist, Math.hypot(dx, dy, dz));
+ }
+ if (maxDist <= 0) return null;
+ const near = Math.max(
+ 2,
+ Math.min(25, Number.isFinite(minDist) ? minDist * 0.5 : 8),
+ );
+ const far = Math.max(
+ near + 10,
+ Math.min(Math.max(radius, 100), maxDist * 1.08 + 10),
+ );
+ const proj = Matrix4Math.perspective(Math.PI / 2, 1, near, far);
+ const viewProjs = POINT_CUBE_FACES.map(({ dir, up }) => {
+ const view = Matrix4Math.lookAt(
+ position,
+ Vector3Math.add(position, dir),
+ up,
+ );
+ return Matrix4Math.multiply(proj, view);
+ }) as [Mat4, Mat4, Mat4, Mat4, Mat4, Mat4];
+
+ return {
+ viewProjs,
+ near,
+ far,
+ texelWorldSize: (2 * Math.max(maxCasterDist, near)) / mapSize,
+ };
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/surface-params.test.ts b/packages/webgpu-renderers/src/renderer3d/surface-params.test.ts
new file mode 100644
index 00000000..c1859111
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/surface-params.test.ts
@@ -0,0 +1,73 @@
+import { describe, expect, it } from "vitest";
+import { packSurfaceParams, SURFACE_PARAMS_FLOATS } from "./surface-params.js";
+
+function pack(params: Parameters<typeof packSurfaceParams>[0]): Float32Array {
+ const out = new Float32Array(SURFACE_PARAMS_FLOATS + 4).fill(-7);
+ packSurfaceParams(params, out, 2);
+ return out;
+}
+
+describe("packSurfaceParams Substrate lobes", () => {
+ it("packs neutral lobes when nothing is set and stays inside its slot", () => {
+ const out = pack({});
+ expect(out[0]).toBe(-7);
+ expect(out[1]).toBe(-7);
+ expect(out[2 + SURFACE_PARAMS_FLOATS]).toBe(-7);
+ const at = (i: number) => out[2 + i];
+ expect(at(20)).toBe(0); // clearcoat
+ expect(at(21)).toBeCloseTo(0.08); // clearcoatRoughness
+ expect(at(22)).toBe(0); // anisotropy
+ expect(at(24)).toBe(0); // subsurface
+ expect(at(28)).toBe(0); // sheen
+ expect(at(32)).toBe(0); // iridescence
+ expect(at(35)).toBe(0); // triplanar off
+ });
+
+ it("packs every lobe, clamping out-of-range input", () => {
+ const out = pack({
+ clearcoat: 2,
+ clearcoatRoughness: 0,
+ anisotropy: -3,
+ anisotropyRotation: 90,
+ subsurface: 0.5,
+ subsurfaceColor: [0.1, 0.2, 0.3],
+ sheen: 0.75,
+ sheenColor: "#ff0000",
+ sheenRoughness: 0,
+ iridescence: { intensity: 0.4, thickness: 5000, ior: 1.6 },
+ triplanar: { scale: 2, sharpness: 40 },
+ });
+ const at = (i: number) => out[2 + i] ?? Number.NaN;
+ expect(at(20)).toBe(1);
+ expect(at(21)).toBeCloseTo(0.03);
+ expect(at(22)).toBe(-1);
+ expect(at(23)).toBeCloseTo(Math.PI / 2);
+ expect([at(24), at(25), at(26), at(27)].map((v) => +v.toFixed(3))).toEqual([
+ 0.5, 0.1, 0.2, 0.3,
+ ]);
+ expect([at(28), at(29), at(30), at(31)]).toEqual([0.75, 1, 0, 0]);
+ expect(at(18)).toBeCloseTo(0.05); // sheenRoughness floor
+ expect(at(19)).toBe(16); // triplanar sharpness ceiling
+ expect(at(32)).toBeCloseTo(0.4);
+ expect(at(33)).toBe(1200);
+ expect(at(34)).toBeCloseTo(1.6);
+ expect(at(35)).toBeCloseTo(2 / 256);
+ });
+
+ it("lets flat props override the object forms", () => {
+ const out = pack({
+ iridescence: { intensity: 1, thickness: 300 },
+ iridescenceThickness: 650,
+ iridescenceIor: 1.2,
+ triplanar: true,
+ triplanarScale: 4,
+ triplanarSharpness: 2,
+ });
+ const at = (i: number) => out[2 + i];
+ expect(at(32)).toBe(1);
+ expect(at(33)).toBe(650);
+ expect(at(34)).toBeCloseTo(1.2);
+ expect(at(35)).toBeCloseTo(4 / 256);
+ expect(at(19)).toBe(2);
+ });
+});
diff --git a/packages/webgpu-renderers/src/renderer3d/surface-params.ts b/packages/webgpu-renderers/src/renderer3d/surface-params.ts
index 3c69cbce..c183b7b7 100644
--- a/packages/webgpu-renderers/src/renderer3d/surface-params.ts
+++ b/packages/webgpu-renderers/src/renderer3d/surface-params.ts
@@ -2,6 +2,22 @@ import type { ToonShading } from "@framefields/core";
import { parseColor } from "../color.js";
import type { Color } from "../renderer2d/index.js";
+export interface IridescenceOptions {
+ /** Thin-film interference blend intensity 0..1 (default 1 when object is passed). */
+ intensity?: number;
+ /** Thin-film thickness in nanometers 100..1000 (default 400). */
+ thickness?: number;
+ /** Thin-film refractive index 1.05..2.5 (default 1.3). */
+ ior?: number;
+}
+
+export interface TriplanarOptions {
+ /** Tiling scale (repeats per 256 world px, or raw UV/px if < 0.05; default 1). */
+ scale?: number;
+ /** Normal blend exponent across projection axes 1..16 (default 4). */
+ sharpness?: number;
+}
+
/** Physically based surface inputs every 3D draw (quad, text, mesh) accepts. */
export interface SurfaceParams {
toon?: ToonShading;
@@ -16,10 +32,103 @@ export interface SurfaceParams {
emissiveIntensity?: number;
/** Casts into a `castShadow` light's shadow map (meshes default on, quads off). */
castShadow?: boolean;
+ /** Secondary dielectric clearcoat/varnish lobe 0..1 (car paint, wet coats, polished lacquer). */
+ clearcoat?: number;
+ /** Roughness of the clearcoat layer 0..1 (default 0.08). */
+ clearcoatRoughness?: number;
+ /** Anisotropic GGX highlight stretch -1..1 (brushed metal, satin, hair; 0 = isotropic). */
+ anisotropy?: number;
+ /** Brush tangent rotation in degrees around the surface normal (default 0). */
+ anisotropyRotation?: number;
+ /** Subsurface wrap diffuse and translucent back-scattering 0..1 (wax, jade, skin, foliage). */
+ subsurface?: number;
+ /** Subsurface scatter & transmission tint color (default "#ff6b4a"). */
+ subsurfaceColor?: string | Color | [number, number, number];
+ /** Charlie cloth / velvet grazing retro-reflection lobe 0..1. */
+ sheen?: number;
+ /** Cloth sheen tint color (default "#ffffff"). */
+ sheenColor?: string | Color | [number, number, number];
+ /** Cloth sheen microfiber roughness 0.05..1 (default 0.5). */
+ sheenRoughness?: number;
+ /** Thin-film spectral iridescence 0..1 or `{ intensity, thickness, ior }`. */
+ iridescence?: number | IridescenceOptions;
+ /** Thin-film thickness in nanometers 100..1000 (default 400). */
+ iridescenceThickness?: number;
+ /** Thin-film refractive index 1.05..2.5 (default 1.3). */
+ iridescenceIor?: number;
+ /** Seamless world-space triplanar texture projection for UV-less or procedural meshes. */
+ triplanar?: boolean | TriplanarOptions;
+ /** World-space triplanar tiling scale (repeats per 256 world px; >0 enables triplanar). */
+ triplanarScale?: number;
+ /** World-space triplanar axis blend sharpness 1..16 (default 4). */
+ triplanarSharpness?: number;
+}
+
+/** Shared model uniforms: PBR, emissive, three art-direction vec4s, and four Substrate multi-lobe vec4s. */
+export const SURFACE_PARAMS_FLOATS = 36;
+
+function parseRgbTriplet(
+ input: string | Color | [number, number, number] | undefined,
+ fallback: [number, number, number],
+): [number, number, number] {
+ if (input === undefined || input === null) return fallback;
+ if (Array.isArray(input)) {
+ return [
+ input[0] ?? fallback[0],
+ input[1] ?? fallback[1],
+ input[2] ?? fallback[2],
+ ];
+ }
+ const c = parseColor(input as string | Color);
+ return [c.r * c.a, c.g * c.a, c.b * c.a];
}
-/** Shared model uniforms: PBR, emissive, and three art-direction vec4s. */
-export const SURFACE_PARAMS_FLOATS = 20;
+function resolveTriplanarParams(data: SurfaceParams): [number, number] {
+ const tri = data.triplanar;
+ const rawScale =
+ data.triplanarScale ??
+ (typeof tri === "object" && tri !== null
+ ? (tri.scale ?? 1)
+ : tri === true
+ ? 1
+ : 0);
+ const sharpness = Math.max(
+ 1,
+ Math.min(
+ 16,
+ data.triplanarSharpness ??
+ (typeof tri === "object" && tri !== null ? (tri.sharpness ?? 4) : 4),
+ ),
+ );
+ if (rawScale <= 0) return [0, sharpness];
+ const uvFreq = rawScale < 0.05 ? rawScale : rawScale / 256;
+ return [uvFreq, sharpness];
+}
+
+function resolveIridescenceParams(
+ data: SurfaceParams,
+): [number, number, number] {
+ const irid = data.iridescence;
+ const intensity =
+ typeof irid === "number"
+ ? irid
+ : irid && typeof irid === "object"
+ ? (irid.intensity ?? 1)
+ : 0;
+ const thickness =
+ data.iridescenceThickness ??
+ (irid && typeof irid === "object" ? irid.thickness : undefined) ??
+ 400;
+ const ior =
+ data.iridescenceIor ??
+ (irid && typeof irid === "object" ? irid.ior : undefined) ??
+ 1.3;
+ return [
+ Math.max(0, Math.min(1, intensity)),
+ Math.max(100, Math.min(1200, thickness)),
+ Math.max(1.05, Math.min(2.5, ior)),
+ ];
+}
export function packSurfaceParams(
data: SurfaceParams,
@@ -31,6 +140,18 @@ export function packSurfaceParams(
out[offset + 1] = data.metallic ?? 0;
out[offset + 2] = 0;
out[offset + 3] = 0;
+
+ const e = data.emissive;
+ if (e === undefined || e === null) {
+ out.fill(0, offset + 4, offset + 8);
+ } else {
+ const rgb = parseRgbTriplet(e, [0, 0, 0]);
+ out[offset + 4] = rgb[0];
+ out[offset + 5] = rgb[1];
+ out[offset + 6] = rgb[2];
+ out[offset + 7] = data.emissiveIntensity ?? 1;
+ }
+
const shadow = data.toon?.shadowColor
? parseColor(data.toon.shadowColor)
: null;
@@ -44,24 +165,48 @@ export function packSurfaceParams(
[light.r, light.g, light.b, data.toon?.threshold ?? 0.075],
offset + 12,
);
+ const [triplanarFreq, triplanarSharpness] = resolveTriplanarParams(data);
+ const sheenRoughness = Math.max(
+ 0.05,
+ Math.min(1, data.sheenRoughness ?? 0.5),
+ );
out.set(
- [data.toon?.softness ?? 0.25, data.outline === false ? 0 : 1, 0, 0],
+ [
+ data.toon?.softness ?? 0.25,
+ data.outline === false ? 0 : 1,
+ sheenRoughness,
+ triplanarSharpness,
+ ],
offset + 16,
);
- const e = data.emissive;
- if (e === undefined || e === null) {
- out.fill(0, offset + 4, offset + 8);
- return;
- }
- let rgb: [number, number, number];
- if (Array.isArray(e)) {
- rgb = [e[0], e[1], e[2]];
- } else {
- const c = parseColor(e as string | Color);
- rgb = [c.r * c.a, c.g * c.a, c.b * c.a];
- }
- out[offset + 4] = rgb[0];
- out[offset + 5] = rgb[1];
- out[offset + 6] = rgb[2];
- out[offset + 7] = data.emissiveIntensity ?? 1;
+
+ // offset + 20: substrateClearcoatAniso [clearcoat, clearcoatRoughness, anisotropy, anisotropyRotationRad]
+ out[offset + 20] = Math.max(0, Math.min(1, data.clearcoat ?? 0));
+ out[offset + 21] = Math.max(
+ 0.03,
+ Math.min(1, data.clearcoatRoughness ?? 0.08),
+ );
+ out[offset + 22] = Math.max(-1, Math.min(1, data.anisotropy ?? 0));
+ out[offset + 23] = ((data.anisotropyRotation ?? 0) * Math.PI) / 180;
+
+ // offset + 24: substrateSubsurface [subsurface, r, g, b]
+ const sssRgb = parseRgbTriplet(data.subsurfaceColor, [1.0, 0.42, 0.29]);
+ out[offset + 24] = Math.max(0, Math.min(1, data.subsurface ?? 0));
+ out[offset + 25] = sssRgb[0];
+ out[offset + 26] = sssRgb[1];
+ out[offset + 27] = sssRgb[2];
+
+ // offset + 28: substrateSheen [sheen, r, g, b]
+ const sheenRgb = parseRgbTriplet(data.sheenColor, [1, 1, 1]);
+ out[offset + 28] = Math.max(0, Math.min(1, data.sheen ?? 0));
+ out[offset + 29] = sheenRgb[0];
+ out[offset + 30] = sheenRgb[1];
+ out[offset + 31] = sheenRgb[2];
+
+ // offset + 32: substrateIridescence [intensity, thicknessNm, ior, triplanarUvFreq]
+ const [iridInt, iridThick, iridIor] = resolveIridescenceParams(data);
+ out[offset + 32] = iridInt;
+ out[offset + 33] = iridThick;
+ out[offset + 34] = iridIor;
+ out[offset + 35] = triplanarFreq;
}
diff --git a/packages/webgpu-renderers/src/renderer3d/volumetric-fog-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/volumetric-fog-pipeline.ts
new file mode 100644
index 00000000..f9e41c77
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/volumetric-fog-pipeline.ts
@@ -0,0 +1,524 @@
+/**
+ * Half-resolution Volumetric Fog & Shadowed Light-Shaft (God Ray) pipeline
+ * with depth-aware bilateral upsampling.
+ *
+ * Reuses the 3D scene's @group(0) CameraUniforms and @group(3) Lights/ShadowMap
+ * bind groups directly so every directional, spot, point, and rect light —
+ * including multi-light PCF / omnidirectional shadow maps — carves volumetric
+ * shafts through the participating medium with zero extra shadow passes.
+ */
+
+import type { ResolvedVolumetricFog } from "../math3d/environment3d.js";
+import { BufferPool } from "../renderer2d/buffer-pool.js";
+import { lighting3dWgsl, lightsBindingsWgsl } from "../shaders/lighting3d.js";
+import { createLightsLayout } from "./quad3d-pipeline.js";
+
+const VOLUMETRIC_UNIFORM_FLOATS = 24;
+const VOLUMETRIC_UNIFORM_BYTES = VOLUMETRIC_UNIFORM_FLOATS * 4;
+
+const volumetricFogWgsl = `
+${lighting3dWgsl}
+
+struct VolumetricFogUniforms {
+ fogColorDensity : vec4<f32>, // rgb = linear fog scatter color, w = base density
+ scatterParams : vec4<f32>, // x = anisotropy g, y = heightFalloff, z = baseHeight, w = maxDistance
+ marchParams : vec4<f32>, // x = steps, y = lightIntensity, z = ambientIntensity, w = frameIndex
+ noiseParams : vec4<f32>, // x = noiseScale, y = noiseStrength, z = octaves, w = timeSeconds
+ windVelocity : vec4<f32>, // xyz = wind speed (px/s), w = depthSigma
+ resParams : vec4<f32>, // xy = halfRes (w, h), zw = fullRes (w, h)
+};
+
+@group(0) @binding(0) var<uniform> cam : CameraUniforms;
+@group(1) @binding(0) var<uniform> vfog : VolumetricFogUniforms;
+@group(1) @binding(1) var linearDepthTex : texture_2d<f32>;
+@group(1) @binding(2) var sceneColorTex : texture_2d<f32>;
+@group(1) @binding(3) var halfFogTex : texture_2d<f32>;
+@group(1) @binding(4) var linearSampler : sampler;
+${lightsBindingsWgsl}
+
+struct VsOut {
+ @builtin(position) pos : vec4<f32>,
+ @location(0) uv : vec2<f32>,
+};
+
+@vertex
+fn vs_fullscreen(@builtin(vertex_index) vid : u32) -> VsOut {
+ var pos = array<vec2<f32>, 3>(
+ vec2<f32>(-1.0, -1.0),
+ vec2<f32>(3.0, -1.0),
+ vec2<f32>(-1.0, 3.0)
+ );
+ var out : VsOut;
+ let p = pos[vid];
+ out.pos = vec4<f32>(p, 0.0, 1.0);
+ out.uv = vec2<f32>(p.x * 0.5 + 0.5, 0.5 - p.y * 0.5);
+ return out;
+}
+
+// Jorge Jimenez's Interleaved Gradient Noise for jittering raymarch steps.
+fn interleavedGradientNoise(pixel : vec2<f32>, frame : f32) -> f32 {
+ let p = pixel + 5.588238 * (frame % 64.0);
+ return fract(52.9829189 * fract(0.06711056 * p.x + 0.00583715 * p.y));
+}
+
+fn hash31(p3 : vec3<f32>) -> f32 {
+ var p = fract(p3 * 0.1031);
+ p += dot(p, p.zyx + 31.32);
+ return fract((p.x + p.y) * p.z);
+}
+
+fn valueNoise3D(x : vec3<f32>) -> f32 {
+ let i = floor(x);
+ let f = fract(x);
+ let u = f * f * (3.0 - 2.0 * f);
+ let n000 = hash31(i + vec3<f32>(0.0, 0.0, 0.0));
+ let n100 = hash31(i + vec3<f32>(1.0, 0.0, 0.0));
+ let n010 = hash31(i + vec3<f32>(0.0, 1.0, 0.0));
+ let n110 = hash31(i + vec3<f32>(1.0, 1.0, 0.0));
+ let n001 = hash31(i + vec3<f32>(0.0, 0.0, 1.0));
+ let n101 = hash31(i + vec3<f32>(1.0, 0.0, 1.0));
+ let n011 = hash31(i + vec3<f32>(0.0, 1.0, 1.0));
+ let n111 = hash31(i + vec3<f32>(1.0, 1.0, 1.0));
+ let nx00 = mix(n000, n100, u.x);
+ let nx10 = mix(n010, n110, u.x);
+ let nx01 = mix(n001, n101, u.x);
+ let nx11 = mix(n101, n111, u.x);
+ let nxy0 = mix(nx00, nx10, u.y);
+ let nxy1 = mix(nx01, nx11, u.y);
+ return mix(nxy0, nxy1, u.z);
+}
+
+fn fbmNoise3D(worldPos : vec3<f32>, scale : f32, octaves : i32) -> f32 {
+ var q = (worldPos - vfog.windVelocity.xyz * vfog.noiseParams.w) * scale;
+ var val = 0.0;
+ var amp = 0.55;
+ var norm = 0.0;
+ for (var o = 0; o < 3; o++) {
+ if (o >= octaves) {
+ break;
+ }
+ val += amp * valueNoise3D(q);
+ norm += amp;
+ q = q * 2.03 + vec3<f32>(17.1, 31.7, 11.3);
+ amp *= 0.5;
+ }
+ return select(0.5, val / max(norm, 1e-4), norm > 0.0);
+}
+
+// Henyey-Greenstein phase function P(cosTheta, g).
+fn henyeyGreenstein(cosTheta : f32, g : f32) -> f32 {
+ let g2 = g * g;
+ let denom = max(1e-4, 1.0 + g2 - 2.0 * g * cosTheta);
+ return (1.0 - g2) / (4.0 * PI * denom * sqrt(denom));
+}
+
+fn reconstructRayDir(uv : vec2<f32>) -> vec3<f32> {
+ let ndc = vec2<f32>(uv.x * 2.0 - 1.0, 1.0 - uv.y * 2.0);
+ let worldH = cam.invViewProjMatrix * vec4<f32>(ndc, 0.5, 1.0);
+ let worldPt = worldH.xyz / max(abs(worldH.w), 1e-6);
+ return normalize(worldPt - cam.cameraPos.xyz);
+}
+
+@fragment
+fn fs_raymarch(in : VsOut) -> @location(0) vec4<f32> {
+ let baseDensity = vfog.fogColorDensity.w;
+ if (baseDensity <= 1e-6) {
+ return vec4<f32>(0.0, 0.0, 0.0, 1.0);
+ }
+
+ let rayOrigin = cam.cameraPos.xyz;
+ let rayDir = reconstructRayDir(in.uv);
+ let camFwd = normalize(cam.cameraDir.xyz);
+ let cosView = max(0.1, dot(rayDir, camFwd));
+
+ let rawDepth = textureSampleLevel(linearDepthTex, linearSampler, in.uv, 0.0).r;
+ let maxDist = vfog.scatterParams.w;
+ let surfaceDist = select(maxDist, rawDepth / cosView, rawDepth > 0.05);
+ let marchDist = clamp(surfaceDist, 0.0, maxDist);
+ if (marchDist <= 1.0) {
+ return vec4<f32>(0.0, 0.0, 0.0, 1.0);
+ }
+
+ let steps = clamp(i32(vfog.marchParams.x), 8, 64);
+ let dt = marchDist / f32(steps);
+ let jitter = interleavedGradientNoise(in.pos.xy, vfog.marchParams.w);
+
+ let g = vfog.scatterParams.x;
+ let heightFalloff = vfog.scatterParams.y;
+ let baseHeight = vfog.scatterParams.z;
+ let lightScale = vfog.marchParams.y;
+ let ambientScale = vfog.marchParams.z;
+ let noiseScale = vfog.noiseParams.x;
+ let noiseStrength = vfog.noiseParams.y;
+ let noiseOctaves = i32(vfog.noiseParams.z);
+ let fogTint = vfog.fogColorDensity.rgb;
+ let ambientIrr = (lights.envHorizon.rgb + lights.ambientLinear.rgb + srgbToLinear(lights.ambientColor.rgb) * 0.35) * ambientScale;
+
+ // Scale user density (e.g. 0.02) into world-px extinction (1 / px).
+ let sigmaBase = baseDensity * 0.012;
+ let numLights = i32(lights.params.x);
+
+ var transmittance = 1.0;
+ var accumLight = vec3<f32>(0.0);
+
+ for (var s = 0; s < 64; s++) {
+ if (s >= steps || transmittance < 0.01) {
+ break;
+ }
+ let t = (f32(s) + jitter) * dt;
+ let samplePos = rayOrigin + rayDir * t;
+
+ // Exponential height falloff (in Y-down coordinates, ground is at +baseHeight;
+ // higher altitude above ground means smaller Y, i.e. baseHeight - samplePos.y > 0).
+ let altitudeAboveGround = max(0.0, baseHeight - samplePos.y);
+ let heightFactor = exp(-heightFalloff * altitudeAboveGround);
+
+ var densityMod = heightFactor;
+ if (noiseStrength > 0.001 && noiseOctaves > 0) {
+ let n = fbmNoise3D(samplePos, noiseScale, noiseOctaves);
+ densityMod *= mix(1.0, clamp(n * 1.65, 0.0, 2.0), noiseStrength);
+ }
+
+ let sigmaT = sigmaBase * densityMod;
+ if (sigmaT <= 1e-6) {
+ continue;
+ }
+
+ var stepScatter = ambientIrr;
+ if (lightScale > 0.001) {
+ for (var i = 0; i < numLights; i++) {
+ let ls = sampleLight(i, samplePos);
+ if (!ls.valid) {
+ continue;
+ }
+ let vis = volumetricShadowFactor(samplePos, i);
+ if (vis <= 0.001) {
+ continue;
+ }
+ let light = directLights[i];
+ // Forward scattering angle: incoming light (-ls.L) vs outgoing view ray to eye (-rayDir),
+ // i.e. dot(-ls.L, -rayDir) == dot(rayDir, ls.L).
+ let cosTheta = dot(rayDir, ls.L);
+ let phase = henyeyGreenstein(cosTheta, g);
+ let lightRad = srgbToLinear(light.colorIntensity.rgb) * light.colorIntensity.w * ls.attenuation;
+ stepScatter += lightRad * (phase * vis * lightScale * 4.0 * PI);
+ }
+ }
+
+ let stepTrans = exp(-sigmaT * dt);
+ let integWeight = 1.0 - stepTrans;
+ accumLight += transmittance * stepScatter * fogTint * integWeight;
+ transmittance *= stepTrans;
+ }
+
+ return vec4<f32>(accumLight, clamp(transmittance, 0.0, 1.0));
+}
+
+@fragment
+fn fs_upsample(in : VsOut) -> @location(0) vec4<f32> {
+ let baseSrgb = textureSampleLevel(sceneColorTex, linearSampler, in.uv, 0.0);
+ let centerDepth = textureSampleLevel(linearDepthTex, linearSampler, in.uv, 0.0).r;
+ let halfTexel = 1.0 / max(vfog.resParams.xy, vec2<f32>(1.0));
+
+ let depthSigma = max(10.0, vfog.windVelocity.w);
+ var sumFog = vec4<f32>(0.0);
+ var sumWeight = 0.0;
+
+ for (var oy = -1; oy <= 1; oy++) {
+ let wy = select(1.0, 2.0, oy == 0);
+ for (var ox = -1; ox <= 1; ox++) {
+ let wx = select(1.0, 2.0, ox == 0);
+ let spatialW = (wx * wy) * 0.0625;
+ let offset = vec2<f32>(f32(ox), f32(oy)) * 0.75 * halfTexel;
+ let sampleUv = clamp(in.uv + offset, vec2<f32>(0.001), vec2<f32>(0.999));
+ let fogSample = textureSampleLevel(halfFogTex, linearSampler, sampleUv, 0.0);
+ let sampleDepth = textureSampleLevel(linearDepthTex, linearSampler, sampleUv, 0.0).r;
+ let depthDiff = abs(centerDepth - sampleDepth);
+ let wDepth = exp(-depthDiff / depthSigma);
+ let w = spatialW * max(wDepth, 1e-3);
+ sumFog += fogSample * w;
+ sumWeight += w;
+ }
+ }
+
+ let fog = select(vec4<f32>(0.0, 0.0, 0.0, 1.0), sumFog / sumWeight, sumWeight > 1e-5);
+ let transmittance = clamp(fog.a, 0.0, 1.0);
+ let fogAlpha = clamp(1.0 - transmittance, 0.0, 1.0);
+ let inScattered = max(vec3<f32>(0.0), fog.rgb);
+ let straightFogLinear = inScattered / max(fogAlpha, 1e-4);
+ let mappedFogLinear = select(
+ clamp(straightFogLinear, vec3<f32>(0.0), vec3<f32>(1.0)),
+ toneMap(straightFogLinear),
+ lights.params.z > 0.5
+ );
+ let premulFogSrgb = linearToSrgb(mappedFogLinear) * fogAlpha;
+ let finalRgb = baseSrgb.rgb * transmittance + premulFogSrgb;
+ let finalAlpha = clamp(baseSrgb.a * transmittance + fogAlpha, 0.0, 1.0);
+ return vec4<f32>(clamp(finalRgb, vec3<f32>(0.0), vec3<f32>(1.0)), finalAlpha);
+}
+`;
+
+export interface VolumetricFogRenderParams {
+ encoder: GPUCommandEncoder;
+ sourceColorTex: GPUTexture;
+ linearDepthTex: GPUTexture;
+ halfFogTex: GPUTexture;
+ destView: GPUTextureView;
+ cameraBindGroup: GPUBindGroup;
+ lightsBindGroup: GPUBindGroup;
+ fog: ResolvedVolumetricFog;
+ fullWidth: number;
+ fullHeight: number;
+ halfWidth: number;
+ halfHeight: number;
+ timeSeconds: number;
+ frameIndex: number;
+}
+
+export class VolumetricFogPipeline {
+ private device: GPUDevice;
+ private raymarchPipeline: GPURenderPipeline;
+ private upsamplePipeline: GPURenderPipeline;
+ private fogBindGroupLayout: GPUBindGroupLayout;
+ private emptyGroup2Layout: GPUBindGroupLayout;
+ private emptyGroup2BindGroup: GPUBindGroup;
+ private linearSampler: GPUSampler;
+ private bufferPool: BufferPool;
+ private uniformScratch = new Float32Array(VOLUMETRIC_UNIFORM_FLOATS);
+
+ constructor(device: GPUDevice, format: GPUTextureFormat = "bgra8unorm") {
+ this.device = device;
+ this.bufferPool = new BufferPool(
+ VOLUMETRIC_UNIFORM_BYTES,
+ GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ );
+
+ this.linearSampler = device.createSampler({
+ magFilter: "linear",
+ minFilter: "linear",
+ addressModeU: "clamp-to-edge",
+ addressModeV: "clamp-to-edge",
+ });
+
+ const cameraLayout = device.createBindGroupLayout({
+ label: "VolumetricFogCameraLayout",
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT,
+ buffer: { type: "uniform" },
+ },
+ ],
+ });
+ const lightsLayout = createLightsLayout(device);
+
+ this.fogBindGroupLayout = device.createBindGroupLayout({
+ label: "VolumetricFogBindGroupLayout",
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.FRAGMENT,
+ buffer: { type: "uniform" },
+ },
+ {
+ binding: 1,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float" },
+ },
+ {
+ binding: 2,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float" },
+ },
+ {
+ binding: 3,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float" },
+ },
+ {
+ binding: 4,
+ visibility: GPUShaderStage.FRAGMENT,
+ sampler: { type: "filtering" },
+ },
+ ],
+ });
+
+ this.emptyGroup2Layout = device.createBindGroupLayout({
+ label: "VolumetricFogEmptyGroup2Layout",
+ entries: [],
+ });
+ this.emptyGroup2BindGroup = device.createBindGroup({
+ layout: this.emptyGroup2Layout,
+ entries: [],
+ });
+
+ const pipelineLayout = device.createPipelineLayout({
+ label: "VolumetricFogPipelineLayout",
+ bindGroupLayouts: [
+ cameraLayout,
+ this.fogBindGroupLayout,
+ this.emptyGroup2Layout,
+ lightsLayout,
+ ],
+ });
+
+ const shaderModule = device.createShaderModule({
+ label: "VolumetricFogShader",
+ code: volumetricFogWgsl,
+ });
+
+ this.raymarchPipeline = device.createRenderPipeline({
+ label: "VolumetricFogRaymarchPipeline",
+ layout: pipelineLayout,
+ vertex: {
+ module: shaderModule,
+ entryPoint: "vs_fullscreen",
+ },
+ fragment: {
+ module: shaderModule,
+ entryPoint: "fs_raymarch",
+ targets: [{ format: "rgba16float" }],
+ },
+ primitive: { topology: "triangle-list" },
+ });
+
+ this.upsamplePipeline = device.createRenderPipeline({
+ label: "VolumetricFogUpsamplePipeline",
+ layout: pipelineLayout,
+ vertex: {
+ module: shaderModule,
+ entryPoint: "vs_fullscreen",
+ },
+ fragment: {
+ module: shaderModule,
+ entryPoint: "fs_upsample",
+ targets: [{ format }],
+ },
+ primitive: { topology: "triangle-list" },
+ });
+ }
+
+ resetPools(): void {
+ this.bufferPool.reset();
+ }
+
+ destroy(): void {
+ this.bufferPool.destroy();
+ }
+
+ render(params: VolumetricFogRenderParams): void {
+ const {
+ encoder,
+ sourceColorTex,
+ linearDepthTex,
+ halfFogTex,
+ destView,
+ cameraBindGroup,
+ lightsBindGroup,
+ fog,
+ fullWidth,
+ fullHeight,
+ halfWidth,
+ halfHeight,
+ timeSeconds,
+ frameIndex,
+ } = params;
+
+ const u = this.uniformScratch;
+ u[0] = fog.color[0];
+ u[1] = fog.color[1];
+ u[2] = fog.color[2];
+ u[3] = fog.density;
+
+ u[4] = fog.anisotropy;
+ u[5] = fog.heightFalloff;
+ u[6] = fog.baseHeight;
+ u[7] = fog.maxDistance;
+
+ u[8] = fog.steps;
+ u[9] = fog.lightIntensity;
+ u[10] = fog.ambientIntensity;
+ u[11] = frameIndex;
+
+ u[12] = fog.noiseScale;
+ u[13] = fog.noiseStrength;
+ u[14] = fog.noiseOctaves;
+ u[15] = timeSeconds;
+
+ u[16] = fog.noiseSpeed[0];
+ u[17] = fog.noiseSpeed[1];
+ u[18] = fog.noiseSpeed[2];
+ u[19] = 85.0; // bilateral depth sigma in world px
+
+ u[20] = halfWidth;
+ u[21] = halfHeight;
+ u[22] = fullWidth;
+ u[23] = fullHeight;
+
+ const uniformBuffer = this.bufferPool.getBuffer(this.device, u);
+
+ const depthView = linearDepthTex.createView();
+ const colorView = sourceColorTex.createView();
+ const halfFogView = halfFogTex.createView();
+
+ // Pass 1: Half-resolution volumetric raymarching into halfFogView
+ const raymarchBindGroup = this.device.createBindGroup({
+ layout: this.fogBindGroupLayout,
+ entries: [
+ { binding: 0, resource: { buffer: uniformBuffer } },
+ { binding: 1, resource: depthView },
+ { binding: 2, resource: colorView },
+ { binding: 3, resource: depthView },
+ { binding: 4, resource: this.linearSampler },
+ ],
+ });
+
+ const rayPass = encoder.beginRenderPass({
+ colorAttachments: [
+ {
+ view: halfFogView,
+ loadOp: "clear",
+ clearValue: { r: 0, g: 0, b: 0, a: 1 },
+ storeOp: "store",
+ },
+ ],
+ });
+ rayPass.setPipeline(this.raymarchPipeline);
+ rayPass.setBindGroup(0, cameraBindGroup);
+ rayPass.setBindGroup(1, raymarchBindGroup);
+ rayPass.setBindGroup(2, this.emptyGroup2BindGroup);
+ rayPass.setBindGroup(3, lightsBindGroup);
+ rayPass.draw(3, 1, 0, 0);
+ rayPass.end();
+
+ // Pass 2: Full-resolution depth-aware bilateral upsample & composite into destView
+ const upsampleBindGroup = this.device.createBindGroup({
+ layout: this.fogBindGroupLayout,
+ entries: [
+ { binding: 0, resource: { buffer: uniformBuffer } },
+ { binding: 1, resource: depthView },
+ { binding: 2, resource: colorView },
+ { binding: 3, resource: halfFogView },
+ { binding: 4, resource: this.linearSampler },
+ ],
+ });
+
+ const upPass = encoder.beginRenderPass({
+ colorAttachments: [
+ {
+ view: destView,
+ loadOp: "clear",
+ clearValue: { r: 0, g: 0, b: 0, a: 0 },
+ storeOp: "store",
+ },
+ ],
+ });
+ upPass.setPipeline(this.upsamplePipeline);
+ upPass.setBindGroup(0, cameraBindGroup);
+ upPass.setBindGroup(1, upsampleBindGroup);
+ upPass.setBindGroup(2, this.emptyGroup2BindGroup);
+ upPass.setBindGroup(3, lightsBindGroup);
+ upPass.draw(3, 1, 0, 0);
+ upPass.end();
+ }
+}
diff --git a/packages/webgpu-renderers/src/shaders/lighting3d.ts b/packages/webgpu-renderers/src/shaders/lighting3d.ts
index 0173462d..61d30187 100644
--- a/packages/webgpu-renderers/src/shaders/lighting3d.ts
+++ b/packages/webgpu-renderers/src/shaders/lighting3d.ts
@@ -38,6 +38,8 @@ struct DirectLight {
shadowMatrix : mat4x4<f32>,
shadowParams : vec4<f32>,
shadowBias : vec4<f32>,
+ rectRight : vec4<f32>,
+ rectUp : vec4<f32>,
};
struct LightsUniforms {
@@ -54,7 +56,8 @@ struct LightsUniforms {
shadowParams : vec4<f32>, // x = enabled, y = shadowed light index, z = softness (texels), w = map size
shadowBias : vec4<f32>, // x = depth bias, y = normal offset (world px), z = darkness 0..1, w = unused
shadowMatrix : mat4x4<f32>,
- floorParams : vec4<f32>, // x = floorY, y = fadeDistance, z = isReflectionPass, w = unused
+ floorParams : vec4<f32>, // x = floorY, y = fadeDistance, z = isReflectionPass, w = environment intensity
+ envSh : array<vec4<f32>, 9>, // HDRI irradiance / PI as L2 SH (rgb), environment space
};
const PI = 3.14159265359;
@@ -185,6 +188,12 @@ fn envRadiance(dirIn: vec3<f32>, rough: f32) -> vec3<f32> {
let s = sin(rot);
let d = normalize(vec3<f32>(c * dirIn.x + s * dirIn.z, dirIn.y, -s * dirIn.x + c * dirIn.z));
let up = -d.y;
+ if (lights.envBottom.w >= 1.5) {
+ // GGX-prefiltered cubemap: mip = perceptual roughness * max mip.
+ let maxMip = max(0.0, lights.envBottom.w - 2.0);
+ let mip = clamp(rough, 0.0, 1.0) * maxMip;
+ return textureSampleLevel(envMap, envSampler, d, mip).rgb * max(0.0, lights.floorParams.w);
+ }
let spread = 0.35 + rough * 0.9;
let tUp = smoothstep(-0.05, spread, up);
let tDown = smoothstep(-0.05, spread, -up);
@@ -201,6 +210,28 @@ fn envRadiance(dirIn: vec3<f32>, rough: f32) -> vec3<f32> {
return col;
}
+/// Cosine-weighted mean radiance arriving around normal n (irradiance / PI):
+/// what a Lambertian surface of albedo c reflects as c * envIrradiance(n).
+fn envIrradiance(nIn: vec3<f32>) -> vec3<f32> {
+ if (lights.envBottom.w < 1.5) {
+ return envRadiance(nIn, 1.0);
+ }
+ let rot = lights.envTop.w;
+ let c = cos(rot);
+ let s = sin(rot);
+ let n = normalize(vec3<f32>(c * nIn.x + s * nIn.z, nIn.y, -s * nIn.x + c * nIn.z));
+ var e = lights.envSh[0].rgb * 0.282095;
+ e += lights.envSh[1].rgb * (0.488603 * n.y);
+ e += lights.envSh[2].rgb * (0.488603 * n.z);
+ e += lights.envSh[3].rgb * (0.488603 * n.x);
+ e += lights.envSh[4].rgb * (1.092548 * n.x * n.y);
+ e += lights.envSh[5].rgb * (1.092548 * n.y * n.z);
+ e += lights.envSh[6].rgb * (0.315392 * (3.0 * n.z * n.z - 1.0));
+ e += lights.envSh[7].rgb * (1.092548 * n.x * n.z);
+ e += lights.envSh[8].rgb * (0.546274 * (n.x * n.x - n.y * n.y));
+ return max(e, vec3<f32>(0.0)) * max(0.0, lights.floorParams.w);
+}
+
// Karis' analytic approximation of the split-sum environment BRDF.
fn envBrdfApprox(f0: vec3<f32>, rough: f32, nDotV: f32) -> vec3<f32> {
let c0 = vec4<f32>(-1.0, -0.0275, -0.572, 0.022);
@@ -220,6 +251,69 @@ var<private> POISSON : array<vec2<f32>, 12> = array<vec2<f32>, 12>(
vec2<f32>(0.896, 0.412), vec2<f32>(-0.322, -0.933), vec2<f32>(-0.792, -0.598),
);
+struct PointCubeProj {
+ uv : vec2<f32>,
+ ndcZ : f32,
+ face : i32,
+ valid : bool,
+};
+
+fn projectPointCubeShadow(d: vec3<f32>, near: f32, far: f32) -> PointCubeProj {
+ var out = PointCubeProj(vec2<f32>(0.5), 0.0, 0, false);
+ let ax = abs(d.x);
+ let ay = abs(d.y);
+ let az = abs(d.z);
+ var xEye = 0.0;
+ var yEye = 0.0;
+ var zEye = 0.0;
+ if (ax >= ay && ax >= az) {
+ if (d.x >= 0.0) {
+ out.face = 0;
+ xEye = -d.z;
+ yEye = -d.y;
+ zEye = d.x;
+ } else {
+ out.face = 1;
+ xEye = d.z;
+ yEye = -d.y;
+ zEye = -d.x;
+ }
+ } else if (ay >= ax && ay >= az) {
+ if (d.y >= 0.0) {
+ out.face = 2;
+ xEye = d.x;
+ yEye = d.z;
+ zEye = d.y;
+ } else {
+ out.face = 3;
+ xEye = d.x;
+ yEye = -d.z;
+ zEye = -d.y;
+ }
+ } else {
+ if (d.z >= 0.0) {
+ out.face = 4;
+ xEye = d.x;
+ yEye = -d.y;
+ zEye = d.z;
+ } else {
+ out.face = 5;
+ xEye = -d.x;
+ yEye = -d.y;
+ zEye = -d.z;
+ }
+ }
+ if (zEye <= near || zEye >= far) {
+ return out;
+ }
+ let ndcX = xEye / zEye;
+ let ndcY = yEye / zEye;
+ out.uv = vec2<f32>(ndcX * 0.5 + 0.5, 0.5 - ndcY * 0.5);
+ out.ndcZ = (far - (near * far) / zEye) / max(far - near, 1e-4);
+ out.valid = true;
+ return out;
+}
+
/// 1 = fully lit, 0 = fully shadowed, for light index i.
fn shadowFactor(worldPos: vec3<f32>, N: vec3<f32>, i: i32) -> f32 {
let shadow = directLights[i];
@@ -227,23 +321,77 @@ fn shadowFactor(worldPos: vec3<f32>, N: vec3<f32>, i: i32) -> f32 {
return 1.0;
}
let p = worldPos + N * shadow.shadowBias.y;
+ let texel = 1.0 / max(shadow.shadowParams.y, 1.0);
+ let radius = shadow.shadowParams.x * texel;
+ let baseLayer = i32(shadow.shadowParams.z);
+
+ if (shadow.shadowParams.w > 1.5) {
+ let near = max(0.5, shadow.shadowBias.w);
+ let far = max(near + 1.0, shadow.shadowMatrix[0][0]);
+ let cp = projectPointCubeShadow(p - shadow.posType.xyz, near, far);
+ if (!cp.valid) {
+ return 1.0;
+ }
+ let depthRef = cp.ndcZ - shadow.shadowBias.x;
+ let layer = baseLayer + cp.face;
+ var lit = 0.0;
+ for (var k = 0; k < 12; k++) {
+ let sampleUv = clamp(cp.uv + POISSON[k] * radius, vec2<f32>(texel), vec2<f32>(1.0 - texel));
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, sampleUv, layer, depthRef);
+ }
+ lit = lit / 12.0;
+ return mix(1.0, lit, shadow.shadowBias.z);
+ }
+
let lp = shadow.shadowMatrix * vec4<f32>(p, 1.0);
let ndc = lp.xyz / lp.w;
let uv = vec2<f32>(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5);
if (lp.w <= 0.0 || uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0 || ndc.z > 1.0) {
return 1.0;
}
- let texel = 1.0 / max(shadow.shadowParams.y, 1.0);
- let radius = shadow.shadowParams.x * texel;
let depthRef = ndc.z - shadow.shadowBias.x;
var lit = 0.0;
for (var k = 0; k < 12; k++) {
- lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[k] * radius, i32(shadow.shadowParams.z), depthRef);
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[k] * radius, baseLayer, depthRef);
}
lit = lit / 12.0;
return mix(1.0, lit, shadow.shadowBias.z);
}
+/// Hardware-comparison shadow lookup with 4-tap soft penumbra for volumetric raymarching.
+fn volumetricShadowFactor(worldPos: vec3<f32>, i: i32) -> f32 {
+ let shadow = directLights[i];
+ if (shadow.shadowParams.w < 0.5) {
+ return 1.0;
+ }
+ let texel = 1.0 / max(shadow.shadowParams.y, 1.0);
+ let radius = shadow.shadowParams.x * texel * 0.85;
+ let baseLayer = i32(shadow.shadowParams.z);
+ if (shadow.shadowParams.w > 1.5) {
+ let near = max(0.5, shadow.shadowBias.w);
+ let far = max(near + 1.0, shadow.shadowMatrix[0][0]);
+ let cp = projectPointCubeShadow(worldPos - shadow.posType.xyz, near, far);
+ if (!cp.valid) {
+ return 1.0;
+ }
+ let lit = textureSampleCompareLevel(shadowMap, shadowSampler, cp.uv, baseLayer + cp.face, cp.ndcZ - shadow.shadowBias.x);
+ return mix(1.0, lit, shadow.shadowBias.z);
+ }
+ let lp = shadow.shadowMatrix * vec4<f32>(worldPos, 1.0);
+ let ndc = lp.xyz / lp.w;
+ let uv = vec2<f32>(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5);
+ if (lp.w <= 0.0 || uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0 || ndc.z > 1.0) {
+ return 1.0;
+ }
+ let depthRef = ndc.z - shadow.shadowBias.x;
+ var lit = 0.0;
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[0] * radius, baseLayer, depthRef);
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[3] * radius, baseLayer, depthRef);
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[5] * radius, baseLayer, depthRef);
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[8] * radius, baseLayer, depthRef);
+ return mix(1.0, lit * 0.25, shadow.shadowBias.z);
+}
+
// ── Lights ────────────────────────────────────────────────────────────
struct LightSample {
@@ -264,6 +412,42 @@ fn sampleLight(i: i32, worldPos: vec3<f32>) -> LightSample {
if (lightType < 1.5) {
return out;
}
+ if (lightType > 3.5) {
+ // Rect area light (lightType == 4.0)
+ let lightPos = light.posType.xyz;
+ let lightForward = normalize(light.dirRadius.xyz);
+ let right = light.rectRight.xyz;
+ let halfW = max(0.5, light.rectRight.w);
+ let up = light.rectUp.xyz;
+ let halfH = max(0.5, light.rectUp.w);
+ let twoSided = light.spotParams.w > 0.5;
+ let rel = worldPos - lightPos;
+ let frontDist = dot(rel, lightForward);
+ if (!twoSided && frontDist <= 0.0) {
+ return out;
+ }
+ let uCoord = clamp(dot(rel, right), -halfW, halfW);
+ let vCoord = clamp(dot(rel, up), -halfH, halfH);
+ let nearestPt = lightPos + right * uCoord + up * vCoord;
+ let toLight = nearestPt - worldPos;
+ let dist = length(toLight);
+ let radius = max(1.0, light.dirRadius.w);
+ if (dist > radius) {
+ return out;
+ }
+ out.L = select(normalize(-lightForward), toLight / dist, dist > 1e-4);
+ let emitCos = select(max(0.0, dot(-out.L, lightForward)), abs(dot(-out.L, lightForward)), twoSided);
+ if (emitCos <= 0.0) {
+ return out;
+ }
+ let dNorm = clamp(dist / radius, 0.0, 1.0);
+ let decay = max(0.5, light.spotParams.z);
+ let area = 4.0 * halfW * halfH;
+ let areaFactor = clamp(area / (area + PI * dist * dist), 0.12, 1.0);
+ out.attenuation = pow(clamp(1.0 - dNorm, 0.0, 1.0), decay) * emitCos * (0.45 + 0.55 * areaFactor);
+ out.valid = true;
+ return out;
+ }
let toLight = light.posType.xyz - worldPos;
let dist = length(toLight);
let radius = max(1.0, light.dirRadius.w);
@@ -286,6 +470,32 @@ fn sampleLight(i: i32, worldPos: vec3<f32>) -> LightSample {
return out;
}
+// Representative point on a rectangular area light along reflection vector R.
+fn rectRepresentativeDirection(light: DirectLight, worldPos: vec3<f32>, R: vec3<f32>, fallbackL: vec3<f32>) -> vec3<f32> {
+ let lightPos = light.posType.xyz;
+ let lightForward = normalize(light.dirRadius.xyz);
+ let right = light.rectRight.xyz;
+ let halfW = max(0.5, light.rectRight.w);
+ let up = light.rectUp.xyz;
+ let halfH = max(0.5, light.rectUp.w);
+ let denom = dot(R, lightForward);
+ let centerDist = length(lightPos - worldPos);
+ var tRay = centerDist;
+ if (abs(denom) > 1e-3) {
+ let tHit = dot(lightPos - worldPos, lightForward) / denom;
+ if (tHit > 0.0) {
+ tRay = tHit;
+ }
+ }
+ let planeHit = worldPos + R * tRay - lightPos;
+ let uSpec = clamp(dot(planeHit, right), -halfW, halfW);
+ let vSpec = clamp(dot(planeHit, up), -halfH, halfH);
+ let specPt = lightPos + right * uSpec + up * vSpec;
+ let toSpec = specPt - worldPos;
+ let len = length(toSpec);
+ return select(fallbackL, toSpec / len, len > 1e-4);
+}
+
// Blinn-Phong in display space: the original look, plus shadows when a
// light casts them.
fn applyLighting3D(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32>, material: vec4<f32>) -> vec3<f32> {
@@ -297,6 +507,7 @@ fn applyLighting3D(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<
if (dot(N, V) < 0.0) {
N = -N;
}
+ let R = reflect(-V, N);
let shininess = max(1.0, material.x);
let specularInt = material.y;
let ambientInt = material.z;
@@ -315,7 +526,8 @@ fn applyLighting3D(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<
let diffuse = baseColor * lightColor * (NdotL * intensity * ls.attenuation);
var specular = vec3<f32>(0.0);
if (NdotL > 0.0 && specularInt > 0.001) {
- let H = normalize(ls.L + V);
+ let specL = select(ls.L, rectRepresentativeDirection(light, worldPos, R, ls.L), light.posType.w > 3.5);
+ let H = normalize(specL + V);
let NdotH = max(dot(N, H), 0.0);
specular = lightColor * (pow(NdotH, shininess) * specularInt * intensity * ls.attenuation);
}
@@ -324,14 +536,55 @@ fn applyLighting3D(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<
return totalIllumination;
}
-// GGX / Smith-correlated / Schlick, image-based light from envRadiance.
-// Returns linear scene radiance (unbounded).
-fn shadePhysical(worldPos: vec3<f32>, worldNormal: vec3<f32>, albedo: vec3<f32>, rough: f32, metal: f32, ambientInt: f32) -> vec3<f32> {
+struct SubstrateParams {
+ clearcoat : vec4<f32>, // x = clearcoat (0..1), y = clearcoatRoughness (0.03..1), z = anisotropy (-1..1), w = anisotropyRotation (rad)
+ subsurface : vec4<f32>, // x = subsurface (0..1), yzw = subsurfaceColor (sRGB)
+ sheen : vec4<f32>, // x = sheen (0..1), yzw = sheenColor (sRGB)
+ iridescence : vec4<f32>, // x = intensity (0..1), y = thickness nm (100..1200), z = film ior (1.3), w = triplanarUvFreq
+};
+
+fn defaultSubstrate() -> SubstrateParams {
+ return SubstrateParams(
+ vec4<f32>(0.0, 0.08, 0.0, 0.0),
+ vec4<f32>(0.0, 1.0, 0.42, 0.29),
+ vec4<f32>(0.0, 1.0, 1.0, 1.0),
+ vec4<f32>(0.0, 400.0, 1.3, 0.0),
+ );
+}
+
+// Airy / Belcour thin-film spectral interference at RGB primary wavelengths (650, 510, 475 nm).
+fn evalIridescence(cosTheta1: f32, thicknessNm: f32, filmIor: f32, baseF0: vec3<f32>) -> vec3<f32> {
+ let n2 = max(1.05, filmIor);
+ let sin2Theta2 = (1.0 - cosTheta1 * cosTheta1) / (n2 * n2);
+ let cosTheta2 = sqrt(max(0.0, 1.0 - sin2Theta2));
+ let opd = 2.0 * n2 * thicknessNm * cosTheta2;
+ let lambdas = vec3<f32>(650.0, 510.0, 475.0);
+ let phi = (2.0 * PI * opd) / lambdas + vec3<f32>(PI);
+ let f0_12 = pow((n2 - 1.0) / (n2 + 1.0), 2.0);
+ let r12 = vec3<f32>(f0_12 + (1.0 - f0_12) * pow(clamp(1.0 - cosTheta1, 0.0, 1.0), 5.0));
+ let r23 = baseF0 + (vec3<f32>(1.0) - baseF0) * pow(clamp(1.0 - cosTheta2, 0.0, 1.0), 5.0);
+ let interf = r12 + r23 + 2.0 * sqrt(max(r12 * r23, vec3<f32>(0.0))) * cos(phi);
+ return clamp(interf, vec3<f32>(0.0), vec3<f32>(1.0));
+}
+
+// Multi-lobe Substrate PBR: GGX (isotropic or anisotropic) + Thin-Film Iridescence +
+// Subsurface wrap/transmission + Charlie Cloth Sheen + Clearcoat secondary lobe + IBL.
+fn shadePhysicalSubstrate(
+ worldPos: vec3<f32>,
+ worldNormal: vec3<f32>,
+ albedo: vec3<f32>,
+ rough: f32,
+ metal: f32,
+ ambientInt: f32,
+ sheenRoughness: f32,
+ substrate: SubstrateParams,
+) -> vec3<f32> {
var N = normalize(worldNormal);
let V = normalize(cam.cameraPos.xyz - worldPos);
if (dot(N, V) < 0.0) {
N = -N;
}
+ let R = reflect(-V, N);
let nDotV = max(dot(N, V), 1e-4);
let r = clamp(rough, 0.03, 1.0);
let a = r * r;
@@ -339,6 +592,32 @@ fn shadePhysical(worldPos: vec3<f32>, worldNormal: vec3<f32>, albedo: vec3<f32>,
let f0 = mix(vec3<f32>(0.04), albedo, metal);
let diffuseColor = albedo * (1.0 - metal);
+ let cc = clamp(substrate.clearcoat.x, 0.0, 1.0);
+ let ccRough = clamp(substrate.clearcoat.y, 0.03, 1.0);
+ let ccA = ccRough * ccRough;
+ let ccA2 = ccA * ccA;
+ let aniso = clamp(substrate.clearcoat.z, -0.95, 0.95);
+ let hasAniso = abs(aniso) > 1e-3;
+ let rot = substrate.clearcoat.w;
+ let upRef = select(vec3<f32>(0.0, 1.0, 0.0), vec3<f32>(1.0, 0.0, 0.0), abs(N.y) > 0.99);
+ let T0 = normalize(cross(upRef, N));
+ let B0 = cross(N, T0);
+ let cRot = cos(rot);
+ let sRot = sin(rot);
+ let T = T0 * cRot + B0 * sRot;
+ let B = -T0 * sRot + B0 * cRot;
+ let at = max(0.001, a * (1.0 + aniso));
+ let ab = max(0.001, a * (1.0 - aniso));
+
+ let sssWeight = clamp(substrate.subsurface.x, 0.0, 1.0);
+ let sssColor = srgbToLinear(substrate.subsurface.yzw);
+ let sheenWeight = clamp(substrate.sheen.x, 0.0, 1.0);
+ let sheenColor = srgbToLinear(substrate.sheen.yzw) * sheenWeight;
+ let sheenAlpha = max(0.07, sheenRoughness * sheenRoughness);
+ let iridWeight = clamp(substrate.iridescence.x, 0.0, 1.0);
+ let iridThickness = max(100.0, substrate.iridescence.y);
+ let iridIor = max(1.05, substrate.iridescence.z);
+
var lo = vec3<f32>(0.0);
let numLights = i32(lights.params.x);
for (var i = 0; i < numLights; i++) {
@@ -346,35 +625,152 @@ fn shadePhysical(worldPos: vec3<f32>, worldNormal: vec3<f32>, albedo: vec3<f32>,
if (!ls.valid) {
continue;
}
- let light = directLights[i];
- let nDotL = max(dot(N, ls.L), 0.0);
- if (nDotL <= 0.0) {
+ let rawNdotL = dot(N, ls.L);
+ let nDotL = max(rawNdotL, 0.0);
+ if (nDotL <= 0.0 && sssWeight <= 0.001) {
continue;
}
- let H = normalize(ls.L + V);
+ let light = directLights[i];
+ let isRect = light.posType.w > 3.5;
+ let specL = select(ls.L, rectRepresentativeDirection(light, worldPos, R, ls.L), isRect);
+ let nDotLSpec = max(dot(N, specL), 1e-4);
+ let H = normalize(specL + V);
let nDotH = max(dot(N, H), 0.0);
let vDotH = max(dot(V, H), 0.0);
- let dDen = nDotH * nDotH * (a2 - 1.0) + 1.0;
- let D = a2 / (PI * dDen * dDen);
- let gv = nDotL * sqrt(nDotV * nDotV * (1.0 - a2) + a2);
- let gl = nDotV * sqrt(nDotL * nDotL * (1.0 - a2) + a2);
- let Vis = 0.5 / max(gv + gl, 1e-5);
- let F = f0 + (vec3<f32>(1.0) - f0) * pow(1.0 - vDotH, 5.0);
- let spec = D * Vis * F;
+
+ var D = 0.0;
+ var Vis = 0.0;
+ if (hasAniso) {
+ let tDotH = dot(T, H);
+ let bDotH = dot(B, H);
+ let atb = at * ab;
+ let vAniso = vec3<f32>(ab * tDotH, at * bDotH, atb * nDotH);
+ let v2 = dot(vAniso, vAniso);
+ let w2 = atb / max(v2, 1e-6);
+ D = atb * w2 * w2 * (1.0 / PI);
+ let tDotV = dot(T, V);
+ let bDotV = dot(B, V);
+ let tDotL = dot(T, specL);
+ let bDotL = dot(B, specL);
+ let lambdaV = nDotLSpec * length(vec3<f32>(at * tDotV, ab * bDotV, nDotV));
+ let lambdaL = nDotV * length(vec3<f32>(at * tDotL, ab * bDotL, nDotLSpec));
+ Vis = 0.5 / max(lambdaV + lambdaL, 1e-5);
+ } else {
+ var a2Spec = a2;
+ var energyNorm = 1.0;
+ if (isRect) {
+ let rectSpan = sqrt(max(1.0, light.rectRight.w * light.rectUp.w));
+ let centerDist = max(50.0, length(light.posType.xyz - worldPos));
+ let aRect = clamp(a + 0.35 * (rectSpan / centerDist), 0.03, 1.0);
+ a2Spec = aRect * aRect;
+ energyNorm = clamp((a * a) / max(a2Spec, 1e-4), 0.15, 1.0);
+ }
+ let dDen = nDotH * nDotH * (a2Spec - 1.0) + 1.0;
+ D = (a2Spec * energyNorm) / (PI * dDen * dDen);
+ let gv = nDotLSpec * sqrt(nDotV * nDotV * (1.0 - a2) + a2);
+ let gl = nDotV * sqrt(nDotLSpec * nDotLSpec * (1.0 - a2) + a2);
+ Vis = 0.5 / max(gv + gl, 1e-5);
+ }
+
+ var F = f0 + (vec3<f32>(1.0) - f0) * pow(1.0 - vDotH, 5.0);
+ if (iridWeight > 0.001) {
+ let fIrid = evalIridescence(vDotH, iridThickness, iridIor, f0);
+ F = mix(F, fIrid, iridWeight);
+ }
+
+ let spec = select(vec3<f32>(0.0), D * Vis * F, nDotL > 0.0);
let diff = (vec3<f32>(1.0) - F) * diffuseColor / PI;
- // x PI so a white light of intensity 1 lights a white wall about as
- // brightly as the legacy shading does.
+
+ var directLobe = diff * nDotL + spec * select(0.0, nDotLSpec, nDotL > 0.0);
+
+ // Subsurface wrap diffuse + terminator bleed + translucent back-scattering
+ if (sssWeight > 0.001) {
+ let wrap = 0.45 * sssWeight;
+ let wrapNdotL = max(0.0, (rawNdotL + wrap) / ((1.0 + wrap) * (1.0 + wrap)));
+ let terminator = smoothstep(0.0, wrap + 0.3, max(0.0, rawNdotL + wrap)) * (1.0 - nDotL);
+ let wrapDiff = diffuseColor * wrapNdotL + sssColor * (diffuseColor * 0.6 + vec3<f32>(0.4)) * terminator * sssWeight * 0.55;
+ let backL = normalize(ls.L + N * 0.35);
+ // Light crossing a convex body travels far through its middle and
+ // briefly near its silhouette: how closely the exit normal follows
+ // the light's travel direction stands in for that chord length, so
+ // back-lit solids glow at the rim and stay dense at the core.
+ let chord = clamp(dot(N, -ls.L), 0.0, 1.0);
+ let transmittance = exp(-chord * mix(6.0, 1.5, sssWeight));
+ let backTrans = pow(clamp(dot(V, -backL), 0.0, 1.0), 3.0) * transmittance * sssWeight * 0.65;
+ let sssContrib = (wrapDiff + sssColor * backTrans) / PI;
+ directLobe = mix(diff * nDotL, sssContrib, sssWeight) + spec * select(0.0, nDotLSpec, nDotL > 0.0);
+ }
+
+ // Charlie cloth / velvet sheen lobe (Estevez & Kulla)
+ if (sheenWeight > 0.001 && nDotL > 0.0) {
+ let invR = 1.0 / sheenAlpha;
+ let sin2h = max(1.0 - nDotH * nDotH, 1e-4);
+ let dCharlie = (2.0 + invR) * pow(sin2h, invR * 0.5) / (2.0 * PI);
+ let vCharlie = 1.0 / max(4.0 * (nDotL + nDotV - nDotL * nDotV), 1e-4);
+ directLobe += sheenColor * dCharlie * vCharlie * nDotL;
+ }
+
+ // Clearcoat secondary dielectric varnish lobe (IOR 1.5 -> F0 = 0.04)
+ if (cc > 0.001 && nDotL > 0.0) {
+ let Fc = 0.04 + 0.96 * pow(1.0 - vDotH, 5.0);
+ let ccDen = nDotH * nDotH * (ccA2 - 1.0) + 1.0;
+ let Dc = ccA2 / (PI * ccDen * ccDen);
+ let VisC = 0.25 / max(vDotH * vDotH, 1e-4);
+ let clearcoatSpec = Dc * VisC * Fc * nDotLSpec;
+ directLobe = directLobe * (1.0 - cc * Fc) + vec3<f32>(cc * clearcoatSpec);
+ }
+
let radiance = srgbToLinear(light.colorIntensity.rgb) * light.colorIntensity.w * ls.attenuation * PI;
- lo += (diff + spec) * radiance * nDotL * shadowFactor(worldPos, N, i);
+ let shadow = select(1.0, shadowFactor(worldPos, N, i), nDotL > 0.0);
+ lo += directLobe * radiance * shadow;
}
- let R = reflect(-V, N);
- let irradiance = envRadiance(N, 1.0) + lights.ambientLinear.rgb;
- let specEnv = envRadiance(R, r) * envBrdfApprox(f0, r, nDotV);
- lo += (diffuseColor * irradiance + specEnv) * ambientInt;
+ var R_ibl = R;
+ if (hasAniso) {
+ let anisoDir = select(B, T, aniso < 0.0);
+ let anisoTangent = cross(anisoDir, V);
+ let anisoNormal = cross(anisoTangent, anisoDir);
+ let bentN = normalize(mix(N, normalize(anisoNormal), abs(aniso) * 0.6));
+ R_ibl = reflect(-V, bentN);
+ }
+
+ var f0_ibl = f0;
+ if (iridWeight > 0.001) {
+ let fIridView = evalIridescence(nDotV, iridThickness, iridIor, f0);
+ f0_ibl = mix(f0, fIridView, iridWeight);
+ }
+
+ let irradiance = envIrradiance(N) + lights.ambientLinear.rgb;
+ var diffAmbient = diffuseColor * irradiance;
+ if (sssWeight > 0.001) {
+ let backIrradiance = envIrradiance(-N) * 0.35 + irradiance * 0.25;
+ diffAmbient += sssColor * (diffuseColor * 0.5 + vec3<f32>(0.5)) * backIrradiance * sssWeight * 0.45;
+ }
+ let specEnv = envRadiance(R_ibl, r) * envBrdfApprox(f0_ibl, r, nDotV);
+ var iblTotal = diffAmbient + specEnv;
+
+ if (sheenWeight > 0.001) {
+ // The Charlie lobe is broad and peaks at grazing angles rather than
+ // around the mirror direction, so cloth gathers the environment's
+ // irradiance, weighted by the lobe's grazing-heavy directional albedo.
+ let sheenAlbedo = pow(1.0 - nDotV, 3.0) * 0.4 + 0.1 * (1.0 - sheenRoughness * 0.5);
+ iblTotal += sheenColor * irradiance * sheenAlbedo;
+ }
+
+ if (cc > 0.001) {
+ let Fc_ibl = 0.04 + 0.96 * pow(1.0 - nDotV, 5.0);
+ let ccEnv = envRadiance(R, ccRough) * Fc_ibl * cc;
+ iblTotal = iblTotal * (1.0 - cc * Fc_ibl) + ccEnv;
+ }
+
+ lo += iblTotal * ambientInt;
return lo;
}
+fn shadePhysical(worldPos: vec3<f32>, worldNormal: vec3<f32>, albedo: vec3<f32>, rough: f32, metal: f32, ambientInt: f32) -> vec3<f32> {
+ return shadePhysicalSubstrate(worldPos, worldNormal, albedo, rough, metal, ambientInt, 0.5, defaultSubstrate());
+}
+
struct Shaded {
/** Display-encoded (sRGB) color for the color target. */
display : vec3<f32>,
@@ -421,13 +817,26 @@ fn geometryPixel(depth: f32, normal: vec3<f32>, outlined: f32) -> vec4<f32> {
return vec4<f32>(depth, oct * 0.5 + 0.5, outlined);
}
-/// Shades a surface for the scene's mode.
-/// material: x shininess, y specularIntensity, z ambientIntensity, w mode (0 unlit).
-/// pbr: x roughness (<0 = default), y metallic, z/w unused.
-/// emissive: rgb sRGB color, w strength.
-fn shadeSurfaceFull(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32>, material: vec4<f32>, pbr: vec4<f32>, emissive: vec4<f32>, toonShadow: vec4<f32>, toonLight: vec4<f32>, toonParams: vec4<f32>) -> Shaded {
+fn hasActiveSubstrate(s: SubstrateParams) -> bool {
+ return s.clearcoat.x > 0.001 || abs(s.clearcoat.z) > 0.001 || s.subsurface.x > 0.001 || s.sheen.x > 0.001 || s.iridescence.x > 0.001;
+}
+
+/// Shades a surface with full Substrate multi-lobe material support.
+fn shadeSurfaceSubstrate(
+ worldPos: vec3<f32>,
+ worldNormal: vec3<f32>,
+ baseColor: vec3<f32>,
+ material: vec4<f32>,
+ pbr: vec4<f32>,
+ emissive: vec4<f32>,
+ toonShadow: vec4<f32>,
+ toonLight: vec4<f32>,
+ toonParams: vec4<f32>,
+ substrate: SubstrateParams,
+) -> Shaded {
var out: Shaded;
- if (lights.params.z < 0.5) {
+ let usePhysical = lights.params.z >= 0.5 || (material.w >= 0.5 && material.w <= 1.5 && hasActiveSubstrate(substrate));
+ if (!usePhysical) {
var c = baseColor;
if (material.w > 1.5) {
c = shadeToon(worldPos, worldNormal, baseColor, material.z, toonShadow, toonLight, toonParams);
@@ -446,17 +855,20 @@ fn shadeSurfaceFull(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3
radiance = srgbToLinear(shadeToon(worldPos, worldNormal, baseColor, material.z, toonShadow, toonLight, toonParams));
} else {
let rough = select(pbr.x, 0.5, pbr.x < 0.0);
- radiance = shadePhysical(worldPos, worldNormal, srgbToLinear(baseColor), rough, clamp(pbr.y, 0.0, 1.0), material.z);
+ let sheenRough = select(0.5, toonParams.z, toonParams.z > 0.01);
+ radiance = shadePhysicalSubstrate(worldPos, worldNormal, srgbToLinear(baseColor), rough, clamp(pbr.y, 0.0, 1.0), material.z, sheenRough, substrate);
}
radiance += srgbToLinear(emissive.rgb) * emissive.w;
var display: vec3<f32>;
if (material.w < 0.5) {
// Unlit layers keep their exact authored colors.
display = clamp(radiance, vec3<f32>(0.0), vec3<f32>(1.0));
- } else {
+ } else if (lights.params.z >= 0.5) {
display = toneMap(radiance);
+ } else {
+ display = toneMapNeutral(radiance);
}
- var hdr = radiance * lights.ambientLinear.w;
+ var hdr = radiance * max(lights.ambientLinear.w, 1.0);
if (lights.fog.w > 0.5) {
let dist = distance(worldPos, cam.cameraPos.xyz);
let f = smoothstep(lights.fogRange.x, max(lights.fogRange.y, lights.fogRange.x + 1.0), dist) * lights.fogRange.z;
@@ -479,6 +891,11 @@ fn shadeSurfaceFull(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3
return out;
}
+/// Shades a surface for the scene's mode.
+fn shadeSurfaceFull(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32>, material: vec4<f32>, pbr: vec4<f32>, emissive: vec4<f32>, toonShadow: vec4<f32>, toonLight: vec4<f32>, toonParams: vec4<f32>) -> Shaded {
+ return shadeSurfaceSubstrate(worldPos, worldNormal, baseColor, material, pbr, emissive, toonShadow, toonLight, toonParams, defaultSubstrate());
+}
+
`;
/** Group-3 bindings every lit 3D shader declares. */
@@ -487,4 +904,6 @@ export const lightsBindingsWgsl = `
@group(3) @binding(1) var shadowMap : texture_depth_2d_array;
@group(3) @binding(2) var shadowSampler : sampler_comparison;
@group(3) @binding(3) var<storage, read> directLights : array<DirectLight>;
+@group(3) @binding(4) var envMap : texture_cube<f32>;
+@group(3) @binding(5) var envSampler : sampler;
`;
diff --git a/packages/webgpu-renderers/src/shaders/mesh3d.ts b/packages/webgpu-renderers/src/shaders/mesh3d.ts
index 26fc7a81..8e887dbd 100644
--- a/packages/webgpu-renderers/src/shaders/mesh3d.ts
+++ b/packages/webgpu-renderers/src/shaders/mesh3d.ts
@@ -14,18 +14,22 @@ export const mesh3dWgsl = `
${lighting3dWgsl}
struct ModelUniforms {
- modelMatrix : mat4x4<f32>,
- normalMatrix : mat4x4<f32>,
- colorTint : vec4<f32>,
- params : vec4<f32>, // x = opacity, y = twoSided, z = isSkinned, w = flags (bit 0 instanced, bit 1 map UVs, bit 2 matrix instances)
- materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit, 2=toon)
- shade : vec4<f32>, // rgb = toon shadow-side color multiplier, a = alpha cutoff (0 = none)
- pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
- emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
- toonShadow : vec4<f32>, // rgb shadow-side multiplier, w diffuse modelling
- toonLight : vec4<f32>, // rgb lit-side multiplier, w terminator threshold
- toonParams : vec4<f32>, // x terminator softness, y outline participation
- maps : vec4<f32>, // x = normal scale (0 = no map), y = occlusion strength, z = has metallic-roughness map, w = has emissive map
+ modelMatrix : mat4x4<f32>,
+ normalMatrix : mat4x4<f32>,
+ colorTint : vec4<f32>,
+ params : vec4<f32>, // x = opacity, y = twoSided, z = isSkinned, w = flags (bit 0 instanced, bit 1 map UVs, bit 2 matrix instances)
+ materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit, 2=toon)
+ shade : vec4<f32>, // rgb = toon shadow-side color multiplier, a = alpha cutoff (0 = none)
+ pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
+ emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
+ toonShadow : vec4<f32>, // rgb shadow-side multiplier, w diffuse modelling
+ toonLight : vec4<f32>, // rgb lit-side multiplier, w terminator threshold
+ toonParams : vec4<f32>, // x terminator softness, y outline participation, z sheenRoughness, w triplanarSharpness
+ substrateClearcoat : vec4<f32>, // x = clearcoat, y = clearcoatRoughness, z = anisotropy, w = anisotropyRotationRad
+ substrateSubsurface : vec4<f32>, // x = subsurface, yzw = subsurfaceColor (sRGB)
+ substrateSheen : vec4<f32>, // x = sheen, yzw = sheenColor (sRGB)
+ substrateIridescence : vec4<f32>, // x = iridescence, y = thicknessNm, z = filmIor, w = triplanarUvFreq
+ maps : vec4<f32>, // x = normal scale (0 = no map), y = occlusion strength, z = has metallic-roughness map, w = has emissive map
};
struct SkinningUniforms {
@@ -254,19 +258,50 @@ fn shadeGlass(in: VertexOutput, Nin: vec3<f32>, tint: vec3<f32>, alpha: f32, rim
return out;
}
+fn sampleTriplanar4(tex: texture_2d<f32>, p: vec3<f32>, dp1: vec3<f32>, dp2: vec3<f32>, weights: vec3<f32>) -> vec4<f32> {
+ let sx = textureSampleGrad(tex, texSampler, fract(p.zy), dp1.zy, dp2.zy);
+ let sy = textureSampleGrad(tex, texSampler, fract(p.xz), dp1.xz, dp2.xz);
+ let sz = textureSampleGrad(tex, texSampler, fract(p.xy), dp1.xy, dp2.xy);
+ return sx * weights.x + sy * weights.y + sz * weights.z;
+}
+
fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {
// Every sample and derivative up front, in uniform control flow.
var texColor = textureSample(mainTex, texSampler, in.uv);
- let normalSample = textureSample(normalTex, texSampler, in.normalUv).xyz;
- let mrSample = textureSample(metalRoughTex, texSampler, in.mrUv);
- let aoSample = textureSample(occlusionTex, texSampler, in.aoUv).r;
- let emissiveSample = textureSample(emissiveTex, texSampler, in.emissiveUv).rgb;
+ var normalSample = textureSample(normalTex, texSampler, in.normalUv).xyz;
+ var mrSample = textureSample(metalRoughTex, texSampler, in.mrUv);
+ var aoSample = textureSample(occlusionTex, texSampler, in.aoUv).r;
+ var emissiveSample = textureSample(emissiveTex, texSampler, in.emissiveUv).rgb;
let dp1 = dpdx(in.worldPos);
let dp2 = dpdy(in.worldPos);
let duv1 = dpdx(in.normalUv);
let duv2 = dpdy(in.normalUv);
let rimWidth = fwidth(1.0 - abs(dot(normalize(in.worldNormal), normalize(cam.cameraPos.xyz - in.worldPos))));
+ let triFreq = model.substrateIridescence.w;
+ if (triFreq > 0.0) {
+ let sharpness = max(1.0, model.toonParams.w);
+ let absN = max(abs(normalize(in.worldNormal)), vec3<f32>(1e-4));
+ let wPow = pow(absN, vec3<f32>(sharpness));
+ let wTri = wPow / max(wPow.x + wPow.y + wPow.z, 1e-5);
+ let pTri = in.worldPos * triFreq;
+ let dp1Tri = dp1 * triFreq;
+ let dp2Tri = dp2 * triFreq;
+ texColor = sampleTriplanar4(mainTex, pTri, dp1Tri, dp2Tri, wTri);
+ if (model.maps.x > 0.0) {
+ normalSample = sampleTriplanar4(normalTex, pTri, dp1Tri, dp2Tri, wTri).xyz;
+ }
+ if (model.maps.z > 0.5) {
+ mrSample = sampleTriplanar4(metalRoughTex, pTri, dp1Tri, dp2Tri, wTri);
+ }
+ if (model.maps.y > 0.0) {
+ aoSample = sampleTriplanar4(occlusionTex, pTri, dp1Tri, dp2Tri, wTri).r;
+ }
+ if (model.maps.w > 0.5) {
+ emissiveSample = sampleTriplanar4(emissiveTex, pTri, dp1Tri, dp2Tri, wTri).rgb;
+ }
+ }
+
if (!isFront && model.params.y < 0.5) {
discard;
}
@@ -309,7 +344,13 @@ fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {
if (model.params.w > 0.5 && (in.emissive.w > 0.0 || dot(in.emissive.rgb, in.emissive.rgb) > 0.0)) {
emissive = in.emissive;
}
- let shaded = shadeSurfaceFull(in.worldPos, N, baseRgb, material, pbr, emissive, model.toonShadow, model.toonLight, model.toonParams);
+ let substrate = SubstrateParams(
+ model.substrateClearcoat,
+ model.substrateSubsurface,
+ model.substrateSheen,
+ model.substrateIridescence,
+ );
+ let shaded = shadeSurfaceSubstrate(in.worldPos, N, baseRgb, material, pbr, emissive, model.toonShadow, model.toonLight, model.toonParams, substrate);
out.color = vec4<f32>(shaded.display * alpha, alpha);
out.hdr = vec4<f32>(shaded.hdr * alpha, alpha);
return out;
diff --git a/packages/webgpu-renderers/src/shaders/quad3d.ts b/packages/webgpu-renderers/src/shaders/quad3d.ts
index fb1dd6ff..85311cfc 100644
--- a/packages/webgpu-renderers/src/shaders/quad3d.ts
+++ b/packages/webgpu-renderers/src/shaders/quad3d.ts
@@ -13,21 +13,25 @@ ${lighting3dWgsl}
const RELIEF_NORMAL_CELLS : f32 = 4.0;
struct ModelUniforms {
- modelMatrix : mat4x4<f32>,
- normalMatrix : mat4x4<f32>,
- colorTint : vec4<f32>,
- borderColor : vec4<f32>,
- params : vec4<f32>, // x = opacity, y = borderRadius, z = twoSided (1.0 = yes), w = borderWidth
- dimensions : vec4<f32>, // x = width, y = height, zw = texture extent the layer covers (0 = whole texture)
- materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit)
- pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
- emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
- toonShadow : vec4<f32>, // rgb shadow-side multiplier, w diffuse modelling
- toonLight : vec4<f32>, // rgb lit-side multiplier, w terminator threshold
- toonParams : vec4<f32>, // x terminator softness, y outline participation
- relief : vec4<f32>, // x = amount (px), y = on, w = grid cells per side
- reliefRect : vec4<f32>, // xy = where the layer's box starts in reliefTex (0..1), zw = its size
- reliefEye : vec4<f32>, // xyz = the eye the surface keeps its look from (the default camera)
+ modelMatrix : mat4x4<f32>,
+ normalMatrix : mat4x4<f32>,
+ colorTint : vec4<f32>,
+ borderColor : vec4<f32>,
+ params : vec4<f32>, // x = opacity, y = borderRadius, z = twoSided (1.0 = yes), w = borderWidth
+ dimensions : vec4<f32>, // x = width, y = height, zw = texture extent the layer covers (0 = whole texture)
+ materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit)
+ pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
+ emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
+ toonShadow : vec4<f32>, // rgb shadow-side multiplier, w diffuse modelling
+ toonLight : vec4<f32>, // rgb lit-side multiplier, w terminator threshold
+ toonParams : vec4<f32>, // x terminator softness, y outline participation, z sheenRoughness, w triplanarSharpness
+ substrateClearcoat : vec4<f32>, // x = clearcoat, y = clearcoatRoughness, z = anisotropy, w = anisotropyRotationRad
+ substrateSubsurface : vec4<f32>, // x = subsurface, yzw = subsurfaceColor (sRGB)
+ substrateSheen : vec4<f32>, // x = sheen, yzw = sheenColor (sRGB)
+ substrateIridescence : vec4<f32>, // x = iridescence, y = thicknessNm, z = filmIor, w = triplanarUvFreq
+ relief : vec4<f32>, // x = amount (px), y = on, w = grid cells per side
+ reliefRect : vec4<f32>, // xy = where the layer's box starts in reliefTex (0..1), zw = its size
+ reliefEye : vec4<f32>, // xyz = the eye the surface keeps its look from (the default camera)
};
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@@ -219,7 +223,13 @@ fn shadeQuad(in: VertexOutput, isFront: bool) -> QuadFragment {
discard;
}
- let shaded = shadeSurfaceFull(in.worldPos, in.worldNormal, baseRgb, model.materialParams, model.pbr, model.emissive, model.toonShadow, model.toonLight, model.toonParams);
+ let substrate = SubstrateParams(
+ model.substrateClearcoat,
+ model.substrateSubsurface,
+ model.substrateSheen,
+ model.substrateIridescence,
+ );
+ let shaded = shadeSurfaceSubstrate(in.worldPos, in.worldNormal, baseRgb, model.materialParams, model.pbr, model.emissive, model.toonShadow, model.toonLight, model.toonParams, substrate);
var out: QuadFragment;
out.color = vec4<f32>(shaded.display * alpha, alpha);
out.hdr = vec4<f32>(shaded.hdr * alpha, alpha);
diff --git a/packages/webgpu-renderers/src/shaders/slug3d.ts b/packages/webgpu-renderers/src/shaders/slug3d.ts
index e74f8b46..0dc2dbfc 100644
--- a/packages/webgpu-renderers/src/shaders/slug3d.ts
+++ b/packages/webgpu-renderers/src/shaders/slug3d.ts
@@ -13,15 +13,19 @@ export const slug3dWgsl = `
${lighting3dWgsl}
struct TextModelUniforms {
- modelMatrix : mat4x4<f32>, // paragraph px -> world
- normalMatrix : mat4x4<f32>,
- params : vec4<f32>, // x = opacity, y = slant, z = twoSided (1.0 = yes), w = pull toward the eye
- materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit)
- pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
- emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
- toonShadow : vec4<f32>, // rgb shadow-side multiplier, w diffuse modelling
- toonLight : vec4<f32>, // rgb lit-side multiplier, w terminator threshold
- toonParams : vec4<f32>, // x terminator softness, y outline participation
+ modelMatrix : mat4x4<f32>, // paragraph px -> world
+ normalMatrix : mat4x4<f32>,
+ params : vec4<f32>, // x = opacity, y = slant, z = twoSided (1.0 = yes), w = pull toward the eye
+ materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit)
+ pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
+ emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
+ toonShadow : vec4<f32>, // rgb shadow-side multiplier, w diffuse modelling
+ toonLight : vec4<f32>, // rgb lit-side multiplier, w terminator threshold
+ toonParams : vec4<f32>, // x terminator softness, y outline participation, z sheenRoughness, w triplanarSharpness
+ substrateClearcoat : vec4<f32>, // x = clearcoat, y = clearcoatRoughness, z = anisotropy, w = anisotropyRotationRad
+ substrateSubsurface : vec4<f32>, // x = subsurface, yzw = subsurfaceColor (sRGB)
+ substrateSheen : vec4<f32>, // x = sheen, yzw = sheenColor (sRGB)
+ substrateIridescence : vec4<f32>, // x = iridescence, y = thicknessNm, z = filmIor, w = triplanarUvFreq
};
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@@ -154,7 +158,13 @@ fn shadeText(in : VSOut, fw : vec2<f32>) -> TextFragment {
if (shaded.a < 0.0001) {
return out;
}
- let lit = shadeSurfaceFull(in.worldPos, in.worldNormal, shaded.rgb / shaded.a, model.materialParams, model.pbr, model.emissive, model.toonShadow, model.toonLight, model.toonParams);
+ let substrate = SubstrateParams(
+ model.substrateClearcoat,
+ model.substrateSubsurface,
+ model.substrateSheen,
+ model.substrateIridescence,
+ );
+ let lit = shadeSurfaceSubstrate(in.worldPos, in.worldNormal, shaded.rgb / shaded.a, model.materialParams, model.pbr, model.emissive, model.toonShadow, model.toonLight, model.toonParams, substrate);
out.color = vec4<f32>(lit.display * shaded.a, shaded.a);
out.hdr = vec4<f32>(lit.hdr * shaded.a, shaded.a);
return out;
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index de59be46..32ec104b 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -456,10 +456,12 @@ comp.add([
toneMapping: "neutral", // default; "agx" filmic, "aces" punchy, "none"
rotation: 30, // turn the reflections
fog: { color: BG, start: 2000, end: 5000 }, // surfaces fade into the background color
+ volumetricFog: { density: 0.025, anisotropy: 0.6, noise: { strength: 0.4, speed: [18, -6, 10] } }, // participating medium + god rays
bloom: true, // or { intensity: 0.3, threshold: 1.6, radius: 6 }
floor: { y: 200, color: "#111115", reflectivity: 0.8, roughness: 0.25, fadeDistance: 800 }, // glossy floor reflection
}),
Layer.directionalLight({ x: 400, y: -1200, z: -600, targetX: 960, targetY: 540, castShadow: true }),
+ Layer.rectLight({ x: 960, y: -300, z: -400, targetX: 960, targetY: 540, width: 320, height: 140, intensity: 2, castShadow: true }),
]);
Layer3D.sphere({ radius: 90, color: "#e8e8e8", metallic: 1, roughness: 0.15 }); // chrome
@@ -469,7 +471,11 @@ Layer3D.cylinder({ ..., height: 400, color: "#111", emissive: "#4da3ff", emissiv
- **Tone mapping `"neutral"` (default) keeps brand colors as authored**: orange stays orange. Use `"agx"` for a filmic roll-off.
- **Bloom only takes light well past white** (emissive ≳ 3, hot speculars), so ordinary bright surfaces don't glow. Raise `emissiveIntensity` to make something glow.
-- **Shadows:** directional and spot lights can each cast shadows, subject to device texture-array limits. Meshes cast by default (`castShadow: false` opts out); flat layers (planes, cards) cast only with `castShadow: true`. Everything receives.
+- **Shadows:** `directional`, `spot`, `rect` (`Layer.rectLight` / `Light.rect`), and omnidirectional `point` (`Layer.pointLight` / `Light.point`, 6-face cube shadow map) lights can each cast shadows, subject to device texture-array limits. Meshes cast by default (`castShadow: false` opts out); flat layers (planes, cards) cast only with `castShadow: true`. Everything receives.
+- **Rect Area Lights (`Layer.rectLight` / `Light.rect`):** studio softbox / LED panel lights with `width`, `height`, `twoSided`, `rotation`, `x/y/z`, `targetX/Y/Z`, `intensity`, `radius`, `decay`, and `castShadow`. Uses solid-angle diffuse irradiance and representative-point GGX specular highlights that stretch across rough and glossy surfaces. Numeric light props accept `Signal`s directly (e.g. `Layer.rectLight({ intensity: bassSignal, width: 300, height: 120 })`).
+- **Volumetric Fog & Light Shafts (`volumetricFog`):** `Layer.environment("night", { volumetricFog: true | { density, anisotropy, heightFalloff, baseHeight, color, lightIntensity, ambientIntensity, maxDistance, steps, noise: { scale, strength, speed, octaves } } })` raymarches participating media at half resolution with depth-aware bilateral upsampling. Any shadow-casting `directional`, `spot`, `rect`, or `point` light automatically carves volumetric light shafts (god rays) through the fog behind occluders. `density`, `anisotropy`, and `noise.strength` accept numbers or live `Signal` objects directly.
+- **Substrate lobes (physical shading, needs an environment):** every `Layer3D.*` mesh, `Layer.model` and 3D card accepts `clearcoat` / `clearcoatRoughness` (car paint, lacquer), `anisotropy` (-1..1) / `anisotropyRotation` (deg; brushed metal), `subsurface` / `subsurfaceColor` (wax, jade, skin), `sheen` / `sheenColor` / `sheenRoughness` (velvet, cloth), `iridescence` (0..1 or `{ intensity, thickness, ior }`; soap, oil, beetle shell) with `iridescenceThickness` (nm) / `iridescenceIor`, and `triplanar: true | { scale, sharpness }` for world-space textures on UV-less meshes. Numeric lobes take numbers or live `Signal`s (`clearcoat: bass`). Example: `Layer3D.sphere({ radius: 90, color: "#8a0f1c", metallic: 0.6, roughness: 0.35, clearcoat: 1, clearcoatRoughness: 0.05 })`.
+- **HDRI environments:** pass a `.hdr` or `.exr` path instead of a preset: `Layer.environment("assets/hdri/studio.exr", { intensity: 1.2, rotation: 90 })` (or `{ hdri: path }`). The panorama lights and reflects every physical surface; preset names keep working; relative paths resolve from the working directory. EXR files must use lossless compression (NONE, RLE, ZIPS, ZIP, PIZ or PXR24); re-save B44/DWA files as ZIP.
- **Art-directed materials:** `material: "toon"` works on meshes, flat 3D cards and native 3D text. Its `toon` options are `shadowColor` (shadow-side albedo multiplier; model shade color or `#80808c`), `lightColor` (lit-side multiplier; white), `threshold` (terminator cosine; 0.075), `softness` (transition width; 0.25) and `diffuse` (0 flat through 1 Lambert; 0). Toon surfaces use all scene lights, their colors, point/spot attenuation and real cast shadows; environment exposure, tone mapping, fog and emissive bloom still apply. Lower `threshold` to wrap light around a surface, or raise `shadowColor` for a bright shadow floor on illustrated cards. Physical and unlit materials are independent of these options.
- **Geometry outlines (ink):** `Layer.environment("neutral", { outlines: true })`, or `outlines: { color: "#191721", width: 2, referenceHeight: 1080, creaseAngle: 45, depthThreshold: 0.01, opacity: 1 }`. `width` is px at `referenceHeight`, scaled with the output height; start near 1 px at export height. Ink comes from depth and normals, so textures, shadows and coplanar seams get none; `creaseAngle: 180` keeps only silhouettes. `outline: false` on a layer/model excludes it but keeps it occluding ink behind it (for already-inked artwork). Ink follows fog and runs before depth blur and bloom; heavy lens blur spreads it into wide bands. Separate from the image-space `Toon` effect. `Layer3D.cube({ faces: { front: Layer.box(...) } })` faces inherit the cube's `toon`, `outline`, emissive and shadow settings unless set per face. Example: `examples/34_spatial_styling` in the engine repo.
- **3D pivots:** `anchorX/Y` choose the point held fixed during scale/rotation; changing an anchor at identity transform leaves the layer in place. Use `anchorY: 1` to keep a character's feet planted while scaling.
diff --git a/specs/ue58-engine-roadmap.md b/specs/ue58-engine-roadmap.md
new file mode 100644
index 00000000..893bef28
--- /dev/null
+++ b/specs/ue58-engine-roadmap.md
@@ -0,0 +1,407 @@
+# Spec: Framefields Next-Gen 3D, Material, Lighting & Procedural Engine Roadmap (UE 5.8+ Parity)
+
+## Goal
+
+Bridge the visual fidelity, procedural world-building, material expressiveness, and character/crowd animation gap between **Unreal Engine 5.8+** and **Framefields**—while staying 100% true to Framefields' architecture:
+
+- **Deterministic, code-first `(composition, frame) -> pixels`** in headless Node.js on native WebGPU (Dawn).
+- **Zero editor bloat**: every capability is exposed as a declarative, type-safe TypeScript builder on `Layer`, `Layer3D`, `Lighting`, `Physics`, or `Environment3DOptions`, serializable through `CompositorProgramSchema` (`packages/compositions/src/program/schema.ts`), and linted by `comp.lint()` (`packages/compositions/src/program/lint.ts`).
+- **Composable with the existing stack**: `Camera3D`, `Light3D` (`@group(3)` storage lights + `texture_depth_2d_array` shadow maps), `Mesh3DPipeline`, `Swarm3DPipeline`, `Physics.world3d` (Rapier), `Signal`, `AudioSignal`, and `comp.vision`.
+
+---
+
+## Architectural Pillars & Roadmap Overview
+
+| Phase | Pillar | UE 5.8+ Inspiration | Framefields Surface & Pipeline |
+|---|---|---|---|
+| **Phase 1** | **Volumetric Fog, Light Shafts & Area Lights** | Volumetric Fog Froxels + MegaLights Area Lights | `Layer.environment(..., { volumetricFog })` (`VolumetricFogPipeline`) + `Layer.rectLight(...)` (`lightType: "rect"` LTC shading + dual-paraboloid/omni point shadows in `@group(3)`) |
+| **Phase 2** | **Substrate-Grade Multi-Lobe PBR Materials & HDRI** | Substrate (`Clearcoat`, `Anisotropy`, `Subsurface`, `Sheen`, `Triplanar`) + HDR Skylight | Extended `SurfaceParams` (`SURFACE_PARAMS_FLOATS`), `mesh3d.wgsl` multi-lobe BRDF, `triplanar` UV-less mapping, and `.hdr` / `.exr` equirectangular IBL prefiltering (`HdrEnvironmentLoader`) |
+| **Phase 3** | **Lumen-Lite: Screen-Space GI & Reflections (SSR / SSGI)** | Lumen Lite (60fps screen-space + depth-hierarchy GI & reflections) | `SsrGiPipeline` in `packages/webgpu-renderers/src/renderer3d/` consuming `GeometryResolvePipeline` depth/normals + hierarchical Z (Hi-Z) raymarching |
+| **Phase 4** | **Mesh Terrain, SDF Booleans & PCG Surface Scatter** | UE 5.8 Mesh Terrain + PCG Framework + Procedural Vegetation | `Layer3D.terrain(...)`, `Layer3D.sdf(...)` (Dual Contouring / Marching Cubes CSG), and `Layer3D.scatter(...)` (Poisson-disk surface instancing + GPU vertex wind sway) |
+| **Phase 5** | **Character Rigging, IK, Pose Blending & VAT Crowds** | Control Rig Physics + Direct Mesh Controls + MetaHuman Crowds | Multi-clip animation blending, Two-Bone IK & spring bones in `GltfLoader`/`Renderer3D`, plus Vertex Animation Textures (VAT) in `Layer3D.instances` for 10,000+ skinned crowd agents |
+| **Phase 6** | **Voronoi Destruction & Per-Vertex Motion Blur** | Chaos Destruction + Per-Bone Velocity Motion Blur | `Physics.fracture(mesh, options)` (3D Voronoi cell shattering into `Physics.world3d` + `Layer3D.instances`) and previous-frame skinned/morphed vertex velocity in `MotionBlurPipeline` |
+
+---
+
+## Phase 1: Volumetric Atmosphere, God Rays & Area Lights
+
+### 1.1 Motivation
+Currently, `Environment3DFog` (`packages/webgpu-renderers/src/math3d/environment3d.ts`) is an analytic distance fade (`mix(surfaceColor, fog.color, fogFactor)`) evaluated only on opaque/transparent surfaces. Spotlights and directional lights cannot cast volumetric shafts ("god rays") through empty air, and point/rect lights cannot cast shadows or rectangular softbox highlights on products.
+
+### 1.2 Volumetric Fog & Light Shafts (`VolumetricFogPipeline`)
+Add `volumetricFog` to `Environment3DOptions`:
+
+```ts
+export interface Environment3DVolumetricFog {
+ /** Scattering albedo tint (CSS color). Default "#ffffff". */
+ color?: string;
+ /** Global extinction/density coefficient (0..1). Default 0.025. */
+ density?: number;
+ /** Henyey-Greenstein phase anisotropy (-0.9..0.9; >0 = strong forward scattering / god rays). Default 0.55. */
+ anisotropy?: number;
+ /** Exponential height falloff in world Y px (y down). Optional. */
+ heightFalloff?: { baseY: number; falloff: number };
+ /** 3D animated curl/FBM density noise for patchy theatrical smoke/clouds. */
+ noise?: { scale?: number; strength?: number; speed?: number; velocity?: [number, number, number] };
+ /** Raymarch steps per pixel (16..96). Default 48. */
+ steps?: number;
+ /** Max raymarch distance in world px. Default 6000. */
+ maxDistance?: number;
+}
+```
+
+#### GPU Pipeline (`packages/webgpu-renderers/src/renderer3d/volumetric-fog-pipeline.ts`)
+- Runs after opaque 3D geometry + `GeometryResolvePipeline` (which resolves single-sample linear depth) and before `BloomPipeline` / tone mapping so hot light shafts contribute to HDR bloom.
+- **Half-resolution Temporal/Interleaved Raymarch + Bilateral Upsample**:
+ 1. For each pixel, reconstruct world-space ray from camera eye to `min(sceneLinearDepth, maxDistance)`.
+ 2. Jitter start offset along the ray using Interleaved Gradient Noise (IGN) seeded deterministically by `(pixelXY, frame)` to eliminate banding at 32–48 steps.
+ 3. At each step $\mathbf{p}_k$, evaluate local density $\sigma_t(\mathbf{p}_k)$ (base density $\times$ height falloff $\times$ 3D simplex/FBM noise).
+ 4. Loop over `@group(3) @binding(3) var<storage, read> directLights : array<DirectLight>`:
+ - Evaluate light attenuation (directional, point, spot cone) and Henyey-Greenstein phase function $P(\cos\theta, g)$.
+ - Sample `@group(3) @binding(1) var shadowMaps : texture_depth_2d_array` for shadow-casting lights (`shadowParams0.w > 0.5`) so occluders carve sharp volumetric shadow shafts.
+ 5. Accumulate in-scattered radiance $\mathbf{L}_{\text{scatter}}$ and transmittance $T = \exp(-\int \sigma_t \, ds)$.
+ 6. Depth-aware bilateral upsample onto the linear HDR scene target: $\mathbf{L}_{\text{out}} = \mathbf{L}_{\text{scene}} \cdot T + \mathbf{L}_{\text{scatter}}$.
+
+### 1.3 Rect Area Lights & Omnidirectional Point Shadows
+Extend `LightKind` in `packages/webgpu-renderers/src/math3d/light3d.ts`:
+
+```ts
+export type LightKind = "ambient" | "directional" | "point" | "spot" | "rect";
+
+export interface RectLight3DOptions extends Light3DOptions {
+ lightType: "rect";
+ /** Width of the rectangular emitter in world px. Default 400. */
+ width?: number;
+ /** Height of the rectangular emitter in world px. Default 200. */
+ height?: number;
+ /** Whether the rect light emits from both sides. Default false. */
+ twoSided?: boolean;
+}
+```
+
+#### WGSL Shading (`mesh3d.ts` / `quad3d.ts`)
+- Pack `typeVal = 4.0` for `"rect"` lights; store `width`, `height`, and right/up tangent basis vectors in `DirectLight` (`spotParams` + unused padding floats in the 40-float `DirectLight` record).
+- Evaluate **Linearly Transformed Cosines (LTC)** (or analytic closest-point representative sphere/rectangle approximation with horizon clipping) for GGX specular + Lambertian diffuse, producing true rectangular softbox reflections on curved product surfaces.
+- **Omnidirectional Point Light Shadows**:
+ - For `point` lights with `castShadow: true`, allocate 2 layers in the existing `shadowMaps : texture_depth_2d_array` using **Dual-Paraboloid Shadow Mapping** (front hemisphere $+Z$ and back hemisphere $-Z$), requiring only 2 layers per point light instead of 6 cubemap faces and fitting cleanly into the existing `texture_depth_2d_array` architecture.
+
+---
+
+## Phase 2: Substrate-Grade Multi-Lobe PBR Materials & Real HDRI Environments
+
+### 2.1 Motivation
+UE 5.8's Substrate material system unifies automotive clearcoat, brushed metal anisotropy, skin/wax subsurface scattering, cloth sheen, and thin-film iridescence. Meanwhile, product and studio films frequently need real `.hdr` / `.exr` studio panoramas and UV-less triplanar projection for procedural meshes (`Layer3D.shard`, `Layer3D.extrude`, `Layer3D.terrain`).
+
+### 2.2 Multi-Lobe `SurfaceParams` (`packages/webgpu-renderers/src/renderer3d/surface-params.ts`)
+Extend `SurfaceParams` and `SURFACE_PARAMS_FLOATS` (from `20` to `32` floats = `8 × vec4<f32>`):
+
+```ts
+export interface SurfaceParams {
+ // Existing fields: roughness, metallic, emissive, emissiveIntensity, castShadow, toon, outline
+ /** Clearcoat layer intensity 0..1 (automotive paint, lacquered plastic, wet surfaces). Default 0. */
+ clearcoat?: number;
+ /** Roughness of the clearcoat layer 0..1. Default 0.08. */
+ clearcoatRoughness?: number;
+ /** Anisotropy strength -1..1 (brushed aluminum, vinyl grooves, satin hair). Default 0. */
+ anisotropy?: number;
+ /** Anisotropy tangent rotation in degrees 0..360. Default 0. */
+ anisotropyRotation?: number;
+ /** Subsurface scattering / wrap translucency 0..1 (skin, wax, marble, leaves, gummies). Default 0. */
+ subsurface?: number;
+ /** Subsurface scattering bleed color (CSS color; defaults to warm shift of base color). */
+ subsurfaceColor?: string | Color | [number, number, number];
+ /** Cloth / velvet micro-fiber sheen intensity 0..1. Default 0. */
+ sheen?: number;
+ /** Sheen tint color (CSS color). Default "#ffffff". */
+ sheenColor?: string | Color | [number, number, number];
+ /** Thin-film soap/oil/holo iridescence strength 0..1 and film thickness in nm (200..800). */
+ iridescence?: number | { intensity?: number; thickness?: number; ior?: number };
+ /** World-space triplanar texture projection (eliminates UV stretching on procedural/extruded meshes). */
+ triplanar?: boolean | { scale?: number; sharpness?: number };
+}
+```
+
+#### WGSL Multi-Lobe BRDF (`packages/webgpu-renderers/src/shaders/mesh3d.ts`)
+1. **Clearcoat Lobe**: Secondary isotropic GGX lobe with $F_0 = 0.04$ (IOR 1.5), roughness $\alpha_c = \text{clearcoatRoughness}^2$, attenuating the base substrate energy by $(1 - F_c \cdot \text{clearcoat})$.
+2. **Anisotropic GGX**: Stretch microfacet roughness along tangent $\mathbf{t}$ and bitangent $\mathbf{b}$ rotated by `anisotropyRotation`: $\alpha_t = \alpha (1 + \text{aniso})$, $\alpha_b = \alpha (1 - \text{aniso})$.
+3. **Subsurface Approximation**: Burley screen-free wrap diffuse + back-lit thickness transmission (sampling the light's shadow map depth thickness $d = z_{\text{surface}} - z_{\text{blocker}}$ when available, plus view-dependent back-scatter $\max(0, -\mathbf{v} \cdot (\mathbf{l} + \mathbf{n} \cdot \text{distortion}))^p \cdot \text{subsurfaceColor}$).
+4. **Charlie Sheen Lobe**: Estevez–Kulla (`Charlie`) retro-reflective grazing lobe for fabrics and velvet.
+5. **Thin-Film Iridescence**: Belcour airy-phase spectral interference shifting $F(\cos\theta)$ by optical path difference $2 n_{\text{film}} d \cos\theta_t$.
+6. **Triplanar Mapping**: When `triplanar` is active, sample albedo/normal/roughness maps along $(yz, xz, xy)$ world planes blended by $|\mathbf{n}_{\text{world}}|^{\text{sharpness}}$.
+
+### 2.3 Equirectangular `.hdr` / `.exr` IBL (`HdrEnvironmentLoader`)
+Extend `Layer.environment(presetOrPath, options)` to accept a file path (`"assets/hdri/studio_small.hdr"`):
+- Decode Radiance `.hdr` (RGBE) in pure TypeScript (`packages/webgpu-renderers/src/loaders/hdr-loader.ts`).
+- On first upload, run a one-shot WebGPU compute pass to:
+ 1. Convert equirectangular `rgba16float` into a mipmapped cubemap (`256×256×6`).
+ 2. Convolve GGX prefiltered specular mip chain (`mip 0..6` mapped to `roughness 0..1`).
+ 3. Project diffuse irradiance into L2 Spherical Harmonics (`9 × vec4<f32>`).
+- Bound in `@group(3)` so any scene can switch between zero-asset procedural studio presets and real captured HDRIs with a single string change.
+
+**Status (implemented):** `renderer3d/hdr-environment.ts` prefilters on the GPU (Karis split-sum, filtered importance sampling, 256 samples) into an `rgba16float` cube whose face is a quarter of the panorama width (8..256) with up to 7 mips (roughness `m / (mips - 1)`). SH projection runs on the CPU from a ≤256-wide copy (`projectIrradianceSh`) and rides in `LightsUniforms.envSh`. `.exr` is decoded by `loaders/exr-loader.ts` (scanline or tiled, NONE/RLE/ZIPS/ZIP/PIZ/PXR24, HALF/FLOAT/UINT), checked against files written by the reference OpenEXR library; lossy B44/DWA/HTJ2K are rejected with a clear error.
+
+---
+
+## Phase 3: Lumen-Lite — Screen-Space Reflections (SSR) & Diffuse GI (SSGI)
+
+### 3.1 Motivation
+Planar floor reflection (`FloorReflectionPipeline`) only reflects across a single flat horizontal plane `y = floor.y`. Vertical walls, curved metallic objects, wet streets, and adjacent diffuse surfaces do not reflect or bounce light from nearby geometry or emissive neon signs.
+
+### 3.2 `SsrGiPipeline` (`packages/webgpu-renderers/src/renderer3d/ssr-gi-pipeline.ts`)
+Add `gi` and `ssr` options to `Environment3DOptions`:
+
+```ts
+export interface Environment3DSsrOptions {
+ /** Max reflection intensity 0..1. Default 0.85. */
+ intensity?: number;
+ /** Max roughness that traces screen-space rays (0..1). Default 0.6. */
+ maxRoughness?: number;
+ /** Surface thickness threshold in world px before a ray passes behind an occluder. Default 120. */
+ thickness?: number;
+ /** Hi-Z march steps (16..64). Default 32. */
+ steps?: number;
+}
+
+export interface Environment3DGiOptions {
+ /** Indirect diffuse bounce strength 0..4. Default 1.0. */
+ intensity?: number;
+ /** Screen-space gather radius in world px. Default 400. */
+ radius?: number;
+ /** Rays per pixel (4..16). Default 6. */
+ samples?: number;
+}
+```
+
+#### GPU Architecture
+1. **Depth Mip Pyramid (Hi-Z)**:
+ - After `GeometryResolvePipeline` produces single-sample `linearDepth` and `worldNormal` textures, a compute pass generates a min-depth mip chain (`Hi-Z`, 6 levels).
+2. **Hi-Z Screen-Space Reflections (SSR)**:
+ - Half-res compute pass marches reflection rays $\mathbf{r} = \text{reflect}(-\mathbf{v}, \mathbf{n})$ (jittered by GGX VNDF lobe for `roughness > 0.05`) through the Hi-Z pyramid, skipping empty space in $O(\log N)$ steps and falling back smoothly to the studio/HDRI environment at screen edges.
+3. **Screen-Space Bent-Normal Diffuse GI (SSGI)**:
+ - Upgrading `SsaoPipeline` (`packages/webgpu-renderers/src/renderer3d/ssao-pipeline.ts`) into a combined **GTAO + Bent-Normal + 1-Bounce Color Bleed** pass:
+ - While horizon-scanning depth slices around each pixel, sample the direct-lit HDR color buffer at visible horizon hits, accumulating indirect diffuse irradiance $\mathbf{E}_{\text{indirect}}$ and bent normal $\mathbf{n}_{\text{bent}}$.
+ - Edge-preserving spatial denoise (normal + depth bilateral filter) composites indirect diffuse + specular before bloom and tone mapping.
+
+---
+
+## Phase 4: Procedural Terrain, SDF Booleans & PCG Surface Scatter
+
+### 4.1 Motivation
+UE 5.8's headline world-building features are **Mesh Terrain** (caves, overhangs, cliffs beyond 2.5D heightfields) and **PCG** (rule-driven surface scattering). In Framefields, bringing these to `Layer3D` turns 20 lines of TypeScript into rich landscapes, organic boolean sculptures, and populated worlds.
+
+### 4.2 `Layer3D.terrain(options)`
+Add procedural terrain generation to `packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts` and `packages/framefields/src/shapes3d.ts`:
+
+```ts
+export interface Terrain3DOptions extends Primitive3DNodeOptions {
+ width: number;
+ depth: number;
+ /** Peak-to-trough vertical relief in world px. Default 300. */
+ height?: number;
+ /** Grid subdivisions along X and Z (e.g. [128, 128]). Default [128, 128]. */
+ segments?: [number, number];
+ /** Multi-octave FBM/ridged-multifractal noise parameters. */
+ noise?: {
+ seed?: number;
+ scale?: number;
+ octaves?: number;
+ persistence?: number;
+ lacunarity?: number;
+ /** "fbm" (rolling hills) | "ridged" (sharp mountain crests) | "terrace" (stepped plateaus). */
+ ridgeMode?: "fbm" | "ridged" | "terrace";
+ terraceSteps?: number;
+ };
+ /** Optional 3D overhang/arch warp displacement amplitude in px (UE 5.8 Mesh Terrain style). */
+ overhangWarp?: number;
+ /** Optional custom height function `(u: number, v: number) => number` in [0, 1]. */
+ heightFn?: (u: number, v: number) => number;
+ /** Automatic vertex-color biome splatting by altitude and slope. */
+ colorByElevation?: {
+ lowColor: string;
+ midColor?: string;
+ highColor: string;
+ cliffColor?: string;
+ /** Slope cosine below which `cliffColor` blends in (default 0.65). */
+ cliffThreshold?: number;
+ };
+ flatShaded?: boolean;
+}
+```
+- Generates vertex colors (`COLOR_0`) from elevation and surface normal slope (`n.y`), which `Mesh3DPipeline` already multiplies with base material properties—giving multi-material snow/rock/grass/sand terrains in a single draw call with zero texture assets.
+- Integrates with `Physics.world3d` via Rapier's `trimesh` / `heightfield` collider so rigid bodies and `Layer3D.swarm` particles roll down valleys and collide with ridges.
+
+### 4.3 `Layer3D.sdf(options)` — Signed Distance Field & CSG Boolean Meshing
+Allow authoring smooth boolean CSG shapes and organic metaballs meshed via Surface Nets / Dual Contouring:
+
+```ts
+export type SdfPrimitive =
+ | { op?: "union" | "subtract" | "intersect"; smoothness?: number; shape: "sphere"; center?: [number, number, number]; radius: number }
+ | { op?: "union" | "subtract" | "intersect"; smoothness?: number; shape: "box"; center?: [number, number, number]; size: [number, number, number]; radius?: number }
+ | { op?: "union" | "subtract" | "intersect"; smoothness?: number; shape: "cylinder" | "capsule" | "torus"; center?: [number, number, number]; radius: number; height?: number; tube?: number }
+ | { op?: "union" | "subtract" | "intersect"; smoothness?: number; shape: "gyroid"; scale: number; thickness: number };
+
+export interface Sdf3DOptions extends Primitive3DNodeOptions {
+ /** List of SDF primitives combined in order via smooth/hard CSG operations (`smin` / `smax`). */
+ primitives: SdfPrimitive[];
+ /** Bounding volume [width, height, depth] in world px. */
+ bounds: [number, number, number];
+ /** Voxel grid resolution (16..128). Default 64. */
+ resolution?: number;
+}
+```
+- Evaluates the composite SDF on a 3D grid and extracts a smooth, watertight triangle mesh with analytic gradient normals $\nabla f(\mathbf{x}) / |\nabla f(\mathbf{x})|$, cached in `getPrimitiveGeometry`.
+
+### 4.4 `Layer3D.scatter(options)` — Declarative PCG Surface Scattering & Wind
+Build directly on `Layer3D.instances` to scatter thousands of rocks, trees, crystals, or buildings across any `Layer3D` surface or `terrain`:
+
+```ts
+export interface Scatter3DOptions extends Omit<Instances3DOptions, "count" | "instanceAt"> {
+ /** Target surface mesh (`Layer3D.terrain(...)` or any `Model3DNode`) to scatter onto. */
+ surface: AnimatableNode<Model3DNode>;
+ /** Number of instances to place (or Poisson-disk `minDistance` in px). */
+ count: number;
+ minDistance?: number;
+ seed?: number;
+ /** Filter candidate surface triangles by slope angle (degrees, e.g. [0, 35]) and Y altitude range. */
+ slopeRange?: [number, number];
+ elevationRange?: [number, number];
+ /** Align instance Y axis to surface normal (`1` = full normal alignment, `0` = stay vertical). Default 0.3. */
+ alignToNormal?: number;
+ /** Random uniform scale range `[min, max]` and random Y-axis yaw. */
+ scaleRange?: [number, number];
+ randomYaw?: boolean;
+ /** Optional spline/circle exclusion zones where density drops to 0 (e.g. clearing a road or stage). */
+ exclude?: Array<{ x: number; z: number; radius: number; falloff?: number }>;
+ /** Procedural GPU/CPU wind sway applied per instance (`frequency`, `amplitudeDeg`, `waveScale`). */
+ wind?: { amplitudeDeg?: number; frequency?: number; direction?: [number, number]; waveScale?: number };
+}
+```
+- Samples surface triangles weighted by area, filters by `slopeRange` / `elevationRange` / `exclude`, enforces Poisson-disk spacing via a spatial hash grid, and outputs an `Instances3DNode` rendered in 1 GPU draw call per mesh variant.
+
+---
+
+## Phase 5: Character Animation Blending, IK, Spring Bones & Skinned VAT Crowds
+
+### 5.1 Motivation
+In `examples/35_robot_uprising` and character-driven films, `Layer.model` currently plays a single glTF animation clip at a time, and `Layer3D.instances` only supports rigid unskinned meshes. We need multi-clip pose blending, procedural LookAt/Two-Bone IK, secondary spring physics on bones, and **instanced crowds of thousands of animated characters**.
+
+### 5.2 Multi-Clip Blending, Two-Bone IK & Spring Bones on `Layer.model`
+Extend `Model3DNode` and `GltfLoader` (`packages/webgpu-renderers/src/loaders/gltf-loader.ts`):
+
+```ts
+export interface ModelAnimationLayer {
+ /** Clip name or index in the glTF asset. */
+ clip: string | number;
+ /** Blend weight 0..1 (number, Signal, or keyframed prop). Default 1. */
+ weight?: number;
+ /** Playback speed multiplier. Default 1. */
+ speed?: number;
+ /** Time offset in seconds. Default 0. */
+ timeOffset?: number;
+ /** Optional bone mask (only affect descendants of these joint names). */
+ mask?: string[];
+}
+
+export interface TwoBoneIkConstraint {
+ /** [rootJointName, midJointName, effectorJointName], e.g. ["UpperArm.L", "LowerArm.L", "Hand.L"]. */
+ chain: [string, string, string];
+ /** World or model-local target position [x, y, z] (or driver `(frame) => [x, y, z]`). */
+ target: [number, number, number] | ((frame: number) => [number, number, number]);
+ /** Pole vector target for knee/elbow bend direction. */
+ poleTarget?: [number, number, number];
+ weight?: number;
+}
+
+export interface LookAtConstraint {
+ joint: string;
+ target: [number, number, number] | ((frame: number) => [number, number, number]);
+ weight?: number;
+ maxAngleDeg?: number;
+}
+
+export interface SpringBoneChain {
+ /** Root joint name(s) for secondary jiggle/follow-through (cape, tail, antenna, hair). */
+ joints: string[];
+ stiffness?: number;
+ damping?: number;
+ gravity?: [number, number, number];
+}
+```
+
+#### Deterministic Evaluation Order per Frame
+1. Sample each active `ModelAnimationLayer` TRS (translation, rotation quaternion, scale) and blend via normalized quaternion `slerp` / `nlerp` (with optional upper-body/lower-body `mask`).
+2. Solve `LookAtConstraint` (clamped swing quaternion) and analytic **Two-Bone IK** (law of cosines for knee/elbow bend angle + pole-vector twist) in joint hierarchy order.
+3. Step `SpringBoneChain` Verlet / harmonic spring state (pre-simulated deterministically across frames `[0..totalFrames]` on build, identical to `Physics.world3d`, so random-access frame rendering stays 100% deterministic).
+4. Upload final `skinningMatrices` (`16 floats × jointCount`) to `Mesh3DPipeline`.
+
+### 5.3 Skinned Crowds via Vertex Animation Textures (VAT) in `Layer3D.instances`
+To render 500–10,000 animated characters (robots, soldiers, pedestrians) in 1 GPU draw call:
+- When `Layer3D.instances({ mesh: Layer.model({ src: "robot.glb", ... }), crowd: { clips: ["walk", "idle", "run"] }, ... })` is constructed:
+ 1. **Bake Joint Palette Atlas once per model**: Sample the model's skinning joint matrices across each animation clip at 30/60 Hz into a 2D `rgba32float` texture (`width = jointCount * 4`, `height = totalBakedFrames`).
+ 2. Pack `(clipIndex, animPhaseOffset, animSpeed)` into the unused bits of `InstanceData.emissive.w` (or a companion per-instance `vec4<f32>`).
+ 3. In `mesh3d.wgsl` (`vs_main`), when both `isInstanced` and `isSkinned` are active, each vertex fetches its 4 influencing bone matrices from the baked joint palette storage/texture buffer at the instance's `(clipIndex, frame * speed + phaseOffset)`.
+- Result: **5,000 independently walking/running skinned 3D characters with shadows, outlines, and bloom in a single draw call**.
+
+---
+
+## Phase 6: Voronoi Mesh Fracture & Per-Vertex Velocity Motion Blur
+
+### 6.1 `Physics.fracture` — Deterministic Voronoi Shattering
+Enable objects (statues, walls, typography, product cases) to remain a single seamless mesh until a designated impact frame, then shatter into physical 3D Voronoi fragments:
+
+```ts
+export interface Fracture3DOptions {
+ /** Source solid (`Layer3D.*` primitive, `Layer3D.extrudedText`, or `Layer.model`). */
+ mesh: AnimatableNode<Model3DNode>;
+ /** Number of Voronoi fracture cells (8..256). Default 48. */
+ cells?: number;
+ /** Optional world/local impact point where Voronoi seeds cluster more densely. */
+ impactPoint?: [number, number, number];
+ /** Deterministic seed. Default 1. */
+ seed?: number;
+ /** Frame at which the intact mesh swaps to dynamic Rapier rigid bodies (or `"onImpact"`). */
+ fractureFrame?: number | "onImpact";
+ /** Optional interior surface material/color for newly exposed fracture faces. */
+ interiorColor?: string;
+ interiorRoughness?: number;
+ /** Outward radial impulse speed applied at `fractureFrame` (px/s). */
+ explosionSpeed?: number;
+}
+```
+- Uses the convex half-space plane clipper from `Layer3D.shard` generalized to **3D Voronoi cell clipping** (clipping the source mesh against the bisector planes $\mathbf{n}_{ij} = \mathbf{p}_j - \mathbf{p}_i$ of its $K$-nearest seed neighbors).
+- Registers each cell with `Physics.world3d` (holding `type: "fixed"` until `fractureFrame`, then switching to `"dynamic"` with `explosionSpeed` impulse) and returns a single `Layer3D.instances` node that renders all cells in one draw call.
+
+### 6.2 Per-Vertex & Per-Instance Velocity Motion Blur (`MotionBlurPipeline`)
+Currently, `MotionBlurPipeline` (`packages/webgpu-renderers/src/renderer3d/motion-blur-pipeline.ts`) reconstructs pixel velocity only from the camera's previous view-projection matrix.
+- Upgrade the MRT velocity pass to accept `prevModelMatrix`, `prevSkinningMatrices`, and `prevInstanceBuffer` (`Swarm3DPipeline` / `Layer3D.instances`).
+- Fast-moving robot limbs, tumbling shards, and fractured debris get authentic directional per-object and per-bone shutter motion blur.
+
+---
+
+## Schema, Lint & Skill Synchronization Requirements
+
+Every phase must keep the engine harness self-consistent:
+1. **`packages/compositions/src/program/schema.ts`**:
+ - Add Zod schemas for `volumetricFog`, `ssr`, `gi`, `rectLight`, `SurfaceParams` extensions (`clearcoat`, `anisotropy`, `subsurface`, `sheen`, `iridescence`, `triplanar`), `Layer3D.terrain`, `Layer3D.sdf`, `Layer3D.scatter`, `ModelAnimationLayer`, `ik`, `springBones`, and `Physics.fracture`.
+2. **`packages/compositions/src/program/lint.ts`**:
+ - Warn if `clearcoat`, `anisotropy`, or `ssr` is used without `Layer.environment`.
+ - Warn if `volumetricFog` is enabled with zero direct lights in the scene.
+ - Warn if `Layer3D.scatter` filters (`slopeRange` / `elevationRange`) reject 100% of candidate surface points.
+3. **Agent Skills & Catalog**:
+ - Update `nodes/node-compositor/SKILL.md` and `plugins/framefields/skills/framefields-compose/SKILL.md`.
+ - Keep `packages/compositions/src/program/skillmd-check.test.ts` and `pnpm run check:plugins` passing.
+
+---
+
+## Verification Ladder (Per Phase)
+
+1. **Typecheck**: `tsc --noEmit -p .` in `@framefields/core`, `@framefields/webgpu-renderers`, `@framefields/compositions`, `@framefields/node-compositor`, and `framefields`.
+2. **Schema & Lint Tests**: Unit tests in `packages/compositions` verifying `CompositorProgramSchema.parse(comp.toSpec())` and `comp.lint()` diagnostics.
+3. **Headless GPU Render Tests**: Vitest pixel-probe tests in `packages/webgpu-renderers/src/renderer3d/renderer3d.test.ts` verifying:
+ - Volumetric light shafts occluded by shadow casters.
+ - Clearcoat / anisotropic / subsurface / HDRI shading energy conservation.
+ - Hi-Z SSR reflection of off-floor emissive objects and SSGI color bleeding.
+ - Watertight `Layer3D.terrain` and `Layer3D.sdf` meshes + deterministic `Layer3D.scatter` placement.
+ - Two-Bone IK target convergence and instanced VAT skinned crowd rendering in 1 draw call.
+4. **Visual Reference Film (`examples/`)**: Render frame grids (`renderFrameGrid`) to inspect visual quality before merging each phase.
+