Skip to content

Code-first video & image, rendered natively on WebGPU.

Merge main: cube cameras with the 3D roadmap features

okanaslccdb70d011a9
diff --cc .changeset/cube-camera-3d-features.md
index 00000000,00000000..1f5b794d
new file mode 100644
--- /dev/null
+++ b/.changeset/cube-camera-3d-features.md
@@@ -1,0 -1,0 +1,5 @@@
++---
++"framefields": patch
++---
++
++Cube cameras draw volumetric fog on every face. Combining a cube camera with motion blur or screen-space reflections and GI, which need one perspective view, throws a clear error, and `comp.lint()` reports `camera_cube_motion_blur` and `camera_cube_screen_space` (as it does `camera_cube_dof`).
diff --cc nodes/node-compositor/SKILL.md
index 5f50822d,cd2488f8..9e667b85
--- a/nodes/node-compositor/SKILL.md
+++ b/nodes/node-compositor/SKILL.md
@@@ -211,17 -211,9 +211,18 @@@ Each track represents animatable proper
- **Picture-in-Picture:** a `flex` row with two `media` nodes (`fit: "cover"`, each `grow: 1`).
- **Audio-Reactive 3D Models:** a `kind: "model3d"` node loading an OBJ/FBX/GLTF/STL/PLY mesh with `audioDeform: { audioTrackId, mode: "normal_extrusion" | "radial_pulse" | "harmonic_wave" | "twist" | "ripple", amplitudeMultiplier, damping }`, deformed directly in VRAM via WebGPU compute shaders driven by audio frequency spectra and transient energies. `twist` turns each slice about Y by its height (harder on the bass); `ripple` runs concentric waves across XY, struck by the drums. `audioTrackId` may be an audio layer's id or an audio file path.
- **Several 3D scenes:** a `box`/`flex`/`block` with a numeric `width` and `height` that holds a `camera` and the layers it films renders as its own scene (one holding only a camera stays a shot in a camera cut list): a nested composition with only its own camera, lights and environment, on its own clock (children start from the box's frame 0), placed in its parent like a media layer. Transitions, opacity and mattes on the box then blend finished pictures. Without that, every 3D layer shares one world and the active camera (the latest to start).
-- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; directional and spot lights can each cast shadows, up to the device texture-array layer limit.
+- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; `directional`, `spot`, `rect` (1 layer each), and `point` lights (6 cube-face layers) can each cast shadows, up to the device texture-array layer limit.
+- **Rect Area Lights (`lightType: "rect"`):** rectangular softbox/panel area lights (`width`, `height`, `twoSided`, `rotation`, `x/y/z`, `targetX/Y/Z`) use solid-angle subtended diffuse irradiance and Karis representative-point GGX specular shading, and can cast perspective frustum shadows (`castShadow: true`).
+- **Volumetric Fog & Light Shafts (`volumetricFog` in `environment`):** `kind: "light"` with `lightType: "environment"` and `volumetricFog: true | { density, anisotropy, heightFalloff, baseHeight, color, lightIntensity, ambientIntensity, maxDistance, steps, noise: { scale, strength, speed, octaves } }` runs half-resolution raymarching with Interleaved Gradient Noise jitter, Henyey-Greenstein phase scattering, 3D FBM wind-animated density noise, and multi-light shadow-map sampling (carving god rays behind shadow casters), followed by depth-aware bilateral upsampling. `volumetricFogDensity`, `volumetricFogAnisotropy`, and `volumetricFogNoiseStrength` can be animated or driven by `Signal`s.
+- **Substrate multi-lobe materials:** physical-shading surfaces (any scene with an `environment` light) on `model3d` nodes, 3D cards and 3D text accept extra lobes on top of `roughness`/`metallic`: `clearcoat` 0..1 + `clearcoatRoughness` (default 0.08; IOR 1.5 GGX varnish that attenuates the base), `anisotropy` -1..1 + `anisotropyRotation` degrees (tangent-space anisotropic GGX: brushed metal, satin), `subsurface` 0..1 + `subsurfaceColor` (default `#ff6b4a`; wrap diffuse, terminator bleed and back-lit transmission), `sheen` 0..1 + `sheenColor` + `sheenRoughness` (default 0.5; Charlie cloth/velvet lobe), `iridescence` 0..1 or `{ intensity, thickness, ior }` + `iridescenceThickness` nm (100..1200, default 400) + `iridescenceIor` (default 1.3; RGB thin-film interference), and `triplanar: true | { scale, sharpness }` + `triplanarScale` (repeats per 256 world px) + `triplanarSharpness` (1..16, default 4) for world-space UV-less texture projection. All numeric lobe props are animatable.
+- **HDRI environments:** an environment light's `hdri` takes an equirectangular panorama: a Radiance `.hdr` (RGBE) or OpenEXR `.exr` file path (resolved from the working directory) or the file's bytes (told apart by magic bytes); it then replaces the procedural preset as the image-based light for reflections and ambient, honoring `intensity`, `rotation` and `exposure`. On first use per device a compute pass prefilters it into a GGX radiance cubemap (faces up to 256², one mip per roughness step, split-sum importance sampling) for reflections, and the CPU projects it onto L2 spherical harmonics for diffuse irradiance; both are cached (up to 8 panoramas per device, least recently used destroyed). `.exr` reads single-part scanline or tiled files (level 0 of mip/ripmaps) with NONE, RLE, ZIPS, ZIP, PIZ or PXR24 compression and HALF/FLOAT/UINT `R`,`G`,`B` (or `Y`) channels; lossy B44/B44A/DWAA/DWAB/HTJ2K, deep and multi-part files fail with an error naming the cause.
+- **Screen-space reflections & GI (`ssr`, `gi` in `environment`):** `ssr: true | { intensity (0..1, default 0.85), maxRoughness (0..1, 0.6), thickness (world px, 120), steps (16..64, 32) }` makes glossy physical surfaces reflect whatever the camera sees (walls, curved metal, emissives off the floor plane); `gi: true | { intensity (0..4, 1), radius (world px, 400), samples (4..16, 6) }` adds ambient occlusion and one diffuse bounce of visible light (color bleeding from neon and lit walls). Either one renders the 3D scene twice: a radiance pass, then Hi-Z reflection tracing and a horizon-scan GI gather (bilateral-denoised) on it, then the shading pass, which swaps each surface's environment reflection for the traced one (glossy ones read a blurred radiance pyramid matching their lobe, so output is noise-free and identical every frame) and occludes its ambient light before tone mapping and fog. Screen-space limits apply: anything off screen or hidden behind other geometry reflects the environment instead; the planar `floor` reflection is unaffected. Needs physical shading; non-physical (legacy, toon) surfaces ignore it.
+- **Procedural geometry (`primitive3d.type: "terrain" | "sdf"`):** terrain is a seeded fractal heightfield (`noise: { seed, scale px, octaves, persistence, lacunarity, ridgeMode: "fbm" | "ridged" | "terrace", terraceSteps }`, or `heights` 0..1 on the `(segments + 1)²` grid), `height` px of relief above the node's y (up is -y), optional `overhangWarp` px, `colorByElevation { lowColor, midColor?, highColor, cliffColor?, cliffThreshold, bands }` as vertex colors, closed into a block `base` px thick. `sdf` combines `primitives` (sphere, box, cylinder, capsule, torus, gyroid; `op` union / subtract / intersect, `smoothness` px, `center`, `rotation`, `color`) clipped to `bounds` and meshes them watertight with Surface Nets (`resolution` 16..160 cells). Both are strictly validated and rebuilt from the spec when `modelData` is lost.
+- **Scatter (`scatter3d` on an `instances3d` node):** `{ surface: <model node spec>, count, minDistance, seed, slopeRange deg, elevationRange 0..1, alignToNormal, scaleRange, randomYaw, exclude: [{ x, z, radius, falloff }], sink }`. The compositor loads the surface (primitive, in-memory or file), samples it by area once (cached) and seats each instance on its variant's base; instance i draws variant i % n, in the variant's own material color under the node tint.
+- **Wind (`wind` on any model node):** `{ amplitudeDeg (8), frequency Hz (0.4), direction [x, z], waveScale px (600) }` bends vertices downwind in the vertex shader by height above the mesh base (instances sway out of phase along a traveling wave); deterministic per frame, shadows included.
+- **Texture layers (`textureLayer: { node }` on a model):** hoisted like a matte to a sibling with `textureOf`, drawn alone each frame (video plays), then bound as the base-color texture of every mesh the model queues, read through its rect in that picture (`uvRect`); without its own geometry it fills the frame (the whole UV square).
- **Art-directed 3D lighting:** `material: "toon"` on meshes, flat 3D cards and native 3D text uses the shared direct lights, colors, point/spot attenuation, cast shadows, environment fog and tone mapping. Optional `toon: { shadowColor, lightColor, threshold, softness, diffuse }` controls the shadow/lit albedo multipliers, terminator cosine/width and Lambert modelling (defaults: model shade color or `#80808c`, white, 0.075, 0.25, 0). Lit/physical materials retain their own shading.
-- **Cube cameras:** a camera whose `lens.projection` is `"equirectangular"` or `"fisheye"`, or that has `relativity`, draws the scene as six 90° faces into a mipmapped cube (`cubeFaceCameras`, layer order +X −X +Y −Y +Z −Z; the right-handed view and left-handed cube sampling differ by a z reflection, `CUBE_FLIP`) and `Renderer3D.remapLens` turns it into the output picture. Each face gets its own floor reflection and outline pass (ink width rescaled to the output's density at the view centre); shadows are drawn once; bloom reads the remapped radiance. `relativity` aberrates every ray (`cos θ = (cos θ' − β)/(1 − β cos θ')`) and applies a spectral Doppler/beaming matrix tabulated in ln D. Depth of field throws; glass gets no 2D backdrop.
++- **Cube cameras:** a camera whose `lens.projection` is `"equirectangular"` or `"fisheye"`, or that has `relativity`, draws the scene as six 90° faces into a mipmapped cube (`cubeFaceCameras`, layer order +X −X +Y −Y +Z −Z; the right-handed view and left-handed cube sampling differ by a z reflection, `CUBE_FLIP`) and `Renderer3D.remapLens` turns it into the output picture. Each face gets its own floor reflection, outline pass (ink width rescaled to the output's density at the view centre) and volumetric fog; shadows are drawn once; bloom reads the remapped radiance. Screen-space reflections and GI and motion blur need one perspective screen, so with a cube camera they throw, like depth of field. `relativity` aberrates every ray (`cos θ = (cos θ' − β)/(1 − β cos θ')`) and applies a spectral Doppler/beaming matrix tabulated in ln D. Glass gets no 2D backdrop.
- **Geometry outlines:** an environment light's optional `outlines: true | { color, width, referenceHeight, creaseAngle, depthThreshold, opacity }` runs a depth/normal ink pass before camera depth blur and bloom (defaults: `#16131c`, 1.5 px at height 1080, 45°, 0.01, 1). It reuses the linear-depth attachment's spare channels for octahedral surface normals and outline participation. Inverse-depth curvature avoids false contours on perspective planes; normal discontinuities find creases without inking texture/shadow/triangulation seams. `outline: false` on a layer/model opts out while retaining depth occlusion; `creaseAngle: 180` disables creases. Ink fades into the environment fog. 3D `anchorX/Y` pivots remain fixed under scale/rotation and leave identity placement unchanged.
With MSAA enabled, retain original scene color and geometry samples and shade ink per sample before resolving color. A single-sample ink pass after the scene resolve loses subpixel coverage between outlined surfaces. Lens blur uses a separate nearest-sample geometry resolve. The scene's geometry is still drawn once.
- **Transparent meshes:** glTF `BLEND` materials and meshes with reduced layer, material, tint, or vertex alpha draw after opaque items, back to front by the camera-space center of each mesh’s transformed bounds, without writing depth. This is object-level sorting: intersecting meshes, self-overlap, skinned/deformed bounds, translucent flat layers, glass, and transparent shadows have separate limitations.
diff --cc nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 58170994,f29d95c2..edda4300
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@@ -45,13 -41,11 +45,14 @@@ import
Camera3D,
type Camera3DPose,
type Color,
+ type CrowdBake,
+ cubeFaceCameras,
type DrawMesh3DOpts,
type DrawQuad3DOpts,
+ type Environment3DFloor,
type Environment3DOptions,
type ExtrudedTextGeometryOptions,
+ FRACTURE_INTERIOR_MATERIAL,
fitSpansToText,
type GPUMeshBuffers,
generateExtrudedTextGeometry,
@@@ -89,28 -82,9 +90,29 @@@
} from "@framefields/webgpu-renderers";
import { compileTimeline, seekTimeline } from "../shared/compiler.js";
import type { CompositorOperation } from "../shared/config.js";
+import {
+ type CrowdInstanceAnim,
+ crowdBake,
+ crowdRecord,
+ namedMorphWeights,
+ paletteBuffer,
+ poseCharacter,
+} from "./character.js";
import { shouldPaintContainerBackground } from "./container-bg.js";
+ import { type CubeCamera, resolveCubeCamera } from "./cube-camera.js";
+import { fracturedModel, fracturePose, fractureWhole } from "./fracture.js";
import { compatibleMeshDraws } from "./mesh-instancing.js";
+import { type MotionHistory, motionHistory } from "./motion-history.js";
+import {
+ composeQuat,
+ meshWind,
+ NO_PRE,
+ type SurfaceNodeSpec,
+ scatterTransforms,
+ surfaceModel,
+ type VariantPre,
+ variantBottom,
+} from "./procedural3d.js";
import {
buildLayerMatrix,
layerRasterScale,
@@@ -1353,9 -1242,8 +1355,11 @@@ export const CompositorWebGPURenderer:
/** Where a relief keeps its look from: the default camera at the scene's field of view. */
let reliefEye: [number, number, number] = [0, 0, 0];
let dofEnabled = false;
+ /** Shutter motion blur of the 3D scene, and what its draws looked like a frame ago. */
+ let motionBlur: CameraMotionBlur | null = null;
+ let history: MotionHistory | null = null;
+ /** Set when the camera draws a cube map and remaps it through its lens. */
+ let cubeCamera: CubeCamera | null = null;
const sceneLights: Light3DOptions[] = [];
let sceneEnvironment: Environment3DOptions | null = null;
let sceneBloom: BloomOptions | null = null;
@@@ -1460,12 -1348,20 +1464,26 @@@
camStub?.maxBlurRadius ?? camOp?.dof?.maxBlurRadius ?? 24;
}
+ const blur = (camOp as CameraNode | undefined)?.motionBlur;
+ if (blur && fps) {
+ motionBlur = blur === true ? {} : blur;
+ history = motionHistory(ctx.device, compiled.key);
+ }
+
+ cubeCamera = resolveCubeCamera(
+ camOp,
+ camera,
+ camStub?.relativitySpeed ?? camOp?.relativity?.speed ?? 0,
+ nativeWidth,
+ nativeHeight,
+ ctx.device.limits.maxTextureDimension2D,
+ );
+ if (cubeCamera && dofEnabled) {
+ throw new Error(
+ `Camera '${camOp?.id}' is a cube camera (${camOp?.relativity ? "relativity" : camOp?.lens?.projection}) with depth of field: depth of field needs a single perspective view.`,
+ );
+ }
+
// Collect Active Lights
for (const lvm of lightVMs) {
const lop = lvm.operation as any;
@@@ -2004,115 -1698,119 +2022,149 @@@
...glass,
];
- // Planar floor reflection pass (if environment.floor is active)
- // One view of the scene: its floor reflection, then every item.
++ // Screen-space light, motion blur and depth of field need one
++ // perspective picture; a cube camera draws six faces and remaps them.
++ if (cubeCamera) {
++ const single = [
++ screenIndirectEnabled && "screen-space reflections and GI (ssr, gi)",
++ motionBlur && "motion blur",
++ ].filter(Boolean);
++ if (single.length) {
++ throw new Error(
++ `A cube camera (${cubeCamera.remap.projection ?? "lens"}) cannot draw ${single.join(" or ")}: they need a single perspective view.`,
++ );
++ }
++ }
++
const floorSpec = sceneEnvironment?.floor;
- let blurredReflectionTex: GPUTexture | null = null;
- if (
- floorSpec &&
- typeof floorSpec.y === "number" &&
- camera.eye[1] < floorSpec.y
- ) {
- const drawView = (
++ const sceneShadow = shadow.length ? shadow : null;
++ /** The floor's mirror image from `view`, blurred by its roughness, when the eye is above it. */
++ const floorReflection = (
+ view: Camera3DPose,
+ width: number,
+ height: number,
- colorView: GPUTextureView,
- geometryView: GPUTextureView | undefined,
- hdrTargetView: GPUTextureView | undefined,
- /** The 2D picture glass refracts; a cube face has none. */
- backdrop: GPUTexture | undefined,
- ): GPUBuffer => {
- let blurredReflectionTex: GPUTexture | null = null;
++ ): GPUTexture | null => {
+ if (
- floorSpec &&
- typeof floorSpec.y === "number" &&
- view.eye[1] < floorSpec.y
++ !floorSpec ||
++ typeof floorSpec.y !== "number" ||
++ view.eye[1] >= floorSpec.y
+ ) {
- const reflectedCamera: Camera3DPose = {
- ...view,
- eye: [view.eye[0], 2 * floorSpec.y - view.eye[1], view.eye[2]],
- target: [
- view.target[0],
- 2 * floorSpec.y - view.target[1],
- view.target[2],
- ],
- up: [view.up[0], view.up[1], view.up[2]],
- };
- const {
- pass: reflPass,
- cameraBindGroup: reflCamBind,
- lightsBindGroup: reflLightsBind,
- reflectionTexture: reflTex,
- } = r3d.beginReflectionPass(
- encoder,
- width,
- height,
- reflectedCamera,
- sceneLights,
- { environment: sceneEnvironment, shadow: null },
- );
- for (const item of ordered) {
- if (item.kind === "quad") {
- r3d.drawTextureQuad(
- reflPass,
- reflCamBind,
- reflLightsBind,
- item.texture,
- item.opts,
- false,
- );
- } else if (item.kind === "slug") {
- r3d.drawSlugText(
- reflPass,
- reflCamBind,
- reflLightsBind,
- item.batches,
- item.opts,
- item.stackBase,
- false,
- );
- } else if (item.kind === "mesh") {
- r3d.drawMesh3D(
- reflPass,
- reflCamBind,
- reflLightsBind,
- item.meshBuffers ?? item.mesh,
- item.texture,
- item.opts,
- false,
- );
- }
- }
- reflPass.end();
- blurredReflectionTex = r3d.blurReflection(
- encoder,
- reflTex,
- floorSpec.roughness ?? 0.35,
- );
++ return null;
+ }
-
+ const reflectedCamera: Camera3DPose = {
- ...camera,
- eye: [camera.eye[0], 2 * floorSpec.y - camera.eye[1], camera.eye[2]],
++ ...view,
++ eye: [view.eye[0], 2 * floorSpec.y - view.eye[1], view.eye[2]],
+ target: [
- camera.target[0],
- 2 * floorSpec.y - camera.target[1],
- camera.target[2],
++ view.target[0],
++ 2 * floorSpec.y - view.target[1],
++ view.target[2],
+ ],
- up: [camera.up[0], camera.up[1], camera.up[2]],
++ up: [view.up[0], view.up[1], view.up[2]],
+ };
const {
- pass: pass3d,
- cameraBindGroup,
- cameraBuffer,
- lightsBindGroup,
- } = r3d.beginPass(
+ pass: reflPass,
+ cameraBindGroup: reflCamBind,
+ lightsBindGroup: reflLightsBind,
+ reflectionTexture: reflTex,
+ } = r3d.beginReflectionPass(
encoder,
- nativeWidth,
- nativeHeight,
- colorView,
+ width,
+ height,
- view,
+ reflectedCamera,
+ sceneLights,
+ { environment: sceneEnvironment, shadow: null },
+ );
+ for (const item of ordered) {
+ if (item.kind === "quad") {
+ r3d.drawTextureQuad(
+ reflPass,
+ reflCamBind,
+ reflLightsBind,
+ item.texture,
+ item.opts,
+ false,
+ );
+ } else if (item.kind === "slug") {
+ r3d.drawSlugText(
+ reflPass,
+ reflCamBind,
+ reflLightsBind,
+ item.batches,
+ item.opts,
+ item.stackBase,
+ false,
+ );
+ } else if (item.kind === "mesh") {
+ r3d.drawMesh3D(
+ reflPass,
+ reflCamBind,
+ reflLightsBind,
+ item.meshBuffers ?? item.mesh,
+ item.texture,
+ item.opts,
+ false,
+ );
+ }
+ }
+ reflPass.end();
- blurredReflectionTex = r3d.blurReflection(
++ return r3d.blurReflection(
+ encoder,
+ reflTex,
+ floorSpec.roughness ?? 0.35,
+ );
- }
++ };
+
- const sceneShadow = shadow.length ? shadow : null;
- /** One forward pass over the scene: floor, then the sorted draws. */
++ /** Where one draw of the batch goes, and how. */
++ interface ViewTarget {
++ view: Camera3DPose;
++ width: number;
++ height: number;
++ color: GPUTextureView;
++ geometry: GPUTextureView | undefined;
++ /** Where the radiance resolves instead of `hdrSource()` (a cube face). */
++ hdr?: GPUTextureView;
++ reflection: GPUTexture | null;
++ }
+ /**
- * One draw of the batch. `glassBackdrop` is what glass refracts;
- * null leaves the glass out (the pre-pass that makes that picture).
++ * One forward pass over the batch: floor, then the sorted draws.
++ * `glassBackdrop` is the 2D picture glass refracts: undefined draws
++ * glass over the 3D scene alone (a cube face), null leaves the glass
++ * out (the pre-pass that makes that picture).
+ */
- const drawScene = (
++ const drawView = (
++ target: ViewTarget,
+ screenIndirect: "radiance" | "apply" | undefined,
+ writeHdr: boolean,
- glassBackdrop: GPUTexture | null = curTex,
++ glassBackdrop: GPUTexture | null | undefined,
+ ) => {
+ const begun = r3d.beginPass(
+ encoder,
- scene3dView,
- nativeWidth,
- nativeHeight,
- camera,
++ target.color,
++ target.width,
++ target.height,
++ target.view,
"clear",
{ r: 0, g: 0, b: 0, a: 0 },
- linearDepthView,
- geometryView,
++ target.geometry,
sceneLights,
{
environment: sceneEnvironment,
- shadow: shadow.length ? shadow : null,
+ shadow: sceneShadow,
+ screenIndirect,
},
- Boolean(sceneBloom),
- hdrTargetView,
+ writeHdr,
++ target.hdr,
);
-
- if (floorSpec && blurredReflectionTex) {
+ const { pass, cameraBindGroup, lightsBindGroup } = begun;
- if (floorSpec && blurredReflectionTex) {
++ if (floorSpec && target.reflection) {
r3d.drawFloor(
- pass3d,
+ pass,
cameraBindGroup,
lightsBindGroup,
-- blurredReflectionTex,
++ target.reflection,
floorSpec,
geometryEnabled,
- Boolean(sceneBloom),
+ writeHdr,
);
}
for (const item of ordered) {
@@@ -2143,47 -1840,94 +2195,172 @@@
lightsBindGroup,
item.meshBuffers ?? item.mesh,
item.texture,
- isGlass(item) ? { ...item.opts, backdrop } : item.opts,
+ isGlass(item)
- ? { ...item.opts, backdrop: glassBackdrop ?? curTex }
++ ? { ...item.opts, backdrop: glassBackdrop ?? undefined }
+ : item.opts,
geometryEnabled,
);
}
}
-
- pass3d.end();
- return cameraBuffer;
+ pass.end();
+ return begun;
};
- // Screen-space reflections and GI read the scene's own light: draw
- // its radiance once, trace it, then shade with what was found.
- if (screenIndirectEnabled && sceneEnvironment) {
- const { cameraBuffer: radianceCamera } = drawScene("radiance", true);
- r3d.traceScreenIndirect(encoder, radianceCamera, sceneEnvironment);
- }
- // Glass refracts the scene behind it, not just the layers under the
- // batch: draw everything else first and lay it over them.
- let glassBackdrop = curTex;
- if (glass.length) {
- drawScene(screenIndirectEnabled ? "apply" : undefined, false, null);
- glassBackdrop = ctx.renderer.composite(
- encoder,
++ const sceneSeconds = shiftedFrame / Math.max(1, fps ?? 24);
+ let final3dTex = scene3dTex;
++ let mainReflection: GPUTexture | null = null;
++ let mainPass: ReturnType<typeof drawView> | null = null;
+ if (cubeCamera) {
+ // Six 90° faces around the eye, then the lens's picture of them.
+ const size = cubeCamera.faceSize;
+ const cube = r3d.getOrCreateCubeTexture(size, false);
+ const hdrCube = sceneBloom
+ ? r3d.getOrCreateCubeTexture(size, true)
+ : null;
+ // Ink as wide on screen as without a cube: a face has size / 2
+ // pixels per radian at its centre, the output centerDensity.
+ const faceOutlines = outlines && {
+ ...outlines,
+ referenceHeight:
+ ((outlines.referenceHeight ?? 1080) *
+ 2 *
+ cubeCamera.centerDensity) /
+ nativeHeight,
+ };
- const faceTex = faceOutlines
- ? ctx.renderer.getTemporaryTexture(size, size, [
- curTex,
- targetTexture,
- scene3dTex,
- ...curExcludes,
- ...itemTextures,
- ])
++ // A face is drawn into its own texture when a post pass (ink,
++ // fog) reads it and writes the cube face.
++ const post = Boolean(faceOutlines) || volumetricFogEnabled;
++ const faceExcludes = [
+ curTex,
++ targetTexture,
+ scene3dTex,
- "normal",
- [targetTexture, scene3dTex, ...curExcludes],
++ ...curExcludes,
++ ...itemTextures,
++ ];
++ const faceTex = post
++ ? ctx.renderer.getTemporaryTexture(size, size, faceExcludes)
+ : null;
- const faceGeometry = faceOutlines
++ const fogTex =
++ faceOutlines && volumetricFogEnabled
++ ? ctx.renderer.getTemporaryTexture(size, size, [
++ ...faceExcludes,
++ ...(faceTex ? [faceTex] : []),
++ ])
++ : null;
++ const faceGeometry = geometryEnabled
+ ? r3d.getOrCreateLinearDepthTexture(size, size)
+ : null;
+ cubeFaceCameras(camera.eye, camera.near, camera.far).forEach(
+ (face, i) => {
+ const faceView = r3d.cubeFaceView(cube, i);
- drawView(
- face,
- size,
- size,
- faceTex ? faceTex.createView() : faceView,
- faceGeometry?.createView(),
- hdrCube ? r3d.cubeFaceView(hdrCube, i) : undefined,
++ const { cameraBindGroup, lightsBindGroup } = drawView(
++ {
++ view: face,
++ width: size,
++ height: size,
++ color: faceTex ? faceTex.createView() : faceView,
++ geometry: faceGeometry?.createView(),
++ hdr: hdrCube ? r3d.cubeFaceView(hdrCube, i) : undefined,
++ reflection: floorReflection(face, size, size),
++ },
++ undefined,
++ Boolean(sceneBloom),
++ // A face has no 2D picture behind it: glass refracts the 3D scene.
+ undefined,
+ );
- if (faceTex && faceGeometry && faceOutlines) {
++ if (!faceTex || !faceGeometry) return;
++ if (volumetricFogEnabled)
++ r3d.resolveGeometry(encoder, faceGeometry);
++ let source = faceTex;
++ if (faceOutlines) {
++ const inked = fogTex ?? null;
+ r3d.applyOutlines(
+ encoder,
+ faceTex,
+ faceGeometry,
- faceView,
++ inked ? inked.createView() : faceView,
+ face,
+ faceOutlines,
+ sceneEnvironment?.fog,
+ );
++ if (!inked) return;
++ source = inked;
++ }
++ if (volumetricFogEnabled && sceneEnvironment) {
++ r3d.applyVolumetricFog(
++ encoder,
++ source,
++ faceGeometry,
++ faceView,
++ cameraBindGroup,
++ lightsBindGroup,
++ sceneEnvironment,
++ sceneSeconds,
++ shiftedFrame,
++ );
+ }
+ },
+ );
+ r3d.remapLens(
+ encoder,
+ cube,
+ hdrCube,
+ scene3dView,
+ nativeWidth,
+ nativeHeight,
+ cubeCamera.remap,
+ );
- }
- const cameraBuffer = cubeCamera
- ? null
- : drawView(
- camera,
- nativeWidth,
- nativeHeight,
- scene3dView,
- linearDepthView,
- undefined,
++ } else {
++ mainReflection = floorReflection(camera, nativeWidth, nativeHeight);
++ const target: ViewTarget = {
++ view: camera,
++ width: nativeWidth,
++ height: nativeHeight,
++ color: scene3dView,
++ geometry: linearDepthView,
++ reflection: mainReflection,
++ };
++ // Screen-space reflections and GI read the scene's own light: draw
++ // its radiance once, trace it, then shade with what was found.
++ if (screenIndirectEnabled && sceneEnvironment) {
++ const { cameraBuffer: radianceCamera } = drawView(
++ target,
++ "radiance",
++ true,
+ curTex,
+ );
- if (!cubeCamera && outlines && linearDepthTex) {
- if (dofEnabled) r3d.resolveGeometry(encoder, linearDepthTex);
++ r3d.traceScreenIndirect(encoder, radianceCamera, sceneEnvironment);
++ }
++ // Glass refracts the scene behind it, not just the layers under
++ // the batch: draw everything else first and lay it over them.
++ let glassBackdrop = curTex;
++ if (glass.length) {
++ drawView(
++ target,
++ screenIndirectEnabled ? "apply" : undefined,
++ false,
++ null,
++ );
++ glassBackdrop = ctx.renderer.composite(
++ encoder,
++ curTex,
++ scene3dTex,
++ "normal",
++ [targetTexture, scene3dTex, ...curExcludes],
++ );
++ }
++ mainPass = drawView(
++ target,
++ screenIndirectEnabled ? "apply" : undefined,
++ Boolean(sceneBloom),
++ glassBackdrop,
+ );
+ }
- const { cameraBindGroup, cameraBuffer, lightsBindGroup } = drawScene(
- screenIndirectEnabled ? "apply" : undefined,
- Boolean(sceneBloom),
- glassBackdrop,
- );
+
- let final3dTex = scene3dTex;
- if (outlines && linearDepthTex) {
++ if (mainPass && outlines && linearDepthTex) {
+ if (dofEnabled || volumetricFogEnabled) {
+ r3d.resolveGeometry(encoder, linearDepthTex);
+ }
const outlinedTex = ctx.renderer.getTemporaryTexture(
nativeWidth,
nativeHeight,
@@@ -2200,33 -1944,7 +2377,38 @@@
);
final3dTex = outlinedTex;
}
- if (volumetricFogEnabled && sceneEnvironment && linearDepthTex) {
- if (dofEnabled && linearDepthTex && cameraBuffer) {
++ if (
++ mainPass &&
++ volumetricFogEnabled &&
++ sceneEnvironment &&
++ linearDepthTex
++ ) {
+ const vfogTex = ctx.renderer.getTemporaryTexture(
+ nativeWidth,
+ nativeHeight,
+ [
+ curTex,
+ targetTexture,
+ scene3dTex,
+ final3dTex,
+ linearDepthTex,
+ ...curExcludes,
+ ],
+ );
+ r3d.applyVolumetricFog(
+ encoder,
+ final3dTex,
+ linearDepthTex,
+ vfogTex.createView(),
- cameraBindGroup,
- lightsBindGroup,
++ mainPass.cameraBindGroup,
++ mainPass.lightsBindGroup,
+ sceneEnvironment,
- shiftedFrame / Math.max(1, fps ?? 24),
++ sceneSeconds,
+ shiftedFrame,
+ );
+ final3dTex = vfogTex;
+ }
- if (dofEnabled && linearDepthTex) {
++ if (mainPass && dofEnabled && linearDepthTex) {
const dofResultTex = ctx.renderer.getTemporaryTexture(
nativeWidth,
nativeHeight,
@@@ -2237,29 -1955,11 +2419,29 @@@
final3dTex,
linearDepthTex,
dofResultTex.createView(),
-- cameraBuffer,
++ mainPass.cameraBuffer,
);
final3dTex = dofResultTex;
}
+ if (motionBlur && history) {
+ const blurredTex = ctx.renderer.getTemporaryTexture(
+ nativeWidth,
+ nativeHeight,
+ [curTex, targetTexture, scene3dTex, final3dTex, ...curExcludes],
+ );
+ blurScene(
+ history,
+ camera,
+ final3dTex,
+ blurredTex,
+ motionBlur,
- floorSpec && blurredReflectionTex ? floorSpec : undefined,
++ floorSpec && mainReflection ? floorSpec : undefined,
+ );
+ final3dTex = blurredTex;
+ }
+ if (history) recordMotion(history, camera);
+
if (sceneBloom) {
const bloomTex = ctx.renderer.getTemporaryTexture(
nativeWidth,
diff --cc packages/compositions/src/program/lint.test.ts
index 730a633d,5fdea5b5..311da331
--- a/packages/compositions/src/program/lint.test.ts
+++ b/packages/compositions/src/program/lint.test.ts
@@@ -247,4 -240,4 +247,36 @@@ describe("lintComposition", () =>
// Callers that don't know about fonts aren't told.
expect(lint().map((d) => d.code)).not.toContain("missing_emoji_font");
});
++
++ it("flags cube cameras combined with single-view effects", () => {
++ const cube = {
++ id: "cam",
++ kind: "camera",
++ lens: { projection: "equirectangular" },
++ };
++ expect(codes([{ ...cube, dof: { enabled: true } }])).toContain(
++ "camera_cube_dof",
++ );
++ expect(codes([{ ...cube, motionBlur: true }])).toContain(
++ "camera_cube_motion_blur",
++ );
++ expect(
++ codes([
++ cube,
++ { id: "env", kind: "light", lightType: "environment", ssr: true },
++ ]),
++ ).toContain("camera_cube_screen_space");
++ // A perspective camera takes all of them.
++ const plain = codes([
++ { id: "cam", kind: "camera", motionBlur: true, dof: { enabled: true } },
++ {
++ id: "env",
++ kind: "light",
++ lightType: "environment",
++ ssr: true,
++ gi: true,
++ },
++ ]);
++ expect(plain.filter((c) => c.startsWith("camera_cube"))).toEqual([]);
++ });
});
diff --cc packages/compositions/src/program/lint.ts
index 6cb0aacf,2d14fbc0..6c53d0a1
--- a/packages/compositions/src/program/lint.ts
+++ b/packages/compositions/src/program/lint.ts
@@@ -109,6 -99,39 +109,50 @@@ function solidColor(value: unknown): st
return c && c[3] > 0.99 ? value : null;
}
+ type Push = (
+ severity: RenderDiagnostic["severity"],
+ code: string,
+ message: string,
+ nodeId?: string,
+ ) => void;
+
-/** Cube cameras (panoramic lenses, relativity) and what they cannot combine with. */
-function lintCamera(node: AnyNode, id: string, push: Push): void {
++/**
++ * Cube cameras (panoramic lenses, relativity) and what they cannot combine
++ * with. Returns whether the camera draws a cube.
++ */
++function lintCamera(node: AnyNode, id: string, push: Push): boolean {
+ const lens = node.lens as { projection?: string; fov?: number } | undefined;
+ const projection = lens?.projection ?? "perspective";
+ if (projection === "perspective" && (lens?.fov ?? 0) > 160)
+ push(
+ "error",
+ "camera_fov_range",
+ `'${id}' has a ${lens?.fov}° perspective lens: perspective goes up to 160°. Use projection "fisheye" for wider views.`,
+ id,
+ );
+ const relativity = node.relativity as { speed?: number } | undefined;
+ const cube =
+ projection !== "perspective" ||
+ (relativity?.speed ?? 0) > 0 ||
+ animates(node, ["relativitySpeed"]);
+ const dof = node.dof as { enabled?: boolean } | undefined;
+ if (cube && (dof?.enabled || animates(node, ["focusDistance"])))
+ push(
+ "error",
+ "camera_cube_dof",
+ `'${id}' is a cube camera (${relativity ? "relativity" : projection}) with depth of field: depth of field needs a single perspective view.`,
+ id,
+ );
++ if (cube && node.motionBlur)
++ push(
++ "error",
++ "camera_cube_motion_blur",
++ `'${id}' is a cube camera (${relativity ? "relativity" : projection}) with motionBlur: shutter motion blur needs a single perspective view.`,
++ id,
++ );
++ return cube;
+ }
+
export function lintComposition(
config: Pick<CompositorProgramConfig, "width" | "height" | "layout"> & {
backgroundColor?: string;
@@@ -131,8 -154,6 +175,10 @@@
const pbrNodes: string[] = [];
const emojiText: string[] = [];
const seenIds = new Set<string>();
+ let directLights = 0;
+ let fogLight: string | undefined;
++ let cubeCamera: string | undefined;
++ let screenSpaceLight: string | undefined;
const visit = (node: AnyNode, backdrop: string | null, atRoot: boolean) => {
const id = String(node.id ?? "?");
@@@ -179,15 -200,8 +225,17 @@@
const type = node.lightType ?? "directional";
if (type === "environment") hasEnvironment = true;
if (type === "ambient") ambientLights.push(id);
+ if (DIRECT_LIGHTS.has(String(type))) directLights++;
+ if (type === "environment" && node.volumetricFog) fogLight = id;
++ if (type === "environment" && (node.ssr || node.gi))
++ screenSpaceLight = id;
if (node.castShadow) {
- if (type !== "directional" && type !== "spot")
+ if (
+ type !== "directional" &&
+ type !== "spot" &&
+ type !== "point" &&
+ type !== "rect"
+ )
push(
"warning",
"shadow_unsupported_light",
@@@ -197,7 -211,10 +245,10 @@@
}
return;
}
- if (node.kind === "camera") return;
+ if (node.kind === "camera") {
- lintCamera(node, id, push);
++ if (lintCamera(node, id, push)) cubeCamera = id;
+ return;
+ }
if (node.kind === "media") {
const src = node.inputHandleId;
@@@ -382,15 -381,6 +433,24 @@@
);
}
++ if (cubeCamera && screenSpaceLight) {
++ push(
++ "error",
++ "camera_cube_screen_space",
++ `'${cubeCamera}' is a cube camera, and '${screenSpaceLight}' turns on ssr / gi: screen-space reflections and GI need a single perspective view. Use the environment's floor for reflections.`,
++ screenSpaceLight,
++ );
++ }
++
+ if (fogLight && directLights === 0) {
+ push(
+ "warning",
+ "volumetric_fog_without_lights",
+ `'${fogLight}' turns on volumetricFog, but the scene has no directional, spot, point or rect light: fog scatters only direct light, so there are no light shafts. Add a light (castShadow for shafts).`,
+ fogLight,
+ );
+ }
+
if (has3D && pbrNodes.length > 0 && !hasEnvironment) {
push(
"info",
diff --cc packages/compositions/src/program/schema.ts
index 324c73e1,fb57a2ac..06faf140
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@@ -1428,26 -1314,38 +1454,56 @@@ export const CameraShakeSchema =
})
.strict();
-export type CameraShake = z.infer<typeof CameraShakeSchema>;
+export type CameraShake = z.input<typeof CameraShakeSchema>;
+
+/**
+ * Per-object shutter motion blur for the 3D scene: every mesh, skinned
+ * joint, instance and fractured piece streaks by its own motion over the
+ * shutter (and the camera's), reconstructed from a velocity buffer.
+ */
+export const CameraMotionBlurSchema = z
+ .object({
+ /** Shutter angle, degrees: 180 is the film look, 360 the whole frame interval. Default 180. */
+ shutter: z.number().min(1).max(720).default(180),
+ /** Taps along each pixel's motion, 4..64. Default 20. */
+ samples: z.number().int().min(4).max(64).default(20),
+ /** Longest streak from a pixel, px (also the tile size). Default 48. */
+ maxBlur: z.number().min(4).max(160).default(48),
+ })
+ .strict();
+
+export type CameraMotionBlur = z.input<typeof CameraMotionBlurSchema>;
+ /**
+ * The camera moves through the scene near the speed of light. What it sees
+ * is physics, not grading: rays crowd towards the direction of motion
+ * (aberration), colours shift blue ahead and red behind (Doppler) and ahead
+ * brightens (beaming). Always drawn by a cube camera.
+ */
+ export const CameraRelativitySchema = z
+ .object({
+ /** β = v / c. Animate it with the `relativitySpeed` prop. */
+ speed: z.number().min(0).max(0.9999).default(0),
+ /** World-space direction of motion. Default: where the camera looks. */
+ direction: z.tuple([z.number(), z.number(), z.number()]).optional(),
+ /** Shift colours along the spectrum. Default true. */
+ doppler: z.boolean().default(true),
+ /** Brighten ahead, dim behind. Default true. */
+ beaming: z.boolean().default(true),
+ /**
+ * Exposure for the shifted light: "auto" (default) meters the centre of
+ * the view, so what the camera looks at keeps its brightness at rest;
+ * a number is fixed stops (0 = the true, often blinding, brightness).
+ */
+ exposure: z
+ .union([z.literal("auto"), z.number().min(-64).max(16)])
+ .default("auto"),
+ })
+ .strict();
+
+ /** Relativity as authored: fields with schema defaults may be left out. */
+ export type CameraRelativity = z.input<typeof CameraRelativitySchema>;
+
export interface CameraNode {
id: string;
kind: "camera";
@@@ -1464,8 -1362,7 +1520,9 @@@
lens?: CameraLens;
dof?: CameraDoF;
shake?: CameraShake;
+ /** Shutter motion blur of the 3D scene; `true` takes the defaults. */
+ motionBlur?: boolean | CameraMotionBlur;
+ relativity?: CameraRelativity;
startFrame?: number;
durationFrames?: number;
animation?: LayerAnimationSpec;
@@@ -1488,7 -1385,7 +1545,8 @@@ export const CameraNodeSchema: z.ZodTyp
lens: CameraLensSchema.optional(),
dof: CameraDoFSchema.optional(),
shake: CameraShakeSchema.optional(),
+ motionBlur: z.union([z.boolean(), CameraMotionBlurSchema]).optional(),
+ relativity: CameraRelativitySchema.optional(),
startFrame: z.number().int().min(0).default(0),
durationFrames: z.number().int().min(1).optional(),
animation: LayerAnimationSchema.default({ tracks: [] }),
diff --cc packages/framefields/src/cube-camera-render.test.ts
index 00000000,dbabd00f..6a8c2b84
mode 000000,100644..100644
--- a/packages/framefields/src/cube-camera-render.test.ts
+++ b/packages/framefields/src/cube-camera-render.test.ts
@@@ -1,0 -1,236 +1,330 @@@
+ import { describe, expect, it } from "vitest";
+ import {
+ CameraAnimation,
+ type CameraNode,
+ Composition,
+ Layer,
+ Layer3D,
+ } from "./index.js";
+ import { computeMse, decodePngPixels, getPixel } from "./test-utils/pixels.js";
+
+ const EYE = { x: 400, y: 300, z: 0 } as const;
+ const DISTANCE = 800;
+
+ /** Unlit markers in the six axis directions around the eye. */
+ const MARKERS = [
+ { id: "front", dir: [0, 0, 1], color: "#ff0000" },
+ { id: "back", dir: [0, 0, -1], color: "#00ffff" },
+ { id: "right", dir: [1, 0, 0], color: "#00ff00" },
+ { id: "left", dir: [-1, 0, 0], color: "#0000ff" },
+ { id: "up", dir: [0, -1, 0], color: "#ffff00" },
+ { id: "down", dir: [0, 1, 0], color: "#ff00ff" },
+ ] as const;
+
+ function scene(
+ width: number,
+ height: number,
+ camera: Partial<CameraNode>,
+ markers: readonly (typeof MARKERS)[number][] = MARKERS,
+ ) {
+ const comp = new Composition({
+ width,
+ height,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#000000",
+ });
+ comp.add([
+ Layer.camera({
+ x: EYE.x,
+ y: EYE.y,
+ z: EYE.z,
+ targetX: EYE.x,
+ targetY: EYE.y,
+ targetZ: EYE.z + 1000,
+ ...camera,
+ }),
+ Layer.ambientLight("#ffffff", 1),
+ ...markers.map((m) =>
+ Layer3D.sphere({
+ id: m.id,
+ radius: 150,
+ x: EYE.x + m.dir[0] * DISTANCE,
+ y: EYE.y + m.dir[1] * DISTANCE,
+ z: EYE.z + m.dir[2] * DISTANCE,
+ color: m.color,
+ material: "unlit",
+ }),
+ ),
+ ]);
+ return comp;
+ }
+
+ const near = (
+ p: { r: number; g: number; b: number },
+ hex: string,
+ tolerance = 40,
+ ) => {
+ const v = Number.parseInt(hex.slice(1), 16);
+ return (
+ Math.abs(p.r - ((v >> 16) & 255)) <= tolerance &&
+ Math.abs(p.g - ((v >> 8) & 255)) <= tolerance &&
+ Math.abs(p.b - (v & 255)) <= tolerance
+ );
+ };
+
+ /** Mean x of the pixels close to `hex`. */
+ function centroidX(
+ img: Awaited<ReturnType<typeof decodePngPixels>>,
+ hex: string,
+ ): number {
+ let sum = 0;
+ let count = 0;
+ for (let y = 0; y < img.height; y += 2) {
+ for (let x = 0; x < img.width; x += 2) {
+ if (near(getPixel(img, x, y), hex)) {
+ sum += x;
+ count++;
+ }
+ }
+ }
+ return count ? sum / count : Number.NaN;
+ }
+
+ describe("cube cameras", () => {
+ it("draws the whole sphere around the eye as an equirectangular panorama", async () => {
+ const img = await decodePngPixels(
+ await scene(1024, 512, {
+ lens: { projection: "equirectangular" },
+ }).renderFrame({ frame: 0 }),
+ );
+ // Forward in the middle, right and left a quarter turn either side,
+ // behind at both edges, up at the top and down at the bottom.
+ expect(near(getPixel(img, 512, 256), "#ff0000")).toBe(true);
+ expect(near(getPixel(img, 768, 256), "#00ff00")).toBe(true);
+ expect(near(getPixel(img, 256, 256), "#0000ff")).toBe(true);
+ expect(near(getPixel(img, 3, 256), "#00ffff")).toBe(true);
+ expect(near(getPixel(img, 1020, 256), "#00ffff")).toBe(true);
+ expect(near(getPixel(img, 512, 6), "#ffff00")).toBe(true);
+ expect(near(getPixel(img, 512, 505), "#ff00ff")).toBe(true);
+ // Between markers: empty sky.
+ expect(near(getPixel(img, 640, 256), "#000000", 10)).toBe(true);
+ });
+
+ it("matches the ordinary camera at rest", async () => {
+ const plain = scene(640, 360, { lens: { fov: 60 } }, [MARKERS[0]]);
+ const cube = scene(
+ 640,
+ 360,
+ { lens: { fov: 60 }, relativity: { speed: 0 } },
+ [MARKERS[0]],
+ );
+ const a = await decodePngPixels(await plain.renderFrame({ frame: 0 }));
+ const b = await decodePngPixels(await cube.renderFrame({ frame: 0 }));
+ expect(near(getPixel(b, 320, 180), "#ff0000")).toBe(true);
+ // Resampling softens the sphere's edge by a pixel at most.
+ expect(computeMse(a.pixels, b.pixels)).toBeLessThan(40);
+ });
+
+ it("aberrates the view towards the direction of motion", async () => {
+ // A marker 70° to the right: at rest, outside a 100° view even counting
+ // its 10.8° angular radius.
+ const angle = (70 * Math.PI) / 180;
+ const side = {
+ id: "side",
+ dir: [Math.sin(angle), 0, Math.cos(angle)],
+ color: "#00ff00",
+ } as unknown as (typeof MARKERS)[number];
+ const render = (speed: number) =>
+ scene(
+ 600,
+ 600,
+ {
+ lens: { fov: 100 },
+ relativity: { speed, doppler: false, beaming: false },
+ },
+ [side],
+ ).renderFrame({ frame: 0 });
+ const rest = await decodePngPixels(await render(0));
+ expect(Number.isNaN(centroidX(rest, "#00ff00"))).toBe(true);
+ // At β = 0.8 it is seen at cos θ' = (cos θ + β) / (1 + β cos θ) ≈ 26.3°.
+ const beta = 0.8;
+ const seen = Math.acos(
+ (Math.cos(angle) + beta) / (1 + beta * Math.cos(angle)),
+ );
+ const fast = await decodePngPixels(await render(beta));
+ const expected =
+ 300 + (300 * Math.tan(seen)) / Math.tan((50 * Math.PI) / 180);
+ expect(centroidX(fast, "#00ff00")).toBeCloseTo(expected, -1);
+ });
+
+ it("shifts light blue ahead and red behind", async () => {
+ const grey = [
+ { id: "ahead", dir: [0, 0, 1], color: "#808080" },
+ { id: "behind", dir: [0, 0, -1], color: "#808080" },
+ ] as unknown as (typeof MARKERS)[number][];
+ const render = (relativity: CameraNode["relativity"]) =>
+ scene(
+ 1024,
+ 512,
+ { lens: { projection: "equirectangular" }, relativity },
+ grey,
+ ).renderFrame({ frame: 0 });
+ const sum = (p: { r: number; g: number; b: number }) => p.r + p.g + p.b;
+ const hue = await decodePngPixels(
+ await render({ speed: 0.3, beaming: false, exposure: 0 }),
+ );
+ const ahead = getPixel(hue, 512, 256);
+ const behind = getPixel(hue, 2, 256);
+ expect(ahead.b).toBeGreaterThan(ahead.r + 20);
+ expect(behind.r).toBeGreaterThan(behind.b + 20);
+ // True brightness: beaming brightens ahead and dims behind.
+ const beamed = await decodePngPixels(
+ await render({ speed: 0.3, exposure: 0 }),
+ );
+ expect(sum(getPixel(beamed, 512, 256))).toBeGreaterThan(sum(ahead));
+ expect(sum(getPixel(beamed, 2, 256))).toBeLessThan(sum(behind));
+ // Auto exposure meters the centre: ahead keeps its brightness at rest.
+ const metered = await decodePngPixels(await render({ speed: 0.3 }));
+ const luma = (p: { r: number; g: number; b: number }) =>
+ 0.2126 * p.r + 0.7152 * p.g + 0.0722 * p.b;
+ expect(luma(getPixel(metered, 512, 256))).toBeCloseTo(128, -1.3);
+ });
+
+ it("animates the speed with relativitySpeed", async () => {
+ const angle = (70 * Math.PI) / 180;
+ const side = {
+ id: "side",
+ dir: [Math.sin(angle), 0, Math.cos(angle)],
+ color: "#00ff00",
+ } as unknown as (typeof MARKERS)[number];
+ const comp = scene(
+ 600,
+ 600,
+ {
+ lens: { fov: 100 },
+ relativity: { doppler: false, beaming: false },
+ animation: CameraAnimation.camera().fromTo("relativitySpeed", 0, 0.8, {
+ start: 0,
+ end: 10,
+ ease: "none",
+ }),
+ },
+ [side],
+ );
+ const rest = await decodePngPixels(await comp.renderFrame({ frame: 0 }));
+ const fast = await decodePngPixels(await comp.renderFrame({ frame: 10 }));
+ expect(Number.isNaN(centroidX(rest, "#00ff00"))).toBe(true);
+ expect(centroidX(fast, "#00ff00")).toBeGreaterThan(300);
+ });
+
+ it("sees behind the camera with a 360° fisheye", async () => {
+ const img = await decodePngPixels(
+ await scene(600, 600, {
+ lens: { projection: "fisheye", fov: 360 },
+ }).renderFrame({ frame: 0 }),
+ );
+ expect(near(getPixel(img, 300, 300), "#ff0000")).toBe(true);
+ // Behind the eye is the rim of the circle; outside it, nothing.
+ expect(near(getPixel(img, 300, 4), "#00ffff")).toBe(true);
+ expect(near(getPixel(img, 300, 596), "#00ffff")).toBe(true);
+ expect(getPixel(img, 10, 10).a).toBe(255);
+ expect(near(getPixel(img, 10, 10), "#000000", 10)).toBe(true);
+ // Right, at 90°, is halfway out.
+ expect(near(getPixel(img, 450, 300), "#00ff00")).toBe(true);
+ });
++
++ it("draws volumetric fog on every face", async () => {
++ const foggy = (fog: boolean) => {
++ const comp = scene(
++ 800,
++ 400,
++ { lens: { projection: "equirectangular" } },
++ [],
++ );
++ comp.add([
++ // Something to draw, off every sampled direction: an empty batch has no 3D pass.
++ Layer3D.sphere({
++ radius: 20,
++ x: EYE.x + 500,
++ y: EYE.y + 500,
++ z: EYE.z + 500,
++ color: "#ffffff",
++ }),
++ Layer.environment("neutral", {
++ ...(fog && {
++ volumetricFog: {
++ density: 0.05,
++ ambientIntensity: 0.6,
++ maxDistance: 3000,
++ },
++ }),
++ }),
++ Layer.directionalLight({
++ x: 0,
++ y: -2000,
++ z: -1000,
++ targetX: EYE.x,
++ targetY: EYE.y,
++ targetZ: EYE.z,
++ }),
++ ]);
++ return comp;
++ };
++ const clear = await decodePngPixels(
++ await foggy(false).renderFrame({ frame: 0 }),
++ );
++ const fogged = await decodePngPixels(
++ await foggy(true).renderFrame({ frame: 0 }),
++ );
++ const lum = (img: typeof clear, x: number, y: number) => {
++ const p = getPixel(img, x, y);
++ return p.r + p.g + p.b;
++ };
++ // Ahead, behind, left, right, above and below: each face is in the fog.
++ for (const [x, y] of [
++ [400, 200],
++ [10, 200],
++ [200, 200],
++ [600, 200],
++ [400, 20],
++ [400, 380],
++ ] as const) {
++ expect(lum(clear, x, y), `${x},${y}`).toBeLessThan(10);
++ expect(lum(fogged, x, y), `${x},${y}`).toBeGreaterThan(30);
++ }
++ });
++
++ it("refuses motion blur, which needs one perspective view", async () => {
++ await expect(
++ scene(400, 200, {
++ lens: { projection: "equirectangular" },
++ motionBlur: true,
++ }).renderFrame({ frame: 3 }),
++ ).rejects.toThrow(/motion blur/);
++ });
++
++ it("draws glass on its faces, refracting the 3D scene", async () => {
++ const withGlass = (glass: boolean) => {
++ const comp = scene(800, 400, { lens: { projection: "equirectangular" } });
++ if (glass)
++ comp.add(
++ Layer3D.sphere({
++ radius: 160,
++ x: EYE.x,
++ y: EYE.y,
++ z: EYE.z + 400,
++ material: "glass",
++ }),
++ );
++ return comp;
++ };
++ const plain = await decodePngPixels(
++ await withGlass(false).renderFrame({ frame: 0 }),
++ );
++ const glassy = await decodePngPixels(
++ await withGlass(true).renderFrame({ frame: 0 }),
++ );
++ expect(computeMse(plain.pixels, glassy.pixels)).toBeGreaterThan(5);
++ });
+ });
diff --cc packages/webgpu-renderers/src/math3d/index.ts
index b9e1bc3a,d9157005..cd65ee44
--- a/packages/webgpu-renderers/src/math3d/index.ts
+++ b/packages/webgpu-renderers/src/math3d/index.ts
@@@ -2,6 -3,6 +3,7 @@@ export * from "./cube-lens.js"
export * from "./environment3d.js";
export * from "./light3d.js";
export * from "./matrix4.js";
+export * from "./noise.js";
export * from "./path3d.js";
+ export * from "./relativistic-doppler.js";
export * from "./vector3.js";
diff --cc packages/webgpu-renderers/src/renderer3d/index.ts
index 402850a2,b7fafa41..1085778e
--- a/packages/webgpu-renderers/src/renderer3d/index.ts
+++ b/packages/webgpu-renderers/src/renderer3d/index.ts
@@@ -1,9 -1,8 +1,10 @@@
export * from "./audio-mesh-deform-pipeline.js";
export * from "./bloom-pipeline.js";
export * from "./floor-reflection-pipeline.js";
+export * from "./fracture-geometry.js";
export * from "./glass-pipeline.js";
+export * from "./hdr-environment.js";
+ export * from "./lens-remap-pipeline.js";
export * from "./matte3d.js";
export * from "./mesh3d-pipeline.js";
export * from "./motion-blur-pipeline.js";
diff --cc packages/webgpu-renderers/src/renderer3d/renderer3d.ts
index 0cab3663,cfb3484d..d3e11d02
--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
@@@ -62,11 -53,11 +62,15 @@@ import { type BloomOptions, BloomPipeli
import { FloorReflectionPipeline } from "./floor-reflection-pipeline.js";
import { GeometryOutlinePipeline } from "./geometry-outline-pipeline.js";
import { GeometryResolvePipeline } from "./geometry-resolve-pipeline.js";
+import {
+ getOrCreateHdrEnvironment,
+ type HdrEnvironment,
+} from "./hdr-environment.js";
import { decodeKtx2, isKtx2 } from "./ktx2.js";
+ import {
+ type LensRemapOptions,
+ LensRemapPipeline,
+ } from "./lens-remap-pipeline.js";
import {
type GPUMeshBuffers,
Mesh3DPipeline,
@@@ -426,12 -312,8 +431,13 @@@ export class Renderer3D
private bloomPipeline: BloomPipeline | null = null;
private outlinePipeline: GeometryOutlinePipeline | null = null;
private geometryResolvePipeline: GeometryResolvePipeline | null = null;
+ private volumetricFogPipeline: VolumetricFogPipeline | null = null;
+ private ssrGiPipeline: SsrGiPipeline | null = null;
+ private screenIndirect: ScreenIndirectTextures | null = null;
+ private placeholderScreenIndirect: ScreenIndirectTextures | null = null;
+ private screenSampler: GPUSampler | null = null;
private lastCameraBuffer: GPUBuffer | null = null;
+ private lensPipeline: LensRemapPipeline | null = null;
/**
* Samples per pixel in the 3D pass. Mesh silhouettes (extruded text,
@@@ -609,12 -476,8 +617,13 @@@
this.attachmentsUsed.clear();
this.outlinePipeline?.reset();
this.bloomPipeline?.resetPools();
+ this.volumetricFogPipeline?.resetPools();
+ this.ssrGiPipeline?.resetPools();
+ this.lensPipeline?.resetPools();
this.cameraPool.reset();
+ this.velocityCameraPool.reset();
+ this.quadMotionPool.reset();
+ this.motionBlurPipeline?.resetPools();
// The cached camera buffer goes back to the pool and may be handed
// out again (e.g. to a shadow pass's light camera): never reuse it
// across a reset.
@@@ -663,9 -526,70 +672,70 @@@
/** The HDR radiance target the last `beginPass(…, writeHdr)` drew into. */
hdrSource(): GPUTexture | null {
- return this.hdrTexture;
+ return this.hdrBlurred ?? this.hdrTexture;
}
+ /**
+ * A cube camera's target: six `size`² faces with a full mip chain, drawn
+ * into face by face and sampled by `remapLens`. `hdr` holds the scene
+ * radiance (the bloom source) instead of display colour.
+ */
+ getOrCreateCubeTexture(size: number, hdr: boolean): GPUTexture {
+ return this.attachment(
+ hdr ? "Renderer3DCubeHdr" : "Renderer3DCube",
+ size,
+ size,
+ {
+ format: hdr ? "rgba16float" : this.format,
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
+ layers: 6,
+ mipLevelCount: LensRemapPipeline.mipLevels(size),
+ },
+ );
+ }
+
+ /** Face `face` (WebGPU layer order) of a cube target, as a render attachment. */
+ cubeFaceView(cube: GPUTexture, face: number): GPUTextureView {
+ return cube.createView({
+ dimension: "2d",
+ baseArrayLayer: face,
+ arrayLayerCount: 1,
+ baseMipLevel: 0,
+ mipLevelCount: 1,
+ });
+ }
+
+ /**
+ * Draws a cube camera's picture into `dest` (`width` × `height`) through
+ * its lens. With an `hdr` cube the radiance is remapped too and becomes
+ * `hdrSource()`, so bloom glows where the lens put the light.
+ */
+ remapLens(
+ encoder: GPUCommandEncoder,
+ color: GPUTexture,
+ hdr: GPUTexture | null,
+ dest: GPUTextureView,
+ width: number,
+ height: number,
+ options: LensRemapOptions,
+ ): void {
+ this.lensPipeline ??= new LensRemapPipeline(this.device, this.format);
+ const destHdr = hdr
+ ? this.getOrCreateHdrTexture(width, height).createView()
+ : null;
+ this.lensPipeline.remap(
+ encoder,
+ color,
+ hdr,
+ dest,
+ destHdr,
+ width,
+ height,
+ options,
+ );
+ }
+
private getOrCreateHdrTexture(width: number, height: number): GPUTexture {
this.hdrTexture = this.attachment("Renderer3DHdrRadiance", width, height, {
format: "rgba16float",
@@@ -1444,9 -1134,10 +1518,11 @@@
}
this.passWritesHdr = writeHdr;
+ this.hdrBlurred = null;
if (writeHdr) {
- this.hdrTexture = this.getOrCreateHdrTexture(surfaceWidth, surfaceHeight);
+ const hdrView =
+ hdrTargetView ??
+ this.getOrCreateHdrTexture(surfaceWidth, surfaceHeight).createView();
if (msaa) {
this.msaaHdrTexture = this.getOrCreateMsaaTexture(
this.msaaHdrTexture,
@@@ -2977,11 -2182,8 +3053,12 @@@
for (const texture of this.attachments.values()) texture.destroy();
this.attachments.clear();
this.bloomPipeline?.destroy();
+ this.lensPipeline?.destroy();
this.outlinePipeline?.destroy();
+ this.volumetricFogPipeline?.destroy();
+ this.ssrGiPipeline?.destroy();
+ for (const t of Object.values(this.placeholderScreenIndirect ?? {}))
+ t.destroy();
this.depthTexture?.destroy();
for (const texture of this.shadowTextures.values()) texture.destroy();
this.shadowTextures.clear();
diff --cc plugins/framefields/skills/framefields-compose/SKILL.md
index 8216efdd,6947efd5..71c2523c
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@@ -243,6 -240,20 +243,20 @@@ const camAnim = CameraAnimation.camera(
});
```
+ **Cube cameras.** `lens.projection` picks the picture: `"perspective"` (default), `"equirectangular"` (the whole sphere around the eye, 360 × 180, for VR/360 video; `fov` is ignored), or `"fisheye"` (equidistant; `fov` is the full angle across the frame's height, up to 360°, so it can see behind the camera). Anything but perspective draws the scene as six 90° cube faces around the eye and remaps them, with mipmaps, so it costs about six renders. `lens.cubeSize` overrides the face size (default: matches the output's sharpness at the centre of the view, up to 4096).
+
+ **Relativity.** `relativity: { speed, direction?, doppler?, beaming?, exposure? }` films the scene as seen by a camera moving at `speed` × c (0 to 0.9999) along `direction` (world space, default where the camera looks). It is physics, not grading: rays crowd towards the direction of motion (aberration; objects at the side swing forward and look turned, not squashed), colours shift blue ahead and red behind (spectral Doppler, each colour modelled as a reflectance lit by daylight) and ahead brightens with beaming. `exposure: "auto"` (default) meters the centre of the view so what the camera looks at keeps its brightness; a number is fixed stops, where `0` is the true, often blinding, light. Animate the speed with the `relativitySpeed` prop, e.g. `CameraAnimation.camera().fromTo("relativitySpeed", 0, 0.99, { start: 0, end: 90, ease: "expo.in" })`. A camera with `relativity` is always a cube camera, even at speed 0, and at rest matches the ordinary camera. Works with every projection.
+
+ ```typescript
+ Layer.camera({
+ x: 0, y: -170, z: 0, targetX: 0, targetY: -170, targetZ: 1000,
+ lens: { fov: 90 },
+ relativity: { speed: 0.9 },
+ });
+ ```
+
-Cube cameras draw outlines, shadows, floor reflections and bloom, but not depth of field (`comp.lint()` reports `camera_cube_dof`), and glass refracts only the 3D scene, not the 2D layers below.
++Cube cameras draw outlines, shadows, floor reflections, volumetric fog and bloom, but not depth of field, motion blur or screen-space reflections and GI, which need one perspective view (`comp.lint()` reports `camera_cube_dof`, `camera_cube_motion_blur`, `camera_cube_screen_space`; rendering throws). Glass refracts only the 3D scene, not the 2D layers below.
+
### 4. Procedural & Audio-Reactive Drivers
Properties can be bound to Perlin wiggles, harmonic springs, or audio-extracted signals:
```typescript