Skip to content

Code-first video & image, rendered natively on WebGPU.

Expand scene lighting, shadows and skinning capacity

okanasl7a1bac85f246
diff --git a/.changeset/scene-capacity.md b/.changeset/scene-capacity.md
new file mode 100644
index 00000000..aadeb597
--- /dev/null
+++ b/.changeset/scene-capacity.md
@@ -0,0 +1,7 @@
+---
+"@framefields/webgpu-renderers": patch
+"@framefields/node-compositor": patch
+"@framefields/compositions": patch
+---
+
+Support variable numbers of direct lights and skin joints, and render independent shadows for multiple directional and spot lights.
diff --git a/nodes/node-compositor/SKILL.md b/nodes/node-compositor/SKILL.md
index 45502a05..b202be6c 100644
--- a/nodes/node-compositor/SKILL.md
+++ b/nodes/node-compositor/SKILL.md
@@ -208,6 +208,7 @@ Each track represents animatable property modifications:
- **Watermarking a Video:** a `flex` row (`align: "end"`, `justify: "end"`, full canvas) containing a `media` node bound to the PNG input; low `opacity`.
- **Picture-in-Picture:** a `flex` row with two `media` nodes (`fit: "cover"`, each `grow: 1`).
- **Audio-Reactive 3D Models:** a `kind: "model3d"` node loading an OBJ/FBX/GLTF/STL/PLY mesh with `audioDeform: { audioTrackId, mode: "normal_extrusion" | "radial_pulse" | "harmonic_wave" | "twist" | "ripple", amplitudeMultiplier, damping }`, deformed directly in VRAM via WebGPU compute shaders driven by audio frequency spectra and transient energies. `twist` turns each slice about Y by its height (harder on the bass); `ripple` runs concentric waves across XY, struck by the drums. `audioTrackId` may be an audio layer's id or an audio file path.
+- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; directional and spot lights can each cast shadows, up to the device texture-array layer limit.
- **Transparent meshes:** glTF `BLEND` materials and meshes with reduced layer, material, tint, or vertex alpha draw after opaque items, back to front by the camera-space center of each mesh’s transformed bounds, without writing depth. This is object-level sorting: intersecting meshes, self-overlap, skinned/deformed bounds, translucent flat layers, glass, and transparent shadows have separate limitations.
- **Mesh wireframe:** `wireframe: true` on a `model3d` draws triangle edges as one-pixel lines, including triangulation and back edges; it also applies to skinned and audio-deformed meshes and their shadows.
- **glTF scenes:** a glTF/GLB `model3d` keeps its node hierarchy, every skin and its animation clips (translation, rotation, scale and morph weights; linear, step and cubic-spline). Draco and meshopt compressed geometry and all seven core primitive modes are supported; per-map UV sets and texture transforms are independent. Basis KTX2 (ETC1S/UASTC) textures transcode offline to RGBA8 with their mip levels. Unsupported required extensions fail clearly. Morph targets run on the GPU before skinning and audio deformation. Skinned meshes are posed per frame from the clip (`animationName`, `animationTime`/`animationProgress`, `loop`); base-colour textures (with mipmaps), alpha `MASK`/`BLEND` and VRM MToon materials (`material: "toon"`, picked automatically when `material` is unset) render as authored. `center`/`normalizeSize` apply above the scene graph, so skinning stays intact.
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 208fc313..6e60e856 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -48,7 +48,6 @@ import {
generateExtrudedTextGeometry,
getPrimitiveGeometry,
jointMatrices,
- Light3D,
type Light3DOptions,
type Mat4,
Matrix4Math,
@@ -1416,12 +1415,20 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
linearDepthView = linearDepthTex.createView();
}
- // Shadow map from the first castShadow light: meshes cast unless
+ // Shadow maps for castShadow lights: meshes cast unless
// they opt out, flat layers only when they opt in; everything
// in the pass can receive.
- let shadow: ReturnType<typeof r3d.renderShadowMap> = null;
- const casterIndex = Light3D.shadowCasterIndex(sceneLights);
- if (casterIndex >= 0) {
+ const shadow: NonNullable<ReturnType<typeof r3d.renderShadowMap>>[] = [];
+ const shadowLights = sceneLights
+ .filter((light) => (light.lightType ?? light.type) !== "ambient")
+ .map((light, index) => ({ light, index }))
+ .filter(
+ ({ light }) =>
+ light.castShadow &&
+ ((light.lightType ?? light.type) === "directional" ||
+ (light.lightType ?? light.type) === "spot"),
+ );
+ if (shadowLights.length > 0) {
const casters: ShadowCaster3D[] = [];
const receivers: ReturnType<typeof r3d.worldBounds>[] = [];
for (const item of pending3DItems) {
@@ -1449,18 +1456,22 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
if (item.opts.castShadow === true) casters.push(caster);
}
}
- const shadowLight = sceneLights.filter(
- (l) => (l.lightType ?? l.type) !== "ambient",
- )[casterIndex];
- if (shadowLight && casters.length > 0) {
- shadow = r3d.renderShadowMap(
- encoder,
- shadowLight,
- casterIndex,
- casters,
- receivers,
- { mapSize: shadowMapSize },
- );
+ if (casters.length > 0) {
+ for (const { light, index } of shadowLights) {
+ const result = r3d.renderShadowMap(
+ encoder,
+ light,
+ index,
+ casters,
+ receivers,
+ {
+ mapSize: shadowMapSize,
+ layer: shadow.length,
+ layerCount: shadowLights.length,
+ },
+ );
+ if (result) shadow.push(result);
+ }
}
}
@@ -1584,7 +1595,10 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
{ r: 0, g: 0, b: 0, a: 0 },
linearDepthView,
sceneLights,
- { environment: sceneEnvironment, shadow },
+ {
+ environment: sceneEnvironment,
+ shadow: shadow.length ? shadow : null,
+ },
Boolean(sceneBloom),
);
@@ -3016,9 +3030,9 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
if (!posed && clip && clip.duration > 0) {
const currentSec = clipSec;
if (model.skeleton && model.skeleton.bones.length > 0) {
- const numBones = Math.min(128, model.skeleton.bones.length);
- skinningMatrices = new Float32Array(128 * 16);
- for (let b = 0; b < 128; b++) {
+ const numBones = model.skeleton.bones.length;
+ skinningMatrices = new Float32Array(numBones * 16);
+ for (let b = 0; b < numBones; b++) {
skinningMatrices[b * 16] = 1;
skinningMatrices[b * 16 + 5] = 1;
skinningMatrices[b * 16 + 10] = 1;
diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.ts
index c630008b..053b382d 100644
--- a/packages/compositions/src/program/lint.ts
+++ b/packages/compositions/src/program/lint.ts
@@ -119,7 +119,6 @@ export function lintComposition(
const ambientLights: string[] = [];
let has3D = false;
const pbrNodes: string[] = [];
- const shadowLights: string[] = [];
const emojiText: string[] = [];
const visit = (node: AnyNode, backdrop: string | null, atRoot: boolean) => {
@@ -158,8 +157,7 @@ export function lintComposition(
if (type === "environment") hasEnvironment = true;
if (type === "ambient") ambientLights.push(id);
if (node.castShadow) {
- if (type === "directional" || type === "spot") shadowLights.push(id);
- else
+ if (type !== "directional" && type !== "spot")
push(
"warning",
"shadow_unsupported_light",
@@ -335,13 +333,7 @@ export function lintComposition(
`The environment already fills shadows with light; ambient light ${ambientLights.join(", ")} adds flat light on top (remove it, or lower it, if the scene looks washed out).`,
);
}
- if (shadowLights.length > 1) {
- push(
- "info",
- "multiple_shadow_lights",
- `${shadowLights.length} lights cast shadows (${shadowLights.join(", ")}); only the first one does.`,
- );
- }
+
if (has3D && pbrNodes.length > 0 && !hasEnvironment) {
push(
"info",
diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.ts
index 66129e3b..5f8ddbbf 100644
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@ -1350,7 +1350,7 @@ export interface LightNode {
decay?: number;
angle?: number;
penumbra?: number;
- /** Directional / spot: render a shadow map (one shadow light per scene). */
+ /** Directional / spot: render a shadow map. */
castShadow?: boolean;
/** Shadow edge blur in shadow-map texels. Default 2.5. */
shadowSoftness?: number;
diff --git a/packages/framefields/src/gltf-scene-render.test.ts b/packages/framefields/src/gltf-scene-render.test.ts
index ac23bb4f..26f141a4 100644
--- a/packages/framefields/src/gltf-scene-render.test.ts
+++ b/packages/framefields/src/gltf-scene-render.test.ts
@@ -602,6 +602,49 @@ describe("glTF scene rendering", () => {
expect(end(250, 200)[0]).toBeGreaterThan(200);
});
+ it("renders a skin whose vertices reference joint 129", async () => {
+ const g = new GlbBuilder();
+ const q = rect(-100, -20, -60, 20);
+ const file = path.join(dir, "large-skin.glb");
+ const joints = new Uint16Array([
+ 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0,
+ ]);
+ const weights = new Float32Array([
+ 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0,
+ ]);
+ const nodes = [
+ { mesh: 0, skin: 0 },
+ ...Array.from({ length: 129 }, (_, i) => ({
+ translation: [i === 128 ? 90 : 0, 0, 0],
+ })),
+ ];
+ await fs.writeFile(
+ file,
+ g.build({
+ nodes,
+ scenes: [{ nodes: nodes.map((_, i) => i) }],
+ skins: [{ joints: Array.from({ length: 129 }, (_, i) => i + 1) }],
+ meshes: [
+ {
+ primitives: [
+ {
+ attributes: {
+ POSITION: g.accessor(new Float32Array(q.positions), "VEC3"),
+ JOINTS_0: g.accessor(joints, "VEC4"),
+ WEIGHTS_0: g.accessor(weights, "VEC4"),
+ },
+ indices: g.accessor(new Uint16Array(q.indices), "SCALAR"),
+ },
+ ],
+ },
+ ],
+ }),
+ );
+ const px = await render(file);
+ expect(px(210, 200)[0]).toBeGreaterThan(200);
+ expect(px(120, 200)[0]).toBeLessThan(10);
+ });
+
it("applies morph deltas before the skin's joint transform", async () => {
const g = new GlbBuilder();
const q = rect(-100, -20, -60, 20);
diff --git a/packages/framefields/src/lights-3d-render.test.ts b/packages/framefields/src/lights-3d-render.test.ts
index 5524460d..a3ab77ee 100644
--- a/packages/framefields/src/lights-3d-render.test.ts
+++ b/packages/framefields/src/lights-3d-render.test.ts
@@ -3,6 +3,69 @@ import { Composition, Layer, LayerAnimation } from "./index.js";
import { computeMse, decodePngPixels, getPixel } from "./test-utils/pixels.js";
describe("WebGPU 3D Lighting & Shading Conformance", () => {
+ it("renders direct lights beyond the eighth", async () => {
+ const comp = new Composition({
+ width: 400,
+ height: 400,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#000000",
+ });
+ comp.add(
+ Layer.camera({
+ x: 200,
+ y: 200,
+ z: -600,
+ targetX: 200,
+ targetY: 200,
+ targetZ: 0,
+ }),
+ );
+ for (let i = 0; i < 8; i++)
+ comp.add(
+ Layer.directionalLight({
+ color: "#ffffff",
+ intensity: 0,
+ x: 200,
+ y: 200,
+ z: -600,
+ targetX: 200,
+ targetY: 200,
+ targetZ: 0,
+ }),
+ );
+ comp.add(
+ Layer.directionalLight({
+ color: "#ff0000",
+ intensity: 1,
+ x: 200,
+ y: 200,
+ z: -600,
+ targetX: 200,
+ targetY: 200,
+ targetZ: 0,
+ }),
+ );
+ comp.add(
+ Layer.box({
+ is3D: true,
+ x: 100,
+ y: 100,
+ width: 200,
+ height: 200,
+ background: "#ffffff",
+ material: "lit",
+ }),
+ );
+ const pixel = getPixel(
+ await decodePngPixels(await comp.renderFrame({ frame: 0 })),
+ 200,
+ 200,
+ );
+ expect(pixel.r).toBeGreaterThan(100);
+ expect(pixel.g).toBeLessThan(30);
+ });
+
it("renders unlit mode when no lights are present (backward compatibility)", async () => {
const comp = new Composition({
width: 400,
diff --git a/packages/framefields/src/multiple-shadow-render.test.ts b/packages/framefields/src/multiple-shadow-render.test.ts
new file mode 100644
index 00000000..c846895f
--- /dev/null
+++ b/packages/framefields/src/multiple-shadow-render.test.ts
@@ -0,0 +1,97 @@
+import { Renderer3D } from "@framefields/webgpu-renderers";
+import { describe, expect, it, vi } from "vitest";
+import { Composition, Layer, Layer3D } from "./index.js";
+import { decodePngPixels } from "./test-utils/pixels.js";
+
+describe("multiple shadow lights", () => {
+ it("renders the shadow maps for both directional lights", async () => {
+ const spy = vi.spyOn(Renderer3D.prototype, "renderShadowMap");
+ try {
+ const render = async (left: boolean, right: boolean) => {
+ const c = new Composition({
+ width: 600,
+ height: 500,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#101010",
+ });
+ c.add(
+ Layer.camera({
+ x: 300,
+ y: -300,
+ z: -750,
+ targetX: 300,
+ targetY: 250,
+ targetZ: 0,
+ }),
+ );
+ c.add(Layer.ambientLight("#ffffff", 0.12));
+ c.add(
+ Layer.directionalLight({
+ x: -400,
+ y: -500,
+ z: -300,
+ targetX: 300,
+ targetY: 300,
+ targetZ: 0,
+ color: "#ffffff",
+ intensity: 0.55,
+ castShadow: left,
+ }),
+ );
+ c.add(
+ Layer.directionalLight({
+ x: 1000,
+ y: -500,
+ z: -300,
+ targetX: 300,
+ targetY: 300,
+ targetZ: 0,
+ color: "#ffffff",
+ intensity: 0.55,
+ castShadow: right,
+ }),
+ );
+ c.add(
+ Layer3D.roundedBox({
+ x: 300,
+ y: 330,
+ z: 0,
+ width: 500,
+ height: 20,
+ depth: 500,
+ radius: 0,
+ color: "#cccccc",
+ material: "lit",
+ }),
+ );
+ c.add(
+ Layer3D.sphere({
+ x: 300,
+ y: 240,
+ z: 0,
+ radius: 60,
+ color: "#ffffff",
+ material: "lit",
+ }),
+ );
+ return (await decodePngPixels(await c.renderFrame({ frame: 0 })))
+ .pixels;
+ };
+ const none = await render(false, false);
+ const left = await render(true, false);
+ const right = await render(false, true);
+ spy.mockClear();
+ const both = await render(true, true);
+ expect(new Set(spy.mock.calls.map((call) => call[2])).size).toBe(2);
+ const diff = (a: Uint8ClampedArray, b: Uint8ClampedArray) =>
+ a.reduce((sum, v, i) => sum + Math.abs(v - (b[i] ?? 0)), 0);
+ expect(diff(none, left)).toBeGreaterThan(1000);
+ expect(diff(none, right)).toBeGreaterThan(1000);
+ expect(diff(left, both)).toBeGreaterThan(1000);
+ expect(diff(right, both)).toBeGreaterThan(1000);
+ } finally {
+ spy.mockRestore();
+ }
+ });
+});
diff --git a/packages/webgpu-renderers/src/loaders/fbx-loader.ts b/packages/webgpu-renderers/src/loaders/fbx-loader.ts
index c86ab72a..d8d7af0b 100644
--- a/packages/webgpu-renderers/src/loaders/fbx-loader.ts
+++ b/packages/webgpu-renderers/src/loaders/fbx-loader.ts
@@ -6,7 +6,7 @@ import {
accumulateVertexNormals,
resolveModelTransform,
} from "./mesh-utils.js";
-import { MAX_JOINTS, poseNodes, type Quat } from "./pose.js";
+import { poseNodes, type Quat } from "./pose.js";
import {
isSupportedTexture,
readTextureFile,
@@ -675,10 +675,6 @@ export class FbxLoader {
influences.set(cp, list);
}
}
- if (joints.length > MAX_JOINTS)
- throw new Error(
- `FBX skin has ${joints.length} bones; the mesh shader holds ${MAX_JOINTS}`,
- );
if (joints.length > 0) {
skinIndex = skins.length;
skins.push({
diff --git a/packages/webgpu-renderers/src/loaders/gltf-loader.ts b/packages/webgpu-renderers/src/loaders/gltf-loader.ts
index 92525876..e608a718 100644
--- a/packages/webgpu-renderers/src/loaders/gltf-loader.ts
+++ b/packages/webgpu-renderers/src/loaders/gltf-loader.ts
@@ -3,7 +3,7 @@ import path from "node:path";
import draco from "draco3dgltf";
import { MeshoptDecoder } from "meshoptimizer/decoder";
import { type Mat4, Matrix4Math, type Vec3 } from "../math3d/index.js";
-import { MAX_JOINTS, poseNodes } from "./pose.js";
+import { poseNodes } from "./pose.js";
import { textureMime } from "./texture-files.js";
import type {
AnimationClip3D,
@@ -1250,13 +1250,6 @@ export class GltfLoader {
});
}
- for (const skin of skins) {
- if (skin.joints.length > MAX_JOINTS)
- throw new Error(
- `glTF skin has ${skin.joints.length} joints; the mesh shader holds ${MAX_JOINTS}`,
- );
- }
-
if (!Number.isFinite(minX)) {
minX = minY = minZ = 0;
maxX = maxY = maxZ = 0;
diff --git a/packages/webgpu-renderers/src/loaders/pose.ts b/packages/webgpu-renderers/src/loaders/pose.ts
index 34564b6e..97cf3087 100644
--- a/packages/webgpu-renderers/src/loaders/pose.ts
+++ b/packages/webgpu-renderers/src/loaders/pose.ts
@@ -13,9 +13,6 @@ import type {
NodeAnimationTrack3D,
} from "./types.js";
-/** Joints per skin the mesh shader holds (mesh3d.wgsl jointMatrices). */
-export const MAX_JOINTS = 128;
-
export type Quat = [number, number, number, number];
/** Column-major translate · rotate(quaternion) · scale. */
@@ -204,7 +201,7 @@ export function poseNodes(
return nodes.map((_, i) => resolve(i));
}
-/** The joint matrices of one skin for a posed scene graph, padded with identity to MAX_JOINTS. */
+/** The joint matrices of one skin for a posed scene graph. */
export function jointMatrices(
model: Model3DData,
skinIndex: number,
@@ -212,8 +209,8 @@ export function jointMatrices(
): Float32Array {
const skin = model.skins?.[skinIndex];
if (!skin) throw new Error(`model has no skin ${skinIndex}`);
- const out = new Float32Array(MAX_JOINTS * 16);
- for (let j = 0; j < MAX_JOINTS; j++) {
+ const out = new Float32Array(Math.max(1, skin.joints.length) * 16);
+ for (let j = 0; j < Math.max(1, skin.joints.length); j++) {
const node = skin.joints[j];
if (node === undefined) {
out.set(Matrix4Math.identity(), j * 16);
diff --git a/packages/webgpu-renderers/src/math3d/light3d.ts b/packages/webgpu-renderers/src/math3d/light3d.ts
index afcc2dcb..882b4fcd 100644
--- a/packages/webgpu-renderers/src/math3d/light3d.ts
+++ b/packages/webgpu-renderers/src/math3d/light3d.ts
@@ -58,6 +58,7 @@ export interface GPULightData {
spotParams: [number, number, number, number];
}
+/** Slots retained in the legacy uniform prefix; actual direct lights use storage. */
export const MAX_LIGHTS = 8;
/**
* 16 (ambient) + 16 (params) + 8 * 64 (lights), then the physical-shading
@@ -284,9 +285,7 @@ export const Light3D = {
ambientG += col[1] * int;
ambientB += col[2] * int;
} else {
- if (directLights.length < MAX_LIGHTS) {
- directLights.push(Light3D.processLight(l));
- }
+ directLights.push(Light3D.processLight(l));
}
}
@@ -338,13 +337,46 @@ export const Light3D = {
return buffer;
},
+ /** Packs every direct light for the variable-length GPU storage binding. */
+ packDirectLights(
+ lights: Light3DOptions[],
+ shadows: ShadowUniformInfo[] = [],
+ ): Float32Array {
+ const direct = lights.filter(
+ (light) => (light.lightType ?? light.type) !== "ambient",
+ );
+ const data = new Float32Array(Math.max(1, direct.length) * 40);
+ const shadowLayers = new Map(
+ shadows.map((shadow, layer) => [shadow.lightIndex, layer]),
+ );
+ for (const [i, light] of direct.entries()) {
+ const packed = Light3D.processLight(light);
+ const offset = i * 40;
+ data.set(packed.posType, offset);
+ data.set(packed.dirRadius, offset + 4);
+ data.set(packed.colorIntensity, offset + 8);
+ data.set(packed.spotParams, offset + 12);
+ const layer = shadowLayers.get(i);
+ const shadow = layer === undefined ? undefined : shadows[layer];
+ if (shadow && layer !== undefined) {
+ for (let j = 0; j < 16; j++)
+ data[offset + 16 + j] = shadow.matrix[j] ?? 0;
+ data.set([shadow.softness, shadow.mapSize, layer, 1], offset + 32);
+ data.set(
+ [shadow.bias, shadow.normalOffset, shadow.darkness, 0],
+ offset + 36,
+ );
+ }
+ }
+ return data;
+ },
+
/** Index (among non-ambient lights) of the light that should cast shadows, or -1. */
shadowCasterIndex(lights: Light3DOptions[]): number {
let index = 0;
for (const l of lights) {
const kind = l.lightType ?? l.type;
if (kind === "ambient") continue;
- if (index >= MAX_LIGHTS) break;
if (l.castShadow && (kind === "directional" || kind === "spot")) {
return index;
}
diff --git a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
index a823e4a3..da693ae8 100644
--- a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
@@ -52,7 +52,7 @@ export interface MeshUniformData extends SurfaceParams {
/** Extra per-map UV vertex attributes are present. */
hasMapUvs?: boolean;
isSkinned?: boolean;
- skinningMatrices?: Float32Array; // 128 * 16 floats
+ skinningMatrices?: Float32Array; // 16 floats per joint
/** Hardware-instanced draw path. */
isInstanced?: boolean;
/** Storage buffer containing `array<InstanceData>` (64 bytes per instance). */
@@ -159,10 +159,9 @@ export class Mesh3DPipeline {
(52 + SURFACE_PARAMS_FLOATS) * 4,
GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
);
- // 128 bones * 16 floats * 4 bytes = 8192 bytes
this.skinningPool = new BufferPool(
- 8192,
- GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ 64,
+ GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
);
const module = device.createShaderModule({
@@ -194,7 +193,7 @@ export class Mesh3DPipeline {
{
binding: 1,
visibility: GPUShaderStage.VERTEX,
- buffer: { type: "uniform" },
+ buffer: { type: "read-only-storage" },
},
matteLayoutEntry(2),
matteLayoutEntry(3),
@@ -393,17 +392,12 @@ export class Mesh3DPipeline {
{ width: 1, height: 1 },
);
- // Default identity skinning buffer (128 identity matrices)
- const idMatrices = new Float32Array(128 * 16);
- for (let b = 0; b < 128; b++) {
- idMatrices[b * 16] = 1;
- idMatrices[b * 16 + 5] = 1;
- idMatrices[b * 16 + 10] = 1;
- idMatrices[b * 16 + 15] = 1;
- }
+ // A non-skinned draw still binds one identity matrix.
+ const idMatrices = new Float32Array(16);
+ idMatrices[0] = idMatrices[5] = idMatrices[10] = idMatrices[15] = 1;
this.defaultSkinningBuffer = device.createBuffer({
- size: 8192,
- usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ size: 64,
+ usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
mappedAtCreation: true,
});
new Float32Array(this.defaultSkinningBuffer.getMappedRange()).set(
@@ -501,6 +495,15 @@ export class Mesh3DPipeline {
const modelBuffer = this.modelPool.getBuffer(this.device, this.modelData);
+ if (
+ data.skinningMatrices &&
+ data.skinningMatrices.byteLength >
+ this.device.limits.maxStorageBufferBindingSize
+ ) {
+ throw new Error(
+ `Skinning data requires ${data.skinningMatrices.byteLength} bytes, exceeding the device storage binding limit of ${this.device.limits.maxStorageBufferBindingSize} bytes`,
+ );
+ }
const skinningBuffer =
data.isSkinned && data.skinningMatrices
? this.skinningPool.getBuffer(this.device, data.skinningMatrices)
diff --git a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
index 47506fd0..35a6a599 100644
--- a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
@@ -33,13 +33,18 @@ export function createLightsLayout(
{
binding: 1,
visibility: GPUShaderStage.FRAGMENT,
- texture: { sampleType: "depth" },
+ texture: { sampleType: "depth", viewDimension: "2d-array" },
},
{
binding: 2,
visibility: GPUShaderStage.FRAGMENT,
sampler: { type: "comparison" },
},
+ {
+ binding: 3,
+ visibility: GPUShaderStage.FRAGMENT,
+ buffer: { type: "read-only-storage" },
+ },
],
});
}
diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
index 81bd0753..c6081179 100644
--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
@@ -162,7 +162,7 @@ export interface DrawMesh3DOpts extends SurfaceParams {
transparent?: boolean;
/** Whether vertex skinning is enabled */
isSkinned?: boolean;
- /** Skinning matrices (128 * 16 floats) */
+ /** Skinning matrices (16 floats per joint) */
skinningMatrices?: Float32Array;
/** Pre-computed model matrix overriding position/rotation/scale if supplied */
modelMatrix?: Mat4;
@@ -219,7 +219,7 @@ export interface Scene3DWorld {
/** Present = physically based shading (see `Environment3DOptions`). */
environment?: Environment3DOptions | null;
/** From `renderShadowMap`, when a light casts shadows. */
- shadow?: ShadowUniformInfo | null;
+ shadow?: ShadowUniformInfo | ShadowUniformInfo[] | null;
}
/** Something that can fall into a shadow map. */
@@ -235,6 +235,9 @@ export type ShadowCaster3D =
export interface ShadowMapOptions3D {
/** Texels per side. Default 2048. */
mapSize?: number;
+ /** Layer and total layers in the shadow texture array. */
+ layer?: number;
+ layerCount?: number;
/** Edge blur in texels. Default 2.5. */
softness?: number;
/** 0..1. Default 1. */
@@ -255,6 +258,7 @@ export class Renderer3D {
private dofBindGroupLayout: GPUBindGroupLayout;
private cameraPool: BufferPool;
private lightsPool: BufferPool;
+ private directLightsPool: BufferPool;
private instancePool: BufferPool;
private cameraUniformBindGroupCache = new UniformBindGroupCache();
private samplerCache = new SamplerCache();
@@ -286,6 +290,7 @@ export class Renderer3D {
private modelCache = new Map<string, Model3DData>();
private lastCameraPose: Camera3DPose | null = null;
private shadowTexture: GPUTexture | null = null;
+ private shadowTextures = new Map<string, GPUTexture>();
private placeholderShadowTexture: GPUTexture | null = null;
private shadowSampler: GPUSampler | null = null;
private meshBounds = new WeakMap<Mesh3DData, Aabb>();
@@ -347,6 +352,11 @@ export class Renderer3D {
GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
);
+ this.directLightsPool = new BufferPool(
+ 64,
+ GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
+ );
+
this.instancePool = new BufferPool(
65536,
GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
@@ -413,6 +423,7 @@ export class Renderer3D {
this.lastCameraPose = null;
this.lastCameraBuffer = null;
this.lightsPool.reset();
+ this.directLightsPool.reset();
this.instancePool.reset();
this.quad3dPipeline.resetPools();
this.slug3dPipeline.resetPools();
@@ -684,7 +695,7 @@ export class Renderer3D {
const lightsData = Light3D.packLightsUniforms(
lights ?? [],
world?.environment,
- world?.shadow,
+ Array.isArray(world?.shadow) ? world.shadow[0] : world?.shadow,
);
return this.lightsPool.getBuffer(this.device, lightsData);
}
@@ -693,7 +704,15 @@ export class Renderer3D {
private createLightsBindGroup(
lightsBuffer: GPUBuffer,
withShadow: boolean,
+ lights: Light3DOptions[],
+ shadows: ShadowUniformInfo[],
): GPUBindGroup {
+ const data = Light3D.packDirectLights(lights, shadows);
+ if (data.byteLength > this.device.limits.maxStorageBufferBindingSize)
+ throw new Error(
+ `Direct lights require ${data.byteLength} bytes, exceeding the device storage binding limit of ${this.device.limits.maxStorageBufferBindingSize} bytes`,
+ );
+ const directBuffer = this.directLightsPool.getBuffer(this.device, data);
this.shadowSampler ??= this.device.createSampler({
label: "ShadowComparisonSampler",
compare: "less-equal",
@@ -718,8 +737,9 @@ export class Renderer3D {
layout: this.quad3dPipeline.lightsLayout,
entries: [
{ binding: 0, resource: { buffer: lightsBuffer } },
- { binding: 1, resource: map.createView() },
+ { binding: 1, resource: map.createView({ dimension: "2d-array" }) },
{ binding: 2, resource: this.shadowSampler },
+ { binding: 3, resource: { buffer: directBuffer } },
],
});
}
@@ -788,6 +808,12 @@ export class Renderer3D {
processed.dirRadius[2],
];
const mapSize = options.mapSize ?? 2048;
+ const layer = options.layer ?? 0;
+ const layerCount = options.layerCount ?? 1;
+ if (layerCount > this.device.limits.maxTextureArrayLayers)
+ throw new Error(
+ `Shadow maps require ${layerCount} texture layers, exceeding the device limit of ${this.device.limits.maxTextureArrayLayers}`,
+ );
const casterBounds = casters.map((c) => this.worldBounds(c));
const frustum = fitShadowFrustum(
{ type: kind, position, direction, angle: light.angle },
@@ -797,16 +823,23 @@ export class Renderer3D {
);
if (!frustum) return null;
- if (!this.shadowTexture || this.shadowTexture.width !== mapSize) {
- this.shadowTexture?.destroy();
- this.shadowTexture = this.device.createTexture({
+ const key = `${mapSize}:${layerCount}`;
+ let texture = this.shadowTextures.get(key);
+ if (!texture) {
+ texture = this.device.createTexture({
label: "ShadowMap",
- size: { width: mapSize, height: mapSize },
+ size: {
+ width: mapSize,
+ height: mapSize,
+ depthOrArrayLayers: layerCount,
+ },
format: SHADOW_DEPTH_FORMAT,
usage:
GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
});
+ this.shadowTextures.set(key, texture);
}
+ this.shadowTexture = texture;
// The light's "camera": only viewProj matters to the vertex shaders.
const lightCam = new Float32Array(80);
@@ -830,7 +863,11 @@ export class Renderer3D {
label: "ShadowMapPass",
colorAttachments: [],
depthStencilAttachment: {
- view: this.shadowTexture.createView(),
+ view: this.shadowTexture.createView({
+ dimension: "2d",
+ baseArrayLayer: layer,
+ arrayLayerCount: 1,
+ }),
depthClearValue: 1.0,
depthLoadOp: "clear",
depthStoreOp: "store",
@@ -922,6 +959,12 @@ export class Renderer3D {
const lightsBindGroup = this.createLightsBindGroup(
lightsBuffer,
Boolean(world?.shadow),
+ lights ?? [],
+ world?.shadow
+ ? Array.isArray(world.shadow)
+ ? world.shadow
+ : [world.shadow]
+ : [],
);
const msaa = this.sampleCount > 1;
@@ -1052,13 +1095,19 @@ export class Renderer3D {
const lightsData = Light3D.packLightsUniforms(
lights ?? [],
world?.environment,
- world?.shadow,
+ Array.isArray(world?.shadow) ? world.shadow[0] : world?.shadow,
true,
);
const lightsBuffer = this.lightsPool.getBuffer(this.device, lightsData);
const lightsBindGroup = this.createLightsBindGroup(
lightsBuffer,
Boolean(world?.shadow),
+ lights ?? [],
+ world?.shadow
+ ? Array.isArray(world.shadow)
+ ? world.shadow
+ : [world.shadow]
+ : [],
);
const msaa = this.sampleCount > 1;
@@ -1967,7 +2016,8 @@ export class Renderer3D {
destroy(): void {
this.bloomPipeline?.destroy();
this.depthTexture?.destroy();
- this.shadowTexture?.destroy();
+ for (const texture of this.shadowTextures.values()) texture.destroy();
+ this.shadowTextures.clear();
this.placeholderShadowTexture?.destroy();
this.linearDepthTexture?.destroy();
this.msaaColorTexture?.destroy();
@@ -1986,6 +2036,7 @@ export class Renderer3D {
this.reflectionBlurredTexture?.destroy();
this.cameraPool.destroy();
this.lightsPool.destroy();
+ this.directLightsPool.destroy();
this.instancePool.destroy();
this.cameraUniformBindGroupCache.destroy();
}
diff --git a/packages/webgpu-renderers/src/shaders/lighting3d.ts b/packages/webgpu-renderers/src/shaders/lighting3d.ts
index c6fc47b5..c240c097 100644
--- a/packages/webgpu-renderers/src/shaders/lighting3d.ts
+++ b/packages/webgpu-renderers/src/shaders/lighting3d.ts
@@ -30,6 +30,16 @@ struct GPULight {
spotParams : vec4<f32>, // x = cos(outer), y = cos(inner), z = decay, w = unused
};
+struct DirectLight {
+ posType : vec4<f32>,
+ dirRadius : vec4<f32>,
+ colorIntensity : vec4<f32>,
+ spotParams : vec4<f32>,
+ shadowMatrix : mat4x4<f32>,
+ shadowParams : vec4<f32>,
+ shadowBias : vec4<f32>,
+};
+
struct LightsUniforms {
ambientColor : vec4<f32>, // rgb = ambient color * intensity (sRGB), w = unused
params : vec4<f32>, // x = numLights, y = hasLights, z = physical (1), w = tone mapping (0 none, 1 ACES, 2 AgX, 3 PBR Neutral)
@@ -212,25 +222,26 @@ var<private> POISSON : array<vec2<f32>, 12> = array<vec2<f32>, 12>(
/// 1 = fully lit, 0 = fully shadowed, for light index i.
fn shadowFactor(worldPos: vec3<f32>, N: vec3<f32>, i: i32) -> f32 {
- if (lights.shadowParams.x < 0.5 || i32(lights.shadowParams.y) != i) {
+ let shadow = directLights[i];
+ if (shadow.shadowParams.w < 0.5) {
return 1.0;
}
- let p = worldPos + N * lights.shadowBias.y;
- let lp = lights.shadowMatrix * vec4<f32>(p, 1.0);
+ let p = worldPos + N * shadow.shadowBias.y;
+ let lp = shadow.shadowMatrix * vec4<f32>(p, 1.0);
let ndc = lp.xyz / lp.w;
let uv = vec2<f32>(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5);
if (lp.w <= 0.0 || uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0 || ndc.z > 1.0) {
return 1.0;
}
- let texel = 1.0 / max(lights.shadowParams.w, 1.0);
- let radius = lights.shadowParams.z * texel;
- let depthRef = ndc.z - lights.shadowBias.x;
+ let texel = 1.0 / max(shadow.shadowParams.y, 1.0);
+ let radius = shadow.shadowParams.x * texel;
+ let depthRef = ndc.z - shadow.shadowBias.x;
var lit = 0.0;
for (var k = 0; k < 12; k++) {
- lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[k] * radius, depthRef);
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[k] * radius, i32(shadow.shadowParams.z), depthRef);
}
lit = lit / 12.0;
- return mix(1.0, lit, lights.shadowBias.z);
+ return mix(1.0, lit, shadow.shadowBias.z);
}
// ── Lights ────────────────────────────────────────────────────────────
@@ -243,7 +254,7 @@ struct LightSample {
fn sampleLight(i: i32, worldPos: vec3<f32>) -> LightSample {
var out = LightSample(vec3<f32>(0.0, 0.0, 1.0), 1.0, false);
- let light = lights.lights[i];
+ let light = directLights[i];
let lightType = light.posType.w;
if (lightType > 0.5 && lightType < 1.5) {
out.L = normalize(-light.dirRadius.xyz);
@@ -292,15 +303,12 @@ fn applyLighting3D(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<
var totalIllumination = baseColor * (lights.ambientColor.rgb * ambientInt);
let numLights = i32(lights.params.x);
- for (var i = 0; i < 8; i++) {
- if (i >= numLights) {
- break;
- }
+ for (var i = 0; i < numLights; i++) {
let ls = sampleLight(i, worldPos);
if (!ls.valid) {
continue;
}
- let light = lights.lights[i];
+ let light = directLights[i];
let lightColor = light.colorIntensity.rgb;
let intensity = light.colorIntensity.w;
let NdotL = max(dot(N, ls.L), 0.0);
@@ -333,15 +341,12 @@ fn shadePhysical(worldPos: vec3<f32>, worldNormal: vec3<f32>, albedo: vec3<f32>,
var lo = vec3<f32>(0.0);
let numLights = i32(lights.params.x);
- for (var i = 0; i < 8; i++) {
- if (i >= numLights) {
- break;
- }
+ for (var i = 0; i < numLights; i++) {
let ls = sampleLight(i, worldPos);
if (!ls.valid) {
continue;
}
- let light = lights.lights[i];
+ let light = directLights[i];
let nDotL = max(dot(N, ls.L), 0.0);
if (nDotL <= 0.0) {
continue;
@@ -439,6 +444,7 @@ fn shadeSurface(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32
/** Group-3 bindings every lit 3D shader declares. */
export const lightsBindingsWgsl = `
@group(3) @binding(0) var<uniform> lights : LightsUniforms;
-@group(3) @binding(1) var shadowMap : texture_depth_2d;
+@group(3) @binding(1) var shadowMap : texture_depth_2d_array;
@group(3) @binding(2) var shadowSampler : sampler_comparison;
+@group(3) @binding(3) var<storage, read> directLights : array<DirectLight>;
`;
diff --git a/packages/webgpu-renderers/src/shaders/mesh3d.ts b/packages/webgpu-renderers/src/shaders/mesh3d.ts
index 7b02d92a..c00acca5 100644
--- a/packages/webgpu-renderers/src/shaders/mesh3d.ts
+++ b/packages/webgpu-renderers/src/shaders/mesh3d.ts
@@ -1,6 +1,6 @@
/**
* WebGPU WGSL shaders for 3D static and skinned meshes.
- * Supports GPU Linear Blend Skinning (up to 128 bones),
+ * Supports GPU Linear Blend Skinning,
* the shared 3D surface shading (Blinn-Phong, or physical GGX + image-based light
* in scenes with an environment) or toon shading, shadow maps, emissive,
* alpha-cutoff (glTF MASK) materials,
@@ -26,7 +26,7 @@ struct ModelUniforms {
};
struct SkinningUniforms {
- jointMatrices : array<mat4x4<f32>, 128>,
+ jointMatrices : array<mat4x4<f32>>,
};
struct InstanceData {
@@ -38,7 +38,7 @@ struct InstanceData {
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@group(1) @binding(0) var<uniform> model : ModelUniforms;
-@group(1) @binding(1) var<uniform> skinning : SkinningUniforms;
+@group(1) @binding(1) var<storage, read> skinning : SkinningUniforms;
${matte3dWgsl(1, 2)}@group(1) @binding(3) var backdropTex : texture_2d<f32>;
@group(1) @binding(4) var<storage, read> instances : array<InstanceData>;
@group(2) @binding(0) var texSampler : sampler;
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index c6eec60f..428c0d21 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -100,7 +100,7 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani
| `Layer.caption(src, options)` | `"media"` | SRT-driven synchronized subtitle engine. Bottom-anchored auto-wrapping cues. |
| `Layer.chart(options, props)` | `"box"` | Charts built from native nodes (d3 scales and geometry). `type`: `"line" \| "area" \| "bar" \| "scatter" \| "candlestick" \| "pie" \| "donut"`. Options: `width`, `height`, `series: [{ name, data, color, type, curve, pointRadius }]` or `data`, `categories`, `stacked`, `xAxis`/`yAxis` (`min`, `max`, `ticks`, `grid`, `format`: a d3-format string like `"$,.0f"`), `legend`, `valueLabels`, `fontFamily`, `fontSize`, `colors`, `animate: { start, duration, stagger, ease } \| false`. `props` positions the chart box. Children get ids `<id>-bar-<series>-<i>`, `<id>-line-<series>`, `<id>-slice-<i>`. Keep scatter charts to a few hundred points: each point is a node. |
| `Layer.camera(options)` | `"camera"` | 3D scene camera with FOV, lens rack focus, and orbital coordinates. |
-| `Layer.light(options)` | `"light"` | 3D lighting primitive (`ambient`, `directional`, `point`, `spot`). Directional and spot take `castShadow: true` (one per scene), `shadowSoftness`, `shadowDarkness`. |
+| `Layer.light(options)` | `"light"` | 3D lighting primitive (`ambient`, `directional`, `point`, `spot`). Directional and spot take `castShadow: true`, `shadowSoftness`, `shadowDarkness`. |
| `Layer.environment(preset, options)` | `"light"` | The 3D scene's world: studio image-based light, exposure, tone mapping, fog, bloom. **Adding one switches the scene to physically based shading.** See *3D Look* below. |
| `Layer.sequence(entries, options)` | `"box"` | Scenes back to back with `Transition`s between them; returns a box whose `totalFrames` is the length. See *Scenes & Transitions*. |
| `Layer.model(options)` | `"model3d"` | 3D mesh loader (`.gltf`, `.glb`, `.obj`, `.fbx`, `.stl`, `.ply`). Supports skinning and audio deformation compute shaders. |
@@ -444,7 +444,7 @@ Layer3D.cylinder({ ..., color: "#111", emissive: "#4da3ff", emissiveIntensity: 3
- **Tone mapping `"neutral"` (default) keeps brand colors as authored**: orange stays orange. Use `"agx"` for a filmic roll-off.
- **Bloom only takes light well past white** (emissive ≳ 3, hot speculars), so ordinary bright surfaces don't glow. Raise `emissiveIntensity` to make something glow.
-- **Shadows:** one `castShadow` directional or spot light per scene. Meshes cast by default (`castShadow: false` opts out); flat layers (planes, cards) cast only with `castShadow: true`. Everything receives.
+- **Shadows:** directional and spot lights can each cast shadows, subject to device texture-array limits. Meshes cast by default (`castShadow: false` opts out); flat layers (planes, cards) cast only with `castShadow: true`. Everything receives.
- **Fog** fades to `fog.color` exactly, so set it to the composition background for a seamless horizon.
- **Models:** glTF materials bring base color, normal, metallic-roughness, occlusion and emissive maps; OBJ (`map_Kd`, `norm` in the `.mtl`) and FBX bring diffuse and normal maps, looked up next to the model. Textures can be PNG, JPEG, WebP, GIF, TIFF, AVIF or Basis KTX2 (ETC1S/UASTC, transcoded offline to RGBA8); TGA, PSD and DDS are skipped with a warning. Draco and meshopt-compressed glTF geometry is decoded by the asynchronous compositor loader; individual maps retain their own UV sets and texture transforms. glTF morph weights animate on the GPU before skinning. Unknown required glTF extensions fail clearly. Vertex colors (VOX, PLY, OFF, glTF `COLOR_0`) multiply the base color.
- **Model orientation:** the world is y-down, so a y-up model (most glTF, OBJ and FBX files) needs `rotateX: 180` to stand upright.
diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.md
index 461ecdb9..94f8f21e 100644
--- a/plugins/framefields/skills/framefields-render/SKILL.md
+++ b/plugins/framefields/skills/framefields-render/SKILL.md
@@ -301,7 +301,6 @@ if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop
| `offscreen_layer` | warning | A static absolute layer entirely outside the canvas |
| `shadow_unsupported_light` | warning | `castShadow` on a point or ambient light |
| `pbr_without_environment` | info | `roughness` / `metallic` set but no `Layer.environment()` (no physical shading) |
-| `multiple_shadow_lights` | info | Several `castShadow` lights; only the first casts |
| `static_track` | info | Every keyframe on a track holds the same value |
The document checks also run on their own, before any render: `comp.lint()` returns the same `{ severity, code, message, layerId }` entries. Run it in Step 1, right after typechecking.