Skip to content

Code-first video & image, rendered natively on WebGPU.

misc rendering imprv

okanaslcae47571d460
diff --git a/README.md b/README.md
index 36d4752c..52c750ce 100644
--- a/README.md
+++ b/README.md
@@ -112,7 +112,7 @@ Modern automated video generation is usually constrained by the architectures of
| 🎯 | **Deterministic Frame-Accurate Clock** | Absolute frame clocks, discrete sample points, and frame-accurate audio BeatGrids. No floating timers, no drift, no dropped frames. |
| πŸ”  | **Analytic, Resolution-Independent Type** | The Slug algorithm evaluates glyph contours per-pixel in WGSL β€” no texture atlases, no scaling artifacts, razor-sharp from 10 px to 10,000 px. |
| 🎨 | **Photoshop-Inspired Tonal & Spatial VFX** | 50+ modular GPU shaders: Curves, Levels, Selective Color, 3D LUTs, Halftone, Film Grain, Unsharp Mask, Mesh Warp, and Screen-Space Relighting. |
-| 🧊 | **Unified 3D & 2D Depth Compositing** | Nest 2D flex/box trees inside 3D homography planes, multiplane rigs, and meshes (OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF), with PBR glass and SSAO. |
+| 🧊 | **Unified 3D & 2D Depth Compositing** | Nest 2D flex/box trees inside 3D homography planes, multiplane rigs, and meshes (OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF), with GGX PBR, image-based light, shadows, bloom and SSAO. |
| πŸ”Š | **Built-in Procedural Audio DSP** | Multi-track soundtracks, deterministic procedural transition SFX (whoosh, impact, riser), and reactive signals that drive visuals from audio. |
| πŸ‘οΈ | **On-Device Neural Vision** | Object tracking, instance segmentation, multi-person pose, and person mattes from Apache-2.0 ONNX models β€” feeding reactive signals without a round trip to disk. |
| ☁️ | **Cloud-Native & CI/CD Ready** | ~200–400 MB RAM per worker (vs. 2–4 GB for Chromium), ideal for serverless GPU render clusters (AWS G4/G5, Modal, RunPod, Kubernetes). |
@@ -135,12 +135,15 @@ A comprehensive suite of professional image/video shader nodes in [`nodes/`](nod
- **Stylization & grain** β€” Film Grain (Gaussian emulsion with spatial seed variation), Halftone (mono/RGB/CMYK, adjustable dot shape & angle), Gradient Map, High Pass.
- **Optics & lens** β€” Bilateral Gaussian Blur, Unsharp Mask, Vignette, Refraction Caustics, PBR Glassmorphism with chromatic dispersion ([`PBRGlass`](packages/framefields/src/effects)).
- **Distortion & warping** β€” Displacement Maps, Liquify, Mesh Warp, Corner Pin homography.
+- **Gradients, mattes & transitions** β€” CSS `linear-` / `radial-` / `conic-gradient()` strings as any background or fill (blended in OKLab), track mattes (`node.withMatte(layer, "alpha" | "luma" …)`) for wipes, reveals and video-in-type, and [`Layer.sequence`](packages/framefields/src/transitions.ts) with `Transition.push` / `wipe` / `iris` / `crossfade` / `zoom` / `whip` between scenes.
### 3. Unified 3D Scene Graph, Camera & Mesh Shading
- **Calibrated camera rig** β€” LookAt and Turntable cameras ([`Camera3D`](packages/webgpu-renderers/src/math3d/camera3d.ts)) calibrated so `z = 0` matches 2D canvas pixel coordinates 1:1.
- **3D layout primitives** β€” [`Layer3D.cube`](packages/framefields/src/shapes3d.ts), `carousel`, `prism`, `plane`, `grid` with unified depth-buffer testing, plus solid smooth-shaded meshes: `roundedBox` (with `taper`, for keycaps and device bodies), `sphere` and `cylinder`.
- **Zero-dependency model parsers** β€” OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF.
-- **Skeletal animation & shading** β€” 128-bone Linear Blend Skinning, Blinn-Phong & PBR multi-light shading, PCSS/Poisson contact shadows, SSAO, and optical DoF.
+- **Physically based look** β€” add [`Layer.environment("studio")`](packages/webgpu-renderers/src/math3d/environment3d.ts) and the scene shades in linear light with GGX `roughness` / `metallic`, a procedural studio environment for image-based light and reflections, `emissive` surfaces, exposure, and Khronos PBR Neutral / AgX / ACES tone mapping. Scenes without one keep the classic Blinn-Phong look.
+- **Shadows, bloom & fog** β€” soft PCF shadow maps from a `castShadow` directional or spot light, HDR bloom that only takes light brighter than white, and distance fog that fades into the background color. [`Lighting.studio()`](packages/framefields/src/lighting.ts) / `soft` / `sunset` / `dramatic` rigs set all of it up in one call.
+- **Skeletal animation & materials** β€” 128-bone Linear Blend Skinning; glTF base color, normal, metallic-roughness, occlusion and emissive maps; SSAO and optical DoF.
- **Physical motion blur** β€” 180Β° shutter motion blur with per-vertex velocity vectors packed into `rg16float` MRT buffers.
### 4. Audio Layers, Procedural SFX & Reactive Signals
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 0d5087f6..1ce50a64 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -20,16 +20,19 @@ import { TemporalDeflickerPipeline } from "@framefields/tensor-webgpu";
import {
AudioLatentTrackCache,
type BlendMode,
+ type BloomOptions,
Camera3D,
type Camera3DPose,
type Color,
type DrawMesh3DOpts,
type DrawQuad3DOpts,
+ type Environment3DOptions,
type ExtrudedTextGeometryOptions,
type GPUMeshBuffers,
generateExtrudedTextGeometry,
getPrimitiveGeometry,
jointMatrices,
+ Light3D,
type Light3DOptions,
type Mat4,
Matrix4Math,
@@ -42,6 +45,7 @@ import {
type RenderContextValue,
Renderer3D,
ScreenSpaceRelightPipeline,
+ type ShadowCaster3D,
type ShapeFillConfig,
type ShapeGeometryConfig,
SlugFontCache,
@@ -561,6 +565,45 @@ function drawProjectedQuad(
renderPass.end();
}
+/**
+ * Emissive and shadow participation for a 3D draw: animated values first,
+ * then the node's own, then (meshes) the model file's material.
+ */
+function surfaceProps(
+ target: any,
+ lop: any,
+ mat?: { emissive?: [number, number, number]; emissiveStrength?: number },
+): {
+ emissive?: string | [number, number, number];
+ emissiveIntensity?: number;
+ castShadow?: boolean;
+} {
+ // glTF emissiveFactor is linear; draws take display (sRGB) colors.
+ const toSrgb = (v: number) =>
+ v <= 0.0031308 ? 12.92 * v : 1.055 * v ** (1 / 2.4) - 0.055;
+ const emissive =
+ target?.emissive ??
+ lop.emissive ??
+ (mat?.emissive
+ ? ([
+ toSrgb(mat.emissive[0]),
+ toSrgb(mat.emissive[1]),
+ toSrgb(mat.emissive[2]),
+ ] as [number, number, number])
+ : undefined);
+ const out: ReturnType<typeof surfaceProps> = {};
+ if (emissive !== undefined) {
+ out.emissive = emissive;
+ out.emissiveIntensity =
+ target?.emissiveIntensity ??
+ lop.emissiveIntensity ??
+ mat?.emissiveStrength ??
+ 1;
+ }
+ if (lop.castShadow !== undefined) out.castShadow = lop.castShadow;
+ return out;
+}
+
function computeContainerScissor(
matrix: DOMMatrix,
width: number,
@@ -871,6 +914,9 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
let camera: Camera3DPose | null = null;
let dofEnabled = false;
const sceneLights: Light3DOptions[] = [];
+ let sceneEnvironment: Environment3DOptions | null = null;
+ let sceneBloom: BloomOptions | null = null;
+ let shadowMapSize: number | undefined;
if (is3DScene) {
const camOp = cameraVM?.operation as any;
@@ -993,7 +1039,31 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
| "ambient"
| "directional"
| "point"
- | "spot";
+ | "spot"
+ | "environment";
+ if (lightType === "environment") {
+ // The scene's world: the last visible one wins.
+ const intensity = target?.intensity ?? lop.intensity;
+ sceneEnvironment = {
+ preset: lop.preset,
+ top: lop.top,
+ horizon: lop.horizon,
+ bottom: lop.bottom,
+ softboxes: target?.softboxes ?? lop.softboxes,
+ rotation: target?.rotation ?? lop.rotation,
+ exposure: target?.exposure ?? lop.exposure,
+ toneMapping: lop.toneMapping,
+ fog: lop.fog,
+ ...(intensity !== undefined && { intensity }),
+ };
+ sceneBloom =
+ lop.bloom === true
+ ? {}
+ : lop.bloom && typeof lop.bloom === "object"
+ ? { ...lop.bloom }
+ : null;
+ continue;
+ }
const rawColor = target?.color ?? lop.color ?? "#ffffff";
const color = parseColor(rawColor);
const intensity = target?.intensity ?? lop.intensity ?? 1.0;
@@ -1018,7 +1088,13 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
angle,
penumbra,
decay,
+ castShadow: Boolean(lop.castShadow),
+ shadowSoftness: target?.shadowSoftness ?? lop.shadowSoftness,
+ shadowDarkness: target?.shadowDarkness ?? lop.shadowDarkness,
});
+ if (lop.castShadow && lop.shadowMapSize) {
+ shadowMapSize = lop.shadowMapSize;
+ }
}
}
@@ -1041,6 +1117,11 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
meshBuffers?: GPUMeshBuffers;
};
const pending3DItems: Pending3DItem[] = [];
+ /** Textures the queued 3D items still read: their layers and mattes. */
+ const pendingTextures = (): GPUTexture[] =>
+ pending3DItems
+ .flatMap((q) => [q.texture, q.opts.matte])
+ .filter((t): t is GPUTexture => Boolean(t));
const flush3DPass = (
curTex: GPUTexture,
@@ -1068,9 +1149,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
}
}
- const itemTextures = pending3DItems
- .map((q) => (q.kind === "quad" ? q.texture : q.texture))
- .filter((t): t is GPUTexture => Boolean(t));
+ const itemTextures = pendingTextures();
const scene3dTex = ctx.renderer.getTemporaryTexture(
nativeWidth,
@@ -1089,6 +1168,54 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
linearDepthView = linearDepthTex.createView();
}
+ // Shadow map from the first castShadow light: meshes cast unless
+ // they opt out, flat layers only when they opt in; everything
+ // in the pass can receive.
+ let shadow: ReturnType<typeof r3d.renderShadowMap> = null;
+ const casterIndex = Light3D.shadowCasterIndex(sceneLights);
+ if (casterIndex >= 0) {
+ const casters: ShadowCaster3D[] = [];
+ const receivers: ReturnType<typeof r3d.worldBounds>[] = [];
+ for (const item of pending3DItems) {
+ if (item.kind === "mesh") {
+ const caster: ShadowCaster3D = {
+ kind: "mesh",
+ mesh: item.mesh,
+ buffers: item.meshBuffers,
+ opts: item.opts,
+ };
+ receivers.push(r3d.worldBounds(caster));
+ if (
+ item.opts.castShadow !== false &&
+ item.opts.material !== "unlit"
+ ) {
+ casters.push(caster);
+ }
+ } else if (item.kind === "quad") {
+ const caster: ShadowCaster3D = {
+ kind: "quad",
+ texture: item.texture,
+ opts: item.opts,
+ };
+ receivers.push(r3d.worldBounds(caster));
+ if (item.opts.castShadow === true) casters.push(caster);
+ }
+ }
+ const shadowLight = sceneLights.filter(
+ (l) => (l.lightType ?? l.type) !== "ambient",
+ )[casterIndex];
+ if (shadowLight && casters.length > 0) {
+ shadow = r3d.renderShadowMap(
+ encoder,
+ shadowLight,
+ casterIndex,
+ casters,
+ receivers,
+ { mapSize: shadowMapSize },
+ );
+ }
+ }
+
const {
pass: pass3d,
cameraBindGroup,
@@ -1104,6 +1231,8 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
{ r: 0, g: 0, b: 0, a: 0 },
linearDepthView,
sceneLights,
+ { environment: sceneEnvironment, shadow },
+ Boolean(sceneBloom),
);
for (const item of pending3DItems) {
@@ -1158,6 +1287,22 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
final3dTex = dofResultTex;
}
+ if (sceneBloom) {
+ const bloomTex = ctx.renderer.getTemporaryTexture(
+ nativeWidth,
+ nativeHeight,
+ [curTex, targetTexture, final3dTex, scene3dTex, ...curExcludes],
+ );
+ r3d.applyBloom(
+ encoder,
+ final3dTex,
+ bloomTex,
+ sceneBloom,
+ r3d.hdrSource(),
+ );
+ final3dTex = bloomTex;
+ }
+
const nextTex = ctx.renderer.composite(
encoder,
curTex,
@@ -1193,9 +1338,16 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
let activeTex = initialTex;
let activeView = initialView;
let activeExcludes = excludes;
+ // Matte layers are not drawn on their own: their owner draws them.
+ const mattesByOwner = new Map<string, VirtualMediaData>();
+ for (const vm of vms) {
+ const owner = (vm.operation as any)?.matteOf;
+ if (owner) mattesByOwner.set(owner, vm);
+ }
for (const child of sorted) {
const lop = child.operation as any;
if (lop.kind === "camera" || lop.kind === "light") continue;
+ if (lop.matteOf) continue;
const targetId = lop.id || lop.inputHandleId;
const target = targetId ? targetsById[targetId] : undefined;
const isChild3D =
@@ -1234,6 +1386,149 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
continue;
}
nodeIndex += 1;
+ const matteVM = lop.matteId ? mattesByOwner.get(lop.id) : undefined;
+ if (matteVM) {
+ // Draws a layer alone on transparent. 3D content it queues is
+ // rendered into it right away, in a pass of its own.
+ const isolate = async (vm: VirtualMediaData, index: number) => {
+ const queued = pending3DItems.splice(0);
+ const busy = [
+ targetTexture,
+ activeTex,
+ ...activeExcludes,
+ ...queued.flatMap((q) => [q.texture, q.opts.matte]),
+ ].filter((t): t is GPUTexture => Boolean(t));
+ const tex = ctx.renderer.getTemporaryTexture(
+ targetW,
+ targetH,
+ busy,
+ );
+ const view = tex.createView();
+ ctx.renderer
+ .beginFrame(
+ encoder,
+ view,
+ { r: 0, g: 0, b: 0, a: 0 },
+ targetW,
+ targetH,
+ "clear",
+ )
+ .end();
+ const out = await renderLayerVM(
+ vm,
+ index,
+ tex,
+ view,
+ busy.filter((t) => t !== targetTexture),
+ accumulatedOpacity,
+ parentMatrix,
+ parentRect,
+ targetW,
+ targetH,
+ parent3DMatrix,
+ false,
+ );
+ const drawn = out?.finalTex ?? tex;
+ const flat = flush3DPass(drawn, drawn.createView(), busy).finalTex;
+ pending3DItems.push(...queued);
+ return flat;
+ };
+ const mode: string = (matteVM.operation as any).matteMode ?? "alpha";
+ const blend =
+ mode === "luma"
+ ? "luma-in"
+ : mode === "luma-inverted"
+ ? "luma-out"
+ : mode === "alpha-inverted"
+ ? "destination-out"
+ : "destination-in";
+
+ if (isChild3D) {
+ // A 3D layer is matted inside the 3D pass, so it keeps its
+ // depth against the rest of the scene: the matte is resolved
+ // to screen coverage, and everything the owner queues shows
+ // only where that coverage does.
+ let coverage = await isolate(matteVM, nodeIndex);
+ activeExcludes = [...activeExcludes, coverage];
+ nodeIndex += 1;
+ if (mode !== "alpha") {
+ const white = ctx.renderer.getTemporaryTexture(
+ targetW,
+ targetH,
+ [targetTexture, activeTex, ...activeExcludes, ...pendingTextures()],
+ );
+ ctx.renderer
+ .beginFrame(
+ encoder,
+ white.createView(),
+ { r: 1, g: 1, b: 1, a: 1 },
+ targetW,
+ targetH,
+ "clear",
+ )
+ .end();
+ coverage = ctx.renderer.composite(encoder, white, coverage, blend, [
+ targetTexture,
+ activeTex,
+ ...activeExcludes,
+ ...pendingTextures(),
+ white,
+ ]);
+ activeExcludes = [...activeExcludes, coverage];
+ }
+ const before = pending3DItems.length;
+ const res = await renderLayerVM(
+ child,
+ nodeIndex,
+ activeTex,
+ activeView,
+ activeExcludes,
+ accumulatedOpacity,
+ parentMatrix,
+ parentRect,
+ targetW,
+ targetH,
+ parent3DMatrix,
+ isRootScene,
+ );
+ for (const item of pending3DItems.slice(before)) {
+ item.opts.matte = coverage;
+ }
+ if (res) {
+ if (res.finalTex !== activeTex) {
+ activeExcludes = [...activeExcludes, res.finalTex];
+ }
+ activeTex = res.finalTex;
+ activeView = res.finalView;
+ }
+ continue;
+ }
+
+ // Track matte: owner and matte each drawn alone, then the
+ // owner keeps only what the matte's alpha / brightness allows.
+ const contentTex = await isolate(child, nodeIndex);
+ activeExcludes = [...activeExcludes, contentTex];
+ nodeIndex += 1;
+ const matteTex = await isolate(matteVM, nodeIndex);
+ const masked = ctx.renderer.composite(
+ encoder,
+ contentTex,
+ matteTex,
+ blend,
+ [targetTexture, activeTex, ...activeExcludes, matteTex],
+ );
+ const next = ctx.renderer.composite(
+ encoder,
+ activeTex,
+ masked,
+ "normal",
+ [targetTexture, ...activeExcludes, matteTex],
+ );
+ activeExcludes = [...activeExcludes, next];
+ activeTex = next;
+ activeView = next.createView();
+ continue;
+ }
const res = await renderLayerVM(
child,
nodeIndex,
@@ -1288,9 +1583,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
// layer of the same size must not be handed one from the pool.
const excludes = [
...baseExcludes,
- ...pending3DItems
- .map((q) => q.texture)
- .filter((t): t is GPUTexture => Boolean(t)),
+ ...pendingTextures(),
];
const lop = nodeVM.operation as any;
if (lop.kind === "camera" || lop.kind === "light") return;
@@ -1743,9 +2036,11 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
ambientIntensity:
target?.ambientIntensity ?? lop.ambientIntensity ?? 1.0,
metallic: target?.metallic ?? lop.metallic ?? mat?.metallic,
+ ...surfaceProps(target, lop, mat),
audioDeform: meshAudioDeform,
};
+ if (mat) meshOpts.maps = await r3d.materialMaps(mat);
pending3DItems.push({
kind: "mesh",
mesh,
@@ -2041,9 +2336,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
activeTex,
targetTexture,
...excludes,
- ...pending3DItems
- .map((q) => q.texture)
- .filter((t): t is GPUTexture => Boolean(t)),
+ ...pendingTextures(),
];
const groupTex = ctx.renderer.getTemporaryTexture(
texW,
@@ -2226,6 +2519,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
target?.specularIntensity ?? lop.specularIntensity,
ambientIntensity: target?.ambientIntensity ?? lop.ambientIntensity,
metallic: target?.metallic ?? lop.metallic,
+ ...surfaceProps(target, lop),
...(modelMatrix && { modelMatrix }),
};
@@ -3632,6 +3926,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
specularIntensity: target?.specularIntensity ?? lop.specularIntensity,
ambientIntensity: target?.ambientIntensity ?? lop.ambientIntensity,
metallic: target?.metallic ?? lop.metallic,
+ ...surfaceProps(target, lop),
...(modelMatrix && { modelMatrix }),
};
diff --git a/packages/compositions/src/program/animation.test.ts b/packages/compositions/src/program/animation.test.ts
index de421029..a2e7a4f3 100644
--- a/packages/compositions/src/program/animation.test.ts
+++ b/packages/compositions/src/program/animation.test.ts
@@ -156,3 +156,53 @@ describe("LayerAnimation.keys", () => {
expect(anim.tracks[0]!.keyframes.map((k) => k.frame)).toEqual([0, 10]);
});
});
+
+describe("LayerAnimation track merging", () => {
+ it("replaces a key on an occupied frame instead of duplicating it", () => {
+ const anim = LayerAnimation.create().keys("y", [
+ [0, 0],
+ [10, 5],
+ [10, 8],
+ ]);
+ expect(anim.tracks[0].keyframes.map((k) => [k.frame, k.value])).toEqual([
+ [0, 0],
+ [10, 8],
+ ]);
+ });
+
+ it("merges behaviours authored separately into one sorted track per prop", () => {
+ const enter = LayerAnimation.keys("y", [
+ [0, -300],
+ [20, 0],
+ ]);
+ const press = LayerAnimation.keys("y", [
+ [40, 0],
+ [42, 6],
+ [48, 0],
+ ]).keys("opacity", [[0, 1]]);
+ const wave = LayerAnimation.create()
+ .keys("y", [
+ [48, 2],
+ [60, -18],
+ ])
+ .wiggle("x", { amplitude: 4 });
+ const merged = LayerAnimation.merge(enter, press, wave);
+ const y = merged.tracks.filter((t) => t.prop === "y");
+ expect(y).toHaveLength(1);
+ expect(y[0].keyframes.map((k) => [k.frame, k.value])).toEqual([
+ [0, -300],
+ [20, 0],
+ [40, 0],
+ [42, 6],
+ [48, 2],
+ [60, -18],
+ ]);
+ expect(merged.tracks.map((t) => t.prop).sort()).toEqual([
+ "opacity",
+ "x",
+ "y",
+ ]);
+ // Inputs are left untouched.
+ expect(enter.tracks[0].keyframes).toHaveLength(2);
+ });
+});
diff --git a/packages/compositions/src/program/animation.ts b/packages/compositions/src/program/animation.ts
index 56716a54..8583b1d6 100644
--- a/packages/compositions/src/program/animation.ts
+++ b/packages/compositions/src/program/animation.ts
@@ -128,6 +128,38 @@ export interface CameraFollowPathOptions {
ease?: string | EaseRef;
}
+const isKeyframeTrack = (t: AnimationTrack) =>
+ !t.source || t.source.type === "keyframe";
+
+/**
+ * Merges keyframe tracks that animate the same prop into one sorted track,
+ * the later key winning on a shared frame. Procedural tracks (signal,
+ * wiggle, spring…) pass through untouched.
+ */
+export function mergeAnimationTracks(
+ tracks: readonly AnimationTrack[],
+): AnimationTrack[] {
+ const out: AnimationTrack[] = [];
+ const byProp = new Map<string, AnimationTrack>();
+ for (const t of tracks) {
+ if (!isKeyframeTrack(t)) {
+ out.push(t);
+ continue;
+ }
+ const existing = byProp.get(t.prop);
+ if (!existing) {
+ const copy = { ...t, keyframes: [...(t.keyframes ?? [])] };
+ byProp.set(t.prop, copy);
+ out.push(copy);
+ continue;
+ }
+ const frames = new Map(existing.keyframes.map((k) => [k.frame, k]));
+ for (const k of t.keyframes ?? []) frames.set(k.frame, k);
+ existing.keyframes = [...frames.values()].sort((a, b) => a.frame - b.frame);
+ }
+ return out;
+}
+
function evaluateEasingProgress(
ease: string | EaseRef | undefined,
t: number,
@@ -483,12 +515,17 @@ export class LayerAnimation implements LayerAnimationSpec {
}
const parsedEase = parseEase(ease);
- track.keyframes.push({
+ const kf = {
id: `k_${track.keyframes.length}_f${frame}`,
frame,
value,
...(parsedEase && { ease: parsedEase }),
- });
+ };
+ // A key on an occupied frame replaces it (later wins), so behaviours
+ // can be layered in any order without tripping duplicate-frame errors.
+ const at = track.keyframes.findIndex((k) => k.frame === frame);
+ if (at >= 0) track.keyframes[at] = kf;
+ else track.keyframes.push(kf);
track.keyframes.sort((a, b) => a.frame - b.frame);
return this;
@@ -978,6 +1015,28 @@ export class LayerAnimation implements LayerAnimationSpec {
return node;
}
+ /**
+ * Folds other animations into this one. Keyframe tracks on the same prop
+ * merge into one track (a later key wins on a shared frame), so separate
+ * behaviours β€” an entrance, presses, a wave β€” can be authored apart:
+ * `LayerAnimation.merge(rainIn, presses, wave)`.
+ */
+ public merge(
+ ...others: Array<LayerAnimation | { tracks: AnimationTrack[] }>
+ ): this {
+ this.tracks = mergeAnimationTracks([
+ ...this.tracks,
+ ...others.flatMap((o) => o.tracks),
+ ]);
+ return this;
+ }
+
+ public static merge(
+ ...anims: Array<LayerAnimation | { tracks: AnimationTrack[] }>
+ ): LayerAnimation {
+ return new LayerAnimation().merge(...anims);
+ }
+
/**
* Returns the validated LayerAnimation specification object.
*/
diff --git a/packages/compositions/src/program/evaluator.test.ts b/packages/compositions/src/program/evaluator.test.ts
index df04fac5..3bd807ef 100644
--- a/packages/compositions/src/program/evaluator.test.ts
+++ b/packages/compositions/src/program/evaluator.test.ts
@@ -531,11 +531,11 @@ describe("Evaluator β€” Signal to Color Modulation", () => {
});
expect(c1).toBe("#ffffff");
- // Signal = 0.5 -> intermediate gray
+ // Signal = 0.5 -> perceptual (OKLab L = 0.5) mid gray, not the sRGB average
const cMid = evaluateTrackAtFrame(track, 0, 24, {
signals: { sig1: 0.5 },
});
- expect(cMid).toBe("#808080");
+ expect(cMid).toBe("#636363");
});
it("switches color abruptly with threshold mode", () => {
diff --git a/packages/compositions/src/program/evaluator.ts b/packages/compositions/src/program/evaluator.ts
index b04b2ca9..22979b10 100644
--- a/packages/compositions/src/program/evaluator.ts
+++ b/packages/compositions/src/program/evaluator.ts
@@ -1,3 +1,4 @@
+import { mixColors } from "@framefields/core";
import { signalRegistry } from "@framefields/webgpu-renderers";
import type {
AnimationTrack,
@@ -557,20 +558,9 @@ function parseHexColor(hex: string): [number, number, number, number] {
return [255, 255, 255, 1];
}
+/** Color keyframes blend perceptually (OKLab) and accept any CSS color. */
function lerpColor(c1: string, c2: string, t: number): string {
- if (!c1.startsWith("#") || !c2.startsWith("#")) return t < 0.5 ? c1 : c2;
- const [r1, g1, b1, a1] = parseHexColor(c1);
- const [r2, g2, b2, a2] = parseHexColor(c2);
- const r = Math.round(lerp(r1, r2, t));
- const g = Math.round(lerp(g1, g2, t));
- const b = Math.round(lerp(b1, b2, t));
- const a = lerp(a1, a2, t);
-
- const toHex = (n: number) => n.toString(16).padStart(2, "0");
- if (a >= 0.999) {
- return `#${toHex(r)}${toHex(g)}${toHex(b)}`;
- }
- return `rgba(${r}, ${g}, ${b}, ${Math.round(a * 100) / 100})`;
+ return mixColors(c1, c2, t);
}
function hueToRgb(p: number, q: number, t: number): number {
diff --git a/packages/compositions/src/program/index.ts b/packages/compositions/src/program/index.ts
index e899431f..7fc1e151 100644
--- a/packages/compositions/src/program/index.ts
+++ b/packages/compositions/src/program/index.ts
@@ -9,6 +9,8 @@
export * from "../utils/layout/resolve-layout.js";
export * from "./animation.js";
export * from "./evaluator.js";
+export * from "./lint.js";
+export * from "./mattes.js";
export * from "./schema.js";
export * from "./sequence.js";
export * from "./stagger.js";
diff --git a/packages/compositions/src/program/lint.test.ts b/packages/compositions/src/program/lint.test.ts
new file mode 100644
index 00000000..7f64c8de
--- /dev/null
+++ b/packages/compositions/src/program/lint.test.ts
@@ -0,0 +1,128 @@
+import { describe, expect, it } from "vitest";
+import { lintComposition } from "./lint.js";
+
+const base = { width: 1920, height: 1080, backgroundColor: "#000000" };
+const codes = (layout: unknown[], extra = {}) =>
+ lintComposition({ ...base, ...extra, layout: layout as never }).map(
+ (d) => d.code,
+ );
+
+describe("lintComposition", () => {
+ it("flags unreadable colors and gradients", () => {
+ expect(
+ codes([
+ { id: "a", kind: "box", background: "lienar-gradient(red, blue)" },
+ ]),
+ ).toEqual(["invalid_gradient"]);
+ expect(codes([{ id: "a", kind: "box", background: "#ff5a1" }])).toEqual([
+ "unknown_color",
+ ]);
+ expect(
+ codes([
+ {
+ id: "a",
+ kind: "box",
+ background: "linear-gradient(to right, nope, blue)",
+ },
+ ]),
+ ).toEqual(["invalid_gradient"]);
+ expect(
+ codes([{ id: "a", kind: "box", background: "oklch(0.7 0.1 200)" }]),
+ ).toEqual([]);
+ });
+
+ it("flags text that disappears into its backdrop", () => {
+ const layout = [
+ {
+ id: "card",
+ kind: "box",
+ background: "#ffffff",
+ children: [{ id: "t", kind: "text", fill: "#eeeeee" }],
+ },
+ ];
+ expect(codes(layout)).toContain("low_contrast_text");
+ expect(codes([{ id: "t", kind: "text", fill: "#ffffff" }])).toEqual([]);
+ });
+
+ it("does not judge text drawn over an image or panel it cannot see", () => {
+ const overImage = [
+ { id: "photo", kind: "image", src: "a.png" },
+ { id: "t", kind: "text", fill: "#ffffff" },
+ ];
+ expect(codes(overImage, { backgroundColor: "#ffffff" })).not.toContain("low_contrast_text");
+ const onPanel = [
+ { id: "panel", kind: "box", background: "#ffffff" },
+ { id: "t", kind: "text", fill: "#ffffff", zIndex: -1 },
+ ];
+ // Drawn before the panel (lower z): the backdrop is still known.
+ expect(codes(onPanel, { backgroundColor: "#ffffff" })).toContain("low_contrast_text");
+ });
+
+ it("flags static layers parked off-canvas, but not ones that move in", () => {
+ const off = {
+ id: "o",
+ kind: "box",
+ position: "absolute",
+ x: 2000,
+ y: 0,
+ width: 100,
+ height: 100,
+ };
+ expect(codes([off])).toContain("offscreen_layer");
+ expect(
+ codes([
+ {
+ ...off,
+ animation: {
+ tracks: [
+ {
+ prop: "x",
+ keyframes: [
+ { frame: 0, value: 2000 },
+ { frame: 10, value: 0 },
+ ],
+ },
+ ],
+ },
+ },
+ ]),
+ ).not.toContain("offscreen_layer");
+ });
+
+ it("flags lights and materials that do nothing", () => {
+ const c = codes([
+ { id: "p", kind: "light", lightType: "point", castShadow: true },
+ { id: "m", kind: "model3d", metallic: 1 },
+ ]);
+ expect(c).toContain("shadow_unsupported_light");
+ expect(c).toContain("pbr_without_environment");
+ expect(
+ codes([
+ { id: "e", kind: "light", lightType: "environment" },
+ { id: "m", kind: "model3d", metallic: 1 },
+ ]),
+ ).toEqual([]);
+ });
+
+ it("flags tracks whose keyframes never change", () => {
+ expect(
+ codes([
+ {
+ id: "s",
+ kind: "box",
+ animation: {
+ tracks: [
+ {
+ prop: "opacity",
+ keyframes: [
+ { frame: 0, value: 1 },
+ { frame: 9, value: 1 },
+ ],
+ },
+ ],
+ },
+ },
+ ]),
+ ).toEqual(["static_track"]);
+ });
+});
diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.ts
new file mode 100644
index 00000000..7989a30f
--- /dev/null
+++ b/packages/compositions/src/program/lint.ts
@@ -0,0 +1,253 @@
+/**
+ * Static checks for things that render "fine" but not as intended: a color
+ * the parser can't read (drawn transparent), text that disappears into its
+ * background, a layer parked off-canvas, a shadow on a light that can't cast
+ * one. Pure and fast; `renderVideo({ qa })` folds the findings into its report.
+ */
+import {
+ Color,
+ parseCssColor,
+ parseCssGradient,
+ type RenderDiagnostic,
+} from "@framefields/core";
+import type { CompositorProgramConfig, LayoutNode } from "./schema.js";
+
+type AnyNode = LayoutNode & Record<string, unknown>;
+
+const COLOR_PROPS = [
+ "background",
+ "fill",
+ "fillColor",
+ "strokeColor",
+ "borderColor",
+ "gradientEndColor",
+ "emissive",
+ "color",
+] as const;
+
+/** Contrast below this makes text hard to read (WCAG large-text minimum). */
+const MIN_TEXT_CONTRAST = 3;
+
+function tracksOf(
+ node: AnyNode,
+): Array<{
+ prop: string;
+ keyframes?: Array<{ value: unknown }>;
+ source?: { type?: string };
+}> {
+ const a = node.animation as { tracks?: unknown[] } | undefined;
+ return (a?.tracks ?? []) as never;
+}
+
+const animates = (node: AnyNode, props: string[]) =>
+ tracksOf(node).some((t) => props.includes(t.prop));
+
+const CONTAINERS = new Set(["box", "flex", "block", "group"]);
+
+/** Whether a layer puts pixels behind whatever is drawn after it (text over text aside). */
+function paints(node: AnyNode): boolean {
+ if (node.kind === "text" || node.kind === "light" || node.kind === "camera")
+ return false;
+ if (!CONTAINERS.has(String(node.kind))) return true;
+ const bg = node.background;
+ if (typeof bg === "string" && bg !== "" && bg !== "transparent") return true;
+ return ((node.children as AnyNode[] | undefined) ?? []).some(paints);
+}
+
+const byZ = (nodes: AnyNode[]) =>
+ [...nodes].sort(
+ (a, b) => (Number(a.zIndex) || 0) - (Number(b.zIndex) || 0),
+ );
+
+function solidColor(value: unknown): string | null {
+ if (typeof value !== "string" || value.includes("gradient(")) return null;
+ const c = parseCssColor(value);
+ return c && c[3] > 0.99 ? value : null;
+}
+
+export function lintComposition(
+ config: Pick<CompositorProgramConfig, "width" | "height" | "layout"> & {
+ backgroundColor?: string;
+ },
+): RenderDiagnostic[] {
+ const out: RenderDiagnostic[] = [];
+ const push = (
+ severity: RenderDiagnostic["severity"],
+ code: string,
+ message: string,
+ layerId?: string,
+ ) => out.push({ severity, code, message, ...(layerId && { layerId }) });
+
+ const W = config.width;
+ const H = config.height;
+ let hasEnvironment = false;
+ const ambientLights: string[] = [];
+ let has3D = false;
+ const pbrNodes: string[] = [];
+ const shadowLights: string[] = [];
+
+ const visit = (node: AnyNode, backdrop: string | null, atRoot: boolean) => {
+ const id = String(node.id ?? "?");
+
+ for (const prop of COLOR_PROPS) {
+ const v = node[prop];
+ if (typeof v !== "string" || v === "" || v === "transparent") continue;
+ if (v.includes("gradient(")) {
+ if (!parseCssGradient(v)) {
+ push(
+ "error",
+ "invalid_gradient",
+ `\`${prop}\` on '${id}' is not a gradient this renderer reads: ${v}. It draws nothing.`,
+ id,
+ );
+ }
+ } else if (!parseCssColor(v)) {
+ push(
+ "warning",
+ "unknown_color",
+ `\`${prop}: "${v}"\` on '${id}' is not a color (hex, rgb(), hsl(), oklch() or a CSS name). It draws transparent.`,
+ id,
+ );
+ }
+ }
+
+ if (node.kind === "light") {
+ const type = node.lightType ?? "directional";
+ if (type === "environment") hasEnvironment = true;
+ if (type === "ambient") ambientLights.push(id);
+ if (node.castShadow) {
+ if (type === "directional" || type === "spot") shadowLights.push(id);
+ else
+ push(
+ "warning",
+ "shadow_unsupported_light",
+ `'${id}' is a ${String(type)} light: only directional and spot lights cast shadows.`,
+ id,
+ );
+ }
+ return;
+ }
+ if (node.kind === "camera") return;
+
+ const is3D =
+ Boolean(node.is3D) || (typeof node.z === "number" && node.z !== 0);
+ if (is3D || node.kind === "model3d") has3D = true;
+ if (node.roughness !== undefined || node.metallic !== undefined)
+ pbrNodes.push(id);
+
+ // Text that vanishes into what is behind it.
+ if (
+ node.kind === "text" &&
+ backdrop &&
+ !animates(node, ["fill", "color"])
+ ) {
+ const fg = solidColor(node.fill ?? node.color);
+ const behind = solidColor(node.background) ?? backdrop;
+ if (fg) {
+ const ratio = Color.contrast(fg, behind);
+ if (ratio < MIN_TEXT_CONTRAST) {
+ push(
+ "warning",
+ "low_contrast_text",
+ `Text '${id}' (${fg}) on ${behind} has contrast ${ratio.toFixed(2)}:1 (aim for β‰₯ ${MIN_TEXT_CONTRAST}; ${Color.readableOn(behind)} reads well).`,
+ id,
+ );
+ }
+ }
+ }
+
+ // A static layer entirely outside the canvas never shows.
+ if (
+ atRoot &&
+ !is3D &&
+ node.position === "absolute" &&
+ typeof node.x === "number" &&
+ typeof node.y === "number" &&
+ typeof node.width === "number" &&
+ typeof node.height === "number" &&
+ !animates(node, [
+ "x",
+ "y",
+ "translateX",
+ "translateY",
+ "scale",
+ "rotation",
+ ])
+ ) {
+ const { x, y, width, height } = node as unknown as Record<string, number>;
+ if (x + width <= 0 || y + height <= 0 || x >= W || y >= H) {
+ push(
+ "warning",
+ "offscreen_layer",
+ `'${id}' sits entirely outside the ${W}Γ—${H} canvas (x ${x}, y ${y}) and never moves in.`,
+ id,
+ );
+ }
+ }
+
+ for (const t of tracksOf(node)) {
+ const kfs = t.keyframes ?? [];
+ if (
+ (!t.source || t.source.type === "keyframe") &&
+ kfs.length >= 2 &&
+ kfs.every(
+ (k) => JSON.stringify(k.value) === JSON.stringify(kfs[0].value),
+ )
+ ) {
+ push(
+ "info",
+ "static_track",
+ `'${id}' animates \`${t.prop}\` but every keyframe holds ${JSON.stringify(kfs[0].value)}.`,
+ id,
+ );
+ }
+ }
+
+ const ownBackdrop = solidColor(node.background) ?? backdrop;
+ const children = node.children as AnyNode[] | undefined;
+ // A full-canvas container passes "root" status down (scene boxes).
+ const fullFrame =
+ node.x === 0 && node.y === 0 && node.width === W && node.height === H;
+ visitLevel(children ?? [], ownBackdrop, atRoot && fullFrame);
+ };
+
+ // The backdrop is known only until something paints over it: text after
+ // an image, shape or panel at the same level sits on unknown pixels.
+ const visitLevel = (
+ nodes: AnyNode[],
+ backdrop: string | null,
+ atRoot: boolean,
+ ) => {
+ let known = backdrop;
+ for (const n of byZ(nodes)) {
+ visit(n, known, atRoot);
+ if (paints(n)) known = null;
+ }
+ };
+
+ const rootBackdrop = solidColor(config.backgroundColor) ?? null;
+ visitLevel(config.layout as AnyNode[], rootBackdrop, true);
+
+ if (hasEnvironment && ambientLights.length > 0) {
+ push(
+ "info",
+ "ambient_with_environment",
+ `The environment already fills shadows with light; ambient light ${ambientLights.join(", ")} adds flat light on top (remove it, or lower it, if the scene looks washed out).`,
+ );
+ }
+ if (shadowLights.length > 1) {
+ push(
+ "info",
+ "multiple_shadow_lights",
+ `${shadowLights.length} lights cast shadows (${shadowLights.join(", ")}); only the first one does.`,
+ );
+ }
+ if (has3D && pbrNodes.length > 0 && !hasEnvironment) {
+ push(
+ "info",
+ "pbr_without_environment",
+ `roughness / metallic are set (${pbrNodes.slice(0, 3).join(", ")}${pbrNodes.length > 3 ? ", …" : ""}) but the scene has no Layer.environment(): add one for physically based shading, reflections and tone mapping.`,
+ );
+ }
+ return out;
+}
diff --git a/packages/compositions/src/program/mattes.ts b/packages/compositions/src/program/mattes.ts
new file mode 100644
index 00000000..af641e34
--- /dev/null
+++ b/packages/compositions/src/program/mattes.ts
@@ -0,0 +1,54 @@
+/**
+ * Track mattes: a node's `matte` is hoisted out into its own layer right
+ * after it, so layout places it like any sibling and the renderer can draw
+ * owner and matte separately and combine them.
+ */
+import type { LayoutNode, MatteMode } from "./schema.js";
+
+type AnyNode = LayoutNode & Record<string, unknown>;
+
+const GEOMETRY = ["x", "y", "width", "height"] as const;
+
+function hoistOne(owner: AnyNode): AnyNode[] {
+ const matte = owner.matte as { node: AnyNode; mode?: MatteMode } | undefined;
+ const { matte: _drop, ...rest } = owner;
+ const self = { ...rest } as AnyNode;
+ if (Array.isArray(self.children)) {
+ self.children = hoistMattes(self.children as AnyNode[]) as never;
+ }
+ if (!matte?.node) return [self];
+
+ const src = matte.node;
+ const id = (src.id as string | undefined) ?? `${String(owner.id)}__matte`;
+ const node: AnyNode = {
+ position: "absolute",
+ ...src,
+ id,
+ matteOf: String(owner.id),
+ matteMode: matte.mode ?? "alpha",
+ };
+ // With no geometry of its own the matte covers its owner's box.
+ if (GEOMETRY.every((k) => src[k] === undefined)) {
+ for (const k of GEOMETRY) {
+ if (typeof owner[k] === "number") node[k] = owner[k];
+ }
+ }
+ // Timed with the owner: its keyframes start when the owner appears.
+ const ownerStart =
+ typeof owner.startFrame === "number" ? owner.startFrame : 0;
+ node.startFrame =
+ ownerStart + (typeof src.startFrame === "number" ? src.startFrame : 0);
+ if (src.durationFrames === undefined && owner.durationFrames !== undefined) {
+ node.durationFrames = owner.durationFrames;
+ }
+ if (Array.isArray(node.children)) {
+ node.children = hoistMattes(node.children as AnyNode[]) as never;
+ }
+ self.matteId = id;
+ return [self, node];
+}
+
+/** Returns the tree with every `matte` turned into a sibling matte layer. */
+export function hoistMattes<T extends LayoutNode>(nodes: readonly T[]): T[] {
+ return nodes.flatMap((n) => hoistOne(n as AnyNode)) as unknown as T[];
+}
diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.ts
index 01a35d11..532d5629 100644
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@ -156,9 +156,21 @@ export const AnimatablePropSchema = z.enum([
"wireframe",
"scaleX",
"scaleY",
+ // Physically based surfaces & world
+ "emissive",
+ "emissiveIntensity",
+ "exposure",
]);
export type AnimatableProp = z.infer<typeof AnimatablePropSchema>;
+export const MatteModeSchema = z.enum([
+ "alpha",
+ "alpha-inverted",
+ "luma",
+ "luma-inverted",
+]);
+export type MatteMode = z.infer<typeof MatteModeSchema>;
+
export const MaterialTypeSchema = z.enum([
"lit",
"unlit",
@@ -566,6 +578,28 @@ const BaseNodeFields = {
fresnelPower: z.number().min(1.0).max(10.0).optional(),
/** Environment reflection intensity [0, 10]. */
envIntensity: z.number().min(0).max(10.0).optional(),
+ /** Light the surface gives off (3D layers): a CSS color added on top of lighting. */
+ emissive: ColorSchema.optional(),
+ /** Emissive multiplier; above 1 glows past white (feeds bloom). Default 1. */
+ emissiveIntensity: z.number().min(0).optional(),
+ /** Falls into a `castShadow` light's shadow map. Meshes default on, flat layers off. */
+ castShadow: z.boolean().optional(),
+ /**
+ * Track matte: another layer whose alpha or brightness masks this one.
+ * Laid out like a sibling (same parent), timed with this node; with no
+ * x/y/width/height of its own it covers this node's box.
+ */
+ matte: z
+ .object({
+ node: z.custom<unknown>((v) => typeof v === "object" && v !== null),
+ mode: MatteModeSchema.optional(),
+ })
+ .optional(),
+ /** Set by the compiler: the id of this node's hoisted matte layer. */
+ matteId: z.string().optional(),
+ /** Set by the compiler: this layer is the matte of node `matteOf` (not drawn by itself). */
+ matteOf: z.string().optional(),
+ matteMode: MatteModeSchema.optional(),
/** Per-node motion: tracks keyed by node id. */
animation: LayerAnimationSchema.default({ tracks: [] }),
/** Vision configuration or effect attached directly to the node. */
@@ -618,6 +652,13 @@ export interface CompositionNodeBase {
transmission?: number;
dispersion?: number;
fresnelPower?: number;
+ emissive?: string;
+ emissiveIntensity?: number;
+ castShadow?: boolean;
+ matte?: { node: unknown; mode?: MatteMode };
+ matteId?: string;
+ matteOf?: string;
+ matteMode?: MatteMode;
envIntensity?: number;
startFrame?: number;
durationFrames?: number;
@@ -1026,14 +1067,57 @@ export const CameraNodeSchema: z.ZodType<CameraNode> = z
})
.strict() as unknown as z.ZodType<CameraNode>;
-export type LightType = "ambient" | "directional" | "point" | "spot";
+export type LightType =
+ | "ambient"
+ | "directional"
+ | "point"
+ | "spot"
+ | "environment";
export const LightTypeSchema = z.enum([
"ambient",
"directional",
"point",
"spot",
+ /** Scene world: image-based light, tone mapping, exposure, fog. Turns on physical shading. */
+ "environment",
]);
+export const EnvironmentPresetSchema = z.enum([
+ "studio",
+ "soft",
+ "sunset",
+ "night",
+ "neutral",
+]);
+export type EnvironmentPreset = z.infer<typeof EnvironmentPresetSchema>;
+
+export const ToneMappingSchema = z.enum(["neutral", "agx", "aces", "none"]);
+export type ToneMapping = z.infer<typeof ToneMappingSchema>;
+
+export const EnvironmentFogSchema = z
+ .object({
+ color: ColorSchema,
+ start: z.number().min(0),
+ end: z.number().min(0),
+ opacity: z.number().min(0).max(1).optional(),
+ })
+ .strict();
+export type EnvironmentFog = z.infer<typeof EnvironmentFogSchema>;
+
+export const BloomConfigSchema = z
+ .object({
+ /** Glow strength. Default 0.2. */
+ intensity: z.number().min(0).max(10).optional(),
+ /** Scene radiance where glow starts (1 = white). Default 2.5. */
+ threshold: z.number().min(0).max(16).optional(),
+ /** Soft transition below the threshold. Default 0.4. */
+ knee: z.number().min(0).max(1).optional(),
+ /** Spread in mip levels (1..8). Default 6. */
+ radius: z.number().min(1).max(8).optional(),
+ })
+ .strict();
+export type BloomConfig = z.infer<typeof BloomConfigSchema>;
+
export interface LightNode {
id: string;
kind: "light";
@@ -1051,6 +1135,26 @@ export interface LightNode {
decay?: number;
angle?: number;
penumbra?: number;
+ /** Directional / spot: render a shadow map (one shadow light per scene). */
+ castShadow?: boolean;
+ /** Shadow edge blur in shadow-map texels. Default 2.5. */
+ shadowSoftness?: number;
+ /** 0..1, how dark shadows get. Default 1. */
+ shadowDarkness?: number;
+ /** Shadow map resolution (texels per side). Default 2048. */
+ shadowMapSize?: number;
+ // environment
+ preset?: EnvironmentPreset;
+ top?: string;
+ horizon?: string;
+ bottom?: string;
+ softboxes?: number;
+ rotation?: number;
+ exposure?: number;
+ toneMapping?: ToneMapping;
+ fog?: EnvironmentFog;
+ /** Glow around emissive surfaces and hot highlights: `true` or settings. */
+ bloom?: boolean | BloomConfig;
startFrame?: number;
durationFrames?: number;
animation?: LayerAnimationSpec;
@@ -1074,6 +1178,20 @@ export const LightNodeSchema: z.ZodType<LightNode> = z
decay: z.number().min(0).default(2.0),
angle: z.number().min(0).max(180).default(45),
penumbra: z.number().min(0).max(1).default(0.2),
+ castShadow: z.boolean().optional(),
+ shadowSoftness: z.number().min(0).max(64).optional(),
+ shadowDarkness: z.number().min(0).max(1).optional(),
+ shadowMapSize: z.number().int().min(256).max(8192).optional(),
+ preset: EnvironmentPresetSchema.optional(),
+ top: ColorSchema.optional(),
+ horizon: ColorSchema.optional(),
+ bottom: ColorSchema.optional(),
+ softboxes: z.number().min(0).optional(),
+ rotation: z.number().optional(),
+ exposure: z.number().min(-16).max(16).optional(),
+ toneMapping: ToneMappingSchema.optional(),
+ fog: EnvironmentFogSchema.optional(),
+ bloom: z.union([z.boolean(), BloomConfigSchema]).optional(),
startFrame: z.number().int().min(0).default(0),
durationFrames: z.number().int().min(1).optional(),
animation: LayerAnimationSchema.default({ tracks: [] }),
diff --git a/packages/compositions/src/program/sequence.ts b/packages/compositions/src/program/sequence.ts
index e85868d4..30717b5a 100644
--- a/packages/compositions/src/program/sequence.ts
+++ b/packages/compositions/src/program/sequence.ts
@@ -15,6 +15,10 @@ import type { CompositorProgramConfig, LayoutNode } from "./schema.js";
/**
* Camera Transition Type across scene boundaries.
+ *
+ * @deprecated Recorded on scenes but not rendered. Use framefields'
+ * `Layer.sequence([...])` with `Transition.push` / `wipe` / `iris` /
+ * `crossfade` / `zoom` / `whip`, which render as keyframes and mattes.
*/
export const SceneTransitionTypeSchema = z.enum([
"cut", // Instant zero-frame hard cut
diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.ts
index e90271bc..fe83b558 100644
--- a/packages/compositions/src/program/to-virtual-media.ts
+++ b/packages/compositions/src/program/to-virtual-media.ts
@@ -11,6 +11,7 @@
* Pure + deterministic: same doc β†’ same tree. No width/height resolution
* here β€” geometry comes from `computeLayout` at render time.
*/
+
import {
createVirtualMedia,
type DataType,
@@ -18,6 +19,7 @@ import {
type VirtualMediaData,
} from "@framefields/core";
import { collectInlineSignals } from "./inline-signals.js";
+import { hoistMattes } from "./mattes.js";
import type { CompositorProgramConfig, LayoutNode } from "./schema.js";
import { computeStaggerDelay } from "./stagger.js";
@@ -313,7 +315,7 @@ export function compositorToProgram(
(config as any).onRequestFrame ?? (options as any).onRequestFrame,
}),
},
- children: config.layout.map((n) =>
+ children: hoistMattes(config.layout).map((n) =>
nodeToVirtualMedia(n, resolveMedia, fps, 0, durationMs),
),
} as unknown as VirtualMediaData;
diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts
index 6ca21cc8..fca15a8d 100644
--- a/packages/core/src/index.ts
+++ b/packages/core/src/index.ts
@@ -5,6 +5,8 @@ export * from "./node-metadata.js";
export * from "./render-diagnostics.js";
export * from "./signals/index.js";
export * from "./types/index.js";
+export * from "./utils/color.js";
+export * from "./utils/gradient.js";
export * from "./utils/is-equal.js";
export * from "./utils/result-cache.js";
export * from "./utils/svg.js";
diff --git a/packages/core/src/utils/color.test.ts b/packages/core/src/utils/color.test.ts
new file mode 100644
index 00000000..a9192c2c
--- /dev/null
+++ b/packages/core/src/utils/color.test.ts
@@ -0,0 +1,92 @@
+import { describe, expect, it } from "vitest";
+import { Color, mixColors, parseCssColor, sampleColorStops } from "./color.js";
+
+const round = (c: number[] | null) => c?.map((v) => Math.round(v * 255));
+
+describe("parseCssColor", () => {
+ it("parses hex, rgb, hsl, oklch and named colors", () => {
+ expect(round(parseCssColor("#f80"))).toEqual([255, 136, 0, 255]);
+ expect(round(parseCssColor("#ff880080"))).toEqual([255, 136, 0, 128]);
+ expect(round(parseCssColor("rgb(10, 20, 30)"))).toEqual([10, 20, 30, 255]);
+ expect(round(parseCssColor("rgb(10 20 30 / 50%)"))).toEqual([
+ 10, 20, 30, 128,
+ ]);
+ expect(round(parseCssColor("rgba(255,0,0,0.5)"))).toEqual([255, 0, 0, 128]);
+ expect(round(parseCssColor("hsl(120, 100%, 50%)"))).toEqual([
+ 0, 255, 0, 255,
+ ]);
+ expect(round(parseCssColor("hsl(240deg 100% 50%)"))).toEqual([
+ 0, 0, 255, 255,
+ ]);
+ expect(round(parseCssColor("oklch(1 0 0)"))).toEqual([255, 255, 255, 255]);
+ expect(round(parseCssColor("oklch(0% 0 0)"))).toEqual([0, 0, 0, 255]);
+ expect(round(parseCssColor("RebeccaPurple"))).toEqual([102, 51, 153, 255]);
+ expect(parseCssColor("not-a-color")).toBeNull();
+ expect(parseCssColor("#ggg")).toBeNull();
+ });
+
+ it("round-trips sRGB through OKLab", () => {
+ for (const hex of ["#ff5a1f", "#1f6bff", "#123456", "#ffffff", "#000000"]) {
+ expect(mixColors(hex, hex, 0.5)).toBe(hex);
+ }
+ });
+});
+
+describe("mixColors", () => {
+ it("hits the endpoints and blends perceptually", () => {
+ expect(mixColors("#ff0000", "#0000ff", 0)).toBe("#ff0000");
+ expect(mixColors("#ff0000", "#0000ff", 1)).toBe("#0000ff");
+ // OKLab midpoint of black/white is L = 0.5, darker than the sRGB average.
+ expect(mixColors("#000000", "#ffffff", 0.5)).toBe("#636363");
+ });
+
+ it("fades to transparent without drifting through grey", () => {
+ const mid = parseCssColor(mixColors("#ff0000", "transparent", 0.5))!;
+ expect(mid[0]).toBeCloseTo(1, 2);
+ expect(mid[1]).toBeCloseTo(0, 2);
+ expect(mid[3]).toBeCloseTo(0.5, 1);
+ });
+});
+
+describe("Color helpers", () => {
+ it("builds colors from hsl and oklch", () => {
+ expect(Color.hsl(0, 1, 0.5)).toBe("#ff0000");
+ expect(Color.oklch(1, 0, 0)).toBe("#ffffff");
+ expect(Color.rgb(255, 0, 0, 0.5)).toBe("#ff000080");
+ expect(Color.hex("rgb(255 136 0)")).toBe("#ff8800");
+ });
+
+ it("samples ramps and steps", () => {
+ const ramp = Color.ramp(["#000000", "#ffffff"]);
+ expect(ramp(0)).toBe("#000000");
+ expect(ramp(1)).toBe("#ffffff");
+ expect(Color.steps(["#ff0000", "#00ff00", "#0000ff"], 3)).toEqual([
+ "#ff0000",
+ "#00ff00",
+ "#0000ff",
+ ]);
+ expect(
+ sampleColorStops(
+ [
+ { offset: 1, color: "#0000ff" },
+ { offset: 0, color: "#ff0000" },
+ ],
+ 0,
+ ),
+ ).toBe("#ff0000");
+ });
+
+ it("adjusts lightness, alpha and contrast", () => {
+ const base = "#3366cc";
+ const lighter = parseCssColor(Color.lighten(base, 0.2))!;
+ const orig = parseCssColor(base)!;
+ expect(lighter[0] + lighter[1] + lighter[2]).toBeGreaterThan(
+ orig[0] + orig[1] + orig[2],
+ );
+ expect(Color.alpha("#000", 0.4)).toBe("#00000066");
+ expect(Color.contrast("#000000", "#ffffff")).toBeCloseTo(21, 0);
+ expect(Color.readableOn("#111111")).toBe("#ffffff");
+ expect(Color.readableOn("#f5f5f5")).toBe("#000000");
+ expect(Color.saturate("#ff0000", 0)).toMatch(/^#([0-9a-f]{2})\1\1$/);
+ });
+});
diff --git a/packages/core/src/utils/color.ts b/packages/core/src/utils/color.ts
new file mode 100644
index 00000000..bc599185
--- /dev/null
+++ b/packages/core/src/utils/color.ts
@@ -0,0 +1,404 @@
+/**
+ * CSS color parsing, perceptual mixing and authoring helpers, shared by the
+ * schema evaluator (color tracks), the GPU renderers (fills) and the public
+ * `Color` helper. Pure and dependency-free.
+ *
+ * Every helper returns a CSS string any color prop accepts: `#rrggbb`, or
+ * `#rrggbbaa` when not fully opaque.
+ */
+
+/** Straight (non-premultiplied) sRGB, each channel 0..1. */
+export type Rgba = [r: number, g: number, b: number, a: number];
+
+const NAMED: Record<string, string> = {
+ transparent: "#00000000",
+ black: "#000000",
+ white: "#ffffff",
+ red: "#ff0000",
+ green: "#008000",
+ lime: "#00ff00",
+ blue: "#0000ff",
+ yellow: "#ffff00",
+ cyan: "#00ffff",
+ aqua: "#00ffff",
+ magenta: "#ff00ff",
+ fuchsia: "#ff00ff",
+ orange: "#ffa500",
+ purple: "#800080",
+ pink: "#ffc0cb",
+ gray: "#808080",
+ grey: "#808080",
+ silver: "#c0c0c0",
+ maroon: "#800000",
+ navy: "#000080",
+ teal: "#008080",
+ olive: "#808000",
+ gold: "#ffd700",
+ coral: "#ff7f50",
+ salmon: "#fa8072",
+ crimson: "#dc143c",
+ indigo: "#4b0082",
+ violet: "#ee82ee",
+ turquoise: "#40e0d0",
+ tomato: "#ff6347",
+ chocolate: "#d2691e",
+ tan: "#d2b48c",
+ beige: "#f5f5dc",
+ ivory: "#fffff0",
+ khaki: "#f0e68c",
+ lavender: "#e6e6fa",
+ mint: "#98ff98",
+ skyblue: "#87ceeb",
+ slategray: "#708090",
+ dimgray: "#696969",
+ lightgray: "#d3d3d3",
+ darkgray: "#a9a9a9",
+ whitesmoke: "#f5f5f5",
+ hotpink: "#ff69b4",
+ deeppink: "#ff1493",
+ dodgerblue: "#1e90ff",
+ royalblue: "#4169e1",
+ steelblue: "#4682b4",
+ midnightblue: "#191970",
+ limegreen: "#32cd32",
+ forestgreen: "#228b22",
+ seagreen: "#2e8b57",
+ darkorange: "#ff8c00",
+ orangered: "#ff4500",
+ firebrick: "#b22222",
+ rebeccapurple: "#663399",
+};
+
+const clamp01 = (v: number) => (v < 0 ? 0 : v > 1 ? 1 : v);
+
+function parseHex(h: string): Rgba | null {
+ if (!/^[0-9a-f]+$/i.test(h)) return null;
+ const x = (i: number, n: number) =>
+ parseInt(n === 1 ? h[i] + h[i] : h.slice(i, i + 2), 16) / 255;
+ switch (h.length) {
+ case 3:
+ return [x(0, 1), x(1, 1), x(2, 1), 1];
+ case 4:
+ return [x(0, 1), x(1, 1), x(2, 1), x(3, 1)];
+ case 6:
+ return [x(0, 2), x(2, 2), x(4, 2), 1];
+ case 8:
+ return [x(0, 2), x(2, 2), x(4, 2), x(6, 2)];
+ default:
+ return null;
+ }
+}
+
+/** Splits `rgb(1 2 3 / 50%)` or `rgb(1, 2, 3, .5)` into its argument tokens. */
+function fnArgs(body: string): string[] {
+ return body
+ .replace(/\s*\/\s*/, " ")
+ .split(/[\s,]+/)
+ .filter(Boolean);
+}
+
+/** A number, or a percentage scaled so 100% = `pct100`. */
+function num(tok: string | undefined, pct100: number): number {
+ if (tok === undefined || tok === "none") return 0;
+ if (tok.endsWith("%")) return (parseFloat(tok) / 100) * pct100;
+ return parseFloat(tok);
+}
+
+function hue(tok: string | undefined): number {
+ if (tok === undefined) return 0;
+ if (tok.endsWith("turn")) return parseFloat(tok) * 360;
+ if (tok.endsWith("rad")) return (parseFloat(tok) * 180) / Math.PI;
+ return parseFloat(tok);
+}
+
+function alpha(tok: string | undefined): number {
+ return tok === undefined ? 1 : clamp01(num(tok, 1));
+}
+
+function hslToRgb(h: number, s: number, l: number): [number, number, number] {
+ const a = s * Math.min(l, 1 - l);
+ const f = (n: number) => {
+ const k = (n + h / 30) % 12;
+ return l - a * Math.max(-1, Math.min(k - 3, 9 - k, 1));
+ };
+ return [f(0), f(8), f(4)];
+}
+
+// ── OKLab (BjΓΆrn Ottosson) ─────────────────────────────────────────────
+
+const toLinear = (c: number) =>
+ c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4;
+const toGamma = (c: number) =>
+ c <= 0.0031308 ? 12.92 * c : 1.055 * c ** (1 / 2.4) - 0.055;
+
+/** sRGB (0..1) β†’ OKLab [L, a, b]. */
+export function rgbToOklab(
+ r: number,
+ g: number,
+ b: number,
+): [number, number, number] {
+ const lr = toLinear(r);
+ const lg = toLinear(g);
+ const lb = toLinear(b);
+ const l = Math.cbrt(
+ 0.4122214708 * lr + 0.5363325363 * lg + 0.0514459929 * lb,
+ );
+ const m = Math.cbrt(
+ 0.2119034982 * lr + 0.6806995451 * lg + 0.1073969566 * lb,
+ );
+ const s = Math.cbrt(
+ 0.0883024619 * lr + 0.2817188376 * lg + 0.6299787005 * lb,
+ );
+ return [
+ 0.2104542553 * l + 0.793617785 * m - 0.0040720468 * s,
+ 1.9779984951 * l - 2.428592205 * m + 0.4505937099 * s,
+ 0.0259040371 * l + 0.7827717662 * m - 0.808675766 * s,
+ ];
+}
+
+/** OKLab β†’ sRGB (0..1, clamped to gamut). */
+export function oklabToRgb(
+ L: number,
+ a: number,
+ b: number,
+): [number, number, number] {
+ const l = (L + 0.3963377774 * a + 0.2158037573 * b) ** 3;
+ const m = (L - 0.1055613458 * a - 0.0638541728 * b) ** 3;
+ const s = (L - 0.0894841775 * a - 1.291485548 * b) ** 3;
+ return [
+ clamp01(toGamma(4.0767416621 * l - 3.3077115913 * m + 0.2309699292 * s)),
+ clamp01(toGamma(-1.2684380046 * l + 2.6097574011 * m - 0.3413193965 * s)),
+ clamp01(toGamma(-0.0041960863 * l - 0.7034186147 * m + 1.707614701 * s)),
+ ];
+}
+
+/**
+ * Parses any CSS color this engine accepts: hex (3/4/6/8), `rgb()`/`rgba()`,
+ * `hsl()`/`hsla()`, `oklch()`, `oklab()` (comma or space syntax, `/ alpha`)
+ * and common named colors. Returns `null` when the string is not a color.
+ */
+export function parseCssColor(input: string): Rgba | null {
+ const s = input.trim().toLowerCase();
+ if (!s) return null;
+ if (s.startsWith("#")) return parseHex(s.slice(1));
+ const named = NAMED[s];
+ if (named) return parseHex(named.slice(1));
+
+ const m = /^([a-z]+)\((.*)\)$/.exec(s);
+ if (!m) return null;
+ const [, fn, body] = m;
+ const t = fnArgs(body);
+ switch (fn) {
+ case "rgb":
+ case "rgba":
+ return [
+ clamp01(num(t[0], 255) / 255),
+ clamp01(num(t[1], 255) / 255),
+ clamp01(num(t[2], 255) / 255),
+ alpha(t[3]),
+ ];
+ case "hsl":
+ case "hsla": {
+ const [r, g, b] = hslToRgb(
+ ((hue(t[0]) % 360) + 360) % 360,
+ clamp01(num(t[1], 100) / 100),
+ clamp01(num(t[2], 100) / 100),
+ );
+ return [r, g, b, alpha(t[3])];
+ }
+ case "oklch": {
+ const L = num(t[0], 1);
+ const C = num(t[1], 0.4);
+ const H = (hue(t[2]) * Math.PI) / 180;
+ const [r, g, b] = oklabToRgb(L, C * Math.cos(H), C * Math.sin(H));
+ return [r, g, b, alpha(t[3])];
+ }
+ case "oklab": {
+ const [r, g, b] = oklabToRgb(
+ num(t[0], 1),
+ num(t[1], 0.4),
+ num(t[2], 0.4),
+ );
+ return [r, g, b, alpha(t[3])];
+ }
+ default:
+ return null;
+ }
+}
+
+/** Formats straight sRGB as `#rrggbb` (or `#rrggbbaa` when translucent). */
+export function formatHex([r, g, b, a]: Rgba): string {
+ const h = (v: number) =>
+ Math.round(clamp01(v) * 255)
+ .toString(16)
+ .padStart(2, "0");
+ return `#${h(r)}${h(g)}${h(b)}${a < 0.999 ? h(a) : ""}`;
+}
+
+const parseOr = (c: string): Rgba => parseCssColor(c) ?? [0, 0, 0, 0];
+
+/**
+ * Mixes two colors perceptually (in OKLab), `t` = 0 β†’ `a`, 1 β†’ `b`.
+ * Midpoints keep their lightness and saturation instead of going muddy
+ * the way a plain sRGB lerp does.
+ */
+export function mixColors(a: string, b: string, t: number): string {
+ const ca = parseOr(a);
+ const cb = parseOr(b);
+ // Transparent ends take the other end's hue, so fades don't go grey.
+ const pa = ca[3] < 1e-4 ? cb : ca;
+ const pb = cb[3] < 1e-4 ? ca : cb;
+ const la = rgbToOklab(pa[0], pa[1], pa[2]);
+ const lb = rgbToOklab(pb[0], pb[1], pb[2]);
+ const k = clamp01(t);
+ const [r, g, bl] = oklabToRgb(
+ la[0] + (lb[0] - la[0]) * k,
+ la[1] + (lb[1] - la[1]) * k,
+ la[2] + (lb[2] - la[2]) * k,
+ );
+ return formatHex([r, g, bl, ca[3] + (cb[3] - ca[3]) * k]);
+}
+
+export interface ColorStopLike {
+ offset: number;
+ color: string;
+}
+
+/** Samples a multi-stop gradient (stops in any order) at `t` ∈ [0, 1]. */
+export function sampleColorStops(
+ stops: readonly (string | ColorStopLike)[],
+ t: number,
+): string {
+ const list = normalizeColorStops(stops);
+ if (list.length === 0) return "#00000000";
+ if (t <= list[0].offset) return list[0].color;
+ for (let i = 1; i < list.length; i++) {
+ const b = list[i];
+ if (t <= b.offset) {
+ const a = list[i - 1];
+ const span = b.offset - a.offset;
+ return mixColors(a.color, b.color, span > 0 ? (t - a.offset) / span : 1);
+ }
+ }
+ return list[list.length - 1].color;
+}
+
+/** Bare color strings become evenly spaced stops; result is sorted by offset. */
+export function normalizeColorStops(
+ stops: readonly (string | ColorStopLike)[],
+): ColorStopLike[] {
+ const n = stops.length;
+ return stops
+ .map((s, i) =>
+ typeof s === "string"
+ ? { offset: n > 1 ? i / (n - 1) : 0, color: s }
+ : { offset: clamp01(s.offset), color: s.color },
+ )
+ .sort((a, b) => a.offset - b.offset);
+}
+
+function oklchOf(c: string): [number, number, number, number] {
+ const [r, g, b, a] = parseOr(c);
+ const [L, A, B] = rgbToOklab(r, g, b);
+ return [L, Math.hypot(A, B), (Math.atan2(B, A) * 180) / Math.PI, a];
+}
+
+function fromOklch(L: number, C: number, H: number, a: number): string {
+ const h = (H * Math.PI) / 180;
+ const [r, g, b] = oklabToRgb(L, C * Math.cos(h), C * Math.sin(h));
+ return formatHex([r, g, b, a]);
+}
+
+/**
+ * Authoring helpers for colors. Every function returns a hex string, so the
+ * result drops straight into any color prop or keyframe.
+ *
+ * ```ts
+ * Color.oklch(0.7, 0.15, 250) // perceptual: equal L reads equally bright
+ * Color.hsl(212, 0.95, 0.74)
+ * Color.mix("#ff5a1f", "#1f6bff", 0.5) // OKLab, no muddy midpoint
+ * const wave = Color.ramp(["#3b82f6", "#8b5cf6", "#ec4899"]);
+ * wave(0.5) // sample a gradient
+ * Color.lighten("#ff5a1f", 0.1), Color.alpha("#000", 0.4)
+ * ```
+ */
+export const Color = {
+ /** Parses any supported CSS color to straight sRGB 0..1, or null. */
+ parse: parseCssColor,
+ /** Normalizes any supported CSS color to hex. */
+ hex(c: string): string {
+ return formatHex(parseOr(c));
+ },
+ /** `r, g, b` in 0..255, `a` in 0..1. */
+ rgb(r: number, g: number, b: number, a = 1): string {
+ return formatHex([r / 255, g / 255, b / 255, a]);
+ },
+ /** Hue in degrees; saturation and lightness in 0..1. */
+ hsl(h: number, s: number, l: number, a = 1): string {
+ const [r, g, b] = hslToRgb(((h % 360) + 360) % 360, clamp01(s), clamp01(l));
+ return formatHex([r, g, b, a]);
+ },
+ /**
+ * Perceptual lightness 0..1, chroma ~0..0.37, hue in degrees. Colors with
+ * the same `l` look equally bright, which makes balanced palettes easy.
+ */
+ oklch(l: number, c: number, h: number, a = 1): string {
+ return fromOklch(l, c, h, a);
+ },
+ /** Perceptual (OKLab) mix; `t` = 0 β†’ `a`, 1 β†’ `b`. */
+ mix: mixColors,
+ /** A gradient sampler: `Color.ramp(["#f00", "#00f"])(0.25)`. */
+ ramp(stops: readonly (string | ColorStopLike)[]): (t: number) => string {
+ const list = normalizeColorStops(stops);
+ return (t: number) => sampleColorStops(list, t);
+ },
+ /** `n` evenly spaced swatches along a gradient. */
+ steps(stops: readonly (string | ColorStopLike)[], n: number): string[] {
+ const list = normalizeColorStops(stops);
+ return Array.from({ length: n }, (_, i) =>
+ sampleColorStops(list, n > 1 ? i / (n - 1) : 0),
+ );
+ },
+ /** Raises perceptual lightness by `amount` (0..1). */
+ lighten(c: string, amount: number): string {
+ const [L, C, H, a] = oklchOf(c);
+ return fromOklch(clamp01(L + amount), C, H, a);
+ },
+ darken(c: string, amount: number): string {
+ const [L, C, H, a] = oklchOf(c);
+ return fromOklch(clamp01(L - amount), C, H, a);
+ },
+ /** Scales chroma: 0 = grey, 1 = unchanged, >1 = more vivid. */
+ saturate(c: string, factor: number): string {
+ const [L, C, H, a] = oklchOf(c);
+ return fromOklch(L, C * factor, H, a);
+ },
+ /** Rotates hue by `degrees`, keeping perceptual lightness. */
+ rotate(c: string, degrees: number): string {
+ const [L, C, H, a] = oklchOf(c);
+ return fromOklch(L, C, H + degrees, a);
+ },
+ /** Replaces alpha. */
+ alpha(c: string, a: number): string {
+ const [r, g, b] = parseOr(c);
+ return formatHex([r, g, b, clamp01(a)]);
+ },
+ /** WCAG relative-luminance contrast ratio (1..21). β‰₯ 4.5 for body text. */
+ contrast(a: string, b: string): number {
+ const lum = (c: string) => {
+ const [r, g, bl] = parseOr(c);
+ return (
+ 0.2126 * toLinear(r) + 0.7152 * toLinear(g) + 0.0722 * toLinear(bl)
+ );
+ };
+ const [hi, lo] = [lum(a), lum(b)].sort((x, y) => y - x);
+ return (hi + 0.05) / (lo + 0.05);
+ },
+ /** Black or white, whichever reads better on `background`. */
+ readableOn(background: string): string {
+ return Color.contrast(background, "#ffffff") >=
+ Color.contrast(background, "#000000")
+ ? "#ffffff"
+ : "#000000";
+ },
+} as const;
diff --git a/packages/core/src/utils/gradient.test.ts b/packages/core/src/utils/gradient.test.ts
new file mode 100644
index 00000000..2ba41a79
--- /dev/null
+++ b/packages/core/src/utils/gradient.test.ts
@@ -0,0 +1,109 @@
+import { describe, expect, it } from "vitest";
+import {
+ Gradient,
+ isCssGradient,
+ limitGradientStops,
+ parseCssGradient,
+} from "./gradient.js";
+
+const offsets = (s: string) =>
+ parseCssGradient(s)?.stops.map((x) => +x.offset.toFixed(3));
+
+describe("parseCssGradient", () => {
+ it("parses linear angles, sides and stop positions", () => {
+ expect(parseCssGradient("linear-gradient(#000, #fff)")?.angle).toBe(180);
+ expect(
+ parseCssGradient("linear-gradient(to right, #000, #fff)")?.angle,
+ ).toBe(90);
+ expect(
+ parseCssGradient("linear-gradient(to top left, red, blue)")?.angle,
+ ).toBe(315);
+ expect(
+ parseCssGradient("linear-gradient(0.25turn, red, blue)")?.angle,
+ ).toBe(90);
+ expect(offsets("linear-gradient(45deg, red, lime 30%, blue)")).toEqual([
+ 0, 0.3, 1,
+ ]);
+ expect(offsets("linear-gradient(red, lime, blue, white)")).toEqual([
+ 0, 0.333, 0.667, 1,
+ ]);
+ // Two-position stops make a hard band; later stops never go backwards.
+ expect(offsets("linear-gradient(red 20%, blue 10% 40%)")).toEqual([
+ 0.2, 0.2, 0.4,
+ ]);
+ // Unitless 0 is a valid position: hard-edged bands.
+ expect(offsets("linear-gradient(red 0 50%, blue 0)")).toEqual([
+ 0, 0.5, 0.5,
+ ]);
+ });
+
+ it("parses colors with functional syntax inside stops", () => {
+ const g = parseCssGradient(
+ "linear-gradient(90deg, rgb(255 0 0 / 50%), hsl(240 100% 50%) 80%)",
+ );
+ expect(g?.stops).toHaveLength(2);
+ expect(g?.stops[0].color[3]).toBeCloseTo(0.5, 2);
+ expect(g?.stops[1].color[2]).toBeCloseTo(1, 2);
+ });
+
+ it("parses radial and conic configuration", () => {
+ const r = parseCssGradient(
+ "radial-gradient(circle at 25% 75%, #fff, transparent)",
+ );
+ expect(r?.type).toBe("radial");
+ expect(r?.shape).toBe("circle");
+ expect(r?.center).toEqual([0.25, 0.75]);
+ expect(
+ parseCssGradient("radial-gradient(120px at center, red, blue)")?.radius,
+ ).toEqual([120, 120]);
+ const c = parseCssGradient(
+ "conic-gradient(from 90deg at left top, red, blue 0.5turn)",
+ );
+ expect(c?.from).toBe(90);
+ expect(c?.center).toEqual([0, 0]);
+ expect(c?.stops[1].offset).toBeCloseTo(0.5);
+ // No config: the first part is a stop.
+ expect(parseCssGradient("radial-gradient(red, blue)")?.stops).toHaveLength(
+ 2,
+ );
+ });
+
+ it("rejects non-gradients and bad colors", () => {
+ expect(parseCssGradient("#ff0000")).toBeNull();
+ expect(
+ parseCssGradient("linear-gradient(to right, nope, #fff)"),
+ ).toBeNull();
+ expect(isCssGradient("conic-gradient(red, blue)")).toBe(true);
+ expect(isCssGradient("red")).toBe(false);
+ });
+
+ it("resamples long stop lists to the GPU limit", () => {
+ const g = parseCssGradient(
+ `linear-gradient(${Array.from({ length: 12 }, (_, i) => (i % 2 ? "#000" : "#fff")).join(", ")})`,
+ )!;
+ const lim = limitGradientStops(g.stops);
+ expect(lim).toHaveLength(8);
+ expect(lim[0].offset).toBe(0);
+ expect(lim[7].offset).toBe(1);
+ });
+});
+
+describe("Gradient helpers", () => {
+ it("write strings the parser reads back", () => {
+ const lin = parseCssGradient(
+ Gradient.linear(["#f00", { color: "#00f", offset: 0.6 }], 135),
+ )!;
+ expect(lin.angle).toBe(135);
+ expect(lin.stops[1].offset).toBeCloseTo(0.6);
+ const rad = parseCssGradient(
+ Gradient.radial(["#fff", "#000"], { at: [0.2, 0.3], circle: true }),
+ )!;
+ expect(rad.center[0]).toBeCloseTo(0.2);
+ expect(rad.shape).toBe("circle");
+ const con = parseCssGradient(
+ Gradient.conic(["red", "blue"], { from: 45 }),
+ )!;
+ expect(con.type).toBe("conic");
+ expect(con.from).toBe(45);
+ });
+});
diff --git a/packages/core/src/utils/gradient.ts b/packages/core/src/utils/gradient.ts
new file mode 100644
index 00000000..4d29ebc1
--- /dev/null
+++ b/packages/core/src/utils/gradient.ts
@@ -0,0 +1,370 @@
+/**
+ * CSS gradients (`linear-gradient`, `radial-gradient`, `conic-gradient`) as
+ * values for any `background` or shape `fill`, plus `Gradient` helpers that
+ * write them. Colors blend perceptually (OKLab), not through muddy sRGB.
+ */
+import { parseCssColor, type Rgba, rgbToOklab } from "./color.js";
+
+export type GradientType = "linear" | "radial" | "conic";
+
+export interface ParsedGradientStop {
+ /** 0..1 along the gradient line / radius / turn. */
+ offset: number;
+ /** Straight sRGB 0..1. */
+ color: Rgba;
+}
+
+export interface ParsedGradient {
+ type: GradientType;
+ /** Linear: CSS angle in degrees (0 = to top, 90 = to right, 180 = to bottom). */
+ angle: number;
+ /** Radial / conic center as a fraction of the box (0.5, 0.5 = middle). */
+ center: [number, number];
+ /** Radial: circle or ellipse sized to the farthest corner, or explicit px radii. */
+ shape: "circle" | "ellipse";
+ radius?: [number, number];
+ /** Conic: start angle in degrees (0 = top, clockwise). */
+ from: number;
+ stops: ParsedGradientStop[];
+}
+
+/** Gradients this renderer draws at most this many stops (extra ones are resampled). */
+export const MAX_GRADIENT_STOPS = 8;
+
+const GRADIENT_RE = /^\s*(linear|radial|conic)-gradient\(\s*([\s\S]*)\)\s*$/i;
+
+export function isCssGradient(value: unknown): value is string {
+ return typeof value === "string" && GRADIENT_RE.test(value);
+}
+
+/** Splits on commas that are not inside parentheses. */
+function splitTopLevel(s: string): string[] {
+ const out: string[] = [];
+ let depth = 0;
+ let cur = "";
+ for (const ch of s) {
+ if (ch === "(") depth++;
+ if (ch === ")") depth--;
+ if (ch === "," && depth === 0) {
+ out.push(cur.trim());
+ cur = "";
+ } else cur += ch;
+ }
+ if (cur.trim()) out.push(cur.trim());
+ return out;
+}
+
+function parseAngle(tok: string): number | null {
+ const m = /^(-?[\d.]+)(deg|turn|rad|grad)?$/i.exec(tok.trim());
+ if (!m) return null;
+ const v = parseFloat(m[1]);
+ switch ((m[2] ?? "deg").toLowerCase()) {
+ case "turn":
+ return v * 360;
+ case "rad":
+ return (v * 180) / Math.PI;
+ case "grad":
+ return v * 0.9;
+ default:
+ return v;
+ }
+}
+
+const SIDE_ANGLES: Record<string, number> = {
+ top: 0,
+ right: 90,
+ bottom: 180,
+ left: 270,
+ "top right": 45,
+ "right top": 45,
+ "bottom right": 135,
+ "right bottom": 135,
+ "bottom left": 225,
+ "left bottom": 225,
+ "top left": 315,
+ "left top": 315,
+};
+
+const POSITION_KEYWORDS: Record<string, number> = {
+ left: 0,
+ top: 0,
+ center: 0.5,
+ right: 1,
+ bottom: 1,
+};
+
+/** `at 30% 70%`, `at left top`, `at center`. */
+function parsePosition(tokens: string[]): [number, number] {
+ const vals = tokens.map((t) =>
+ t in POSITION_KEYWORDS
+ ? {
+ v: POSITION_KEYWORDS[t],
+ axis:
+ t === "top" || t === "bottom"
+ ? "y"
+ : t === "left" || t === "right"
+ ? "x"
+ : "any",
+ }
+ : { v: t.endsWith("%") ? parseFloat(t) / 100 : 0.5, axis: "any" },
+ );
+ let x = 0.5;
+ let y = 0.5;
+ if (vals.length === 1) {
+ if (vals[0].axis === "y") y = vals[0].v;
+ else x = vals[0].v;
+ } else if (vals.length >= 2) {
+ if (vals[0].axis === "y" || vals[1].axis === "x") {
+ y = vals[0].v;
+ x = vals[1].v;
+ } else {
+ x = vals[0].v;
+ y = vals[1].v;
+ }
+ }
+ return [x, y];
+}
+
+/** Parses one `color [pos] [pos]` stop list entry; positions in 0..1 (or px later). */
+function parseStop(s: string): { color: Rgba; positions: number[] } | null {
+ // The color is a leading function call or word; the rest are positions.
+ const m = /^((?:[a-z]+\([^)]*\))|#[0-9a-f]+|[a-z]+)\s*(.*)$/i.exec(s.trim());
+ if (!m) return null;
+ const color = parseCssColor(m[1]);
+ if (!color) return null;
+ // Percentages anywhere; angles on conic gradients (as a fraction of a turn).
+ const positions: number[] = [];
+ for (const p of m[2].split(/\s+/).filter(Boolean)) {
+ if (p.endsWith("%")) positions.push(parseFloat(p) / 100);
+ else if (p === "0")
+ positions.push(0); // CSS allows a unitless zero
+ else if (/(deg|turn|rad|grad)$/i.test(p)) {
+ const a = parseAngle(p);
+ if (a !== null) positions.push(a / 360);
+ }
+ }
+ return { color, positions: positions.filter((p) => Number.isFinite(p)) };
+}
+
+/** CSS stop fixup: clamp ascending, spread unpositioned stops evenly. */
+function resolveStops(
+ raw: Array<{ color: Rgba; position?: number }>,
+): ParsedGradientStop[] {
+ const n = raw.length;
+ if (n === 0) return [];
+ const pos: (number | undefined)[] = raw.map((r) => r.position);
+ if (pos[0] === undefined) pos[0] = 0;
+ if (pos[n - 1] === undefined) pos[n - 1] = 1;
+ let maxSoFar = -Infinity;
+ for (let i = 0; i < n; i++) {
+ const p = pos[i];
+ if (p !== undefined) {
+ pos[i] = Math.max(p, maxSoFar);
+ maxSoFar = pos[i] as number;
+ }
+ }
+ for (let i = 1; i < n; i++) {
+ if (pos[i] !== undefined) continue;
+ let j = i;
+ while (pos[j] === undefined) j++;
+ const a = pos[i - 1] as number;
+ const b = pos[j] as number;
+ for (let k = i; k < j; k++)
+ pos[k] = a + ((b - a) * (k - i + 1)) / (j - i + 1);
+ }
+ return raw.map((r, i) => ({ offset: pos[i] as number, color: r.color }));
+}
+
+const cache = new Map<string, ParsedGradient | null>();
+
+/** Parses a CSS gradient string; `null` when it is not one this engine draws. */
+export function parseCssGradient(input: string): ParsedGradient | null {
+ const hit = cache.get(input);
+ if (hit !== undefined) return hit;
+ const result = parseUncached(input);
+ if (cache.size > 512) cache.clear();
+ cache.set(input, result);
+ return result;
+}
+
+function parseUncached(input: string): ParsedGradient | null {
+ const m = GRADIENT_RE.exec(input);
+ if (!m) return null;
+ const type = m[1].toLowerCase() as GradientType;
+ const parts = splitTopLevel(m[2]);
+ const g: ParsedGradient = {
+ type,
+ angle: 180,
+ center: [0.5, 0.5],
+ shape: "ellipse",
+ from: 0,
+ stops: [],
+ };
+ const first = parts[0]?.toLowerCase().trim() ?? "";
+ let consumedFirst = false;
+ if (type === "linear") {
+ if (first.startsWith("to ")) {
+ const side = first.slice(3).trim().replace(/\s+/g, " ");
+ if (side in SIDE_ANGLES) {
+ g.angle = SIDE_ANGLES[side];
+ consumedFirst = true;
+ }
+ } else {
+ const a = parseAngle(first);
+ if (a !== null) {
+ g.angle = a;
+ consumedFirst = true;
+ }
+ }
+ } else {
+ const tokens = first.split(/\s+/).filter(Boolean);
+ const at = tokens.indexOf("at");
+ const before = at >= 0 ? tokens.slice(0, at) : tokens;
+ const isConfig =
+ at >= 0 ||
+ before.some(
+ (t) =>
+ t === "circle" ||
+ t === "ellipse" ||
+ t === "from" ||
+ /^closest|^farthest/.test(t) ||
+ (type === "radial" && /px$/.test(t)),
+ );
+ if (isConfig) {
+ consumedFirst = true;
+ if (at >= 0) g.center = parsePosition(tokens.slice(at + 1));
+ if (type === "radial") {
+ if (before.includes("circle")) g.shape = "circle";
+ const px = before
+ .filter((t) => /px$/.test(t))
+ .map((t) => parseFloat(t));
+ if (px.length === 1) {
+ g.shape = "circle";
+ g.radius = [px[0], px[0]];
+ } else if (px.length >= 2) g.radius = [px[0], px[1]];
+ } else {
+ const f = before.indexOf("from");
+ if (f >= 0 && before[f + 1]) g.from = parseAngle(before[f + 1]) ?? 0;
+ }
+ }
+ }
+ const raw: Array<{ color: Rgba; position?: number }> = [];
+ for (const p of consumedFirst ? parts.slice(1) : parts) {
+ const stop = parseStop(p);
+ if (!stop) return null;
+ if (stop.positions.length === 0) raw.push({ color: stop.color });
+ for (const pos of stop.positions)
+ raw.push({ color: stop.color, position: pos });
+ }
+ if (raw.length === 0) return null;
+ if (raw.length === 1) raw.push({ ...raw[0] });
+ g.stops = resolveStops(raw);
+ return g;
+}
+
+/** Up to MAX_GRADIENT_STOPS stops: longer lists are resampled evenly. */
+export function limitGradientStops(
+ stops: ParsedGradientStop[],
+): ParsedGradientStop[] {
+ if (stops.length <= MAX_GRADIENT_STOPS) return stops;
+ const out: ParsedGradientStop[] = [];
+ const lo = stops[0].offset;
+ const hi = stops[stops.length - 1].offset;
+ for (let i = 0; i < MAX_GRADIENT_STOPS; i++) {
+ const t = lo + ((hi - lo) * i) / (MAX_GRADIENT_STOPS - 1);
+ let k = 1;
+ while (k < stops.length - 1 && stops[k].offset < t) k++;
+ const a = stops[k - 1];
+ const b = stops[k];
+ const span = b.offset - a.offset;
+ const f = span > 0 ? Math.min(1, Math.max(0, (t - a.offset) / span)) : 1;
+ out.push({
+ offset: t,
+ color: a.color.map((v, j) => v + (b.color[j] - v) * f) as Rgba,
+ });
+ }
+ return out;
+}
+
+/**
+ * Stops as OKLab + alpha for the GPU. A transparent stop borrows its
+ * neighbor's color, so fading to `transparent` keeps the hue instead of
+ * passing through grey (CSS premultiplied interpolation does the same).
+ */
+export function gradientStopsToOklab(
+ stops: ParsedGradientStop[],
+): Array<[number, number, number, number]> {
+ const lab = stops.map((s) => rgbToOklab(s.color[0], s.color[1], s.color[2]));
+ return stops.map((s, i) => {
+ let src = lab[i];
+ if (s.color[3] < 1e-4) {
+ const prev = stops
+ .slice(0, i)
+ .reverse()
+ .findIndex((p) => p.color[3] > 1e-4);
+ const next = stops.slice(i + 1).findIndex((p) => p.color[3] > 1e-4);
+ const pi = prev >= 0 ? i - 1 - prev : -1;
+ const ni = next >= 0 ? i + 1 + next : -1;
+ if (pi >= 0 && ni >= 0) {
+ const a = lab[pi];
+ const b = lab[ni];
+ src = [(a[0] + b[0]) / 2, (a[1] + b[1]) / 2, (a[2] + b[2]) / 2];
+ } else if (pi >= 0) src = lab[pi];
+ else if (ni >= 0) src = lab[ni];
+ }
+ return [src[0], src[1], src[2], s.color[3]];
+ });
+}
+
+type StopInput = string | { color: string; offset: number };
+
+function stopList(stops: readonly StopInput[]): string {
+ return stops
+ .map((s) =>
+ typeof s === "string"
+ ? s
+ : `${s.color} ${Math.round(s.offset * 10000) / 100}%`,
+ )
+ .join(", ");
+}
+
+/**
+ * Writes CSS gradient strings for `background` / `fill`:
+ *
+ * ```ts
+ * Layer.box({ background: Gradient.linear(["#ff5a1f", "#7c3aed"], 135) })
+ * Layer.shape("circle", { fill: Gradient.radial(["#fff", "transparent"]) })
+ * Gradient.conic(["red", "yellow", "lime", "aqua", "blue", "magenta", "red"])
+ * ```
+ * Plain strings work too: `"linear-gradient(to right, #000, #fff 80%)"`.
+ */
+export const Gradient = {
+ /** `angle` in degrees: 0 = bottom→top, 90 = left→right, 180 = top→bottom (default). */
+ linear(stops: readonly StopInput[], angle = 180): string {
+ return `linear-gradient(${angle}deg, ${stopList(stops)})`;
+ },
+ /** Ellipse to the farthest corner, centered unless `at` ([0..1, 0..1]) says otherwise. */
+ radial(
+ stops: readonly StopInput[],
+ options: { at?: [number, number]; circle?: boolean; radius?: number } = {},
+ ): string {
+ const at = options.at ?? [0.5, 0.5];
+ const shape =
+ options.radius !== undefined
+ ? `${options.radius}px`
+ : options.circle
+ ? "circle"
+ : "ellipse";
+ return `radial-gradient(${shape} at ${at[0] * 100}% ${at[1] * 100}%, ${stopList(stops)})`;
+ },
+ /** A sweep around `at`, starting at `from` degrees (0 = top, clockwise). */
+ conic(
+ stops: readonly StopInput[],
+ options: { from?: number; at?: [number, number] } = {},
+ ): string {
+ const at = options.at ?? [0.5, 0.5];
+ return `conic-gradient(from ${options.from ?? 0}deg at ${at[0] * 100}% ${at[1] * 100}%, ${stopList(stops)})`;
+ },
+ parse: parseCssGradient,
+ is: isCssGradient,
+} as const;
diff --git a/packages/framefields/src/font-manager.test.ts b/packages/framefields/src/font-manager.test.ts
index c95b9b60..7939e4ff 100644
--- a/packages/framefields/src/font-manager.test.ts
+++ b/packages/framefields/src/font-manager.test.ts
@@ -157,3 +157,56 @@ describe("FontManager Singleton & Typography Binding", () => {
},
);
});
+
+describe.skipIf(!hasInter)("FontManager.measure", () => {
+ beforeAll(async () => {
+ await FontManager.register({
+ family: "Inter",
+ source: path.resolve(import.meta.dirname, "../../..", INTER),
+ });
+ });
+
+ it("measures width, lines and per-glyph carets with the real font", () => {
+ const m = FontManager.measure("Hello", {
+ fontFamily: "Inter",
+ fontSize: 100,
+ });
+ expect(m.lines).toBe(1);
+ expect(m.width).toBeGreaterThan(150);
+ expect(m.width).toBeLessThan(350);
+ expect(m.glyphs.map((g) => g.char).join("")).toBe("Hello");
+ expect(m.caretX(0)).toBe(m.glyphs[0].x);
+ // Carets advance monotonically and the last one sits at the ink edge.
+ for (let i = 1; i <= 5; i++) {
+ expect(m.caretX(i)).toBeGreaterThan(m.caretX(i - 1));
+ }
+ expect(Math.abs(m.caretX(5) - m.width)).toBeLessThan(2);
+ // "l" is narrower than "H" in Inter.
+ expect(m.glyphs[2].width).toBeLessThan(m.glyphs[0].width);
+ });
+
+ it("wraps to the given width and scales with font size", () => {
+ const small = FontManager.measure("Hello world", {
+ fontFamily: "Inter",
+ fontSize: 50,
+ });
+ const big = FontManager.measure("Hello world", {
+ fontFamily: "Inter",
+ fontSize: 100,
+ });
+ expect(big.width / small.width).toBeCloseTo(2, 1);
+ const wrapped = FontManager.measure("Hello world", {
+ fontFamily: "Inter",
+ fontSize: 100,
+ width: big.width * 0.6,
+ });
+ expect(wrapped.lines).toBe(2);
+ expect(wrapped.glyphs.find((g) => g.char === "w")?.line).toBe(1);
+ });
+
+ it("throws a clear error for unregistered fonts", () => {
+ expect(() => FontManager.measure("x", { fontFamily: "Nope" })).toThrow(
+ /not registered/,
+ );
+ });
+});
diff --git a/packages/framefields/src/fonts/index.ts b/packages/framefields/src/fonts/index.ts
index 11c60d00..39a4fcd9 100644
--- a/packages/framefields/src/fonts/index.ts
+++ b/packages/framefields/src/fonts/index.ts
@@ -1,5 +1,8 @@
export {
FontManager,
+ type MeasuredGlyph,
+ type MeasureTextOptions,
type RegisteredFont,
type RegisterFontOptions,
+ type TextMetrics,
} from "./manager.js";
diff --git a/packages/framefields/src/fonts/manager.ts b/packages/framefields/src/fonts/manager.ts
index 78c17ad7..44605a92 100644
--- a/packages/framefields/src/fonts/manager.ts
+++ b/packages/framefields/src/fonts/manager.ts
@@ -3,6 +3,7 @@ import path from "node:path";
import {
registerHeadlessFont,
SlugFontCache,
+ SlugGeometry,
} from "@framefields/webgpu-renderers";
import * as fontkit from "fontkit";
@@ -160,6 +161,49 @@ async function resolveSourceBuffer(
/**
* Central singleton manager for font registration, TTF validation, and compositor typography binding.
*/
+export interface MeasureTextOptions {
+ fontFamily: string;
+ fontSize?: number;
+ fontWeight?: string | number;
+ /** Extra px between characters, as on `Layer.text`. */
+ letterSpacing?: number;
+ /** Multiplier of fontSize when < 10, else px. Default 1.2. */
+ lineHeight?: number;
+ /** Wrap width in px; omit for one line per `\n`. */
+ width?: number;
+ align?: "left" | "center" | "right";
+}
+
+export interface MeasuredGlyph {
+ char: string;
+ /** Index into the string (code units), matching `text.slice`. */
+ index: number;
+ /** Left edge of the glyph, relative to the text box. */
+ x: number;
+ /** Baseline-relative top of the line box. */
+ y: number;
+ /** Advance width in px. */
+ width: number;
+ line: number;
+}
+
+export interface TextMetrics {
+ width: number;
+ height: number;
+ lines: number;
+ /** Line box height in px. */
+ lineHeight: number;
+ glyphs: MeasuredGlyph[];
+ /**
+ * X of a caret placed after `count` characters (0 = before the first),
+ * relative to the text box. Pair with a `.keys("text", …)` reveal to make
+ * a cursor follow typing exactly.
+ */
+ caretX(count: number): number;
+ /** Width of the first `count` characters (same as `caretX` on one line). */
+ widthOf(count: number): number;
+}
+
export class FontManager {
private static instance: FontManager | null = null;
private readonly fonts = new Map<string, RegisteredFont>();
@@ -202,6 +246,80 @@ export class FontManager {
SlugFontCache.setEmojiFont(buffer, filePath);
}
+ /**
+ * Measures text with the real font, exactly as `Layer.text` lays it out:
+ * box size, line count, per-glyph positions and caret offsets. Use it to
+ * size containers, place cursors, underline a word or center by ink
+ * instead of guessing advances. The family must be registered first.
+ *
+ * ```ts
+ * const m = FontManager.measure("Hello", { fontFamily: "Inter", fontSize: 60 });
+ * m.width; m.caretX(3); m.glyphs[1].x
+ * ```
+ */
+ public static measure(
+ text: string,
+ options: MeasureTextOptions,
+ ): TextMetrics {
+ const fontSize = options.fontSize ?? 48;
+ const font = SlugFontCache.getFont(
+ options.fontFamily,
+ options.fontWeight,
+ null,
+ );
+ if (!font) {
+ throw new Error(
+ `FontManager.measure: font "${options.fontFamily}" is not registered. ` +
+ "Call `await FontManager.register(...)` first.",
+ );
+ }
+ const lineHeight =
+ options.lineHeight !== undefined && options.lineHeight < 10
+ ? options.lineHeight * fontSize
+ : (options.lineHeight ?? fontSize * 1.2);
+ const letterSpacing = options.letterSpacing ?? 0;
+ const { width, height, layout } = SlugGeometry.measure(
+ text,
+ font,
+ fontSize,
+ letterSpacing,
+ lineHeight,
+ options.width,
+ );
+ const scale = fontSize / font.unitsPerEm;
+ // Map glyphs back to string indices (layout counts graphemes).
+ const glyphs: MeasuredGlyph[] = [];
+ let cursor = 0;
+ for (const g of layout.glyphs) {
+ const at = text.indexOf(g.char, cursor);
+ const index = at >= 0 ? at : cursor;
+ cursor = index + g.char.length;
+ glyphs.push({
+ char: g.char,
+ index,
+ x: g.x,
+ y: g.y,
+ width: g.cp.advanceWidth * (g.fontScale ?? scale) + letterSpacing,
+ line: g.lineIndex,
+ });
+ }
+ const caretX = (count: number): number => {
+ if (count <= 0 || glyphs.length === 0) return glyphs[0]?.x ?? 0;
+ const before = glyphs.filter((g) => g.index < count);
+ const last = before[before.length - 1];
+ return last ? last.x + last.width - letterSpacing : 0;
+ };
+ return {
+ width,
+ height,
+ lines: layout.linesCount,
+ lineHeight,
+ glyphs,
+ caretX,
+ widthOf: (count: number) => caretX(count) - (glyphs[0]?.x ?? 0),
+ };
+ }
+
/**
* Retrieves registered font metadata by family name.
*/
diff --git a/packages/framefields/src/gradient-render.test.ts b/packages/framefields/src/gradient-render.test.ts
new file mode 100644
index 00000000..217c6396
--- /dev/null
+++ b/packages/framefields/src/gradient-render.test.ts
@@ -0,0 +1,116 @@
+import { Canvas, loadImage } from "skia-canvas";
+import { describe, expect, it } from "vitest";
+import { Composition, Gradient, Layer } from "./index.js";
+
+async function decode(png: Buffer) {
+ const img = await loadImage(png);
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ const { data } = ctx.getImageData(0, 0, img.width, img.height);
+ return (x: number, y: number) => {
+ const i = (Math.round(y) * img.width + Math.round(x)) * 4;
+ return { r: data[i]!, g: data[i + 1]!, b: data[i + 2]!, a: data[i + 3]! };
+ };
+}
+
+function comp() {
+ return new Composition({
+ width: 400,
+ height: 200,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#000000",
+ });
+}
+
+describe("CSS gradients as backgrounds and fills", () => {
+ it("paints a multi-stop linear gradient box background", async () => {
+ const c = comp();
+ c.add(
+ Layer.box({
+ id: "bg",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 400,
+ height: 200,
+ background: "linear-gradient(to right, #ff0000, #00ff00 50%, #0000ff)",
+ }),
+ );
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ const left = at(4, 100);
+ const mid = at(200, 100);
+ const right = at(396, 100);
+ expect(left.r).toBeGreaterThan(230);
+ expect(mid.g).toBeGreaterThan(230);
+ expect(mid.r).toBeLessThan(40);
+ expect(right.b).toBeGreaterThan(230);
+ // Perceptual blend: halfway red→green stays bright, not a muddy dark olive.
+ const q = at(100, 100);
+ expect(q.r + q.g).toBeGreaterThan(300);
+ });
+
+ it("keeps rounded corners on gradient backgrounds", async () => {
+ const c = comp();
+ c.add(
+ Layer.box({
+ id: "card",
+ position: "absolute",
+ x: 100,
+ y: 20,
+ width: 200,
+ height: 160,
+ borderRadius: 60,
+ background: Gradient.linear(["#ffffff", "#ffffff"]),
+ }),
+ );
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ expect(at(102, 22).r).toBeLessThan(20); // outside the corner
+ expect(at(200, 100).r).toBeGreaterThan(240);
+ });
+
+ it("fills a shape with a radial gradient fading to transparent", async () => {
+ const c = comp();
+ c.add(
+ Layer.shape("rect", {
+ id: "glow",
+ position: "absolute",
+ x: 100,
+ y: 0,
+ width: 200,
+ height: 200,
+ fill: Gradient.radial(["#ffffff", "transparent"], { circle: true }),
+ }),
+ );
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ expect(at(200, 100).r).toBeGreaterThan(220);
+ expect(at(103, 3).r).toBeLessThan(30);
+ expect(at(150, 100).r).toBeLessThan(at(200, 100).r);
+ });
+
+ it("sweeps a conic gradient clockwise from the top", async () => {
+ const c = comp();
+ c.add(
+ Layer.shape("rect", {
+ id: "wheel",
+ position: "absolute",
+ x: 100,
+ y: 0,
+ width: 200,
+ height: 200,
+ fill: "conic-gradient(#ff0000 0 25%, #00ff00 0 50%, #0000ff 0 75%, #ffffff 0)",
+ }),
+ );
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ const tr = at(260, 40); // top-right quadrant: first quarter turn
+ const br = at(260, 160);
+ const bl = at(140, 160);
+ const tl = at(140, 40);
+ expect(tr.r).toBeGreaterThan(200);
+ expect(tr.g).toBeLessThan(60);
+ expect(br.g).toBeGreaterThan(200);
+ expect(bl.b).toBeGreaterThan(200);
+ expect(tl.r + tl.g + tl.b).toBeGreaterThan(700);
+ });
+});
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 266ae0ee..85c357b0 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -17,8 +17,11 @@ export {
import {
FontManager,
+ type MeasuredGlyph,
+ type MeasureTextOptions,
type RegisteredFont,
type RegisterFontOptions,
+ type TextMetrics,
} from "./fonts/index.js";
import { autoId } from "./ids.js";
import { normalizeTextSpans, type TextSpan } from "./rich-text.js";
@@ -52,7 +55,14 @@ export {
type TextSpanItem,
type TextSpanMark,
} from "./rich-text.js";
-export { FontManager, type RegisteredFont, type RegisterFontOptions };
+export {
+ FontManager,
+ type MeasuredGlyph,
+ type MeasureTextOptions,
+ type RegisteredFont,
+ type RegisterFontOptions,
+ type TextMetrics,
+};
export {
CannyComputePipeline,
@@ -122,12 +132,16 @@ import {
type CompositorProgramConfig,
type CompositorToProgramOptions,
compositorToProgram,
+ type EnvironmentPreset,
type FlexNode,
LayerAnimation,
type LayoutNode,
type LightNode,
+ lintComposition,
+ type MatteMode,
type MediaNode,
type Model3DNode,
+ mergeAnimationTracks,
type ShapeNode,
type TextNode,
} from "@framefields/compositions/program";
@@ -142,6 +156,7 @@ import {
type MediaSourceInput,
type MeshAudioDeformConfig,
ProgrammaticSignal,
+ type RenderDiagnostic,
type Signal,
type VirtualMediaData,
} from "@framefields/core";
@@ -239,6 +254,12 @@ import {
type Plane3DOptions,
type Prism3DOptions,
} from "./shapes3d.js";
+import {
+ type SceneSequenceOptions,
+ type SequenceEntry,
+ type SequenceNode,
+ sequence,
+} from "./transitions.js";
export interface SectionOptions {
readonly id?: string;
@@ -406,6 +427,8 @@ export {
AudioSpectrumTypographySchema,
applyStaggerToChildren,
type BlockNode,
+ type BloomConfig,
+ BloomConfigSchema,
type BoxNode,
BoxNodeSchema,
buildPathLUT,
@@ -445,6 +468,10 @@ export {
type EaseRef,
type EnhancedTextNode,
EnhancedTextNodeSchema,
+ type EnvironmentFog,
+ EnvironmentFogSchema,
+ type EnvironmentPreset,
+ EnvironmentPresetSchema,
type ExpressionEvaluationContext,
type ExpressionSelector,
ExpressionSelectorSchema,
@@ -473,6 +500,7 @@ export {
generatePathSegments,
getScramblePool,
getShuffledOrder,
+ hoistMattes,
type Keyframe,
type KeyframeTuple,
type KineticMarqueeConfig,
@@ -488,12 +516,16 @@ export {
LineSpacingDynamicsSchema,
type LongShadowConfig,
LongShadowConfigSchema,
+ lintComposition,
type MaterialType,
MaterialTypeSchema,
+ type MatteMode,
+ MatteModeSchema,
type MediaNode,
MediaNodeSchema,
type Model3DNode,
Model3DNodeSchema,
+ mergeAnimationTracks,
type ParticleDissolveConfig,
ParticleDissolveConfigSchema,
type PathChainLUT,
@@ -557,6 +589,8 @@ export {
TextSpanSchema,
type TextWigglySelector,
TextWigglySelectorSchema,
+ type ToneMapping,
+ ToneMappingSchema,
type TrackSource,
type TypewriterAnimator,
TypewriterAnimatorSchema,
@@ -745,6 +779,11 @@ export {
Vision,
type VisionProps,
} from "./effects/index.js";
+export {
+ Lighting,
+ type LightingRig,
+ type LightingRigOptions,
+} from "./lighting.js";
export {
type Carousel3DOptions,
type Cube3DOptions,
@@ -760,8 +799,17 @@ export {
type RoundedBox3DOptions,
type Shape3DContainer,
type Sphere3DOptions,
+ type Surface3DOptions,
type Text3DOptions,
} from "./shapes3d.js";
+export {
+ type SceneSequenceOptions,
+ type SequenceEntry,
+ type SequenceNode,
+ Transition,
+ type TransitionDirection,
+ type TransitionSpec,
+} from "./transitions.js";
export interface CompositionOptions {
width: number;
@@ -1776,8 +1824,39 @@ export class Composition {
: outputPathOrOptions;
const renderer = opts.renderer ?? new HeadlessMediaRenderer();
const result = await renderer.renderVideo(this.toVirtualMedia(), opts);
+ if (result.qa) {
+ // Fold the document checks into the output checks.
+ const lint = this.lint();
+ const issues = [
+ ...result.qa.issues,
+ ...lint.map((d) => ({
+ severity: d.severity,
+ code: d.code,
+ message: d.message,
+ ...(d.layerId !== undefined && { layerId: d.layerId }),
+ })),
+ ];
+ result.qa = {
+ ...result.qa,
+ issues,
+ // Lint warnings advise; only a lint error fails the render.
+ passed:
+ result.qa.passed && !lint.some((d) => d.severity === "error"),
+ };
+ }
return deliverToOutputPath(result, opts.outputPath);
}
+
+ /**
+ * Checks the document for things that render without error but not as
+ * intended: unreadable colors or gradients (drawn transparent),
+ * low-contrast text, layers parked off-canvas, shadows on lights that
+ * cannot cast them, physical materials without an environment, tracks that
+ * never change. `renderVideo({ qa })` includes these automatically.
+ */
+ public lint(): RenderDiagnostic[] {
+ return lintComposition(this.toSpec());
+ }
}
/**
@@ -1785,7 +1864,6 @@ export class Composition {
*/
export type AnimatableNode<T extends LayoutNode> = T & {
- mask?: unknown;
glass?: Record<string, unknown>;
audioDeform?: MeshAudioDeformConfig | Record<string, unknown>;
crop?: CropProps | Crop;
@@ -1827,7 +1905,16 @@ export type AnimatableNode<T extends LayoutNode> = T & {
},
options?: PinToObjectOptions,
): AnimatableNode<T>;
- withMask(mask: unknown): AnimatableNode<T>;
+ /**
+ * Track matte: `matte` (any layer) masks this one by its alpha or
+ * brightness. It is laid out like a sibling (same parent) and timed with
+ * this node; with no x/y/width/height it covers this node's box. Animate
+ * the matte for wipes and reveals: text through a sliding box, video
+ * inside type, an iris circle.
+ */
+ withMatte(matte: LayoutNode, mode?: MatteMode): AnimatableNode<T>;
+ /** Alias of {@link withMatte}. */
+ withMask(mask: LayoutNode, mode?: MatteMode): AnimatableNode<T>;
withPbrGlass(options?: Record<string, unknown>): AnimatableNode<T>;
deformWithAudio(
config?: MeshAudioDeformConfig | Record<string, unknown>,
@@ -1861,24 +1948,21 @@ function withAnimation<T extends LayoutNode>(node: T): AnimatableNode<T> {
| LayerAnimation[]
)[]
) => {
- const flatAnims = anims.flat();
- if (
- flatAnims.length > 1 ||
- (flatAnims.length === 1 && Array.isArray(anims[0]))
- ) {
- const allTracks = flatAnims.flatMap((a) =>
+ // Every call layers onto what is already there: keyframe tracks on
+ // the same prop merge (a later key wins on a shared frame).
+ const incoming = anims
+ .flat()
+ .flatMap((a) =>
a instanceof LayerAnimation
? a.tracks
: ((a as { tracks?: AnimationTrack[] }).tracks ?? []),
);
- target.animation = { tracks: allTracks };
- } else if (flatAnims.length === 1) {
- const anim = flatAnims[0];
- target.animation =
- anim instanceof LayerAnimation
- ? anim.toSpec()
- : (anim as { tracks: AnimationTrack[] });
- }
+ const existing =
+ (target.animation as { tracks?: AnimationTrack[] } | undefined)
+ ?.tracks ?? [];
+ target.animation = {
+ tracks: mergeAnimationTracks([...existing, ...incoming]),
+ };
return target;
},
enumerable: false,
@@ -1944,15 +2028,18 @@ function withAnimation<T extends LayoutNode>(node: T): AnimatableNode<T> {
configurable: true,
});
- Object.defineProperty(target, "withMask", {
- value: (mask: unknown) => {
- target.mask = mask;
- return target;
- },
- enumerable: false,
- writable: true,
- configurable: true,
- });
+ const withMatte = (matte: LayoutNode, mode?: MatteMode) => {
+ target.matte = { node: matte, ...(mode && { mode }) };
+ return target;
+ };
+ for (const name of ["withMatte", "withMask"]) {
+ Object.defineProperty(target, name, {
+ value: withMatte,
+ enumerable: false,
+ writable: true,
+ configurable: true,
+ });
+ }
Object.defineProperty(target, "withPbrGlass", {
value: (options?: Record<string, unknown>) => {
@@ -2265,14 +2352,19 @@ export const Layer = {
| "star"
| "arrow"
| "path" = "rect",
- options: Partial<ShapeNode> = {},
+ options: Partial<ShapeNode> & {
+ /** Alias of `fillColor`: a color or a CSS gradient string. */
+ fill?: string;
+ } = {},
): AnimatableNode<ShapeNode> =>
withAnimation({
id: options.id ?? autoId("shape"),
kind: "shape",
shapeType: options.shapeType ?? shapeType,
- fillColor: options.fillColor ?? "#3b82f6",
...options,
+ // `fill` and `fillColor` are the same thing; either may hold a
+ // color or a CSS gradient.
+ fillColor: options.fillColor ?? options.fill ?? "#3b82f6",
}),
flex: (options: Partial<FlexNode> = {}): AnimatableNode<FlexNode> =>
withAnimation({
@@ -2540,6 +2632,21 @@ export const Layer = {
intensity: number = 0.5,
options: Partial<LightNode> = {},
): AnimatableNode<LightNode> => Light.ambient(color, intensity, options),
+ /**
+ * Scenes back to back with transitions between them; see `Transition`.
+ * `comp.add(Layer.sequence([{ scene: hero(), duration: 90 },
+ * { scene: next(), duration: 120, transition: Transition.push("left") }]))`.
+ * The result's `totalFrames` is the sequence length.
+ */
+ sequence: (
+ entries: SequenceEntry[],
+ options?: SceneSequenceOptions,
+ ): SequenceNode => sequence(entries, options),
+ /** See {@link Light.environment}: `Layer.environment("studio", { exposure: 0.3 })`. */
+ environment: (
+ presetOrOptions: EnvironmentPreset | EnvironmentOptions = "studio",
+ options: EnvironmentOptions = {},
+ ): AnimatableNode<LightNode> => Light.environment(presetOrOptions, options),
directionalLight: (
options: Partial<LightNode> = {},
): AnimatableNode<LightNode> => Light.directional(options),
@@ -2654,7 +2761,51 @@ export const Layer = {
}),
};
+/** World settings for `Layer.environment` (see `LightNode` for every field). */
+export type EnvironmentOptions = Partial<
+ Pick<
+ LightNode,
+ | "id"
+ | "preset"
+ | "top"
+ | "horizon"
+ | "bottom"
+ | "intensity"
+ | "softboxes"
+ | "rotation"
+ | "exposure"
+ | "toneMapping"
+ | "fog"
+ | "bloom"
+ | "startFrame"
+ | "durationFrames"
+ >
+>;
+
export const Light = {
+ /**
+ * The scene's world: a procedural studio environment that lights and
+ * reflects off every surface, plus exposure, tone mapping and fog. Adding
+ * one switches the scene to physically based shading (GGX, real
+ * `roughness` / `metallic`, `emissive` that blooms past white).
+ */
+ environment: (
+ presetOrOptions: EnvironmentPreset | EnvironmentOptions = "studio",
+ options: EnvironmentOptions = {},
+ ): AnimatableNode<LightNode> => {
+ const opts =
+ typeof presetOrOptions === "string"
+ ? { preset: presetOrOptions, ...options }
+ : { ...presetOrOptions, ...options };
+ return withAnimation({
+ kind: "light",
+ lightType: "environment",
+ preset: "studio",
+ intensity: 1,
+ ...opts,
+ id: opts.id ?? autoId("environment"),
+ });
+ },
ambient: (
color: string = "#ffffff",
intensity: number = 0.5,
diff --git a/packages/framefields/src/lighting.ts b/packages/framefields/src/lighting.ts
new file mode 100644
index 00000000..66d97ddf
--- /dev/null
+++ b/packages/framefields/src/lighting.ts
@@ -0,0 +1,165 @@
+import type {
+ EnvironmentPreset,
+ LightNode,
+ ToneMapping,
+} from "@framefields/compositions/program";
+import { autoId } from "./ids.js";
+import {
+ type AnimatableNode,
+ type EnvironmentOptions,
+ Light,
+} from "./index.js";
+
+export interface LightingRigOptions {
+ /** What the lights aim at (world px). Default: the canvas center at z 0. */
+ target?: [number, number, number];
+ /** How far the lights sit from the target. Default 1600. */
+ distance?: number;
+ /** Turns the whole rig (lights and environment) around the vertical axis, degrees. */
+ rotation?: number;
+ /** Scales every light. Default 1. */
+ intensity?: number;
+ /** Exposure in stops. Default 0. */
+ exposure?: number;
+ toneMapping?: ToneMapping;
+ /** Key light casts a soft shadow. Default true. */
+ shadows?: boolean;
+ /** Shadow edge blur in texels. Default 3. */
+ shadowSoftness?: number;
+ /** Fog fading into the background, e.g. `{ color: BG, start: 2500, end: 6000 }`. */
+ fog?: EnvironmentOptions["fog"];
+ /** Glow around emissive surfaces and highlights: `true` or settings. */
+ bloom?: EnvironmentOptions["bloom"];
+ /** Prefix for the rig's node ids. */
+ id?: string;
+}
+
+interface RigSpec {
+ environment: EnvironmentPreset;
+ envIntensity: number;
+ /** Light directions relative to the target, before `rotation`; y is down. */
+ key: { dir: [number, number, number]; color: string; intensity: number };
+ rim?: { dir: [number, number, number]; color: string; intensity: number };
+ fill?: { dir: [number, number, number]; color: string; intensity: number };
+}
+
+const RIGS = {
+ /** Product photography: soft overhead key with a shadow, cool rim, studio reflections. */
+ studio: {
+ environment: "studio",
+ envIntensity: 1,
+ key: { dir: [-0.55, -1, -0.7], color: "#fff6ec", intensity: 1.1 },
+ rim: { dir: [0.9, -0.5, 0.9], color: "#dbe8ff", intensity: 0.9 },
+ },
+ /** Bright, airy, low contrast: lifestyle and UI-on-device shots. */
+ soft: {
+ environment: "soft",
+ envIntensity: 1,
+ key: { dir: [-0.4, -1, -0.8], color: "#ffffff", intensity: 1.0 },
+ },
+ /** Low warm sun, long shadows, cool sky fill. */
+ sunset: {
+ environment: "sunset",
+ envIntensity: 0.9,
+ key: { dir: [-1, -0.35, -0.3], color: "#ffb27a", intensity: 2.2 },
+ fill: { dir: [0.8, -0.6, -0.4], color: "#7f9cff", intensity: 0.35 },
+ },
+ /** Moody: dark world, hard key, strong rim. Pair with emissive accents. */
+ dramatic: {
+ environment: "night",
+ envIntensity: 0.8,
+ key: { dir: [-0.8, -1, -0.35], color: "#ffffff", intensity: 2.4 },
+ rim: { dir: [0.7, -0.4, 1], color: "#8fb4ff", intensity: 1.8 },
+ },
+} satisfies Record<string, RigSpec>;
+
+export type LightingRig = keyof typeof RIGS;
+
+function rotateY(
+ [x, y, z]: [number, number, number],
+ degrees: number,
+): [number, number, number] {
+ const r = (degrees * Math.PI) / 180;
+ const c = Math.cos(r);
+ const s = Math.sin(r);
+ return [c * x + s * z, y, -s * x + c * z];
+}
+
+function aim(
+ dir: [number, number, number],
+ target: [number, number, number],
+ distance: number,
+ rotation: number,
+): Pick<LightNode, "x" | "y" | "z" | "targetX" | "targetY" | "targetZ"> {
+ const [dx, dy, dz] = rotateY(dir, rotation);
+ const len = Math.hypot(dx, dy, dz) || 1;
+ return {
+ x: target[0] + (dx / len) * distance,
+ y: target[1] + (dy / len) * distance,
+ z: target[2] + (dz / len) * distance,
+ targetX: target[0],
+ targetY: target[1],
+ targetZ: target[2],
+ };
+}
+
+function rig(
+ name: LightingRig,
+ options: LightingRigOptions,
+): AnimatableNode<LightNode>[] {
+ const spec: RigSpec = RIGS[name];
+ const target = options.target ?? [960, 540, 0];
+ const distance = options.distance ?? 1600;
+ const rotation = options.rotation ?? 0;
+ const k = options.intensity ?? 1;
+ const id = options.id ?? autoId(`lighting-${name}`);
+ const nodes: AnimatableNode<LightNode>[] = [
+ Light.environment(spec.environment, {
+ id: `${id}-environment`,
+ intensity: spec.envIntensity * k,
+ rotation,
+ exposure: options.exposure,
+ toneMapping: options.toneMapping,
+ fog: options.fog,
+ bloom: options.bloom,
+ }),
+ Light.directional({
+ id: `${id}-key`,
+ color: spec.key.color,
+ intensity: spec.key.intensity * k,
+ ...aim(spec.key.dir, target, distance, rotation),
+ castShadow: options.shadows ?? true,
+ shadowSoftness: options.shadowSoftness ?? 3,
+ }),
+ ];
+ for (const [part, light] of [
+ ["rim", spec.rim],
+ ["fill", spec.fill],
+ ] as const) {
+ if (!light) continue;
+ nodes.push(
+ Light.directional({
+ id: `${id}-${part}`,
+ color: light.color,
+ intensity: light.intensity * k,
+ ...aim(light.dir, target, distance, rotation),
+ }),
+ );
+ }
+ return nodes;
+}
+
+/**
+ * Ready-made lighting: an environment (physical shading, tone mapping) plus
+ * placed key / rim / fill lights, the key casting a soft shadow. Add the
+ * returned nodes to the composition: `comp.add(Lighting.studio({ target }))`.
+ *
+ * Tweak from there rather than starting from zero: `intensity`, `exposure`,
+ * `rotation` (turns the rig), `shadowSoftness`, `fog`.
+ */
+export const Lighting = {
+ studio: (options: LightingRigOptions = {}) => rig("studio", options),
+ soft: (options: LightingRigOptions = {}) => rig("soft", options),
+ sunset: (options: LightingRigOptions = {}) => rig("sunset", options),
+ dramatic: (options: LightingRigOptions = {}) => rig("dramatic", options),
+};
diff --git a/packages/framefields/src/material-maps-render.test.ts b/packages/framefields/src/material-maps-render.test.ts
new file mode 100644
index 00000000..61c35890
--- /dev/null
+++ b/packages/framefields/src/material-maps-render.test.ts
@@ -0,0 +1,188 @@
+import fs from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { Canvas, loadImage } from "skia-canvas";
+import { afterAll, describe, expect, it } from "vitest";
+import { Composition, Layer } from "./index.js";
+
+async function decode(png: Buffer) {
+ const img = await loadImage(png);
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ const { data } = ctx.getImageData(0, 0, img.width, img.height);
+ return (x: number, y: number) => {
+ const i = (Math.round(y) * img.width + Math.round(x)) * 4;
+ return 0.2126 * data[i]! + 0.7152 * data[i + 1]! + 0.0722 * data[i + 2]!;
+ };
+}
+
+async function solidPng(rgb: [number, number, number]): Promise<string> {
+ const c = new Canvas(4, 4);
+ const ctx = c.getContext("2d");
+ ctx.fillStyle = `rgb(${rgb.join(",")})`;
+ ctx.fillRect(0, 0, 4, 4);
+ return `data:image/png;base64,${(await c.toBuffer("png")).toString("base64")}`;
+}
+
+const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-maps-"));
+afterAll(() => fs.rm(dir, { recursive: true, force: true }));
+
+/**
+ * A 400Γ—400 quad in the z = 0 plane facing the camera (-z), u along +x and
+ * v down the image (+y), with the given material maps.
+ */
+async function quadGltf(
+ name: string,
+ material: Record<string, unknown>,
+ images: string[],
+): Promise<string> {
+ const pos = new Float32Array([
+ -200, -200, 0, 200, -200, 0, -200, 200, 0, 200, 200, 0,
+ ]);
+ const nrm = new Float32Array([0, 0, -1, 0, 0, -1, 0, 0, -1, 0, 0, -1]);
+ const uv = new Float32Array([0, 0, 1, 0, 0, 1, 1, 1]);
+ const idx = new Uint16Array([0, 2, 1, 1, 2, 3]);
+ const bin = Buffer.concat(
+ [pos, nrm, uv, idx].map((a) => Buffer.from(a.buffer)),
+ );
+ const doc = {
+ asset: { version: "2.0" },
+ buffers: [
+ {
+ byteLength: bin.length,
+ uri: `data:application/octet-stream;base64,${bin.toString("base64")}`,
+ },
+ ],
+ bufferViews: [
+ { buffer: 0, byteOffset: 0, byteLength: 48 },
+ { buffer: 0, byteOffset: 48, byteLength: 48 },
+ { buffer: 0, byteOffset: 96, byteLength: 32 },
+ { buffer: 0, byteOffset: 128, byteLength: 12 },
+ ],
+ accessors: [
+ {
+ bufferView: 0,
+ componentType: 5126,
+ count: 4,
+ type: "VEC3",
+ min: [-200, -200, 0],
+ max: [200, 200, 0],
+ },
+ { bufferView: 1, componentType: 5126, count: 4, type: "VEC3" },
+ { bufferView: 2, componentType: 5126, count: 4, type: "VEC2" },
+ { bufferView: 3, componentType: 5123, count: 6, type: "SCALAR" },
+ ],
+ images: images.map((uri) => ({ uri })),
+ textures: images.map((_, i) => ({ source: i })),
+ materials: [material],
+ meshes: [
+ {
+ primitives: [
+ {
+ attributes: { POSITION: 0, NORMAL: 1, TEXCOORD_0: 2 },
+ indices: 3,
+ material: 0,
+ },
+ ],
+ },
+ ],
+ nodes: [{ mesh: 0 }],
+ scenes: [{ nodes: [0] }],
+ scene: 0,
+ };
+ const file = path.join(dir, `${name}.gltf`);
+ await fs.writeFile(file, JSON.stringify(doc));
+ return file;
+}
+
+async function brightness(src: string, light: [number, number, number]) {
+ const comp = new Composition({
+ width: 400,
+ height: 400,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#000",
+ });
+ comp.add([
+ Layer.camera({
+ x: 200,
+ y: 200,
+ z: -800,
+ targetX: 200,
+ targetY: 200,
+ targetZ: 0,
+ }),
+ Layer.directionalLight({
+ x: 200 + light[0],
+ y: 200 + light[1],
+ z: light[2],
+ targetX: 200,
+ targetY: 200,
+ targetZ: 0,
+ intensity: 1,
+ }),
+ Layer.gltf(src, {
+ id: "quad",
+ x: 200,
+ y: 200,
+ z: 0,
+ is3D: true,
+ specularIntensity: 0,
+ }),
+ ]);
+ const at = await decode(await comp.renderFrame({ frame: 0 }));
+ return at(200, 200);
+}
+
+describe("glTF material maps", () => {
+ it("tilts the surface along +u with the normal map's red channel", async () => {
+ const src = await quadGltf("red", { normalTexture: { index: 0 } }, [
+ await solidPng([224, 128, 200]),
+ ]);
+ const fromRight = await brightness(src, [700, 0, -700]);
+ const fromLeft = await brightness(src, [-700, 0, -700]);
+ expect(fromRight).toBeGreaterThan(fromLeft + 25);
+ });
+
+ it("tilts toward the top of the image with the green channel (glTF +Y up)", async () => {
+ const src = await quadGltf("green", { normalTexture: { index: 0 } }, [
+ await solidPng([128, 224, 200]),
+ ]);
+ const fromAbove = await brightness(src, [0, -700, -700]); // y is down
+ const fromBelow = await brightness(src, [0, 700, -700]);
+ expect(fromAbove).toBeGreaterThan(fromBelow + 25);
+ });
+
+ it("darkens ambient light with the occlusion map", async () => {
+ const ambientOnly = async (src: string) => {
+ const comp = new Composition({
+ width: 400,
+ height: 400,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#000",
+ });
+ comp.add([
+ Layer.camera({
+ x: 200,
+ y: 200,
+ z: -800,
+ targetX: 200,
+ targetY: 200,
+ targetZ: 0,
+ }),
+ Layer.ambientLight("#ffffff", 1),
+ Layer.gltf(src, { id: "quad", x: 200, y: 200, z: 0, is3D: true }),
+ ]);
+ return (await decode(await comp.renderFrame({ frame: 0 })))(200, 200);
+ };
+ const plain = await ambientOnly(await quadGltf("plain", {}, []));
+ const occluded = await ambientOnly(
+ await quadGltf("ao", { occlusionTexture: { index: 0 } }, [
+ await solidPng([60, 60, 60]),
+ ]),
+ );
+ expect(occluded).toBeLessThan(plain * 0.5);
+ });
+});
diff --git a/packages/framefields/src/matte-render.test.ts b/packages/framefields/src/matte-render.test.ts
new file mode 100644
index 00000000..959f03fe
--- /dev/null
+++ b/packages/framefields/src/matte-render.test.ts
@@ -0,0 +1,233 @@
+import { Canvas, loadImage } from "skia-canvas";
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ hoistMattes,
+ Layer,
+ Layer3D,
+ LayerAnimation,
+} from "./index.js";
+
+async function decode(png: Buffer) {
+ const img = await loadImage(png);
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ const { data } = ctx.getImageData(0, 0, img.width, img.height);
+ return (x: number, y: number) => {
+ const i = (Math.round(y) * img.width + Math.round(x)) * 4;
+ return { r: data[i]!, g: data[i + 1]!, b: data[i + 2]! };
+ };
+}
+
+function comp() {
+ return new Composition({
+ width: 400,
+ height: 200,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#000000",
+ });
+}
+
+const red = (extra: Record<string, unknown> = {}) =>
+ Layer.box({
+ id: "red",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 400,
+ height: 200,
+ background: "#ff0000",
+ ...extra,
+ });
+
+describe("track mattes", () => {
+ it("hoists a matte into a sibling timed and sized like its owner", () => {
+ const [owner, matte] = hoistMattes([
+ {
+ ...red({ startFrame: 10, durationFrames: 20 }),
+ matte: { node: { id: "m", kind: "box" }, mode: "luma" },
+ } as never,
+ ]) as Array<Record<string, unknown>>;
+ expect(owner.matteId).toBe("m");
+ expect(owner.matte).toBeUndefined();
+ expect(matte).toMatchObject({
+ id: "m",
+ matteOf: "red",
+ matteMode: "luma",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 400,
+ height: 200,
+ startFrame: 10,
+ durationFrames: 20,
+ });
+ });
+
+ it("shows the owner only inside an alpha matte", async () => {
+ const c = comp();
+ c.add(
+ red().withMatte(
+ Layer.box({
+ id: "window",
+ position: "absolute",
+ x: 100,
+ y: 50,
+ width: 100,
+ height: 100,
+ background: "#ffffff",
+ }),
+ ),
+ );
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ expect(at(150, 100).r).toBeGreaterThan(240); // inside the window
+ expect(at(20, 20).r).toBeLessThan(10); // outside: hidden
+ expect(at(150, 100).g).toBeLessThan(10); // the matte itself is not drawn
+ });
+
+ it("inverts and uses brightness", async () => {
+ const inverted = comp();
+ inverted.add(
+ red().withMatte(
+ Layer.box({
+ id: "hole",
+ position: "absolute",
+ x: 100,
+ y: 50,
+ width: 100,
+ height: 100,
+ background: "#fff",
+ }),
+ "alpha-inverted",
+ ),
+ );
+ const inv = await decode(await inverted.renderFrame({ frame: 0 }));
+ expect(inv(150, 100).r).toBeLessThan(10);
+ expect(inv(20, 20).r).toBeGreaterThan(240);
+
+ const luma = comp();
+ luma.add(
+ red().withMatte(
+ Layer.box({
+ id: "ramp",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 400,
+ height: 200,
+ background: "linear-gradient(to right, #000, #fff)",
+ }),
+ "luma",
+ ),
+ );
+ const l = await decode(await luma.renderFrame({ frame: 0 }));
+ expect(l(10, 100).r).toBeLessThan(20);
+ expect(l(390, 100).r).toBeGreaterThan(220);
+ });
+
+ it("animates a wipe through the matte, keyed from the owner's start", async () => {
+ const c = comp();
+ c.add(
+ red({ startFrame: 10 }).withMatte(
+ Layer.box({
+ id: "wipe",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 400,
+ height: 200,
+ background: "#fff",
+ }).animate(
+ LayerAnimation.keys("x", [
+ [0, -400],
+ [20, 0],
+ ]),
+ ),
+ ),
+ );
+ const halfway = await decode(await c.renderFrame({ frame: 20 }));
+ expect(halfway(100, 100).r).toBeGreaterThan(240);
+ expect(halfway(300, 100).r).toBeLessThan(10);
+ });
+
+ describe("on 3D layers", () => {
+ const card = (
+ id: string,
+ background: string,
+ z: number,
+ [x, y, width, height]: number[],
+ ) =>
+ Layer.box({ id, position: "absolute", x, y, width, height, z, background });
+ const scene = () => {
+ const c = comp();
+ c.add(
+ Layer.camera({
+ x: 200,
+ y: 100,
+ z: -1000,
+ targetX: 200,
+ targetY: 100,
+ targetZ: 0,
+ }),
+ );
+ return c;
+ };
+
+ it("mattes in screen space and keeps depth against the scene", async () => {
+ const c = scene();
+ c.add([
+ card("back", "#00ff00", 300, [-400, -200, 1200, 600]),
+ card("front", "#0000ff", -100, [40, 40, 120, 120]),
+ card("red", "#ff0000", 50, [-100, -50, 600, 300]).withMatte(
+ Layer.box({
+ id: "left-half",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 200,
+ height: 200,
+ background: "#fff",
+ }),
+ ),
+ ]);
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ expect(at(150, 120).r).toBeGreaterThan(200); // shown inside the matte
+ expect(at(250, 100).g).toBeGreaterThan(200); // matted out: no depth written
+ expect(at(250, 100).r).toBeLessThan(20);
+ expect(at(175, 100).b).toBeGreaterThan(200); // a nearer layer still covers it
+ });
+
+ it("mattes meshes, by brightness", async () => {
+ const c = scene();
+ c.add(
+ Layer3D.roundedBox({
+ id: "cube",
+ width: 300,
+ height: 300,
+ depth: 300,
+ radius: 10,
+ x: 200,
+ y: 100,
+ color: "#ff0000",
+ material: "unlit",
+ }).withMatte(
+ Layer.box({
+ id: "split",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 400,
+ height: 200,
+ background: "linear-gradient(to right, #000 50%, #fff 50%)",
+ }),
+ "luma-inverted",
+ ),
+ );
+ const at = await decode(await c.renderFrame({ frame: 0 }));
+ expect(at(180, 100).r).toBeGreaterThan(200);
+ expect(at(220, 100).r).toBeLessThan(20);
+ });
+ });
+});
diff --git a/packages/framefields/src/physical-shading-render.test.ts b/packages/framefields/src/physical-shading-render.test.ts
new file mode 100644
index 00000000..5aaf2a53
--- /dev/null
+++ b/packages/framefields/src/physical-shading-render.test.ts
@@ -0,0 +1,196 @@
+import { Canvas, loadImage } from "skia-canvas";
+import { describe, expect, it } from "vitest";
+import { Composition, Layer, Layer3D, Lighting } from "./index.js";
+
+async function decode(png: Buffer) {
+ const img = await loadImage(png);
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ const { data } = ctx.getImageData(0, 0, img.width, img.height);
+ return {
+ at(x: number, y: number) {
+ const i = (Math.round(y) * img.width + Math.round(x)) * 4;
+ return { r: data[i]!, g: data[i + 1]!, b: data[i + 2]! };
+ },
+ };
+}
+
+const lum = (c: { r: number; g: number; b: number }) =>
+ 0.2126 * c.r + 0.7152 * c.g + 0.0722 * c.b;
+
+/** Camera above-front of (400, 300, 0), a floor at y 360. */
+function stage(world: unknown[] = []) {
+ const comp = new Composition({
+ width: 800,
+ height: 600,
+ fps: 30,
+ durationMs: 1000,
+ backgroundColor: "#101012",
+ });
+ comp.add([
+ Layer.camera({
+ x: 400,
+ y: 300 - 500,
+ z: -700,
+ targetX: 400,
+ targetY: 300,
+ targetZ: 0,
+ }),
+ ...(world as never[]),
+ ]);
+ return comp;
+}
+
+const floor = () =>
+ Layer3D.roundedBox({
+ id: "floor",
+ width: 3000,
+ height: 10,
+ depth: 3000,
+ radius: 2,
+ x: 400,
+ y: 365,
+ color: "#808080",
+ });
+
+describe("physical shading (Layer.environment)", () => {
+ it("keeps a brand color's hue under the default neutral tone mapping", async () => {
+ const comp = stage([
+ ...Lighting.studio({ target: [400, 300, 0], shadows: false }),
+ ]);
+ comp.add(
+ Layer3D.roundedBox({
+ id: "box",
+ width: 220,
+ height: 220,
+ depth: 220,
+ radius: 20,
+ x: 400,
+ y: 300,
+ color: "#ff5a1f",
+ }),
+ );
+ const img = await decode(await comp.renderFrame({ frame: 0 }));
+ const c = img.at(400, 300);
+ expect(c.r).toBeGreaterThan(180);
+ expect(c.r).toBeGreaterThan(c.g * 1.6);
+ expect(c.g).toBeGreaterThan(c.b);
+ });
+
+ it("makes a polished metal reflect the environment instead of a flat tint", async () => {
+ const render = async (metallic: number) => {
+ const comp = stage([Layer.environment("studio")]);
+ comp.add(
+ Layer3D.sphere({
+ id: "ball",
+ radius: 160,
+ x: 400,
+ y: 300,
+ color: "#d0d0d0",
+ metallic,
+ roughness: 0.1,
+ }),
+ );
+ return decode(await comp.renderFrame({ frame: 0 }));
+ };
+ const metal = await render(1);
+ const plastic = await render(0);
+ // A mirror ball is dark where it reflects the dark floor and bright in
+ // the softbox reflections: far more contrast than a matte ball.
+ const samples = [
+ [400, 200],
+ [400, 300],
+ [400, 420],
+ [310, 300],
+ [490, 300],
+ ] as const;
+ const spread = (img: Awaited<ReturnType<typeof decode>>) => {
+ const ls = samples.map(([x, y]) => lum(img.at(x, y)));
+ return Math.max(...ls) - Math.min(...ls);
+ };
+ expect(spread(metal)).toBeGreaterThan(spread(plastic));
+ });
+
+ it("casts a shadow from a castShadow light onto the floor", async () => {
+ const render = async (castShadow: boolean) => {
+ const comp = stage([
+ Layer.environment("neutral", { intensity: 0.3 }),
+ Layer.directionalLight({
+ x: 400,
+ y: -800,
+ z: 0,
+ targetX: 400,
+ targetY: 360,
+ targetZ: 0,
+ intensity: 1.5,
+ castShadow,
+ }),
+ floor(),
+ Layer3D.roundedBox({
+ id: "occluder",
+ width: 200,
+ height: 20,
+ depth: 200,
+ radius: 4,
+ x: 400,
+ y: 150,
+ color: "#808080",
+ }),
+ ]);
+ return decode(await comp.renderFrame({ frame: 0 }));
+ };
+ const lit = await render(false);
+ const shadowed = await render(true);
+ // The floor straight below the occluder (seen past its front edge).
+ const below = (img: Awaited<ReturnType<typeof decode>>) =>
+ lum(img.at(400, 340));
+ expect(below(shadowed)).toBeLessThan(below(lit) * 0.75);
+ // Floor well away from the occluder is untouched.
+ expect(
+ Math.abs(lum(shadowed.at(120, 520)) - lum(lit.at(120, 520))),
+ ).toBeLessThan(6);
+ });
+
+ it("lights a dark surface with emissive and blooms past it", async () => {
+ const render = async (bloom: boolean) => {
+ const comp = stage([Layer.environment("night", { bloom })]);
+ comp.add(
+ Layer3D.sphere({
+ id: "lamp",
+ radius: 80,
+ x: 400,
+ y: 300,
+ color: "#111111",
+ emissive: "#3388ff",
+ emissiveIntensity: 8,
+ }),
+ );
+ return decode(await comp.renderFrame({ frame: 0 }));
+ };
+ const plain = await render(false);
+ const glowing = await render(true);
+ const core = plain.at(400, 300);
+ expect(core.b).toBeGreaterThan(200);
+ expect(core.b).toBeGreaterThan(core.r);
+ // Just outside the sphere: dark without bloom, blue with it.
+ const halo = glowing.at(400 + 110, 300);
+ const bare = plain.at(400 + 110, 300);
+ expect(halo.b).toBeGreaterThan(bare.b + 15);
+ expect(halo.b).toBeGreaterThan(halo.r);
+ });
+
+ it("fades distant surfaces into the fog color", async () => {
+ const comp = stage([
+ Layer.environment("soft", {
+ fog: { color: "#ff0000", start: 600, end: 1500 },
+ }),
+ floor(),
+ ]);
+ const img = await decode(await comp.renderFrame({ frame: 0 }));
+ const far = img.at(400, 180);
+ const near = img.at(400, 590);
+ expect(far.r).toBeGreaterThan(far.g + 80);
+ expect(Math.abs(near.r - near.g)).toBeLessThan(25);
+ });
+});
diff --git a/packages/framefields/src/shapes3d.ts b/packages/framefields/src/shapes3d.ts
index 6f10f33f..63cc2f42 100644
--- a/packages/framefields/src/shapes3d.ts
+++ b/packages/framefields/src/shapes3d.ts
@@ -17,7 +17,44 @@ import {
import { autoId } from "./ids.js";
import { type AnimatableNode, Layer } from "./index.js";
-export interface CubeFaceConfig {
+/** Physically based surface options every 3D builder accepts. */
+export interface Surface3DOptions {
+ /** Light the surface gives off: a CSS color added on top of lighting. */
+ emissive?: string;
+ /** Emissive multiplier; above 1 glows past white and blooms. Default 1. */
+ emissiveIntensity?: number;
+ /** Falls into a `castShadow` light's shadow map (meshes default on, flat layers off). */
+ castShadow?: boolean;
+}
+
+const MATERIAL_KEYS = [
+ "material",
+ "shininess",
+ "roughness",
+ "specularIntensity",
+ "metallic",
+ "ambientIntensity",
+ "ior",
+ "transmission",
+ "dispersion",
+ "fresnelPower",
+ "envIntensity",
+ "emissive",
+ "emissiveIntensity",
+ "castShadow",
+] as const;
+
+/** The material and surface options that were actually given. */
+function materialProps(o: object): Record<string, unknown> {
+ const src = o as Record<string, unknown>;
+ const out: Record<string, unknown> = {};
+ for (const k of MATERIAL_KEYS) {
+ if (src[k] !== undefined) out[k] = src[k];
+ }
+ return out;
+}
+
+export interface CubeFaceConfig extends Surface3DOptions {
background?: string;
borderRadius?: number;
borderWidth?: number;
@@ -39,7 +76,7 @@ export interface CubeFaceConfig {
export type CubeFaceInput = CubeFaceConfig | LayoutNode | string;
-export interface Cube3DOptions {
+export interface Cube3DOptions extends Surface3DOptions {
id?: string;
size: number | [number, number, number];
x?: number;
@@ -115,7 +152,7 @@ export interface Prism3DOptions {
opacity?: number;
}
-export interface Plane3DOptions {
+export interface Plane3DOptions extends Surface3DOptions {
id?: string;
width: number;
height: number;
@@ -160,7 +197,7 @@ export interface Grid3DOptions {
opacity?: number;
}
-export interface Text3DOptions {
+export interface Text3DOptions extends Surface3DOptions {
id?: string;
text: string;
fontSize?: number;
@@ -222,7 +259,7 @@ export interface ExtrudedText3DOptions extends Text3DOptions {
}
/** Node props shared by the mesh primitives (roundedBox, sphere, cylinder). */
-export interface Primitive3DNodeOptions {
+export interface Primitive3DNodeOptions extends Surface3DOptions {
id?: string;
x?: number;
y?: number;
@@ -309,6 +346,11 @@ function primitiveNode(
specularIntensity: o.specularIntensity ?? 0.5,
ambientIntensity: o.ambientIntensity ?? 1.0,
metallic: o.metallic,
+ ...materialProps({
+ emissive: o.emissive,
+ emissiveIntensity: o.emissiveIntensity,
+ castShadow: o.castShadow,
+ }),
twoSided: false,
startFrame: o.startFrame,
durationFrames: o.durationFrames,
@@ -427,6 +469,11 @@ export const Layer3D = {
metallic: cfg.metallic ?? options.metallic,
ambientIntensity:
cfg.ambientIntensity ?? options.ambientIntensity ?? 1.0,
+ ...materialProps({
+ emissive: cfg.emissive ?? options.emissive,
+ emissiveIntensity: cfg.emissiveIntensity ?? options.emissiveIntensity,
+ castShadow: cfg.castShadow ?? options.castShadow,
+ }),
background: cfg.background ?? defaultColor,
borderRadius: cfg.borderRadius ?? 0,
borderWidth: cfg.borderWidth,
@@ -644,6 +691,7 @@ export const Layer3D = {
background: options.background ?? "#3b82f6",
borderRadius: options.borderRadius ?? 0,
opacity: options.opacity ?? 1,
+ ...materialProps(options),
children: options.children ?? [],
});
},
@@ -753,6 +801,7 @@ export const Layer3D = {
specularIntensity: options.specularIntensity ?? 0.8,
ambientIntensity: options.ambientIntensity ?? 1.0,
metallic: options.metallic,
+ ...materialProps(options),
opacity: options.opacity ?? 1,
background: options.background,
borderRadius: options.borderRadius,
@@ -833,6 +882,7 @@ export const Layer3D = {
specularIntensity: options.specularIntensity ?? 0.8,
ambientIntensity: options.ambientIntensity ?? 1.0,
metallic: options.metallic,
+ ...materialProps(options),
twoSided: options.twoSided ?? true,
// What the compositor rebuilds the mesh from when modelData did not
// survive (a spec sent as JSON): the same tolerance as here.
diff --git a/packages/framefields/src/transitions-render.test.ts b/packages/framefields/src/transitions-render.test.ts
new file mode 100644
index 00000000..8eb3cb83
--- /dev/null
+++ b/packages/framefields/src/transitions-render.test.ts
@@ -0,0 +1,93 @@
+import { Canvas, loadImage } from "skia-canvas";
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ Layer,
+ Transition,
+ type TransitionSpec,
+} from "./index.js";
+
+async function decode(png: Buffer) {
+ const img = await loadImage(png);
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ const { data } = ctx.getImageData(0, 0, img.width, img.height);
+ return (x: number, y: number) => {
+ const i = (Math.round(y) * img.width + Math.round(x)) * 4;
+ return { r: data[i]!, g: data[i + 1]!, b: data[i + 2]! };
+ };
+}
+
+const W = 400;
+const H = 200;
+const scene = (id: string, background: string) =>
+ Layer.box({ id, width: W, height: H, background });
+
+/** Red scene (30 frames) then blue scene (30 frames) through `t`. */
+async function render(t: TransitionSpec, frame: number) {
+ const seq = Layer.sequence(
+ [
+ { scene: scene("a", "#ff0000"), duration: 30 },
+ { scene: scene("b", "#0000ff"), duration: 30, transition: t },
+ ],
+ { width: W, height: H },
+ );
+ const comp = new Composition({
+ width: W,
+ height: H,
+ fps: 30,
+ durationMs: (seq.totalFrames / 30) * 1000,
+ backgroundColor: "#000000",
+ });
+ comp.add(seq);
+ return { at: await decode(await comp.renderFrame({ frame })), seq };
+}
+
+describe("Layer.sequence + Transition", () => {
+ it("overlaps scenes by the transition length", async () => {
+ const { seq } = await render(Transition.crossfade(10), 0);
+ expect(seq.sceneStarts).toEqual([0, 20]);
+ expect(seq.totalFrames).toBe(50);
+ const cut = await render(Transition.cut(), 0);
+ expect(cut.seq.sceneStarts).toEqual([0, 30]);
+ });
+
+ it("crossfades", async () => {
+ const before = (await render(Transition.crossfade(10), 15)).at(200, 100);
+ const mid = (await render(Transition.crossfade(10), 25)).at(200, 100);
+ const after = (await render(Transition.crossfade(10), 35)).at(200, 100);
+ expect(before.r).toBeGreaterThan(240);
+ expect(mid.r).toBeGreaterThan(60);
+ expect(mid.b).toBeGreaterThan(60);
+ expect(after.b).toBeGreaterThan(240);
+ expect(after.r).toBeLessThan(10);
+ });
+
+ it("pushes the old scene out as the new one comes in", async () => {
+ const { at } = await render(Transition.push("left", 10, "none"), 25);
+ // Halfway: red has moved left by half, blue fills the right half.
+ expect(at(100, 100).r).toBeGreaterThan(240);
+ expect(at(300, 100).b).toBeGreaterThan(240);
+ });
+
+ it("wipes with a matte edge", async () => {
+ const { at } = await render(Transition.wipe("right", 10, "none"), 25);
+ // Edge travelling right: the left half is already revealed.
+ expect(at(100, 100).b).toBeGreaterThan(240);
+ expect(at(300, 100).r).toBeGreaterThan(240);
+ });
+
+ it("opens an iris from the center", async () => {
+ const { at } = await render(Transition.iris(10, { ease: "none" }), 25);
+ expect(at(200, 100).b).toBeGreaterThan(240);
+ expect(at(5, 5).r).toBeGreaterThan(240);
+ });
+
+ it("fades through the background", async () => {
+ const { at } = await render(Transition.fadeThrough(10, "none"), 25);
+ const c = at(200, 100);
+ expect(c.r).toBeLessThan(40);
+ expect(c.b).toBeLessThan(40);
+ });
+});
diff --git a/packages/framefields/src/transitions.ts b/packages/framefields/src/transitions.ts
new file mode 100644
index 00000000..70535e98
--- /dev/null
+++ b/packages/framefields/src/transitions.ts
@@ -0,0 +1,383 @@
+/**
+ * Scene sequencing with transitions. `Layer.sequence` lays scenes end to end
+ * and overlaps neighbours by each transition's length; the transition itself
+ * is plain keyframes (and, for wipes and irises, a track matte) merged into
+ * the scenes, so it renders like anything else and stays editable.
+ */
+import {
+ type AnimationTrack,
+ type BoxNode,
+ LayerAnimation,
+ type LayoutNode,
+} from "@framefields/compositions/program";
+import { autoId } from "./ids.js";
+import { type AnimatableNode, Layer } from "./index.js";
+
+export type TransitionDirection = "left" | "right" | "up" | "down";
+
+/** What a transition needs to know to animate two scenes. */
+interface TransitionContext {
+ /** Scene size (px). */
+ width: number;
+ height: number;
+ /** Overlap length in frames. */
+ frames: number;
+ /** Outgoing scene: local frame where the overlap starts, and its rest position. */
+ outStart: number;
+ outRest: { x: number; y: number };
+ /** Incoming scene rest position (the overlap starts at its local frame 0). */
+ inRest: { x: number; y: number };
+}
+
+interface TransitionParts {
+ outgoing?: LayerAnimation;
+ incoming?: LayerAnimation;
+ /** A matte revealing the incoming scene (laid out like the scene, keyed from its start). */
+ incomingMatte?: LayoutNode;
+ /** Static props set on both scenes (e.g. motion blur for a whip). */
+ sceneProps?: Record<string, unknown>;
+}
+
+export interface TransitionSpec {
+ readonly kind: string;
+ /** Overlap between the two scenes, in frames (0 = hard cut). */
+ readonly frames: number;
+ /** @internal */
+ build(ctx: TransitionContext): TransitionParts;
+}
+
+const offsetFor = (dir: TransitionDirection, w: number, h: number) => {
+ switch (dir) {
+ case "left":
+ return { dx: -w, dy: 0 };
+ case "right":
+ return { dx: w, dy: 0 };
+ case "up":
+ return { dx: 0, dy: -h };
+ default:
+ return { dx: 0, dy: h };
+ }
+};
+
+function spec(
+ kind: string,
+ frames: number,
+ build: (ctx: TransitionContext) => TransitionParts,
+): TransitionSpec {
+ return { kind, frames: Math.max(0, Math.round(frames)), build };
+}
+
+/**
+ * Scene-to-scene transitions for `Layer.sequence`. `frames` is how long the
+ * two scenes overlap; `ease` shapes the move (default "power3.inOut").
+ */
+export const Transition = {
+ /** Instant cut. */
+ cut: (): TransitionSpec => spec("cut", 0, () => ({})),
+
+ /** The incoming scene fades in over the outgoing one. */
+ crossfade: (frames = 15, ease = "sine.inOut"): TransitionSpec =>
+ spec("crossfade", frames, (c) => ({
+ incoming: LayerAnimation.keys("opacity", [
+ [0, 0],
+ [c.frames, 1, ease],
+ ]),
+ })),
+
+ /** Out to the background, then in: a dip to the composition's background color. */
+ fadeThrough: (frames = 20, ease = "sine.inOut"): TransitionSpec =>
+ spec("fadeThrough", frames, (c) => {
+ const half = Math.max(1, Math.floor(c.frames / 2));
+ return {
+ outgoing: LayerAnimation.keys("opacity", [
+ [c.outStart, 1],
+ [c.outStart + half, 0, ease],
+ ]),
+ incoming: LayerAnimation.keys("opacity", [
+ [0, 0],
+ [half, 0],
+ [c.frames, 1, ease],
+ ]),
+ };
+ }),
+
+ /** Both scenes move together: the new one pushes the old one out toward `direction`. */
+ push: (
+ direction: TransitionDirection = "left",
+ frames = 18,
+ ease = "power3.inOut",
+ ): TransitionSpec =>
+ spec("push", frames, (c) => {
+ const { dx, dy } = offsetFor(direction, c.width, c.height);
+ const out = LayerAnimation.create();
+ const inc = LayerAnimation.create();
+ if (dx !== 0) {
+ out.keys("x", [
+ [c.outStart, c.outRest.x],
+ [c.outStart + c.frames, c.outRest.x + dx, ease],
+ ]);
+ inc.keys("x", [
+ [0, c.inRest.x - dx],
+ [c.frames, c.inRest.x, ease],
+ ]);
+ } else {
+ out.keys("y", [
+ [c.outStart, c.outRest.y],
+ [c.outStart + c.frames, c.outRest.y + dy, ease],
+ ]);
+ inc.keys("y", [
+ [0, c.inRest.y - dy],
+ [c.frames, c.inRest.y, ease],
+ ]);
+ }
+ return { outgoing: out, incoming: inc };
+ }),
+
+ /** The new scene slides in over the old one, which stays put and dims slightly. */
+ slide: (
+ direction: TransitionDirection = "left",
+ frames = 18,
+ ease = "power3.out",
+ ): TransitionSpec =>
+ spec("slide", frames, (c) => {
+ const { dx, dy } = offsetFor(direction, c.width, c.height);
+ const prop = dx !== 0 ? "x" : "y";
+ const rest = dx !== 0 ? c.inRest.x : c.inRest.y;
+ return {
+ outgoing: LayerAnimation.keys("opacity", [
+ [c.outStart, 1],
+ [c.outStart + c.frames, 0.6, ease],
+ ]),
+ incoming: LayerAnimation.keys(prop, [
+ [0, rest - (dx || dy)],
+ [c.frames, rest, ease],
+ ]),
+ };
+ }),
+
+ /** Old scene grows and fades, new one settles in from slightly small. */
+ zoom: (frames = 18, ease = "power2.inOut"): TransitionSpec =>
+ spec("zoom", frames, (c) => ({
+ outgoing: LayerAnimation.create()
+ .keys("scale", [
+ [c.outStart, 1],
+ [c.outStart + c.frames, 1.25, ease],
+ ])
+ .keys("opacity", [
+ [c.outStart, 1],
+ [c.outStart + c.frames, 0, ease],
+ ]),
+ incoming: LayerAnimation.create()
+ .keys("scale", [
+ [0, 0.85],
+ [c.frames, 1, ease],
+ ])
+ .keys("opacity", [
+ [0, 0],
+ [c.frames, 1, ease],
+ ]),
+ })),
+
+ /** A fast, motion-blurred push: the camera whips to the next scene. */
+ whip: (
+ direction: TransitionDirection = "left",
+ frames = 10,
+ ): TransitionSpec => {
+ const push = Transition.push(direction, frames, "expo.inOut");
+ return spec("whip", frames, (c) => ({
+ ...push.build(c),
+ sceneProps: { motionBlurShutter: 360 },
+ }));
+ },
+
+ /** A hard edge sweeps across, revealing the new scene. */
+ wipe: (
+ direction: TransitionDirection = "left",
+ frames = 18,
+ ease = "power2.inOut",
+ ): TransitionSpec =>
+ spec("wipe", frames, (c) => {
+ // `direction` is where the edge travels: "left" reveals from the right.
+ const { dx, dy } = offsetFor(direction, c.width, c.height);
+ const prop = dx !== 0 ? "x" : "y";
+ const rest = dx !== 0 ? c.inRest.x : c.inRest.y;
+ return {
+ incomingMatte: Layer.box({
+ id: autoId("wipe-matte"),
+ position: "absolute",
+ x: c.inRest.x,
+ y: c.inRest.y,
+ width: c.width,
+ height: c.height,
+ background: "#ffffff",
+ }).animate(
+ LayerAnimation.keys(prop, [
+ [0, rest - (dx || dy)],
+ [c.frames, rest, ease],
+ ]),
+ ),
+ };
+ }),
+
+ /** A circle opens from the center (or `at`, as fractions of the frame) to reveal the new scene. */
+ iris: (
+ frames = 20,
+ options: { at?: [number, number]; ease?: string } = {},
+ ): TransitionSpec =>
+ spec("iris", frames, (c) => {
+ const [fx, fy] = options.at ?? [0.5, 0.5];
+ const cx = c.inRest.x + fx * c.width;
+ const cy = c.inRest.y + fy * c.height;
+ // Big enough to cover the farthest corner.
+ const r = Math.hypot(
+ Math.max(fx, 1 - fx) * c.width,
+ Math.max(fy, 1 - fy) * c.height,
+ );
+ return {
+ incomingMatte: Layer.shape("circle", {
+ id: autoId("iris-matte"),
+ position: "absolute",
+ x: cx - r,
+ y: cy - r,
+ width: r * 2,
+ height: r * 2,
+ fillColor: "#ffffff",
+ }).animate(
+ LayerAnimation.keys("scale", [
+ [0, 0],
+ [c.frames, 1, options.ease ?? "power2.inOut"],
+ ]),
+ ),
+ };
+ }),
+};
+
+export interface SequenceEntry {
+ /** A full-frame layer (usually a box holding the scene). */
+ scene: AnimatableNode<LayoutNode> | LayoutNode;
+ /** Frames this scene is on screen, transitions included. */
+ duration: number;
+ /** How this scene arrives from the previous one (default: cut). */
+ transition?: TransitionSpec;
+}
+
+export interface SceneSequenceOptions {
+ id?: string;
+ /** Scene size used by moving transitions (default: each scene's own width/height, else 1920Γ—1080). */
+ width?: number;
+ height?: number;
+ /** First frame of the whole sequence. Default 0. */
+ startFrame?: number;
+}
+
+export type SequenceNode = AnimatableNode<BoxNode> & {
+ /** Total length in frames: put it in the composition's duration. */
+ readonly totalFrames: number;
+ /** Where each scene starts (composition frames, relative to the sequence). */
+ readonly sceneStarts: readonly number[];
+};
+
+function sizeOf(node: LayoutNode, axis: "width" | "height", fallback: number) {
+ const v = (node as unknown as Record<string, unknown>)[axis];
+ return typeof v === "number" ? v : fallback;
+}
+
+/**
+ * Scenes back to back, each overlapping the previous one by its transition.
+ * Scene timing is set for you (`startFrame` / `durationFrames`), so every
+ * scene keys its own animation from its local frame 0.
+ */
+export function sequence(
+ entries: SequenceEntry[],
+ options: SceneSequenceOptions = {},
+): SequenceNode {
+ const scenes: LayoutNode[] = [];
+ const starts: number[] = [];
+ let cursor = 0;
+ entries.forEach((entry, i) => {
+ const t = i === 0 ? undefined : entry.transition;
+ const overlap = Math.min(t?.frames ?? 0, entries[i - 1]?.duration ?? 0);
+ const start = Math.max(0, cursor - overlap);
+ const duration = Math.max(1, Math.round(entry.duration));
+ starts.push(start);
+ const node = entry.scene as LayoutNode & Record<string, unknown>;
+ // Scenes stack in one frame rather than flowing one under the other.
+ if (node.position === undefined || node.position === "relative") {
+ node.position = "absolute";
+ node.x ??= 0;
+ node.y ??= 0;
+ }
+ node.startFrame = start;
+ node.durationFrames = duration;
+ scenes.push(node);
+ cursor = start + duration;
+ });
+
+ entries.forEach((entry, i) => {
+ if (i === 0 || !entry.transition || entry.transition.frames === 0) return;
+ const prev = scenes[i - 1] as LayoutNode & Record<string, unknown>;
+ const cur = scenes[i] as LayoutNode & Record<string, unknown>;
+ const frames = Math.min(entry.transition.frames, entries[i - 1].duration);
+ const width = options.width ?? sizeOf(cur, "width", 1920);
+ const height = options.height ?? sizeOf(cur, "height", 1080);
+ const parts = entry.transition.build({
+ width,
+ height,
+ frames,
+ outStart: entries[i - 1].duration - frames,
+ outRest: {
+ x: typeof prev.x === "number" ? prev.x : 0,
+ y: typeof prev.y === "number" ? prev.y : 0,
+ },
+ inRest: {
+ x: typeof cur.x === "number" ? cur.x : 0,
+ y: typeof cur.y === "number" ? cur.y : 0,
+ },
+ });
+ const merge = (
+ node: LayoutNode & Record<string, unknown>,
+ anim?: LayerAnimation,
+ ) => {
+ if (!anim) return;
+ const existing =
+ (node.animation as { tracks?: AnimationTrack[] } | undefined)?.tracks ??
+ [];
+ node.animation = LayerAnimation.merge(
+ { tracks: existing },
+ anim,
+ ).toSpec();
+ };
+ merge(prev, parts.outgoing);
+ merge(cur, parts.incoming);
+ if (parts.sceneProps) {
+ Object.assign(prev, parts.sceneProps);
+ Object.assign(cur, parts.sceneProps);
+ }
+ if (parts.incomingMatte) {
+ cur.matte = { node: parts.incomingMatte, mode: "alpha" };
+ }
+ });
+
+ const root = Layer.box({
+ id: options.id ?? autoId("sequence"),
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width:
+ options.width ?? sizeOf(scenes[0] ?? ({} as LayoutNode), "width", 1920),
+ height:
+ options.height ?? sizeOf(scenes[0] ?? ({} as LayoutNode), "height", 1080),
+ startFrame: options.startFrame ?? 0,
+ durationFrames: Math.max(1, cursor),
+ children: scenes,
+ });
+ Object.defineProperty(root, "totalFrames", {
+ value: cursor,
+ enumerable: false,
+ });
+ Object.defineProperty(root, "sceneStarts", {
+ value: starts,
+ enumerable: false,
+ });
+ return root as SequenceNode;
+}
diff --git a/packages/webgpu-renderers/src/color.ts b/packages/webgpu-renderers/src/color.ts
index 95422297..9b4ceef5 100644
--- a/packages/webgpu-renderers/src/color.ts
+++ b/packages/webgpu-renderers/src/color.ts
@@ -1,79 +1,13 @@
+import { parseCssColor } from "@framefields/core";
import type { Color } from "./renderer2d/index.js";
-export type { Color }
-/**
- * CSS named colors β†’ normalized RGBA.
- * Covers the most commonly used names; hex and functional notation
- * are handled by the parser below.
- */
-const NAMED_COLORS: Record<string, Color> = {
- transparent: { r: 0, g: 0, b: 0, a: 0 },
- black: { r: 0, g: 0, b: 0, a: 1 },
- white: { r: 1, g: 1, b: 1, a: 1 },
- red: { r: 1, g: 0, b: 0, a: 1 },
- green: { r: 0, g: 0.502, b: 0, a: 1 }, // CSS green = #008000
- lime: { r: 0, g: 1, b: 0, a: 1 },
- blue: { r: 0, g: 0, b: 1, a: 1 },
- yellow: { r: 1, g: 1, b: 0, a: 1 },
- cyan: { r: 0, g: 1, b: 1, a: 1 },
- aqua: { r: 0, g: 1, b: 1, a: 1 },
- magenta: { r: 1, g: 0, b: 1, a: 1 },
- fuchsia: { r: 1, g: 0, b: 1, a: 1 },
- orange: { r: 1, g: 0.647, b: 0, a: 1 },
- purple: { r: 0.502, g: 0, b: 0.502, a: 1 },
- pink: { r: 1, g: 0.753, b: 0.796, a: 1 },
- gray: { r: 0.502, g: 0.502, b: 0.502, a: 1 },
- grey: { r: 0.502, g: 0.502, b: 0.502, a: 1 },
-};
-
-const RGBA_RE =
- /^rgba?\(\s*(\d{1,3})\s*,\s*(\d{1,3})\s*,\s*(\d{1,3})\s*(?:,\s*([\d.]+)\s*)?\)$/;
-
-function parseHex(hex: string): Color | null {
- const clean = hex.startsWith("#") ? hex.slice(1) : hex;
- const len = clean.length;
-
- if (len === 3) {
- return {
- r: parseInt(clean[0] + clean[0], 16) / 255,
- g: parseInt(clean[1] + clean[1], 16) / 255,
- b: parseInt(clean[2] + clean[2], 16) / 255,
- a: 1,
- };
- }
- if (len === 4) {
- return {
- r: parseInt(clean[0] + clean[0], 16) / 255,
- g: parseInt(clean[1] + clean[1], 16) / 255,
- b: parseInt(clean[2] + clean[2], 16) / 255,
- a: parseInt(clean[3] + clean[3], 16) / 255,
- };
- }
- if (len === 6) {
- return {
- r: parseInt(clean.slice(0, 2), 16) / 255,
- g: parseInt(clean.slice(2, 4), 16) / 255,
- b: parseInt(clean.slice(4, 6), 16) / 255,
- a: 1,
- };
- }
- if (len === 8) {
- return {
- r: parseInt(clean.slice(0, 2), 16) / 255,
- g: parseInt(clean.slice(2, 4), 16) / 255,
- b: parseInt(clean.slice(4, 6), 16) / 255,
- a: parseInt(clean.slice(6, 8), 16) / 255,
- };
- }
- return null;
-}
-
+export type { Color };
/**
* Parse any CSS color string into a normalized {r,g,b,a} Color
* with channel values in [0,1].
*
* Supports:
* - Hex: #RGB, #RGBA, #RRGGBB, #RRGGBBAA
- * - Functional: rgb(r,g,b), rgba(r,g,b,a)
+ * - Functional: rgb()/rgba(), hsl()/hsla(), oklch(), oklab() (comma or space syntax)
* - Named: black, white, red, green, blue, yellow, transparent, etc.
* - Color objects pass through unchanged.
*
@@ -94,27 +28,10 @@ export function parseColor(color: Color | string | null | undefined): Color {
return { r: 0, g: 0, b: 0, a: 0 };
}
- // Hex
- if (trimmed.startsWith("#")) {
- const parsed = parseHex(trimmed);
- if (parsed) return parsed;
- }
-
- // rgb() / rgba()
- const rgbaMatch = RGBA_RE.exec(trimmed);
- if (rgbaMatch) {
- return {
- r: parseInt(rgbaMatch[1], 10) / 255,
- g: parseInt(rgbaMatch[2], 10) / 255,
- b: parseInt(rgbaMatch[3], 10) / 255,
- a: rgbaMatch[4] !== undefined ? parseFloat(rgbaMatch[4]) : 1,
- };
- }
-
- // Named color
- const named = NAMED_COLORS[trimmed.toLowerCase()];
- if (named) {
- return { ...named };
+ // Hex, rgb()/rgba(), hsl(), oklch(), oklab() and named colors.
+ const parsed = parseCssColor(trimmed);
+ if (parsed) {
+ return { r: parsed[0], g: parsed[1], b: parsed[2], a: parsed[3] };
}
// Fallback: transparent black
diff --git a/packages/webgpu-renderers/src/loaders/gltf-loader.ts b/packages/webgpu-renderers/src/loaders/gltf-loader.ts
index ce8a8e21..fc2463af 100644
--- a/packages/webgpu-renderers/src/loaders/gltf-loader.ts
+++ b/packages/webgpu-renderers/src/loaders/gltf-loader.ts
@@ -111,13 +111,18 @@ interface GltfMaterial {
metallicFactor?: number;
roughnessFactor?: number;
baseColorTexture?: { index: number; texCoord?: number };
+ metallicRoughnessTexture?: { index: number; texCoord?: number };
};
+ normalTexture?: { index: number; texCoord?: number; scale?: number };
+ occlusionTexture?: { index: number; texCoord?: number; strength?: number };
+ emissiveTexture?: { index: number; texCoord?: number };
emissiveFactor?: [number, number, number];
doubleSided?: boolean;
alphaMode?: "OPAQUE" | "MASK" | "BLEND";
alphaCutoff?: number;
extensions?: {
KHR_materials_unlit?: Record<string, never>;
+ KHR_materials_emissive_strength?: { emissiveStrength?: number };
/** VRM 1.0 toon material: the base is lit flat, the shadowed side tinted by shadeColorFactor. */
VRMC_materials_mtoon?: { shadeColorFactor?: [number, number, number] };
};
@@ -204,7 +209,8 @@ export class GltfLoader {
* Checks if the given buffer starts with the GLB binary container magic bytes.
*/
public static isGlb(buffer: Uint8Array | ArrayBuffer): boolean {
- const bytes = buffer instanceof Uint8Array ? buffer : new Uint8Array(buffer);
+ const bytes =
+ buffer instanceof Uint8Array ? buffer : new Uint8Array(buffer);
if (bytes.length < 12) return false;
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
return view.getUint32(0, true) === GLB_MAGIC;
@@ -253,13 +259,23 @@ export class GltfLoader {
if (base64Index !== -1) {
const b64 = buf.uri.substring(base64Index + 7);
const rawBuf = Buffer.from(b64, "base64");
- binaryBuffers[i] = new Uint8Array(rawBuf.buffer, rawBuf.byteOffset, rawBuf.byteLength);
+ binaryBuffers[i] = new Uint8Array(
+ rawBuf.buffer,
+ rawBuf.byteOffset,
+ rawBuf.byteLength,
+ );
}
} else if (basePath) {
- const resolved = path.isAbsolute(buf.uri) ? buf.uri : path.join(basePath, buf.uri);
+ const resolved = path.isAbsolute(buf.uri)
+ ? buf.uri
+ : path.join(basePath, buf.uri);
if (fs.existsSync(resolved)) {
const fileBuf = fs.readFileSync(resolved);
- binaryBuffers[i] = new Uint8Array(fileBuf.buffer, fileBuf.byteOffset, fileBuf.byteLength);
+ binaryBuffers[i] = new Uint8Array(
+ fileBuf.buffer,
+ fileBuf.byteOffset,
+ fileBuf.byteLength,
+ );
}
}
}
@@ -269,7 +285,10 @@ export class GltfLoader {
return GltfLoader.buildModel(doc, binaryBuffers, options, basePath);
}
- private static unpackGlb(bytes: Uint8Array): { doc: GltfDocument; binBuffer?: Uint8Array } {
+ private static unpackGlb(bytes: Uint8Array): {
+ doc: GltfDocument;
+ binBuffer?: Uint8Array;
+ } {
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
const magic = view.getUint32(0, true);
const totalLength = view.getUint32(8, true);
@@ -374,7 +393,9 @@ export class GltfLoader {
const comma = image.uri.indexOf(",");
const mime = image.uri.slice(5, image.uri.indexOf(";"));
return {
- bytes: new Uint8Array(Buffer.from(image.uri.slice(comma + 1), "base64")),
+ bytes: new Uint8Array(
+ Buffer.from(image.uri.slice(comma + 1), "base64"),
+ ),
mime,
};
}
@@ -415,6 +436,12 @@ export class GltfLoader {
: "opaque",
alphaCutoff: mat.alphaMode === "MASK" ? (mat.alphaCutoff ?? 0.5) : 0,
};
+ const e = mat.emissiveFactor;
+ if (e && (e[0] > 0 || e[1] > 0 || e[2] > 0)) {
+ material.emissive = [e[0], e[1], e[2]];
+ material.emissiveStrength =
+ mat.extensions?.KHR_materials_emissive_strength?.emissiveStrength ?? 1;
+ }
const mtoon = mat.extensions?.VRMC_materials_mtoon;
if (mtoon) {
material.shading = "toon";
@@ -442,6 +469,32 @@ export class GltfLoader {
wrap === 33071 ? "clamp" : wrap === 33648 ? "mirror" : "repeat";
}
}
+ const mapBytes = (ref?: { index: number }) => {
+ const tex = ref ? doc.textures?.[ref.index] : undefined;
+ return tex?.source !== undefined
+ ? GltfLoader.imageBytes(doc, binaryBuffers, tex.source, basePath)?.bytes
+ : undefined;
+ };
+ const normal = mapBytes(mat.normalTexture);
+ if (normal) {
+ material.normalTextureBuffer = normal;
+ material.normalScale = mat.normalTexture?.scale ?? 1;
+ }
+ const mr = mapBytes(pbr?.metallicRoughnessTexture);
+ if (mr) {
+ material.metallicRoughnessTextureBuffer = mr;
+ // With a map the factors scale it, and glTF's default factor is 1.
+ material.roughness = pbr?.roughnessFactor ?? 1;
+ material.metallic = pbr?.metallicFactor ?? 1;
+ }
+ const occlusion = mapBytes(mat.occlusionTexture);
+ if (occlusion) {
+ material.occlusionTextureBuffer = occlusion;
+ material.occlusionStrength = mat.occlusionTexture?.strength ?? 1;
+ }
+ // glTF's emissiveFactor defaults to 0: a map alone does not glow.
+ const emissiveMap = material.emissive && mapBytes(mat.emissiveTexture);
+ if (emissiveMap) material.emissiveTextureBuffer = emissiveMap;
return material;
}
@@ -476,8 +529,16 @@ export class GltfLoader {
const node = chan.target.node;
if (!sampler || node === undefined || chan.target.path === "weights")
continue;
- const times = GltfLoader.readFloats(doc, binaryBuffers, sampler.input).data;
- const values = GltfLoader.readFloats(doc, binaryBuffers, sampler.output).data;
+ const times = GltfLoader.readFloats(
+ doc,
+ binaryBuffers,
+ sampler.input,
+ ).data;
+ const values = GltfLoader.readFloats(
+ doc,
+ binaryBuffers,
+ sampler.output,
+ ).data;
duration = Math.max(duration, times[times.length - 1] ?? 0);
const track: NodeAnimationTrack3D = {
nodeName: nodes[node]?.name ?? `node_${node}`,
@@ -513,14 +574,21 @@ export class GltfLoader {
// Material keys must be unique: glTF names are optional and may repeat.
const materialKeys = (doc.materials ?? []).map((mat, idx, all) => {
const name = mat.name ?? `material_${idx}`;
- return all.findIndex((m, j) => (m.name ?? `material_${j}`) === name) === idx
+ return all.findIndex((m, j) => (m.name ?? `material_${j}`) === name) ===
+ idx
? name
: `${name}#${idx}`;
});
const materials: Record<string, Material3D> = {};
(doc.materials ?? []).forEach((mat, idx) => {
const key = materialKeys[idx]!;
- materials[key] = GltfLoader.buildMaterial(doc, binaryBuffers, mat, key, basePath);
+ materials[key] = GltfLoader.buildMaterial(
+ doc,
+ binaryBuffers,
+ mat,
+ key,
+ basePath,
+ );
});
const nodes = GltfLoader.buildNodes(doc);
@@ -528,7 +596,8 @@ export class GltfLoader {
joints: skin.joints,
inverseBindMatrices:
skin.inverseBindMatrices !== undefined
- ? GltfLoader.readFloats(doc, binaryBuffers, skin.inverseBindMatrices).data
+ ? GltfLoader.readFloats(doc, binaryBuffers, skin.inverseBindMatrices)
+ .data
: Float32Array.from(
skin.joints.flatMap(() => Matrix4Math.identity()),
),
@@ -540,7 +609,9 @@ export class GltfLoader {
n.mesh !== undefined ? [{ mesh: n.mesh, node, skin: n.skin }] : [],
);
const instances: { mesh: number; node?: number; skin?: number }[] =
- placed.length > 0 ? placed : (doc.meshes ?? []).map((_, mesh) => ({ mesh }));
+ placed.length > 0
+ ? placed
+ : (doc.meshes ?? []).map((_, mesh) => ({ mesh }));
const model: Model3DData = {
name: "glTF_Model",
@@ -559,8 +630,12 @@ export class GltfLoader {
};
const rest = poseNodes(model, undefined, 0);
- let minX = Infinity, minY = Infinity, minZ = Infinity;
- let maxX = -Infinity, maxY = -Infinity, maxZ = -Infinity;
+ let minX = Infinity,
+ minY = Infinity,
+ minZ = Infinity;
+ let maxX = -Infinity,
+ maxY = -Infinity,
+ maxZ = -Infinity;
const grow = (x: number, y: number, z: number) => {
if (x < minX) minX = x;
if (x > maxX) maxX = x;
@@ -577,7 +652,9 @@ export class GltfLoader {
if (prim.attributes.POSITION === undefined) return;
if (prim.mode !== undefined && prim.mode !== 4) return;
const read = (acc: number | undefined) =>
- acc === undefined ? undefined : GltfLoader.readFloats(doc, binaryBuffers, acc);
+ acc === undefined
+ ? undefined
+ : GltfLoader.readFloats(doc, binaryBuffers, acc);
const pos = read(prim.attributes.POSITION)!;
const numVerts = pos.count;
@@ -594,13 +671,18 @@ export class GltfLoader {
if (prim.indices !== undefined) {
indices = GltfLoader.readIndices(doc, binaryBuffers, prim.indices);
} else {
- indices = numVerts > 65535 ? new Uint32Array(numVerts) : new Uint16Array(numVerts);
+ indices =
+ numVerts > 65535
+ ? new Uint32Array(numVerts)
+ : new Uint16Array(numVerts);
for (let i = 0; i < numVerts; i++) indices[i] = i;
}
// Skinned vertices are in bind (model) space; others sit under their node.
- const skinned = inst.skin !== undefined && prim.attributes.JOINTS_0 !== undefined;
- const place = !skinned && inst.node !== undefined ? rest[inst.node] : undefined;
+ const skinned =
+ inst.skin !== undefined && prim.attributes.JOINTS_0 !== undefined;
+ const place =
+ !skinned && inst.node !== undefined ? rest[inst.node] : undefined;
for (let i = 0; i < numVerts; i++) {
const x = pos.data[i * 3]!;
const y = pos.data[i * 3 + 1]!;
@@ -615,13 +697,24 @@ export class GltfLoader {
id: `gltf_mesh_${inst.mesh}_${primIdx}${inst.node !== undefined ? `_n${inst.node}` : ""}`,
name: meshDef.name ?? `Mesh_${inst.mesh}`,
positions: pos.data,
- normals: read(prim.attributes.NORMAL)?.data ?? new Float32Array(numVerts * 3),
- uvs: read(prim.attributes.TEXCOORD_0)?.data ?? new Float32Array(numVerts * 2),
+ normals:
+ read(prim.attributes.NORMAL)?.data ??
+ new Float32Array(numVerts * 3),
+ uvs:
+ read(prim.attributes.TEXCOORD_0)?.data ??
+ new Float32Array(numVerts * 2),
colors: rgba,
- jointIndices: skinned ? read(prim.attributes.JOINTS_0)?.data : undefined,
- jointWeights: skinned ? read(prim.attributes.WEIGHTS_0)?.data : undefined,
+ jointIndices: skinned
+ ? read(prim.attributes.JOINTS_0)?.data
+ : undefined,
+ jointWeights: skinned
+ ? read(prim.attributes.WEIGHTS_0)?.data
+ : undefined,
indices,
- materialName: prim.material !== undefined ? materialKeys[prim.material] : undefined,
+ materialName:
+ prim.material !== undefined
+ ? materialKeys[prim.material]
+ : undefined,
node: inst.node,
skin: skinned ? inst.skin : undefined,
});
@@ -640,7 +733,12 @@ export class GltfLoader {
maxX = maxY = maxZ = 0;
}
- const maxDimension = Math.max(maxX - minX, maxY - minY, maxZ - minZ, 0.0001);
+ const maxDimension = Math.max(
+ maxX - minX,
+ maxY - minY,
+ maxZ - minZ,
+ 0.0001,
+ );
const fit =
options.normalizeSize && options.normalizeSize > 0
? options.normalizeSize / maxDimension
@@ -674,9 +772,14 @@ export class GltfLoader {
model.bounds = {
min: lo,
max: hi,
- center: [(lo[0] + hi[0]) * 0.5, (lo[1] + hi[1]) * 0.5, (lo[2] + hi[2]) * 0.5],
+ center: [
+ (lo[0] + hi[0]) * 0.5,
+ (lo[1] + hi[1]) * 0.5,
+ (lo[2] + hi[2]) * 0.5,
+ ],
size: [hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2]],
- boundingSphereRadius: Math.hypot(hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2]) * 0.5,
+ boundingSphereRadius:
+ Math.hypot(hi[0] - lo[0], hi[1] - lo[1], hi[2] - lo[2]) * 0.5,
};
return model;
}
diff --git a/packages/webgpu-renderers/src/loaders/types.ts b/packages/webgpu-renderers/src/loaders/types.ts
index 6fe7538c..23a86953 100644
--- a/packages/webgpu-renderers/src/loaders/types.ts
+++ b/packages/webgpu-renderers/src/loaders/types.ts
@@ -48,6 +48,20 @@ export interface Material3D {
shading?: "lit" | "unlit" | "toon";
/** Toon shading: the color multiplier of the shadowed side. */
shadeColor?: [number, number, number];
+ /** Linear-ish emissive color 0..1 (glTF emissiveFactor). */
+ emissive?: [number, number, number];
+ /** Emissive multiplier (glTF KHR_materials_emissive_strength). */
+ emissiveStrength?: number;
+ /** Encoded tangent-space normal map (glTF convention: +Y up / OpenGL). */
+ normalTextureBuffer?: Uint8Array;
+ normalScale?: number;
+ /** Encoded map: G = roughness, B = metallic (glTF). */
+ metallicRoughnessTextureBuffer?: Uint8Array;
+ /** Encoded map: R = ambient occlusion. */
+ occlusionTextureBuffer?: Uint8Array;
+ occlusionStrength?: number;
+ /** Encoded emissive color map (multiplies `emissive`). */
+ emissiveTextureBuffer?: Uint8Array;
}
/** A scene-graph node: its rest transform and parent (glTF node semantics). */
diff --git a/packages/webgpu-renderers/src/math3d/environment3d.ts b/packages/webgpu-renderers/src/math3d/environment3d.ts
new file mode 100644
index 00000000..60d6f346
--- /dev/null
+++ b/packages/webgpu-renderers/src/math3d/environment3d.ts
@@ -0,0 +1,174 @@
+/**
+ * Scene-wide "world" settings for physically based 3D: a procedural studio
+ * environment (image-based light), exposure, tone mapping and fog. A scene
+ * with an environment is shaded physically; one without keeps the original
+ * Blinn-Phong look.
+ */
+import { parseCssColor } from "@framefields/core";
+
+/**
+ * "neutral" (default, Khronos PBR Neutral): colors as authored, highlights
+ * rolled off: right for products, brands and UI. "agx": filmic, desaturates
+ * hot highlights gracefully. "aces": punchy film contrast, shifts hues.
+ */
+export type ToneMapping = "neutral" | "agx" | "aces" | "none";
+
+export type EnvironmentPreset =
+ | "studio"
+ | "soft"
+ | "sunset"
+ | "night"
+ | "neutral";
+
+export interface Environment3DFog {
+ /** Display color surfaces fade into; match the background for a seamless horizon. */
+ color: string;
+ /** Distance from the camera (px) where fog starts. */
+ start: number;
+ /** Distance where fog reaches `opacity`. */
+ end: number;
+ /** Max fog coverage 0..1. Default 1. */
+ opacity?: number;
+}
+
+export interface Environment3DOptions {
+ preset?: EnvironmentPreset;
+ /** Overrides the preset's sky (above), horizon and ground colors. */
+ top?: string;
+ horizon?: string;
+ bottom?: string;
+ /** Scales the whole environment's light. Default 1. */
+ intensity?: number;
+ /** Strength of the studio softboxes reflected in glossy surfaces. */
+ softboxes?: number;
+ /** Turns the environment around the vertical axis, degrees. */
+ rotation?: number;
+ /** Exposure in stops: +1 doubles brightness. Default 0. */
+ exposure?: number;
+ /** Default "neutral". */
+ toneMapping?: ToneMapping;
+ fog?: Environment3DFog;
+}
+
+interface PresetDef {
+ top: string;
+ horizon: string;
+ bottom: string;
+ softboxes: number;
+}
+
+export const ENVIRONMENT_PRESETS: Record<EnvironmentPreset, PresetDef> = {
+ /** Neutral photo studio: bright overhead softbox, grey sweep, dark floor bounce. */
+ studio: {
+ top: "#9da3ad",
+ horizon: "#5d626b",
+ bottom: "#202226",
+ softboxes: 3.5,
+ },
+ /** Bright, low-contrast daylight room. */
+ soft: {
+ top: "#f1efea",
+ horizon: "#cfcac1",
+ bottom: "#77726a",
+ softboxes: 1.2,
+ },
+ /** Warm horizon, cool sky. */
+ sunset: {
+ top: "#4d63a3",
+ horizon: "#ff9a5c",
+ bottom: "#3b2a28",
+ softboxes: 1.0,
+ },
+ /** Deep blue night with faint highlights; pair with emissive and point lights. */
+ night: {
+ top: "#121a3a",
+ horizon: "#1f2950",
+ bottom: "#05060b",
+ softboxes: 0.5,
+ },
+ /** Flat grey: image-based light without studio reflections. */
+ neutral: {
+ top: "#9a9a9a",
+ horizon: "#808080",
+ bottom: "#5c5c5c",
+ softboxes: 0,
+ },
+};
+
+const TONE_MAPPING_CODE: Record<ToneMapping, number> = {
+ none: 0,
+ aces: 1,
+ agx: 2,
+ neutral: 3,
+};
+
+function linearRgb(color: string): [number, number, number] {
+ const c = parseCssColor(color) ?? [0, 0, 0, 1];
+ const lin = (v: number) =>
+ v <= 0.04045 ? v / 12.92 : ((v + 0.055) / 1.055) ** 2.4;
+ return [lin(c[0]), lin(c[1]), lin(c[2])];
+}
+
+export interface ResolvedEnvironment {
+ top: [number, number, number];
+ horizon: [number, number, number];
+ bottom: [number, number, number];
+ softboxes: number;
+ rotationRad: number;
+ exposureMultiplier: number;
+ toneMapping: number;
+ fog?: {
+ color: [number, number, number];
+ start: number;
+ end: number;
+ opacity: number;
+ };
+}
+
+/** Linear radiance and shader codes for an environment. */
+export function resolveEnvironment(
+ opts: Environment3DOptions,
+): ResolvedEnvironment {
+ const preset = ENVIRONMENT_PRESETS[opts.preset ?? "studio"];
+ const k = opts.intensity ?? 1;
+ const scale = (c: [number, number, number]): [number, number, number] => [
+ c[0] * k,
+ c[1] * k,
+ c[2] * k,
+ ];
+ return {
+ top: scale(linearRgb(opts.top ?? preset.top)),
+ horizon: scale(linearRgb(opts.horizon ?? preset.horizon)),
+ bottom: scale(linearRgb(opts.bottom ?? preset.bottom)),
+ softboxes: (opts.softboxes ?? preset.softboxes) * k,
+ rotationRad: ((opts.rotation ?? 0) * Math.PI) / 180,
+ exposureMultiplier: 2 ** (opts.exposure ?? 0),
+ toneMapping: TONE_MAPPING_CODE[opts.toneMapping ?? "neutral"],
+ fog: opts.fog
+ ? {
+ color: linearRgb(opts.fog.color),
+ start: opts.fog.start,
+ end: opts.fog.end,
+ opacity: Math.max(0, Math.min(1, opts.fog.opacity ?? 1)),
+ }
+ : undefined,
+ };
+}
+
+/** Where a shadow-casting light's depth map lives, for the lit shaders. */
+export interface ShadowUniformInfo {
+ /** Light view-projection (world β†’ shadow clip). */
+ matrix: ArrayLike<number>;
+ /** Index of the casting light among the packed (non-ambient) lights. */
+ lightIndex: number;
+ /** Shadow map size in texels. */
+ mapSize: number;
+ /** Blur radius in texels. */
+ softness: number;
+ /** Depth bias in NDC. */
+ bias: number;
+ /** Pushes the lookup along the normal, world px. */
+ normalOffset: number;
+ /** 0 = invisible shadows, 1 = fully dark. */
+ darkness: number;
+}
diff --git a/packages/webgpu-renderers/src/math3d/index.ts b/packages/webgpu-renderers/src/math3d/index.ts
index 3d0b7cbc..ac4aafbc 100644
--- a/packages/webgpu-renderers/src/math3d/index.ts
+++ b/packages/webgpu-renderers/src/math3d/index.ts
@@ -1,5 +1,6 @@
-export * from "./vector3.js";
-export * from "./matrix4.js";
export * from "./camera3d.js";
-export * from "./path3d.js";
+export * from "./environment3d.js";
export * from "./light3d.js";
+export * from "./matrix4.js";
+export * from "./path3d.js";
+export * from "./vector3.js";
diff --git a/packages/webgpu-renderers/src/math3d/light3d.ts b/packages/webgpu-renderers/src/math3d/light3d.ts
index f4c81393..8833051c 100644
--- a/packages/webgpu-renderers/src/math3d/light3d.ts
+++ b/packages/webgpu-renderers/src/math3d/light3d.ts
@@ -5,6 +5,11 @@
* and binary uniform buffer packing for WGSL shaders.
*/
+import {
+ type Environment3DOptions,
+ resolveEnvironment,
+ type ShadowUniformInfo,
+} from "./environment3d.js";
import { type Mat4, Matrix4Math } from "./matrix4.js";
import { type Vec3, Vector3Math } from "./vector3.js";
@@ -14,7 +19,12 @@ export interface Light3DOptions {
id?: string;
lightType?: LightKind;
type?: LightKind;
- color?: string | [number, number, number] | [number, number, number, number] | { r: number; g: number; b: number; a?: number } | unknown;
+ color?:
+ | string
+ | [number, number, number]
+ | [number, number, number, number]
+ | { r: number; g: number; b: number; a?: number }
+ | unknown;
intensity?: number;
x?: number;
y?: number;
@@ -29,6 +39,12 @@ export interface Light3DOptions {
decay?: number; // Attenuation exponent (default 2.0)
angle?: number; // Outer cone half-angle in degrees (default 45)
penumbra?: number; // Softness fraction [0, 1] (default 0.2)
+ /** Directional or spot: render a shadow map (the first such light wins). */
+ castShadow?: boolean;
+ /** Shadow edge blur in shadow-map texels (default 2). */
+ shadowSoftness?: number;
+ /** 0..1, how dark shadows get (default 1). */
+ shadowDarkness?: number;
}
export interface GPULightData {
@@ -43,7 +59,11 @@ export interface GPULightData {
}
export const MAX_LIGHTS = 8;
-export const LIGHT_UNIFORM_SIZE_BYTES = 544; // 16 (ambient) + 16 (params) + 8 * 64 (lights)
+/**
+ * 16 (ambient) + 16 (params) + 8 * 64 (lights), then the physical-shading
+ * block appended after them: 8 vec4 (128) + shadow matrix (64).
+ */
+export const LIGHT_UNIFORM_SIZE_BYTES = 544 + 128 + 64;
export interface LightsUniformData {
ambientColor: [number, number, number, number];
@@ -55,7 +75,12 @@ export interface LightsUniformData {
* Parses CSS / hex / rgb color strings, arrays, or color objects into normalized linear [r, g, b] floats in [0, 1].
*/
export function parseColorToRgb(
- color?: string | [number, number, number] | [number, number, number, number] | { r: number; g: number; b: number; a?: number } | unknown,
+ color?:
+ | string
+ | [number, number, number]
+ | [number, number, number, number]
+ | { r: number; g: number; b: number; a?: number }
+ | unknown,
): [number, number, number] {
if (!color) return [1, 1, 1];
if (Array.isArray(color)) {
@@ -89,15 +114,23 @@ export function parseColorToRgb(
}
}
- const rgbMatch = str.match(/rgba?\s*\(\s*([\d.]+)\s*,\s*([\d.]+)\s*,\s*([\d.]+)/);
+ const rgbMatch = str.match(
+ /rgba?\s*\(\s*([\d.]+)\s*,\s*([\d.]+)\s*,\s*([\d.]+)/,
+ );
if (rgbMatch) {
const r = Number.parseFloat(rgbMatch[1]) / 255;
const g = Number.parseFloat(rgbMatch[2]) / 255;
const b = Number.parseFloat(rgbMatch[3]) / 255;
- return [Math.max(0, Math.min(1, r)), Math.max(0, Math.min(1, g)), Math.max(0, Math.min(1, b))];
+ return [
+ Math.max(0, Math.min(1, r)),
+ Math.max(0, Math.min(1, g)),
+ Math.max(0, Math.min(1, b)),
+ ];
}
- const hslMatch = str.match(/hsla?\s*\(\s*([\d.]+)\s*,\s*([\d.]+)%?\s*,\s*([\d.]+)%?/);
+ const hslMatch = str.match(
+ /hsla?\s*\(\s*([\d.]+)\s*,\s*([\d.]+)%?\s*,\s*([\d.]+)%?/,
+ );
if (hslMatch) {
const h = Number.parseFloat(hslMatch[1]) / 360;
const s = Number.parseFloat(hslMatch[2]) / 100;
@@ -136,10 +169,22 @@ export const Light3D = {
}
// Transpose of the inverted matrix
const normMat: Mat4 = [
- inv[0], inv[4], inv[8], 0,
- inv[1], inv[5], inv[9], 0,
- inv[2], inv[6], inv[10], 0,
- 0, 0, 0, 1,
+ inv[0],
+ inv[4],
+ inv[8],
+ 0,
+ inv[1],
+ inv[5],
+ inv[9],
+ 0,
+ inv[2],
+ inv[6],
+ inv[10],
+ 0,
+ 0,
+ 0,
+ 0,
+ 1,
];
return normMat;
},
@@ -202,10 +247,15 @@ export const Light3D = {
/**
* Packs active scene lights into a Float32Array matching WGSL LightsUniforms layout.
*/
- packLightsUniforms(lights: Light3DOptions[]): Float32Array {
+ packLightsUniforms(
+ lights: Light3DOptions[],
+ environment?: Environment3DOptions | null,
+ shadow?: ShadowUniformInfo | null,
+ ): Float32Array {
const buffer = new Float32Array(LIGHT_UNIFORM_SIZE_BYTES / 4);
+ Light3D.packPhysicalBlock(buffer, lights ?? [], environment, shadow);
- if (!lights || lights.length === 0) {
+ if ((!lights || lights.length === 0) && !environment) {
// No lights: params.y = 0.0 indicates unlit mode
buffer[4] = 0.0; // lightCount = 0
buffer[5] = 0.0; // hasLights = 0.0 (unlit)
@@ -242,8 +292,7 @@ export const Light3D = {
// Offset 4: params (vec4<f32>)
buffer[4] = directLights.length;
buffer[5] = 1.0; // hasLights = 1.0 (lit mode)
- buffer[6] = 0.0;
- buffer[7] = 0.0;
+ // z = physical shading, w = tone mapping (set by packPhysicalBlock)
// Offset 8 onwards: array<GPULightData, 8> (each light = 16 floats = 64 bytes)
let offset = 8;
@@ -281,4 +330,73 @@ export const Light3D = {
return buffer;
},
+
+ /** Index (among non-ambient lights) of the light that should cast shadows, or -1. */
+ shadowCasterIndex(lights: Light3DOptions[]): number {
+ let index = 0;
+ for (const l of lights) {
+ const kind = l.lightType ?? l.type;
+ if (kind === "ambient") continue;
+ if (index >= MAX_LIGHTS) break;
+ if (l.castShadow && (kind === "directional" || kind === "spot")) {
+ return index;
+ }
+ index++;
+ }
+ return -1;
+ },
+
+ /** Fills params.zw and the appended physical-shading block (offset 136 on). */
+ packPhysicalBlock(
+ buffer: Float32Array,
+ lights: Light3DOptions[],
+ environment?: Environment3DOptions | null,
+ shadow?: ShadowUniformInfo | null,
+ ): void {
+ const base = 8 + MAX_LIGHTS * 16; // first float after the lights array
+ let ambR = 0;
+ let ambG = 0;
+ let ambB = 0;
+ for (const l of lights) {
+ if ((l.lightType ?? l.type) !== "ambient") continue;
+ const [r, g, b] = parseColorToRgb(l.color);
+ const k = l.intensity ?? 1.0;
+ const lin = (v: number) =>
+ v <= 0.04045 ? v / 12.92 : ((v + 0.055) / 1.055) ** 2.4;
+ ambR += lin(r) * k;
+ ambG += lin(g) * k;
+ ambB += lin(b) * k;
+ }
+ buffer[base + 0] = ambR;
+ buffer[base + 1] = ambG;
+ buffer[base + 2] = ambB;
+ buffer[base + 3] = 1; // exposure multiplier
+
+ if (environment) {
+ const env = resolveEnvironment(environment);
+ buffer[6] = 1; // physical shading
+ buffer[7] = env.toneMapping;
+ buffer[base + 3] = env.exposureMultiplier;
+ buffer.set([...env.top, env.rotationRad], base + 4);
+ buffer.set([...env.horizon, env.softboxes], base + 8);
+ buffer.set([...env.bottom, 1], base + 12);
+ if (env.fog) {
+ buffer.set([...env.fog.color, 1], base + 16);
+ buffer.set([env.fog.start, env.fog.end, env.fog.opacity, 0], base + 20);
+ }
+ }
+
+ if (shadow && shadow.lightIndex >= 0) {
+ buffer.set(
+ [1, shadow.lightIndex, shadow.softness, shadow.mapSize],
+ base + 24,
+ );
+ buffer.set(
+ [shadow.bias, shadow.normalOffset, shadow.darkness, 0],
+ base + 28,
+ );
+ for (let i = 0; i < 16; i++)
+ buffer[base + 32 + i] = shadow.matrix[i] ?? 0;
+ }
+ },
};
diff --git a/packages/webgpu-renderers/src/renderer2d/effect-pipeline.ts b/packages/webgpu-renderers/src/renderer2d/effect-pipeline.ts
index 1b2fa9a0..c2ca7a16 100644
--- a/packages/webgpu-renderers/src/renderer2d/effect-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer2d/effect-pipeline.ts
@@ -153,6 +153,8 @@ const BLEND_MODE_MAP: Record<BlendMode, number> = {
lighter: 23,
copy: 24,
xor: 25,
+ "luma-in": 26,
+ "luma-out": 27,
};
export class EffectPipeline {
@@ -261,7 +263,10 @@ export class EffectPipeline {
});
const motionBlurPipelineLayout = device.createPipelineLayout({
- bindGroupLayouts: [this.motionBlurUniformLayout, this.singleTextureLayout],
+ bindGroupLayouts: [
+ this.motionBlurUniformLayout,
+ this.singleTextureLayout,
+ ],
});
const motionBlurModule = device.createShaderModule({
diff --git a/packages/webgpu-renderers/src/renderer2d/gradient-uniforms.ts b/packages/webgpu-renderers/src/renderer2d/gradient-uniforms.ts
new file mode 100644
index 00000000..81d9d550
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer2d/gradient-uniforms.ts
@@ -0,0 +1,82 @@
+import {
+ gradientStopsToOklab,
+ limitGradientStops,
+ type ParsedGradient,
+ type ParsedGradientStop,
+ parseCssGradient,
+} from "@framefields/core";
+import { GRADIENT_UNIFORM_FLOATS } from "../shaders/gradient.js";
+
+export interface GradientBox {
+ x: number;
+ y: number;
+ width: number;
+ height: number;
+}
+
+const GRADIENT_TYPE_CODE = { linear: 1, radial: 2, conic: 3 } as const;
+
+/** The parsed gradient when `value` is a CSS gradient string. */
+export function asGradient(value: unknown): ParsedGradient | null {
+ return typeof value === "string" && value.includes("gradient(")
+ ? parseCssGradient(value)
+ : null;
+}
+
+/**
+ * Packs a gradient laid over `box` (local px) into `out` at `offset`
+ * (`GRADIENT_UNIFORM_FLOATS` floats). Pass `null` to clear (solid fill).
+ */
+export function packGradientUniforms(
+ gradient: ParsedGradient | null,
+ box: GradientBox,
+ out: Float32Array,
+ offset: number,
+): void {
+ out.fill(0, offset, offset + GRADIENT_UNIFORM_FLOATS);
+ if (!gradient) return;
+ const stops: ParsedGradientStop[] = limitGradientStops(gradient.stops);
+ const w = Math.max(box.width, 1e-3);
+ const h = Math.max(box.height, 1e-3);
+ const cx = box.x + gradient.center[0] * w;
+ const cy = box.y + gradient.center[1] * h;
+ out[offset] = GRADIENT_TYPE_CODE[gradient.type];
+ out[offset + 1] = stops.length;
+ out[offset + 4] = gradient.type === "linear" ? box.x + w / 2 : cx;
+ out[offset + 5] = gradient.type === "linear" ? box.y + h / 2 : cy;
+ if (gradient.type === "linear") {
+ // CSS: 0deg points up, 90deg right; the line spans corner to corner.
+ const r = (gradient.angle * Math.PI) / 180;
+ const dx = Math.sin(r);
+ const dy = -Math.cos(r);
+ const len = Math.abs(w * dx) + Math.abs(h * dy);
+ out[offset + 6] = dx / len;
+ out[offset + 7] = dy / len;
+ } else if (gradient.type === "radial") {
+ let rx: number;
+ let ry: number;
+ if (gradient.radius) {
+ [rx, ry] = gradient.radius;
+ } else {
+ // farthest-corner, as CSS sizes radial gradients by default.
+ const fx = Math.max(cx - box.x, box.x + w - cx);
+ const fy = Math.max(cy - box.y, box.y + h - cy);
+ if (gradient.shape === "circle") {
+ rx = ry = Math.hypot(fx, fy);
+ } else {
+ rx = fx * Math.SQRT2;
+ ry = fy * Math.SQRT2;
+ }
+ }
+ out[offset + 6] = rx;
+ out[offset + 7] = ry;
+ } else {
+ out[offset + 6] = (gradient.from * Math.PI) / 180;
+ }
+ stops.forEach((s, i) => {
+ out[offset + 8 + i] = s.offset;
+ });
+ gradientStopsToOklab(stops).forEach((lab, i) => {
+ out.set(lab, offset + 16 + i * 4);
+ });
+}
diff --git a/packages/webgpu-renderers/src/renderer2d/index.ts b/packages/webgpu-renderers/src/renderer2d/index.ts
index b751a3db..7c291323 100644
--- a/packages/webgpu-renderers/src/renderer2d/index.ts
+++ b/packages/webgpu-renderers/src/renderer2d/index.ts
@@ -19,8 +19,8 @@ import { SolidPipeline } from "./solid-pipeline.js";
import { TransformStack } from "./transform-stack.js";
export * from "./path-geometry.js";
-export * from "./shape-fill-pipeline.js";
export type { PathDrawOpts } from "./path-pipeline.js";
+export * from "./shape-fill-pipeline.js";
export interface Rect {
x: number;
@@ -71,7 +71,11 @@ export type BlendMode =
| "destination-atop"
| "lighter"
| "copy"
- | "xor";
+ | "xor"
+ /** Keeps the base where the overlay is bright (luma matte). */
+ | "luma-in"
+ /** Keeps the base where the overlay is dark (inverted luma matte). */
+ | "luma-out";
export interface DrawOpts {
opacity?: number;
diff --git a/packages/webgpu-renderers/src/renderer2d/shape-fill-pipeline.ts b/packages/webgpu-renderers/src/renderer2d/shape-fill-pipeline.ts
index 4a741d0a..80248dc4 100644
--- a/packages/webgpu-renderers/src/renderer2d/shape-fill-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer2d/shape-fill-pipeline.ts
@@ -1,6 +1,9 @@
+import { gradientStopsToOklab } from "@framefields/core";
import { parseColor } from "../color.js";
+import { GRADIENT_UNIFORM_FLOATS } from "../shaders/gradient.js";
import { shapeFillWgsl } from "../shaders/shape-fill.js";
import { BufferPool } from "./buffer-pool.js";
+import { asGradient, packGradientUniforms } from "./gradient-uniforms.js";
import type { Color, DrawOpts, Rect } from "./index.js";
import type { TransformStack } from "./transform-stack.js";
@@ -8,6 +11,7 @@ export type ShapeFillType = "solid" | "linear" | "radial" | "none";
export interface ShapeFillConfig {
type?: ShapeFillType;
+ /** A color, or a CSS gradient string (linear/radial/conic, any stops) laid over the shape's bounds. */
color?: Color | string;
color2?: Color | string;
startX?: number;
@@ -25,7 +29,7 @@ export class ShapeFillPipeline {
private uniformLayout: GPUBindGroupLayout;
private uniformPool: BufferPool;
private vertexPool: BufferPool;
- private uniformData = new Float32Array(28);
+ private uniformData = new Float32Array(20 + GRADIENT_UNIFORM_FLOATS);
constructor(device: GPUDevice, format: GPUTextureFormat) {
this.device = device;
@@ -119,40 +123,18 @@ export class ShapeFillPipeline {
opts?.transform,
);
- let fillTypeNum = 0;
- if (fill.type === "linear") fillTypeNum = 1;
- else if (fill.type === "radial") fillTypeNum = 2;
-
- this.uniformData[15] = fillTypeNum;
-
- const c1 = parseColor(fill.color ?? "#ffffff");
- this.uniformData[16] = c1.r;
- this.uniformData[17] = c1.g;
- this.uniformData[18] = c1.b;
- this.uniformData[19] = c1.a;
-
- const c2 = parseColor(fill.color2 ?? fill.color ?? "#ffffff");
- this.uniformData[20] = c2.r;
- this.uniformData[21] = c2.g;
- this.uniformData[22] = c2.b;
- this.uniformData[23] = c2.a;
-
- if (fillTypeNum === 1) {
- this.uniformData[24] = fill.startX ?? bounds.x;
- this.uniformData[25] = fill.startY ?? bounds.y;
- this.uniformData[26] = fill.endX ?? bounds.x + bounds.width;
- this.uniformData[27] = fill.endY ?? bounds.y + bounds.height;
- } else if (fillTypeNum === 2) {
- this.uniformData[24] = fill.cx ?? bounds.x + bounds.width / 2;
- this.uniformData[25] = fill.cy ?? bounds.y + bounds.height / 2;
- this.uniformData[26] =
- fill.radius ?? Math.max(bounds.width, bounds.height) / 2;
- this.uniformData[27] = 0;
+ this.uniformData[15] = 0;
+ const css = asGradient(fill.color);
+ if (css) {
+ this.uniformData.fill(0, 16, 20);
+ packGradientUniforms(css, bounds, this.uniformData, 20);
} else {
- this.uniformData[24] = 0;
- this.uniformData[25] = 0;
- this.uniformData[26] = 0;
- this.uniformData[27] = 0;
+ const c1 = parseColor(fill.color ?? "#ffffff");
+ this.uniformData[16] = c1.r;
+ this.uniformData[17] = c1.g;
+ this.uniformData[18] = c1.b;
+ this.uniformData[19] = c1.a;
+ this.packTwoColorGradient(fill, bounds, c1);
}
const uniformBuffer = this.uniformPool.getBuffer(
@@ -173,6 +155,55 @@ export class ShapeFillPipeline {
pass.draw(vertices.length / 2);
}
+ /** The `type: "linear" | "radial"` two-color fill (color β†’ color2). */
+ private packTwoColorGradient(
+ fill: ShapeFillConfig,
+ bounds: Rect,
+ c1: Color,
+ ): void {
+ const o = 20;
+ this.uniformData.fill(0, o, o + GRADIENT_UNIFORM_FLOATS);
+ if (fill.type !== "linear" && fill.type !== "radial") return;
+ const c2 = parseColor(fill.color2 ?? fill.color ?? "#ffffff");
+ const d = this.uniformData;
+ if (fill.type === "linear") {
+ const x0 = fill.startX ?? bounds.x;
+ const y0 = fill.startY ?? bounds.y;
+ const x1 = fill.endX ?? bounds.x + bounds.width;
+ const y1 = fill.endY ?? bounds.y + bounds.height;
+ const dx = x1 - x0;
+ const dy = y1 - y0;
+ const lenSq = Math.max(dx * dx + dy * dy, 1e-6);
+ d.set(
+ [1, 2, 0, 0, (x0 + x1) / 2, (y0 + y1) / 2, dx / lenSq, dy / lenSq],
+ o,
+ );
+ } else {
+ const r = fill.radius ?? Math.max(bounds.width, bounds.height) / 2;
+ d.set(
+ [
+ 2,
+ 2,
+ 0,
+ 0,
+ fill.cx ?? bounds.x + bounds.width / 2,
+ fill.cy ?? bounds.y + bounds.height / 2,
+ r,
+ r,
+ ],
+ o,
+ );
+ }
+ d[o + 8] = 0;
+ d[o + 9] = 1;
+ const lab = gradientStopsToOklab([
+ { offset: 0, color: [c1.r, c1.g, c1.b, c1.a] },
+ { offset: 1, color: [c2.r, c2.g, c2.b, c2.a] },
+ ]);
+ d.set(lab[0], o + 16);
+ d.set(lab[1], o + 20);
+ }
+
destroy(): void {
this.uniformPool.destroy();
this.vertexPool.destroy();
diff --git a/packages/webgpu-renderers/src/renderer2d/solid-pipeline.ts b/packages/webgpu-renderers/src/renderer2d/solid-pipeline.ts
index 845c2465..68a70ac1 100644
--- a/packages/webgpu-renderers/src/renderer2d/solid-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer2d/solid-pipeline.ts
@@ -1,9 +1,11 @@
import { parseColor } from "../color.js";
+import { GRADIENT_UNIFORM_FLOATS } from "../shaders/gradient.js";
import { solidWgsl } from "../shaders/solid.js";
import { BufferPool } from "./buffer-pool.js";
+import { asGradient, packGradientUniforms } from "./gradient-uniforms.js";
import type { Color, DrawOpts, Rect } from "./index.js";
-import { UniformBindGroupCache } from "./uniform-bind-group-cache.js";
import type { TransformStack } from "./transform-stack.js";
+import { UniformBindGroupCache } from "./uniform-bind-group-cache.js";
export class SolidPipeline {
private device: GPUDevice;
@@ -12,7 +14,7 @@ export class SolidPipeline {
private uniformPool: BufferPool;
private vertexPool: BufferPool;
private uniformBindGroupCache = new UniformBindGroupCache();
- private uniformData = new Float32Array(24);
+ private uniformData = new Float32Array(24 + GRADIENT_UNIFORM_FLOATS);
private vertexData = new Float32Array(8);
// Correct layout for uniformData:
// transformCol0: vec4 (4)
@@ -21,7 +23,8 @@ export class SolidPipeline {
// params: vec4 (4)
// color: vec4 (4)
// rect: vec4 (4)
- // Total: 24 floats
+ // gradient: GradientUniforms (48)
+ // Total: 72 floats
constructor(device: GPUDevice, format: GPUTextureFormat) {
this.device = device;
@@ -104,7 +107,9 @@ export class SolidPipeline {
cornerRadius = 0,
opts?: DrawOpts,
): void {
- const c = parseColor(color);
+ // A CSS gradient string paints the rect with that gradient.
+ const gradient = asGradient(color);
+ const c = gradient ? { r: 0, g: 0, b: 0, a: 1 } : parseColor(color);
const opacity = opts?.opacity ?? 1.0;
transformStack.packIntoBuffer(
@@ -126,6 +131,7 @@ export class SolidPipeline {
this.uniformData[21] = rect.y;
this.uniformData[22] = rect.width;
this.uniformData[23] = rect.height;
+ packGradientUniforms(gradient, rect, this.uniformData, 24);
const uniformBuffer = this.uniformPool.getBuffer(
this.device,
diff --git a/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.ts
new file mode 100644
index 00000000..6ddcaa9f
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.ts
@@ -0,0 +1,433 @@
+/**
+ * Bloom: light that spills past bright pixels, the way a lens and film do.
+ *
+ * A mip chain at half, quarter, … resolution: a soft-knee threshold picks the
+ * bright parts (emissive surfaces, hot speculars), 13-tap downsamples spread
+ * them, tent-filtered upsamples add every level back up (wide and tight glow
+ * together), and the result is added over the source. Works on premultiplied
+ * color: glow over a transparent pixel raises its alpha, so a bloomed 3D
+ * layer still glows when composited over the 2D layers below.
+ */
+
+export interface BloomOptions {
+ /** Glow strength. Default 0.2. */
+ intensity?: number;
+ /**
+ * Brightness where glow starts. With an HDR source (3D scenes) it is scene
+ * radiance, default 2.5 = only light well past white (emissive ≳ 3,
+ * hot speculars). Without one it is display brightness, default 0.85.
+ */
+ threshold?: number;
+ /** Soft transition below the threshold, 0..1. Default 0.4. */
+ knee?: number;
+ /** Glow spread: mip levels used, 1..8. Default 6. */
+ radius?: number;
+ /** Tints the glow (CSS-free rgb 0..1). Default white. */
+ tint?: [number, number, number];
+}
+
+const bloomWgsl = /* wgsl */ `
+struct BloomUniforms {
+ texel : vec2<f32>, // 1 / source size
+ threshold : f32,
+ knee : f32,
+ intensity : f32,
+ mode : f32,
+ _pad : vec2<f32>,
+ tint : vec4<f32>,
+};
+
+@group(0) @binding(0) var<uniform> u : BloomUniforms;
+@group(0) @binding(1) var samp : sampler;
+@group(0) @binding(2) var src : texture_2d<f32>;
+@group(0) @binding(3) var base : texture_2d<f32>;
+
+struct VsOut {
+ @builtin(position) pos : vec4<f32>,
+ @location(0) uv : vec2<f32>,
+};
+
+@vertex
+fn vs(@builtin(vertex_index) i : u32) -> VsOut {
+ let p = vec2<f32>(f32((i << 1u) & 2u), f32(i & 2u));
+ var o : VsOut;
+ o.pos = vec4<f32>(p * 2.0 - 1.0, 0.0, 1.0);
+ o.uv = vec2<f32>(p.x, 1.0 - p.y);
+ return o;
+}
+
+fn s(uv : vec2<f32>) -> vec3<f32> {
+ return textureSampleLevel(src, samp, uv, 0.0).rgb;
+}
+
+// Jimenez's 13-tap downsample (Call of Duty: AW): stable, no fireflies.
+fn down13(uv : vec2<f32>) -> vec3<f32> {
+ let t = u.texel;
+ let a = s(uv + t * vec2<f32>(-2.0, -2.0));
+ let b = s(uv + t * vec2<f32>(0.0, -2.0));
+ let c = s(uv + t * vec2<f32>(2.0, -2.0));
+ let d = s(uv + t * vec2<f32>(-2.0, 0.0));
+ let e = s(uv);
+ let f = s(uv + t * vec2<f32>(2.0, 0.0));
+ let g = s(uv + t * vec2<f32>(-2.0, 2.0));
+ let h = s(uv + t * vec2<f32>(0.0, 2.0));
+ let i = s(uv + t * vec2<f32>(2.0, 2.0));
+ let j = s(uv + t * vec2<f32>(-1.0, -1.0));
+ let k = s(uv + t * vec2<f32>(1.0, -1.0));
+ let l = s(uv + t * vec2<f32>(-1.0, 1.0));
+ let m = s(uv + t * vec2<f32>(1.0, 1.0));
+ return e * 0.125 + (a + c + g + i) * 0.03125 + (b + d + f + h) * 0.0625 + (j + k + l + m) * 0.125;
+}
+
+// Soft-knee threshold (keeps hue, fades in below the threshold).
+fn prefilter(c : vec3<f32>) -> vec3<f32> {
+ let br = max(c.r, max(c.g, c.b));
+ let kneeW = u.threshold * u.knee + 1e-4;
+ var soft = clamp(br - u.threshold + kneeW, 0.0, 2.0 * kneeW);
+ soft = soft * soft / (4.0 * kneeW);
+ let w = max(soft, br - u.threshold) / max(br, 1e-4);
+ return c * max(w, 0.0);
+}
+
+fn srgbToLinear(c : vec3<f32>) -> vec3<f32> {
+ let lo = c / 12.92;
+ let hi = pow((max(c, vec3<f32>(0.0)) + 0.055) / 1.055, vec3<f32>(2.4));
+ return select(hi, lo, c <= vec3<f32>(0.04045));
+}
+
+fn linearToSrgb(c : vec3<f32>) -> vec3<f32> {
+ let x = clamp(c, vec3<f32>(0.0), vec3<f32>(1.0));
+ let lo = x * 12.92;
+ let hi = 1.055 * pow(x, vec3<f32>(1.0 / 2.4)) - 0.055;
+ return select(hi, lo, x <= vec3<f32>(0.0031308));
+}
+
+// mode 0: the source is linear scene radiance; mode 1: display-encoded color.
+@fragment
+fn fs_prefilter(in : VsOut) -> @location(0) vec4<f32> {
+ var c = down13(in.uv);
+ if (u.mode > 0.5) {
+ c = srgbToLinear(c);
+ }
+ return vec4<f32>(prefilter(c), 1.0);
+}
+
+@fragment
+fn fs_down(in : VsOut) -> @location(0) vec4<f32> {
+ return vec4<f32>(down13(in.uv), 1.0);
+}
+
+// 3x3 tent upsample; added onto the next-larger level by the blend state.
+@fragment
+fn fs_up(in : VsOut) -> @location(0) vec4<f32> {
+ let t = u.texel;
+ var c = s(in.uv) * 4.0;
+ c += (s(in.uv + vec2<f32>(-t.x, 0.0)) + s(in.uv + vec2<f32>(t.x, 0.0)) + s(in.uv + vec2<f32>(0.0, -t.y)) + s(in.uv + vec2<f32>(0.0, t.y))) * 2.0;
+ c += s(in.uv + vec2<f32>(-t.x, -t.y)) + s(in.uv + vec2<f32>(t.x, -t.y)) + s(in.uv + vec2<f32>(-t.x, t.y)) + s(in.uv + vec2<f32>(t.x, t.y));
+ return vec4<f32>(c / 16.0, 1.0);
+}
+
+// Adds the glow in linear light over the premultiplied display image.
+@fragment
+fn fs_composite(in : VsOut) -> @location(0) vec4<f32> {
+ let orig = textureSampleLevel(base, samp, in.uv, 0.0);
+ let glow = max(textureSampleLevel(src, samp, in.uv, 0.0).rgb * u.intensity * u.tint.rgb, vec3<f32>(0.0));
+ var lin = vec3<f32>(0.0);
+ if (orig.a > 0.0001) {
+ lin = srgbToLinear(orig.rgb / orig.a) * orig.a;
+ }
+ let sum = lin + glow;
+ let a = clamp(max(orig.a, max(glow.r, max(glow.g, glow.b))), 0.0, 1.0);
+ if (a <= 0.0001) {
+ return vec4<f32>(0.0);
+ }
+ return vec4<f32>(linearToSrgb(sum / a) * a, a);
+}
+`;
+
+import { BufferPool } from "../renderer2d/buffer-pool.js";
+
+const MIP_FORMAT: GPUTextureFormat = "rgba16float";
+
+export class BloomPipeline {
+ private device: GPUDevice;
+ private layout: GPUBindGroupLayout;
+ private prefilterPipe: GPURenderPipeline;
+ private downPipe: GPURenderPipeline;
+ private upPipe: GPURenderPipeline;
+ private compositePipes = new Map<GPUTextureFormat, GPURenderPipeline>();
+ private module: GPUShaderModule;
+ private pipelineLayout: GPUPipelineLayout;
+ private sampler: GPUSampler;
+ private mips: GPUTexture[] = [];
+ private mipKey = "";
+ private placeholder: GPUTexture;
+ private uniformPool = new BufferPool(
+ 48,
+ GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ );
+
+ constructor(device: GPUDevice) {
+ this.device = device;
+ this.module = device.createShaderModule({
+ label: "bloom.wgsl",
+ code: bloomWgsl,
+ });
+ this.layout = device.createBindGroupLayout({
+ label: "BloomLayout",
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.FRAGMENT,
+ buffer: { type: "uniform" },
+ },
+ {
+ binding: 1,
+ visibility: GPUShaderStage.FRAGMENT,
+ sampler: { type: "filtering" },
+ },
+ {
+ binding: 2,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float" },
+ },
+ {
+ binding: 3,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float" },
+ },
+ ],
+ });
+ this.pipelineLayout = device.createPipelineLayout({
+ bindGroupLayouts: [this.layout],
+ });
+ const make = (entryPoint: string, additive: boolean) =>
+ device.createRenderPipeline({
+ label: `Bloom:${entryPoint}`,
+ layout: this.pipelineLayout,
+ vertex: { module: this.module, entryPoint: "vs" },
+ fragment: {
+ module: this.module,
+ entryPoint,
+ targets: [
+ {
+ format: MIP_FORMAT,
+ ...(additive && {
+ blend: {
+ color: {
+ srcFactor: "one",
+ dstFactor: "one",
+ operation: "add",
+ },
+ alpha: {
+ srcFactor: "one",
+ dstFactor: "one",
+ operation: "add",
+ },
+ },
+ }),
+ },
+ ],
+ },
+ primitive: { topology: "triangle-list" },
+ });
+ this.prefilterPipe = make("fs_prefilter", false);
+ this.downPipe = make("fs_down", false);
+ this.upPipe = make("fs_up", true);
+ this.sampler = device.createSampler({
+ label: "BloomSampler",
+ magFilter: "linear",
+ minFilter: "linear",
+ addressModeU: "clamp-to-edge",
+ addressModeV: "clamp-to-edge",
+ });
+ this.placeholder = device.createTexture({
+ size: { width: 1, height: 1 },
+ format: MIP_FORMAT,
+ usage: GPUTextureUsage.TEXTURE_BINDING,
+ });
+ }
+
+ private compositePipe(format: GPUTextureFormat): GPURenderPipeline {
+ let pipe = this.compositePipes.get(format);
+ if (!pipe) {
+ pipe = this.device.createRenderPipeline({
+ label: "Bloom:composite",
+ layout: this.pipelineLayout,
+ vertex: { module: this.module, entryPoint: "vs" },
+ fragment: {
+ module: this.module,
+ entryPoint: "fs_composite",
+ targets: [{ format }],
+ },
+ primitive: { topology: "triangle-list" },
+ });
+ this.compositePipes.set(format, pipe);
+ }
+ return pipe;
+ }
+
+ private ensureMips(width: number, height: number, levels: number): void {
+ const key = `${width}x${height}x${levels}`;
+ if (key === this.mipKey) return;
+ for (const t of this.mips) t.destroy();
+ this.mips = [];
+ let w = width;
+ let h = height;
+ for (let i = 0; i < levels; i++) {
+ w = Math.max(1, Math.floor(w / 2));
+ h = Math.max(1, Math.floor(h / 2));
+ this.mips.push(
+ this.device.createTexture({
+ label: `BloomMip${i}`,
+ size: { width: w, height: h },
+ format: MIP_FORMAT,
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
+ }),
+ );
+ if (w === 1 && h === 1) break;
+ }
+ this.mipKey = key;
+ }
+
+ private uniforms(
+ texelW: number,
+ texelH: number,
+ opts: Required<Omit<BloomOptions, "radius">> & { mode: number },
+ ): GPUBuffer {
+ const data = new Float32Array([
+ texelW,
+ texelH,
+ opts.threshold,
+ opts.knee,
+ opts.intensity,
+ opts.mode,
+ 0,
+ 0,
+ opts.tint[0],
+ opts.tint[1],
+ opts.tint[2],
+ 1,
+ ]);
+ return this.uniformPool.getBuffer(this.device, data);
+ }
+
+ private pass(
+ encoder: GPUCommandEncoder,
+ pipe: GPURenderPipeline,
+ target: GPUTextureView,
+ src: GPUTexture,
+ base: GPUTexture,
+ uniform: GPUBuffer,
+ load: GPULoadOp,
+ ): void {
+ const bindGroup = this.device.createBindGroup({
+ layout: this.layout,
+ entries: [
+ { binding: 0, resource: { buffer: uniform } },
+ { binding: 1, resource: this.sampler },
+ { binding: 2, resource: src.createView() },
+ { binding: 3, resource: base.createView() },
+ ],
+ });
+ const pass = encoder.beginRenderPass({
+ colorAttachments: [
+ {
+ view: target,
+ loadOp: load,
+ storeOp: "store",
+ clearValue: { r: 0, g: 0, b: 0, a: 0 },
+ },
+ ],
+ });
+ pass.setPipeline(pipe);
+ pass.setBindGroup(0, bindGroup);
+ pass.draw(3);
+ pass.end();
+ }
+
+ /**
+ * Writes `source` (premultiplied, display-encoded) plus its bloom into
+ * `dest` (same size). `hdrSource`, when given, is linear scene radiance
+ * of the same image: glow is taken from it instead of from `source`.
+ */
+ apply(
+ encoder: GPUCommandEncoder,
+ source: GPUTexture,
+ dest: GPUTexture,
+ options: BloomOptions = {},
+ hdrSource?: GPUTexture,
+ ): void {
+ const display = !hdrSource;
+ const toLinear = (v: number) =>
+ v <= 0.04045 ? v / 12.92 : ((v + 0.055) / 1.055) ** 2.4;
+ const threshold = options.threshold ?? (display ? 0.85 : 2.5);
+ const opts = {
+ intensity: options.intensity ?? 0.2,
+ threshold: display ? toLinear(threshold) : threshold,
+ knee: options.knee ?? 0.4,
+ tint: options.tint ?? ([1, 1, 1] as [number, number, number]),
+ mode: display ? 1 : 0,
+ };
+ const levels = Math.max(1, Math.min(8, Math.round(options.radius ?? 6)));
+ this.ensureMips(source.width, source.height, levels);
+ const mips = this.mips;
+ const u = (w: number, h: number) => this.uniforms(1 / w, 1 / h, opts);
+
+ const glowSource = hdrSource ?? source;
+ this.pass(
+ encoder,
+ this.prefilterPipe,
+ mips[0].createView(),
+ glowSource,
+ this.placeholder,
+ u(glowSource.width, glowSource.height),
+ "clear",
+ );
+ for (let i = 1; i < mips.length; i++) {
+ const prev = mips[i - 1];
+ this.pass(
+ encoder,
+ this.downPipe,
+ mips[i].createView(),
+ prev,
+ this.placeholder,
+ u(prev.width, prev.height),
+ "clear",
+ );
+ }
+ for (let i = mips.length - 1; i > 0; i--) {
+ const small = mips[i];
+ this.pass(
+ encoder,
+ this.upPipe,
+ mips[i - 1].createView(),
+ small,
+ this.placeholder,
+ u(small.width, small.height),
+ "load",
+ );
+ }
+ this.pass(
+ encoder,
+ this.compositePipe(dest.format),
+ dest.createView(),
+ mips[0],
+ source,
+ u(mips[0].width, mips[0].height),
+ "clear",
+ );
+ }
+
+ /** Call once per frame before `apply`: recycles the per-pass uniforms. */
+ resetPools(): void {
+ this.uniformPool.reset();
+ }
+
+ destroy(): void {
+ for (const t of this.mips) t.destroy();
+ this.placeholder.destroy();
+ this.uniformPool.destroy();
+ }
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/index.ts b/packages/webgpu-renderers/src/renderer3d/index.ts
index 190adcc0..d61f52cf 100644
--- a/packages/webgpu-renderers/src/renderer3d/index.ts
+++ b/packages/webgpu-renderers/src/renderer3d/index.ts
@@ -1,10 +1,15 @@
export * from "./audio-mesh-deform-pipeline.js";
+export * from "./bloom-pipeline.js";
export * from "./glass-pipeline.js";
export * from "./mesh3d-pipeline.js";
export * from "./motion-blur-pipeline.js";
export * from "./primitive-geometry.js";
export * from "./quad3d-pipeline.js";
export * from "./renderer3d.js";
+export * from "./shadow-depth.js";
+export * from "./shadow-frustum.js";
export * from "./shadow-pipeline.js";
export * from "./ssao-pipeline.js";
+export * from "./surface-params.js";
export * from "./text-extruder.js";
+export * from "./matte3d.js";
diff --git a/packages/webgpu-renderers/src/renderer3d/matte3d.ts b/packages/webgpu-renderers/src/renderer3d/matte3d.ts
new file mode 100644
index 00000000..8de9a381
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/matte3d.ts
@@ -0,0 +1,32 @@
+/** The per-draw matte binding of the 3D pipelines (see `shaders/matte3d.ts`). */
+export function matteLayoutEntry(binding: number): GPUBindGroupLayoutEntry {
+ return {
+ binding,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "float" },
+ };
+}
+
+const fullCoverage = new WeakMap<GPUDevice, GPUTextureView>();
+
+/** A 1Γ—1 opaque white view: full coverage, for draws without a matte. */
+export function fullCoverageView(device: GPUDevice): GPUTextureView {
+ let view = fullCoverage.get(device);
+ if (!view) {
+ const tex = device.createTexture({
+ label: "Matte3DFullCoverage",
+ size: [1, 1],
+ format: "rgba8unorm",
+ usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST,
+ });
+ device.queue.writeTexture(
+ { texture: tex },
+ new Uint8Array([255, 255, 255, 255]),
+ { bytesPerRow: 4 },
+ [1, 1],
+ );
+ view = tex.createView();
+ fullCoverage.set(device, view);
+ }
+ return view;
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
index ca809556..8165447f 100644
--- a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
@@ -1,7 +1,30 @@
import { BufferPool } from "../renderer2d/buffer-pool.js";
import { mesh3dWgsl } from "../shaders/mesh3d.js";
+import { fullCoverageView, matteLayoutEntry } from "./matte3d.js";
+import { BloomPipelineVariants } from "./pipeline-variants.js";
+import { createLightsLayout } from "./quad3d-pipeline.js";
+import { SHADOW_DEPTH_STENCIL } from "./shadow-depth.js";
+import {
+ packSurfaceParams,
+ SURFACE_PARAMS_FLOATS,
+ type SurfaceParams,
+} from "./surface-params.js";
-export interface MeshUniformData {
+/** A mesh material's extra maps (glTF conventions); absent maps are neutral. */
+export interface MeshMaterialMaps {
+ /** Tangent-space normal map (+Y up). */
+ normal?: GPUTexture;
+ normalScale?: number;
+ /** G = roughness, B = metallic; multiplies the roughness / metallic factors. */
+ metallicRoughness?: GPUTexture;
+ /** R = ambient occlusion. */
+ occlusion?: GPUTexture;
+ occlusionStrength?: number;
+ /** Multiplies the emissive color. */
+ emissive?: GPUTexture;
+}
+
+export interface MeshUniformData extends SurfaceParams {
modelMatrix: Float32Array | number[];
normalMatrix?: Float32Array | number[];
colorTint?: [number, number, number, number];
@@ -11,8 +34,12 @@ export interface MeshUniformData {
isSkinned?: boolean;
skinningMatrices?: Float32Array; // 128 * 16 floats
material?: "lit" | "unlit" | "toon";
+ /** Screen-space track matte: the mesh shows where this texture's alpha does. */
+ matte?: GPUTexture;
/** Toon shading: the shadowed side's color multiplier. */
shadeColor?: [number, number, number];
+ /** Normal / metallic-roughness / occlusion / emissive maps. */
+ maps?: MeshMaterialMaps;
/** Texels with alpha below this are discarded (glTF MASK). */
alphaCutoff?: number;
shininess?: number;
@@ -33,6 +60,9 @@ export class Mesh3DPipeline {
private device: GPUDevice;
public pipelineSingle: GPURenderPipeline;
public pipelineMrt: GPURenderPipeline;
+ private bloomVariants: BloomPipelineVariants;
+ /** Depth only (skinning included): draws a caster into a shadow map. */
+ public pipelineShadow: GPURenderPipeline;
public cameraLayout: GPUBindGroupLayout;
public modelLayout: GPUBindGroupLayout;
public textureLayout: GPUBindGroupLayout;
@@ -40,10 +70,12 @@ export class Mesh3DPipeline {
private modelPool: BufferPool;
private skinningPool: BufferPool;
public defaultWhiteTexture: GPUTexture;
+ /** (0.5, 0.5, 1): an unperturbed tangent-space normal. */
+ private flatNormalTexture: GPUTexture;
private defaultSkinningBuffer: GPUBuffer;
- // 16 (model) + 16 (normal) + 4 (tint) + 4 (params) + 4 (materialParams) + 4 (shade) = 48 floats (192 bytes)
- private modelData = new Float32Array(48);
+ // 16 (model) + 16 (normal) + 4 (tint) + 4 (params) + 4 (materialParams) + 4 (shade) + 8 (pbr, emissive) + 4 (maps) = 60 floats (240 bytes)
+ private modelData = new Float32Array(52 + SURFACE_PARAMS_FLOATS);
constructor(
device: GPUDevice,
@@ -56,7 +88,7 @@ export class Mesh3DPipeline {
) {
this.device = device;
this.modelPool = new BufferPool(
- 192,
+ (52 + SURFACE_PARAMS_FLOATS) * 4,
GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
);
// 128 bones * 16 floats * 4 bytes = 8192 bytes
@@ -96,6 +128,7 @@ export class Mesh3DPipeline {
visibility: GPUShaderStage.VERTEX,
buffer: { type: "uniform" },
},
+ matteLayoutEntry(2),
],
});
@@ -107,26 +140,17 @@ export class Mesh3DPipeline {
visibility: GPUShaderStage.FRAGMENT,
sampler: { type: "filtering" },
},
- {
- binding: 1,
+ // 1 base color, 2 normal, 3 metallic-roughness, 4 occlusion, 5 emissive
+ ...[1, 2, 3, 4, 5].map((binding) => ({
+ binding,
visibility: GPUShaderStage.FRAGMENT,
- texture: { sampleType: "float" },
- },
+ texture: { sampleType: "float" as const },
+ })),
],
});
this.lightsLayout =
- lightsLayout ??
- device.createBindGroupLayout({
- label: "Mesh3DLightsLayout",
- entries: [
- {
- binding: 0,
- visibility: GPUShaderStage.FRAGMENT,
- buffer: { type: "uniform" },
- },
- ],
- });
+ lightsLayout ?? createLightsLayout(device, "Mesh3DLightsLayout");
const pipelineLayout = device.createPipelineLayout({
bindGroupLayouts: [
@@ -233,6 +257,31 @@ export class Mesh3DPipeline {
multisample: { count: sampleCount },
});
+ this.bloomVariants = new BloomPipelineVariants(device, {
+ label: "Mesh3DPipeline",
+ layout: pipelineLayout,
+ vertex: vertexState,
+ module,
+ format,
+ depthStencil: {
+ format: depthFormat,
+ depthWriteEnabled: true,
+ depthCompare: "less-equal",
+ },
+ primitive: { topology: "triangle-list", cullMode: "none" },
+ multisample: { count: sampleCount },
+ });
+
+ this.pipelineShadow = device.createRenderPipeline({
+ label: "Mesh3DPipelineShadow",
+ layout: device.createPipelineLayout({
+ bindGroupLayouts: [this.cameraLayout, this.modelLayout],
+ }),
+ vertex: vertexState,
+ depthStencil: SHADOW_DEPTH_STENCIL,
+ primitive: { topology: "triangle-list", cullMode: "none" },
+ });
+
// 1x1 White dummy texture
this.defaultWhiteTexture = device.createTexture({
label: "Mesh3DDefaultWhiteTexture",
@@ -247,6 +296,19 @@ export class Mesh3DPipeline {
{ width: 1, height: 1 },
);
+ this.flatNormalTexture = device.createTexture({
+ label: "Mesh3DFlatNormalTexture",
+ size: { width: 1, height: 1 },
+ format: "rgba8unorm",
+ usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.COPY_DST,
+ });
+ device.queue.writeTexture(
+ { texture: this.flatNormalTexture },
+ new Uint8Array([128, 128, 255, 255]),
+ { bytesPerRow: 4, rowsPerImage: 1 },
+ { width: 1, height: 1 },
+ );
+
// Default identity skinning buffer (128 identity matrices)
const idMatrices = new Float32Array(128 * 16);
for (let b = 0; b < 128; b++) {
@@ -271,16 +333,7 @@ export class Mesh3DPipeline {
this.skinningPool.reset();
}
- draw(
- pass: GPURenderPassEncoder,
- cameraBindGroup: GPUBindGroup,
- lightsBindGroup: GPUBindGroup,
- meshBuffers: GPUMeshBuffers,
- texture: GPUTexture | undefined,
- sampler: GPUSampler,
- data: MeshUniformData,
- useMrt = false,
- ): void {
+ private packModel(data: MeshUniformData): GPUBindGroup {
const mat = data.modelMatrix;
for (let i = 0; i < 16; i++) {
this.modelData[i] = mat[i] ?? 0;
@@ -324,35 +377,72 @@ export class Mesh3DPipeline {
this.modelData[45] = shade[1];
this.modelData[46] = shade[2];
this.modelData[47] = data.alphaCutoff ?? 0;
+ packSurfaceParams(data, this.modelData, 48);
+ const maps = data.maps;
+ this.modelData[56] = maps?.normal ? (maps.normalScale ?? 1) : 0;
+ this.modelData[57] = maps?.occlusion ? (maps.occlusionStrength ?? 1) : 0;
+ this.modelData[58] = maps?.metallicRoughness ? 1 : 0;
+ this.modelData[59] = maps?.emissive ? 1 : 0;
- const modelBuffer = this.modelPool.getBuffer(
- this.device,
- this.modelData,
- );
+ const modelBuffer = this.modelPool.getBuffer(this.device, this.modelData);
const skinningBuffer =
data.isSkinned && data.skinningMatrices
? this.skinningPool.getBuffer(this.device, data.skinningMatrices)
: this.defaultSkinningBuffer;
- const modelBindGroup = this.device.createBindGroup({
+ return this.device.createBindGroup({
layout: this.modelLayout,
entries: [
{ binding: 0, resource: { buffer: modelBuffer } },
{ binding: 1, resource: { buffer: skinningBuffer } },
+ {
+ binding: 2,
+ resource: data.matte?.createView() ?? fullCoverageView(this.device),
+ },
],
});
+ }
- const actualTexture = texture ?? this.defaultWhiteTexture;
+ draw(
+ pass: GPURenderPassEncoder,
+ cameraBindGroup: GPUBindGroup,
+ lightsBindGroup: GPUBindGroup,
+ meshBuffers: GPUMeshBuffers,
+ texture: GPUTexture | undefined,
+ sampler: GPUSampler,
+ data: MeshUniformData,
+ useMrt = false,
+ useBloom = false,
+ ): void {
+ const modelBindGroup = this.packModel(data);
+ const white = this.defaultWhiteTexture;
+ const maps = data.maps;
const textureBindGroup = this.device.createBindGroup({
layout: this.textureLayout,
entries: [
{ binding: 0, resource: sampler },
- { binding: 1, resource: actualTexture.createView() },
+ { binding: 1, resource: (texture ?? white).createView() },
+ {
+ binding: 2,
+ resource: (maps?.normal ?? this.flatNormalTexture).createView(),
+ },
+ {
+ binding: 3,
+ resource: (maps?.metallicRoughness ?? white).createView(),
+ },
+ { binding: 4, resource: (maps?.occlusion ?? white).createView() },
+ { binding: 5, resource: (maps?.emissive ?? white).createView() },
],
});
- pass.setPipeline(useMrt ? this.pipelineMrt : this.pipelineSingle);
+ pass.setPipeline(
+ useBloom
+ ? this.bloomVariants.get(useMrt)
+ : useMrt
+ ? this.pipelineMrt
+ : this.pipelineSingle,
+ );
pass.setBindGroup(0, cameraBindGroup);
pass.setBindGroup(1, modelBindGroup);
pass.setBindGroup(2, textureBindGroup);
@@ -362,10 +452,26 @@ export class Mesh3DPipeline {
pass.drawIndexed(meshBuffers.indexCount);
}
+ /** Draws the mesh into a shadow map (`lightCameraBindGroup` holds the light's view-projection). */
+ drawShadow(
+ pass: GPURenderPassEncoder,
+ lightCameraBindGroup: GPUBindGroup,
+ meshBuffers: GPUMeshBuffers,
+ data: MeshUniformData,
+ ): void {
+ pass.setPipeline(this.pipelineShadow);
+ pass.setBindGroup(0, lightCameraBindGroup);
+ pass.setBindGroup(1, this.packModel(data));
+ pass.setVertexBuffer(0, meshBuffers.vertexBuffer);
+ pass.setIndexBuffer(meshBuffers.indexBuffer, meshBuffers.indexFormat);
+ pass.drawIndexed(meshBuffers.indexCount);
+ }
+
destroy(): void {
this.modelPool.destroy();
this.skinningPool.destroy();
this.defaultWhiteTexture.destroy();
+ this.flatNormalTexture.destroy();
this.defaultSkinningBuffer.destroy();
}
}
diff --git a/packages/webgpu-renderers/src/renderer3d/pipeline-variants.ts b/packages/webgpu-renderers/src/renderer3d/pipeline-variants.ts
new file mode 100644
index 00000000..b35bb281
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/pipeline-variants.ts
@@ -0,0 +1,70 @@
+/**
+ * The 3D pipelines draw into one of four target sets: color; color + linear
+ * depth (DoF); and either of those plus an HDR radiance target (bloom). The
+ * bloom variants are built on first use from the same descriptor parts.
+ */
+
+/** Premultiplied-alpha "over". */
+export const PREMULTIPLIED_OVER: GPUBlendState = {
+ color: {
+ srcFactor: "one",
+ dstFactor: "one-minus-src-alpha",
+ operation: "add",
+ },
+ alpha: {
+ srcFactor: "one",
+ dstFactor: "one-minus-src-alpha",
+ operation: "add",
+ },
+};
+
+/** Linear-depth MRT target (DoF) and HDR radiance target (bloom). */
+export const LINEAR_DEPTH_FORMAT: GPUTextureFormat = "rgba16float";
+export const HDR_FORMAT: GPUTextureFormat = "rgba16float";
+
+export interface PipelineVariantBase {
+ label: string;
+ layout: GPUPipelineLayout;
+ vertex: GPUVertexState;
+ module: GPUShaderModule;
+ format: GPUTextureFormat;
+ depthStencil: GPUDepthStencilState;
+ primitive: GPUPrimitiveState;
+ multisample: GPUMultisampleState;
+}
+
+export class BloomPipelineVariants {
+ private cache = new Map<boolean, GPURenderPipeline>();
+
+ constructor(
+ private device: GPUDevice,
+ private base: PipelineVariantBase,
+ ) {}
+
+ /** The pipeline writing color (+ linear depth when `useMrt`) + HDR radiance. */
+ get(useMrt: boolean): GPURenderPipeline {
+ let pipe = this.cache.get(useMrt);
+ if (!pipe) {
+ const b = this.base;
+ pipe = this.device.createRenderPipeline({
+ label: `${b.label}${useMrt ? "MrtBloom" : "Bloom"}`,
+ layout: b.layout,
+ vertex: b.vertex,
+ fragment: {
+ module: b.module,
+ entryPoint: useMrt ? "fs_mrt_bloom" : "fs_single_bloom",
+ targets: [
+ { format: b.format, blend: PREMULTIPLIED_OVER },
+ ...(useMrt ? [{ format: LINEAR_DEPTH_FORMAT }] : []),
+ { format: HDR_FORMAT, blend: PREMULTIPLIED_OVER },
+ ],
+ },
+ depthStencil: b.depthStencil,
+ primitive: b.primitive,
+ multisample: b.multisample,
+ });
+ this.cache.set(useMrt, pipe);
+ }
+ return pipe;
+ }
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
index 67d52c33..e176986e 100644
--- a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts
@@ -1,7 +1,45 @@
import { BufferPool } from "../renderer2d/buffer-pool.js";
import { quad3dWgsl } from "../shaders/quad3d.js";
+import { fullCoverageView, matteLayoutEntry } from "./matte3d.js";
+import { BloomPipelineVariants } from "./pipeline-variants.js";
+import { SHADOW_DEPTH_STENCIL } from "./shadow-depth.js";
+import {
+ packSurfaceParams,
+ SURFACE_PARAMS_FLOATS,
+ type SurfaceParams,
+} from "./surface-params.js";
-export interface ModelUniformData {
+/**
+ * Group 3 of every lit 3D shader: the lights uniform, then the shadow map and
+ * its comparison sampler (a 1Γ—1 cleared map when no light casts shadows).
+ */
+export function createLightsLayout(
+ device: GPUDevice,
+ label = "Lights3DLayout",
+): GPUBindGroupLayout {
+ return device.createBindGroupLayout({
+ label,
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.FRAGMENT,
+ buffer: { type: "uniform" },
+ },
+ {
+ binding: 1,
+ visibility: GPUShaderStage.FRAGMENT,
+ texture: { sampleType: "depth" },
+ },
+ {
+ binding: 2,
+ visibility: GPUShaderStage.FRAGMENT,
+ sampler: { type: "comparison" },
+ },
+ ],
+ });
+}
+
+export interface ModelUniformData extends SurfaceParams {
modelMatrix: Float32Array | number[];
normalMatrix?: Float32Array | number[];
colorTint?: [number, number, number, number];
@@ -15,6 +53,8 @@ export interface ModelUniformData {
/** Fraction of the texture the layer covers, [u, v] (default the whole texture) */
textureExtent?: [number, number];
material?: "lit" | "unlit";
+ /** Screen-space track matte: the layer shows where this texture's alpha does. */
+ matte?: GPUTexture;
shininess?: number;
roughness?: number;
specularIntensity?: number;
@@ -30,6 +70,8 @@ export type MaterialUniformData = Pick<
| "specularIntensity"
| "ambientIntensity"
| "metallic"
+ | "emissive"
+ | "emissiveIntensity"
>;
/** [shininess, specularIntensity, ambientIntensity, materialMode] as the 3D shaders' lighting reads them. */
@@ -53,6 +95,9 @@ export class Quad3DPipeline {
private device: GPUDevice;
public pipelineSingle: GPURenderPipeline;
public pipelineMrt: GPURenderPipeline;
+ private bloomVariants: BloomPipelineVariants;
+ /** Depth-only, alpha-tested: draws a caster into a shadow map. */
+ public pipelineShadow: GPURenderPipeline;
public cameraLayout: GPUBindGroupLayout;
public modelLayout: GPUBindGroupLayout;
public textureLayout: GPUBindGroupLayout;
@@ -60,8 +105,8 @@ export class Quad3DPipeline {
private modelPool: BufferPool;
private vertexBuffer: GPUBuffer;
private indexBuffer: GPUBuffer;
- // 16 (model) + 16 (normal) + 4 (tint) + 4 (borderColor) + 4 (params) + 4 (dimensions) + 4 (materialParams) = 52 floats (208 bytes)
- private modelData = new Float32Array(52);
+ // 16 (model) + 16 (normal) + 4 (tint) + 4 (borderColor) + 4 (params) + 4 (dimensions) + 4 (materialParams) + 8 (pbr, emissive) = 60 floats (240 bytes)
+ private modelData = new Float32Array(52 + SURFACE_PARAMS_FLOATS);
constructor(
device: GPUDevice,
@@ -72,7 +117,7 @@ export class Quad3DPipeline {
) {
this.device = device;
this.modelPool = new BufferPool(
- 208,
+ (52 + SURFACE_PARAMS_FLOATS) * 4,
GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
);
@@ -100,6 +145,7 @@ export class Quad3DPipeline {
visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT,
buffer: { type: "uniform" },
},
+ matteLayoutEntry(1),
],
});
@@ -119,16 +165,7 @@ export class Quad3DPipeline {
],
});
- this.lightsLayout = device.createBindGroupLayout({
- label: "Quad3DLightsLayout",
- entries: [
- {
- binding: 0,
- visibility: GPUShaderStage.FRAGMENT,
- buffer: { type: "uniform" },
- },
- ],
- });
+ this.lightsLayout = createLightsLayout(device, "Quad3DLightsLayout");
const pipelineLayout = device.createPipelineLayout({
bindGroupLayouts: [
@@ -234,6 +271,36 @@ export class Quad3DPipeline {
});
// Unit quad centered at origin: [-0.5, -0.5] to [0.5, 0.5] with normal [0, 0, -1]
+ this.bloomVariants = new BloomPipelineVariants(device, {
+ label: "Quad3DPipeline",
+ layout: pipelineLayout,
+ vertex: vertexState,
+ module,
+ format,
+ depthStencil: {
+ format: depthFormat,
+ depthWriteEnabled: true,
+ depthCompare: "less-equal",
+ },
+ primitive: { topology: "triangle-list", cullMode: "none" },
+ multisample: { count: sampleCount },
+ });
+
+ this.pipelineShadow = device.createRenderPipeline({
+ label: "Quad3DPipelineShadow",
+ layout: device.createPipelineLayout({
+ bindGroupLayouts: [
+ this.cameraLayout,
+ this.modelLayout,
+ this.textureLayout,
+ ],
+ }),
+ vertex: vertexState,
+ fragment: { module, entryPoint: "fs_shadow", targets: [] },
+ depthStencil: SHADOW_DEPTH_STENCIL,
+ primitive: { topology: "triangle-list", cullMode: "none" },
+ });
+
const vertices = new Float32Array([
-0.5, -0.5, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.5, -0.5, 0.0, 1.0, 0.0, 0.0,
0.0, -1.0, 0.5, 0.5, 0.0, 1.0, 1.0, 0.0, 0.0, -1.0, -0.5, 0.5, 0.0, 0.0,
@@ -262,15 +329,7 @@ export class Quad3DPipeline {
this.modelPool.reset();
}
- draw(
- pass: GPURenderPassEncoder,
- cameraBindGroup: GPUBindGroup,
- lightsBindGroup: GPUBindGroup,
- texture: GPUTexture,
- sampler: GPUSampler,
- data: ModelUniformData,
- useMrt = false,
- ): void {
+ private packModel(data: ModelUniformData): GPUBindGroup {
const mat = data.modelMatrix;
for (let i = 0; i < 16; i++) {
this.modelData[i] = mat[i] ?? 0;
@@ -304,23 +363,54 @@ export class Quad3DPipeline {
this.modelData[47] = data.textureExtent?.[1] ?? 0;
this.modelData.set(packMaterialParams(data), 48);
+ packSurfaceParams(data, this.modelData, 52);
const modelBuffer = this.modelPool.getBuffer(this.device, this.modelData);
-
- const modelBindGroup = this.device.createBindGroup({
+ return this.device.createBindGroup({
layout: this.modelLayout,
- entries: [{ binding: 0, resource: { buffer: modelBuffer } }],
+ entries: [
+ { binding: 0, resource: { buffer: modelBuffer } },
+ {
+ binding: 1,
+ resource: data.matte?.createView() ?? fullCoverageView(this.device),
+ },
+ ],
});
+ }
- const textureBindGroup = this.device.createBindGroup({
+ private textureBindGroup(
+ texture: GPUTexture,
+ sampler: GPUSampler,
+ ): GPUBindGroup {
+ return this.device.createBindGroup({
layout: this.textureLayout,
entries: [
{ binding: 0, resource: sampler },
{ binding: 1, resource: texture.createView() },
],
});
+ }
+
+ draw(
+ pass: GPURenderPassEncoder,
+ cameraBindGroup: GPUBindGroup,
+ lightsBindGroup: GPUBindGroup,
+ texture: GPUTexture,
+ sampler: GPUSampler,
+ data: ModelUniformData,
+ useMrt = false,
+ useBloom = false,
+ ): void {
+ const modelBindGroup = this.packModel(data);
+ const textureBindGroup = this.textureBindGroup(texture, sampler);
- pass.setPipeline(useMrt ? this.pipelineMrt : this.pipelineSingle);
+ pass.setPipeline(
+ useBloom
+ ? this.bloomVariants.get(useMrt)
+ : useMrt
+ ? this.pipelineMrt
+ : this.pipelineSingle,
+ );
pass.setBindGroup(0, cameraBindGroup);
pass.setBindGroup(1, modelBindGroup);
pass.setBindGroup(2, textureBindGroup);
@@ -330,6 +420,23 @@ export class Quad3DPipeline {
pass.drawIndexed(6);
}
+ /** Draws the quad into a shadow map (`lightCameraBindGroup` holds the light's view-projection). */
+ drawShadow(
+ pass: GPURenderPassEncoder,
+ lightCameraBindGroup: GPUBindGroup,
+ texture: GPUTexture,
+ sampler: GPUSampler,
+ data: ModelUniformData,
+ ): void {
+ pass.setPipeline(this.pipelineShadow);
+ pass.setBindGroup(0, lightCameraBindGroup);
+ pass.setBindGroup(1, this.packModel(data));
+ pass.setBindGroup(2, this.textureBindGroup(texture, sampler));
+ pass.setVertexBuffer(0, this.vertexBuffer);
+ pass.setIndexBuffer(this.indexBuffer, "uint16");
+ pass.drawIndexed(6);
+ }
+
destroy(): void {
this.modelPool.destroy();
this.vertexBuffer.destroy();
diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
index 71415c39..e199c9a0 100644
--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
@@ -1,17 +1,5 @@
-import {
- Camera3D,
- type Camera3DPose,
- Light3D,
- type Light3DOptions,
- LIGHT_UNIFORM_SIZE_BYTES,
- type Mat4,
- Matrix4Math,
- type Vec3,
- Vector3Math,
-} from "../math3d/index.js";
-import { BufferPool } from "../renderer2d/buffer-pool.js";
-import { SamplerCache } from "../renderer2d/sampler-cache.js";
-import { dofBokehWgsl } from "../shaders/dof-bokeh.js";
+import type { MeshAudioDeformConfig } from "@framefields/core";
+import { type Color, parseColor } from "../color.js";
import {
FbxLoader,
GltfLoader,
@@ -34,14 +22,42 @@ import type {
Mesh3DData,
Model3DData,
} from "../loaders/types.js";
-import { type Color, parseColor } from "../color.js";
-import { type GPUMeshBuffers, Mesh3DPipeline } from "./mesh3d-pipeline.js";
-import { Quad3DPipeline } from "./quad3d-pipeline.js";
-import { Slug3DPipeline } from "./slug3d-pipeline.js";
+import type {
+ Environment3DOptions,
+ ShadowUniformInfo,
+} from "../math3d/environment3d.js";
+import {
+ Camera3D,
+ type Camera3DPose,
+ LIGHT_UNIFORM_SIZE_BYTES,
+ Light3D,
+ type Light3DOptions,
+ type Mat4,
+ Matrix4Math,
+ type Vec3,
+ Vector3Math,
+} from "../math3d/index.js";
+import { BufferPool } from "../renderer2d/buffer-pool.js";
+import { SamplerCache } from "../renderer2d/sampler-cache.js";
+import { UniformBindGroupCache } from "../renderer2d/uniform-bind-group-cache.js";
+import { dofBokehWgsl } from "../shaders/dof-bokeh.js";
import type { SlugGlyphBatch } from "../slug/slug-pipeline.js";
import { AudioMeshDeformPipeline } from "./audio-mesh-deform-pipeline.js";
-import { UniformBindGroupCache } from "../renderer2d/uniform-bind-group-cache.js";
-import type { MeshAudioDeformConfig } from "@framefields/core";
+import { type BloomOptions, BloomPipeline } from "./bloom-pipeline.js";
+import {
+ type GPUMeshBuffers,
+ Mesh3DPipeline,
+ type MeshMaterialMaps,
+} from "./mesh3d-pipeline.js";
+import { Quad3DPipeline } from "./quad3d-pipeline.js";
+import { SHADOW_DEPTH_FORMAT } from "./shadow-depth.js";
+import {
+ type Aabb,
+ fitShadowFrustum,
+ transformAabb,
+} from "./shadow-frustum.js";
+import { Slug3DPipeline } from "./slug3d-pipeline.js";
+import type { SurfaceParams } from "./surface-params.js";
const ADDRESS_MODES: Record<"repeat" | "clamp" | "mirror", GPUAddressMode> = {
repeat: "repeat",
@@ -49,7 +65,7 @@ const ADDRESS_MODES: Record<"repeat" | "clamp" | "mirror", GPUAddressMode> = {
mirror: "mirror-repeat",
};
-export interface DrawQuad3DOpts {
+export interface DrawQuad3DOpts extends SurfaceParams {
/** 3D world position [x, y, z] */
position: [number, number, number];
/** Layer surface width */
@@ -92,6 +108,11 @@ export interface DrawQuad3DOpts {
ambientIntensity?: number;
/** Metallic factor [0, 1] */
metallic?: number;
+ /**
+ * Screen-space track matte, the size of the pass: the layer shows where
+ * this texture's alpha does and keeps its place in the depth buffer.
+ */
+ matte?: GPUTexture;
}
/**
@@ -101,7 +122,12 @@ export interface DrawQuad3DOpts {
*/
const TEXT_STACK_PULL = 5e-4;
-export interface DrawMesh3DOpts {
+export interface DrawMesh3DOpts extends SurfaceParams {
+ /**
+ * Screen-space track matte, the size of the pass: the layer shows where
+ * this texture's alpha does and keeps its place in the depth buffer.
+ */
+ matte?: GPUTexture;
/** 3D world position [x, y, z] */
position?: Vec3;
/** Euler rotations in degrees [rx, ry, rz] */
@@ -128,6 +154,8 @@ export interface DrawMesh3DOpts {
material?: "lit" | "unlit" | "toon";
/** Toon shading: the shadowed side's color multiplier */
shadeColor?: [number, number, number];
+ /** Normal / metallic-roughness / occlusion / emissive maps (see `materialMaps`). */
+ maps?: MeshMaterialMaps;
/** Discard texels whose alpha is below this (glTF MASK) */
alphaCutoff?: number;
/** Texture addressing outside [0, 1] */
@@ -151,6 +179,33 @@ export interface DrawMesh3DOpts {
};
}
+/** Scene-wide world settings for a 3D pass. */
+export interface Scene3DWorld {
+ /** Present = physically based shading (see `Environment3DOptions`). */
+ environment?: Environment3DOptions | null;
+ /** From `renderShadowMap`, when a light casts shadows. */
+ shadow?: ShadowUniformInfo | null;
+}
+
+/** Something that can fall into a shadow map. */
+export type ShadowCaster3D =
+ | {
+ kind: "mesh";
+ mesh: Mesh3DData;
+ buffers?: GPUMeshBuffers;
+ opts: DrawMesh3DOpts;
+ }
+ | { kind: "quad"; texture: GPUTexture; opts: DrawQuad3DOpts };
+
+export interface ShadowMapOptions3D {
+ /** Texels per side. Default 2048. */
+ mapSize?: number;
+ /** Edge blur in texels. Default 2.5. */
+ softness?: number;
+ /** 0..1. Default 1. */
+ darkness?: number;
+}
+
export class Renderer3D {
private device: GPUDevice;
public format: GPUTextureFormat;
@@ -163,22 +218,34 @@ export class Renderer3D {
private cameraPool: BufferPool;
private lightsPool: BufferPool;
private cameraUniformBindGroupCache = new UniformBindGroupCache();
- private lightsUniformBindGroupCache = new UniformBindGroupCache();
private samplerCache = new SamplerCache();
private depthTexture: GPUTexture | null = null;
private linearDepthTexture: GPUTexture | null = null;
// Multisampled targets the pass draws into; they resolve into the caller's views.
private msaaColorTexture: GPUTexture | null = null;
private msaaLinearDepthTexture: GPUTexture | null = null;
+ private msaaHdrTexture: GPUTexture | null = null;
+ private hdrTexture: GPUTexture | null = null;
+ /** Whether the current pass also writes HDR radiance (bloom). */
+ private passWritesHdr = false;
private cameraUniformData = new Float32Array(80); // 320 bytes = 80 floats
private meshBuffersCache = new WeakMap<Mesh3DData, GPUMeshBuffers>();
private materialTextures = new WeakMap<
Material3D,
Promise<GPUTexture | undefined>
>();
+ private materialMapCache = new WeakMap<
+ Material3D,
+ Promise<MeshMaterialMaps>
+ >();
/** Parsed models by source and load options: a model file is read once, not every frame. */
private modelCache = new Map<string, Model3DData>();
private lastCameraPose: Camera3DPose | null = null;
+ private shadowTexture: GPUTexture | null = null;
+ private placeholderShadowTexture: GPUTexture | null = null;
+ private shadowSampler: GPUSampler | null = null;
+ private meshBounds = new WeakMap<Mesh3DData, Aabb>();
+ private bloomPipeline: BloomPipeline | null = null;
private lastCameraBuffer: GPUBuffer | null = null;
/**
@@ -279,7 +346,13 @@ export class Renderer3D {
}
resetPools(): void {
+ this.bloomPipeline?.resetPools();
this.cameraPool.reset();
+ // The cached camera buffer goes back to the pool and may be handed
+ // out again (e.g. to a shadow pass's light camera): never reuse it
+ // across a reset.
+ this.lastCameraPose = null;
+ this.lastCameraBuffer = null;
this.lightsPool.reset();
this.quad3dPipeline.resetPools();
this.slug3dPipeline.resetPools();
@@ -325,6 +398,29 @@ export class Renderer3D {
});
}
+ /** The HDR radiance target the last `beginPass(…, writeHdr)` drew into. */
+ hdrSource(): GPUTexture | null {
+ return this.hdrTexture;
+ }
+
+ private getOrCreateHdrTexture(width: number, height: number): GPUTexture {
+ if (
+ !this.hdrTexture ||
+ this.hdrTexture.width !== width ||
+ this.hdrTexture.height !== height
+ ) {
+ this.hdrTexture?.destroy();
+ this.hdrTexture = this.device.createTexture({
+ label: "Renderer3DHdrRadiance",
+ size: { width, height },
+ format: "rgba16float",
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
+ });
+ }
+ return this.hdrTexture;
+ }
+
getOrCreateLinearDepthTexture(width: number, height: number): GPUTexture {
if (
!this.linearDepthTexture ||
@@ -427,11 +523,211 @@ export class Renderer3D {
);
}
- public getOrCreateLightsBuffer(lights?: Light3DOptions[]): GPUBuffer {
- const lightsData = Light3D.packLightsUniforms(lights ?? []);
+ public getOrCreateLightsBuffer(
+ lights?: Light3DOptions[],
+ world?: Scene3DWorld,
+ ): GPUBuffer {
+ const lightsData = Light3D.packLightsUniforms(
+ lights ?? [],
+ world?.environment,
+ world?.shadow,
+ );
return this.lightsPool.getBuffer(this.device, lightsData);
}
+ /** Lights uniform + shadow map + comparison sampler (group 3 of every lit shader). */
+ private createLightsBindGroup(
+ lightsBuffer: GPUBuffer,
+ withShadow: boolean,
+ ): GPUBindGroup {
+ this.shadowSampler ??= this.device.createSampler({
+ label: "ShadowComparisonSampler",
+ compare: "less-equal",
+ magFilter: "linear",
+ minFilter: "linear",
+ addressModeU: "clamp-to-edge",
+ addressModeV: "clamp-to-edge",
+ });
+ this.placeholderShadowTexture ??= this.device.createTexture({
+ label: "ShadowMapPlaceholder",
+ size: { width: 1, height: 1 },
+ format: SHADOW_DEPTH_FORMAT,
+ usage:
+ GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.RENDER_ATTACHMENT,
+ });
+ const map =
+ withShadow && this.shadowTexture
+ ? this.shadowTexture
+ : this.placeholderShadowTexture;
+ return this.device.createBindGroup({
+ label: "Lights3DBindGroup",
+ layout: this.quad3dPipeline.lightsLayout,
+ entries: [
+ { binding: 0, resource: { buffer: lightsBuffer } },
+ { binding: 1, resource: map.createView() },
+ { binding: 2, resource: this.shadowSampler },
+ ],
+ });
+ }
+
+ /** Local bounds of a mesh, computed once. */
+ private meshAabb(mesh: Mesh3DData): Aabb {
+ let box = this.meshBounds.get(mesh);
+ if (!box) {
+ const min: Vec3 = [Infinity, Infinity, Infinity];
+ const max: Vec3 = [-Infinity, -Infinity, -Infinity];
+ const p = mesh.positions;
+ for (let i = 0; i + 2 < p.length; i += 3) {
+ for (let k = 0; k < 3; k++) {
+ const v = p[i + k];
+ if (v < min[k]) min[k] = v;
+ if (v > max[k]) max[k] = v;
+ }
+ }
+ box = Number.isFinite(min[0])
+ ? { min, max }
+ : { min: [0, 0, 0], max: [0, 0, 0] };
+ this.meshBounds.set(mesh, box);
+ }
+ return box;
+ }
+
+ /** World bounds of a mesh or quad draw, for fitting shadow frusta. */
+ worldBounds(item: ShadowCaster3D): Aabb {
+ if (item.kind === "mesh") {
+ return transformAabb(
+ this.meshAabb(item.mesh),
+ this.meshModelMatrix(item.opts),
+ );
+ }
+ return transformAabb(
+ { min: [-0.5, -0.5, 0], max: [0.5, 0.5, 0] },
+ this.quadModelMatrix(item.opts),
+ );
+ }
+
+ /**
+ * Renders the shadow map for `light` (a directional or spot light with
+ * `castShadow`) from `casters`, fitting its frustum around them and deep
+ * enough for `receivers`. Pass the result to `beginPass` as `world.shadow`.
+ */
+ renderShadowMap(
+ encoder: GPUCommandEncoder,
+ light: Light3DOptions,
+ lightIndex: number,
+ casters: ShadowCaster3D[],
+ receivers: Aabb[],
+ options: ShadowMapOptions3D = {},
+ ): ShadowUniformInfo | null {
+ const kind = light.lightType ?? light.type;
+ if (kind !== "directional" && kind !== "spot") return null;
+ const processed = Light3D.processLight(light);
+ const position: Vec3 = [
+ processed.posType[0],
+ processed.posType[1],
+ processed.posType[2],
+ ];
+ const direction: Vec3 = [
+ processed.dirRadius[0],
+ processed.dirRadius[1],
+ processed.dirRadius[2],
+ ];
+ const mapSize = options.mapSize ?? 2048;
+ const casterBounds = casters.map((c) => this.worldBounds(c));
+ const frustum = fitShadowFrustum(
+ { type: kind, position, direction, angle: light.angle },
+ casterBounds,
+ receivers,
+ mapSize,
+ );
+ if (!frustum) return null;
+
+ if (!this.shadowTexture || this.shadowTexture.width !== mapSize) {
+ this.shadowTexture?.destroy();
+ this.shadowTexture = this.device.createTexture({
+ label: "ShadowMap",
+ size: { width: mapSize, height: mapSize },
+ format: SHADOW_DEPTH_FORMAT,
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
+ });
+ }
+
+ // The light's "camera": only viewProj matters to the vertex shaders.
+ const lightCam = new Float32Array(80);
+ lightCam.set(frustum.viewProj, 32);
+ lightCam.set(frustum.viewProj, 0);
+ lightCam[5] = 1;
+ lightCam.set(frustum.viewProj, 16);
+ lightCam[64] = position[0];
+ lightCam[65] = position[1];
+ lightCam[66] = position[2];
+ lightCam[68] = direction[0];
+ lightCam[69] = direction[1];
+ lightCam[70] = direction[2];
+ lightCam[76] = mapSize;
+ lightCam[77] = mapSize;
+ const lightCamBindGroup = this.createCameraBindGroup(
+ this.cameraPool.getBuffer(this.device, lightCam),
+ );
+
+ const pass = encoder.beginRenderPass({
+ label: "ShadowMapPass",
+ colorAttachments: [],
+ depthStencilAttachment: {
+ view: this.shadowTexture.createView(),
+ depthClearValue: 1.0,
+ depthLoadOp: "clear",
+ depthStoreOp: "store",
+ },
+ });
+ const sampler = this.samplerCache.getSampler(this.device);
+ for (const c of casters) {
+ if (c.kind === "mesh") {
+ const m = this.meshModelMatrix(c.opts);
+ this.mesh3dPipeline.drawShadow(
+ pass,
+ lightCamBindGroup,
+ c.buffers ?? this.getOrCreateMeshBuffers(c.mesh),
+ {
+ modelMatrix: m,
+ normalMatrix: c.opts.normalMatrix ?? Light3D.computeNormalMatrix(m),
+ isSkinned: c.opts.isSkinned,
+ skinningMatrices: c.opts.skinningMatrices,
+ },
+ );
+ } else {
+ const m = this.quadModelMatrix(c.opts);
+ this.quad3dPipeline.drawShadow(
+ pass,
+ lightCamBindGroup,
+ c.texture,
+ sampler,
+ {
+ modelMatrix: m,
+ normalMatrix: m,
+ width: c.opts.width,
+ height: c.opts.height,
+ textureExtent: c.opts.textureExtent,
+ opacity: c.opts.opacity,
+ colorTint: c.opts.colorTint,
+ },
+ );
+ }
+ }
+ pass.end();
+
+ return {
+ matrix: frustum.viewProj,
+ lightIndex,
+ mapSize,
+ softness: options.softness ?? light.shadowSoftness ?? 2.5,
+ bias: kind === "directional" ? 0.0006 : 0.00008,
+ normalOffset: frustum.texelWorldSize * 1.5,
+ darkness: options.darkness ?? light.shadowDarkness ?? 1,
+ };
+ }
+
beginPass(
encoder: GPUCommandEncoder,
colorTargetView: GPUTextureView,
@@ -442,6 +738,9 @@ export class Renderer3D {
clearColor: GPUColor = { r: 0, g: 0, b: 0, a: 0 },
linearDepthTargetView?: GPUTextureView,
lights?: Light3DOptions[],
+ world?: Scene3DWorld,
+ /** Also write unclamped scene radiance (`hdrTexture()`), the bloom source. */
+ writeHdr = false,
): {
pass: GPURenderPassEncoder;
cameraBindGroup: GPUBindGroup;
@@ -456,11 +755,10 @@ export class Renderer3D {
);
const cameraBindGroup = this.createCameraBindGroup(cameraBuffer);
- const lightsBuffer = this.getOrCreateLightsBuffer(lights);
- const lightsBindGroup = this.lightsUniformBindGroupCache.getBindGroup(
- this.device,
- this.quad3dPipeline.lightsLayout,
+ const lightsBuffer = this.getOrCreateLightsBuffer(lights, world);
+ const lightsBindGroup = this.createLightsBindGroup(
lightsBuffer,
+ Boolean(world?.shadow),
);
const msaa = this.sampleCount > 1;
@@ -527,6 +825,28 @@ export class Renderer3D {
);
}
+ this.passWritesHdr = writeHdr;
+ if (writeHdr) {
+ this.hdrTexture = this.getOrCreateHdrTexture(surfaceWidth, surfaceHeight);
+ if (msaa) {
+ this.msaaHdrTexture = this.getOrCreateMsaaTexture(
+ this.msaaHdrTexture,
+ "Renderer3DMsaaHdr",
+ "rgba16float",
+ surfaceWidth,
+ surfaceHeight,
+ );
+ }
+ colorAttachments.push(
+ attach(
+ this.hdrTexture.createView(),
+ msaa ? this.msaaHdrTexture : null,
+ { r: 0, g: 0, b: 0, a: 0 },
+ "clear",
+ ),
+ );
+ }
+
const pass = encoder.beginRenderPass({
colorAttachments,
depthStencilAttachment: {
@@ -653,8 +973,12 @@ export class Renderer3D {
specularIntensity: opts.specularIntensity ?? 0.5,
ambientIntensity: opts.ambientIntensity ?? 1.0,
metallic: opts.metallic,
+ emissive: opts.emissive,
+ emissiveIntensity: opts.emissiveIntensity,
+ matte: opts.matte,
},
useMrt,
+ this.passWritesHdr,
);
}
}
@@ -722,8 +1046,33 @@ export class Renderer3D {
specularIntensity,
ambientIntensity,
metallic,
+ emissive: opts.emissive,
+ emissiveIntensity: opts.emissiveIntensity,
+ matte: opts.matte,
},
useMrt,
+ this.passWritesHdr,
+ );
+ }
+
+ /** T(pos) * Rz * Ry * Rx * S, or the draw's precomputed model matrix. */
+ private meshModelMatrix(opts: DrawMesh3DOpts): Mat4 {
+ if (opts.modelMatrix) return opts.modelMatrix;
+ const {
+ position = [0, 0, 0],
+ rotation = [0, 0, 0],
+ scale = [1, 1, 1],
+ } = opts;
+ let m = Matrix4Math.translate(position[0], position[1], position[2]);
+ const rxRad = (rotation[0] * Math.PI) / 180;
+ const ryRad = (rotation[1] * Math.PI) / 180;
+ const rzRad = (rotation[2] * Math.PI) / 180;
+ if (rzRad !== 0) m = Matrix4Math.multiply(m, Matrix4Math.rotateZ(rzRad));
+ if (ryRad !== 0) m = Matrix4Math.multiply(m, Matrix4Math.rotateY(ryRad));
+ if (rxRad !== 0) m = Matrix4Math.multiply(m, Matrix4Math.rotateX(rxRad));
+ return Matrix4Math.multiply(
+ m,
+ Matrix4Math.scale(scale[0], scale[1], scale[2]),
);
}
@@ -861,9 +1210,6 @@ export class Renderer3D {
: this.getOrCreateMeshBuffers(meshOrBuffers);
const {
- position = [0, 0, 0],
- rotation = [0, 0, 0],
- scale = [1, 1, 1],
opacity = 1.0,
colorTint = [1, 1, 1, 1],
twoSided = true,
@@ -881,29 +1227,7 @@ export class Renderer3D {
metallic,
} = opts;
- let m: Mat4;
- if (opts.modelMatrix) {
- m = opts.modelMatrix;
- } else {
- m = Matrix4Math.identity();
- m = Matrix4Math.multiply(
- m,
- Matrix4Math.translate(position[0], position[1], position[2]),
- );
-
- const rxRad = (rotation[0] * Math.PI) / 180;
- const ryRad = (rotation[1] * Math.PI) / 180;
- const rzRad = (rotation[2] * Math.PI) / 180;
-
- if (rzRad !== 0) m = Matrix4Math.multiply(m, Matrix4Math.rotateZ(rzRad));
- if (ryRad !== 0) m = Matrix4Math.multiply(m, Matrix4Math.rotateY(ryRad));
- if (rxRad !== 0) m = Matrix4Math.multiply(m, Matrix4Math.rotateX(rxRad));
-
- m = Matrix4Math.multiply(
- m,
- Matrix4Math.scale(scale[0], scale[1], scale[2]),
- );
- }
+ const m = this.meshModelMatrix(opts);
const normalMatrix = opts.normalMatrix ?? Light3D.computeNormalMatrix(m);
const address = ADDRESS_MODES[textureWrap];
@@ -936,8 +1260,13 @@ export class Renderer3D {
specularIntensity,
ambientIntensity,
metallic,
+ emissive: opts.emissive,
+ emissiveIntensity: opts.emissiveIntensity,
+ matte: opts.matte,
+ maps: opts.maps,
},
useMrt,
+ this.passWritesHdr,
);
}
@@ -980,10 +1309,60 @@ export class Renderer3D {
return pending;
}
- private async uploadMaterialTexture(
+ /**
+ * The material's normal, metallic-roughness, occlusion and emissive maps
+ * on the GPU (each decoded once); absent maps are left out.
+ */
+ materialMaps(material: Material3D): Promise<MeshMaterialMaps> {
+ let pending = this.materialMapCache.get(material);
+ if (!pending) {
+ pending = (async () => {
+ const label = material.name;
+ const [normal, metallicRoughness, occlusion, emissive] =
+ await Promise.all([
+ this.uploadImage(material.normalTextureBuffer, `${label}:normal`),
+ this.uploadImage(
+ material.metallicRoughnessTextureBuffer,
+ `${label}:metallicRoughness`,
+ ),
+ this.uploadImage(
+ material.occlusionTextureBuffer,
+ `${label}:occlusion`,
+ ),
+ this.uploadImage(
+ material.emissiveTextureBuffer,
+ `${label}:emissive`,
+ ),
+ ]);
+ return {
+ ...(normal && { normal, normalScale: material.normalScale ?? 1 }),
+ ...(metallicRoughness && { metallicRoughness }),
+ ...(occlusion && {
+ occlusion,
+ occlusionStrength: material.occlusionStrength ?? 1,
+ }),
+ ...(emissive && { emissive }),
+ };
+ })();
+ this.materialMapCache.set(material, pending);
+ }
+ return pending;
+ }
+
+ private uploadMaterialTexture(
material: Material3D,
): Promise<GPUTexture | undefined> {
- const encoded = material.diffuseTextureBuffer;
+ return this.uploadImage(
+ material.diffuseTextureBuffer,
+ `Material:${material.name}`,
+ );
+ }
+
+ /** Decodes an encoded image into a texture with a full mip chain. */
+ private async uploadImage(
+ encoded: Uint8Array | ArrayBuffer | undefined,
+ label: string,
+ ): Promise<GPUTexture | undefined> {
if (!encoded) return undefined;
const sharp = (await import(/* webpackIgnore: true */ "sharp")).default;
const source = Buffer.from(
@@ -993,7 +1372,7 @@ export class Renderer3D {
if (!width || !height) return undefined;
const mipLevelCount = Math.floor(Math.log2(Math.max(width, height))) + 1;
const texture = this.device.createTexture({
- label: `Material:${material.name}`,
+ label,
size: { width, height },
mipLevelCount,
format: "rgba8unorm",
@@ -1017,6 +1396,25 @@ export class Renderer3D {
return texture;
}
+ /** Writes `source` with bloom into `dest` (see `BloomOptions`). */
+ applyBloom(
+ encoder: GPUCommandEncoder,
+ source: GPUTexture,
+ dest: GPUTexture,
+ options: BloomOptions = {},
+ /** Scene radiance to threshold (from a `writeHdr` pass); default `source`. */
+ hdrSource?: GPUTexture | null,
+ ): void {
+ this.bloomPipeline ??= new BloomPipeline(this.device);
+ this.bloomPipeline.apply(
+ encoder,
+ source,
+ dest,
+ options,
+ hdrSource ?? undefined,
+ );
+ }
+
applyDepthOfField(
encoder: GPUCommandEncoder,
sourceColorTex: GPUTexture,
@@ -1053,17 +1451,21 @@ export class Renderer3D {
}
destroy(): void {
+ this.bloomPipeline?.destroy();
this.depthTexture?.destroy();
+ this.shadowTexture?.destroy();
+ this.placeholderShadowTexture?.destroy();
this.linearDepthTexture?.destroy();
this.msaaColorTexture?.destroy();
this.msaaLinearDepthTexture?.destroy();
+ this.msaaHdrTexture?.destroy();
+ this.hdrTexture?.destroy();
this.quad3dPipeline.destroy();
this.slug3dPipeline.destroy();
this.mesh3dPipeline.destroy();
this.cameraPool.destroy();
this.lightsPool.destroy();
this.cameraUniformBindGroupCache.destroy();
- this.lightsUniformBindGroupCache.destroy();
}
}
diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-depth.ts b/packages/webgpu-renderers/src/renderer3d/shadow-depth.ts
new file mode 100644
index 00000000..c83dcbfd
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/shadow-depth.ts
@@ -0,0 +1,12 @@
+/** Shadow maps: a depth32float target rendered from a light's point of view. */
+export const SHADOW_DEPTH_FORMAT: GPUTextureFormat = "depth32float";
+
+/** Depth state for shadow-map passes: slope-scaled bias fights acne on steep faces. */
+export const SHADOW_DEPTH_STENCIL: GPUDepthStencilState = {
+ format: SHADOW_DEPTH_FORMAT,
+ depthWriteEnabled: true,
+ depthCompare: "less-equal",
+ depthBias: 2,
+ depthBiasSlopeScale: 2.5,
+ depthBiasClamp: 0.01,
+};
diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts
new file mode 100644
index 00000000..fba87876
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.test.ts
@@ -0,0 +1,114 @@
+import { describe, expect, it } from "vitest";
+import { Light3D } from "../math3d/light3d.js";
+import type { Mat4 } from "../math3d/matrix4.js";
+import {
+ type Aabb,
+ fitShadowFrustum,
+ transformAabb,
+} from "./shadow-frustum.js";
+
+function project(m: Mat4, p: [number, number, number]) {
+ const x = m[0] * p[0] + m[4] * p[1] + m[8] * p[2] + m[12];
+ const y = m[1] * p[0] + m[5] * p[1] + m[9] * p[2] + m[13];
+ const z = m[2] * p[0] + m[6] * p[1] + m[10] * p[2] + m[14];
+ const w = m[3] * p[0] + m[7] * p[1] + m[11] * p[2] + m[15];
+ return [x / w, y / w, z / w];
+}
+
+const box = (cx: number, cy: number, cz: number, r: number): Aabb => ({
+ min: [cx - r, cy - r, cz - r],
+ max: [cx + r, cy + r, cz + r],
+});
+
+describe("fitShadowFrustum", () => {
+ it("fits a directional light tightly around casters, deep enough for receivers", () => {
+ const caster = box(0, 0, 0, 100);
+ const floor: Aabb = { min: [-5000, 200, -5000], max: [5000, 210, 5000] };
+ const f = fitShadowFrustum(
+ { type: "directional", position: [0, -1000, 0], direction: [0, 1, 0] },
+ [caster],
+ [caster, floor],
+ 2048,
+ );
+ expect(f).not.toBeNull();
+ for (const p of [caster.min, caster.max]) {
+ const [x, y, z] = project(f!.viewProj, p as [number, number, number]);
+ expect(Math.abs(x)).toBeLessThanOrEqual(1);
+ expect(Math.abs(y)).toBeLessThanOrEqual(1);
+ expect(z).toBeGreaterThanOrEqual(0);
+ expect(z).toBeLessThanOrEqual(1);
+ }
+ // The floor below the caster is inside the depth range, and farther.
+ const [, , floorZ] = project(f!.viewProj, [0, 205, 0]);
+ const [, , casterZ] = project(f!.viewProj, [0, 0, 0]);
+ expect(floorZ).toBeLessThanOrEqual(1);
+ expect(floorZ).toBeGreaterThan(casterZ);
+ // Texels go to the caster, not the 10000 px floor.
+ expect(f!.texelWorldSize).toBeLessThan(0.2);
+ });
+
+ it("aims a spot light's perspective frustum along its direction", () => {
+ const caster = box(0, 0, 500, 50);
+ const f = fitShadowFrustum(
+ { type: "spot", position: [0, 0, 0], direction: [0, 0, 1], angle: 30 },
+ [caster],
+ [caster],
+ 1024,
+ );
+ const [x, y, z] = project(f!.viewProj, [0, 0, 500]);
+ expect(Math.abs(x)).toBeLessThan(1e-6);
+ expect(Math.abs(y)).toBeLessThan(1e-6);
+ expect(z).toBeGreaterThan(0);
+ expect(z).toBeLessThan(1);
+ });
+
+ it("returns null without casters", () => {
+ expect(
+ fitShadowFrustum(
+ { type: "directional", position: [0, 0, 0], direction: [0, 1, 0] },
+ [],
+ [],
+ 1024,
+ ),
+ ).toBeNull();
+ });
+
+ it("transforms bounds by a model matrix", () => {
+ const m: Mat4 = [2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 2, 0, 10, 20, 30, 1];
+ expect(transformAabb({ min: [-1, -1, -1], max: [1, 1, 1] }, m)).toEqual({
+ min: [8, 18, 28],
+ max: [12, 22, 32],
+ });
+ });
+});
+
+describe("Light3D physical block", () => {
+ it("switches on physical shading and tone mapping only with an environment", () => {
+ const legacy = Light3D.packLightsUniforms([
+ { type: "ambient", intensity: 1 },
+ ]);
+ expect(legacy[6]).toBe(0);
+ const physical = Light3D.packLightsUniforms([], { preset: "studio" });
+ expect(physical[5]).toBe(1); // lit even with no lights
+ expect(physical[6]).toBe(1);
+ expect(physical[7]).toBe(3); // neutral by default
+ const agx = Light3D.packLightsUniforms([], {
+ toneMapping: "agx",
+ exposure: 1,
+ });
+ expect(agx[7]).toBe(2);
+ expect(agx[8 + 8 * 16 + 3]).toBeCloseTo(2); // exposure +1 stop
+ });
+
+ it("indexes the shadow light among non-ambient lights", () => {
+ const lights = [
+ { type: "ambient" as const },
+ { type: "point" as const },
+ { type: "directional" as const, castShadow: true },
+ ];
+ expect(Light3D.shadowCasterIndex(lights)).toBe(1);
+ expect(
+ Light3D.shadowCasterIndex([{ type: "point" as const, castShadow: true }]),
+ ).toBe(-1);
+ });
+});
diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts
new file mode 100644
index 00000000..a22a0674
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/shadow-frustum.ts
@@ -0,0 +1,160 @@
+/**
+ * Fits a shadow-casting light's view-projection around the scene: tight on
+ * the casters (so the map's texels go where shadows are), deep enough to
+ * cover every receiver (a floor far bigger than the casters still darkens).
+ */
+import { type Mat4, Matrix4Math } from "../math3d/matrix4.js";
+import { type Vec3, Vector3Math } from "../math3d/vector3.js";
+
+export interface Aabb {
+ min: Vec3;
+ max: Vec3;
+}
+
+export interface ShadowLight {
+ type: "directional" | "spot";
+ /** World position (spot); directional lights only use `direction`. */
+ position: Vec3;
+ /** Unit direction the light travels. */
+ direction: Vec3;
+ /** Spot cone half-angle in degrees. */
+ angle?: number;
+}
+
+export interface ShadowFrustum {
+ viewProj: Mat4;
+ /** World px covered by one shadow-map texel (at the casters, for spots). */
+ texelWorldSize: number;
+}
+
+export function aabbCorners(box: Aabb): Vec3[] {
+ const { min, max } = box;
+ return [
+ [min[0], min[1], min[2]],
+ [max[0], min[1], min[2]],
+ [min[0], max[1], min[2]],
+ [max[0], max[1], min[2]],
+ [min[0], min[1], max[2]],
+ [max[0], min[1], max[2]],
+ [min[0], max[1], max[2]],
+ [max[0], max[1], max[2]],
+ ];
+}
+
+/** World-space bounds of a local box under a column-major model matrix. */
+export function transformAabb(box: Aabb, m: ArrayLike<number>): Aabb {
+ const min: Vec3 = [Infinity, Infinity, Infinity];
+ const max: Vec3 = [-Infinity, -Infinity, -Infinity];
+ for (const [x, y, z] of aabbCorners(box)) {
+ const w = [
+ m[0] * x + m[4] * y + m[8] * z + m[12],
+ m[1] * x + m[5] * y + m[9] * z + m[13],
+ m[2] * x + m[6] * y + m[10] * z + m[14],
+ ];
+ for (let i = 0; i < 3; i++) {
+ if (w[i] < min[i]) min[i] = w[i];
+ if (w[i] > max[i]) max[i] = w[i];
+ }
+ }
+ return { min, max };
+}
+
+function transformPoint(m: Mat4, p: Vec3): Vec3 {
+ return [
+ m[0] * p[0] + m[4] * p[1] + m[8] * p[2] + m[12],
+ m[1] * p[0] + m[5] * p[1] + m[9] * p[2] + m[13],
+ m[2] * p[0] + m[6] * p[1] + m[10] * p[2] + m[14],
+ ];
+}
+
+/** An up vector not parallel to `dir` (world y points down). */
+function upFor(dir: Vec3): Vec3 {
+ return Math.abs(dir[1]) > 0.98 ? [0, 0, 1] : [0, -1, 0];
+}
+
+export function fitShadowFrustum(
+ light: ShadowLight,
+ casters: Aabb[],
+ receivers: Aabb[],
+ mapSize: number,
+): ShadowFrustum | null {
+ if (casters.length === 0) return null;
+ const dir = Vector3Math.normalize(light.direction);
+ const casterPts = casters.flatMap(aabbCorners);
+ const allPts = [...casterPts, ...receivers.flatMap(aabbCorners)];
+
+ if (light.type === "directional") {
+ const center: Vec3 = [0, 0, 0];
+ for (const p of casterPts) {
+ center[0] += p[0] / casterPts.length;
+ center[1] += p[1] / casterPts.length;
+ center[2] += p[2] / casterPts.length;
+ }
+ const eye = Vector3Math.subtract(
+ center,
+ Vector3Math.multiplyScalar(dir, 1),
+ );
+ const view = Matrix4Math.lookAt(eye, center, upFor(dir));
+ let minX = Infinity;
+ let maxX = -Infinity;
+ let minY = Infinity;
+ let maxY = -Infinity;
+ for (const p of casterPts) {
+ const v = transformPoint(view, p);
+ minX = Math.min(minX, v[0]);
+ maxX = Math.max(maxX, v[0]);
+ minY = Math.min(minY, v[1]);
+ maxY = Math.max(maxY, v[1]);
+ }
+ let minZ = Infinity;
+ let maxZ = -Infinity;
+ for (const p of allPts) {
+ const z = transformPoint(view, p)[2];
+ minZ = Math.min(minZ, z);
+ maxZ = Math.max(maxZ, z);
+ }
+ // Room for blurred edges, and square texels.
+ const half = Math.max(maxX - minX, maxY - minY) * 0.5 * 1.08 + 4;
+ const cx = (minX + maxX) / 2;
+ const cy = (minY + maxY) / 2;
+ const depthPad = Math.max(10, (maxZ - minZ) * 0.02);
+ const proj = Matrix4Math.orthographic(
+ cx - half,
+ cx + half,
+ cy - half,
+ cy + half,
+ minZ - depthPad,
+ maxZ + depthPad,
+ );
+ return {
+ viewProj: Matrix4Math.multiply(proj, view),
+ texelWorldSize: (2 * half) / mapSize,
+ };
+ }
+
+ const eye = light.position;
+ const view = Matrix4Math.lookAt(eye, Vector3Math.add(eye, dir), upFor(dir));
+ let near = Infinity;
+ let far = 0;
+ let casterDist = 0;
+ for (const p of allPts) {
+ const z = transformPoint(view, p)[2];
+ if (z > 0) {
+ near = Math.min(near, z);
+ far = Math.max(far, z);
+ }
+ }
+ for (const p of casterPts) {
+ casterDist = Math.max(casterDist, transformPoint(view, p)[2]);
+ }
+ if (!Number.isFinite(near) || far <= 0) return null;
+ near = Math.max(1, near * 0.9);
+ far = far * 1.05 + 1;
+ const fovRad = (Math.min(170, 2 * (light.angle ?? 45) + 4) * Math.PI) / 180;
+ const proj = Matrix4Math.perspective(fovRad, 1, near, far);
+ return {
+ viewProj: Matrix4Math.multiply(proj, view),
+ texelWorldSize:
+ (2 * Math.tan(fovRad / 2) * Math.max(casterDist, near)) / mapSize,
+ };
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts
index fe756b66..5f981bbc 100644
--- a/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts
@@ -13,6 +13,11 @@ export interface ShadowMapOptions {
slopeBias?: number;
}
+/**
+ * @deprecated A texture holder only; nothing renders into it. Shadows are
+ * drawn by `Renderer3D.renderShadowMap` (a `castShadow` directional or spot
+ * light in the scene).
+ */
export class ShadowPipeline {
public shadowDepthTexture: GPUTexture;
public shadowDepthView: GPUTextureView;
@@ -26,7 +31,8 @@ export class ShadowPipeline {
label: "shadow-depth-map",
size: [resolution, resolution, 1],
format: "depth32float",
- usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,
});
this.shadowDepthView = this.shadowDepthTexture.createView({
@@ -50,7 +56,14 @@ export class ShadowPipeline {
): Mat4 {
const viewMatrix = Matrix4Math.lookAt(lightPos, targetPos, [0, 1, 0]);
const half = orthoSize * 0.5;
- const projMatrix = Matrix4Math.orthographic(-half, half, -half, half, near, far);
+ const projMatrix = Matrix4Math.orthographic(
+ -half,
+ half,
+ -half,
+ half,
+ near,
+ far,
+ );
this.lightViewProjMatrix = Matrix4Math.multiply(projMatrix, viewMatrix);
return this.lightViewProjMatrix;
}
diff --git a/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts
index 5a62b3c5..94b0bfa5 100644
--- a/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts
@@ -2,10 +2,13 @@ import type { Mat4 } from "../math3d/index.js";
import { BufferPool } from "../renderer2d/buffer-pool.js";
import { slug3dWgsl } from "../shaders/slug3d.js";
import type { SlugFont } from "../slug/slug-loader.js";
+import { fullCoverageView, matteLayoutEntry } from "./matte3d.js";
+import { BloomPipelineVariants } from "./pipeline-variants.js";
import {
type MaterialUniformData,
packMaterialParams,
} from "./quad3d-pipeline.js";
+import { packSurfaceParams, SURFACE_PARAMS_FLOATS } from "./surface-params.js";
export interface Slug3DUniformData extends MaterialUniformData {
/** Paragraph px to world */
@@ -17,6 +20,8 @@ export interface Slug3DUniformData extends MaterialUniformData {
twoSided: boolean;
/** Fraction of the distance to the eye the glyphs slide toward it, to stack on their plane */
depthPull: number;
+ /** Screen-space track matte: the text shows where this texture's alpha does. */
+ matte?: GPUTexture;
}
const INSTANCE_FLOATS = 24;
@@ -26,13 +31,14 @@ export class Slug3DPipeline {
private device: GPUDevice;
private pipelineSingle: GPURenderPipeline;
private pipelineMrt: GPURenderPipeline;
+ private bloomVariants: BloomPipelineVariants;
private modelLayout: GPUBindGroupLayout;
private fontLayout: GPUBindGroupLayout;
private modelPool: BufferPool;
private instancePool: BufferPool;
private fontBindGroups = new WeakMap<GPUTexture, GPUBindGroup>();
- // 16 (model) + 16 (normal) + 4 (params) + 4 (materialParams) = 40 floats (160 bytes)
- private modelData = new Float32Array(40);
+ // 16 (model) + 16 (normal) + 4 (params) + 4 (materialParams) + 8 (pbr, emissive) = 48 floats (192 bytes)
+ private modelData = new Float32Array(40 + SURFACE_PARAMS_FLOATS);
constructor(
device: GPUDevice,
@@ -45,7 +51,7 @@ export class Slug3DPipeline {
) {
this.device = device;
this.modelPool = new BufferPool(
- 160,
+ (40 + SURFACE_PARAMS_FLOATS) * 4,
GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
);
this.instancePool = new BufferPool(
@@ -66,6 +72,7 @@ export class Slug3DPipeline {
visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT,
buffer: { type: "uniform" },
},
+ matteLayoutEntry(1),
],
});
@@ -163,6 +170,17 @@ export class Slug3DPipeline {
primitive,
multisample,
});
+
+ this.bloomVariants = new BloomPipelineVariants(device, {
+ label: "Slug3DPipeline",
+ layout,
+ vertex,
+ module,
+ format,
+ depthStencil,
+ primitive,
+ multisample,
+ });
}
resetPools(): void {
@@ -179,6 +197,7 @@ export class Slug3DPipeline {
instanceCount: number,
data: Slug3DUniformData,
useMrt = false,
+ useBloom = false,
): void {
if (instanceCount === 0 || !font.curvesTex || !font.bandsTex) return;
@@ -189,18 +208,31 @@ export class Slug3DPipeline {
this.modelData[34] = data.twoSided ? 1.0 : 0.0;
this.modelData[35] = data.depthPull;
this.modelData.set(packMaterialParams(data), 36);
+ packSurfaceParams(data, this.modelData, 40);
const modelBuffer = this.modelPool.getBuffer(this.device, this.modelData);
const modelBindGroup = this.device.createBindGroup({
layout: this.modelLayout,
- entries: [{ binding: 0, resource: { buffer: modelBuffer } }],
+ entries: [
+ { binding: 0, resource: { buffer: modelBuffer } },
+ {
+ binding: 1,
+ resource: data.matte?.createView() ?? fullCoverageView(this.device),
+ },
+ ],
});
const instanceBuffer = this.instancePool.getBuffer(
this.device,
instances.subarray(0, instanceCount * INSTANCE_FLOATS),
);
- pass.setPipeline(useMrt ? this.pipelineMrt : this.pipelineSingle);
+ pass.setPipeline(
+ useBloom
+ ? this.bloomVariants.get(useMrt)
+ : useMrt
+ ? this.pipelineMrt
+ : this.pipelineSingle,
+ );
pass.setBindGroup(0, cameraBindGroup);
pass.setBindGroup(1, modelBindGroup);
pass.setBindGroup(2, this.fontBindGroup(font.curvesTex, font.bandsTex));
diff --git a/packages/webgpu-renderers/src/renderer3d/surface-params.ts b/packages/webgpu-renderers/src/renderer3d/surface-params.ts
new file mode 100644
index 00000000..2afaf3c1
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/surface-params.ts
@@ -0,0 +1,46 @@
+import { parseColor } from "../color.js";
+import type { Color } from "../renderer2d/index.js";
+
+/** Physically based surface inputs every 3D draw (quad, text, mesh) accepts. */
+export interface SurfaceParams {
+ /** Microsurface roughness 0 (mirror) .. 1 (chalk). Physical scenes default to 0.5. */
+ roughness?: number;
+ /** 0 = dielectric (plastic, paint, wood), 1 = metal. */
+ metallic?: number;
+ /** Light the surface gives off (a CSS color), added on top of lighting. */
+ emissive?: string | Color | [number, number, number];
+ /** Emissive multiplier; values above 1 glow past white and feed bloom. Default 1. */
+ emissiveIntensity?: number;
+ /** Casts into a `castShadow` light's shadow map (meshes default on, quads off). */
+ castShadow?: boolean;
+}
+
+/** Floats appended to each 3D model uniform block: pbr vec4 + emissive vec4. */
+export const SURFACE_PARAMS_FLOATS = 8;
+
+export function packSurfaceParams(
+ data: SurfaceParams,
+ out: Float32Array,
+ offset: number,
+): void {
+ out[offset] = data.roughness ?? -1;
+ out[offset + 1] = data.metallic ?? 0;
+ out[offset + 2] = 0;
+ out[offset + 3] = 0;
+ const e = data.emissive;
+ if (e === undefined || e === null) {
+ out.fill(0, offset + 4, offset + 8);
+ return;
+ }
+ let rgb: [number, number, number];
+ if (Array.isArray(e)) {
+ rgb = [e[0], e[1], e[2]];
+ } else {
+ const c = parseColor(e as string | Color);
+ rgb = [c.r * c.a, c.g * c.a, c.b * c.a];
+ }
+ out[offset + 4] = rgb[0];
+ out[offset + 5] = rgb[1];
+ out[offset + 6] = rgb[2];
+ out[offset + 7] = data.emissiveIntensity ?? 1;
+}
diff --git a/packages/webgpu-renderers/src/shaders/composite.ts b/packages/webgpu-renderers/src/shaders/composite.ts
index 8fff1f76..b834f47a 100644
--- a/packages/webgpu-renderers/src/shaders/composite.ts
+++ b/packages/webgpu-renderers/src/shaders/composite.ts
@@ -85,6 +85,9 @@ fn set_sat(c: vec3<f32>, s: f32) -> vec3<f32> {
case 23u: { return s + d; } // lighter
case 24u: { return s; } // copy
case 25u: { return s * (1.0 - da) + d * (1.0 - sa); } // xor
+ // Luma mattes: the overlay's brightness (premultiplied, so transparent = black) masks the base.
+ case 26u: { return d * clamp(dot(s.rgb, vec3<f32>(0.2126, 0.7152, 0.0722)), 0.0, 1.0); } // luma-in
+ case 27u: { return d * (1.0 - clamp(dot(s.rgb, vec3<f32>(0.2126, 0.7152, 0.0722)), 0.0, 1.0)); } // luma-out
default: {
if (sa <= 0.0) { return d; }
if (da <= 0.0) { return s; }
diff --git a/packages/webgpu-renderers/src/shaders/gradient.ts b/packages/webgpu-renderers/src/shaders/gradient.ts
new file mode 100644
index 00000000..a033d059
--- /dev/null
+++ b/packages/webgpu-renderers/src/shaders/gradient.ts
@@ -0,0 +1,69 @@
+/**
+ * Multi-stop linear / radial / conic gradients, blended in OKLab, shared by
+ * the solid (rounded rect backgrounds) and shape-fill pipelines. The including
+ * shader embeds a `GradientUniforms` in its uniform block.
+ */
+export const gradientWgsl = `
+struct GradientUniforms {
+ info : vec4<f32>, // x = type (0 none, 1 linear, 2 radial, 3 conic), y = stop count
+ geom : vec4<f32>, // linear: cx, cy, dir.x / len, dir.y / len; radial: cx, cy, rx, ry; conic: cx, cy, from (rad)
+ offsets : array<vec4<f32>, 2>,
+ stops : array<vec4<f32>, 8>, // OKLab + alpha
+};
+
+fn gradientOklabToSrgb(lab: vec3<f32>) -> vec3<f32> {
+ let l = pow(lab.x + 0.3963377774 * lab.y + 0.2158037573 * lab.z, 3.0);
+ let m = pow(lab.x - 0.1055613458 * lab.y - 0.0638541728 * lab.z, 3.0);
+ let s = pow(lab.x - 0.0894841775 * lab.y - 1.291485548 * lab.z, 3.0);
+ let lin = clamp(vec3<f32>(
+ 4.0767416621 * l - 3.3077115913 * m + 0.2309699292 * s,
+ -1.2684380046 * l + 2.6097574011 * m - 0.3413193965 * s,
+ -0.0041960863 * l - 0.7034186147 * m + 1.707614701 * s,
+ ), vec3<f32>(0.0), vec3<f32>(1.0));
+ let lo = lin * 12.92;
+ let hi = 1.055 * pow(lin, vec3<f32>(1.0 / 2.4)) - 0.055;
+ return select(hi, lo, lin <= vec3<f32>(0.0031308));
+}
+
+/// Position along the gradient (0..1, unclamped for linear/radial) at local point p.
+fn gradientT(g: GradientUniforms, p: vec2<f32>) -> f32 {
+ let d = p - g.geom.xy;
+ let kind = g.info.x;
+ if (kind < 1.5) {
+ return dot(d, g.geom.zw) + 0.5;
+ }
+ if (kind < 2.5) {
+ return length(d / max(g.geom.zw, vec2<f32>(1e-4)));
+ }
+ let a = atan2(d.x, -d.y) - g.geom.z;
+ return fract(a / 6.28318530718);
+}
+
+/// Straight sRGB color + alpha of the gradient at local point p.
+fn evalGradient(g: GradientUniforms, p: vec2<f32>) -> vec4<f32> {
+ var offs = g.offsets;
+ var stops = g.stops;
+ let n = i32(g.info.y);
+ let t = gradientT(g, p);
+ var lab = stops[0];
+ if (t > offs[0].x) {
+ lab = stops[n - 1];
+ for (var i = 1; i < 8; i++) {
+ if (i >= n) {
+ break;
+ }
+ let b = offs[i / 4][i % 4];
+ if (t <= b) {
+ let a = offs[(i - 1) / 4][(i - 1) % 4];
+ let f = select(1.0, (t - a) / (b - a), b - a > 1e-5);
+ lab = mix(stops[i - 1], stops[i], clamp(f, 0.0, 1.0));
+ break;
+ }
+ }
+ }
+ return vec4<f32>(gradientOklabToSrgb(lab.xyz), lab.w);
+}
+`;
+
+/** Floats a `GradientUniforms` block takes. */
+export const GRADIENT_UNIFORM_FLOATS = 48;
diff --git a/packages/webgpu-renderers/src/shaders/lighting3d.ts b/packages/webgpu-renderers/src/shaders/lighting3d.ts
index 80124f67..9d440d3d 100644
--- a/packages/webgpu-renderers/src/shaders/lighting3d.ts
+++ b/packages/webgpu-renderers/src/shaders/lighting3d.ts
@@ -1,8 +1,15 @@
/**
- * Camera and light uniforms plus Blinn-Phong lighting shared by the 3D quad
- * and 3D text shaders, so text on a plane is lit exactly like a textured quad.
- * The including shader declares `cam : CameraUniforms` and
- * `lights : LightsUniforms`.
+ * Camera and light uniforms plus the surface shading shared by the 3D quad,
+ * 3D text and mesh shaders, so every surface in a scene is lit the same way.
+ * The including shader declares `cam : CameraUniforms` and binds
+ * `lightsBindingsWgsl` (lights, shadow map, shadow sampler) at group 3.
+ *
+ * Two shading modes, picked per scene by `lights.params.z`:
+ * - legacy (no `Layer.environment`): Blinn-Phong in display space, exactly
+ * as before, plus shadows and emissive when a scene asks for them;
+ * - physical (`Layer.environment(...)` present): linear-light GGX PBR with a
+ * procedural studio environment (image-based light), emissive, exposure,
+ * AgX / ACES tone mapping and display-space fog.
*/
export const lighting3dWgsl = `
struct CameraUniforms {
@@ -19,99 +26,407 @@ struct CameraUniforms {
struct GPULight {
posType : vec4<f32>, // xyz = position, w = type (0=ambient, 1=directional, 2=point, 3=spot)
dirRadius : vec4<f32>, // xyz = normalized direction, w = radius / range
- colorIntensity : vec4<f32>, // rgb = linear color, w = intensity
+ colorIntensity : vec4<f32>, // rgb = sRGB color, w = intensity
spotParams : vec4<f32>, // x = cos(outer), y = cos(inner), z = decay, w = unused
};
struct LightsUniforms {
- ambientColor : vec4<f32>, // rgb = ambient color * intensity, w = unused
- params : vec4<f32>, // x = numLights, y = hasLights (1.0 = lit, 0.0 = unlit), z = unused, w = unused
- lights : array<GPULight, 8>,
+ ambientColor : vec4<f32>, // rgb = ambient color * intensity (sRGB), w = unused
+ params : vec4<f32>, // x = numLights, y = hasLights, z = physical (1), w = tone mapping (0 none, 1 ACES, 2 AgX, 3 PBR Neutral)
+ lights : array<GPULight, 8>,
+ // Appended after the original layout so older readers of the prefix still line up.
+ ambientLinear : vec4<f32>, // rgb = ambient irradiance (linear), w = exposure multiplier
+ envTop : vec4<f32>, // rgb = sky radiance (linear), w = environment rotation (rad)
+ envHorizon : vec4<f32>, // rgb = horizon radiance, w = softbox strength
+ envBottom : vec4<f32>, // rgb = ground radiance, w = environment enabled
+ fog : vec4<f32>, // rgb = fog color (display linear), w = enabled
+ fogRange : vec4<f32>, // x = start distance, y = full distance, z = max opacity, w = unused
+ shadowParams : vec4<f32>, // x = enabled, y = shadowed light index, z = softness (texels), w = map size
+ shadowBias : vec4<f32>, // x = depth bias, y = normal offset (world px), z = darkness 0..1, w = unused
+ shadowMatrix : mat4x4<f32>,
};
+const PI = 3.14159265359;
+
+fn srgbToLinear(c: vec3<f32>) -> vec3<f32> {
+ let lo = c / 12.92;
+ let hi = pow((max(c, vec3<f32>(0.0)) + 0.055) / 1.055, vec3<f32>(2.4));
+ return select(hi, lo, c <= vec3<f32>(0.04045));
+}
+
+fn linearToSrgb(c: vec3<f32>) -> vec3<f32> {
+ let x = clamp(c, vec3<f32>(0.0), vec3<f32>(1.0));
+ let lo = x * 12.92;
+ let hi = 1.055 * pow(x, vec3<f32>(1.0 / 2.4)) - 0.055;
+ return select(hi, lo, x <= vec3<f32>(0.0031308));
+}
+
+// ── Tone mapping (linear scene β†’ linear display, 0..1) ─────────────────
+
+fn rrtAndOdtFit(v: vec3<f32>) -> vec3<f32> {
+ let a = v * (v + 0.0245786) - 0.000090537;
+ let b = v * (0.983729 * v + 0.4329510) + 0.238081;
+ return a / b;
+}
+
+// Stephen Hill's fitted ACES (RRT + sRGB ODT).
+fn toneMapAces(c: vec3<f32>) -> vec3<f32> {
+ let x = c / 0.6;
+ var v = vec3<f32>(
+ dot(vec3<f32>(0.59719, 0.35458, 0.04823), x),
+ dot(vec3<f32>(0.07600, 0.90834, 0.01566), x),
+ dot(vec3<f32>(0.02840, 0.13383, 0.83777), x),
+ );
+ v = rrtAndOdtFit(v);
+ v = vec3<f32>(
+ dot(vec3<f32>(1.60475, -0.53108, -0.07367), v),
+ dot(vec3<f32>(-0.10208, 1.10813, -0.00605), v),
+ dot(vec3<f32>(-0.00327, -0.07276, 1.07602), v),
+ );
+ return clamp(v, vec3<f32>(0.0), vec3<f32>(1.0));
+}
+
+fn agxContrast(x: vec3<f32>) -> vec3<f32> {
+ let x2 = x * x;
+ let x4 = x2 * x2;
+ return 15.5 * x4 * x2 - 40.14 * x4 * x + 31.96 * x4 - 6.868 * x2 * x
+ + 0.4298 * x2 + 0.1191 * x - 0.00232;
+}
+
+// AgX (Troy Sobotka), base look: graceful highlight roll-off that keeps
+// saturated emissives from clipping to flat primaries.
+fn toneMapAgx(c: vec3<f32>) -> vec3<f32> {
+ let inset = mat3x3<f32>(
+ 0.842479062253094, 0.0423282422610123, 0.0423756549057051,
+ 0.0784335999999992, 0.878468636469772, 0.0784336,
+ 0.0792237451477643, 0.0791661274605434, 0.879142973793104,
+ );
+ let outset = mat3x3<f32>(
+ 1.19687900512017, -0.0528968517574562, -0.0529716355144438,
+ -0.0980208811401368, 1.15190312990417, -0.0980434501171241,
+ -0.0990297440797205, -0.0989611768448433, 1.15107367264116,
+ );
+ let minEv = -12.47393;
+ let maxEv = 4.026069;
+ var v = inset * max(c, vec3<f32>(0.0));
+ v = clamp(log2(max(v, vec3<f32>(1e-10))), vec3<f32>(minEv), vec3<f32>(maxEv));
+ v = (v - minEv) / (maxEv - minEv);
+ v = agxContrast(v);
+ v = outset * v;
+ return clamp(pow(max(v, vec3<f32>(0.0)), vec3<f32>(2.2)), vec3<f32>(0.0), vec3<f32>(1.0));
+}
+
+// Khronos PBR Neutral: base colors come out as authored (hue and
+// saturation kept) up to near-white; only highlights are compressed.
+fn toneMapNeutral(cIn: vec3<f32>) -> vec3<f32> {
+ let startCompression = 0.8 - 0.04;
+ let desaturation = 0.15;
+ var c = max(cIn, vec3<f32>(0.0));
+ let x = min(c.r, min(c.g, c.b));
+ let offset = select(0.04, x - 6.25 * x * x, x < 0.08);
+ c = c - offset;
+ let peak = max(c.r, max(c.g, c.b));
+ if (peak < startCompression) {
+ return c;
+ }
+ let d = 1.0 - startCompression;
+ let newPeak = 1.0 - d * d / (peak + d - startCompression);
+ c = c * (newPeak / peak);
+ let g = 1.0 - 1.0 / (desaturation * (peak - newPeak) + 1.0);
+ return mix(c, vec3<f32>(newPeak), g);
+}
+
+fn toneMap(c: vec3<f32>) -> vec3<f32> {
+ let exposed = c * lights.ambientLinear.w;
+ let mode = lights.params.w;
+ if (mode > 2.5) {
+ return toneMapNeutral(exposed);
+ }
+ if (mode > 1.5) {
+ return toneMapAgx(exposed);
+ }
+ if (mode > 0.5) {
+ return toneMapAces(exposed);
+ }
+ return clamp(exposed, vec3<f32>(0.0), vec3<f32>(1.0));
+}
+
+// ── Procedural studio environment (image-based light) ─────────────────
+
+// A rectangular-ish softbox seen from the scene: bright inside its angular
+// radius, blurring out with roughness while keeping roughly the same energy.
+fn softbox(d: vec3<f32>, axis: vec3<f32>, radius: f32, rough: f32) -> f32 {
+ let cosA = dot(d, axis);
+ let edge = cos(radius + rough * 0.9);
+ let width = 0.004 + rough * rough * 0.5;
+ let lobe = smoothstep(edge - width, edge + width * 0.25, cosA);
+ return lobe / (1.0 + rough * rough * 6.0);
+}
+
+/// Radiance arriving from direction d (world space; y points down), as a
+/// surface of the given roughness sees it.
+fn envRadiance(dirIn: vec3<f32>, rough: f32) -> vec3<f32> {
+ if (lights.envBottom.w < 0.5) {
+ return vec3<f32>(0.0);
+ }
+ let rot = lights.envTop.w;
+ let c = cos(rot);
+ let s = sin(rot);
+ let d = normalize(vec3<f32>(c * dirIn.x + s * dirIn.z, dirIn.y, -s * dirIn.x + c * dirIn.z));
+ let up = -d.y;
+ let spread = 0.35 + rough * 0.9;
+ let tUp = smoothstep(-0.05, spread, up);
+ let tDown = smoothstep(-0.05, spread, -up);
+ var col = mix(lights.envHorizon.rgb, lights.envTop.rgb, tUp);
+ col = mix(col, lights.envBottom.rgb, tDown);
+ let k = lights.envHorizon.w;
+ if (k > 0.0) {
+ // Key softbox overhead-front-left, a rim strip behind-right, a fill strip left.
+ let key = softbox(d, normalize(vec3<f32>(-0.35, -0.85, -0.4)), 0.32, rough);
+ let rim = softbox(d, normalize(vec3<f32>(0.75, -0.35, 0.55)), 0.2, rough);
+ let fill = softbox(d, normalize(vec3<f32>(-0.92, -0.15, -0.2)), 0.16, rough);
+ col += vec3<f32>(k * (key * 1.0 + rim * 0.7 + fill * 0.35));
+ }
+ return col;
+}
+
+// Karis' analytic approximation of the split-sum environment BRDF.
+fn envBrdfApprox(f0: vec3<f32>, rough: f32, nDotV: f32) -> vec3<f32> {
+ let c0 = vec4<f32>(-1.0, -0.0275, -0.572, 0.022);
+ let c1 = vec4<f32>(1.0, 0.0425, 1.04, -0.04);
+ let r = rough * c0 + c1;
+ let a004 = min(r.x * r.x, exp2(-9.28 * nDotV)) * r.x + r.y;
+ let ab = vec2<f32>(-1.04, 1.04) * a004 + r.zw;
+ return f0 * ab.x + ab.y;
+}
+
+// ── Shadows ───────────────────────────────────────────────────────────
+
+var<private> POISSON : array<vec2<f32>, 12> = array<vec2<f32>, 12>(
+ vec2<f32>(-0.326, -0.406), vec2<f32>(-0.840, -0.074), vec2<f32>(-0.696, 0.457),
+ vec2<f32>(-0.203, 0.621), vec2<f32>(0.962, -0.195), vec2<f32>(0.473, -0.480),
+ vec2<f32>(0.519, 0.767), vec2<f32>(0.185, -0.893), vec2<f32>(0.507, 0.064),
+ vec2<f32>(0.896, 0.412), vec2<f32>(-0.322, -0.933), vec2<f32>(-0.792, -0.598),
+);
+
+/// 1 = fully lit, 0 = fully shadowed, for light index i.
+fn shadowFactor(worldPos: vec3<f32>, N: vec3<f32>, i: i32) -> f32 {
+ if (lights.shadowParams.x < 0.5 || i32(lights.shadowParams.y) != i) {
+ return 1.0;
+ }
+ let p = worldPos + N * lights.shadowBias.y;
+ let lp = lights.shadowMatrix * vec4<f32>(p, 1.0);
+ let ndc = lp.xyz / lp.w;
+ let uv = vec2<f32>(ndc.x * 0.5 + 0.5, 0.5 - ndc.y * 0.5);
+ if (lp.w <= 0.0 || uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0 || ndc.z > 1.0) {
+ return 1.0;
+ }
+ let texel = 1.0 / max(lights.shadowParams.w, 1.0);
+ let radius = lights.shadowParams.z * texel;
+ let depthRef = ndc.z - lights.shadowBias.x;
+ var lit = 0.0;
+ for (var k = 0; k < 12; k++) {
+ lit += textureSampleCompareLevel(shadowMap, shadowSampler, uv + POISSON[k] * radius, depthRef);
+ }
+ lit = lit / 12.0;
+ return mix(1.0, lit, lights.shadowBias.z);
+}
+
+// ── Lights ────────────────────────────────────────────────────────────
+
+struct LightSample {
+ L : vec3<f32>,
+ attenuation : f32,
+ valid : bool,
+};
+
+fn sampleLight(i: i32, worldPos: vec3<f32>) -> LightSample {
+ var out = LightSample(vec3<f32>(0.0, 0.0, 1.0), 1.0, false);
+ let light = lights.lights[i];
+ let lightType = light.posType.w;
+ if (lightType > 0.5 && lightType < 1.5) {
+ out.L = normalize(-light.dirRadius.xyz);
+ out.valid = true;
+ return out;
+ }
+ if (lightType < 1.5) {
+ return out;
+ }
+ let toLight = light.posType.xyz - worldPos;
+ let dist = length(toLight);
+ let radius = max(1.0, light.dirRadius.w);
+ if (dist > radius) {
+ return out;
+ }
+ out.L = normalize(toLight);
+ let dNorm = clamp(dist / radius, 0.0, 1.0);
+ let decay = max(0.5, light.spotParams.z);
+ out.attenuation = pow(clamp(1.0 - dNorm, 0.0, 1.0), decay);
+ if (lightType > 2.5) {
+ let spotDir = normalize(light.dirRadius.xyz);
+ let cosAngle = dot(-out.L, spotDir);
+ if (cosAngle < light.spotParams.x) {
+ return out;
+ }
+ out.attenuation *= smoothstep(light.spotParams.x, light.spotParams.y, cosAngle);
+ }
+ out.valid = true;
+ return out;
+}
+
+// Blinn-Phong in display space: the original look, plus shadows when a
+// light casts them.
fn applyLighting3D(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32>, material: vec4<f32>) -> vec3<f32> {
- // If no lights in scene or unlit material mode, return unlit base color
if (lights.params.y < 0.5 || material.w < 0.5) {
return baseColor;
}
-
var N = normalize(worldNormal);
let V = normalize(cam.cameraPos.xyz - worldPos);
if (dot(N, V) < 0.0) {
N = -N;
}
-
let shininess = max(1.0, material.x);
let specularInt = material.y;
let ambientInt = material.z;
-
- // 1. Ambient component
+
var totalIllumination = baseColor * (lights.ambientColor.rgb * ambientInt);
-
let numLights = i32(lights.params.x);
for (var i = 0; i < 8; i++) {
if (i >= numLights) {
break;
}
+ let ls = sampleLight(i, worldPos);
+ if (!ls.valid) {
+ continue;
+ }
let light = lights.lights[i];
- let lightType = light.posType.w; // 1 = directional, 2 = point, 3 = spot
let lightColor = light.colorIntensity.rgb;
let intensity = light.colorIntensity.w;
-
- var L = vec3<f32>(0.0, 0.0, 1.0);
- var attenuation = 1.0;
-
- if (lightType > 0.5 && lightType < 1.5) {
- // Directional Light
- L = normalize(-light.dirRadius.xyz);
- } else if (lightType >= 1.5) {
- // Point or Spot Light
- let toLight = light.posType.xyz - worldPos;
- let dist = length(toLight);
- let radius = max(1.0, light.dirRadius.w);
- if (dist > radius) {
- continue;
- }
- L = normalize(toLight);
-
- // Distance attenuation
- let dNorm = clamp(dist / radius, 0.0, 1.0);
- let decay = max(0.5, light.spotParams.z);
- attenuation = pow(clamp(1.0 - dNorm, 0.0, 1.0), decay);
-
- if (lightType > 2.5) {
- // Spot Light cone angle and penumbra falloff
- let spotDir = normalize(light.dirRadius.xyz);
- let cosAngle = dot(-L, spotDir);
- let cosOuter = light.spotParams.x;
- let cosInner = light.spotParams.y;
-
- if (cosAngle < cosOuter) {
- continue;
- }
- let spotFalloff = smoothstep(cosOuter, cosInner, cosAngle);
- attenuation *= spotFalloff;
- }
- } else {
- continue;
- }
-
- // Diffuse Lambertian
- let NdotL = max(dot(N, L), 0.0);
- let diffuse = baseColor * lightColor * (NdotL * intensity * attenuation);
-
- // Specular Blinn-Phong
+ let NdotL = max(dot(N, ls.L), 0.0);
+ let diffuse = baseColor * lightColor * (NdotL * intensity * ls.attenuation);
var specular = vec3<f32>(0.0);
if (NdotL > 0.0 && specularInt > 0.001) {
- let H = normalize(L + V);
+ let H = normalize(ls.L + V);
let NdotH = max(dot(N, H), 0.0);
- let specFactor = pow(NdotH, shininess);
- specular = lightColor * (specFactor * specularInt * intensity * attenuation);
+ specular = lightColor * (pow(NdotH, shininess) * specularInt * intensity * ls.attenuation);
}
-
- totalIllumination += diffuse + specular;
+ totalIllumination += (diffuse + specular) * shadowFactor(worldPos, N, i);
}
-
return totalIllumination;
}
+
+// GGX / Smith-correlated / Schlick, image-based light from envRadiance.
+// Returns linear scene radiance (unbounded).
+fn shadePhysical(worldPos: vec3<f32>, worldNormal: vec3<f32>, albedo: vec3<f32>, rough: f32, metal: f32, ambientInt: f32) -> vec3<f32> {
+ var N = normalize(worldNormal);
+ let V = normalize(cam.cameraPos.xyz - worldPos);
+ if (dot(N, V) < 0.0) {
+ N = -N;
+ }
+ let nDotV = max(dot(N, V), 1e-4);
+ let r = clamp(rough, 0.03, 1.0);
+ let a = r * r;
+ let a2 = a * a;
+ let f0 = mix(vec3<f32>(0.04), albedo, metal);
+ let diffuseColor = albedo * (1.0 - metal);
+
+ var lo = vec3<f32>(0.0);
+ let numLights = i32(lights.params.x);
+ for (var i = 0; i < 8; i++) {
+ if (i >= numLights) {
+ break;
+ }
+ let ls = sampleLight(i, worldPos);
+ if (!ls.valid) {
+ continue;
+ }
+ let light = lights.lights[i];
+ let nDotL = max(dot(N, ls.L), 0.0);
+ if (nDotL <= 0.0) {
+ continue;
+ }
+ let H = normalize(ls.L + V);
+ let nDotH = max(dot(N, H), 0.0);
+ let vDotH = max(dot(V, H), 0.0);
+ let dDen = nDotH * nDotH * (a2 - 1.0) + 1.0;
+ let D = a2 / (PI * dDen * dDen);
+ let gv = nDotL * sqrt(nDotV * nDotV * (1.0 - a2) + a2);
+ let gl = nDotV * sqrt(nDotL * nDotL * (1.0 - a2) + a2);
+ let Vis = 0.5 / max(gv + gl, 1e-5);
+ let F = f0 + (vec3<f32>(1.0) - f0) * pow(1.0 - vDotH, 5.0);
+ let spec = D * Vis * F;
+ let diff = (vec3<f32>(1.0) - F) * diffuseColor / PI;
+ // x PI so a white light of intensity 1 lights a white wall about as
+ // brightly as the legacy shading does.
+ let radiance = srgbToLinear(light.colorIntensity.rgb) * light.colorIntensity.w * ls.attenuation * PI;
+ lo += (diff + spec) * radiance * nDotL * shadowFactor(worldPos, N, i);
+ }
+
+ let R = reflect(-V, N);
+ let irradiance = envRadiance(N, 1.0) + lights.ambientLinear.rgb;
+ let specEnv = envRadiance(R, r) * envBrdfApprox(f0, r, nDotV);
+ lo += (diffuseColor * irradiance + specEnv) * ambientInt;
+ return lo;
+}
+
+struct Shaded {
+ /** Display-encoded (sRGB) color for the color target. */
+ display : vec3<f32>,
+ /** Unclamped scene-referred linear radiance, for the bloom target. */
+ hdr : vec3<f32>,
+};
+
+/// Shades a surface for the scene's mode.
+/// material: x shininess, y specularIntensity, z ambientIntensity, w mode (0 unlit).
+/// pbr: x roughness (<0 = default), y metallic, z/w unused.
+/// emissive: rgb sRGB color, w strength.
+fn shadeSurfaceFull(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32>, material: vec4<f32>, pbr: vec4<f32>, emissive: vec4<f32>) -> Shaded {
+ var out: Shaded;
+ if (lights.params.z < 0.5) {
+ var c = baseColor;
+ if (material.w >= 0.5) {
+ c = applyLighting3D(worldPos, worldNormal, baseColor, material);
+ }
+ c += emissive.rgb * emissive.w;
+ out.display = c;
+ out.hdr = srgbToLinear(clamp(c, vec3<f32>(0.0), vec3<f32>(1.0))) * max(max(c.r, max(c.g, c.b)), 1.0);
+ return out;
+ }
+ var radiance: vec3<f32>;
+ if (material.w < 0.5) {
+ radiance = srgbToLinear(baseColor);
+ } else {
+ let rough = select(pbr.x, 0.5, pbr.x < 0.0);
+ radiance = shadePhysical(worldPos, worldNormal, srgbToLinear(baseColor), rough, clamp(pbr.y, 0.0, 1.0), material.z);
+ }
+ radiance += srgbToLinear(emissive.rgb) * emissive.w;
+ var display: vec3<f32>;
+ if (material.w < 0.5) {
+ // Unlit layers keep their exact authored colors.
+ display = clamp(radiance, vec3<f32>(0.0), vec3<f32>(1.0));
+ } else {
+ display = toneMap(radiance);
+ }
+ var hdr = radiance * lights.ambientLinear.w;
+ if (lights.fog.w > 0.5) {
+ let dist = distance(worldPos, cam.cameraPos.xyz);
+ let f = smoothstep(lights.fogRange.x, max(lights.fogRange.y, lights.fogRange.x + 1.0), dist) * lights.fogRange.z;
+ display = mix(display, lights.fog.rgb, f);
+ hdr = hdr * (1.0 - f);
+ }
+ out.display = linearToSrgb(display);
+ out.hdr = hdr;
+ return out;
+}
+
+fn shadeSurface(worldPos: vec3<f32>, worldNormal: vec3<f32>, baseColor: vec3<f32>, material: vec4<f32>, pbr: vec4<f32>, emissive: vec4<f32>) -> vec3<f32> {
+ return shadeSurfaceFull(worldPos, worldNormal, baseColor, material, pbr, emissive).display;
+}
+`;
+
+/** Group-3 bindings every lit 3D shader declares. */
+export const lightsBindingsWgsl = `
+@group(3) @binding(0) var<uniform> lights : LightsUniforms;
+@group(3) @binding(1) var shadowMap : texture_depth_2d;
+@group(3) @binding(2) var shadowSampler : sampler_comparison;
`;
diff --git a/packages/webgpu-renderers/src/shaders/matte3d.ts b/packages/webgpu-renderers/src/shaders/matte3d.ts
new file mode 100644
index 00000000..c21984ca
--- /dev/null
+++ b/packages/webgpu-renderers/src/shaders/matte3d.ts
@@ -0,0 +1,20 @@
+/**
+ * Screen-space track matte for 3D draws. The compositor resolves a matte of
+ * any mode into a coverage texture (alpha = how much of the layer shows);
+ * the draw scales its alpha by it at each pixel and discards where nothing
+ * shows, so a matted layer keeps its place in the depth buffer. Unmatted
+ * draws bind a 1Γ—1 white texture (full coverage).
+ *
+ * The including shader declares `cam : CameraUniforms`.
+ */
+export function matte3dWgsl(group: number, binding: number): string {
+ return `
+@group(${group}) @binding(${binding}) var matteTex : texture_2d<f32>;
+
+fn matteCoverage(fragPos : vec2<f32>) -> f32 {
+ let dims = vec2<f32>(textureDimensions(matteTex));
+ let p = floor(fragPos * dims / max(cam.screenParams.xy, vec2<f32>(1.0)));
+ return textureLoad(matteTex, vec2<i32>(clamp(p, vec2<f32>(0.0), dims - 1.0)), 0).a;
+}
+`;
+}
diff --git a/packages/webgpu-renderers/src/shaders/mesh3d.ts b/packages/webgpu-renderers/src/shaders/mesh3d.ts
index 1690cf1b..c7abad67 100644
--- a/packages/webgpu-renderers/src/shaders/mesh3d.ts
+++ b/packages/webgpu-renderers/src/shaders/mesh3d.ts
@@ -1,22 +1,17 @@
/**
* WebGPU WGSL shaders for 3D static and skinned meshes.
* Supports GPU Linear Blend Skinning (up to 128 bones),
- * Multi-Light Blinn-Phong illumination (Ambient, Directional, Point, Spot) or toon shading,
+ * the shared 3D surface shading (Blinn-Phong, or physical GGX + image-based light
+ * in scenes with an environment) or toon shading, shadow maps, emissive,
* alpha-cutoff (glTF MASK) materials,
* Material parameters, normal transformations, and linear depth MRT output for Bokeh DoF.
*/
+import { lighting3dWgsl, lightsBindingsWgsl } from "./lighting3d.js";
+import { matte3dWgsl } from "./matte3d.js";
+
export const mesh3dWgsl = `
-struct CameraUniforms {
- viewMatrix : mat4x4<f32>,
- projMatrix : mat4x4<f32>,
- viewProjMatrix : mat4x4<f32>,
- invViewProjMatrix : mat4x4<f32>,
- cameraPos : vec4<f32>, // xyz = eye, w = near
- cameraDir : vec4<f32>, // xyz = forward, w = far
- dofParams : vec4<f32>, // x = focusDistance, y = fStop, z = maxBlurRadius, w = dofEnabled
- screenParams : vec4<f32>, // x = surfaceWidth, y = surfaceHeight, z = time, w = unused
-};
+${lighting3dWgsl}
struct ModelUniforms {
modelMatrix : mat4x4<f32>,
@@ -25,31 +20,25 @@ struct ModelUniforms {
params : vec4<f32>, // x = opacity, y = twoSided (1.0 = yes), z = isSkinned (1.0 = yes), w = wireframe
materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit, 2=toon)
shade : vec4<f32>, // rgb = toon shadow-side color multiplier, a = alpha cutoff (0 = none)
+ pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
+ emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
+ maps : vec4<f32>, // x = normal scale (0 = no map), y = occlusion strength, z = has metallic-roughness map, w = has emissive map
};
struct SkinningUniforms {
jointMatrices : array<mat4x4<f32>, 128>,
};
-struct GPULight {
- posType : vec4<f32>, // xyz = position, w = type (0=ambient, 1=directional, 2=point, 3=spot)
- dirRadius : vec4<f32>, // xyz = normalized direction, w = radius / range
- colorIntensity : vec4<f32>, // rgb = linear color, w = intensity
- spotParams : vec4<f32>, // x = cos(outer), y = cos(inner), z = decay, w = unused
-};
-
-struct LightsUniforms {
- ambientColor : vec4<f32>, // rgb = ambient color * intensity, w = unused
- params : vec4<f32>, // x = numLights, y = hasLights (1.0 = lit, 0.0 = unlit), z = unused, w = unused
- lights : array<GPULight, 8>,
-};
-
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@group(1) @binding(0) var<uniform> model : ModelUniforms;
@group(1) @binding(1) var<uniform> skinning : SkinningUniforms;
-@group(2) @binding(0) var texSampler : sampler;
+${matte3dWgsl(1, 2)}@group(2) @binding(0) var texSampler : sampler;
@group(2) @binding(1) var mainTex : texture_2d<f32>;
-@group(3) @binding(0) var<uniform> lights : LightsUniforms;
+@group(2) @binding(2) var normalTex : texture_2d<f32>;
+@group(2) @binding(3) var metalRoughTex : texture_2d<f32>;
+@group(2) @binding(4) var occlusionTex : texture_2d<f32>;
+@group(2) @binding(5) var emissiveTex : texture_2d<f32>;
+${lightsBindingsWgsl}
struct VertexInput {
@location(0) position : vec3<f32>,
@@ -145,88 +134,88 @@ fn toonLighting(in: VertexOutput, baseColor: vec3<f32>) -> vec3<f32> {
return baseColor * mix(model.shade.rgb, vec3<f32>(1.0), lit);
}
-fn calculateLighting(in: VertexOutput, isFront: bool, baseColor: vec3<f32>) -> vec3<f32> {
- if (lights.params.y < 0.5 || model.materialParams.w < 0.5) {
- return baseColor;
- }
- if (model.materialParams.w > 1.5) {
- return toonLighting(in, baseColor);
- }
-
- var N = normalize(in.worldNormal);
- let V = normalize(cam.cameraPos.xyz - in.worldPos);
- if (dot(N, V) < 0.0) {
- N = -N;
- }
-
- let shininess = max(1.0, model.materialParams.x);
- let specularInt = model.materialParams.y;
- let ambientInt = model.materialParams.z;
+struct MeshFragment {
+ color : vec4<f32>, // premultiplied display color
+ hdr : vec4<f32>, // premultiplied scene radiance (bloom source)
+};
- var totalIllumination = baseColor * (lights.ambientColor.rgb * ambientInt);
+// Normal mapping without tangents: a cotangent frame from screen-space
+// derivatives (SchΓΌler, "Followup: Normal Mapping Without Precomputed Tangents").
+fn perturbNormal(N: vec3<f32>, p: vec3<f32>, uv: vec2<f32>, tn: vec3<f32>, dp1: vec3<f32>, dp2: vec3<f32>, duv1: vec2<f32>, duv2: vec2<f32>) -> vec3<f32> {
+ let dp2perp = cross(dp2, N);
+ let dp1perp = cross(N, dp1);
+ // The derivation assumes cross(dp1, dp2) points along N; with this
+ // engine's y-down view basis it can point away, mirroring the frame.
+ let handed = select(1.0, -1.0, dot(cross(dp1, dp2), N) < 0.0);
+ let T = (dp2perp * duv1.x + dp1perp * duv2.x) * handed;
+ let B = (dp2perp * duv1.y + dp1perp * duv2.y) * handed;
+ let invmax = inverseSqrt(max(max(dot(T, T), dot(B, B)), 1e-20));
+ // glTF maps are +Y up while v grows down the image: green points along -B.
+ return normalize(T * (tn.x * invmax) - B * (tn.y * invmax) + N * tn.z);
+}
- let numLights = i32(lights.params.x);
- for (var i = 0; i < 8; i++) {
- if (i >= numLights) {
- break;
- }
- let light = lights.lights[i];
- let lightType = light.posType.w;
- let lightColor = light.colorIntensity.rgb;
- let intensity = light.colorIntensity.w;
+fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {
+ // Every sample and derivative up front, in uniform control flow.
+ var texColor = textureSample(mainTex, texSampler, in.uv);
+ let normalSample = textureSample(normalTex, texSampler, in.uv).xyz;
+ let mrSample = textureSample(metalRoughTex, texSampler, in.uv);
+ let aoSample = textureSample(occlusionTex, texSampler, in.uv).r;
+ let emissiveSample = textureSample(emissiveTex, texSampler, in.uv).rgb;
+ let dp1 = dpdx(in.worldPos);
+ let dp2 = dpdy(in.worldPos);
+ let duv1 = dpdx(in.uv);
+ let duv2 = dpdy(in.uv);
- var L = vec3<f32>(0.0, 0.0, 1.0);
- var attenuation = 1.0;
+ if (!isFront && model.params.y < 0.5) {
+ discard;
+ }
+ if (texColor.a < model.shade.a) {
+ discard;
+ }
+ let alpha = texColor.a * model.params.x * model.colorTint.a * matteCoverage(in.clipPos.xy);
+ if (alpha <= 0.001) {
+ discard;
+ }
- if (lightType > 0.5 && lightType < 1.5) {
- // Directional Light
- L = normalize(-light.dirRadius.xyz);
- } else if (lightType >= 1.5) {
- // Point or Spot Light
- let toLight = light.posType.xyz - in.worldPos;
- let dist = length(toLight);
- let radius = max(1.0, light.dirRadius.w);
- if (dist > radius) {
- continue;
- }
- L = normalize(toLight);
-
- let dNorm = clamp(dist / radius, 0.0, 1.0);
- let decay = max(0.5, light.spotParams.z);
- attenuation = pow(clamp(1.0 - dNorm, 0.0, 1.0), decay);
-
- if (lightType > 2.5) {
- // Spot Light
- let spotDir = normalize(light.dirRadius.xyz);
- let cosAngle = dot(-L, spotDir);
- let cosOuter = light.spotParams.x;
- let cosInner = light.spotParams.y;
-
- if (cosAngle < cosOuter) {
- continue;
- }
- let spotFalloff = smoothstep(cosOuter, cosInner, cosAngle);
- attenuation *= spotFalloff;
- }
- } else {
- continue;
+ let baseRgb = texColor.rgb * model.colorTint.rgb;
+ var out: MeshFragment;
+ if (model.materialParams.w > 1.5) {
+ var toon = baseRgb;
+ if (lights.params.y >= 0.5) {
+ toon = toonLighting(in, baseRgb);
}
+ out.color = vec4<f32>(toon * alpha, alpha);
+ out.hdr = vec4<f32>(srgbToLinear(toon) * alpha, alpha);
+ return out;
+ }
- let NdotL = max(dot(N, L), 0.0);
- let diffuse = baseColor * lightColor * (NdotL * intensity * attenuation);
-
- var specular = vec3<f32>(0.0);
- if (NdotL > 0.0 && specularInt > 0.001) {
- let H = normalize(L + V);
- let NdotH = max(dot(N, H), 0.0);
- let specFactor = pow(NdotH, shininess);
- specular = lightColor * (specFactor * specularInt * intensity * attenuation);
+ var N = normalize(in.worldNormal);
+ if (model.maps.x > 0.0) {
+ var tn = normalSample * 2.0 - 1.0;
+ tn = vec3<f32>(tn.xy * model.maps.x, tn.z);
+ // The frame follows the surface side the camera sees.
+ var Nf = N;
+ if (dot(Nf, cam.cameraPos.xyz - in.worldPos) < 0.0) {
+ Nf = -Nf;
}
-
- totalIllumination += diffuse + specular;
+ N = perturbNormal(Nf, in.worldPos, in.uv, tn, dp1, dp2, duv1, duv2);
}
-
- return totalIllumination;
+ var pbr = model.pbr;
+ if (model.maps.z > 0.5) {
+ pbr.x = select(pbr.x, 1.0, pbr.x < 0.0) * mrSample.g;
+ pbr.y = pbr.y * mrSample.b;
+ }
+ var material = model.materialParams;
+ material.z = material.z * mix(1.0, aoSample, clamp(model.maps.y, 0.0, 1.0));
+ var emissive = model.emissive;
+ if (model.maps.w > 0.5) {
+ // Linear multiply in display space is close enough for an 8-bit map.
+ emissive = vec4<f32>(emissive.rgb * emissiveSample, emissive.w);
+ }
+ let shaded = shadeSurfaceFull(in.worldPos, N, baseRgb, material, pbr, emissive);
+ out.color = vec4<f32>(shaded.display * alpha, alpha);
+ out.hdr = vec4<f32>(shaded.hdr * alpha, alpha);
+ return out;
}
struct MrtOutput {
@@ -234,48 +223,46 @@ struct MrtOutput {
@location(1) depth : vec4<f32>,
};
-@fragment
-fn fs_mrt(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtOutput {
- if (!isFront && model.params.y < 0.5) {
- discard;
- }
+struct BloomOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) hdr : vec4<f32>,
+};
- var texColor = textureSample(mainTex, texSampler, in.uv);
- if (texColor.a < model.shade.a) {
- discard;
- }
- let alpha = texColor.a * model.params.x * model.colorTint.a;
- if (alpha <= 0.001) {
- discard;
- }
+struct MrtBloomOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) depth : vec4<f32>,
+ @location(2) hdr : vec4<f32>,
+};
- let baseRgb = texColor.rgb * model.colorTint.rgb;
- let litRgb = calculateLighting(in, isFront, baseRgb);
- let premulColor = vec4<f32>(litRgb * alpha, alpha);
+@fragment
+fn fs_single(in: VertexOutput, @builtin(front_facing) isFront: bool) -> @location(0) vec4<f32> {
+ return shadeMesh(in, isFront).color;
+}
+@fragment
+fn fs_mrt(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtOutput {
var out: MrtOutput;
- out.color = premulColor;
+ out.color = shadeMesh(in, isFront).color;
out.depth = vec4<f32>(in.linearDepth, 0.0, 0.0, 1.0);
return out;
}
@fragment
-fn fs_single(in: VertexOutput, @builtin(front_facing) isFront: bool) -> @location(0) vec4<f32> {
- if (!isFront && model.params.y < 0.5) {
- discard;
- }
-
- var texColor = textureSample(mainTex, texSampler, in.uv);
- if (texColor.a < model.shade.a) {
- discard;
- }
- let alpha = texColor.a * model.params.x * model.colorTint.a;
- if (alpha <= 0.001) {
- discard;
- }
+fn fs_single_bloom(in: VertexOutput, @builtin(front_facing) isFront: bool) -> BloomOutput {
+ let f = shadeMesh(in, isFront);
+ var out: BloomOutput;
+ out.color = f.color;
+ out.hdr = f.hdr;
+ return out;
+}
- let baseRgb = texColor.rgb * model.colorTint.rgb;
- let litRgb = calculateLighting(in, isFront, baseRgb);
- return vec4<f32>(litRgb * alpha, alpha);
+@fragment
+fn fs_mrt_bloom(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtBloomOutput {
+ let f = shadeMesh(in, isFront);
+ var out: MrtBloomOutput;
+ out.color = f.color;
+ out.depth = vec4<f32>(in.linearDepth, 0.0, 0.0, 1.0);
+ out.hdr = f.hdr;
+ return out;
}
`;
diff --git a/packages/webgpu-renderers/src/shaders/quad3d.ts b/packages/webgpu-renderers/src/shaders/quad3d.ts
index cafb3ddd..035e508c 100644
--- a/packages/webgpu-renderers/src/shaders/quad3d.ts
+++ b/packages/webgpu-renderers/src/shaders/quad3d.ts
@@ -4,7 +4,8 @@
* normal matrix transformations for non-uniform scaling, and linear depth MRT output.
*/
-import { lighting3dWgsl } from "./lighting3d.js";
+import { lighting3dWgsl, lightsBindingsWgsl } from "./lighting3d.js";
+import { matte3dWgsl } from "./matte3d.js";
export const quad3dWgsl = `
${lighting3dWgsl}
@@ -16,13 +17,15 @@ struct ModelUniforms {
params : vec4<f32>, // x = opacity, y = borderRadius, z = twoSided (1.0 = yes), w = borderWidth
dimensions : vec4<f32>, // x = width, y = height, zw = texture extent the layer covers (0 = whole texture)
materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit)
+ pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
+ emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
};
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@group(1) @binding(0) var<uniform> model : ModelUniforms;
-@group(2) @binding(0) var texSampler : sampler;
+${matte3dWgsl(1, 1)}@group(2) @binding(0) var texSampler : sampler;
@group(2) @binding(1) var mainTex : texture_2d<f32>;
-@group(3) @binding(0) var<uniform> lights : LightsUniforms;
+${lightsBindingsWgsl}
struct VertexInput {
@location(0) position : vec3<f32>,
@@ -76,10 +79,6 @@ fn vs_main(in: VertexInput) -> VertexOutput {
return out;
}
-fn calculateLighting(in: VertexOutput, isFront: bool, baseColor: vec3<f32>) -> vec3<f32> {
- return applyLighting3D(in.worldPos, in.worldNormal, baseColor, model.materialParams);
-}
-
fn sdRoundedRectWithNormal(p: vec2<f32>, b: vec2<f32>, r: f32) -> vec3<f32> {
let q = abs(p) - b + vec2<f32>(r);
let sx = select(1.0, -1.0, p.x < 0.0);
@@ -103,13 +102,12 @@ fn sdRoundedRectWithNormal(p: vec2<f32>, b: vec2<f32>, r: f32) -> vec3<f32> {
}
}
-struct MrtOutput {
- @location(0) color : vec4<f32>,
- @location(1) depth : vec4<f32>,
+struct QuadFragment {
+ color : vec4<f32>, // premultiplied display color
+ hdr : vec4<f32>, // premultiplied scene radiance (bloom source)
};
-@fragment
-fn fs_mrt(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtOutput {
+fn shadeQuad(in: VertexOutput, isFront: bool) -> QuadFragment {
// One-sided quads cull against their declared normal, not the hardware
// winding: the Y-down view basis mirrors winding, so front_facing reads
// camera-facing quads as back faces.
@@ -155,73 +153,73 @@ fn fs_mrt(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtOutput {
let unmultipliedRgb = select(texColor.rgb / texAlpha, texColor.rgb, texAlpha < 0.001);
let baseRgb = mix(unmultipliedRgb * model.colorTint.rgb, model.borderColor.rgb, borderFactor);
- let alpha = mix(texAlpha, 1.0, borderFactor) * model.params.x * model.colorTint.a * outerAlpha;
+ let alpha = mix(texAlpha, 1.0, borderFactor) * model.params.x * model.colorTint.a * outerAlpha * matteCoverage(in.clipPos.xy);
if (alpha <= 0.001) {
discard;
}
- let litRgb = calculateLighting(in, isFront, baseRgb);
- let premulColor = vec4<f32>(litRgb * alpha, alpha);
-
+ let shaded = shadeSurfaceFull(in.worldPos, in.worldNormal, baseRgb, model.materialParams, model.pbr, model.emissive);
+ var out: QuadFragment;
+ out.color = vec4<f32>(shaded.display * alpha, alpha);
+ out.hdr = vec4<f32>(shaded.hdr * alpha, alpha);
+ return out;
+}
+
+struct MrtOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) depth : vec4<f32>,
+};
+
+struct BloomOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) hdr : vec4<f32>,
+};
+
+struct MrtBloomOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) depth : vec4<f32>,
+ @location(2) hdr : vec4<f32>,
+};
+
+@fragment
+fn fs_single(in: VertexOutput, @builtin(front_facing) isFront: bool) -> @location(0) vec4<f32> {
+ return shadeQuad(in, isFront).color;
+}
+
+@fragment
+fn fs_mrt(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtOutput {
var out: MrtOutput;
- out.color = premulColor;
+ out.color = shadeQuad(in, isFront).color;
out.depth = vec4<f32>(in.linearDepth, 0.0, 0.0, 1.0);
return out;
}
@fragment
-fn fs_single(in: VertexOutput, @builtin(front_facing) isFront: bool) -> @location(0) vec4<f32> {
- // One-sided quads cull against their declared normal, not the hardware
- // winding: the Y-down view basis mirrors winding, so front_facing reads
- // camera-facing quads as back faces.
- if (model.params.z < 0.5 && dot(in.worldNormal, cam.cameraPos.xyz - in.worldPos) < 0.0) {
- discard;
- }
-
- var outerAlpha = 1.0;
- var borderFactor = 0.0;
- let br = model.params.y;
- let borderWidth = model.params.w;
- let hasDimensions = model.dimensions.x > 0.0 && model.dimensions.y > 0.0;
-
- if (hasDimensions) {
- let halfSize = model.dimensions.xy * 0.5;
- let normData = sdRoundedRectWithNormal(in.localPos, halfSize, br);
-
- let uvDx = dpdx(in.localPos);
- let uvDy = dpdy(in.localPos);
-
- let dPdx = normData.y * uvDx.x + normData.z * uvDx.y;
- let dPdy = normData.y * uvDy.x + normData.z * uvDy.y;
- let gradLen = max(sqrt(dPdx * dPdx + dPdy * dPdy), 0.001);
-
- let distPixel = normData.x / gradLen;
- outerAlpha = clamp(0.5 - distPixel, 0.0, 1.0);
- if (outerAlpha <= 0.001) {
- discard;
- }
-
- if (borderWidth > 0.0 && model.borderColor.a > 0.0) {
- let borderWidthPixel = max(borderWidth / gradLen, 0.0);
- let innerAlpha = clamp(0.5 - (distPixel + borderWidthPixel), 0.0, 1.0);
- borderFactor = clamp(outerAlpha - innerAlpha, 0.0, 1.0) * model.borderColor.a;
- }
- }
-
+fn fs_single_bloom(in: VertexOutput, @builtin(front_facing) isFront: bool) -> BloomOutput {
+ let f = shadeQuad(in, isFront);
+ var out: BloomOutput;
+ out.color = f.color;
+ out.hdr = f.hdr;
+ return out;
+}
+
+@fragment
+fn fs_mrt_bloom(in: VertexOutput, @builtin(front_facing) isFront: bool) -> MrtBloomOutput {
+ let f = shadeQuad(in, isFront);
+ var out: MrtBloomOutput;
+ out.color = f.color;
+ out.depth = vec4<f32>(in.linearDepth, 0.0, 0.0, 1.0);
+ out.hdr = f.hdr;
+ return out;
+}
+
+// Shadow-map pass: depth only, alpha-tested so a cut-out casts its shape.
+@fragment
+fn fs_shadow(in: VertexOutput) {
let clampedUv = clamp(in.uv, vec2<f32>(0.0), vec2<f32>(1.0));
- let dxUV = dpdx(in.uv);
- let dyUV = dpdy(in.uv);
- var texColor = textureSampleGrad(mainTex, texSampler, clampedUv, dxUV, dyUV);
- let texAlpha = texColor.a;
-
- let unmultipliedRgb = select(texColor.rgb / texAlpha, texColor.rgb, texAlpha < 0.001);
- let baseRgb = mix(unmultipliedRgb * model.colorTint.rgb, model.borderColor.rgb, borderFactor);
- let alpha = mix(texAlpha, 1.0, borderFactor) * model.params.x * model.colorTint.a * outerAlpha;
- if (alpha <= 0.001) {
+ let a = textureSampleLevel(mainTex, texSampler, clampedUv, 0.0).a * model.params.x * model.colorTint.a;
+ if (a < 0.5) {
discard;
}
-
- let litRgb = calculateLighting(in, isFront, baseRgb);
- return vec4<f32>(litRgb * alpha, alpha);
}
`;
diff --git a/packages/webgpu-renderers/src/shaders/shape-fill.ts b/packages/webgpu-renderers/src/shaders/shape-fill.ts
index 68e5fd70..aa75f967 100644
--- a/packages/webgpu-renderers/src/shaders/shape-fill.ts
+++ b/packages/webgpu-renderers/src/shaders/shape-fill.ts
@@ -1,12 +1,15 @@
+import { gradientWgsl } from "./gradient.js";
+
export const shapeFillWgsl = `
+${gradientWgsl}
+
struct Uniforms {
transformCol0 : vec4<f32>,
transformCol1 : vec4<f32>,
transformCol2 : vec4<f32>,
- params : vec4<f32>, // x = opacity, y = surfaceWidth, z = surfaceHeight, w = fillType (0=solid, 1=linear, 2=radial)
- color : vec4<f32>, // solid color or gradient start
- color2 : vec4<f32>, // gradient end
- gradCoords : vec4<f32>, // linear: (x0, y0, x1, y1), radial: (cx, cy, radius, unused)
+ params : vec4<f32>, // x = opacity, y = surfaceWidth, z = surfaceHeight, w = unused
+ color : vec4<f32>, // solid color (gradient.info.x = 0)
+ gradient : GradientUniforms,
};
@group(0) @binding(0) var<uniform> u : Uniforms;
@@ -29,23 +32,8 @@ struct VSOut {
@fragment fn fs(in : VSOut) -> @location(0) vec4<f32> {
var outColor = u.color;
- let fillType = u.params.w;
-
- if (fillType > 0.5 && fillType < 1.5) {
- // Linear gradient between p0 and p1
- let p0 = u.gradCoords.xy;
- let p1 = u.gradCoords.zw;
- let d = p1 - p0;
- let lenSq = dot(d, d);
- let t = clamp(dot(in.localPos - p0, d) / max(lenSq, 1e-6), 0.0, 1.0);
- outColor = mix(u.color, u.color2, t);
- } else if (fillType >= 1.5) {
- // Radial gradient from center out to radius
- let center = u.gradCoords.xy;
- let radius = max(u.gradCoords.z, 1e-6);
- let dist = length(in.localPos - center);
- let t = clamp(dist / radius, 0.0, 1.0);
- outColor = mix(u.color, u.color2, t);
+ if (u.gradient.info.x > 0.5) {
+ outColor = evalGradient(u.gradient, in.localPos);
}
let alpha = outColor.a * u.params.x;
diff --git a/packages/webgpu-renderers/src/shaders/slug3d.ts b/packages/webgpu-renderers/src/shaders/slug3d.ts
index 4f2f308b..7a50b80c 100644
--- a/packages/webgpu-renderers/src/shaders/slug3d.ts
+++ b/packages/webgpu-renderers/src/shaders/slug3d.ts
@@ -5,7 +5,8 @@
* angle, with no intermediate texture to magnify or minify.
*/
-import { lighting3dWgsl } from "./lighting3d.js";
+import { lighting3dWgsl, lightsBindingsWgsl } from "./lighting3d.js";
+import { matte3dWgsl } from "./matte3d.js";
import { slugCoverageWgsl } from "./slug.js";
export const slug3dWgsl = `
@@ -16,13 +17,15 @@ struct TextModelUniforms {
normalMatrix : mat4x4<f32>,
params : vec4<f32>, // x = opacity, y = slant, z = twoSided (1.0 = yes), w = pull toward the eye
materialParams : vec4<f32>, // x = shininess, y = specularIntensity, z = ambientIntensity, w = materialMode (0=unlit, 1=lit)
+ pbr : vec4<f32>, // x = roughness (<0 = default), y = metallic
+ emissive : vec4<f32>, // rgb = sRGB emissive color, w = strength
};
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@group(1) @binding(0) var<uniform> model : TextModelUniforms;
-@group(2) @binding(0) var curvesTex : texture_2d<f32>;
+${matte3dWgsl(1, 1)}@group(2) @binding(0) var curvesTex : texture_2d<f32>;
@group(2) @binding(1) var bandsTex : texture_2d<u32>;
-@group(3) @binding(0) var<uniform> lights : LightsUniforms;
+${lightsBindingsWgsl}
${slugCoverageWgsl}
@@ -119,11 +122,18 @@ fn pixelsPerUnitAlong(c: vec4<f32>, dc: vec4<f32>) -> vec2<f32> {
return out;
}
-// Lit, premultiplied glyph colour; alpha 0 means discard.
-fn shadeText(in : VSOut, fw : vec2<f32>) -> vec4<f32> {
+struct TextFragment {
+ color : vec4<f32>, // premultiplied display color; alpha 0 = discard
+ hdr : vec4<f32>, // premultiplied scene radiance (bloom source)
+};
+
+fn shadeText(in : VSOut, fw : vec2<f32>) -> TextFragment {
+ var out : TextFragment;
+ out.color = vec4<f32>(0.0);
+ out.hdr = vec4<f32>(0.0);
// One-sided text culls against its plane's normal, as one-sided quads do.
if (model.params.z < 0.5 && dot(in.worldNormal, cam.cameraPos.xyz - in.worldPos) < 0.0) {
- return vec4<f32>(0.0);
+ return out;
}
let glyph = SlugGlyph(in.glyphScale, in.bandScale, in.bandMax, in.bandsTexCoords);
let shaded = shadeSlugGlyph(
@@ -138,19 +148,28 @@ fn shadeText(in : VSOut, fw : vec2<f32>) -> vec4<f32> {
in.fx.z,
);
if (shaded.a < 0.0001) {
- return vec4<f32>(0.0);
+ return out;
}
- let lit = applyLighting3D(in.worldPos, in.worldNormal, shaded.rgb / shaded.a, model.materialParams);
- return vec4<f32>(lit * shaded.a, shaded.a);
+ let lit = shadeSurfaceFull(in.worldPos, in.worldNormal, shaded.rgb / shaded.a, model.materialParams, model.pbr, model.emissive);
+ out.color = vec4<f32>(lit.display * shaded.a, shaded.a);
+ out.hdr = vec4<f32>(lit.hdr * shaded.a, shaded.a);
+ return out;
}
-@fragment fn fs_single(in : VSOut) -> @location(0) vec4<f32> {
+fn textFragment(in : VSOut) -> TextFragment {
let fw = max(abs(dpdx(in.uv)) + abs(dpdy(in.uv)), vec2<f32>(0.000001));
- let color = shadeText(in, fw);
- if (color.a < 0.0001) {
+ let m = matteCoverage(in.pos.xy);
+ var f = shadeText(in, fw);
+ f.color *= m;
+ f.hdr *= m;
+ if (f.color.a < 0.0001) {
discard;
}
- return color;
+ return f;
+}
+
+@fragment fn fs_single(in : VSOut) -> @location(0) vec4<f32> {
+ return textFragment(in).color;
}
struct MrtOutput {
@@ -158,15 +177,38 @@ struct MrtOutput {
@location(1) depth : vec4<f32>,
};
+struct BloomOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) hdr : vec4<f32>,
+};
+
+struct MrtBloomOutput {
+ @location(0) color : vec4<f32>,
+ @location(1) depth : vec4<f32>,
+ @location(2) hdr : vec4<f32>,
+};
+
@fragment fn fs_mrt(in : VSOut) -> MrtOutput {
- let fw = max(abs(dpdx(in.uv)) + abs(dpdy(in.uv)), vec2<f32>(0.000001));
- let color = shadeText(in, fw);
- if (color.a < 0.0001) {
- discard;
- }
var out : MrtOutput;
- out.color = color;
+ out.color = textFragment(in).color;
+ out.depth = vec4<f32>(in.linearDepth, 0.0, 0.0, 1.0);
+ return out;
+}
+
+@fragment fn fs_single_bloom(in : VSOut) -> BloomOutput {
+ let f = textFragment(in);
+ var out : BloomOutput;
+ out.color = f.color;
+ out.hdr = f.hdr;
+ return out;
+}
+
+@fragment fn fs_mrt_bloom(in : VSOut) -> MrtBloomOutput {
+ let f = textFragment(in);
+ var out : MrtBloomOutput;
+ out.color = f.color;
out.depth = vec4<f32>(in.linearDepth, 0.0, 0.0, 1.0);
+ out.hdr = f.hdr;
return out;
}
`;
diff --git a/packages/webgpu-renderers/src/shaders/solid.ts b/packages/webgpu-renderers/src/shaders/solid.ts
index e8d680ad..297dc196 100644
--- a/packages/webgpu-renderers/src/shaders/solid.ts
+++ b/packages/webgpu-renderers/src/shaders/solid.ts
@@ -1,4 +1,8 @@
+import { gradientWgsl } from "./gradient.js";
+
export const solidWgsl = `
+${gradientWgsl}
+
struct Uniforms {
transformCol0 : vec4<f32>,
transformCol1 : vec4<f32>,
@@ -6,6 +10,7 @@ struct Uniforms {
params : vec4<f32>, // x = opacity, y = surfaceWidth, z = surfaceHeight, w = cornerRadius
color : vec4<f32>,
rect : vec4<f32>, // x, y, width, height
+ gradient : GradientUniforms, // info.x = 0: solid \`color\`
};
@group(0) @binding(0) var<uniform> u : Uniforms;
@@ -38,11 +43,15 @@ fn sdRoundedRect(p: vec2<f32>, b: vec2<f32>, r: f32) -> f32 {
let d = sdRoundedRect(in.localPos - center, halfSize, radius);
let edgeAlpha = 1.0 - smoothstep(0.0, 1.0, d);
- let alpha = u.color.a * u.params.x * edgeAlpha;
+ var fill = u.color;
+ if (u.gradient.info.x > 0.5) {
+ fill = evalGradient(u.gradient, in.localPos);
+ }
+ let alpha = fill.a * u.params.x * edgeAlpha;
if (alpha <= 0.0) {
discard;
}
- return vec4<f32>(u.color.rgb * alpha, alpha);
+ return vec4<f32>(fill.rgb * alpha, alpha);
}
`;
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index 99cece90..7714a47d 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -92,13 +92,15 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani
| `Layer.svg(src, options)` | `"media"` | Vector SVG asset. Options: `fit: "contain"`. |
| `Layer.lottie(src, options)` | `"media"` | Lottie vector animation file. |
| `Layer.audio(src, options)` | `"media"` | Audio track layer. Options: `volume`, `startFrame`, `durationFrames`. |
-| `Layer.shape(type, options)` | `"shape"` | Parametric vector graphics (`"rect"`, `"circle"`, `"ellipse"`, `"polygon"`, `"star"`, `"arrow"`, `"path"`). Options: `d` (SVG path data with arcs), `strokeWidth`, `strokeColor`, `trimStart`, `trimEnd`. |
+| `Layer.shape(type, options)` | `"shape"` | Parametric vector graphics (`"rect"`, `"circle"`, `"ellipse"`, `"polygon"`, `"star"`, `"arrow"`, `"path"`). Options: `fill` (a color or CSS gradient), `d` (SVG path data with arcs), `strokeWidth`, `strokeColor`, `trimStart`, `trimEnd`. |
| `Layer.caption(src, options)` | `"media"` | SRT-driven synchronized subtitle engine. Bottom-anchored auto-wrapping cues. |
| `Layer.chart(options, props)` | `"box"` | Charts built from native nodes (d3 scales and geometry). `type`: `"line" \| "area" \| "bar" \| "scatter" \| "candlestick" \| "pie" \| "donut"`. Options: `width`, `height`, `series: [{ name, data, color, type, curve, pointRadius }]` or `data`, `categories`, `stacked`, `xAxis`/`yAxis` (`min`, `max`, `ticks`, `grid`, `format`: a d3-format string like `"$,.0f"`), `legend`, `valueLabels`, `fontFamily`, `fontSize`, `colors`, `animate: { start, duration, stagger, ease } \| false`. `props` positions the chart box. Children get ids `<id>-bar-<series>-<i>`, `<id>-line-<series>`, `<id>-slice-<i>`. Keep scatter charts to a few hundred points: each point is a node. |
| `Layer.camera(options)` | `"camera"` | 3D scene camera with FOV, lens rack focus, and orbital coordinates. |
-| `Layer.light(options)` | `"light"` | 3D lighting primitive (`ambient`, `directional`, `point`, `spot`). |
+| `Layer.light(options)` | `"light"` | 3D lighting primitive (`ambient`, `directional`, `point`, `spot`). Directional and spot take `castShadow: true` (one per scene), `shadowSoftness`, `shadowDarkness`. |
+| `Layer.environment(preset, options)` | `"light"` | The 3D scene's world: studio image-based light, exposure, tone mapping, fog, bloom. **Adding one switches the scene to physically based shading.** See *3D Look* below. |
+| `Layer.sequence(entries, options)` | `"box"` | Scenes back to back with `Transition`s between them; returns a box whose `totalFrames` is the length. See *Scenes & Transitions*. |
| `Layer.model(options)` | `"model3d"` | 3D mesh loader (`.gltf`, `.glb`, `.obj`, `.fbx`, `.stl`, `.ply`). Supports skinning and audio deformation compute shaders. |
-| `Layer3D.roundedBox / sphere / cylinder(options)` | `"model3d"` | Procedural solid meshes, centered on `x`/`y`/`z` (y is down, so "top" is -y). `roundedBox`: `width`, `height`, `depth`, `radius`, `segments`, `taper` (top face scale, <1 for a keycap). `sphere`: `radius`. `cylinder`: `radius` or `radiusTop`/`radiusBottom`, `height`, `openEnded`. `color` is baked in; `tint` multiplies it and animates as the `color` prop. Equal shapes share one GPU buffer, so hundreds of copies are cheap. |
+| `Layer3D.roundedBox / sphere / cylinder(options)` | `"model3d"` | Procedural solid meshes, centered on `x`/`y`/`z` (y is down, so "top" is -y). `roundedBox`: `width`, `height`, `depth`, `radius`, `segments`, `taper` (top face scale, <1 for a keycap). `sphere`: `radius`. `cylinder`: `radius` or `radiusTop`/`radiusBottom`, `height`, `openEnded`. `color` is baked in; `tint` multiplies it and animates as the `color` prop. Surface: `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `castShadow`. Equal shapes share one GPU buffer, so hundreds of copies are cheap. |
---
@@ -231,6 +233,142 @@ Anything else (a number, `undefined`) throws. Signals are sampled at the **compo
---
+## Color, Gradients & Text Metrics
+
+Every color prop accepts hex, `rgb()`, `hsl()`, `oklch()`, `oklab()` and CSS names. Color keyframes blend **perceptually (OKLab)**, so red β†’ green passes through yellow, not mud. `Color` builds colors without hand-rolled math; every helper returns a hex string:
+
+```typescript
+import { Color, Gradient } from "framefields";
+
+Color.oklch(0.72, 0.16, 250); // equal L = equal perceived brightness: balanced palettes
+Color.mix("#ff5a1f", "#1f6bff", 0.5); // perceptual mix
+const wave = Color.ramp(["#3b82f6", "#8b5cf6", "#ec4899"]); wave(0.3);
+Color.steps(["#0E0D0C", "#FF5A1F"], 5); // 5 swatches
+Color.lighten(EMBER, 0.1); Color.alpha("#000", 0.4); Color.readableOn(bg);
+Color.contrast(fg, bg); // WCAG ratio; β‰₯ 4.5 body text, β‰₯ 3 large
+```
+
+**Gradients are CSS strings**, usable in any `background` (boxes, flex, cards; rounded corners kept) and any shape `fill`. Linear, radial and conic, up to 8 stops, hard stops (`red 0 50%, blue 0`), `transparent` fades keep their hue:
+
+```typescript
+Layer.box({ background: "linear-gradient(135deg, #ff5a1f, #ffb800 60%, #7c3aed)" });
+Layer.shape("circle", { fill: Gradient.radial(["#fff", "transparent"]) });
+Layer.box({ background: Gradient.conic(["red", "yellow", "lime", "aqua", "blue", "magenta", "red"]) });
+```
+
+**Measure text with the real font** instead of guessing advances (place carets, underline a word, size a pill):
+
+```typescript
+const m = FontManager.measure("Hello, world", { fontFamily: "Inter", fontSize: 64 });
+m.width; m.height; m.lines; m.caretX(5); m.glyphs[0].x; // px, relative to the text box
+```
+
+---
+
+## Composing Animations
+
+`.animate()` **layers** onto what a node already has: call it several times, or pass several animations. Keyframe tracks on the same prop merge into one sorted track; a later key wins on a shared frame. Author behaviours separately and combine them:
+
+```typescript
+const rainIn = LayerAnimation.keys("y", [[0, -300], [d, -300], [d + 12, 0, "back.out(1.5)"]]);
+const press = LayerAnimation.keys("y", [[40, 0], [42, 6, "power2.in"], [47, 0, "power2.out"]]);
+const glow = LayerAnimation.keys("color", [[42, GLOW], [60, REST, "sine.inOut"]]);
+key.animate(rainIn).animate(press, glow); // or LayerAnimation.merge(rainIn, press, glow)
+```
+
+---
+
+## Track Mattes (masks, wipes, reveals)
+
+`node.withMatte(matteLayer, mode)` shows `node` only where the matte is opaque (`"alpha"`, default), transparent (`"alpha-inverted"`), bright (`"luma"`) or dark (`"luma-inverted"`). The matte is laid out like a **sibling** (same parent; absolute), keyed from the node's start, and covers the node's box when it has no `x`/`y`/`width`/`height`. It is never drawn by itself. On a 3D layer (or model) the matte works in screen space inside the 3D pass, so the layer still sorts by depth and its hidden part does not block what is behind it; a matte with its own `z` is projected like any 3D layer.
+
+```typescript
+// Text revealed by a sliding bar
+Layer.text("LAUNCH", { id: "title", position: "absolute", x: 560, y: 480, width: 800, height: 120, fontSize: 120 })
+ .withMatte(
+ Layer.box({ position: "absolute", x: 560, y: 480, width: 800, height: 120, background: "#fff" })
+ .animate(LayerAnimation.keys("x", [[0, -240], [24, 560, "expo.out"]])),
+ );
+
+// Video inside type: the text is the matte
+Layer.video("assets/surf.mp4", { position: "absolute", x: 0, y: 0, width: W, height: H, fit: "cover" })
+ .withMatte(Layer.text("OCEAN", { position: "absolute", x: 0, y: 340, width: W, fontSize: 380, align: "center", fill: "#fff" }));
+```
+
+---
+
+## Scenes & Transitions
+
+`Layer.sequence` places full-frame scenes end to end, overlapping each pair by its transition, and writes the scenes' `startFrame` / `durationFrames` for you (each scene keys from its own frame 0). Transitions are ordinary keyframes and mattes merged into the scenes:
+
+```typescript
+import { Layer, Transition } from "framefields";
+
+const film = Layer.sequence([
+ { scene: hero(), duration: beats(8) },
+ { scene: features(), duration: beats(12), transition: Transition.push("left", 18) },
+ { scene: proof(), duration: beats(8), transition: Transition.wipe("right", 16) },
+ { scene: outro(), duration: beats(6), transition: Transition.iris(20) },
+], { width: W, height: H });
+comp.add(film); // film.totalFrames β†’ composition length
+```
+
+| Transition | Look |
+|---|---|
+| `cut()` | hard cut |
+| `crossfade(frames)` | new scene fades in over the old |
+| `fadeThrough(frames)` | dip to the background color and back |
+| `push(dir, frames, ease)` | new scene pushes the old one out |
+| `slide(dir, frames)` | new scene slides over the old (which dims) |
+| `zoom(frames)` | old scene grows and fades, new one settles in |
+| `whip(dir, frames)` | fast motion-blurred push |
+| `wipe(dir, frames)` | hard edge sweeps across (matte) |
+| `iris(frames, { at })` | circle opens from a point (matte) |
+
+---
+
+## 3D Look: Environment, Materials, Shadows, Bloom
+
+A scene **without** `Layer.environment` keeps the classic Blinn-Phong look. **Add one** and the whole 3D pass becomes physically based: linear-light GGX shading where `roughness` and `metallic` behave like in Blender, a procedural studio environment that lights and reflects off everything, exposure, tone mapping, and optional fog and bloom.
+
+**Fastest path to a great product shot: a lighting rig.** It returns an environment plus placed key/rim(/fill) lights, the key casting a soft shadow:
+
+```typescript
+import { Lighting } from "framefields";
+
+comp.add(Lighting.studio({ target: [W / 2, H / 2, 0] })); // also: soft, sunset, dramatic
+comp.add(Lighting.dramatic({ target: [W / 2, H / 2, 0], bloom: true, fog: { color: BG, start: 2500, end: 6000 } }));
+// knobs: intensity, exposure (stops), rotation (turns the rig), shadowSoftness, shadows: false
+```
+
+Or by hand:
+
+```typescript
+comp.add([
+ Layer.environment("studio", { // presets: studio | soft | sunset | night | neutral
+ exposure: 0.3, // stops
+ toneMapping: "neutral", // default; "agx" filmic, "aces" punchy, "none"
+ rotation: 30, // turn the reflections
+ fog: { color: BG, start: 2000, end: 5000 }, // surfaces fade into the background color
+ bloom: true, // or { intensity: 0.3, threshold: 1.6, radius: 6 }
+ }),
+ Layer.directionalLight({ x: 400, y: -1200, z: -600, targetX: 960, targetY: 540, castShadow: true }),
+]);
+
+Layer3D.sphere({ radius: 90, color: "#e8e8e8", metallic: 1, roughness: 0.15 }); // chrome
+Layer3D.roundedBox({ ..., color: "#ff5a1f", roughness: 0.45 }); // satin plastic
+Layer3D.cylinder({ ..., color: "#111", emissive: "#4da3ff", emissiveIntensity: 3 }); // neon tube
+```
+
+- **Tone mapping `"neutral"` (default) keeps brand colors as authored**: orange stays orange. Use `"agx"` for a filmic roll-off.
+- **Bloom only takes light well past white** (emissive ≳ 3, hot speculars), so ordinary bright surfaces don't glow. Raise `emissiveIntensity` to make something glow.
+- **Shadows:** one `castShadow` directional or spot light per scene. Meshes cast by default (`castShadow: false` opts out); flat layers (planes, cards) cast only with `castShadow: true`. Everything receives.
+- **Fog** fades to `fog.color` exactly, so set it to the composition background for a seamless horizon.
+- **Models:** glTF materials bring base color, normal, metallic-roughness, occlusion and emissive maps.
+- `emissive` / `emissiveIntensity` / `exposure` animate like any prop (`.keys("emissiveIntensity", …)`).
+
+---
+
## Critical Authoring Rules & Gotchas
1. **Local Keyframe Coordinates:** Keyframes are strictly relative to the node's own timeline window (`0` = node appearance). Never offset keyframe frames by the parent's `startFrame`.
@@ -241,3 +379,4 @@ Anything else (a number, `undefined`) throws. Signals are sampled at the **compo
6. **Relative vs. Absolute:** Providing `x` or `y` without `position: "relative"` turns the node into an absolute-positioned layer removed from flex flow.
7. **Audio Layer Duration:** Every `Layer.audio()` must be assigned `durationFrames` to prevent audio truncation in headless rendering.
8. **Premultiplied Alpha:** Alpha shaders operate with premultiplied alpha. Never use `"src-alpha"` factor in custom blends.
+9. **Lint before rendering:** `comp.lint()` lists colors/gradients that would draw transparent, low-contrast text, off-canvas layers, shadow settings that do nothing and `roughness`/`metallic` without an environment. `renderVideo({ qa: true })` includes these findings.
diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.md
index 0ad249f3..5d9f9c0d 100644
--- a/plugins/framefields/skills/framefields-render/SKILL.md
+++ b/plugins/framefields/skills/framefields-render/SKILL.md
@@ -34,6 +34,9 @@ if (!parsed.success) {
console.error("Schema validation failed:", parsed.error.format());
process.exit(1);
}
+
+// 3. Lint: things that render without error but not as intended
+for (const d of comp.lint()) console.log(`${d.severity} ${d.code}: ${d.message}`);
```
Typecheck the project:
@@ -248,6 +251,16 @@ if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop
| `frozen_video` | info | Picture unchanged for β‰₯ `minFrozenSec` (2 s); fine for held title cards |
| `audio_silence` | info | β‰₯ `minSilenceSec` (2 s) below `silenceDbfs` (-60) |
| `audio_silent` | info | The whole track is silent |
+| `invalid_gradient` | error | A `background` / `fill` gradient string that does not parse: it draws nothing |
+| `unknown_color` | warning | A color string that is not a color: it draws transparent |
+| `low_contrast_text` | warning | Text below 3:1 contrast against the solid color behind it (suggests a readable one) |
+| `offscreen_layer` | warning | A static absolute layer entirely outside the canvas |
+| `shadow_unsupported_light` | warning | `castShadow` on a point or ambient light |
+| `pbr_without_environment` | info | `roughness` / `metallic` set but no `Layer.environment()` (no physical shading) |
+| `multiple_shadow_lights` | info | Several `castShadow` lights; only the first casts |
+| `static_track` | info | Every keyframe on a track holds the same value |
+
+The document checks also run on their own, before any render: `comp.lint()` returns the same `{ severity, code, message, layerId }` entries. Run it in Step 1, right after typechecking.
`passed` is false when any error or warning is present. `qa.audio.integratedLufs` is ITU-R BS.1770 loudness (streaming platforms normalize to about -14 LUFS); `qa.audio.peakDbfs` is the sample peak.