Skip to content

Code-first video & image, rendered natively on WebGPU.

feat(3d): procedural terrain, SDF, scatter with wind; character rigs and skinned crowds

okanasl4ff400b8e3aa
diff --git a/nodes/node-compositor/SKILL.md b/nodes/node-compositor/SKILL.md
index 88623fe7..65c0d521 100644
--- a/nodes/node-compositor/SKILL.md
+++ b/nodes/node-compositor/SKILL.md
@@ -217,6 +217,10 @@ Each track represents animatable property modifications:
- **Substrate multi-lobe materials:** physical-shading surfaces (any scene with an `environment` light) on `model3d` nodes, 3D cards and 3D text accept extra lobes on top of `roughness`/`metallic`: `clearcoat` 0..1 + `clearcoatRoughness` (default 0.08; IOR 1.5 GGX varnish that attenuates the base), `anisotropy` -1..1 + `anisotropyRotation` degrees (tangent-space anisotropic GGX: brushed metal, satin), `subsurface` 0..1 + `subsurfaceColor` (default `#ff6b4a`; wrap diffuse, terminator bleed and back-lit transmission), `sheen` 0..1 + `sheenColor` + `sheenRoughness` (default 0.5; Charlie cloth/velvet lobe), `iridescence` 0..1 or `{ intensity, thickness, ior }` + `iridescenceThickness` nm (100..1200, default 400) + `iridescenceIor` (default 1.3; RGB thin-film interference), and `triplanar: true | { scale, sharpness }` + `triplanarScale` (repeats per 256 world px) + `triplanarSharpness` (1..16, default 4) for world-space UV-less texture projection. All numeric lobe props are animatable.
- **HDRI environments:** an environment light's `hdri` takes an equirectangular panorama: a Radiance `.hdr` (RGBE) or OpenEXR `.exr` file path (resolved from the working directory) or the file's bytes (told apart by magic bytes); it then replaces the procedural preset as the image-based light for reflections and ambient, honoring `intensity`, `rotation` and `exposure`. On first use per device a compute pass prefilters it into a GGX radiance cubemap (faces up to 256², one mip per roughness step, split-sum importance sampling) for reflections, and the CPU projects it onto L2 spherical harmonics for diffuse irradiance; both are cached (up to 8 panoramas per device, least recently used destroyed). `.exr` reads single-part scanline or tiled files (level 0 of mip/ripmaps) with NONE, RLE, ZIPS, ZIP, PIZ or PXR24 compression and HALF/FLOAT/UINT `R`,`G`,`B` (or `Y`) channels; lossy B44/B44A/DWAA/DWAB/HTJ2K, deep and multi-part files fail with an error naming the cause.
- **Screen-space reflections & GI (`ssr`, `gi` in `environment`):** `ssr: true | { intensity (0..1, default 0.85), maxRoughness (0..1, 0.6), thickness (world px, 120), steps (16..64, 32) }` makes glossy physical surfaces reflect whatever the camera sees (walls, curved metal, emissives off the floor plane); `gi: true | { intensity (0..4, 1), radius (world px, 400), samples (4..16, 6) }` adds ambient occlusion and one diffuse bounce of visible light (color bleeding from neon and lit walls). Either one renders the 3D scene twice: a radiance pass, then Hi-Z reflection tracing and a horizon-scan GI gather (bilateral-denoised) on it, then the shading pass, which swaps each surface's environment reflection for the traced one (glossy ones read a blurred radiance pyramid matching their lobe, so output is noise-free and identical every frame) and occludes its ambient light before tone mapping and fog. Screen-space limits apply: anything off screen or hidden behind other geometry reflects the environment instead; the planar `floor` reflection is unaffected. Needs physical shading; non-physical (legacy, toon) surfaces ignore it.
+- **Procedural geometry (`primitive3d.type: "terrain" | "sdf"`):** terrain is a seeded fractal heightfield (`noise: { seed, scale px, octaves, persistence, lacunarity, ridgeMode: "fbm" | "ridged" | "terrace", terraceSteps }`, or `heights` 0..1 on the `(segments + 1)²` grid), `height` px of relief above the node's y (up is -y), optional `overhangWarp` px, `colorByElevation { lowColor, midColor?, highColor, cliffColor?, cliffThreshold, bands }` as vertex colors, closed into a block `base` px thick. `sdf` combines `primitives` (sphere, box, cylinder, capsule, torus, gyroid; `op` union / subtract / intersect, `smoothness` px, `center`, `rotation`, `color`) clipped to `bounds` and meshes them watertight with Surface Nets (`resolution` 16..160 cells). Both are strictly validated and rebuilt from the spec when `modelData` is lost.
+- **Scatter (`scatter3d` on an `instances3d` node):** `{ surface: <model node spec>, count, minDistance, seed, slopeRange deg, elevationRange 0..1, alignToNormal, scaleRange, randomYaw, exclude: [{ x, z, radius, falloff }], sink }`. The compositor loads the surface (primitive, in-memory or file), samples it by area once (cached) and seats each instance on its variant's base; instance i draws variant i % n, in the variant's own material color under the node tint.
+- **Wind (`wind` on any model node):** `{ amplitudeDeg (8), frequency Hz (0.4), direction [x, z], waveScale px (600) }` bends vertices downwind in the vertex shader by height above the mesh base (instances sway out of phase along a traveling wave); deterministic per frame, shadows included.
+- **Texture layers (`textureLayer: { node }` on a model):** hoisted like a matte to a sibling with `textureOf`, drawn alone each frame (video plays), then bound as the base-color texture of every mesh the model queues, read through its rect in that picture (`uvRect`); without its own geometry it fills the frame (the whole UV square).
- **Art-directed 3D lighting:** `material: "toon"` on meshes, flat 3D cards and native 3D text uses the shared direct lights, colors, point/spot attenuation, cast shadows, environment fog and tone mapping. Optional `toon: { shadowColor, lightColor, threshold, softness, diffuse }` controls the shadow/lit albedo multipliers, terminator cosine/width and Lambert modelling (defaults: model shade color or `#80808c`, white, 0.075, 0.25, 0). Lit/physical materials retain their own shading.
- **Geometry outlines:** an environment light's optional `outlines: true | { color, width, referenceHeight, creaseAngle, depthThreshold, opacity }` runs a depth/normal ink pass before camera depth blur and bloom (defaults: `#16131c`, 1.5 px at height 1080, 45°, 0.01, 1). It reuses the linear-depth attachment's spare channels for octahedral surface normals and outline participation. Inverse-depth curvature avoids false contours on perspective planes; normal discontinuities find creases without inking texture/shadow/triangulation seams. `outline: false` on a layer/model opts out while retaining depth occlusion; `creaseAngle: 180` disables creases. Ink fades into the environment fog. 3D `anchorX/Y` pivots remain fixed under scale/rotation and leave identity placement unchanged.
With MSAA enabled, retain original scene color and geometry samples and shade ink per sample before resolving color. A single-sample ink pass after the scene resolve loses subpixel coverage between outlined surfaces. Lens blur uses a separate nearest-sample geometry resolve. The scene's geometry is still drawn once.
@@ -224,6 +228,8 @@ Each track represents animatable property modifications:
- **Mesh wireframe:** `wireframe: true` on a `model3d` draws triangle edges as one-pixel lines, including triangulation and back edges; it also applies to skinned and audio-deformed meshes and their shadows.
- **glTF scenes:** a glTF/GLB `model3d` keeps its node hierarchy, every skin and its animation clips (translation, rotation, scale and morph weights; linear, step and cubic-spline). Draco and meshopt compressed geometry and all seven core primitive modes are supported; per-map UV sets and texture transforms are independent. Basis KTX2 (ETC1S/UASTC) textures transcode offline to RGBA8 with their mip levels. Unsupported required extensions fail clearly. Morph targets run on the GPU before skinning and audio deformation. Skinned meshes are posed per frame from the clip (`animationName`, `animationTime`/`animationProgress`, `loop`); base-colour textures (with mipmaps), alpha `MASK`/`BLEND` and VRM MToon materials (`material: "toon"`, picked automatically when `material` is unset) render as authored. `center`/`normalizeSize` apply above the scene graph, so skinning stays intact.
- **Procedural Shards (`Layer3D.shard`):** Procedural polyhedron primitive generated by Sutherland-Hodgman convex half-space clipping against Fibonacci lattice normals with PCG jitter. Faceted flat normals, automatic planar UVs, Rapier 3D convex hull collision.
+- **Character rigs (`animationLayers`, `ik`, `lookAt`, `springBones` on a glTF `model3d`):** clips blend in order from the rest pose, each over the pose beneath by `weight` (number, `[[frame, w]]` keys, or `(frame) => w` in process), slerped per joint and restricted to `mask` joints and their descendants; `speed`, `timeOffset`, `loop` per layer. Then `lookAt` (joint `axis`, default glTF +z, clamped to `maxAngleDeg`) and analytic two-bone `ik` (`chain: [root, mid, effector]`, `poleTarget` sets the bend side); targets are scene coordinates (point, keys or function) mapped into model space through the model's transform. `springBones` (each root's chain down its first child; `stiffness`, `damping`, scene `gravity`) are Verlet-simulated from frame 0 and cached per layer, so frames come out the same in any order. A node without these keeps the single-clip path.
+- **Skinned crowds (`crowd` on an instances node):** `{ clips, fps (30), seed, speedRange, phase: "random" | "sync" }`; each mesh of each variant model bakes its joint matrices (or its node matrix, for rigid node meshes) for every clip once (`count + 1` samples per loop) into a GPU palette; instances take a second record (clip range, phase, playback rate) and skin in the vertex shader, interpolating two baked samples. Per instance `clip`, `animOffset`, `animSpeed` override the seeded defaults. One draw call per mesh, shadows included. Instance (and scatter) variants apply their own node `rotateX/Y/Z` and `scale` under each instance's transform.
- **Automatic mesh instancing:** consecutive opaque draws of the same shared mesh with matching material state use one GPU draw with per-object model and normal matrices. Transparent, skinned, morphed, audio-deformed and glass meshes retain separate draws; shadows retain their individual draw submissions.
- **Instanced 3D Meshes (`instances3d`):** `kind: "model3d"` with `instances3d` renders hundreds or thousands of mesh instances in 1 GPU draw call (`drawIndexed(indexCount, instanceCount)`). Supports single or multi-mesh variants with round-robin or variant indexing, dynamic per-instance transforms, and per-instance emissive / color modulation. In `framefields`, created with `Layer3D.instances({ count, mesh, instanceAt, bodies, emissiveBySpeed })`.
- **GPU 3D Swarms (`swarm3d`):** `kind: "model3d"` with `swarm3d` runs continuous 3D particle simulation in a WebGPU compute shader with 3D curl noise, 3D vortex fields (`swirl`, `pull`, `axial`), ground plane collisions with Coulomb sliding friction and rotational roll torque, and kinetic heat accumulation/cooling, packing directly into GPU instance buffers.
diff --git a/nodes/node-compositor/src/renderers/character.ts b/nodes/node-compositor/src/renderers/character.ts
new file mode 100644
index 00000000..587d865d
--- /dev/null
+++ b/nodes/node-compositor/src/renderers/character.ts
@@ -0,0 +1,295 @@
+/**
+ * Character animation for the compositor: poses a model from its clip
+ * layers, look-at and IK constraints (targets in scene coordinates) and
+ * spring bones (replayed from frame 0 and cached, so frames render the
+ * same in any order); and the per-instance records of skinned crowds.
+ */
+import type {
+ CrowdConfig,
+ LookAtConstraint,
+ Model3DNode,
+ NumberTrack,
+ TwoBoneIkConstraint,
+ Vec3Track,
+} from "@framefields/compositions/program";
+import {
+ type AnimationClip3D,
+ applyConstraints,
+ bakeCrowdPalette,
+ blendClipLayers,
+ type CrowdBake,
+ createRandom,
+ type Mat4,
+ Matrix4Math,
+ type Model3DData,
+ poseNodes,
+ type RigClipLayer,
+ type RigConstraints,
+ type RigPose,
+ resolveClips,
+ SpringBones,
+ type Vec3,
+} from "@framefields/webgpu-renderers";
+
+/** A number track's value at `frame`. */
+export function numberAt(
+ track: NumberTrack | undefined,
+ frame: number,
+ fallback: number,
+): number {
+ if (track === undefined) return fallback;
+ if (typeof track === "number") return track;
+ if (typeof track === "function") return track(frame);
+ return keyed(track, frame, 1)[0] ?? fallback;
+}
+
+/** A point track's value at `frame`. */
+export function vec3At(track: Vec3Track, frame: number): Vec3 {
+ if (typeof track === "function") return track(frame);
+ if (track.length === 3 && typeof track[0] === "number")
+ return track as unknown as Vec3;
+ return keyed(track as number[][], frame, 3) as Vec3;
+}
+
+/** Linear interpolation over `[[frame, …values]]` keys, held past the ends. */
+function keyed(keys: number[][], frame: number, size: number): number[] {
+ const sorted = keys;
+ const first = sorted[0]!;
+ const last = sorted[sorted.length - 1]!;
+ if (frame <= first[0]!) return first.slice(1, 1 + size);
+ if (frame >= last[0]!) return last.slice(1, 1 + size);
+ let k = 0;
+ while (k + 1 < sorted.length && sorted[k + 1]![0]! <= frame) k++;
+ const a = sorted[k]!;
+ const b = sorted[k + 1]!;
+ const u = (frame - a[0]!) / Math.max(1e-9, b[0]! - a[0]!);
+ return a.slice(1, 1 + size).map((v, i) => v + (b[i + 1]! - v) * u);
+}
+
+const transformPoint = (m: Mat4, p: Vec3): Vec3 => [
+ m[0] * p[0] + m[4] * p[1] + m[8] * p[2] + m[12],
+ m[1] * p[0] + m[5] * p[1] + m[9] * p[2] + m[13],
+ m[2] * p[0] + m[6] * p[1] + m[10] * p[2] + m[14],
+];
+const transformDir = (m: Mat4, v: Vec3): Vec3 => [
+ m[0] * v[0] + m[4] * v[1] + m[8] * v[2],
+ m[1] * v[0] + m[5] * v[1] + m[9] * v[2],
+ m[2] * v[0] + m[6] * v[1] + m[10] * v[2],
+];
+
+type CharacterNode = Pick<
+ Model3DNode,
+ "animationLayers" | "ik" | "lookAt" | "springBones" | "loop"
+>;
+
+/** Whether a node needs the rig evaluator (anything beyond one plain clip). */
+export function hasRig(n: CharacterNode): boolean {
+ return Boolean(
+ n.animationLayers?.length ||
+ n.ik?.length ||
+ n.lookAt?.length ||
+ n.springBones?.length,
+ );
+}
+
+export interface PoseInput {
+ model: Model3DData;
+ node: CharacterNode;
+ /** The layer's own frame. */
+ frame: number;
+ fps: number;
+ /** The single clip and its time when there are no `animationLayers`. */
+ clip: AnimationClip3D | undefined;
+ clipSec: number;
+ /** Scene ← model (the model's world transform, root transform included). */
+ sceneRoot: Mat4;
+ /** Identifies the node across frames (spring state). */
+ key: string;
+}
+
+function clipLayers(p: PoseInput, frame: number): RigClipLayer[] {
+ const layers = p.node.animationLayers;
+ if (!layers?.length) {
+ return p.clip ? [{ clip: p.clip, time: p.clipSec, weight: 1 }] : [];
+ }
+ const out: RigClipLayer[] = [];
+ for (const l of layers) {
+ const [clip] = resolveClips(p.model, [l.clip]);
+ if (!clip) continue;
+ let time = (l.timeOffset ?? 0) + (frame / p.fps) * (l.speed ?? 1);
+ if (clip.duration > 0) {
+ time =
+ (l.loop ?? true)
+ ? ((time % clip.duration) + clip.duration) % clip.duration
+ : Math.min(Math.max(time, 0), clip.duration);
+ }
+ out.push({
+ clip,
+ time,
+ weight: numberAt(l.weight, frame, 1),
+ mask: l.mask,
+ });
+ }
+ return out;
+}
+
+function constraintsAt(
+ p: PoseInput,
+ frame: number,
+ toModel: Mat4,
+): RigConstraints {
+ const point = (t: Vec3Track) => transformPoint(toModel, vec3At(t, frame));
+ return {
+ ik: (p.node.ik ?? []).map((c: TwoBoneIkConstraint) => ({
+ chain: c.chain,
+ target: point(c.target),
+ pole: c.poleTarget ? point(c.poleTarget) : undefined,
+ weight: numberAt(c.weight, frame, 1),
+ })),
+ lookAt: (p.node.lookAt ?? []).map((c: LookAtConstraint) => ({
+ joint: c.joint,
+ target: point(c.target),
+ weight: numberAt(c.weight, frame, 1),
+ maxAngleDeg: c.maxAngleDeg ?? 70,
+ axis: c.axis ?? [0, 0, 1],
+ })),
+ };
+}
+
+/** The pose before springs: blended clips, then constraints. */
+function basePose(p: PoseInput, frame: number, toModel: Mat4): RigPose {
+ const rig = blendClipLayers(p.model, clipLayers(p, frame));
+ applyConstraints(rig, constraintsAt(p, frame, toModel));
+ return rig;
+}
+
+interface SpringState {
+ model: Model3DData;
+ config: string;
+ springs: SpringBones;
+ /** Last frame stepped. */
+ frame: number;
+}
+
+const springStates = new Map<string, SpringState>();
+
+function springsAt(
+ p: PoseInput,
+ toModel: Mat4,
+ rig: RigPose,
+): SpringBones | undefined {
+ const chains = p.node.springBones;
+ if (!chains?.length) return undefined;
+ const config = JSON.stringify([chains, p.node.animationLayers, p.fps]);
+ let s = springStates.get(p.key);
+ const frame = Math.max(0, Math.floor(p.frame));
+ if (!s || s.model !== p.model || s.config !== config || s.frame > frame) {
+ const gravityModel = (g: Vec3 | undefined): Vec3 =>
+ transformDir(toModel, g ?? [0, 0, 0]);
+ s = {
+ model: p.model,
+ config,
+ springs: new SpringBones(
+ rig,
+ chains.map((c) => ({ ...c, gravity: gravityModel(c.gravity) })),
+ ),
+ frame: -1,
+ };
+ if (springStates.size > 256) springStates.clear();
+ springStates.set(p.key, s);
+ }
+ for (let f = s.frame + 1; f <= frame; f++) {
+ s.springs.step(basePose(p, f, toModel), 1 / p.fps);
+ s.frame = f;
+ }
+ return s.springs;
+}
+
+/** Model-space node matrices for this frame. */
+export function poseCharacter(p: PoseInput): Mat4[] {
+ if (!hasRig(p.node)) return poseNodes(p.model, p.clip, p.clipSec);
+ const toModel = Matrix4Math.invert(p.sceneRoot) ?? Matrix4Math.identity();
+ const rig = basePose(p, p.frame, toModel);
+ springsAt(p, toModel, rig)?.apply(rig);
+ return rig.world();
+}
+
+// --- Crowds -------------------------------------------------------------------
+
+const bakes = new WeakMap<Model3DData, Map<string, CrowdBake>>();
+const palettes = new WeakMap<Float32Array, GPUBuffer>();
+
+/** A baked palette on the GPU, uploaded once. */
+export function paletteBuffer(
+ device: GPUDevice,
+ palette: Float32Array,
+): GPUBuffer {
+ let buffer = palettes.get(palette);
+ if (!buffer) {
+ buffer = device.createBuffer({
+ label: "CrowdPalette",
+ size: Math.max(64, palette.byteLength),
+ usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
+ });
+ device.queue.writeBuffer(buffer, 0, palette);
+ palettes.set(palette, buffer);
+ }
+ return buffer;
+}
+
+/** The model's crowd palette for these clips, baked once. */
+export function crowdBake(
+ model: Model3DData,
+ crowd: CrowdConfig,
+): CrowdBake | undefined {
+ const key = JSON.stringify([crowd.clips, crowd.fps ?? 30]);
+ let byKey = bakes.get(model);
+ if (!byKey) {
+ byKey = new Map();
+ bakes.set(model, byKey);
+ }
+ let bake = byKey.get(key);
+ if (!bake) {
+ const clips = resolveClips(model, crowd.clips);
+ if (!clips.length) return undefined;
+ bake = bakeCrowdPalette(model, clips, crowd.fps ?? 30);
+ byKey.set(key, bake);
+ }
+ return bake;
+}
+
+export interface CrowdInstanceAnim {
+ clip?: number;
+ animOffset?: number;
+ animSpeed?: number;
+}
+
+/**
+ * Instance `i`'s second record: (first baked frame, frames per loop, phase
+ * in frames, frames per second of playback). Unset fields come from the seed: clip i % n, a random
+ * phase, a speed within `speedRange`.
+ */
+export function crowdRecord(
+ bake: CrowdBake,
+ crowd: CrowdConfig,
+ i: number,
+ anim: CrowdInstanceAnim,
+): [number, number, number, number] {
+ const rand = createRandom((crowd.seed ?? 1) * 7919 + i * 104729);
+ const pick = rand();
+ const speedDraw = rand();
+ const clipIndex = (anim.clip ?? i) % bake.clips.length;
+ const range = bake.clips[Math.max(0, clipIndex)]!;
+ const [s0, s1] = crowd.speedRange ?? [1, 1];
+ const offset =
+ anim.animOffset ??
+ ((crowd.phase ?? "random") === "random" ? pick * range.duration : 0);
+ // Baked samples per second of this clip's time.
+ const rate = range.duration > 0 ? range.count / range.duration : bake.fps;
+ return [
+ range.base,
+ range.count,
+ offset * rate,
+ (anim.animSpeed ?? s0 + (s1 - s0) * speedDraw) * rate,
+ ];
+}
diff --git a/nodes/node-compositor/src/renderers/procedural3d.ts b/nodes/node-compositor/src/renderers/procedural3d.ts
new file mode 100644
index 00000000..8d096ea5
--- /dev/null
+++ b/nodes/node-compositor/src/renderers/procedural3d.ts
@@ -0,0 +1,172 @@
+/**
+ * Layer3D.scatter and wind for the compositor: where scattered instances
+ * stand (computed once per scatter, then reused every frame) and the wind
+ * a mesh sways in at a given frame.
+ */
+import type {
+ Scatter3DSpec,
+ WindConfig,
+} from "@framefields/compositions/program";
+import {
+ getPrimitiveGeometry,
+ type Mat4,
+ Matrix4Math,
+ type MeshWind,
+ type Model3DData,
+ type PrimitiveGeometryOptions,
+ type Quat,
+ rotateByQuat,
+ scatterOnSurface,
+ seatOnSurface,
+} from "@framefields/webgpu-renderers";
+
+/** The static node fields a scatter surface is placed by. */
+export interface SurfaceNodeSpec {
+ x?: number;
+ y?: number;
+ z?: number;
+ rotateX?: number;
+ rotateY?: number;
+ rotateZ?: number;
+ scale?: number;
+ scaleX?: number;
+ scaleY?: number;
+ scaleZ?: number;
+ modelData?: unknown;
+ primitive3d?: unknown;
+ src?: string;
+ modelFormat?: string;
+ center?: boolean;
+ normalizeSize?: number;
+}
+
+/** T · Rz · Ry · Rx · S: the transform the compositor draws a model with. */
+export function nodeMatrix(n: SurfaceNodeSpec): Mat4 {
+ const rad = Math.PI / 180;
+ const s = n.scale ?? 1;
+ let m = Matrix4Math.translate(n.x ?? 0, n.y ?? 0, n.z ?? 0);
+ if (n.rotateZ)
+ m = Matrix4Math.multiply(m, Matrix4Math.rotateZ(n.rotateZ * rad));
+ if (n.rotateY)
+ m = Matrix4Math.multiply(m, Matrix4Math.rotateY(n.rotateY * rad));
+ if (n.rotateX)
+ m = Matrix4Math.multiply(m, Matrix4Math.rotateX(n.rotateX * rad));
+ return Matrix4Math.multiply(
+ m,
+ Matrix4Math.scale(
+ s * (n.scaleX ?? 1),
+ s * (n.scaleY ?? 1),
+ s * (n.scaleZ ?? 1),
+ ),
+ );
+}
+
+const isModel = (v: unknown): v is Model3DData =>
+ typeof v === "object" && v !== null && "meshes" in v;
+
+/** The surface's mesh: in memory, rebuilt from its primitive spec, or loaded. */
+export async function surfaceModel(
+ n: SurfaceNodeSpec,
+ load: (src: string, n: SurfaceNodeSpec) => Promise<Model3DData | undefined>,
+): Promise<Model3DData | undefined> {
+ if (isModel(n.modelData)) return n.modelData;
+ if (n.primitive3d)
+ return getPrimitiveGeometry(n.primitive3d as PrimitiveGeometryOptions);
+ if (n.src) return load(n.src, n);
+ return undefined;
+}
+
+export interface ScatterTransform {
+ x: number;
+ y: number;
+ z: number;
+ rotation: [number, number, number, number];
+ scale: number;
+}
+
+/**
+ * A mesh variant's own node rotation and uniform scale, applied under each
+ * instance's (a y-up glTF passed as `Layer.glb(src, { rotateX: 180 })`
+ * stands up in a crowd or scatter as it would on its own).
+ */
+export interface VariantPre {
+ q: Quat;
+ s: number;
+}
+
+export const NO_PRE: VariantPre = { q: [0, 0, 0, 1], s: 1 };
+
+export function composeQuat(a: Quat, b: Quat): Quat {
+ return [
+ a[3] * b[0] + a[0] * b[3] + a[1] * b[2] - a[2] * b[1],
+ a[3] * b[1] - a[0] * b[2] + a[1] * b[3] + a[2] * b[0],
+ a[3] * b[2] + a[0] * b[1] - a[1] * b[0] + a[2] * b[3],
+ a[3] * b[3] - a[0] * b[0] - a[1] * b[1] - a[2] * b[2],
+ ];
+}
+
+/** The variant's lowest point after its own rotation and scale: the largest y (up is -y). */
+export function variantBottom(model: Model3DData, pre: VariantPre): number {
+ const b = model.bounds;
+ if (!b) return 0;
+ let bottom = Number.NEGATIVE_INFINITY;
+ for (let k = 0; k < 8; k++) {
+ const corner: [number, number, number] = [
+ k & 1 ? b.max[0] : b.min[0],
+ k & 2 ? b.max[1] : b.min[1],
+ k & 4 ? b.max[2] : b.min[2],
+ ];
+ bottom = Math.max(bottom, rotateByQuat(pre.q, corner)[1] * pre.s);
+ }
+ return bottom;
+}
+
+const cache = new WeakMap<object, { key: string; out: ScatterTransform[] }>();
+
+/**
+ * Instance transforms for a scatter: points on the surface, each seated on
+ * its variant's base (instance i draws variant i % bottoms.length; bottoms
+ * already include the variant's own rotation and scale).
+ */
+export function scatterTransforms(
+ scatter: Scatter3DSpec,
+ surface: Model3DData,
+ bottoms: number[],
+): ScatterTransform[] {
+ const surfaceNode = scatter.surface as SurfaceNodeSpec;
+ const key = JSON.stringify([bottoms, surface.bounds?.size]);
+ const hit = cache.get(scatter);
+ if (hit && hit.key === key) return hit.out;
+ const { points } = scatterOnSurface(
+ { model: surface, matrix: nodeMatrix(surfaceNode) },
+ scatter,
+ );
+ const n = Math.max(1, bottoms.length);
+ const out = points.map((p, i) => {
+ const [x, y, z] = seatOnSurface(p, bottoms[i % n] ?? 0, scatter.sink ?? 0);
+ return { x, y, z, rotation: p.rotation, scale: p.scale };
+ });
+ cache.set(scatter, { key, out });
+ return out;
+}
+
+/** The wind a mesh of `model` sways in at `seconds`, or undefined when calm. */
+export function meshWind(
+ wind: WindConfig | undefined,
+ model: Model3DData,
+ seconds: number,
+): MeshWind | undefined {
+ if (!wind) return undefined;
+ const b = model.bounds;
+ const height = b ? b.size[1] : 0;
+ if (!b || height <= 0) return undefined;
+ return {
+ amplitudeDeg: wind.amplitudeDeg ?? 8,
+ frequency: wind.frequency ?? 0.4,
+ direction: wind.direction ?? [1, 0],
+ waveScale: wind.waveScale ?? 600,
+ time: seconds,
+ baseY: b.max[1],
+ height,
+ };
+}
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 9281700b..8fac3a33 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -4,11 +4,13 @@ import { resolveMediaSourceUrl } from "@framefields/client-utils";
import { decodeAudioSource } from "@framefields/compositions";
import {
type CompositionNodeBase,
+ type CrowdConfig,
collectNodeOps,
computeLayout,
initLayout,
type LayoutNode,
type Rect,
+ type Scatter3DSpec,
} from "@framefields/compositions/program";
import type { MeshAudioDeformConfig, TextSpan } from "@framefields/core";
import {
@@ -41,6 +43,7 @@ import {
Camera3D,
type Camera3DPose,
type Color,
+ type CrowdBake,
type DrawMesh3DOpts,
type DrawQuad3DOpts,
type Environment3DOptions,
@@ -55,12 +58,12 @@ import {
type Mat4,
Matrix4Math,
type Mesh3DData,
+ type MeshCrowd,
type Model3DData,
measureMaxCaptionHeight,
type PrimitiveGeometryOptions,
parseColor,
parseDashArray,
- poseNodes,
type RenderContextValue,
Renderer3D,
resolveTextSpans,
@@ -81,8 +84,25 @@ import {
} from "@framefields/webgpu-renderers";
import { compileTimeline, seekTimeline } from "../shared/compiler.js";
import type { CompositorOperation } from "../shared/config.js";
+import {
+ type CrowdInstanceAnim,
+ crowdBake,
+ crowdRecord,
+ paletteBuffer,
+ poseCharacter,
+} from "./character.js";
import { shouldPaintContainerBackground } from "./container-bg.js";
import { compatibleMeshDraws } from "./mesh-instancing.js";
+import {
+ composeQuat,
+ meshWind,
+ NO_PRE,
+ type SurfaceNodeSpec,
+ scatterTransforms,
+ surfaceModel,
+ type VariantPre,
+ variantBottom,
+} from "./procedural3d.js";
import {
buildLayerMatrix,
layerRasterScale,
@@ -2143,15 +2163,17 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
// Matte and relief layers are not drawn on their own: their owner draws them.
const mattesByOwner = new Map<string, VirtualMediaData>();
const reliefsByOwner = new Map<string, VirtualMediaData>();
+ const texturesByOwner = new Map<string, VirtualMediaData>();
for (const vm of vms) {
const op = vm.operation as any;
if (op?.matteOf) mattesByOwner.set(op.matteOf, vm);
if (op?.reliefOf) reliefsByOwner.set(op.reliefOf, vm);
+ if (op?.textureOf) texturesByOwner.set(op.textureOf, vm);
}
for (const child of sorted) {
const lop = child.operation as any;
if (lop.kind === "camera" || lop.kind === "light") continue;
- if (lop.matteOf || lop.reliefOf) continue;
+ if (lop.matteOf || lop.reliefOf || lop.textureOf) continue;
const targetId = lop.id || lop.inputHandleId;
const target = targetId ? targetsById[targetId] : undefined;
const isChild3D =
@@ -2246,6 +2268,34 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
amount: lop.reliefAmount ?? 0,
});
}
+ // A model's texture layer is drawn flat first; the meshes the
+ // model queues then read it through its place in that picture.
+ const textureVM = lop.textureLayerId
+ ? texturesByOwner.get(lop.id)
+ : undefined;
+ let wrapTexture: (() => void) | undefined;
+ if (textureVM) {
+ const texture = await isolate(textureVM, nodeIndex);
+ activeExcludes = [...activeExcludes, texture];
+ nodeIndex += 1;
+ const r = rects[lop.textureLayerId];
+ const uvRect: [number, number, number, number] = r
+ ? [
+ r.x / targetW,
+ r.y / targetH,
+ r.width / targetW,
+ r.height / targetH,
+ ]
+ : [0, 0, 1, 1];
+ const from = pending3DItems.length;
+ wrapTexture = () => {
+ for (const item of pending3DItems.slice(from)) {
+ if (item.kind !== "mesh") continue;
+ item.texture = texture;
+ item.opts.uvRect = uvRect;
+ }
+ };
+ }
const matteVM = lop.matteId ? mattesByOwner.get(lop.id) : undefined;
if (matteVM) {
const mode: string = (matteVM.operation as any).matteMode ?? "alpha";
@@ -2316,6 +2366,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
for (const item of pending3DItems.slice(before)) {
item.opts.matte = coverage;
}
+ wrapTexture?.();
if (res) {
if (res.finalTex !== activeTex) {
activeExcludes = [...activeExcludes, res.finalTex];
@@ -2365,6 +2416,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
parent3DMatrix,
isRootScene,
);
+ wrapTexture?.();
if (res) {
if (res.finalTex !== activeTex) {
activeExcludes = [...activeExcludes, res.finalTex];
@@ -2582,6 +2634,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
: [];
const variants: Model3DData[] = [];
+ const variantPres: VariantPre[] = [];
for (const m of rawMeshes) {
let vModel: Model3DData | undefined;
if (typeof m === "object" && m !== null) {
@@ -2604,10 +2657,44 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
}
if (vModel && vModel.meshes && vModel.meshes.length > 0) {
variants.push(vModel);
+ const isNode = !("meshes" in m);
+ variantPres.push(
+ isNode
+ ? {
+ q: quatFromEulerAngles(
+ m.rotateX ?? 0,
+ m.rotateY ?? 0,
+ m.rotateZ ?? 0,
+ ),
+ s: m.scale ?? 1,
+ }
+ : NO_PRE,
+ );
}
}
if (variants.length === 0 && model) {
variants.push(model);
+ variantPres.push(NO_PRE);
+ }
+ if (lop.scatter3d && variants.length > 0) {
+ // Layer3D.scatter: transforms from the surface, seated per variant.
+ const scatter = lop.scatter3d as Scatter3DSpec;
+ const surface = await surfaceModel(
+ scatter.surface as SurfaceNodeSpec,
+ (src, n) =>
+ r3d.loadModelAsync(src, (n.modelFormat ?? "auto") as never, {
+ center: n.center,
+ normalizeSize: n.normalizeSize,
+ }),
+ );
+ instOpt.transforms = surface
+ ? scatterTransforms(
+ scatter,
+ surface,
+ variants.map((v, k) => variantBottom(v, variantPres[k]!)),
+ )
+ : [];
+ instOpt.count = instOpt.transforms.length;
}
if (variants.length > 0) {
const count =
@@ -2626,13 +2713,33 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
for (let i = 0; i < count; i++) {
bucketCounts[i % numVariants]++;
}
+ // Skinned crowds: each instance takes a second record (clip, phase, speed).
+ const crowdCfg = lop.crowd as CrowdConfig | undefined;
+ const crowdBakes = variants.map((v) =>
+ crowdCfg ? crowdBake(v, crowdCfg) : undefined,
+ );
+ const strides = crowdBakes.map((b) => (b ? 32 : 16));
+ const crowdOpts = (
+ bake: CrowdBake | undefined,
+ mesh: Mesh3DData,
+ ): MeshCrowd | undefined => {
+ const baked = bake?.meshes.get(mesh);
+ if (!bake || !baked) return undefined;
+ return {
+ palette: paletteBuffer(ctx.device, baked.palette),
+ jointCount: baked.jointCount,
+ time: (shiftedFrame ?? 0) / (fps || 30),
+ };
+ };
const bucketItems: Float32Array[] = [];
const bucketBounds: Array<{
min: [number, number, number];
max: [number, number, number];
}> = [];
for (let k = 0; k < numVariants; k++) {
- bucketItems.push(new Float32Array(bucketCounts[k] * 16));
+ bucketItems.push(
+ new Float32Array(bucketCounts[k] * strides[k]),
+ );
bucketBounds.push({
min: [Infinity, Infinity, Infinity],
max: [-Infinity, -Infinity, -Infinity],
@@ -2655,7 +2762,8 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const cursor = bucketCursors[k];
const bData = bucketItems[k];
const bBox = bucketBounds[k];
- const off = cursor * 16;
+ const off = cursor * strides[k];
+ let ianim: CrowdInstanceAnim = {};
let ix = 0;
let iy = 0;
@@ -2745,6 +2853,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
} else if (typeof instOpt.instanceAt === "function") {
const t = instOpt.instanceAt(i, shiftedFrame ?? 0, fps || 30);
if (!t) continue;
+ ianim = t;
ix = t.x ?? 0;
iy = t.y ?? 0;
iz = t.z ?? 0;
@@ -2806,6 +2915,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
instOpt.transforms[i]
) {
const t = instOpt.transforms[i];
+ ianim = t;
ix = t.x ?? 0;
iy = t.y ?? 0;
iz = t.z ?? 0;
@@ -2867,6 +2977,15 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
}
bucketCursors[k]++;
+ // The variant's own rotation and scale, under the instance's.
+ const pre = variantPres[k]!;
+ if (pre !== NO_PRE) {
+ [iqx, iqy, iqz, iqw] = composeQuat(
+ [iqx, iqy, iqz, iqw],
+ pre.q,
+ );
+ iscale *= pre.s;
+ }
bData[off + 0] = ix;
bData[off + 1] = iy;
bData[off + 2] = iz;
@@ -2883,8 +3002,16 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
bData[off + 13] = ieg;
bData[off + 14] = ieb;
bData[off + 15] = iew;
+ const bake = crowdBakes[k];
+ if (bake && crowdCfg) {
+ bData.set(crowdRecord(bake, crowdCfg, i, ianim), off + 16);
+ }
- const rad = 50 * iscale;
+ const vb = variants[k].bounds;
+ const rad =
+ (vb
+ ? vb.boundingSphereRadius + Math.hypot(...vb.center)
+ : 50) * iscale;
bBox.min[0] = Math.min(bBox.min[0], ix - rad);
bBox.min[1] = Math.min(bBox.min[1], iy - rad);
bBox.min[2] = Math.min(bBox.min[2], iz - rad);
@@ -2900,7 +3027,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const activeSlice =
bCount === bucketCounts[k]
? bucketItems[k]
- : bucketItems[k].subarray(0, bCount * 16);
+ : bucketItems[k].subarray(0, bCount * strides[k]);
const instanceBuffer = r3d.uploadInstanceData(activeSlice);
const bounds = Number.isFinite(bucketBounds[k].min[0])
? bucketBounds[k]
@@ -2910,10 +3037,17 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const mat = mesh.materialName
? v.materials[mesh.materialName]
: undefined;
+ // The variant's own material color, under the node's tint.
+ const diffuse = mat?.diffuseColor ?? [1, 1, 1, 1];
const meshOpts: DrawMesh3DOpts = {
modelMatrix: finalModelMatrix,
opacity: modelOpacity * (mat?.opacity ?? 1.0),
- colorTint,
+ colorTint: [
+ colorTint[0] * diffuse[0],
+ colorTint[1] * diffuse[1],
+ colorTint[2] * diffuse[2],
+ colorTint[3] * (diffuse[3] ?? 1),
+ ],
transparent: false,
twoSided: target?.twoSided ?? lop.twoSided ?? true,
wireframe: Boolean(target?.wireframe ?? lop.wireframe),
@@ -2947,6 +3081,12 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
instanceBuffer,
instanceCount: bCount,
bounds,
+ wind: meshWind(
+ lop.wind,
+ v,
+ (shiftedFrame ?? 0) / (fps || 30),
+ ),
+ crowd: crowdOpts(crowdBakes[k], mesh),
};
pending3DItems.push({
kind: "mesh",
@@ -3397,7 +3537,18 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
finalModelMatrix,
model.rootTransform ?? Matrix4Math.identity(),
);
- const posed = model.nodes ? poseNodes(model, clip, clipSec) : undefined;
+ const posed = model.nodes
+ ? poseCharacter({
+ model,
+ node: lop,
+ frame: shiftedFrame ?? 0,
+ fps: fps || 30,
+ clip,
+ clipSec,
+ sceneRoot,
+ key: String(lop.id),
+ })
+ : undefined;
const skinCache = new Map<number, Float32Array>();
const placeMesh = (
mesh: Mesh3DData,
@@ -3689,6 +3840,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
dispersion: target?.dispersion ?? lop.dispersion,
thickness: target?.thickness ?? lop.thickness,
audioDeform: meshAudioDeform,
+ wind: meshWind(lop.wind, model, (shiftedFrame ?? 0) / (fps || 30)),
};
if (mat) meshOpts.maps = await r3d.materialMaps(mat);
diff --git a/packages/compositions/src/program/character-schema.ts b/packages/compositions/src/program/character-schema.ts
new file mode 100644
index 00000000..98f39e99
--- /dev/null
+++ b/packages/compositions/src/program/character-schema.ts
@@ -0,0 +1,105 @@
+/**
+ * Schemas for character animation on model nodes: layered clips, two-bone
+ * IK, look-at, spring bones, and skinned crowds on instance nodes.
+ *
+ * Time-varying values are tracks: a constant, keyframes `[[frame, …value]]`
+ * (linear, in the layer's own frames), or, in process, a function of the
+ * frame (not serializable; a spec sent as JSON keeps constants and keys).
+ */
+import { z } from "zod";
+
+const Vec3Schema = z.tuple([z.number(), z.number(), z.number()]);
+const isFunction = (v: unknown) => typeof v === "function";
+
+/** A number over time: constant, `[[frame, value], …]` keys, or `(frame) => value`. */
+export const NumberTrackSchema = z.union([
+ z.number(),
+ z.array(z.tuple([z.number(), z.number()])).min(1),
+ z.custom<(frame: number) => number>(isFunction),
+]);
+export type NumberTrack = z.infer<typeof NumberTrackSchema>;
+
+/** A point over time: `[x, y, z]`, `[[frame, x, y, z], …]` keys, or `(frame) => [x, y, z]`. */
+export const Vec3TrackSchema = z.union([
+ Vec3Schema,
+ z.array(z.tuple([z.number(), z.number(), z.number(), z.number()])).min(1),
+ z.custom<(frame: number) => [number, number, number]>(isFunction),
+]);
+export type Vec3Track = z.infer<typeof Vec3TrackSchema>;
+
+const ClipRefSchema = z.union([z.string(), z.number().int().min(0)]);
+
+export const ModelAnimationLayerSchema = z
+ .object({
+ /** Clip name or index in the model file. */
+ clip: ClipRefSchema,
+ /** 0..1 over the layers beneath. Default 1. */
+ weight: NumberTrackSchema.optional(),
+ /** Playback speed. Default 1. */
+ speed: z.number().optional(),
+ /** Seconds into the clip at the layer's first frame. Default 0. */
+ timeOffset: z.number().optional(),
+ /** Only these joints and their descendants take this layer. */
+ mask: z.array(z.string()).optional(),
+ /** Default true; false holds the last pose. */
+ loop: z.boolean().optional(),
+ })
+ .strict();
+export type ModelAnimationLayer = z.infer<typeof ModelAnimationLayerSchema>;
+
+export const TwoBoneIkSchema = z
+ .object({
+ /** [root, mid, effector] joint names, e.g. ["UpperArm.L", "LowerArm.L", "Hand.L"]. */
+ chain: z.tuple([z.string(), z.string(), z.string()]),
+ /** Where the effector reaches, in scene coordinates. */
+ target: Vec3TrackSchema,
+ /** The knee / elbow bends toward this point, scene coordinates. */
+ poleTarget: Vec3TrackSchema.optional(),
+ weight: NumberTrackSchema.optional(),
+ })
+ .strict();
+export type TwoBoneIkConstraint = z.infer<typeof TwoBoneIkSchema>;
+
+export const LookAtSchema = z
+ .object({
+ joint: z.string(),
+ /** Scene coordinates. */
+ target: Vec3TrackSchema,
+ weight: NumberTrackSchema.optional(),
+ /** Largest turn from the animated pose, degrees. Default 70. */
+ maxAngleDeg: z.number().min(0).max(180).optional(),
+ /** The joint's local axis that faces the target. Default [0, 0, 1] (glTF forward). */
+ axis: Vec3Schema.optional(),
+ })
+ .strict();
+export type LookAtConstraint = z.infer<typeof LookAtSchema>;
+
+export const SpringBoneChainSchema = z
+ .object({
+ /** Root joints of chains (cape, tail, antenna, hair); each runs down its first child. */
+ joints: z.array(z.string()).min(1),
+ /** Pull back toward the animated pose, per second. Default 6. */
+ stiffness: z.number().min(0).optional(),
+ /** Share of velocity lost per second, 0..1. Default 0.6. */
+ damping: z.number().min(0).max(1).optional(),
+ /** Scene acceleration, px/s² (+y is down). Default [0, 0, 0]. */
+ gravity: Vec3Schema.optional(),
+ })
+ .strict();
+export type SpringBoneChain = z.infer<typeof SpringBoneChainSchema>;
+
+export const CrowdSchema = z
+ .object({
+ /** Clips the instances play (names or indices); instance i plays clip i % n unless it says. */
+ clips: z.array(ClipRefSchema).min(1),
+ /** Baked samples per second. Default 30. */
+ fps: z.number().min(1).max(120).optional(),
+ /** Seeds each instance's phase and speed. */
+ seed: z.number().optional(),
+ /** Random playback speed per instance. Default [1, 1]. */
+ speedRange: z.tuple([z.number().min(0), z.number().min(0)]).optional(),
+ /** "random" (default) staggers instances through their clip; "sync" starts all together. */
+ phase: z.enum(["random", "sync"]).optional(),
+ })
+ .strict();
+export type CrowdConfig = z.infer<typeof CrowdSchema>;
diff --git a/packages/compositions/src/program/index.ts b/packages/compositions/src/program/index.ts
index aa0489ae..4c64317e 100644
--- a/packages/compositions/src/program/index.ts
+++ b/packages/compositions/src/program/index.ts
@@ -8,11 +8,13 @@
export * from "../utils/layout/resolve-layout.js";
export * from "./animation.js";
+export * from "./character-schema.js";
export * from "./evaluator.js";
export * from "./lint.js";
export * from "./mattes.js";
export * from "./media-signal.js";
export * from "./path-morph.js";
+export * from "./procedural-schema.js";
export * from "./schema.js";
export * from "./sequence.js";
export * from "./stagger.js";
diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.ts
index 2ee25293..cb1e9db0 100644
--- a/packages/compositions/src/program/lint.ts
+++ b/packages/compositions/src/program/lint.ts
@@ -11,6 +11,7 @@ import {
parseCssGradient,
type RenderDiagnostic,
} from "@framefields/core";
+import type { Scatter3DSpec } from "./procedural-schema.js";
import type { CompositorProgramConfig, LayoutNode } from "./schema.js";
type AnyNode = LayoutNode & Record<string, unknown>;
@@ -61,8 +62,17 @@ export interface LintOptions {
* the emoji would draw nothing. Unknown (omitted): not checked.
*/
emojiFont?: boolean;
+ /**
+ * Share of a scatter's surface area its slope and altitude filters keep
+ * (0..1), or undefined when the surface can't be measured here (a model
+ * file not loaded yet). framefields passes the real mesher; this module
+ * has no geometry of its own.
+ */
+ scatterCoverage?: (scatter: Scatter3DSpec) => number | undefined;
}
+const DIRECT_LIGHTS = new Set(["directional", "spot", "point", "rect"]);
+
/** Contrast below this makes text hard to read (WCAG large-text minimum). */
const MIN_TEXT_CONTRAST = 3;
@@ -121,6 +131,8 @@ export function lintComposition(
const pbrNodes: string[] = [];
const emojiText: string[] = [];
const seenIds = new Set<string>();
+ let directLights = 0;
+ let fogLight: string | undefined;
const visit = (node: AnyNode, backdrop: string | null, atRoot: boolean) => {
const id = String(node.id ?? "?");
@@ -167,6 +179,8 @@ export function lintComposition(
const type = node.lightType ?? "directional";
if (type === "environment") hasEnvironment = true;
if (type === "ambient") ambientLights.push(id);
+ if (DIRECT_LIGHTS.has(String(type))) directLights++;
+ if (type === "environment" && node.volumetricFog) fogLight = id;
if (node.castShadow) {
if (
type !== "directional" &&
@@ -219,6 +233,18 @@ export function lintComposition(
const is3D =
Boolean(node.is3D) || (typeof node.z === "number" && node.z !== 0);
if (is3D || node.kind === "model3d") has3D = true;
+ const scatter = node.scatter3d as Scatter3DSpec | undefined;
+ if (
+ scatter &&
+ scatter.count > 0 &&
+ options.scatterCoverage?.(scatter) === 0
+ )
+ push(
+ "warning",
+ "scatter_filters_reject_all",
+ `'${id}' scatters onto a surface its slopeRange / elevationRange filters reject entirely, so no instance is placed. Widen the ranges (slope is degrees from level, elevation 0..1 from the surface's lowest to highest point).`,
+ id,
+ );
if (node.roughness !== undefined || node.metallic !== undefined)
pbrNodes.push(id);
@@ -350,6 +376,15 @@ export function lintComposition(
);
}
+ if (fogLight && directLights === 0) {
+ push(
+ "warning",
+ "volumetric_fog_without_lights",
+ `'${fogLight}' turns on volumetricFog, but the scene has no directional, spot, point or rect light: fog scatters only direct light, so there are no light shafts. Add a light (castShadow for shafts).`,
+ fogLight,
+ );
+ }
+
if (has3D && pbrNodes.length > 0 && !hasEnvironment) {
push(
"info",
diff --git a/packages/compositions/src/program/mattes.ts b/packages/compositions/src/program/mattes.ts
index 825ff25a..42135f85 100644
--- a/packages/compositions/src/program/mattes.ts
+++ b/packages/compositions/src/program/mattes.ts
@@ -1,7 +1,8 @@
/**
- * Track mattes and reliefs: a node's `matte` or `relief` is hoisted out into
- * its own layer right after it, so layout places it like any sibling and the
- * renderer can draw owner and companion separately and combine them.
+ * Track mattes, reliefs and texture layers: a node's `matte`, `relief` or
+ * `textureLayer` is hoisted out into its own layer right after it, so layout
+ * places it like any sibling and the renderer can draw owner and companion
+ * separately and combine them.
*/
import type { LayoutNode, MatteMode } from "./schema.js";
@@ -38,13 +39,45 @@ function companion(
return node;
}
-function hoistOne(owner: AnyNode): AnyNode[] {
+/** The picture a mesh's UVs read: the frame's size from the top-left corner unless placed. */
+export interface HoistOptions {
+ width?: number;
+ height?: number;
+}
+
+function textureCompanion(
+ owner: AnyNode,
+ src: AnyNode,
+ size: HoistOptions,
+): AnyNode {
+ const node = companion(owner, src, "texture", {
+ textureOf: String(owner.id),
+ is3D: false,
+ z: 0,
+ });
+ // A model has no box to cover: the texture's own geometry, else the frame.
+ if (GEOMETRY.every((k) => src[k] === undefined)) {
+ node.x = 0;
+ node.y = 0;
+ if (size.width !== undefined) node.width = size.width;
+ if (size.height !== undefined) node.height = size.height;
+ }
+ return node;
+}
+
+function hoistOne(owner: AnyNode, size: HoistOptions): AnyNode[] {
const matte = owner.matte as { node: AnyNode; mode?: MatteMode } | undefined;
const relief = owner.relief as { node: AnyNode; amount?: number } | undefined;
- const { matte: _matte, relief: _relief, ...rest } = owner;
+ const texture = owner.textureLayer as { node: AnyNode } | undefined;
+ const {
+ matte: _matte,
+ relief: _relief,
+ textureLayer: _texture,
+ ...rest
+ } = owner;
const self = { ...rest } as AnyNode;
if (Array.isArray(self.children)) {
- self.children = hoistMattes(self.children as AnyNode[]) as never;
+ self.children = hoistMattes(self.children as AnyNode[], size) as never;
}
const out = [self];
if (matte?.node) {
@@ -66,10 +99,18 @@ function hoistOne(owner: AnyNode): AnyNode[] {
self.reliefAmount = relief.amount ?? 0;
out.push(node);
}
+ if (texture?.node) {
+ const node = textureCompanion(owner, texture.node, size);
+ self.textureLayerId = node.id as string;
+ out.push(node);
+ }
return out;
}
-/** Returns the tree with every `matte` and `relief` turned into a sibling companion layer. */
-export function hoistMattes<T extends LayoutNode>(nodes: readonly T[]): T[] {
- return nodes.flatMap((n) => hoistOne(n as AnyNode)) as unknown as T[];
+/** Returns the tree with every `matte`, `relief` and `textureLayer` turned into a sibling companion layer. */
+export function hoistMattes<T extends LayoutNode>(
+ nodes: readonly T[],
+ size: HoistOptions = {},
+): T[] {
+ return nodes.flatMap((n) => hoistOne(n as AnyNode, size)) as unknown as T[];
}
diff --git a/packages/compositions/src/program/procedural-schema.ts b/packages/compositions/src/program/procedural-schema.ts
new file mode 100644
index 00000000..180b5ba3
--- /dev/null
+++ b/packages/compositions/src/program/procedural-schema.ts
@@ -0,0 +1,215 @@
+/**
+ * Schemas for procedural 3D content: terrain, SDF sculptures, surface
+ * scattering and wind. A model node's `primitive3d` is checked against the
+ * terrain / SDF schema when it names one of them.
+ */
+import { z } from "zod";
+
+const Vec3Schema = z.tuple([z.number(), z.number(), z.number()]);
+const RangeSchema = (min: number, max: number) =>
+ z.tuple([z.number().min(min).max(max), z.number().min(min).max(max)]);
+const ShadingSchema = z.enum(["lit", "unlit", "toon"]);
+
+const PrimitiveSurfaceFields = {
+ color: z.string().optional(),
+ material: ShadingSchema.optional(),
+ shininess: z.number().min(0).optional(),
+ roughness: z.number().min(0).max(1).optional(),
+ metallic: z.number().min(0).max(1).optional(),
+};
+
+export const TerrainNoiseSchema = z
+ .object({
+ seed: z.number().optional(),
+ /** Size of the largest landforms, px. */
+ scale: z.number().positive().optional(),
+ octaves: z.number().int().min(1).max(10).optional(),
+ persistence: z.number().min(0).max(1).optional(),
+ lacunarity: z.number().min(1).max(4).optional(),
+ ridgeMode: z.enum(["fbm", "ridged", "terrace"]).optional(),
+ terraceSteps: z.number().int().min(2).max(32).optional(),
+ })
+ .strict();
+
+export const TerrainColorByElevationSchema = z
+ .object({
+ lowColor: z.string(),
+ midColor: z.string().optional(),
+ highColor: z.string(),
+ cliffColor: z.string().optional(),
+ cliffThreshold: z.number().min(0).max(1).optional(),
+ bands: RangeSchema(0, 1).optional(),
+ })
+ .strict();
+
+export const TerrainGeometrySchema = z
+ .object({
+ type: z.literal("terrain"),
+ width: z.number().positive(),
+ depth: z.number().positive(),
+ height: z.number().min(0).optional(),
+ segments: z
+ .union([
+ z.number().int().min(1).max(512),
+ z.tuple([
+ z.number().int().min(1).max(512),
+ z.number().int().min(1).max(512),
+ ]),
+ ])
+ .optional(),
+ noise: TerrainNoiseSchema.optional(),
+ heights: z.array(z.number().min(0).max(1)).optional(),
+ overhangWarp: z.number().min(0).optional(),
+ colorByElevation: TerrainColorByElevationSchema.optional(),
+ flatShaded: z.boolean().optional(),
+ base: z.number().min(0).optional(),
+ ...PrimitiveSurfaceFields,
+ })
+ .strict();
+
+const SdfBaseFields = {
+ op: z.enum(["union", "subtract", "intersect"]).optional(),
+ smoothness: z.number().min(0).optional(),
+ center: Vec3Schema.optional(),
+ rotation: Vec3Schema.optional(),
+ color: z.string().optional(),
+};
+
+export const SdfPrimitiveSchema = z.discriminatedUnion("shape", [
+ z
+ .object({
+ ...SdfBaseFields,
+ shape: z.literal("sphere"),
+ radius: z.number().positive(),
+ })
+ .strict(),
+ z
+ .object({
+ ...SdfBaseFields,
+ shape: z.literal("box"),
+ size: z.tuple([
+ z.number().positive(),
+ z.number().positive(),
+ z.number().positive(),
+ ]),
+ radius: z.number().min(0).optional(),
+ })
+ .strict(),
+ z
+ .object({
+ ...SdfBaseFields,
+ shape: z.literal("cylinder"),
+ radius: z.number().positive(),
+ height: z.number().positive(),
+ })
+ .strict(),
+ z
+ .object({
+ ...SdfBaseFields,
+ shape: z.literal("capsule"),
+ radius: z.number().positive(),
+ height: z.number().positive(),
+ })
+ .strict(),
+ z
+ .object({
+ ...SdfBaseFields,
+ shape: z.literal("torus"),
+ radius: z.number().positive(),
+ tube: z.number().positive(),
+ })
+ .strict(),
+ z
+ .object({
+ ...SdfBaseFields,
+ shape: z.literal("gyroid"),
+ scale: z.number().positive(),
+ thickness: z.number().positive(),
+ })
+ .strict(),
+]);
+
+export const SdfGeometrySchema = z
+ .object({
+ type: z.literal("sdf"),
+ primitives: z.array(SdfPrimitiveSchema).min(1),
+ bounds: z.tuple([
+ z.number().positive(),
+ z.number().positive(),
+ z.number().positive(),
+ ]),
+ resolution: z.number().int().min(16).max(160).optional(),
+ flatShaded: z.boolean().optional(),
+ ...PrimitiveSurfaceFields,
+ })
+ .strict();
+
+/** `primitive3d`: free-form for the classic primitives, strict for terrain and SDF. */
+export const Primitive3DSpecSchema = z.unknown().superRefine((value, ctx) => {
+ const type = (value as { type?: unknown } | null)?.type;
+ const schema =
+ type === "terrain"
+ ? TerrainGeometrySchema
+ : type === "sdf"
+ ? SdfGeometrySchema
+ : undefined;
+ if (!schema) return;
+ const parsed = schema.safeParse(value);
+ if (parsed.success) return;
+ for (const issue of parsed.error.issues) {
+ ctx.addIssue({ ...issue, code: "custom", message: issue.message });
+ }
+});
+
+export const WindSchema = z
+ .object({
+ /** Peak lean, degrees. Default 8. */
+ amplitudeDeg: z.number().min(0).max(90).optional(),
+ /** Sways per second. Default 0.4. */
+ frequency: z.number().min(0).max(20).optional(),
+ /** Downwind on the ground plane [x, z]. Default [1, 0]. */
+ direction: z.tuple([z.number(), z.number()]).optional(),
+ /** Gust wavelength across the field, px. Default 600. */
+ waveScale: z.number().positive().optional(),
+ })
+ .strict();
+export type WindConfig = z.infer<typeof WindSchema>;
+
+export const ScatterExclusionSchema = z
+ .object({
+ x: z.number(),
+ z: z.number(),
+ radius: z.number().min(0),
+ falloff: z.number().min(0).optional(),
+ })
+ .strict();
+
+export const Scatter3DSchema = z
+ .object({
+ /** The surface node (any model3d node: terrain, sdf, primitive, file). */
+ surface: z.custom<Record<string, unknown>>(
+ (v) => typeof v === "object" && v !== null,
+ ),
+ count: z.number().int().min(0).max(200_000),
+ minDistance: z.number().min(0).optional(),
+ seed: z.number().optional(),
+ slopeRange: RangeSchema(0, 180).optional(),
+ elevationRange: RangeSchema(0, 1).optional(),
+ alignToNormal: z.number().min(0).max(1).optional(),
+ scaleRange: z
+ .tuple([z.number().positive(), z.number().positive()])
+ .optional(),
+ randomYaw: z.boolean().optional(),
+ exclude: z.array(ScatterExclusionSchema).optional(),
+ /** Push each instance this far into the surface, px. */
+ sink: z.number().optional(),
+ })
+ .strict();
+export type Scatter3DSpec = z.infer<typeof Scatter3DSchema>;
+
+/** A layer drawn offscreen every frame and wrapped onto a model's surface. */
+export const TextureLayerSchema = z
+ .object({
+ node: z.custom<unknown>((v) => typeof v === "object" && v !== null),
+ })
+ .strict();
diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.ts
index 3d43b70c..f8ca0705 100644
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@ -27,6 +27,26 @@ import {
validateMediaTimeRemap,
} from "@framefields/core";
import { ColorSchema } from "@framefields/node-sdk";
+import {
+ type CrowdConfig,
+ CrowdSchema,
+ type LookAtConstraint,
+ LookAtSchema,
+ type ModelAnimationLayer,
+ ModelAnimationLayerSchema,
+ type SpringBoneChain,
+ SpringBoneChainSchema,
+ type TwoBoneIkConstraint,
+ TwoBoneIkSchema,
+} from "./character-schema.js";
+import {
+ Primitive3DSpecSchema,
+ Scatter3DSchema,
+ type Scatter3DSpec,
+ TextureLayerSchema,
+ type WindConfig,
+ WindSchema,
+} from "./procedural-schema.js";
export type { TextSpan, TextSpanMark };
export type { ToonShading, GeometryOutlineOptions };
@@ -934,6 +954,8 @@ const BaseNodeFields = {
/** Set by the compiler: this layer is the relief of node `reliefOf` (not drawn by itself). */
reliefOf: z.string().optional(),
reliefAmount: z.number().optional(),
+ /** Set by the compiler: this layer is the surface texture of model `textureOf` (not drawn by itself). */
+ textureOf: z.string().optional(),
/** Per-node motion: tracks keyed by node id. */
animation: LayerAnimationSchema.default({ tracks: [] }),
/** Vision configuration or effect attached directly to the node. */
@@ -1016,6 +1038,7 @@ export interface CompositionNodeBase {
reliefId?: string;
reliefOf?: string;
reliefAmount?: number;
+ textureOf?: string;
envIntensity?: number;
startFrame?: number;
durationFrames?: number;
@@ -1783,6 +1806,24 @@ export interface Model3DNode {
instances3d?: unknown;
/** Layer3D.swarm GPU simulation and instancing options. */
swarm3d?: unknown;
+ /** Layer3D.scatter: instances placed on another model's surface. */
+ scatter3d?: Scatter3DSpec;
+ /** Procedural wind sway (bends vertices downwind by height above the mesh base). */
+ wind?: WindConfig;
+ /** Any layer (image, video, layout) drawn each frame and wrapped onto the mesh by its UVs. */
+ textureLayer?: { node: unknown };
+ /** Set by the compiler: the id of the hoisted texture layer. */
+ textureLayerId?: string;
+ /** Clips blended in order, each over the pose beneath by its weight (replaces animationName). */
+ animationLayers?: ModelAnimationLayer[];
+ /** Two-bone IK: limbs reach scene-space targets. */
+ ik?: TwoBoneIkConstraint[];
+ /** Joints (heads, eyes) turn toward scene-space targets. */
+ lookAt?: LookAtConstraint[];
+ /** Secondary motion simulated from frame 0 (capes, tails, hair). */
+ springBones?: SpringBoneChain[];
+ /** Instance nodes: every instance a skinned character on its own clip, phase and speed. */
+ crowd?: CrowdConfig;
}
export type LayoutNode =
@@ -2104,9 +2145,18 @@ export const Model3DNodeSchema: z.ZodType<Model3DNode> = z
color: ColorSchema.optional(),
colorTint: ColorSchema.optional(),
text3dOptions: z.unknown().optional(),
- primitive3d: z.unknown().optional(),
+ primitive3d: Primitive3DSpecSchema.optional(),
instances3d: z.unknown().optional(),
swarm3d: z.unknown().optional(),
+ scatter3d: Scatter3DSchema.optional(),
+ wind: WindSchema.optional(),
+ textureLayer: TextureLayerSchema.optional(),
+ textureLayerId: z.string().optional(),
+ animationLayers: z.array(ModelAnimationLayerSchema).optional(),
+ ik: z.array(TwoBoneIkSchema).optional(),
+ lookAt: z.array(LookAtSchema).optional(),
+ springBones: z.array(SpringBoneChainSchema).optional(),
+ crowd: CrowdSchema.optional(),
})
.strict() as unknown as z.ZodType<Model3DNode>;
diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.ts
index 9540c9cb..f381f4d3 100644
--- a/packages/compositions/src/program/to-virtual-media.ts
+++ b/packages/compositions/src/program/to-virtual-media.ts
@@ -473,7 +473,10 @@ export function compositorToProgram(
}
: {}),
},
- children: hoistMattes(config.layout).map((n) =>
+ children: hoistMattes(config.layout, {
+ width: config.width,
+ height: config.height,
+ }).map((n) =>
nodeToVirtualMedia(
n,
resolveMedia,
diff --git a/packages/framefields/src/character-render.test.ts b/packages/framefields/src/character-render.test.ts
new file mode 100644
index 00000000..6a7f9111
--- /dev/null
+++ b/packages/framefields/src/character-render.test.ts
@@ -0,0 +1,325 @@
+/**
+ * Character animation through the compositor: clip layers, IK, look-at,
+ * spring bones and skinned crowds, on a GLB built here: a 20-wide strip
+ * skinned to a three-joint chain (root at 0, knee 80 up, foot 160 up,
+ * the strip reaching 200), with a "fold" clip (knee +90° about z over 1 s)
+ * and a "curl" clip (knee -90°).
+ */
+import fs from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import {
+ Composition,
+ CompositorProgramSchema,
+ Layer,
+ Layer3D,
+ type LayoutNode,
+} from "./index.js";
+import { GlbBuilder } from "./test-utils/glb.js";
+import { decodePngPixels } from "./test-utils/pixels.js";
+
+const SIZE = 400;
+const C = SIZE / 2;
+
+let dir = "";
+let file = "";
+
+function quarter(sign: number): Float32Array {
+ const s = Math.SQRT1_2;
+ return new Float32Array([0, 0, 0, 1, 0, 0, sign * s, s]);
+}
+
+function buildRig(): Buffer {
+ const g = new GlbBuilder();
+ const ys = [0, 40, 80, 120, 160, 200];
+ const joint = [0, 0, 1, 1, 2, 2];
+ const positions: number[] = [];
+ const joints: number[] = [];
+ const weights: number[] = [];
+ ys.forEach((y, k) => {
+ for (const x of [-10, 10]) {
+ positions.push(x, y, 0);
+ joints.push(joint[k]!, 0, 0, 0);
+ weights.push(1, 0, 0, 0);
+ }
+ });
+ const indices: number[] = [];
+ for (let k = 0; k + 1 < ys.length; k++) {
+ const a = k * 2;
+ indices.push(a, a + 1, a + 3, a, a + 3, a + 2);
+ }
+ const ibm = [0, -80, -160].flatMap((ty) => [
+ 1,
+ 0,
+ 0,
+ 0,
+ 0,
+ 1,
+ 0,
+ 0,
+ 0,
+ 0,
+ 1,
+ 0,
+ 0,
+ ty,
+ 0,
+ 1,
+ ]);
+ const times = g.accessor(new Float32Array([0, 1]), "SCALAR");
+ const clip = (name: string, sign: number) => ({
+ name,
+ channels: [{ sampler: 0, target: { node: 2, path: "rotation" } }],
+ samplers: [{ input: times, output: g.accessor(quarter(sign), "VEC4") }],
+ });
+ return g.build({
+ scenes: [{ nodes: [0, 1] }],
+ nodes: [
+ { mesh: 0, skin: 0 },
+ { name: "root", children: [2] },
+ { name: "knee", translation: [0, 80, 0], children: [3] },
+ { name: "foot", translation: [0, 80, 0] },
+ ],
+ skins: [
+ {
+ joints: [1, 2, 3],
+ inverseBindMatrices: g.accessor(new Float32Array(ibm), "MAT4"),
+ },
+ ],
+ meshes: [
+ {
+ primitives: [
+ {
+ attributes: {
+ POSITION: g.accessor(new Float32Array(positions), "VEC3"),
+ JOINTS_0: g.accessor(new Uint16Array(joints), "VEC4"),
+ WEIGHTS_0: g.accessor(new Float32Array(weights), "VEC4"),
+ },
+ indices: g.accessor(new Uint16Array(indices), "SCALAR"),
+ material: 0,
+ },
+ ],
+ },
+ ],
+ materials: [{ pbrMetallicRoughness: { baseColorFactor: [1, 1, 1, 1] } }],
+ animations: [clip("fold", 1), clip("curl", -1)],
+ });
+}
+
+beforeAll(async () => {
+ dir = await fs.mkdtemp(path.join(os.tmpdir(), "framefields-character-"));
+ file = path.join(dir, "leg.glb");
+ await fs.writeFile(file, buildRig());
+});
+afterAll(async () => {
+ await fs.rm(dir, { recursive: true, force: true });
+});
+
+function scene(layers: LayoutNode[]): Composition {
+ const comp = new Composition({
+ width: SIZE,
+ height: SIZE,
+ fps: 30,
+ durationFrames: 120,
+ backgroundColor: "#000000",
+ });
+ comp.add([
+ Layer.camera({
+ x: C,
+ y: C,
+ // A 50° field of view at this distance maps one model unit to one pixel.
+ z: -C / Math.tan((25 * Math.PI) / 180),
+ targetX: C,
+ targetY: C,
+ targetZ: 0,
+ }),
+ ...layers,
+ ]);
+ return comp;
+}
+
+/** The leg, its root at the frame's center; model +y is up on screen. */
+const leg = (extra: Record<string, unknown> = {}) =>
+ Layer.glb(file, {
+ id: "leg",
+ x: C,
+ y: C,
+ z: 0,
+ rotateX: 180,
+ material: "unlit",
+ ...extra,
+ });
+
+/** Whether model point (x, y) is drawn (white), for a model rooted at screen (ox, oy). */
+async function probe(comp: Composition, frame = 0) {
+ const { pixels } = await decodePngPixels(await comp.renderFrame({ frame }));
+ return (x: number, y: number, ox = C, oy = C) => {
+ const i = (Math.round(oy - y) * SIZE + Math.round(ox + x)) * 4;
+ return (pixels[i] ?? 0) > 128;
+ };
+}
+
+describe("character animation", () => {
+ it("blends clip layers by weight, over the pose beneath", async () => {
+ const at = (weight: number) =>
+ probe(
+ scene([
+ leg({
+ animationLayers: [
+ { clip: "fold", timeOffset: 1, loop: false },
+ { clip: "curl", timeOffset: 1, loop: false, weight },
+ ],
+ }),
+ ]),
+ );
+ // Folded +90°: the shin points -x. Blended halfway back toward -90°… the
+ // two cancel at weight 0.5 by slerp: straight up again.
+ const folded = await at(0);
+ expect(folded(-60, 80)).toBe(true);
+ expect(folded(0, 140)).toBe(false);
+ const straight = await at(0.5);
+ expect(straight(0, 140)).toBe(true);
+ const curled = await at(1);
+ expect(curled(60, 80)).toBe(true);
+ });
+
+ it("reaches an IK target with the foot, the knee toward the pole", async () => {
+ // Target model (100, 60): scene (C + 100, C - 60).
+ const lit = await probe(
+ scene([
+ leg({
+ ik: [
+ {
+ chain: ["root", "knee", "foot"],
+ target: [C + 100, C - 60, 0],
+ poleTarget: [C + 200, C - 200, 0],
+ },
+ ],
+ }),
+ ]),
+ );
+ expect(lit(100, 60)).toBe(true);
+ expect(lit(0, 140)).toBe(false);
+ // The knee, 80 from both root and target, sits on the pole's side of
+ // the root→target line: (50, 30) + 54.8 · (-0.514, 0.857).
+ const knee: [number, number] = [22, 77];
+ expect(lit(...knee)).toBe(true);
+ });
+
+ it("turns a joint toward a look-at target within its limit", async () => {
+ const look = (max: number) =>
+ probe(
+ scene([
+ leg({
+ lookAt: [
+ {
+ joint: "root",
+ target: [C + 300, C, 0],
+ axis: [0, 1, 0],
+ maxAngleDeg: max,
+ },
+ ],
+ }),
+ ]),
+ );
+ const full = await look(180);
+ expect(full(150, 0)).toBe(true);
+ expect(full(0, 150)).toBe(false);
+ const limited = await look(45);
+ expect(limited(150 * Math.SQRT1_2, 150 * Math.SQRT1_2)).toBe(true);
+ });
+
+ it("simulates spring bones deterministically from frame 0", async () => {
+ const comp = scene([
+ leg({
+ // The foot's chain swings under sideways gravity; the knee clip is still.
+ springBones: [
+ {
+ joints: ["knee"],
+ stiffness: 0,
+ damping: 0.9,
+ gravity: [600, 0, 0],
+ },
+ ],
+ }),
+ ]);
+ const first = await probe(comp, 0);
+ expect(first(0, 140)).toBe(true);
+ const later = await probe(comp, 40);
+ expect(later(0, 150)).toBe(false);
+ // Rendering an earlier frame, then frame 40 again, gives the same picture.
+ await comp.renderFrame({ frame: 10 });
+ const again = await comp.renderFrame({ frame: 40 });
+ expect(again).toEqual(await comp.renderFrame({ frame: 40 }));
+ const { pixels: a } = await decodePngPixels(again);
+ const fresh = scene([
+ leg({
+ springBones: [
+ {
+ joints: ["knee"],
+ stiffness: 0,
+ damping: 0.9,
+ gravity: [600, 0, 0],
+ },
+ ],
+ }),
+ ]);
+ const { pixels: b } = await decodePngPixels(
+ await fresh.renderFrame({ frame: 40 }),
+ );
+ expect(a).toEqual(b);
+ });
+
+ it("draws a skinned crowd, each instance at its own point in its clip", async () => {
+ // The mesh node's own rotateX (the leg is y-up) applies under each instance's.
+ const crowd = Layer3D.instances({
+ mesh: leg(),
+ x: C,
+ y: C,
+ z: 0,
+ material: "unlit",
+ crowd: { clips: ["fold", "curl"], phase: "sync" },
+ transforms: [
+ { x: -120, y: 0, z: 0, clip: 0, animOffset: 0 },
+ { x: 0, y: 0, z: 0, clip: 0, animOffset: 0.5 },
+ { x: 120, y: 0, z: 0, clip: 1, animOffset: 0.98 },
+ ],
+ });
+ const lit = await probe(scene([crowd]));
+ // Straight, folded 45° left, curled ~90° right.
+ expect(lit(0, 140, C - 120)).toBe(true);
+ expect(lit(-40, 120, C)).toBe(true);
+ expect(lit(0, 150, C)).toBe(false);
+ expect(lit(60, 80, C + 120)).toBe(true);
+ expect(lit(0, 150, C + 120)).toBe(false);
+ });
+
+ it("keeps character options schema-valid", () => {
+ const comp = scene([
+ leg({
+ animationLayers: [
+ {
+ clip: "fold",
+ weight: [
+ [0, 0],
+ [30, 1],
+ ],
+ mask: ["knee"],
+ },
+ ],
+ ik: [{ chain: ["root", "knee", "foot"], target: [[0, 1, 2, 3]] }],
+ lookAt: [{ joint: "root", target: [0, 0, 0], maxAngleDeg: 30 }],
+ springBones: [{ joints: ["knee"], gravity: [0, 980, 0] }],
+ }),
+ Layer3D.instances({
+ mesh: leg(),
+ count: 2,
+ crowd: { clips: ["fold"], fps: 24 },
+ }),
+ ]);
+ expect(CompositorProgramSchema.safeParse(comp.toSpec()).success).toBe(true);
+ const bad = scene([leg({ springBones: [{ joints: [] }] })]);
+ expect(CompositorProgramSchema.safeParse(bad.toSpec()).success).toBe(false);
+ });
+});
diff --git a/packages/framefields/src/gltf-scene-render.test.ts b/packages/framefields/src/gltf-scene-render.test.ts
index 26f141a4..c5d546aa 100644
--- a/packages/framefields/src/gltf-scene-render.test.ts
+++ b/packages/framefields/src/gltf-scene-render.test.ts
@@ -10,91 +10,11 @@ import path from "node:path";
import { Canvas, loadImage } from "skia-canvas";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { Composition, Layer } from "./index.js";
+import { GlbBuilder } from "./test-utils/glb.js";
const SIZE = 400;
const BG = "#000000";
-/** Accumulates typed arrays into one binary buffer and the accessors that view them. */
-class GlbBuilder {
- private chunks: Buffer[] = [];
- private length = 0;
- readonly bufferViews: object[] = [];
- readonly accessors: object[] = [];
-
- private view(data: Buffer): number {
- const pad = (4 - (this.length % 4)) % 4;
- if (pad) {
- this.chunks.push(Buffer.alloc(pad));
- this.length += pad;
- }
- this.bufferViews.push({
- buffer: 0,
- byteOffset: this.length,
- byteLength: data.length,
- });
- this.chunks.push(data);
- this.length += data.length;
- return this.bufferViews.length - 1;
- }
-
- accessor(
- data: Float32Array | Uint16Array,
- type: "SCALAR" | "VEC2" | "VEC3" | "VEC4" | "MAT4",
- ): number {
- const bufferView = this.view(
- Buffer.from(data.buffer, data.byteOffset, data.byteLength),
- );
- const size = { SCALAR: 1, VEC2: 2, VEC3: 3, VEC4: 4, MAT4: 16 }[type];
- this.accessors.push({
- bufferView,
- componentType: data instanceof Float32Array ? 5126 : 5123,
- count: data.length / size,
- type,
- });
- return this.accessors.length - 1;
- }
-
- image(png: Buffer): number {
- return this.view(png);
- }
-
- build(json: Record<string, unknown>): Buffer {
- const bin = Buffer.concat(this.chunks);
- const doc = Buffer.from(
- JSON.stringify({
- asset: { version: "2.0" },
- ...json,
- buffers: [{ byteLength: bin.length }],
- bufferViews: this.bufferViews,
- accessors: this.accessors,
- }),
- );
- const docPadded = Buffer.concat([
- doc,
- Buffer.alloc((4 - (doc.length % 4)) % 4, 0x20),
- ]);
- const binPadded = Buffer.concat([
- bin,
- Buffer.alloc((4 - (bin.length % 4)) % 4),
- ]);
- const header = Buffer.alloc(12);
- header.writeUInt32LE(0x46546c67, 0);
- header.writeUInt32LE(2, 4);
- header.writeUInt32LE(12 + 8 + docPadded.length + 8 + binPadded.length, 8);
- const chunk = (data: Buffer, type: number) => {
- const h = Buffer.alloc(8);
- h.writeUInt32LE(data.length, 0);
- h.writeUInt32LE(type, 4);
- return Buffer.concat([h, data]);
- };
- return Buffer.concat([
- header,
- chunk(docPadded, 0x4e4f534a),
- chunk(binPadded, 0x004e4942),
- ]);
- }
-}
-
/** An axis-aligned rectangle in the XY plane as two triangles. */
function rect(x0: number, y0: number, x1: number, y1: number) {
return {
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index a18449a6..57c6c33e 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -987,6 +987,33 @@ export {
type Torus3DOptions,
type Tube3DOptions,
} from "./shapes3d.js";
+
+import { scatterCoverage } from "./procedural3d.js";
+
+export type {
+ CrowdConfig,
+ LookAtConstraint,
+ ModelAnimationLayer,
+ NumberTrack,
+ SpringBoneChain,
+ TwoBoneIkConstraint,
+ Vec3Track,
+ WindConfig,
+} from "@framefields/compositions/program";
+export type {
+ ScatterPoint,
+ SdfPrimitive,
+ TerrainColorByElevation,
+ TerrainNoiseOptions,
+} from "@framefields/webgpu-renderers";
+export type {
+ Scatter3DNode,
+ Scatter3DOptions,
+ Sdf3DOptions,
+ Terrain3DNode,
+ Terrain3DOptions,
+ TerrainSurface,
+} from "./procedural3d.js";
export { type StyleName, Styles } from "./styles.js";
export type { Swarm3DOptions } from "./swarm3d.js";
export {
@@ -2427,6 +2454,7 @@ export class Composition {
public lint(): RenderDiagnostic[] {
return lintComposition(this.toSpec(), {
emojiFont: FontManager.hasEmojiFont(),
+ scatterCoverage,
});
}
}
diff --git a/packages/framefields/src/physics/backend.ts b/packages/framefields/src/physics/backend.ts
index 98f1cb56..307fe3d5 100644
--- a/packages/framefields/src/physics/backend.ts
+++ b/packages/framefields/src/physics/backend.ts
@@ -32,7 +32,9 @@ export type Shape3D =
/** Axis along y. */
| { kind: "cylinder"; halfHeight: number; radius: number }
/** Flat [x0, y0, z0, …] around the body origin. */
- | { kind: "hull"; points: number[] };
+ | { kind: "hull"; points: number[] }
+ /** Exact triangle mesh (static / kinematic bodies: terrain, sculptures). */
+ | { kind: "trimesh"; vertices: number[]; indices: number[] };
interface BodySpecBase extends MaterialSpec {
type: BodyType;
@@ -500,5 +502,10 @@ function shapeDesc3D(R: Rapier3D, shape: Shape3D): RAPIER3D.ColliderDesc {
);
return desc;
}
+ case "trimesh":
+ return R.ColliderDesc.trimesh(
+ new Float32Array(shape.vertices),
+ new Uint32Array(shape.indices),
+ );
}
}
diff --git a/packages/framefields/src/physics/world3d.ts b/packages/framefields/src/physics/world3d.ts
index 9b0c7f10..390175e3 100644
--- a/packages/framefields/src/physics/world3d.ts
+++ b/packages/framefields/src/physics/world3d.ts
@@ -62,10 +62,12 @@ export interface Body3DOptions extends Omit<BodyCommonOptions, "lockRotation"> {
lockRotation?: boolean | { x?: boolean; y?: boolean; z?: boolean };
/**
* Collider: `"auto"` (default) reads the Layer3D primitive (box, sphere,
- * cylinder) and falls back to the mesh's convex hull for models and
- * tapered or uneven shapes. Force `"box"`, `"sphere"`, `"cylinder"` or `"hull"`.
+ * cylinder), uses the exact triangles of `Layer3D.terrain` / `Layer3D.sdf`
+ * on static bodies, and falls back to the mesh's convex hull for models and
+ * tapered or uneven shapes. Force `"box"`, `"sphere"`, `"cylinder"`,
+ * `"hull"` or `"mesh"` (exact triangles; static or kinematic bodies only).
*/
- shape?: "auto" | "box" | "sphere" | "cylinder" | "hull";
+ shape?: "auto" | "box" | "sphere" | "cylinder" | "hull" | "mesh";
/** Collider size override `[width, height, depth]`, px (before the layer's scale). */
size?: [number, number, number];
/** px/s. */
@@ -734,6 +736,15 @@ function colliderShape3D(t: LayerLike, options: Body3DOptions): Shape3D {
return { kind: "hull", points: pts.map((v, i) => v * scale[i % 3]) };
};
+ const exact =
+ mode === "mesh" ||
+ (mode === "auto" &&
+ (options.type ?? "dynamic") !== "dynamic" &&
+ (prim.type === "terrain" || prim.type === "sdf"));
+ if (exact) {
+ const mesh = meshTriangles(t.modelData, scale);
+ if (mesh) return mesh;
+ }
if (options.size || mode === "box") {
const [w, h, d] = options.size ?? boxSize(t, prim) ?? [0, 0, 0];
if (w > 0 && h > 0 && d > 0) {
@@ -847,6 +858,36 @@ function boxSize(t: LayerLike, prim: PrimitiveSpec): Vec3 | undefined {
return undefined;
}
+/** All of a model's triangles, scaled: an exact (non-convex) collider. */
+function meshTriangles(modelData: unknown, scale: Vec3): Shape3D | undefined {
+ const meshes = (
+ modelData as
+ | {
+ meshes?: Array<{
+ positions?: ArrayLike<number>;
+ indices?: ArrayLike<number>;
+ topology?: string;
+ }>;
+ }
+ | undefined
+ )?.meshes;
+ if (!meshes?.length) return undefined;
+ const vertices: number[] = [];
+ const indices: number[] = [];
+ for (const m of meshes) {
+ if (!m.positions || !m.indices) continue;
+ if (m.topology && m.topology !== "triangle-list") continue;
+ const base = vertices.length / 3;
+ for (let i = 0; i < m.positions.length; i++)
+ vertices.push(m.positions[i]! * scale[i % 3]!);
+ for (let i = 0; i < m.indices.length; i++)
+ indices.push(m.indices[i]! + base);
+ }
+ return indices.length >= 3
+ ? { kind: "trimesh", vertices, indices }
+ : undefined;
+}
+
/** Every vertex of a model, flat xyz, at most ~4k points (the hull needs no more). */
function meshPoints(modelData: unknown): number[] | undefined {
const meshes = (
@@ -891,5 +932,11 @@ function scaleShape(shape: Shape3D, s: number): Shape3D {
};
case "hull":
return { kind: "hull", points: shape.points.map((v) => v * s) };
+ case "trimesh":
+ return {
+ kind: "trimesh",
+ vertices: shape.vertices.map((v) => v * s),
+ indices: shape.indices,
+ };
}
}
diff --git a/packages/framefields/src/procedural3d-render.test.ts b/packages/framefields/src/procedural3d-render.test.ts
new file mode 100644
index 00000000..2b2a0e8b
--- /dev/null
+++ b/packages/framefields/src/procedural3d-render.test.ts
@@ -0,0 +1,272 @@
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ CompositorProgramSchema,
+ Layer,
+ Layer3D,
+ type LayoutNode,
+} from "./index.js";
+import { Physics } from "./physics/index.js";
+import { decodePngPixels } from "./test-utils/pixels.js";
+
+const W = 320;
+const H = 180;
+
+type Rgb = [number, number, number];
+
+function comp(layers: LayoutNode[], frames = 1): Composition {
+ const c = new Composition({
+ width: W,
+ height: H,
+ fps: 30,
+ durationFrames: frames,
+ backgroundColor: "#000000",
+ });
+ c.add([
+ Layer.camera({
+ x: 160,
+ y: 40,
+ z: -520,
+ targetX: 160,
+ targetY: 300,
+ targetZ: 0,
+ }),
+ ...layers,
+ ]);
+ return c;
+}
+
+async function pixels(c: Composition, frame = 0): Promise<Uint8ClampedArray> {
+ return (await decodePngPixels(await c.renderFrame({ frame }))).pixels;
+}
+
+function count(px: Uint8ClampedArray, test: (rgb: Rgb) => boolean): number {
+ let n = 0;
+ for (let i = 0; i < px.length; i += 4)
+ if (test([px[i]!, px[i + 1]!, px[i + 2]!])) n++;
+ return n;
+}
+
+function mean(
+ px: Uint8ClampedArray,
+ x0: number,
+ y0: number,
+ x1: number,
+ y1: number,
+): Rgb {
+ const sum: Rgb = [0, 0, 0];
+ let n = 0;
+ for (let y = y0; y < y1; y++)
+ for (let x = x0; x < x1; x++) {
+ const i = (y * W + x) * 4;
+ for (let c = 0; c < 3; c++) sum[c] += px[i + c]!;
+ n++;
+ }
+ return sum.map((v) => v / n) as Rgb;
+}
+
+const red = ([r, g, b]: Rgb) => r > 150 && g < 80 && b < 80;
+const green = ([r, g, b]: Rgb) => g > 120 && r < 90 && b < 90;
+const blue = ([r, g, b]: Rgb) => b > 150 && r < 80 && g < 80;
+
+/** Rolling ground under the camera; its valley floor at y = 320. */
+const ground = (extra: Record<string, unknown> = {}) =>
+ Layer3D.terrain({
+ id: "ground",
+ x: 160,
+ y: 320,
+ z: 0,
+ width: 900,
+ depth: 900,
+ height: 80,
+ segments: 64,
+ noise: { seed: 4, octaves: 4 },
+ material: "unlit",
+ ...extra,
+ });
+
+const flat = () => ground({ heightFn: () => 0.5, color: "#20c040" });
+
+describe("Layer3D.terrain / sdf / scatter", () => {
+ it("colors terrain by altitude", async () => {
+ const c = comp([
+ ground({
+ colorByElevation: { lowColor: "#00c000", highColor: "#ffffff" },
+ }),
+ ]);
+ const px = await pixels(c);
+ expect(count(px, green)).toBeGreaterThan(500);
+ expect(
+ count(px, ([r, g, b]) => r > 220 && g > 220 && b > 220),
+ ).toBeGreaterThan(200);
+ expect(await pixels(c)).toEqual(px);
+ });
+
+ it("wraps any layer onto a mesh as its texture", async () => {
+ const half = (x: number, background: string) =>
+ Layer.box({
+ position: "absolute",
+ x,
+ y: 0,
+ width: W / 2,
+ height: H,
+ background,
+ });
+ const halves = Layer.box({
+ width: W,
+ height: H,
+ children: [half(0, "#ff0000"), half(W / 2, "#0000ff")],
+ });
+ const px = await pixels(
+ comp([ground({ heightFn: () => 0, texture: halves })]),
+ );
+ // u runs left to right across the terrain's x.
+ expect(red(mean(px, 40, 150, 120, 175))).toBe(true);
+ expect(blue(mean(px, 200, 150, 280, 175))).toBe(true);
+ // The texture layer is not drawn flat on its own: the sky stays black.
+ expect(mean(px, 0, 0, W, 10).every((v) => v < 5)).toBe(true);
+ });
+
+ it("meshes SDF booleans with their primitives' colors", async () => {
+ const px = await pixels(
+ comp([
+ Layer3D.sdf({
+ x: 160,
+ y: 230,
+ z: 0,
+ rotateY: 35,
+ rotateX: 30,
+ bounds: [220, 220, 220],
+ resolution: 48,
+ material: "unlit",
+ primitives: [
+ {
+ shape: "box",
+ size: [160, 160, 160],
+ radius: 12,
+ color: "#ff2020",
+ },
+ { shape: "sphere", radius: 105, op: "intersect", smoothness: 6 },
+ {
+ shape: "cylinder",
+ radius: 45,
+ height: 400,
+ op: "subtract",
+ color: "#2020ff",
+ },
+ ],
+ }),
+ ]),
+ );
+ expect(count(px, red)).toBeGreaterThan(800);
+ expect(count(px, blue)).toBeGreaterThan(100);
+ });
+
+ it("scatters deterministic instances that stand on the surface", async () => {
+ const terrain = flat();
+ const scatter = (seed: number) =>
+ Layer3D.scatter({
+ id: "rocks",
+ surface: terrain,
+ mesh: Layer3D.sphere({
+ radius: 10,
+ color: "#ff0000",
+ material: "unlit",
+ }),
+ material: "unlit",
+ count: 40,
+ minDistance: 40,
+ seed,
+ exclude: [{ x: 160, z: 0, radius: 120 }],
+ });
+ const a = scatter(9);
+ const points = a.points();
+ expect(points.length).toBe(40);
+ for (const p of points) {
+ expect(p.position[1]).toBeCloseTo(
+ terrain.heightAt(p.position[0], p.position[2]),
+ 3,
+ );
+ expect(
+ Math.hypot(p.position[0] - 160, p.position[2]),
+ ).toBeGreaterThanOrEqual(120);
+ }
+ const px = await pixels(comp([terrain, a]));
+ expect(count(px, red)).toBeGreaterThan(150);
+ expect(await pixels(comp([terrain, scatter(9)]))).toEqual(px);
+ expect(await pixels(comp([terrain, scatter(10)]))).not.toEqual(px);
+ // The clearing in front of the camera stays grass.
+ expect(green(mean(px, 150, 120, 170, 135))).toBe(true);
+ });
+
+ it("sways scattered meshes in the wind, frame by frame", async () => {
+ const scene = (wind: boolean) =>
+ comp(
+ [
+ flat(),
+ Layer3D.scatter({
+ surface: flat(),
+ mesh: Layer3D.capsule({ radius: 6, length: 60, color: "#ff0000" }),
+ material: "unlit",
+ count: 30,
+ seed: 2,
+ alignToNormal: 0,
+ ...(wind && { wind: { amplitudeDeg: 25, frequency: 1 } }),
+ }),
+ ],
+ 30,
+ );
+ const still = scene(false);
+ expect(await pixels(still, 0)).toEqual(await pixels(still, 7));
+ const windy = scene(true);
+ expect(await pixels(windy, 0)).not.toEqual(await pixels(windy, 7));
+ });
+
+ it("lands physics bodies on the terrain's own triangles", async () => {
+ const world = await Physics.world3d({ fps: 30, frames: 90 });
+ const terrain = flat();
+ world.body(terrain, { type: "fixed" });
+ const ball = world.body(
+ Layer3D.sphere({ radius: 20, x: 200, y: 0, z: 40 }),
+ { restitution: 0, angularDamping: 4, linearDamping: 2 },
+ );
+ world.simulate();
+ const y = ball.y.get({ frame: 89, fps: 30 } as never);
+ expect(y).toBeCloseTo(terrain.heightAt(200, 40) - 20, 0);
+ });
+
+ it("validates specs and lints filters that reject the whole surface", () => {
+ const terrain = flat();
+ const ok = comp([
+ terrain,
+ Layer3D.scatter({
+ surface: terrain,
+ mesh: Layer3D.sphere({ radius: 5 }),
+ count: 10,
+ wind: { amplitudeDeg: 5 },
+ }),
+ Layer3D.sdf({
+ bounds: [100, 100, 100],
+ primitives: [{ shape: "gyroid", scale: 40, thickness: 6 }],
+ }),
+ ]);
+ expect(CompositorProgramSchema.safeParse(ok.toSpec()).success).toBe(true);
+ expect(ok.lint().some((d) => d.code === "scatter_filters_reject_all")).toBe(
+ false,
+ );
+ const bad = comp([ground({ segments: 9999 })]);
+ expect(CompositorProgramSchema.safeParse(bad.toSpec()).success).toBe(false);
+ const empty = comp([
+ terrain,
+ Layer3D.scatter({
+ surface: terrain,
+ mesh: Layer3D.sphere({ radius: 5 }),
+ count: 10,
+ slopeRange: [40, 60],
+ }),
+ ]);
+ expect(
+ empty.lint().some((d) => d.code === "scatter_filters_reject_all"),
+ ).toBe(true);
+ });
+});
diff --git a/packages/framefields/src/procedural3d.ts b/packages/framefields/src/procedural3d.ts
new file mode 100644
index 00000000..31470a87
--- /dev/null
+++ b/packages/framefields/src/procedural3d.ts
@@ -0,0 +1,298 @@
+/**
+ * Procedural 3D builders behind `Layer3D.terrain`, `Layer3D.sdf` and
+ * `Layer3D.scatter`. Each returns an ordinary model node, so it takes
+ * materials, lights, physics, layer textures (image, video, any layout) and
+ * animation like every other Layer3D mesh.
+ */
+import type {
+ LayoutNode,
+ Model3DNode,
+ Scatter3DSpec,
+ WindConfig,
+} from "@framefields/compositions/program";
+import {
+ createTerrainSampler,
+ getPrimitiveGeometry,
+ type Mat4,
+ Matrix4Math,
+ type Model3DData,
+ type PrimitiveGeometryOptions,
+ type ScatterOptions,
+ type ScatterPoint,
+ type SdfGeometryOptions,
+ scatterOnSurface,
+ type TerrainGeometryOptions,
+ type TerrainSampler,
+ terrainSegments,
+} from "@framefields/webgpu-renderers";
+import type { AnimatableNode } from "./index.js";
+import {
+ type Instances3DOptions,
+ Layer3D,
+ type Primitive3DNodeOptions,
+ primitiveNode,
+} from "./shapes3d.js";
+
+type GeometryKeys = "type" | "color" | "material";
+
+export type Terrain3DOptions = Primitive3DNodeOptions &
+ Omit<TerrainGeometryOptions, GeometryKeys> & {
+ /**
+ * Custom heights: `(u, v) => 0..1` over the grid (u across x, v across
+ * z, both 0..1 from the near-left corner). Sampled into `heights`.
+ */
+ heightFn?: (u: number, v: number) => number;
+ };
+
+export type Sdf3DOptions = Primitive3DNodeOptions &
+ Omit<SdfGeometryOptions, GeometryKeys>;
+
+export type Scatter3DOptions = Omit<
+ Instances3DOptions,
+ "count" | "instanceAt" | "transforms" | "bodies"
+> &
+ ScatterOptions & {
+ /** The surface: `Layer3D.terrain`, `Layer3D.sdf`, any primitive or `Layer.model`. */
+ surface: AnimatableNode<Model3DNode>;
+ /** Push each instance this far into the surface, px. Default 0. */
+ sink?: number;
+ /** Procedural wind sway; the meshes bend from their base. */
+ wind?: WindConfig;
+ };
+
+/** Where a model node's mesh sits: its static transform. */
+function staticMatrix(n: Partial<Model3DNode>): Mat4 {
+ const rad = Math.PI / 180;
+ const s = n.scale ?? 1;
+ let m = Matrix4Math.translate(n.x ?? 0, n.y ?? 0, n.z ?? 0);
+ if (n.rotateZ)
+ m = Matrix4Math.multiply(m, Matrix4Math.rotateZ(n.rotateZ * rad));
+ if (n.rotateY)
+ m = Matrix4Math.multiply(m, Matrix4Math.rotateY(n.rotateY * rad));
+ if (n.rotateX)
+ m = Matrix4Math.multiply(m, Matrix4Math.rotateX(n.rotateX * rad));
+ return Matrix4Math.multiply(
+ m,
+ Matrix4Math.scale(
+ s * (n.scaleX ?? 1),
+ s * (n.scaleY ?? 1),
+ s * (n.scaleZ ?? 1),
+ ),
+ );
+}
+
+/** A terrain node's helpers for standing other layers on it. */
+export interface TerrainSurface {
+ /**
+ * Surface y at (x, z) in the terrain's parent space (where its own x, y,
+ * z live), so a camera, light, video screen or model can stand on it.
+ * Follows the terrain's position and scale; rotation is not applied.
+ */
+ heightAt(x: number, z: number): number;
+ /** Unit surface normal at (x, z), parent space (unrotated). */
+ normalAt(x: number, z: number): [number, number, number];
+ /** Normalized altitude 0..1 at (x, z). */
+ elevationAt(x: number, z: number): number;
+}
+
+export type Terrain3DNode = AnimatableNode<Model3DNode> & TerrainSurface;
+
+function defineHidden(target: object, values: Record<string, unknown>): void {
+ for (const [key, value] of Object.entries(values)) {
+ Object.defineProperty(target, key, {
+ value,
+ enumerable: false,
+ configurable: true,
+ });
+ }
+}
+
+function surfaceHelpers(
+ sampler: TerrainSampler,
+ o: Terrain3DOptions,
+): TerrainSurface {
+ const s = o.scale ?? 1;
+ const sx = s * (o.scaleX ?? 1);
+ const sy = s * (o.scaleY ?? 1);
+ const sz = s * (o.scaleZ ?? 1);
+ const local = (x: number, z: number): [number, number] => [
+ (x - (o.x ?? 0)) / sx,
+ (z - (o.z ?? 0)) / sz,
+ ];
+ return {
+ heightAt: (x, z) => (o.y ?? 0) + sampler.heightAt(...local(x, z)) * sy,
+ elevationAt: (x, z) => sampler.elevationAt(...local(x, z)),
+ normalAt(x, z) {
+ const n = sampler.normalAt(...local(x, z));
+ // Normals take the inverse scale.
+ const m = [n[0] / sx, n[1] / sy, n[2] / sz];
+ const len = Math.hypot(m[0]!, m[1]!, m[2]!) || 1;
+ return [m[0]! / len, m[1]! / len, m[2]! / len];
+ },
+ };
+}
+
+/** Procedural terrain (see `Layer3D.terrain`). */
+export function terrain3d(options: Terrain3DOptions): Terrain3DNode {
+ const { heightFn, ...o } = options;
+ const geometry: TerrainGeometryOptions = {
+ type: "terrain",
+ width: o.width,
+ depth: o.depth,
+ height: o.height,
+ segments: o.segments,
+ noise: o.noise,
+ heights: o.heights,
+ overhangWarp: o.overhangWarp,
+ colorByElevation: o.colorByElevation,
+ flatShaded: o.flatShaded,
+ base: o.base,
+ };
+ if (heightFn) {
+ const [cols, rows] = terrainSegments(geometry);
+ const heights: number[] = [];
+ for (let j = 0; j <= rows; j++)
+ for (let i = 0; i <= cols; i++)
+ heights.push(heightFn(i / cols, j / rows));
+ geometry.heights = heights;
+ }
+ const node = primitiveNode("terrain", geometry, o) as Terrain3DNode;
+ defineHidden(
+ node,
+ surfaceHelpers(
+ createTerrainSampler(geometry),
+ options,
+ ) as unknown as Record<string, unknown>,
+ );
+ return node;
+}
+
+/** A signed-distance sculpture (see `Layer3D.sdf`). */
+export function sdf3d(options: Sdf3DOptions): AnimatableNode<Model3DNode> {
+ return primitiveNode(
+ "sdf",
+ {
+ type: "sdf",
+ primitives: options.primitives,
+ bounds: options.bounds,
+ resolution: options.resolution,
+ flatShaded: options.flatShaded,
+ },
+ options,
+ );
+}
+
+const SURFACE_KEYS = [
+ "x",
+ "y",
+ "z",
+ "rotateX",
+ "rotateY",
+ "rotateZ",
+ "scale",
+ "scaleX",
+ "scaleY",
+ "scaleZ",
+ "primitive3d",
+ "src",
+ "modelFormat",
+ "center",
+ "normalizeSize",
+] as const;
+
+/** What the renderer rebuilds and places the surface from. */
+function surfaceSpec(surface: Partial<Model3DNode>): Record<string, unknown> {
+ const out: Record<string, unknown> = {};
+ for (const k of SURFACE_KEYS) {
+ if (surface[k] !== undefined) out[k] = surface[k];
+ }
+ // In-memory meshes with nothing to rebuild them from travel as they are.
+ if (!surface.primitive3d && !surface.src && surface.modelData)
+ out.modelData = surface.modelData;
+ return out;
+}
+
+/** The surface's mesh when it is already in memory (primitives, terrain, SDF). */
+function surfaceModel(surface: Partial<Model3DNode>): Model3DData | undefined {
+ const data = surface.modelData as Model3DData | undefined;
+ if (data && "meshes" in data) return data;
+ if (surface.primitive3d)
+ return getPrimitiveGeometry(
+ surface.primitive3d as PrimitiveGeometryOptions,
+ );
+ return undefined;
+}
+
+const SCATTER_KEYS = [
+ "count",
+ "minDistance",
+ "seed",
+ "slopeRange",
+ "elevationRange",
+ "alignToNormal",
+ "scaleRange",
+ "randomYaw",
+ "exclude",
+ "sink",
+] as const;
+
+export type Scatter3DNode = AnimatableNode<Model3DNode> & {
+ /**
+ * The scattered points (surface position, normal, rotation, scale) in the
+ * surface's parent space, for placing other layers among the instances.
+ * Empty for a surface loaded from a file (its mesh is read at render).
+ */
+ points(): ScatterPoint[];
+};
+
+/** Instances scattered on a surface (see `Layer3D.scatter`). */
+export function scatter3d(options: Scatter3DOptions): Scatter3DNode {
+ const { surface, wind, ...rest } = options;
+ const spec: Record<string, unknown> = {
+ surface: surfaceSpec(surface as Partial<Model3DNode>),
+ };
+ const instanceOptions: Record<string, unknown> = { ...rest };
+ for (const k of SCATTER_KEYS) {
+ if (rest[k] !== undefined) spec[k] = rest[k];
+ delete instanceOptions[k];
+ }
+ const node = Layer3D.instances(
+ instanceOptions as unknown as Instances3DOptions,
+ ) as Scatter3DNode;
+ const target = node as unknown as Model3DNode;
+ target.scatter3d = spec as unknown as Scatter3DSpec;
+ if (wind) target.wind = wind;
+ defineHidden(node, {
+ points: (): ScatterPoint[] => {
+ const model = surfaceModel(surface as Partial<Model3DNode>);
+ if (!model) return [];
+ return scatterOnSurface(
+ { model, matrix: staticMatrix(surface as Partial<Model3DNode>) },
+ spec as unknown as ScatterOptions,
+ ).points;
+ },
+ });
+ return node;
+}
+
+/**
+ * Share of a scatter's surface its slope / altitude filters keep (for
+ * `lint`), or undefined when the surface is a file not loaded yet.
+ */
+export function scatterCoverage(spec: Scatter3DSpec): number | undefined {
+ const surface = spec.surface as Partial<Model3DNode>;
+ const model = surfaceModel(surface);
+ if (!model) return undefined;
+ return scatterOnSurface(
+ { model, matrix: staticMatrix(surface) },
+ { ...spec, count: 0 },
+ ).acceptedArea;
+}
+
+/** `texture` on a Layer3D mesh: an image path, or any layer drawn each frame. */
+export function textureLayerOf(
+ texture: string | LayoutNode,
+ image: (src: string) => LayoutNode,
+): { node: LayoutNode } {
+ return { node: typeof texture === "string" ? image(texture) : texture };
+}
diff --git a/packages/framefields/src/shapes3d.ts b/packages/framefields/src/shapes3d.ts
index 43a71718..1146df7e 100644
--- a/packages/framefields/src/shapes3d.ts
+++ b/packages/framefields/src/shapes3d.ts
@@ -1,11 +1,13 @@
import type {
BoxNode,
+ CrowdConfig,
IridescenceConfig,
LayoutNode,
MaterialType,
Model3DNode,
ToonShading,
TriplanarConfig,
+ WindConfig,
} from "@framefields/compositions/program";
import {
type CapsuleGeometryOptions,
@@ -37,6 +39,17 @@ import {
type SignalOrNumber,
} from "./index.js";
import type { PhysicsBody3D } from "./physics/world3d.js";
+import {
+ type Scatter3DNode,
+ type Scatter3DOptions,
+ type Sdf3DOptions,
+ scatter3d,
+ sdf3d,
+ type Terrain3DNode,
+ type Terrain3DOptions,
+ terrain3d,
+ textureLayerOf,
+} from "./procedural3d.js";
import { createSwarm3D, type Swarm3DOptions } from "./swarm3d.js";
/** Physically based surface options every 3D builder accepts. */
@@ -498,6 +511,16 @@ export interface Primitive3DNodeOptions extends Surface3DOptions {
opacity?: number;
startFrame?: number;
durationFrames?: number;
+ /**
+ * Base-color texture wrapped by the mesh's UVs: an image path, or any
+ * layer (`Layer.video`, `Layer.image`, a layout tree with text…) drawn
+ * fresh every frame, so video plays on the surface. Without its own
+ * x / y / width / height the layer fills the frame, which maps to the
+ * whole UV square. Combine with `triplanar` to project it in world space.
+ */
+ texture?: string | LayoutNode;
+ /** Procedural wind sway: vertices bend downwind by height above the mesh base. */
+ wind?: WindConfig;
}
export type RoundedBox3DOptions = Primitive3DNodeOptions &
@@ -554,6 +577,12 @@ export interface InstanceTransform3D {
color?: string | [number, number, number, number];
emissive?: string | [number, number, number];
emissiveIntensity?: number;
+ /** Crowds: index into `crowd.clips` (default: instance index % clip count). */
+ clip?: number;
+ /** Crowds: seconds into the clip at frame 0 (default: seeded random, or 0 with `phase: "sync"`). */
+ animOffset?: number;
+ /** Crowds: playback speed (default: seeded within `crowd.speedRange`). */
+ animSpeed?: number;
}
export interface Instances3DOptions extends Surface3DOptions {
@@ -579,6 +608,15 @@ export interface Instances3DOptions extends Surface3DOptions {
intensity?: number;
};
emissiveByImpact?: { color: string; intensity?: number };
+ /** Procedural wind sway: each instance bends downwind from its base. */
+ wind?: WindConfig;
+ /**
+ * Skinned crowd: the mesh is an animated model (`Layer.glb`, `Layer.gltf`)
+ * and every instance plays one of `clips` at its own phase and speed,
+ * skinned on the GPU from a palette baked once: thousands of characters,
+ * one draw call per mesh.
+ */
+ crowd?: CrowdConfig;
x?: number;
y?: number;
z?: number;
@@ -622,7 +660,8 @@ function normalizePathSpec(input: Path3D | TubePathSpec): TubePathSpec {
return input;
}
-function primitiveNode(
+/** A Layer3D mesh node built from a procedural geometry spec. */
+export function primitiveNode(
prefix: string,
geometry: PrimitiveGeometryOptions,
o: Primitive3DNodeOptions,
@@ -668,6 +707,10 @@ function primitiveNode(
wireframe: o.wireframe,
startFrame: o.startFrame,
durationFrames: o.durationFrames,
+ ...(o.wind && { wind: o.wind }),
+ ...(o.texture !== undefined && {
+ textureLayer: textureLayerOf(o.texture, (src) => Layer.image(src)),
+ }),
});
}
@@ -1615,6 +1658,39 @@ export const Layer3D = {
* Renders hundreds or thousands of 3D meshes in 1 draw call per mesh variant,
* driven by Physics.world3d rigid bodies, procedural callbacks, or transform arrays.
*/
+ /**
+ * Procedural terrain: seeded fractal noise ("fbm" hills, "ridged"
+ * mountains, "terrace" plateaus) or your own heights, optional cliff
+ * overhangs, biome colors by altitude and slope, closed underneath into a
+ * solid block. The node's y is the valley floor; peaks rise `height` px
+ * above it. The node carries `heightAt(x, z)` / `normalAt(x, z)` for
+ * standing cameras, lights, video screens and models on the ground.
+ */
+ terrain(options: Terrain3DOptions): Terrain3DNode {
+ return terrain3d(options);
+ },
+
+ /**
+ * A sculpture of signed-distance primitives (sphere, box, cylinder,
+ * capsule, torus, gyroid) joined in order by union / subtract /
+ * intersect, hard or smooth (`smoothness` px), meshed watertight inside
+ * `bounds`. Colored primitives blend where their joins blend.
+ */
+ sdf(options: Sdf3DOptions): AnimatableNode<Model3DNode> {
+ return sdf3d(options);
+ },
+
+ /**
+ * Scatters instances of one or more meshes over a surface (terrain, SDF,
+ * any primitive or `Layer.model` file): area-weighted, deterministic per
+ * `seed`, filtered by slope / altitude / exclusion zones, spaced by
+ * `minDistance`, each standing on its base. One draw call per mesh
+ * variant. `wind` sways them; `.points()` returns where they stand.
+ */
+ scatter(options: Scatter3DOptions): Scatter3DNode {
+ return scatter3d(options);
+ },
+
instances(options: Instances3DOptions): AnimatableNode<Model3DNode> {
const primaryMesh = Array.isArray(options.mesh)
? options.mesh[0]
@@ -1642,6 +1718,8 @@ export const Layer3D = {
startFrame: options.startFrame,
durationFrames: options.durationFrames,
instances3d: { ...options },
+ ...(options.wind && { wind: options.wind }),
+ ...(options.crowd && { crowd: options.crowd }),
...materialProps(options),
};
if (primaryMesh) {
diff --git a/packages/framefields/src/test-utils/glb.ts b/packages/framefields/src/test-utils/glb.ts
new file mode 100644
index 00000000..7cfab1b5
--- /dev/null
+++ b/packages/framefields/src/test-utils/glb.ts
@@ -0,0 +1,82 @@
+/** Builds GLB files in tests: typed arrays become accessors in one binary chunk. */
+
+/** Accumulates typed arrays into one binary buffer and the accessors that view them. */
+export class GlbBuilder {
+ private chunks: Buffer[] = [];
+ private length = 0;
+ readonly bufferViews: object[] = [];
+ readonly accessors: object[] = [];
+
+ private view(data: Buffer): number {
+ const pad = (4 - (this.length % 4)) % 4;
+ if (pad) {
+ this.chunks.push(Buffer.alloc(pad));
+ this.length += pad;
+ }
+ this.bufferViews.push({
+ buffer: 0,
+ byteOffset: this.length,
+ byteLength: data.length,
+ });
+ this.chunks.push(data);
+ this.length += data.length;
+ return this.bufferViews.length - 1;
+ }
+
+ accessor(
+ data: Float32Array | Uint16Array,
+ type: "SCALAR" | "VEC2" | "VEC3" | "VEC4" | "MAT4",
+ ): number {
+ const bufferView = this.view(
+ Buffer.from(data.buffer, data.byteOffset, data.byteLength),
+ );
+ const size = { SCALAR: 1, VEC2: 2, VEC3: 3, VEC4: 4, MAT4: 16 }[type];
+ this.accessors.push({
+ bufferView,
+ componentType: data instanceof Float32Array ? 5126 : 5123,
+ count: data.length / size,
+ type,
+ });
+ return this.accessors.length - 1;
+ }
+
+ image(png: Buffer): number {
+ return this.view(png);
+ }
+
+ build(json: Record<string, unknown>): Buffer {
+ const bin = Buffer.concat(this.chunks);
+ const doc = Buffer.from(
+ JSON.stringify({
+ asset: { version: "2.0" },
+ ...json,
+ buffers: [{ byteLength: bin.length }],
+ bufferViews: this.bufferViews,
+ accessors: this.accessors,
+ }),
+ );
+ const docPadded = Buffer.concat([
+ doc,
+ Buffer.alloc((4 - (doc.length % 4)) % 4, 0x20),
+ ]);
+ const binPadded = Buffer.concat([
+ bin,
+ Buffer.alloc((4 - (bin.length % 4)) % 4),
+ ]);
+ const header = Buffer.alloc(12);
+ header.writeUInt32LE(0x46546c67, 0);
+ header.writeUInt32LE(2, 4);
+ header.writeUInt32LE(12 + 8 + docPadded.length + 8 + binPadded.length, 8);
+ const chunk = (data: Buffer, type: number) => {
+ const h = Buffer.alloc(8);
+ h.writeUInt32LE(data.length, 0);
+ h.writeUInt32LE(type, 4);
+ return Buffer.concat([h, data]);
+ };
+ return Buffer.concat([
+ header,
+ chunk(docPadded, 0x4e4f534a),
+ chunk(binPadded, 0x004e4942),
+ ]);
+ }
+}
diff --git a/packages/webgpu-renderers/src/loaders/crowd.ts b/packages/webgpu-renderers/src/loaders/crowd.ts
new file mode 100644
index 00000000..ccf7f3f5
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/crowd.ts
@@ -0,0 +1,100 @@
+/**
+ * Vertex-animation crowds: each clip a model can play, sampled at a fixed
+ * rate into a palette of model-space joint matrices per mesh, so instances
+ * can skin themselves on the GPU at their own clip, phase and speed.
+ */
+import { Matrix4Math } from "../math3d/index.js";
+import { jointMatrices, poseNodes } from "./pose.js";
+import type { AnimationClip3D, Mesh3DData, Model3DData } from "./types.js";
+
+export interface CrowdClipRange {
+ name: string;
+ /** First baked frame of the clip in every palette. */
+ base: number;
+ /** Frames in one loop of the clip; `count + 1` are baked (the last at its end), so playback interpolates up to the loop point and wraps cleanly. */
+ count: number;
+ duration: number;
+}
+
+export interface CrowdMeshPalette {
+ /** `frames × jointCount` column-major 4×4s. */
+ palette: Float32Array;
+ jointCount: number;
+}
+
+export interface CrowdBake {
+ /** Target samples per second; each clip bakes round(duration · fps) per loop. */
+ fps: number;
+ clips: CrowdClipRange[];
+ meshes: Map<Mesh3DData, CrowdMeshPalette>;
+}
+
+/** The model's clips by name or index; unknown ones are skipped. */
+export function resolveClips(
+ model: Model3DData,
+ clips: Array<string | number>,
+): AnimationClip3D[] {
+ const out: AnimationClip3D[] = [];
+ for (const c of clips) {
+ const clip =
+ typeof c === "number"
+ ? model.animations[c]
+ : model.animations.find((a) => a.name === c);
+ if (clip) out.push(clip);
+ }
+ return out;
+}
+
+/** A mesh's matrices at one posed frame: its skin's joints, or its node as joint 0. */
+function meshFrame(
+ model: Model3DData,
+ mesh: Mesh3DData,
+ world: ReturnType<typeof poseNodes>,
+): Float32Array {
+ if (mesh.skin !== undefined) return jointMatrices(model, mesh.skin, world);
+ const m =
+ mesh.node !== undefined && world[mesh.node]
+ ? world[mesh.node]!
+ : Matrix4Math.identity();
+ return new Float32Array(m);
+}
+
+/** Bakes every clip at `fps` for every mesh of the model. */
+export function bakeCrowdPalette(
+ model: Model3DData,
+ clips: AnimationClip3D[],
+ fps = 30,
+): CrowdBake {
+ const ranges: CrowdClipRange[] = [];
+ const times: number[] = [];
+ const clipOf: AnimationClip3D[] = [];
+ for (const clip of clips) {
+ const count = Math.max(1, Math.round(clip.duration * fps));
+ ranges.push({
+ name: clip.name,
+ base: times.length,
+ count,
+ duration: clip.duration,
+ });
+ for (let f = 0; f <= count; f++) {
+ times.push((f / count) * clip.duration);
+ clipOf.push(clip);
+ }
+ }
+ const meshes = new Map<Mesh3DData, CrowdMeshPalette>();
+ const frames = times.map((t, i) => poseNodes(model, clipOf[i], t));
+ for (const mesh of model.meshes) {
+ const first = meshFrame(
+ model,
+ mesh,
+ frames[0] ?? poseNodes(model, undefined, 0),
+ );
+ const jointCount = first.length / 16;
+ const palette = new Float32Array(Math.max(1, frames.length) * first.length);
+ frames.forEach((world, f) => {
+ palette.set(meshFrame(model, mesh, world), f * first.length);
+ });
+ meshes.set(mesh, { palette, jointCount });
+ }
+ return { fps, clips: ranges, meshes };
+}
diff --git a/packages/webgpu-renderers/src/loaders/index.ts b/packages/webgpu-renderers/src/loaders/index.ts
index bc64b787..f0bf3048 100644
--- a/packages/webgpu-renderers/src/loaders/index.ts
+++ b/packages/webgpu-renderers/src/loaders/index.ts
@@ -1,4 +1,5 @@
export * from "./3ds-loader.js";
+export * from "./crowd.js";
export * from "./exr-loader.js";
export * from "./fbx-loader.js";
export * from "./gltf-loader.js";
@@ -7,6 +8,7 @@ export * from "./obj-loader.js";
export * from "./off-loader.js";
export * from "./ply-loader.js";
export * from "./pose.js";
+export * from "./rig.js";
export * from "./stl-loader.js";
export * from "./types.js";
export * from "./vox-loader.js";
diff --git a/packages/webgpu-renderers/src/loaders/pose.ts b/packages/webgpu-renderers/src/loaders/pose.ts
index 97cf3087..f9b1ffcb 100644
--- a/packages/webgpu-renderers/src/loaders/pose.ts
+++ b/packages/webgpu-renderers/src/loaders/pose.ts
@@ -154,12 +154,19 @@ export function sampleMorphWeights(
: (mesh.morphWeights ?? Array(count).fill(0));
}
-/** Each node's local matrix with the clip's tracks applied at `time` (seconds). */
-function localMatrices(
+/** Every node's local translation, rotation (unit quaternion) and scale. */
+export interface LocalPose {
+ t: Vec3[];
+ r: Quat[];
+ s: Vec3[];
+}
+
+/** The rest pose with the clip's tracks applied at `time` (seconds). */
+export function sampleLocalPose(
nodes: ModelNode3D[],
clip: AnimationClip3D | undefined,
time: number,
-): Mat4[] {
+): LocalPose {
const t = nodes.map((n) => [...n.translation] as Vec3);
const r = nodes.map((n) => [...n.rotation] as Quat);
const s = nodes.map((n) => [...n.scale] as Vec3);
@@ -174,20 +181,21 @@ function localMatrices(
if (track.scales)
s[i] = sampleChannel(track, track.scales, 3, time) as Vec3;
}
- return nodes.map((n, i) => n.matrix ?? trsMatrix(t[i]!, r[i]!, s[i]!));
+ return { t, r, s };
}
-/**
- * Model-space matrix of every node, posed by `clip` at `time` seconds (the
- * rest pose without a clip). Parents may follow their children in the list.
- */
-export function poseNodes(
- model: Model3DData,
- clip: AnimationClip3D | undefined,
- time: number,
+/** Each node's local matrix (a fixed glTF `matrix` wins over TRS). */
+export function localPoseMatrices(
+ nodes: ModelNode3D[],
+ pose: LocalPose,
): Mat4[] {
- const nodes = model.nodes ?? [];
- const local = localMatrices(nodes, clip, time);
+ return nodes.map(
+ (n, i) => n.matrix ?? trsMatrix(pose.t[i]!, pose.r[i]!, pose.s[i]!),
+ );
+}
+
+/** Model-space matrices from local ones. Parents may follow their children in the list. */
+export function composeWorld(nodes: ModelNode3D[], local: Mat4[]): Mat4[] {
const world: (Mat4 | undefined)[] = new Array(nodes.length);
const resolve = (i: number): Mat4 => {
const done = world[i];
@@ -201,6 +209,22 @@ export function poseNodes(
return nodes.map((_, i) => resolve(i));
}
+/**
+ * Model-space matrix of every node, posed by `clip` at `time` seconds (the
+ * rest pose without a clip). Parents may follow their children in the list.
+ */
+export function poseNodes(
+ model: Model3DData,
+ clip: AnimationClip3D | undefined,
+ time: number,
+): Mat4[] {
+ const nodes = model.nodes ?? [];
+ return composeWorld(
+ nodes,
+ localPoseMatrices(nodes, sampleLocalPose(nodes, clip, time)),
+ );
+}
+
/** The joint matrices of one skin for a posed scene graph. */
export function jointMatrices(
model: Model3DData,
diff --git a/packages/webgpu-renderers/src/loaders/rig.test.ts b/packages/webgpu-renderers/src/loaders/rig.test.ts
new file mode 100644
index 00000000..6cf40aae
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/rig.test.ts
@@ -0,0 +1,211 @@
+import { describe, expect, it } from "vitest";
+import type { Vec3 } from "../math3d/index.js";
+import { sampleLocalPose } from "./pose.js";
+import {
+ applyConstraints,
+ blendClipLayers,
+ RigPose,
+ SpringBones,
+ solveLookAt,
+ solveTwoBoneIk,
+} from "./rig.js";
+import type { AnimationClip3D, Model3DData, ModelNode3D } from "./types.js";
+
+const node = (name: string, parent: number, t: Vec3): ModelNode3D => ({
+ name,
+ parent,
+ translation: t,
+ rotation: [0, 0, 0, 1],
+ scale: [1, 1, 1],
+});
+
+/** A straight leg down +y: hip at the origin, knee 100 below, foot 100 below that; a 3-joint tail. */
+const NODES: ModelNode3D[] = [
+ node("hip", -1, [0, 0, 0]),
+ node("knee", 0, [0, 100, 0]),
+ node("foot", 1, [0, 100, 0]),
+ node("tail0", -1, [50, 0, 0]),
+ node("tail1", 3, [40, 0, 0]),
+ node("tail2", 4, [40, 0, 0]),
+];
+
+/** Turns the hip a quarter turn about z over one second. */
+const SWING: AnimationClip3D = {
+ name: "swing",
+ duration: 1,
+ fps: 30,
+ tracks: [
+ {
+ nodeName: "hip",
+ node: 0,
+ times: new Float32Array([0, 1]),
+ rotations: new Float32Array([
+ 0,
+ 0,
+ 0,
+ 1,
+ 0,
+ 0,
+ Math.SQRT1_2,
+ Math.SQRT1_2,
+ ]),
+ },
+ {
+ nodeName: "tail0",
+ node: 3,
+ times: new Float32Array([0, 1]),
+ translations: new Float32Array([50, 0, 0, 50, -200, 0]),
+ },
+ ],
+} as unknown as AnimationClip3D;
+
+const MODEL = {
+ meshes: [],
+ materials: {},
+ animations: [SWING],
+ nodes: NODES,
+} as unknown as Model3DData;
+
+const rest = () => new RigPose(NODES, sampleLocalPose(NODES, undefined, 0));
+const dist = (a: Vec3, b: Vec3) =>
+ Math.hypot(a[0] - b[0], a[1] - b[1], a[2] - b[2]);
+
+describe("rig", () => {
+ it("blends clip layers by weight and mask", () => {
+ const full = blendClipLayers(MODEL, [{ clip: SWING, time: 1, weight: 1 }]);
+ // A quarter turn about z sends the foot from +y to -x.
+ const foot = full.position(2);
+ expect(foot[0]).toBeCloseTo(-200, 3);
+ expect(foot[1]).toBeCloseTo(0, 3);
+ const half = blendClipLayers(MODEL, [
+ { clip: SWING, time: 1, weight: 0.5 },
+ ]);
+ // Half the rotation: 45°.
+ expect(half.position(2)[0]).toBeCloseTo(-200 * Math.SQRT1_2, 3);
+ const masked = blendClipLayers(MODEL, [
+ { clip: SWING, time: 1, weight: 1, mask: ["tail0"] },
+ ]);
+ expect(masked.position(2)).toEqual([0, 200, 0]);
+ expect(masked.position(3)[1]).toBeCloseTo(-200, 3);
+ });
+
+ it("reaches an IK target and bends toward the pole", () => {
+ for (const target of [
+ [60, 120, 0],
+ [-80, 90, 40],
+ [0, 150, -60],
+ ] as Vec3[]) {
+ for (const pole of [
+ [0, 100, 200],
+ [200, 100, 0],
+ ] as Vec3[]) {
+ const rig = rest();
+ solveTwoBoneIk(rig, {
+ chain: ["hip", "knee", "foot"],
+ target,
+ pole,
+ weight: 1,
+ });
+ expect(dist(rig.position(2), target)).toBeLessThan(0.01);
+ // Bone lengths hold.
+ expect(dist(rig.position(0), rig.position(1))).toBeCloseTo(100, 4);
+ expect(dist(rig.position(1), rig.position(2))).toBeCloseTo(100, 4);
+ // The knee sits on the pole's side of the hip→foot line.
+ const knee = rig.position(1);
+ const axis = target.map((v) => v / Math.hypot(...target)) as Vec3;
+ const off = (p: Vec3) => {
+ const d = p[0] * axis[0] + p[1] * axis[1] + p[2] * axis[2];
+ return p.map((v, i) => v - d * axis[i]!) as Vec3;
+ };
+ const k = off(knee);
+ const p = off(pole);
+ expect(k[0] * p[0] + k[1] * p[1] + k[2] * p[2]).toBeGreaterThan(0);
+ }
+ }
+ // Out of reach: the leg straightens toward it.
+ const far = rest();
+ solveTwoBoneIk(far, {
+ chain: ["hip", "knee", "foot"],
+ target: [400, 0, 0],
+ weight: 1,
+ });
+ expect(far.position(2)[0]).toBeGreaterThan(199);
+ // Half weight lands between.
+ const half = rest();
+ solveTwoBoneIk(half, {
+ chain: ["hip", "knee", "foot"],
+ target: [60, 120, 0],
+ weight: 0.5,
+ });
+ expect(dist(half.position(2), [60, 120, 0])).toBeGreaterThan(5);
+ expect(dist(half.position(2), [0, 200, 0])).toBeGreaterThan(5);
+ });
+
+ it("turns a joint toward a target within its limit", () => {
+ const rig = rest();
+ solveLookAt(rig, {
+ joint: "hip",
+ target: [100, 0, 0],
+ weight: 1,
+ maxAngleDeg: 180,
+ axis: [0, 1, 0],
+ });
+ expect(rig.position(2)[0]).toBeCloseTo(200, 3);
+ const limited = rest();
+ solveLookAt(limited, {
+ joint: "hip",
+ target: [100, 0, 0],
+ weight: 1,
+ maxAngleDeg: 30,
+ axis: [0, 1, 0],
+ });
+ const f = limited.position(2);
+ expect((Math.atan2(f[0], f[1]) * 180) / Math.PI).toBeCloseTo(30, 3);
+ const both = rest();
+ applyConstraints(both, {
+ lookAt: [
+ {
+ joint: "tail0",
+ target: [50, 100, 0],
+ weight: 1,
+ maxAngleDeg: 90,
+ axis: [1, 0, 0],
+ },
+ ],
+ ik: [{ chain: ["hip", "knee", "foot"], target: [50, 150, 0], weight: 1 }],
+ });
+ expect(dist(both.position(2), [50, 150, 0])).toBeLessThan(0.01);
+ expect(both.position(5)[1]).toBeCloseTo(80, 3);
+ });
+
+ it("simulates spring bones deterministically, lagging then settling", () => {
+ const run = (frames: number) => {
+ const springs = new SpringBones(rest(), [
+ { joints: ["tail0"], stiffness: 4, damping: 0.5, gravity: [0, 0, 0] },
+ ]);
+ let rig = rest();
+ for (let f = 0; f <= frames; f++) {
+ rig = blendClipLayers(MODEL, [
+ { clip: SWING, time: Math.min(1, f / 30), weight: 1 },
+ ]);
+ springs.step(rig, 1 / 30);
+ springs.apply(rig);
+ }
+ return rig;
+ };
+ const mid = run(4);
+ const animated = blendClipLayers(MODEL, [
+ { clip: SWING, time: 4 / 30, weight: 1 },
+ ]);
+ // The root rides the animation; just after it starts moving (-y) the tip trails.
+ expect(dist(mid.position(3), animated.position(3))).toBeLessThan(1e-6);
+ expect(mid.position(5)[1]).toBeGreaterThan(animated.position(5)[1] + 3);
+ // Lengths hold.
+ expect(dist(mid.position(4), mid.position(5))).toBeCloseTo(40, 3);
+ expect(run(4).position(5)).toEqual(mid.position(5));
+ // Long after the motion stops it settles back onto the animated pose.
+ const settled = run(400);
+ const end = blendClipLayers(MODEL, [{ clip: SWING, time: 1, weight: 1 }]);
+ expect(dist(settled.position(5), end.position(5))).toBeLessThan(0.5);
+ });
+});
diff --git a/packages/webgpu-renderers/src/loaders/rig.ts b/packages/webgpu-renderers/src/loaders/rig.ts
new file mode 100644
index 00000000..8ada3df7
--- /dev/null
+++ b/packages/webgpu-renderers/src/loaders/rig.ts
@@ -0,0 +1,428 @@
+/**
+ * Character rigs on a model's scene graph: layered clip blending with bone
+ * masks, analytic two-bone IK with a pole target, clamped look-at, and
+ * deterministic spring bones (follow-through simulated frame by frame from
+ * frame 0, so any frame renders the same whether reached in order or not).
+ *
+ * Everything works in the model's own space; the caller maps scene
+ * targets in (and gravity) through the inverse of the model's transform.
+ */
+import type { Mat4, Vec3 } from "../math3d/index.js";
+import {
+ composeWorld,
+ type LocalPose,
+ localPoseMatrices,
+ type Quat,
+ sampleLocalPose,
+ slerp,
+} from "./pose.js";
+import type { AnimationClip3D, Model3DData, ModelNode3D } from "./types.js";
+
+// --- Quaternion and vector helpers -------------------------------------
+
+const qMul = (a: Quat, b: Quat): Quat => [
+ a[3] * b[0] + a[0] * b[3] + a[1] * b[2] - a[2] * b[1],
+ a[3] * b[1] - a[0] * b[2] + a[1] * b[3] + a[2] * b[0],
+ a[3] * b[2] + a[0] * b[1] - a[1] * b[0] + a[2] * b[3],
+ a[3] * b[3] - a[0] * b[0] - a[1] * b[1] - a[2] * b[2],
+];
+const qConj = (q: Quat): Quat => [-q[0], -q[1], -q[2], q[3]];
+const sub = (a: Vec3, b: Vec3): Vec3 => [a[0] - b[0], a[1] - b[1], a[2] - b[2]];
+const add = (a: Vec3, b: Vec3): Vec3 => [a[0] + b[0], a[1] + b[1], a[2] + b[2]];
+const scale = (a: Vec3, s: number): Vec3 => [a[0] * s, a[1] * s, a[2] * s];
+const dot = (a: Vec3, b: Vec3) => a[0] * b[0] + a[1] * b[1] + a[2] * b[2];
+const cross = (a: Vec3, b: Vec3): Vec3 => [
+ a[1] * b[2] - a[2] * b[1],
+ a[2] * b[0] - a[0] * b[2],
+ a[0] * b[1] - a[1] * b[0],
+];
+const len = (a: Vec3) => Math.hypot(a[0], a[1], a[2]);
+const norm = (a: Vec3): Vec3 => scale(a, 1 / (len(a) || 1));
+const clamp = (v: number, lo: number, hi: number) =>
+ Math.min(hi, Math.max(lo, v));
+
+function qRotate(q: Quat, v: Vec3): Vec3 {
+ const t = scale(cross([q[0], q[1], q[2]], v), 2);
+ return add(add(v, scale(t, q[3])), cross([q[0], q[1], q[2]], t));
+}
+
+function axisAngle(axis: Vec3, angle: number): Quat {
+ const a = norm(axis);
+ const s = Math.sin(angle / 2);
+ return [a[0] * s, a[1] * s, a[2] * s, Math.cos(angle / 2)];
+}
+
+/** Shortest rotation taking direction `a` onto direction `b`. */
+function qBetween(a: Vec3, b: Vec3): Quat {
+ const u = norm(a);
+ const v = norm(b);
+ const d = dot(u, v);
+ if (d < -0.999999) {
+ const ortho =
+ Math.abs(u[0]) < 0.9 ? cross(u, [1, 0, 0]) : cross(u, [0, 1, 0]);
+ return axisAngle(ortho, Math.PI);
+ }
+ const c = cross(u, v);
+ const q: Quat = [c[0], c[1], c[2], 1 + d];
+ const l = Math.hypot(...q);
+ return [q[0] / l, q[1] / l, q[2] / l, q[3] / l];
+}
+
+const position = (m: Mat4): Vec3 => [m[12], m[13], m[14]];
+
+// --- Pose assembly -------------------------------------------------------
+
+export interface RigClipLayer {
+ clip: AnimationClip3D;
+ /** Seconds into the clip. */
+ time: number;
+ /** 0..1: how far this layer overrides the pose beneath it. */
+ weight: number;
+ /** Joint names: only these joints and their descendants take this layer. */
+ mask?: string[];
+}
+
+export interface RigTwoBoneIk {
+ /** [root, mid, effector] joint names, e.g. ["UpperLeg.L", "LowerLeg.L", "Foot.L"]. */
+ chain: [string, string, string];
+ /** Where the effector reaches, model space. */
+ target: Vec3;
+ /** The mid joint (knee, elbow) bends toward this point, model space. */
+ pole?: Vec3;
+ weight: number;
+}
+
+export interface RigLookAt {
+ joint: string;
+ target: Vec3;
+ weight: number;
+ /** Largest turn away from the animated pose, degrees. */
+ maxAngleDeg: number;
+ /** The joint's local axis that should point at the target. */
+ axis: Vec3;
+}
+
+export interface RigConstraints {
+ ik?: RigTwoBoneIk[];
+ lookAt?: RigLookAt[];
+}
+
+/** A pose being edited: local TRS plus the matching model-space matrices. */
+export class RigPose {
+ readonly nodes: ModelNode3D[];
+ readonly local: LocalPose;
+ private worldCache: Mat4[] | null = null;
+ private byName: Map<string, number>;
+
+ constructor(nodes: ModelNode3D[], local: LocalPose) {
+ this.nodes = nodes;
+ this.local = local;
+ this.byName = new Map(nodes.map((n, i) => [n.name, i]));
+ }
+
+ index(name: string): number | undefined {
+ return this.byName.get(name);
+ }
+
+ world(): Mat4[] {
+ this.worldCache ??= composeWorld(
+ this.nodes,
+ localPoseMatrices(this.nodes, this.local),
+ );
+ return this.worldCache;
+ }
+
+ position(i: number): Vec3 {
+ return position(this.world()[i]!);
+ }
+
+ /** Model-space rotation of node `i` (scale ignored). */
+ globalRotation(i: number): Quat {
+ let q = this.local.r[i]!;
+ for (let p = this.nodes[i]!.parent; p >= 0; p = this.nodes[p]!.parent) {
+ q = qMul(this.local.r[p]!, q);
+ }
+ return q;
+ }
+
+ parentRotation(i: number): Quat {
+ const p = this.nodes[i]!.parent;
+ return p >= 0 ? this.globalRotation(p) : [0, 0, 0, 1];
+ }
+
+ /** Call after editing `local` directly. */
+ touch(): void {
+ this.worldCache = null;
+ }
+
+ /** Sets node `i`'s model-space rotation, blended from its current one by `weight`. */
+ setGlobalRotation(i: number, q: Quat, weight = 1): void {
+ const local = qMul(qConj(this.parentRotation(i)), q);
+ this.local.r[i] =
+ weight >= 1 ? local : slerp(this.local.r[i]!, local, weight);
+ this.worldCache = null;
+ }
+
+ /** Every node at or below the named joints. */
+ maskSet(names: string[]): Set<number> {
+ const roots = new Set(
+ names.map((n) => this.byName.get(n)).filter((i) => i !== undefined),
+ );
+ const out = new Set<number>();
+ for (let i = 0; i < this.nodes.length; i++) {
+ for (let p = i; p >= 0; p = this.nodes[p]!.parent) {
+ if (roots.has(p)) {
+ out.add(i);
+ break;
+ }
+ }
+ }
+ return out;
+ }
+}
+
+/**
+ * The rest pose with each layer blended over the pose beneath it by its
+ * weight (translations and scales lerped, rotations slerped), restricted to
+ * the layer's mask.
+ */
+export function blendClipLayers(
+ model: Model3DData,
+ layers: RigClipLayer[],
+): RigPose {
+ const nodes = model.nodes ?? [];
+ const pose = sampleLocalPose(nodes, undefined, 0);
+ const rig = new RigPose(nodes, pose);
+ for (const layer of layers) {
+ const w = clamp(layer.weight, 0, 1);
+ if (w <= 0) continue;
+ const lp = sampleLocalPose(nodes, layer.clip, layer.time);
+ const mask = layer.mask?.length ? rig.maskSet(layer.mask) : undefined;
+ for (let i = 0; i < nodes.length; i++) {
+ if (mask && !mask.has(i)) continue;
+ const t = pose.t[i]!;
+ const s = pose.s[i]!;
+ pose.t[i] = add(t, scale(sub(lp.t[i]!, t), w));
+ pose.s[i] = add(s, scale(sub(lp.s[i]!, s), w));
+ pose.r[i] = slerp(pose.r[i]!, lp.r[i]!, w);
+ }
+ }
+ return rig;
+}
+
+/**
+ * Analytic two-bone IK (law of cosines; D. Holden's formulation): bends the
+ * mid joint to the angle that puts the effector at the target's distance,
+ * swings the root onto the target, then twists the chain about the
+ * root→target axis so the mid joint faces the pole.
+ */
+export function solveTwoBoneIk(rig: RigPose, c: RigTwoBoneIk): void {
+ const ids = c.chain.map((n) => rig.index(n));
+ if (ids.some((i) => i === undefined)) return;
+ const [ia, ib, ic] = ids as [number, number, number];
+ const w = clamp(c.weight, 0, 1);
+ if (w <= 0) return;
+ const before: [Quat, Quat] = [rig.local.r[ia]!, rig.local.r[ib]!];
+ const a = rig.position(ia);
+ const b = rig.position(ib);
+ const cc = rig.position(ic);
+ const eps = 1e-4;
+ const lab = len(sub(b, a));
+ const lcb = len(sub(cc, b));
+ const lat = clamp(len(sub(c.target, a)), eps, lab + lcb - eps);
+ const acos = (v: number) => Math.acos(clamp(v, -1, 1));
+ const acAb0 = acos(dot(norm(sub(cc, a)), norm(sub(b, a))));
+ const baBc0 = acos(dot(norm(sub(a, b)), norm(sub(cc, b))));
+ const acAb1 = acos((lcb * lcb - lab * lab - lat * lat) / (-2 * lab * lat));
+ const baBc1 = acos((lat * lat - lab * lab - lcb * lcb) / (-2 * lab * lcb));
+ // The bend plane: the current one, or the pole's when the chain is straight.
+ let axis0 = cross(sub(cc, a), sub(b, a));
+ if (len(axis0) < 1e-6 && c.pole) axis0 = cross(sub(cc, a), sub(c.pole, a));
+ if (len(axis0) < 1e-6) axis0 = cross(sub(cc, a), [0, 0, 1]);
+ const ga = rig.globalRotation(ia);
+ const gb = rig.globalRotation(ib);
+ // Both turns about the bend axis; the mid joint also rides the root's.
+ const r0 = axisAngle(axis0, acAb1 - acAb0);
+ const r1 = axisAngle(axis0, baBc1 - baBc0);
+ rig.setGlobalRotation(ia, qMul(r0, ga));
+ rig.setGlobalRotation(ib, qMul(r0, qMul(r1, gb)));
+ // Swing the whole chain so the effector lands on the target.
+ const reach = sub(rig.position(ic), a);
+ rig.setGlobalRotation(
+ ia,
+ qMul(qBetween(reach, sub(c.target, a)), rig.globalRotation(ia)),
+ );
+ if (c.pole) twistToPole(rig, ia, ib, a, c.target, c.pole);
+ if (w < 1) {
+ const solved: [Quat, Quat] = [rig.local.r[ia]!, rig.local.r[ib]!];
+ rig.local.r[ia] = slerp(before[0], solved[0], w);
+ rig.local.r[ib] = slerp(before[1], solved[1], w);
+ rig.touch();
+ }
+}
+
+/** Turns the chain about root→target so the mid joint points toward the pole. */
+function twistToPole(
+ rig: RigPose,
+ ia: number,
+ ib: number,
+ a: Vec3,
+ target: Vec3,
+ pole: Vec3,
+): void {
+ const axis = norm(sub(target, a));
+ const flat = (v: Vec3) => sub(v, scale(axis, dot(v, axis)));
+ const mid = flat(sub(rig.position(ib), a));
+ const want = flat(sub(pole, a));
+ if (len(mid) < 1e-6 || len(want) < 1e-6) return;
+ const m = norm(mid);
+ const p = norm(want);
+ const angle = Math.atan2(dot(cross(m, p), axis), dot(m, p));
+ rig.setGlobalRotation(
+ ia,
+ qMul(axisAngle(axis, angle), rig.globalRotation(ia)),
+ );
+}
+
+/** Turns a joint so its `axis` points at the target, at most `maxAngleDeg`. */
+export function solveLookAt(rig: RigPose, c: RigLookAt): void {
+ const i = rig.index(c.joint);
+ const w = clamp(c.weight, 0, 1);
+ if (i === undefined || w <= 0) return;
+ const g = rig.globalRotation(i);
+ const facing = qRotate(g, c.axis);
+ const want = sub(c.target, rig.position(i));
+ if (len(want) < 1e-6) return;
+ let swing = qBetween(facing, want);
+ const angle = 2 * Math.acos(clamp(swing[3], -1, 1));
+ const max = (clamp(c.maxAngleDeg, 0, 180) * Math.PI) / 180;
+ if (angle > max) {
+ const ax: Vec3 = [swing[0], swing[1], swing[2]];
+ swing = len(ax) > 1e-9 ? axisAngle(ax, max) : [0, 0, 0, 1];
+ }
+ rig.setGlobalRotation(i, qMul(swing, g), w);
+}
+
+/** Look-at first (a head turns), then IK (limbs reach), in the given order. */
+export function applyConstraints(rig: RigPose, c: RigConstraints): void {
+ for (const l of c.lookAt ?? []) solveLookAt(rig, l);
+ for (const k of c.ik ?? []) solveTwoBoneIk(rig, k);
+}
+
+// --- Spring bones ------------------------------------------------------------
+
+export interface RigSpringChain {
+ /** Root joints; each chain runs down its first child to the tip. */
+ joints: string[];
+ /** Pull back toward the animated pose, per second. Default 6. */
+ stiffness?: number;
+ /** Share of velocity lost per second, 0..1-ish. Default 0.6. */
+ damping?: number;
+ /** Model-space acceleration, px/s². Default [0, 0, 0]. */
+ gravity?: Vec3;
+}
+
+interface Chain {
+ ids: number[];
+ stiffness: number;
+ damping: number;
+ gravity: Vec3;
+ cur: Vec3[];
+ prev: Vec3[];
+}
+
+/** Each root's chain: the joint, its first child, that one's first child… */
+function chainFrom(rig: RigPose, root: number): number[] {
+ const ids = [root];
+ for (;;) {
+ const last = ids[ids.length - 1]!;
+ const child = rig.nodes.findIndex((n) => n.parent === last);
+ if (child < 0) return ids;
+ ids.push(child);
+ }
+}
+
+/**
+ * Verlet springs on joint chains. `step` advances one frame from the
+ * animated pose; `apply` turns the chain's joints toward the simulated
+ * points. State only moves forward, so callers replay from frame 0.
+ */
+export class SpringBones {
+ private chains: Chain[] = [];
+ private started = false;
+
+ constructor(
+ rig: RigPose,
+ springs: RigSpringChain[],
+ private substeps = 4,
+ ) {
+ for (const s of springs) {
+ for (const name of s.joints) {
+ const root = rig.index(name);
+ if (root === undefined) continue;
+ const ids = chainFrom(rig, root);
+ if (ids.length < 2) continue;
+ this.chains.push({
+ ids,
+ stiffness: s.stiffness ?? 6,
+ damping: clamp(s.damping ?? 0.6, 0, 1),
+ gravity: s.gravity ?? [0, 0, 0],
+ cur: [],
+ prev: [],
+ });
+ }
+ }
+ }
+
+ /** Advances every chain by `dt` seconds toward the animated `rig`. */
+ step(rig: RigPose, dt: number): void {
+ for (const c of this.chains) {
+ const animated = c.ids.map((i) => rig.position(i));
+ if (!this.started) {
+ c.cur = animated.map((p) => [...p] as Vec3);
+ c.prev = animated.map((p) => [...p] as Vec3);
+ continue;
+ }
+ const h = dt / this.substeps;
+ for (let s = 0; s < this.substeps; s++) this.substep(c, animated, h);
+ }
+ this.started = true;
+ }
+
+ private substep(c: Chain, animated: Vec3[], h: number): void {
+ c.cur[0] = animated[0]!;
+ c.prev[0] = animated[0]!;
+ const keep = (1 - c.damping) ** h;
+ const pull = Math.min(1, c.stiffness * h);
+ for (let k = 1; k < c.ids.length; k++) {
+ const parent = c.cur[k - 1]!;
+ const rest = len(sub(animated[k]!, animated[k - 1]!));
+ // Where the animation puts this point, carried along by the parent.
+ const goal = add(parent, sub(animated[k]!, animated[k - 1]!));
+ const p = c.cur[k]!;
+ let next = add(p, scale(sub(p, c.prev[k]!), keep));
+ next = add(next, scale(sub(goal, next), pull));
+ next = add(next, scale(c.gravity, h * h));
+ const dir = sub(next, parent);
+ next = len(dir) > 1e-9 ? add(parent, scale(norm(dir), rest)) : goal;
+ c.prev[k] = p;
+ c.cur[k] = next;
+ }
+ }
+
+ /** Rotates each chain joint so its child lies at the simulated point. */
+ apply(rig: RigPose): void {
+ for (const c of this.chains) {
+ for (let k = 0; k + 1 < c.ids.length; k++) {
+ const i = c.ids[k]!;
+ const from = sub(rig.position(c.ids[k + 1]!), rig.position(i));
+ const to = sub(c.cur[k + 1]!, rig.position(i));
+ if (len(from) < 1e-9 || len(to) < 1e-9) continue;
+ rig.setGlobalRotation(
+ i,
+ qMul(qBetween(from, to), rig.globalRotation(i)),
+ );
+ }
+ }
+ }
+}
diff --git a/packages/webgpu-renderers/src/math3d/index.ts b/packages/webgpu-renderers/src/math3d/index.ts
index ac4aafbc..b9e1bc3a 100644
--- a/packages/webgpu-renderers/src/math3d/index.ts
+++ b/packages/webgpu-renderers/src/math3d/index.ts
@@ -2,5 +2,6 @@ export * from "./camera3d.js";
export * from "./environment3d.js";
export * from "./light3d.js";
export * from "./matrix4.js";
+export * from "./noise.js";
export * from "./path3d.js";
export * from "./vector3.js";
diff --git a/packages/webgpu-renderers/src/math3d/noise.ts b/packages/webgpu-renderers/src/math3d/noise.ts
new file mode 100644
index 00000000..39c21432
--- /dev/null
+++ b/packages/webgpu-renderers/src/math3d/noise.ts
@@ -0,0 +1,204 @@
+/**
+ * Seeded, deterministic gradient noise (Perlin, improved) and the fractal
+ * sums procedural geometry is built from. Same seed, same field: on every
+ * machine and every frame.
+ */
+
+/** mulberry32: a small, fast, well-distributed 32-bit PRNG. Returns [0, 1). */
+export function createRandom(seed: number): () => number {
+ let a = (Math.floor(seed) ^ 0x9e3779b9) >>> 0;
+ return () => {
+ a = (a + 0x6d2b79f5) >>> 0;
+ let t = a;
+ t = Math.imul(t ^ (t >>> 15), t | 1);
+ t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+ };
+}
+
+const fade = (t: number) => t * t * t * (t * (t * 6 - 15) + 10);
+const lerp = (a: number, b: number, t: number) => a + (b - a) * t;
+
+/** The 12 cube-edge gradients of improved Perlin noise (z ignored in 2D). */
+const GRAD3 = [
+ [1, 1, 0],
+ [-1, 1, 0],
+ [1, -1, 0],
+ [-1, -1, 0],
+ [1, 0, 1],
+ [-1, 0, 1],
+ [1, 0, -1],
+ [-1, 0, -1],
+ [0, 1, 1],
+ [0, -1, 1],
+ [0, 1, -1],
+ [0, -1, -1],
+] as const;
+
+const GRAD2 = [
+ [1, 0],
+ [-1, 0],
+ [0, 1],
+ [0, -1],
+ [Math.SQRT1_2, Math.SQRT1_2],
+ [-Math.SQRT1_2, Math.SQRT1_2],
+ [Math.SQRT1_2, -Math.SQRT1_2],
+ [-Math.SQRT1_2, -Math.SQRT1_2],
+] as const;
+
+export interface Noise {
+ /** Gradient noise in about [-1, 1]. */
+ noise2(x: number, y: number): number;
+ /** Gradient noise in about [-1, 1]. */
+ noise3(x: number, y: number, z: number): number;
+}
+
+/** A gradient noise field whose lattice is shuffled by `seed`. */
+export function createNoise(seed = 1): Noise {
+ const rand = createRandom(seed);
+ const p = new Uint8Array(256);
+ for (let i = 0; i < 256; i++) p[i] = i;
+ for (let i = 255; i > 0; i--) {
+ const j = Math.floor(rand() * (i + 1));
+ const t = p[i]!;
+ p[i] = p[j]!;
+ p[j] = t;
+ }
+ const perm = new Uint8Array(512);
+ for (let i = 0; i < 512; i++) perm[i] = p[i & 255]!;
+
+ const g2 = (h: number, x: number, y: number) => {
+ const g = GRAD2[h & 7]!;
+ return g[0] * x + g[1] * y;
+ };
+ const g3 = (h: number, x: number, y: number, z: number) => {
+ const g = GRAD3[h % 12]!;
+ return g[0] * x + g[1] * y + g[2] * z;
+ };
+
+ return {
+ noise2(x, y) {
+ const xi = Math.floor(x);
+ const yi = Math.floor(y);
+ const xf = x - xi;
+ const yf = y - yi;
+ const X = xi & 255;
+ const Y = yi & 255;
+ const aa = perm[perm[X]! + Y]!;
+ const ab = perm[perm[X]! + Y + 1]!;
+ const ba = perm[perm[X + 1]! + Y]!;
+ const bb = perm[perm[X + 1]! + Y + 1]!;
+ const u = fade(xf);
+ const v = fade(yf);
+ // Scaled so the field reaches about ±1.
+ return (
+ 1.41 *
+ lerp(
+ lerp(g2(aa, xf, yf), g2(ba, xf - 1, yf), u),
+ lerp(g2(ab, xf, yf - 1), g2(bb, xf - 1, yf - 1), u),
+ v,
+ )
+ );
+ },
+ noise3(x, y, z) {
+ const xi = Math.floor(x);
+ const yi = Math.floor(y);
+ const zi = Math.floor(z);
+ const xf = x - xi;
+ const yf = y - yi;
+ const zf = z - zi;
+ const X = xi & 255;
+ const Y = yi & 255;
+ const Z = zi & 255;
+ const A = perm[X]! + Y;
+ const B = perm[X + 1]! + Y;
+ const AA = perm[A]! + Z;
+ const AB = perm[A + 1]! + Z;
+ const BA = perm[B]! + Z;
+ const BB = perm[B + 1]! + Z;
+ const u = fade(xf);
+ const v = fade(yf);
+ const w = fade(zf);
+ return lerp(
+ lerp(
+ lerp(g3(perm[AA]!, xf, yf, zf), g3(perm[BA]!, xf - 1, yf, zf), u),
+ lerp(
+ g3(perm[AB]!, xf, yf - 1, zf),
+ g3(perm[BB]!, xf - 1, yf - 1, zf),
+ u,
+ ),
+ v,
+ ),
+ lerp(
+ lerp(
+ g3(perm[AA + 1]!, xf, yf, zf - 1),
+ g3(perm[BA + 1]!, xf - 1, yf, zf - 1),
+ u,
+ ),
+ lerp(
+ g3(perm[AB + 1]!, xf, yf - 1, zf - 1),
+ g3(perm[BB + 1]!, xf - 1, yf - 1, zf - 1),
+ u,
+ ),
+ v,
+ ),
+ w,
+ );
+ },
+ };
+}
+
+export type FractalMode = "fbm" | "ridged" | "terrace";
+
+export interface FractalOptions {
+ octaves: number;
+ persistence: number;
+ lacunarity: number;
+ mode: FractalMode;
+}
+
+/**
+ * Fractal sum of `octaves` noise layers at (x, y), in [0, 1] before any
+ * normalization: "fbm" rolls, "ridged" folds each octave into sharp crests
+ * (Musgrave's ridged multifractal: each octave weighted by the last, so
+ * detail gathers on ridges, not in valleys). Terracing is applied later,
+ * over the normalized field.
+ */
+export function fractal2(
+ noise: Noise,
+ x: number,
+ y: number,
+ o: FractalOptions,
+): number {
+ let sum = 0;
+ let norm = 0;
+ let amp = 1;
+ let freq = 1;
+ let weight = 1;
+ for (let i = 0; i < o.octaves; i++) {
+ // Offset each octave so their lattices never line up.
+ const n = noise.noise2(x * freq + i * 17.31, y * freq - i * 9.77);
+ if (o.mode === "ridged") {
+ let r = 1 - Math.abs(n);
+ r *= r;
+ r *= weight;
+ weight = Math.min(1, Math.max(0, r * 2));
+ sum += r * amp;
+ } else {
+ sum += (n * 0.5 + 0.5) * amp;
+ }
+ norm += amp;
+ amp *= o.persistence;
+ freq *= o.lacunarity;
+ }
+ return norm > 0 ? sum / norm : 0;
+}
+
+/** Stepped plateaus: each step flat for most of its width, then a steep riser. */
+export function terrace(h: number, steps: number): number {
+ const k = h * steps;
+ const base = Math.floor(k);
+ const f = k - base;
+ const t = Math.min(1, Math.max(0, (f - 0.7) / 0.3));
+ return Math.min(1, (base + t * t * (3 - 2 * t)) / steps);
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/geometry-arrays.ts b/packages/webgpu-renderers/src/renderer3d/geometry-arrays.ts
new file mode 100644
index 00000000..462282df
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/geometry-arrays.ts
@@ -0,0 +1,124 @@
+/**
+ * Flat vertex arrays procedural meshes are assembled in before they become a
+ * Mesh3DData. Winding follows the primitives': for a triangle (a, b, c),
+ * cross(b - a, c - a) points out of the surface.
+ */
+export interface GeometryArrays {
+ positions: number[];
+ normals: number[];
+ uvs: number[];
+ /** RGBA per vertex, display (sRGB) 0..1; empty for none. */
+ colors: number[];
+ indices: number[];
+}
+
+export type Rgba = [number, number, number, number];
+type V3 = readonly [number, number, number];
+
+export function emptyArrays(): GeometryArrays {
+ return { positions: [], normals: [], uvs: [], colors: [], indices: [] };
+}
+
+export function pushVertex(
+ g: GeometryArrays,
+ p: V3,
+ n: V3,
+ uv: readonly [number, number],
+ color?: Rgba,
+): number {
+ g.positions.push(p[0], p[1], p[2]);
+ g.normals.push(n[0], n[1], n[2]);
+ g.uvs.push(uv[0], uv[1]);
+ if (color) g.colors.push(color[0], color[1], color[2], color[3]);
+ return g.positions.length / 3 - 1;
+}
+
+/** Appends triangle (a, b, c), or (a, c, b) when `flip`. */
+export function pushTriangle(
+ g: GeometryArrays,
+ a: number,
+ b: number,
+ c: number,
+ flip = false,
+): void {
+ if (flip) g.indices.push(a, c, b);
+ else g.indices.push(a, b, c);
+}
+
+/**
+ * Smooth vertex normals: each vertex takes the area-weighted sum of its
+ * triangles' face normals.
+ */
+export function accumulateNormals(g: GeometryArrays): void {
+ const p = g.positions;
+ const n = new Float64Array(p.length);
+ const idx = g.indices;
+ for (let t = 0; t < idx.length; t += 3) {
+ const a = idx[t]! * 3;
+ const b = idx[t + 1]! * 3;
+ const c = idx[t + 2]! * 3;
+ const ex = p[b]! - p[a]!;
+ const ey = p[b + 1]! - p[a + 1]!;
+ const ez = p[b + 2]! - p[a + 2]!;
+ const fx = p[c]! - p[a]!;
+ const fy = p[c + 1]! - p[a + 1]!;
+ const fz = p[c + 2]! - p[a + 2]!;
+ const cx = ey * fz - ez * fy;
+ const cy = ez * fx - ex * fz;
+ const cz = ex * fy - ey * fx;
+ for (const v of [a, b, c]) {
+ n[v] += cx;
+ n[v + 1] += cy;
+ n[v + 2] += cz;
+ }
+ }
+ for (let v = 0; v < n.length; v += 3) {
+ const len = Math.hypot(n[v]!, n[v + 1]!, n[v + 2]!) || 1;
+ g.normals[v] = n[v]! / len;
+ g.normals[v + 1] = n[v + 1]! / len;
+ g.normals[v + 2] = n[v + 2]! / len;
+ }
+}
+
+/** Every triangle gets its own three vertices and its face normal: faceted shading. */
+export function flatten(g: GeometryArrays): GeometryArrays {
+ const out = emptyArrays();
+ const p = g.positions;
+ const hasColor = g.colors.length > 0;
+ for (let t = 0; t < g.indices.length; t += 3) {
+ const tri = [g.indices[t]!, g.indices[t + 1]!, g.indices[t + 2]!];
+ const [a, b, c] = tri.map((i) => i * 3) as [number, number, number];
+ const ex = p[b]! - p[a]!;
+ const ey = p[b + 1]! - p[a + 1]!;
+ const ez = p[b + 2]! - p[a + 2]!;
+ const fx = p[c]! - p[a]!;
+ const fy = p[c + 1]! - p[a + 1]!;
+ const fz = p[c + 2]! - p[a + 2]!;
+ let nx = ey * fz - ez * fy;
+ let ny = ez * fx - ex * fz;
+ let nz = ex * fy - ey * fx;
+ const len = Math.hypot(nx, ny, nz) || 1;
+ nx /= len;
+ ny /= len;
+ nz /= len;
+ // A face's color: the mean of its corners'.
+ let color: Rgba | undefined;
+ if (hasColor) {
+ color = [0, 0, 0, 0];
+ for (const i of tri)
+ for (let k = 0; k < 4; k++) color[k] += g.colors[i * 4 + k]! / 3;
+ }
+ for (const i of tri) {
+ out.indices.push(
+ pushVertex(
+ out,
+ [p[i * 3]!, p[i * 3 + 1]!, p[i * 3 + 2]!],
+ [nx, ny, nz],
+ [g.uvs[i * 2]!, g.uvs[i * 2 + 1]!],
+ color,
+ ),
+ );
+ }
+ }
+ return out;
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/index.ts b/packages/webgpu-renderers/src/renderer3d/index.ts
index 9c3aee44..bb0ca9cb 100644
--- a/packages/webgpu-renderers/src/renderer3d/index.ts
+++ b/packages/webgpu-renderers/src/renderer3d/index.ts
@@ -9,6 +9,8 @@ export * from "./motion-blur-pipeline.js";
export * from "./primitive-geometry.js";
export * from "./quad3d-pipeline.js";
export * from "./renderer3d.js";
+export * from "./scatter.js";
+export * from "./sdf-geometry.js";
export * from "./shadow-depth.js";
export * from "./shadow-frustum.js";
export * from "./shadow-pipeline.js";
@@ -16,5 +18,6 @@ export * from "./ssao-pipeline.js";
export * from "./ssr-gi-pipeline.js";
export * from "./surface-params.js";
export * from "./swarm3d-pipeline.js";
+export * from "./terrain-geometry.js";
export * from "./text-extruder.js";
export * from "./volumetric-fog-pipeline.js";
diff --git a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
index b4e38844..7de0df52 100644
--- a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
@@ -40,6 +40,41 @@ export interface MeshGlass {
frost: number;
}
+/**
+ * Procedural wind: vertices bend downwind by how far they sit above the
+ * mesh's base, swaying in a traveling wave (instances sway out of phase).
+ */
+export interface MeshWind {
+ /** Peak lean, degrees. */
+ amplitudeDeg: number;
+ /** Sways per second. */
+ frequency: number;
+ /** Downwind direction on the ground plane [x, z]. */
+ direction: [number, number];
+ /** Gust wavelength across the field, px. */
+ waveScale: number;
+ /** Seconds. */
+ time: number;
+ /** The mesh's base (its largest local y: up is -y) and height, px. */
+ baseY: number;
+ height: number;
+}
+
+/** Floats in the mesh `ModelUniforms` struct. */
+export const MESH_MODEL_FLOATS = 52 + SURFACE_PARAMS_FLOATS + 16;
+
+/**
+ * Skinned crowd (vertex animation): every instance skins from a baked
+ * joint palette at its own clip, phase and speed (`bakeCrowdPalette`).
+ */
+export interface MeshCrowd {
+ /** `palette` uploaded once: `frames × jointCount` model-space joint matrices. */
+ palette: GPUBuffer;
+ jointCount: number;
+ /** Seconds. */
+ time: number;
+}
+
export interface MeshUniformData extends SurfaceParams {
modelMatrix: Float32Array | number[];
normalMatrix?: Float32Array | number[];
@@ -80,6 +115,11 @@ export interface MeshUniformData extends SurfaceParams {
specularIntensity?: number;
ambientIntensity?: number;
metallic?: number;
+ /** Base-color UV window [x, y, width, height]: uv' = xy + uv · wh. */
+ uvRect?: [number, number, number, number];
+ wind?: MeshWind;
+ /** Instances carry a second record each and skin from a baked palette. */
+ crowd?: MeshCrowd;
}
export interface GPUMeshBuffers {
@@ -95,6 +135,26 @@ export interface GPUMeshBuffers {
wireIndexCount?: number;
}
+/** uvRect, then wind (amplitude rad, angular frequency, wave scale, time), then (dir x, dir z, baseY, 1 / height). */
+function packUvAndWind(out: Float32Array, at: number, data: MeshUniformData) {
+ const uv = data.uvRect ?? [0, 0, 1, 1];
+ out.set(uv, at);
+ const w = data.wind;
+ if (!w || w.amplitudeDeg === 0 || w.height <= 0) {
+ out.fill(0, at + 4, at + 12);
+ return;
+ }
+ const len = Math.hypot(w.direction[0], w.direction[1]) || 1;
+ out[at + 4] = (w.amplitudeDeg * Math.PI) / 180;
+ out[at + 5] = w.frequency * Math.PI * 2;
+ out[at + 6] = (Math.PI * 2) / Math.max(1, w.waveScale);
+ out[at + 7] = w.time;
+ out[at + 8] = w.direction[0] / len;
+ out[at + 9] = w.direction[1] / len;
+ out[at + 10] = w.baseY;
+ out[at + 11] = 1 / w.height;
+}
+
export class Mesh3DPipeline {
private device: GPUDevice;
public pipelineSingle: GPURenderPipeline;
@@ -145,7 +205,7 @@ export class Mesh3DPipeline {
private retiredColorBuffers: GPUBuffer[] = [];
// Transform, tint, draw/material params, shade, shared surface block, maps.
- private modelData = new Float32Array(52 + SURFACE_PARAMS_FLOATS);
+ private modelData = new Float32Array(MESH_MODEL_FLOATS);
constructor(
device: GPUDevice,
@@ -158,7 +218,7 @@ export class Mesh3DPipeline {
) {
this.device = device;
this.modelPool = new BufferPool(
- (52 + SURFACE_PARAMS_FLOATS) * 4,
+ MESH_MODEL_FLOATS * 4,
GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
);
this.skinningPool = new BufferPool(
@@ -451,12 +511,13 @@ export class Mesh3DPipeline {
this.modelData[36] = data.opacity ?? 1.0;
this.modelData[37] = data.twoSided ? 1.0 : 0.0;
- this.modelData[38] = data.isSkinned ? 1.0 : 0.0;
+ this.modelData[38] = data.isSkinned || data.crowd ? 1.0 : 0.0;
// Bit 0 selects instance data; bit 1 selects the extra map UVs.
this.modelData[39] =
(data.isInstanced || data.instanceBuffer ? 1 : 0) +
(data.hasMapUvs ? 2 : 0) +
- (data.matrixInstances ? 4 : 0);
+ (data.matrixInstances ? 4 : 0) +
+ (data.crowd ? 8 : 0);
const shininess =
data.roughness !== undefined
@@ -489,6 +550,12 @@ export class Mesh3DPipeline {
: 0;
this.modelData[mapsOffset + 2] = maps?.metallicRoughness ? 1 : 0;
this.modelData[mapsOffset + 3] = maps?.emissive ? 1 : 0;
+ packUvAndWind(this.modelData, mapsOffset + 4, data);
+ const crowd = data.crowd;
+ this.modelData.set(
+ crowd ? [crowd.time, 0, crowd.jointCount, 1] : [0, 0, 0, 0],
+ mapsOffset + 16,
+ );
const glass = data.glass;
if (glass) {
// shade.rgb and pbr.zw are unused outside toon: glass parameters.
@@ -510,8 +577,9 @@ export class Mesh3DPipeline {
`Skinning data requires ${data.skinningMatrices.byteLength} bytes, exceeding the device storage binding limit of ${this.device.limits.maxStorageBufferBindingSize} bytes`,
);
}
- const skinningBuffer =
- data.isSkinned && data.skinningMatrices
+ const skinningBuffer = data.crowd
+ ? data.crowd.palette
+ : data.isSkinned && data.skinningMatrices
? this.skinningPool.getBuffer(this.device, data.skinningMatrices)
: this.defaultSkinningBuffer;
diff --git a/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts b/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts
index b6531053..5d08d42f 100644
--- a/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts
+++ b/packages/webgpu-renderers/src/renderer3d/primitive-geometry.ts
@@ -3,6 +3,12 @@ import type { Material3D, Mesh3DData, Model3DData } from "../loaders/types.js";
import { Path3D } from "../math3d/path3d.js";
import type { Vec3 } from "../math3d/vector3.js";
import { sampleSvgArc } from "../renderer2d/path-geometry.js";
+import type { GeometryArrays } from "./geometry-arrays.js";
+import { buildSdf, type SdfGeometryOptions } from "./sdf-geometry.js";
+import {
+ buildTerrain,
+ type TerrainGeometryOptions,
+} from "./terrain-geometry.js";
import {
type Contour,
type CurveFlattening,
@@ -301,7 +307,9 @@ export type PrimitiveGeometryOptions =
| TubeGeometryOptions
| Line3DGeometryOptions
| Points3DGeometryOptions
- | ShardGeometryOptions;
+ | ShardGeometryOptions
+ | TerrainGeometryOptions
+ | SdfGeometryOptions;
class MeshBuilder {
positions: number[] = [];
@@ -2532,6 +2540,15 @@ function shard(o: ShardGeometryOptions, mb: MeshBuilder): void {
}
}
+/** Takes over arrays built outside the builder (terrain, SDF). */
+function adopt(mb: MeshBuilder, g: GeometryArrays): void {
+ mb.positions = g.positions;
+ mb.normals = g.normals;
+ mb.uvs = g.uvs;
+ mb.colors = g.colors;
+ mb.indices = g.indices;
+}
+
function finalizeModel3DData(
name: string,
meshes: Mesh3DData[],
@@ -2606,6 +2623,8 @@ export function generatePrimitiveGeometry(
else if (options.type === "line3d") line3d(options, mb);
else if (options.type === "points3d") points3d(options, mb);
else if (options.type === "shard") shard(options, mb);
+ else if (options.type === "terrain") adopt(mb, buildTerrain(options));
+ else if (options.type === "sdf") adopt(mb, buildSdf(options));
else {
throw new Error(
`Unknown 3D primitive: ${(options as { type?: string }).type}`,
diff --git a/packages/webgpu-renderers/src/renderer3d/procedural-geometry.test.ts b/packages/webgpu-renderers/src/renderer3d/procedural-geometry.test.ts
new file mode 100644
index 00000000..70cd6c12
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/procedural-geometry.test.ts
@@ -0,0 +1,292 @@
+import { describe, expect, it } from "vitest";
+import type { Mesh3DData } from "../loaders/types.js";
+import { Matrix4Math } from "../math3d/matrix4.js";
+import { generatePrimitiveGeometry } from "./primitive-geometry.js";
+import { scatterOnSurface, seatOnSurface } from "./scatter.js";
+import { SdfField, type SdfGeometryOptions } from "./sdf-geometry.js";
+import {
+ createTerrainSampler,
+ type TerrainGeometryOptions,
+ terrainHeights,
+} from "./terrain-geometry.js";
+
+/**
+ * Closed and consistently wound: welded by position, every edge is used by
+ * exactly two triangles, once in each direction.
+ */
+function expectWatertight(mesh: Mesh3DData): void {
+ const p = mesh.positions;
+ const weld = new Map<string, number>();
+ const id = (i: number) => {
+ const k = `${p[i * 3]!.toFixed(3)},${p[i * 3 + 1]!.toFixed(3)},${p[i * 3 + 2]!.toFixed(3)}`;
+ let v = weld.get(k);
+ if (v === undefined) {
+ v = weld.size;
+ weld.set(k, v);
+ }
+ return v;
+ };
+ const edges = new Map<string, number>();
+ const idx = mesh.indices;
+ let degenerate = 0;
+ for (let t = 0; t < idx.length; t += 3) {
+ const v = [id(idx[t]!), id(idx[t + 1]!), id(idx[t + 2]!)];
+ if (v[0] === v[1] || v[1] === v[2] || v[0] === v[2]) {
+ degenerate++;
+ continue;
+ }
+ for (let e = 0; e < 3; e++) {
+ const k = `${v[e]}>${v[(e + 1) % 3]}`;
+ edges.set(k, (edges.get(k) ?? 0) + 1);
+ }
+ }
+ const bad: string[] = [];
+ for (const [k, n] of edges) {
+ const [a, b] = k.split(">");
+ if (n !== 1 || edges.get(`${b}>${a}`) !== 1) bad.push(k);
+ }
+ expect(bad.length).toBe(0);
+ expect(edges.size).toBeGreaterThan(0);
+ expect(degenerate).toBeLessThan(idx.length / 3);
+}
+
+const TERRAIN: TerrainGeometryOptions = {
+ type: "terrain",
+ width: 800,
+ depth: 600,
+ height: 200,
+ segments: [48, 36],
+ noise: { seed: 7, ridgeMode: "ridged" },
+};
+
+describe("terrain geometry", () => {
+ it("spans exactly the requested relief and is deterministic", () => {
+ const h = terrainHeights(TERRAIN);
+ expect(Math.min(...h)).toBeCloseTo(0, 6);
+ expect(Math.max(...h)).toBeCloseTo(1, 6);
+ expect(Array.from(terrainHeights(TERRAIN))).toEqual(Array.from(h));
+ const other = terrainHeights({
+ ...TERRAIN,
+ noise: { ...TERRAIN.noise, seed: 8 },
+ });
+ expect(Array.from(other)).not.toEqual(Array.from(h));
+ });
+
+ it("closes into a watertight block, warped or faceted", () => {
+ for (const extra of [
+ {},
+ { overhangWarp: 40 },
+ { flatShaded: true },
+ { noise: { ridgeMode: "terrace" as const, terraceSteps: 5 } },
+ ]) {
+ const model = generatePrimitiveGeometry({ ...TERRAIN, ...extra });
+ expectWatertight(model.meshes[0]!);
+ }
+ });
+
+ it("colors faces by altitude and slope", () => {
+ const model = generatePrimitiveGeometry({
+ ...TERRAIN,
+ colorByElevation: {
+ lowColor: "#00ff00",
+ highColor: "#ffffff",
+ cliffColor: "#ff0000",
+ cliffThreshold: 0.2,
+ },
+ });
+ const mesh = model.meshes[0]!;
+ const colors = mesh.colors!;
+ expect(colors.length).toBe((mesh.positions.length / 3) * 4);
+ // The lowest vertex is green, the highest white.
+ let lo = 0;
+ let hi = 0;
+ const p = mesh.positions;
+ const count = (48 + 1) * (36 + 1);
+ for (let v = 0; v < count; v++) {
+ if (p[v * 3 + 1]! > p[lo * 3 + 1]!) lo = v;
+ if (p[v * 3 + 1]! < p[hi * 3 + 1]!) hi = v;
+ }
+ expect(colors[lo * 4 + 1]).toBeGreaterThan(0.9);
+ expect(colors[lo * 4]).toBeLessThan(0.1);
+ expect(colors[hi * 4 + 2]).toBeGreaterThan(0.6);
+ });
+
+ it("samples the surface other layers stand on", () => {
+ const s = createTerrainSampler(TERRAIN);
+ const h = terrainHeights(TERRAIN);
+ // Grid points match exactly; the corner is (-w/2, -d/2).
+ expect(s.heightAt(-400, -300)).toBeCloseTo(-h[0]! * 200, 6);
+ const n = s.normalAt(10, 20);
+ expect(Math.hypot(...n)).toBeCloseTo(1, 6);
+ expect(n[1]).toBeLessThan(0);
+ // Custom heights: a ramp rising toward +x.
+ const ramp = createTerrainSampler({
+ type: "terrain",
+ width: 100,
+ depth: 100,
+ height: 50,
+ segments: 2,
+ heights: [0, 0.5, 1, 0, 0.5, 1, 0, 0.5, 1],
+ });
+ expect(ramp.heightAt(0, 0)).toBeCloseTo(-25);
+ expect(ramp.heightAt(25, 10)).toBeCloseTo(-37.5);
+ expect(ramp.normalAt(0, 0)[0]).toBeLessThan(0);
+ });
+});
+
+const SCULPT: SdfGeometryOptions = {
+ type: "sdf",
+ bounds: [300, 300, 300],
+ resolution: 40,
+ primitives: [
+ { shape: "box", size: [200, 200, 200], radius: 20, color: "#ff0000" },
+ { shape: "sphere", radius: 130, op: "intersect", smoothness: 10 },
+ {
+ shape: "cylinder",
+ radius: 50,
+ height: 400,
+ op: "subtract",
+ smoothness: 8,
+ color: "#0000ff",
+ },
+ { shape: "torus", radius: 110, tube: 18, center: [0, -60, 0] },
+ ],
+};
+
+describe("sdf geometry", () => {
+ it("combines shapes with hard and smooth booleans", () => {
+ const union = new SdfField({
+ type: "sdf",
+ bounds: [400, 400, 400],
+ primitives: [
+ { shape: "sphere", radius: 50, center: [-40, 0, 0] },
+ { shape: "sphere", radius: 50, center: [40, 0, 0], smoothness: 30 },
+ ],
+ });
+ // The smooth union bulges past both spheres between them.
+ expect(union.distance(0, 0, 0)).toBeLessThan(-15);
+ expect(union.distance(0, -32, 0)).toBeLessThan(0);
+ expect(new SdfField({ ...SCULPT }).distance(0, 0, 0)).toBeGreaterThan(0);
+ // A rotated box: 90° about z swaps its x and y extents.
+ const turned = new SdfField({
+ type: "sdf",
+ bounds: [400, 400, 400],
+ primitives: [{ shape: "box", size: [200, 20, 20], rotation: [0, 0, 90] }],
+ });
+ expect(turned.distance(0, 90, 0)).toBeLessThan(0);
+ expect(turned.distance(90, 0, 0)).toBeGreaterThan(0);
+ });
+
+ it("meshes a watertight surface with gradient normals", () => {
+ const model = generatePrimitiveGeometry(SCULPT);
+ const mesh = model.meshes[0]!;
+ expectWatertight(mesh);
+ const field = new SdfField(SCULPT);
+ const p = mesh.positions;
+ const n = mesh.normals;
+ for (let v = 0; v < p.length / 3; v += 37) {
+ const [x, y, z] = [p[v * 3]!, p[v * 3 + 1]!, p[v * 3 + 2]!];
+ expect(Math.abs(field.distance(x, y, z))).toBeLessThan(6);
+ // Stepping along the normal leaves the solid.
+ const s = 4;
+ expect(
+ field.distance(
+ x + n[v * 3]! * s,
+ y + n[v * 3 + 1]! * s,
+ z + n[v * 3 + 2]! * s,
+ ),
+ ).toBeGreaterThan(field.distance(x, y, z));
+ }
+ // The cut through the middle takes the cutter's blue.
+ const colors = mesh.colors!;
+ let blue = 0;
+ for (let v = 0; v < p.length / 3; v++) {
+ if (Math.hypot(p[v * 3]!, p[v * 3 + 2]!) < 55 && colors[v * 4 + 2]! > 0.8)
+ blue++;
+ }
+ expect(blue).toBeGreaterThan(10);
+ });
+
+ it("clips an unbounded gyroid to its bounds", () => {
+ const model = generatePrimitiveGeometry({
+ type: "sdf",
+ bounds: [200, 200, 200],
+ resolution: 32,
+ primitives: [{ shape: "gyroid", scale: 80, thickness: 12 }],
+ });
+ expectWatertight(model.meshes[0]!);
+ const b = model.bounds!;
+ expect(b.max[0]).toBeLessThanOrEqual(101);
+ expect(b.min[1]).toBeGreaterThanOrEqual(-101);
+ });
+});
+
+describe("scatter", () => {
+ const terrain = generatePrimitiveGeometry(TERRAIN);
+ const matrix = Matrix4Math.translate(1000, 500, 0);
+
+ it("places the same points for the same seed, on the surface", () => {
+ const opts = { count: 200, seed: 3, minDistance: 25 };
+ const a = scatterOnSurface({ model: terrain, matrix }, opts);
+ const b = scatterOnSurface({ model: terrain, matrix }, opts);
+ expect(a.points.length).toBe(200);
+ expect(b.points).toEqual(a.points);
+ const sampler = createTerrainSampler(TERRAIN);
+ for (const pt of a.points) {
+ const [x, y, z] = pt.position;
+ expect(y - 500).toBeCloseTo(sampler.heightAt(x - 1000, z), 3);
+ }
+ // Poisson spacing.
+ let closest = Number.POSITIVE_INFINITY;
+ for (let i = 0; i < a.points.length; i++)
+ for (let j = i + 1; j < a.points.length; j++) {
+ const p = a.points[i]!.position;
+ const q = a.points[j]!.position;
+ closest = Math.min(
+ closest,
+ Math.hypot(p[0] - q[0], p[1] - q[1], p[2] - q[2]),
+ );
+ }
+ expect(closest).toBeGreaterThanOrEqual(25);
+ });
+
+ it("filters by slope, altitude and exclusion zones", () => {
+ const high = scatterOnSurface(
+ { model: terrain },
+ { count: 150, elevationRange: [0.6, 1], slopeRange: [0, 30] },
+ );
+ expect(high.points.length).toBeGreaterThan(0);
+ for (const p of high.points) {
+ expect(p.elevation).toBeGreaterThan(0.55);
+ expect((Math.acos(-p.normal[1]) * 180) / Math.PI).toBeLessThanOrEqual(
+ 30.001,
+ );
+ }
+ const cleared = scatterOnSurface(
+ { model: terrain },
+ { count: 300, exclude: [{ x: 0, z: 0, radius: 150 }] },
+ );
+ for (const p of cleared.points)
+ expect(Math.hypot(p.position[0], p.position[2])).toBeGreaterThanOrEqual(
+ 150,
+ );
+ const none = scatterOnSurface(
+ { model: terrain },
+ { count: 50, slopeRange: [0, 30], elevationRange: [1.1, 2] },
+ );
+ expect(none.points.length).toBe(0);
+ expect(none.acceptedArea).toBe(0);
+ });
+
+ it("stands instances on their base, leaning toward the normal", () => {
+ const [pt] = scatterOnSurface(
+ { model: terrain },
+ { count: 1, alignToNormal: 1, randomYaw: false, scaleRange: [2, 2] },
+ ).points;
+ // A mesh whose lowest point is local y = 10 (up is -y) rests on the surface.
+ const at = seatOnSurface(pt!, 10);
+ const n = pt!.normal;
+ for (let k = 0; k < 3; k++)
+ expect(at[k]! - n[k]! * 20).toBeCloseTo(pt!.position[k]!, 5);
+ });
+});
diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
index d091f905..9d7e12e1 100644
--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
@@ -61,7 +61,9 @@ import { decodeKtx2, isKtx2 } from "./ktx2.js";
import {
type GPUMeshBuffers,
Mesh3DPipeline,
+ type MeshCrowd,
type MeshMaterialMaps,
+ type MeshWind,
} from "./mesh3d-pipeline.js";
import { MorphPipeline } from "./morph-pipeline.js";
import { Quad3DPipeline, type Relief3D } from "./quad3d-pipeline.js";
@@ -238,6 +240,12 @@ export interface DrawMesh3DOpts extends SurfaceParams {
matrixInstances?: boolean;
/** Optional overall bounding box (e.g. for instanced clusters or swarms) for shadow frustum fitting. */
bounds?: Aabb;
+ /** Base-color UV window [x, y, width, height] (a layer texture's place in its pass). */
+ uvRect?: [number, number, number, number];
+ /** Procedural wind sway (`MeshWind`). */
+ wind?: MeshWind;
+ /** Skinned crowd: instances skin from a baked palette (`MeshCrowd`). */
+ crowd?: MeshCrowd;
}
/** Scene-wide world settings for a 3D pass. */
@@ -1057,6 +1065,8 @@ export class Renderer3D {
(c.opts.instanceCount && c.opts.instanceCount > 1),
),
instanceBuffer: c.opts.instanceBuffer,
+ wind: c.opts.wind,
+ crowd: c.opts.crowd,
},
c.opts.instanceCount ?? 1,
c.opts.firstInstance ?? 0,
@@ -2079,6 +2089,9 @@ export class Renderer3D {
outline: opts.outline,
matte: opts.matte,
maps: opts.maps,
+ uvRect: opts.uvRect,
+ wind: opts.wind,
+ crowd: opts.crowd,
clearcoat: opts.clearcoat,
clearcoatRoughness: opts.clearcoatRoughness,
anisotropy: opts.anisotropy,
diff --git a/packages/webgpu-renderers/src/renderer3d/scatter.ts b/packages/webgpu-renderers/src/renderer3d/scatter.ts
new file mode 100644
index 00000000..8500a996
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/scatter.ts
@@ -0,0 +1,353 @@
+/**
+ * Procedural surface scattering: deterministic points on any mesh (terrain,
+ * SDF sculpture, primitive or loaded model), sampled by triangle area,
+ * filtered by slope, altitude and exclusion zones, spaced by a Poisson-disk
+ * test on a spatial hash, and oriented between world up and the surface.
+ */
+
+import { poseNodes, type Quat } from "../loaders/pose.js";
+import type { Model3DData } from "../loaders/types.js";
+import { type Mat4, Matrix4Math } from "../math3d/matrix4.js";
+import { createRandom } from "../math3d/noise.js";
+
+type V3 = [number, number, number];
+
+export interface ScatterExclusion {
+ /** Center in the surface's parent space, px. */
+ x: number;
+ z: number;
+ /** Nothing grows inside this radius, px. */
+ radius: number;
+ /** Density ramps back to full over this many px past `radius`. Default 0. */
+ falloff?: number;
+}
+
+export interface ScatterOptions {
+ /** Points to place (fewer when spacing or filters leave no room). */
+ count: number;
+ /** Poisson-disk spacing: no two points closer than this, px. */
+ minDistance?: number;
+ seed?: number;
+ /** Allowed slope in degrees from level ground. Default [0, 89]. */
+ slopeRange?: [number, number];
+ /** Allowed altitude 0..1 across the surface's vertical extent (0 = lowest). Default [0, 1]. */
+ elevationRange?: [number, number];
+ /** 0 keeps instances upright, 1 stands them on the surface normal. Default 0.3. */
+ alignToNormal?: number;
+ /** Uniform scale drawn per point. Default [1, 1]. */
+ scaleRange?: [number, number];
+ /** Random turn around the instance's up axis. Default true. */
+ randomYaw?: boolean;
+ exclude?: ScatterExclusion[];
+}
+
+export interface ScatterPoint {
+ /** On the surface, in its parent space. */
+ position: V3;
+ /** Unit face normal there. */
+ normal: V3;
+ rotation: Quat;
+ scale: number;
+ /** Normalized altitude 0..1. */
+ elevation: number;
+}
+
+export interface ScatterSurface {
+ model: Model3DData;
+ /** The surface layer's transform (its parent space ← model space). */
+ matrix?: Mat4;
+}
+
+export interface ScatterResult {
+ points: ScatterPoint[];
+ /** Share of the surface's area its slope and altitude filters keep, 0..1. */
+ acceptedArea: number;
+}
+
+const UP: V3 = [0, -1, 0];
+
+function transformPoint(m: Mat4, x: number, y: number, z: number): V3 {
+ return [
+ m[0] * x + m[4] * y + m[8] * z + m[12],
+ m[1] * x + m[5] * y + m[9] * z + m[13],
+ m[2] * x + m[6] * y + m[10] * z + m[14],
+ ];
+}
+
+/** Every triangle of the model in the surface's parent space, flat xyz × 3. */
+export function surfaceTriangles(surface: ScatterSurface): Float64Array {
+ const { model } = surface;
+ const root = Matrix4Math.multiply(
+ surface.matrix ?? Matrix4Math.identity(),
+ model.rootTransform ?? Matrix4Math.identity(),
+ );
+ const posed = model.nodes ? poseNodes(model, undefined, 0) : undefined;
+ const tris: number[] = [];
+ for (const mesh of model.meshes) {
+ if (mesh.topology && mesh.topology !== "triangle-list") continue;
+ const node =
+ posed && mesh.node !== undefined && mesh.skin === undefined
+ ? posed[mesh.node]
+ : undefined;
+ const m = node ? Matrix4Math.multiply(root, node) : root;
+ const p = mesh.positions;
+ for (const i of mesh.indices) {
+ tris.push(...transformPoint(m, p[i * 3]!, p[i * 3 + 1]!, p[i * 3 + 2]!));
+ }
+ }
+ return new Float64Array(tris);
+}
+
+function quatMul(a: Quat, b: Quat): Quat {
+ return [
+ a[3] * b[0] + a[0] * b[3] + a[1] * b[2] - a[2] * b[1],
+ a[3] * b[1] - a[0] * b[2] + a[1] * b[3] + a[2] * b[0],
+ a[3] * b[2] + a[0] * b[1] - a[1] * b[0] + a[2] * b[3],
+ a[3] * b[3] - a[0] * b[0] - a[1] * b[1] - a[2] * b[2],
+ ];
+}
+
+/** Shortest rotation taking unit `a` onto unit `b`. */
+function quatBetween(a: V3, b: V3): Quat {
+ const d = a[0] * b[0] + a[1] * b[1] + a[2] * b[2];
+ if (d < -0.999999) return [1, 0, 0, 0];
+ const c: V3 = [
+ a[1] * b[2] - a[2] * b[1],
+ a[2] * b[0] - a[0] * b[2],
+ a[0] * b[1] - a[1] * b[0],
+ ];
+ const q: Quat = [c[0], c[1], c[2], 1 + d];
+ const len = Math.hypot(...q);
+ return [q[0] / len, q[1] / len, q[2] / len, q[3] / len];
+}
+
+/** Rotates `v` by unit quaternion `q`. */
+export function rotateByQuat(q: Quat, v: V3): V3 {
+ const t: V3 = [
+ 2 * (q[1] * v[2] - q[2] * v[1]),
+ 2 * (q[2] * v[0] - q[0] * v[2]),
+ 2 * (q[0] * v[1] - q[1] * v[0]),
+ ];
+ return [
+ v[0] + q[3] * t[0] + (q[1] * t[2] - q[2] * t[1]),
+ v[1] + q[3] * t[1] + (q[2] * t[0] - q[0] * t[2]),
+ v[2] + q[3] * t[2] + (q[0] * t[1] - q[1] * t[0]),
+ ];
+}
+
+interface Candidates {
+ /** Accepted triangle indices and their cumulative areas. */
+ tris: Int32Array;
+ cumulative: Float64Array;
+ normals: Float64Array;
+ minY: number;
+ maxY: number;
+ acceptedArea: number;
+}
+
+/** Face normal of triangle `t`, and its area. */
+function face(tri: Float64Array, t: number): { n: V3; area: number } {
+ const o = t * 9;
+ const ex = tri[o + 3]! - tri[o]!;
+ const ey = tri[o + 4]! - tri[o + 1]!;
+ const ez = tri[o + 5]! - tri[o + 2]!;
+ const fx = tri[o + 6]! - tri[o]!;
+ const fy = tri[o + 7]! - tri[o + 1]!;
+ const fz = tri[o + 8]! - tri[o + 2]!;
+ const c: V3 = [ey * fz - ez * fy, ez * fx - ex * fz, ex * fy - ey * fx];
+ const len = Math.hypot(...c);
+ return {
+ n: len > 0 ? [c[0] / len, c[1] / len, c[2] / len] : [0, -1, 0],
+ area: len / 2,
+ };
+}
+
+/** The triangles the slope and altitude filters keep, weighted by area. */
+function candidates(tri: Float64Array, o: ScatterOptions): Candidates {
+ const count = tri.length / 9;
+ let minY = Number.POSITIVE_INFINITY;
+ let maxY = Number.NEGATIVE_INFINITY;
+ for (let i = 1; i < tri.length; i += 3) {
+ minY = Math.min(minY, tri[i]!);
+ maxY = Math.max(maxY, tri[i]!);
+ }
+ const spanY = maxY - minY > 1e-9 ? maxY - minY : 1;
+ const [s0, s1] = o.slopeRange ?? [0, 89];
+ const [e0, e1] = o.elevationRange ?? [0, 1];
+ const keep: number[] = [];
+ const cumulative: number[] = [];
+ const normals: number[] = [];
+ let total = 0;
+ let all = 0;
+ for (let t = 0; t < count; t++) {
+ const { n, area } = face(tri, t);
+ all += area;
+ if (area <= 0) continue;
+ const slope = (Math.acos(Math.max(-1, Math.min(1, -n[1]))) * 180) / Math.PI;
+ const cy = (tri[t * 9 + 1]! + tri[t * 9 + 4]! + tri[t * 9 + 7]!) / 3;
+ const elevation = (maxY - cy) / spanY;
+ if (slope < s0 || slope > s1 || elevation < e0 || elevation > e1) continue;
+ total += area;
+ keep.push(t);
+ cumulative.push(total);
+ normals.push(...n);
+ }
+ return {
+ tris: Int32Array.from(keep),
+ cumulative: Float64Array.from(cumulative),
+ normals: Float64Array.from(normals),
+ minY,
+ maxY,
+ acceptedArea: all > 0 ? total / all : 0,
+ };
+}
+
+/** Index of the first cumulative area above `x`. */
+function pick(cumulative: Float64Array, x: number): number {
+ let lo = 0;
+ let hi = cumulative.length - 1;
+ while (lo < hi) {
+ const mid = (lo + hi) >> 1;
+ if (cumulative[mid]! > x) hi = mid;
+ else lo = mid + 1;
+ }
+ return lo;
+}
+
+/** Whether exclusion zones let a point at (x, z) grow, given a uniform draw `r`. */
+function allowed(
+ zones: ScatterExclusion[] | undefined,
+ x: number,
+ z: number,
+ r: number,
+): boolean {
+ for (const e of zones ?? []) {
+ const d = Math.hypot(x - e.x, z - e.z) - e.radius;
+ if (d < 0) return false;
+ const f = e.falloff ?? 0;
+ if (f > 0 && d < f && r > d / f) return false;
+ }
+ return true;
+}
+
+/** A spatial hash of accepted points, for the Poisson-disk distance test. */
+class PoissonGrid {
+ private cells = new Map<string, V3[]>();
+ constructor(private spacing: number) {}
+
+ private key(x: number, y: number, z: number): string {
+ const s = this.spacing;
+ return `${Math.floor(x / s)},${Math.floor(y / s)},${Math.floor(z / s)}`;
+ }
+
+ fits(p: V3): boolean {
+ const s = this.spacing;
+ for (let dx = -1; dx <= 1; dx++)
+ for (let dy = -1; dy <= 1; dy++)
+ for (let dz = -1; dz <= 1; dz++) {
+ const near = this.cells.get(
+ this.key(p[0] + dx * s, p[1] + dy * s, p[2] + dz * s),
+ );
+ for (const q of near ?? []) {
+ if (Math.hypot(p[0] - q[0], p[1] - q[1], p[2] - q[2]) < s)
+ return false;
+ }
+ }
+ return true;
+ }
+
+ add(p: V3): void {
+ const k = this.key(...p);
+ const list = this.cells.get(k);
+ if (list) list.push(p);
+ else this.cells.set(k, [p]);
+ }
+}
+
+function orient(n: V3, o: ScatterOptions, rand: () => number): Quat {
+ const a = Math.max(0, Math.min(1, o.alignToNormal ?? 0.3));
+ const u: V3 = [
+ UP[0] + (n[0] - UP[0]) * a,
+ UP[1] + (n[1] - UP[1]) * a,
+ UP[2] + (n[2] - UP[2]) * a,
+ ];
+ const len = Math.hypot(...u) || 1;
+ const align = quatBetween(UP, [u[0] / len, u[1] / len, u[2] / len]);
+ const yaw = (o.randomYaw ?? true) ? rand() * Math.PI * 2 : 0;
+ // Turn around the instance's own up (-y) first, then lean onto the surface.
+ const spin: Quat = [0, -Math.sin(yaw / 2), 0, Math.cos(yaw / 2)];
+ return quatMul(align, spin);
+}
+
+/** Scatters points on `surface`. Same inputs, same points. */
+export function scatterOnSurface(
+ surface: ScatterSurface,
+ o: ScatterOptions,
+): ScatterResult {
+ const tri = surfaceTriangles(surface);
+ const c = candidates(tri, o);
+ const points: ScatterPoint[] = [];
+ const total = c.cumulative[c.cumulative.length - 1] ?? 0;
+ const count = Math.max(0, Math.floor(o.count));
+ if (total <= 0 || count === 0)
+ return { points, acceptedArea: c.acceptedArea };
+ const rand = createRandom(o.seed ?? 1);
+ const spacing = Math.max(0, o.minDistance ?? 0);
+ const poisson = spacing > 0 ? new PoissonGrid(spacing) : undefined;
+ const [sc0, sc1] = o.scaleRange ?? [1, 1];
+ const spanY = c.maxY - c.minY > 1e-9 ? c.maxY - c.minY : 1;
+ const attempts = count * 30;
+ for (let a = 0; a < attempts && points.length < count; a++) {
+ const k = pick(c.cumulative, rand() * total);
+ const t = c.tris[k]! * 9;
+ // Uniform on the triangle.
+ const r1 = Math.sqrt(rand());
+ const r2 = rand();
+ const w0 = 1 - r1;
+ const w1 = r1 * (1 - r2);
+ const w2 = r1 * r2;
+ const p: V3 = [0, 1, 2].map(
+ (d) => w0 * tri[t + d]! + w1 * tri[t + 3 + d]! + w2 * tri[t + 6 + d]!,
+ ) as V3;
+ const gate = rand();
+ const yawDraw = createRandom(Math.floor(rand() * 2 ** 31));
+ const scaleDraw = rand();
+ if (!allowed(o.exclude, p[0], p[2], gate)) continue;
+ if (poisson && !poisson.fits(p)) continue;
+ poisson?.add(p);
+ const n: V3 = [
+ c.normals[k * 3]!,
+ c.normals[k * 3 + 1]!,
+ c.normals[k * 3 + 2]!,
+ ];
+ points.push({
+ position: p,
+ normal: n,
+ rotation: orient(n, o, yawDraw),
+ scale: sc0 + (sc1 - sc0) * scaleDraw,
+ elevation: (c.maxY - p[1]) / spanY,
+ });
+ }
+ return { points, acceptedArea: c.acceptedArea };
+}
+
+/**
+ * Where an instance's origin goes so its lowest point (local max y, as up
+ * is -y) rests on the scattered point, sunk `sink` px into the surface.
+ */
+export function seatOnSurface(
+ point: ScatterPoint,
+ bottomY: number,
+ sink = 0,
+): V3 {
+ const lift = rotateByQuat(point.rotation, [
+ 0,
+ bottomY * point.scale - sink,
+ 0,
+ ]);
+ return [
+ point.position[0] - lift[0],
+ point.position[1] - lift[1],
+ point.position[2] - lift[2],
+ ];
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/sdf-geometry.ts b/packages/webgpu-renderers/src/renderer3d/sdf-geometry.ts
new file mode 100644
index 00000000..1caef94b
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/sdf-geometry.ts
@@ -0,0 +1,459 @@
+/**
+ * Signed distance field sculpting: primitives combined in order by hard or
+ * smooth boolean operations (union, subtract, intersect), meshed with Surface
+ * Nets into a watertight surface whose normals are the field's gradient.
+ * Colored primitives blend where their smooth joins blend.
+ *
+ * Local space follows the primitives': centered on the origin, up is -y;
+ * cylinders and capsules stand along y.
+ */
+import {
+ emptyArrays,
+ flatten,
+ type GeometryArrays,
+ pushTriangle,
+ pushVertex,
+ type Rgba,
+} from "./geometry-arrays.js";
+import { parseHexOrRgbColor } from "./text-extruder.js";
+
+export type SdfOp = "union" | "subtract" | "intersect";
+
+interface SdfPrimitiveBase {
+ /** How this shape combines with everything before it. Default "union". */
+ op?: SdfOp;
+ /** Blend radius of the join, px (0 = a hard edge). Default 0. */
+ smoothness?: number;
+ /** Center, px, in the sculpture's local space. Default [0, 0, 0]. */
+ center?: [number, number, number];
+ /** Euler degrees [x, y, z]. */
+ rotation?: [number, number, number];
+ /** Surface color where this shape forms the surface (cuts too). */
+ color?: string;
+}
+
+export type SdfPrimitive = SdfPrimitiveBase &
+ (
+ | { shape: "sphere"; radius: number }
+ | {
+ shape: "box";
+ size: [number, number, number];
+ /** Corner rounding, px. */
+ radius?: number;
+ }
+ | { shape: "cylinder" | "capsule"; radius: number; height: number }
+ | { shape: "torus"; radius: number; tube: number }
+ | {
+ shape: "gyroid";
+ /** Period of the lattice, px. */
+ scale: number;
+ /** Wall thickness, px. */
+ thickness: number;
+ }
+ );
+
+export interface SdfGeometryOptions {
+ type: "sdf";
+ primitives: SdfPrimitive[];
+ /** The sculpture is clipped to this box [width, height, depth], px. */
+ bounds: [number, number, number];
+ /** Cells along the longest side of `bounds`, 16..160. Default 64. */
+ resolution?: number;
+ /** Faceted shading. */
+ flatShaded?: boolean;
+ color?: string;
+ material?: "lit" | "unlit" | "toon";
+ shininess?: number;
+ roughness?: number;
+ metallic?: number;
+}
+
+type V3 = [number, number, number];
+
+interface Prepared {
+ prim: SdfPrimitive;
+ op: SdfOp;
+ k: number;
+ center: V3;
+ /** Rows of the inverse rotation (world → shape). */
+ inv: [V3, V3, V3] | null;
+ color: Rgba | undefined;
+}
+
+function rotationRows(deg: V3 | undefined): [V3, V3, V3] | null {
+ if (!deg || (deg[0] === 0 && deg[1] === 0 && deg[2] === 0)) return null;
+ const [x, y, z] = deg.map((d) => (d * Math.PI) / 180) as V3;
+ const [cx, sx, cy, sy, cz, sz] = [
+ Math.cos(x),
+ Math.sin(x),
+ Math.cos(y),
+ Math.sin(y),
+ Math.cos(z),
+ Math.sin(z),
+ ];
+ // R = Rz · Ry · Rx (the node transform order); its inverse is its transpose.
+ const r: [V3, V3, V3] = [
+ [cz * cy, cz * sy * sx - sz * cx, cz * sy * cx + sz * sx],
+ [sz * cy, sz * sy * sx + cz * cx, sz * sy * cx - cz * sx],
+ [-sy, cy * sx, cy * cx],
+ ];
+ return [
+ [r[0][0], r[1][0], r[2][0]],
+ [r[0][1], r[1][1], r[2][1]],
+ [r[0][2], r[1][2], r[2][2]],
+ ];
+}
+
+function shapeDistance(s: SdfPrimitive, x: number, y: number, z: number) {
+ switch (s.shape) {
+ case "sphere":
+ return Math.hypot(x, y, z) - s.radius;
+ case "box": {
+ const r = Math.max(0, s.radius ?? 0);
+ const qx = Math.abs(x) - s.size[0] / 2 + r;
+ const qy = Math.abs(y) - s.size[1] / 2 + r;
+ const qz = Math.abs(z) - s.size[2] / 2 + r;
+ const outside = Math.hypot(
+ Math.max(qx, 0),
+ Math.max(qy, 0),
+ Math.max(qz, 0),
+ );
+ return outside + Math.min(Math.max(qx, qy, qz), 0) - r;
+ }
+ case "cylinder": {
+ const dr = Math.hypot(x, z) - s.radius;
+ const dy = Math.abs(y) - s.height / 2;
+ return (
+ Math.min(Math.max(dr, dy), 0) +
+ Math.hypot(Math.max(dr, 0), Math.max(dy, 0))
+ );
+ }
+ case "capsule": {
+ const half = Math.max(0, s.height / 2 - s.radius);
+ const cy = Math.max(-half, Math.min(half, y));
+ return Math.hypot(x, y - cy, z) - s.radius;
+ }
+ case "torus":
+ return Math.hypot(Math.hypot(x, z) - s.radius, y) - s.tube;
+ case "gyroid": {
+ const f = (2 * Math.PI) / Math.max(1e-3, s.scale);
+ const g =
+ Math.sin(x * f) * Math.cos(y * f) +
+ Math.sin(y * f) * Math.cos(z * f) +
+ Math.sin(z * f) * Math.cos(x * f);
+ // |∇g| peaks near 1.5·f: a conservative distance estimate.
+ return Math.abs(g) / (1.5 * f) - s.thickness / 2;
+ }
+ }
+}
+
+interface Sample {
+ d: number;
+ color: Rgba;
+}
+
+/**
+ * The composite field. Joins follow Quilez's polynomial smooth min / max;
+ * the blend weight `h` they compute also mixes the two sides' colors.
+ */
+export class SdfField {
+ private prims: Prepared[];
+ private half: V3;
+ private base: Rgba;
+ readonly colored: boolean;
+
+ constructor(o: SdfGeometryOptions) {
+ this.base = parseHexOrRgbColor(o.color ?? "#ffffff");
+ this.prims = o.primitives.map((prim) => ({
+ prim,
+ op: prim.op ?? "union",
+ k: Math.max(0, prim.smoothness ?? 0),
+ center: prim.center ?? [0, 0, 0],
+ inv: rotationRows(prim.rotation),
+ color: prim.color ? parseHexOrRgbColor(prim.color) : undefined,
+ }));
+ this.colored = this.prims.some((p) => p.color);
+ this.half = [o.bounds[0] / 2, o.bounds[1] / 2, o.bounds[2] / 2];
+ }
+
+ private local(p: Prepared, x: number, y: number, z: number): number {
+ const dx = x - p.center[0];
+ const dy = y - p.center[1];
+ const dz = z - p.center[2];
+ if (!p.inv) return shapeDistance(p.prim, dx, dy, dz);
+ const [a, b, c] = p.inv;
+ return shapeDistance(
+ p.prim,
+ a[0] * dx + a[1] * dy + a[2] * dz,
+ b[0] * dx + b[1] * dy + b[2] * dz,
+ c[0] * dx + c[1] * dy + c[2] * dz,
+ );
+ }
+
+ sample(x: number, y: number, z: number): Sample {
+ let d = Number.POSITIVE_INFINITY;
+ let color = this.base;
+ for (const p of this.prims) {
+ const e = this.local(p, x, y, z);
+ const c = p.color ?? (p.op === "union" ? this.base : color);
+ // The other operand: inverted for a cut.
+ const o = p.op === "subtract" ? -e : e;
+ if (!Number.isFinite(d)) {
+ // Nothing to cut or intersect yet: the first union starts the shape.
+ if (p.op === "union") {
+ d = e;
+ color = c;
+ }
+ continue;
+ }
+ const k = p.k;
+ if (p.op === "union") {
+ const h = k > 0 ? clamp01(0.5 + (0.5 * (o - d)) / k) : o < d ? 0 : 1;
+ d = o + (d - o) * h - k * h * (1 - h);
+ color = mixRgba(c, color, h);
+ } else {
+ const h = k > 0 ? clamp01(0.5 + (0.5 * (d - o)) / k) : d > o ? 1 : 0;
+ d = o + (d - o) * h + k * h * (1 - h);
+ color = mixRgba(c, color, h);
+ }
+ }
+ // Clipped to the bounds box: the mesh always closes.
+ const [hx, hy, hz] = this.half;
+ const qx = Math.abs(x) - hx;
+ const qy = Math.abs(y) - hy;
+ const qz = Math.abs(z) - hz;
+ const box =
+ Math.hypot(Math.max(qx, 0), Math.max(qy, 0), Math.max(qz, 0)) +
+ Math.min(Math.max(qx, qy, qz), 0);
+ return { d: Math.max(d, box), color };
+ }
+
+ distance(x: number, y: number, z: number): number {
+ return this.sample(x, y, z).d;
+ }
+
+ /** Unit outward normal: the normalized central-difference gradient. */
+ normal(x: number, y: number, z: number, e: number): V3 {
+ const gx = this.distance(x + e, y, z) - this.distance(x - e, y, z);
+ const gy = this.distance(x, y + e, z) - this.distance(x, y - e, z);
+ const gz = this.distance(x, y, z + e) - this.distance(x, y, z - e);
+ const len = Math.hypot(gx, gy, gz) || 1;
+ return [gx / len, gy / len, gz / len];
+ }
+}
+
+const clamp01 = (v: number) => Math.min(1, Math.max(0, v));
+const mixRgba = (a: Rgba, b: Rgba, t: number): Rgba => [
+ a[0] + (b[0] - a[0]) * t,
+ a[1] + (b[1] - a[1]) * t,
+ a[2] + (b[2] - a[2]) * t,
+ a[3] + (b[3] - a[3]) * t,
+];
+
+interface Grid {
+ nx: number;
+ ny: number;
+ nz: number;
+ cell: number;
+ origin: V3;
+ values: Float32Array;
+}
+
+/** Field values on a lattice that covers the bounds with one cell to spare on every side. */
+function sampleGrid(field: SdfField, o: SdfGeometryOptions): Grid {
+ const res = Math.min(160, Math.max(16, Math.round(o.resolution ?? 64)));
+ const cell = Math.max(...o.bounds) / res;
+ const count = (s: number) => Math.ceil(s / cell) + 3;
+ const [nx, ny, nz] = [
+ count(o.bounds[0]),
+ count(o.bounds[1]),
+ count(o.bounds[2]),
+ ];
+ const origin: V3 = [
+ (-(nx - 1) * cell) / 2,
+ (-(ny - 1) * cell) / 2,
+ (-(nz - 1) * cell) / 2,
+ ];
+ const values = new Float32Array(nx * ny * nz);
+ for (let k = 0; k < nz; k++) {
+ for (let j = 0; j < ny; j++) {
+ for (let i = 0; i < nx; i++) {
+ values[(k * ny + j) * nx + i] = field.distance(
+ origin[0] + i * cell,
+ origin[1] + j * cell,
+ origin[2] + k * cell,
+ );
+ }
+ }
+ }
+ return { nx, ny, nz, cell, origin, values };
+}
+
+const CORNERS: V3[] = [
+ [0, 0, 0],
+ [1, 0, 0],
+ [0, 1, 0],
+ [1, 1, 0],
+ [0, 0, 1],
+ [1, 0, 1],
+ [0, 1, 1],
+ [1, 1, 1],
+];
+/** The cube's 12 edges as corner index pairs. */
+const EDGES: Array<[number, number]> = [
+ [0, 1],
+ [2, 3],
+ [4, 5],
+ [6, 7],
+ [0, 2],
+ [1, 3],
+ [4, 6],
+ [5, 7],
+ [0, 4],
+ [1, 5],
+ [2, 6],
+ [3, 7],
+];
+
+/** The surface point of a cell: the mean of its edges' zero crossings. */
+function cellPoint(g: Grid, i: number, j: number, k: number): V3 | null {
+ const vals = CORNERS.map(
+ ([a, b, c]) => g.values[((k + c) * g.ny + j + b) * g.nx + i + a]!,
+ );
+ let sum: V3 = [0, 0, 0];
+ let n = 0;
+ for (const [a, b] of EDGES) {
+ const va = vals[a]!;
+ const vb = vals[b]!;
+ if (va < 0 === vb < 0) continue;
+ const t = va / (va - vb);
+ const ca = CORNERS[a]!;
+ const cb = CORNERS[b]!;
+ sum = [
+ sum[0] + ca[0] + (cb[0] - ca[0]) * t,
+ sum[1] + ca[1] + (cb[1] - ca[1]) * t,
+ sum[2] + ca[2] + (cb[2] - ca[2]) * t,
+ ];
+ n++;
+ }
+ if (n === 0) return null;
+ return [
+ g.origin[0] + (i + sum[0] / n) * g.cell,
+ g.origin[1] + (j + sum[1] / n) * g.cell,
+ g.origin[2] + (k + sum[2] / n) * g.cell,
+ ];
+}
+
+/** Box-projected UVs: the face of `bounds` the normal points at. */
+function boxUv(p: V3, n: V3, bounds: V3): [number, number] {
+ const ax = Math.abs(n[0]);
+ const ay = Math.abs(n[1]);
+ const az = Math.abs(n[2]);
+ const u = (v: number, s: number) => v / s + 0.5;
+ if (ay >= ax && ay >= az) return [u(p[0], bounds[0]), u(p[2], bounds[2])];
+ if (ax >= az) return [u(p[2], bounds[2]), u(p[1], bounds[1])];
+ return [u(p[0], bounds[0]), u(p[1], bounds[1])];
+}
+
+/** One vertex per surface cell, normal from the field's gradient. */
+function placeVertices(
+ g: Grid,
+ field: SdfField,
+ o: SdfGeometryOptions,
+ out: GeometryArrays,
+): Int32Array {
+ const index = new Int32Array((g.nx - 1) * (g.ny - 1) * (g.nz - 1)).fill(-1);
+ for (let k = 0; k < g.nz - 1; k++) {
+ for (let j = 0; j < g.ny - 1; j++) {
+ for (let i = 0; i < g.nx - 1; i++) {
+ const p = cellPoint(g, i, j, k);
+ if (!p) continue;
+ const n = field.normal(p[0], p[1], p[2], g.cell * 0.5);
+ const color = field.colored
+ ? field.sample(p[0], p[1], p[2]).color
+ : undefined;
+ index[(k * (g.ny - 1) + j) * (g.nx - 1) + i] = pushVertex(
+ out,
+ p,
+ n,
+ boxUv(p, n, o.bounds),
+ color,
+ );
+ }
+ }
+ }
+ return index;
+}
+
+/**
+ * A quad for every lattice edge the surface crosses, joining the four
+ * cells around it; it faces from inside (negative) to outside.
+ */
+function stitch(g: Grid, index: Int32Array, out: GeometryArrays): void {
+ const cx = g.nx - 1;
+ const cy = g.ny - 1;
+ const cell = (i: number, j: number, k: number) =>
+ index[(k * cy + j) * cx + i]!;
+ const value = (i: number, j: number, k: number) =>
+ g.values[(k * g.ny + j) * g.nx + i]!;
+ const axes: Array<{
+ step: V3;
+ around: [V3, V3, V3, V3];
+ }> = [
+ {
+ step: [1, 0, 0],
+ around: [
+ [0, 0, 0],
+ [0, -1, 0],
+ [0, -1, -1],
+ [0, 0, -1],
+ ],
+ },
+ {
+ step: [0, 1, 0],
+ around: [
+ [0, 0, 0],
+ [0, 0, -1],
+ [-1, 0, -1],
+ [-1, 0, 0],
+ ],
+ },
+ {
+ step: [0, 0, 1],
+ around: [
+ [0, 0, 0],
+ [-1, 0, 0],
+ [-1, -1, 0],
+ [0, -1, 0],
+ ],
+ },
+ ];
+ for (let k = 1; k < g.nz - 1; k++) {
+ for (let j = 1; j < g.ny - 1; j++) {
+ for (let i = 1; i < g.nx - 1; i++) {
+ const v0 = value(i, j, k);
+ for (const { step, around } of axes) {
+ const v1 = value(i + step[0], j + step[1], k + step[2]);
+ if (v0 < 0 === v1 < 0) continue;
+ const q = around.map(([a, b, c]) => cell(i + a, j + b, k + c));
+ if (q.some((v) => v < 0)) continue;
+ // Each `around` cycle winds toward +axis; the surface faces
+ // along +axis where the field leaves the solid going that way.
+ const flip = v0 >= 0;
+ pushTriangle(out, q[0]!, q[1]!, q[2]!, flip);
+ pushTriangle(out, q[0]!, q[2]!, q[3]!, flip);
+ }
+ }
+ }
+ }
+}
+
+/** Builds the sculpture's vertex arrays. */
+export function buildSdf(o: SdfGeometryOptions): GeometryArrays {
+ const field = new SdfField(o);
+ const grid = sampleGrid(field, o);
+ const out = emptyArrays();
+ const index = placeVertices(grid, field, o, out);
+ stitch(grid, index, out);
+ return o.flatShaded ? flatten(out) : out;
+}
diff --git a/packages/webgpu-renderers/src/renderer3d/terrain-geometry.ts b/packages/webgpu-renderers/src/renderer3d/terrain-geometry.ts
new file mode 100644
index 00000000..400627b0
--- /dev/null
+++ b/packages/webgpu-renderers/src/renderer3d/terrain-geometry.ts
@@ -0,0 +1,424 @@
+/**
+ * Procedural terrain: a heightfield grid lifted by seeded fractal noise (or
+ * a caller's height samples), optionally warped sideways on its cliffs into
+ * overhangs, colored by altitude and slope, and closed underneath into a
+ * watertight block.
+ *
+ * Local space: x across `width`, z across `depth`, both centered; the valley
+ * floor (height 0) at y = 0 and the highest peak at y = -height (up is -y).
+ */
+import {
+ createNoise,
+ type FractalMode,
+ fractal2,
+ type Noise,
+ terrace,
+} from "../math3d/noise.js";
+import {
+ accumulateNormals,
+ emptyArrays,
+ flatten,
+ type GeometryArrays,
+ pushTriangle,
+ pushVertex,
+ type Rgba,
+} from "./geometry-arrays.js";
+import { parseHexOrRgbColor } from "./text-extruder.js";
+
+export interface TerrainNoiseOptions {
+ seed?: number;
+ /** Size of the largest landforms, px. Default: a third of the longer side. */
+ scale?: number;
+ /** Noise layers summed, 1..10. Default 6. */
+ octaves?: number;
+ /** Amplitude kept per octave, 0..1. Default 0.5. */
+ persistence?: number;
+ /** Frequency gained per octave, 1..4. Default 2. */
+ lacunarity?: number;
+ /** "fbm" rolling hills, "ridged" sharp crests, "terrace" stepped plateaus. Default "fbm". */
+ ridgeMode?: FractalMode;
+ /** Plateaus for "terrace", 2..32. Default 6. */
+ terraceSteps?: number;
+}
+
+export interface TerrainColorByElevation {
+ lowColor: string;
+ midColor?: string;
+ highColor: string;
+ /** Steep faces (slope cosine below `cliffThreshold`) blend to this. */
+ cliffColor?: string;
+ /** Slope cosine (1 = flat) below which cliffs show. Default 0.65. */
+ cliffThreshold?: number;
+ /** Normalized altitudes where low turns mid and mid turns high. Default [0.35, 0.7]. */
+ bands?: [number, number];
+}
+
+export interface TerrainGeometryOptions {
+ type: "terrain";
+ width: number;
+ depth: number;
+ /** Peak-to-valley relief, px. Default 300. */
+ height?: number;
+ /** Grid cells along x and z (or both), 1..512. Default 128. */
+ segments?: number | [number, number];
+ noise?: TerrainNoiseOptions;
+ /**
+ * Heights 0..1 on the (segments + 1)² grid, row by row from the near
+ * edge (-z) and left to right: a DEM, or a sampled height function.
+ * Replaces the noise.
+ */
+ heights?: number[];
+ /** Sideways warp of steep faces into overhangs, px. Default 0. */
+ overhangWarp?: number;
+ colorByElevation?: TerrainColorByElevation;
+ /** Faceted, low-poly shading. */
+ flatShaded?: boolean;
+ /** Thickness of the solid under the valley floor, px. Default 10% of `height`. */
+ base?: number;
+ color?: string;
+ material?: "lit" | "unlit" | "toon";
+ shininess?: number;
+ roughness?: number;
+ metallic?: number;
+}
+
+const clamp = (v: number, lo: number, hi: number) =>
+ Math.min(hi, Math.max(lo, v));
+const smooth = (e0: number, e1: number, x: number) => {
+ const t = clamp((x - e0) / (e1 - e0), 0, 1);
+ return t * t * (3 - 2 * t);
+};
+
+export function terrainSegments(o: TerrainGeometryOptions): [number, number] {
+ const s = o.segments ?? 128;
+ const [sx, sz] = typeof s === "number" ? [s, s] : s;
+ return [clamp(Math.round(sx), 1, 512), clamp(Math.round(sz), 1, 512)];
+}
+
+/** Normalized heights 0..1 on the (sx + 1) × (sz + 1) grid. */
+export function terrainHeights(o: TerrainGeometryOptions): Float32Array {
+ const [sx, sz] = terrainSegments(o);
+ const n = (sx + 1) * (sz + 1);
+ const out = new Float32Array(n);
+ if (o.heights) {
+ for (let i = 0; i < n; i++) out[i] = clamp(o.heights[i] ?? 0, 0, 1);
+ return out;
+ }
+ const nz = o.noise ?? {};
+ const noise = createNoise(nz.seed ?? 1);
+ const mode = nz.ridgeMode ?? "fbm";
+ const fractal = {
+ octaves: clamp(Math.round(nz.octaves ?? 6), 1, 10),
+ persistence: clamp(nz.persistence ?? 0.5, 0, 1),
+ lacunarity: clamp(nz.lacunarity ?? 2, 1, 4),
+ mode,
+ };
+ const scale = Math.max(1, nz.scale ?? Math.max(o.width, o.depth) / 3);
+ let lo = Number.POSITIVE_INFINITY;
+ let hi = Number.NEGATIVE_INFINITY;
+ for (let j = 0; j <= sz; j++) {
+ for (let i = 0; i <= sx; i++) {
+ const x = (i / sx - 0.5) * o.width;
+ const z = (j / sz - 0.5) * o.depth;
+ const h = fractal2(noise, x / scale, z / scale, fractal);
+ out[j * (sx + 1) + i] = h;
+ lo = Math.min(lo, h);
+ hi = Math.max(hi, h);
+ }
+ }
+ // Stretched to span exactly 0..1, so `height` is the true relief.
+ const span = hi - lo > 1e-9 ? hi - lo : 1;
+ const steps = clamp(Math.round(nz.terraceSteps ?? 6), 2, 32);
+ for (let k = 0; k < n; k++) {
+ const h = (out[k]! - lo) / span;
+ out[k] = mode === "terrace" ? terrace(h, steps) : h;
+ }
+ return out;
+}
+
+/**
+ * Height at grid-space (gx, gy) on the mesh's own triangles: each cell
+ * splits along its (i + 1, j)–(i, j + 1) diagonal, as `buildTop` does.
+ */
+function onTriangles(
+ h: Float32Array,
+ sx: number,
+ sz: number,
+ gx: number,
+ gy: number,
+): number {
+ const x = clamp(gx, 0, sx);
+ const y = clamp(gy, 0, sz);
+ const i = Math.min(Math.floor(x), sx - 1);
+ const j = Math.min(Math.floor(y), sz - 1);
+ const fx = x - i;
+ const fy = y - j;
+ const at = (a: number, b: number) => h[b * (sx + 1) + a]!;
+ if (fx + fy <= 1) {
+ const a = at(i, j);
+ return a + (at(i + 1, j) - a) * fx + (at(i, j + 1) - a) * fy;
+ }
+ const d = at(i + 1, j + 1);
+ return d + (at(i, j + 1) - d) * (1 - fx) + (at(i + 1, j) - d) * (1 - fy);
+}
+
+export interface TerrainSampler {
+ /** Surface y (local, up is -y) above local (x, z); clamped to the edges. */
+ heightAt(x: number, z: number): number;
+ /** Unit surface normal at local (x, z). */
+ normalAt(x: number, z: number): [number, number, number];
+ /** Normalized altitude 0..1 at local (x, z). */
+ elevationAt(x: number, z: number): number;
+}
+
+/**
+ * Samples the terrain's unwarped surface (exactly its triangles), so other layers (a camera, a
+ * video screen, a glTF model, a light) can be stood on it.
+ */
+export function createTerrainSampler(
+ o: TerrainGeometryOptions,
+): TerrainSampler {
+ const [sx, sz] = terrainSegments(o);
+ const h = terrainHeights(o);
+ const relief = o.height ?? 300;
+ const grid = (x: number, z: number): [number, number] => [
+ (x / o.width + 0.5) * sx,
+ (z / o.depth + 0.5) * sz,
+ ];
+ const elevationAt = (x: number, z: number) =>
+ onTriangles(h, sx, sz, ...grid(x, z));
+ const heightAt = (x: number, z: number) => -elevationAt(x, z) * relief;
+ return {
+ heightAt,
+ elevationAt,
+ normalAt(x, z) {
+ const ex = o.width / sx;
+ const ez = o.depth / sz;
+ const dx = (heightAt(x + ex, z) - heightAt(x - ex, z)) / (2 * ex);
+ const dz = (heightAt(x, z + ez) - heightAt(x, z - ez)) / (2 * ez);
+ // Surface y = f(x, z): the up-facing normal is (f_x, -1, f_z).
+ const len = Math.hypot(dx, 1, dz);
+ return [dx / len, -1 / len, dz / len];
+ },
+ };
+}
+
+interface Palette {
+ low: Rgba;
+ mid?: Rgba;
+ high: Rgba;
+ cliff?: Rgba;
+ threshold: number;
+ bands: [number, number];
+}
+
+function palette(c: TerrainColorByElevation | undefined): Palette | undefined {
+ if (!c) return undefined;
+ return {
+ low: parseHexOrRgbColor(c.lowColor),
+ mid: c.midColor ? parseHexOrRgbColor(c.midColor) : undefined,
+ high: parseHexOrRgbColor(c.highColor),
+ cliff: c.cliffColor ? parseHexOrRgbColor(c.cliffColor) : undefined,
+ threshold: clamp(c.cliffThreshold ?? 0.65, 0, 1),
+ bands: c.bands ?? [0.35, 0.7],
+ };
+}
+
+const mix = (a: Rgba, b: Rgba, t: number): Rgba => [
+ a[0] + (b[0] - a[0]) * t,
+ a[1] + (b[1] - a[1]) * t,
+ a[2] + (b[2] - a[2]) * t,
+ a[3] + (b[3] - a[3]) * t,
+];
+
+/** Biome color at normalized altitude `e` on a face whose up-ness is `up` (n·up). */
+function biome(p: Palette, e: number, up: number): Rgba {
+ const [b0, b1] = p.bands;
+ const blend = 0.06;
+ let c: Rgba;
+ if (p.mid) {
+ c = mix(p.low, p.mid, smooth(b0 - blend, b0 + blend, e));
+ c = mix(c, p.high, smooth(b1 - blend, b1 + blend, e));
+ } else {
+ const b = (b0 + b1) / 2;
+ c = mix(p.low, p.high, smooth(b - blend, b + blend, e));
+ }
+ if (p.cliff) {
+ c = mix(c, p.cliff, smooth(p.threshold + 0.08, p.threshold - 0.08, up));
+ }
+ return c;
+}
+
+/**
+ * Pushes steep faces sideways along their downhill direction by 3D noise
+ * that changes with altitude, so cliffs fold out over their own feet.
+ */
+function warpCliffs(
+ g: GeometryArrays,
+ topCount: number,
+ amount: number,
+ noise: Noise,
+ freq: number,
+): void {
+ const p = g.positions;
+ const n = g.normals;
+ for (let v = 0; v < topCount; v++) {
+ const k = v * 3;
+ const up = -n[k + 1]!;
+ const steep = 1 - up * up;
+ if (steep < 1e-3) continue;
+ const hl = Math.hypot(n[k]!, n[k + 2]!) || 1;
+ const w =
+ amount *
+ steep *
+ (0.5 +
+ 0.5 * noise.noise3(p[k]! * freq, p[k + 1]! * freq, p[k + 2]! * freq));
+ p[k] += (n[k]! / hl) * w;
+ p[k + 2] += (n[k + 2]! / hl) * w;
+ }
+}
+
+/** The top surface: one vertex per grid point. */
+function buildTop(
+ o: TerrainGeometryOptions,
+ h: Float32Array,
+ sx: number,
+ sz: number,
+): GeometryArrays {
+ const g = emptyArrays();
+ const relief = o.height ?? 300;
+ for (let j = 0; j <= sz; j++) {
+ for (let i = 0; i <= sx; i++) {
+ const x = (i / sx - 0.5) * o.width;
+ const z = (j / sz - 0.5) * o.depth;
+ // v = 0 on the far edge: a map draped on it reads upright from the front.
+ pushVertex(
+ g,
+ [x, -h[j * (sx + 1) + i]! * relief, z],
+ [0, -1, 0],
+ [i / sx, 1 - j / sz],
+ );
+ }
+ }
+ for (let j = 0; j < sz; j++) {
+ for (let i = 0; i < sx; i++) {
+ const a = j * (sx + 1) + i;
+ const b = a + 1;
+ const c = a + sx + 1;
+ const d = c + 1;
+ // cross(b - a, c - a) = cross(+x, +z) points up (-y).
+ pushTriangle(g, a, b, c);
+ pushTriangle(g, b, d, c);
+ }
+ }
+ accumulateNormals(g);
+ return g;
+}
+
+/** The top grid's rim, one side at a time: near (-z), far (+z), left (-x), right (+x). */
+function rimIndices(side: number, sx: number, sz: number): number[] {
+ const out: number[] = [];
+ if (side === 0) for (let i = 0; i <= sx; i++) out.push(i);
+ if (side === 1) for (let i = 0; i <= sx; i++) out.push(sz * (sx + 1) + i);
+ if (side === 2) for (let j = 0; j <= sz; j++) out.push(j * (sx + 1));
+ if (side === 3) for (let j = 0; j <= sz; j++) out.push(j * (sx + 1) + sx);
+ return out;
+}
+
+const OUTWARD: Array<[number, number, number]> = [
+ [0, 0, -1],
+ [0, 0, 1],
+ [-1, 0, 0],
+ [1, 0, 0],
+];
+
+/**
+ * Four walls from the top's rim down to y = base, and a floor fanned from
+ * its center to the walls' feet: a closed solid with no T-junctions.
+ */
+function closeBlock(
+ g: GeometryArrays,
+ sx: number,
+ sz: number,
+ base: number,
+ sideColor: Rgba | undefined,
+): void {
+ const p = g.positions;
+ const at = (v: number) => [p[v * 3]!, p[v * 3 + 1]!, p[v * 3 + 2]!] as const;
+ const feet: Array<[number, number][]> = [];
+ for (let side = 0; side < 4; side++) {
+ const n = OUTWARD[side]!;
+ const ring = rimIndices(side, sx, sz);
+ const pairs = ring.map((v, k): [number, number] => {
+ const pos = at(v);
+ const u = k / (ring.length - 1);
+ return [
+ pushVertex(g, pos, n, [u, 0], sideColor),
+ pushVertex(g, [pos[0], base, pos[2]], n, [u, 1], sideColor),
+ ];
+ });
+ // Rims run toward +x / +z: the near and right walls wind as built.
+ const flip = side === 1 || side === 2;
+ for (let k = 0; k + 1 < pairs.length; k++) {
+ const [t0, b0] = pairs[k]!;
+ const [t1, b1] = pairs[k + 1]!;
+ pushTriangle(g, t0, b0, t1, flip);
+ pushTriangle(g, t1, b0, b1, flip);
+ }
+ feet.push(pairs.map(([, b]) => [b, 0]));
+ }
+ // The floor's boundary loop: near → right → far (reversed) → left (reversed).
+ const loop = [
+ ...feet[0]!,
+ ...feet[3]!.slice(1),
+ ...feet[1]!.slice(0, -1).reverse(),
+ ...feet[2]!.slice(1, -1).reverse(),
+ ].map(([v]) => v);
+ const down = [0, 1, 0] as const;
+ let cx = 0;
+ let cz = 0;
+ for (const v of loop) {
+ cx += p[v * 3]! / loop.length;
+ cz += p[v * 3 + 2]! / loop.length;
+ }
+ const floorUv = (x: number, z: number): [number, number] => [x, z];
+ const center = pushVertex(
+ g,
+ [cx, base, cz],
+ down,
+ floorUv(0.5, 0.5),
+ sideColor,
+ );
+ const rim = loop.map((v) =>
+ pushVertex(g, at(v), down, floorUv(0, 0), sideColor),
+ );
+ for (let k = 0; k < rim.length; k++) {
+ pushTriangle(g, center, rim[(k + 1) % rim.length]!, rim[k]!);
+ }
+}
+
+/** Builds the terrain's vertex arrays. */
+export function buildTerrain(o: TerrainGeometryOptions): GeometryArrays {
+ const [sx, sz] = terrainSegments(o);
+ const h = terrainHeights(o);
+ const relief = o.height ?? 300;
+ let g = buildTop(o, h, sx, sz);
+ const topCount = g.positions.length / 3;
+ const warp = o.overhangWarp ?? 0;
+ if (warp > 0) {
+ const noise = createNoise((o.noise?.seed ?? 1) + 101);
+ warpCliffs(g, topCount, warp, noise, 3 / Math.max(1, relief));
+ accumulateNormals(g);
+ }
+ const pal = palette(o.colorByElevation);
+ if (pal) {
+ for (let v = 0; v < topCount; v++) {
+ g.colors.push(...biome(pal, h[v]!, -g.normals[v * 3 + 1]!));
+ }
+ }
+ const base = o.base ?? relief * 0.1;
+ const side = pal ? mix(pal.cliff ?? pal.low, [0, 0, 0, 1], 0.35) : undefined;
+ closeBlock(g, sx, sz, base, side);
+ if (o.flatShaded) g = flatten(g);
+ return g;
+}
diff --git a/packages/webgpu-renderers/src/shaders/mesh3d.ts b/packages/webgpu-renderers/src/shaders/mesh3d.ts
index 0d1875c0..362ac5c6 100644
--- a/packages/webgpu-renderers/src/shaders/mesh3d.ts
+++ b/packages/webgpu-renderers/src/shaders/mesh3d.ts
@@ -30,6 +30,10 @@ struct ModelUniforms {
substrateSheen : vec4<f32>, // x = sheen, yzw = sheenColor (sRGB)
substrateIridescence : vec4<f32>, // x = iridescence, y = thicknessNm, z = filmIor, w = triplanarUvFreq
maps : vec4<f32>, // x = normal scale (0 = no map), y = occlusion strength, z = has metallic-roughness map, w = has emissive map
+ uvRect : vec4<f32>, // base-color uv' = xy + uv * zw
+ wind : vec4<f32>, // x = amplitude rad (0 = calm), y = angular frequency, z = wave number per px, w = time s
+ windFrame : vec4<f32>, // xy = downwind (x, z), z = mesh base y, w = 1 / mesh height
+ crowd : vec4<f32>, // x = time s, z = joints per baked frame
};
struct SkinningUniforms {
@@ -88,6 +92,31 @@ fn rotateVecByQuat(v: vec3<f32>, q: vec4<f32>) -> vec3<f32> {
return v + q.w * t + cross(q.xyz, t);
}
+// Wind lean in radians at a plant rooted at ground point root: a steady
+// push plus a traveling wave and a faster flutter, so neighbors sway together
+// and the far field lags the near one.
+fn windAngle(root: vec2<f32>) -> f32 {
+ let w = model.wind;
+ let phase = dot(root, model.windFrame.xy) * w.z - w.w * w.y;
+ return w.x * (0.55 + 0.35 * sin(phase) + 0.1 * sin(phase * 2.7 + 1.3));
+}
+
+// Bends a vertex (mesh space, scale s) downwind: the base stays put,
+// the tip leans furthest, along an arc so the stem keeps its length.
+fn windBend(local: vec3<f32>, angle: f32, s: f32) -> vec3<f32> {
+ let h = max(model.windFrame.z - local.y, 0.0);
+ let k = clamp(h * model.windFrame.w, 0.0, 1.0);
+ let a = angle * k * k;
+ let downwind = vec3<f32>(model.windFrame.x, 0.0, model.windFrame.y);
+ return downwind * (sin(a) * h * s) + vec3<f32>(0.0, (1.0 - cos(a)) * h * s, 0.0);
+}
+
+// A crowd joint: two baked frames of the palette, blended.
+fn crowdJoint(base0: u32, base1: u32, t: f32, joint: u32) -> mat4x4<f32> {
+ return skinning.jointMatrices[base0 + joint] * (1.0 - t) +
+ skinning.jointMatrices[base1 + joint] * t;
+}
+
@vertex
fn vs_main(
in: VertexInput,
@@ -98,12 +127,33 @@ fn vs_main(
var localPos = in.position;
var localNorm = in.normal;
+ // Crowd instances (flag bit 3) carry a second record: (clip's first
+ // baked frame, its frames per loop, phase in frames, frames per second), and skin from a
+ // baked joint palette at their own point in their own clip.
+ let flags = u32(model.params.w);
+ let crowd = (flags & 8u) != 0u;
+ var palette0 = 0u;
+ var palette1 = 0u;
+ var paletteT = 0.0;
+ if (crowd) {
+ let anim = instances[instanceIndex * 2u + 1u].posScale;
+ let count = max(anim.y, 1.0);
+ let f = anim.z + model.crowd.x * anim.w;
+ let wrapped = f - floor(f / count) * count;
+ let f0 = floor(wrapped);
+ let jc = u32(model.crowd.z);
+ palette0 = (u32(anim.x) + u32(f0)) * jc;
+ // Each clip bakes count + 1 samples (its end too): no wrap inside a loop.
+ palette1 = (u32(anim.x) + u32(f0) + 1u) * jc;
+ paletteT = wrapped - f0;
+ }
+
// Linear Blend Skinning (LBS) on GPU
if (model.params.z > 0.5) {
- let j0 = i32(in.joints.x);
- let j1 = i32(in.joints.y);
- let j2 = i32(in.joints.z);
- let j3 = i32(in.joints.w);
+ let j0 = u32(in.joints.x);
+ let j1 = u32(in.joints.y);
+ let j2 = u32(in.joints.z);
+ let j3 = u32(in.joints.w);
let w0 = in.weights.x;
let w1 = in.weights.y;
@@ -111,12 +161,24 @@ fn vs_main(
let w3 = in.weights.w;
let totalWeight = w0 + w1 + w2 + w3;
- if (totalWeight > 0.001) {
- let skinMat = skinning.jointMatrices[j0] * w0 +
- skinning.jointMatrices[j1] * w1 +
- skinning.jointMatrices[j2] * w2 +
- skinning.jointMatrices[j3] * w3;
-
+ var skinMat = mat4x4<f32>();
+ if (crowd) {
+ // A node-attached (rigid) mesh bakes its node as joint 0.
+ if (totalWeight <= 0.001) {
+ skinMat = crowdJoint(palette0, palette1, paletteT, 0u);
+ } else {
+ skinMat = crowdJoint(palette0, palette1, paletteT, j0) * w0 +
+ crowdJoint(palette0, palette1, paletteT, j1) * w1 +
+ crowdJoint(palette0, palette1, paletteT, j2) * w2 +
+ crowdJoint(palette0, palette1, paletteT, j3) * w3;
+ }
+ } else if (totalWeight > 0.001) {
+ skinMat = skinning.jointMatrices[j0] * w0 +
+ skinning.jointMatrices[j1] * w1 +
+ skinning.jointMatrices[j2] * w2 +
+ skinning.jointMatrices[j3] * w3;
+ }
+ if (crowd || totalWeight > 0.001) {
let pos4 = skinMat * vec4<f32>(in.position, 1.0);
localPos = pos4.xyz;
@@ -131,6 +193,7 @@ fn vs_main(
var instColor = vec4<f32>(1.0);
var instEmissive = vec4<f32>(0.0);
+ let windy = model.wind.x != 0.0;
if ((u32(model.params.w) & 4u) != 0u) {
let a = instances[instanceIndex * 2u];
let b = instances[instanceIndex * 2u + 1u];
@@ -139,14 +202,21 @@ fn vs_main(
localPos = (instanceModel * vec4<f32>(localPos, 1.0)).xyz;
localNorm = (instanceNormal * vec4<f32>(localNorm, 0.0)).xyz;
} else if ((u32(model.params.w) & 1u) != 0u) {
- let inst = instances[instanceIndex];
+ let inst = instances[select(instanceIndex, instanceIndex * 2u, crowd)];
let scaledPos = localPos * inst.posScale.w;
- localPos = rotateVecByQuat(scaledPos, inst.quat) + inst.posScale.xyz;
+ var bend = vec3<f32>(0.0);
+ if (windy) {
+ bend = windBend(localPos, windAngle(inst.posScale.xz), inst.posScale.w);
+ }
+ localPos = rotateVecByQuat(scaledPos, inst.quat) + inst.posScale.xyz + bend;
localNorm = rotateVecByQuat(localNorm, inst.quat);
instColor = inst.color;
instEmissive = inst.emissive;
} else {
localPos.x += dummyInst * 0.0;
+ if (windy) {
+ localPos += windBend(localPos, windAngle(vec2<f32>(0.0)), 1.0);
+ }
}
let worldPos4 = model.modelMatrix * vec4<f32>(localPos, 1.0);
@@ -156,7 +226,7 @@ fn vs_main(
out.worldNormal = normalize(norm4.xyz);
out.clipPos = cam.viewProjMatrix * worldPos4;
- out.uv = in.uv;
+ out.uv = model.uvRect.xy + in.uv * model.uvRect.zw;
let hasMapUvs = (u32(model.params.w) & 2u) != 0u;
out.normalUv = select(in.uv, in.normalUv, hasMapUvs);
out.mrUv = select(in.uv, in.mrUv, hasMapUvs);
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index 5fe3759b..e01fa5b4 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -106,6 +106,9 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani
| `Layer3D.extrude / lathe / tube / line / points(options)` | `"model3d"` | Custom & path-driven 3D geometry. `extrude`: extrude a 2D SVG path `d` (with holes) or `contour`/`holes` (`depth`, `bevelThickness`, `bevelSize`, `bevelSegments`, `bevelColor`). `lathe`: revolve a 2D `points` profile `[r, y][]` around the Y axis (`segments`, `phiStart`, `phiLength`, `closed`, `flatShaded`). `tube`: sweep a solid or thick-walled hollow pipe (`radius`, `innerRadius`, `radiusTaper`, `closed`, `capped`) along a `Path3D` or 3D control `points` using rotation-minimizing frames. `line`: 3D polyline (`points` or `Path3D`, `dashSize`, `gapSize`, per-vertex `colors`). `points`: 3D point cloud (`points`, `pointSize` for lit sphere dots or `0` for `point-list`, per-point `colors`). |
| `Layer3D.instances(options)` | `"model3d"` | Hardware-instanced single or multi-variant mesh rendering in 1 GPU draw call. Options: `count`, `mesh` / `meshes` (array of Layer3D primitives or meshes), `instanceAt(index, frame): { position, rotation, scale, color, emissive }`, or bind directly to Rapier `Physics.world3d` bodies via `bodies: Body3D[]` with `emissiveBySpeed` and `colorRamp`. |
| `Layer3D.swarm(options)` | `"model3d"` | Continuous 3D GPU particle swarm with compute-driven physics, 3D curl turbulence, vortex/cyclone field (`swirl`, `pull`, `axial`), ground plane collisions with Coulomb sliding friction and rotational roll torque, kinetic heat accumulation/cooling, and zero-copy packing directly into GPU instance buffers. |
+| `Layer3D.terrain(options)` | `"model3d"` | Procedural landscape, closed underneath. `width`, `depth`, `height` (relief px above the node's y, the valley floor), `segments`, `noise: { seed, scale, octaves, persistence, lacunarity, ridgeMode: "fbm" \| "ridged" \| "terrace", terraceSteps }`, or `heightFn(u, v) => 0..1` / `heights`, `overhangWarp`, `colorByElevation: { lowColor, midColor, highColor, cliffColor, cliffThreshold, bands }`, `flatShaded`. Returns a node with `heightAt(x, z)`, `normalAt(x, z)`, `elevationAt(x, z)` to stand cameras, lights, video screens and models on the ground. Static `Physics.world3d` bodies collide with its exact triangles. |
+| `Layer3D.sdf(options)` | `"model3d"` | Boolean sculpture: `primitives: [{ shape: "sphere" \| "box" \| "cylinder" \| "capsule" \| "torus" \| "gyroid", op: "union" \| "subtract" \| "intersect", smoothness (px blend), center, rotation, color, …size }]` meshed watertight inside `bounds: [w, h, d]` at `resolution` (16..160). Colors blend across smooth joins; a colored cutter paints its cut. |
+| `Layer3D.scatter(options)` | `"model3d"` | Scatter `mesh` (one or an array of variants, any Layer3D mesh or `Layer.model`) over `surface` (terrain, sdf, primitive or model file): `count`, `minDistance` (Poisson spacing), `seed`, `slopeRange` (deg from level), `elevationRange` (0..1), `alignToNormal`, `scaleRange`, `randomYaw`, `exclude: [{ x, z, radius, falloff }]`, `sink`, `wind`. Deterministic, one draw call per variant; `.points()` returns where they stand. |
---
@@ -401,6 +404,52 @@ Where depth jumps (an object's outline) the surface stretches across with the ed
---
+### Characters: blending, IK, look-at, springs, crowds
+
+glTF characters (`Layer.glb` / `Layer.gltf`) take layered clips and procedural constraints. Joint names come from the file. Targets are scene coordinates: a point, keys `[[frame, x, y, z]]`, or `(frame) => [x, y, z]`.
+
+```ts
+const hero = Layer.glb("hero.glb", {
+ x: 960, y: 900, z: 0, rotateX: 180, scale: 3,
+ animationLayers: [
+ { clip: "Walk" },
+ { clip: "Run", weight: [[0, 0], [48, 1]] }, // crossfade walk → run
+ { clip: "Wave", mask: ["Spine2"], weight: 0.8 }, // upper body only
+ ],
+ lookAt: [{ joint: "Head", target: (f) => [700 + f * 4, 600, -200], maxAngleDeg: 60 }],
+ ik: [{ chain: ["UpperArm.R", "LowerArm.R", "Hand.R"], target: [1100, 650, -80], poleTarget: [1200, 800, 200] }],
+ springBones: [{ joints: ["Tail1", "Cape"], stiffness: 4, damping: 0.6, gravity: [0, 980, 0] }],
+});
+// 3,000 walkers in one draw call per mesh, each at its own phase and speed:
+const extras = Layer3D.scatter({
+ surface: land, mesh: Layer.glb("walker.glb", { rotateX: 180, scale: 2 }), count: 3000, minDistance: 40,
+ crowd: { clips: ["Walk", "Idle"], speedRange: [0.8, 1.2] },
+});
+```
+
+Rules: a crowd mesh keeps its own `rotateX` / `scale` under each instance (y-up glTF: `rotateX: 180`). Spring bones follow the clip's motion and the model's current orientation, not the layer's keyframed travel. Per-instance `clip`, `animOffset`, `animSpeed` in `transforms` / `instanceAt` override the crowd's seeded choices.
+
+### Procedural worlds: terrain, sculptures, scatter
+
+Terrain, SDF sculptures and scatters are ordinary model nodes: they take materials, lights, shadows, SSR/GI, physics, keyframes and a `texture` that can be any layer (an image path, `Layer.video`, a layout tree) drawn fresh each frame.
+
+```ts
+const land = Layer3D.terrain({
+ x: 960, y: 900, z: 0, width: 4000, depth: 4000, height: 600,
+ noise: { seed: 3, ridgeMode: "ridged" },
+ colorByElevation: { lowColor: "#3d6b2a", midColor: "#7a6a4f", highColor: "#f4f4f4", cliffColor: "#5a5048" },
+});
+const pines = Layer3D.scatter({
+ surface: land, mesh: [Layer.gltf("pine.glb"), Layer3D.capsule({ radius: 8, length: 60, color: "#2f5d2a" })],
+ count: 800, minDistance: 60, slopeRange: [0, 30], elevationRange: [0.05, 0.6],
+ exclude: [{ x: 960, z: 0, radius: 300, falloff: 120 }], wind: { amplitudeDeg: 6 },
+});
+// Stand any layer on the ground: a video screen in the clearing.
+const screen = Layer3D.plane({ x: 960, y: land.heightAt(960, 0) - 160, z: 0, width: 480, height: 270, children: [Layer.video("clip.mp4", { width: 480, height: 270 })] });
+```
+
+Rules: the terrain's y is its valley floor, peaks rise `height` px above it (up is -y); `heightAt` follows the node's position and scale but not its rotation. Scatter reads the surface's static transform, so don't keyframe the surface. Make a terrain or sculpture `type: "fixed"` in `Physics.world3d` for exact-triangle collisions. `lint()` warns when `slopeRange` / `elevationRange` reject the whole surface.
+
## Scenes & Transitions
`Layer.sequence` places full-frame scenes end to end, overlapping each pair by its transition, and writes the scenes' `startFrame` / `durationFrames` for you (each scene keys from its own frame 0). Transitions are ordinary keyframes and mattes merged into the scenes.
diff --git a/specs/ue58-engine-roadmap.md b/specs/ue58-engine-roadmap.md
index 5821a8cf..69933b74 100644
--- a/specs/ue58-engine-roadmap.md
+++ b/specs/ue58-engine-roadmap.md
@@ -280,6 +280,8 @@ export interface Scatter3DOptions extends Omit<Instances3DOptions, "count" | "in
```
- Samples surface triangles weighted by area, filters by `slopeRange` / `elevationRange` / `exclude`, enforces Poisson-disk spacing via a spatial hash grid, and outputs an `Instances3DNode` rendered in 1 GPU draw call per mesh variant.
+**Status (implemented):** `renderer3d/terrain-geometry.ts`, `sdf-geometry.ts`, `scatter.ts` (pure, deterministic, unit-tested for watertightness and placement), `math3d/noise.ts` (seeded improved Perlin), SDK `framefields/src/procedural3d.ts`, compositor `renderers/procedural3d.ts`, schemas in `compositions/src/program/procedural-schema.ts`. Deviations and additions: (1) Terrain's valley floor sits at the node's y and the field is normalized so `height` is the exact relief; the sheet closes into a solid block (`base`), so it is watertight. `heightFn` is sampled into `heights` (the spec stays JSON); the node carries `heightAt` / `normalAt` / `elevationAt`, exact on the mesh's own triangles, for standing any other layer on the ground. (2) `overhangWarp` pushes steep faces sideways along their downhill direction by altitude-varying 3D noise (folds, not caves; caves belong to `Layer3D.sdf`). (3) SDF meshing is Surface Nets with topological (not geometric) winding; every primitive may carry `color`, blended by the smooth min/max weight, and `rotation`; the field is clipped to `bounds` so the mesh always closes. Gyroids use a conservative distance estimate. (4) Scatter runs in the compositor from the surface node's spec (so loaded model files work as surfaces), cached per scatter; `elevationRange` is normalized 0..1 over the surface's vertical extent; instances stand on their variant's base (`sink` adjusts). `.points()` exposes the placement in the SDK. (5) Wind is a `wind` option on any model node, bent in the mesh vertex shader by height above the mesh base (a traveling-wave lean plus flutter), so shadows sway too. (6) Integration with other layers: `texture` on every Layer3D mesh (and `textureLayer` on `Layer.model`) takes an image path or any layer (video, image, layout), hoisted like a matte and drawn offscreen each frame, then bound as the mesh's base-color map through its rect (`uvRect`). (7) Physics: `shape: "mesh"` (auto for fixed / kinematic terrain and SDF bodies) builds an exact Rapier trimesh. (8) Lint: `scatter_filters_reject_all` (framefields supplies the mesher through `LintOptions.scatterCoverage`, keeping the program module geometry-free) and `volumetric_fog_without_lights`. Fixed in passing: instanced meshes ignored their own material color.
+
---
## Phase 5: Character Animation Blending, IK, Spring Bones & Skinned VAT Crowds
@@ -344,6 +346,8 @@ To render 500–10,000 animated characters (robots, soldiers, pedestrians) in 1
3. In `mesh3d.wgsl` (`vs_main`), when both `isInstanced` and `isSkinned` are active, each vertex fetches its 4 influencing bone matrices from the baked joint palette storage/texture buffer at the instance's `(clipIndex, frame * speed + phaseOffset)`.
- Result: **5,000 independently walking/running skinned 3D characters with shadows, outlines, and bloom in a single draw call**.
+**Status (implemented):** rig evaluator `webgpu-renderers/src/loaders/rig.ts` (pure, unit-tested), crowd baking `loaders/crowd.ts`, compositor `renderers/character.ts`, schemas `compositions/src/program/character-schema.ts`. The field is `animationLayers` (`animation` already holds keyframe tracks). Deviations: (1) Layers blend in order, each overriding the pose beneath by its weight (slerp per joint) from the rest pose, so a crossfade is two layers with the top one's weight keyed; weights and targets are tracks: a constant, linear `[[frame, …]]` keys (JSON) or a function (in process). (2) Look-at runs before IK (a head turns, then limbs reach); look-at takes the joint's local `axis` (default glTF +z). IK targets and poles are scene coordinates, mapped through the inverse of the model's transform; the pole twists the solved chain about root→target. (3) Spring bones simulate in model space (follow-through from the clips, gravity mapped by the model's current orientation), replayed from frame 0 and cached per layer; stiffness is a per-second pull toward the animated pose, damping a per-second velocity loss. (4) Crowds bake a palette per mesh into a storage buffer (the mesh's skinning binding), not a texture; each clip bakes `count + 1` samples so playback interpolates to the loop point without wrapping; rigid node-attached meshes bake their node as joint 0. Each crowd instance carries a second 64-byte record (clip range, phase, playback rate) instead of packing bits into `emissive.w`. (5) Instance and scatter variants now apply their own node rotation and scale under each instance's, so a y-up glTF (`rotateX: 180`) stands upright in crowds and scatters.
+
---
## Phase 6: Voronoi Mesh Fracture & Per-Vertex Velocity Motion Blur