Code-first video & image, rendered natively on WebGPU.
fix(3d): render each 3D scene in isolation; stop per-frame GPU leaks
okanasl01954c0c2ad2
diff --git a/.changeset/scene-isolation.md b/.changeset/scene-isolation.mdnew file mode 100644index 00000000..a14d2061--- /dev/null+++ b/.changeset/scene-isolation.md@@ -0,0 +1,8 @@+---+"@framefields/compositions": minor+"@framefields/node-compositor": patch+"@framefields/webgpu-renderers": patch+"framefields": minor+---++3D scenes render in isolation. A box with a fixed size that holds a camera is its own scene: its camera, lights and environment see only its layers, it runs on its own clock, and it is placed like a picture, so `Layer.sequence` transitions between 3D scenes blend two finished pictures instead of drawing both through one camera. Signal tracks take `{ clock: "layer" }` to sample at the layer's own frame. `comp.lint()` reports `duplicate_id`. Per-pass 3D buffers are recycled once per frame (per command encoder), so a frame holding several 3D scenes never reuses a buffer an earlier pass still reads, and bloom mip chains live until the frame is done. Size-dependent 3D render targets (depth, HDR, MSAA, reflections) are cached per size and kept for the frame, so scenes of different sizes can share a frame.diff --git a/nodes/node-compositor/SKILL.md b/nodes/node-compositor/SKILL.mdindex 54a2d4d6..2094abb8 100644--- a/nodes/node-compositor/SKILL.md+++ b/nodes/node-compositor/SKILL.md@@ -210,6 +210,7 @@ Each track represents animatable property modifications:- **Watermarking a Video:** a `flex` row (`align: "end"`, `justify: "end"`, full canvas) containing a `media` node bound to the PNG input; low `opacity`.- **Picture-in-Picture:** a `flex` row with two `media` nodes (`fit: "cover"`, each `grow: 1`).- **Audio-Reactive 3D Models:** a `kind: "model3d"` node loading an OBJ/FBX/GLTF/STL/PLY mesh with `audioDeform: { audioTrackId, mode: "normal_extrusion" | "radial_pulse" | "harmonic_wave" | "twist" | "ripple", amplitudeMultiplier, damping }`, deformed directly in VRAM via WebGPU compute shaders driven by audio frequency spectra and transient energies. `twist` turns each slice about Y by its height (harder on the bass); `ripple` runs concentric waves across XY, struck by the drums. `audioTrackId` may be an audio layer's id or an audio file path.+- **Several 3D scenes:** a `box`/`flex`/`block` with a numeric `width` and `height` that holds a `camera` and the layers it films renders as its own scene (one holding only a camera stays a shot in a camera cut list): a nested composition with only its own camera, lights and environment, on its own clock (children start from the box's frame 0), placed in its parent like a media layer. Transitions, opacity and mattes on the box then blend finished pictures. Without that, every 3D layer shares one world and the active camera (the latest to start).- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; directional and spot lights can each cast shadows, up to the device texture-array layer limit.- **Transparent meshes:** glTF `BLEND` materials and meshes with reduced layer, material, tint, or vertex alpha draw after opaque items, back to front by the camera-space center of each mesh’s transformed bounds, without writing depth. This is object-level sorting: intersecting meshes, self-overlap, skinned/deformed bounds, translucent flat layers, glass, and transparent shadows have separate limitations.- **Mesh wireframe:** `wireframe: true` on a `model3d` draws triangle edges as one-pixel lines, including triangulation and back edges; it also applies to skinned and audio-deformed meshes and their shadows.diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.tsindex 13f21780..a6b0ab7b 100644--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts@@ -225,6 +225,9 @@ const swarmStateMap = new WeakMap<object, Swarm3DState>();const deviceRenderer3DCache = new WeakMap<GPUDevice, Map<string, Renderer3D>>();/** One Renderer3D per device, target format and MSAA sample count. */+/** The command encoder the 3D pools were last recycled for, per device. */+const poolEncoder = new WeakMap<GPUDevice, GPUCommandEncoder>();+function getRenderer3D(device: GPUDevice,format: GPUTextureFormat,@@ -990,13 +993,18 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {ctx.renderer.format ?? "rgba8unorm",sampleCount3d,);- r3d.resetPools();-const relightPipe = getScreenSpaceRelightPipeline(ctx.device,ctx.renderer.format ?? "rgba8unorm",);- relightPipe.resetPools();+ // The pools recycle per-pass uniforms once per frame, and a frame is one+ // command encoder: a nested scene drawn into the same encoder must not+ // recycle buffers the passes encoded before it still read.+ if (poolEncoder.get(ctx.device) !== encoder) {+ poolEncoder.set(ctx.device, encoder);+ r3d.resetPools();+ relightPipe.resetPools();+ }// 2. Initial clearconst clearColor = parseColor(vop.backgroundColor);diff --git a/packages/compositions/src/program/animation.ts b/packages/compositions/src/program/animation.tsindex 72e0ad3d..1165b203 100644--- a/packages/compositions/src/program/animation.ts+++ b/packages/compositions/src/program/animation.ts@@ -4,6 +4,7 @@ import type {AnimationTrack,EaseRef,LayerAnimationSpec,+ SignalClock,TrackSource,} from "./schema.js";@@ -66,6 +67,13 @@ export interface SignalTrackOptions {mode?: "continuous" | "accumulate";threshold?: number;debounceFrames?: number;+ /**+ * `"composition"` (default) samples the signal at the composition frame,+ * as audio analysis and shared signals need. `"layer"` samples it at the+ * layer's own frame, counted from its start like keyframes: for motion+ * authored per scene, which then plays right inside `Layer.sequence`.+ */+ clock?: SignalClock;}export interface ColorSignalOptions {@@ -963,6 +971,7 @@ export class LayerAnimation<TProp extends AnimatableProp = AnimatableProp>accumulateThreshold: options.threshold ?? 0.15,debounceFrames: options.debounceFrames ?? 2,channel: options.channel ?? "primary",+ ...(options.clock && { clock: options.clock }),};if (inline) attachInlineSignal(source, inline);diff --git a/packages/compositions/src/program/evaluator.test.ts b/packages/compositions/src/program/evaluator.test.tsindex 3bd807ef..69212095 100644--- a/packages/compositions/src/program/evaluator.test.ts+++ b/packages/compositions/src/program/evaluator.test.ts@@ -502,6 +502,32 @@ describe("Container Staggering", () => {});describe("Evaluator — Signal to Color Modulation", () => {+ it("reads a signal on the composition clock unless the track asks for its layer's", () => {+ const ramp = [0, 10, 20, 30, 40, 50];+ const track = (clock?: "composition" | "layer"): AnimationTrack => ({+ id: "sig-clock",+ prop: "x",+ source: {+ type: "signal",+ inputHandleId: "ramp",+ multiplier: 1,+ offset: 0,+ smoothingWindowFrames: 0,+ ...(clock && { clock }),+ },+ keyframes: [],+ });+ const at = (clock?: "composition" | "layer") =>+ evaluateTrackAtFrame(track(clock), 4, 30, {+ startFrame: 3,+ signals: { ramp },+ });+ // A layer starting on frame 3, seen on composition frame 4.+ expect(at()).toBe(40);+ expect(at("composition")).toBe(40);+ expect(at("layer")).toBe(10);+ });+it("interpolates colors based on signal amplitude", () => {const track: AnimationTrack = {id: "sig-color-1",diff --git a/packages/compositions/src/program/evaluator.ts b/packages/compositions/src/program/evaluator.tsindex c2bde79a..cc3ba07a 100644--- a/packages/compositions/src/program/evaluator.ts+++ b/packages/compositions/src/program/evaluator.ts@@ -787,7 +787,8 @@ export function evaluateTrackAtFrame(}case "signal": {- const sigVal = evaluateSignal(source, frame, fps, context.signals);+ const at = source.clock === "layer" ? localFrame : frame;+ const sigVal = evaluateSignal(source, at, fps, context.signals);if (COLOR_PROPS.has(track.prop)) {const src = source as {colorMode?: "interpolate" | "hueRotate" | "threshold";diff --git a/packages/compositions/src/program/lint.test.ts b/packages/compositions/src/program/lint.test.tsindex 2f9ce14a..5fdea5b5 100644--- a/packages/compositions/src/program/lint.test.ts+++ b/packages/compositions/src/program/lint.test.ts@@ -8,6 +8,14 @@ const codes = (layout: unknown[], extra = {}) =>);describe("lintComposition", () => {+ it("flags two layers with one id, wherever they sit in the tree", () => {+ expect(+ codes([+ { id: "scene-a", kind: "box", children: [{ id: "hero", kind: "box" }] },+ { id: "scene-b", kind: "box", children: [{ id: "hero", kind: "box" }] },+ ]),+ ).toEqual(["duplicate_id"]);+ });it("flags unreadable colors and gradients", () => {expect(codes([diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.tsindex 053b382d..957b4040 100644--- a/packages/compositions/src/program/lint.ts+++ b/packages/compositions/src/program/lint.ts@@ -120,9 +120,20 @@ export function lintComposition(let has3D = false;const pbrNodes: string[] = [];const emojiText: string[] = [];+ const seenIds = new Set<string>();const visit = (node: AnyNode, backdrop: string | null, atRoot: boolean) => {const id = String(node.id ?? "?");+ if (typeof node.id === "string") {+ if (seenIds.has(node.id))+ push(+ "error",+ "duplicate_id",+ `Two layers share the id '${id}': the renderer keeps one and drops the other. Give every layer its own id (scenes reused in one composition need their own prefix).`,+ id,+ );+ seenIds.add(node.id);+ }if (options.emojiFont === false &&node.kind === "text" &&diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.tsindex abf5f343..0f9d1fc2 100644--- a/packages/compositions/src/program/schema.ts+++ b/packages/compositions/src/program/schema.ts@@ -469,6 +469,13 @@ export type Keyframe = z.infer<typeof KeyframeSchema>;export const SignalModeSchema = z.enum(["continuous", "accumulate"]);export type SignalMode = z.infer<typeof SignalModeSchema>;+/**+ * Which clock a signal track reads: the composition's (default), as audio+ * analysis and shared signals need, or its layer's own, from the layer's+ * start frame, as keyframes do: motion authored for one scene of a sequence.+ */+export const SignalClockSchema = z.enum(["composition", "layer"]);+export type SignalClock = z.infer<typeof SignalClockSchema>;const RawTrackSourceSchema = z.discriminatedUnion("type", [z.object({@@ -489,6 +496,7 @@ const RawTrackSourceSchema = z.discriminatedUnion("type", [accumulateThreshold: z.number().optional(),debounceFrames: z.number().int().min(0).optional(),channel: z.string().optional(),+ clock: SignalClockSchema.optional(),}),z.object({type: z.literal("wiggle"),@@ -530,6 +538,7 @@ export const TrackSourceSchema = z.preprocess((val: unknown) => {accumulateThreshold: thresh,debounceFrames: obj.debounceFrames ?? 2,channel: typeof obj.channel === "string" ? obj.channel : "primary",+ ...(obj.clock !== undefined && { clock: obj.clock }),};}return val;diff --git a/packages/compositions/src/program/to-virtual-media.test.ts b/packages/compositions/src/program/to-virtual-media.test.tsindex cb80967b..73bb77d4 100644--- a/packages/compositions/src/program/to-virtual-media.test.ts+++ b/packages/compositions/src/program/to-virtual-media.test.ts@@ -6,7 +6,7 @@ import {} from "@framefields/core";import { describe, expect, it } from "vitest";import type { CompositorProgramConfig, LayoutNode } from "./schema.js";-import { compositorToProgram } from "./to-virtual-media.js";+import { compositorToProgram, isScene3D } from "./to-virtual-media.js";const doc = {width: 1920,@@ -441,3 +441,75 @@ describe("compositorToProgram", () => {expect(textOp?.effects).toEqual([{ op: "FilmGrain", strength: 0.1 }]);});});++describe("compositorToProgram: 3D scenes", () => {+ const scene = (id: string, startFrame: number): LayoutNode =>+ ({+ id,+ kind: "box",+ position: "absolute",+ width: 1080,+ height: 1920,+ background: "#202020",+ startFrame,+ durationFrames: 60,+ children: [+ { id: `${id}-cam`, kind: "camera", z: -800 },+ {+ id: `${id}-cube`,+ kind: "box",+ is3D: true,+ width: 100,+ height: 100,+ startFrame: 10,+ },+ ],+ }) as unknown as LayoutNode;+ const ramp = [0, 1, 2];+ const program = compositorToProgram(+ {+ width: 1080,+ height: 1920,+ fps: 30,+ mode: "Video",+ layout: [scene("a", 0), scene("b", 50)],+ signals: { ramp },+ } as unknown as CompositorProgramConfig,+ { isVideoMode: true, fps: 30, durationMs: 4000 },+ );+ const [a, b] = program.children as VirtualMediaData[];++ it("places a container holding a camera as media whose content is its own composition", () => {+ const op = b.operation as Record<string, unknown>;+ expect(op.kind).toBe("media");+ expect(op.startFrame).toBe(50);+ expect(op.background).toBeUndefined();+ const nested = b.children?.[0];+ expect(nested?.operation.op).toBe("Compositor");+ expect((nested?.operation as Record<string, unknown>).backgroundColor).toBe(+ "#202020",+ );+ expect((nested?.operation as Record<string, unknown>).signals).toEqual({+ ramp,+ });+ });+ it("leaves a box holding only a camera as a shot over the shared world", () => {+ expect(+ isScene3D({+ id: "shot",+ kind: "box",+ width: 1080,+ height: 1920,+ children: [{ id: "shot-cam", kind: "camera" }],+ } as unknown as LayoutNode),+ ).toBe(false);+ });+ it("keeps the scene's layers on the scene's own clock", () => {+ const layers = b.children?.[0].children ?? [];+ const cube = layers.find(+ (l) => (l.operation as { id?: string }).id === "b-cube",+ );+ expect(cube?.operation.startFrame).toBe(10);+ expect(a.children?.[0].children?.length).toBe(2);+ });+});diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.tsindex 9d32fd80..9540c9cb 100644--- a/packages/compositions/src/program/to-virtual-media.ts+++ b/packages/compositions/src/program/to-virtual-media.ts@@ -64,6 +64,27 @@ export type MediaSignalResolver = (layer: MediaSignalLayer,) => ResolvedMediaTimeRemap;+/**+ * A container holding a camera and the layers it films is a 3D scene of its+ * own: its camera, lights and environment see only its layers. It needs a+ * fixed size to render at. A container holding only a camera (and lights) is+ * a shot in a camera cut list over the shared world, and stays one.+ */+export function isScene3D(node: LayoutNode): boolean {+ if (+ (node.kind !== "box" && node.kind !== "flex" && node.kind !== "block") ||+ typeof node.width !== "number" ||+ typeof node.height !== "number" ||+ !Array.isArray(node.children)+ )+ return false;+ const kids = node.children;+ return (+ kids.some((c) => c.kind === "camera") &&+ kids.some((c) => c.kind !== "camera" && c.kind !== "light")+ );+}+function nodeToVirtualMedia(node: LayoutNode,resolveMedia: CompositorToProgramOptions["resolveMedia"],@@ -224,6 +245,10 @@ function nodeToVirtualMedia(? Math.round(node.durationFrames): inheritedDurationFrames;const nodeChildren = node.children;+ // A 3D scene renders as a composition of its own on its own clock, which+ // this layer then places like a picture: transitions, opacity and mattes+ // on the layer blend whole scenes, each through its own camera.+ const scene = isScene3D(node);children = nodeChildren.map((c, idx) => {const delay = computeStaggerDelay(idx,@@ -231,7 +256,7 @@ function nodeToVirtualMedia(staggerFrames,staggerDirection,);- const childOffset = inheritedOffset + nodeStart + delay;+ const childOffset = scene ? delay : inheritedOffset + nodeStart + delay;return nodeToVirtualMedia(c,resolveMedia,@@ -242,6 +267,33 @@ function nodeToVirtualMedia(resolveSignal,);});+ if (scene) {+ const frames =+ nodeDuration ?? Math.round(((compDurationMs ?? 10000) / 1000) * fps);+ const durationMs = (frames / fps) * 1000;+ media = {+ metadata: {+ width: node.width as number,+ height: node.height as number,+ fps,+ durationMs,+ },+ operation: {+ op: "Compositor",+ width: node.width as number,+ height: node.height as number,+ fps,+ mode: "Video",+ dataType: "Video",+ volume: 1,+ ...(typeof node.background === "string" && {+ backgroundColor: node.background,+ }),+ },+ children,+ } as unknown as VirtualMediaData;+ children = [media];+ }}let dataType: DataType = "Image";@@ -346,6 +398,12 @@ function nodeToVirtualMedia(operation: {op: "CompositorLayer",...node,+ // A scene is placed as media: its nested composition paints its background.+ ...(isScene3D(node) && {+ kind: "media",+ background: undefined,+ children: undefined,+ }),startFrame,durationFrames: dur,dataType,@@ -428,6 +486,8 @@ export function compositorToProgram(),} as unknown as VirtualMediaData;+ shareSignals(rootMedia.children ?? [], programSignals);+const effectsList: ProgramEffect[] = Array.isArray(options.effects)? options.effects: Array.isArray(config.effects)@@ -471,10 +531,26 @@ export function compositorToProgram(return rootMedia;}-/** Recursively gather every node op from a render tree (in order). */+/** Hands every nested scene composition the program's signals, which its tracks read. */+function shareSignals(+ nodes: readonly VirtualMediaData[],+ signals: Record<string, unknown>,+) {+ for (const vm of nodes) {+ if (vm?.operation?.op === "Compositor")+ (vm.operation as { signals?: unknown }).signals = signals;+ shareSignals(vm?.children ?? [], signals);+ }+}++/**+ * Recursively gather every node op from a render tree (in order), stopping at+ * nested scene compositions: their layers belong to their own render.+ */export function collectNodeOps(nodes: VirtualMediaData[]): VirtualMediaData[] {const out: VirtualMediaData[] = [];const walk = (vm: VirtualMediaData) => {+ if (vm?.operation?.op === "Compositor") return;if (vm?.operation?.op === "CompositorLayer") {out.push(vm);}diff --git a/packages/framefields/src/render-memory.test.ts b/packages/framefields/src/render-memory.test.tsnew file mode 100644index 00000000..71861389--- /dev/null+++ b/packages/framefields/src/render-memory.test.ts@@ -0,0 +1,60 @@+import { describe, expect, it } from "vitest";+import {+ Composition,+ HeadlessMediaRenderer,+ Layer,+ Layer3D,+ LayerAnimation,+} from "./index.js";++type Proto = Record<string, (...args: unknown[]) => unknown>;++describe("render memory", () => {+ it("recycles 3D buffers from frame to frame instead of allocating new ones", async () => {+ const comp = new Composition({+ width: 200,+ height: 200,+ fps: 30,+ durationFrames: 60,+ backgroundColor: "#000000",+ });+ comp.add([+ Layer.camera({ x: 100, y: 100, z: -400, targetX: 100, targetY: 100 }),+ Layer.ambientLight("#ffffff", 1),+ ...[0, 1, 2, 3].map((i) =>+ Layer3D.roundedBox({+ x: 40 + i * 40,+ y: 100,+ z: 0,+ width: 30,+ height: 30,+ depth: 30,+ color: "#ff8800",+ }).animate(+ LayerAnimation.create().keys("rotateY", [+ [0, 0],+ [60, 360],+ ]),+ ),+ ),+ ]);+ const renderer = new HeadlessMediaRenderer();+ for (let f = 0; f < 4; f++) await comp.renderFrame({ frame: f, renderer });+ const proto = (globalThis as unknown as { GPUDevice: { prototype: Proto } })+ .GPUDevice.prototype;+ const createBuffer = proto.createBuffer;+ let created = 0;+ proto.createBuffer = function (...args: unknown[]) {+ created++;+ return createBuffer.apply(this, args);+ };+ try {+ for (let f = 4; f < 34; f++)+ await comp.renderFrame({ frame: f, renderer });+ } finally {+ proto.createBuffer = createBuffer;+ }+ // Thirty more frames of the same scene reuse the pools: next to nothing new.+ expect(created).toBeLessThan(30);+ });+});diff --git a/packages/framefields/src/scene-isolation-render.test.ts b/packages/framefields/src/scene-isolation-render.test.tsnew file mode 100644index 00000000..935c4a15--- /dev/null+++ b/packages/framefields/src/scene-isolation-render.test.ts@@ -0,0 +1,90 @@+import { Canvas, loadImage } from "skia-canvas";+import { describe, expect, it } from "vitest";+import {+ Composition,+ HeadlessMediaRenderer,+ Layer,+ Layer3D,+ Transition,+} from "./index.js";++async function centre(png: Buffer) {+ const img = await loadImage(png);+ const canvas = new Canvas(img.width, img.height);+ const ctx = canvas.getContext("2d");+ ctx.drawImage(img, 0, 0);+ const [r, g, b] = ctx.getImageData(img.width / 2, img.height / 2, 1, 1).data;+ return { r, g, b };+}++/** A scene that films one glowing cube at `x` through its own camera. */+function scene(x: number, color: string) {+ return Layer.box({+ width: 200,+ height: 200,+ background: "#000000",+ children: [+ Layer.camera({+ x,+ y: 100,+ z: -400,+ targetX: x,+ targetY: 100,+ targetZ: 0,+ }),+ Layer.ambientLight("#ffffff", 1),+ Layer3D.roundedBox({+ x,+ y: 100,+ z: 0,+ width: 160,+ height: 160,+ depth: 160,+ radius: 4,+ color,+ emissive: color,+ emissiveIntensity: 1,+ }),+ ],+ });+}++describe("3D scenes in a sequence", () => {+ it("renders each scene through its own camera, and blends them in a transition", async () => {+ const comp = new Composition({+ width: 200,+ height: 200,+ fps: 30,+ durationFrames: 50,+ backgroundColor: "#000000",+ });+ // The cubes stand far apart: one shared world filmed by either camera+ // would show only one of them.+ comp.add(+ Layer.sequence(+ [+ { scene: scene(100, "#ff0000"), duration: 30 },+ {+ scene: scene(5000, "#0000ff"),+ duration: 30,+ transition: Transition.crossfade(10),+ },+ ],+ { width: 200, height: 200 },+ ),+ );+ const renderer = new HeadlessMediaRenderer();+ const at = async (frame: number) =>+ centre(await comp.renderFrame({ frame, renderer }));+ const first = await at(10);+ const both = await at(25);+ const second = await at(40);+ expect(first.r).toBeGreaterThan(150);+ expect(first.b).toBeLessThan(60);+ expect(second.b).toBeGreaterThan(150);+ expect(second.r).toBeLessThan(60);+ // Mid-crossfade both cubes show, each through its own camera.+ expect(both.r).toBeGreaterThan(50);+ expect(both.b).toBeGreaterThan(50);+ });+});diff --git a/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.tsindex 6ddcaa9f..c1e2b621 100644--- a/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/bloom-pipeline.ts@@ -159,8 +159,13 @@ export class BloomPipeline {private module: GPUShaderModule;private pipelineLayout: GPUPipelineLayout;private sampler: GPUSampler;- private mips: GPUTexture[] = [];- private mipKey = "";+ /**+ * Mip chains by size and depth. Scenes with different bloom share a frame+ * (one command buffer), so a chain stays alive until the frame is done:+ * chains not used in a frame are destroyed at the next `resetPools`.+ */+ private chains = new Map<string, GPUTexture[]>();+ private usedThisFrame = new Set<string>();private placeholder: GPUTexture;private uniformPool = new BufferPool(48,@@ -267,17 +272,22 @@ export class BloomPipeline {return pipe;}- private ensureMips(width: number, height: number, levels: number): void {+ private mipChain(+ width: number,+ height: number,+ levels: number,+ ): GPUTexture[] {const key = `${width}x${height}x${levels}`;- if (key === this.mipKey) return;- for (const t of this.mips) t.destroy();- this.mips = [];+ this.usedThisFrame.add(key);+ const cached = this.chains.get(key);+ if (cached) return cached;+ const chain: GPUTexture[] = [];let w = width;let h = height;for (let i = 0; i < levels; i++) {w = Math.max(1, Math.floor(w / 2));h = Math.max(1, Math.floor(h / 2));- this.mips.push(+ chain.push(this.device.createTexture({label: `BloomMip${i}`,size: { width: w, height: h },@@ -288,7 +298,8 @@ export class BloomPipeline {);if (w === 1 && h === 1) break;}- this.mipKey = key;+ this.chains.set(key, chain);+ return chain;}private uniforms(@@ -371,8 +382,7 @@ export class BloomPipeline {mode: display ? 1 : 0,};const levels = Math.max(1, Math.min(8, Math.round(options.radius ?? 6)));- this.ensureMips(source.width, source.height, levels);- const mips = this.mips;+ const mips = this.mipChain(source.width, source.height, levels);const u = (w: number, h: number) => this.uniforms(1 / w, 1 / h, opts);const glowSource = hdrSource ?? source;@@ -420,13 +430,24 @@ export class BloomPipeline {);}- /** Call once per frame before `apply`: recycles the per-pass uniforms. */+ /**+ * Call once per frame before `apply`: recycles the per-pass uniforms and+ * frees the mip chains the last frame did not use.+ */resetPools(): void {this.uniformPool.reset();+ for (const [key, chain] of this.chains) {+ if (this.usedThisFrame.has(key)) continue;+ for (const t of chain) t.destroy();+ this.chains.delete(key);+ }+ this.usedThisFrame.clear();}destroy(): void {- for (const t of this.mips) t.destroy();+ for (const chain of this.chains.values())+ for (const t of chain) t.destroy();+ this.chains.clear();this.placeholder.destroy();this.uniformPool.destroy();}diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.tsindex 120a2548..166d7cf2 100644--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts@@ -418,7 +418,41 @@ export class Renderer3D {});}+ /**+ * Size-dependent render targets, by label, size, samples and usage. One+ * frame (one command buffer) can render 3D scenes of different sizes, so+ * a target is never destroyed while the frame may use it: targets the last+ * frame did not use are destroyed at the next `resetPools`.+ */+ private attachments = new Map<string, GPUTexture>();+ private attachmentsUsed = new Set<string>();+ private attachment(+ label: string,+ width: number,+ height: number,+ desc: { format: GPUTextureFormat; usage: number; sampleCount?: number },+ ): GPUTexture {+ const key = `${label}:${width}x${height}:${desc.sampleCount ?? 1}:${desc.usage}`;+ this.attachmentsUsed.add(key);+ let texture = this.attachments.get(key);+ if (!texture) {+ texture = this.device.createTexture({+ label,+ size: { width, height },+ ...desc,+ });+ this.attachments.set(key, texture);+ }+ return texture;+ }+resetPools(): void {+ for (const [key, texture] of this.attachments) {+ if (this.attachmentsUsed.has(key)) continue;+ texture.destroy();+ this.attachments.delete(key);+ }+ this.attachmentsUsed.clear();this.bloomPipeline?.resetPools();this.cameraPool.reset();// The cached camera buffer goes back to the pool and may be handed@@ -437,37 +471,27 @@ export class Renderer3D {}getOrCreateDepthTexture(width: number, height: number): GPUTexture {- if (- !this.depthTexture ||- this.depthTexture.width !== width ||- this.depthTexture.height !== height- ) {- this.depthTexture?.destroy();- this.depthTexture = this.device.createTexture({- label: "Renderer3DDepthTexture",- size: { width, height },+ this.depthTexture = this.attachment(+ "Renderer3DDepthTexture",+ width,+ height,+ {format: "depth24plus",sampleCount: this.sampleCount,usage: GPUTextureUsage.RENDER_ATTACHMENT,- });- }+ },+ );return this.depthTexture;}private getOrCreateMsaaTexture(- current: GPUTexture | null,+ _current: GPUTexture | null,label: string,format: GPUTextureFormat,width: number,height: number,): GPUTexture {- if (current && current.width === width && current.height === height) {- return current;- }- current?.destroy();- return this.device.createTexture({- label,- size: { width, height },+ return this.attachment(label, width, height, {format,sampleCount: this.sampleCount,usage: GPUTextureUsage.RENDER_ATTACHMENT,@@ -480,110 +504,85 @@ export class Renderer3D {}private getOrCreateHdrTexture(width: number, height: number): GPUTexture {- if (- !this.hdrTexture ||- this.hdrTexture.width !== width ||- this.hdrTexture.height !== height- ) {- this.hdrTexture?.destroy();- this.hdrTexture = this.device.createTexture({- label: "Renderer3DHdrRadiance",- size: { width, height },- format: "rgba16float",- usage:- GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,- });- }+ this.hdrTexture = this.attachment("Renderer3DHdrRadiance", width, height, {+ format: "rgba16float",++ usage:+ GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,+ });return this.hdrTexture;}getOrCreateLinearDepthTexture(width: number, height: number): GPUTexture {- if (- !this.linearDepthTexture ||- this.linearDepthTexture.width !== width ||- this.linearDepthTexture.height !== height- ) {- this.linearDepthTexture?.destroy();- this.linearDepthTexture = this.device.createTexture({- label: "Renderer3DLinearDepthTexture",- size: { width, height },+ this.linearDepthTexture = this.attachment(+ "Renderer3DLinearDepthTexture",+ width,+ height,+ {format: "rgba16float",+usage:GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,- });- }+ },+ );return this.linearDepthTexture;}getOrCreateReflectionTexture(width: number, height: number): GPUTexture {- if (- !this.reflectionColorTexture ||- this.reflectionColorTexture.width !== width ||- this.reflectionColorTexture.height !== height- ) {- this.reflectionColorTexture?.destroy();- this.reflectionColorTexture = this.device.createTexture({- label: "Renderer3DReflectionColor",- size: { width, height },+ this.reflectionColorTexture = this.attachment(+ "Renderer3DReflectionColor",+ width,+ height,+ {format: this.format,+usage:GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,- });- }+ },+ );return this.reflectionColorTexture;}getOrCreateReflectionMsaaTexture(width: number, height: number): GPUTexture {- if (- !this.reflectionMsaaColorTexture ||- this.reflectionMsaaColorTexture.width !== width ||- this.reflectionMsaaColorTexture.height !== height- ) {- this.reflectionMsaaColorTexture?.destroy();- this.reflectionMsaaColorTexture = this.device.createTexture({- label: "Renderer3DReflectionMsaaColor",- size: { width, height },+ this.reflectionMsaaColorTexture = this.attachment(+ "Renderer3DReflectionMsaaColor",+ width,+ height,+ {format: this.format,sampleCount: this.sampleCount,usage: GPUTextureUsage.RENDER_ATTACHMENT,- });- }+ },+ );return this.reflectionMsaaColorTexture;}getOrCreateReflectionDepthTexture(width: number, height: number): GPUTexture {- if (- !this.reflectionDepthTexture ||- this.reflectionDepthTexture.width !== width ||- this.reflectionDepthTexture.height !== height- ) {- this.reflectionDepthTexture?.destroy();- this.reflectionDepthTexture = this.device.createTexture({- label: "Renderer3DReflectionDepth",- size: { width, height },+ this.reflectionDepthTexture = this.attachment(+ "Renderer3DReflectionDepth",+ width,+ height,+ {format: "depth24plus",sampleCount: this.sampleCount,usage: GPUTextureUsage.RENDER_ATTACHMENT,- });- }+ },+ );return this.reflectionDepthTexture;}getOrCreateReflectionTempTexture(width: number, height: number): GPUTexture {- if (- !this.reflectionTempTexture ||- this.reflectionTempTexture.width !== width ||- this.reflectionTempTexture.height !== height- ) {- this.reflectionTempTexture?.destroy();- this.reflectionTempTexture = this.device.createTexture({- label: "Renderer3DReflectionTemp",- size: { width, height },+ this.reflectionTempTexture = this.attachment(+ "Renderer3DReflectionTemp",+ width,+ height,+ {format: this.format,+usage:GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,- });- }+ },+ );return this.reflectionTempTexture;}@@ -591,20 +590,17 @@ export class Renderer3D {width: number,height: number,): GPUTexture {- if (- !this.reflectionBlurredTexture ||- this.reflectionBlurredTexture.width !== width ||- this.reflectionBlurredTexture.height !== height- ) {- this.reflectionBlurredTexture?.destroy();- this.reflectionBlurredTexture = this.device.createTexture({- label: "Renderer3DReflectionBlurred",- size: { width, height },+ this.reflectionBlurredTexture = this.attachment(+ "Renderer3DReflectionBlurred",+ width,+ height,+ {format: this.format,+usage:GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TEXTURE_BINDING,- });- }+ },+ );return this.reflectionBlurredTexture;}@@ -2021,6 +2017,8 @@ export class Renderer3D {}destroy(): void {+ for (const texture of this.attachments.values()) texture.destroy();+ this.attachments.clear();this.bloomPipeline?.destroy();this.depthTexture?.destroy();for (const texture of this.shadowTextures.values()) texture.destroy();diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.mdindex 324bc747..a181dac1 100644--- a/plugins/framefields/skills/framefields-compose/SKILL.md+++ b/plugins/framefields/skills/framefields-compose/SKILL.md@@ -104,7 +104,7 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani| `Layer.environment(preset, options)` | `"light"` | The 3D scene's world: studio image-based light, exposure, tone mapping, fog, bloom. **Adding one switches the scene to physically based shading.** See *3D Look* below. || `Layer.sequence(entries, options)` | `"box"` | Scenes back to back with `Transition`s between them; returns a box whose `totalFrames` is the length. See *Scenes & Transitions*. || `Layer.model(options)` | `"model3d"` | 3D mesh loader (`.gltf`, `.glb`, `.obj`, `.fbx`, `.stl`, `.ply`). Supports skinning and audio deformation compute shaders. |-| `Layer3D.roundedBox / sphere / cylinder / capsule / torus / ring / prism / pyramid / polyhedron / icosahedron / icosphere / shard(options)` | `"model3d"` | Procedural solid meshes, centered on `x`/`y`/`z` (y is down, so "top" is -y). `roundedBox`: `width`, `height`, `depth`, `radius`, `segments`, `taper`. `sphere`: `radius`. `cylinder`: `radius` or `radiusTop`/`radiusBottom`, `height`, `openEnded`, `flatShaded`. `capsule`: `radius`, `height` (cylindrical shaft) or `totalHeight`. `torus`: `radius`, `tubeRadius`, `arc`. `ring`: flat annulus (`innerRadius`, `outerRadius`). `prism` / `pyramid`: solid capped flat-shaded N-gon prism or pyramid (`sides`, `radius`, `height`). `polyhedron` (`type: "tetrahedron" \| "octahedron" \| "dodecahedron" \| "icosahedron"`), `icosahedron`, `icosphere` (`radius`, `detail`, `flatShaded`). `shard`: procedural flat-faceted polyhedron rock/crystal (`radius`, `cuts`, `irregularity`, `seed`). Surface: `color`, `tint`, `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `material` (`"lit" \| "glass" \| "acrylic" \| "frosted"`), `castShadow`, `wireframe`. Shared GPU buffers cache identical shapes. |+| `Layer3D.roundedBox / sphere / cylinder / capsule / torus / ring / prism / pyramid / polyhedron / icosahedron / icosphere / shard(options)` | `"model3d"` | Procedural solid meshes, centered on `x`/`y`/`z` (y is down, so "top" is -y). `roundedBox`: `width`, `height`, `depth`, `radius`, `segments`, `taper`. `sphere`: `radius`. `cylinder`: `radius` or `radiusTop`/`radiusBottom`, `height`, `openEnded`, `flatShaded`. `capsule`: `radius`, `height` (cylindrical shaft) or `totalHeight`. `torus`: `radius`, `tube` (tube radius), `arc`. `ring`: flat annulus (`innerRadius`, `outerRadius`). `prism` / `pyramid`: solid capped flat-shaded N-gon prism or pyramid (`sides`, `radius`, `height`). `polyhedron` (`type: "tetrahedron" \| "octahedron" \| "dodecahedron" \| "icosahedron"`), `icosahedron`, `icosphere` (`radius`, `detail`, `flatShaded`). `shard`: procedural flat-faceted polyhedron rock/crystal (`radius`, `cuts`, `irregularity`, `seed`). Surface: `color`, `tint`, `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `material` (`"lit" \| "glass" \| "acrylic" \| "frosted"`), `castShadow`, `wireframe`. Shared GPU buffers cache identical shapes. || `Layer3D.extrude / lathe / tube / line / points(options)` | `"model3d"` | Custom & path-driven 3D geometry. `extrude`: extrude a 2D SVG path `d` (with holes) or `contour`/`holes` (`depth`, `bevelThickness`, `bevelSize`, `bevelSegments`, `bevelColor`). `lathe`: revolve a 2D `points` profile `[r, y][]` around the Y axis (`segments`, `phiStart`, `phiLength`, `closed`, `flatShaded`). `tube`: sweep a solid or thick-walled hollow pipe (`radius`, `innerRadius`, `radiusTaper`, `closed`, `capped`) along a `Path3D` or 3D control `points` using rotation-minimizing frames. `line`: 3D polyline (`points` or `Path3D`, `dashSize`, `gapSize`, per-vertex `colors`). `points`: 3D point cloud (`points`, `pointSize` for lit sphere dots or `0` for `point-list`, per-point `colors`). || `Layer3D.instances(options)` | `"model3d"` | Hardware-instanced single or multi-variant mesh rendering in 1 GPU draw call. Options: `count`, `mesh` / `meshes` (array of Layer3D primitives or meshes), `instanceAt(index, frame): { position, rotation, scale, color, emissive }`, or bind directly to Rapier `Physics.world3d` bodies via `bodies: Body3D[]` with `emissiveBySpeed` and `colorRamp`. || `Layer3D.swarm(options)` | `"model3d"` | Continuous 3D GPU particle swarm with compute-driven physics, 3D curl turbulence, vortex/cyclone field (`swirl`, `pull`, `axial`), ground plane collisions with Coulomb sliding friction and rotational roll torque, kinetic heat accumulation/cooling, and zero-copy packing directly into GPU instance buffers. |@@ -271,7 +271,9 @@ const beatAnim2 = LayerAnimation.create()object is carried on its track and registered with the program automatically (under a generated`inline_signal_N` id, so the spec stays plain JSON); a string is looked up in `comp.addSignal` names.Anything else (a number, `undefined`) throws. Signals are sampled at the **composition** frame-(`ctx.frame`), not the layer's local frame.+(`ctx.frame`) by default, which shared and audio signals need. Motion authored for one scene+(`Signal.fromArray` of per-frame poses) passes `{ clock: "layer" }` to sample at the layer's own+frame, counted from its start like keyframes, so it plays right inside `Layer.sequence`.To pull signals out of audio, use `AudioSignal`, which runs on the GPU and accepts audio or video files.`AudioSignal.extract(src, { fps })` gives `primary`, `beat`, `bass` and `energy` envelopes.@@ -405,7 +407,9 @@ Where depth jumps (an object's outline) the surface stretches across with the ed## Scenes & Transitions-`Layer.sequence` places full-frame scenes end to end, overlapping each pair by its transition, and writes the scenes' `startFrame` / `durationFrames` for you (each scene keys from its own frame 0). Transitions are ordinary keyframes and mattes merged into the scenes:+`Layer.sequence` places full-frame scenes end to end, overlapping each pair by its transition, and writes the scenes' `startFrame` / `durationFrames` for you (each scene keys from its own frame 0). Transitions are ordinary keyframes and mattes merged into the scenes.++**3D scenes:** a box with a fixed `width`/`height` that holds a `Layer.camera` and the layers it films is a 3D scene of its own (a box holding only a camera stays a shot in a camera cut list over the shared world): its camera, lights and environment see only its layers, it renders on its own clock, and the box is placed like a picture. So a crossfade, whip or iris between two 3D scenes blends two finished pictures, each through its own camera. Give layer ids a per-scene prefix when the same scene code builds several scenes (`comp.lint()` reports `duplicate_id`).```typescriptimport { Layer, Transition } from "framefields";diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.mdindex 5c5927ee..940a6735 100644--- a/plugins/framefields/skills/framefields-render/SKILL.md+++ b/plugins/framefields/skills/framefields-render/SKILL.md@@ -423,6 +423,7 @@ if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop| `unknown_color` | warning | A color string that is not a color: it draws transparent || `low_contrast_text` | warning | Text below 3:1 contrast against the solid color behind it (suggests a readable one) || `offscreen_layer` | warning | A static absolute layer entirely outside the canvas |+| `duplicate_id` | error | Two layers share an id: the renderer keeps one and drops the other || `shadow_unsupported_light` | warning | `castShadow` on a point or ambient light || `pbr_without_environment` | info | `roughness` / `metallic` set but no `Layer.environment()` (no physical shading) || `static_track` | info | Every keyframe on a track holds the same value |