Code-first video & image, rendered natively on WebGPU.
feat: GPU particles, glass refraction, Trails/Datamosh, physics fixes; new dancer cut
okanasl741f6d21eb97
diff --git a/.changeset/physics-rapier.md b/.changeset/physics-rapier.mdindex f570bbbc..bb4dafdf 100644--- a/.changeset/physics-rapier.md+++ b/.changeset/physics-rapier.md@@ -7,3 +7,7 @@ Add rigid-body physics: `Physics.world2d` / `Physics.world3d` (Rapier, determini`world.field()` adds position-dependent forces (wind, vortices, flow fields). `world.live(comp)` steps a world as the composition renders instead of all at once: before each frame draws it advances to it, after that frame's inputs are ready, giving the same result as `simulate()`.`comp.vision(clip)` analyses a clip's own frames on demand (pose keypoints and a soft silhouette) and shares the results with everything that reads them; every render path and the preview await it before drawing a frame, and results persist on disk in Node. `comp.onPrepareFrame()` registers async per-frame work, and the renderer awaits an `onRequestFrame` that returns a promise. `colorSignal` takes `multiplier` and `offset`.++`Layer.particles` adds GPU particle systems: a compute-shader simulation (emitters, curl-noise flow, attractors, gravity, drag, floors and bounds, a performer's silhouette push) drawn in one instanced call, with snapshots so scrubbing back restores state. New temporal effects: `Trails` (echoes, light painting, feedback tunnels) and `Datamosh` (pixels dragged along optical flow between keyframes).++3D `glass`, `acrylic` and `frosted` materials now refract what is behind them on screen (with dispersion, thickness, tint and studio reflections) instead of drawing as translucent plastic. Particle emitters can move: `emitter.x` / `emitter.y` take drivers, and births spread along the path between frames (light painting from a tracked hand).diff --git a/examples/28_physics_dancer/README.md b/examples/28_physics_dancer/README.mdindex ba7ca65a..df62a769 100644--- a/examples/28_physics_dancer/README.md+++ b/examples/28_physics_dancer/README.md@@ -1,8 +1,8 @@# 28 — Physics dancer-A 6.6-second shot where rigid-body physics meets on-device vision. Word cards fall onto a-dancer's raised hands and head and tumble to the floor; a cloud of ~800 particles flows around-her as the silk dress sweeps through it.+A 6.6-second shot that puts most of the engine in one frame: glass word cards are thrown in from+both sides and bounce off a dancer's raised arms and head, the last one landing on her hands; a+cloud of embers parts around her silk dress. Renders in about 20 seconds.```bashpnpm fonts # Syne from google/fonts → assets/fonts/@@ -19,13 +19,9 @@ The clip is example 19's `assets/dancer.mp4`; generate it there with `pnpm asset| Piece | API ||---|---|| Pose and silhouette, frame by frame as it renders | `comp.vision(clip).person`, placed with `.place({ width, height, fit })`; results persist in `~/.cache/framefields/vision` |-| Physics that steps with the render | `world.live(comp)`: before each frame draws, its vision is ready and the world advances to it |-| Arms and head that catch the cards | kinematic circle bodies along shoulder → elbow → wrist (and past the wrist for the hand), `follow`ing `person.point(name, frame)` |-| The dress pushing particles and cards | `world.field()` with `person.sample(frame, x, y)`: out along the silhouette's gradient, much harder where the silhouette just arrived (the sweep) |-| Fluid feel | small circles with `linearDamping` 2.2, near-zero gravity, a slow swirling current and a soft pull home |-| Particles behind her | a second copy of the clip, cut out with `.withVision({ mode: "matte" })`, above the particles |-| Hot particles | `.colorSignal("fillColor", body.speed, { multiplier: 1 / 900, … })` |-| Screen shake | the whole stage's `y` driven by `world.impact` |--Cards are sized with `FontManager.measure`, so the text always fits. A live world can't look-ahead, so they drop at fixed offsets from where her head starts and find her hands on the way down.+| 3D physics that steps with the render | `Physics.world3d(…)` + `world.live(comp)`: before each frame draws, its vision is ready and the world advances to it |+| Glass cards | `Layer3D.roundedBox({ material: "glass" })` as the rigid body, the word as unlit blue `Layer3D.extrudedText` following the body's `x`/`y`/`z` signals just in front of it; `lockRotation: true` keeps every word upright |+| Thrown in from the sides | each card's launch `velocity` solved from gravity and a 0.7 s flight so the arc crosses her head and arms; the finale drops from above with a lower `gravityScale` |+| Arms and head that catch them | kinematic spheres along shoulder → elbow → wrist (and past it for the hand), `follow`ing `person.point(name, frame)`; without a camera layer the 3D z = 0 plane matches the video 1:1 |+| Embers | `Layer.particles` with `push: { person }` (her silhouette, hardest where it sweeps in), curl `noise`, `colorBySpeed` |+| Embers behind her | a second copy of the clip, cut out with `.withVision({ mode: "matte" })`, above the embers |diff --git a/examples/28_physics_dancer/src/film.ts b/examples/28_physics_dancer/src/film.tsindex 89198b0f..7b1f6e84 100644--- a/examples/28_physics_dancer/src/film.ts+++ b/examples/28_physics_dancer/src/film.ts@@ -1,30 +1,31 @@/**- * A dancer in a field of physics: word cards drop onto her raised hands and- * head, and a cloud of particles flows around her as the dress sweeps through.+ * Glass word cards are thrown in from both sides onto a dancer's raised hands while she sweeps a cloud+ * of embers aside.*- * Nothing is analysed up front: `comp.vision` analyses each frame of the clip- * just before it renders, and the live physics world steps to that frame+ * Nothing is analysed up front: `comp.vision` reads each frame of the clip+ * just before it renders, and the live 3D physics world steps to that frame* right after, so the preview starts at once and fills in as it plays.*- * - Her arms and head are kinematic colliders that follow the tracked pose.- * - Her silhouette (dress included) is a force field that pushes everything out,- * hardest where it is sweeping in.- * - A vision cutout of the same clip sits above the particles, so they pass behind her.+ * - Each card is a 3D glass slab (Rapier rigid body) that refracts+ * everything behind it, with its word in solid blue welded in front.+ * - Her arms and head are kinematic colliders that follow the tracked pose;+ * her silhouette (dress included) pushes the embers out of the way.+ * - A vision cutout of the clip sits above the embers, so they pass behind her.*/import fs from "node:fs";import {Composition,FontManager,Layer,+ Layer3D,LayerAnimation,type PersonTrack,Physics,- type PhysicsBody2D,- type PhysicsWorld2D,+ type PhysicsBody3D,+ type PhysicsWorld3D,type PoseKeypointName,} from "framefields";import {- CREAM,DANCER,DURATION,EMBER,@@ -34,27 +35,16 @@ import {GOLD,H,HOT,- INK,RUST,registerFonts,- VIOLET,W,} from "./theme.js";-const WORDS = [- // `dx`: where it drops, from her head at the first frame (live worlds can't look ahead).- { text: "PHYSICS", bg: EMBER, fg: INK, at: 30, dx: -40 },- { text: "MEETS", bg: INK, fg: CREAM, at: 46, dx: 120 },- { text: "VISION", bg: CREAM, fg: INK, at: 62, dx: -120 },- { text: "IN", bg: GOLD, fg: INK, at: 78, dx: 60 },- { text: "FRAMEFIELDS", bg: VIOLET, fg: CREAM, at: 94, dx: 0 },-];-const CARD_FONT_SIZE = 64;-const PARTICLE_COLORS = [INK, RUST, EMBER, GOLD];+export { DURATION as DURATION_FRAMES } from "./theme.js";-/** Balls along a limb (u = 0 at `a`, 1 at `b`, beyond 1 past `b`) that follow the pose. */+/** Spheres along a limb (u = 0 at `a`, 1 at `b`, beyond 1 past it) that follow the pose, in the z = 0 plane. */function limb(- world: PhysicsWorld2D,+ world: PhysicsWorld3D,person: PersonTrack,a: PoseKeypointName,b: PoseKeypointName,@@ -69,126 +59,161 @@ function limb(};const [x, y] = at(0);world.body(- { x: x - r, y: y - r, width: 2 * r, height: 2 * r },+ { x, y, z: 0, width: 2 * r, height: 2 * r, depth: 2 * r },{type: "kinematic",- shape: "circle",- friction: 0.9,- follow: { x: (f) => at(f)[0] - r, y: (f) => at(f)[1] - r },+ shape: "sphere",+ friction: 0.8,+ follow: { x: (f) => at(f)[0], y: (f) => at(f)[1], z: 0 },},);}}-/** Word cards sized to their text, dropped around where her head starts. */-function cards(world: PhysicsWorld2D, person: PersonTrack) {- return WORDS.map((w, i) => {- const m = FontManager.measure(w.text, {+/** The framefields blues. */+const BLUE = "#3B82F6";+const BLUE_DEEP = "#1D4ED8";+const NAVY = "#0B1E4D";++const CARDS = [+ // `from`: the side it is thrown in from; `dx`: where on her it is aimed, from her head.+ { text: "PHYSICS", color: BLUE, at: 24, from: "left", dx: -30 },+ { text: "MEETS", color: NAVY, at: 38, from: "right", dx: 50 },+ { text: "VISION", color: BLUE, at: 52, from: "left", dx: 10 },+ { text: "IN", color: BLUE_DEEP, at: 66, from: "right", dx: -20 },+ // The finale drops straight down onto her, slower to read.+ { text: "FRAMEFIELDS", color: BLUE_DEEP, at: 82, from: "top", dx: 0 },+];+const CARD_FONT = 76;+const CARD_DEPTH = 34;+const GRAVITY = 2600;+/** Seconds from the edge of the frame to her. */+const FLIGHT = 0.7;++/**+ * Word cards: a 3D glass slab (the rigid body) with its word in solid blue+ * riding just in front of it. Cards keep their slight tilt and never turn,+ * so the words stay upright and readable while the cards bounce off her.+ */+function glassCards(world: PhysicsWorld3D, person: PersonTrack) {+ const head = person.point("nose", 0);+ return CARDS.map((card, i) => {+ const m = FontManager.measure(card.text, {fontFamily: "Syne",- fontSize: CARD_FONT_SIZE,+ fontSize: CARD_FONT,});- const width = Math.ceil(m.width + 72);- const height = Math.ceil(m.height + 40);- const head = person.point("nose", 0);- const layer = Layer.box({+ const width = Math.ceil(m.width + 84);+ const height = Math.ceil(m.height + 52);+ // Thrown in from the side on an arc that crosses her head and raised+ // arms: launch velocity solved from gravity and the flight time.+ const target = { x: head.x + card.dx, y: head.y + 40 };+ const fromTop = card.from === "top";+ const x = fromTop+ ? target.x+ : card.from === "left"+ ? -width / 2 - 40+ : W + width / 2 + 40;+ const y = fromTop ? -height - 60 : 430 - (i % 2) * 50;+ const velocity: [number, number, number] = fromTop+ ? [0, 0, 0]+ : [+ (target.x - x) / FLIGHT,+ (target.y - y - 0.5 * GRAVITY * FLIGHT * FLIGHT) / FLIGHT,+ 0,+ ];+ const tilt = (i % 2 ? 1 : -1) * (4 + i * 2);+ const slab = Layer3D.roundedBox({id: `card-${i}`,- position: "absolute",- x: head.x + w.dx - width / 2,- y: -height - 200,width,height,- rotation: (i % 2 ? 1 : -1) * (6 + i * 2),- borderRadius: 18,- background: w.bg,- children: [- Layer.text(w.text, {- fontFamily: "Syne",- fontSize: CARD_FONT_SIZE,- fill: w.fg,- width,- height,- align: "center",- verticalAlign: "middle",- }),- ],+ depth: CARD_DEPTH,+ radius: 18,+ x,+ y,+ z: 0,+ rotateZ: tilt,+ color: "#eef5ff",+ material: "glass",+ roughness: 0.03,});- const body = world.body(layer, {- restitution: 0.25,- friction: 0.7,- density: 1.5,- angularDamping: 1.2,- startFrame: w.at,+ const body = world.body(slab, {+ restitution: fromTop ? 0.1 : 0.3,+ friction: fromTop ? 0.9 : 0.5,+ // The finale falls slower, so it reads on the way down.+ gravityScale: fromTop ? 0.45 : 1,+ density: 1.2,+ startFrame: card.at,+ velocity,+ lockRotation: true,});- return { layer, body };+ const words = Layer3D.extrudedText({+ id: `card-${i}-text`,+ text: card.text,+ fontFamily: "Syne",+ fontSize: CARD_FONT,+ depth: 3,+ align: "center",+ verticalAlign: "middle",+ fill: card.color,+ material: "unlit",+ x,+ y,+ z: -CARD_DEPTH / 2 - 2,+ rotateZ: tilt,+ }).animate(+ LayerAnimation.create()+ .signal("x", body.x)+ .signal("y", body.y)+ .signal("z", body.z, { offset: -CARD_DEPTH / 2 - 2 }),+ );+ return { layer: slab, words, body };});}-/** A loose cloud of small particles around her, avoiding where she stands. */-function particles(world: PhysicsWorld2D, person: PersonTrack) {- let seed = 7;- const rand = () => {- seed = (seed * 16807) % 2147483647;- return seed / 2147483647;- };- const out: Array<{- layer: ReturnType<typeof Layer.shape>;- body: PhysicsBody2D;- color: string;- home: [number, number];- }> = [];- for (let y = 430; y < FLOOR - 10; y += 24) {- for (let x = 20; x < W - 20; x += 24) {- const px = x + (rand() - 0.5) * 12;- const py = y + (rand() - 0.5) * 12;- if (person.sample(0, px, py).alpha > 0.15 || rand() > 0.5) continue;- const r = 6 + rand() * 6;- const color =- PARTICLE_COLORS[Math.floor(rand() * PARTICLE_COLORS.length)];- const layer = Layer.shape("circle", {- position: "absolute",- x: px - r,- y: py - r,- width: 2 * r,- height: 2 * r,- fill: color,- });- const body = world.body(layer, {- restitution: 0.05,- friction: 0.02,- density: 0.25,- linearDamping: 2.2,- gravityScale: 0.06,- lockRotation: true,- });- out.push({ layer, body, color, home: [px, py] });- }- }- return out;-}--/** Out of the silhouette: `strength` where she is, plus `sweep` where she is arriving. */-function silhouettePush(person: PersonTrack, strength: number, sweep: number) {- return (s: {- x: number;- y: number;- frame: number;- }): [number, number] | null => {- const now = person.sample(s.frame, s.x, s.y);- if (now.alpha < 0.02) return null;- const before = person.sample(s.frame - 1, s.x, s.y);- const g = Math.hypot(now.gx, now.gy) || 1;- const k =- now.alpha * strength + Math.max(0, now.alpha - before.alpha) * sweep;- return [(-now.gx / g) * k, (-now.gy / g) * k];- };+/** Embers in the lower half: her silhouette pushes them, a slow current keeps them drifting. */+function embers(person: PersonTrack) {+ return Layer.particles({+ id: "embers",+ width: W,+ height: H,+ max: 24000,+ seed: 7,+ emitter: {+ shape: "rect",+ x: 0,+ y: 380,+ width: W,+ height: FLOOR - 390,+ rate: 3600,+ },+ lifetime: [4, 6.5],+ speed: [0, 25],+ drag: 2.2,+ gravity: -6,+ noise: { strength: 240, scale: 170, speed: 0.25 },+ push: { person, strength: 9000, sweep: 120000 },+ floor: FLOOR,+ bounce: 0.1,+ size: [11, 8],+ sizeJitter: 0.6,+ fade: [0.15, 0.3],+ color: [RUST, EMBER, GOLD, HOT],+ colorBySpeed: 500,+ prewarm: 4,+ });}-export { DURATION as DURATION_FRAMES } from "./theme.js";+/** Keeps letters near the picture plane, so they stay where her arms are. */+const keepInPlane = (s: { z: number }): [number, number, number] => [+ 0,+ 0,+ -s.z * 6,+];export interface Film {comp: Composition;- world: PhysicsWorld2D;- cards: Array<{ id: string; body: PhysicsBody2D }>;+ world: PhysicsWorld3D;+ letters: Array<{ id: string; body: PhysicsBody3D }>;}export async function buildFilm(): Promise<Composition> {@@ -211,107 +236,61 @@ export async function buildScene(): Promise<Film> {);}const dancer = comp.vision(DANCER);- // The particles are laid out around her first pose: analyse just that now.+ // Letters drop around her first pose: analyse just that now.await dancer.ready(0);const person = dancer.person.place({ width: W, height: H, fit: "cover" });- const world = await Physics.world2d({+ // Without a camera layer the 3D world's z = 0 plane lines up 1:1 with the+ // video, so pose points are 3D positions too.+ const world = await Physics.world3d({fps: FPS,frames: DURATION,- gravity: 2600,+ gravity: GRAVITY,substeps: 6,- impactDecay: 5,});- world.walls({ width: W, height: FLOOR });-+ // Only a floor: cards come in from beyond the frame edges.+ world.ground({ y: FLOOR });for (const side of ["left", "right"] as const) {- limb(world, person, `${side}Shoulder`, `${side}Elbow`, [0, 0.5, 1], 26);+ limb(world, person, `${side}Shoulder`, `${side}Elbow`, [0, 0.5, 1], 30);limb(world,person,`${side}Elbow`,`${side}Wrist`,[0.33, 0.66, 1, 1.25],- 24,+ 28,);}- limb(world, person, "leftShoulder", "rightShoulder", [0.5], 40);- limb(world, person, "nose", "nose", [0], 52);-- const words = cards(world, person);- const cloud = particles(world, person);- const cloudBodies = cloud.map((p) => p.body);- const home = new Map(cloud.map((p) => [p.body, p.home]));-- world.field(silhouettePush(person, 11000, 140000), { bodies: cloudBodies });- world.field(silhouettePush(person, 16000, 60000), {- bodies: words.map((w) => w.body),+ limb(world, person, "leftShoulder", "rightShoulder", [0.5], 46);+ limb(world, person, "nose", "nose", [0], 58);+ const letters = glassCards(world, person);+ world.field(keepInPlane, {+ bodies: letters.map((l) => l.body),});- // A slow swirling current keeps the cloud alive; a soft pull home keeps it on stage.- world.field(- (s) => {- const t = s.frame / FPS;- const [hx, hy] = home.get(s.body) ?? [s.x, s.y];- return [- 140 * Math.sin(s.y * 0.009 + t * 1.7) + (hx - s.x) * 0.35,- 110 * Math.cos(s.x * 0.007 - t * 1.3) + (hy - s.y) * 0.6,- ];- },- { bodies: cloudBodies },- );- // Steps as the composition renders, after each frame's vision is ready.world.live(comp);- // Particles glow as they speed up (900 px/s is fully hot).- for (const p of cloud) {- p.layer.animate(- LayerAnimation.create().colorSignal("fillColor", p.body.speed, {- colorA: p.color,- colorB: HOT,- multiplier: 1 / 900,- }),- );- }-- comp.add(- Layer.box({- id: "stage",+ const video = (id: string, muted: boolean) =>+ Layer.video(DANCER, {+ id,position: "absolute",x: 0,y: 0,width: W,height: H,- children: [- Layer.video(DANCER, {- id: "plate",- position: "absolute",- x: 0,- y: 0,- width: W,- height: H,- fit: "cover",- }),- ...cloud.map((p) => p.layer),- Layer.video(DANCER, {- id: "cutout",- position: "absolute",- x: 0,- y: 0,- width: W,- height: H,- fit: "cover",- muted: true,- }).withVision({ mode: "matte" }),- ...words.map((w) => w.layer),- ],- }).animate(- LayerAnimation.create().signal("y", world.impact, { multiplier: 10 }),- ),- );+ fit: "cover",+ muted,+ });++ comp.add([+ video("plate", false),+ embers(person),+ video("cutout", true).withVision({ mode: "matte" }),+ ...letters.flatMap((l) => [l.layer, l.words]),+ ]);return {comp,world,- cards: words.map((w) => ({ id: String(w.layer.id), body: w.body })),+ letters: letters.map((l) => ({ id: String(l.layer.id), body: l.body })),};}diff --git a/examples/28_physics_dancer/src/render.ts b/examples/28_physics_dancer/src/render.tsindex b8fc9fdc..73c0c9c7 100644--- a/examples/28_physics_dancer/src/render.ts+++ b/examples/28_physics_dancer/src/render.ts@@ -22,7 +22,7 @@ if (mode === "preview") {process.exit(0);}-const { comp, world, cards } = await buildScene();+const { comp, world, letters } = await buildScene();if (mode === "sim") {const started = Date.now();@@ -30,7 +30,7 @@ if (mode === "sim") {console.log(`Analysed and simulated in ${((Date.now() - started) / 1000).toFixed(1)} s`,);- for (const { id, body } of cards) {+ for (const { id, body } of letters) {const hit = body.collisions[0];console.log(`${id}: first contact at frame ${hit?.frame ?? "-"} (${hit?.with.type ?? "none"})`,diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.tsindex 06013f21..29f31cf4 100644--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts@@ -1148,7 +1148,18 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {Boolean(sceneBloom),);- for (const item of pending3DItems) {+ // Glass reads what is behind the batch (the canvas so far), and goes+ // last so opaque meshes are already in the depth buffer.+ const isGlass = (item: Pending3DItem) =>+ item.kind === "mesh" &&+ (item.opts.material === "glass" ||+ item.opts.material === "acrylic" ||+ item.opts.material === "frosted");+ const ordered = [+ ...pending3DItems.filter((i) => !isGlass(i)),+ ...pending3DItems.filter(isGlass),+ ];+ for (const item of ordered) {if (item.kind === "quad") {r3d.drawTextureQuad(pass3d,@@ -1175,7 +1186,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {lightsBindGroup,item.meshBuffers ?? item.mesh,item.texture,- item.opts,+ isGlass(item) ? { ...item.opts, backdrop: curTex } : item.opts,dofEnabled,);}@@ -1954,6 +1965,9 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {target?.ambientIntensity ?? lop.ambientIntensity ?? 1.0,metallic: target?.metallic ?? lop.metallic ?? mat?.metallic,...surfaceProps(target, lop, mat),+ ior: target?.ior ?? lop.ior,+ dispersion: target?.dispersion ?? lop.dispersion,+ thickness: target?.thickness ?? lop.thickness,audioDeform: meshAudioDeform,};diff --git a/nodes/node-datamosh/package.json b/nodes/node-datamosh/package.jsonnew file mode 100644index 00000000..76cea0b9--- /dev/null+++ b/nodes/node-datamosh/package.json@@ -0,0 +1,47 @@+{+ "name": "@framefields/node-datamosh",+ "version": "2.2.0",+ "private": false,+ "framefields": {+ "type": "Datamosh",+ "kind": "effect",+ "schema": "DatamoshNodeConfigSchema"+ },+ "type": "module",+ "license": "Apache-2.0",+ "exports": {+ ".": {+ "types": "./dist/index.d.mts",+ "development": "./src/index.ts",+ "import": "./dist/index.mjs",+ "default": "./dist/index.mjs"+ },+ "./renderer": {+ "types": "./dist/renderer.d.mts",+ "development": "./src/renderers/index.ts",+ "import": "./dist/renderer.mjs",+ "default": "./dist/renderer.mjs"+ }+ },+ "scripts": {+ "build": "tsdown",+ "dev": "tsdown --watch",+ "clean": "rm -rf dist .turbo",+ "typecheck": "tsc --noEmit"+ },+ "dependencies": {+ "@framefields/node-sdk": "workspace:*",+ "@framefields/tensor-webgpu": "workspace:*",+ "zod": "^4.3.6"+ },+ "devDependencies": {+ "@framefields/tsconfig": "workspace:*",+ "@types/node": "^24.1.0",+ "tsdown": "^0.19.0",+ "typescript": "^5.8.3",+ "webgpu": "^0.4.0"+ },+ "publishConfig": {+ "access": "public"+ }+}diff --git a/nodes/node-datamosh/src/index.ts b/nodes/node-datamosh/src/index.tsnew file mode 100644index 00000000..944d702e--- /dev/null+++ b/nodes/node-datamosh/src/index.ts@@ -0,0 +1,2 @@+export * from "./renderers/index.js";+export * from "./shared/index.js";diff --git a/nodes/node-datamosh/src/renderers/index.ts b/nodes/node-datamosh/src/renderers/index.tsnew file mode 100644index 00000000..28b94a6e--- /dev/null+++ b/nodes/node-datamosh/src/renderers/index.ts@@ -0,0 +1,7 @@+import { defineRenderer } from "@framefields/node-sdk/renderer";+import { DatamoshWebGPURenderer } from "./webgpu-renderer.js";++export default defineRenderer({+ WebGPURenderer: DatamoshWebGPURenderer,+});+export { DatamoshWebGPURenderer };diff --git a/nodes/node-datamosh/src/renderers/webgpu-renderer.ts b/nodes/node-datamosh/src/renderers/webgpu-renderer.tsnew file mode 100644index 00000000..724878f1--- /dev/null+++ b/nodes/node-datamosh/src/renderers/webgpu-renderer.ts@@ -0,0 +1,274 @@+/// <reference types="webgpu" />+/**+ * Datamosh: instead of each new frame, the previous output dragged along the+ * layer's optical flow, until a keyframe (I-frame) shows the clean picture.+ * History lives in textures on the effect object and advances one frame at a+ * time; after a jump the frames since the last keyframe are replayed.+ */+import type { WebGPUNodeRenderer } from "@framefields/node-sdk";+import {+ drawChildToTexture,+ drawFullscreenPass,+ drawTextureToTarget,+ FULLSCREEN_QUAD_WGSL,+ fragmentBindGroupLayout,+ perDevice,+ type WebGPUNodeRendererArgs,+} from "@framefields/node-sdk/gpu";+import { OpticalFlowComputePipeline } from "@framefields/tensor-webgpu";++const HISTORY_FORMAT: GPUTextureFormat = "rgba16float";++interface DatamoshValues {+ keyframes: number[];+ strength: number;+ bleed: number;+ warmup: number;+}++const MOSH_WGSL = /* wgsl */ `+struct U { a : vec4<f32>, b : vec4<f32> };+@group(0) @binding(0) var<uniform> u : U;+@group(0) @binding(1) var moshTex : texture_2d<f32>;+@group(0) @binding(2) var currentTex : texture_2d<f32>;+@group(0) @binding(3) var flowTex : texture_2d<f32>;+@group(0) @binding(4) var samp : sampler;+${FULLSCREEN_QUAD_WGSL}++@fragment fn fs(in : VSOut) -> @location(0) vec4<f32> {+ let strength = u.a.x;+ let bleed = u.a.y;+ let inv = u.a.zw;+ let cur0 = textureSampleLevel(currentTex, samp, in.uv, 0.0);+ if (u.b.x > 0.5) { return cur0; }+ let dims = vec2<f32>(textureDimensions(flowTex));+ let coord = vec2<i32>(clamp(in.uv * dims, vec2<f32>(0.0), dims - 1.0));+ let flow = textureLoad(flowTex, coord, 0).xy;+ let cur = textureSampleLevel(currentTex, samp, in.uv, 0.0);+ // The old picture moves with the new motion instead of being replaced.+ let moved = textureSampleLevel(moshTex, samp, clamp(in.uv - flow * inv * strength, vec2<f32>(0.0), vec2<f32>(1.0)), 0.0);+ var out = mix(moved, cur, bleed);+ if (moved.a < 0.01) { out = cur; }+ return out;+}+`;++const resources = perDevice((device: GPUDevice) => {+ const layout = fragmentBindGroupLayout(+ device,+ ["uniform", "texture", "texture", "texture", "sampler"],+ "datamosh_layout",+ );+ const module = device.createShaderModule({+ code: MOSH_WGSL,+ label: "datamosh.wgsl",+ });+ return {+ layout,+ pipeline: device.createRenderPipeline({+ label: "datamosh",+ layout: device.createPipelineLayout({ bindGroupLayouts: [layout] }),+ vertex: { module, entryPoint: "vs" },+ fragment: {+ module,+ entryPoint: "fs",+ targets: [{ format: HISTORY_FORMAT }],+ },+ primitive: { topology: "triangle-strip" },+ }),+ flow: new OpticalFlowComputePipeline(device),+ sampler: device.createSampler({ magFilter: "linear", minFilter: "linear" }),+ };+});++interface History {+ device: GPUDevice;+ width: number;+ height: number;+ /** Ping-pong history; `front` holds the latest. */+ front: GPUTexture;+ back: GPUTexture;+ /** Datamosh: the layer's previous clean frame. */+ prev: GPUTexture | null;+ /** Layer frame `front` holds; -1 = none. */+ frame: number;+}++interface FeedbackEffect {+ resolveUniforms(ctx?: unknown): Record<string, unknown>;+ feedbackHistory?: History;+}++function historyFor(+ effect: FeedbackEffect,+ device: GPUDevice,+ width: number,+ height: number,+ format: GPUTextureFormat,+): History {+ const h = effect.feedbackHistory;+ if (h && h.device === device && h.width === width && h.height === height)+ return h;+ h?.front.destroy();+ h?.back.destroy();+ h?.prev?.destroy();+ const make = (label: string, fmt: GPUTextureFormat) =>+ device.createTexture({+ label,+ size: [width, height],+ format: fmt,+ usage:+ GPUTextureUsage.RENDER_ATTACHMENT |+ GPUTextureUsage.TEXTURE_BINDING |+ GPUTextureUsage.COPY_SRC |+ GPUTextureUsage.COPY_DST,+ });+ const next: History = {+ device,+ width,+ height,+ front: make("feedback_front", HISTORY_FORMAT),+ back: make("feedback_back", HISTORY_FORMAT),+ prev: make("feedback_prev", format),+ frame: -1,+ };+ effect.feedbackHistory = next;+ return next;+}++async function childAt(+ args: WebGPUNodeRendererArgs,+ frame: number,+ compositionStart: number,+ h: History,+) {+ const child = args.props.virtualMedia?.children?.[0];+ if (!child) return null;+ return drawChildToTexture(args, child, {+ exclude: [h.front, h.back, ...(h.prev ? [h.prev] : [])],+ props: {+ ...args.props,+ virtualMedia: child,+ frame,+ compositionFrame: compositionStart + frame,+ },+ });+}++function swap(h: History): void {+ const t = h.front;+ h.front = h.back;+ h.back = t;+}++async function stepDatamosh(+ args: WebGPUNodeRendererArgs,+ h: History,+ v: DatamoshValues,+ frame: number,+ compositionStart: number,+ key: boolean,+): Promise<void> {+ const cur = await childAt(args, frame, compositionStart, h);+ if (!cur || !h.prev) return;+ const r = resources(args.ctx.device);+ const { encoder } = args;+ const clean = key || h.frame < 0;+ // On a keyframe the flow is unused; the current frame stands in for it.+ const flow = clean+ ? cur.texture+ : r.flow.execute(h.prev, cur.texture, {}, encoder);+ const uniforms = new Float32Array([+ v.strength,+ v.bleed,+ 1 / h.width,+ 1 / h.height,+ clean ? 1 : 0,+ 0,+ 0,+ 0,+ ]);+ drawFullscreenPass(encoder, h.back.createView(), r.pipeline, [+ args.ctx.device.createBindGroup({+ layout: r.layout,+ entries: [+ {+ binding: 0,+ resource: { buffer: args.ctx.renderer.getTemporaryBuffer(uniforms) },+ },+ { binding: 1, resource: h.front.createView() },+ { binding: 2, resource: cur.view },+ { binding: 3, resource: flow.createView() },+ { binding: 4, resource: r.sampler },+ ],+ }),+ ]);+ encoder.copyTextureToTexture({ texture: cur.texture }, { texture: h.prev }, [+ h.width,+ h.height,+ 1,+ ]);+ swap(h);+ h.frame = frame;+}++function num(v: unknown, fallback: number): number {+ return typeof v === "number" && Number.isFinite(v) ? v : fallback;+}++export const DatamoshWebGPURenderer: WebGPUNodeRenderer = async (args) => {+ const { ctx, pass, targetWidth, targetHeight, props } = args;+ const op = props.virtualMedia?.operation as+ | { op?: string; effect?: FeedbackEffect }+ | undefined;+ if (op?.op !== "Datamosh" || !op.effect?.resolveUniforms) return;+ if (!props.virtualMedia?.children?.[0]) return;+ pass.end();++ const fps = props.fps || 30;+ const local = Math.max(0, Math.round(props.frame ?? 0));+ const compositionFrame = Math.round(props.compositionFrame ?? local);+ const compositionStart = compositionFrame - local;+ const raw = op.effect.resolveUniforms({+ frame: compositionFrame,+ fps,+ time: compositionFrame / fps,+ });+ const h = historyFor(+ op.effect,+ ctx.device,+ targetWidth,+ targetHeight,+ ctx.renderer.format,+ );++ {+ const v: DatamoshValues = {+ keyframes: Array.isArray(raw.keyframes)+ ? (raw.keyframes as number[]).map(Math.round)+ : [],+ strength: num(raw.strength, 1),+ bleed: Math.min(1, Math.max(0, num(raw.bleed, 0))),+ warmup: Math.max(1, Math.round(num(raw.warmup, 48))),+ };+ if (h.frame !== local) {+ let start = local;+ if (h.frame !== local - 1) {+ const lastKey = Math.max(0, ...v.keyframes.filter((k) => k <= local));+ start = Math.max(lastKey, local - v.warmup);+ h.frame = -1;+ }+ for (let f = start; f <= local; f++) {+ await stepDatamosh(+ args,+ h,+ v,+ f,+ compositionStart,+ f === 0 || v.keyframes.includes(f) || h.frame < 0,+ );+ }+ }+ }+ drawTextureToTarget(args, h.front);+};diff --git a/nodes/node-datamosh/src/shared/config.ts b/nodes/node-datamosh/src/shared/config.tsnew file mode 100644index 00000000..110ddc45--- /dev/null+++ b/nodes/node-datamosh/src/shared/config.ts@@ -0,0 +1,31 @@+import { configBuilder } from "@framefields/node-sdk";+import { z } from "zod";++export const datamoshConfig = configBuilder()+ .field("keyframes", z.array(z.number().int().min(0)).default([]), {+ bindable: false,+ label: "Keyframes",+ description:+ "Layer frames that show the clean picture again (I-frames). The first frame always does.",+ })+ .field("strength", z.number().min(0).max(10).default(1), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Strength",+ description: "How far pixels are dragged along the motion.",+ })+ .field("bleed", z.number().min(0).max(1).default(0), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Bleed",+ description: "Clean picture mixed back in per frame, 0–1.",+ })+ .field("warmup", z.number().int().min(1).max(240).default(48), {+ bindable: false,+ label: "Warmup",+ description: "Most frames replayed to rebuild the smear after a jump.",+ })+ .build();++export const DatamoshNodeConfigSchema = datamoshConfig.schema;+export type DatamoshNodeConfig = z.infer<typeof DatamoshNodeConfigSchema>;diff --git a/nodes/node-datamosh/src/shared/index.ts b/nodes/node-datamosh/src/shared/index.tsnew file mode 100644index 00000000..d1bc97e8--- /dev/null+++ b/nodes/node-datamosh/src/shared/index.ts@@ -0,0 +1 @@+export * from "./config.js";diff --git a/nodes/node-datamosh/tsconfig.json b/nodes/node-datamosh/tsconfig.jsonnew file mode 100644index 00000000..5a0b963f--- /dev/null+++ b/nodes/node-datamosh/tsconfig.json@@ -0,0 +1,12 @@+{+ "extends": "@framefields/tsconfig/node-library.json",+ "compilerOptions": {+ "baseUrl": ".",+ "paths": {+ "@/*": ["./src/*"]+ },+ "experimentalDecorators": true,+ "emitDecoratorMetadata": true+ },+ "include": ["src"]+}diff --git a/nodes/node-datamosh/tsdown.config.ts b/nodes/node-datamosh/tsdown.config.tsnew file mode 100644index 00000000..73f5c2fc--- /dev/null+++ b/nodes/node-datamosh/tsdown.config.ts@@ -0,0 +1,13 @@+import { defineConfig } from "tsdown";++export default defineConfig({+ entry: {+ index: "src/index.ts",+ renderer: "src/renderers/index.ts",+ },+ format: ["esm"],+ dts: true,+ clean: true,+ sourcemap: true,+ treeshake: true,+});diff --git a/nodes/node-particles/package.json b/nodes/node-particles/package.jsonnew file mode 100644index 00000000..1bad4dae--- /dev/null+++ b/nodes/node-particles/package.json@@ -0,0 +1,44 @@+{+ "name": "@framefields/node-particles",+ "version": "2.2.0",+ "private": false,+ "framefields": {+ "type": "Particles",+ "kind": "source"+ },+ "type": "module",+ "license": "Apache-2.0",+ "exports": {+ ".": {+ "types": "./dist/index.d.mts",+ "development": "./src/index.ts",+ "import": "./dist/index.mjs",+ "default": "./dist/index.mjs"+ },+ "./renderer": {+ "types": "./dist/renderer.d.mts",+ "development": "./src/renderers/index.ts",+ "import": "./dist/renderer.mjs",+ "default": "./dist/renderer.mjs"+ }+ },+ "scripts": {+ "build": "tsdown",+ "dev": "tsdown --watch",+ "clean": "rm -rf dist .turbo",+ "typecheck": "tsc --noEmit"+ },+ "dependencies": {+ "@framefields/node-sdk": "workspace:*"+ },+ "devDependencies": {+ "@framefields/tsconfig": "workspace:*",+ "@types/node": "^24.1.0",+ "tsdown": "^0.19.0",+ "typescript": "^5.8.3",+ "webgpu": "^0.4.0"+ },+ "publishConfig": {+ "access": "public"+ }+}diff --git a/nodes/node-particles/src/index.ts b/nodes/node-particles/src/index.tsnew file mode 100644index 00000000..944d702e--- /dev/null+++ b/nodes/node-particles/src/index.ts@@ -0,0 +1,2 @@+export * from "./renderers/index.js";+export * from "./shared/index.js";diff --git a/nodes/node-particles/src/renderers/index.ts b/nodes/node-particles/src/renderers/index.tsnew file mode 100644index 00000000..b8f1f065--- /dev/null+++ b/nodes/node-particles/src/renderers/index.ts@@ -0,0 +1,7 @@+import { defineRenderer } from "@framefields/node-sdk/renderer";+import { ParticlesWebGPURenderer } from "./webgpu-renderer.js";++export default defineRenderer({+ WebGPURenderer: ParticlesWebGPURenderer,+});+export { ParticlesWebGPURenderer };diff --git a/nodes/node-particles/src/renderers/webgpu-renderer.ts b/nodes/node-particles/src/renderers/webgpu-renderer.tsnew file mode 100644index 00000000..0dffeaef--- /dev/null+++ b/nodes/node-particles/src/renderers/webgpu-renderer.ts@@ -0,0 +1,735 @@+/// <reference types="webgpu" />+/**+ * GPU particles: a compute pass spawns and simulates every particle, then+ * one instanced draw renders them all. State lives on the GPU and is+ * stepped frame by frame; snapshots every SNAPSHOT_EVERY frames make+ * scrubbing back (preview) a restore plus a few steps instead of a replay+ * from the start. Same GPU, same frames, same result.+ */+import type { WebGPUNodeRenderer } from "@framefields/node-sdk";+import {+ isParticleProgram,+ type ParticleFrame,+ type ParticleMask,+ type ParticleProgram,+ type ParticleStatic,+} from "../shared/program.js";++const PARTICLE_BYTES = 32;+const PARAM_VEC4S = 32;+const MAX_ATTRACTORS = 8;+const MAX_COLORS = 8;+const SNAPSHOT_EVERY = 24;+const MAX_SNAPSHOTS = 64;+const WORKGROUP = 64;++const COMMON_WGSL = /* wgsl */ `+struct Particle {+ pos : vec2<f32>,+ vel : vec2<f32>,+ age : f32,+ life : f32,+ jitter : f32,+ _pad : f32,+};++struct Params { v : array<vec4<f32>, ${PARAM_VEC4S}> };+`;++const SIM_WGSL = /* wgsl */ `+${COMMON_WGSL}+@group(0) @binding(0) var<uniform> P : Params;+@group(0) @binding(1) var<storage, read_write> particles : array<Particle>;+@group(0) @binding(2) var<storage, read> mask : array<f32>;++fn pcg(v : u32) -> u32 {+ let state = v * 747796405u + 2891336453u;+ let word = ((state >> ((state >> 28u) + 4u)) ^ state) * 277803737u;+ return (word >> 22u) ^ word;+}+fn rnd(idx : u32, channel : u32, seed : u32) -> f32 {+ return f32(pcg(idx * 16u + channel + pcg(seed))) / 4294967295.0;+}++fn hash2(p : vec2<f32>) -> f32 {+ let q = fract(p * vec2<f32>(0.1031, 0.1030));+ let r = q + dot(q, q.yx + 33.33);+ return fract((r.x + r.y) * r.x);+}+fn vnoise(p : vec2<f32>) -> f32 {+ let i = floor(p);+ let f = fract(p);+ let u = f * f * (3.0 - 2.0 * f);+ let a = hash2(i);+ let b = hash2(i + vec2<f32>(1.0, 0.0));+ let c = hash2(i + vec2<f32>(0.0, 1.0));+ let d = hash2(i + vec2<f32>(1.0, 1.0));+ return mix(mix(a, b, u.x), mix(c, d, u.x), u.y);+}+fn fbm(p : vec2<f32>) -> f32 {+ return vnoise(p) * 0.6 + vnoise(p * 2.03 + 17.0) * 0.3 + vnoise(p * 4.01 + 41.0) * 0.1;+}+// Divergence-free flow: the curl of a scalar noise field.+fn curl(p : vec2<f32>) -> vec2<f32> {+ let e = 0.02;+ let dx = fbm(p + vec2<f32>(e, 0.0)) - fbm(p - vec2<f32>(e, 0.0));+ let dy = fbm(p + vec2<f32>(0.0, e)) - fbm(p - vec2<f32>(0.0, e));+ return vec2<f32>(dy, -dx) / (2.0 * e);+}++fn maskAt(cell : vec2<f32>, offset : u32) -> f32 {+ let w = i32(P.v[18].x);+ let h = i32(P.v[18].y);+ let c0 = vec2<i32>(floor(cell));+ let t = cell - floor(cell);+ var s = 0.0;+ for (var k = 0; k < 4; k++) {+ let o = vec2<i32>(k & 1, k >> 1);+ let c = c0 + o;+ var v = 0.0;+ if (c.x >= 0 && c.y >= 0 && c.x < w && c.y < h) {+ v = mask[offset + u32(c.y * w + c.x)];+ }+ let wx = select(1.0 - t.x, t.x, o.x == 1);+ let wy = select(1.0 - t.y, t.y, o.y == 1);+ s += v * wx * wy;+ }+ return s;+}++fn spawn(i : u32, idx : u32, birthFrac : f32) {+ let birthAge = birthFrac * P.v[1].z;+ let seed = u32(P.v[1].y);+ let shape = u32(P.v[4].x);+ let ex = P.v[4].y; let ey = P.v[4].z; let ew = P.v[4].w;+ let eh = P.v[5].x; let er = P.v[5].y;+ let r0 = rnd(idx, 0u, seed); let r1 = rnd(idx, 1u, seed);+ var pos = vec2<f32>(ex, ey);+ var center = vec2<f32>(ex, ey);+ if (shape == 1u) { pos = vec2<f32>(ex + ew * r0, ey + eh * r0); center = vec2<f32>(ex + ew * 0.5, ey + eh * 0.5); }+ else if (shape == 2u) { pos = vec2<f32>(ex + ew * r0, ey + eh * r1); center = vec2<f32>(ex + ew * 0.5, ey + eh * 0.5); }+ else if (shape == 3u) {+ let a = r0 * 6.2831853; let rr = er * sqrt(r1);+ pos = vec2<f32>(ex + cos(a) * rr, ey + sin(a) * rr);+ } else if (shape == 4u) {+ let a = r0 * 6.2831853;+ pos = vec2<f32>(ex + cos(a) * er, ey + sin(a) * er);+ }+ let radial = P.v[1].w > 0.5;+ var dir = P.v[6].x + (rnd(idx, 2u, seed) - 0.5) * P.v[6].y;+ if (radial) {+ let d = pos - center;+ if (dot(d, d) > 1e-6) { dir = atan2(d.y, d.x) + (rnd(idx, 2u, seed) - 0.5) * P.v[6].y; }+ }+ // A moving emitter: older births of the frame start nearer last frame's spot.+ if (P.v[30].z > 0.5) { pos += (P.v[30].xy - vec2<f32>(ex, ey)) * birthFrac; }+ let speed = mix(P.v[5].z, P.v[5].w, rnd(idx, 3u, seed));+ let vel = vec2<f32>(cos(dir), sin(dir)) * speed;+ var p : Particle;+ p.vel = vel;+ p.pos = pos + vel * birthAge;+ p.age = birthAge;+ p.life = mix(P.v[6].z, P.v[6].w, rnd(idx, 4u, seed));+ p.jitter = 1.0 - P.v[16].w * rnd(idx, 5u, seed);+ p._pad = 0.0;+ particles[i] = p;+}++@compute @workgroup_size(${WORKGROUP})+fn simulate(@builtin(global_invocation_id) gid : vec3<u32>) {+ let i = gid.x;+ let maxN = u32(P.v[1].x);+ if (i >= maxN) { return; }+ let dt = P.v[0].x;+ let time = P.v[0].y;+ let emitStart = u32(P.v[0].z);+ let emitCount = u32(P.v[0].w);+ let frameDt = P.v[1].z;++ if (emitCount > 0u) {+ let k = (i + maxN - (emitStart % maxN)) % maxN;+ if (k < emitCount) {+ // Births spread over the frame, oldest first.+ spawn(i, emitStart + k, 1.0 - (f32(k) + 0.5) / f32(emitCount));+ return;+ }+ }++ var p = particles[i];+ if (p.life <= 0.0) { return; }+ p.age += dt;+ if (p.age >= p.life) { p.life = 0.0; particles[i] = p; return; }++ var acc = vec2<f32>(P.v[2].z, P.v[2].w);+ let ns = P.v[3].y;+ if (ns != 0.0) {+ let sc = max(P.v[3].z, 1.0);+ acc += curl(p.pos / sc + vec2<f32>(time * P.v[3].w, time * P.v[3].w * 0.7)) * ns;+ }+ let nAttr = u32(P.v[7].w);+ for (var a = 0u; a < nAttr; a++) {+ let at = P.v[8u + a];+ let d = at.xy - p.pos;+ let dist = length(d) + 1e-3;+ let fall = exp(-(dist * dist) / max(at.w * at.w, 1.0));+ acc += d / dist * at.z * fall;+ }+ if (P.v[16].x > 0.5) {+ // Silhouette push: out along the gradient, harder where it just arrived.+ let comp = p.pos + P.v[18].zw;+ let cell = (comp - P.v[17].xy) / P.v[17].zw - 0.5;+ let cellCount = u32(P.v[18].x * P.v[18].y);+ let a0 = maskAt(cell, 0u);+ if (a0 > 0.02) {+ let gx = maskAt(cell + vec2<f32>(1.0, 0.0), 0u) - maskAt(cell - vec2<f32>(1.0, 0.0), 0u);+ let gy = maskAt(cell + vec2<f32>(0.0, 1.0), 0u) - maskAt(cell - vec2<f32>(0.0, 1.0), 0u);+ let g = vec2<f32>(gx, gy);+ let gl = length(g);+ if (gl > 1e-5) {+ let before = maskAt(cell, cellCount);+ let k = a0 * P.v[16].y + max(0.0, a0 - before) * P.v[16].z;+ acc -= g / gl * k;+ }+ }+ }++ p.vel += acc * dt;+ p.vel *= exp(-P.v[3].x * dt);+ p.pos += p.vel * dt;++ let w = P.v[2].x; let h = P.v[2].y;+ let floorY = P.v[7].y;+ let bounce = P.v[7].z;+ if (floorY >= 0.0 && p.pos.y > floorY) { p.pos.y = floorY; p.vel.y = -abs(p.vel.y) * bounce; p.vel.x *= 0.9; }+ let mode = u32(P.v[7].x);+ if (mode == 1u) {+ if (p.pos.x < 0.0 || p.pos.y < 0.0 || p.pos.x > w || p.pos.y > h) { p.life = 0.0; }+ } else if (mode == 2u) {+ if (p.pos.x < 0.0) { p.pos.x = 0.0; p.vel.x = abs(p.vel.x) * bounce; }+ if (p.pos.x > w) { p.pos.x = w; p.vel.x = -abs(p.vel.x) * bounce; }+ if (p.pos.y < 0.0) { p.pos.y = 0.0; p.vel.y = abs(p.vel.y) * bounce; }+ if (p.pos.y > h) { p.pos.y = h; p.vel.y = -abs(p.vel.y) * bounce; }+ } else if (mode == 3u) {+ p.pos = vec2<f32>(p.pos.x - floor(p.pos.x / w) * w, p.pos.y - floor(p.pos.y / h) * h);+ }+ particles[i] = p;+}+`;++const DRAW_WGSL = /* wgsl */ `+${COMMON_WGSL}+@group(0) @binding(0) var<uniform> P : Params;+@group(0) @binding(1) var<storage, read> particles : array<Particle>;++struct VOut {+ @builtin(position) pos : vec4<f32>,+ @location(0) uv : vec2<f32>,+ @location(1) color : vec4<f32>,+ @location(2) aspect : f32,+};++fn gradient(t : f32) -> vec4<f32> {+ let n = i32(P.v[21].x);+ if (n <= 1) { return P.v[22]; }+ let x = clamp(t, 0.0, 1.0) * f32(n - 1);+ let i = min(i32(floor(x)), n - 2);+ return mix(P.v[22 + i], P.v[23 + i], x - f32(i));+}++@vertex+fn vs(@builtin(vertex_index) vi : u32, @builtin(instance_index) ii : u32) -> VOut {+ var out : VOut;+ let p = particles[ii];+ if (p.life <= 0.0) {+ out.pos = vec4<f32>(2.0, 2.0, 0.0, 1.0);+ out.color = vec4<f32>(0.0);+ return out;+ }+ let t = p.age / p.life;+ let size = mix(P.v[19].x, P.v[19].y, t) * p.jitter;+ var corner = vec2<f32>(0.0);+ switch (vi) {+ case 0u: { corner = vec2<f32>(-1.0, -1.0); }+ case 1u: { corner = vec2<f32>(1.0, -1.0); }+ case 2u: { corner = vec2<f32>(-1.0, 1.0); }+ case 3u: { corner = vec2<f32>(-1.0, 1.0); }+ case 4u: { corner = vec2<f32>(1.0, -1.0); }+ default: { corner = vec2<f32>(1.0, 1.0); }+ }+ let speed = length(p.vel);+ var axisX = vec2<f32>(1.0, 0.0);+ var halfLen = size * 0.5;+ let shape = u32(P.v[20].x);+ if (shape == 3u && speed > 1e-3) {+ axisX = p.vel / speed;+ halfLen = size * 0.5 + speed * P.v[20].y * 0.5;+ }+ let axisY = vec2<f32>(-axisX.y, axisX.x);+ let local = axisX * corner.x * halfLen + axisY * corner.y * size * 0.5;+ let px = (p.pos + local) * P.v[21].yz;+ let tsize = P.v[2].xy * P.v[21].yz;+ out.pos = vec4<f32>(px.x / tsize.x * 2.0 - 1.0, 1.0 - px.y / tsize.y * 2.0, 0.0, 1.0);+ out.uv = corner;+ out.aspect = halfLen / max(size * 0.5, 1e-3);++ var c = gradient(t);+ if (P.v[20].z > 0.5) { c = gradient(speed / max(P.v[20].w, 1e-3)); }+ let fadeIn = max(P.v[19].z, 1e-4);+ let fadeOut = max(P.v[19].w, 1e-4);+ let a = c.a * smoothstep(0.0, fadeIn, t) * (1.0 - smoothstep(1.0 - fadeOut, 1.0, t)) * P.v[21].w;+ out.color = vec4<f32>(c.rgb * a, a);+ return out;+}++@fragment+fn fs(in : VOut) -> @location(0) vec4<f32> {+ let shape = u32(P.v[20].x);+ var m = 1.0;+ if (shape == 0u) {+ let d = length(in.uv);+ let e = max(fwidth(d), 1e-3);+ m = 1.0 - smoothstep(1.0 - e, 1.0, d);+ } else if (shape == 1u) {+ let d = length(in.uv);+ m = exp(-3.5 * d * d) * (1.0 - smoothstep(0.9, 1.0, d));+ } else if (shape == 3u) {+ // A capsule along the motion, soft at the sides.+ let ax = abs(in.uv.x) * in.aspect;+ let ex = max(ax - (in.aspect - 1.0), 0.0);+ let d = length(vec2<f32>(ex, in.uv.y));+ m = 1.0 - smoothstep(0.5, 1.0, d);+ }+ return in.color * m;+}+`;++interface SystemState {+ device: GPUDevice;+ particles: GPUBuffer;+ /** Frame the GPU buffer holds, relative to the layer's start; -1 = empty. */+ frame: number;+ /** Particles born so far (the next birth's index) and the fractional carry. */+ emitted: number;+ carry: number;+ snapshots: Map<number, { buffer: GPUBuffer; emitted: number; carry: number }>;+ maskBuffer: GPUBuffer | null;+ maskSize: number;+ ubo: GPUBuffer;+}++interface Pipelines {+ simLayout: GPUBindGroupLayout;+ sim: GPUComputePipeline;+ drawLayout: GPUBindGroupLayout;+ draw: Map<string, GPURenderPipeline>;+ drawModule: GPUShaderModule;+}++const pipelines = new WeakMap<GPUDevice, Pipelines>();++function getPipelines(device: GPUDevice): Pipelines {+ let p = pipelines.get(device);+ if (p) return p;+ const simLayout = device.createBindGroupLayout({+ label: "particles_sim_layout",+ entries: [+ {+ binding: 0,+ visibility: GPUShaderStage.COMPUTE,+ buffer: { type: "uniform" },+ },+ {+ binding: 1,+ visibility: GPUShaderStage.COMPUTE,+ buffer: { type: "storage" },+ },+ {+ binding: 2,+ visibility: GPUShaderStage.COMPUTE,+ buffer: { type: "read-only-storage" },+ },+ ],+ });+ const sim = device.createComputePipeline({+ label: "particles_sim",+ layout: device.createPipelineLayout({ bindGroupLayouts: [simLayout] }),+ compute: {+ module: device.createShaderModule({+ label: "particles_sim.wgsl",+ code: SIM_WGSL,+ }),+ entryPoint: "simulate",+ },+ });+ const drawLayout = device.createBindGroupLayout({+ label: "particles_draw_layout",+ entries: [+ {+ binding: 0,+ visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT,+ buffer: { type: "uniform" },+ },+ {+ binding: 1,+ visibility: GPUShaderStage.VERTEX,+ buffer: { type: "read-only-storage" },+ },+ ],+ });+ p = {+ simLayout,+ sim,+ drawLayout,+ draw: new Map(),+ drawModule: device.createShaderModule({+ label: "particles_draw.wgsl",+ code: DRAW_WGSL,+ }),+ };+ pipelines.set(device, p);+ return p;+}++const BLENDS: Record<ParticleStatic["blend"], GPUBlendState> = {+ normal: {+ color: {+ srcFactor: "one",+ dstFactor: "one-minus-src-alpha",+ operation: "add",+ },+ alpha: {+ srcFactor: "one",+ dstFactor: "one-minus-src-alpha",+ operation: "add",+ },+ },+ add: {+ color: { srcFactor: "one", dstFactor: "one", operation: "add" },+ alpha: { srcFactor: "one", dstFactor: "one", operation: "add" },+ },+ screen: {+ color: { srcFactor: "one", dstFactor: "one-minus-src", operation: "add" },+ alpha: {+ srcFactor: "one",+ dstFactor: "one-minus-src-alpha",+ operation: "add",+ },+ },+};++function drawPipeline(+ device: GPUDevice,+ format: GPUTextureFormat,+ blend: ParticleStatic["blend"],+): GPURenderPipeline {+ const p = getPipelines(device);+ const key = `${format}|${blend}`;+ let pipe = p.draw.get(key);+ if (!pipe) {+ pipe = device.createRenderPipeline({+ label: `particles_draw_${blend}`,+ layout: device.createPipelineLayout({ bindGroupLayouts: [p.drawLayout] }),+ vertex: { module: p.drawModule, entryPoint: "vs" },+ fragment: {+ module: p.drawModule,+ entryPoint: "fs",+ targets: [{ format, blend: BLENDS[blend] }],+ },+ primitive: { topology: "triangle-list" },+ });+ p.draw.set(key, pipe);+ }+ return pipe;+}++const SHAPES = { circle: 0, soft: 1, square: 2, spark: 3 } as const;+const EMITTERS = { point: 0, line: 1, rect: 2, circle: 3, ring: 4 } as const;+const BOUNDS = { none: 0, kill: 1, bounce: 2, wrap: 3 } as const;++/** Packs static settings and one frame's inputs into the shared parameter block. */+function packParams(+ s: ParticleStatic,+ f: ParticleFrame,+ out: Float32Array,+ sim: {+ dt: number;+ time: number;+ emitStart: number;+ emitCount: number;+ frameDt: number;+ },+ view: { scaleX: number; scaleY: number },+): void {+ out.fill(0);+ const v = (i: number, a: number, b: number, c: number, d: number) =>+ out.set([a, b, c, d], i * 4);+ const deg = Math.PI / 180;+ v(0, sim.dt, sim.time, sim.emitStart, sim.emitCount);+ v(1, s.max, s.seed, sim.frameDt, s.emitter.radial ? 1 : 0);+ v(2, s.layer.width, s.layer.height, f.gravity[0], f.gravity[1]);+ v(3, f.drag, f.noise.strength, f.noise.scale, f.noise.speed);+ const e = s.emitter;+ const ex = f.emitter?.x ?? e.x;+ const ey = f.emitter?.y ?? e.y;+ v(4, EMITTERS[e.shape], ex, ey, e.width);+ if (f.emitter) v(30, f.emitter.previousX, f.emitter.previousY, 1, 0);+ v(5, e.height, e.radius, s.speed[0], s.speed[1]);+ v(6, s.direction * deg, s.spread * deg, s.lifetime[0], s.lifetime[1]);+ const attractors = f.attractors.slice(0, MAX_ATTRACTORS);+ v(7, BOUNDS[s.bounds], s.floor ?? -1, s.bounce, attractors.length);+ attractors.forEach((a, i) => v(8 + i, a.x, a.y, a.strength, a.radius));+ const m = f.push?.mask;+ v(16, m ? 1 : 0, f.push?.strength ?? 0, f.push?.sweep ?? 0, s.sizeJitter);+ v(17, m?.originX ?? 0, m?.originY ?? 0, m?.scaleX ?? 1, m?.scaleY ?? 1);+ v(18, m?.width ?? 0, m?.height ?? 0, s.layer.x, s.layer.y);+ v(19, s.size[0], s.size[1], s.fade[0], s.fade[1]);+ v(+ 20,+ SHAPES[s.shape],+ s.stretch,+ s.colorBySpeed !== null ? 1 : 0,+ s.colorBySpeed ?? 1,+ );+ const colors = s.colors.slice(0, MAX_COLORS);+ v(21, colors.length, view.scaleX, view.scaleY, s.opacity);+ colors.forEach((c, i) => v(22 + i, c[0], c[1], c[2], c[3]));+}++function stateFor(program: ParticleProgram, device: GPUDevice): SystemState {+ const existing = program.particleState as SystemState | undefined;+ if (existing && existing.device === device) return existing;+ const s = program.particleStatic;+ const state: SystemState = {+ device,+ particles: device.createBuffer({+ label: "particles",+ size: s.max * PARTICLE_BYTES,+ usage:+ GPUBufferUsage.STORAGE |+ GPUBufferUsage.COPY_SRC |+ GPUBufferUsage.COPY_DST,+ }),+ frame: -1,+ emitted: 0,+ carry: 0,+ snapshots: new Map(),+ maskBuffer: null,+ maskSize: 0,+ ubo: device.createBuffer({+ label: "particles_params",+ size: PARAM_VEC4S * 16,+ usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,+ }),+ };+ program.particleState = state;+ return state;+}++/** Uploads this frame's (and the previous frame's) silhouette; a 1-float buffer when there is none. */+function uploadMask(state: SystemState, f: ParticleFrame): GPUBuffer {+ const m = f.push?.mask;+ const cells = m ? m.width * m.height : 0;+ const size = Math.max(16, cells * 2 * 4);+ if (!state.maskBuffer || state.maskSize < size) {+ state.maskBuffer?.destroy();+ state.maskBuffer = state.device.createBuffer({+ label: "particles_mask",+ size,+ usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,+ });+ state.maskSize = size;+ }+ if (m) {+ const both = new Float32Array(cells * 2);+ both.set(m.data.subarray(0, cells));+ both.set((f.push?.previous ?? m).data.subarray(0, cells), cells);+ state.device.queue.writeBuffer(state.maskBuffer, 0, both);+ }+ return state.maskBuffer;+}++/** Steps the system from the frame it holds to `frame` (layer-local), restoring a snapshot to go back. */+function advance(+ program: ParticleProgram,+ state: SystemState,+ frame: number,+ compositionStart: number,+ fps: number,+): void {+ const { device } = state;+ const s = program.particleStatic;+ if (frame < state.frame || state.frame < 0) {+ let best = -1;+ for (const k of state.snapshots.keys())+ if (k <= frame && k > best) best = k;+ const encoder = device.createCommandEncoder({ label: "particles_restore" });+ if (best >= 0) {+ const snap = state.snapshots.get(best) as {+ buffer: GPUBuffer;+ emitted: number;+ carry: number;+ };+ encoder.copyBufferToBuffer(+ snap.buffer,+ 0,+ state.particles,+ 0,+ s.max * PARTICLE_BYTES,+ );+ state.emitted = snap.emitted;+ state.carry = snap.carry;+ state.frame = best;+ } else {+ encoder.clearBuffer(state.particles);+ state.emitted = 0;+ state.carry = 0;+ state.frame = -1;+ }+ device.queue.submit([encoder.finish()]);+ }++ const p = getPipelines(device);+ const params = new Float32Array(PARAM_VEC4S * 4);+ const substeps = Math.max(1, Math.round(s.substeps));+ const frameDt = 1 / fps;+ const prewarmFrames = Math.round(s.prewarm * fps);++ // state.frame = -1 means "before the first frame": prewarm first, then frame 0.+ for (let f = state.frame + 1; f <= frame; f++) {+ const steps = f === 0 ? prewarmFrames + 1 : 1;+ for (let w = 0; w < steps; w++) {+ const simFrame = f === 0 ? w - prewarmFrames : f;+ const inputs = program.particleFrame(+ compositionStart + Math.max(0, simFrame),+ fps,+ );+ state.carry += Math.max(0, inputs.rate) * frameDt;+ let born = Math.floor(state.carry);+ state.carry -= born;+ born += Math.max(0, Math.round(inputs.burst));+ born = Math.min(born, s.max);+ const mask = uploadMask(state, inputs);+ const bindGroup = device.createBindGroup({+ layout: p.simLayout,+ entries: [+ { binding: 0, resource: { buffer: state.ubo } },+ { binding: 1, resource: { buffer: state.particles } },+ { binding: 2, resource: { buffer: mask } },+ ],+ });+ // One submit per substep: queue writes are ordered with submits, so+ // each substep reads its own parameters.+ for (let k = 0; k < substeps; k++) {+ packParams(+ s,+ inputs,+ params,+ {+ dt: frameDt / substeps,+ time: (simFrame + k / substeps) * frameDt,+ emitStart: state.emitted,+ emitCount: k === 0 ? born : 0,+ frameDt,+ },+ { scaleX: 1, scaleY: 1 },+ );+ device.queue.writeBuffer(state.ubo, 0, params);+ const encoder = device.createCommandEncoder({+ label: "particles_step",+ });+ const pass = encoder.beginComputePass({ label: "particles_sim" });+ pass.setPipeline(p.sim);+ pass.setBindGroup(0, bindGroup);+ pass.dispatchWorkgroups(Math.ceil(s.max / WORKGROUP));+ pass.end();+ device.queue.submit([encoder.finish()]);+ }+ state.emitted += born;+ }+ state.frame = f;+ if (+ f % SNAPSHOT_EVERY === 0 &&+ !state.snapshots.has(f) &&+ state.snapshots.size < MAX_SNAPSHOTS+ ) {+ const buffer = device.createBuffer({+ label: `particles_snapshot_${f}`,+ size: s.max * PARTICLE_BYTES,+ usage: GPUBufferUsage.COPY_SRC | GPUBufferUsage.COPY_DST,+ });+ const encoder = device.createCommandEncoder({+ label: "particles_snapshot",+ });+ encoder.copyBufferToBuffer(+ state.particles,+ 0,+ buffer,+ 0,+ s.max * PARTICLE_BYTES,+ );+ device.queue.submit([encoder.finish()]);+ state.snapshots.set(f, {+ buffer,+ emitted: state.emitted,+ carry: state.carry,+ });+ }+ }+}++export const ParticlesWebGPURenderer: WebGPUNodeRenderer = async (args) => {+ const { ctx, encoder, pass, targetView, targetWidth, targetHeight, props } =+ args;+ const op = props.virtualMedia?.operation as+ | { op?: string; effect?: unknown }+ | undefined;+ if (op?.op !== "Particles" || !isParticleProgram(op.effect)) return;+ const program = op.effect;+ const s = program.particleStatic;++ const fps = props.fps || 30;+ const local = Math.max(0, Math.round(props.frame ?? 0));+ const compositionFrame = Math.round(props.compositionFrame ?? local);+ const compositionStart = compositionFrame - local;++ // Simulation runs in its own submissions, ahead of this frame's draw.+ const state = stateFor(program, ctx.device);+ advance(program, state, local, compositionStart, fps);++ pass.end();+ const params = new Float32Array(PARAM_VEC4S * 4);+ packParams(+ s,+ program.particleFrame(compositionFrame, fps),+ params,+ { dt: 0, time: 0, emitStart: 0, emitCount: 0, frameDt: 1 / fps },+ {+ scaleX: targetWidth / s.layer.width,+ scaleY: targetHeight / s.layer.height,+ },+ );+ const ubo = ctx.renderer.getTemporaryBuffer(params);+ const draw = ctx.renderer.beginFrame(+ encoder,+ targetView,+ { r: 0, g: 0, b: 0, a: 0 },+ targetWidth,+ targetHeight,+ "load",+ );+ draw.setPipeline(drawPipeline(ctx.device, ctx.renderer.format, s.blend));+ draw.setBindGroup(+ 0,+ ctx.device.createBindGroup({+ layout: getPipelines(ctx.device).drawLayout,+ entries: [+ { binding: 0, resource: { buffer: ubo } },+ { binding: 1, resource: { buffer: state.particles } },+ ],+ }),+ );+ draw.draw(6, s.max);+ draw.end();+};++export type { ParticleMask };diff --git a/nodes/node-particles/src/shared/index.ts b/nodes/node-particles/src/shared/index.tsnew file mode 100644index 00000000..939ffda6--- /dev/null+++ b/nodes/node-particles/src/shared/index.ts@@ -0,0 +1 @@+export * from "./program.js";diff --git a/nodes/node-particles/src/shared/program.ts b/nodes/node-particles/src/shared/program.tsnew file mode 100644index 00000000..d62cbd11--- /dev/null+++ b/nodes/node-particles/src/shared/program.ts@@ -0,0 +1,118 @@+/**+ * The contract between a particle system's authoring object (framefields'+ * `Particles` effect, carried on the op as `effect`) and this renderer.+ * Static settings are read once per system; per-frame inputs (driven values,+ * a performer's silhouette) are asked for each simulated frame.+ */++export type EmitterShape = "point" | "line" | "rect" | "circle" | "ring";+export type ParticleShape = "circle" | "soft" | "square" | "spark";+export type ParticleBlend = "normal" | "add" | "screen";+export type ParticleBounds = "none" | "kill" | "bounce" | "wrap";++export interface ParticleStatic {+ /** Capacity: the most particles alive at once. */+ max: number;+ seed: number;+ /** Layer box in composition px (sampling silhouettes needs the origin). */+ layer: { x: number; y: number; width: number; height: number };+ emitter: {+ shape: EmitterShape;+ x: number;+ y: number;+ width: number;+ height: number;+ radius: number;+ /** Aim away from the emitter's center instead of along `direction`. */+ radial: boolean;+ };+ /** Seconds. */+ lifetime: [number, number];+ /** px/s. */+ speed: [number, number];+ /** Degrees, 0 = right, -90 = up. */+ direction: number;+ /** Degrees around `direction`. */+ spread: number;+ bounds: ParticleBounds;+ /** Floor line in layer px, or null. */+ floor: number | null;+ bounce: number;+ /** px at birth and death. */+ size: [number, number];+ /** 0–1 random scale per particle. */+ sizeJitter: number;+ /** Fractions of life spent fading in and out. */+ fade: [number, number];+ shape: ParticleShape;+ /** Spark length per px/s of speed. */+ stretch: number;+ blend: ParticleBlend;+ /** Up to 8 colors, evenly spaced over life (or speed). */+ colors: Array<[number, number, number, number]>;+ /** Color by speed instead of age: speed (px/s) that reaches the last color. */+ colorBySpeed: number | null;+ opacity: number;+ substeps: number;+ /** Seconds simulated before the first frame, so it starts full. */+ prewarm: number;+}++export interface ParticleAttractor {+ x: number;+ y: number;+ /** px/s² at the center; negative repels. */+ strength: number;+ /** Falloff radius, px. */+ radius: number;+}++/** A silhouette grid (0–1 per cell) and how composition px map onto it. */+export interface ParticleMask {+ data: Float32Array;+ width: number;+ height: number;+ /** cell = (px − origin) / scale − 0.5 */+ originX: number;+ originY: number;+ scaleX: number;+ scaleY: number;+}++export interface ParticleFrame {+ /** Emitter position this frame, when it moves (layer px); births spread back toward `previous`. */+ emitter?: { x: number; y: number; previousX: number; previousY: number };+ /** Particles per second this frame. */+ rate: number;+ /** Extra particles born this frame. */+ burst: number;+ gravity: [number, number];+ /** Velocity damping per second. */+ drag: number;+ noise: { strength: number; scale: number; speed: number };+ attractors: ParticleAttractor[];+ push: {+ strength: number;+ /** Extra push where the silhouette just arrived. */+ sweep: number;+ mask: ParticleMask | null;+ previous: ParticleMask | null;+ } | null;+}++export interface ParticleProgram {+ readonly particleStatic: ParticleStatic;+ /** Inputs for composition frame `frame` (layer-local frames are the effect's concern). */+ particleFrame(frame: number, fps: number): ParticleFrame;+ /** Renderer-owned state, kept on the authoring object so it lives as long as the system. */+ particleState?: unknown;+}++export function isParticleProgram(value: unknown): value is ParticleProgram {+ return (+ typeof value === "object" &&+ value !== null &&+ "particleStatic" in value &&+ typeof (value as ParticleProgram).particleFrame === "function"+ );+}diff --git a/nodes/node-particles/tsconfig.json b/nodes/node-particles/tsconfig.jsonnew file mode 100644index 00000000..5a0b963f--- /dev/null+++ b/nodes/node-particles/tsconfig.json@@ -0,0 +1,12 @@+{+ "extends": "@framefields/tsconfig/node-library.json",+ "compilerOptions": {+ "baseUrl": ".",+ "paths": {+ "@/*": ["./src/*"]+ },+ "experimentalDecorators": true,+ "emitDecoratorMetadata": true+ },+ "include": ["src"]+}diff --git a/nodes/node-particles/tsdown.config.ts b/nodes/node-particles/tsdown.config.tsnew file mode 100644index 00000000..73f5c2fc--- /dev/null+++ b/nodes/node-particles/tsdown.config.ts@@ -0,0 +1,13 @@+import { defineConfig } from "tsdown";++export default defineConfig({+ entry: {+ index: "src/index.ts",+ renderer: "src/renderers/index.ts",+ },+ format: ["esm"],+ dts: true,+ clean: true,+ sourcemap: true,+ treeshake: true,+});diff --git a/nodes/node-trails/package.json b/nodes/node-trails/package.jsonnew file mode 100644index 00000000..3400fbc3--- /dev/null+++ b/nodes/node-trails/package.json@@ -0,0 +1,46 @@+{+ "name": "@framefields/node-trails",+ "version": "2.2.0",+ "private": false,+ "framefields": {+ "type": "Trails",+ "kind": "effect",+ "schema": "TrailsNodeConfigSchema"+ },+ "type": "module",+ "license": "Apache-2.0",+ "exports": {+ ".": {+ "types": "./dist/index.d.mts",+ "development": "./src/index.ts",+ "import": "./dist/index.mjs",+ "default": "./dist/index.mjs"+ },+ "./renderer": {+ "types": "./dist/renderer.d.mts",+ "development": "./src/renderers/index.ts",+ "import": "./dist/renderer.mjs",+ "default": "./dist/renderer.mjs"+ }+ },+ "scripts": {+ "build": "tsdown",+ "dev": "tsdown --watch",+ "clean": "rm -rf dist .turbo",+ "typecheck": "tsc --noEmit"+ },+ "dependencies": {+ "@framefields/node-sdk": "workspace:*",+ "zod": "^4.3.6"+ },+ "devDependencies": {+ "@framefields/tsconfig": "workspace:*",+ "@types/node": "^24.1.0",+ "tsdown": "^0.19.0",+ "typescript": "^5.8.3",+ "webgpu": "^0.4.0"+ },+ "publishConfig": {+ "access": "public"+ }+}diff --git a/nodes/node-trails/src/index.ts b/nodes/node-trails/src/index.tsnew file mode 100644index 00000000..944d702e--- /dev/null+++ b/nodes/node-trails/src/index.ts@@ -0,0 +1,2 @@+export * from "./renderers/index.js";+export * from "./shared/index.js";diff --git a/nodes/node-trails/src/renderers/index.ts b/nodes/node-trails/src/renderers/index.tsnew file mode 100644index 00000000..ad407f02--- /dev/null+++ b/nodes/node-trails/src/renderers/index.ts@@ -0,0 +1,7 @@+import { defineRenderer } from "@framefields/node-sdk/renderer";+import { TrailsWebGPURenderer } from "./webgpu-renderer.js";++export default defineRenderer({+ WebGPURenderer: TrailsWebGPURenderer,+});+export { TrailsWebGPURenderer };diff --git a/nodes/node-trails/src/renderers/webgpu-renderer.ts b/nodes/node-trails/src/renderers/webgpu-renderer.tsnew file mode 100644index 00000000..6b888265--- /dev/null+++ b/nodes/node-trails/src/renderers/webgpu-renderer.ts@@ -0,0 +1,272 @@+/// <reference types="webgpu" />+/**+ * Trails: the layer over a decaying, optionally zooming / turning / drifting+ * / hue-shifting copy of its own history (echoes, light painting, feedback+ * tunnels). History lives in textures on the effect object and advances one+ * frame at a time; after a jump the last frames are replayed to rebuild it.+ */+import type { WebGPUNodeRenderer } from "@framefields/node-sdk";+import {+ drawChildToTexture,+ drawFullscreenPass,+ drawTextureToTarget,+ FULLSCREEN_QUAD_WGSL,+ fragmentBindGroupLayout,+ perDevice,+ type WebGPUNodeRendererArgs,+} from "@framefields/node-sdk/gpu";+import type { TrailsMode } from "../shared/index.js";++const HISTORY_FORMAT: GPUTextureFormat = "rgba16float";++interface TrailsValues {+ decay: number;+ mode: TrailsMode;+ zoom: number;+ rotation: number;+ offsetX: number;+ offsetY: number;+ hueShift: number;+ warmup: number;+}++const TRAILS_WGSL = /* wgsl */ `+struct U { a : vec4<f32>, b : vec4<f32> };+@group(0) @binding(0) var<uniform> u : U;+@group(0) @binding(1) var historyTex : texture_2d<f32>;+@group(0) @binding(2) var currentTex : texture_2d<f32>;+@group(0) @binding(3) var samp : sampler;+${FULLSCREEN_QUAD_WGSL}++// Hue rotation in YIQ: linear, so it works on premultiplied color.+fn hueRotate(c : vec3<f32>, angle : f32) -> vec3<f32> {+ let yiq = mat3x3<f32>(0.299, 0.596, 0.211, 0.587, -0.274, -0.523, 0.114, -0.322, 0.312) * c;+ let cs = cos(angle);+ let sn = sin(angle);+ let rot = vec3<f32>(yiq.x, yiq.y * cs - yiq.z * sn, yiq.y * sn + yiq.z * cs);+ return mat3x3<f32>(1.0, 1.0, 1.0, 0.956, -0.272, -1.106, 0.621, -0.647, 1.703) * rot;+}++@fragment fn fs(in : VSOut) -> @location(0) vec4<f32> {+ let decay = u.a.x;+ let mode = u.a.y;+ let zoom = max(u.a.z, 1e-3);+ let angle = u.a.w;+ let offset = u.b.xy;+ let hue = u.b.z;+ let fresh = u.b.w;+ let cur = textureSampleLevel(currentTex, samp, in.uv, 0.0);+ if (fresh > 0.5) { return cur; }+ let c = in.uv - 0.5;+ let cs = cos(-angle);+ let sn = sin(-angle);+ let src = vec2<f32>(c.x * cs - c.y * sn, c.x * sn + c.y * cs) / zoom + 0.5 - offset;+ var hist = vec4<f32>(0.0);+ if (all(src >= vec2<f32>(0.0)) && all(src <= vec2<f32>(1.0))) {+ hist = textureSampleLevel(historyTex, samp, src, 0.0);+ }+ if (hue != 0.0) { hist = vec4<f32>(max(hueRotate(hist.rgb, hue), vec3<f32>(0.0)), hist.a); }+ hist *= decay;+ if (mode < 0.5) { return cur + hist * (1.0 - cur.a); }+ if (mode < 1.5) { return vec4<f32>(cur.rgb + hist.rgb, min(cur.a + hist.a, 1.0)); }+ if (mode < 2.5) { return max(cur, hist); }+ // blend: a running average, so opaque footage shows its own echoes.+ return mix(cur, hist / max(decay, 1e-4), decay);+}+`;++const resources = perDevice((device: GPUDevice) => {+ const layout = fragmentBindGroupLayout(+ device,+ ["uniform", "texture", "texture", "sampler"],+ "trails_layout",+ );+ const module = device.createShaderModule({+ code: TRAILS_WGSL,+ label: "trails.wgsl",+ });+ return {+ layout,+ pipeline: device.createRenderPipeline({+ label: "trails",+ layout: device.createPipelineLayout({ bindGroupLayouts: [layout] }),+ vertex: { module, entryPoint: "vs" },+ fragment: {+ module,+ entryPoint: "fs",+ targets: [{ format: HISTORY_FORMAT }],+ },+ primitive: { topology: "triangle-strip" },+ }),+ sampler: device.createSampler({ magFilter: "linear", minFilter: "linear" }),+ };+});++interface History {+ device: GPUDevice;+ width: number;+ height: number;+ /** Ping-pong history; `front` holds the latest. */+ front: GPUTexture;+ back: GPUTexture;+ /** Layer frame `front` holds; -1 = none. */+ frame: number;+}++interface FeedbackEffect {+ resolveUniforms(ctx?: unknown): Record<string, unknown>;+ feedbackHistory?: History;+}++function historyFor(+ effect: FeedbackEffect,+ device: GPUDevice,+ width: number,+ height: number,+): History {+ const h = effect.feedbackHistory;+ if (h && h.device === device && h.width === width && h.height === height)+ return h;+ h?.front.destroy();+ h?.back.destroy();+ const make = (label: string) =>+ device.createTexture({+ label,+ size: [width, height],+ format: HISTORY_FORMAT,+ usage:+ GPUTextureUsage.RENDER_ATTACHMENT |+ GPUTextureUsage.TEXTURE_BINDING |+ GPUTextureUsage.COPY_SRC |+ GPUTextureUsage.COPY_DST,+ });+ const next: History = {+ device,+ width,+ height,+ front: make("trails_front"),+ back: make("trails_back"),+ frame: -1,+ };+ effect.feedbackHistory = next;+ return next;+}++async function childAt(+ args: WebGPUNodeRendererArgs,+ frame: number,+ compositionStart: number,+ h: History,+) {+ const child = args.props.virtualMedia?.children?.[0];+ if (!child) return null;+ return drawChildToTexture(args, child, {+ exclude: [h.front, h.back],+ props: {+ ...args.props,+ virtualMedia: child,+ frame,+ compositionFrame: compositionStart + frame,+ },+ });+}++function swap(h: History): void {+ const t = h.front;+ h.front = h.back;+ h.back = t;+}++async function stepTrails(+ args: WebGPUNodeRendererArgs,+ h: History,+ v: TrailsValues,+ frame: number,+ compositionStart: number,+ fresh: boolean,+): Promise<void> {+ const cur = await childAt(args, frame, compositionStart, h);+ if (!cur) return;+ const r = resources(args.ctx.device);+ const uniforms = new Float32Array([+ v.decay,+ { over: 0, add: 1, max: 2, blend: 3 }[v.mode] ?? 0,+ v.zoom,+ (v.rotation * Math.PI) / 180,+ v.offsetX / h.width,+ v.offsetY / h.height,+ (v.hueShift * Math.PI) / 180,+ fresh ? 1 : 0,+ ]);+ drawFullscreenPass(args.encoder, h.back.createView(), r.pipeline, [+ args.ctx.device.createBindGroup({+ layout: r.layout,+ entries: [+ {+ binding: 0,+ resource: { buffer: args.ctx.renderer.getTemporaryBuffer(uniforms) },+ },+ { binding: 1, resource: h.front.createView() },+ { binding: 2, resource: cur.view },+ { binding: 3, resource: r.sampler },+ ],+ }),+ ]);+ swap(h);+ h.frame = frame;+}++function num(v: unknown, fallback: number): number {+ return typeof v === "number" && Number.isFinite(v) ? v : fallback;+}++export const TrailsWebGPURenderer: WebGPUNodeRenderer = async (args) => {+ const { ctx, pass, targetWidth, targetHeight, props } = args;+ const op = props.virtualMedia?.operation as+ | { op?: string; effect?: FeedbackEffect }+ | undefined;+ if (op?.op !== "Trails" || !op.effect?.resolveUniforms) return;+ if (!props.virtualMedia?.children?.[0]) return;+ pass.end();++ const fps = props.fps || 30;+ const local = Math.max(0, Math.round(props.frame ?? 0));+ const compositionFrame = Math.round(props.compositionFrame ?? local);+ const compositionStart = compositionFrame - local;+ const raw = op.effect.resolveUniforms({+ frame: compositionFrame,+ fps,+ time: compositionFrame / fps,+ });+ const h = historyFor(op.effect, ctx.device, targetWidth, targetHeight);++ {+ const v: TrailsValues = {+ decay: Math.min(1, Math.max(0, num(raw.decay, 0.9))),+ mode: (raw.mode as TrailsValues["mode"]) ?? "blend",+ zoom: num(raw.zoom, 1),+ rotation: num(raw.rotation, 0),+ offsetX: num(raw.offsetX, 0),+ offsetY: num(raw.offsetY, 0),+ hueShift: num(raw.hueShift, 0),+ warmup: Math.max(0, Math.round(num(raw.warmup, 60))),+ };+ if (h.frame !== local) {+ let start = local;+ if (h.frame === local - 1) start = local;+ else {+ // Rebuild: replay until older frames have faded below 0.5%.+ const needed =+ v.decay > 0 && v.decay < 1+ ? Math.ceil(Math.log(0.005) / Math.log(v.decay))+ : v.warmup;+ start = Math.max(0, local - Math.min(v.warmup, needed));+ h.frame = -1;+ }+ for (let f = start; f <= local; f++) {+ await stepTrails(args, h, v, f, compositionStart, h.frame < 0);+ }+ }+ }+ drawTextureToTarget(args, h.front);+};diff --git a/nodes/node-trails/src/shared/config.ts b/nodes/node-trails/src/shared/config.tsnew file mode 100644index 00000000..3ef5f334--- /dev/null+++ b/nodes/node-trails/src/shared/config.ts@@ -0,0 +1,61 @@+import { configBuilder } from "@framefields/node-sdk";+import { z } from "zod";++export const TRAILS_MODES = ["blend", "over", "add", "max"] as const;++export const trailsConfig = configBuilder()+ .field("decay", z.number().min(0).max(1).default(0.9), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Decay",+ description:+ "History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails.",+ })+ .field("mode", z.enum(TRAILS_MODES).default("blend"), {+ bindable: false,+ label: "Mode",+ description:+ "blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest.",+ })+ .field("zoom", z.number().min(0.5).max(2).default(1), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Zoom",+ description: "History scale per frame; 1.02 pulls the past into a tunnel.",+ })+ .field("rotation", z.number().min(-45).max(45).default(0), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Rotation",+ description: "History rotation per frame, degrees.",+ })+ .field("offsetX", z.number().min(-500).max(500).default(0), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Offset X",+ description: "History drift per frame, px.",+ })+ .field("offsetY", z.number().min(-500).max(500).default(0), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Offset Y",+ description: "History drift per frame, px.",+ })+ .field("hueShift", z.number().min(-180).max(180).default(0), {+ bindable: true,+ dataTypes: ["Number", "Signal"],+ label: "Hue Shift",+ description:+ "History hue rotation per frame, degrees (psychedelic cycling).",+ })+ .field("warmup", z.number().int().min(0).max(240).default(60), {+ bindable: false,+ label: "Warmup",+ description:+ "Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render).",+ })+ .build();++export const TrailsNodeConfigSchema = trailsConfig.schema;+export type TrailsNodeConfig = z.infer<typeof TrailsNodeConfigSchema>;+export type TrailsMode = (typeof TRAILS_MODES)[number];diff --git a/nodes/node-trails/src/shared/index.ts b/nodes/node-trails/src/shared/index.tsnew file mode 100644index 00000000..d1bc97e8--- /dev/null+++ b/nodes/node-trails/src/shared/index.ts@@ -0,0 +1 @@+export * from "./config.js";diff --git a/nodes/node-trails/tsconfig.json b/nodes/node-trails/tsconfig.jsonnew file mode 100644index 00000000..5a0b963f--- /dev/null+++ b/nodes/node-trails/tsconfig.json@@ -0,0 +1,12 @@+{+ "extends": "@framefields/tsconfig/node-library.json",+ "compilerOptions": {+ "baseUrl": ".",+ "paths": {+ "@/*": ["./src/*"]+ },+ "experimentalDecorators": true,+ "emitDecoratorMetadata": true+ },+ "include": ["src"]+}diff --git a/nodes/node-trails/tsdown.config.ts b/nodes/node-trails/tsdown.config.tsnew file mode 100644index 00000000..73f5c2fc--- /dev/null+++ b/nodes/node-trails/tsdown.config.ts@@ -0,0 +1,13 @@+import { defineConfig } from "tsdown";++export default defineConfig({+ entry: {+ index: "src/index.ts",+ renderer: "src/renderers/index.ts",+ },+ format: ["esm"],+ dts: true,+ clean: true,+ sourcemap: true,+ treeshake: true,+});diff --git a/packages/framefields/package.json b/packages/framefields/package.jsonindex 64b74ecb..9f8466dc 100644--- a/packages/framefields/package.json+++ b/packages/framefields/package.json@@ -146,6 +146,7 @@"@framefields/node-mesh-warp": "workspace:*","@framefields/node-modulate": "workspace:*","@framefields/node-paint": "workspace:*",+ "@framefields/node-particles": "workspace:*","@framefields/node-patch-heal": "workspace:*","@framefields/node-refine-edge": "workspace:*","@framefields/node-refraction-caustics": "workspace:*",diff --git a/packages/framefields/src/clip-vision.ts b/packages/framefields/src/clip-vision.tsindex a0df0a9d..dd07098d 100644--- a/packages/framefields/src/clip-vision.ts+++ b/packages/framefields/src/clip-vision.ts@@ -457,6 +457,35 @@ export class PersonTrack {return { x: read("x"), y: read("y"), visibility: read("visibility") };}+ /**+ * @internal The softened silhouette grid at a composition frame and how+ * placed px map onto it (`cell = (px − origin) / scale − 0.5`), for GPU+ * consumers such as particles. Null when no person was found.+ */+ silhouetteGrid(frame: number): {+ data: Float32Array;+ width: number;+ height: number;+ originX: number;+ originY: number;+ scaleX: number;+ scaleY: number;+ } | null {+ const grid = this.clip.blurredMask(Math.round(this.clip.clipFrame(frame)));+ if (!grid) return null;+ const { cell, width, height } = this.clip.grid();+ const t = this.transform();+ return {+ data: grid,+ width,+ height,+ originX: t.ox,+ originY: t.oy,+ scaleX: cell * t.sx,+ scaleY: cell * t.sy,+ };+ }+/*** The silhouette near a point at a composition frame: `alpha` 0–1 (softened* over a few cells; inside > 0.5) and its gradient `gx`, `gy` per px,diff --git a/packages/framefields/src/effects/generated/catalog.json b/packages/framefields/src/effects/generated/catalog.jsonindex 42968b1d..71cbad13 100644--- a/packages/framefields/src/effects/generated/catalog.json+++ b/packages/framefields/src/effects/generated/catalog.json@@ -712,6 +712,51 @@}]},+ {+ "name": "Datamosh",+ "op": "Datamosh",+ "kind": "effect",+ "factory": "datamosh",+ "node": "node-datamosh",+ "aliases": [],+ "custom": null,+ "props": [+ {+ "name": "keyframes",+ "type": "number[]",+ "signal": false,+ "default": [],+ "description": "Layer frames that show the clean picture again (I-frames). The first frame always does."+ },+ {+ "name": "strength",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 1,+ "min": 0,+ "max": 10,+ "description": "How far pixels are dragged along the motion."+ },+ {+ "name": "bleed",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 0,+ "min": 0,+ "max": 1,+ "description": "Clean picture mixed back in per frame, 0–1."+ },+ {+ "name": "warmup",+ "type": "number",+ "signal": false,+ "default": 48,+ "min": 1,+ "max": 240,+ "description": "Most frames replayed to rebuild the smear after a jump."+ }+ ]+ },{"name": "DisplacementMap","op": "DisplacementMap",@@ -2097,6 +2142,88 @@}]},+ {+ "name": "Trails",+ "op": "Trails",+ "kind": "effect",+ "factory": "trails",+ "node": "node-trails",+ "aliases": [],+ "custom": null,+ "props": [+ {+ "name": "decay",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 0.9,+ "min": 0,+ "max": 1,+ "description": "History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails."+ },+ {+ "name": "mode",+ "type": "\"blend\" | \"over\" | \"add\" | \"max\"",+ "signal": false,+ "default": "blend",+ "enum": ["blend", "over", "add", "max"],+ "description": "blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest."+ },+ {+ "name": "zoom",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 1,+ "min": 0.5,+ "max": 2,+ "description": "History scale per frame; 1.02 pulls the past into a tunnel."+ },+ {+ "name": "rotation",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 0,+ "min": -45,+ "max": 45,+ "description": "History rotation per frame, degrees."+ },+ {+ "name": "offsetX",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 0,+ "min": -500,+ "max": 500,+ "description": "History drift per frame, px."+ },+ {+ "name": "offsetY",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 0,+ "min": -500,+ "max": 500,+ "description": "History drift per frame, px."+ },+ {+ "name": "hueShift",+ "type": "EffectProp<number>",+ "signal": true,+ "default": 0,+ "min": -180,+ "max": 180,+ "description": "History hue rotation per frame, degrees (psychedelic cycling)."+ },+ {+ "name": "warmup",+ "type": "number",+ "signal": false,+ "default": 60,+ "min": 0,+ "max": 240,+ "description": "Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render)."+ }+ ]+ },{"name": "Vignette","op": "Vignette",diff --git a/packages/framefields/src/effects/generated/classes.ts b/packages/framefields/src/effects/generated/classes.tsindex 150c3545..2043c83f 100644--- a/packages/framefields/src/effects/generated/classes.ts+++ b/packages/framefields/src/effects/generated/classes.ts@@ -1217,6 +1217,54 @@ export class CornerPin extends Effect<CornerPinProps> {}}+export interface DatamoshProps {+ /**+ * Layer frames that show the clean picture again (I-frames). The first frame always does.+ * @default []+ */+ keyframes?: number[];+ /**+ * How far pixels are dragged along the motion.+ * Range 0 to 10. Accepts a signal.+ * @default 1+ */+ strength?: EffectProp<number>;+ /**+ * Clean picture mixed back in per frame, 0–1.+ * Range 0 to 1. Accepts a signal.+ * @default 0+ */+ bleed?: EffectProp<number>;+ /**+ * Most frames replayed to rebuild the smear after a jump.+ * Range 1 to 240.+ * @default 48+ */+ warmup?: number;+}++const datamoshDefaults: DatamoshProps = {+ keyframes: [],+ strength: 1,+ bleed: 0,+ warmup: 48,+};++export class Datamosh extends Effect<DatamoshProps> {+ public readonly op = "Datamosh";++ public declare keyframes: number[];+ public declare strength: number;+ public declare bleed: number;+ public declare warmup: number;++ constructor(config: DatamoshProps = {}) {+ super(+ withDefaults("Datamosh", datamoshDefaults, config, effectMeta.Datamosh),+ );+ }+}+export interface DisplacementMapProps {/*** Horizontal displacement strength in pixels. Can be modulated by a static number or a dynamic signal.@@ -2627,6 +2675,84 @@ export class StereoPanning extends Effect<StereoPanningProps> {}}+export interface TrailsProps {+ /**+ * History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails.+ * Range 0 to 1. Accepts a signal.+ * @default 0.9+ */+ decay?: EffectProp<number>;+ /**+ * blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest.+ * @default "blend"+ */+ mode?: "blend" | "over" | "add" | "max";+ /**+ * History scale per frame; 1.02 pulls the past into a tunnel.+ * Range 0.5 to 2. Accepts a signal.+ * @default 1+ */+ zoom?: EffectProp<number>;+ /**+ * History rotation per frame, degrees.+ * Range -45 to 45. Accepts a signal.+ * @default 0+ */+ rotation?: EffectProp<number>;+ /**+ * History drift per frame, px.+ * Range -500 to 500. Accepts a signal.+ * @default 0+ */+ offsetX?: EffectProp<number>;+ /**+ * History drift per frame, px.+ * Range -500 to 500. Accepts a signal.+ * @default 0+ */+ offsetY?: EffectProp<number>;+ /**+ * History hue rotation per frame, degrees (psychedelic cycling).+ * Range -180 to 180. Accepts a signal.+ * @default 0+ */+ hueShift?: EffectProp<number>;+ /**+ * Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render).+ * Range 0 to 240.+ * @default 60+ */+ warmup?: number;+}++const trailsDefaults: TrailsProps = {+ decay: 0.9,+ mode: "blend",+ zoom: 1,+ rotation: 0,+ offsetX: 0,+ offsetY: 0,+ hueShift: 0,+ warmup: 60,+};++export class Trails extends Effect<TrailsProps> {+ public readonly op = "Trails";++ public declare decay: number;+ public declare mode: "blend" | "over" | "add" | "max";+ public declare zoom: number;+ public declare rotation: number;+ public declare offsetX: number;+ public declare offsetY: number;+ public declare hueShift: number;+ public declare warmup: number;++ constructor(config: TrailsProps = {}) {+ super(withDefaults("Trails", trailsDefaults, config, effectMeta.Trails));+ }+}+export interface VignetteProps {/*** The intensity of the vignette darkening effect. 0 = no vignette.diff --git a/packages/framefields/src/effects/generated/index.ts b/packages/framefields/src/effects/generated/index.tsindex a03cb351..fb2cadae 100644--- a/packages/framefields/src/effects/generated/index.ts+++ b/packages/framefields/src/effects/generated/index.ts@@ -29,6 +29,8 @@ export {CornerPin,type CornerPinPointsItem,type CornerPinProps,+ Datamosh,+ type DatamoshProps,DisplacementMap,type DisplacementMapProps,ExtractLUT,@@ -78,6 +80,8 @@ export {StereoPanning,type StereoPanningProps,type TonalShift,+ Trails,+ type TrailsProps,Vignette,type VignetteProps,} from "./classes.js";diff --git a/packages/framefields/src/effects/generated/meta.ts b/packages/framefields/src/effects/generated/meta.tsindex 67175de6..f0c87007 100644--- a/packages/framefields/src/effects/generated/meta.ts+++ b/packages/framefields/src/effects/generated/meta.ts@@ -308,6 +308,21 @@ export const effectMeta = {CornerPin: {points: {},},+ Datamosh: {+ keyframes: {},+ strength: {+ min: 0,+ max: 10,+ },+ bleed: {+ min: 0,+ max: 1,+ },+ warmup: {+ min: 1,+ max: 240,+ },+ },DisplacementMap: {strengthX: {min: 0,@@ -941,6 +956,39 @@ export const effectMeta = {max: 1,},},+ Trails: {+ decay: {+ min: 0,+ max: 1,+ },+ mode: {+ enum: ["blend", "over", "add", "max"],+ },+ zoom: {+ min: 0.5,+ max: 2,+ },+ rotation: {+ min: -45,+ max: 45,+ },+ offsetX: {+ min: -500,+ max: 500,+ },+ offsetY: {+ min: -500,+ max: 500,+ },+ hueShift: {+ min: -180,+ max: 180,+ },+ warmup: {+ min: 0,+ max: 240,+ },+ },Vignette: {strength: {min: 0,diff --git a/packages/framefields/src/effects/generated/registry.ts b/packages/framefields/src/effects/generated/registry.tsindex 9390c894..78037ed2 100644--- a/packages/framefields/src/effects/generated/registry.ts+++ b/packages/framefields/src/effects/generated/registry.ts@@ -28,6 +28,8 @@ import {type ColorKeyProps,CornerPin,type CornerPinProps,+ Datamosh,+ type DatamoshProps,DisplacementMap,type DisplacementMapProps,ExtractLUT,@@ -62,6 +64,8 @@ import {type RotateProps,StereoPanning,type StereoPanningProps,+ Trails,+ type TrailsProps,Vignette,type VignetteProps,} from "./classes.js";@@ -85,6 +89,7 @@ export const effectFactories = {colorBalance: (config?: ColorBalanceProps) => new ColorBalance(config),colorKey: (config?: ColorKeyProps) => new ColorKey(config),cornerPin: (config?: CornerPinProps) => new CornerPin(config),+ datamosh: (config?: DatamoshProps) => new Datamosh(config),displacementMap: (config?: DisplacementMapProps) =>new DisplacementMap(config),extractLUT: (config?: ExtractLUTProps) => new ExtractLUT(config),@@ -104,5 +109,6 @@ export const effectFactories = {resizerScaler: (config?: ResizerScalerProps) => new ResizerScaler(config),rotate: (config?: RotateProps) => new Rotate(config),stereoPanning: (config?: StereoPanningProps) => new StereoPanning(config),+ trails: (config?: TrailsProps) => new Trails(config),vignette: (config?: VignetteProps) => new Vignette(config),} as const;diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.tsindex 20acb7aa..9239e8b0 100644--- a/packages/framefields/src/index.ts+++ b/packages/framefields/src/index.ts@@ -241,6 +241,7 @@ import {Vision,type VisionProps,} from "./effects/index.js";+import { Particles, type ParticlesOptions } from "./particles.js";import {type FrameGridOptions,HeadlessMediaRenderer,@@ -709,6 +710,8 @@ export {type CurvesChannelPoint,type CurvesProps,CustomEffect,+ Datamosh,+ type DatamoshProps,type DeflickerProps,DepthOfField,type DepthOfFieldProps,@@ -781,6 +784,8 @@ export {TileOffset,type TileOffsetEdgeMode,type TileOffsetProps,+ Trails,+ type TrailsProps,UnsharpMask,type UnsharpMaskProps,Vignette,@@ -793,6 +798,14 @@ export {type LightingRig,type LightingRigOptions,} from "./lighting.js";+export {+ type EmitterShape,+ type ParticleBlend,+ type ParticleBounds,+ type ParticleShape,+ Particles,+ type ParticlesOptions,+} from "./particles.js";export {type Body2DOptions,type Body3DOptions,@@ -1160,10 +1173,14 @@ export class Composition {return clip;}- /** @internal Analyses every `comp.vision` clip for a composition frame. */+ /**+ * @internal Analyses every `comp.vision` clip up to a composition frame+ * (frames already analysed are skipped), so consumers that step frame by+ * frame — physics, particles — always find every frame before it.+ */public async prepareVision(frame: number): Promise<void> {await Promise.all(- [...this._clipVision.values()].map((clip) => clip.ready(frame)),+ [...this._clipVision.values()].map((clip) => clip.ready(0, frame)),);}@@ -2505,6 +2522,41 @@ export const Layer = {children: options.children ?? [],...options,}),+ /**+ * A GPU particle system as a layer: tens of thousands of particles+ * simulated in a compute shader and drawn in one call (see+ * {@link ParticlesOptions}). Positions are in the layer's own px; drivers+ * (rate, gravity, noise, attractors) are read every frame, and a+ * performer can push particles with `push: { person }`.+ */+ particles: (options: ParticlesOptions): AnimatableNode<MediaNode> => {+ const system = new Particles(options);+ // A media layer whose source is the particle op itself: it draws into+ // the layer's own texture, composited like any clip.+ const source = {+ metadata: { width: options.width, height: options.height },+ operation: { ...system.toOperation(), effect: system, dataType: "Video" },+ children: [],+ };+ return withAnimation({+ id: options.id ?? autoId("particles"),+ kind: "media",+ dataType: "Video",+ src: source,+ position: "absolute",+ x: options.x ?? 0,+ y: options.y ?? 0,+ width: options.width,+ height: options.height,+ ...(options.startFrame !== undefined && {+ startFrame: options.startFrame,+ }),+ ...(options.durationFrames !== undefined && {+ durationFrames: options.durationFrames,+ }),+ ...(options.zIndex !== undefined && { zIndex: options.zIndex }),+ } as unknown as MediaNode);+ },section: (source: string | MediaNode | Composition,targetOrOptions: ObjectTrackSignals | SectionOptions = {},diff --git a/packages/framefields/src/particles-render.test.ts b/packages/framefields/src/particles-render.test.tsnew file mode 100644index 00000000..dd981cff--- /dev/null+++ b/packages/framefields/src/particles-render.test.ts@@ -0,0 +1,250 @@+/**+ * GPU particles and temporal effects through the real renderer: particles+ * appear where they are emitted, scrubbing back gives the same frame, and+ * Trails / Datamosh remember earlier frames.+ */+import fs from "node:fs/promises";+import os from "node:os";+import path from "node:path";+import { describe, expect, it } from "vitest";+import {+ Composition,+ Datamosh,+ HeadlessMediaRenderer,+ Layer,+ Layer3D,+ LayerAnimation,+ Trails,+} from "./index.js";+import { decodePngPixels, getPixel } from "./test-utils/pixels.js";++const FPS = 30;++function stage(frames = 60) {+ return new Composition({+ width: 200,+ height: 200,+ fps: FPS,+ durationFrames: frames,+ backgroundColor: "#000000",+ });+}++async function frame(+ c: Composition,+ f: number,+ renderer?: HeadlessMediaRenderer,+) {+ return decodePngPixels(+ await c.renderFrame({ frame: f, ...(renderer && { renderer }) }),+ );+}++function brightPixels(+ img: Awaited<ReturnType<typeof decodePngPixels>>,+ x0 = 0,+ x1 = img.width,+) {+ let n = 0;+ for (let y = 0; y < img.height; y++) {+ for (let x = x0; x < x1; x++) if (getPixel(img, x, y).r > 60) n++;+ }+ return n;+}++describe("Layer.particles", () => {+ it("emits, moves and draws particles", async () => {+ const c = stage();+ c.add(+ Layer.particles({+ width: 200,+ height: 200,+ max: 2000,+ emitter: { shape: "point", x: 100, y: 180, rate: 600 },+ direction: -90,+ spread: 20,+ speed: [200, 260],+ lifetime: [3, 3],+ size: [6, 6],+ color: "#ffffff",+ fade: [0.001, 0.001],+ }),+ );+ const renderer = new HeadlessMediaRenderer();+ const early = await frame(c, 3, renderer);+ const later = await frame(c, 20, renderer);+ // Near the emitter early, risen ~150 px by frame 20.+ expect(getPixel(early, 100, 172).r).toBeGreaterThan(100);+ expect(brightPixels(early)).toBeGreaterThan(5);+ expect(getPixel(later, 100, 60).r).toBeGreaterThan(100);+ // Nothing left or right of the narrow fountain.+ expect(brightPixels(later, 0, 40) + brightPixels(later, 160, 200)).toBe(0);+ });++ it("gives the same frame when scrubbing back (snapshots + replay)", async () => {+ const c = stage(80);+ c.add(+ Layer.particles({+ width: 200,+ height: 200,+ max: 3000,+ emitter: { shape: "rect", rate: 900 },+ noise: { strength: 400, scale: 40 },+ lifetime: [1, 2],+ size: [4, 2],+ seed: 3,+ }),+ );+ const renderer = new HeadlessMediaRenderer();+ const a = await frame(c, 30, renderer);+ await frame(c, 70, renderer);+ const b = await frame(c, 30, renderer);+ expect(Buffer.from(b.pixels).equals(Buffer.from(a.pixels))).toBe(true);+ });+});++describe("moving emitters and glass", () => {+ it("emits where a driven emitter is each frame", async () => {+ const c = stage(40);+ c.add(+ Layer.particles({+ width: 200,+ height: 200,+ max: 3000,+ emitter: { shape: "point", x: (f) => 20 + f * 4, y: 100, rate: 1200 },+ speed: [0, 5],+ lifetime: [0.2, 0.2],+ size: [6, 6],+ fade: [0.001, 0.001],+ color: "#ffffff",+ }),+ );+ const renderer = new HeadlessMediaRenderer();+ let img = await frame(c, 0, renderer);+ for (let f = 1; f <= 30; f++) img = await frame(c, f, renderer);+ // Emitter at x = 140 now; the trail of the last 0.2 s reaches back ~24 px.+ expect(getPixel(img, 139, 100).r).toBeGreaterThan(100);+ expect(getPixel(img, 125, 100).r).toBeGreaterThan(100);+ expect(getPixel(img, 40, 100).r).toBeLessThan(20);+ });++ it("glass shows what is behind it instead of a flat surface", async () => {+ const scene = (material: "glass" | "lit") => {+ const c = stage(2);+ c.add([+ Layer.box({+ position: "absolute",+ x: 0,+ y: 0,+ width: 100,+ height: 200,+ background: "#ff0000",+ }),+ Layer.box({+ position: "absolute",+ x: 100,+ y: 0,+ width: 100,+ height: 200,+ background: "#0000ff",+ }),+ Layer3D.roundedBox({+ width: 120,+ height: 120,+ depth: 120,+ x: 100,+ y: 100,+ z: 0,+ color: "#ffffff",+ material,+ }),+ ]);+ return c;+ };+ const glass = await frame(scene("glass"), 0);+ const left = getPixel(glass, 70, 100);+ const right = getPixel(glass, 130, 100);+ expect(left.r).toBeGreaterThan(left.b + 60);+ expect(right.b).toBeGreaterThan(right.r + 60);+ const lit = await frame(scene("lit"), 0);+ // An opaque white box hides the split.+ expect(+ Math.abs(getPixel(lit, 70, 100).r - getPixel(lit, 130, 100).r),+ ).toBeLessThan(30);+ });+});++describe("temporal effects", () => {+ /** Renders a 30-frame clip of a white square sliding right 4 px per frame on black. */+ async function slidingClip(): Promise<string> {+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-temporal-"));+ const file = path.join(dir, "slide.mp4");+ const c = stage(30);+ c.add(+ Layer.box({+ position: "absolute",+ x: 0,+ y: 80,+ width: 40,+ height: 40,+ background: "#ffffff",+ }).animate(+ LayerAnimation.keys("x", [+ [0, 0],+ [29, 116],+ ]),+ ),+ );+ const r = await c.renderVideo({ outputPath: file, quality: "lossless" });+ await r.cleanup?.();+ return file;+ }++ function withEffect(clip: string, effect: Trails | Datamosh | null) {+ const c = stage(30);+ const layer = Layer.video(clip, {+ position: "absolute",+ x: 0,+ y: 0,+ width: 200,+ height: 200,+ fit: "cover",+ muted: true,+ });+ c.add(effect ? layer.withEffect(effect) : layer);+ return c;+ }++ it("Trails leaves a fading echo behind a moving subject", async () => {+ const clip = await slidingClip();+ const renderer = new HeadlessMediaRenderer();+ const trails = withEffect(clip, new Trails({ decay: 0.8 }));+ let img = await frame(trails, 0, renderer);+ for (let f = 1; f <= 15; f++) img = await frame(trails, f, renderer);+ const plain = await frame(withEffect(clip, null), 15);+ // Square now covers x 60–100; x 50 was covered a few frames ago.+ expect(getPixel(plain, 50, 100).r).toBeLessThan(10);+ const echo = getPixel(img, 50, 100).r;+ expect(echo).toBeGreaterThan(20);+ expect(echo).toBeLessThan(250);+ // Far behind it, the echo has faded out.+ expect(getPixel(img, 5, 100).r).toBeLessThan(20);+ // Jumping straight to a frame replays enough history to match.+ const jumped = await frame(trails, 15, new HeadlessMediaRenderer());+ expect(Math.abs(getPixel(jumped, 50, 100).r - echo)).toBeLessThan(6);+ });++ it("Datamosh shows the clean picture on keyframes", async () => {+ const clip = await slidingClip();+ const renderer = new HeadlessMediaRenderer();+ const mosh = withEffect(clip, new Datamosh({ keyframes: [0, 12] }));+ for (let f = 0; f <= 12; f++) await frame(mosh, f, renderer);+ const atKey = await frame(mosh, 12, renderer);+ const plain = await frame(withEffect(clip, null), 12);+ expect(getPixel(atKey, 70, 100).r).toBeCloseTo(+ getPixel(plain, 70, 100).r,+ -1,+ );+ expect(getPixel(atKey, 20, 100).r).toBeLessThan(20);+ });+});diff --git a/packages/framefields/src/particles.ts b/packages/framefields/src/particles.tsnew file mode 100644index 00000000..1966140e--- /dev/null+++ b/packages/framefields/src/particles.ts@@ -0,0 +1,253 @@+import { Effect } from "@framefields/core";+import type {+ EmitterShape,+ ParticleBlend,+ ParticleBounds,+ ParticleFrame,+ ParticleProgram,+ ParticleShape,+ ParticleStatic,+} from "@framefields/node-particles";+import { parseColor } from "@framefields/webgpu-renderers";+import type { PersonTrack } from "./clip-vision.js";+import { type PhysicsDriver, sampleDriver } from "./physics/core.js";++export type { EmitterShape, ParticleBlend, ParticleBounds, ParticleShape };++type Range = number | [number, number];++export interface ParticlesOptions {+ /** Particle layer box in composition px (the particles' own space; default the whole canvas). */+ x?: number;+ y?: number;+ width: number;+ height: number;+ id?: string;+ startFrame?: number;+ durationFrames?: number;+ zIndex?: number;+ /** Most particles alive at once (default 20000). Costs GPU memory, not time when idle. */+ max?: number;+ seed?: number;+ emitter?: {+ /** `"rect"` (default, the whole layer), `"point"`, `"line"`, `"circle"` (area) or `"ring"` (edge). Layer px. */+ shape?: EmitterShape;+ /** Emitter position (layer px). A driver moves it: `(f) => person.point("rightWrist", f).x` paints with light. */+ x?: PhysicsDriver;+ y?: PhysicsDriver;+ width?: number;+ height?: number;+ radius?: number;+ /** Aim away from the emitter's center instead of along `direction`. */+ radial?: boolean;+ /** Particles per second (default 1000); a driver for bursts of activity. */+ rate?: PhysicsDriver;+ /** Extra particles at given layer frames. */+ bursts?: Array<{ frame: number; count: number }>;+ };+ /** Seconds each particle lives (default [2, 4]). */+ lifetime?: Range;+ /** Launch speed, px/s (default [40, 160]). */+ speed?: Range;+ /** Launch direction, degrees (0 = right, -90 = up; default -90). */+ direction?: number;+ /** Spread around `direction`, degrees (default 360). */+ spread?: number;+ /** px/s². A single driver pulls down. */+ gravity?: PhysicsDriver | [PhysicsDriver, PhysicsDriver];+ /** Velocity damping per second (default 0.4). Higher reads as thicker air or liquid. */+ drag?: PhysicsDriver;+ /** Swirling, divergence-free flow (curl noise): smoke, ink, flow fields. */+ noise?: { strength?: PhysicsDriver; scale?: number; speed?: number };+ /** Points that pull (positive) or push (negative), px/s² at the center, falling off over `radius`. Layer px. Up to 8. */+ attractors?: Array<{+ x: PhysicsDriver;+ y: PhysicsDriver;+ strength: PhysicsDriver;+ radius?: number;+ }>;+ /**+ * A performer pushes particles out of their silhouette, harder where it+ * is sweeping in: `person` from `comp.vision(clip).person.place(…)`.+ */+ push?: {+ person: PersonTrack;+ strength?: PhysicsDriver;+ sweep?: PhysicsDriver;+ };+ /** What happens at the layer's edges: `"none"` (default), `"kill"`, `"bounce"`, `"wrap"`. */+ bounds?: ParticleBounds;+ /** A floor line in layer px that particles bounce on. */+ floor?: number;+ /** Bounciness at the floor and bounced edges (default 0.3). */+ bounce?: number;+ /** px at birth → death (default [6, 0]). */+ size?: Range;+ /** Random size variation, 0–1 (default 0.5). */+ sizeJitter?: number;+ /** Fractions of life to fade in and out (default [0.1, 0.3]). */+ fade?: [number, number];+ /** `"circle"` (default), `"soft"` (glow), `"square"`, `"spark"` (streaks along motion). */+ shape?: ParticleShape;+ /** Spark length per px/s (default 0.04). */+ stretch?: number;+ /** `"normal"` (default), `"add"` (glows build up), `"screen"`. */+ blend?: ParticleBlend;+ /** One color, or up to 8 spread over each particle's life (default white). */+ color?: string | string[];+ /** Color by speed instead of age: the px/s that reaches the last color. */+ colorBySpeed?: number;+ opacity?: number;+ /** Solver steps per frame (default 2). */+ substeps?: number;+ /** Seconds simulated before the first frame, so it starts already full. */+ prewarm?: number;+}++const range = (+ r: Range | undefined,+ fallback: [number, number],+): [number, number] =>+ r === undefined ? fallback : typeof r === "number" ? [r, r] : r;++/**+ * A GPU particle system: tens of thousands of particles simulated in a+ * compute shader and drawn in one call. Used through `Layer.particles()`.+ * It steps as the composition renders (snapshots make scrubbing back+ * cheap), and its drivers are read every frame.+ */+export class Particles+ extends Effect<Record<string, never>>+ implements ParticleProgram+{+ public readonly op = "Particles";+ readonly particleStatic: ParticleStatic;+ particleState?: unknown;+ private readonly options: ParticlesOptions;++ constructor(options: ParticlesOptions) {+ super({});+ this.options = options;+ const e = options.emitter ?? {};+ const staticPos = (d: PhysicsDriver | undefined) =>+ typeof d === "number" ? d : undefined;+ const width = options.width;+ const height = options.height;+ const colors = (+ Array.isArray(options.color)+ ? options.color+ : [options.color ?? "#ffffff"]+ ).map((c): [number, number, number, number] => {+ const p = parseColor(c);+ return [p.r, p.g, p.b, p.a];+ });+ this.particleStatic = {+ max: Math.max(1, Math.round(options.max ?? 20000)),+ seed: Math.round(options.seed ?? 1) >>> 0,+ layer: { x: options.x ?? 0, y: options.y ?? 0, width, height },+ emitter: {+ shape: e.shape ?? "rect",+ x:+ staticPos(e.x) ??+ (e.shape === "circle" || e.shape === "ring" || e.shape === "point"+ ? width / 2+ : 0),+ y:+ staticPos(e.y) ??+ (e.shape === "circle" || e.shape === "ring" || e.shape === "point"+ ? height / 2+ : 0),+ width: e.width ?? width,+ height: e.height ?? (e.shape === "line" ? 0 : height),+ radius: e.radius ?? Math.min(width, height) / 4,+ radial: e.radial ?? false,+ },+ lifetime: range(options.lifetime, [2, 4]),+ speed: range(options.speed, [40, 160]),+ direction: options.direction ?? -90,+ spread: options.spread ?? 360,+ bounds: options.bounds ?? "none",+ floor: options.floor ?? null,+ bounce: options.bounce ?? 0.3,+ size: range(options.size, [6, 0]),+ sizeJitter: options.sizeJitter ?? 0.5,+ fade: options.fade ?? [0.1, 0.3],+ shape: options.shape ?? "circle",+ stretch: options.stretch ?? 0.04,+ blend: options.blend ?? "normal",+ colors: colors.slice(0, 8),+ colorBySpeed: options.colorBySpeed ?? null,+ opacity: options.opacity ?? 1,+ substeps: options.substeps ?? 2,+ prewarm: options.prewarm ?? 0,+ };+ }++ particleFrame(frame: number, fps: number): ParticleFrame {+ const o = this.options;+ const read = (d: PhysicsDriver | undefined, fallback: number) =>+ d === undefined ? fallback : sampleDriver(d, frame, fps);+ const layerFrame = frame - (o.startFrame ?? 0);+ const burst = (o.emitter?.bursts ?? [])+ .filter((b) => Math.round(b.frame) === layerFrame)+ .reduce((n, b) => n + b.count, 0);+ const g = o.gravity;+ const gravity: [number, number] = Array.isArray(g)+ ? [read(g[0], 0), read(g[1], 0)]+ : [0, read(g, 0)];+ let push: ParticleFrame["push"] = null;+ if (o.push) {+ push = {+ strength: read(o.push.strength, 6000),+ sweep: read(o.push.sweep, 60000),+ mask: o.push.person.silhouetteGrid(frame),+ previous: o.push.person.silhouetteGrid(Math.max(0, frame - 1)),+ };+ }+ const ex = o.emitter?.x;+ const ey = o.emitter?.y;+ const moving =+ (ex !== undefined && typeof ex !== "number") ||+ (ey !== undefined && typeof ey !== "number");+ const at = (f: number) => ({+ x:+ ex === undefined+ ? this.particleStatic.emitter.x+ : sampleDriver(ex, f, fps),+ y:+ ey === undefined+ ? this.particleStatic.emitter.y+ : sampleDriver(ey, f, fps),+ });+ let emitter: ParticleFrame["emitter"];+ if (moving) {+ const now = at(frame);+ const before = at(Math.max(0, frame - 1));+ emitter = {+ x: now.x,+ y: now.y,+ previousX: before.x,+ previousY: before.y,+ };+ }+ return {+ ...(emitter && { emitter }),+ rate: read(o.emitter?.rate, 1000),+ burst,+ gravity,+ drag: read(o.drag, 0.4),+ noise: {+ strength: read(o.noise?.strength, 0),+ scale: o.noise?.scale ?? 300,+ speed: o.noise?.speed ?? 0.3,+ },+ attractors: (o.attractors ?? []).map((a) => ({+ x: read(a.x, 0),+ y: read(a.y, 0),+ strength: read(a.strength, 0),+ radius: a.radius ?? 200,+ })),+ push,+ };+ }+}diff --git a/packages/framefields/src/physics/backend.ts b/packages/framefields/src/physics/backend.tsindex 6f684e00..98f1cb56 100644--- a/packages/framefields/src/physics/backend.ts+++ b/packages/framefields/src/physics/backend.ts@@ -55,6 +55,8 @@ export interface BodySpec2D extends BodySpecBase {}export interface BodySpec3D extends BodySpecBase {+ /** Per-axis rotation, when only some axes are locked: [x, y, z] allowed. */+ enabledRotations?: [boolean, boolean, boolean];position: Vec3;rotation: Quat;velocity: Vec3;@@ -306,13 +308,24 @@ class RapierWorld3D implements SimWorld3D {.setTranslation(spec.position[0], spec.position[1], spec.position[2]).setRotation({ x: qx, y: qy, z: qz, w: qw }).setLinvel(spec.velocity[0], spec.velocity[1], spec.velocity[2])- .setAngvel(vec3(spec.angularVelocity))+ // Locked axes start still too (Rapier keeps an initial spin otherwise).+ .setAngvel(+ vec3(+ spec.lockRotation+ ? [0, 0, 0]+ : (spec.angularVelocity.map((w, i) =>+ spec.enabledRotations && !spec.enabledRotations[i] ? 0 : w,+ ) as Vec3),+ ),+ ).setLinearDamping(spec.linearDamping).setAngularDamping(spec.angularDamping).setGravityScale(spec.gravityScale).setCcdEnabled(spec.ccd).setEnabled(spec.enabled);if (spec.lockRotation) desc.lockRotations();+ else if (spec.enabledRotations)+ desc.enabledRotations(...spec.enabledRotations);const body = this.world.createRigidBody(desc);const shape = shapeDesc3D(R, spec.shape);const collider = this.world.createCollider(@@ -439,7 +452,8 @@ function shapeDesc2D(R: Rapier2D, shape: Shape2D): RAPIER2D.ColliderDesc {case "ball":return R.ColliderDesc.ball(shape.radius);case "box": {- const r = Math.min(shape.radius ?? 0, shape.hx, shape.hy);+ // The rounded border wraps a core box; keep the core solid (a flat core has no mass).+ const r = Math.min(shape.radius ?? 0, shape.hx / 2, shape.hy / 2);return r > 0? R.ColliderDesc.roundCuboid(shape.hx - r, shape.hy - r, r): R.ColliderDesc.cuboid(shape.hx, shape.hy);@@ -460,7 +474,13 @@ function shapeDesc3D(R: Rapier3D, shape: Shape3D): RAPIER3D.ColliderDesc {case "ball":return R.ColliderDesc.ball(shape.radius);case "box": {- const r = Math.min(shape.radius ?? 0, shape.hx, shape.hy, shape.hz);+ // The rounded border wraps a core box; keep the core solid (a flat core has no mass).+ const r = Math.min(+ shape.radius ?? 0,+ shape.hx / 2,+ shape.hy / 2,+ shape.hz / 2,+ );return r > 0? R.ColliderDesc.roundCuboid(shape.hx - r,diff --git a/packages/framefields/src/physics/core.ts b/packages/framefields/src/physics/core.tsindex 08622852..27d42811 100644--- a/packages/framefields/src/physics/core.ts+++ b/packages/framefields/src/physics/core.ts@@ -200,11 +200,28 @@ export class Recording {return arr;}+ /**+ * A per-frame signal over a channel. Creating it does nothing; the first+ * value read runs the simulation (offline worlds), so signals can be bound+ * while bodies are still being added.+ */signal(name: string): FrameSignal<number> {- this.ensure();let sig = this.signals.get(name);if (!sig) {- sig = Signal.fromArray(this.channel(name), this.fps);+ const arr = this.channel(name);+ const fps = this.fps;+ sig = Signal.builder({+ type: "custom",+ fn: (ctx: { frame: number; fps?: number }) => {+ this.ensure();+ const f =+ ctx.fps && ctx.fps !== fps+ ? (ctx.frame / ctx.fps) * fps+ : ctx.frame;+ const i = Math.max(0, Math.min(arr.length - 1, Math.round(f)));+ return arr[i];+ },+ }) as unknown as FrameSignal<number>;this.signals.set(name, sig);}return sig;diff --git a/packages/framefields/src/physics/physics.test.ts b/packages/framefields/src/physics/physics.test.tsindex 6d5b1a8d..1fcf110f 100644--- a/packages/framefields/src/physics/physics.test.ts+++ b/packages/framefields/src/physics/physics.test.ts@@ -262,8 +262,11 @@ describe("Physics.world2d", () => {/x.*y/,);const b = world.body(card("a", 0, 0));- void b.y;- expect(() => world.body(card("b", 0, 0))).toThrow(/already started/);+ // Holding a signal is fine; reading a value runs the simulation and locks.+ const y = b.y;+ world.body(card("b", 0, 0));+ at(y, 0);+ expect(() => world.body(card("c", 0, 0))).toThrow(/already started/);});});@@ -382,6 +385,51 @@ describe("Physics.world3d", () => {]);});+ it("drops thin rounded slabs (rounding never empties the core)", async () => {+ const world = await Physics.world3d({ fps: FPS, frames: 40 });+ world.ground({ y: 800 });+ const slab = world.body(+ Layer3D.roundedBox({+ width: 300,+ height: 120,+ depth: 34,+ radius: 18,+ x: 400,+ y: 100,+ z: 0,+ }),+ { lockRotation: true },+ );+ const pill = await Physics.world2d({ fps: FPS, frames: 40 });+ pill.ground({ y: 800 });+ const p = pill.body(card("pill", 0, 100, { borderRadius: 50 }));+ world.simulate();+ pill.simulate();+ expect(at(slab.y, 39)).toBeGreaterThan(400);+ expect(at(p.y, 39)).toBeGreaterThan(400);+ });++ it("locks rotation per axis", async () => {+ const world = await Physics.world3d({ fps: FPS, frames: 60 });+ world.ground({ y: 600 });+ const card3 = world.body(+ Layer3D.roundedBox({+ width: 200,+ height: 60,+ depth: 30,+ x: 0,+ y: 0,+ z: 0,+ rotateZ: 30,+ rotateX: 0,+ }),+ { lockRotation: { x: true, y: true }, angularVelocity: [200, 200, 0] },+ );+ world.simulate();+ expect(Math.abs(at(card3.rotateX, 59))).toBeLessThan(0.01);+ expect(Math.abs(at(card3.rotateY, 59))).toBeLessThan(0.01);+ });+it("converts rotations with the renderer's Euler order", () => {for (const e of [[10, 20, 30],diff --git a/packages/framefields/src/physics/world3d.ts b/packages/framefields/src/physics/world3d.tsindex 748414d6..3db494fb 100644--- a/packages/framefields/src/physics/world3d.ts+++ b/packages/framefields/src/physics/world3d.ts@@ -42,7 +42,12 @@ export interface PhysicsBox3D {rotateZ?: number;}-export interface Body3DOptions extends BodyCommonOptions {+export interface Body3DOptions extends Omit<BodyCommonOptions, "lockRotation"> {+ /**+ * Never rotates (`true`), or only around some axes: `{ x: true, y: true }`+ * keeps a card facing the camera while it still spins in the picture plane.+ */+ lockRotation?: boolean | { x?: boolean; y?: boolean; z?: boolean };/*** Collider: `"auto"` (default) reads the Layer3D primitive (box, sphere,* cylinder) and falls back to the mesh's convex hull for models and@@ -267,7 +272,7 @@ export class PhysicsBody3D implements CoreBody<Vec3, Quat> {["y", "y"],["z", "z"],];- if (!this.options.lockRotation) {+ if (this.options.lockRotation !== true) {out.push(["rotateX", "rotateX"],["rotateY", "rotateY"],@@ -381,7 +386,14 @@ export class PhysicsWorld3D extends WorldCore<Vec3, Quat, PhysicsBody3D> {linearDamping: options.linearDamping ?? 0,angularDamping: options.angularDamping ?? 0,gravityScale: options.gravityScale ?? 1,- lockRotation: options.lockRotation ?? false,+ lockRotation: options.lockRotation === true,+ ...(typeof options.lockRotation === "object" && {+ enabledRotations: [+ !options.lockRotation.x,+ !options.lockRotation.y,+ !options.lockRotation.z,+ ] as [boolean, boolean, boolean],+ }),ccd: options.ccd ?? false,enabled: true,sensor: options.sensor ?? false,diff --git a/packages/renderer/package.json b/packages/renderer/package.jsonindex f6ea91c1..1e67c339 100644--- a/packages/renderer/package.json+++ b/packages/renderer/package.json@@ -62,6 +62,7 @@"@framefields/node-corner-pin": "workspace:*","@framefields/node-crop": "workspace:*","@framefields/node-curves": "workspace:*",+ "@framefields/node-datamosh": "workspace:*","@framefields/node-deflicker": "workspace:*","@framefields/node-displacement-map": "workspace:*","@framefields/node-extract-frame": "workspace:*",@@ -81,6 +82,7 @@"@framefields/node-modulate": "workspace:*","@framefields/node-noise-generator": "workspace:*","@framefields/node-paint": "workspace:*",+ "@framefields/node-particles": "workspace:*","@framefields/node-patch-heal": "workspace:*","@framefields/node-refine-edge": "workspace:*","@framefields/node-refraction-caustics": "workspace:*",@@ -94,6 +96,7 @@"@framefields/node-signal-gate": "workspace:*","@framefields/node-stereo-panning": "workspace:*","@framefields/node-tile-offset": "workspace:*",+ "@framefields/node-trails": "workspace:*","@framefields/node-unsharp-mask": "workspace:*","@framefields/node-vignette": "workspace:*","@framefields/node-vision": "workspace:*",diff --git a/packages/renderer/src/generated/node-renderers.ts b/packages/renderer/src/generated/node-renderers.tsindex 42bd861d..1948f127 100644--- a/packages/renderer/src/generated/node-renderers.ts+++ b/packages/renderer/src/generated/node-renderers.ts@@ -106,6 +106,11 @@ export const BUILTIN_NODE_RENDERERS: readonly BuiltinNodeRenderer[] = [ops: ["Curves"],load: () => import("@framefields/node-curves/renderer"),},+ {+ packageName: "@framefields/node-datamosh",+ ops: ["Datamosh"],+ load: () => import("@framefields/node-datamosh/renderer"),+ },{packageName: "@framefields/node-deflicker",ops: ["TemporalDeflicker", "Deflicker"],@@ -196,6 +201,11 @@ export const BUILTIN_NODE_RENDERERS: readonly BuiltinNodeRenderer[] = [ops: ["Paint"],load: () => import("@framefields/node-paint/renderer"),},+ {+ packageName: "@framefields/node-particles",+ ops: ["Particles"],+ load: () => import("@framefields/node-particles/renderer"),+ },{packageName: "@framefields/node-patch-heal",ops: ["PatchHeal"],@@ -256,6 +266,11 @@ export const BUILTIN_NODE_RENDERERS: readonly BuiltinNodeRenderer[] = [ops: ["TileOffset"],load: () => import("@framefields/node-tile-offset/renderer"),},+ {+ packageName: "@framefields/node-trails",+ ops: ["Trails"],+ load: () => import("@framefields/node-trails/renderer"),+ },{packageName: "@framefields/node-unsharp-mask",ops: ["UnsharpMask"],diff --git a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.tsindex f2be4594..f9a83739 100644--- a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts@@ -29,6 +29,17 @@ export interface MeshMaterialMaps {emissive?: GPUTexture;}+export interface MeshGlass {+ /** Index of refraction (1.5 glass, 1.33 water). */+ ior: number;+ /** Red/blue split of the refraction, 0–0.2 (rainbow fringes). */+ dispersion: number;+ /** How far the backdrop bends at the edges, px. */+ thickness: number;+ /** Frosted blur, 0–1. */+ frost: number;+}+export interface MeshUniformData extends SurfaceParams {modelMatrix: Float32Array | number[];normalMatrix?: Float32Array | number[];@@ -41,6 +52,13 @@ export interface MeshUniformData extends SurfaceParams {material?: "lit" | "unlit" | "toon";/** Screen-space track matte: the mesh shows where this texture's alpha does. */matte?: GPUTexture;+ /**+ * Glass: the surface shows `backdrop` (what is behind it on screen) bent by+ * refraction, tinted by the color, under Fresnel reflections.+ */+ glass?: MeshGlass;+ /** Screen-sized picture behind the 3D batch, read by glass. */+ backdrop?: GPUTexture;/** Toon shading: the shadowed side's color multiplier. */shadeColor?: [number, number, number];/** Normal / metallic-roughness / occlusion / emissive maps. */@@ -134,6 +152,7 @@ export class Mesh3DPipeline {buffer: { type: "uniform" },},matteLayoutEntry(2),+ matteLayoutEntry(3),],});@@ -304,6 +323,15 @@ export class Mesh3DPipeline {this.modelData[57] = maps?.occlusion ? (maps.occlusionStrength ?? 1) : 0;this.modelData[58] = maps?.metallicRoughness ? 1 : 0;this.modelData[59] = maps?.emissive ? 1 : 0;+ const glass = data.glass;+ if (glass) {+ // shade.rgb and pbr.zw are unused outside toon: glass parameters.+ this.modelData[44] = glass.dispersion;+ this.modelData[45] = glass.thickness;+ this.modelData[46] = glass.frost;+ this.modelData[50] = 1;+ this.modelData[51] = glass.ior;+ }const modelBuffer = this.modelPool.getBuffer(this.device, this.modelData);@@ -321,6 +349,12 @@ export class Mesh3DPipeline {binding: 2,resource: data.matte?.createView() ?? fullCoverageView(this.device),},+ {+ binding: 3,+ resource:+ (data.glass && data.backdrop?.createView()) ||+ fullCoverageView(this.device),+ },],});}diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.tsindex e199c9a0..00d1355a 100644--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts@@ -123,6 +123,14 @@ export interface DrawQuad3DOpts extends SurfaceParams {const TEXT_STACK_PULL = 5e-4;export interface DrawMesh3DOpts extends SurfaceParams {+ /** Refractive index for glass materials (default 1.5). */+ ior?: number;+ /** Rainbow split for glass materials, 0–0.2 (default 0.04). */+ dispersion?: number;+ /** Edge bend for glass materials, px (default 80). */+ thickness?: number;+ /** Screen-sized picture behind this 3D batch; glass materials refract it. */+ backdrop?: GPUTexture;/*** Screen-space track matte, the size of the pass: the layer shows where* this texture's alpha does and keeps its place in the depth buffer.@@ -151,7 +159,14 @@ export interface DrawMesh3DOpts extends SurfaceParams {/** Pre-computed normal matrix */normalMatrix?: Mat4;/** Material shading mode */- material?: "lit" | "unlit" | "toon";+ material?:+ | "lit"+ | "unlit"+ | "toon"+ | "glass"+ | "acrylic"+ | "frosted"+ | "metallic";/** Toon shading: the shadowed side's color multiplier */shadeColor?: [number, number, number];/** Normal / metallic-roughness / occlusion / emissive maps (see `materialMaps`). */@@ -975,7 +990,7 @@ export class Renderer3D {metallic: opts.metallic,emissive: opts.emissive,emissiveIntensity: opts.emissiveIntensity,- matte: opts.matte,+ matte: opts.matte,},useMrt,this.passWritesHdr,@@ -1236,6 +1251,17 @@ export class Renderer3D {addressModeV: address,});+ const glassy =+ material === "glass" || material === "acrylic" || material === "frosted";+ const glass = glassy+ ? {+ ior: opts.ior ?? (material === "acrylic" ? 1.49 : 1.5),+ dispersion: opts.dispersion ?? (material === "glass" ? 0.04 : 0.015),+ thickness: opts.thickness ?? 80,+ frost: material === "frosted" ? (roughness ?? 0.5) : 0,+ }+ : undefined;+this.mesh3dPipeline.draw(pass,cameraBindGroup,@@ -1244,6 +1270,8 @@ export class Renderer3D {texture,sampler,{+ glass,+ backdrop: opts.backdrop,modelMatrix: m,normalMatrix,colorTint,@@ -1252,11 +1280,12 @@ export class Renderer3D {wireframe,isSkinned,skinningMatrices,- material,+ material:+ material === "unlit" || material === "toon" ? material : "lit",shadeColor,alphaCutoff,shininess,- roughness,+ roughness: glassy ? Math.min(roughness ?? 0.05, 0.3) : roughness,specularIntensity,ambientIntensity,metallic,diff --git a/packages/webgpu-renderers/src/shaders/mesh3d.ts b/packages/webgpu-renderers/src/shaders/mesh3d.tsindex c7abad67..1bf89c65 100644--- a/packages/webgpu-renderers/src/shaders/mesh3d.ts+++ b/packages/webgpu-renderers/src/shaders/mesh3d.ts@@ -32,7 +32,8 @@ struct SkinningUniforms {@group(0) @binding(0) var<uniform> cam : CameraUniforms;@group(1) @binding(0) var<uniform> model : ModelUniforms;@group(1) @binding(1) var<uniform> skinning : SkinningUniforms;-${matte3dWgsl(1, 2)}@group(2) @binding(0) var texSampler : sampler;+${matte3dWgsl(1, 2)}@group(1) @binding(3) var backdropTex : texture_2d<f32>;+@group(2) @binding(0) var texSampler : sampler;@group(2) @binding(1) var mainTex : texture_2d<f32>;@group(2) @binding(2) var normalTex : texture_2d<f32>;@group(2) @binding(3) var metalRoughTex : texture_2d<f32>;@@ -154,6 +155,75 @@ fn perturbNormal(N: vec3<f32>, p: vec3<f32>, uv: vec2<f32>, tn: vec3<f32>, dp1:return normalize(T * (tn.x * invmax) - B * (tn.y * invmax) + N * tn.z);}+// Glass: what is behind the 3D batch on screen (backdrop), bent by+// refraction along the surface normal with a red/blue split, frosted by a+// disc blur, tinted by the color, under Fresnel-weighted reflections.+// shade.rgb = (dispersion, thickness px, frost), pbr.w = ior.+fn backdropAt(uv: vec2<f32>) -> vec3<f32> {+ return textureSampleLevel(backdropTex, texSampler, clamp(uv, vec2<f32>(0.001), vec2<f32>(0.999)), 0.0).rgb;+}++// rimWidth: how much 1 - N·V changes across this pixel (fwidth, taken in+// uniform control flow by the caller). Rim effects narrower than a pixel are+// faded out, or per-pixel shading turns them into stair-stepped lines that+// MSAA (coverage only) cannot smooth.+fn shadeGlass(in: VertexOutput, Nin: vec3<f32>, tint: vec3<f32>, alpha: f32, rimWidth: f32) -> MeshFragment {+ let V = normalize(cam.cameraPos.xyz - in.worldPos);+ var N = Nin;+ if (dot(N, V) < 0.0) { N = -N; }+ let NdotV = clamp(dot(N, V), 0.0, 1.0);+ // Also fade toward edge-on: a face seen at a grazing angle is a sliver+ // under a pixel tall (a slab's bottom), which would alias the same way.+ let rimAA = clamp(1.0 - rimWidth * 2.5, 0.0, 1.0) * smoothstep(0.03, 0.22, NdotV);+ let rim = (1.0 - NdotV) * rimAA;+ let ior = max(model.pbr.w, 1.0001);+ let screen = max(cam.screenParams.xy, vec2<f32>(1.0));+ let uv = in.clipPos.xy / screen;+ // Screen-space bend: the normal's in-plane part, stronger toward the rim.+ let bend = N.xy * model.shade.g * (1.0 - 1.0 / ior) * (0.6 + rim) * rimAA / screen;+ let disp = model.shade.r;+ let frost = model.shade.b * 22.0 / screen;+ var col = vec3<f32>(0.0);+ let taps = select(1, 8, model.shade.b > 0.001);+ for (var i = 0; i < taps; i++) {+ let a = f32(i) * 2.399963;+ let r = sqrt((f32(i) + 0.5) / f32(taps));+ let o = select(vec2<f32>(0.0), vec2<f32>(cos(a), sin(a)) * r * frost, taps > 1);+ col += vec3<f32>(+ backdropAt(uv - bend * (1.0 + disp * 6.0) + o).r,+ backdropAt(uv - bend + o).g,+ backdropAt(uv - bend * (1.0 - disp * 6.0) + o).b,+ );+ }+ col /= f32(taps);+ // Light travels further through the glass toward the rim: the tint+ // deepens and the body darkens there, which is what outlines glass.+ let depthTint = mix(vec3<f32>(1.0), tint, 0.55 + 0.45 * rim);+ let transmitted = col * depthTint * (1.0 - 0.45 * rim * rim);+ let f0 = pow((ior - 1.0) / (ior + 1.0), 2.0);+ let F = f0 + (1.0 - f0) * pow(rim, 5.0);+ let lit = shadeSurfaceFull(in.worldPos, N, vec3<f32>(0.0), model.materialParams, vec4<f32>(model.pbr.x, 0.0, 0.0, 0.0), vec4<f32>(0.0));+ // Grazing faces mirror the room; keep that faint so dark scenes stay dark+ // and the studio highlights below carry the shape.+ let sheen = vec3<f32>(0.9, 0.94, 1.0) * F * 0.22 + tint * 0.05 * (0.4 + rim);+ // A studio in the reflection: a sharp key and two strip softboxes, which+ // is most of what makes glass read as glass.+ let R = reflect(-V, N);+ // Tight: a flat face only flashes as it turns through the light.+ let key = pow(max(dot(R, normalize(vec3<f32>(-0.45, -0.75, -0.5))), 0.0), 600.0) * 1.8;+ // Two tall softboxes: narrow bands in the reflection, brightest above.+ let strips = (1.0 - smoothstep(0.03, 0.07, abs(R.x + 0.45))) * smoothstep(0.3, -0.5, R.y) * 1.1+ + (1.0 - smoothstep(0.02, 0.05, abs(R.x - 0.6))) * smoothstep(0.4, -0.3, R.y) * 0.7;+ let studio = vec3<f32>(1.0, 0.98, 0.95) * (key + strips * (0.25 + F));+ // A thin bright line where the surface turns away: glass silhouettes.+ let edge = vec3<f32>(1.0, 0.99, 0.97) * smoothstep(0.8, 0.97, rim) * 0.55;+ let rgb = transmitted * (1.0 - F) + lit.display + sheen + studio + edge;+ var out: MeshFragment;+ out.color = vec4<f32>(rgb * alpha, alpha);+ out.hdr = vec4<f32>(srgbToLinear(transmitted * (1.0 - F)) * alpha + lit.hdr * alpha + srgbToLinear(sheen + studio + edge) * alpha, alpha);+ return out;+}+fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {// Every sample and derivative up front, in uniform control flow.var texColor = textureSample(mainTex, texSampler, in.uv);@@ -165,6 +235,7 @@ fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {let dp2 = dpdy(in.worldPos);let duv1 = dpdx(in.uv);let duv2 = dpdy(in.uv);+ let rimWidth = fwidth(1.0 - abs(dot(normalize(in.worldNormal), normalize(cam.cameraPos.xyz - in.worldPos))));if (!isFront && model.params.y < 0.5) {discard;@@ -205,6 +276,9 @@ fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {pbr.x = select(pbr.x, 1.0, pbr.x < 0.0) * mrSample.g;pbr.y = pbr.y * mrSample.b;}+ if (model.pbr.z > 0.5) {+ return shadeGlass(in, N, baseRgb, alpha, rimWidth);+ }var material = model.materialParams;material.z = material.z * mix(1.0, aoSample, clamp(model.maps.y, 0.0, 1.0));var emissive = model.emissive;diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.mdindex a7d58639..16898c2a 100644--- a/plugins/framefields/skills/framefields-compose/SKILL.md+++ b/plugins/framefields/skills/framefields-compose/SKILL.md@@ -366,6 +366,7 @@ Layer3D.cylinder({ ..., color: "#111", emissive: "#4da3ff", emissiveIntensity: 3- **Fog** fades to `fog.color` exactly, so set it to the composition background for a seamless horizon.- **Models:** glTF materials bring base color, normal, metallic-roughness, occlusion and emissive maps.- `emissive` / `emissiveIntensity` / `exposure` animate like any prop (`.keys("emissiveIntensity", …)`).+- **Glass:** `material: "glass" | "acrylic" | "frosted"` on any Layer3D primitive, extruded text (`Layer3D.extrudedText`) or model refracts whatever is behind it on screen — the video, 2D layers, particles — with a rainbow fringe (`dispersion`, default 0.04), bend strength `thickness` (px, default 80) and `ior` (1.5). `color` tints the glass (deeper toward the edges); `frosted` blurs by `roughness`. Glass needs no environment: it draws its own studio reflections, and against a dark grade it reads best — skip `Lighting.*` rigs for glass-only scenes (their environment washes glass out). Glass shows only 2D content and opaque 3D drawn before it, not other glass.---@@ -406,7 +407,7 @@ stage.animate(LayerAnimation.create().signal("y", world.impact, { multiplier: 14- Reads are synchronous and need the frame analysed: inside a live world, while rendering, or after `await dancer.ready(frame)` (e.g. `ready(0)` to lay things out around the first pose). A live world can't look ahead, so place things from early frames.- Layer a vision cutout of the same clip (`Layer.video(src).withVision({ mode: "matte" })`) above the particles so they pass behind the performer.- Live worlds bind signals only (no `bakeKeyframes()`); `await world.advanceTo(frame)` steps one without rendering (e.g. to read `collisions`).-- **Particles today:** up to ~600 small `Layer.shape("circle")` bodies with `linearDamping` ≈ 1–1.5, low `friction`, `gravityScale` < 1 read as a fluid; tint them by speed with `.colorSignal("fillColor", body.speed, { colorA, colorB, multiplier: 1 / 900 })` (`multiplier`/`offset` scale the signal into 0–1).+- **Particles:** for clouds, dust, sparks, fluids and flow fields use `Layer.particles` (GPU, below), not hundreds of physics bodies. Keep physics bodies for a few dozen things that must collide and stack (cards, letters, products).- **Keyframes instead of signals:** `world.bakeKeyframes({ tolerance: 0.25 })` in place of `simulate()` writes plain, simplified linear keyframes (editable, plain JSON).- **3D:** `Physics.world3d` takes `Layer3D.roundedBox / sphere / cylinder` and models (convex hull of the mesh); same options with 3-vectors, `world.hinge(a, b, at, axis)`, `world.walls({ width, height, depth })`.- **Timing:** signals are on the composition clock. In a scene that starts at frame `S`, pass `startFrame: S` to the world. Before it and after `frames` the signals hold their first/last value. Add every body, joint and force before `simulate()` (or before reading any signal): the world then locks.@@ -415,6 +416,34 @@ stage.animate(LayerAnimation.create().signal("y", world.impact, { multiplier: 14---+## GPU Particles (`Layer.particles`)++Tens of thousands of particles simulated in a compute shader and drawn in one call — sparks, embers, dust, snow, smoke, ink, flow fields, light painting, a fluid a performer moves through. A particle layer is a box in composition px; positions inside it are layer px.++```typescript+comp.add(Layer.particles({+ width: W, height: H, max: 20000, seed: 7,+ emitter: { shape: "rect", x: 0, y: H * 0.6, width: W, height: 200, rate: 3000 }, // point | line | rect | circle | ring; radial+ lifetime: [4, 6], speed: [0, 40], direction: -90, spread: 360,+ gravity: 20, drag: 1.5, // px/s², per second+ noise: { strength: 260, scale: 170, speed: 0.25 }, // curl-noise flow: smoke, ink, flow fields+ attractors: [{ x: W / 2, y: H / 2, strength: 900, radius: 200 }], // negative strength repels+ push: { person, strength: 9000, sweep: 120000 }, // a performer's silhouette pushes them (comp.vision)+ floor: H - 60, bounce: 0.2, bounds: "none", // none | kill | bounce | wrap+ size: [10, 4], sizeJitter: 0.5, fade: [0.1, 0.3],+ shape: "circle", // circle | soft (glow) | square | spark (streaks along motion)+ color: ["#ffd23f", "#ff5a1f", "#7a2e14"], // over life, or by speed with colorBySpeed: 600+ blend: "normal", // add for glows and light painting+ prewarm: 3, // seconds simulated before frame 0: starts full+}));+```++- `emitter.x` / `emitter.y`, `rate`, `gravity`, `drag`, `noise.strength`, attractor positions/strengths and `push` strengths take a number, a signal (audio, physics `impact`) or `(frame) => value`, read every frame: bursts on the beat, wind gusts, an attractor following a tracked hand (`person.point("rightWrist", f)`). `emitter.bursts: [{ frame, count }]` adds one-off explosions.+- Recipes: **light painting** an additive `"soft"` point emitter whose `x`/`y` follow a tracked wrist (`(f) => person.point("rightWrist", f).x`), lifetime ~1 s, small speed, births spread along the hand's path so fast moves stay continuous; **sparks/fountain** `shape: "spark"`, `blend: "add"`, gravity, `floor`; **smoke/ink** `shape: "soft"`, low opacity, `noise`, sizes growing; **snow/dust** `rect` emitter above the frame, small gravity, `noise`, `bounds: "wrap"`; **swirl** `ring` emitter + attractor + `colorBySpeed`; **fluid around a dancer** `push: { person }` + `drag` 2 + gentle noise, and a vision cutout of the clip above the particle layer.+- The system steps as frames render and snapshots itself every 24 frames, so scrubbing back restores instead of replaying from the start; the same GPU gives the same frames.++---+## Critical Authoring Rules & Gotchas1. **Local Keyframe Coordinates:** Keyframes are strictly relative to the node's own timeline window (`0` = node appearance). Never offset keyframe frames by the parent's `startFrame`.diff --git a/plugins/framefields/skills/framefields-effects/SKILL.md b/plugins/framefields/skills/framefields-effects/SKILL.mdindex d4b52ebc..13ff997e 100644--- a/plugins/framefields/skills/framefields-effects/SKILL.md+++ b/plugins/framefields/skills/framefields-effects/SKILL.md@@ -22,7 +22,9 @@ See [references/effects-catalog.md](references/effects-catalog.md) for the generThere are 4 main attachment modes in the engine:### 1. Layer-Level Effect Pipeline-Attach one or more effects to an individual media, box, or text layer:+Attach one or more effects to a media layer (video, image, SVG, Lottie). Box and text layers take only `TemporalDeflicker` and `Relight3D` directly; to treat a group or text as an image, render it into a media layer first.++```typescriptimport { Layer, Curves, Vignette, Blur } from "framefields";@@ -109,6 +111,11 @@ const pngBuffer = await clip.renderFrame({ atMs: 1500 });- **`MotionBlur`**: Directional shutter simulation with custom shutter angles and velocity clamps.- **`TemporalDeflicker`**: Multi-frame optical flow blending to eliminate flickering in generative AI clips or high-speed footage.+### Temporal (effects with memory)+- **`Trails`**: the layer over a decaying copy of its own past. `mode: "blend"` (default; echoes on full-frame footage), `"over"` (cutouts and text), `"add"` (light painting on dark scenes), `"max"` (light trails). `decay` 0.8 short echo → 0.97 long trails; per-frame `zoom` (1.02 = tunnel), `rotation`, `offsetX/Y`, `hueShift` (psychedelic cycling).+- **`Datamosh`**: the previous picture dragged along the new frame's optical flow instead of being refreshed, until a `keyframes` frame (I-frame) shows the clean picture. Put keyframes where you want it clean; leave a cut out to smear one shot with the next one's motion. `strength`, `bleed` (clean picture mixed back per frame).+- Both step as frames render; after a jump (scrubbing, a single `renderFrame`) they replay the last frames (`warmup`) to rebuild the history.+### 3. 3D Spatial Lighting & Materials- **`Relight3D`**: Screen-space normal map relighting with point, spot, and directional lights, specular highlights, roughness, and metallic uniforms.- **`SSAO`**: Screen-space ambient occlusion generating contact shadows from depth buffers.diff --git a/plugins/framefields/skills/framefields-effects/references/effects-catalog.md b/plugins/framefields/skills/framefields-effects/references/effects-catalog.mdindex 613e207b..20a05cd8 100644--- a/plugins/framefields/skills/framefields-effects/references/effects-catalog.md+++ b/plugins/framefields/skills/framefields-effects/references/effects-catalog.md@@ -3,7 +3,7 @@> Generated by `pnpm run sync:effects-catalog` from the node schemas (via `pnpm run generate:effects`) and `packages/framefields/src/effects/classes.ts`.> Do not edit by hand — a stale catalog fails `pnpm run check:plugins`.-52 effect classes are available from the package root and the `/effects` subpath:+54 effect classes are available from the package root and the `/effects` subpath:```tsimport { Blur, Curves, FilmGrain } from "framefields";@@ -34,6 +34,7 @@ Set `FRAMEFIELDS_STRICT_PROPS=1` to make an unknown prop or an out-of-range valu| `ColorBalance` | `ColorBalance` | `shadows`, `midtones`, `highlights`, `preserveLuminosity` || `ColorKey` | `ColorKey` | `keyColor`, `similarity`, `smoothness`, `spillSuppression`, `colorSpace`, `spillSuppressionType` || `CornerPin` | `CornerPin` | `points` |+| `Datamosh` | `Datamosh` | `keyframes`, `strength`, `bleed`, `warmup` || `DisplacementMap` | `DisplacementMap` | `strengthX`, `strengthY`, `xChannel`, `yChannel`, `wrapMode` || `ExtractLUT` | `ExtractLUT` | `strategy`, `samplePoints` || `Flip` | `Flip` | `horizontal`, `vertical`, `diagonal`, `mode`, `symmetry` |@@ -51,6 +52,7 @@ Set `FRAMEFIELDS_STRICT_PROPS=1` to make an unknown prop or an out-of-range valu| `ResizerScaler` | `ResizerScaler` | `aspectRatioPreset`, `resolutionPreset`, `targetWidth`, `targetHeight`, `fitMode`, `zoom`, `offsetX`, `offsetY`, `backgroundMode`, `backgroundColor`, `backgroundColor2`, `blurRadius`, `backgroundBrightness`, `anchorX`, `anchorY` || `Rotate` | `Rotate` | `angle`, `fit`, `scale` || `StereoPanning` | `StereoPanning` | `pan` |+| `Trails` | `Trails` | `decay`, `mode`, `zoom`, `rotation`, `offsetX`, `offsetY`, `hueShift`, `warmup` || `Vignette` | `Vignette` | `strength`, `radius`, `softness`, `roundness`, `centerX`, `centerY` || `Levels` | `Levels` | `master`, `red`, `green`, `blue` || `FilmGrain` | `FilmGrain` | `strength`, `size`, `monochrome`, `animated`, `speed`, `shadows`, `midtones`, `highlights` |@@ -283,6 +285,21 @@ new CornerPin({ /* props */ });| --- | --- | --- || `points` | `CornerPinPointsItem[]` | — |+### Datamosh++```ts+import { Datamosh } from "framefields/effects";++new Datamosh({ strength: 1, bleed: 0, warmup: 48 });+```++| Prop | Type | Default | Notes |+| --- | --- | --- | --- |+| `keyframes` | `number[]` | `[]` | Layer frames that show the clean picture again (I-frames). The first frame always does. |+| `strength` | `EffectProp<number>` | `1` | Range 0 to 10. How far pixels are dragged along the motion. |+| `bleed` | `EffectProp<number>` | `0` | Range 0 to 1. Clean picture mixed back in per frame, 0–1. |+| `warmup` | `number` | `48` | Range 1 to 240. Most frames replayed to rebuild the smear after a jump. |+### DisplacementMap```ts@@ -616,6 +633,25 @@ new StereoPanning({ pan: 0 });| --- | --- | --- | --- || `pan` | `EffectProp<number>` | `0` | Range -1 to 1. Stereo panning value: -1 (full left) to 1 (full right), 0 is center. Can be modulated by a static number or dynamic signal. |+### Trails++```ts+import { Trails } from "framefields/effects";++new Trails({ decay: 0.9, mode: "blend", zoom: 1 });+```++| Prop | Type | Default | Notes |+| --- | --- | --- | --- |+| `decay` | `EffectProp<number>` | `0.9` | Range 0 to 1. History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails. |+| `mode` | `"blend" \| "over" \| "add" \| "max"` | `"blend"` | blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest. |+| `zoom` | `EffectProp<number>` | `1` | Range 0.5 to 2. History scale per frame; 1.02 pulls the past into a tunnel. |+| `rotation` | `EffectProp<number>` | `0` | Range -45 to 45. History rotation per frame, degrees. |+| `offsetX` | `EffectProp<number>` | `0` | Range -500 to 500. History drift per frame, px. |+| `offsetY` | `EffectProp<number>` | `0` | Range -500 to 500. History drift per frame, px. |+| `hueShift` | `EffectProp<number>` | `0` | Range -180 to 180. History hue rotation per frame, degrees (psychedelic cycling). |+| `warmup` | `number` | `60` | Range 0 to 240. Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render). |+### Vignette```ts@@ -1025,4 +1061,4 @@ new Modulate({ hue: 0, brightness: 1.0, contrast: 1.0 });Beyond these classes, the engine ships node plugins under `nodes/` (audio DSP, generators,signal maths, vision). Each is a package named `@framefields/node-<name>`:-`node-apply-lut`, `node-audio-compressor`, `node-audio-delay`, `node-audio-fade`, `node-audio-noise-gate`, `node-audio-parametric-eq`, `node-audio-reverb`, `node-audio-signal-extractor`, `node-blur`, `node-camera-parallax-3d`, `node-canvas-generator`, `node-caption-editor`, `node-caption-generator`, `node-channel-merger`, `node-channel-splitter`, `node-color-balance`, `node-colorkey`, `node-compositor`, `node-corner-pin`, `node-crop`, `node-curves`, `node-deflicker`, `node-displacement-map`, `node-extract-frame`, `node-extract-lut`, `node-film-grain`, `node-flip`, `node-gradient-map`, `node-halftone-screen`, `node-high-pass`, `node-kenburns`, `node-layer-style`, `node-levels`, `node-liquify`, `node-mask-math`, `node-mesh-warp`, `node-modulate`, `node-noise-generator`, `node-paint`, `node-patch-heal`, `node-procedural-vfx`, `node-refine-edge`, `node-refraction-caustics`, `node-relight-3d`, `node-resizer-scaler`, `node-rotate`, `node-selective-color`, `node-shadows-highlights`, `node-shape-generator`, `node-signal-gate`, `node-signal-math`, `node-signal`, `node-stereo-panning`, `node-text`, `node-tile-offset`, `node-unsharp-mask`, `node-vignette`, `node-vision`+`node-apply-lut`, `node-audio-compressor`, `node-audio-delay`, `node-audio-fade`, `node-audio-noise-gate`, `node-audio-parametric-eq`, `node-audio-reverb`, `node-audio-signal-extractor`, `node-blur`, `node-camera-parallax-3d`, `node-canvas-generator`, `node-caption-editor`, `node-caption-generator`, `node-channel-merger`, `node-channel-splitter`, `node-color-balance`, `node-colorkey`, `node-compositor`, `node-corner-pin`, `node-crop`, `node-curves`, `node-datamosh`, `node-deflicker`, `node-displacement-map`, `node-extract-frame`, `node-extract-lut`, `node-film-grain`, `node-flip`, `node-gradient-map`, `node-halftone-screen`, `node-high-pass`, `node-kenburns`, `node-layer-style`, `node-levels`, `node-liquify`, `node-mask-math`, `node-mesh-warp`, `node-modulate`, `node-noise-generator`, `node-paint`, `node-particles`, `node-patch-heal`, `node-procedural-vfx`, `node-refine-edge`, `node-refraction-caustics`, `node-relight-3d`, `node-resizer-scaler`, `node-rotate`, `node-selective-color`, `node-shadows-highlights`, `node-shape-generator`, `node-signal-gate`, `node-signal-math`, `node-signal`, `node-stereo-panning`, `node-text`, `node-tile-offset`, `node-trails`, `node-unsharp-mask`, `node-vignette`, `node-vision`diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex 653d39d0..28e73282 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -432,28 +432,6 @@ importers:specifier: ^5.8.3version: 5.9.3- examples/e_kofte:- dependencies:- framefields:- specifier: workspace:*- version: link:../../packages/framefields- mediabunny:- specifier: ^1.61.6- version: 1.61.6- devDependencies:- '@framefields/tsconfig':- specifier: workspace:*- version: link:../../packages/tsconfig- '@types/node':- specifier: ^24.1.0- version: 24.10.14- tsx:- specifier: ^4.20.3- version: 4.21.0- typescript:- specifier: ^5.8.3- version: 5.9.3-nodes/node-apply-lut:dependencies:'@framefields/core':@@ -1090,6 +1068,34 @@ importers:specifier: ^5.8.3version: 5.8.3+ nodes/node-datamosh:+ dependencies:+ '@framefields/node-sdk':+ specifier: workspace:*+ version: link:../../packages/node-sdk+ '@framefields/tensor-webgpu':+ specifier: workspace:*+ version: link:../../packages/tensor-webgpu+ zod:+ specifier: ^4.3.6+ version: 4.3.6+ devDependencies:+ '@framefields/tsconfig':+ specifier: workspace:*+ version: link:../../packages/tsconfig+ '@types/node':+ specifier: ^24.1.0+ version: 24.10.14+ tsdown:+ specifier: ^0.19.0+ version: 0.19.0([email protected])([email protected])+ typescript:+ specifier: ^5.8.3+ version: 5.9.3+ webgpu:+ specifier: ^0.4.0+ version: 0.4.0+nodes/node-deflicker:dependencies:'@framefields/core':@@ -1624,6 +1630,28 @@ importers:specifier: ^5.8.3version: 5.8.3+ nodes/node-particles:+ dependencies:+ '@framefields/node-sdk':+ specifier: workspace:*+ version: link:../../packages/node-sdk+ devDependencies:+ '@framefields/tsconfig':+ specifier: workspace:*+ version: link:../../packages/tsconfig+ '@types/node':+ specifier: ^24.1.0+ version: 24.10.14+ tsdown:+ specifier: ^0.19.0+ version: 0.19.0([email protected])([email protected])+ typescript:+ specifier: ^5.8.3+ version: 5.9.3+ webgpu:+ specifier: ^0.4.0+ version: 0.4.0+nodes/node-patch-heal:dependencies:'@framefields/core':@@ -2108,6 +2136,31 @@ importers:specifier: ^0.4.0version: 0.4.0+ nodes/node-trails:+ dependencies:+ '@framefields/node-sdk':+ specifier: workspace:*+ version: link:../../packages/node-sdk+ zod:+ specifier: ^4.3.6+ version: 4.3.6+ devDependencies:+ '@framefields/tsconfig':+ specifier: workspace:*+ version: link:../../packages/tsconfig+ '@types/node':+ specifier: ^24.1.0+ version: 24.10.14+ tsdown:+ specifier: ^0.19.0+ version: 0.19.0([email protected])([email protected])+ typescript:+ specifier: ^5.8.3+ version: 5.9.3+ webgpu:+ specifier: ^0.4.0+ version: 0.4.0+nodes/node-unsharp-mask:dependencies:'@framefields/core':@@ -2465,6 +2518,9 @@ importers:'@framefields/node-paint':specifier: workspace:*version: link:../../nodes/node-paint+ '@framefields/node-particles':+ specifier: workspace:*+ version: link:../../nodes/node-particles'@framefields/node-patch-heal':specifier: workspace:*version: link:../../nodes/node-patch-heal@@ -2662,6 +2718,9 @@ importers:'@framefields/node-curves':specifier: workspace:*version: link:../../nodes/node-curves+ '@framefields/node-datamosh':+ specifier: workspace:*+ version: link:../../nodes/node-datamosh'@framefields/node-deflicker':specifier: workspace:*version: link:../../nodes/node-deflicker@@ -2719,6 +2778,9 @@ importers:'@framefields/node-paint':specifier: workspace:*version: link:../../nodes/node-paint+ '@framefields/node-particles':+ specifier: workspace:*+ version: link:../../nodes/node-particles'@framefields/node-patch-heal':specifier: workspace:*version: link:../../nodes/node-patch-heal@@ -2758,6 +2820,9 @@ importers:'@framefields/node-tile-offset':specifier: workspace:*version: link:../../nodes/node-tile-offset+ '@framefields/node-trails':+ specifier: workspace:*+ version: link:../../nodes/node-trails'@framefields/node-unsharp-mask':specifier: workspace:*version: link:../../nodes/node-unsharp-mask