Skip to content

Code-first video & image, rendered natively on WebGPU.

feat: GPU particles, glass refraction, Trails/Datamosh, physics fixes; new dancer cut

okanasl741f6d21eb97
diff --git a/.changeset/physics-rapier.md b/.changeset/physics-rapier.md
index f570bbbc..bb4dafdf 100644
--- a/.changeset/physics-rapier.md
+++ b/.changeset/physics-rapier.md
@@ -7,3 +7,7 @@ Add rigid-body physics: `Physics.world2d` / `Physics.world3d` (Rapier, determini
`world.field()` adds position-dependent forces (wind, vortices, flow fields). `world.live(comp)` steps a world as the composition renders instead of all at once: before each frame draws it advances to it, after that frame's inputs are ready, giving the same result as `simulate()`.
`comp.vision(clip)` analyses a clip's own frames on demand (pose keypoints and a soft silhouette) and shares the results with everything that reads them; every render path and the preview await it before drawing a frame, and results persist on disk in Node. `comp.onPrepareFrame()` registers async per-frame work, and the renderer awaits an `onRequestFrame` that returns a promise. `colorSignal` takes `multiplier` and `offset`.
+
+`Layer.particles` adds GPU particle systems: a compute-shader simulation (emitters, curl-noise flow, attractors, gravity, drag, floors and bounds, a performer's silhouette push) drawn in one instanced call, with snapshots so scrubbing back restores state. New temporal effects: `Trails` (echoes, light painting, feedback tunnels) and `Datamosh` (pixels dragged along optical flow between keyframes).
+
+3D `glass`, `acrylic` and `frosted` materials now refract what is behind them on screen (with dispersion, thickness, tint and studio reflections) instead of drawing as translucent plastic. Particle emitters can move: `emitter.x` / `emitter.y` take drivers, and births spread along the path between frames (light painting from a tracked hand).
diff --git a/examples/28_physics_dancer/README.md b/examples/28_physics_dancer/README.md
index ba7ca65a..df62a769 100644
--- a/examples/28_physics_dancer/README.md
+++ b/examples/28_physics_dancer/README.md
@@ -1,8 +1,8 @@
# 28 — Physics dancer
-A 6.6-second shot where rigid-body physics meets on-device vision. Word cards fall onto a
-dancer's raised hands and head and tumble to the floor; a cloud of ~800 particles flows around
-her as the silk dress sweeps through it.
+A 6.6-second shot that puts most of the engine in one frame: glass word cards are thrown in from
+both sides and bounce off a dancer's raised arms and head, the last one landing on her hands; a
+cloud of embers parts around her silk dress. Renders in about 20 seconds.
```bash
pnpm fonts # Syne from google/fonts → assets/fonts/
@@ -19,13 +19,9 @@ The clip is example 19's `assets/dancer.mp4`; generate it there with `pnpm asset
| Piece | API |
|---|---|
| Pose and silhouette, frame by frame as it renders | `comp.vision(clip).person`, placed with `.place({ width, height, fit })`; results persist in `~/.cache/framefields/vision` |
-| Physics that steps with the render | `world.live(comp)`: before each frame draws, its vision is ready and the world advances to it |
-| Arms and head that catch the cards | kinematic circle bodies along shoulder → elbow → wrist (and past the wrist for the hand), `follow`ing `person.point(name, frame)` |
-| The dress pushing particles and cards | `world.field()` with `person.sample(frame, x, y)`: out along the silhouette's gradient, much harder where the silhouette just arrived (the sweep) |
-| Fluid feel | small circles with `linearDamping` 2.2, near-zero gravity, a slow swirling current and a soft pull home |
-| Particles behind her | a second copy of the clip, cut out with `.withVision({ mode: "matte" })`, above the particles |
-| Hot particles | `.colorSignal("fillColor", body.speed, { multiplier: 1 / 900, … })` |
-| Screen shake | the whole stage's `y` driven by `world.impact` |
-
-Cards are sized with `FontManager.measure`, so the text always fits. A live world can't look
-ahead, so they drop at fixed offsets from where her head starts and find her hands on the way down.
+| 3D physics that steps with the render | `Physics.world3d(…)` + `world.live(comp)`: before each frame draws, its vision is ready and the world advances to it |
+| Glass cards | `Layer3D.roundedBox({ material: "glass" })` as the rigid body, the word as unlit blue `Layer3D.extrudedText` following the body's `x`/`y`/`z` signals just in front of it; `lockRotation: true` keeps every word upright |
+| Thrown in from the sides | each card's launch `velocity` solved from gravity and a 0.7 s flight so the arc crosses her head and arms; the finale drops from above with a lower `gravityScale` |
+| Arms and head that catch them | kinematic spheres along shoulder → elbow → wrist (and past it for the hand), `follow`ing `person.point(name, frame)`; without a camera layer the 3D z = 0 plane matches the video 1:1 |
+| Embers | `Layer.particles` with `push: { person }` (her silhouette, hardest where it sweeps in), curl `noise`, `colorBySpeed` |
+| Embers behind her | a second copy of the clip, cut out with `.withVision({ mode: "matte" })`, above the embers |
diff --git a/examples/28_physics_dancer/src/film.ts b/examples/28_physics_dancer/src/film.ts
index 89198b0f..7b1f6e84 100644
--- a/examples/28_physics_dancer/src/film.ts
+++ b/examples/28_physics_dancer/src/film.ts
@@ -1,30 +1,31 @@
/**
- * A dancer in a field of physics: word cards drop onto her raised hands and
- * head, and a cloud of particles flows around her as the dress sweeps through.
+ * Glass word cards are thrown in from both sides onto a dancer's raised hands while she sweeps a cloud
+ * of embers aside.
*
- * Nothing is analysed up front: `comp.vision` analyses each frame of the clip
- * just before it renders, and the live physics world steps to that frame
+ * Nothing is analysed up front: `comp.vision` reads each frame of the clip
+ * just before it renders, and the live 3D physics world steps to that frame
* right after, so the preview starts at once and fills in as it plays.
*
- * - Her arms and head are kinematic colliders that follow the tracked pose.
- * - Her silhouette (dress included) is a force field that pushes everything out,
- * hardest where it is sweeping in.
- * - A vision cutout of the same clip sits above the particles, so they pass behind her.
+ * - Each card is a 3D glass slab (Rapier rigid body) that refracts
+ * everything behind it, with its word in solid blue welded in front.
+ * - Her arms and head are kinematic colliders that follow the tracked pose;
+ * her silhouette (dress included) pushes the embers out of the way.
+ * - A vision cutout of the clip sits above the embers, so they pass behind her.
*/
import fs from "node:fs";
import {
Composition,
FontManager,
Layer,
+ Layer3D,
LayerAnimation,
type PersonTrack,
Physics,
- type PhysicsBody2D,
- type PhysicsWorld2D,
+ type PhysicsBody3D,
+ type PhysicsWorld3D,
type PoseKeypointName,
} from "framefields";
import {
- CREAM,
DANCER,
DURATION,
EMBER,
@@ -34,27 +35,16 @@ import {
GOLD,
H,
HOT,
- INK,
RUST,
registerFonts,
- VIOLET,
W,
} from "./theme.js";
-const WORDS = [
- // `dx`: where it drops, from her head at the first frame (live worlds can't look ahead).
- { text: "PHYSICS", bg: EMBER, fg: INK, at: 30, dx: -40 },
- { text: "MEETS", bg: INK, fg: CREAM, at: 46, dx: 120 },
- { text: "VISION", bg: CREAM, fg: INK, at: 62, dx: -120 },
- { text: "IN", bg: GOLD, fg: INK, at: 78, dx: 60 },
- { text: "FRAMEFIELDS", bg: VIOLET, fg: CREAM, at: 94, dx: 0 },
-];
-const CARD_FONT_SIZE = 64;
-const PARTICLE_COLORS = [INK, RUST, EMBER, GOLD];
+export { DURATION as DURATION_FRAMES } from "./theme.js";
-/** Balls along a limb (u = 0 at `a`, 1 at `b`, beyond 1 past `b`) that follow the pose. */
+/** Spheres along a limb (u = 0 at `a`, 1 at `b`, beyond 1 past it) that follow the pose, in the z = 0 plane. */
function limb(
- world: PhysicsWorld2D,
+ world: PhysicsWorld3D,
person: PersonTrack,
a: PoseKeypointName,
b: PoseKeypointName,
@@ -69,126 +59,161 @@ function limb(
};
const [x, y] = at(0);
world.body(
- { x: x - r, y: y - r, width: 2 * r, height: 2 * r },
+ { x, y, z: 0, width: 2 * r, height: 2 * r, depth: 2 * r },
{
type: "kinematic",
- shape: "circle",
- friction: 0.9,
- follow: { x: (f) => at(f)[0] - r, y: (f) => at(f)[1] - r },
+ shape: "sphere",
+ friction: 0.8,
+ follow: { x: (f) => at(f)[0], y: (f) => at(f)[1], z: 0 },
},
);
}
}
-/** Word cards sized to their text, dropped around where her head starts. */
-function cards(world: PhysicsWorld2D, person: PersonTrack) {
- return WORDS.map((w, i) => {
- const m = FontManager.measure(w.text, {
+/** The framefields blues. */
+const BLUE = "#3B82F6";
+const BLUE_DEEP = "#1D4ED8";
+const NAVY = "#0B1E4D";
+
+const CARDS = [
+ // `from`: the side it is thrown in from; `dx`: where on her it is aimed, from her head.
+ { text: "PHYSICS", color: BLUE, at: 24, from: "left", dx: -30 },
+ { text: "MEETS", color: NAVY, at: 38, from: "right", dx: 50 },
+ { text: "VISION", color: BLUE, at: 52, from: "left", dx: 10 },
+ { text: "IN", color: BLUE_DEEP, at: 66, from: "right", dx: -20 },
+ // The finale drops straight down onto her, slower to read.
+ { text: "FRAMEFIELDS", color: BLUE_DEEP, at: 82, from: "top", dx: 0 },
+];
+const CARD_FONT = 76;
+const CARD_DEPTH = 34;
+const GRAVITY = 2600;
+/** Seconds from the edge of the frame to her. */
+const FLIGHT = 0.7;
+
+/**
+ * Word cards: a 3D glass slab (the rigid body) with its word in solid blue
+ * riding just in front of it. Cards keep their slight tilt and never turn,
+ * so the words stay upright and readable while the cards bounce off her.
+ */
+function glassCards(world: PhysicsWorld3D, person: PersonTrack) {
+ const head = person.point("nose", 0);
+ return CARDS.map((card, i) => {
+ const m = FontManager.measure(card.text, {
fontFamily: "Syne",
- fontSize: CARD_FONT_SIZE,
+ fontSize: CARD_FONT,
});
- const width = Math.ceil(m.width + 72);
- const height = Math.ceil(m.height + 40);
- const head = person.point("nose", 0);
- const layer = Layer.box({
+ const width = Math.ceil(m.width + 84);
+ const height = Math.ceil(m.height + 52);
+ // Thrown in from the side on an arc that crosses her head and raised
+ // arms: launch velocity solved from gravity and the flight time.
+ const target = { x: head.x + card.dx, y: head.y + 40 };
+ const fromTop = card.from === "top";
+ const x = fromTop
+ ? target.x
+ : card.from === "left"
+ ? -width / 2 - 40
+ : W + width / 2 + 40;
+ const y = fromTop ? -height - 60 : 430 - (i % 2) * 50;
+ const velocity: [number, number, number] = fromTop
+ ? [0, 0, 0]
+ : [
+ (target.x - x) / FLIGHT,
+ (target.y - y - 0.5 * GRAVITY * FLIGHT * FLIGHT) / FLIGHT,
+ 0,
+ ];
+ const tilt = (i % 2 ? 1 : -1) * (4 + i * 2);
+ const slab = Layer3D.roundedBox({
id: `card-${i}`,
- position: "absolute",
- x: head.x + w.dx - width / 2,
- y: -height - 200,
width,
height,
- rotation: (i % 2 ? 1 : -1) * (6 + i * 2),
- borderRadius: 18,
- background: w.bg,
- children: [
- Layer.text(w.text, {
- fontFamily: "Syne",
- fontSize: CARD_FONT_SIZE,
- fill: w.fg,
- width,
- height,
- align: "center",
- verticalAlign: "middle",
- }),
- ],
+ depth: CARD_DEPTH,
+ radius: 18,
+ x,
+ y,
+ z: 0,
+ rotateZ: tilt,
+ color: "#eef5ff",
+ material: "glass",
+ roughness: 0.03,
});
- const body = world.body(layer, {
- restitution: 0.25,
- friction: 0.7,
- density: 1.5,
- angularDamping: 1.2,
- startFrame: w.at,
+ const body = world.body(slab, {
+ restitution: fromTop ? 0.1 : 0.3,
+ friction: fromTop ? 0.9 : 0.5,
+ // The finale falls slower, so it reads on the way down.
+ gravityScale: fromTop ? 0.45 : 1,
+ density: 1.2,
+ startFrame: card.at,
+ velocity,
+ lockRotation: true,
});
- return { layer, body };
+ const words = Layer3D.extrudedText({
+ id: `card-${i}-text`,
+ text: card.text,
+ fontFamily: "Syne",
+ fontSize: CARD_FONT,
+ depth: 3,
+ align: "center",
+ verticalAlign: "middle",
+ fill: card.color,
+ material: "unlit",
+ x,
+ y,
+ z: -CARD_DEPTH / 2 - 2,
+ rotateZ: tilt,
+ }).animate(
+ LayerAnimation.create()
+ .signal("x", body.x)
+ .signal("y", body.y)
+ .signal("z", body.z, { offset: -CARD_DEPTH / 2 - 2 }),
+ );
+ return { layer: slab, words, body };
});
}
-/** A loose cloud of small particles around her, avoiding where she stands. */
-function particles(world: PhysicsWorld2D, person: PersonTrack) {
- let seed = 7;
- const rand = () => {
- seed = (seed * 16807) % 2147483647;
- return seed / 2147483647;
- };
- const out: Array<{
- layer: ReturnType<typeof Layer.shape>;
- body: PhysicsBody2D;
- color: string;
- home: [number, number];
- }> = [];
- for (let y = 430; y < FLOOR - 10; y += 24) {
- for (let x = 20; x < W - 20; x += 24) {
- const px = x + (rand() - 0.5) * 12;
- const py = y + (rand() - 0.5) * 12;
- if (person.sample(0, px, py).alpha > 0.15 || rand() > 0.5) continue;
- const r = 6 + rand() * 6;
- const color =
- PARTICLE_COLORS[Math.floor(rand() * PARTICLE_COLORS.length)];
- const layer = Layer.shape("circle", {
- position: "absolute",
- x: px - r,
- y: py - r,
- width: 2 * r,
- height: 2 * r,
- fill: color,
- });
- const body = world.body(layer, {
- restitution: 0.05,
- friction: 0.02,
- density: 0.25,
- linearDamping: 2.2,
- gravityScale: 0.06,
- lockRotation: true,
- });
- out.push({ layer, body, color, home: [px, py] });
- }
- }
- return out;
-}
-
-/** Out of the silhouette: `strength` where she is, plus `sweep` where she is arriving. */
-function silhouettePush(person: PersonTrack, strength: number, sweep: number) {
- return (s: {
- x: number;
- y: number;
- frame: number;
- }): [number, number] | null => {
- const now = person.sample(s.frame, s.x, s.y);
- if (now.alpha < 0.02) return null;
- const before = person.sample(s.frame - 1, s.x, s.y);
- const g = Math.hypot(now.gx, now.gy) || 1;
- const k =
- now.alpha * strength + Math.max(0, now.alpha - before.alpha) * sweep;
- return [(-now.gx / g) * k, (-now.gy / g) * k];
- };
+/** Embers in the lower half: her silhouette pushes them, a slow current keeps them drifting. */
+function embers(person: PersonTrack) {
+ return Layer.particles({
+ id: "embers",
+ width: W,
+ height: H,
+ max: 24000,
+ seed: 7,
+ emitter: {
+ shape: "rect",
+ x: 0,
+ y: 380,
+ width: W,
+ height: FLOOR - 390,
+ rate: 3600,
+ },
+ lifetime: [4, 6.5],
+ speed: [0, 25],
+ drag: 2.2,
+ gravity: -6,
+ noise: { strength: 240, scale: 170, speed: 0.25 },
+ push: { person, strength: 9000, sweep: 120000 },
+ floor: FLOOR,
+ bounce: 0.1,
+ size: [11, 8],
+ sizeJitter: 0.6,
+ fade: [0.15, 0.3],
+ color: [RUST, EMBER, GOLD, HOT],
+ colorBySpeed: 500,
+ prewarm: 4,
+ });
}
-export { DURATION as DURATION_FRAMES } from "./theme.js";
+/** Keeps letters near the picture plane, so they stay where her arms are. */
+const keepInPlane = (s: { z: number }): [number, number, number] => [
+ 0,
+ 0,
+ -s.z * 6,
+];
export interface Film {
comp: Composition;
- world: PhysicsWorld2D;
- cards: Array<{ id: string; body: PhysicsBody2D }>;
+ world: PhysicsWorld3D;
+ letters: Array<{ id: string; body: PhysicsBody3D }>;
}
export async function buildFilm(): Promise<Composition> {
@@ -211,107 +236,61 @@ export async function buildScene(): Promise<Film> {
);
}
const dancer = comp.vision(DANCER);
- // The particles are laid out around her first pose: analyse just that now.
+ // Letters drop around her first pose: analyse just that now.
await dancer.ready(0);
const person = dancer.person.place({ width: W, height: H, fit: "cover" });
- const world = await Physics.world2d({
+ // Without a camera layer the 3D world's z = 0 plane lines up 1:1 with the
+ // video, so pose points are 3D positions too.
+ const world = await Physics.world3d({
fps: FPS,
frames: DURATION,
- gravity: 2600,
+ gravity: GRAVITY,
substeps: 6,
- impactDecay: 5,
});
- world.walls({ width: W, height: FLOOR });
-
+ // Only a floor: cards come in from beyond the frame edges.
+ world.ground({ y: FLOOR });
for (const side of ["left", "right"] as const) {
- limb(world, person, `${side}Shoulder`, `${side}Elbow`, [0, 0.5, 1], 26);
+ limb(world, person, `${side}Shoulder`, `${side}Elbow`, [0, 0.5, 1], 30);
limb(
world,
person,
`${side}Elbow`,
`${side}Wrist`,
[0.33, 0.66, 1, 1.25],
- 24,
+ 28,
);
}
- limb(world, person, "leftShoulder", "rightShoulder", [0.5], 40);
- limb(world, person, "nose", "nose", [0], 52);
-
- const words = cards(world, person);
- const cloud = particles(world, person);
- const cloudBodies = cloud.map((p) => p.body);
- const home = new Map(cloud.map((p) => [p.body, p.home]));
-
- world.field(silhouettePush(person, 11000, 140000), { bodies: cloudBodies });
- world.field(silhouettePush(person, 16000, 60000), {
- bodies: words.map((w) => w.body),
+ limb(world, person, "leftShoulder", "rightShoulder", [0.5], 46);
+ limb(world, person, "nose", "nose", [0], 58);
+ const letters = glassCards(world, person);
+ world.field(keepInPlane, {
+ bodies: letters.map((l) => l.body),
});
- // A slow swirling current keeps the cloud alive; a soft pull home keeps it on stage.
- world.field(
- (s) => {
- const t = s.frame / FPS;
- const [hx, hy] = home.get(s.body) ?? [s.x, s.y];
- return [
- 140 * Math.sin(s.y * 0.009 + t * 1.7) + (hx - s.x) * 0.35,
- 110 * Math.cos(s.x * 0.007 - t * 1.3) + (hy - s.y) * 0.6,
- ];
- },
- { bodies: cloudBodies },
- );
- // Steps as the composition renders, after each frame's vision is ready.
world.live(comp);
- // Particles glow as they speed up (900 px/s is fully hot).
- for (const p of cloud) {
- p.layer.animate(
- LayerAnimation.create().colorSignal("fillColor", p.body.speed, {
- colorA: p.color,
- colorB: HOT,
- multiplier: 1 / 900,
- }),
- );
- }
-
- comp.add(
- Layer.box({
- id: "stage",
+ const video = (id: string, muted: boolean) =>
+ Layer.video(DANCER, {
+ id,
position: "absolute",
x: 0,
y: 0,
width: W,
height: H,
- children: [
- Layer.video(DANCER, {
- id: "plate",
- position: "absolute",
- x: 0,
- y: 0,
- width: W,
- height: H,
- fit: "cover",
- }),
- ...cloud.map((p) => p.layer),
- Layer.video(DANCER, {
- id: "cutout",
- position: "absolute",
- x: 0,
- y: 0,
- width: W,
- height: H,
- fit: "cover",
- muted: true,
- }).withVision({ mode: "matte" }),
- ...words.map((w) => w.layer),
- ],
- }).animate(
- LayerAnimation.create().signal("y", world.impact, { multiplier: 10 }),
- ),
- );
+ fit: "cover",
+ muted,
+ });
+
+ comp.add([
+ video("plate", false),
+ embers(person),
+ video("cutout", true).withVision({ mode: "matte" }),
+ ...letters.flatMap((l) => [l.layer, l.words]),
+ ]);
return {
comp,
world,
- cards: words.map((w) => ({ id: String(w.layer.id), body: w.body })),
+ letters: letters.map((l) => ({ id: String(l.layer.id), body: l.body })),
};
}
diff --git a/examples/28_physics_dancer/src/render.ts b/examples/28_physics_dancer/src/render.ts
index b8fc9fdc..73c0c9c7 100644
--- a/examples/28_physics_dancer/src/render.ts
+++ b/examples/28_physics_dancer/src/render.ts
@@ -22,7 +22,7 @@ if (mode === "preview") {
process.exit(0);
}
-const { comp, world, cards } = await buildScene();
+const { comp, world, letters } = await buildScene();
if (mode === "sim") {
const started = Date.now();
@@ -30,7 +30,7 @@ if (mode === "sim") {
console.log(
`Analysed and simulated in ${((Date.now() - started) / 1000).toFixed(1)} s`,
);
- for (const { id, body } of cards) {
+ for (const { id, body } of letters) {
const hit = body.collisions[0];
console.log(
`${id}: first contact at frame ${hit?.frame ?? "-"} (${hit?.with.type ?? "none"})`,
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 06013f21..29f31cf4 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -1148,7 +1148,18 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
Boolean(sceneBloom),
);
- for (const item of pending3DItems) {
+ // Glass reads what is behind the batch (the canvas so far), and goes
+ // last so opaque meshes are already in the depth buffer.
+ const isGlass = (item: Pending3DItem) =>
+ item.kind === "mesh" &&
+ (item.opts.material === "glass" ||
+ item.opts.material === "acrylic" ||
+ item.opts.material === "frosted");
+ const ordered = [
+ ...pending3DItems.filter((i) => !isGlass(i)),
+ ...pending3DItems.filter(isGlass),
+ ];
+ for (const item of ordered) {
if (item.kind === "quad") {
r3d.drawTextureQuad(
pass3d,
@@ -1175,7 +1186,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
lightsBindGroup,
item.meshBuffers ?? item.mesh,
item.texture,
- item.opts,
+ isGlass(item) ? { ...item.opts, backdrop: curTex } : item.opts,
dofEnabled,
);
}
@@ -1954,6 +1965,9 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
target?.ambientIntensity ?? lop.ambientIntensity ?? 1.0,
metallic: target?.metallic ?? lop.metallic ?? mat?.metallic,
...surfaceProps(target, lop, mat),
+ ior: target?.ior ?? lop.ior,
+ dispersion: target?.dispersion ?? lop.dispersion,
+ thickness: target?.thickness ?? lop.thickness,
audioDeform: meshAudioDeform,
};
diff --git a/nodes/node-datamosh/package.json b/nodes/node-datamosh/package.json
new file mode 100644
index 00000000..76cea0b9
--- /dev/null
+++ b/nodes/node-datamosh/package.json
@@ -0,0 +1,47 @@
+{
+ "name": "@framefields/node-datamosh",
+ "version": "2.2.0",
+ "private": false,
+ "framefields": {
+ "type": "Datamosh",
+ "kind": "effect",
+ "schema": "DatamoshNodeConfigSchema"
+ },
+ "type": "module",
+ "license": "Apache-2.0",
+ "exports": {
+ ".": {
+ "types": "./dist/index.d.mts",
+ "development": "./src/index.ts",
+ "import": "./dist/index.mjs",
+ "default": "./dist/index.mjs"
+ },
+ "./renderer": {
+ "types": "./dist/renderer.d.mts",
+ "development": "./src/renderers/index.ts",
+ "import": "./dist/renderer.mjs",
+ "default": "./dist/renderer.mjs"
+ }
+ },
+ "scripts": {
+ "build": "tsdown",
+ "dev": "tsdown --watch",
+ "clean": "rm -rf dist .turbo",
+ "typecheck": "tsc --noEmit"
+ },
+ "dependencies": {
+ "@framefields/node-sdk": "workspace:*",
+ "@framefields/tensor-webgpu": "workspace:*",
+ "zod": "^4.3.6"
+ },
+ "devDependencies": {
+ "@framefields/tsconfig": "workspace:*",
+ "@types/node": "^24.1.0",
+ "tsdown": "^0.19.0",
+ "typescript": "^5.8.3",
+ "webgpu": "^0.4.0"
+ },
+ "publishConfig": {
+ "access": "public"
+ }
+}
diff --git a/nodes/node-datamosh/src/index.ts b/nodes/node-datamosh/src/index.ts
new file mode 100644
index 00000000..944d702e
--- /dev/null
+++ b/nodes/node-datamosh/src/index.ts
@@ -0,0 +1,2 @@
+export * from "./renderers/index.js";
+export * from "./shared/index.js";
diff --git a/nodes/node-datamosh/src/renderers/index.ts b/nodes/node-datamosh/src/renderers/index.ts
new file mode 100644
index 00000000..28b94a6e
--- /dev/null
+++ b/nodes/node-datamosh/src/renderers/index.ts
@@ -0,0 +1,7 @@
+import { defineRenderer } from "@framefields/node-sdk/renderer";
+import { DatamoshWebGPURenderer } from "./webgpu-renderer.js";
+
+export default defineRenderer({
+ WebGPURenderer: DatamoshWebGPURenderer,
+});
+export { DatamoshWebGPURenderer };
diff --git a/nodes/node-datamosh/src/renderers/webgpu-renderer.ts b/nodes/node-datamosh/src/renderers/webgpu-renderer.ts
new file mode 100644
index 00000000..724878f1
--- /dev/null
+++ b/nodes/node-datamosh/src/renderers/webgpu-renderer.ts
@@ -0,0 +1,274 @@
+/// <reference types="webgpu" />
+/**
+ * Datamosh: instead of each new frame, the previous output dragged along the
+ * layer's optical flow, until a keyframe (I-frame) shows the clean picture.
+ * History lives in textures on the effect object and advances one frame at a
+ * time; after a jump the frames since the last keyframe are replayed.
+ */
+import type { WebGPUNodeRenderer } from "@framefields/node-sdk";
+import {
+ drawChildToTexture,
+ drawFullscreenPass,
+ drawTextureToTarget,
+ FULLSCREEN_QUAD_WGSL,
+ fragmentBindGroupLayout,
+ perDevice,
+ type WebGPUNodeRendererArgs,
+} from "@framefields/node-sdk/gpu";
+import { OpticalFlowComputePipeline } from "@framefields/tensor-webgpu";
+
+const HISTORY_FORMAT: GPUTextureFormat = "rgba16float";
+
+interface DatamoshValues {
+ keyframes: number[];
+ strength: number;
+ bleed: number;
+ warmup: number;
+}
+
+const MOSH_WGSL = /* wgsl */ `
+struct U { a : vec4<f32>, b : vec4<f32> };
+@group(0) @binding(0) var<uniform> u : U;
+@group(0) @binding(1) var moshTex : texture_2d<f32>;
+@group(0) @binding(2) var currentTex : texture_2d<f32>;
+@group(0) @binding(3) var flowTex : texture_2d<f32>;
+@group(0) @binding(4) var samp : sampler;
+${FULLSCREEN_QUAD_WGSL}
+
+@fragment fn fs(in : VSOut) -> @location(0) vec4<f32> {
+ let strength = u.a.x;
+ let bleed = u.a.y;
+ let inv = u.a.zw;
+ let cur0 = textureSampleLevel(currentTex, samp, in.uv, 0.0);
+ if (u.b.x > 0.5) { return cur0; }
+ let dims = vec2<f32>(textureDimensions(flowTex));
+ let coord = vec2<i32>(clamp(in.uv * dims, vec2<f32>(0.0), dims - 1.0));
+ let flow = textureLoad(flowTex, coord, 0).xy;
+ let cur = textureSampleLevel(currentTex, samp, in.uv, 0.0);
+ // The old picture moves with the new motion instead of being replaced.
+ let moved = textureSampleLevel(moshTex, samp, clamp(in.uv - flow * inv * strength, vec2<f32>(0.0), vec2<f32>(1.0)), 0.0);
+ var out = mix(moved, cur, bleed);
+ if (moved.a < 0.01) { out = cur; }
+ return out;
+}
+`;
+
+const resources = perDevice((device: GPUDevice) => {
+ const layout = fragmentBindGroupLayout(
+ device,
+ ["uniform", "texture", "texture", "texture", "sampler"],
+ "datamosh_layout",
+ );
+ const module = device.createShaderModule({
+ code: MOSH_WGSL,
+ label: "datamosh.wgsl",
+ });
+ return {
+ layout,
+ pipeline: device.createRenderPipeline({
+ label: "datamosh",
+ layout: device.createPipelineLayout({ bindGroupLayouts: [layout] }),
+ vertex: { module, entryPoint: "vs" },
+ fragment: {
+ module,
+ entryPoint: "fs",
+ targets: [{ format: HISTORY_FORMAT }],
+ },
+ primitive: { topology: "triangle-strip" },
+ }),
+ flow: new OpticalFlowComputePipeline(device),
+ sampler: device.createSampler({ magFilter: "linear", minFilter: "linear" }),
+ };
+});
+
+interface History {
+ device: GPUDevice;
+ width: number;
+ height: number;
+ /** Ping-pong history; `front` holds the latest. */
+ front: GPUTexture;
+ back: GPUTexture;
+ /** Datamosh: the layer's previous clean frame. */
+ prev: GPUTexture | null;
+ /** Layer frame `front` holds; -1 = none. */
+ frame: number;
+}
+
+interface FeedbackEffect {
+ resolveUniforms(ctx?: unknown): Record<string, unknown>;
+ feedbackHistory?: History;
+}
+
+function historyFor(
+ effect: FeedbackEffect,
+ device: GPUDevice,
+ width: number,
+ height: number,
+ format: GPUTextureFormat,
+): History {
+ const h = effect.feedbackHistory;
+ if (h && h.device === device && h.width === width && h.height === height)
+ return h;
+ h?.front.destroy();
+ h?.back.destroy();
+ h?.prev?.destroy();
+ const make = (label: string, fmt: GPUTextureFormat) =>
+ device.createTexture({
+ label,
+ size: [width, height],
+ format: fmt,
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT |
+ GPUTextureUsage.TEXTURE_BINDING |
+ GPUTextureUsage.COPY_SRC |
+ GPUTextureUsage.COPY_DST,
+ });
+ const next: History = {
+ device,
+ width,
+ height,
+ front: make("feedback_front", HISTORY_FORMAT),
+ back: make("feedback_back", HISTORY_FORMAT),
+ prev: make("feedback_prev", format),
+ frame: -1,
+ };
+ effect.feedbackHistory = next;
+ return next;
+}
+
+async function childAt(
+ args: WebGPUNodeRendererArgs,
+ frame: number,
+ compositionStart: number,
+ h: History,
+) {
+ const child = args.props.virtualMedia?.children?.[0];
+ if (!child) return null;
+ return drawChildToTexture(args, child, {
+ exclude: [h.front, h.back, ...(h.prev ? [h.prev] : [])],
+ props: {
+ ...args.props,
+ virtualMedia: child,
+ frame,
+ compositionFrame: compositionStart + frame,
+ },
+ });
+}
+
+function swap(h: History): void {
+ const t = h.front;
+ h.front = h.back;
+ h.back = t;
+}
+
+async function stepDatamosh(
+ args: WebGPUNodeRendererArgs,
+ h: History,
+ v: DatamoshValues,
+ frame: number,
+ compositionStart: number,
+ key: boolean,
+): Promise<void> {
+ const cur = await childAt(args, frame, compositionStart, h);
+ if (!cur || !h.prev) return;
+ const r = resources(args.ctx.device);
+ const { encoder } = args;
+ const clean = key || h.frame < 0;
+ // On a keyframe the flow is unused; the current frame stands in for it.
+ const flow = clean
+ ? cur.texture
+ : r.flow.execute(h.prev, cur.texture, {}, encoder);
+ const uniforms = new Float32Array([
+ v.strength,
+ v.bleed,
+ 1 / h.width,
+ 1 / h.height,
+ clean ? 1 : 0,
+ 0,
+ 0,
+ 0,
+ ]);
+ drawFullscreenPass(encoder, h.back.createView(), r.pipeline, [
+ args.ctx.device.createBindGroup({
+ layout: r.layout,
+ entries: [
+ {
+ binding: 0,
+ resource: { buffer: args.ctx.renderer.getTemporaryBuffer(uniforms) },
+ },
+ { binding: 1, resource: h.front.createView() },
+ { binding: 2, resource: cur.view },
+ { binding: 3, resource: flow.createView() },
+ { binding: 4, resource: r.sampler },
+ ],
+ }),
+ ]);
+ encoder.copyTextureToTexture({ texture: cur.texture }, { texture: h.prev }, [
+ h.width,
+ h.height,
+ 1,
+ ]);
+ swap(h);
+ h.frame = frame;
+}
+
+function num(v: unknown, fallback: number): number {
+ return typeof v === "number" && Number.isFinite(v) ? v : fallback;
+}
+
+export const DatamoshWebGPURenderer: WebGPUNodeRenderer = async (args) => {
+ const { ctx, pass, targetWidth, targetHeight, props } = args;
+ const op = props.virtualMedia?.operation as
+ | { op?: string; effect?: FeedbackEffect }
+ | undefined;
+ if (op?.op !== "Datamosh" || !op.effect?.resolveUniforms) return;
+ if (!props.virtualMedia?.children?.[0]) return;
+ pass.end();
+
+ const fps = props.fps || 30;
+ const local = Math.max(0, Math.round(props.frame ?? 0));
+ const compositionFrame = Math.round(props.compositionFrame ?? local);
+ const compositionStart = compositionFrame - local;
+ const raw = op.effect.resolveUniforms({
+ frame: compositionFrame,
+ fps,
+ time: compositionFrame / fps,
+ });
+ const h = historyFor(
+ op.effect,
+ ctx.device,
+ targetWidth,
+ targetHeight,
+ ctx.renderer.format,
+ );
+
+ {
+ const v: DatamoshValues = {
+ keyframes: Array.isArray(raw.keyframes)
+ ? (raw.keyframes as number[]).map(Math.round)
+ : [],
+ strength: num(raw.strength, 1),
+ bleed: Math.min(1, Math.max(0, num(raw.bleed, 0))),
+ warmup: Math.max(1, Math.round(num(raw.warmup, 48))),
+ };
+ if (h.frame !== local) {
+ let start = local;
+ if (h.frame !== local - 1) {
+ const lastKey = Math.max(0, ...v.keyframes.filter((k) => k <= local));
+ start = Math.max(lastKey, local - v.warmup);
+ h.frame = -1;
+ }
+ for (let f = start; f <= local; f++) {
+ await stepDatamosh(
+ args,
+ h,
+ v,
+ f,
+ compositionStart,
+ f === 0 || v.keyframes.includes(f) || h.frame < 0,
+ );
+ }
+ }
+ }
+ drawTextureToTarget(args, h.front);
+};
diff --git a/nodes/node-datamosh/src/shared/config.ts b/nodes/node-datamosh/src/shared/config.ts
new file mode 100644
index 00000000..110ddc45
--- /dev/null
+++ b/nodes/node-datamosh/src/shared/config.ts
@@ -0,0 +1,31 @@
+import { configBuilder } from "@framefields/node-sdk";
+import { z } from "zod";
+
+export const datamoshConfig = configBuilder()
+ .field("keyframes", z.array(z.number().int().min(0)).default([]), {
+ bindable: false,
+ label: "Keyframes",
+ description:
+ "Layer frames that show the clean picture again (I-frames). The first frame always does.",
+ })
+ .field("strength", z.number().min(0).max(10).default(1), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Strength",
+ description: "How far pixels are dragged along the motion.",
+ })
+ .field("bleed", z.number().min(0).max(1).default(0), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Bleed",
+ description: "Clean picture mixed back in per frame, 0–1.",
+ })
+ .field("warmup", z.number().int().min(1).max(240).default(48), {
+ bindable: false,
+ label: "Warmup",
+ description: "Most frames replayed to rebuild the smear after a jump.",
+ })
+ .build();
+
+export const DatamoshNodeConfigSchema = datamoshConfig.schema;
+export type DatamoshNodeConfig = z.infer<typeof DatamoshNodeConfigSchema>;
diff --git a/nodes/node-datamosh/src/shared/index.ts b/nodes/node-datamosh/src/shared/index.ts
new file mode 100644
index 00000000..d1bc97e8
--- /dev/null
+++ b/nodes/node-datamosh/src/shared/index.ts
@@ -0,0 +1 @@
+export * from "./config.js";
diff --git a/nodes/node-datamosh/tsconfig.json b/nodes/node-datamosh/tsconfig.json
new file mode 100644
index 00000000..5a0b963f
--- /dev/null
+++ b/nodes/node-datamosh/tsconfig.json
@@ -0,0 +1,12 @@
+{
+ "extends": "@framefields/tsconfig/node-library.json",
+ "compilerOptions": {
+ "baseUrl": ".",
+ "paths": {
+ "@/*": ["./src/*"]
+ },
+ "experimentalDecorators": true,
+ "emitDecoratorMetadata": true
+ },
+ "include": ["src"]
+}
diff --git a/nodes/node-datamosh/tsdown.config.ts b/nodes/node-datamosh/tsdown.config.ts
new file mode 100644
index 00000000..73f5c2fc
--- /dev/null
+++ b/nodes/node-datamosh/tsdown.config.ts
@@ -0,0 +1,13 @@
+import { defineConfig } from "tsdown";
+
+export default defineConfig({
+ entry: {
+ index: "src/index.ts",
+ renderer: "src/renderers/index.ts",
+ },
+ format: ["esm"],
+ dts: true,
+ clean: true,
+ sourcemap: true,
+ treeshake: true,
+});
diff --git a/nodes/node-particles/package.json b/nodes/node-particles/package.json
new file mode 100644
index 00000000..1bad4dae
--- /dev/null
+++ b/nodes/node-particles/package.json
@@ -0,0 +1,44 @@
+{
+ "name": "@framefields/node-particles",
+ "version": "2.2.0",
+ "private": false,
+ "framefields": {
+ "type": "Particles",
+ "kind": "source"
+ },
+ "type": "module",
+ "license": "Apache-2.0",
+ "exports": {
+ ".": {
+ "types": "./dist/index.d.mts",
+ "development": "./src/index.ts",
+ "import": "./dist/index.mjs",
+ "default": "./dist/index.mjs"
+ },
+ "./renderer": {
+ "types": "./dist/renderer.d.mts",
+ "development": "./src/renderers/index.ts",
+ "import": "./dist/renderer.mjs",
+ "default": "./dist/renderer.mjs"
+ }
+ },
+ "scripts": {
+ "build": "tsdown",
+ "dev": "tsdown --watch",
+ "clean": "rm -rf dist .turbo",
+ "typecheck": "tsc --noEmit"
+ },
+ "dependencies": {
+ "@framefields/node-sdk": "workspace:*"
+ },
+ "devDependencies": {
+ "@framefields/tsconfig": "workspace:*",
+ "@types/node": "^24.1.0",
+ "tsdown": "^0.19.0",
+ "typescript": "^5.8.3",
+ "webgpu": "^0.4.0"
+ },
+ "publishConfig": {
+ "access": "public"
+ }
+}
diff --git a/nodes/node-particles/src/index.ts b/nodes/node-particles/src/index.ts
new file mode 100644
index 00000000..944d702e
--- /dev/null
+++ b/nodes/node-particles/src/index.ts
@@ -0,0 +1,2 @@
+export * from "./renderers/index.js";
+export * from "./shared/index.js";
diff --git a/nodes/node-particles/src/renderers/index.ts b/nodes/node-particles/src/renderers/index.ts
new file mode 100644
index 00000000..b8f1f065
--- /dev/null
+++ b/nodes/node-particles/src/renderers/index.ts
@@ -0,0 +1,7 @@
+import { defineRenderer } from "@framefields/node-sdk/renderer";
+import { ParticlesWebGPURenderer } from "./webgpu-renderer.js";
+
+export default defineRenderer({
+ WebGPURenderer: ParticlesWebGPURenderer,
+});
+export { ParticlesWebGPURenderer };
diff --git a/nodes/node-particles/src/renderers/webgpu-renderer.ts b/nodes/node-particles/src/renderers/webgpu-renderer.ts
new file mode 100644
index 00000000..0dffeaef
--- /dev/null
+++ b/nodes/node-particles/src/renderers/webgpu-renderer.ts
@@ -0,0 +1,735 @@
+/// <reference types="webgpu" />
+/**
+ * GPU particles: a compute pass spawns and simulates every particle, then
+ * one instanced draw renders them all. State lives on the GPU and is
+ * stepped frame by frame; snapshots every SNAPSHOT_EVERY frames make
+ * scrubbing back (preview) a restore plus a few steps instead of a replay
+ * from the start. Same GPU, same frames, same result.
+ */
+import type { WebGPUNodeRenderer } from "@framefields/node-sdk";
+import {
+ isParticleProgram,
+ type ParticleFrame,
+ type ParticleMask,
+ type ParticleProgram,
+ type ParticleStatic,
+} from "../shared/program.js";
+
+const PARTICLE_BYTES = 32;
+const PARAM_VEC4S = 32;
+const MAX_ATTRACTORS = 8;
+const MAX_COLORS = 8;
+const SNAPSHOT_EVERY = 24;
+const MAX_SNAPSHOTS = 64;
+const WORKGROUP = 64;
+
+const COMMON_WGSL = /* wgsl */ `
+struct Particle {
+ pos : vec2<f32>,
+ vel : vec2<f32>,
+ age : f32,
+ life : f32,
+ jitter : f32,
+ _pad : f32,
+};
+
+struct Params { v : array<vec4<f32>, ${PARAM_VEC4S}> };
+`;
+
+const SIM_WGSL = /* wgsl */ `
+${COMMON_WGSL}
+@group(0) @binding(0) var<uniform> P : Params;
+@group(0) @binding(1) var<storage, read_write> particles : array<Particle>;
+@group(0) @binding(2) var<storage, read> mask : array<f32>;
+
+fn pcg(v : u32) -> u32 {
+ let state = v * 747796405u + 2891336453u;
+ let word = ((state >> ((state >> 28u) + 4u)) ^ state) * 277803737u;
+ return (word >> 22u) ^ word;
+}
+fn rnd(idx : u32, channel : u32, seed : u32) -> f32 {
+ return f32(pcg(idx * 16u + channel + pcg(seed))) / 4294967295.0;
+}
+
+fn hash2(p : vec2<f32>) -> f32 {
+ let q = fract(p * vec2<f32>(0.1031, 0.1030));
+ let r = q + dot(q, q.yx + 33.33);
+ return fract((r.x + r.y) * r.x);
+}
+fn vnoise(p : vec2<f32>) -> f32 {
+ let i = floor(p);
+ let f = fract(p);
+ let u = f * f * (3.0 - 2.0 * f);
+ let a = hash2(i);
+ let b = hash2(i + vec2<f32>(1.0, 0.0));
+ let c = hash2(i + vec2<f32>(0.0, 1.0));
+ let d = hash2(i + vec2<f32>(1.0, 1.0));
+ return mix(mix(a, b, u.x), mix(c, d, u.x), u.y);
+}
+fn fbm(p : vec2<f32>) -> f32 {
+ return vnoise(p) * 0.6 + vnoise(p * 2.03 + 17.0) * 0.3 + vnoise(p * 4.01 + 41.0) * 0.1;
+}
+// Divergence-free flow: the curl of a scalar noise field.
+fn curl(p : vec2<f32>) -> vec2<f32> {
+ let e = 0.02;
+ let dx = fbm(p + vec2<f32>(e, 0.0)) - fbm(p - vec2<f32>(e, 0.0));
+ let dy = fbm(p + vec2<f32>(0.0, e)) - fbm(p - vec2<f32>(0.0, e));
+ return vec2<f32>(dy, -dx) / (2.0 * e);
+}
+
+fn maskAt(cell : vec2<f32>, offset : u32) -> f32 {
+ let w = i32(P.v[18].x);
+ let h = i32(P.v[18].y);
+ let c0 = vec2<i32>(floor(cell));
+ let t = cell - floor(cell);
+ var s = 0.0;
+ for (var k = 0; k < 4; k++) {
+ let o = vec2<i32>(k & 1, k >> 1);
+ let c = c0 + o;
+ var v = 0.0;
+ if (c.x >= 0 && c.y >= 0 && c.x < w && c.y < h) {
+ v = mask[offset + u32(c.y * w + c.x)];
+ }
+ let wx = select(1.0 - t.x, t.x, o.x == 1);
+ let wy = select(1.0 - t.y, t.y, o.y == 1);
+ s += v * wx * wy;
+ }
+ return s;
+}
+
+fn spawn(i : u32, idx : u32, birthFrac : f32) {
+ let birthAge = birthFrac * P.v[1].z;
+ let seed = u32(P.v[1].y);
+ let shape = u32(P.v[4].x);
+ let ex = P.v[4].y; let ey = P.v[4].z; let ew = P.v[4].w;
+ let eh = P.v[5].x; let er = P.v[5].y;
+ let r0 = rnd(idx, 0u, seed); let r1 = rnd(idx, 1u, seed);
+ var pos = vec2<f32>(ex, ey);
+ var center = vec2<f32>(ex, ey);
+ if (shape == 1u) { pos = vec2<f32>(ex + ew * r0, ey + eh * r0); center = vec2<f32>(ex + ew * 0.5, ey + eh * 0.5); }
+ else if (shape == 2u) { pos = vec2<f32>(ex + ew * r0, ey + eh * r1); center = vec2<f32>(ex + ew * 0.5, ey + eh * 0.5); }
+ else if (shape == 3u) {
+ let a = r0 * 6.2831853; let rr = er * sqrt(r1);
+ pos = vec2<f32>(ex + cos(a) * rr, ey + sin(a) * rr);
+ } else if (shape == 4u) {
+ let a = r0 * 6.2831853;
+ pos = vec2<f32>(ex + cos(a) * er, ey + sin(a) * er);
+ }
+ let radial = P.v[1].w > 0.5;
+ var dir = P.v[6].x + (rnd(idx, 2u, seed) - 0.5) * P.v[6].y;
+ if (radial) {
+ let d = pos - center;
+ if (dot(d, d) > 1e-6) { dir = atan2(d.y, d.x) + (rnd(idx, 2u, seed) - 0.5) * P.v[6].y; }
+ }
+ // A moving emitter: older births of the frame start nearer last frame's spot.
+ if (P.v[30].z > 0.5) { pos += (P.v[30].xy - vec2<f32>(ex, ey)) * birthFrac; }
+ let speed = mix(P.v[5].z, P.v[5].w, rnd(idx, 3u, seed));
+ let vel = vec2<f32>(cos(dir), sin(dir)) * speed;
+ var p : Particle;
+ p.vel = vel;
+ p.pos = pos + vel * birthAge;
+ p.age = birthAge;
+ p.life = mix(P.v[6].z, P.v[6].w, rnd(idx, 4u, seed));
+ p.jitter = 1.0 - P.v[16].w * rnd(idx, 5u, seed);
+ p._pad = 0.0;
+ particles[i] = p;
+}
+
+@compute @workgroup_size(${WORKGROUP})
+fn simulate(@builtin(global_invocation_id) gid : vec3<u32>) {
+ let i = gid.x;
+ let maxN = u32(P.v[1].x);
+ if (i >= maxN) { return; }
+ let dt = P.v[0].x;
+ let time = P.v[0].y;
+ let emitStart = u32(P.v[0].z);
+ let emitCount = u32(P.v[0].w);
+ let frameDt = P.v[1].z;
+
+ if (emitCount > 0u) {
+ let k = (i + maxN - (emitStart % maxN)) % maxN;
+ if (k < emitCount) {
+ // Births spread over the frame, oldest first.
+ spawn(i, emitStart + k, 1.0 - (f32(k) + 0.5) / f32(emitCount));
+ return;
+ }
+ }
+
+ var p = particles[i];
+ if (p.life <= 0.0) { return; }
+ p.age += dt;
+ if (p.age >= p.life) { p.life = 0.0; particles[i] = p; return; }
+
+ var acc = vec2<f32>(P.v[2].z, P.v[2].w);
+ let ns = P.v[3].y;
+ if (ns != 0.0) {
+ let sc = max(P.v[3].z, 1.0);
+ acc += curl(p.pos / sc + vec2<f32>(time * P.v[3].w, time * P.v[3].w * 0.7)) * ns;
+ }
+ let nAttr = u32(P.v[7].w);
+ for (var a = 0u; a < nAttr; a++) {
+ let at = P.v[8u + a];
+ let d = at.xy - p.pos;
+ let dist = length(d) + 1e-3;
+ let fall = exp(-(dist * dist) / max(at.w * at.w, 1.0));
+ acc += d / dist * at.z * fall;
+ }
+ if (P.v[16].x > 0.5) {
+ // Silhouette push: out along the gradient, harder where it just arrived.
+ let comp = p.pos + P.v[18].zw;
+ let cell = (comp - P.v[17].xy) / P.v[17].zw - 0.5;
+ let cellCount = u32(P.v[18].x * P.v[18].y);
+ let a0 = maskAt(cell, 0u);
+ if (a0 > 0.02) {
+ let gx = maskAt(cell + vec2<f32>(1.0, 0.0), 0u) - maskAt(cell - vec2<f32>(1.0, 0.0), 0u);
+ let gy = maskAt(cell + vec2<f32>(0.0, 1.0), 0u) - maskAt(cell - vec2<f32>(0.0, 1.0), 0u);
+ let g = vec2<f32>(gx, gy);
+ let gl = length(g);
+ if (gl > 1e-5) {
+ let before = maskAt(cell, cellCount);
+ let k = a0 * P.v[16].y + max(0.0, a0 - before) * P.v[16].z;
+ acc -= g / gl * k;
+ }
+ }
+ }
+
+ p.vel += acc * dt;
+ p.vel *= exp(-P.v[3].x * dt);
+ p.pos += p.vel * dt;
+
+ let w = P.v[2].x; let h = P.v[2].y;
+ let floorY = P.v[7].y;
+ let bounce = P.v[7].z;
+ if (floorY >= 0.0 && p.pos.y > floorY) { p.pos.y = floorY; p.vel.y = -abs(p.vel.y) * bounce; p.vel.x *= 0.9; }
+ let mode = u32(P.v[7].x);
+ if (mode == 1u) {
+ if (p.pos.x < 0.0 || p.pos.y < 0.0 || p.pos.x > w || p.pos.y > h) { p.life = 0.0; }
+ } else if (mode == 2u) {
+ if (p.pos.x < 0.0) { p.pos.x = 0.0; p.vel.x = abs(p.vel.x) * bounce; }
+ if (p.pos.x > w) { p.pos.x = w; p.vel.x = -abs(p.vel.x) * bounce; }
+ if (p.pos.y < 0.0) { p.pos.y = 0.0; p.vel.y = abs(p.vel.y) * bounce; }
+ if (p.pos.y > h) { p.pos.y = h; p.vel.y = -abs(p.vel.y) * bounce; }
+ } else if (mode == 3u) {
+ p.pos = vec2<f32>(p.pos.x - floor(p.pos.x / w) * w, p.pos.y - floor(p.pos.y / h) * h);
+ }
+ particles[i] = p;
+}
+`;
+
+const DRAW_WGSL = /* wgsl */ `
+${COMMON_WGSL}
+@group(0) @binding(0) var<uniform> P : Params;
+@group(0) @binding(1) var<storage, read> particles : array<Particle>;
+
+struct VOut {
+ @builtin(position) pos : vec4<f32>,
+ @location(0) uv : vec2<f32>,
+ @location(1) color : vec4<f32>,
+ @location(2) aspect : f32,
+};
+
+fn gradient(t : f32) -> vec4<f32> {
+ let n = i32(P.v[21].x);
+ if (n <= 1) { return P.v[22]; }
+ let x = clamp(t, 0.0, 1.0) * f32(n - 1);
+ let i = min(i32(floor(x)), n - 2);
+ return mix(P.v[22 + i], P.v[23 + i], x - f32(i));
+}
+
+@vertex
+fn vs(@builtin(vertex_index) vi : u32, @builtin(instance_index) ii : u32) -> VOut {
+ var out : VOut;
+ let p = particles[ii];
+ if (p.life <= 0.0) {
+ out.pos = vec4<f32>(2.0, 2.0, 0.0, 1.0);
+ out.color = vec4<f32>(0.0);
+ return out;
+ }
+ let t = p.age / p.life;
+ let size = mix(P.v[19].x, P.v[19].y, t) * p.jitter;
+ var corner = vec2<f32>(0.0);
+ switch (vi) {
+ case 0u: { corner = vec2<f32>(-1.0, -1.0); }
+ case 1u: { corner = vec2<f32>(1.0, -1.0); }
+ case 2u: { corner = vec2<f32>(-1.0, 1.0); }
+ case 3u: { corner = vec2<f32>(-1.0, 1.0); }
+ case 4u: { corner = vec2<f32>(1.0, -1.0); }
+ default: { corner = vec2<f32>(1.0, 1.0); }
+ }
+ let speed = length(p.vel);
+ var axisX = vec2<f32>(1.0, 0.0);
+ var halfLen = size * 0.5;
+ let shape = u32(P.v[20].x);
+ if (shape == 3u && speed > 1e-3) {
+ axisX = p.vel / speed;
+ halfLen = size * 0.5 + speed * P.v[20].y * 0.5;
+ }
+ let axisY = vec2<f32>(-axisX.y, axisX.x);
+ let local = axisX * corner.x * halfLen + axisY * corner.y * size * 0.5;
+ let px = (p.pos + local) * P.v[21].yz;
+ let tsize = P.v[2].xy * P.v[21].yz;
+ out.pos = vec4<f32>(px.x / tsize.x * 2.0 - 1.0, 1.0 - px.y / tsize.y * 2.0, 0.0, 1.0);
+ out.uv = corner;
+ out.aspect = halfLen / max(size * 0.5, 1e-3);
+
+ var c = gradient(t);
+ if (P.v[20].z > 0.5) { c = gradient(speed / max(P.v[20].w, 1e-3)); }
+ let fadeIn = max(P.v[19].z, 1e-4);
+ let fadeOut = max(P.v[19].w, 1e-4);
+ let a = c.a * smoothstep(0.0, fadeIn, t) * (1.0 - smoothstep(1.0 - fadeOut, 1.0, t)) * P.v[21].w;
+ out.color = vec4<f32>(c.rgb * a, a);
+ return out;
+}
+
+@fragment
+fn fs(in : VOut) -> @location(0) vec4<f32> {
+ let shape = u32(P.v[20].x);
+ var m = 1.0;
+ if (shape == 0u) {
+ let d = length(in.uv);
+ let e = max(fwidth(d), 1e-3);
+ m = 1.0 - smoothstep(1.0 - e, 1.0, d);
+ } else if (shape == 1u) {
+ let d = length(in.uv);
+ m = exp(-3.5 * d * d) * (1.0 - smoothstep(0.9, 1.0, d));
+ } else if (shape == 3u) {
+ // A capsule along the motion, soft at the sides.
+ let ax = abs(in.uv.x) * in.aspect;
+ let ex = max(ax - (in.aspect - 1.0), 0.0);
+ let d = length(vec2<f32>(ex, in.uv.y));
+ m = 1.0 - smoothstep(0.5, 1.0, d);
+ }
+ return in.color * m;
+}
+`;
+
+interface SystemState {
+ device: GPUDevice;
+ particles: GPUBuffer;
+ /** Frame the GPU buffer holds, relative to the layer's start; -1 = empty. */
+ frame: number;
+ /** Particles born so far (the next birth's index) and the fractional carry. */
+ emitted: number;
+ carry: number;
+ snapshots: Map<number, { buffer: GPUBuffer; emitted: number; carry: number }>;
+ maskBuffer: GPUBuffer | null;
+ maskSize: number;
+ ubo: GPUBuffer;
+}
+
+interface Pipelines {
+ simLayout: GPUBindGroupLayout;
+ sim: GPUComputePipeline;
+ drawLayout: GPUBindGroupLayout;
+ draw: Map<string, GPURenderPipeline>;
+ drawModule: GPUShaderModule;
+}
+
+const pipelines = new WeakMap<GPUDevice, Pipelines>();
+
+function getPipelines(device: GPUDevice): Pipelines {
+ let p = pipelines.get(device);
+ if (p) return p;
+ const simLayout = device.createBindGroupLayout({
+ label: "particles_sim_layout",
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.COMPUTE,
+ buffer: { type: "uniform" },
+ },
+ {
+ binding: 1,
+ visibility: GPUShaderStage.COMPUTE,
+ buffer: { type: "storage" },
+ },
+ {
+ binding: 2,
+ visibility: GPUShaderStage.COMPUTE,
+ buffer: { type: "read-only-storage" },
+ },
+ ],
+ });
+ const sim = device.createComputePipeline({
+ label: "particles_sim",
+ layout: device.createPipelineLayout({ bindGroupLayouts: [simLayout] }),
+ compute: {
+ module: device.createShaderModule({
+ label: "particles_sim.wgsl",
+ code: SIM_WGSL,
+ }),
+ entryPoint: "simulate",
+ },
+ });
+ const drawLayout = device.createBindGroupLayout({
+ label: "particles_draw_layout",
+ entries: [
+ {
+ binding: 0,
+ visibility: GPUShaderStage.VERTEX | GPUShaderStage.FRAGMENT,
+ buffer: { type: "uniform" },
+ },
+ {
+ binding: 1,
+ visibility: GPUShaderStage.VERTEX,
+ buffer: { type: "read-only-storage" },
+ },
+ ],
+ });
+ p = {
+ simLayout,
+ sim,
+ drawLayout,
+ draw: new Map(),
+ drawModule: device.createShaderModule({
+ label: "particles_draw.wgsl",
+ code: DRAW_WGSL,
+ }),
+ };
+ pipelines.set(device, p);
+ return p;
+}
+
+const BLENDS: Record<ParticleStatic["blend"], GPUBlendState> = {
+ normal: {
+ color: {
+ srcFactor: "one",
+ dstFactor: "one-minus-src-alpha",
+ operation: "add",
+ },
+ alpha: {
+ srcFactor: "one",
+ dstFactor: "one-minus-src-alpha",
+ operation: "add",
+ },
+ },
+ add: {
+ color: { srcFactor: "one", dstFactor: "one", operation: "add" },
+ alpha: { srcFactor: "one", dstFactor: "one", operation: "add" },
+ },
+ screen: {
+ color: { srcFactor: "one", dstFactor: "one-minus-src", operation: "add" },
+ alpha: {
+ srcFactor: "one",
+ dstFactor: "one-minus-src-alpha",
+ operation: "add",
+ },
+ },
+};
+
+function drawPipeline(
+ device: GPUDevice,
+ format: GPUTextureFormat,
+ blend: ParticleStatic["blend"],
+): GPURenderPipeline {
+ const p = getPipelines(device);
+ const key = `${format}|${blend}`;
+ let pipe = p.draw.get(key);
+ if (!pipe) {
+ pipe = device.createRenderPipeline({
+ label: `particles_draw_${blend}`,
+ layout: device.createPipelineLayout({ bindGroupLayouts: [p.drawLayout] }),
+ vertex: { module: p.drawModule, entryPoint: "vs" },
+ fragment: {
+ module: p.drawModule,
+ entryPoint: "fs",
+ targets: [{ format, blend: BLENDS[blend] }],
+ },
+ primitive: { topology: "triangle-list" },
+ });
+ p.draw.set(key, pipe);
+ }
+ return pipe;
+}
+
+const SHAPES = { circle: 0, soft: 1, square: 2, spark: 3 } as const;
+const EMITTERS = { point: 0, line: 1, rect: 2, circle: 3, ring: 4 } as const;
+const BOUNDS = { none: 0, kill: 1, bounce: 2, wrap: 3 } as const;
+
+/** Packs static settings and one frame's inputs into the shared parameter block. */
+function packParams(
+ s: ParticleStatic,
+ f: ParticleFrame,
+ out: Float32Array,
+ sim: {
+ dt: number;
+ time: number;
+ emitStart: number;
+ emitCount: number;
+ frameDt: number;
+ },
+ view: { scaleX: number; scaleY: number },
+): void {
+ out.fill(0);
+ const v = (i: number, a: number, b: number, c: number, d: number) =>
+ out.set([a, b, c, d], i * 4);
+ const deg = Math.PI / 180;
+ v(0, sim.dt, sim.time, sim.emitStart, sim.emitCount);
+ v(1, s.max, s.seed, sim.frameDt, s.emitter.radial ? 1 : 0);
+ v(2, s.layer.width, s.layer.height, f.gravity[0], f.gravity[1]);
+ v(3, f.drag, f.noise.strength, f.noise.scale, f.noise.speed);
+ const e = s.emitter;
+ const ex = f.emitter?.x ?? e.x;
+ const ey = f.emitter?.y ?? e.y;
+ v(4, EMITTERS[e.shape], ex, ey, e.width);
+ if (f.emitter) v(30, f.emitter.previousX, f.emitter.previousY, 1, 0);
+ v(5, e.height, e.radius, s.speed[0], s.speed[1]);
+ v(6, s.direction * deg, s.spread * deg, s.lifetime[0], s.lifetime[1]);
+ const attractors = f.attractors.slice(0, MAX_ATTRACTORS);
+ v(7, BOUNDS[s.bounds], s.floor ?? -1, s.bounce, attractors.length);
+ attractors.forEach((a, i) => v(8 + i, a.x, a.y, a.strength, a.radius));
+ const m = f.push?.mask;
+ v(16, m ? 1 : 0, f.push?.strength ?? 0, f.push?.sweep ?? 0, s.sizeJitter);
+ v(17, m?.originX ?? 0, m?.originY ?? 0, m?.scaleX ?? 1, m?.scaleY ?? 1);
+ v(18, m?.width ?? 0, m?.height ?? 0, s.layer.x, s.layer.y);
+ v(19, s.size[0], s.size[1], s.fade[0], s.fade[1]);
+ v(
+ 20,
+ SHAPES[s.shape],
+ s.stretch,
+ s.colorBySpeed !== null ? 1 : 0,
+ s.colorBySpeed ?? 1,
+ );
+ const colors = s.colors.slice(0, MAX_COLORS);
+ v(21, colors.length, view.scaleX, view.scaleY, s.opacity);
+ colors.forEach((c, i) => v(22 + i, c[0], c[1], c[2], c[3]));
+}
+
+function stateFor(program: ParticleProgram, device: GPUDevice): SystemState {
+ const existing = program.particleState as SystemState | undefined;
+ if (existing && existing.device === device) return existing;
+ const s = program.particleStatic;
+ const state: SystemState = {
+ device,
+ particles: device.createBuffer({
+ label: "particles",
+ size: s.max * PARTICLE_BYTES,
+ usage:
+ GPUBufferUsage.STORAGE |
+ GPUBufferUsage.COPY_SRC |
+ GPUBufferUsage.COPY_DST,
+ }),
+ frame: -1,
+ emitted: 0,
+ carry: 0,
+ snapshots: new Map(),
+ maskBuffer: null,
+ maskSize: 0,
+ ubo: device.createBuffer({
+ label: "particles_params",
+ size: PARAM_VEC4S * 16,
+ usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ }),
+ };
+ program.particleState = state;
+ return state;
+}
+
+/** Uploads this frame's (and the previous frame's) silhouette; a 1-float buffer when there is none. */
+function uploadMask(state: SystemState, f: ParticleFrame): GPUBuffer {
+ const m = f.push?.mask;
+ const cells = m ? m.width * m.height : 0;
+ const size = Math.max(16, cells * 2 * 4);
+ if (!state.maskBuffer || state.maskSize < size) {
+ state.maskBuffer?.destroy();
+ state.maskBuffer = state.device.createBuffer({
+ label: "particles_mask",
+ size,
+ usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
+ });
+ state.maskSize = size;
+ }
+ if (m) {
+ const both = new Float32Array(cells * 2);
+ both.set(m.data.subarray(0, cells));
+ both.set((f.push?.previous ?? m).data.subarray(0, cells), cells);
+ state.device.queue.writeBuffer(state.maskBuffer, 0, both);
+ }
+ return state.maskBuffer;
+}
+
+/** Steps the system from the frame it holds to `frame` (layer-local), restoring a snapshot to go back. */
+function advance(
+ program: ParticleProgram,
+ state: SystemState,
+ frame: number,
+ compositionStart: number,
+ fps: number,
+): void {
+ const { device } = state;
+ const s = program.particleStatic;
+ if (frame < state.frame || state.frame < 0) {
+ let best = -1;
+ for (const k of state.snapshots.keys())
+ if (k <= frame && k > best) best = k;
+ const encoder = device.createCommandEncoder({ label: "particles_restore" });
+ if (best >= 0) {
+ const snap = state.snapshots.get(best) as {
+ buffer: GPUBuffer;
+ emitted: number;
+ carry: number;
+ };
+ encoder.copyBufferToBuffer(
+ snap.buffer,
+ 0,
+ state.particles,
+ 0,
+ s.max * PARTICLE_BYTES,
+ );
+ state.emitted = snap.emitted;
+ state.carry = snap.carry;
+ state.frame = best;
+ } else {
+ encoder.clearBuffer(state.particles);
+ state.emitted = 0;
+ state.carry = 0;
+ state.frame = -1;
+ }
+ device.queue.submit([encoder.finish()]);
+ }
+
+ const p = getPipelines(device);
+ const params = new Float32Array(PARAM_VEC4S * 4);
+ const substeps = Math.max(1, Math.round(s.substeps));
+ const frameDt = 1 / fps;
+ const prewarmFrames = Math.round(s.prewarm * fps);
+
+ // state.frame = -1 means "before the first frame": prewarm first, then frame 0.
+ for (let f = state.frame + 1; f <= frame; f++) {
+ const steps = f === 0 ? prewarmFrames + 1 : 1;
+ for (let w = 0; w < steps; w++) {
+ const simFrame = f === 0 ? w - prewarmFrames : f;
+ const inputs = program.particleFrame(
+ compositionStart + Math.max(0, simFrame),
+ fps,
+ );
+ state.carry += Math.max(0, inputs.rate) * frameDt;
+ let born = Math.floor(state.carry);
+ state.carry -= born;
+ born += Math.max(0, Math.round(inputs.burst));
+ born = Math.min(born, s.max);
+ const mask = uploadMask(state, inputs);
+ const bindGroup = device.createBindGroup({
+ layout: p.simLayout,
+ entries: [
+ { binding: 0, resource: { buffer: state.ubo } },
+ { binding: 1, resource: { buffer: state.particles } },
+ { binding: 2, resource: { buffer: mask } },
+ ],
+ });
+ // One submit per substep: queue writes are ordered with submits, so
+ // each substep reads its own parameters.
+ for (let k = 0; k < substeps; k++) {
+ packParams(
+ s,
+ inputs,
+ params,
+ {
+ dt: frameDt / substeps,
+ time: (simFrame + k / substeps) * frameDt,
+ emitStart: state.emitted,
+ emitCount: k === 0 ? born : 0,
+ frameDt,
+ },
+ { scaleX: 1, scaleY: 1 },
+ );
+ device.queue.writeBuffer(state.ubo, 0, params);
+ const encoder = device.createCommandEncoder({
+ label: "particles_step",
+ });
+ const pass = encoder.beginComputePass({ label: "particles_sim" });
+ pass.setPipeline(p.sim);
+ pass.setBindGroup(0, bindGroup);
+ pass.dispatchWorkgroups(Math.ceil(s.max / WORKGROUP));
+ pass.end();
+ device.queue.submit([encoder.finish()]);
+ }
+ state.emitted += born;
+ }
+ state.frame = f;
+ if (
+ f % SNAPSHOT_EVERY === 0 &&
+ !state.snapshots.has(f) &&
+ state.snapshots.size < MAX_SNAPSHOTS
+ ) {
+ const buffer = device.createBuffer({
+ label: `particles_snapshot_${f}`,
+ size: s.max * PARTICLE_BYTES,
+ usage: GPUBufferUsage.COPY_SRC | GPUBufferUsage.COPY_DST,
+ });
+ const encoder = device.createCommandEncoder({
+ label: "particles_snapshot",
+ });
+ encoder.copyBufferToBuffer(
+ state.particles,
+ 0,
+ buffer,
+ 0,
+ s.max * PARTICLE_BYTES,
+ );
+ device.queue.submit([encoder.finish()]);
+ state.snapshots.set(f, {
+ buffer,
+ emitted: state.emitted,
+ carry: state.carry,
+ });
+ }
+ }
+}
+
+export const ParticlesWebGPURenderer: WebGPUNodeRenderer = async (args) => {
+ const { ctx, encoder, pass, targetView, targetWidth, targetHeight, props } =
+ args;
+ const op = props.virtualMedia?.operation as
+ | { op?: string; effect?: unknown }
+ | undefined;
+ if (op?.op !== "Particles" || !isParticleProgram(op.effect)) return;
+ const program = op.effect;
+ const s = program.particleStatic;
+
+ const fps = props.fps || 30;
+ const local = Math.max(0, Math.round(props.frame ?? 0));
+ const compositionFrame = Math.round(props.compositionFrame ?? local);
+ const compositionStart = compositionFrame - local;
+
+ // Simulation runs in its own submissions, ahead of this frame's draw.
+ const state = stateFor(program, ctx.device);
+ advance(program, state, local, compositionStart, fps);
+
+ pass.end();
+ const params = new Float32Array(PARAM_VEC4S * 4);
+ packParams(
+ s,
+ program.particleFrame(compositionFrame, fps),
+ params,
+ { dt: 0, time: 0, emitStart: 0, emitCount: 0, frameDt: 1 / fps },
+ {
+ scaleX: targetWidth / s.layer.width,
+ scaleY: targetHeight / s.layer.height,
+ },
+ );
+ const ubo = ctx.renderer.getTemporaryBuffer(params);
+ const draw = ctx.renderer.beginFrame(
+ encoder,
+ targetView,
+ { r: 0, g: 0, b: 0, a: 0 },
+ targetWidth,
+ targetHeight,
+ "load",
+ );
+ draw.setPipeline(drawPipeline(ctx.device, ctx.renderer.format, s.blend));
+ draw.setBindGroup(
+ 0,
+ ctx.device.createBindGroup({
+ layout: getPipelines(ctx.device).drawLayout,
+ entries: [
+ { binding: 0, resource: { buffer: ubo } },
+ { binding: 1, resource: { buffer: state.particles } },
+ ],
+ }),
+ );
+ draw.draw(6, s.max);
+ draw.end();
+};
+
+export type { ParticleMask };
diff --git a/nodes/node-particles/src/shared/index.ts b/nodes/node-particles/src/shared/index.ts
new file mode 100644
index 00000000..939ffda6
--- /dev/null
+++ b/nodes/node-particles/src/shared/index.ts
@@ -0,0 +1 @@
+export * from "./program.js";
diff --git a/nodes/node-particles/src/shared/program.ts b/nodes/node-particles/src/shared/program.ts
new file mode 100644
index 00000000..d62cbd11
--- /dev/null
+++ b/nodes/node-particles/src/shared/program.ts
@@ -0,0 +1,118 @@
+/**
+ * The contract between a particle system's authoring object (framefields'
+ * `Particles` effect, carried on the op as `effect`) and this renderer.
+ * Static settings are read once per system; per-frame inputs (driven values,
+ * a performer's silhouette) are asked for each simulated frame.
+ */
+
+export type EmitterShape = "point" | "line" | "rect" | "circle" | "ring";
+export type ParticleShape = "circle" | "soft" | "square" | "spark";
+export type ParticleBlend = "normal" | "add" | "screen";
+export type ParticleBounds = "none" | "kill" | "bounce" | "wrap";
+
+export interface ParticleStatic {
+ /** Capacity: the most particles alive at once. */
+ max: number;
+ seed: number;
+ /** Layer box in composition px (sampling silhouettes needs the origin). */
+ layer: { x: number; y: number; width: number; height: number };
+ emitter: {
+ shape: EmitterShape;
+ x: number;
+ y: number;
+ width: number;
+ height: number;
+ radius: number;
+ /** Aim away from the emitter's center instead of along `direction`. */
+ radial: boolean;
+ };
+ /** Seconds. */
+ lifetime: [number, number];
+ /** px/s. */
+ speed: [number, number];
+ /** Degrees, 0 = right, -90 = up. */
+ direction: number;
+ /** Degrees around `direction`. */
+ spread: number;
+ bounds: ParticleBounds;
+ /** Floor line in layer px, or null. */
+ floor: number | null;
+ bounce: number;
+ /** px at birth and death. */
+ size: [number, number];
+ /** 0–1 random scale per particle. */
+ sizeJitter: number;
+ /** Fractions of life spent fading in and out. */
+ fade: [number, number];
+ shape: ParticleShape;
+ /** Spark length per px/s of speed. */
+ stretch: number;
+ blend: ParticleBlend;
+ /** Up to 8 colors, evenly spaced over life (or speed). */
+ colors: Array<[number, number, number, number]>;
+ /** Color by speed instead of age: speed (px/s) that reaches the last color. */
+ colorBySpeed: number | null;
+ opacity: number;
+ substeps: number;
+ /** Seconds simulated before the first frame, so it starts full. */
+ prewarm: number;
+}
+
+export interface ParticleAttractor {
+ x: number;
+ y: number;
+ /** px/s² at the center; negative repels. */
+ strength: number;
+ /** Falloff radius, px. */
+ radius: number;
+}
+
+/** A silhouette grid (0–1 per cell) and how composition px map onto it. */
+export interface ParticleMask {
+ data: Float32Array;
+ width: number;
+ height: number;
+ /** cell = (px − origin) / scale − 0.5 */
+ originX: number;
+ originY: number;
+ scaleX: number;
+ scaleY: number;
+}
+
+export interface ParticleFrame {
+ /** Emitter position this frame, when it moves (layer px); births spread back toward `previous`. */
+ emitter?: { x: number; y: number; previousX: number; previousY: number };
+ /** Particles per second this frame. */
+ rate: number;
+ /** Extra particles born this frame. */
+ burst: number;
+ gravity: [number, number];
+ /** Velocity damping per second. */
+ drag: number;
+ noise: { strength: number; scale: number; speed: number };
+ attractors: ParticleAttractor[];
+ push: {
+ strength: number;
+ /** Extra push where the silhouette just arrived. */
+ sweep: number;
+ mask: ParticleMask | null;
+ previous: ParticleMask | null;
+ } | null;
+}
+
+export interface ParticleProgram {
+ readonly particleStatic: ParticleStatic;
+ /** Inputs for composition frame `frame` (layer-local frames are the effect's concern). */
+ particleFrame(frame: number, fps: number): ParticleFrame;
+ /** Renderer-owned state, kept on the authoring object so it lives as long as the system. */
+ particleState?: unknown;
+}
+
+export function isParticleProgram(value: unknown): value is ParticleProgram {
+ return (
+ typeof value === "object" &&
+ value !== null &&
+ "particleStatic" in value &&
+ typeof (value as ParticleProgram).particleFrame === "function"
+ );
+}
diff --git a/nodes/node-particles/tsconfig.json b/nodes/node-particles/tsconfig.json
new file mode 100644
index 00000000..5a0b963f
--- /dev/null
+++ b/nodes/node-particles/tsconfig.json
@@ -0,0 +1,12 @@
+{
+ "extends": "@framefields/tsconfig/node-library.json",
+ "compilerOptions": {
+ "baseUrl": ".",
+ "paths": {
+ "@/*": ["./src/*"]
+ },
+ "experimentalDecorators": true,
+ "emitDecoratorMetadata": true
+ },
+ "include": ["src"]
+}
diff --git a/nodes/node-particles/tsdown.config.ts b/nodes/node-particles/tsdown.config.ts
new file mode 100644
index 00000000..73f5c2fc
--- /dev/null
+++ b/nodes/node-particles/tsdown.config.ts
@@ -0,0 +1,13 @@
+import { defineConfig } from "tsdown";
+
+export default defineConfig({
+ entry: {
+ index: "src/index.ts",
+ renderer: "src/renderers/index.ts",
+ },
+ format: ["esm"],
+ dts: true,
+ clean: true,
+ sourcemap: true,
+ treeshake: true,
+});
diff --git a/nodes/node-trails/package.json b/nodes/node-trails/package.json
new file mode 100644
index 00000000..3400fbc3
--- /dev/null
+++ b/nodes/node-trails/package.json
@@ -0,0 +1,46 @@
+{
+ "name": "@framefields/node-trails",
+ "version": "2.2.0",
+ "private": false,
+ "framefields": {
+ "type": "Trails",
+ "kind": "effect",
+ "schema": "TrailsNodeConfigSchema"
+ },
+ "type": "module",
+ "license": "Apache-2.0",
+ "exports": {
+ ".": {
+ "types": "./dist/index.d.mts",
+ "development": "./src/index.ts",
+ "import": "./dist/index.mjs",
+ "default": "./dist/index.mjs"
+ },
+ "./renderer": {
+ "types": "./dist/renderer.d.mts",
+ "development": "./src/renderers/index.ts",
+ "import": "./dist/renderer.mjs",
+ "default": "./dist/renderer.mjs"
+ }
+ },
+ "scripts": {
+ "build": "tsdown",
+ "dev": "tsdown --watch",
+ "clean": "rm -rf dist .turbo",
+ "typecheck": "tsc --noEmit"
+ },
+ "dependencies": {
+ "@framefields/node-sdk": "workspace:*",
+ "zod": "^4.3.6"
+ },
+ "devDependencies": {
+ "@framefields/tsconfig": "workspace:*",
+ "@types/node": "^24.1.0",
+ "tsdown": "^0.19.0",
+ "typescript": "^5.8.3",
+ "webgpu": "^0.4.0"
+ },
+ "publishConfig": {
+ "access": "public"
+ }
+}
diff --git a/nodes/node-trails/src/index.ts b/nodes/node-trails/src/index.ts
new file mode 100644
index 00000000..944d702e
--- /dev/null
+++ b/nodes/node-trails/src/index.ts
@@ -0,0 +1,2 @@
+export * from "./renderers/index.js";
+export * from "./shared/index.js";
diff --git a/nodes/node-trails/src/renderers/index.ts b/nodes/node-trails/src/renderers/index.ts
new file mode 100644
index 00000000..ad407f02
--- /dev/null
+++ b/nodes/node-trails/src/renderers/index.ts
@@ -0,0 +1,7 @@
+import { defineRenderer } from "@framefields/node-sdk/renderer";
+import { TrailsWebGPURenderer } from "./webgpu-renderer.js";
+
+export default defineRenderer({
+ WebGPURenderer: TrailsWebGPURenderer,
+});
+export { TrailsWebGPURenderer };
diff --git a/nodes/node-trails/src/renderers/webgpu-renderer.ts b/nodes/node-trails/src/renderers/webgpu-renderer.ts
new file mode 100644
index 00000000..6b888265
--- /dev/null
+++ b/nodes/node-trails/src/renderers/webgpu-renderer.ts
@@ -0,0 +1,272 @@
+/// <reference types="webgpu" />
+/**
+ * Trails: the layer over a decaying, optionally zooming / turning / drifting
+ * / hue-shifting copy of its own history (echoes, light painting, feedback
+ * tunnels). History lives in textures on the effect object and advances one
+ * frame at a time; after a jump the last frames are replayed to rebuild it.
+ */
+import type { WebGPUNodeRenderer } from "@framefields/node-sdk";
+import {
+ drawChildToTexture,
+ drawFullscreenPass,
+ drawTextureToTarget,
+ FULLSCREEN_QUAD_WGSL,
+ fragmentBindGroupLayout,
+ perDevice,
+ type WebGPUNodeRendererArgs,
+} from "@framefields/node-sdk/gpu";
+import type { TrailsMode } from "../shared/index.js";
+
+const HISTORY_FORMAT: GPUTextureFormat = "rgba16float";
+
+interface TrailsValues {
+ decay: number;
+ mode: TrailsMode;
+ zoom: number;
+ rotation: number;
+ offsetX: number;
+ offsetY: number;
+ hueShift: number;
+ warmup: number;
+}
+
+const TRAILS_WGSL = /* wgsl */ `
+struct U { a : vec4<f32>, b : vec4<f32> };
+@group(0) @binding(0) var<uniform> u : U;
+@group(0) @binding(1) var historyTex : texture_2d<f32>;
+@group(0) @binding(2) var currentTex : texture_2d<f32>;
+@group(0) @binding(3) var samp : sampler;
+${FULLSCREEN_QUAD_WGSL}
+
+// Hue rotation in YIQ: linear, so it works on premultiplied color.
+fn hueRotate(c : vec3<f32>, angle : f32) -> vec3<f32> {
+ let yiq = mat3x3<f32>(0.299, 0.596, 0.211, 0.587, -0.274, -0.523, 0.114, -0.322, 0.312) * c;
+ let cs = cos(angle);
+ let sn = sin(angle);
+ let rot = vec3<f32>(yiq.x, yiq.y * cs - yiq.z * sn, yiq.y * sn + yiq.z * cs);
+ return mat3x3<f32>(1.0, 1.0, 1.0, 0.956, -0.272, -1.106, 0.621, -0.647, 1.703) * rot;
+}
+
+@fragment fn fs(in : VSOut) -> @location(0) vec4<f32> {
+ let decay = u.a.x;
+ let mode = u.a.y;
+ let zoom = max(u.a.z, 1e-3);
+ let angle = u.a.w;
+ let offset = u.b.xy;
+ let hue = u.b.z;
+ let fresh = u.b.w;
+ let cur = textureSampleLevel(currentTex, samp, in.uv, 0.0);
+ if (fresh > 0.5) { return cur; }
+ let c = in.uv - 0.5;
+ let cs = cos(-angle);
+ let sn = sin(-angle);
+ let src = vec2<f32>(c.x * cs - c.y * sn, c.x * sn + c.y * cs) / zoom + 0.5 - offset;
+ var hist = vec4<f32>(0.0);
+ if (all(src >= vec2<f32>(0.0)) && all(src <= vec2<f32>(1.0))) {
+ hist = textureSampleLevel(historyTex, samp, src, 0.0);
+ }
+ if (hue != 0.0) { hist = vec4<f32>(max(hueRotate(hist.rgb, hue), vec3<f32>(0.0)), hist.a); }
+ hist *= decay;
+ if (mode < 0.5) { return cur + hist * (1.0 - cur.a); }
+ if (mode < 1.5) { return vec4<f32>(cur.rgb + hist.rgb, min(cur.a + hist.a, 1.0)); }
+ if (mode < 2.5) { return max(cur, hist); }
+ // blend: a running average, so opaque footage shows its own echoes.
+ return mix(cur, hist / max(decay, 1e-4), decay);
+}
+`;
+
+const resources = perDevice((device: GPUDevice) => {
+ const layout = fragmentBindGroupLayout(
+ device,
+ ["uniform", "texture", "texture", "sampler"],
+ "trails_layout",
+ );
+ const module = device.createShaderModule({
+ code: TRAILS_WGSL,
+ label: "trails.wgsl",
+ });
+ return {
+ layout,
+ pipeline: device.createRenderPipeline({
+ label: "trails",
+ layout: device.createPipelineLayout({ bindGroupLayouts: [layout] }),
+ vertex: { module, entryPoint: "vs" },
+ fragment: {
+ module,
+ entryPoint: "fs",
+ targets: [{ format: HISTORY_FORMAT }],
+ },
+ primitive: { topology: "triangle-strip" },
+ }),
+ sampler: device.createSampler({ magFilter: "linear", minFilter: "linear" }),
+ };
+});
+
+interface History {
+ device: GPUDevice;
+ width: number;
+ height: number;
+ /** Ping-pong history; `front` holds the latest. */
+ front: GPUTexture;
+ back: GPUTexture;
+ /** Layer frame `front` holds; -1 = none. */
+ frame: number;
+}
+
+interface FeedbackEffect {
+ resolveUniforms(ctx?: unknown): Record<string, unknown>;
+ feedbackHistory?: History;
+}
+
+function historyFor(
+ effect: FeedbackEffect,
+ device: GPUDevice,
+ width: number,
+ height: number,
+): History {
+ const h = effect.feedbackHistory;
+ if (h && h.device === device && h.width === width && h.height === height)
+ return h;
+ h?.front.destroy();
+ h?.back.destroy();
+ const make = (label: string) =>
+ device.createTexture({
+ label,
+ size: [width, height],
+ format: HISTORY_FORMAT,
+ usage:
+ GPUTextureUsage.RENDER_ATTACHMENT |
+ GPUTextureUsage.TEXTURE_BINDING |
+ GPUTextureUsage.COPY_SRC |
+ GPUTextureUsage.COPY_DST,
+ });
+ const next: History = {
+ device,
+ width,
+ height,
+ front: make("trails_front"),
+ back: make("trails_back"),
+ frame: -1,
+ };
+ effect.feedbackHistory = next;
+ return next;
+}
+
+async function childAt(
+ args: WebGPUNodeRendererArgs,
+ frame: number,
+ compositionStart: number,
+ h: History,
+) {
+ const child = args.props.virtualMedia?.children?.[0];
+ if (!child) return null;
+ return drawChildToTexture(args, child, {
+ exclude: [h.front, h.back],
+ props: {
+ ...args.props,
+ virtualMedia: child,
+ frame,
+ compositionFrame: compositionStart + frame,
+ },
+ });
+}
+
+function swap(h: History): void {
+ const t = h.front;
+ h.front = h.back;
+ h.back = t;
+}
+
+async function stepTrails(
+ args: WebGPUNodeRendererArgs,
+ h: History,
+ v: TrailsValues,
+ frame: number,
+ compositionStart: number,
+ fresh: boolean,
+): Promise<void> {
+ const cur = await childAt(args, frame, compositionStart, h);
+ if (!cur) return;
+ const r = resources(args.ctx.device);
+ const uniforms = new Float32Array([
+ v.decay,
+ { over: 0, add: 1, max: 2, blend: 3 }[v.mode] ?? 0,
+ v.zoom,
+ (v.rotation * Math.PI) / 180,
+ v.offsetX / h.width,
+ v.offsetY / h.height,
+ (v.hueShift * Math.PI) / 180,
+ fresh ? 1 : 0,
+ ]);
+ drawFullscreenPass(args.encoder, h.back.createView(), r.pipeline, [
+ args.ctx.device.createBindGroup({
+ layout: r.layout,
+ entries: [
+ {
+ binding: 0,
+ resource: { buffer: args.ctx.renderer.getTemporaryBuffer(uniforms) },
+ },
+ { binding: 1, resource: h.front.createView() },
+ { binding: 2, resource: cur.view },
+ { binding: 3, resource: r.sampler },
+ ],
+ }),
+ ]);
+ swap(h);
+ h.frame = frame;
+}
+
+function num(v: unknown, fallback: number): number {
+ return typeof v === "number" && Number.isFinite(v) ? v : fallback;
+}
+
+export const TrailsWebGPURenderer: WebGPUNodeRenderer = async (args) => {
+ const { ctx, pass, targetWidth, targetHeight, props } = args;
+ const op = props.virtualMedia?.operation as
+ | { op?: string; effect?: FeedbackEffect }
+ | undefined;
+ if (op?.op !== "Trails" || !op.effect?.resolveUniforms) return;
+ if (!props.virtualMedia?.children?.[0]) return;
+ pass.end();
+
+ const fps = props.fps || 30;
+ const local = Math.max(0, Math.round(props.frame ?? 0));
+ const compositionFrame = Math.round(props.compositionFrame ?? local);
+ const compositionStart = compositionFrame - local;
+ const raw = op.effect.resolveUniforms({
+ frame: compositionFrame,
+ fps,
+ time: compositionFrame / fps,
+ });
+ const h = historyFor(op.effect, ctx.device, targetWidth, targetHeight);
+
+ {
+ const v: TrailsValues = {
+ decay: Math.min(1, Math.max(0, num(raw.decay, 0.9))),
+ mode: (raw.mode as TrailsValues["mode"]) ?? "blend",
+ zoom: num(raw.zoom, 1),
+ rotation: num(raw.rotation, 0),
+ offsetX: num(raw.offsetX, 0),
+ offsetY: num(raw.offsetY, 0),
+ hueShift: num(raw.hueShift, 0),
+ warmup: Math.max(0, Math.round(num(raw.warmup, 60))),
+ };
+ if (h.frame !== local) {
+ let start = local;
+ if (h.frame === local - 1) start = local;
+ else {
+ // Rebuild: replay until older frames have faded below 0.5%.
+ const needed =
+ v.decay > 0 && v.decay < 1
+ ? Math.ceil(Math.log(0.005) / Math.log(v.decay))
+ : v.warmup;
+ start = Math.max(0, local - Math.min(v.warmup, needed));
+ h.frame = -1;
+ }
+ for (let f = start; f <= local; f++) {
+ await stepTrails(args, h, v, f, compositionStart, h.frame < 0);
+ }
+ }
+ }
+ drawTextureToTarget(args, h.front);
+};
diff --git a/nodes/node-trails/src/shared/config.ts b/nodes/node-trails/src/shared/config.ts
new file mode 100644
index 00000000..3ef5f334
--- /dev/null
+++ b/nodes/node-trails/src/shared/config.ts
@@ -0,0 +1,61 @@
+import { configBuilder } from "@framefields/node-sdk";
+import { z } from "zod";
+
+export const TRAILS_MODES = ["blend", "over", "add", "max"] as const;
+
+export const trailsConfig = configBuilder()
+ .field("decay", z.number().min(0).max(1).default(0.9), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Decay",
+ description:
+ "History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails.",
+ })
+ .field("mode", z.enum(TRAILS_MODES).default("blend"), {
+ bindable: false,
+ label: "Mode",
+ description:
+ "blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest.",
+ })
+ .field("zoom", z.number().min(0.5).max(2).default(1), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Zoom",
+ description: "History scale per frame; 1.02 pulls the past into a tunnel.",
+ })
+ .field("rotation", z.number().min(-45).max(45).default(0), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Rotation",
+ description: "History rotation per frame, degrees.",
+ })
+ .field("offsetX", z.number().min(-500).max(500).default(0), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Offset X",
+ description: "History drift per frame, px.",
+ })
+ .field("offsetY", z.number().min(-500).max(500).default(0), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Offset Y",
+ description: "History drift per frame, px.",
+ })
+ .field("hueShift", z.number().min(-180).max(180).default(0), {
+ bindable: true,
+ dataTypes: ["Number", "Signal"],
+ label: "Hue Shift",
+ description:
+ "History hue rotation per frame, degrees (psychedelic cycling).",
+ })
+ .field("warmup", z.number().int().min(0).max(240).default(60), {
+ bindable: false,
+ label: "Warmup",
+ description:
+ "Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render).",
+ })
+ .build();
+
+export const TrailsNodeConfigSchema = trailsConfig.schema;
+export type TrailsNodeConfig = z.infer<typeof TrailsNodeConfigSchema>;
+export type TrailsMode = (typeof TRAILS_MODES)[number];
diff --git a/nodes/node-trails/src/shared/index.ts b/nodes/node-trails/src/shared/index.ts
new file mode 100644
index 00000000..d1bc97e8
--- /dev/null
+++ b/nodes/node-trails/src/shared/index.ts
@@ -0,0 +1 @@
+export * from "./config.js";
diff --git a/nodes/node-trails/tsconfig.json b/nodes/node-trails/tsconfig.json
new file mode 100644
index 00000000..5a0b963f
--- /dev/null
+++ b/nodes/node-trails/tsconfig.json
@@ -0,0 +1,12 @@
+{
+ "extends": "@framefields/tsconfig/node-library.json",
+ "compilerOptions": {
+ "baseUrl": ".",
+ "paths": {
+ "@/*": ["./src/*"]
+ },
+ "experimentalDecorators": true,
+ "emitDecoratorMetadata": true
+ },
+ "include": ["src"]
+}
diff --git a/nodes/node-trails/tsdown.config.ts b/nodes/node-trails/tsdown.config.ts
new file mode 100644
index 00000000..73f5c2fc
--- /dev/null
+++ b/nodes/node-trails/tsdown.config.ts
@@ -0,0 +1,13 @@
+import { defineConfig } from "tsdown";
+
+export default defineConfig({
+ entry: {
+ index: "src/index.ts",
+ renderer: "src/renderers/index.ts",
+ },
+ format: ["esm"],
+ dts: true,
+ clean: true,
+ sourcemap: true,
+ treeshake: true,
+});
diff --git a/packages/framefields/package.json b/packages/framefields/package.json
index 64b74ecb..9f8466dc 100644
--- a/packages/framefields/package.json
+++ b/packages/framefields/package.json
@@ -146,6 +146,7 @@
"@framefields/node-mesh-warp": "workspace:*",
"@framefields/node-modulate": "workspace:*",
"@framefields/node-paint": "workspace:*",
+ "@framefields/node-particles": "workspace:*",
"@framefields/node-patch-heal": "workspace:*",
"@framefields/node-refine-edge": "workspace:*",
"@framefields/node-refraction-caustics": "workspace:*",
diff --git a/packages/framefields/src/clip-vision.ts b/packages/framefields/src/clip-vision.ts
index a0df0a9d..dd07098d 100644
--- a/packages/framefields/src/clip-vision.ts
+++ b/packages/framefields/src/clip-vision.ts
@@ -457,6 +457,35 @@ export class PersonTrack {
return { x: read("x"), y: read("y"), visibility: read("visibility") };
}
+ /**
+ * @internal The softened silhouette grid at a composition frame and how
+ * placed px map onto it (`cell = (px − origin) / scale − 0.5`), for GPU
+ * consumers such as particles. Null when no person was found.
+ */
+ silhouetteGrid(frame: number): {
+ data: Float32Array;
+ width: number;
+ height: number;
+ originX: number;
+ originY: number;
+ scaleX: number;
+ scaleY: number;
+ } | null {
+ const grid = this.clip.blurredMask(Math.round(this.clip.clipFrame(frame)));
+ if (!grid) return null;
+ const { cell, width, height } = this.clip.grid();
+ const t = this.transform();
+ return {
+ data: grid,
+ width,
+ height,
+ originX: t.ox,
+ originY: t.oy,
+ scaleX: cell * t.sx,
+ scaleY: cell * t.sy,
+ };
+ }
+
/**
* The silhouette near a point at a composition frame: `alpha` 0–1 (softened
* over a few cells; inside > 0.5) and its gradient `gx`, `gy` per px,
diff --git a/packages/framefields/src/effects/generated/catalog.json b/packages/framefields/src/effects/generated/catalog.json
index 42968b1d..71cbad13 100644
--- a/packages/framefields/src/effects/generated/catalog.json
+++ b/packages/framefields/src/effects/generated/catalog.json
@@ -712,6 +712,51 @@
}
]
},
+ {
+ "name": "Datamosh",
+ "op": "Datamosh",
+ "kind": "effect",
+ "factory": "datamosh",
+ "node": "node-datamosh",
+ "aliases": [],
+ "custom": null,
+ "props": [
+ {
+ "name": "keyframes",
+ "type": "number[]",
+ "signal": false,
+ "default": [],
+ "description": "Layer frames that show the clean picture again (I-frames). The first frame always does."
+ },
+ {
+ "name": "strength",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 1,
+ "min": 0,
+ "max": 10,
+ "description": "How far pixels are dragged along the motion."
+ },
+ {
+ "name": "bleed",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 0,
+ "min": 0,
+ "max": 1,
+ "description": "Clean picture mixed back in per frame, 0–1."
+ },
+ {
+ "name": "warmup",
+ "type": "number",
+ "signal": false,
+ "default": 48,
+ "min": 1,
+ "max": 240,
+ "description": "Most frames replayed to rebuild the smear after a jump."
+ }
+ ]
+ },
{
"name": "DisplacementMap",
"op": "DisplacementMap",
@@ -2097,6 +2142,88 @@
}
]
},
+ {
+ "name": "Trails",
+ "op": "Trails",
+ "kind": "effect",
+ "factory": "trails",
+ "node": "node-trails",
+ "aliases": [],
+ "custom": null,
+ "props": [
+ {
+ "name": "decay",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 0.9,
+ "min": 0,
+ "max": 1,
+ "description": "History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails."
+ },
+ {
+ "name": "mode",
+ "type": "\"blend\" | \"over\" | \"add\" | \"max\"",
+ "signal": false,
+ "default": "blend",
+ "enum": ["blend", "over", "add", "max"],
+ "description": "blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest."
+ },
+ {
+ "name": "zoom",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 1,
+ "min": 0.5,
+ "max": 2,
+ "description": "History scale per frame; 1.02 pulls the past into a tunnel."
+ },
+ {
+ "name": "rotation",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 0,
+ "min": -45,
+ "max": 45,
+ "description": "History rotation per frame, degrees."
+ },
+ {
+ "name": "offsetX",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 0,
+ "min": -500,
+ "max": 500,
+ "description": "History drift per frame, px."
+ },
+ {
+ "name": "offsetY",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 0,
+ "min": -500,
+ "max": 500,
+ "description": "History drift per frame, px."
+ },
+ {
+ "name": "hueShift",
+ "type": "EffectProp<number>",
+ "signal": true,
+ "default": 0,
+ "min": -180,
+ "max": 180,
+ "description": "History hue rotation per frame, degrees (psychedelic cycling)."
+ },
+ {
+ "name": "warmup",
+ "type": "number",
+ "signal": false,
+ "default": 60,
+ "min": 0,
+ "max": 240,
+ "description": "Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render)."
+ }
+ ]
+ },
{
"name": "Vignette",
"op": "Vignette",
diff --git a/packages/framefields/src/effects/generated/classes.ts b/packages/framefields/src/effects/generated/classes.ts
index 150c3545..2043c83f 100644
--- a/packages/framefields/src/effects/generated/classes.ts
+++ b/packages/framefields/src/effects/generated/classes.ts
@@ -1217,6 +1217,54 @@ export class CornerPin extends Effect<CornerPinProps> {
}
}
+export interface DatamoshProps {
+ /**
+ * Layer frames that show the clean picture again (I-frames). The first frame always does.
+ * @default []
+ */
+ keyframes?: number[];
+ /**
+ * How far pixels are dragged along the motion.
+ * Range 0 to 10. Accepts a signal.
+ * @default 1
+ */
+ strength?: EffectProp<number>;
+ /**
+ * Clean picture mixed back in per frame, 0–1.
+ * Range 0 to 1. Accepts a signal.
+ * @default 0
+ */
+ bleed?: EffectProp<number>;
+ /**
+ * Most frames replayed to rebuild the smear after a jump.
+ * Range 1 to 240.
+ * @default 48
+ */
+ warmup?: number;
+}
+
+const datamoshDefaults: DatamoshProps = {
+ keyframes: [],
+ strength: 1,
+ bleed: 0,
+ warmup: 48,
+};
+
+export class Datamosh extends Effect<DatamoshProps> {
+ public readonly op = "Datamosh";
+
+ public declare keyframes: number[];
+ public declare strength: number;
+ public declare bleed: number;
+ public declare warmup: number;
+
+ constructor(config: DatamoshProps = {}) {
+ super(
+ withDefaults("Datamosh", datamoshDefaults, config, effectMeta.Datamosh),
+ );
+ }
+}
+
export interface DisplacementMapProps {
/**
* Horizontal displacement strength in pixels. Can be modulated by a static number or a dynamic signal.
@@ -2627,6 +2675,84 @@ export class StereoPanning extends Effect<StereoPanningProps> {
}
}
+export interface TrailsProps {
+ /**
+ * History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails.
+ * Range 0 to 1. Accepts a signal.
+ * @default 0.9
+ */
+ decay?: EffectProp<number>;
+ /**
+ * blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest.
+ * @default "blend"
+ */
+ mode?: "blend" | "over" | "add" | "max";
+ /**
+ * History scale per frame; 1.02 pulls the past into a tunnel.
+ * Range 0.5 to 2. Accepts a signal.
+ * @default 1
+ */
+ zoom?: EffectProp<number>;
+ /**
+ * History rotation per frame, degrees.
+ * Range -45 to 45. Accepts a signal.
+ * @default 0
+ */
+ rotation?: EffectProp<number>;
+ /**
+ * History drift per frame, px.
+ * Range -500 to 500. Accepts a signal.
+ * @default 0
+ */
+ offsetX?: EffectProp<number>;
+ /**
+ * History drift per frame, px.
+ * Range -500 to 500. Accepts a signal.
+ * @default 0
+ */
+ offsetY?: EffectProp<number>;
+ /**
+ * History hue rotation per frame, degrees (psychedelic cycling).
+ * Range -180 to 180. Accepts a signal.
+ * @default 0
+ */
+ hueShift?: EffectProp<number>;
+ /**
+ * Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render).
+ * Range 0 to 240.
+ * @default 60
+ */
+ warmup?: number;
+}
+
+const trailsDefaults: TrailsProps = {
+ decay: 0.9,
+ mode: "blend",
+ zoom: 1,
+ rotation: 0,
+ offsetX: 0,
+ offsetY: 0,
+ hueShift: 0,
+ warmup: 60,
+};
+
+export class Trails extends Effect<TrailsProps> {
+ public readonly op = "Trails";
+
+ public declare decay: number;
+ public declare mode: "blend" | "over" | "add" | "max";
+ public declare zoom: number;
+ public declare rotation: number;
+ public declare offsetX: number;
+ public declare offsetY: number;
+ public declare hueShift: number;
+ public declare warmup: number;
+
+ constructor(config: TrailsProps = {}) {
+ super(withDefaults("Trails", trailsDefaults, config, effectMeta.Trails));
+ }
+}
+
export interface VignetteProps {
/**
* The intensity of the vignette darkening effect. 0 = no vignette.
diff --git a/packages/framefields/src/effects/generated/index.ts b/packages/framefields/src/effects/generated/index.ts
index a03cb351..fb2cadae 100644
--- a/packages/framefields/src/effects/generated/index.ts
+++ b/packages/framefields/src/effects/generated/index.ts
@@ -29,6 +29,8 @@ export {
CornerPin,
type CornerPinPointsItem,
type CornerPinProps,
+ Datamosh,
+ type DatamoshProps,
DisplacementMap,
type DisplacementMapProps,
ExtractLUT,
@@ -78,6 +80,8 @@ export {
StereoPanning,
type StereoPanningProps,
type TonalShift,
+ Trails,
+ type TrailsProps,
Vignette,
type VignetteProps,
} from "./classes.js";
diff --git a/packages/framefields/src/effects/generated/meta.ts b/packages/framefields/src/effects/generated/meta.ts
index 67175de6..f0c87007 100644
--- a/packages/framefields/src/effects/generated/meta.ts
+++ b/packages/framefields/src/effects/generated/meta.ts
@@ -308,6 +308,21 @@ export const effectMeta = {
CornerPin: {
points: {},
},
+ Datamosh: {
+ keyframes: {},
+ strength: {
+ min: 0,
+ max: 10,
+ },
+ bleed: {
+ min: 0,
+ max: 1,
+ },
+ warmup: {
+ min: 1,
+ max: 240,
+ },
+ },
DisplacementMap: {
strengthX: {
min: 0,
@@ -941,6 +956,39 @@ export const effectMeta = {
max: 1,
},
},
+ Trails: {
+ decay: {
+ min: 0,
+ max: 1,
+ },
+ mode: {
+ enum: ["blend", "over", "add", "max"],
+ },
+ zoom: {
+ min: 0.5,
+ max: 2,
+ },
+ rotation: {
+ min: -45,
+ max: 45,
+ },
+ offsetX: {
+ min: -500,
+ max: 500,
+ },
+ offsetY: {
+ min: -500,
+ max: 500,
+ },
+ hueShift: {
+ min: -180,
+ max: 180,
+ },
+ warmup: {
+ min: 0,
+ max: 240,
+ },
+ },
Vignette: {
strength: {
min: 0,
diff --git a/packages/framefields/src/effects/generated/registry.ts b/packages/framefields/src/effects/generated/registry.ts
index 9390c894..78037ed2 100644
--- a/packages/framefields/src/effects/generated/registry.ts
+++ b/packages/framefields/src/effects/generated/registry.ts
@@ -28,6 +28,8 @@ import {
type ColorKeyProps,
CornerPin,
type CornerPinProps,
+ Datamosh,
+ type DatamoshProps,
DisplacementMap,
type DisplacementMapProps,
ExtractLUT,
@@ -62,6 +64,8 @@ import {
type RotateProps,
StereoPanning,
type StereoPanningProps,
+ Trails,
+ type TrailsProps,
Vignette,
type VignetteProps,
} from "./classes.js";
@@ -85,6 +89,7 @@ export const effectFactories = {
colorBalance: (config?: ColorBalanceProps) => new ColorBalance(config),
colorKey: (config?: ColorKeyProps) => new ColorKey(config),
cornerPin: (config?: CornerPinProps) => new CornerPin(config),
+ datamosh: (config?: DatamoshProps) => new Datamosh(config),
displacementMap: (config?: DisplacementMapProps) =>
new DisplacementMap(config),
extractLUT: (config?: ExtractLUTProps) => new ExtractLUT(config),
@@ -104,5 +109,6 @@ export const effectFactories = {
resizerScaler: (config?: ResizerScalerProps) => new ResizerScaler(config),
rotate: (config?: RotateProps) => new Rotate(config),
stereoPanning: (config?: StereoPanningProps) => new StereoPanning(config),
+ trails: (config?: TrailsProps) => new Trails(config),
vignette: (config?: VignetteProps) => new Vignette(config),
} as const;
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 20acb7aa..9239e8b0 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -241,6 +241,7 @@ import {
Vision,
type VisionProps,
} from "./effects/index.js";
+import { Particles, type ParticlesOptions } from "./particles.js";
import {
type FrameGridOptions,
HeadlessMediaRenderer,
@@ -709,6 +710,8 @@ export {
type CurvesChannelPoint,
type CurvesProps,
CustomEffect,
+ Datamosh,
+ type DatamoshProps,
type DeflickerProps,
DepthOfField,
type DepthOfFieldProps,
@@ -781,6 +784,8 @@ export {
TileOffset,
type TileOffsetEdgeMode,
type TileOffsetProps,
+ Trails,
+ type TrailsProps,
UnsharpMask,
type UnsharpMaskProps,
Vignette,
@@ -793,6 +798,14 @@ export {
type LightingRig,
type LightingRigOptions,
} from "./lighting.js";
+export {
+ type EmitterShape,
+ type ParticleBlend,
+ type ParticleBounds,
+ type ParticleShape,
+ Particles,
+ type ParticlesOptions,
+} from "./particles.js";
export {
type Body2DOptions,
type Body3DOptions,
@@ -1160,10 +1173,14 @@ export class Composition {
return clip;
}
- /** @internal Analyses every `comp.vision` clip for a composition frame. */
+ /**
+ * @internal Analyses every `comp.vision` clip up to a composition frame
+ * (frames already analysed are skipped), so consumers that step frame by
+ * frame — physics, particles — always find every frame before it.
+ */
public async prepareVision(frame: number): Promise<void> {
await Promise.all(
- [...this._clipVision.values()].map((clip) => clip.ready(frame)),
+ [...this._clipVision.values()].map((clip) => clip.ready(0, frame)),
);
}
@@ -2505,6 +2522,41 @@ export const Layer = {
children: options.children ?? [],
...options,
}),
+ /**
+ * A GPU particle system as a layer: tens of thousands of particles
+ * simulated in a compute shader and drawn in one call (see
+ * {@link ParticlesOptions}). Positions are in the layer's own px; drivers
+ * (rate, gravity, noise, attractors) are read every frame, and a
+ * performer can push particles with `push: { person }`.
+ */
+ particles: (options: ParticlesOptions): AnimatableNode<MediaNode> => {
+ const system = new Particles(options);
+ // A media layer whose source is the particle op itself: it draws into
+ // the layer's own texture, composited like any clip.
+ const source = {
+ metadata: { width: options.width, height: options.height },
+ operation: { ...system.toOperation(), effect: system, dataType: "Video" },
+ children: [],
+ };
+ return withAnimation({
+ id: options.id ?? autoId("particles"),
+ kind: "media",
+ dataType: "Video",
+ src: source,
+ position: "absolute",
+ x: options.x ?? 0,
+ y: options.y ?? 0,
+ width: options.width,
+ height: options.height,
+ ...(options.startFrame !== undefined && {
+ startFrame: options.startFrame,
+ }),
+ ...(options.durationFrames !== undefined && {
+ durationFrames: options.durationFrames,
+ }),
+ ...(options.zIndex !== undefined && { zIndex: options.zIndex }),
+ } as unknown as MediaNode);
+ },
section: (
source: string | MediaNode | Composition,
targetOrOptions: ObjectTrackSignals | SectionOptions = {},
diff --git a/packages/framefields/src/particles-render.test.ts b/packages/framefields/src/particles-render.test.ts
new file mode 100644
index 00000000..dd981cff
--- /dev/null
+++ b/packages/framefields/src/particles-render.test.ts
@@ -0,0 +1,250 @@
+/**
+ * GPU particles and temporal effects through the real renderer: particles
+ * appear where they are emitted, scrubbing back gives the same frame, and
+ * Trails / Datamosh remember earlier frames.
+ */
+import fs from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ Datamosh,
+ HeadlessMediaRenderer,
+ Layer,
+ Layer3D,
+ LayerAnimation,
+ Trails,
+} from "./index.js";
+import { decodePngPixels, getPixel } from "./test-utils/pixels.js";
+
+const FPS = 30;
+
+function stage(frames = 60) {
+ return new Composition({
+ width: 200,
+ height: 200,
+ fps: FPS,
+ durationFrames: frames,
+ backgroundColor: "#000000",
+ });
+}
+
+async function frame(
+ c: Composition,
+ f: number,
+ renderer?: HeadlessMediaRenderer,
+) {
+ return decodePngPixels(
+ await c.renderFrame({ frame: f, ...(renderer && { renderer }) }),
+ );
+}
+
+function brightPixels(
+ img: Awaited<ReturnType<typeof decodePngPixels>>,
+ x0 = 0,
+ x1 = img.width,
+) {
+ let n = 0;
+ for (let y = 0; y < img.height; y++) {
+ for (let x = x0; x < x1; x++) if (getPixel(img, x, y).r > 60) n++;
+ }
+ return n;
+}
+
+describe("Layer.particles", () => {
+ it("emits, moves and draws particles", async () => {
+ const c = stage();
+ c.add(
+ Layer.particles({
+ width: 200,
+ height: 200,
+ max: 2000,
+ emitter: { shape: "point", x: 100, y: 180, rate: 600 },
+ direction: -90,
+ spread: 20,
+ speed: [200, 260],
+ lifetime: [3, 3],
+ size: [6, 6],
+ color: "#ffffff",
+ fade: [0.001, 0.001],
+ }),
+ );
+ const renderer = new HeadlessMediaRenderer();
+ const early = await frame(c, 3, renderer);
+ const later = await frame(c, 20, renderer);
+ // Near the emitter early, risen ~150 px by frame 20.
+ expect(getPixel(early, 100, 172).r).toBeGreaterThan(100);
+ expect(brightPixels(early)).toBeGreaterThan(5);
+ expect(getPixel(later, 100, 60).r).toBeGreaterThan(100);
+ // Nothing left or right of the narrow fountain.
+ expect(brightPixels(later, 0, 40) + brightPixels(later, 160, 200)).toBe(0);
+ });
+
+ it("gives the same frame when scrubbing back (snapshots + replay)", async () => {
+ const c = stage(80);
+ c.add(
+ Layer.particles({
+ width: 200,
+ height: 200,
+ max: 3000,
+ emitter: { shape: "rect", rate: 900 },
+ noise: { strength: 400, scale: 40 },
+ lifetime: [1, 2],
+ size: [4, 2],
+ seed: 3,
+ }),
+ );
+ const renderer = new HeadlessMediaRenderer();
+ const a = await frame(c, 30, renderer);
+ await frame(c, 70, renderer);
+ const b = await frame(c, 30, renderer);
+ expect(Buffer.from(b.pixels).equals(Buffer.from(a.pixels))).toBe(true);
+ });
+});
+
+describe("moving emitters and glass", () => {
+ it("emits where a driven emitter is each frame", async () => {
+ const c = stage(40);
+ c.add(
+ Layer.particles({
+ width: 200,
+ height: 200,
+ max: 3000,
+ emitter: { shape: "point", x: (f) => 20 + f * 4, y: 100, rate: 1200 },
+ speed: [0, 5],
+ lifetime: [0.2, 0.2],
+ size: [6, 6],
+ fade: [0.001, 0.001],
+ color: "#ffffff",
+ }),
+ );
+ const renderer = new HeadlessMediaRenderer();
+ let img = await frame(c, 0, renderer);
+ for (let f = 1; f <= 30; f++) img = await frame(c, f, renderer);
+ // Emitter at x = 140 now; the trail of the last 0.2 s reaches back ~24 px.
+ expect(getPixel(img, 139, 100).r).toBeGreaterThan(100);
+ expect(getPixel(img, 125, 100).r).toBeGreaterThan(100);
+ expect(getPixel(img, 40, 100).r).toBeLessThan(20);
+ });
+
+ it("glass shows what is behind it instead of a flat surface", async () => {
+ const scene = (material: "glass" | "lit") => {
+ const c = stage(2);
+ c.add([
+ Layer.box({
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 100,
+ height: 200,
+ background: "#ff0000",
+ }),
+ Layer.box({
+ position: "absolute",
+ x: 100,
+ y: 0,
+ width: 100,
+ height: 200,
+ background: "#0000ff",
+ }),
+ Layer3D.roundedBox({
+ width: 120,
+ height: 120,
+ depth: 120,
+ x: 100,
+ y: 100,
+ z: 0,
+ color: "#ffffff",
+ material,
+ }),
+ ]);
+ return c;
+ };
+ const glass = await frame(scene("glass"), 0);
+ const left = getPixel(glass, 70, 100);
+ const right = getPixel(glass, 130, 100);
+ expect(left.r).toBeGreaterThan(left.b + 60);
+ expect(right.b).toBeGreaterThan(right.r + 60);
+ const lit = await frame(scene("lit"), 0);
+ // An opaque white box hides the split.
+ expect(
+ Math.abs(getPixel(lit, 70, 100).r - getPixel(lit, 130, 100).r),
+ ).toBeLessThan(30);
+ });
+});
+
+describe("temporal effects", () => {
+ /** Renders a 30-frame clip of a white square sliding right 4 px per frame on black. */
+ async function slidingClip(): Promise<string> {
+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-temporal-"));
+ const file = path.join(dir, "slide.mp4");
+ const c = stage(30);
+ c.add(
+ Layer.box({
+ position: "absolute",
+ x: 0,
+ y: 80,
+ width: 40,
+ height: 40,
+ background: "#ffffff",
+ }).animate(
+ LayerAnimation.keys("x", [
+ [0, 0],
+ [29, 116],
+ ]),
+ ),
+ );
+ const r = await c.renderVideo({ outputPath: file, quality: "lossless" });
+ await r.cleanup?.();
+ return file;
+ }
+
+ function withEffect(clip: string, effect: Trails | Datamosh | null) {
+ const c = stage(30);
+ const layer = Layer.video(clip, {
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: 200,
+ height: 200,
+ fit: "cover",
+ muted: true,
+ });
+ c.add(effect ? layer.withEffect(effect) : layer);
+ return c;
+ }
+
+ it("Trails leaves a fading echo behind a moving subject", async () => {
+ const clip = await slidingClip();
+ const renderer = new HeadlessMediaRenderer();
+ const trails = withEffect(clip, new Trails({ decay: 0.8 }));
+ let img = await frame(trails, 0, renderer);
+ for (let f = 1; f <= 15; f++) img = await frame(trails, f, renderer);
+ const plain = await frame(withEffect(clip, null), 15);
+ // Square now covers x 60–100; x 50 was covered a few frames ago.
+ expect(getPixel(plain, 50, 100).r).toBeLessThan(10);
+ const echo = getPixel(img, 50, 100).r;
+ expect(echo).toBeGreaterThan(20);
+ expect(echo).toBeLessThan(250);
+ // Far behind it, the echo has faded out.
+ expect(getPixel(img, 5, 100).r).toBeLessThan(20);
+ // Jumping straight to a frame replays enough history to match.
+ const jumped = await frame(trails, 15, new HeadlessMediaRenderer());
+ expect(Math.abs(getPixel(jumped, 50, 100).r - echo)).toBeLessThan(6);
+ });
+
+ it("Datamosh shows the clean picture on keyframes", async () => {
+ const clip = await slidingClip();
+ const renderer = new HeadlessMediaRenderer();
+ const mosh = withEffect(clip, new Datamosh({ keyframes: [0, 12] }));
+ for (let f = 0; f <= 12; f++) await frame(mosh, f, renderer);
+ const atKey = await frame(mosh, 12, renderer);
+ const plain = await frame(withEffect(clip, null), 12);
+ expect(getPixel(atKey, 70, 100).r).toBeCloseTo(
+ getPixel(plain, 70, 100).r,
+ -1,
+ );
+ expect(getPixel(atKey, 20, 100).r).toBeLessThan(20);
+ });
+});
diff --git a/packages/framefields/src/particles.ts b/packages/framefields/src/particles.ts
new file mode 100644
index 00000000..1966140e
--- /dev/null
+++ b/packages/framefields/src/particles.ts
@@ -0,0 +1,253 @@
+import { Effect } from "@framefields/core";
+import type {
+ EmitterShape,
+ ParticleBlend,
+ ParticleBounds,
+ ParticleFrame,
+ ParticleProgram,
+ ParticleShape,
+ ParticleStatic,
+} from "@framefields/node-particles";
+import { parseColor } from "@framefields/webgpu-renderers";
+import type { PersonTrack } from "./clip-vision.js";
+import { type PhysicsDriver, sampleDriver } from "./physics/core.js";
+
+export type { EmitterShape, ParticleBlend, ParticleBounds, ParticleShape };
+
+type Range = number | [number, number];
+
+export interface ParticlesOptions {
+ /** Particle layer box in composition px (the particles' own space; default the whole canvas). */
+ x?: number;
+ y?: number;
+ width: number;
+ height: number;
+ id?: string;
+ startFrame?: number;
+ durationFrames?: number;
+ zIndex?: number;
+ /** Most particles alive at once (default 20000). Costs GPU memory, not time when idle. */
+ max?: number;
+ seed?: number;
+ emitter?: {
+ /** `"rect"` (default, the whole layer), `"point"`, `"line"`, `"circle"` (area) or `"ring"` (edge). Layer px. */
+ shape?: EmitterShape;
+ /** Emitter position (layer px). A driver moves it: `(f) => person.point("rightWrist", f).x` paints with light. */
+ x?: PhysicsDriver;
+ y?: PhysicsDriver;
+ width?: number;
+ height?: number;
+ radius?: number;
+ /** Aim away from the emitter's center instead of along `direction`. */
+ radial?: boolean;
+ /** Particles per second (default 1000); a driver for bursts of activity. */
+ rate?: PhysicsDriver;
+ /** Extra particles at given layer frames. */
+ bursts?: Array<{ frame: number; count: number }>;
+ };
+ /** Seconds each particle lives (default [2, 4]). */
+ lifetime?: Range;
+ /** Launch speed, px/s (default [40, 160]). */
+ speed?: Range;
+ /** Launch direction, degrees (0 = right, -90 = up; default -90). */
+ direction?: number;
+ /** Spread around `direction`, degrees (default 360). */
+ spread?: number;
+ /** px/s². A single driver pulls down. */
+ gravity?: PhysicsDriver | [PhysicsDriver, PhysicsDriver];
+ /** Velocity damping per second (default 0.4). Higher reads as thicker air or liquid. */
+ drag?: PhysicsDriver;
+ /** Swirling, divergence-free flow (curl noise): smoke, ink, flow fields. */
+ noise?: { strength?: PhysicsDriver; scale?: number; speed?: number };
+ /** Points that pull (positive) or push (negative), px/s² at the center, falling off over `radius`. Layer px. Up to 8. */
+ attractors?: Array<{
+ x: PhysicsDriver;
+ y: PhysicsDriver;
+ strength: PhysicsDriver;
+ radius?: number;
+ }>;
+ /**
+ * A performer pushes particles out of their silhouette, harder where it
+ * is sweeping in: `person` from `comp.vision(clip).person.place(…)`.
+ */
+ push?: {
+ person: PersonTrack;
+ strength?: PhysicsDriver;
+ sweep?: PhysicsDriver;
+ };
+ /** What happens at the layer's edges: `"none"` (default), `"kill"`, `"bounce"`, `"wrap"`. */
+ bounds?: ParticleBounds;
+ /** A floor line in layer px that particles bounce on. */
+ floor?: number;
+ /** Bounciness at the floor and bounced edges (default 0.3). */
+ bounce?: number;
+ /** px at birth → death (default [6, 0]). */
+ size?: Range;
+ /** Random size variation, 0–1 (default 0.5). */
+ sizeJitter?: number;
+ /** Fractions of life to fade in and out (default [0.1, 0.3]). */
+ fade?: [number, number];
+ /** `"circle"` (default), `"soft"` (glow), `"square"`, `"spark"` (streaks along motion). */
+ shape?: ParticleShape;
+ /** Spark length per px/s (default 0.04). */
+ stretch?: number;
+ /** `"normal"` (default), `"add"` (glows build up), `"screen"`. */
+ blend?: ParticleBlend;
+ /** One color, or up to 8 spread over each particle's life (default white). */
+ color?: string | string[];
+ /** Color by speed instead of age: the px/s that reaches the last color. */
+ colorBySpeed?: number;
+ opacity?: number;
+ /** Solver steps per frame (default 2). */
+ substeps?: number;
+ /** Seconds simulated before the first frame, so it starts already full. */
+ prewarm?: number;
+}
+
+const range = (
+ r: Range | undefined,
+ fallback: [number, number],
+): [number, number] =>
+ r === undefined ? fallback : typeof r === "number" ? [r, r] : r;
+
+/**
+ * A GPU particle system: tens of thousands of particles simulated in a
+ * compute shader and drawn in one call. Used through `Layer.particles()`.
+ * It steps as the composition renders (snapshots make scrubbing back
+ * cheap), and its drivers are read every frame.
+ */
+export class Particles
+ extends Effect<Record<string, never>>
+ implements ParticleProgram
+{
+ public readonly op = "Particles";
+ readonly particleStatic: ParticleStatic;
+ particleState?: unknown;
+ private readonly options: ParticlesOptions;
+
+ constructor(options: ParticlesOptions) {
+ super({});
+ this.options = options;
+ const e = options.emitter ?? {};
+ const staticPos = (d: PhysicsDriver | undefined) =>
+ typeof d === "number" ? d : undefined;
+ const width = options.width;
+ const height = options.height;
+ const colors = (
+ Array.isArray(options.color)
+ ? options.color
+ : [options.color ?? "#ffffff"]
+ ).map((c): [number, number, number, number] => {
+ const p = parseColor(c);
+ return [p.r, p.g, p.b, p.a];
+ });
+ this.particleStatic = {
+ max: Math.max(1, Math.round(options.max ?? 20000)),
+ seed: Math.round(options.seed ?? 1) >>> 0,
+ layer: { x: options.x ?? 0, y: options.y ?? 0, width, height },
+ emitter: {
+ shape: e.shape ?? "rect",
+ x:
+ staticPos(e.x) ??
+ (e.shape === "circle" || e.shape === "ring" || e.shape === "point"
+ ? width / 2
+ : 0),
+ y:
+ staticPos(e.y) ??
+ (e.shape === "circle" || e.shape === "ring" || e.shape === "point"
+ ? height / 2
+ : 0),
+ width: e.width ?? width,
+ height: e.height ?? (e.shape === "line" ? 0 : height),
+ radius: e.radius ?? Math.min(width, height) / 4,
+ radial: e.radial ?? false,
+ },
+ lifetime: range(options.lifetime, [2, 4]),
+ speed: range(options.speed, [40, 160]),
+ direction: options.direction ?? -90,
+ spread: options.spread ?? 360,
+ bounds: options.bounds ?? "none",
+ floor: options.floor ?? null,
+ bounce: options.bounce ?? 0.3,
+ size: range(options.size, [6, 0]),
+ sizeJitter: options.sizeJitter ?? 0.5,
+ fade: options.fade ?? [0.1, 0.3],
+ shape: options.shape ?? "circle",
+ stretch: options.stretch ?? 0.04,
+ blend: options.blend ?? "normal",
+ colors: colors.slice(0, 8),
+ colorBySpeed: options.colorBySpeed ?? null,
+ opacity: options.opacity ?? 1,
+ substeps: options.substeps ?? 2,
+ prewarm: options.prewarm ?? 0,
+ };
+ }
+
+ particleFrame(frame: number, fps: number): ParticleFrame {
+ const o = this.options;
+ const read = (d: PhysicsDriver | undefined, fallback: number) =>
+ d === undefined ? fallback : sampleDriver(d, frame, fps);
+ const layerFrame = frame - (o.startFrame ?? 0);
+ const burst = (o.emitter?.bursts ?? [])
+ .filter((b) => Math.round(b.frame) === layerFrame)
+ .reduce((n, b) => n + b.count, 0);
+ const g = o.gravity;
+ const gravity: [number, number] = Array.isArray(g)
+ ? [read(g[0], 0), read(g[1], 0)]
+ : [0, read(g, 0)];
+ let push: ParticleFrame["push"] = null;
+ if (o.push) {
+ push = {
+ strength: read(o.push.strength, 6000),
+ sweep: read(o.push.sweep, 60000),
+ mask: o.push.person.silhouetteGrid(frame),
+ previous: o.push.person.silhouetteGrid(Math.max(0, frame - 1)),
+ };
+ }
+ const ex = o.emitter?.x;
+ const ey = o.emitter?.y;
+ const moving =
+ (ex !== undefined && typeof ex !== "number") ||
+ (ey !== undefined && typeof ey !== "number");
+ const at = (f: number) => ({
+ x:
+ ex === undefined
+ ? this.particleStatic.emitter.x
+ : sampleDriver(ex, f, fps),
+ y:
+ ey === undefined
+ ? this.particleStatic.emitter.y
+ : sampleDriver(ey, f, fps),
+ });
+ let emitter: ParticleFrame["emitter"];
+ if (moving) {
+ const now = at(frame);
+ const before = at(Math.max(0, frame - 1));
+ emitter = {
+ x: now.x,
+ y: now.y,
+ previousX: before.x,
+ previousY: before.y,
+ };
+ }
+ return {
+ ...(emitter && { emitter }),
+ rate: read(o.emitter?.rate, 1000),
+ burst,
+ gravity,
+ drag: read(o.drag, 0.4),
+ noise: {
+ strength: read(o.noise?.strength, 0),
+ scale: o.noise?.scale ?? 300,
+ speed: o.noise?.speed ?? 0.3,
+ },
+ attractors: (o.attractors ?? []).map((a) => ({
+ x: read(a.x, 0),
+ y: read(a.y, 0),
+ strength: read(a.strength, 0),
+ radius: a.radius ?? 200,
+ })),
+ push,
+ };
+ }
+}
diff --git a/packages/framefields/src/physics/backend.ts b/packages/framefields/src/physics/backend.ts
index 6f684e00..98f1cb56 100644
--- a/packages/framefields/src/physics/backend.ts
+++ b/packages/framefields/src/physics/backend.ts
@@ -55,6 +55,8 @@ export interface BodySpec2D extends BodySpecBase {
}
export interface BodySpec3D extends BodySpecBase {
+ /** Per-axis rotation, when only some axes are locked: [x, y, z] allowed. */
+ enabledRotations?: [boolean, boolean, boolean];
position: Vec3;
rotation: Quat;
velocity: Vec3;
@@ -306,13 +308,24 @@ class RapierWorld3D implements SimWorld3D {
.setTranslation(spec.position[0], spec.position[1], spec.position[2])
.setRotation({ x: qx, y: qy, z: qz, w: qw })
.setLinvel(spec.velocity[0], spec.velocity[1], spec.velocity[2])
- .setAngvel(vec3(spec.angularVelocity))
+ // Locked axes start still too (Rapier keeps an initial spin otherwise).
+ .setAngvel(
+ vec3(
+ spec.lockRotation
+ ? [0, 0, 0]
+ : (spec.angularVelocity.map((w, i) =>
+ spec.enabledRotations && !spec.enabledRotations[i] ? 0 : w,
+ ) as Vec3),
+ ),
+ )
.setLinearDamping(spec.linearDamping)
.setAngularDamping(spec.angularDamping)
.setGravityScale(spec.gravityScale)
.setCcdEnabled(spec.ccd)
.setEnabled(spec.enabled);
if (spec.lockRotation) desc.lockRotations();
+ else if (spec.enabledRotations)
+ desc.enabledRotations(...spec.enabledRotations);
const body = this.world.createRigidBody(desc);
const shape = shapeDesc3D(R, spec.shape);
const collider = this.world.createCollider(
@@ -439,7 +452,8 @@ function shapeDesc2D(R: Rapier2D, shape: Shape2D): RAPIER2D.ColliderDesc {
case "ball":
return R.ColliderDesc.ball(shape.radius);
case "box": {
- const r = Math.min(shape.radius ?? 0, shape.hx, shape.hy);
+ // The rounded border wraps a core box; keep the core solid (a flat core has no mass).
+ const r = Math.min(shape.radius ?? 0, shape.hx / 2, shape.hy / 2);
return r > 0
? R.ColliderDesc.roundCuboid(shape.hx - r, shape.hy - r, r)
: R.ColliderDesc.cuboid(shape.hx, shape.hy);
@@ -460,7 +474,13 @@ function shapeDesc3D(R: Rapier3D, shape: Shape3D): RAPIER3D.ColliderDesc {
case "ball":
return R.ColliderDesc.ball(shape.radius);
case "box": {
- const r = Math.min(shape.radius ?? 0, shape.hx, shape.hy, shape.hz);
+ // The rounded border wraps a core box; keep the core solid (a flat core has no mass).
+ const r = Math.min(
+ shape.radius ?? 0,
+ shape.hx / 2,
+ shape.hy / 2,
+ shape.hz / 2,
+ );
return r > 0
? R.ColliderDesc.roundCuboid(
shape.hx - r,
diff --git a/packages/framefields/src/physics/core.ts b/packages/framefields/src/physics/core.ts
index 08622852..27d42811 100644
--- a/packages/framefields/src/physics/core.ts
+++ b/packages/framefields/src/physics/core.ts
@@ -200,11 +200,28 @@ export class Recording {
return arr;
}
+ /**
+ * A per-frame signal over a channel. Creating it does nothing; the first
+ * value read runs the simulation (offline worlds), so signals can be bound
+ * while bodies are still being added.
+ */
signal(name: string): FrameSignal<number> {
- this.ensure();
let sig = this.signals.get(name);
if (!sig) {
- sig = Signal.fromArray(this.channel(name), this.fps);
+ const arr = this.channel(name);
+ const fps = this.fps;
+ sig = Signal.builder({
+ type: "custom",
+ fn: (ctx: { frame: number; fps?: number }) => {
+ this.ensure();
+ const f =
+ ctx.fps && ctx.fps !== fps
+ ? (ctx.frame / ctx.fps) * fps
+ : ctx.frame;
+ const i = Math.max(0, Math.min(arr.length - 1, Math.round(f)));
+ return arr[i];
+ },
+ }) as unknown as FrameSignal<number>;
this.signals.set(name, sig);
}
return sig;
diff --git a/packages/framefields/src/physics/physics.test.ts b/packages/framefields/src/physics/physics.test.ts
index 6d5b1a8d..1fcf110f 100644
--- a/packages/framefields/src/physics/physics.test.ts
+++ b/packages/framefields/src/physics/physics.test.ts
@@ -262,8 +262,11 @@ describe("Physics.world2d", () => {
/x.*y/,
);
const b = world.body(card("a", 0, 0));
- void b.y;
- expect(() => world.body(card("b", 0, 0))).toThrow(/already started/);
+ // Holding a signal is fine; reading a value runs the simulation and locks.
+ const y = b.y;
+ world.body(card("b", 0, 0));
+ at(y, 0);
+ expect(() => world.body(card("c", 0, 0))).toThrow(/already started/);
});
});
@@ -382,6 +385,51 @@ describe("Physics.world3d", () => {
]);
});
+ it("drops thin rounded slabs (rounding never empties the core)", async () => {
+ const world = await Physics.world3d({ fps: FPS, frames: 40 });
+ world.ground({ y: 800 });
+ const slab = world.body(
+ Layer3D.roundedBox({
+ width: 300,
+ height: 120,
+ depth: 34,
+ radius: 18,
+ x: 400,
+ y: 100,
+ z: 0,
+ }),
+ { lockRotation: true },
+ );
+ const pill = await Physics.world2d({ fps: FPS, frames: 40 });
+ pill.ground({ y: 800 });
+ const p = pill.body(card("pill", 0, 100, { borderRadius: 50 }));
+ world.simulate();
+ pill.simulate();
+ expect(at(slab.y, 39)).toBeGreaterThan(400);
+ expect(at(p.y, 39)).toBeGreaterThan(400);
+ });
+
+ it("locks rotation per axis", async () => {
+ const world = await Physics.world3d({ fps: FPS, frames: 60 });
+ world.ground({ y: 600 });
+ const card3 = world.body(
+ Layer3D.roundedBox({
+ width: 200,
+ height: 60,
+ depth: 30,
+ x: 0,
+ y: 0,
+ z: 0,
+ rotateZ: 30,
+ rotateX: 0,
+ }),
+ { lockRotation: { x: true, y: true }, angularVelocity: [200, 200, 0] },
+ );
+ world.simulate();
+ expect(Math.abs(at(card3.rotateX, 59))).toBeLessThan(0.01);
+ expect(Math.abs(at(card3.rotateY, 59))).toBeLessThan(0.01);
+ });
+
it("converts rotations with the renderer's Euler order", () => {
for (const e of [
[10, 20, 30],
diff --git a/packages/framefields/src/physics/world3d.ts b/packages/framefields/src/physics/world3d.ts
index 748414d6..3db494fb 100644
--- a/packages/framefields/src/physics/world3d.ts
+++ b/packages/framefields/src/physics/world3d.ts
@@ -42,7 +42,12 @@ export interface PhysicsBox3D {
rotateZ?: number;
}
-export interface Body3DOptions extends BodyCommonOptions {
+export interface Body3DOptions extends Omit<BodyCommonOptions, "lockRotation"> {
+ /**
+ * Never rotates (`true`), or only around some axes: `{ x: true, y: true }`
+ * keeps a card facing the camera while it still spins in the picture plane.
+ */
+ lockRotation?: boolean | { x?: boolean; y?: boolean; z?: boolean };
/**
* Collider: `"auto"` (default) reads the Layer3D primitive (box, sphere,
* cylinder) and falls back to the mesh's convex hull for models and
@@ -267,7 +272,7 @@ export class PhysicsBody3D implements CoreBody<Vec3, Quat> {
["y", "y"],
["z", "z"],
];
- if (!this.options.lockRotation) {
+ if (this.options.lockRotation !== true) {
out.push(
["rotateX", "rotateX"],
["rotateY", "rotateY"],
@@ -381,7 +386,14 @@ export class PhysicsWorld3D extends WorldCore<Vec3, Quat, PhysicsBody3D> {
linearDamping: options.linearDamping ?? 0,
angularDamping: options.angularDamping ?? 0,
gravityScale: options.gravityScale ?? 1,
- lockRotation: options.lockRotation ?? false,
+ lockRotation: options.lockRotation === true,
+ ...(typeof options.lockRotation === "object" && {
+ enabledRotations: [
+ !options.lockRotation.x,
+ !options.lockRotation.y,
+ !options.lockRotation.z,
+ ] as [boolean, boolean, boolean],
+ }),
ccd: options.ccd ?? false,
enabled: true,
sensor: options.sensor ?? false,
diff --git a/packages/renderer/package.json b/packages/renderer/package.json
index f6ea91c1..1e67c339 100644
--- a/packages/renderer/package.json
+++ b/packages/renderer/package.json
@@ -62,6 +62,7 @@
"@framefields/node-corner-pin": "workspace:*",
"@framefields/node-crop": "workspace:*",
"@framefields/node-curves": "workspace:*",
+ "@framefields/node-datamosh": "workspace:*",
"@framefields/node-deflicker": "workspace:*",
"@framefields/node-displacement-map": "workspace:*",
"@framefields/node-extract-frame": "workspace:*",
@@ -81,6 +82,7 @@
"@framefields/node-modulate": "workspace:*",
"@framefields/node-noise-generator": "workspace:*",
"@framefields/node-paint": "workspace:*",
+ "@framefields/node-particles": "workspace:*",
"@framefields/node-patch-heal": "workspace:*",
"@framefields/node-refine-edge": "workspace:*",
"@framefields/node-refraction-caustics": "workspace:*",
@@ -94,6 +96,7 @@
"@framefields/node-signal-gate": "workspace:*",
"@framefields/node-stereo-panning": "workspace:*",
"@framefields/node-tile-offset": "workspace:*",
+ "@framefields/node-trails": "workspace:*",
"@framefields/node-unsharp-mask": "workspace:*",
"@framefields/node-vignette": "workspace:*",
"@framefields/node-vision": "workspace:*",
diff --git a/packages/renderer/src/generated/node-renderers.ts b/packages/renderer/src/generated/node-renderers.ts
index 42bd861d..1948f127 100644
--- a/packages/renderer/src/generated/node-renderers.ts
+++ b/packages/renderer/src/generated/node-renderers.ts
@@ -106,6 +106,11 @@ export const BUILTIN_NODE_RENDERERS: readonly BuiltinNodeRenderer[] = [
ops: ["Curves"],
load: () => import("@framefields/node-curves/renderer"),
},
+ {
+ packageName: "@framefields/node-datamosh",
+ ops: ["Datamosh"],
+ load: () => import("@framefields/node-datamosh/renderer"),
+ },
{
packageName: "@framefields/node-deflicker",
ops: ["TemporalDeflicker", "Deflicker"],
@@ -196,6 +201,11 @@ export const BUILTIN_NODE_RENDERERS: readonly BuiltinNodeRenderer[] = [
ops: ["Paint"],
load: () => import("@framefields/node-paint/renderer"),
},
+ {
+ packageName: "@framefields/node-particles",
+ ops: ["Particles"],
+ load: () => import("@framefields/node-particles/renderer"),
+ },
{
packageName: "@framefields/node-patch-heal",
ops: ["PatchHeal"],
@@ -256,6 +266,11 @@ export const BUILTIN_NODE_RENDERERS: readonly BuiltinNodeRenderer[] = [
ops: ["TileOffset"],
load: () => import("@framefields/node-tile-offset/renderer"),
},
+ {
+ packageName: "@framefields/node-trails",
+ ops: ["Trails"],
+ load: () => import("@framefields/node-trails/renderer"),
+ },
{
packageName: "@framefields/node-unsharp-mask",
ops: ["UnsharpMask"],
diff --git a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
index f2be4594..f9a83739 100644
--- a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
+++ b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts
@@ -29,6 +29,17 @@ export interface MeshMaterialMaps {
emissive?: GPUTexture;
}
+export interface MeshGlass {
+ /** Index of refraction (1.5 glass, 1.33 water). */
+ ior: number;
+ /** Red/blue split of the refraction, 0–0.2 (rainbow fringes). */
+ dispersion: number;
+ /** How far the backdrop bends at the edges, px. */
+ thickness: number;
+ /** Frosted blur, 0–1. */
+ frost: number;
+}
+
export interface MeshUniformData extends SurfaceParams {
modelMatrix: Float32Array | number[];
normalMatrix?: Float32Array | number[];
@@ -41,6 +52,13 @@ export interface MeshUniformData extends SurfaceParams {
material?: "lit" | "unlit" | "toon";
/** Screen-space track matte: the mesh shows where this texture's alpha does. */
matte?: GPUTexture;
+ /**
+ * Glass: the surface shows `backdrop` (what is behind it on screen) bent by
+ * refraction, tinted by the color, under Fresnel reflections.
+ */
+ glass?: MeshGlass;
+ /** Screen-sized picture behind the 3D batch, read by glass. */
+ backdrop?: GPUTexture;
/** Toon shading: the shadowed side's color multiplier. */
shadeColor?: [number, number, number];
/** Normal / metallic-roughness / occlusion / emissive maps. */
@@ -134,6 +152,7 @@ export class Mesh3DPipeline {
buffer: { type: "uniform" },
},
matteLayoutEntry(2),
+ matteLayoutEntry(3),
],
});
@@ -304,6 +323,15 @@ export class Mesh3DPipeline {
this.modelData[57] = maps?.occlusion ? (maps.occlusionStrength ?? 1) : 0;
this.modelData[58] = maps?.metallicRoughness ? 1 : 0;
this.modelData[59] = maps?.emissive ? 1 : 0;
+ const glass = data.glass;
+ if (glass) {
+ // shade.rgb and pbr.zw are unused outside toon: glass parameters.
+ this.modelData[44] = glass.dispersion;
+ this.modelData[45] = glass.thickness;
+ this.modelData[46] = glass.frost;
+ this.modelData[50] = 1;
+ this.modelData[51] = glass.ior;
+ }
const modelBuffer = this.modelPool.getBuffer(this.device, this.modelData);
@@ -321,6 +349,12 @@ export class Mesh3DPipeline {
binding: 2,
resource: data.matte?.createView() ?? fullCoverageView(this.device),
},
+ {
+ binding: 3,
+ resource:
+ (data.glass && data.backdrop?.createView()) ||
+ fullCoverageView(this.device),
+ },
],
});
}
diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
index e199c9a0..00d1355a 100644
--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts
@@ -123,6 +123,14 @@ export interface DrawQuad3DOpts extends SurfaceParams {
const TEXT_STACK_PULL = 5e-4;
export interface DrawMesh3DOpts extends SurfaceParams {
+ /** Refractive index for glass materials (default 1.5). */
+ ior?: number;
+ /** Rainbow split for glass materials, 0–0.2 (default 0.04). */
+ dispersion?: number;
+ /** Edge bend for glass materials, px (default 80). */
+ thickness?: number;
+ /** Screen-sized picture behind this 3D batch; glass materials refract it. */
+ backdrop?: GPUTexture;
/**
* Screen-space track matte, the size of the pass: the layer shows where
* this texture's alpha does and keeps its place in the depth buffer.
@@ -151,7 +159,14 @@ export interface DrawMesh3DOpts extends SurfaceParams {
/** Pre-computed normal matrix */
normalMatrix?: Mat4;
/** Material shading mode */
- material?: "lit" | "unlit" | "toon";
+ material?:
+ | "lit"
+ | "unlit"
+ | "toon"
+ | "glass"
+ | "acrylic"
+ | "frosted"
+ | "metallic";
/** Toon shading: the shadowed side's color multiplier */
shadeColor?: [number, number, number];
/** Normal / metallic-roughness / occlusion / emissive maps (see `materialMaps`). */
@@ -975,7 +990,7 @@ export class Renderer3D {
metallic: opts.metallic,
emissive: opts.emissive,
emissiveIntensity: opts.emissiveIntensity,
- matte: opts.matte,
+ matte: opts.matte,
},
useMrt,
this.passWritesHdr,
@@ -1236,6 +1251,17 @@ export class Renderer3D {
addressModeV: address,
});
+ const glassy =
+ material === "glass" || material === "acrylic" || material === "frosted";
+ const glass = glassy
+ ? {
+ ior: opts.ior ?? (material === "acrylic" ? 1.49 : 1.5),
+ dispersion: opts.dispersion ?? (material === "glass" ? 0.04 : 0.015),
+ thickness: opts.thickness ?? 80,
+ frost: material === "frosted" ? (roughness ?? 0.5) : 0,
+ }
+ : undefined;
+
this.mesh3dPipeline.draw(
pass,
cameraBindGroup,
@@ -1244,6 +1270,8 @@ export class Renderer3D {
texture,
sampler,
{
+ glass,
+ backdrop: opts.backdrop,
modelMatrix: m,
normalMatrix,
colorTint,
@@ -1252,11 +1280,12 @@ export class Renderer3D {
wireframe,
isSkinned,
skinningMatrices,
- material,
+ material:
+ material === "unlit" || material === "toon" ? material : "lit",
shadeColor,
alphaCutoff,
shininess,
- roughness,
+ roughness: glassy ? Math.min(roughness ?? 0.05, 0.3) : roughness,
specularIntensity,
ambientIntensity,
metallic,
diff --git a/packages/webgpu-renderers/src/shaders/mesh3d.ts b/packages/webgpu-renderers/src/shaders/mesh3d.ts
index c7abad67..1bf89c65 100644
--- a/packages/webgpu-renderers/src/shaders/mesh3d.ts
+++ b/packages/webgpu-renderers/src/shaders/mesh3d.ts
@@ -32,7 +32,8 @@ struct SkinningUniforms {
@group(0) @binding(0) var<uniform> cam : CameraUniforms;
@group(1) @binding(0) var<uniform> model : ModelUniforms;
@group(1) @binding(1) var<uniform> skinning : SkinningUniforms;
-${matte3dWgsl(1, 2)}@group(2) @binding(0) var texSampler : sampler;
+${matte3dWgsl(1, 2)}@group(1) @binding(3) var backdropTex : texture_2d<f32>;
+@group(2) @binding(0) var texSampler : sampler;
@group(2) @binding(1) var mainTex : texture_2d<f32>;
@group(2) @binding(2) var normalTex : texture_2d<f32>;
@group(2) @binding(3) var metalRoughTex : texture_2d<f32>;
@@ -154,6 +155,75 @@ fn perturbNormal(N: vec3<f32>, p: vec3<f32>, uv: vec2<f32>, tn: vec3<f32>, dp1:
return normalize(T * (tn.x * invmax) - B * (tn.y * invmax) + N * tn.z);
}
+// Glass: what is behind the 3D batch on screen (backdrop), bent by
+// refraction along the surface normal with a red/blue split, frosted by a
+// disc blur, tinted by the color, under Fresnel-weighted reflections.
+// shade.rgb = (dispersion, thickness px, frost), pbr.w = ior.
+fn backdropAt(uv: vec2<f32>) -> vec3<f32> {
+ return textureSampleLevel(backdropTex, texSampler, clamp(uv, vec2<f32>(0.001), vec2<f32>(0.999)), 0.0).rgb;
+}
+
+// rimWidth: how much 1 - N·V changes across this pixel (fwidth, taken in
+// uniform control flow by the caller). Rim effects narrower than a pixel are
+// faded out, or per-pixel shading turns them into stair-stepped lines that
+// MSAA (coverage only) cannot smooth.
+fn shadeGlass(in: VertexOutput, Nin: vec3<f32>, tint: vec3<f32>, alpha: f32, rimWidth: f32) -> MeshFragment {
+ let V = normalize(cam.cameraPos.xyz - in.worldPos);
+ var N = Nin;
+ if (dot(N, V) < 0.0) { N = -N; }
+ let NdotV = clamp(dot(N, V), 0.0, 1.0);
+ // Also fade toward edge-on: a face seen at a grazing angle is a sliver
+ // under a pixel tall (a slab's bottom), which would alias the same way.
+ let rimAA = clamp(1.0 - rimWidth * 2.5, 0.0, 1.0) * smoothstep(0.03, 0.22, NdotV);
+ let rim = (1.0 - NdotV) * rimAA;
+ let ior = max(model.pbr.w, 1.0001);
+ let screen = max(cam.screenParams.xy, vec2<f32>(1.0));
+ let uv = in.clipPos.xy / screen;
+ // Screen-space bend: the normal's in-plane part, stronger toward the rim.
+ let bend = N.xy * model.shade.g * (1.0 - 1.0 / ior) * (0.6 + rim) * rimAA / screen;
+ let disp = model.shade.r;
+ let frost = model.shade.b * 22.0 / screen;
+ var col = vec3<f32>(0.0);
+ let taps = select(1, 8, model.shade.b > 0.001);
+ for (var i = 0; i < taps; i++) {
+ let a = f32(i) * 2.399963;
+ let r = sqrt((f32(i) + 0.5) / f32(taps));
+ let o = select(vec2<f32>(0.0), vec2<f32>(cos(a), sin(a)) * r * frost, taps > 1);
+ col += vec3<f32>(
+ backdropAt(uv - bend * (1.0 + disp * 6.0) + o).r,
+ backdropAt(uv - bend + o).g,
+ backdropAt(uv - bend * (1.0 - disp * 6.0) + o).b,
+ );
+ }
+ col /= f32(taps);
+ // Light travels further through the glass toward the rim: the tint
+ // deepens and the body darkens there, which is what outlines glass.
+ let depthTint = mix(vec3<f32>(1.0), tint, 0.55 + 0.45 * rim);
+ let transmitted = col * depthTint * (1.0 - 0.45 * rim * rim);
+ let f0 = pow((ior - 1.0) / (ior + 1.0), 2.0);
+ let F = f0 + (1.0 - f0) * pow(rim, 5.0);
+ let lit = shadeSurfaceFull(in.worldPos, N, vec3<f32>(0.0), model.materialParams, vec4<f32>(model.pbr.x, 0.0, 0.0, 0.0), vec4<f32>(0.0));
+ // Grazing faces mirror the room; keep that faint so dark scenes stay dark
+ // and the studio highlights below carry the shape.
+ let sheen = vec3<f32>(0.9, 0.94, 1.0) * F * 0.22 + tint * 0.05 * (0.4 + rim);
+ // A studio in the reflection: a sharp key and two strip softboxes, which
+ // is most of what makes glass read as glass.
+ let R = reflect(-V, N);
+ // Tight: a flat face only flashes as it turns through the light.
+ let key = pow(max(dot(R, normalize(vec3<f32>(-0.45, -0.75, -0.5))), 0.0), 600.0) * 1.8;
+ // Two tall softboxes: narrow bands in the reflection, brightest above.
+ let strips = (1.0 - smoothstep(0.03, 0.07, abs(R.x + 0.45))) * smoothstep(0.3, -0.5, R.y) * 1.1
+ + (1.0 - smoothstep(0.02, 0.05, abs(R.x - 0.6))) * smoothstep(0.4, -0.3, R.y) * 0.7;
+ let studio = vec3<f32>(1.0, 0.98, 0.95) * (key + strips * (0.25 + F));
+ // A thin bright line where the surface turns away: glass silhouettes.
+ let edge = vec3<f32>(1.0, 0.99, 0.97) * smoothstep(0.8, 0.97, rim) * 0.55;
+ let rgb = transmitted * (1.0 - F) + lit.display + sheen + studio + edge;
+ var out: MeshFragment;
+ out.color = vec4<f32>(rgb * alpha, alpha);
+ out.hdr = vec4<f32>(srgbToLinear(transmitted * (1.0 - F)) * alpha + lit.hdr * alpha + srgbToLinear(sheen + studio + edge) * alpha, alpha);
+ return out;
+}
+
fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {
// Every sample and derivative up front, in uniform control flow.
var texColor = textureSample(mainTex, texSampler, in.uv);
@@ -165,6 +235,7 @@ fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {
let dp2 = dpdy(in.worldPos);
let duv1 = dpdx(in.uv);
let duv2 = dpdy(in.uv);
+ let rimWidth = fwidth(1.0 - abs(dot(normalize(in.worldNormal), normalize(cam.cameraPos.xyz - in.worldPos))));
if (!isFront && model.params.y < 0.5) {
discard;
@@ -205,6 +276,9 @@ fn shadeMesh(in: VertexOutput, isFront: bool) -> MeshFragment {
pbr.x = select(pbr.x, 1.0, pbr.x < 0.0) * mrSample.g;
pbr.y = pbr.y * mrSample.b;
}
+ if (model.pbr.z > 0.5) {
+ return shadeGlass(in, N, baseRgb, alpha, rimWidth);
+ }
var material = model.materialParams;
material.z = material.z * mix(1.0, aoSample, clamp(model.maps.y, 0.0, 1.0));
var emissive = model.emissive;
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index a7d58639..16898c2a 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -366,6 +366,7 @@ Layer3D.cylinder({ ..., color: "#111", emissive: "#4da3ff", emissiveIntensity: 3
- **Fog** fades to `fog.color` exactly, so set it to the composition background for a seamless horizon.
- **Models:** glTF materials bring base color, normal, metallic-roughness, occlusion and emissive maps.
- `emissive` / `emissiveIntensity` / `exposure` animate like any prop (`.keys("emissiveIntensity", …)`).
+- **Glass:** `material: "glass" | "acrylic" | "frosted"` on any Layer3D primitive, extruded text (`Layer3D.extrudedText`) or model refracts whatever is behind it on screen — the video, 2D layers, particles — with a rainbow fringe (`dispersion`, default 0.04), bend strength `thickness` (px, default 80) and `ior` (1.5). `color` tints the glass (deeper toward the edges); `frosted` blurs by `roughness`. Glass needs no environment: it draws its own studio reflections, and against a dark grade it reads best — skip `Lighting.*` rigs for glass-only scenes (their environment washes glass out). Glass shows only 2D content and opaque 3D drawn before it, not other glass.
---
@@ -406,7 +407,7 @@ stage.animate(LayerAnimation.create().signal("y", world.impact, { multiplier: 14
- Reads are synchronous and need the frame analysed: inside a live world, while rendering, or after `await dancer.ready(frame)` (e.g. `ready(0)` to lay things out around the first pose). A live world can't look ahead, so place things from early frames.
- Layer a vision cutout of the same clip (`Layer.video(src).withVision({ mode: "matte" })`) above the particles so they pass behind the performer.
- Live worlds bind signals only (no `bakeKeyframes()`); `await world.advanceTo(frame)` steps one without rendering (e.g. to read `collisions`).
-- **Particles today:** up to ~600 small `Layer.shape("circle")` bodies with `linearDamping` ≈ 1–1.5, low `friction`, `gravityScale` < 1 read as a fluid; tint them by speed with `.colorSignal("fillColor", body.speed, { colorA, colorB, multiplier: 1 / 900 })` (`multiplier`/`offset` scale the signal into 0–1).
+- **Particles:** for clouds, dust, sparks, fluids and flow fields use `Layer.particles` (GPU, below), not hundreds of physics bodies. Keep physics bodies for a few dozen things that must collide and stack (cards, letters, products).
- **Keyframes instead of signals:** `world.bakeKeyframes({ tolerance: 0.25 })` in place of `simulate()` writes plain, simplified linear keyframes (editable, plain JSON).
- **3D:** `Physics.world3d` takes `Layer3D.roundedBox / sphere / cylinder` and models (convex hull of the mesh); same options with 3-vectors, `world.hinge(a, b, at, axis)`, `world.walls({ width, height, depth })`.
- **Timing:** signals are on the composition clock. In a scene that starts at frame `S`, pass `startFrame: S` to the world. Before it and after `frames` the signals hold their first/last value. Add every body, joint and force before `simulate()` (or before reading any signal): the world then locks.
@@ -415,6 +416,34 @@ stage.animate(LayerAnimation.create().signal("y", world.impact, { multiplier: 14
---
+## GPU Particles (`Layer.particles`)
+
+Tens of thousands of particles simulated in a compute shader and drawn in one call — sparks, embers, dust, snow, smoke, ink, flow fields, light painting, a fluid a performer moves through. A particle layer is a box in composition px; positions inside it are layer px.
+
+```typescript
+comp.add(Layer.particles({
+ width: W, height: H, max: 20000, seed: 7,
+ emitter: { shape: "rect", x: 0, y: H * 0.6, width: W, height: 200, rate: 3000 }, // point | line | rect | circle | ring; radial
+ lifetime: [4, 6], speed: [0, 40], direction: -90, spread: 360,
+ gravity: 20, drag: 1.5, // px/s², per second
+ noise: { strength: 260, scale: 170, speed: 0.25 }, // curl-noise flow: smoke, ink, flow fields
+ attractors: [{ x: W / 2, y: H / 2, strength: 900, radius: 200 }], // negative strength repels
+ push: { person, strength: 9000, sweep: 120000 }, // a performer's silhouette pushes them (comp.vision)
+ floor: H - 60, bounce: 0.2, bounds: "none", // none | kill | bounce | wrap
+ size: [10, 4], sizeJitter: 0.5, fade: [0.1, 0.3],
+ shape: "circle", // circle | soft (glow) | square | spark (streaks along motion)
+ color: ["#ffd23f", "#ff5a1f", "#7a2e14"], // over life, or by speed with colorBySpeed: 600
+ blend: "normal", // add for glows and light painting
+ prewarm: 3, // seconds simulated before frame 0: starts full
+}));
+```
+
+- `emitter.x` / `emitter.y`, `rate`, `gravity`, `drag`, `noise.strength`, attractor positions/strengths and `push` strengths take a number, a signal (audio, physics `impact`) or `(frame) => value`, read every frame: bursts on the beat, wind gusts, an attractor following a tracked hand (`person.point("rightWrist", f)`). `emitter.bursts: [{ frame, count }]` adds one-off explosions.
+- Recipes: **light painting** an additive `"soft"` point emitter whose `x`/`y` follow a tracked wrist (`(f) => person.point("rightWrist", f).x`), lifetime ~1 s, small speed, births spread along the hand's path so fast moves stay continuous; **sparks/fountain** `shape: "spark"`, `blend: "add"`, gravity, `floor`; **smoke/ink** `shape: "soft"`, low opacity, `noise`, sizes growing; **snow/dust** `rect` emitter above the frame, small gravity, `noise`, `bounds: "wrap"`; **swirl** `ring` emitter + attractor + `colorBySpeed`; **fluid around a dancer** `push: { person }` + `drag` 2 + gentle noise, and a vision cutout of the clip above the particle layer.
+- The system steps as frames render and snapshots itself every 24 frames, so scrubbing back restores instead of replaying from the start; the same GPU gives the same frames.
+
+---
+
## Critical Authoring Rules & Gotchas
1. **Local Keyframe Coordinates:** Keyframes are strictly relative to the node's own timeline window (`0` = node appearance). Never offset keyframe frames by the parent's `startFrame`.
diff --git a/plugins/framefields/skills/framefields-effects/SKILL.md b/plugins/framefields/skills/framefields-effects/SKILL.md
index d4b52ebc..13ff997e 100644
--- a/plugins/framefields/skills/framefields-effects/SKILL.md
+++ b/plugins/framefields/skills/framefields-effects/SKILL.md
@@ -22,7 +22,9 @@ See [references/effects-catalog.md](references/effects-catalog.md) for the gener
There are 4 main attachment modes in the engine:
### 1. Layer-Level Effect Pipeline
-Attach one or more effects to an individual media, box, or text layer:
+Attach one or more effects to a media layer (video, image, SVG, Lottie). Box and text layers take only `TemporalDeflicker` and `Relight3D` directly; to treat a group or text as an image, render it into a media layer first.
+
+
```typescript
import { Layer, Curves, Vignette, Blur } from "framefields";
@@ -109,6 +111,11 @@ const pngBuffer = await clip.renderFrame({ atMs: 1500 });
- **`MotionBlur`**: Directional shutter simulation with custom shutter angles and velocity clamps.
- **`TemporalDeflicker`**: Multi-frame optical flow blending to eliminate flickering in generative AI clips or high-speed footage.
+### Temporal (effects with memory)
+- **`Trails`**: the layer over a decaying copy of its own past. `mode: "blend"` (default; echoes on full-frame footage), `"over"` (cutouts and text), `"add"` (light painting on dark scenes), `"max"` (light trails). `decay` 0.8 short echo → 0.97 long trails; per-frame `zoom` (1.02 = tunnel), `rotation`, `offsetX/Y`, `hueShift` (psychedelic cycling).
+- **`Datamosh`**: the previous picture dragged along the new frame's optical flow instead of being refreshed, until a `keyframes` frame (I-frame) shows the clean picture. Put keyframes where you want it clean; leave a cut out to smear one shot with the next one's motion. `strength`, `bleed` (clean picture mixed back per frame).
+- Both step as frames render; after a jump (scrubbing, a single `renderFrame`) they replay the last frames (`warmup`) to rebuild the history.
+
### 3. 3D Spatial Lighting & Materials
- **`Relight3D`**: Screen-space normal map relighting with point, spot, and directional lights, specular highlights, roughness, and metallic uniforms.
- **`SSAO`**: Screen-space ambient occlusion generating contact shadows from depth buffers.
diff --git a/plugins/framefields/skills/framefields-effects/references/effects-catalog.md b/plugins/framefields/skills/framefields-effects/references/effects-catalog.md
index 613e207b..20a05cd8 100644
--- a/plugins/framefields/skills/framefields-effects/references/effects-catalog.md
+++ b/plugins/framefields/skills/framefields-effects/references/effects-catalog.md
@@ -3,7 +3,7 @@
> Generated by `pnpm run sync:effects-catalog` from the node schemas (via `pnpm run generate:effects`) and `packages/framefields/src/effects/classes.ts`.
> Do not edit by hand — a stale catalog fails `pnpm run check:plugins`.
-52 effect classes are available from the package root and the `/effects` subpath:
+54 effect classes are available from the package root and the `/effects` subpath:
```ts
import { Blur, Curves, FilmGrain } from "framefields";
@@ -34,6 +34,7 @@ Set `FRAMEFIELDS_STRICT_PROPS=1` to make an unknown prop or an out-of-range valu
| `ColorBalance` | `ColorBalance` | `shadows`, `midtones`, `highlights`, `preserveLuminosity` |
| `ColorKey` | `ColorKey` | `keyColor`, `similarity`, `smoothness`, `spillSuppression`, `colorSpace`, `spillSuppressionType` |
| `CornerPin` | `CornerPin` | `points` |
+| `Datamosh` | `Datamosh` | `keyframes`, `strength`, `bleed`, `warmup` |
| `DisplacementMap` | `DisplacementMap` | `strengthX`, `strengthY`, `xChannel`, `yChannel`, `wrapMode` |
| `ExtractLUT` | `ExtractLUT` | `strategy`, `samplePoints` |
| `Flip` | `Flip` | `horizontal`, `vertical`, `diagonal`, `mode`, `symmetry` |
@@ -51,6 +52,7 @@ Set `FRAMEFIELDS_STRICT_PROPS=1` to make an unknown prop or an out-of-range valu
| `ResizerScaler` | `ResizerScaler` | `aspectRatioPreset`, `resolutionPreset`, `targetWidth`, `targetHeight`, `fitMode`, `zoom`, `offsetX`, `offsetY`, `backgroundMode`, `backgroundColor`, `backgroundColor2`, `blurRadius`, `backgroundBrightness`, `anchorX`, `anchorY` |
| `Rotate` | `Rotate` | `angle`, `fit`, `scale` |
| `StereoPanning` | `StereoPanning` | `pan` |
+| `Trails` | `Trails` | `decay`, `mode`, `zoom`, `rotation`, `offsetX`, `offsetY`, `hueShift`, `warmup` |
| `Vignette` | `Vignette` | `strength`, `radius`, `softness`, `roundness`, `centerX`, `centerY` |
| `Levels` | `Levels` | `master`, `red`, `green`, `blue` |
| `FilmGrain` | `FilmGrain` | `strength`, `size`, `monochrome`, `animated`, `speed`, `shadows`, `midtones`, `highlights` |
@@ -283,6 +285,21 @@ new CornerPin({ /* props */ });
| --- | --- | --- |
| `points` | `CornerPinPointsItem[]` | — |
+### Datamosh
+
+```ts
+import { Datamosh } from "framefields/effects";
+
+new Datamosh({ strength: 1, bleed: 0, warmup: 48 });
+```
+
+| Prop | Type | Default | Notes |
+| --- | --- | --- | --- |
+| `keyframes` | `number[]` | `[]` | Layer frames that show the clean picture again (I-frames). The first frame always does. |
+| `strength` | `EffectProp<number>` | `1` | Range 0 to 10. How far pixels are dragged along the motion. |
+| `bleed` | `EffectProp<number>` | `0` | Range 0 to 1. Clean picture mixed back in per frame, 0–1. |
+| `warmup` | `number` | `48` | Range 1 to 240. Most frames replayed to rebuild the smear after a jump. |
+
### DisplacementMap
```ts
@@ -616,6 +633,25 @@ new StereoPanning({ pan: 0 });
| --- | --- | --- | --- |
| `pan` | `EffectProp<number>` | `0` | Range -1 to 1. Stereo panning value: -1 (full left) to 1 (full right), 0 is center. Can be modulated by a static number or dynamic signal. |
+### Trails
+
+```ts
+import { Trails } from "framefields/effects";
+
+new Trails({ decay: 0.9, mode: "blend", zoom: 1 });
+```
+
+| Prop | Type | Default | Notes |
+| --- | --- | --- | --- |
+| `decay` | `EffectProp<number>` | `0.9` | Range 0 to 1. History kept per frame, 0–1: 0.8 short echoes, 0.97 long light trails. |
+| `mode` | `"blend" \| "over" \| "add" \| "max"` | `"blend"` | blend: running average (echoes on full-frame footage); over: the layer over its fading past (cutouts, text); add: glows build up (light painting); max: keeps the brightest. |
+| `zoom` | `EffectProp<number>` | `1` | Range 0.5 to 2. History scale per frame; 1.02 pulls the past into a tunnel. |
+| `rotation` | `EffectProp<number>` | `0` | Range -45 to 45. History rotation per frame, degrees. |
+| `offsetX` | `EffectProp<number>` | `0` | Range -500 to 500. History drift per frame, px. |
+| `offsetY` | `EffectProp<number>` | `0` | Range -500 to 500. History drift per frame, px. |
+| `hueShift` | `EffectProp<number>` | `0` | Range -180 to 180. History hue rotation per frame, degrees (psychedelic cycling). |
+| `warmup` | `number` | `60` | Range 0 to 240. Most frames replayed to rebuild the history after a jump (scrubbing, a single frame render). |
+
### Vignette
```ts
@@ -1025,4 +1061,4 @@ new Modulate({ hue: 0, brightness: 1.0, contrast: 1.0 });
Beyond these classes, the engine ships node plugins under `nodes/` (audio DSP, generators,
signal maths, vision). Each is a package named `@framefields/node-<name>`:
-`node-apply-lut`, `node-audio-compressor`, `node-audio-delay`, `node-audio-fade`, `node-audio-noise-gate`, `node-audio-parametric-eq`, `node-audio-reverb`, `node-audio-signal-extractor`, `node-blur`, `node-camera-parallax-3d`, `node-canvas-generator`, `node-caption-editor`, `node-caption-generator`, `node-channel-merger`, `node-channel-splitter`, `node-color-balance`, `node-colorkey`, `node-compositor`, `node-corner-pin`, `node-crop`, `node-curves`, `node-deflicker`, `node-displacement-map`, `node-extract-frame`, `node-extract-lut`, `node-film-grain`, `node-flip`, `node-gradient-map`, `node-halftone-screen`, `node-high-pass`, `node-kenburns`, `node-layer-style`, `node-levels`, `node-liquify`, `node-mask-math`, `node-mesh-warp`, `node-modulate`, `node-noise-generator`, `node-paint`, `node-patch-heal`, `node-procedural-vfx`, `node-refine-edge`, `node-refraction-caustics`, `node-relight-3d`, `node-resizer-scaler`, `node-rotate`, `node-selective-color`, `node-shadows-highlights`, `node-shape-generator`, `node-signal-gate`, `node-signal-math`, `node-signal`, `node-stereo-panning`, `node-text`, `node-tile-offset`, `node-unsharp-mask`, `node-vignette`, `node-vision`
+`node-apply-lut`, `node-audio-compressor`, `node-audio-delay`, `node-audio-fade`, `node-audio-noise-gate`, `node-audio-parametric-eq`, `node-audio-reverb`, `node-audio-signal-extractor`, `node-blur`, `node-camera-parallax-3d`, `node-canvas-generator`, `node-caption-editor`, `node-caption-generator`, `node-channel-merger`, `node-channel-splitter`, `node-color-balance`, `node-colorkey`, `node-compositor`, `node-corner-pin`, `node-crop`, `node-curves`, `node-datamosh`, `node-deflicker`, `node-displacement-map`, `node-extract-frame`, `node-extract-lut`, `node-film-grain`, `node-flip`, `node-gradient-map`, `node-halftone-screen`, `node-high-pass`, `node-kenburns`, `node-layer-style`, `node-levels`, `node-liquify`, `node-mask-math`, `node-mesh-warp`, `node-modulate`, `node-noise-generator`, `node-paint`, `node-particles`, `node-patch-heal`, `node-procedural-vfx`, `node-refine-edge`, `node-refraction-caustics`, `node-relight-3d`, `node-resizer-scaler`, `node-rotate`, `node-selective-color`, `node-shadows-highlights`, `node-shape-generator`, `node-signal-gate`, `node-signal-math`, `node-signal`, `node-stereo-panning`, `node-text`, `node-tile-offset`, `node-trails`, `node-unsharp-mask`, `node-vignette`, `node-vision`
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 653d39d0..28e73282 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -432,28 +432,6 @@ importers:
specifier: ^5.8.3
version: 5.9.3
- examples/e_kofte:
- dependencies:
- framefields:
- specifier: workspace:*
- version: link:../../packages/framefields
- mediabunny:
- specifier: ^1.61.6
- version: 1.61.6
- devDependencies:
- '@framefields/tsconfig':
- specifier: workspace:*
- version: link:../../packages/tsconfig
- '@types/node':
- specifier: ^24.1.0
- version: 24.10.14
- tsx:
- specifier: ^4.20.3
- version: 4.21.0
- typescript:
- specifier: ^5.8.3
- version: 5.9.3
-
nodes/node-apply-lut:
dependencies:
'@framefields/core':
@@ -1090,6 +1068,34 @@ importers:
specifier: ^5.8.3
version: 5.8.3
+ nodes/node-datamosh:
+ dependencies:
+ '@framefields/node-sdk':
+ specifier: workspace:*
+ version: link:../../packages/node-sdk
+ '@framefields/tensor-webgpu':
+ specifier: workspace:*
+ version: link:../../packages/tensor-webgpu
+ zod:
+ specifier: ^4.3.6
+ version: 4.3.6
+ devDependencies:
+ '@framefields/tsconfig':
+ specifier: workspace:*
+ version: link:../../packages/tsconfig
+ '@types/node':
+ specifier: ^24.1.0
+ version: 24.10.14
+ tsdown:
+ specifier: ^0.19.0
+ typescript:
+ specifier: ^5.8.3
+ version: 5.9.3
+ webgpu:
+ specifier: ^0.4.0
+ version: 0.4.0
+
nodes/node-deflicker:
dependencies:
'@framefields/core':
@@ -1624,6 +1630,28 @@ importers:
specifier: ^5.8.3
version: 5.8.3
+ nodes/node-particles:
+ dependencies:
+ '@framefields/node-sdk':
+ specifier: workspace:*
+ version: link:../../packages/node-sdk
+ devDependencies:
+ '@framefields/tsconfig':
+ specifier: workspace:*
+ version: link:../../packages/tsconfig
+ '@types/node':
+ specifier: ^24.1.0
+ version: 24.10.14
+ tsdown:
+ specifier: ^0.19.0
+ typescript:
+ specifier: ^5.8.3
+ version: 5.9.3
+ webgpu:
+ specifier: ^0.4.0
+ version: 0.4.0
+
nodes/node-patch-heal:
dependencies:
'@framefields/core':
@@ -2108,6 +2136,31 @@ importers:
specifier: ^0.4.0
version: 0.4.0
+ nodes/node-trails:
+ dependencies:
+ '@framefields/node-sdk':
+ specifier: workspace:*
+ version: link:../../packages/node-sdk
+ zod:
+ specifier: ^4.3.6
+ version: 4.3.6
+ devDependencies:
+ '@framefields/tsconfig':
+ specifier: workspace:*
+ version: link:../../packages/tsconfig
+ '@types/node':
+ specifier: ^24.1.0
+ version: 24.10.14
+ tsdown:
+ specifier: ^0.19.0
+ typescript:
+ specifier: ^5.8.3
+ version: 5.9.3
+ webgpu:
+ specifier: ^0.4.0
+ version: 0.4.0
+
nodes/node-unsharp-mask:
dependencies:
'@framefields/core':
@@ -2465,6 +2518,9 @@ importers:
'@framefields/node-paint':
specifier: workspace:*
version: link:../../nodes/node-paint
+ '@framefields/node-particles':
+ specifier: workspace:*
+ version: link:../../nodes/node-particles
'@framefields/node-patch-heal':
specifier: workspace:*
version: link:../../nodes/node-patch-heal
@@ -2662,6 +2718,9 @@ importers:
'@framefields/node-curves':
specifier: workspace:*
version: link:../../nodes/node-curves
+ '@framefields/node-datamosh':
+ specifier: workspace:*
+ version: link:../../nodes/node-datamosh
'@framefields/node-deflicker':
specifier: workspace:*
version: link:../../nodes/node-deflicker
@@ -2719,6 +2778,9 @@ importers:
'@framefields/node-paint':
specifier: workspace:*
version: link:../../nodes/node-paint
+ '@framefields/node-particles':
+ specifier: workspace:*
+ version: link:../../nodes/node-particles
'@framefields/node-patch-heal':
specifier: workspace:*
version: link:../../nodes/node-patch-heal
@@ -2758,6 +2820,9 @@ importers:
'@framefields/node-tile-offset':
specifier: workspace:*
version: link:../../nodes/node-tile-offset
+ '@framefields/node-trails':
+ specifier: workspace:*
+ version: link:../../nodes/node-trails
'@framefields/node-unsharp-mask':
specifier: workspace:*
version: link:../../nodes/node-unsharp-mask