Skip to content

Code-first video & image, rendered natively on WebGPU.

feat(lottie): support Layer.lottie with supersampling, warn on unsupported image formats

okanasl933d8ab6a4ba
diff --git a/README.md b/README.md
index d8fe78a8..8559b72a 100644
--- a/README.md
+++ b/README.md
@@ -174,7 +174,25 @@ Layer.chart(
);
```
-### 6. On-Device Vision & Tracking
+### 6. Vector Animations & Lottie
+[`Layer.lottie`](packages/framefields/src/index.ts) plays Lottie JSON (`.json`) and dotLottie (`.lottie`) vector animations natively via ThorVG without browser overhead:
+- **Headless vector decoding** — renders directly on WebGPU in Node.js via WASM/ThorVG.
+- **Unified compositing** — positions with flex/box layout, animates with `LayerAnimation`, tilts into 3D scenes, masks (`.withMatte`), and applies GPU post-processing effects.
+- **Configurable playback** — supports `fit` (`"contain"` | `"cover"` | `"fill"`), `loop`, `speed`, `autoplay`, and lifecycle frames.
+- **Strict format boundary** — raster formats (`.png`, `.jpg`, `.webp`) use `Layer.image()`; passing `.json` or `.lottie` to `Layer.image()` throws/warns and is flagged by `comp.lint()`.
+
+```typescript
+comp.add(
+ Layer.lottie("assets/animations/badge.json", {
+ width: 400,
+ height: 400,
+ fit: "contain",
+ loop: true,
+ })
+);
+```
+
+### 7. On-Device Vision & Tracking
[`@framefields/vision`](packages/vision) runs ONNX models via `onnxruntime-node` (CPU) or `onnxruntime-web` (WebGPU) and wires every result into the same reactive signal surface the rest of Framefields consumes.
> [!TIP]
@@ -240,11 +258,11 @@ Project normalized landmarks to screen space with a configurable camera FOV, the
- **Clear failures** — a model that is the wrong size, fails its checksum, or lacks an expected output raises an error naming the model and its source.
- **Browser entry** — `@framefields/vision/web` re-exports the engine plus `createWebGPUProvider()` / `hasWebGPU()`; `onnxruntime-web` is an optional lazy peer.
-### 7. Headless Conformance & FrameGrid Testing
+### 8. Headless Conformance & FrameGrid Testing
- **Pixel-sampling invariant assertions** — test compositions in Vitest with `skia-canvas` to verify shader math, font coverage, and Mean Squared Error (MSE) temporal deltas.
- **FrameGrid contact sheets** — `comp.renderFrameGrid(...)` outputs sequential-frame contact sheets for instant review of easing, kinetic type, and transitions.
-### 8. Live Preview in the Browser
+### 9. Live Preview in the Browser
- **Runs your composition, not a video** — `startPreview({ entry, export })` serves a localhost WebGPU player that loads the composition's own module and renders its frames live in the browser. Nothing is streamed: the server only hands over the bundle, the project's assets, and the soundtrack mixed by the export engine.
- **Timeline, waveform & frame stepping** — play/pause, scrub, step frame by frame, and read resolution, FPS, duration, and audio status at a glance.
- **One stable URL per project** — the port is derived from the working directory, so re-running the preview replaces the running server and any open tab reloads into the new version by itself. Close the tab and the server shuts down about five seconds later. A reload comes back at the same moment, and `#t=12.5` links to one.
@@ -587,6 +605,7 @@ console.log(`Video rendered successfully to: ${filePath}`);
3. **Carrier match cuts** — carry a visual element (badge, card, cursor, container) across scene boundaries with continuous velocity and position to avoid jarring cuts.
4. **Physical easing vocabulary** — `back.out(1.4–1.7)` for snap-overshoot entrances, `spring` / `expo.out` for decelerating motion, `power2.in` for exits. Reserve `linear` for infinite spinners and time counters.
5. **Headless invariant verification** — verify shader transforms, glyph coverage, and temporal MSE deltas with `skia-canvas` pixel sampling in Vitest before shipping.
+6. **Image vs. Lottie layers** — raster images (`.png`, `.jpg`, `.webp`) use `Layer.image()`. Vector animations (`.json`, `.lottie`) must use `Layer.lottie()` — passing `.json` or `.lottie` to `Layer.image()` throws/warns and is rejected by `comp.lint()` with `unsupported_image_format`.
---
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index 11a71316..f75dca32 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -3287,6 +3287,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
durationMs: layerDurationInMS,
opacity: 1.0,
fps,
+ ...(typeof lop.dpr === "number" && { dpr: lop.dpr }),
// A media layer's effect chain (decode → passes) allocates from the
// shared pool: keep it off the destination this layer composites onto,
// or the raw frame lands there wherever the clip does not cover it.
diff --git a/packages/compositions/src/compositions/canvas-composition.ts b/packages/compositions/src/compositions/canvas-composition.ts
index 5355119d..4c6d24d4 100644
--- a/packages/compositions/src/compositions/canvas-composition.ts
+++ b/packages/compositions/src/compositions/canvas-composition.ts
@@ -506,6 +506,12 @@ export async function drawCompositionTree(
fps,
opacity: Number(op.opacity ?? 1) * Number(props.opacity ?? 1),
isHeadless: props.isHeadless,
+ dpr:
+ typeof (props as { dpr?: unknown }).dpr === "number"
+ ? (props as { dpr: number }).dpr
+ : typeof (op as { dpr?: unknown }).dpr === "number"
+ ? (op as { dpr: number }).dpr
+ : undefined,
});
} else if (mediaType === "GIF") {
await drawGifNode(ctx, pass, {
diff --git a/packages/compositions/src/program/lint.test.ts b/packages/compositions/src/program/lint.test.ts
index 7f64c8de..f7cb19c5 100644
--- a/packages/compositions/src/program/lint.test.ts
+++ b/packages/compositions/src/program/lint.test.ts
@@ -49,13 +49,17 @@ describe("lintComposition", () => {
{ id: "photo", kind: "image", src: "a.png" },
{ id: "t", kind: "text", fill: "#ffffff" },
];
- expect(codes(overImage, { backgroundColor: "#ffffff" })).not.toContain("low_contrast_text");
+ expect(codes(overImage, { backgroundColor: "#ffffff" })).not.toContain(
+ "low_contrast_text",
+ );
const onPanel = [
{ id: "panel", kind: "box", background: "#ffffff" },
{ id: "t", kind: "text", fill: "#ffffff", zIndex: -1 },
];
// Drawn before the panel (lower z): the backdrop is still known.
- expect(codes(onPanel, { backgroundColor: "#ffffff" })).toContain("low_contrast_text");
+ expect(codes(onPanel, { backgroundColor: "#ffffff" })).toContain(
+ "low_contrast_text",
+ );
});
it("flags static layers parked off-canvas, but not ones that move in", () => {
@@ -125,4 +129,47 @@ describe("lintComposition", () => {
]),
).toEqual(["static_track"]);
});
+
+ it("flags unsupported formats given to image layers", () => {
+ expect(
+ codes([
+ {
+ id: "img1",
+ kind: "media",
+ dataType: "Image",
+ inputHandleId: "anim.json",
+ },
+ ]),
+ ).toEqual(["unsupported_image_format"]);
+ expect(
+ codes([
+ {
+ id: "img2",
+ kind: "media",
+ dataType: "Image",
+ inputHandleId: "anim.lottie",
+ },
+ ]),
+ ).toEqual(["unsupported_image_format"]);
+ expect(
+ codes([
+ {
+ id: "img3",
+ kind: "media",
+ dataType: "Image",
+ inputHandleId: "video.mp4",
+ },
+ ]),
+ ).toEqual(["unsupported_image_format"]);
+ expect(
+ codes([
+ {
+ id: "img4",
+ kind: "media",
+ dataType: "Image",
+ inputHandleId: "valid.png",
+ },
+ ]),
+ ).toEqual([]);
+ });
});
diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.ts
index 7989a30f..c09fcacf 100644
--- a/packages/compositions/src/program/lint.ts
+++ b/packages/compositions/src/program/lint.ts
@@ -28,9 +28,7 @@ const COLOR_PROPS = [
/** Contrast below this makes text hard to read (WCAG large-text minimum). */
const MIN_TEXT_CONTRAST = 3;
-function tracksOf(
- node: AnyNode,
-): Array<{
+function tracksOf(node: AnyNode): Array<{
prop: string;
keyframes?: Array<{ value: unknown }>;
source?: { type?: string };
@@ -55,9 +53,7 @@ function paints(node: AnyNode): boolean {
}
const byZ = (nodes: AnyNode[]) =>
- [...nodes].sort(
- (a, b) => (Number(a.zIndex) || 0) - (Number(b.zIndex) || 0),
- );
+ [...nodes].sort((a, b) => (Number(a.zIndex) || 0) - (Number(b.zIndex) || 0));
function solidColor(value: unknown): string | null {
if (typeof value !== "string" || value.includes("gradient(")) return null;
@@ -129,6 +125,37 @@ export function lintComposition(
}
if (node.kind === "camera") return;
+ if (node.kind === "media") {
+ const src = node.inputHandleId;
+ if (typeof src === "string") {
+ const dataType = node.dataType ?? "Video";
+ if (dataType === "Image") {
+ if (/\.(json|lottie)(\?.*)?$/i.test(src)) {
+ push(
+ "error",
+ "unsupported_image_format",
+ `'${id}' is an image layer with source '${src}', which is a Lottie vector animation. Use Layer.lottie() instead.`,
+ id,
+ );
+ } else if (/\.(mp4|mov|webm|mkv)(\?.*)?$/i.test(src)) {
+ push(
+ "error",
+ "unsupported_image_format",
+ `'${id}' is an image layer with video source '${src}'. Use Layer.video() instead.`,
+ id,
+ );
+ } else if (/\.(mp3|wav|ogg|m4a|aac)(\?.*)?$/i.test(src)) {
+ push(
+ "error",
+ "unsupported_image_format",
+ `'${id}' is an image layer with audio source '${src}'. Use Layer.audio() instead.`,
+ id,
+ );
+ }
+ }
+ }
+ }
+
const is3D =
Boolean(node.is3D) || (typeof node.z === "number" && node.z !== 0);
if (is3D || node.kind === "model3d") has3D = true;
diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.ts
index 5882975d..32690e4d 100644
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@ -925,6 +925,7 @@ export interface MediaNode extends CompositionNodeBase {
inputHandleId: string;
dataType?: string;
fit?: "cover" | "contain" | "fill" | "none";
+ dpr?: number;
borderRadius?: number;
borderColor?: string;
borderWidth?: number;
@@ -1438,6 +1439,7 @@ export const MediaNodeSchema: z.ZodType<MediaNode> = z
inputHandleId: z.string().min(1),
dataType: z.string().optional(),
fit: z.enum(["cover", "contain", "fill", "none"]).default("contain"),
+ dpr: z.number().min(1).max(4).optional(),
borderRadius: z.number().min(0).optional(),
borderColor: ColorSchema.optional(),
borderWidth: z.number().min(0).optional(),
diff --git a/packages/compositions/src/program/to-virtual-media.test.ts b/packages/compositions/src/program/to-virtual-media.test.ts
index 6daf364e..6a8fb5a6 100644
--- a/packages/compositions/src/program/to-virtual-media.test.ts
+++ b/packages/compositions/src/program/to-virtual-media.test.ts
@@ -239,4 +239,42 @@ describe("compositorToProgram", () => {
expect((card.operation as any).durationFrames).toBe(144);
expect((card.children![0].operation as any).startFrame).toBe(490);
});
+
+ it("resolves .json and .lottie files as direct media sources with Lottie dataType", () => {
+ const doc: CompositorProgramConfig = {
+ width: 1920,
+ height: 1080,
+ fps: 30,
+ mode: "Video",
+ layout: [
+ {
+ id: "lottie1",
+ kind: "media",
+ inputHandleId: "animation.json",
+ dataType: "Lottie",
+ },
+ {
+ id: "lottie2",
+ kind: "media",
+ inputHandleId: "animation.lottie",
+ dataType: "Lottie",
+ },
+ ],
+ } as any;
+
+ const vm = compositorToProgram(doc, { isVideoMode: true, fps: 30 });
+ expect(vm.children).toHaveLength(2);
+
+ const layer1 = vm.children![0];
+ expect(layer1.children).toHaveLength(1);
+ expect((layer1.children![0].operation as any).op).toBe("source");
+ expect((layer1.children![0].operation as any).dataType).toBe("Lottie");
+ expect((layer1.children![0].operation as any).url).toBe("animation.json");
+
+ const layer2 = vm.children![1];
+ expect(layer2.children).toHaveLength(1);
+ expect((layer2.children![0].operation as any).op).toBe("source");
+ expect((layer2.children![0].operation as any).dataType).toBe("Lottie");
+ expect((layer2.children![0].operation as any).url).toBe("animation.lottie");
+ });
});
diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.ts
index fe83b558..bc70e23b 100644
--- a/packages/compositions/src/program/to-virtual-media.ts
+++ b/packages/compositions/src/program/to-virtual-media.ts
@@ -54,7 +54,7 @@ function nodeToVirtualMedia(
s.startsWith("./") ||
s.startsWith("blob:") ||
s.startsWith("data:") ||
- /\.(png|jpe?g|webp|gif|svg|mp4|webm|mov|mkv|mp3|wav|ogg|m4a|srt|vtt)$/i.test(
+ /\.(png|jpe?g|webp|gif|svg|mp4|webm|mov|mkv|mp3|wav|ogg|m4a|srt|vtt|json|lottie)$/i.test(
s,
));
@@ -73,6 +73,16 @@ function nodeToVirtualMedia(
(node.dataType as DataType) ?? "Video",
);
}
+ if (
+ !media &&
+ !resolveMedia &&
+ sourceRef &&
+ (sourceRef.includes(".") || sourceRef.includes("/"))
+ ) {
+ console.warn(
+ `[Compositor] Media layer '${node.id ?? "?"}' source '${sourceRef}' could not be resolved.`,
+ );
+ }
if (media && (node as any).crop) {
const cropRaw = (node as any).crop;
const cropOp =
diff --git a/packages/framefields/src/builder-schema-conformance.test.ts b/packages/framefields/src/builder-schema-conformance.test.ts
index 18d53743..a1c0fca2 100644
--- a/packages/framefields/src/builder-schema-conformance.test.ts
+++ b/packages/framefields/src/builder-schema-conformance.test.ts
@@ -15,6 +15,7 @@ const builders = {
video: () => Layer.video("clip.mp4"),
image: () => Layer.image("still.png"),
svg: () => Layer.svg("logo.svg"),
+ lottie: () => Layer.lottie("anim.json"),
audio: () => Layer.audio("track.mp3"),
text: () => Layer.text("Hello", { align: "start" }),
richText: () => Layer.text(["Hello ", { text: "world", fontWeight: 700 }]),
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 85c357b0..afe053d9 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -849,6 +849,15 @@ export class Media {
return m;
}
+ static lottie(
+ source: MediaSourceInput,
+ meta?: Partial<MediaMetadata>,
+ ): Media {
+ const m = new Media(source, "Lottie");
+ if (meta) Object.assign(m.virtualMedia.metadata, meta);
+ return m;
+ }
+
static audio(source: MediaSourceInput, meta?: Partial<MediaMetadata>): Media {
const m = new Media(source, "Audio");
if (meta) Object.assign(m.virtualMedia.metadata, meta);
@@ -1840,8 +1849,7 @@ export class Composition {
...result.qa,
issues,
// Lint warnings advise; only a lint error fails the render.
- passed:
- result.qa.passed && !lint.some((d) => d.severity === "error"),
+ passed: result.qa.passed && !lint.some((d) => d.severity === "error"),
};
}
return deliverToOutputPath(result, opts.outputPath);
@@ -2231,15 +2239,34 @@ export const Layer = {
image: (
src: string,
options: Partial<MediaNode> = {},
- ): AnimatableNode<MediaNode> =>
- withAnimation({
+ ): AnimatableNode<MediaNode> => {
+ if (typeof src === "string") {
+ if (
+ /\.(json|lottie)(\?.*)?$/i.test(src) ||
+ src.startsWith("data:application/json")
+ ) {
+ console.warn(
+ `[Layer.image] '${src}' is a Lottie vector animation, not a static image. Use Layer.lottie('${src}', options) instead.`,
+ );
+ } else if (/\.(mp4|mov|webm|mkv)(\?.*)?$/i.test(src)) {
+ console.warn(
+ `[Layer.image] '${src}' is a video file, not a static image. Use Layer.video('${src}', options) instead.`,
+ );
+ } else if (/\.(mp3|wav|ogg|m4a|aac)(\?.*)?$/i.test(src)) {
+ console.warn(
+ `[Layer.image] '${src}' is an audio file, not a static image. Use Layer.audio('${src}', options) instead.`,
+ );
+ }
+ }
+ return withAnimation({
id: options.id ?? autoId("img"),
kind: "media",
inputHandleId: src,
dataType: "Image",
fit: options.fit ?? "cover",
...options,
- }),
+ });
+ },
svg: (
src: string,
options: Partial<MediaNode> = {},
diff --git a/packages/framefields/src/lottie-layer.test.ts b/packages/framefields/src/lottie-layer.test.ts
new file mode 100644
index 00000000..756c282c
--- /dev/null
+++ b/packages/framefields/src/lottie-layer.test.ts
@@ -0,0 +1,82 @@
+import { describe, expect, it, vi } from "vitest";
+import { Composition, Layer, Media } from "./index.js";
+
+describe("Layer.lottie & unsupported format handling", () => {
+ it("resolves Layer.lottie with relative path into a Lottie source operation", () => {
+ const comp = new Composition({ width: 400, height: 400, fps: 30 });
+ comp.add(Layer.lottie("assets/spinner.json", { width: 200, height: 200 }));
+
+ const vm = comp.toVirtualMedia();
+ expect(vm.children).toHaveLength(1);
+
+ const layer = vm.children?.[0];
+ expect(layer).toBeDefined();
+ const op = layer?.operation as Record<string, unknown>;
+ expect(op.kind).toBe("media");
+ expect(op.dataType).toBe("Lottie");
+ expect(layer?.children).toHaveLength(1);
+
+ const sourceOp = layer?.children?.[0]?.operation as Record<string, unknown>;
+ expect(sourceOp.op).toBe("source");
+ expect(sourceOp.dataType).toBe("Lottie");
+ expect(sourceOp.url).toBe("assets/spinner.json");
+ });
+
+ it("creates Media.lottie with Lottie dataType", () => {
+ const media = Media.lottie("animation.json");
+ expect(media.virtualMedia.operation.dataType).toBe("Lottie");
+ expect(media.virtualMedia.operation.op).toBe("source");
+ });
+
+ it("warns when Layer.image is called with a Lottie .json file", () => {
+ const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
+ try {
+ Layer.image("animation.json");
+ expect(warnSpy).toHaveBeenCalledWith(
+ expect.stringContaining(
+ "is a Lottie vector animation, not a static image. Use Layer.lottie(",
+ ),
+ );
+ } finally {
+ warnSpy.mockRestore();
+ }
+ });
+
+ it("warns when Layer.image is called with a .lottie archive", () => {
+ const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});
+ try {
+ Layer.image("badge.lottie");
+ expect(warnSpy).toHaveBeenCalledWith(
+ expect.stringContaining(
+ "is a Lottie vector animation, not a static image. Use Layer.lottie(",
+ ),
+ );
+ } finally {
+ warnSpy.mockRestore();
+ }
+ });
+
+ it("comp.lint() flags Layer.image with .json as unsupported_image_format", () => {
+ const comp = new Composition({ width: 1920, height: 1080 });
+ comp.add(Layer.image("hero-animation.json"));
+
+ const diagnostics = comp.lint();
+ const issue = diagnostics.find(
+ (d) => d.code === "unsupported_image_format",
+ );
+ expect(issue).toBeDefined();
+ expect(issue?.severity).toBe("error");
+ expect(issue?.message).toContain("Use Layer.lottie() instead");
+ });
+
+ it("comp.lint() does not flag valid Layer.lottie", () => {
+ const comp = new Composition({ width: 1920, height: 1080 });
+ comp.add(Layer.lottie("hero-animation.json"));
+
+ const diagnostics = comp.lint();
+ const issue = diagnostics.find(
+ (d) => d.code === "unsupported_image_format",
+ );
+ expect(issue).toBeUndefined();
+ });
+});
diff --git a/packages/webgpu-renderers/src/nodes/image.test.ts b/packages/webgpu-renderers/src/nodes/image.test.ts
index a9ee435c..a286fe64 100644
--- a/packages/webgpu-renderers/src/nodes/image.test.ts
+++ b/packages/webgpu-renderers/src/nodes/image.test.ts
@@ -106,6 +106,15 @@ describe("Image Node", () => {
imageLoader.load("http://test.com/fail.png", true),
).rejects.toThrow();
});
+
+ it("should reject Lottie and JSON animation files with helpful message", async () => {
+ await expect(imageLoader.load("animation.json", true)).rejects.toThrow(
+ "Unsupported image format for 'animation.json'",
+ );
+ await expect(imageLoader.load("animation.lottie", true)).rejects.toThrow(
+ "Unsupported image format for 'animation.lottie'",
+ );
+ });
});
describe("drawImageNode", () => {
diff --git a/packages/webgpu-renderers/src/nodes/image.ts b/packages/webgpu-renderers/src/nodes/image.ts
index 9fcb84e5..d902b9f9 100644
--- a/packages/webgpu-renderers/src/nodes/image.ts
+++ b/packages/webgpu-renderers/src/nodes/image.ts
@@ -18,6 +18,15 @@ class ImageLoader {
let promise = this.pending.get(key);
if (!promise) {
promise = (async () => {
+ if (
+ typeof src === "string" &&
+ (/\.(json|lottie)(\?.*)?$/i.test(src) ||
+ src.startsWith("data:application/json"))
+ ) {
+ throw new Error(
+ `Unsupported image format for '${src}'. Lottie vector animations must be rendered with Layer.lottie(), not Layer.image().`,
+ );
+ }
let arrayBuffer: ArrayBuffer;
const isRemote =
src.startsWith("http://") ||
@@ -83,6 +92,15 @@ export async function drawImageNode(
pass: GPURenderPassEncoder,
props: ImageNodeProps,
): Promise<void> {
+ if (
+ typeof props.src === "string" &&
+ (/\.(json|lottie)(\?.*)?$/i.test(props.src) ||
+ props.src.startsWith("data:application/json"))
+ ) {
+ console.warn(
+ `[ImageRenderer] '${props.src}' is a Lottie vector animation, not a static image. Use Layer.lottie() instead.`,
+ );
+ }
let tex = textureCache.acquire(props.src);
if (!tex) {
diff --git a/packages/webgpu-renderers/src/nodes/lottie.ts b/packages/webgpu-renderers/src/nodes/lottie.ts
index b04ee628..e5ac6cd6 100644
--- a/packages/webgpu-renderers/src/nodes/lottie.ts
+++ b/packages/webgpu-renderers/src/nodes/lottie.ts
@@ -214,13 +214,18 @@ class LottieInstance {
private _width: number;
private _height: number;
- constructor(src: string, width: number, height: number) {
+ constructor(
+ src: string,
+ width: number,
+ height: number,
+ data?: string | ArrayBuffer,
+ ) {
this._width = width;
this._height = height;
this.canvas = new OffscreenCanvas(width, height);
this.dotLottie = new DotLottie({
canvas: this.canvas,
- src: resolveUrl(src),
+ ...(data ? { data } : { src: resolveUrl(src) }),
autoplay: false,
loop: false,
speed: 0,
@@ -242,7 +247,7 @@ class LottieInstance {
reject(
e instanceof Error
? e
- : new Error("Failed to load DotLottie animation"),
+ : new Error(`Failed to load DotLottie animation from '${src}'`),
);
});
});
@@ -312,20 +317,23 @@ function polyfillHeadlessDOM(): void {
}
}
- const docObj = (globalThis.document ??= {
- createElement: (tag: string) => {
- if (
- tag === "canvas" &&
- typeof globalThis.OffscreenCanvas === "function"
- ) {
- return new globalThis.OffscreenCanvas(1, 1);
- }
- return { ...dummyElement };
- },
- getElementsByTagName: () => [],
- documentElement: dummyElement,
- body: dummyElement,
- } as unknown as Document) as unknown as Record<string, unknown>;
+ if (!globalThis.document) {
+ (globalThis as unknown as Record<string, unknown>).document = {
+ createElement: (tag: string) => {
+ if (
+ tag === "canvas" &&
+ typeof globalThis.OffscreenCanvas === "function"
+ ) {
+ return new globalThis.OffscreenCanvas(1, 1);
+ }
+ return { ...dummyElement };
+ },
+ getElementsByTagName: () => [],
+ documentElement: dummyElement,
+ body: dummyElement,
+ } as unknown as Document;
+ }
+ const docObj = globalThis.document as unknown as Record<string, unknown>;
docObj.documentElement ??= dummyElement;
docObj.body ??= dummyElement;
@@ -348,6 +356,15 @@ async function ensureWasmConfigured(): Promise<void> {
typeof document.getElementById === "function";
if (!isBrowser) {
+ if (typeof globalThis.OffscreenCanvas === "undefined") {
+ try {
+ const { Canvas } = await import(
+ /* webpackIgnore: true */ "skia-canvas"
+ );
+ (globalThis as unknown as Record<string, unknown>).OffscreenCanvas =
+ Canvas;
+ } catch {}
+ }
polyfillHeadlessDOM();
// Add Response instanceof patch to handle cross-realm ESM loader issues in worker threads
if (typeof globalThis.Response === "function") {
@@ -426,25 +443,27 @@ class LottieLoader {
// 1. Parse the Lottie JSON if applicable, extract its fonts and register them.
// This runs before the LottieInstance is created, so the text layers are shaped
// with the registered fonts from the start.
- let resolvedSrc = src;
+ const resolvedSrc = src;
+ let lottieData: string | ArrayBuffer | undefined;
try {
let lottieJson: any = null;
const isNode =
typeof process !== "undefined" && Boolean(process.versions?.node);
+ const isRemote =
+ resolvedSrc.startsWith("http://") ||
+ resolvedSrc.startsWith("https://") ||
+ resolvedSrc.startsWith("data:");
if (resolvedSrc.startsWith("data:application/json;base64,")) {
const base64Data = resolvedSrc.split(",")[1];
const jsonStr = decodeBase64(base64Data);
lottieJson = JSON.parse(jsonStr);
+ lottieData = jsonStr;
} else if (resolvedSrc.startsWith("data:application/json,")) {
const decodeData = decodeURIComponent(resolvedSrc.split(",")[1]);
lottieJson = JSON.parse(decodeData);
- } else if (
- isNode &&
- (resolvedSrc.startsWith("/") ||
- resolvedSrc.startsWith("file://")) &&
- resolvedSrc.endsWith(".json")
- ) {
+ lottieData = decodeData;
+ } else if (isNode && !isRemote) {
let filePath = resolvedSrc;
if (filePath.startsWith("file://")) {
try {
@@ -453,25 +472,35 @@ class LottieLoader {
filePath = filePath.replace(/^file:\/\//, "");
}
}
- const fs = await import(/* webpackIgnore: true */ "fs/promises");
- const jsonContent = await fs.readFile(filePath, "utf-8");
- lottieJson = JSON.parse(jsonContent);
- const base64Data = Buffer.from(jsonContent).toString("base64");
- resolvedSrc = `data:application/json;base64,${base64Data}`;
+ const pathModule = await import(
+ /* webpackIgnore: true */ "node:path"
+ );
+ if (!pathModule.isAbsolute(filePath)) {
+ filePath = pathModule.resolve(process.cwd(), filePath);
+ }
+ try {
+ const fs = await import(/* webpackIgnore: true */ "fs/promises");
+ if (filePath.endsWith(".json")) {
+ const jsonContent = await fs.readFile(filePath, "utf-8");
+ lottieJson = JSON.parse(jsonContent);
+ lottieData = jsonContent;
+ } else {
+ const fileBuf = await fs.readFile(filePath);
+ lottieData = fileBuf.buffer.slice(
+ fileBuf.byteOffset,
+ fileBuf.byteOffset + fileBuf.byteLength,
+ );
+ }
+ } catch {
+ // If file read fails (e.g. mocked path in tests), fall back to resolvedSrc
+ }
} else if (
- resolvedSrc.startsWith("http://") ||
- resolvedSrc.startsWith("https://") ||
- resolvedSrc.startsWith("/")
+ isRemote &&
+ (resolvedSrc.endsWith(".json") || resolvedSrc.includes("/assets/"))
) {
- // Only pre-fetch JSON animations. Binary .lottie zip files pack their own fonts.
- if (
- resolvedSrc.endsWith(".json") ||
- resolvedSrc.includes("/assets/")
- ) {
- const response = await fetch(resolveUrl(resolvedSrc));
- if (response.ok) {
- lottieJson = await response.json();
- }
+ const response = await fetch(resolveUrl(resolvedSrc));
+ if (response.ok) {
+ lottieJson = await response.json();
}
}
@@ -495,6 +524,7 @@ class LottieLoader {
resolvedSrc,
resolvedWidth,
resolvedHeight,
+ lottieData,
);
await instance.ready();
@@ -514,8 +544,17 @@ export async function drawLottieNode(
pass: GPURenderPassEncoder,
props: LottieNodeProps,
): Promise<void> {
- const resolvedWidth = Math.max(1, Math.ceil(props.dstRect.width));
- const resolvedHeight = Math.max(1, Math.ceil(props.dstRect.height));
+ const baseWidth = Math.max(1, Math.ceil(props.dstRect.width));
+ const baseHeight = Math.max(1, Math.ceil(props.dstRect.height));
+ const requestedDpr =
+ typeof props.dpr === "number" && props.dpr > 0 ? props.dpr : 2;
+ const maxDim = Math.max(baseWidth, baseHeight);
+ const maxAllowedDpr = Math.max(1, Math.floor(4096 / Math.max(1, maxDim)));
+ const effectiveDpr = Math.min(requestedDpr, maxAllowedDpr);
+
+ const resolvedWidth = Math.max(1, Math.round(baseWidth * effectiveDpr));
+ const resolvedHeight = Math.max(1, Math.round(baseHeight * effectiveDpr));
+
const instance = await lottieLoader.load(
props.src,
resolvedWidth,
diff --git a/packages/webgpu-renderers/src/nodes/types.ts b/packages/webgpu-renderers/src/nodes/types.ts
index 411438c8..951ec6cc 100644
--- a/packages/webgpu-renderers/src/nodes/types.ts
+++ b/packages/webgpu-renderers/src/nodes/types.ts
@@ -45,6 +45,7 @@ export interface LottieNodeProps {
animationId?: string;
stateMachineId?: string;
theme?: string;
+ dpr?: number;
}
export interface SVGNodeProps {
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index 7714a47d..c6fd0947 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -88,9 +88,9 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani
| `Layer.box(options)` | `"box"` | Rectangular container / card. Options: `background`, `borderRadius`, `borderColor`, `borderWidth`, `overflow: "hidden"`, `padding`, `children`. |
| `Layer.flex(options)` | `"flex"` | Auto-layout container (Yoga). Options: `dir: "row" \| "column"`, `gap`, `justify`, `align`, `padding`, `wrap`, `children`. |
| `Layer.video(src, options)` | `"media"` | Video layer decoded via mediabunny. Options: `fit: "cover" \| "contain"`, `trimStartSec`, `muted`, `volume`. |
-| `Layer.image(src, options)` | `"media"` | Static raster image (`.png`, `.jpg`, `.webp`). Options: `fit: "cover" \| "contain"`. |
+| `Layer.image(src, options)` | `"media"` | Static raster image (`.png`, `.jpg`, `.webp`). Options: `fit: "cover" \| "contain"`. For vector animations (`.json`, `.lottie`), use `Layer.lottie`. |
| `Layer.svg(src, options)` | `"media"` | Vector SVG asset. Options: `fit: "contain"`. |
-| `Layer.lottie(src, options)` | `"media"` | Lottie vector animation file. |
+| `Layer.lottie(src, options)` | `"media"` | Lottie vector animation file (`.json`, `.lottie`). Decoded and rendered via ThorVG. Options: `fit: "contain" \| "cover" \| "fill"`, `width`, `height`. |
| `Layer.audio(src, options)` | `"media"` | Audio track layer. Options: `volume`, `startFrame`, `durationFrames`. |
| `Layer.shape(type, options)` | `"shape"` | Parametric vector graphics (`"rect"`, `"circle"`, `"ellipse"`, `"polygon"`, `"star"`, `"arrow"`, `"path"`). Options: `fill` (a color or CSS gradient), `d` (SVG path data with arcs), `strokeWidth`, `strokeColor`, `trimStart`, `trimEnd`. |
| `Layer.caption(src, options)` | `"media"` | SRT-driven synchronized subtitle engine. Bottom-anchored auto-wrapping cues. |
@@ -380,3 +380,4 @@ Layer3D.cylinder({ ..., color: "#111", emissive: "#4da3ff", emissiveIntensity: 3
7. **Audio Layer Duration:** Every `Layer.audio()` must be assigned `durationFrames` to prevent audio truncation in headless rendering.
8. **Premultiplied Alpha:** Alpha shaders operate with premultiplied alpha. Never use `"src-alpha"` factor in custom blends.
9. **Lint before rendering:** `comp.lint()` lists colors/gradients that would draw transparent, low-contrast text, off-canvas layers, shadow settings that do nothing and `roughness`/`metallic` without an environment. `renderVideo({ qa: true })` includes these findings.
+10. **Lottie vs. Image Layers:** Use `Layer.lottie("file.json" | "file.lottie", { width, height, fit })` for Lottie vector animations. Never pass `.json` or `.lottie` to `Layer.image()`, which only decodes raster formats and throws/warns if given a vector animation.
diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.md
index 5d9f9c0d..d8df0995 100644
--- a/plugins/framefields/skills/framefields-render/SKILL.md
+++ b/plugins/framefields/skills/framefields-render/SKILL.md
@@ -252,6 +252,7 @@ if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop
| `audio_silence` | info | ≥ `minSilenceSec` (2 s) below `silenceDbfs` (-60) |
| `audio_silent` | info | The whole track is silent |
| `invalid_gradient` | error | A `background` / `fill` gradient string that does not parse: it draws nothing |
+| `unsupported_image_format` | error | An image layer receiving a `.json` / `.lottie` vector animation, video, or audio file |
| `unknown_color` | warning | A color string that is not a color: it draws transparent |
| `low_contrast_text` | warning | Text below 3:1 contrast against the solid color behind it (suggests a readable one) |
| `offscreen_layer` | warning | A static absolute layer entirely outside the canvas |
diff --git a/plugins/framefields/skills/framefields/SKILL.md b/plugins/framefields/skills/framefields/SKILL.md
index 4ed51691..bbc33609 100644
--- a/plugins/framefields/skills/framefields/SKILL.md
+++ b/plugins/framefields/skills/framefields/SKILL.md
@@ -217,7 +217,7 @@ Add scripts to `package.json`:
### Video assets & Git LFS
-Video inputs (`Layer.video("assets/footage/clip.mp4")`), audio tracks (`Layer.audio(...)`), LUTs, and custom fonts belong in **Git LFS**, while render outputs in `output/` belong in `.gitignore`.
+Video inputs (`Layer.video("assets/footage/clip.mp4")`), Lottie animations (`Layer.lottie(...)`), audio tracks (`Layer.audio(...)`), LUTs, and custom fonts belong in **Git LFS**, while render outputs in `output/` belong in `.gitignore`.
1. **Initialize Git LFS:**
```bash
@@ -232,6 +232,7 @@ Video inputs (`Layer.video("assets/footage/clip.mp4")`), audio tracks (`Layer.au
assets/**/*.mp3 filter=lfs diff=lfs merge=lfs -text
assets/**/*.wav filter=lfs diff=lfs merge=lfs -text
assets/**/*.cube filter=lfs diff=lfs merge=lfs -text
+ assets/**/*.lottie filter=lfs diff=lfs merge=lfs -text
assets/fonts/**/*.ttf filter=lfs diff=lfs merge=lfs -text
assets/fonts/**/*.otf filter=lfs diff=lfs merge=lfs -text
```
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index ad9c1f8a..06b54414 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -375,7 +375,26 @@ importers:
specifier: ^5.8.3
version: 5.9.3
- examples/26_mac_keyboard:
+ examples/26_mac_keyboard_from_3d_primitives:
+ dependencies:
+ framefields:
+ specifier: workspace:*
+ version: link:../../packages/framefields
+ devDependencies:
+ '@framefields/tsconfig':
+ specifier: workspace:*
+ version: link:../../packages/tsconfig
+ '@types/node':
+ specifier: ^24.1.0
+ version: 24.10.14
+ tsx:
+ specifier: ^4.20.3
+ version: 4.21.0
+ typescript:
+ specifier: ^5.8.3
+ version: 5.9.3
+
+ examples/27_rocket_science:
dependencies:
framefields:
specifier: workspace:*