Code-first video & image, rendered natively on WebGPU.
feat(lottie): support Layer.lottie with supersampling, warn on unsupported image formats
okanasl933d8ab6a4ba
diff --git a/README.md b/README.mdindex d8fe78a8..8559b72a 100644--- a/README.md+++ b/README.md@@ -174,7 +174,25 @@ Layer.chart();```-### 6. On-Device Vision & Tracking+### 6. Vector Animations & Lottie+[`Layer.lottie`](packages/framefields/src/index.ts) plays Lottie JSON (`.json`) and dotLottie (`.lottie`) vector animations natively via ThorVG without browser overhead:+- **Headless vector decoding** — renders directly on WebGPU in Node.js via WASM/ThorVG.+- **Unified compositing** — positions with flex/box layout, animates with `LayerAnimation`, tilts into 3D scenes, masks (`.withMatte`), and applies GPU post-processing effects.+- **Configurable playback** — supports `fit` (`"contain"` | `"cover"` | `"fill"`), `loop`, `speed`, `autoplay`, and lifecycle frames.+- **Strict format boundary** — raster formats (`.png`, `.jpg`, `.webp`) use `Layer.image()`; passing `.json` or `.lottie` to `Layer.image()` throws/warns and is flagged by `comp.lint()`.++```typescript+comp.add(+ Layer.lottie("assets/animations/badge.json", {+ width: 400,+ height: 400,+ fit: "contain",+ loop: true,+ })+);+```++### 7. On-Device Vision & Tracking[`@framefields/vision`](packages/vision) runs ONNX models via `onnxruntime-node` (CPU) or `onnxruntime-web` (WebGPU) and wires every result into the same reactive signal surface the rest of Framefields consumes.> [!TIP]@@ -240,11 +258,11 @@ Project normalized landmarks to screen space with a configurable camera FOV, the- **Clear failures** — a model that is the wrong size, fails its checksum, or lacks an expected output raises an error naming the model and its source.- **Browser entry** — `@framefields/vision/web` re-exports the engine plus `createWebGPUProvider()` / `hasWebGPU()`; `onnxruntime-web` is an optional lazy peer.-### 7. Headless Conformance & FrameGrid Testing+### 8. Headless Conformance & FrameGrid Testing- **Pixel-sampling invariant assertions** — test compositions in Vitest with `skia-canvas` to verify shader math, font coverage, and Mean Squared Error (MSE) temporal deltas.- **FrameGrid contact sheets** — `comp.renderFrameGrid(...)` outputs sequential-frame contact sheets for instant review of easing, kinetic type, and transitions.-### 8. Live Preview in the Browser+### 9. Live Preview in the Browser- **Runs your composition, not a video** — `startPreview({ entry, export })` serves a localhost WebGPU player that loads the composition's own module and renders its frames live in the browser. Nothing is streamed: the server only hands over the bundle, the project's assets, and the soundtrack mixed by the export engine.- **Timeline, waveform & frame stepping** — play/pause, scrub, step frame by frame, and read resolution, FPS, duration, and audio status at a glance.- **One stable URL per project** — the port is derived from the working directory, so re-running the preview replaces the running server and any open tab reloads into the new version by itself. Close the tab and the server shuts down about five seconds later. A reload comes back at the same moment, and `#t=12.5` links to one.@@ -587,6 +605,7 @@ console.log(`Video rendered successfully to: ${filePath}`);3. **Carrier match cuts** — carry a visual element (badge, card, cursor, container) across scene boundaries with continuous velocity and position to avoid jarring cuts.4. **Physical easing vocabulary** — `back.out(1.4–1.7)` for snap-overshoot entrances, `spring` / `expo.out` for decelerating motion, `power2.in` for exits. Reserve `linear` for infinite spinners and time counters.5. **Headless invariant verification** — verify shader transforms, glyph coverage, and temporal MSE deltas with `skia-canvas` pixel sampling in Vitest before shipping.+6. **Image vs. Lottie layers** — raster images (`.png`, `.jpg`, `.webp`) use `Layer.image()`. Vector animations (`.json`, `.lottie`) must use `Layer.lottie()` — passing `.json` or `.lottie` to `Layer.image()` throws/warns and is rejected by `comp.lint()` with `unsupported_image_format`.---diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.tsindex 11a71316..f75dca32 100644--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts@@ -3287,6 +3287,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {durationMs: layerDurationInMS,opacity: 1.0,fps,+ ...(typeof lop.dpr === "number" && { dpr: lop.dpr }),// A media layer's effect chain (decode → passes) allocates from the// shared pool: keep it off the destination this layer composites onto,// or the raw frame lands there wherever the clip does not cover it.diff --git a/packages/compositions/src/compositions/canvas-composition.ts b/packages/compositions/src/compositions/canvas-composition.tsindex 5355119d..4c6d24d4 100644--- a/packages/compositions/src/compositions/canvas-composition.ts+++ b/packages/compositions/src/compositions/canvas-composition.ts@@ -506,6 +506,12 @@ export async function drawCompositionTree(fps,opacity: Number(op.opacity ?? 1) * Number(props.opacity ?? 1),isHeadless: props.isHeadless,+ dpr:+ typeof (props as { dpr?: unknown }).dpr === "number"+ ? (props as { dpr: number }).dpr+ : typeof (op as { dpr?: unknown }).dpr === "number"+ ? (op as { dpr: number }).dpr+ : undefined,});} else if (mediaType === "GIF") {await drawGifNode(ctx, pass, {diff --git a/packages/compositions/src/program/lint.test.ts b/packages/compositions/src/program/lint.test.tsindex 7f64c8de..f7cb19c5 100644--- a/packages/compositions/src/program/lint.test.ts+++ b/packages/compositions/src/program/lint.test.ts@@ -49,13 +49,17 @@ describe("lintComposition", () => {{ id: "photo", kind: "image", src: "a.png" },{ id: "t", kind: "text", fill: "#ffffff" },];- expect(codes(overImage, { backgroundColor: "#ffffff" })).not.toContain("low_contrast_text");+ expect(codes(overImage, { backgroundColor: "#ffffff" })).not.toContain(+ "low_contrast_text",+ );const onPanel = [{ id: "panel", kind: "box", background: "#ffffff" },{ id: "t", kind: "text", fill: "#ffffff", zIndex: -1 },];// Drawn before the panel (lower z): the backdrop is still known.- expect(codes(onPanel, { backgroundColor: "#ffffff" })).toContain("low_contrast_text");+ expect(codes(onPanel, { backgroundColor: "#ffffff" })).toContain(+ "low_contrast_text",+ );});it("flags static layers parked off-canvas, but not ones that move in", () => {@@ -125,4 +129,47 @@ describe("lintComposition", () => {]),).toEqual(["static_track"]);});++ it("flags unsupported formats given to image layers", () => {+ expect(+ codes([+ {+ id: "img1",+ kind: "media",+ dataType: "Image",+ inputHandleId: "anim.json",+ },+ ]),+ ).toEqual(["unsupported_image_format"]);+ expect(+ codes([+ {+ id: "img2",+ kind: "media",+ dataType: "Image",+ inputHandleId: "anim.lottie",+ },+ ]),+ ).toEqual(["unsupported_image_format"]);+ expect(+ codes([+ {+ id: "img3",+ kind: "media",+ dataType: "Image",+ inputHandleId: "video.mp4",+ },+ ]),+ ).toEqual(["unsupported_image_format"]);+ expect(+ codes([+ {+ id: "img4",+ kind: "media",+ dataType: "Image",+ inputHandleId: "valid.png",+ },+ ]),+ ).toEqual([]);+ });});diff --git a/packages/compositions/src/program/lint.ts b/packages/compositions/src/program/lint.tsindex 7989a30f..c09fcacf 100644--- a/packages/compositions/src/program/lint.ts+++ b/packages/compositions/src/program/lint.ts@@ -28,9 +28,7 @@ const COLOR_PROPS = [/** Contrast below this makes text hard to read (WCAG large-text minimum). */const MIN_TEXT_CONTRAST = 3;-function tracksOf(- node: AnyNode,-): Array<{+function tracksOf(node: AnyNode): Array<{prop: string;keyframes?: Array<{ value: unknown }>;source?: { type?: string };@@ -55,9 +53,7 @@ function paints(node: AnyNode): boolean {}const byZ = (nodes: AnyNode[]) =>- [...nodes].sort(- (a, b) => (Number(a.zIndex) || 0) - (Number(b.zIndex) || 0),- );+ [...nodes].sort((a, b) => (Number(a.zIndex) || 0) - (Number(b.zIndex) || 0));function solidColor(value: unknown): string | null {if (typeof value !== "string" || value.includes("gradient(")) return null;@@ -129,6 +125,37 @@ export function lintComposition(}if (node.kind === "camera") return;+ if (node.kind === "media") {+ const src = node.inputHandleId;+ if (typeof src === "string") {+ const dataType = node.dataType ?? "Video";+ if (dataType === "Image") {+ if (/\.(json|lottie)(\?.*)?$/i.test(src)) {+ push(+ "error",+ "unsupported_image_format",+ `'${id}' is an image layer with source '${src}', which is a Lottie vector animation. Use Layer.lottie() instead.`,+ id,+ );+ } else if (/\.(mp4|mov|webm|mkv)(\?.*)?$/i.test(src)) {+ push(+ "error",+ "unsupported_image_format",+ `'${id}' is an image layer with video source '${src}'. Use Layer.video() instead.`,+ id,+ );+ } else if (/\.(mp3|wav|ogg|m4a|aac)(\?.*)?$/i.test(src)) {+ push(+ "error",+ "unsupported_image_format",+ `'${id}' is an image layer with audio source '${src}'. Use Layer.audio() instead.`,+ id,+ );+ }+ }+ }+ }+const is3D =Boolean(node.is3D) || (typeof node.z === "number" && node.z !== 0);if (is3D || node.kind === "model3d") has3D = true;diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.tsindex 5882975d..32690e4d 100644--- a/packages/compositions/src/program/schema.ts+++ b/packages/compositions/src/program/schema.ts@@ -925,6 +925,7 @@ export interface MediaNode extends CompositionNodeBase {inputHandleId: string;dataType?: string;fit?: "cover" | "contain" | "fill" | "none";+ dpr?: number;borderRadius?: number;borderColor?: string;borderWidth?: number;@@ -1438,6 +1439,7 @@ export const MediaNodeSchema: z.ZodType<MediaNode> = zinputHandleId: z.string().min(1),dataType: z.string().optional(),fit: z.enum(["cover", "contain", "fill", "none"]).default("contain"),+ dpr: z.number().min(1).max(4).optional(),borderRadius: z.number().min(0).optional(),borderColor: ColorSchema.optional(),borderWidth: z.number().min(0).optional(),diff --git a/packages/compositions/src/program/to-virtual-media.test.ts b/packages/compositions/src/program/to-virtual-media.test.tsindex 6daf364e..6a8fb5a6 100644--- a/packages/compositions/src/program/to-virtual-media.test.ts+++ b/packages/compositions/src/program/to-virtual-media.test.ts@@ -239,4 +239,42 @@ describe("compositorToProgram", () => {expect((card.operation as any).durationFrames).toBe(144);expect((card.children![0].operation as any).startFrame).toBe(490);});++ it("resolves .json and .lottie files as direct media sources with Lottie dataType", () => {+ const doc: CompositorProgramConfig = {+ width: 1920,+ height: 1080,+ fps: 30,+ mode: "Video",+ layout: [+ {+ id: "lottie1",+ kind: "media",+ inputHandleId: "animation.json",+ dataType: "Lottie",+ },+ {+ id: "lottie2",+ kind: "media",+ inputHandleId: "animation.lottie",+ dataType: "Lottie",+ },+ ],+ } as any;++ const vm = compositorToProgram(doc, { isVideoMode: true, fps: 30 });+ expect(vm.children).toHaveLength(2);++ const layer1 = vm.children![0];+ expect(layer1.children).toHaveLength(1);+ expect((layer1.children![0].operation as any).op).toBe("source");+ expect((layer1.children![0].operation as any).dataType).toBe("Lottie");+ expect((layer1.children![0].operation as any).url).toBe("animation.json");++ const layer2 = vm.children![1];+ expect(layer2.children).toHaveLength(1);+ expect((layer2.children![0].operation as any).op).toBe("source");+ expect((layer2.children![0].operation as any).dataType).toBe("Lottie");+ expect((layer2.children![0].operation as any).url).toBe("animation.lottie");+ });});diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.tsindex fe83b558..bc70e23b 100644--- a/packages/compositions/src/program/to-virtual-media.ts+++ b/packages/compositions/src/program/to-virtual-media.ts@@ -54,7 +54,7 @@ function nodeToVirtualMedia(s.startsWith("./") ||s.startsWith("blob:") ||s.startsWith("data:") ||- /\.(png|jpe?g|webp|gif|svg|mp4|webm|mov|mkv|mp3|wav|ogg|m4a|srt|vtt)$/i.test(+ /\.(png|jpe?g|webp|gif|svg|mp4|webm|mov|mkv|mp3|wav|ogg|m4a|srt|vtt|json|lottie)$/i.test(s,));@@ -73,6 +73,16 @@ function nodeToVirtualMedia((node.dataType as DataType) ?? "Video",);}+ if (+ !media &&+ !resolveMedia &&+ sourceRef &&+ (sourceRef.includes(".") || sourceRef.includes("/"))+ ) {+ console.warn(+ `[Compositor] Media layer '${node.id ?? "?"}' source '${sourceRef}' could not be resolved.`,+ );+ }if (media && (node as any).crop) {const cropRaw = (node as any).crop;const cropOp =diff --git a/packages/framefields/src/builder-schema-conformance.test.ts b/packages/framefields/src/builder-schema-conformance.test.tsindex 18d53743..a1c0fca2 100644--- a/packages/framefields/src/builder-schema-conformance.test.ts+++ b/packages/framefields/src/builder-schema-conformance.test.ts@@ -15,6 +15,7 @@ const builders = {video: () => Layer.video("clip.mp4"),image: () => Layer.image("still.png"),svg: () => Layer.svg("logo.svg"),+ lottie: () => Layer.lottie("anim.json"),audio: () => Layer.audio("track.mp3"),text: () => Layer.text("Hello", { align: "start" }),richText: () => Layer.text(["Hello ", { text: "world", fontWeight: 700 }]),diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.tsindex 85c357b0..afe053d9 100644--- a/packages/framefields/src/index.ts+++ b/packages/framefields/src/index.ts@@ -849,6 +849,15 @@ export class Media {return m;}+ static lottie(+ source: MediaSourceInput,+ meta?: Partial<MediaMetadata>,+ ): Media {+ const m = new Media(source, "Lottie");+ if (meta) Object.assign(m.virtualMedia.metadata, meta);+ return m;+ }+static audio(source: MediaSourceInput, meta?: Partial<MediaMetadata>): Media {const m = new Media(source, "Audio");if (meta) Object.assign(m.virtualMedia.metadata, meta);@@ -1840,8 +1849,7 @@ export class Composition {...result.qa,issues,// Lint warnings advise; only a lint error fails the render.- passed:- result.qa.passed && !lint.some((d) => d.severity === "error"),+ passed: result.qa.passed && !lint.some((d) => d.severity === "error"),};}return deliverToOutputPath(result, opts.outputPath);@@ -2231,15 +2239,34 @@ export const Layer = {image: (src: string,options: Partial<MediaNode> = {},- ): AnimatableNode<MediaNode> =>- withAnimation({+ ): AnimatableNode<MediaNode> => {+ if (typeof src === "string") {+ if (+ /\.(json|lottie)(\?.*)?$/i.test(src) ||+ src.startsWith("data:application/json")+ ) {+ console.warn(+ `[Layer.image] '${src}' is a Lottie vector animation, not a static image. Use Layer.lottie('${src}', options) instead.`,+ );+ } else if (/\.(mp4|mov|webm|mkv)(\?.*)?$/i.test(src)) {+ console.warn(+ `[Layer.image] '${src}' is a video file, not a static image. Use Layer.video('${src}', options) instead.`,+ );+ } else if (/\.(mp3|wav|ogg|m4a|aac)(\?.*)?$/i.test(src)) {+ console.warn(+ `[Layer.image] '${src}' is an audio file, not a static image. Use Layer.audio('${src}', options) instead.`,+ );+ }+ }+ return withAnimation({id: options.id ?? autoId("img"),kind: "media",inputHandleId: src,dataType: "Image",fit: options.fit ?? "cover",...options,- }),+ });+ },svg: (src: string,options: Partial<MediaNode> = {},diff --git a/packages/framefields/src/lottie-layer.test.ts b/packages/framefields/src/lottie-layer.test.tsnew file mode 100644index 00000000..756c282c--- /dev/null+++ b/packages/framefields/src/lottie-layer.test.ts@@ -0,0 +1,82 @@+import { describe, expect, it, vi } from "vitest";+import { Composition, Layer, Media } from "./index.js";++describe("Layer.lottie & unsupported format handling", () => {+ it("resolves Layer.lottie with relative path into a Lottie source operation", () => {+ const comp = new Composition({ width: 400, height: 400, fps: 30 });+ comp.add(Layer.lottie("assets/spinner.json", { width: 200, height: 200 }));++ const vm = comp.toVirtualMedia();+ expect(vm.children).toHaveLength(1);++ const layer = vm.children?.[0];+ expect(layer).toBeDefined();+ const op = layer?.operation as Record<string, unknown>;+ expect(op.kind).toBe("media");+ expect(op.dataType).toBe("Lottie");+ expect(layer?.children).toHaveLength(1);++ const sourceOp = layer?.children?.[0]?.operation as Record<string, unknown>;+ expect(sourceOp.op).toBe("source");+ expect(sourceOp.dataType).toBe("Lottie");+ expect(sourceOp.url).toBe("assets/spinner.json");+ });++ it("creates Media.lottie with Lottie dataType", () => {+ const media = Media.lottie("animation.json");+ expect(media.virtualMedia.operation.dataType).toBe("Lottie");+ expect(media.virtualMedia.operation.op).toBe("source");+ });++ it("warns when Layer.image is called with a Lottie .json file", () => {+ const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});+ try {+ Layer.image("animation.json");+ expect(warnSpy).toHaveBeenCalledWith(+ expect.stringContaining(+ "is a Lottie vector animation, not a static image. Use Layer.lottie(",+ ),+ );+ } finally {+ warnSpy.mockRestore();+ }+ });++ it("warns when Layer.image is called with a .lottie archive", () => {+ const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {});+ try {+ Layer.image("badge.lottie");+ expect(warnSpy).toHaveBeenCalledWith(+ expect.stringContaining(+ "is a Lottie vector animation, not a static image. Use Layer.lottie(",+ ),+ );+ } finally {+ warnSpy.mockRestore();+ }+ });++ it("comp.lint() flags Layer.image with .json as unsupported_image_format", () => {+ const comp = new Composition({ width: 1920, height: 1080 });+ comp.add(Layer.image("hero-animation.json"));++ const diagnostics = comp.lint();+ const issue = diagnostics.find(+ (d) => d.code === "unsupported_image_format",+ );+ expect(issue).toBeDefined();+ expect(issue?.severity).toBe("error");+ expect(issue?.message).toContain("Use Layer.lottie() instead");+ });++ it("comp.lint() does not flag valid Layer.lottie", () => {+ const comp = new Composition({ width: 1920, height: 1080 });+ comp.add(Layer.lottie("hero-animation.json"));++ const diagnostics = comp.lint();+ const issue = diagnostics.find(+ (d) => d.code === "unsupported_image_format",+ );+ expect(issue).toBeUndefined();+ });+});diff --git a/packages/webgpu-renderers/src/nodes/image.test.ts b/packages/webgpu-renderers/src/nodes/image.test.tsindex a9ee435c..a286fe64 100644--- a/packages/webgpu-renderers/src/nodes/image.test.ts+++ b/packages/webgpu-renderers/src/nodes/image.test.ts@@ -106,6 +106,15 @@ describe("Image Node", () => {imageLoader.load("http://test.com/fail.png", true),).rejects.toThrow();});++ it("should reject Lottie and JSON animation files with helpful message", async () => {+ await expect(imageLoader.load("animation.json", true)).rejects.toThrow(+ "Unsupported image format for 'animation.json'",+ );+ await expect(imageLoader.load("animation.lottie", true)).rejects.toThrow(+ "Unsupported image format for 'animation.lottie'",+ );+ });});describe("drawImageNode", () => {diff --git a/packages/webgpu-renderers/src/nodes/image.ts b/packages/webgpu-renderers/src/nodes/image.tsindex 9fcb84e5..d902b9f9 100644--- a/packages/webgpu-renderers/src/nodes/image.ts+++ b/packages/webgpu-renderers/src/nodes/image.ts@@ -18,6 +18,15 @@ class ImageLoader {let promise = this.pending.get(key);if (!promise) {promise = (async () => {+ if (+ typeof src === "string" &&+ (/\.(json|lottie)(\?.*)?$/i.test(src) ||+ src.startsWith("data:application/json"))+ ) {+ throw new Error(+ `Unsupported image format for '${src}'. Lottie vector animations must be rendered with Layer.lottie(), not Layer.image().`,+ );+ }let arrayBuffer: ArrayBuffer;const isRemote =src.startsWith("http://") ||@@ -83,6 +92,15 @@ export async function drawImageNode(pass: GPURenderPassEncoder,props: ImageNodeProps,): Promise<void> {+ if (+ typeof props.src === "string" &&+ (/\.(json|lottie)(\?.*)?$/i.test(props.src) ||+ props.src.startsWith("data:application/json"))+ ) {+ console.warn(+ `[ImageRenderer] '${props.src}' is a Lottie vector animation, not a static image. Use Layer.lottie() instead.`,+ );+ }let tex = textureCache.acquire(props.src);if (!tex) {diff --git a/packages/webgpu-renderers/src/nodes/lottie.ts b/packages/webgpu-renderers/src/nodes/lottie.tsindex b04ee628..e5ac6cd6 100644--- a/packages/webgpu-renderers/src/nodes/lottie.ts+++ b/packages/webgpu-renderers/src/nodes/lottie.ts@@ -214,13 +214,18 @@ class LottieInstance {private _width: number;private _height: number;- constructor(src: string, width: number, height: number) {+ constructor(+ src: string,+ width: number,+ height: number,+ data?: string | ArrayBuffer,+ ) {this._width = width;this._height = height;this.canvas = new OffscreenCanvas(width, height);this.dotLottie = new DotLottie({canvas: this.canvas,- src: resolveUrl(src),+ ...(data ? { data } : { src: resolveUrl(src) }),autoplay: false,loop: false,speed: 0,@@ -242,7 +247,7 @@ class LottieInstance {reject(e instanceof Error? e- : new Error("Failed to load DotLottie animation"),+ : new Error(`Failed to load DotLottie animation from '${src}'`),);});});@@ -312,20 +317,23 @@ function polyfillHeadlessDOM(): void {}}- const docObj = (globalThis.document ??= {- createElement: (tag: string) => {- if (- tag === "canvas" &&- typeof globalThis.OffscreenCanvas === "function"- ) {- return new globalThis.OffscreenCanvas(1, 1);- }- return { ...dummyElement };- },- getElementsByTagName: () => [],- documentElement: dummyElement,- body: dummyElement,- } as unknown as Document) as unknown as Record<string, unknown>;+ if (!globalThis.document) {+ (globalThis as unknown as Record<string, unknown>).document = {+ createElement: (tag: string) => {+ if (+ tag === "canvas" &&+ typeof globalThis.OffscreenCanvas === "function"+ ) {+ return new globalThis.OffscreenCanvas(1, 1);+ }+ return { ...dummyElement };+ },+ getElementsByTagName: () => [],+ documentElement: dummyElement,+ body: dummyElement,+ } as unknown as Document;+ }+ const docObj = globalThis.document as unknown as Record<string, unknown>;docObj.documentElement ??= dummyElement;docObj.body ??= dummyElement;@@ -348,6 +356,15 @@ async function ensureWasmConfigured(): Promise<void> {typeof document.getElementById === "function";if (!isBrowser) {+ if (typeof globalThis.OffscreenCanvas === "undefined") {+ try {+ const { Canvas } = await import(+ /* webpackIgnore: true */ "skia-canvas"+ );+ (globalThis as unknown as Record<string, unknown>).OffscreenCanvas =+ Canvas;+ } catch {}+ }polyfillHeadlessDOM();// Add Response instanceof patch to handle cross-realm ESM loader issues in worker threadsif (typeof globalThis.Response === "function") {@@ -426,25 +443,27 @@ class LottieLoader {// 1. Parse the Lottie JSON if applicable, extract its fonts and register them.// This runs before the LottieInstance is created, so the text layers are shaped// with the registered fonts from the start.- let resolvedSrc = src;+ const resolvedSrc = src;+ let lottieData: string | ArrayBuffer | undefined;try {let lottieJson: any = null;const isNode =typeof process !== "undefined" && Boolean(process.versions?.node);+ const isRemote =+ resolvedSrc.startsWith("http://") ||+ resolvedSrc.startsWith("https://") ||+ resolvedSrc.startsWith("data:");if (resolvedSrc.startsWith("data:application/json;base64,")) {const base64Data = resolvedSrc.split(",")[1];const jsonStr = decodeBase64(base64Data);lottieJson = JSON.parse(jsonStr);+ lottieData = jsonStr;} else if (resolvedSrc.startsWith("data:application/json,")) {const decodeData = decodeURIComponent(resolvedSrc.split(",")[1]);lottieJson = JSON.parse(decodeData);- } else if (- isNode &&- (resolvedSrc.startsWith("/") ||- resolvedSrc.startsWith("file://")) &&- resolvedSrc.endsWith(".json")- ) {+ lottieData = decodeData;+ } else if (isNode && !isRemote) {let filePath = resolvedSrc;if (filePath.startsWith("file://")) {try {@@ -453,25 +472,35 @@ class LottieLoader {filePath = filePath.replace(/^file:\/\//, "");}}- const fs = await import(/* webpackIgnore: true */ "fs/promises");- const jsonContent = await fs.readFile(filePath, "utf-8");- lottieJson = JSON.parse(jsonContent);- const base64Data = Buffer.from(jsonContent).toString("base64");- resolvedSrc = `data:application/json;base64,${base64Data}`;+ const pathModule = await import(+ /* webpackIgnore: true */ "node:path"+ );+ if (!pathModule.isAbsolute(filePath)) {+ filePath = pathModule.resolve(process.cwd(), filePath);+ }+ try {+ const fs = await import(/* webpackIgnore: true */ "fs/promises");+ if (filePath.endsWith(".json")) {+ const jsonContent = await fs.readFile(filePath, "utf-8");+ lottieJson = JSON.parse(jsonContent);+ lottieData = jsonContent;+ } else {+ const fileBuf = await fs.readFile(filePath);+ lottieData = fileBuf.buffer.slice(+ fileBuf.byteOffset,+ fileBuf.byteOffset + fileBuf.byteLength,+ );+ }+ } catch {+ // If file read fails (e.g. mocked path in tests), fall back to resolvedSrc+ }} else if (- resolvedSrc.startsWith("http://") ||- resolvedSrc.startsWith("https://") ||- resolvedSrc.startsWith("/")+ isRemote &&+ (resolvedSrc.endsWith(".json") || resolvedSrc.includes("/assets/"))) {- // Only pre-fetch JSON animations. Binary .lottie zip files pack their own fonts.- if (- resolvedSrc.endsWith(".json") ||- resolvedSrc.includes("/assets/")- ) {- const response = await fetch(resolveUrl(resolvedSrc));- if (response.ok) {- lottieJson = await response.json();- }+ const response = await fetch(resolveUrl(resolvedSrc));+ if (response.ok) {+ lottieJson = await response.json();}}@@ -495,6 +524,7 @@ class LottieLoader {resolvedSrc,resolvedWidth,resolvedHeight,+ lottieData,);await instance.ready();@@ -514,8 +544,17 @@ export async function drawLottieNode(pass: GPURenderPassEncoder,props: LottieNodeProps,): Promise<void> {- const resolvedWidth = Math.max(1, Math.ceil(props.dstRect.width));- const resolvedHeight = Math.max(1, Math.ceil(props.dstRect.height));+ const baseWidth = Math.max(1, Math.ceil(props.dstRect.width));+ const baseHeight = Math.max(1, Math.ceil(props.dstRect.height));+ const requestedDpr =+ typeof props.dpr === "number" && props.dpr > 0 ? props.dpr : 2;+ const maxDim = Math.max(baseWidth, baseHeight);+ const maxAllowedDpr = Math.max(1, Math.floor(4096 / Math.max(1, maxDim)));+ const effectiveDpr = Math.min(requestedDpr, maxAllowedDpr);++ const resolvedWidth = Math.max(1, Math.round(baseWidth * effectiveDpr));+ const resolvedHeight = Math.max(1, Math.round(baseHeight * effectiveDpr));+const instance = await lottieLoader.load(props.src,resolvedWidth,diff --git a/packages/webgpu-renderers/src/nodes/types.ts b/packages/webgpu-renderers/src/nodes/types.tsindex 411438c8..951ec6cc 100644--- a/packages/webgpu-renderers/src/nodes/types.ts+++ b/packages/webgpu-renderers/src/nodes/types.ts@@ -45,6 +45,7 @@ export interface LottieNodeProps {animationId?: string;stateMachineId?: string;theme?: string;+ dpr?: number;}export interface SVGNodeProps {diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.mdindex 7714a47d..c6fd0947 100644--- a/plugins/framefields/skills/framefields-compose/SKILL.md+++ b/plugins/framefields/skills/framefields-compose/SKILL.md@@ -88,9 +88,9 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani| `Layer.box(options)` | `"box"` | Rectangular container / card. Options: `background`, `borderRadius`, `borderColor`, `borderWidth`, `overflow: "hidden"`, `padding`, `children`. || `Layer.flex(options)` | `"flex"` | Auto-layout container (Yoga). Options: `dir: "row" \| "column"`, `gap`, `justify`, `align`, `padding`, `wrap`, `children`. || `Layer.video(src, options)` | `"media"` | Video layer decoded via mediabunny. Options: `fit: "cover" \| "contain"`, `trimStartSec`, `muted`, `volume`. |-| `Layer.image(src, options)` | `"media"` | Static raster image (`.png`, `.jpg`, `.webp`). Options: `fit: "cover" \| "contain"`. |+| `Layer.image(src, options)` | `"media"` | Static raster image (`.png`, `.jpg`, `.webp`). Options: `fit: "cover" \| "contain"`. For vector animations (`.json`, `.lottie`), use `Layer.lottie`. || `Layer.svg(src, options)` | `"media"` | Vector SVG asset. Options: `fit: "contain"`. |-| `Layer.lottie(src, options)` | `"media"` | Lottie vector animation file. |+| `Layer.lottie(src, options)` | `"media"` | Lottie vector animation file (`.json`, `.lottie`). Decoded and rendered via ThorVG. Options: `fit: "contain" \| "cover" \| "fill"`, `width`, `height`. || `Layer.audio(src, options)` | `"media"` | Audio track layer. Options: `volume`, `startFrame`, `durationFrames`. || `Layer.shape(type, options)` | `"shape"` | Parametric vector graphics (`"rect"`, `"circle"`, `"ellipse"`, `"polygon"`, `"star"`, `"arrow"`, `"path"`). Options: `fill` (a color or CSS gradient), `d` (SVG path data with arcs), `strokeWidth`, `strokeColor`, `trimStart`, `trimEnd`. || `Layer.caption(src, options)` | `"media"` | SRT-driven synchronized subtitle engine. Bottom-anchored auto-wrapping cues. |@@ -380,3 +380,4 @@ Layer3D.cylinder({ ..., color: "#111", emissive: "#4da3ff", emissiveIntensity: 37. **Audio Layer Duration:** Every `Layer.audio()` must be assigned `durationFrames` to prevent audio truncation in headless rendering.8. **Premultiplied Alpha:** Alpha shaders operate with premultiplied alpha. Never use `"src-alpha"` factor in custom blends.9. **Lint before rendering:** `comp.lint()` lists colors/gradients that would draw transparent, low-contrast text, off-canvas layers, shadow settings that do nothing and `roughness`/`metallic` without an environment. `renderVideo({ qa: true })` includes these findings.+10. **Lottie vs. Image Layers:** Use `Layer.lottie("file.json" | "file.lottie", { width, height, fit })` for Lottie vector animations. Never pass `.json` or `.lottie` to `Layer.image()`, which only decodes raster formats and throws/warns if given a vector animation.diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.mdindex 5d9f9c0d..d8df0995 100644--- a/plugins/framefields/skills/framefields-render/SKILL.md+++ b/plugins/framefields/skills/framefields-render/SKILL.md@@ -252,6 +252,7 @@ if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop| `audio_silence` | info | ≥ `minSilenceSec` (2 s) below `silenceDbfs` (-60) || `audio_silent` | info | The whole track is silent || `invalid_gradient` | error | A `background` / `fill` gradient string that does not parse: it draws nothing |+| `unsupported_image_format` | error | An image layer receiving a `.json` / `.lottie` vector animation, video, or audio file || `unknown_color` | warning | A color string that is not a color: it draws transparent || `low_contrast_text` | warning | Text below 3:1 contrast against the solid color behind it (suggests a readable one) || `offscreen_layer` | warning | A static absolute layer entirely outside the canvas |diff --git a/plugins/framefields/skills/framefields/SKILL.md b/plugins/framefields/skills/framefields/SKILL.mdindex 4ed51691..bbc33609 100644--- a/plugins/framefields/skills/framefields/SKILL.md+++ b/plugins/framefields/skills/framefields/SKILL.md@@ -217,7 +217,7 @@ Add scripts to `package.json`:### Video assets & Git LFS-Video inputs (`Layer.video("assets/footage/clip.mp4")`), audio tracks (`Layer.audio(...)`), LUTs, and custom fonts belong in **Git LFS**, while render outputs in `output/` belong in `.gitignore`.+Video inputs (`Layer.video("assets/footage/clip.mp4")`), Lottie animations (`Layer.lottie(...)`), audio tracks (`Layer.audio(...)`), LUTs, and custom fonts belong in **Git LFS**, while render outputs in `output/` belong in `.gitignore`.1. **Initialize Git LFS:**```bash@@ -232,6 +232,7 @@ Video inputs (`Layer.video("assets/footage/clip.mp4")`), audio tracks (`Layer.auassets/**/*.mp3 filter=lfs diff=lfs merge=lfs -textassets/**/*.wav filter=lfs diff=lfs merge=lfs -textassets/**/*.cube filter=lfs diff=lfs merge=lfs -text+ assets/**/*.lottie filter=lfs diff=lfs merge=lfs -textassets/fonts/**/*.ttf filter=lfs diff=lfs merge=lfs -textassets/fonts/**/*.otf filter=lfs diff=lfs merge=lfs -text```diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex ad9c1f8a..06b54414 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -375,7 +375,26 @@ importers:specifier: ^5.8.3version: 5.9.3- examples/26_mac_keyboard:+ examples/26_mac_keyboard_from_3d_primitives:+ dependencies:+ framefields:+ specifier: workspace:*+ version: link:../../packages/framefields+ devDependencies:+ '@framefields/tsconfig':+ specifier: workspace:*+ version: link:../../packages/tsconfig+ '@types/node':+ specifier: ^24.1.0+ version: 24.10.14+ tsx:+ specifier: ^4.20.3+ version: 4.21.0+ typescript:+ specifier: ^5.8.3+ version: 5.9.3++ examples/27_rocket_science:dependencies:framefields:specifier: workspace:*