Skip to content

Code-first video & image, rendered natively on WebGPU.

feat: variable font axes, 3D scene fixes, raster texture limit

okanasl865acb033237
diff --git a/.changeset/font-variations.md b/.changeset/font-variations.md
new file mode 100644
index 00000000..6f2e5dad
--- /dev/null
+++ b/.changeset/font-variations.md
@@ -0,0 +1,5 @@
+---
+"framefields": minor
+---
+
+Text takes `fontVariations`: a variable font's axes by tag, as CSS `font-variation-settings` (`{ wdth: 68 }` for a condensed cut, `{ opsz: 72 }`, `{ GRAD: 50 }`). Layout measures the text at those axes, 2D and native 3D glyphs draw at them, spans inherit them, and `FontManager.measure` takes them. Text layers used to ignore every axis but weight and the automatic optical size, so a width axis could not be set at all.
diff --git a/.changeset/raster-scale-texture-limit.md b/.changeset/raster-scale-texture-limit.md
new file mode 100644
index 00000000..9393838a
--- /dev/null
+++ b/.changeset/raster-scale-texture-limit.md
@@ -0,0 +1,5 @@
+---
+"framefields": patch
+---
+
+A 2D layer longer than the GPU's texture side at layout scale (a counter's tall reel strip, a long scroll) rasterizes just under that side, and its motion-blur margin stays within it, instead of failing to allocate its texture and drawing nothing.
diff --git a/.changeset/scene-box-effects.md b/.changeset/scene-box-effects.md
new file mode 100644
index 00000000..e2ad84c3
--- /dev/null
+++ b/.changeset/scene-box-effects.md
@@ -0,0 +1,5 @@
+---
+"framefields": patch
+---
+
+Effects on a 3D scene box (a box holding a camera and what it films) now apply, over the finished scene. The scene renders as a nested composition placed like a clip, but its effects were never wrapped around that composition, so every effect on it was silently dropped.
diff --git a/.changeset/scene-layer-vision-clock.md b/.changeset/scene-layer-vision-clock.md
new file mode 100644
index 00000000..ab974c84
--- /dev/null
+++ b/.changeset/scene-layer-vision-clock.md
@@ -0,0 +1,5 @@
+---
+"framefields": patch
+---
+
+Vision clips of layers inside a 3D scene analyse on the composition's clock. Layers in a scene run on the scene's own clock, and the clip linking skipped them: a depth layer inside a scene got a clip with no time remap and a start of 0, so preparing frame N analysed its first N frames, far past the clip's end (a depth relief late in a film analysed thousands of frames). `comp.vision(layer)` now finds layers inside scenes too, at their start on the composition's clock.
diff --git a/.changeset/scene-window-inherit.md b/.changeset/scene-window-inherit.md
new file mode 100644
index 00000000..0f517803
--- /dev/null
+++ b/.changeset/scene-window-inherit.md
@@ -0,0 +1,5 @@
+---
+"framefields": patch
+---
+
+A 3D scene box with no window of its own lasts its parent's whole window. It ended early inside a parent that starts after frame 0: its length was taken from its nested composition's end as if that were an absolute frame, so a scene placed in an act starting at frame 282 drew for only its first few frames and then went black.
diff --git a/assets/fonts b/assets/fonts
new file mode 120000
index 00000000..08395e8a
--- /dev/null
+++ b/assets/fonts
@@ -0,0 +1 @@
+/Users/okanaslankan/frameshop/assets/fonts
\ No newline at end of file
diff --git a/nodes/node-compositor/SKILL.md b/nodes/node-compositor/SKILL.md
index 9e667b85..01e3029f 100644
--- a/nodes/node-compositor/SKILL.md
+++ b/nodes/node-compositor/SKILL.md
@@ -210,7 +210,7 @@ Each track represents animatable property modifications:
- **Watermarking a Video:** a `flex` row (`align: "end"`, `justify: "end"`, full canvas) containing a `media` node bound to the PNG input; low `opacity`.
- **Picture-in-Picture:** a `flex` row with two `media` nodes (`fit: "cover"`, each `grow: 1`).
- **Audio-Reactive 3D Models:** a `kind: "model3d"` node loading an OBJ/FBX/GLTF/STL/PLY mesh with `audioDeform: { audioTrackId, mode: "normal_extrusion" | "radial_pulse" | "harmonic_wave" | "twist" | "ripple", amplitudeMultiplier, damping }`, deformed directly in VRAM via WebGPU compute shaders driven by audio frequency spectra and transient energies. `twist` turns each slice about Y by its height (harder on the bass); `ripple` runs concentric waves across XY, struck by the drums. `audioTrackId` may be an audio layer's id or an audio file path.
-- **Several 3D scenes:** a `box`/`flex`/`block` with a numeric `width` and `height` that holds a `camera` and the layers it films renders as its own scene (one holding only a camera stays a shot in a camera cut list): a nested composition with only its own camera, lights and environment, on its own clock (children start from the box's frame 0), placed in its parent like a media layer. Transitions, opacity and mattes on the box then blend finished pictures. Without that, every 3D layer shares one world and the active camera (the latest to start).
+- **Several 3D scenes:** a `box`/`flex`/`block` with a numeric `width` and `height` that holds a `camera` and the layers it films renders as its own scene (one holding only a camera stays a shot in a camera cut list): a nested composition with only its own camera, lights and environment, on its own clock (children start from the box's frame 0), placed in its parent like a media layer. Its effects run over the finished scene, and without a window of its own it lasts its parent's. Transitions, opacity and mattes on the box then blend finished pictures. Without that, every 3D layer shares one world and the active camera (the latest to start).
- **3D scene capacity:** direct lights and skin joints use dynamically sized GPU storage; `directional`, `spot`, `rect` (1 layer each), and `point` lights (6 cube-face layers) can each cast shadows, up to the device texture-array layer limit.
- **Rect Area Lights (`lightType: "rect"`):** rectangular softbox/panel area lights (`width`, `height`, `twoSided`, `rotation`, `x/y/z`, `targetX/Y/Z`) use solid-angle subtended diffuse irradiance and Karis representative-point GGX specular shading, and can cast perspective frustum shadows (`castShadow: true`).
- **Volumetric Fog & Light Shafts (`volumetricFog` in `environment`):** `kind: "light"` with `lightType: "environment"` and `volumetricFog: true | { density, anisotropy, heightFalloff, baseHeight, color, lightIntensity, ambientIntensity, maxDistance, steps, noise: { scale, strength, speed, octaves } }` runs half-resolution raymarching with Interleaved Gradient Noise jitter, Henyey-Greenstein phase scattering, 3D FBM wind-animated density noise, and multi-light shadow-map sampling (carving god rays behind shadow casters), followed by depth-aware bilateral upsampling. `volumetricFogDensity`, `volumetricFogAnisotropy`, and `volumetricFogNoiseStrength` can be animated or driven by `Signal`s.
diff --git a/nodes/node-compositor/src/renderers/transform.test.ts b/nodes/node-compositor/src/renderers/transform.test.ts
index 21e11190..da18347b 100644
--- a/nodes/node-compositor/src/renderers/transform.test.ts
+++ b/nodes/node-compositor/src/renderers/transform.test.ts
@@ -338,7 +338,21 @@ describe("layerRasterScale", () => {
// A full 1080p layer under a 4-canvas budget doubles at most.
expect(layerRasterScale(4, { width: 1920, height: 1080 }, roomy)).toBe(2);
expect(layerRasterScale(4, size, { ...roomy, effects: 1.6 })).toBe(1.5);
- // Never below layout scale, whatever the limits.
+ // Never below layout scale for a budget or an effect.
expect(layerRasterScale(4, size, { ...roomy, effects: 0.5 })).toBe(1);
+ expect(layerRasterScale(1, { width: 3000, height: 3000 }, roomy)).toBe(1);
+ });
+
+ it("shrinks a layer longer than the texture side to fit it", () => {
+ // A counter's reel strip, 8210 px tall at layout scale.
+ const strip = { width: 112, height: 8210 };
+ const scale = layerRasterScale(1, strip, roomy);
+ expect(scale).toBeLessThan(1);
+ expect(Math.ceil(strip.height * scale)).toBeLessThanOrEqual(8192);
+ expect(Math.ceil(strip.width * scale)).toBeLessThanOrEqual(8192);
+ // Zoomed in, it still fits.
+ expect(
+ Math.ceil(strip.height * layerRasterScale(3, strip, roomy)),
+ ).toBeLessThanOrEqual(8192);
});
});
diff --git a/nodes/node-compositor/src/renderers/transform.ts b/nodes/node-compositor/src/renderers/transform.ts
index 3a6fd7aa..e6fed56b 100644
--- a/nodes/node-compositor/src/renderers/transform.ts
+++ b/nodes/node-compositor/src/renderers/transform.ts
@@ -158,11 +158,13 @@ export const MAX_RASTER_SCALE = 4;
* Pixels per layout px to rasterize a 2D layer at, so it stays sharp when its
* transform onto the canvas scales it up by `axisScale` (the transform's
* larger axis scale). Rounded up to a quarter, so an animated zoom reuses a
- * few texture sizes instead of allocating new ones every frame; never below 1
+ * few texture sizes instead of allocating new ones every frame; not below 1
* (a shrunk layer keeps its layout raster). Capped, rounded down to a quarter,
* by MAX_RASTER_SCALE, by `limits.maxDimension` (the GPU's texture side), by
* `limits.maxPixels` (a pixel budget for one texture) and by
* `limits.effects`, the highest scale the layer's effects keep their look at.
+ * The texture side is a hard limit: a layer longer than it at layout scale
+ * (a tall reel strip, a long scroll) rasterizes just under it, below 1.
*/
export function layerRasterScale(
axisScale: number,
@@ -177,6 +179,8 @@ export function layerRasterScale(
Math.sqrt(limits.maxPixels / (w * h)),
limits.effects,
);
+ const fits = limits.maxDimension / Math.max(w, h);
+ if (fits < 1) return Math.floor(fits * 1024) / 1024;
const wanted = Math.ceil(axisScale * 4 - 1e-6) / 4;
return Math.max(1, Math.min(wanted, Math.floor(cap * 4 + 1e-6) / 4));
}
diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
index edda4300..1c14308c 100644
--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts
+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts
@@ -1269,6 +1269,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
const textStyle = {
fontSize: node.fontSize ?? 48,
fontWeight: node.fontWeight,
+ fontVariations: node.fontVariations,
fontFamily: node.fontFamily,
letterSpacing: node.letterSpacing,
lineHeight: node.lineHeight,
@@ -5842,6 +5843,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
fontFamily: lop.fontFamily,
fontStyle: lop.fontStyle,
fontWeight: lop.fontWeight,
+ fontVariations: lop.fontVariations,
fill: target?.fill ?? target?.color ?? lop.fill,
align: lop.align,
direction: lop.direction,
@@ -6280,8 +6282,17 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {
// to streak past their own bounds instead of smearing edge pixels.
if (!isTrue3D && !(is3D && quad3D)) {
const reach = shutterAngle / 360;
- const padX = Math.ceil(Math.abs(vx) * rasterScale * reach);
- const padY = Math.ceil(Math.abs(vy) * rasterScale * reach);
+ // Within the GPU's texture side: a layer already at it
+ // streaks only as far as the side allows.
+ const side = ctx.device.limits.maxTextureDimension2D;
+ const padX = Math.min(
+ Math.ceil(Math.abs(vx) * rasterScale * reach),
+ Math.max(0, Math.floor((side - texW) / 2)),
+ );
+ const padY = Math.min(
+ Math.ceil(Math.abs(vy) * rasterScale * reach),
+ Math.max(0, Math.floor((side - texH) / 2)),
+ );
blurPadX = padX / rasterScale;
blurPadY = padY / rasterScale;
const paddedTex = ctx.renderer.getTemporaryTexture(
diff --git a/packages/compositions/src/compositions/canvas-composition.ts b/packages/compositions/src/compositions/canvas-composition.ts
index df59d35a..f378e265 100644
--- a/packages/compositions/src/compositions/canvas-composition.ts
+++ b/packages/compositions/src/compositions/canvas-composition.ts
@@ -118,6 +118,7 @@ function paragraphPropsFromTextOp(
verticalAlign: opText.verticalAlign,
fontWeight: opText.fontWeight,
fontStyle: opText.fontStyle,
+ variableAxes: opText.fontVariations,
letterSpacing: opText.letterSpacing,
shadows: opText.shadows,
textBackgroundColor: opText.textBackgroundColor,
diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.ts
index 06faf140..db6ed057 100644
--- a/packages/compositions/src/program/schema.ts
+++ b/packages/compositions/src/program/schema.ts
@@ -1298,6 +1298,13 @@ export interface TextNode extends CompositionNodeBase {
fontSize?: number;
fontWeight?: number | string;
fontStyle?: string;
+ /**
+ * Variable font axes by tag, as CSS `font-variation-settings`: `{ wdth: 68 }`
+ * narrows a font with a width axis, `{ opsz: 72 }` or `{ GRAD: 50 }` set
+ * others. `fontWeight` sets `wght`; `opsz` follows the font size unless set.
+ * Axes the font lacks are ignored. Spans inherit them.
+ */
+ fontVariations?: Record<string, number>;
fill?: string;
align?: "start" | "center" | "end";
/**
@@ -2071,6 +2078,7 @@ const TextFields = {
fontSize: z.number().positive().optional(),
fontWeight: z.union([z.number().int(), z.string()]).optional(),
fontStyle: z.string().optional(),
+ fontVariations: z.record(z.string(), z.number()).optional(),
/** Text color (v1 name kept). */
fill: ColorSchema,
align: z.enum(["start", "center", "end"]).optional(),
diff --git a/packages/compositions/src/program/to-virtual-media.test.ts b/packages/compositions/src/program/to-virtual-media.test.ts
index 73bb77d4..401c883a 100644
--- a/packages/compositions/src/program/to-virtual-media.test.ts
+++ b/packages/compositions/src/program/to-virtual-media.test.ts
@@ -6,7 +6,11 @@ import {
} from "@framefields/core";
import { describe, expect, it } from "vitest";
import type { CompositorProgramConfig, LayoutNode } from "./schema.js";
-import { compositorToProgram, isScene3D } from "./to-virtual-media.js";
+import {
+ collectLayersOnClock,
+ compositorToProgram,
+ isScene3D,
+} from "./to-virtual-media.js";
const doc = {
width: 1920,
@@ -493,6 +497,103 @@ describe("compositorToProgram: 3D scenes", () => {
ramp,
});
});
+ it("runs a scene's effects over its finished composition, once", () => {
+ const fx = compositorToProgram(
+ {
+ width: 1080,
+ height: 1920,
+ fps: 30,
+ mode: "Video",
+ layout: [
+ {
+ ...(scene("fx", 0) as unknown as Record<string, unknown>),
+ effects: [{ op: "Vignette", strength: 80 }],
+ } as unknown as LayoutNode,
+ ],
+ } as unknown as CompositorProgramConfig,
+ { isVideoMode: true, fps: 30, durationMs: 4000 },
+ );
+ const layer = (fx.children as VirtualMediaData[])[0];
+ // The placed picture carries no effects of its own...
+ expect(
+ (layer?.operation as Record<string, unknown>).effects,
+ ).toBeUndefined();
+ // ...they wrap the nested composition, once.
+ const pass = layer?.children?.[0];
+ expect(pass?.operation.op).toBe("Vignette");
+ expect(pass?.children?.[0]?.operation.op).toBe("Compositor");
+ });
+ it("keeps a windowless scene for its parent's whole window", () => {
+ const fx = compositorToProgram(
+ {
+ width: 1080,
+ height: 1920,
+ fps: 30,
+ mode: "Video",
+ layout: [
+ {
+ kind: "box",
+ id: "act",
+ width: 1080,
+ height: 1920,
+ startFrame: 282,
+ durationFrames: 341,
+ children: [
+ {
+ ...(scene("inner", 0) as unknown as Record<string, unknown>),
+ startFrame: undefined,
+ durationFrames: undefined,
+ } as unknown as LayoutNode,
+ ],
+ } as unknown as LayoutNode,
+ ],
+ } as unknown as CompositorProgramConfig,
+ { isVideoMode: true, fps: 30, durationMs: 30000 },
+ );
+ const act = (fx.children as VirtualMediaData[])[0];
+ const inner = act?.children?.[0]?.operation as Record<string, unknown>;
+ expect(inner.startFrame).toBe(282);
+ expect(inner.durationFrames).toBe(341);
+ });
+ it("puts a scene's layers on the composition's clock", () => {
+ const fx = compositorToProgram(
+ {
+ width: 1080,
+ height: 1920,
+ fps: 30,
+ mode: "Video",
+ layout: [
+ {
+ kind: "box",
+ id: "act",
+ width: 1080,
+ height: 1920,
+ startFrame: 200,
+ durationFrames: 300,
+ children: [
+ {
+ ...(scene("inner", 40) as unknown as Record<string, unknown>),
+ durationFrames: 200,
+ } as unknown as LayoutNode,
+ ],
+ } as unknown as LayoutNode,
+ ],
+ } as unknown as CompositorProgramConfig,
+ { isVideoMode: true, fps: 30, durationMs: 30000 },
+ );
+ const starts = new Map(
+ collectLayersOnClock(fx.children as VirtualMediaData[]).map(
+ ({ vm, startFrame }) => [
+ (vm.operation as { id?: string }).id,
+ startFrame,
+ ],
+ ),
+ );
+ expect(starts.get("act")).toBe(200);
+ expect(starts.get("inner")).toBe(240);
+ // The cube starts 10 frames into the scene, on the scene's clock.
+ expect(starts.get("inner-cube")).toBe(250);
+ });
it("leaves a box holding only a camera as a shot over the shared world", () => {
expect(
isScene3D({
diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.ts
index 366c3dcb..c2c3054a 100644
--- a/packages/compositions/src/program/to-virtual-media.ts
+++ b/packages/compositions/src/program/to-virtual-media.ts
@@ -86,6 +86,37 @@ export function isScene3D(node: LayoutNode): boolean {
);
}
+/**
+ * `media` with `node`'s effects applied in order, each a pass over the
+ * picture below it: a clip's frames, or a 3D scene's finished composition.
+ */
+function wrapEffects(
+ media: VirtualMediaData,
+ node: { id?: string; effects?: unknown },
+): VirtualMediaData {
+ if (!Array.isArray(node.effects)) return media;
+ let current = media;
+ for (const eff of node.effects) {
+ const effOp =
+ eff &&
+ typeof (eff as { toOperation?: unknown }).toOperation === "function"
+ ? (eff as { toOperation: () => Record<string, unknown> }).toOperation()
+ : (eff as Record<string, unknown>);
+ current = {
+ metadata: { ...current.metadata },
+ operation: {
+ op: String(effOp.op ?? "effect"),
+ id: node.id ? `${node.id}-${effOp.op ?? "effect"}` : undefined,
+ effect: eff,
+ ...effOp,
+ dataType: current.operation.dataType,
+ },
+ children: [current],
+ } as VirtualMediaData;
+ }
+ return current;
+}
+
function nodeToVirtualMedia(
node: LayoutNode,
resolveMedia: CompositorToProgramOptions["resolveMedia"],
@@ -178,30 +209,7 @@ function nodeToVirtualMedia(
children: [media],
};
}
- if (media && Array.isArray(node.effects)) {
- let currentMedia: VirtualMediaData = media;
- for (const eff of node.effects) {
- const effOp =
- eff &&
- typeof (eff as { toOperation?: unknown }).toOperation === "function"
- ? (
- eff as { toOperation: () => Record<string, unknown> }
- ).toOperation()
- : (eff as Record<string, unknown>);
- currentMedia = {
- metadata: { ...currentMedia.metadata },
- operation: {
- op: String(effOp.op ?? "effect"),
- id: node.id ? `${node.id}-${effOp.op ?? "effect"}` : undefined,
- effect: eff,
- ...effOp,
- dataType: currentMedia.operation.dataType,
- },
- children: [currentMedia],
- };
- }
- media = currentMedia;
- }
+ if (media) media = wrapEffects(media, node);
if (media && node.vision) {
const visionRaw = node.vision as
| { toOperation?: () => Record<string, unknown> }
@@ -319,6 +327,8 @@ function nodeToVirtualMedia(
},
children,
} as unknown as VirtualMediaData;
+ // Its effects run over the finished scene, as over a clip's frames.
+ media = wrapEffects(media, node);
children = [media];
}
}
@@ -361,7 +371,13 @@ function nodeToVirtualMedia(
// are authored parent-relative and its background shows before its first child
// arrives — and stays active until its last child ends.
// Also when its own durationFrames is unusable (0, NaN): `dur` is unset then.
- if (dur === undefined) {
+ // A 3D scene's one child is its own composition, on the scene's clock
+ // from 0: it lasts the scene's inherited duration, not until an
+ // absolute frame (inside a parent that starts late, that would end it
+ // early).
+ if (dur === undefined && isScene3D(node)) {
+ dur = Math.max(1, defaultCompDuration);
+ } else if (dur === undefined) {
const childEnds = children.map(
(c) =>
(c.operation.startFrame ?? 0) +
@@ -425,11 +441,13 @@ function nodeToVirtualMedia(
operation: {
op: "CompositorLayer",
...node,
- // A scene is placed as media: its nested composition paints its background.
+ // A scene is placed as media: its nested composition paints its
+ // background, and its effects already wrap that composition.
...(isScene3D(node) && {
kind: "media",
background: undefined,
children: undefined,
+ effects: undefined,
}),
startFrame,
durationFrames: dur,
@@ -577,6 +595,34 @@ function shareSignals(
* Recursively gather every node op from a render tree (in order), stopping at
* nested scene compositions: their layers belong to their own render.
*/
+/**
+ * Every layer, 3D scenes' layers included, with its start on the
+ * composition's clock. A scene's layers run on the scene's own clock, which
+ * starts where the scene's layer does, so their starts are offset by it.
+ */
+export function collectLayersOnClock(
+ nodes: VirtualMediaData[],
+): Array<{ vm: VirtualMediaData; startFrame: number }> {
+ const out: Array<{ vm: VirtualMediaData; startFrame: number }> = [];
+ const walk = (vm: VirtualMediaData, offset: number, layerStart: number) => {
+ const op = vm?.operation as
+ | { op?: string; startFrame?: number }
+ | undefined;
+ if (op?.op === "Compositor") {
+ for (const c of vm.children ?? []) walk(c, layerStart, layerStart);
+ return;
+ }
+ let start = layerStart;
+ if (op?.op === "CompositorLayer") {
+ start = offset + (op.startFrame ?? 0);
+ out.push({ vm, startFrame: start });
+ }
+ for (const c of vm?.children ?? []) walk(c, offset, start);
+ };
+ for (const n of nodes) walk(n, 0, 0);
+ return out;
+}
+
export function collectNodeOps(nodes: VirtualMediaData[]): VirtualMediaData[] {
const out: VirtualMediaData[] = [];
const walk = (vm: VirtualMediaData) => {
diff --git a/packages/framefields/src/clip-vision.test.ts b/packages/framefields/src/clip-vision.test.ts
index db15c1a2..dcfe793d 100644
--- a/packages/framefields/src/clip-vision.test.ts
+++ b/packages/framefields/src/clip-vision.test.ts
@@ -160,6 +160,45 @@ describe("remapped clip vision", () => {
expect(vision.startFrame).toBe(17);
expect(vision.clipFrame(19)).toBe(25);
});
+ it("binds a layer inside a 3D scene on the composition's clock", () => {
+ const comp = new Composition({
+ width: 64,
+ height: 64,
+ fps: 24,
+ durationFrames: 200,
+ });
+ // The scene starts at 90 (an act at 60, the scene 30 into it); the
+ // clip starts 4 frames into the scene, on the scene's own clock.
+ const layer = Layer.video("clip.mp4", {
+ id: "plate",
+ is3D: true,
+ startFrame: 4,
+ durationFrames: 40,
+ timeRemap: { kind: "rate", sourceStartSec: 1, rate: 0.5 },
+ });
+ comp.add(
+ Layer.box({
+ startFrame: 60,
+ durationFrames: 120,
+ children: [
+ Layer.box({
+ width: 64,
+ height: 64,
+ startFrame: 30,
+ durationFrames: 60,
+ children: [
+ Layer.camera({ x: 32, y: 32, z: -100, targetX: 32, targetY: 32 }),
+ layer,
+ ],
+ }),
+ ],
+ }),
+ );
+ const vision = comp.vision(layer);
+ expect(vision.startFrame).toBe(94);
+ // Its time remap comes along: 2 frames in, 1 s + 2/24 × 0.5 s of source.
+ expect(vision.clipFrame(96)).toBeCloseTo((1 + (2 / 24) * 0.5) * 24, 6);
+ });
it("shares a layer's live signal time with its render", () => {
const comp = new Composition({
width: 64,
diff --git a/packages/framefields/src/font-variations.test.ts b/packages/framefields/src/font-variations.test.ts
new file mode 100644
index 00000000..a263cb97
--- /dev/null
+++ b/packages/framefields/src/font-variations.test.ts
@@ -0,0 +1,128 @@
+/**
+ * `fontVariations` sets a variable font's axes on a text layer, as CSS
+ * `font-variation-settings` does: in layout (the measured box), in the
+ * drawn glyphs, in `FontManager.measure`, and in spans. Inter's optical
+ * size axis makes the check: at 100 px `opsz` follows the size to its 32
+ * maximum, while `{ opsz: 14 }` draws the wider text cut, about 11% wider.
+ */
+import { existsSync } from "node:fs";
+import path from "node:path";
+import { Canvas, loadImage } from "skia-canvas";
+import { beforeAll, describe, expect, it } from "vitest";
+import { Composition, FontManager, Layer, type LayoutNode } from "./index.js";
+
+const INTER = path.resolve(
+ import.meta.dirname,
+ "../../..",
+ "assets/fonts/Inter.ttf",
+);
+const hasInter = existsSync(INTER);
+const W = 1200;
+const H = 160;
+const TEXT_CUT = { opsz: 14 };
+
+/** Leftmost and rightmost columns with dark ink, and the first red column. */
+async function columns(png: Buffer) {
+ const img = await loadImage(png);
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ const data = ctx.getImageData(0, 0, img.width, img.height).data;
+ let left = img.width;
+ let right = -1;
+ let red = img.width;
+ for (let i = 0; i < data.length; i += 4) {
+ const x = (i / 4) % img.width;
+ const [r, g, b] = [data[i] ?? 0, data[i + 1] ?? 0, data[i + 2] ?? 0];
+ if (r < 90 && g < 90 && b < 90) {
+ left = Math.min(left, x);
+ right = Math.max(right, x);
+ }
+ if (r > 200 && g < 60 && b < 60) red = Math.min(red, x);
+ }
+ return { ink: right - left, red };
+}
+
+/** An auto-width text in a row, followed by a red marker the layout places after it. */
+async function render(text: LayoutNode): Promise<Buffer> {
+ const comp = new Composition({
+ width: W,
+ height: H,
+ fps: 30,
+ durationFrames: 1,
+ backgroundColor: "#ffffff",
+ });
+ comp.add(
+ Layer.flex({
+ id: "row",
+ position: "absolute",
+ x: 0,
+ y: 0,
+ width: W,
+ height: H,
+ dir: "row",
+ align: "center",
+ children: [
+ text,
+ Layer.box({
+ id: "marker",
+ width: 12,
+ height: 12,
+ background: "#ff0000",
+ }),
+ ],
+ }),
+ );
+ return comp.renderFrame({ frame: 0 });
+}
+
+const word = (fontVariations?: Record<string, number>) =>
+ Layer.text("framefields", {
+ id: "word",
+ fontFamily: "Inter",
+ fontSize: 100,
+ fill: "#000000",
+ ...(fontVariations && { fontVariations }),
+ }) as LayoutNode;
+
+describe.skipIf(!hasInter)("fontVariations", () => {
+ beforeAll(async () => {
+ await FontManager.register(INTER);
+ });
+
+ it("measures with the axes it is given", () => {
+ const auto = FontManager.measure("framefields", {
+ fontFamily: "Inter",
+ fontSize: 100,
+ });
+ const textCut = FontManager.measure("framefields", {
+ fontFamily: "Inter",
+ fontSize: 100,
+ fontVariations: TEXT_CUT,
+ });
+ expect(textCut.width / auto.width).toBeCloseTo(10946 / 9830, 2);
+ });
+
+ it("lays out and draws the text at those axes", async () => {
+ const auto = await columns(await render(word()));
+ const textCut = await columns(await render(word(TEXT_CUT)));
+ // The glyphs draw wider...
+ expect(textCut.ink / auto.ink).toBeGreaterThan(1.08);
+ // ...and the layout measured the same text, so the marker moved with them.
+ expect(textCut.red - auto.red).toBeGreaterThan(0.08 * auto.ink);
+ }, 60_000);
+
+ it("carries the axes into spans that change only the weight", async () => {
+ const spans = (fontVariations?: Record<string, number>) =>
+ Layer.text(["frame", { text: "fields", fontWeight: 700 }], {
+ id: "word",
+ fontFamily: "Inter",
+ fontSize: 100,
+ fill: "#000000",
+ ...(fontVariations && { fontVariations }),
+ }) as LayoutNode;
+ const auto = await columns(await render(spans()));
+ const textCut = await columns(await render(spans(TEXT_CUT)));
+ expect(textCut.ink / auto.ink).toBeGreaterThan(1.08);
+ }, 60_000);
+});
diff --git a/packages/framefields/src/fonts/manager.ts b/packages/framefields/src/fonts/manager.ts
index 6cb801c8..8ce4f88c 100644
--- a/packages/framefields/src/fonts/manager.ts
+++ b/packages/framefields/src/fonts/manager.ts
@@ -171,6 +171,8 @@ export interface MeasureTextOptions {
fontFamily: string;
fontSize?: number;
fontWeight?: string | number;
+ /** Variable font axes, as on `Layer.text` (`{ wdth: 68 }`). */
+ fontVariations?: Record<string, number>;
/** Extra px between characters, as on `Layer.text`. */
letterSpacing?: number;
/** Multiplier of fontSize when < 10, else px. Default 1.2. */
@@ -284,6 +286,7 @@ export class FontManager {
null,
undefined,
fontSize,
+ options.fontVariations,
);
if (!font) {
throw new Error(
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 7ce25c5e..ee00c481 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -161,7 +161,7 @@ import {
type CameraNode,
type CompositorProgramConfig,
type CompositorToProgramOptions,
- collectNodeOps,
+ collectLayersOnClock,
compositorToProgram,
type EnvironmentPreset,
type EnvironmentVolumetricFog,
@@ -1376,11 +1376,15 @@ export class Composition {
const vm = this.toVirtualMedia({
resolveMediaSignal: this.mediaSignalResolver(true),
});
- const layer = collectNodeOps(vm.children).find(
- (node) =>
+ // Layers inside a 3D scene count too, with their start on this clock.
+ const found = collectLayersOnClock(vm.children).find(
+ ({ vm: node }) =>
(node.operation as unknown as { id?: string }).id ===
sourceOrLayer.id,
);
+ const layer = found && {
+ operation: { ...found.vm.operation, startFrame: found.startFrame },
+ };
if (!layer)
throw new Error(
"Add the media layer to the composition before calling comp.vision(layer)",
@@ -1452,15 +1456,18 @@ export class Composition {
n.vision instanceof Vision &&
!(CLIP_SUBJECT in n.vision)
) {
+ // Scenes' layers too: a layer inside a 3D scene starts on the
+ // scene's clock, and its clip must analyse on this one.
layers ??= new Map(
- collectNodeOps(this.toVirtualMedia().children).map((node) => {
- const op = node.operation as unknown as {
- id?: string;
- startFrame?: number;
- timeRemap?: ResolvedMediaTimeRemap;
- };
- return [op.id, op];
- }),
+ collectLayersOnClock(this.toVirtualMedia().children).map(
+ ({ vm, startFrame }) => {
+ const op = vm.operation as unknown as {
+ id?: string;
+ timeRemap?: ResolvedMediaTimeRemap;
+ };
+ return [op.id, { startFrame, timeRemap: op.timeRemap }];
+ },
+ ),
);
const layer = layers.get(n.id);
const clip = this.clipForLayer(n.src ?? n.inputHandleId, n.vision, {
diff --git a/packages/framefields/src/scene-isolation-render.test.ts b/packages/framefields/src/scene-isolation-render.test.ts
index 184a4d64..a4c9e2e3 100644
--- a/packages/framefields/src/scene-isolation-render.test.ts
+++ b/packages/framefields/src/scene-isolation-render.test.ts
@@ -5,6 +5,7 @@ import {
HeadlessMediaRenderer,
Layer,
Layer3D,
+ Pixelate,
Transition,
} from "./index.js";
@@ -132,4 +133,42 @@ describe("3D scenes in a sequence", () => {
expect(g).toBeLessThan(15);
expect(b).toBeLessThan(15);
});
+ it("applies a scene box's effects over the finished scene", async () => {
+ // 40 px cells over a 200 px frame: pixelated, every cell is one colour.
+ const mixedCells = async (pixelate: boolean) => {
+ const box = scene(100, "#ff0000");
+ const comp = new Composition({
+ width: 200,
+ height: 200,
+ fps: 30,
+ durationFrames: 10,
+ backgroundColor: "#000000",
+ });
+ comp.add(
+ pixelate
+ ? box.withEffects([new Pixelate({ size: 40, shape: "square" })])
+ : box,
+ );
+ const img = await loadImage(await comp.renderFrame({ frame: 2 }));
+ const canvas = new Canvas(img.width, img.height);
+ const ctx = canvas.getContext("2d");
+ ctx.drawImage(img, 0, 0);
+ let mixed = 0;
+ for (let cy = 0; cy < 5; cy++)
+ for (let cx = 0; cx < 5; cx++) {
+ const { data } = ctx.getImageData(cx * 40 + 2, cy * 40 + 2, 36, 36);
+ let lo = 255;
+ let hi = 0;
+ for (let i = 0; i < data.length; i += 4) {
+ lo = Math.min(lo, data[i] ?? 0);
+ hi = Math.max(hi, data[i] ?? 0);
+ }
+ if (hi - lo > 24) mixed++;
+ }
+ return mixed;
+ };
+ // The cube's edges cross cells; the pixelated scene has none mixed.
+ expect(await mixedCells(false)).toBeGreaterThan(0);
+ expect(await mixedCells(true)).toBe(0);
+ });
});
diff --git a/packages/renderers/src/text-measurer.ts b/packages/renderers/src/text-measurer.ts
index 38c30599..fa81572c 100644
--- a/packages/renderers/src/text-measurer.ts
+++ b/packages/renderers/src/text-measurer.ts
@@ -7,6 +7,8 @@ export interface TextMeasureStyle {
fontSize?: number;
fontWeight?: string | number;
fontStyle?: string;
+ /** Variable font axes, as `TextNode.fontVariations`. */
+ fontVariations?: Record<string, number>;
letterSpacing?: number;
lineHeight?: number;
padding?: number;
@@ -62,6 +64,7 @@ export function measureText(
null,
undefined,
fontSize,
+ style.fontVariations,
);
const padding = style.padding ?? 0;
diff --git a/packages/webgpu-renderers/src/slug/slug-font-cache.ts b/packages/webgpu-renderers/src/slug/slug-font-cache.ts
index c49c2859..34fcbe40 100644
--- a/packages/webgpu-renderers/src/slug/slug-font-cache.ts
+++ b/packages/webgpu-renderers/src/slug/slug-font-cache.ts
@@ -667,6 +667,7 @@ export class SlugFontCache {
generated.hasTrueWeight = hasTrueWeight;
generated.weight = weight;
generated.fontFamily = fontFamily;
+ generated.variableAxes = variableAxes;
SlugFontCache.cache.set(cacheKey, generated);
return generated;
} catch (e) {
diff --git a/packages/webgpu-renderers/src/slug/slug-geometry.ts b/packages/webgpu-renderers/src/slug/slug-geometry.ts
index 0108659f..4f2a0f0f 100644
--- a/packages/webgpu-renderers/src/slug/slug-geometry.ts
+++ b/packages/webgpu-renderers/src/slug/slug-geometry.ts
@@ -473,6 +473,7 @@ export class SlugGeometry {
null,
undefined,
spanFontSize,
+ defaultFont.variableAxes,
);
if (resolved) spanFont = resolved;
}
diff --git a/packages/webgpu-renderers/src/slug/slug-loader.ts b/packages/webgpu-renderers/src/slug/slug-loader.ts
index acfbe492..fd390b20 100644
--- a/packages/webgpu-renderers/src/slug/slug-loader.ts
+++ b/packages/webgpu-renderers/src/slug/slug-loader.ts
@@ -35,6 +35,8 @@ export interface SlugFont {
fontFamily?: string;
hasTrueWeight?: boolean;
weight?: number;
+ /** The variable axes this instance was asked for (spans inherit them). */
+ variableAxes?: Record<string, number>;
}
/** `SlugCodePoint.codePoint` of a glyph that only shaping reaches. */
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index 37e7499d..3bb8d8ac 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -74,7 +74,7 @@ All `Layer.*` builders return `AnimatableNode<T>` instances with chainable `.ani
| Builder | Node `kind` | Description & Key Options |
|---|---|---|
-| `Layer.text(content, options)` | `"text"` | Slug GPU vector text. Options: `fontSize`, `fontFamily`, `fill`, `letterSpacing`, `lineHeight`, `align`, `direction`, `animators`, `spans`. Arabic, Hebrew and Persian are shaped and laid out right to left automatically (`direction: "auto"`); set `"rtl"`/`"ltr"` to force it. `align: "start"`/`"end"` follow the direction. |
+| `Layer.text(content, options)` | `"text"` | Slug GPU vector text. Options: `fontSize`, `fontFamily`, `fontWeight`, `fontVariations` (variable font axes, e.g. `{ wdth: 68 }` for a condensed cut; spans inherit them), `fill`, `letterSpacing`, `lineHeight`, `align`, `direction`, `animators`, `spans`. Arabic, Hebrew and Persian are shaped and laid out right to left automatically (`direction: "auto"`); set `"rtl"`/`"ltr"` to force it. `align: "start"`/`"end"` follow the direction. |
| `Layer.box(options)` | `"box"` | Rectangular container / card. Options: `background`, `borderRadius`, `borderColor`, `borderWidth`, `overflow` (`"visible"` by default; `"hidden"` clips children to the box and its rounded corners), `padding`, `children`. |
| `Layer.flex(options)` | `"flex"` | Auto-layout container (Yoga). Options: `dir: "row" \| "column"`, `gap`, `justify`, `align`, `padding`, `wrap`, `children`. |
| `Layer.video(src, options)` | `"media"` | Video layer decoded via mediabunny. Options: `fit: "cover" \| "contain"`, `trimStartSec`, `muted`, `volume`. |