Skip to content

Code-first video & image, rendered natively on WebGPU.

62.3 KB
Raw Blame

framefields

2.14.0

Minor Changes

  • 976c782: Containers no longer clip their children by default: overflow defaults to "visible", as in CSS, so tilted badges, outside strokes and shaking titles are not cut at a parent's edge. Set overflow: "hidden" where children must stay inside a box, such as a rounded card holding a square header or image. Text layers also make room for their own outside stroke and shadows, which used to be cut at the text box, and scale or rotate around their anchor within that room.

  • 01954c0: 3D scenes render in isolation. A box with a fixed size that holds a camera is its own scene: its camera, lights and environment see only its layers, it runs on its own clock, and it is placed like a picture, so Layer.sequence transitions between 3D scenes blend two finished pictures instead of drawing both through one camera. Signal tracks take { clock: "layer" } to sample at the layer's own frame. comp.lint() reports duplicate_id. Per-pass 3D buffers are recycled once per frame (per command encoder), so a frame holding several 3D scenes never reuses a buffer an earlier pass still reads, and bloom mip chains live until the frame is done. Size-dependent 3D render targets (depth, HDR, MSAA, reflections) are cached per size and kept for the frame, so scenes of different sizes can share a frame.

  • 976c782: Add opt-in geometry-aware scene outlines and art-directed toon lighting for meshes, 3D cards and native text. Outline widths scale with export resolution; exclusions retain occlusion, and ink follows fog before camera depth blur and bloom. Toon materials now use colored direct lights, point/spot attenuation and cast shadows, with configurable shadow/lit colors and terminators. Correct 3D card pivot transforms so changing an anchor does not move identity geometry.

    Shade ink before the MSAA color resolve to preserve subpixel coverage between outlined surfaces. Custom cube-face artwork inherits surface settings while retaining explicit face overrides.

Patch Changes

  • 69e2ed8: Text with glyph animators keeps its stroke and shadow colours. The animator colour used to start from the base fill and override every pass, so outlined or shadowed text turned into a solid fill-coloured blob; colour animators now start from each glyph's own fill (spans and highlights included) and tint only the fill pass.

2.13.1

Patch Changes

  • 1835ddf: A hand-drawn arrow's head no longer spins. Annotate's arrow aimed its head from the tip to the last point of the wobbled shaft, a segment a few pixels long that the boil moved on every re-pose, so the head swung round (fully, on short arrows), and the wobbled shaft stopped short of it. The head now follows the clean curve's direction a head's length back from the tip, the same in every pose, and the hand lets go of the shaft toward the tip so the two always meet.
  • 1835ddf: Paths with several subpaths fill as drawn. A path shape's fill joined every subpath into one polygon, so a map's states and islands, a ring or a letter with a counter came out as wedges bridging them. Each subpath is now a contour; contours nest, holes follow SVG's nonzero rule, and each filled contour is triangulated with its holes (earcut). Fills are cached by path data, so a large path is triangulated once rather than every frame. Animating d (path morphs) also passes schema validation now: its keyframes are path data, not numbers, which Annotate's hand-drawn boil relies on.
  • 1835ddf: strokeDashArray renders. It is written as SVG writes it ("12 8", "4,8"), but the string reached the path pipeline as if it were a list of numbers, the dash arithmetic came out NaN, and the whole stroke was dropped: dashed lines, circles and outlines drew nothing. It is now read by SVG's rules (an odd list repeats; a negative, unreadable or zero-length pattern draws solid).
  • 1835ddf: Strokes draw in one call per path. The path pipeline drew every straight piece of a stroke (and of the hairline that antialiases a solid fill) as its own draw, with two buffer uploads each, so a curve, a dashed line or a map's coastline cost thousands of native calls a frame. Each segment is now an instance of one draw: the path's transform, color, width and cap are uploaded once, its segments in one buffer, and the vertex shader builds each segment's quad. Output is unchanged; a scene of map coastlines and a hundred filled circles renders about twice as fast.
  • 1835ddf: A layer keyed up from volume: 0, or keyed out of muted: true, plays. The mix baked a layer's resting volume and mute into its sources (skipping a source at 0 altogether) and the layer's keys only rescaled what was left, so a fade in from a resting volume of 0 stayed silent for good. Props a layer keys are now applied by its keys alone; resting values it does not key are applied as before. A ducked music bed written Layer.audio(music, { volume: 0 }).animate(LayerAnimation.keys("volume", …)) is heard.
  • 1835ddf: A keyframe track holds its first value until it is reached. Baked tracks showed the layer's own value before their first key, so keys("opacity", [[40, 0], [48, 1]]) drew the layer fully opaque for 40 frames and then snapped it out to fade in, and keys("y", [[30, 440], [40, 400]]) sat at 400 before jumping to 440. Draw-time tracks (signals, stepped layers, path morphs) already held the first key, so the same track now plays the same either way. Discrete props (hidden, muted) still step from the layer's own value, as sets do.

2.13.0

Minor Changes

  • a1bc911: Keep media out of git on your own bucket. framefields assets add|push|pull|status stores files by sha256 on any S3-compatible bucket (Cloudflare R2, S3, B2, MinIO) and pins them in framefields.json; a remote's publicUrl (a custom domain or CDN) serves reads without keys, and uploads are marked immutable so CDNs cache them. Cloud renders upload what the bucket lacks, sign reads of private remotes, and workers download and cache them by hash. framefields/storage exposes the client.
  • 7edabea: framefields render <name> renders a named render on this machine and framefields preview <name> plays it in the browser, reading the same framefields.json render as --cloud, so a render writes the same file wherever it runs. Both load the module with the project's own tsx and framefields. A local render first downloads pinned assets that are missing; the preview streams them from their bucket instead (redirecting the page to the remote's publicUrl or a signed URL, and mixing the soundtrack over HTTP), so an agent in a cloud sandbox previews footage without downloading it. init's render and preview scripts now call the CLI. A local render runs QA (qa: true), prints the report and exits non-zero when QA fails. Every framefields command loads the project's .env (variables already set win), so render --cloud and the storage commands find FRAMEFIELDS_API_KEY and remote keys kept there.
  • 786503b: The preview mixes its soundtrack in the browser, around the playhead. The page runs the export's mixer (same decoders, audio effects on its GPU, signals) on 2-second blocks as it plays, each continuing the effect state of the block before, so from the film's start it is the export's mix sample for sample (after a seek the first block warms its effects up from 2 s earlier until the background pass replaces it), and plays them back to back on the AudioContext clock; the rest of the film mixes in the background for the waveform and the master gain. Sources are read in byte ranges, so pinned assets stream from their bucket and the server sends no WAV. The mixing runs in a module worker (the project's bundle, loaded as one) with its own GPU device, so it never stalls the picture; the page mixes if the worker can't, and the server only if neither can. mixAudioTracks and renderAudio take a window (the same samples as that stretch of a full mix), continue (carry the last mix's effect state, so back-to-back windows are exact) and master: false.

Patch Changes

  • 1064591: Audio effects start every mix from silence. Effect state on the GPU (delay lines, reverb tails, envelopes) was keyed by render id and carried into the next mix that reused one, so mixing again under the same id (the preview, block by block) heard the previous mix's echoes: a ping-pong delay was off by up to 0.43. mixAudioTracks now resets it first (WebGPUAudioProcessor.resetState, which keeps compiled pipelines), renderAudio frees each mix's state, and the video export's cleanup, which passed no device and so freed nothing, now releases it.

2.12.0

Minor Changes

  • 66ecdae: Named renders in framefields.json: renders.<name> points at a module and export with its title, description and render defaults (workers, codec, quality, output). framefields render <name> --cloud, framefields check <name> (no name checks every render), framefields renders and renderCloud({ render }) pick them by name, and the dashboard lists them so nobody selects files. init records a film render and its scripts use it.

2.11.0

Minor Changes

  • AudioSignal.pitch(source, { fps, minHz, maxHz }) tracks the fundamental frequency of audio or video on the GPU: YIN period candidates per frame, then a Viterbi decode that rejects octave and subharmonic errors. It returns hz, confidence and voiced signals; hz is interpolated across unvoiced gaps. Use it for engine RPM, melody lines or any pitched source.

  • Split a long render across GPUs. renderVideo({ frames: { start, end }, audio: false }) renders one chunk of frames: it starts on a keyframe, its timestamps start at 0, and trails, datamosh and particles rebuild their state before start. stitchChunks({ chunks, audio, outputPath }) from framefields/renderer joins the chunks and a separate audio-only render (codec: "aac") into one file without re-encoding or a GPU. It refuses chunks encoded with a different codec, size or encoder parameters. planChunks({ totalFrames, samples, workers: "auto" }) decides how many chunks a render should use (more only while each chunk stays at least 85% efficient against the per-worker start-up) and cuts them at equal render cost; renderer.probeRender(comp) measures the frame costs it needs.

    framefields render <entry> --cloud [--workers auto] [--plan] renders the pushed commit of the project's git repo on api.framefields.dev and downloads the video; --plan prints the workers, time and estimated cost without rendering. The same from code: renderCloud() from the new framefields/cloud entry, which needs no GPU. framefields init adds a render:cloud script.

  • ab1cc06: FocusZoom takes drift: a slow push while each shot holds (zoom gained per second, e.g. 0.012), so the picture never stops between camera moves. Defaults to 0, which holds still as before.

  • Depth and relief: footage as a 3D surface.

    • comp.vision(clip, { depth: true }) analyses each frame's relative depth with Depth Anything V2 Small (Apache-2.0, pinned by revision and SHA-256), ahead of render and cached one file per frame. Read it with clip.depthMap(frame); runner.depth(image) runs it on one image.
    • withVision({ mode: "depth" }) draws a clip's depth as gray (white = near), always for the frame on screen; it makes its depth clip when there is none.
    • layer.withRelief(heightLayer, { amount }) lifts a 3D layer into a surface by another layer's brightness, along the default camera's rays: from that lens it is the flat layer, pixel for pixel; a moving camera sees it in 3D, lit by the scene and hiding what passes behind it. With a depth layer, footage joins the 3D scene.
    • Layers inside a matte or relief now link to their comp.vision clips too.
    • The render GPU device now requests the adapter's full buffer limits. Vision models share that device, and one whose intermediates exceed the 128 MiB default binding (Depth Anything's attention) computed on truncated data instead of failing.

Patch Changes

  • Text, shapes, images and SVGs stay sharp when an ancestor scales them up. A 2D leaf now rasterizes at its scale on the canvas (up to 4×) instead of magnifying a layout-size texture, so a camera push or a card at scale: 2 keeps crisp glyphs, round dots and clean borders. Text shadows, masks, motion blur, LayerStyle and Gaussian/box Blur scale with the layer; a leaf whose effect doesn't declare how far it scales keeps its layout-size raster, as do 3D and deflickered leaves. Layers at scale 1 or below render exactly as before.

2.9.3

Patch Changes

  • eeeae1e: SAM ships in framefields: comp.vision(clip, { sam2 | sam3 }) works from a plain npm install framefields. The @framefields/vision-sam peer it asked for was never published, so prompted tracking failed for every npm user. The SAM code is a lazily loaded chunk, fetched only when a clip uses sam2 or sam3; the trackers are exported as framefields/sam. No weights ship: they download on first use, SAM 3's only after you accept Meta's SAM License, as NOTICE.md describes.

2.9.2

Minor Changes

  • c785aa1: sam3: { redetect: N } runs the last text prompt again every N frames and adds the instances no tracked object covers yet: people who walk in, step out from behind someone, or were lost. With merge they join the one object (what it already tracks stays at full resolution); otherwise they become new track IDs up to maxObjects. The text features are encoded once per prompt.
  • a1e37b8: A withVision({ mode: "skeleton" }) layer of a clip that comp.vision analyses now draws the pose that clip matched to its subject, for the frame on screen, instead of running its own pass and drawing the first person detected. A skeleton layer of a SAM clip needs pose: true on the clip and fails clearly without it. Skeleton mode draws only the skeleton: a frame without a pose is empty rather than showing the footage. SAM clip caches written before this change claimed pose they did not track; they are invalidated, so SAM clips re-analyse once.

2.9.0

Minor Changes

  • dac3694: @framefields/vision-sam is now an optional peer of framefields, loaded only when a clip uses sam2 or sam3, so the SDK no longer installs or imports SAM-licensed code by default. Install it next to framefields to use prompted tracking, and import Sam2Tracker/Sam3Tracker from @framefields/vision-sam. Mask-derived object geometry moved to @framefields/vision as maskObject (formerly sam2Object).
  • 7af1f91: Add prompted SAM 2.1 Tiny and SAM 3 video segmentation and semantic text selection to clip vision, decoupled into a dedicated @framefields/vision-sam package under the Meta SAM License. Includes temporal memory, point/box corrections, semantic text selection, cached frame seeking, and time-remapped matte reuse. Selected-object masks, geometry, motion, and optional matched pose are exposed through standard vision signals. Model assets download dynamically at runtime to local cache with SHA-256 verification upon explicit license agreement.
  • 9a82bdb: SAM 2.1 Small: sam2: { model: "small" } tracks at SAM 2's native 1024² from Diffusion Studio's pinned fp16 export (~112 MB, ~1.1 s/frame on an M1 Pro against Tiny's ~0.27 s), for small, thin or detailed subjects and identity after cuts. Sam2Tracker derives every size from the model's constants; sam2Revision(model) replaces SAM2_TINY_REVISION. The framefields-vision skill says when to choose Tiny, Small or SAM 3.
  • aec869b: sam3: { text, merge: true } selects every instance the text matches as one object: one mask and one memory, so a whole crowd costs the same as a single track. maxObjects caps the instances merged (default: all).
  • f1623c0: SAM 3 tracks several objects at once: sam3.maxObjects lets a text prompt select up to N instances (tracks 1…N by score) through one encoder pass per frame, and points prompts take object to seed or correct one track. comp.vision publishes every track in signals.objects and signals.masks; the clip matte is their union. Fixes concurrent comp.vision clips of one source dropping decoded frames (each clip now owns its decoder) and concurrent ORT WebGPU session creation falling back to the CPU (creation is serialized; SAM trackers share the process provider).

Patch Changes

  • 645dd13: SAM 2.1 (Apache-2.0 weights) no longer asks for Meta's SAM License; only SAM 3 does (acceptSamLicense / sam3.acceptLicense / FRAMEFIELDS_ACCEPT_SAM_LICENSE=1). sam2.acceptLicense and Sam2Tracker's acceptLicense are removed. @framefields/vision-sam is Apache-2.0 like every framefields package (SAM 3's weights stay under Meta's SAM License, accepted at runtime); its sources no longer carry a Meta copyright header, and the model is named SAM 3 throughout.
  • 72f6aaa: SAM 3 downloads the image encoder its backend runs fastest: fp16 on WebGPU and CUDA, q4 on the CPU (SAM3_ENCODER, override with sam3.precision). Providers report their backend before any download (SessionProvider.backend(), sessionCompute); the auto provider decides with a one-node probe graph and, on Linux with the NVIDIA driver, tries CUDA first. All SAM 3 trackers in a process share one encoder session and reuse its features per frame. Text-seeded objects now output and remember their seed mask on the prompt frame, as SAM does, instead of a speckled re-decode.

2.8.1

2.8.0

Minor Changes

  • b1a40b4: Bound frame-grid sample generation before allocation, validate numeric sampling inputs, and keep automatically chosen samples inside the exported frame window. Render hooks and pixels now use one timestamp-selected frame; timestamp labels no longer change frame selection through rounding. Composition rendering rejects mismatching FPS overrides.

    Fix sequence duration with outgoing overlaps and delayed layers that previously outlived their scene. Explicit scene-layer windows must fit their owning scene, and overlaps cannot exceed either adjacent scene. Frame indices and frame-range steps must be safe integers; fractional timestamps and FPS remain supported.

    Add framefields check <entry> --json with isolated builder execution, schema/lint/timeline diagnostics, bounded worker lifetime and optional --render frame/grid smoke checks. Scaffolds include check and check:render scripts. Add JSON component/font discovery and an API reference generated from the owning timing/preflight types via framefields api.

    Normalize animation-builder inputs to strict program specs at the layer-builder boundary. Ambient light helpers now also accept an options object, consistently with other light helpers, while preserving positional color/intensity calls. These fixes allow the launch example to pass schema preflight.

Patch Changes

  • 66e7530: Include Noto Color Emoji font by default in framefields init alongside Inter and JetBrains Mono.
  • b5c6470: Sort blended 3D meshes across the scene and render them without depth writes. Apply glTF base-color alpha once.
  • e96a32f: Render model and procedural mesh wireframes as triangle edges, including animated and shadow-casting meshes.
  • 9609991: Render all glTF primitive modes, per-map UV sets and transforms, sparse accessors, animated morph targets, Draco or meshopt compressed geometry, and ETC1S/UASTC KTX2 textures. Reject unsupported required glTF extensions explicitly.

2.7.0

Minor Changes

  • 18aece7: Add forward and reverse media time remapping for video and audio layers, including smooth ramps, video holds, silent audio holds, source range checks and clip vision synchronization. Reverse video decoding uses a bounded frame cache.
  • e56c7f5: Drive media timeRemap from a signal: { kind: "signal", mode: "speed" | "source-time", signal } accepts a signal instance or a named composition signal. Without frame hooks the signal is sampled up front; with hooks it is recorded live after each frame's hooks, replayed for seeks, audio, vision and the preview, and its source bounds are checked as it records.

Patch Changes

  • e56c7f5: Preserve the supplied frame progress when evaluating programmatic signals, and prevent cached frame-signal values from being reused across different frame rates or frame contexts.

  • 5c18fd8: TextReveal type no longer shows the whole text from the first frame when emphasis (or any other glyph animator) is set. The compositor dropped a text layer's text count track whenever the layer also had glyph animators. The two now compose, because animators index the full string. type also no longer reflows: each glyph switches on at its keystroke inside the full layout, so centred and wrapped lines stay put as they type, and the backspace exit removes the last glyph first.

    TextReveal slam words are no longer cut off at the text's box while they land from 2.4×. A component's root and stage boxes no longer clip, since a component's box is its footprint, not a clip. Only the mask-* enter and exit presets still cut to it. This also restores the full drop shadow under CodeBlock and AppFrame windows, which used to show as grey triangles at the bottom corners.

    LowerThird's tag layout scales its one-line pill down to fit the safe area, instead of running past it.

    Breaking for anyone who used it: TextReveal's track style (enter and exit) is removed. TitleCard's kicker now enters with rise.

2.6.2

Patch Changes

  • 7a9e97d: Symbols like ✓ ✗ ★ ☆ ❯ © ™ draw from the text font instead of as emoji. Layout treated every character in U+2600–27BF, and every emoji-capable pictograph, as a color emoji, so a ✓ (which no emoji font has) drew nothing and "© 2026" got an emoji bitmap. Text now follows Unicode's emoji presentation rules: a grapheme is emoji when it defaults to emoji presentation (🚀 ✅ ⚡), carries VS16 (❤️), a skin tone or a keycap, or is a ZWJ sequence. Pictographs that default to text (❤ ✈ ☀ ✔ © ™) use the text font's glyph when it has one, and the emoji font when it doesn't. A glyph missing from the text font is taken from another registered family that has it, so a ✓ in JetBrains Mono draws from Inter instead of as a .notdef box. The missing_emoji_font lint uses the same rule, so it no longer flags © or ✓.

    Terminal also drops a done line's leading check mark when color markup wraps it ({green}✓{/} done), as it already did for a bare ✓ done. The drawn check takes its place. Until now the text ✓ drew nothing, so the second mark went unnoticed.

2.6.1

Patch Changes

  • 126fccb: - ChatThread lasts until a reaction on the last message has popped. Its natural length ended 0.8 s after the last bubble, but a reaction pops 0.6 s after its bubble and springs for longer, so with an exit or a fitted clip the pop was cut off and the reaction never showed.
    • Fonts registered in the browser (the live preview, the docs lab) no longer warn [registerHeadlessFont] … mkdtemp is not a function: the Node-only local copy is skipped in a page, where text draws from the registered source. Headless file:// font paths with spaces resolve too.

2.6.0

Minor Changes

  • 7f58e58: Start projects with framefields init, install fonts with components, and polish the component toolkit.

    • npx framefields init [dir] writes a working project: package.json (ES module, scripts, dependencies, packageManager, and install-script approvals that let only skia-canvas run its script), tsconfig.json, src/theme.ts / film.ts / render.ts, assets/, an LFS .gitattributes, .gitignore and README.md. It also installs Inter and JetBrains Mono and runs git init. It never overwrites: in an existing project, files only gain what they lack. Options: --pm, --install, --emoji, --no-fonts, --no-git, --name. It asks no questions, so it runs unattended in CI. The AI gateway key is optional: init writes .env.example, git-ignores .env, and the generated render.ts loads .env when it exists.
    • Fonts come with the components that need them. framefields add chat-thread / captions installs Noto Color Emoji, and terminal / code-block install JetBrains Mono. Each font is pinned by size and SHA-256, saved with its OFL license, recorded in framefields.json and registered in a generated fonts.ts (await registerFonts()). framefields fonts add <emoji|sans|mono> and fonts list install fonts directly, and --no-fonts skips them. The package itself ships no fonts.
    • comp.lint() reports missing_emoji_font when text has emoji but no emoji font is registered, and the render warning now names the fix. FontManager.hasEmojiFont() is new.
    • Toolkit: withDefaults({ fps, canvas, theme, seed }) and after(node, gap) (also on the root), a canvas base prop, prop typos answered with "did you mean", partNames on every node, a blur option on motion presets, and shared keyframe, color, layout and surface helpers.
    • Themes gain mode, font weights, elevation, springs, color.shadow and syntax. LowerThird, Toasts, Callout, CardStack and ChatThread surfaces get depth (shadow, lit top edge, sheen), and sliding surfaces get motion blur. ChatThread reactions are emoji text drawn from the color emoji font.
    • Fix: fonts registered from a file:// URL whose path has spaces (or a Windows drive letter) are found again. The browser preview failed to load them.

2.5.5

Patch Changes

  • demos

2.5.4

Patch Changes

  • Compositor container fast paths and effect reach optimization:
    • Containers draw background, children, and border directly onto the destination when no isolation or corner masking is required.
    • Effects can declare their spatial reach (EffectReach), allowing the compositor to bound container effect passes (such as LayerStyle drop shadows, glows, outer strokes) to the layer's reach rather than the full canvas.

2.5.3

Patch Changes

  • docs-fixes

2.5.2

Patch Changes

  • Fix preview components entry, registered font sources, and GPU device reuse:
    • Browser engine preview adds framefields/components entry.
    • Fonts registered with FontManager (bytes or URL) load from registered sources in Slug font cache and text measurer.
    • Preview players share a single WebGPU device and clean up on disposal.

2.5.0

Minor Changes

  • 0695665: Initialize components

2.4.3

Patch Changes

  • Strict compile-time type safety across the engine harness:
    • Branded types for coordinate and unit spaces (Pixels, Normalized01, Degrees, Radians, Frames, Seconds, Hertz, Decibels).
    • Discriminated union types and capability-safe layer ports on NodeEffect, Layer nodes, and effect classes.
    • Compile-time exhaustiveness checking with assertNever and type-narrowed animation property targeting.
    • Schema and pipeline unification with generic node operations.

2.4.2

Patch Changes

  • 4f5339e: Clearer render progress, sized for agents' context:
    • renderVideo logs each phase (prepare, frames, audio mix, audio encode, mux, GIF conversion) and a summary with per-phase timings. Frames log at every 10% with fps and ETA instead of every 10 frames, so an export prints about 15 lines whatever its length.
    • A phase that runs long logs Still … lines (after 10 s, then backing off to every 5 min) naming the phase and its progress, so the audio mix or the mux after the last frame no longer looks like a hang. A failed render names the phase it stopped in.
    • onProgress reports the new phases prepare, audio-encode, mux, convert and done alongside render and audio, with elapsedMs, and fps, etaSec, phaseProgress and detail where they apply. The audio mix reports each clip it mixes (mixAudioTracks takes an onProgress option), which also keeps the render worker's stall detector from firing on long mixes.
    • The engine's logs are one short [framefields] … line each on stderr instead of pino JSON, about a fifth of the size. NODE_ENV=production keeps JSON on stdout; FRAMEFIELDS_LOG_FORMAT=json|text picks either. The render worker keeps JSON.
    • Removes the unused video-worker entry from @framefields/renderer.

2.4.1

Patch Changes

  • Speed up audio mixing for long timelines:
    • Decode only the stretch of a source each clip plays, 0.5s early so MP3/AAC decoders warm up; samples match a whole-file decode exactly.
    • Skip subtrees with no source and no sound-producing processor.
    • Pass a lone child's buffer up instead of copying it per level.
    • Read plain value signals once instead of once per sample.
    • Keep up to 8 effect chunks in flight on the GPU before reading back.
    • ffmpeg fallback: drop the 100 MB output cap and decode only the needed range.

2.4.0

Minor Changes

  • dcee493: Add ai: AI models (images, video, music, voice, …) through the framefields AI gateway, authenticated with FRAMEFIELDS_API_KEY and billed to your framefields account. subscribe, run, stream, queue and storage, with no dependencies: import { ai } from "framefields".

2.3.2

Patch Changes

  • 1c5919f: Fix native C++ memory leaks in headless video and still renders:
    • Decouple headless Lottie rendering from skia-canvas, reading straight-alpha pixels directly from the underlying ThorVG WASM linear buffer (_dotLottieCore.get_pixel_buffer()) into an instance-reused upload buffer before writing to the WebGPU texture, eliminating megabytes of native heap leakage per frame.
    • Add frame-based eviction (mediaDecoderCache.tick()) for video and audio decoders of clips that ended over 120 frames ago, and pool still render IDs to prevent decoder re-instantiation across consecutive stills.

2.3.1

Patch Changes

  • 0415bc7: Fix 3D extruded text in the browser preview filling the notches of letters such as V and M. Without skia's outline merge, glyph contours that cross themselves (common in variable fonts) were triangulated as-is; they are now split at their self-crossings and filled with the nonzero rule, matching rendered output.

2.3.0

Minor Changes

  • 303c410: withVision({ mode: "matte" }) (and mask / crop) on a video layer now reuses comp.vision(clip)'s per-frame results when the composition already analyses that clip, instead of segmenting every frame a second time during render. The clip keeps each frame's full-resolution subject mask (merged the same way as the matte), persisted beside its other results in the vision cache, and the cutout uses the mask for the frame being drawn, so it no longer runs one frame behind the plate. Layers whose settings would give a different subject (classes, keyBackground, matteSource: "selfie", a different variant / confidence / maskThreshold / featherRadius, or an attached vision bundle) keep their own segmentation, and so do frames the clip has not analysed.

  • 4625cc7: Add rigid-body physics: Physics.world2d / Physics.world3d (Rapier, deterministic) simulate once while the composition is built and replay as per-frame signals. Bodies come from layers (boxes, circles, convex hulls of shapes, measured text, Layer3D primitives and models); simulate() binds each layer, and every body exposes impact, speed, velocity and pose signals for driving any prop or effect. Includes ground/walls, pins, hinges, welds, springs, ropes, signal-driven forces and kinematic bodies, collision lists, and bakeKeyframes() for plain keyframes.

    world.field() adds position-dependent forces (wind, vortices, flow fields). world.live(comp) steps a world as the composition renders instead of all at once: before each frame draws it advances to it, after that frame's inputs are ready, giving the same result as simulate().

    comp.vision(clip) analyses a clip's own frames on demand (pose keypoints and a soft silhouette) and shares the results with everything that reads them; every render path and the preview await it before drawing a frame, and results persist on disk in Node. comp.onPrepareFrame() registers async per-frame work, and the renderer awaits an onRequestFrame that returns a promise. colorSignal takes multiplier and offset.

    Layer.particles adds GPU particle systems: a compute-shader simulation (emitters, curl-noise flow, attractors, gravity, drag, floors and bounds, a performer's silhouette push) drawn in one instanced call, with snapshots so scrubbing back restores state. New temporal effects: Trails (echoes, light painting, feedback tunnels) and Datamosh (pixels dragged along optical flow between keyframes).

    3D glass, acrylic and frosted materials now refract what is behind them on screen (with dispersion, thickness, tint and studio reflections) instead of drawing as translucent plastic. Particle emitters can move: emitter.x / emitter.y take drivers, and births spread along the path between frames (light painting from a tracked hand).

  • 6685719: Add a stylize effect pack: Pixelate (square, circle/LED and diamond cells), Posterize (levels or retro palettes: GameBoy, teletext, CGA, PICO-8, C64, grayscale, sepia, custom; with Bayer dithering and big pixels), Dither (Bayer, noise and halftone; 1-bit duotone or per channel), Ascii, Toon (cel bands and ink outlines), RGBSplit, CRT (curvature, scanlines, aperture mask, plus VHS tracking and colour bleed), Painterly (Kuwahara oil and watercolour), Contour (topographic lines), Kaleidoscope, OpArt, Iridescence (holographic foil) and Gooey (metaballs).

    Styles.* presets return ready-made effect stacks for common looks (pixelArt, gameBoy, teletext, ledWall, ascii, oneBit, risograph, comic, cartoon, thermal, vhs, crt, glitch, oilPainting, watercolor, topographic, kaleidoscope, opArt, holographic, noir, film): layer.withEffects(Styles.vhs()).

    LayerAnimation.stepped(n) gives a layer stepped timing ("on twos"): every animated value is held for n frames, eases kept. A shape's d is now animatable: keyframed SVG paths morph into each other, resampled and aligned so paths with different point counts, curves and subpaths morph cleanly.

    Layer.vfx() draws ProceduralVFX looks as their own layer (the node was not registered before), and adds an Aurora effect type.

2.2.0

Minor Changes

  • cf65727: Headless rendering for long-lived workers and any Linux machine:

    • Video encoding uses a hardware encoder (NVENC, VideoToolbox) only after testing that it works, and software encoders (libx264, libx265, libvpx, SVT-AV1) otherwise. This fixes H.264/HEVC renders on linux-arm64, where h264_rkmpp was picked without Rockchip hardware, and on NVIDIA GPUs without NVENC (A100, H100). Set FRAMEFIELDS_HW_CODECS=off to always encode in software.
    • WebGPU falls back to a software adapter (e.g. lavapipe) when no GPU adapter is available. The new describeGpu() reports the adapter in use and whether it is a CPU one.
    • renderVideo accepts signal (AbortSignal) and onProgress; resetRenderCaches() clears per-render caches between jobs in one process.
  • 60c35a0: Faster on-device vision, on the GPU by default:

    • Vision layers no longer reload their models every frame. Layers whose options differed (e.g. one with classes: ["person"] and one without) used to close and recreate their ONNX sessions on each frame; they now share sessions through VisionRunner.derive(). This also fixes the thread constructor failed: Resource temporarily unavailable crash with onnxruntime-web installed.
    • Layers that hand vision the same pixels (two panels of one video) share one RTMDet-Ins pass.
    • onnxruntime-web is now an optional dependency of framefields, so inference runs on WebGPU when there is a hardware adapter. Dawn's GPUSupportedFeatures gains the forEach that onnxruntime-web's WebGPU build needs. Software adapters (lavapipe, SwiftShader) use the CPU instead.
    • CPU sessions on Linux size their thread pool from the CPUs the process may use, cgroup quota included, instead of the host's cores. FRAMEFIELDS_VISION_THREADS overrides it.
    • FRAMEFIELDS_VISION_PROVIDER (auto, webgpu, cpu, cuda) forces a backend; cuda uses onnxruntime-node's CUDA EP on Linux x64.

    The smoke project's vision export (three vision layers, 60 frames of 1280×720) on an Apple Silicon Mac: 84.9 s before; 41–43 s on the CPU and 26–29 s on the GPU after. In the renderer worker on a RunPod RTX 3070: 58.7 s before, 22–25 s after.

Patch Changes

  • 9b70fda: Asset and font paths no longer depend on the BASE_URL and R2_CUSTOM_DOMAIN environment variables, left over from the Gatewai web app. With BASE_URL set, a font registered from an absolute path was fetched from $BASE_URL/<path>, and when that failed the engine quietly used a font from the R2_CUSTOM_DOMAIN CDN instead of the project's file. Absolute paths are now always read from disk, and Lottie sources and fonts are used as given instead of being prefixed with http://localhost:8081.

    Removed from @framefields/client-utils: getEnv, BASE_URL, R2_CUSTOM_DOMAIN, configureAssetUrls and the options argument of GetAssetEndpoint, GetFontAssetUrl, resolveMediaSourceUrl and ResolveFileDataUrl. Removed from @framefields/compositions: getFontUrl.

  • 6adfd26: Requires mediabunny and @mediabunny/server 1.61.6 or newer. Encoded video now carries correct colour: older versions converted RGB to YUV with BT.601 and left the output untagged, so reds came out dull and blues shifted. 1.61.5, the first release with the colour fix, also slowed RGB encoding about 5×; 1.61.6 fixes that, and the new minimum keeps installs off 1.61.5.

  • edfe483: resetRenderCaches() also forgets registered font files and deletes fonts downloaded from URLs (clearHeadlessFonts()). Before, the first file registered for a family stayed in use for the life of the process. In a long-lived worker, the next project's "Inter" then loaded the previous project's file, and failed once that checkout was evicted. Downloaded fonts now go to a fresh temp folder per session instead of a shared gatewai-fonts.

  • 5c55a7b: Vision layers no longer copy frames to the CPU. ONNX Runtime's WebGPU sessions now run on the renderer's own device (setVisionDevice), and a compute shader turns the layer's texture straight into each model's input tensor. It does the same resize, letterbox, channel order and normalisation as imageToTensor, within float32 rounding. A small GPU checksum replaces hashing the pixels on the CPU, so layers showing the same frame still share a pass. Frames come back to the CPU only for the opt-in background key (keyBackground).

    VisionRunner methods accept a VisionGpuImage ({ texture, device, width, height }) as well as RGBA pixels. When its sessions run on another device, or on the CPU, the runner reads the texture back itself. WebGPUProvider takes a device option and feeds gpuBuffer inputs with Tensor.fromGpuBuffer.

  • 25edbba: A mask / matte / crop cutout with classes keeps the whole subject again. The class filter ran before the subject was assembled, so the parts COCO labels as other classes were thrown away before they could merge: a flowing dress comes back as a "kite", "umbrella" or "horse", and a classes: ["person"] cutout of a dancer kept only head, arms and torso (about 3% of the frame instead of 12%). The text behind the dancer then showed through the dress on most frames, on every platform. classes now only decides which instance is the subject; overlapping parts of any class merge into it, and the masks a vision bundle receives are still filtered by classes.

    mergeSubjectMask(masks, options) takes { classes, overlap, maxGrowth } instead of positional overlap and maxGrowth, and selectSubjectMask(masks, classes) takes an optional class list.

  • 931435f: Instance segmentation does far less work per frame. The RTMDet-Ins export ran NMS on the CPU in the middle of its graph and returned a full 640×640 mask for every detection: about 40 per frame, 65 MB of floats, computed on the GPU and copied back. The runner now cuts the graph at load time before its first data-dependent step and does the rest itself: top-k, per-class NMS with the export's thresholds, and each mask's three dynamic 1×1 layers at 80×80, only for the detections a layer keeps. detect() computes no masks at all.

    Results match the end2end graph: the same detections, scores and boxes, and masks with IoU above 0.98 on the reference frames (edges differ by one interpolation step). The model file and its SHA-256 check are unchanged; a model without the expected head tensors still runs end to end.

    The smoke project's vision export on an Apple Silicon Mac: 22.3 s → 15 s.

2.2.0-beta.4

Patch Changes

  • 9b70fda: Asset and font paths no longer depend on the BASE_URL and R2_CUSTOM_DOMAIN environment variables, left over from the Gatewai web app. With BASE_URL set, a font registered from an absolute path was fetched from $BASE_URL/<path>, and when that failed the engine quietly used a font from the R2_CUSTOM_DOMAIN CDN instead of the project's file. Absolute paths are now always read from disk, and Lottie sources and fonts are used as given instead of being prefixed with http://localhost:8081.

    Removed from @framefields/client-utils: getEnv, BASE_URL, R2_CUSTOM_DOMAIN, configureAssetUrls and the options argument of GetAssetEndpoint, GetFontAssetUrl, resolveMediaSourceUrl and ResolveFileDataUrl. Removed from @framefields/compositions: getFontUrl.

  • 6adfd26: Requires mediabunny and @mediabunny/server 1.61.6 or newer. Encoded video now carries correct colour: older versions converted RGB to YUV with BT.601 and left the output untagged, so reds came out dull and blues shifted. 1.61.5, the first release with the colour fix, also slowed RGB encoding about 5×; 1.61.6 fixes that, and the new minimum keeps installs off 1.61.5.

  • edfe483: resetRenderCaches() also forgets registered font files and deletes fonts downloaded from URLs (clearHeadlessFonts()). Before, the first file registered for a family stayed in use for the life of the process. In a long-lived worker, the next project's "Inter" then loaded the previous project's file, and failed once that checkout was evicted. Downloaded fonts now go to a fresh temp folder per session instead of a shared gatewai-fonts.

2.2.0-beta.3

Patch Changes

  • 25edbba: A mask / matte / crop cutout with classes keeps the whole subject again. The class filter ran before the subject was assembled, so the parts COCO labels as other classes were thrown away before they could merge: a flowing dress comes back as a "kite", "umbrella" or "horse", and a classes: ["person"] cutout of a dancer kept only head, arms and torso (about 3% of the frame instead of 12%). The text behind the dancer then showed through the dress on most frames, on every platform. classes now only decides which instance is the subject; overlapping parts of any class merge into it, and the masks a vision bundle receives are still filtered by classes.

    mergeSubjectMask(masks, options) takes { classes, overlap, maxGrowth } instead of positional overlap and maxGrowth, and selectSubjectMask(masks, classes) takes an optional class list.

2.2.0-beta.2

Patch Changes

  • 5c55a7b: Vision layers no longer copy frames to the CPU. ONNX Runtime's WebGPU sessions now run on the renderer's own device (setVisionDevice), and a compute shader turns the layer's texture straight into each model's input tensor. It does the same resize, letterbox, channel order and normalisation as imageToTensor, within float32 rounding. A small GPU checksum replaces hashing the pixels on the CPU, so layers showing the same frame still share a pass. Frames come back to the CPU only for the opt-in background key (keyBackground).

    VisionRunner methods accept a VisionGpuImage ({ texture, device, width, height }) as well as RGBA pixels. When its sessions run on another device, or on the CPU, the runner reads the texture back itself. WebGPUProvider takes a device option and feeds gpuBuffer inputs with Tensor.fromGpuBuffer.

  • 931435f: Instance segmentation does far less work per frame. The RTMDet-Ins export ran NMS on the CPU in the middle of its graph and returned a full 640×640 mask for every detection: about 40 per frame, 65 MB of floats, computed on the GPU and copied back. The runner now cuts the graph at load time before its first data-dependent step and does the rest itself: top-k, per-class NMS with the export's thresholds, and each mask's three dynamic 1×1 layers at 80×80, only for the detections a layer keeps. detect() computes no masks at all.

    Results match the end2end graph: the same detections, scores and boxes, and masks with IoU above 0.98 on the reference frames (edges differ by one interpolation step). The model file and its SHA-256 check are unchanged; a model without the expected head tensors still runs end to end.

    The smoke project's vision export on an Apple Silicon Mac: 22.3 s → 15 s.

2.2.0-beta.1

Minor Changes

  • 60c35a0: Faster on-device vision, on the GPU by default:

    • Vision layers no longer reload their models every frame. Layers whose options differed (e.g. one with classes: ["person"] and one without) used to close and recreate their ONNX sessions on each frame; they now share sessions through VisionRunner.derive(). This also fixes the thread constructor failed: Resource temporarily unavailable crash with onnxruntime-web installed.
    • Layers that hand vision the same pixels (two panels of one video) share one RTMDet-Ins pass.
    • onnxruntime-web is now an optional dependency of framefields, so inference runs on WebGPU when there is a hardware adapter. Dawn's GPUSupportedFeatures gains the forEach that onnxruntime-web's WebGPU build needs. Software adapters (lavapipe, SwiftShader) use the CPU instead.
    • CPU sessions on Linux size their thread pool from the CPUs the process may use, cgroup quota included, instead of the host's cores. FRAMEFIELDS_VISION_THREADS overrides it.
    • FRAMEFIELDS_VISION_PROVIDER (auto, webgpu, cpu, cuda) forces a backend; cuda uses onnxruntime-node's CUDA EP on Linux x64.

    The smoke project's vision export (three vision layers, 60 frames of 1280×720) on an Apple Silicon Mac: 84.9 s before; 41–43 s on the CPU and 26–29 s on the GPU after. In the renderer worker on a RunPod RTX 3070: 58.7 s before, 22–25 s after.

2.2.0-beta.0

Minor Changes

  • cf65727: Headless rendering for long-lived workers and any Linux machine:

    • Video encoding uses a hardware encoder (NVENC, VideoToolbox) only after testing that it works, and software encoders (libx264, libx265, libvpx, SVT-AV1) otherwise. This fixes H.264/HEVC renders on linux-arm64, where h264_rkmpp was picked without Rockchip hardware, and on NVIDIA GPUs without NVENC (A100, H100). Set FRAMEFIELDS_HW_CODECS=off to always encode in software.
    • WebGPU falls back to a software adapter (e.g. lavapipe) when no GPU adapter is available. The new describeGpu() reports the adapter in use and whether it is a CPU one.
    • renderVideo accepts signal (AbortSignal) and onProgress; resetRenderCaches() clears per-render caches between jobs in one process.

2.1.3

Patch Changes

  • Measure variable fonts with optical size (opsz) clamped to fontSize across FontManager.measure, measureText, captions, and rich text so layout matches rendered glyph geometry.
  • 71d48b4: Fail fast when referenced local media files cannot be opened or decoded during headless rendering, instead of silently resolving with black frames or silent audio.

2.1.2

Patch Changes

2.1.1

Patch Changes

  • rm aws bundle

2.1.0

Minor Changes

  • 24a4176: Text layers now render Arabic, Hebrew, Persian and other complex scripts. Each run is shaped with the font's OpenType tables (joining forms, lam-alef and other ligatures, kerning, mark placement), paragraphs go through the Unicode Bidirectional Algorithm, and lines wrap and read right to left with embedded Latin words and numbers kept left to right. Glyphs outside the default Latin set are generated on first use instead of rendering as boxes.

    New direction: "auto" | "ltr" | "rtl" option on text and caption layers (default "auto", taken from the first strong character); align: "start" | "end" now follow it instead of always meaning left and right. FontManager.measure accepts direction and reports right-to-left glyphs and carets. The Slug binary format moves to v2 (SLUGGIS2, with glyph ids); v1 files still load.

2.0.11

Patch Changes

  • Physical 3D look, gradients, mattes, transitions and lint for agents:

    • Layer.environment(preset) / Lighting.studio|soft|sunset|dramatic(): GGX PBR with a procedural studio environment, emissive, exposure and tone mapping (PBR Neutral, AgX, ACES), soft shadow maps (castShadow), HDR bloom and fog. glTF normal, metallic-roughness, occlusion and emissive maps are now used.
    • Color helpers (oklch, mix, ramp, lighten, contrast, readableOn, …); hsl(), oklch() and space-separated rgb() accepted everywhere; color keyframes blend in OKLab.
    • CSS linear- / radial- / conic-gradient() with up to 8 stops on any background or shape fill.
    • Track mattes (withMatte(layer, "alpha" | "alpha-inverted" | "luma" | "luma-inverted")), on 2D and 3D layers; 3D layers keep their depth.
    • Layer.sequence([...]) with Transition.crossfade|push|slide|zoom|whip|wipe|iris|fadeThrough.
    • .animate() merges with existing tracks; FontManager.measure() text metrics.
    • comp.lint() (also in renderVideo({ qa })): unreadable colors/gradients, low-contrast text, off-canvas layers, shadow and environment misuse, static tracks.
    • Fixes: Layer.shape({ fill }) was ignored; Layer3D.plane dropped material options; stale 3D camera buffer reuse between frames.

2.0.10

Patch Changes

  • Internal cleanup and maintenance.

2.0.9

Patch Changes

  • Sample effect signals at composition frame across effect nodes, add Rotate effect node and SDK bindings, fix audio layer timing and LUT local file loading, and add WebGPU media size utilities.

2.0.8

Patch Changes

  • Fix text padding asymmetry and background badge rendering by eliminating artificial width offsets and fixing word-wrap token width calculation.

2.0.7

Patch Changes

  • Remove node-svg and node-lottie.

2.0.6

Patch Changes

  • 7ddb9d9: Register all remaining effect and audio DSP nodes with the effects generator and catalog, bundle them in the framefields SDK, and expose their strongly-typed classes, factories, and props for agents and composers.

2.0.5

Patch Changes

  • Fix homepage URL and bump packages.

2.0.4

Patch Changes

  • cf04e38: Preview: drawing no longer stops for good after a frame is shown from the frame cache (stepping or scrubbing back to a frame drawn before).
  • 7ce675d: Preview: the page shows the frame rate playback reaches next to the timecode, and window.framefieldsPreview.state() reports it as playbackFps, with renderMs for the last frame drawn.
  • d70a58b: Preview: Play runs in real time. It draws the frame due on the soundtrack's clock and skips the ones it missed, so playback starts at once and the sound never waits, however slow the device; it no longer renders 3 s ahead first and stalls whenever rendering falls behind.

2.0.3

Patch Changes

  • 4bf477c: Preview: the page's errors and the server's lifecycle are printed where the preview runs, session.closed resolves with why it stopped, the playhead survives reloads and #t= links, and window.framefieldsPreview lets scripts seek, play and read state. The audio mixer no longer prints its progress at info level.

    Preview notes: pause and click the picture (or drag an area) to pin a note to that spot and moment. Notes are saved in .framefields/preview-notes/ (notes.json plus the frame with the spot marked), printed in the preview's output, listed in a Notes panel and on the timeline, and can be ticked off or copied for a chat. notesDir moves them, false turns them off.

    Preview playback: pausing keeps the frames rendered ahead (and keeps filling them), so playing on starts at once; moving the playhead clears them. Fixes a seek that could wait forever when it landed while a buffered frame was drawing.

  • ea9edc3: Preview: drawn frames are cached, so stepping or scrubbing back to a frame you have already seen is a blit instead of a re-render. The cache is full-resolution and bounded by a byte budget, and is disabled for compositions that use vision (whose frames depend on the one before them).

  • 52c2e8b: The live preview now works from the published package. The browser engine (framefields, the player and the Node shims) is prebuilt into dist/preview-engine (about 3 MB) and served to the page as separate modules. At preview time only the project's own files are bundled, against that engine. onnxruntime-web, needed only for vision effects, comes from the project's install when present and otherwise from a pinned CDN build, so its WebAssembly isn't shipped. Fonts referenced by family only (assets/fonts/<Family>.ttf) are served from the project folder, as the export engine reads them.

  • 51b0daf: Rendering performance for the headless / server-GPU path:

    • VRAM telemetry no longer blocks the event loop. The renderer's 5-second stats probe ran nvidia-smi with execSync; it is now sampled asynchronously and cached, so it can no longer stall the frame loop and DMA callbacks on NVIDIA hosts.
    • renderVideo accepts ringCapacity (also FRAMEFIELDS_RING_CAPACITY, clamped 2–8) to tune the DMA staging ring depth. The default stays at 2.
    • renderImage reuses render surfaces across stills instead of allocating and destroying the native render target and readback staging buffer on every call.
    • Headless video decode reuses a single RGBA staging buffer per decoder instead of allocating a full-frame buffer for every decoded frame.

2.0.2

Patch Changes

  • d7bda9e: The preview now serves assets from the project that holds the entry (its nearest .git, else package.json), not just the entry's own directory. A project with no .git, where fonts and media live in a sibling assets/ folder, previously failed to load them. root also accepts several paths now, for assets kept outside the project.

2.0.1

Patch Changes

  • Fix the browser preview failing to load from an installed package. The engine bundles the Node-only preview server, whose top-level createRequire(import.meta.url) ran against the stubbed node:module, so loading framefields in the preview threw (0, Dr.createRequire) is not a function before the player could start. node:module now has a shim, and a test imports every engine entry so a missing shim is caught.

    The npm page now shows the repository README: the package's README is generated from the root one at build and pack time, with its relative asset and repository links rewritten to absolute GitHub URLs so images and links resolve on npmjs.com.

2.0.0

Minor Changes

  • 5c257c2: Layer.chart is rebuilt on d3 and no longer depends on chartgpu, which removes about 130 MB from every install. Charts are now ordinary nodes: bars are boxes, lines, areas and slices are path shapes, and labels are text in the composition's own fonts. They support line, area, bar (grouped or stacked), scatter, candlestick, pie and donut, with a built-in reveal (animate: { start, duration, stagger, ease }).

    Breaking for chart users: Layer.chart takes the new ChartOptions (type, series/data, categories, xAxis/yAxis, legend, valueLabels, …) instead of ChartGPU options, and returns a box. The "chart" node kind, the ChartGPUOptions/Chart*Schema exports, the progress/drawProgress/chartProgress animatable props and LayerAnimation.drawProgress()/chartProgress() are removed.

  • 5c257c2: Add a live preview before rendering. startPreview({ entry, export }) (also from framefields/preview) serves a localhost player for a composition, with the mixed soundtrack and a timeline with the audio waveform. The page loads the module that builds the composition, bundled by esbuild against the browser engine that ships with the package (dist/preview-engine), and renders every frame with WebGPU in the browser, so nothing is streamed from the server. The server serves that bundle, the project's files (by absolute path, with byte ranges for video), and the soundtrack mixed by the export engine. Extruded 3D text (Layer3D.extrudedText) renders in the browser too. Glyph outlines used to go through skia's path simplify, which is Node-only; without skia they are read from fontkit, and overlapping contours (common in variable fonts) are sorted into outlines and holes by winding. Node output is unchanged. Build-time Node code in the composition (node:path, import.meta.url, synchronous fs reads) runs in the browser through shims. Playback never drops frames. Frames are rendered ahead, in order, into a buffer on the GPU and shown at the film's frame rate, on the soundtrack's clock. Playback starts once 3 seconds (or the rest of the film) are ready, and holds with a spinner to refill whenever rendering falls behind. The timeline shows the buffered range. startPreview returns a session (url, close(), closed) and serves the page instead of opening a browser, so an agent can show the URL in its own pane; open: true opens the system browser. The server shuts itself down a few seconds after the last tab closes (idleCloseMs). A reload doesn't trigger this. Each project gets a fixed port, so running the preview again replaces the running one and an open tab reloads into the new version by itself. No Electron. esbuild is a new dependency.

    Rendering consecutive frames is about 3× faster. The compositor no longer re-hashes every keyframe on each frame. renderFrame also accepts a stable renderId, which lets compiled timelines be reused between frames as renderVideo already does, and format: "rgba", which returns raw pixels and skips PNG encoding. The new renderAudio() returns the composition's mixed soundtrack as PCM. The compositor also reuses the previous frame's layout when no layout input changed (the tree, the viewport, media sizes, and animated width, height, gap, padding, fontSize and letterSpacing). Before, it re-ran Yoga over the whole film's tree on every frame.

  • 5c257c2: Vision effects are faster, in export and in the preview. The matte modes (matte, mask, crop) composite on the GPU: only the 8-bit mask is uploaded, instead of building the cut-out at full resolution on the CPU. That CPU path also kept a new full-frame texture for every frame rendered; the new path keeps two per node. Vision results are cached per source frame, keyed by node, source and inference settings, with run-length-encoded masks, so replays, scrubbing back and later renders in the same process skip inference. A frame found in the cache is cut out from its own pixels, without the usual one-frame delay. The preview analyses frames ahead of the playhead while paused, so tracked shots play at full frame rate. Background keying (keyBackground) runs about 10× faster with the same output: flood fills use typed-array queues and a foreground map computed once, and the edge blur reads memory in order. The preview page is served cross-origin isolated, so onnxruntime-web runs its CPU operations on several threads.

    Vision effects also run in the browser preview, on onnxruntime-web with WebGPU. @framefields/vision adds setDefaultSessionProvider(), which the player uses to switch to that runtime. Cached models are read asynchronously. The texture cache now holds evicted textures until the frame being recorded has been submitted, which fixes "Destroyed texture used in a submit" when a frame's recording spans a mid-frame GPU readback. Its pruning also evicts the least recently used textures first, as documented.

Patch Changes

  • 985e6be: LayerAnimation.signal() and .colorSignal() now accept a signal object (Signal.fromArray, Signal.builder, …) as well as a name registered with comp.addSignal. Before, passing the object produced a meaningless handle id and the track silently did nothing (or picked up an unrelated signal). The object is carried on its track and registered with the program under a generated inline_signal_N id, so specs stay plain JSON. Values that are neither a signal nor a name now throw.
  • 5c257c2: Require sharp ^0.35.0, so fresh installs no longer pull the libvips/libheif versions flagged by npm audit. Fix the README samples that did not compile against the published types (quickstart animation, 3D camera orbit and carousel, ColorBalance). The setup skill's tsconfig.json now sets "types": ["node"] so typecheck passes on TypeScript 6+, and its package.json also denies the esbuild install script that tsx brings in, so installs finish without warnings.

1.4.3

Patch Changes

  • Quieter renders: headless GPU, font, and node-discovery messages now log only with LOG_LEVEL=debug, and a local font no longer logs as if it were downloaded. The setup skill now writes package.json with the skia-canvas install-script approval (npm allowScripts, pnpm onlyBuiltDependencies, bun trustedDependencies) before installing, so installs work on package managers that block dependency scripts.

1.4.2

Patch Changes

  • Render logs now report the released version: the monorepo root version is synced on every release, so the logger no longer shows a stale 1.3.0.

1.4.1

Patch Changes

  • Fix issues from the 1.4.0 friction report:

    • Published types no longer import unpublished @framefields/* packages, so signal APIs, frameSignal/timeSignal/progressSignal and layer option types are fully typed instead of any. A build check now guards against regressions.
    • Layer.cube, carousel3d, prism3d and extrudedText no longer leak faces/items into the spec, so CompositorProgramSchema accepts them.
    • scale (and scaleX/scaleY/scaleZ) now applies to preserve-3d containers such as Layer.cube.
    • renderVideo({ outputPath }) returns outputPath as filePath and removes the temp file.

1.4.0

Minor Changes

  • Bundle every built-in node renderer (Compositor, effects, audio) into the published package. Renderers are now registered from a generated static list instead of a scan for a nodes/ directory, so installed projects no longer render blank frames. Set FRAMEFIELDS_NODES_DIR to load extra nodes from disk during local node development.

    Remove React: the framefields/react entry point (CanvasComposition, FramefieldsPlayer, RenderProvider, useRenderContext, useCompositionState) and the react peer dependency are gone. import "framefields" no longer requires react to be installed.

    Relicense all @framefields/* packages and nodes to Apache-2.0.

1.3.2

Patch Changes

  • Agent plugin: add the framefields entry-point skill for setting up a new project, and fix the frame-grid example in framefields-render (showLabels).

1.3.1

Patch Changes

  • init