Skip to content

Code-first video & image, rendered natively on WebGPU.

chore(release): 2.11.0

okanaslfe3a36a08bc3
diff --git a/.changeset/focus-zoom-drift.md b/.changeset/focus-zoom-drift.md
deleted file mode 100644
index 77b12846..00000000
--- a/.changeset/focus-zoom-drift.md
+++ /dev/null
@@ -1,5 +0,0 @@
----
-"framefields": minor
----
-
-FocusZoom takes `drift`: a slow push while each shot holds (zoom gained per second, e.g. `0.012`), so the picture never stops between camera moves. Defaults to 0, which holds still as before.
diff --git a/CLAUDE.md b/CLAUDE.md
index 8082f79f..3b371d31 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -13,7 +13,7 @@ Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turb
- Keep this file current when structure or dev patterns change but do not bloat it.
## Repo map
-- `packages/framefields`: public SDK. `Composition`, `Layer.*`, `LayerAnimation`, `Signal`, effects (`src/effects/classes.ts`), `FontManager`, `renderFrameGrid`. `src/index.ts` is the export surface. CLI in `src/cli/` (`init`, `add`, `list`, `fonts add|list`); `init` never prompts or overwrites, and its file templates are strings in `templates.ts` (a test parses them).
+- `packages/framefields`: public SDK. `Composition`, `Layer.*`, `LayerAnimation`, `Signal`, effects (`src/effects/classes.ts`), `FontManager`, `renderFrameGrid`. `src/index.ts` is the export surface. CLI in `src/cli/` (`init`, `add`, `list`, `fonts add|list`, `check`, `render --cloud`); `framefields/cloud` (`renderCloud`) is the GPU-free client for api.framefields.dev; `init` never prompts or overwrites, and its file templates are strings in `templates.ts` (a test parses them).
- `packages/components`: the component registry `framefields add` copies into user projects (shadcn-style). `defineComponent`/`bind` contract in `src/core/contract.ts`, shared helpers in `src/core/` (keys, color, layout, surface, theme). Components import only `framefields` and `../core`. Registry items declare font needs (e.g. emoji); CLI fonts are pinned by URL, size and SHA-256 in `cli/fonts.ts`. Prop docs in `components.json` are generated (drift test). Tree-shake size budgets are tested.
- `packages/compositions`: Yoga layout (`utils/layout/resolve-layout.ts`), program schema (`program/schema.ts`), lint (`program/lint.ts`), animation/evaluator.
- `packages/webgpu-renderers`: WGSL shaders, 2D/3D pipelines, Slug text (`src/slug/`), audio DSP.
@@ -22,7 +22,7 @@ Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turb
- `packages/core`: shared types, `Effect` base, signals, `emojiPresentation` (the one UTS #51 rule used by layout, preload and lint).
- Others: `renderers`, `media`, `node-sdk`, `server-utils`, `client-utils`, `tensor-webgpu`, `vision`, `tsconfig`.
- `nodes/node-*`: one VFX/audio/layout node each (`src/shared/`, `src/renderers/`, `src/index.ts`). Only `node-compositor` has a `SKILL.md`; update it when Compositor behavior changes (guarded by `skillmd-check.test.ts`).
-- `apps/renderer-service`: HTTP render service (Docker, RunPod).
+- `apps/renderer-service`: HTTP render service (Docker, RunPod). It renders single jobs, including the pieces of a split render (`probe`, `range` chunks, `stitch`); the API repo (`/Users/okanaslankan/framefields-dev/api`, `src/render/`) plans, dispatches and bills them.
- `examples/NN_*`: reference films. Study `19_framefields_film` (structure), `22_framefields_launch` (lyric sync), `28_physics_dancer` (physics, vision, GPU particles) before writing one.
- `plugins/framefields/skills/framefields{,-compose,-effects,-render,-vision}`: agent skills, symlinked from `.agents/skills` and `.claude/skills`.
- `framefields` is the only published package: it bundles every `@framefields/*` workspace package, and its build fails if `dist` still imports one (`scripts/check-bundled-imports.mjs`).
diff --git a/apps/renderer-service/CHANGELOG.md b/apps/renderer-service/CHANGELOG.md
index 6c53cc65..5262b4e1 100644
--- a/apps/renderer-service/CHANGELOG.md
+++ b/apps/renderer-service/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/renderer-service
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/apps/renderer-service/PLAN.md b/apps/renderer-service/PLAN.md
index c6185e9c..55b7c4ff 100644
--- a/apps/renderer-service/PLAN.md
+++ b/apps/renderer-service/PLAN.md
@@ -23,7 +23,7 @@ A job tells the worker which **git repo + ref** holds a framefields project. The
### Non-goals (for this iteration)
-- Splitting one render across several workers by frame range. Section 13 covers this as future work.
+- Deciding how to split one render across several workers. The worker renders the pieces (`probe`, a `video` job with `range`, `audio`, `stitch`; see the README's "Split renders"), and the caller coordinates them: api.framefields.dev does it in its render jobs.
- A queue or scheduler of our own. RunPod or the caller does the queueing.
- Repo encryption filters such as `git-crypt`. These are detected and rejected with a clear error.
- Running untrusted third-party repos on a shared multi-tenant worker. The trust model is in section 9.
@@ -512,6 +512,7 @@ Every failure maps to a stable `code`. `retryable` is what RunPod or the caller
| install | `LOCKFILE_MISSING` / `INSTALL_FAILED` / `INSTALL_TIMEOUT` | no / no / yes | The tail of the install log is attached. |
| install | `ENGINE_NOT_FOUND` | no | The project doesn't depend on, or didn't install, framefields. |
| install / render | `ENGINE_SWITCH` | yes (fresh worker) | The project needs a different framefields version than the worker bound, or its dependencies changed after the worker loaded them. The worker recycles. |
+| render | `ENGINE_UNSUPPORTED` | no | A `probe`, `stitch` or `range` job on a framefields version without split renders. |
| load | `ENTRY_NOT_FOUND` / `ENTRY_COMPILE_FAILED` / `EXPORT_NOT_FOUND` / `EXPORT_INVALID` / `EXPORT_SIDE_EFFECT` / `COMPOSITION_BUILD_FAILED` | no | The user's stack trace is attached and trimmed to the project's files. |
| render | `ASSET_NOT_FOUND` | no | A referenced file is missing. Often an LFS include that is too narrow, so the hint says so. |
| render | `RENDER_FAILED` / `RENDER_STALLED` / `RENDER_TIMEOUT` | no / yes / no | |
@@ -640,7 +641,7 @@ Each phase is its own PR-sized step with tests.
- Pinned SSH host keys, the RunPod templates and `create-template.sh`, `docker-compose.yml` with MinIO, `env.example`, the release workflow, and the README guide as listed above.
- **Still to do:** an amd64 build (CI), the nightly host smoke tests (6.5), and a page on the docs website.
9. **(Later)**
- - Chunked distributed rendering: split a frame range across N jobs and concat with a stream copy.
+ - ✅ Chunked distributed rendering: the worker side is done (`probe`, `range` + `audio: false`, `stitch`). Splitting, retries and backups of slow chunks live in the caller (api.framefields.dev `/v1/render`).
- Webhooks on completion (`output.webhook` with an HMAC signature).
## 14. Test plan
diff --git a/apps/renderer-service/README.md b/apps/renderer-service/README.md
index 26aefd5f..472d5f94 100644
--- a/apps/renderer-service/README.md
+++ b/apps/renderer-service/README.md
@@ -276,7 +276,18 @@ Versions of `framefields` released before worker support (2.1.3 and older) still
- `audio` (`aac`, `mp3`, `opus`);
- `frames` (`frames: [0, 120]` → PNGs);
- `frameGrid` (a contact sheet);
-- `lut`.
+- `lut`;
+- `probe` and `stitch`, for split renders (below).
+
+### Split renders
+
+The worker renders one piece of a split render per job. A coordinator decides the split and sends the jobs; for api.framefields.dev that is the API's render jobs (`POST /v1/render` with `workers: "auto"`).
+
+1. **Probe:** `{ "kind": "probe", "probe": { "workers": "auto", "maxWorkers": 10, "overheadSec": 90 } }` times frames across the timeline and plans the split with the project's own `planChunks`. Send it with an `inline` output: the result is one JSON file, `{ totalFrames, fps, samples, plan: { workers, chunks: [{ start, end, workSec }], workSec, etaSec } }`, and `source.sha` is the commit to pin the other jobs to.
+2. **Chunks and audio:** one `video` job per chunk with `"range": { "from": start, "to": end }, "audio": false`, plus one `audio` job (`aac`, or `opus` for VP9). Each writes to the output bucket.
+3. **Stitch:** `{ "kind": "stitch", "stitch": { "chunks": ["…/chunk-000.mp4", …], "audio": "…/audio.m4a" } }` downloads the files from the output bucket (`stitch.bucket` defaults to `STORAGE_BUCKET`; either must be in `FF_OUTPUT_BUCKETS`), joins them without re-encoding, and uploads the result to the job's output.
+
+These kinds need a framefields version with split renders; older ones fail with `ENGINE_UNSUPPORTED` rather than render the whole film.
### Output targets
@@ -350,6 +361,7 @@ A failed job returns:
| `RENDER_TIMEOUT` / `RENDER_STALLED` | The render took too long, or stopped making progress. |
| `ENGINE_NOT_FOUND` | The project doesn't depend on `framefields`, so there is nothing to render with. |
| `ENGINE_SWITCH` | Retry. The project needs another framefields version, or new dependencies; the worker restarts to load them. |
+| `ENGINE_UNSUPPORTED` | A `probe`, `stitch` or `range` job on a framefields version without split renders. Update framefields in the project. |
| `GPU_DEVICE_LOST` / `RENDER_OOM` | Retry. The worker replaces itself. |
| `GPU_UNAVAILABLE` | The machine has no usable GPU (`FF_GPU=require`), or has one that WebGPU couldn't use. Retry on another worker; on RunPod this happens automatically. Check the GPU's Vulkan driver with `vulkaninfo --summary`. |
| `QA_FAILED` | `render.qa.failOn` matched. The outputs are still uploaded and listed. |
diff --git a/apps/renderer-service/package.json b/apps/renderer-service/package.json
index 11b7799b..387c2f8b 100644
--- a/apps/renderer-service/package.json
+++ b/apps/renderer-service/package.json
@@ -1,7 +1,7 @@
{
"name": "@framefields/renderer-service",
"private": true,
- "version": "2.9.3",
+ "version": "2.11.0",
"type": "module",
"types": "./dist/src/index.d.ts",
"files": [
diff --git a/apps/renderer-service/src/engine.gpu.test.ts b/apps/renderer-service/src/engine.gpu.test.ts
index f7c08197..2881131e 100644
--- a/apps/renderer-service/src/engine.gpu.test.ts
+++ b/apps/renderer-service/src/engine.gpu.test.ts
@@ -173,6 +173,21 @@ gpu("in-process render step (real Node loader)", () => {
expect(r.cancel).toMatchObject({ ok: false, code: "CANCELLED" });
expect(r.after.ok).toBe(true);
expect(r.switch).toMatchObject({ ok: false, code: "ENGINE_SWITCH" });
+ expect(r.probe).toMatchObject({
+ ok: true,
+ files: [{ name: "main", ext: "json" }],
+ });
+ expect(r.probePlan).toMatchObject({
+ totalFrames: 24,
+ fps: 24,
+ plan: { workers: 2 },
+ });
+ expect(r.probePlan.samples).toHaveLength(4);
+ expect(r.probePlan.plan.chunks.at(-1).end).toBe(24);
+ expect(r.chunk).toMatchObject({
+ ok: true,
+ files: [{ name: "main", ext: "mp4" }],
+ });
for (const k of [
"video1",
"video2",
diff --git a/apps/renderer-service/src/errors.ts b/apps/renderer-service/src/errors.ts
index e008736f..41db8b6b 100644
--- a/apps/renderer-service/src/errors.ts
+++ b/apps/renderer-service/src/errors.ts
@@ -56,6 +56,7 @@ const CODES = {
INSTALL_TIMEOUT: { phase: "install", retryable: true },
ENGINE_NOT_FOUND: { phase: "install", retryable: false },
ENGINE_SWITCH: { phase: "render", retryable: true },
+ ENGINE_UNSUPPORTED: { phase: "render", retryable: false },
ENTRY_NOT_FOUND: { phase: "load", retryable: false },
ENTRY_COMPILE_FAILED: { phase: "load", retryable: false },
diff --git a/apps/renderer-service/src/jobs/schema.ts b/apps/renderer-service/src/jobs/schema.ts
index 7bcae544..9fe1c908 100644
--- a/apps/renderer-service/src/jobs/schema.ts
+++ b/apps/renderer-service/src/jobs/schema.ts
@@ -63,18 +63,47 @@ export const RENDER_KINDS = [
"frameGrid",
"lut",
"audio",
+ "probe",
+ "stitch",
] as const;
+/** An object key the worker reads from storage: relative, no "..". */
+const storageKey = z.string().min(1).max(1024).refine(isSafeRelativePath, {
+ message: 'keys must be relative and must not contain ".."',
+});
+
export const renderSchema = z.object({
kind: z.enum(RENDER_KINDS).default("video"),
codec: z.string().max(32).optional(),
audioCodec: z.string().max(32).optional(),
quality: z.string().max(32).optional(),
frames: z.array(z.number().int().min(0)).min(1).max(10_000).optional(),
+ /** kind=video: render only frames `from` (inclusive) to `to` (exclusive), as one chunk of a split render. */
range: z
.object({ from: z.number().int().min(0), to: z.number().int().min(1) })
.refine((r) => r.to > r.from, { message: "range.to must be > range.from" })
.optional(),
+ /** kind=video: `false` writes no audio track (chunks; the audio is its own kind=audio job). */
+ audio: z.boolean().optional(),
+ /** kind=probe: measure frame costs and plan a split render (`planChunks`). */
+ probe: z
+ .object({
+ samples: z.number().int().min(1).max(64).optional(),
+ workers: z.union([z.number().int().min(1), z.literal("auto")]).optional(),
+ maxWorkers: z.number().int().min(1).max(256).optional(),
+ overheadSec: z.number().min(0).max(3600).optional(),
+ efficiency: z.number().gt(0).lt(1).optional(),
+ minChunkFrames: z.number().int().min(1).optional(),
+ })
+ .optional(),
+ /** kind=stitch: chunk files (and the audio file) in the output bucket, joined in order. */
+ stitch: z
+ .object({
+ bucket: z.string().min(1).max(255).optional(),
+ chunks: z.array(storageKey).min(1).max(256),
+ audio: storageKey.optional(),
+ })
+ .optional(),
grid: z
.object({
from: z.number().int().min(0).optional(),
@@ -146,6 +175,20 @@ export const jobSchema = z
message: 'required when render.kind is "frames"',
});
}
+ if (job.render.kind === "stitch" && !job.render.stitch) {
+ ctx.addIssue({
+ code: "custom",
+ path: ["render", "stitch"],
+ message: 'required when render.kind is "stitch"',
+ });
+ }
+ if (job.render.range && job.render.kind !== "video") {
+ ctx.addIssue({
+ code: "custom",
+ path: ["render", "range"],
+ message: 'only applies to render.kind "video"',
+ });
+ }
});
export type JobInput = z.infer<typeof jobSchema>;
diff --git a/apps/renderer-service/src/jobs/validate.ts b/apps/renderer-service/src/jobs/validate.ts
index 402111c6..af3d0f5c 100644
--- a/apps/renderer-service/src/jobs/validate.ts
+++ b/apps/renderer-service/src/jobs/validate.ts
@@ -79,6 +79,7 @@ export function validateJob(
}
validateOutput(input, config);
+ validateStitchInputs(input, config);
for (const name of Object.keys(input.env)) {
if (RESERVED_JOB_ENV.some((re) => re.test(name))) {
@@ -137,6 +138,25 @@ function validateOutput(input: JobInput, config: RendererConfig): void {
}
}
+/** A stitch job reads its chunks with the worker's own storage credentials. */
+function validateStitchInputs(input: JobInput, config: RendererConfig): void {
+ const stitch = input.render.stitch;
+ if (input.render.kind !== "stitch" || !stitch) return;
+ if (config.storage.driver !== "s3") {
+ throw new RenderError(
+ "JOB_INVALID",
+ 'render.kind "stitch" needs STORAGE_DRIVER=s3 on the worker.',
+ );
+ }
+ const bucket = stitch.bucket ?? config.storage.bucket;
+ if (!bucket || !config.storage.outputBuckets.includes(bucket)) {
+ throw new RenderError(
+ "BUCKET_NOT_ALLOWED",
+ `Bucket "${bucket}" is not in FF_OUTPUT_BUCKETS.`,
+ );
+ }
+}
+
function globMatch(pattern: string, value: string): boolean {
const re = new RegExp(
`^${pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*")}$`,
diff --git a/apps/renderer-service/src/output/sinks.ts b/apps/renderer-service/src/output/sinks.ts
index 3d4be55a..41ff0bb6 100644
--- a/apps/renderer-service/src/output/sinks.ts
+++ b/apps/renderer-service/src/output/sinks.ts
@@ -1,6 +1,8 @@
-import { createReadStream } from "node:fs";
+import { createReadStream, createWriteStream } from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
+import type { Readable } from "node:stream";
+import { pipeline } from "node:stream/promises";
import {
GetObjectCommand,
HeadObjectCommand,
@@ -87,6 +89,41 @@ export function s3Client(config: RendererConfig): S3Client {
return client;
}
+/** Downloads an object to `file`: a stitch job's chunks, read with the worker's own credentials. */
+export async function downloadS3(
+ config: RendererConfig,
+ bucket: string,
+ key: string,
+ file: string,
+ signal: AbortSignal,
+): Promise<void> {
+ try {
+ await withRetries(signal, async () => {
+ const res = await s3Client(config).send(
+ new GetObjectCommand({ Bucket: bucket, Key: key }),
+ { abortSignal: signal },
+ );
+ if (!res.Body) throw new Error(`s3://${bucket}/${key} has no body`);
+ await pipeline(res.Body as Readable, createWriteStream(file), {
+ signal,
+ });
+ });
+ } catch (err) {
+ const missing =
+ err instanceof S3ServiceException && err.name === "NoSuchKey";
+ if (
+ missing ||
+ (err instanceof RenderError && /NoSuchKey/.test(err.message))
+ )
+ throw new RenderError(
+ "ASSET_NOT_FOUND",
+ `Stitch input s3://${bucket}/${key} does not exist.`,
+ { phase: "render" },
+ );
+ throw err;
+ }
+}
+
export function publicUrl(
config: RendererConfig,
key: string,
diff --git a/apps/renderer-service/src/render/engine.ts b/apps/renderer-service/src/render/engine.ts
index cea764f1..4e166d46 100644
--- a/apps/renderer-service/src/render/engine.ts
+++ b/apps/renderer-service/src/render/engine.ts
@@ -26,10 +26,32 @@ export interface EngineModule {
resetRenderCaches?: () => void;
/** The adapter the engine renders on (newer framefields versions). */
describeGpu?: () => Promise<GpuAdapter>;
+ /** Split renders (framefields with `renderVideo({ frames })`). */
+ planChunks?: (options: Record<string, unknown>) => unknown;
+ stitchChunks?: (options: {
+ chunks: readonly string[];
+ audio?: string;
+ outputPath: string;
+ signal?: AbortSignal;
+ }) => Promise<{ filePath: string; frames: number; durationSec: number }>;
}
export interface EngineRenderer {
renderLut(vm: never, frame?: number): Promise<string | Buffer>;
+ /** Present with `planChunks`: frame costs across the timeline. */
+ probeRender?(
+ vm: never,
+ options: {
+ samples?: number;
+ signal?: AbortSignal;
+ onSample?: (done: number, total: number) => void;
+ },
+ ): Promise<{
+ totalFrames: number;
+ fps: number;
+ durationMs: number;
+ samples: { frame: number; ms: number }[];
+ }>;
}
export interface BoundEngine {
diff --git a/apps/renderer-service/src/render/render-step.ts b/apps/renderer-service/src/render/render-step.ts
index 4769f42a..8fc322fb 100644
--- a/apps/renderer-service/src/render/render-step.ts
+++ b/apps/renderer-service/src/render/render-step.ts
@@ -10,9 +10,16 @@ import type {
import type { JobInput, RenderSettings } from "../jobs/schema.js";
import type { JobProgress } from "../jobs/store.js";
import { rendererLogger } from "../logger.js";
+import { downloadS3 } from "../output/sinks.js";
import { gpuStatus } from "../worker/gpu.js";
import { workerHealth } from "../worker/health.js";
-import { type BoundEngine, bindEngine, markDepsLoaded } from "./engine.js";
+import {
+ type BoundEngine,
+ bindEngine,
+ type EngineModule,
+ type EngineRenderer,
+ markDepsLoaded,
+} from "./engine.js";
import { clearWorkspaceRequireCache } from "./engine-alias.js";
import { buildPlan, importEntry, type PlanOutput } from "./load.js";
import { openSandbox } from "./sandbox.js";
@@ -62,6 +69,11 @@ export const render: Pipeline["render"] = async (checkout, input, ctx) => {
let dispose: (() => Promise<void>) | undefined;
try {
const work = (async () => {
+ // A stitch joins files already rendered: no composition to load.
+ if (input.render.kind === "stitch")
+ return [
+ await stitchOutput(input.render, outDir, ctx, controller, engine),
+ ];
const files: RenderedFile[] = [];
const plan = await withTimeout(
loadPlan(checkout, input, ctx.jobId, (d) => {
@@ -192,8 +204,11 @@ async function renderOutput(
let lastEmit = 0;
let lastPhase: string | undefined;
let expectedFrames: number | undefined;
+ const chunk =
+ settings.kind === "video" ? chunkOptions(settings, engine) : {};
const result = await output.target.renderVideo({
outputPath: `${base}.${ext}`,
+ ...chunk,
codec,
audioCodec: settings.audioCodec,
quality: settings.quality,
@@ -232,6 +247,25 @@ async function renderOutput(
},
];
}
+ case "probe":
+ return [
+ await probeOutput(
+ output,
+ settings,
+ base,
+ ctx,
+ controller,
+ engine,
+ renderer,
+ stall.touch,
+ count > 1 ? index : undefined,
+ ),
+ ];
+ case "stitch":
+ throw new RenderError(
+ "INTERNAL",
+ "Stitch jobs are handled before the plan loads.",
+ );
case "frames": {
const files: RenderedFile[] = [];
const frames = settings.frames ?? [0];
@@ -316,6 +350,176 @@ async function renderOutput(
}
}
+/** A function the project's framefields must export for this job, or ENGINE_UNSUPPORTED. */
+function requireEngine<K extends "planChunks" | "stitchChunks">(
+ engine: BoundEngine,
+ name: K,
+ what: string,
+): NonNullable<EngineModule[K]> {
+ const fn = engine.mod[name];
+ if (typeof fn !== "function") {
+ throw new RenderError(
+ "ENGINE_UNSUPPORTED",
+ `framefields ${engine.version} can't run ${what}: it has no \`${name}\`.`,
+ {
+ phase: "render",
+ hint: "Update framefields in the project to a version with split renders.",
+ },
+ );
+ }
+ return fn as NonNullable<EngineModule[K]>;
+}
+
+/**
+ * `renderVideo` options for one chunk of a split render. An engine without
+ * them would ignore `frames` and render the whole film, so it is refused.
+ */
+function chunkOptions(
+ settings: RenderSettings,
+ engine: BoundEngine,
+): { frames?: { start: number; end: number }; audio?: false } {
+ if (!settings.range && settings.audio !== false) return {};
+ requireEngine(engine, "stitchChunks", "chunked renders");
+ return {
+ ...(settings.range && {
+ frames: { start: settings.range.from, end: settings.range.to },
+ }),
+ ...(settings.audio === false && { audio: false }),
+ };
+}
+
+/**
+ * kind=probe: times frames across the timeline and plans the split render
+ * with the project's own `planChunks`, written as one JSON file.
+ */
+async function probeOutput(
+ output: PlanOutput,
+ settings: RenderSettings,
+ base: string,
+ ctx: StepContext,
+ controller: AbortController,
+ engine: BoundEngine,
+ renderer: Pick<EngineRenderer, "probeRender">,
+ touch: () => void,
+ outputIndex: number | undefined,
+): Promise<RenderedFile> {
+ const planChunks = requireEngine(engine, "planChunks", "probe jobs");
+ const vm = output.target.toVirtualMedia?.() ?? output.target.virtualMedia;
+ if (!renderer.probeRender || !vm) {
+ throw new RenderError(
+ "EXPORT_INVALID",
+ `Output ${output.name} can't be probed.`,
+ { phase: "render" },
+ );
+ }
+ const options = settings.probe ?? {};
+ const measured = await renderer.probeRender(vm as never, {
+ samples: options.samples,
+ signal: controller.signal,
+ onSample: (done, total) => {
+ touch();
+ ctx.progress({
+ phase: "rendering",
+ output: outputIndex,
+ frame: done,
+ total,
+ message: "probing frame costs",
+ });
+ },
+ });
+ const plan = planChunks({
+ totalFrames: measured.totalFrames,
+ samples: measured.samples,
+ workers: options.workers,
+ maxWorkers: options.maxWorkers,
+ overheadSec: options.overheadSec,
+ efficiency: options.efficiency,
+ minChunkFrames: options.minChunkFrames,
+ });
+ const file = `${base}-probe.json`;
+ await fs.writeFile(file, JSON.stringify({ ...measured, plan }));
+ return {
+ name: output.name,
+ path: file,
+ ext: "json",
+ contentType: "application/json",
+ };
+}
+
+/**
+ * kind=stitch: downloads the chunks (and audio) from the output bucket and
+ * joins them with the project's `stitchChunks`, without re-encoding.
+ */
+async function stitchOutput(
+ settings: RenderSettings,
+ outDir: string,
+ ctx: StepContext,
+ controller: AbortController,
+ engine: BoundEngine,
+): Promise<RenderedFile> {
+ const stitch = settings.stitch;
+ const bucket = stitch?.bucket ?? ctx.config.storage.bucket;
+ if (!stitch || !bucket)
+ throw new RenderError(
+ "JOB_INVALID",
+ "render.stitch needs chunks and a bucket.",
+ );
+ const stitchChunks = requireEngine(engine, "stitchChunks", "stitch jobs");
+ const inDir = path.join(ctx.jobDir, "in");
+ await fs.mkdir(inDir, { recursive: true });
+ const keys = [...stitch.chunks, ...(stitch.audio ? [stitch.audio] : [])];
+ let downloaded = 0;
+ const local = await mapLimit(keys, 4, async (key, i) => {
+ const file = path.join(inDir, `${i}-${path.basename(key)}`);
+ await downloadS3(ctx.config, bucket, key, file, controller.signal);
+ ctx.progress({
+ phase: "rendering",
+ frame: ++downloaded,
+ total: keys.length,
+ message: "downloading chunks",
+ });
+ return file;
+ });
+ const ext = path.extname(stitch.chunks[0]).slice(1) || "mp4";
+ const { contentType } = containerFor(ext === "webm" ? "vp9" : "h264");
+ const outputPath = path.join(outDir, `0-main.${ext}`);
+ ctx.progress({
+ phase: "encoding",
+ message: `stitching ${stitch.chunks.length} chunks`,
+ });
+ const result = await stitchChunks({
+ chunks: local.slice(0, stitch.chunks.length),
+ audio: stitch.audio ? local.at(-1) : undefined,
+ outputPath,
+ signal: controller.signal,
+ });
+ return {
+ name: "main",
+ path: outputPath,
+ ext,
+ contentType,
+ expectedFrames: result.frames,
+ };
+}
+
+/** `Promise.all` over `items`, at most `limit` at a time, results in order. */
+async function mapLimit<T, R>(
+ items: readonly T[],
+ limit: number,
+ fn: (item: T, index: number) => Promise<R>,
+): Promise<R[]> {
+ const results = new Array<R>(items.length);
+ let next = 0;
+ const run = async () => {
+ while (next < items.length) {
+ const i = next++;
+ results[i] = await fn(items[i], i);
+ }
+ };
+ await Promise.all(Array.from({ length: Math.min(limit, items.length) }, run));
+ return results;
+}
+
/**
* What `renderVideo({ onProgress })` reports. Engines before the phase
* breakdown send only "render" and "audio", without the optional fields.
diff --git a/apps/renderer-service/test/render-harness.ts b/apps/renderer-service/test/render-harness.ts
index 6b9df1e5..ef41ae18 100644
--- a/apps/renderer-service/test/render-harness.ts
+++ b/apps/renderer-service/test/render-harness.ts
@@ -202,6 +202,21 @@ results.after = await job("after", {
entry: "src/film.ts",
export: "buildFilm",
});
+// Split renders: plan with the project's engine, then render one chunk.
+results.probe = await job(
+ "probe",
+ { entry: "src/film.ts", export: "buildFilm" },
+ { kind: "probe", probe: { samples: 4, workers: 2, minChunkFrames: 6 } },
+);
+const probeFile = (results.probe as { files?: { path: string }[] }).files?.[0];
+results.probePlan = probeFile
+ ? JSON.parse(await fs.readFile(probeFile.path, "utf8"))
+ : undefined;
+results.chunk = await job(
+ "chunk",
+ { entry: "src/film.ts", export: "buildFilm" },
+ { range: { from: 12, to: 24 }, audio: false },
+);
// Last: a repo pinned to another framefields version can't share this process.
results.switch = await job(
"switch",
diff --git a/examples/19_framefields_film/CHANGELOG.md b/examples/19_framefields_film/CHANGELOG.md
index 08c5b066..36ca91f7 100644
--- a/examples/19_framefields_film/CHANGELOG.md
+++ b/examples/19_framefields_film/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-19-framefields-film
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/19_framefields_film/package.json b/examples/19_framefields_film/package.json
index 69e5637e..58a1543c 100644
--- a/examples/19_framefields_film/package.json
+++ b/examples/19_framefields_film/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-19-framefields-film",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/21_full_circle/CHANGELOG.md b/examples/21_full_circle/CHANGELOG.md
index c72cee9f..e4470b49 100644
--- a/examples/21_full_circle/CHANGELOG.md
+++ b/examples/21_full_circle/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-21-full-circle
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/21_full_circle/package.json b/examples/21_full_circle/package.json
index 7347bfa6..6d6ed77c 100644
--- a/examples/21_full_circle/package.json
+++ b/examples/21_full_circle/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-21-full-circle",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/22_framefields_launch/CHANGELOG.md b/examples/22_framefields_launch/CHANGELOG.md
index 994c017d..13c653b7 100644
--- a/examples/22_framefields_launch/CHANGELOG.md
+++ b/examples/22_framefields_launch/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-22-framefields-launch
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/22_framefields_launch/package.json b/examples/22_framefields_launch/package.json
index 12bf7370..3add25e3 100644
--- a/examples/22_framefields_launch/package.json
+++ b/examples/22_framefields_launch/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-22-framefields-launch",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/23_data_story/CHANGELOG.md b/examples/23_data_story/CHANGELOG.md
index ab170bbf..27fd259a 100644
--- a/examples/23_data_story/CHANGELOG.md
+++ b/examples/23_data_story/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-23-data-story
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/23_data_story/package.json b/examples/23_data_story/package.json
index 6a6ef95e..8dbd6c88 100644
--- a/examples/23_data_story/package.json
+++ b/examples/23_data_story/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-23-data-story",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/24_audio_capabilities/CHANGELOG.md b/examples/24_audio_capabilities/CHANGELOG.md
index 257b1681..0ecc009e 100644
--- a/examples/24_audio_capabilities/CHANGELOG.md
+++ b/examples/24_audio_capabilities/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-24-audio-capabilities
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/24_audio_capabilities/package.json b/examples/24_audio_capabilities/package.json
index d52e8625..e0368bed 100644
--- a/examples/24_audio_capabilities/package.json
+++ b/examples/24_audio_capabilities/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-24-audio-capabilities",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/26_mac_keyboard_from_3d_primitives/CHANGELOG.md b/examples/26_mac_keyboard_from_3d_primitives/CHANGELOG.md
index 1ef52454..545a8dcb 100644
--- a/examples/26_mac_keyboard_from_3d_primitives/CHANGELOG.md
+++ b/examples/26_mac_keyboard_from_3d_primitives/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-26-mac-keyboard
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/26_mac_keyboard_from_3d_primitives/package.json b/examples/26_mac_keyboard_from_3d_primitives/package.json
index 6cdfa579..21581b51 100644
--- a/examples/26_mac_keyboard_from_3d_primitives/package.json
+++ b/examples/26_mac_keyboard_from_3d_primitives/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-26-mac-keyboard",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/27_rtl_text/CHANGELOG.md b/examples/27_rtl_text/CHANGELOG.md
index fb679e84..29a44097 100644
--- a/examples/27_rtl_text/CHANGELOG.md
+++ b/examples/27_rtl_text/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-27-rtl-text
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/27_rtl_text/package.json b/examples/27_rtl_text/package.json
index 5fd2d534..21240488 100644
--- a/examples/27_rtl_text/package.json
+++ b/examples/27_rtl_text/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-27-rtl-text",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/28_physics_dancer/CHANGELOG.md b/examples/28_physics_dancer/CHANGELOG.md
index e35ee7f7..b1e06a35 100644
--- a/examples/28_physics_dancer/CHANGELOG.md
+++ b/examples/28_physics_dancer/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-28-physics-dancer
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/28_physics_dancer/package.json b/examples/28_physics_dancer/package.json
index 1e51725b..9c26acbf 100644
--- a/examples/28_physics_dancer/package.json
+++ b/examples/28_physics_dancer/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-28-physics-dancer",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/29_capabilities_reel/CHANGELOG.md b/examples/29_capabilities_reel/CHANGELOG.md
index 096751eb..867b4295 100644
--- a/examples/29_capabilities_reel/CHANGELOG.md
+++ b/examples/29_capabilities_reel/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-29-capabilities-reel
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/29_capabilities_reel/package.json b/examples/29_capabilities_reel/package.json
index b4d01189..5604a2a1 100644
--- a/examples/29_capabilities_reel/package.json
+++ b/examples/29_capabilities_reel/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-29-capabilities-reel",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/30_product_demo/CHANGELOG.md b/examples/30_product_demo/CHANGELOG.md
index 1f993de1..fd77297f 100644
--- a/examples/30_product_demo/CHANGELOG.md
+++ b/examples/30_product_demo/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-30-product-demo
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/30_product_demo/package.json b/examples/30_product_demo/package.json
index f6e42ea4..31c3feff 100644
--- a/examples/30_product_demo/package.json
+++ b/examples/30_product_demo/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-30-product-demo",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/31_shard_storm/CHANGELOG.md b/examples/31_shard_storm/CHANGELOG.md
index 9b9272a2..73c09f3c 100644
--- a/examples/31_shard_storm/CHANGELOG.md
+++ b/examples/31_shard_storm/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/example-31-shard-storm
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/examples/31_shard_storm/package.json b/examples/31_shard_storm/package.json
index 236cd63a..79d6dc0c 100644
--- a/examples/31_shard_storm/package.json
+++ b/examples/31_shard_storm/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-31-shard-storm",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/examples/32_relay/CHANGELOG.md b/examples/32_relay/CHANGELOG.md
new file mode 100644
index 00000000..f3282652
--- /dev/null
+++ b/examples/32_relay/CHANGELOG.md
@@ -0,0 +1,3 @@
+# @framefields/example-32-relay
+
+## 2.11.0
diff --git a/examples/32_relay/package.json b/examples/32_relay/package.json
index 7dcc485e..7df4d13e 100644
--- a/examples/32_relay/package.json
+++ b/examples/32_relay/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/example-32-relay",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"scripts": {
diff --git a/nodes/node-apply-lut/CHANGELOG.md b/nodes/node-apply-lut/CHANGELOG.md
index 849c3290..2b3496b6 100644
--- a/nodes/node-apply-lut/CHANGELOG.md
+++ b/nodes/node-apply-lut/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-lut
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-apply-lut/package.json b/nodes/node-apply-lut/package.json
index 0292d69c..8b432405 100644
--- a/nodes/node-apply-lut/package.json
+++ b/nodes/node-apply-lut/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-lut",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ApplyLUT",
diff --git a/nodes/node-ascii/CHANGELOG.md b/nodes/node-ascii/CHANGELOG.md
index 92112e8b..cf6e343e 100644
--- a/nodes/node-ascii/CHANGELOG.md
+++ b/nodes/node-ascii/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-ascii
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-ascii/package.json b/nodes/node-ascii/package.json
index 9c32782e..2bbec10d 100644
--- a/nodes/node-ascii/package.json
+++ b/nodes/node-ascii/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-ascii",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Ascii",
diff --git a/nodes/node-audio-compressor/CHANGELOG.md b/nodes/node-audio-compressor/CHANGELOG.md
index 1e73bbf6..764bed51 100644
--- a/nodes/node-audio-compressor/CHANGELOG.md
+++ b/nodes/node-audio-compressor/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-compressor
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-compressor/package.json b/nodes/node-audio-compressor/package.json
index 89add34b..bb5a9a57 100644
--- a/nodes/node-audio-compressor/package.json
+++ b/nodes/node-audio-compressor/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-compressor",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-audio-delay/CHANGELOG.md b/nodes/node-audio-delay/CHANGELOG.md
index 7eb21974..8403949e 100644
--- a/nodes/node-audio-delay/CHANGELOG.md
+++ b/nodes/node-audio-delay/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-delay
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-delay/package.json b/nodes/node-audio-delay/package.json
index 8c088212..20bc34c2 100644
--- a/nodes/node-audio-delay/package.json
+++ b/nodes/node-audio-delay/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-delay",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-audio-fade/CHANGELOG.md b/nodes/node-audio-fade/CHANGELOG.md
index e8868318..96ff3a07 100644
--- a/nodes/node-audio-fade/CHANGELOG.md
+++ b/nodes/node-audio-fade/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-fade
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-fade/package.json b/nodes/node-audio-fade/package.json
index e67b0957..e1ef9384 100644
--- a/nodes/node-audio-fade/package.json
+++ b/nodes/node-audio-fade/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-fade",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-audio-noise-gate/CHANGELOG.md b/nodes/node-audio-noise-gate/CHANGELOG.md
index 46d1c6ca..67f52e03 100644
--- a/nodes/node-audio-noise-gate/CHANGELOG.md
+++ b/nodes/node-audio-noise-gate/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-noise-gate
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-noise-gate/package.json b/nodes/node-audio-noise-gate/package.json
index aa633789..2c9fdff0 100644
--- a/nodes/node-audio-noise-gate/package.json
+++ b/nodes/node-audio-noise-gate/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-noise-gate",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-audio-parametric-eq/CHANGELOG.md b/nodes/node-audio-parametric-eq/CHANGELOG.md
index 453cc70a..89b252b5 100644
--- a/nodes/node-audio-parametric-eq/CHANGELOG.md
+++ b/nodes/node-audio-parametric-eq/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-parametric-eq
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-parametric-eq/package.json b/nodes/node-audio-parametric-eq/package.json
index 13fb1447..a938f4ae 100644
--- a/nodes/node-audio-parametric-eq/package.json
+++ b/nodes/node-audio-parametric-eq/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-parametric-eq",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-audio-reverb/CHANGELOG.md b/nodes/node-audio-reverb/CHANGELOG.md
index 34ec1d8e..02eda667 100644
--- a/nodes/node-audio-reverb/CHANGELOG.md
+++ b/nodes/node-audio-reverb/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-reverb
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-reverb/package.json b/nodes/node-audio-reverb/package.json
index a9087728..ae14517b 100644
--- a/nodes/node-audio-reverb/package.json
+++ b/nodes/node-audio-reverb/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-reverb",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-audio-signal-extractor/CHANGELOG.md b/nodes/node-audio-signal-extractor/CHANGELOG.md
index eca4b992..ff53682c 100644
--- a/nodes/node-audio-signal-extractor/CHANGELOG.md
+++ b/nodes/node-audio-signal-extractor/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-audio-signal-extractor
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-audio-signal-extractor/package.json b/nodes/node-audio-signal-extractor/package.json
index 7a26f4fc..b0b62b09 100644
--- a/nodes/node-audio-signal-extractor/package.json
+++ b/nodes/node-audio-signal-extractor/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-audio-signal-extractor",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "AudioSignalExtractor",
diff --git a/nodes/node-blur/CHANGELOG.md b/nodes/node-blur/CHANGELOG.md
index 40d84d74..e7a44d6f 100644
--- a/nodes/node-blur/CHANGELOG.md
+++ b/nodes/node-blur/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-blur
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-blur/package.json b/nodes/node-blur/package.json
index fa82b27f..404a4456 100644
--- a/nodes/node-blur/package.json
+++ b/nodes/node-blur/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-blur",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Blur",
diff --git a/nodes/node-camera-parallax-3d/CHANGELOG.md b/nodes/node-camera-parallax-3d/CHANGELOG.md
index ba956b60..a4734059 100644
--- a/nodes/node-camera-parallax-3d/CHANGELOG.md
+++ b/nodes/node-camera-parallax-3d/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-camera-parallax-3d
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-camera-parallax-3d/package.json b/nodes/node-camera-parallax-3d/package.json
index 0b00d20e..5ef4f841 100644
--- a/nodes/node-camera-parallax-3d/package.json
+++ b/nodes/node-camera-parallax-3d/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-camera-parallax-3d",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "CameraParallax3D",
diff --git a/nodes/node-canvas-generator/CHANGELOG.md b/nodes/node-canvas-generator/CHANGELOG.md
index 2eed4d3c..e8264400 100644
--- a/nodes/node-canvas-generator/CHANGELOG.md
+++ b/nodes/node-canvas-generator/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-canvas-generator
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-canvas-generator/package.json b/nodes/node-canvas-generator/package.json
index aab07c52..a9a69ea0 100644
--- a/nodes/node-canvas-generator/package.json
+++ b/nodes/node-canvas-generator/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-canvas-generator",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-caption-editor/CHANGELOG.md b/nodes/node-caption-editor/CHANGELOG.md
index 0f9461dd..1939742a 100644
--- a/nodes/node-caption-editor/CHANGELOG.md
+++ b/nodes/node-caption-editor/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-caption-editor
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-caption-editor/package.json b/nodes/node-caption-editor/package.json
index 55241927..dcebadf0 100644
--- a/nodes/node-caption-editor/package.json
+++ b/nodes/node-caption-editor/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-caption-editor",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-caption-generator/CHANGELOG.md b/nodes/node-caption-generator/CHANGELOG.md
index 933b882d..a1e0cd8e 100644
--- a/nodes/node-caption-generator/CHANGELOG.md
+++ b/nodes/node-caption-generator/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-caption-generator
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-caption-generator/package.json b/nodes/node-caption-generator/package.json
index 78b8dcb5..76431450 100644
--- a/nodes/node-caption-generator/package.json
+++ b/nodes/node-caption-generator/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-caption-generator",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-channel-merger/CHANGELOG.md b/nodes/node-channel-merger/CHANGELOG.md
index 9bbd127f..f96c240f 100644
--- a/nodes/node-channel-merger/CHANGELOG.md
+++ b/nodes/node-channel-merger/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-channel-merger
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-channel-merger/package.json b/nodes/node-channel-merger/package.json
index 7ce30758..369c4997 100644
--- a/nodes/node-channel-merger/package.json
+++ b/nodes/node-channel-merger/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-channel-merger",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ChannelMerger",
diff --git a/nodes/node-channel-splitter/CHANGELOG.md b/nodes/node-channel-splitter/CHANGELOG.md
index a0d9bffd..08d4a993 100644
--- a/nodes/node-channel-splitter/CHANGELOG.md
+++ b/nodes/node-channel-splitter/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-channel-splitter
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-channel-splitter/package.json b/nodes/node-channel-splitter/package.json
index c117ca5d..0ae36a90 100644
--- a/nodes/node-channel-splitter/package.json
+++ b/nodes/node-channel-splitter/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-channel-splitter",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ChannelSplitter",
diff --git a/nodes/node-color-balance/CHANGELOG.md b/nodes/node-color-balance/CHANGELOG.md
index 8ee22cd1..c2da772b 100644
--- a/nodes/node-color-balance/CHANGELOG.md
+++ b/nodes/node-color-balance/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-color-balance
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-color-balance/package.json b/nodes/node-color-balance/package.json
index 14262d3f..90f78a0a 100644
--- a/nodes/node-color-balance/package.json
+++ b/nodes/node-color-balance/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-color-balance",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ColorBalance",
diff --git a/nodes/node-colorkey/CHANGELOG.md b/nodes/node-colorkey/CHANGELOG.md
index 429260e4..e61f953a 100644
--- a/nodes/node-colorkey/CHANGELOG.md
+++ b/nodes/node-colorkey/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-colorkey
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-colorkey/package.json b/nodes/node-colorkey/package.json
index bac71b35..2ada0cde 100644
--- a/nodes/node-colorkey/package.json
+++ b/nodes/node-colorkey/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-colorkey",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-compositor/CHANGELOG.md b/nodes/node-compositor/CHANGELOG.md
index b838a989..eb2e50e7 100644
--- a/nodes/node-compositor/CHANGELOG.md
+++ b/nodes/node-compositor/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-compositor
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-compositor/package.json b/nodes/node-compositor/package.json
index 86f0d336..5b251cd3 100644
--- a/nodes/node-compositor/package.json
+++ b/nodes/node-compositor/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-compositor",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Compositor",
diff --git a/nodes/node-contour/CHANGELOG.md b/nodes/node-contour/CHANGELOG.md
index f1098eb5..c5201494 100644
--- a/nodes/node-contour/CHANGELOG.md
+++ b/nodes/node-contour/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-contour
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-contour/package.json b/nodes/node-contour/package.json
index 5641ebe6..62b8193f 100644
--- a/nodes/node-contour/package.json
+++ b/nodes/node-contour/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-contour",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Contour",
diff --git a/nodes/node-corner-pin/CHANGELOG.md b/nodes/node-corner-pin/CHANGELOG.md
index f9a1d069..014730f4 100644
--- a/nodes/node-corner-pin/CHANGELOG.md
+++ b/nodes/node-corner-pin/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-corner-pin
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-corner-pin/package.json b/nodes/node-corner-pin/package.json
index ed67c1df..9f5652a6 100644
--- a/nodes/node-corner-pin/package.json
+++ b/nodes/node-corner-pin/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-corner-pin",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-crop/CHANGELOG.md b/nodes/node-crop/CHANGELOG.md
index ad5f14fa..036e5795 100644
--- a/nodes/node-crop/CHANGELOG.md
+++ b/nodes/node-crop/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-crop
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-crop/package.json b/nodes/node-crop/package.json
index 1392eb74..900663fb 100644
--- a/nodes/node-crop/package.json
+++ b/nodes/node-crop/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-crop",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-crt/CHANGELOG.md b/nodes/node-crt/CHANGELOG.md
index d6448a7e..43ab6181 100644
--- a/nodes/node-crt/CHANGELOG.md
+++ b/nodes/node-crt/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-crt
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-crt/package.json b/nodes/node-crt/package.json
index c749854a..c8658a96 100644
--- a/nodes/node-crt/package.json
+++ b/nodes/node-crt/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-crt",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "CRT",
diff --git a/nodes/node-curves/CHANGELOG.md b/nodes/node-curves/CHANGELOG.md
index f055fdc8..28cdc821 100644
--- a/nodes/node-curves/CHANGELOG.md
+++ b/nodes/node-curves/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-curves
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-curves/package.json b/nodes/node-curves/package.json
index 8df08db4..365b9cc5 100644
--- a/nodes/node-curves/package.json
+++ b/nodes/node-curves/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-curves",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-datamosh/CHANGELOG.md b/nodes/node-datamosh/CHANGELOG.md
index 92bef23d..debe08dd 100644
--- a/nodes/node-datamosh/CHANGELOG.md
+++ b/nodes/node-datamosh/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-datamosh
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-datamosh/package.json b/nodes/node-datamosh/package.json
index f120e320..7b7e4cbd 100644
--- a/nodes/node-datamosh/package.json
+++ b/nodes/node-datamosh/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-datamosh",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Datamosh",
diff --git a/nodes/node-deflicker/CHANGELOG.md b/nodes/node-deflicker/CHANGELOG.md
index 15b33ed5..0afac1c5 100644
--- a/nodes/node-deflicker/CHANGELOG.md
+++ b/nodes/node-deflicker/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-deflicker
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-deflicker/package.json b/nodes/node-deflicker/package.json
index e1d61298..2797931f 100644
--- a/nodes/node-deflicker/package.json
+++ b/nodes/node-deflicker/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-deflicker",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "TemporalDeflicker",
diff --git a/nodes/node-displacement-map/CHANGELOG.md b/nodes/node-displacement-map/CHANGELOG.md
index 4c9ccae2..7b25a3eb 100644
--- a/nodes/node-displacement-map/CHANGELOG.md
+++ b/nodes/node-displacement-map/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-displacement-map
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-displacement-map/package.json b/nodes/node-displacement-map/package.json
index 78eb446c..43f1f386 100644
--- a/nodes/node-displacement-map/package.json
+++ b/nodes/node-displacement-map/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-displacement-map",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "DisplacementMap",
diff --git a/nodes/node-dither/CHANGELOG.md b/nodes/node-dither/CHANGELOG.md
index 91a734a3..f107bb7e 100644
--- a/nodes/node-dither/CHANGELOG.md
+++ b/nodes/node-dither/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-dither
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-dither/package.json b/nodes/node-dither/package.json
index 70683bbc..68daeee4 100644
--- a/nodes/node-dither/package.json
+++ b/nodes/node-dither/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-dither",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Dither",
diff --git a/nodes/node-extract-frame/CHANGELOG.md b/nodes/node-extract-frame/CHANGELOG.md
index 55a75e06..0076b2f9 100644
--- a/nodes/node-extract-frame/CHANGELOG.md
+++ b/nodes/node-extract-frame/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-extract-frame
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-extract-frame/package.json b/nodes/node-extract-frame/package.json
index c867d0af..235c4612 100644
--- a/nodes/node-extract-frame/package.json
+++ b/nodes/node-extract-frame/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-extract-frame",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-extract-lut/CHANGELOG.md b/nodes/node-extract-lut/CHANGELOG.md
index f9a47882..dcf62045 100644
--- a/nodes/node-extract-lut/CHANGELOG.md
+++ b/nodes/node-extract-lut/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-extract-lut
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-extract-lut/package.json b/nodes/node-extract-lut/package.json
index 3eb89c67..04031bdc 100644
--- a/nodes/node-extract-lut/package.json
+++ b/nodes/node-extract-lut/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-extract-lut",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ExtractLUT",
diff --git a/nodes/node-film-grain/CHANGELOG.md b/nodes/node-film-grain/CHANGELOG.md
index 238d6992..6642f88a 100644
--- a/nodes/node-film-grain/CHANGELOG.md
+++ b/nodes/node-film-grain/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-film-grain
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-film-grain/package.json b/nodes/node-film-grain/package.json
index 26bdb8de..3f26c4d8 100644
--- a/nodes/node-film-grain/package.json
+++ b/nodes/node-film-grain/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-film-grain",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "FilmGrain",
diff --git a/nodes/node-flip/CHANGELOG.md b/nodes/node-flip/CHANGELOG.md
index 2b2049b3..ea3ce9dc 100644
--- a/nodes/node-flip/CHANGELOG.md
+++ b/nodes/node-flip/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-flip
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-flip/package.json b/nodes/node-flip/package.json
index cb95b5d4..b0716ed8 100644
--- a/nodes/node-flip/package.json
+++ b/nodes/node-flip/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-flip",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Flip",
diff --git a/nodes/node-gooey/CHANGELOG.md b/nodes/node-gooey/CHANGELOG.md
index bb5e8e6f..07ba62e2 100644
--- a/nodes/node-gooey/CHANGELOG.md
+++ b/nodes/node-gooey/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-gooey
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-gooey/package.json b/nodes/node-gooey/package.json
index 8fda47c5..f0aff623 100644
--- a/nodes/node-gooey/package.json
+++ b/nodes/node-gooey/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-gooey",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Gooey",
diff --git a/nodes/node-gradient-map/CHANGELOG.md b/nodes/node-gradient-map/CHANGELOG.md
index 8b0c389b..d990f240 100644
--- a/nodes/node-gradient-map/CHANGELOG.md
+++ b/nodes/node-gradient-map/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-gradient-map
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-gradient-map/package.json b/nodes/node-gradient-map/package.json
index ec72332c..680a05b6 100644
--- a/nodes/node-gradient-map/package.json
+++ b/nodes/node-gradient-map/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-gradient-map",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "GradientMap",
diff --git a/nodes/node-halftone-screen/CHANGELOG.md b/nodes/node-halftone-screen/CHANGELOG.md
index 786114f2..a7d68c11 100644
--- a/nodes/node-halftone-screen/CHANGELOG.md
+++ b/nodes/node-halftone-screen/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-halftone-screen
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-halftone-screen/package.json b/nodes/node-halftone-screen/package.json
index 93537c9d..c3c80feb 100644
--- a/nodes/node-halftone-screen/package.json
+++ b/nodes/node-halftone-screen/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-halftone-screen",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "HalftoneScreen",
diff --git a/nodes/node-high-pass/CHANGELOG.md b/nodes/node-high-pass/CHANGELOG.md
index 026166cd..5645a2a4 100644
--- a/nodes/node-high-pass/CHANGELOG.md
+++ b/nodes/node-high-pass/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-high-pass
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-high-pass/package.json b/nodes/node-high-pass/package.json
index 9104646c..34bb4e6f 100644
--- a/nodes/node-high-pass/package.json
+++ b/nodes/node-high-pass/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-high-pass",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "HighPass",
diff --git a/nodes/node-iridescence/CHANGELOG.md b/nodes/node-iridescence/CHANGELOG.md
index e1bc6d05..c70c3415 100644
--- a/nodes/node-iridescence/CHANGELOG.md
+++ b/nodes/node-iridescence/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-iridescence
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-iridescence/package.json b/nodes/node-iridescence/package.json
index c392726b..5aba794a 100644
--- a/nodes/node-iridescence/package.json
+++ b/nodes/node-iridescence/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-iridescence",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Iridescence",
diff --git a/nodes/node-kaleidoscope/CHANGELOG.md b/nodes/node-kaleidoscope/CHANGELOG.md
index 03a7a207..9dc51217 100644
--- a/nodes/node-kaleidoscope/CHANGELOG.md
+++ b/nodes/node-kaleidoscope/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-kaleidoscope
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-kaleidoscope/package.json b/nodes/node-kaleidoscope/package.json
index aa977aa7..71c3b948 100644
--- a/nodes/node-kaleidoscope/package.json
+++ b/nodes/node-kaleidoscope/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-kaleidoscope",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Kaleidoscope",
diff --git a/nodes/node-kenburns/CHANGELOG.md b/nodes/node-kenburns/CHANGELOG.md
index f4c40358..3988acab 100644
--- a/nodes/node-kenburns/CHANGELOG.md
+++ b/nodes/node-kenburns/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-kenburns
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-kenburns/package.json b/nodes/node-kenburns/package.json
index 747a333c..e82e214d 100644
--- a/nodes/node-kenburns/package.json
+++ b/nodes/node-kenburns/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-kenburns",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-layer-style/CHANGELOG.md b/nodes/node-layer-style/CHANGELOG.md
index 6b7ebc3b..9df1b075 100644
--- a/nodes/node-layer-style/CHANGELOG.md
+++ b/nodes/node-layer-style/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-layer-style
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-layer-style/package.json b/nodes/node-layer-style/package.json
index daf4bae9..40e61e5d 100644
--- a/nodes/node-layer-style/package.json
+++ b/nodes/node-layer-style/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-layer-style",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "LayerStyle",
diff --git a/nodes/node-levels/CHANGELOG.md b/nodes/node-levels/CHANGELOG.md
index 663a55a1..92ea2945 100644
--- a/nodes/node-levels/CHANGELOG.md
+++ b/nodes/node-levels/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-levels
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-levels/package.json b/nodes/node-levels/package.json
index 63ec55b4..b1fa393a 100644
--- a/nodes/node-levels/package.json
+++ b/nodes/node-levels/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-levels",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-liquify/CHANGELOG.md b/nodes/node-liquify/CHANGELOG.md
index 0408d47c..c9b5a4b4 100644
--- a/nodes/node-liquify/CHANGELOG.md
+++ b/nodes/node-liquify/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-liquify
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-liquify/package.json b/nodes/node-liquify/package.json
index 9a61f46d..0d26bd9e 100644
--- a/nodes/node-liquify/package.json
+++ b/nodes/node-liquify/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-liquify",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Liquify",
diff --git a/nodes/node-mask-math/CHANGELOG.md b/nodes/node-mask-math/CHANGELOG.md
index 75b5872d..a64bb7a5 100644
--- a/nodes/node-mask-math/CHANGELOG.md
+++ b/nodes/node-mask-math/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-mask-math
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-mask-math/package.json b/nodes/node-mask-math/package.json
index 49f42483..ab311e4b 100644
--- a/nodes/node-mask-math/package.json
+++ b/nodes/node-mask-math/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-mask-math",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "MaskMath",
diff --git a/nodes/node-mesh-warp/CHANGELOG.md b/nodes/node-mesh-warp/CHANGELOG.md
index 844ba5e2..4cc39086 100644
--- a/nodes/node-mesh-warp/CHANGELOG.md
+++ b/nodes/node-mesh-warp/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-mesh-warp
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-mesh-warp/package.json b/nodes/node-mesh-warp/package.json
index ce58c52d..8799cc4d 100644
--- a/nodes/node-mesh-warp/package.json
+++ b/nodes/node-mesh-warp/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-mesh-warp",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-modulate/CHANGELOG.md b/nodes/node-modulate/CHANGELOG.md
index fb9971dc..1db0947b 100644
--- a/nodes/node-modulate/CHANGELOG.md
+++ b/nodes/node-modulate/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-modulate
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-modulate/package.json b/nodes/node-modulate/package.json
index a1e56f20..06e6856c 100644
--- a/nodes/node-modulate/package.json
+++ b/nodes/node-modulate/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-modulate",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-noise-generator/CHANGELOG.md b/nodes/node-noise-generator/CHANGELOG.md
index 60db60de..6dbb49e2 100644
--- a/nodes/node-noise-generator/CHANGELOG.md
+++ b/nodes/node-noise-generator/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-noise-generator
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-noise-generator/package.json b/nodes/node-noise-generator/package.json
index 8f939e1d..18b304bf 100644
--- a/nodes/node-noise-generator/package.json
+++ b/nodes/node-noise-generator/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-noise-generator",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "NoiseGenerator",
diff --git a/nodes/node-op-art/CHANGELOG.md b/nodes/node-op-art/CHANGELOG.md
index a648424e..38187829 100644
--- a/nodes/node-op-art/CHANGELOG.md
+++ b/nodes/node-op-art/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-op-art
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-op-art/package.json b/nodes/node-op-art/package.json
index 27b423fd..2e85e1db 100644
--- a/nodes/node-op-art/package.json
+++ b/nodes/node-op-art/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-op-art",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "OpArt",
diff --git a/nodes/node-paint/CHANGELOG.md b/nodes/node-paint/CHANGELOG.md
index f97c51a8..98f3a8bd 100644
--- a/nodes/node-paint/CHANGELOG.md
+++ b/nodes/node-paint/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-paint
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-paint/package.json b/nodes/node-paint/package.json
index a4561990..6db124e5 100644
--- a/nodes/node-paint/package.json
+++ b/nodes/node-paint/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-paint",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-painterly/CHANGELOG.md b/nodes/node-painterly/CHANGELOG.md
index 8ccd0861..eb2bf02c 100644
--- a/nodes/node-painterly/CHANGELOG.md
+++ b/nodes/node-painterly/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-painterly
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-painterly/package.json b/nodes/node-painterly/package.json
index 6472d1c6..43794b8d 100644
--- a/nodes/node-painterly/package.json
+++ b/nodes/node-painterly/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-painterly",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Painterly",
diff --git a/nodes/node-particles/CHANGELOG.md b/nodes/node-particles/CHANGELOG.md
index 4787f916..244afcf3 100644
--- a/nodes/node-particles/CHANGELOG.md
+++ b/nodes/node-particles/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-particles
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-particles/package.json b/nodes/node-particles/package.json
index b7a09104..2942259c 100644
--- a/nodes/node-particles/package.json
+++ b/nodes/node-particles/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-particles",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Particles",
diff --git a/nodes/node-patch-heal/CHANGELOG.md b/nodes/node-patch-heal/CHANGELOG.md
index f6c6c955..0e86b904 100644
--- a/nodes/node-patch-heal/CHANGELOG.md
+++ b/nodes/node-patch-heal/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-patch-heal
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-patch-heal/package.json b/nodes/node-patch-heal/package.json
index c8a8fa55..c74f46fe 100644
--- a/nodes/node-patch-heal/package.json
+++ b/nodes/node-patch-heal/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-patch-heal",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "PatchHeal",
diff --git a/nodes/node-pixelate/CHANGELOG.md b/nodes/node-pixelate/CHANGELOG.md
index 44ac53e6..472e9e7d 100644
--- a/nodes/node-pixelate/CHANGELOG.md
+++ b/nodes/node-pixelate/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-pixelate
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-pixelate/package.json b/nodes/node-pixelate/package.json
index 229f3c8e..f943bef7 100644
--- a/nodes/node-pixelate/package.json
+++ b/nodes/node-pixelate/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-pixelate",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Pixelate",
diff --git a/nodes/node-posterize/CHANGELOG.md b/nodes/node-posterize/CHANGELOG.md
index 95784d81..aecd1700 100644
--- a/nodes/node-posterize/CHANGELOG.md
+++ b/nodes/node-posterize/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-posterize
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-posterize/package.json b/nodes/node-posterize/package.json
index dd53d85c..d42b2692 100644
--- a/nodes/node-posterize/package.json
+++ b/nodes/node-posterize/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-posterize",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Posterize",
diff --git a/nodes/node-procedural-vfx/CHANGELOG.md b/nodes/node-procedural-vfx/CHANGELOG.md
index 8ad5f45a..7723a01b 100644
--- a/nodes/node-procedural-vfx/CHANGELOG.md
+++ b/nodes/node-procedural-vfx/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-procedural-vfx
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-procedural-vfx/package.json b/nodes/node-procedural-vfx/package.json
index 4b8e0961..1ef1fe46 100644
--- a/nodes/node-procedural-vfx/package.json
+++ b/nodes/node-procedural-vfx/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-procedural-vfx",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ProceduralVFX",
diff --git a/nodes/node-refine-edge/CHANGELOG.md b/nodes/node-refine-edge/CHANGELOG.md
index 81820779..04a760d3 100644
--- a/nodes/node-refine-edge/CHANGELOG.md
+++ b/nodes/node-refine-edge/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-refine-edge
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-refine-edge/package.json b/nodes/node-refine-edge/package.json
index 4012918c..311fc860 100644
--- a/nodes/node-refine-edge/package.json
+++ b/nodes/node-refine-edge/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-refine-edge",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "RefineEdge",
diff --git a/nodes/node-refraction-caustics/CHANGELOG.md b/nodes/node-refraction-caustics/CHANGELOG.md
index 7d6cd818..abbc8f69 100644
--- a/nodes/node-refraction-caustics/CHANGELOG.md
+++ b/nodes/node-refraction-caustics/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-refraction-caustics
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-refraction-caustics/package.json b/nodes/node-refraction-caustics/package.json
index e678a248..ad3cb561 100644
--- a/nodes/node-refraction-caustics/package.json
+++ b/nodes/node-refraction-caustics/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-refraction-caustics",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "RefractionCaustics3D",
diff --git a/nodes/node-relight-3d/CHANGELOG.md b/nodes/node-relight-3d/CHANGELOG.md
index 4487bd47..06b52b08 100644
--- a/nodes/node-relight-3d/CHANGELOG.md
+++ b/nodes/node-relight-3d/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-relight-3d
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-relight-3d/package.json b/nodes/node-relight-3d/package.json
index 5e678e44..65cea633 100644
--- a/nodes/node-relight-3d/package.json
+++ b/nodes/node-relight-3d/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-relight-3d",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Relight3D",
diff --git a/nodes/node-resizer-scaler/CHANGELOG.md b/nodes/node-resizer-scaler/CHANGELOG.md
index 58a71e94..b0d50acc 100644
--- a/nodes/node-resizer-scaler/CHANGELOG.md
+++ b/nodes/node-resizer-scaler/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-resizer-scaler
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-resizer-scaler/package.json b/nodes/node-resizer-scaler/package.json
index 7741e506..d91f6aa9 100644
--- a/nodes/node-resizer-scaler/package.json
+++ b/nodes/node-resizer-scaler/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-resizer-scaler",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ResizerScaler",
diff --git a/nodes/node-rgb-split/CHANGELOG.md b/nodes/node-rgb-split/CHANGELOG.md
index d8b09744..a8728089 100644
--- a/nodes/node-rgb-split/CHANGELOG.md
+++ b/nodes/node-rgb-split/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-rgb-split
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-rgb-split/package.json b/nodes/node-rgb-split/package.json
index a2edd2f6..c8de57de 100644
--- a/nodes/node-rgb-split/package.json
+++ b/nodes/node-rgb-split/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-rgb-split",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "RGBSplit",
diff --git a/nodes/node-rotate/CHANGELOG.md b/nodes/node-rotate/CHANGELOG.md
index f403d8c5..5b0bb7e2 100644
--- a/nodes/node-rotate/CHANGELOG.md
+++ b/nodes/node-rotate/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-rotate
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-rotate/package.json b/nodes/node-rotate/package.json
index 70464609..6204c5d4 100644
--- a/nodes/node-rotate/package.json
+++ b/nodes/node-rotate/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-rotate",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Rotate",
diff --git a/nodes/node-selective-color/CHANGELOG.md b/nodes/node-selective-color/CHANGELOG.md
index 86d78ce1..84dd7d42 100644
--- a/nodes/node-selective-color/CHANGELOG.md
+++ b/nodes/node-selective-color/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-selective-color
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-selective-color/package.json b/nodes/node-selective-color/package.json
index 4a5e9f54..00598440 100644
--- a/nodes/node-selective-color/package.json
+++ b/nodes/node-selective-color/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-selective-color",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "SelectiveColor",
diff --git a/nodes/node-shadows-highlights/CHANGELOG.md b/nodes/node-shadows-highlights/CHANGELOG.md
index 5224c497..6a635c32 100644
--- a/nodes/node-shadows-highlights/CHANGELOG.md
+++ b/nodes/node-shadows-highlights/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-shadows-highlights
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-shadows-highlights/package.json b/nodes/node-shadows-highlights/package.json
index 49442679..3e0ee452 100644
--- a/nodes/node-shadows-highlights/package.json
+++ b/nodes/node-shadows-highlights/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-shadows-highlights",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ShadowsHighlights",
diff --git a/nodes/node-shape-generator/CHANGELOG.md b/nodes/node-shape-generator/CHANGELOG.md
index 46d08d7f..74bb7043 100644
--- a/nodes/node-shape-generator/CHANGELOG.md
+++ b/nodes/node-shape-generator/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-shape-generator
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-shape-generator/package.json b/nodes/node-shape-generator/package.json
index 39f35244..db1a4a87 100644
--- a/nodes/node-shape-generator/package.json
+++ b/nodes/node-shape-generator/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-shape-generator",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "ShapeGenerator",
diff --git a/nodes/node-signal-gate/CHANGELOG.md b/nodes/node-signal-gate/CHANGELOG.md
index 487c744a..15dd5a9e 100644
--- a/nodes/node-signal-gate/CHANGELOG.md
+++ b/nodes/node-signal-gate/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-signal-gate
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-signal-gate/package.json b/nodes/node-signal-gate/package.json
index 00cbec09..09ce1fc0 100644
--- a/nodes/node-signal-gate/package.json
+++ b/nodes/node-signal-gate/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-signal-gate",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "SignalGate",
diff --git a/nodes/node-signal-math/CHANGELOG.md b/nodes/node-signal-math/CHANGELOG.md
index ef0a1338..240caef7 100644
--- a/nodes/node-signal-math/CHANGELOG.md
+++ b/nodes/node-signal-math/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-signal-math
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-signal-math/package.json b/nodes/node-signal-math/package.json
index bcb7c644..0b4c2cee 100644
--- a/nodes/node-signal-math/package.json
+++ b/nodes/node-signal-math/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-signal-math",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "SignalMath",
diff --git a/nodes/node-signal/CHANGELOG.md b/nodes/node-signal/CHANGELOG.md
index 681ac893..197d26d0 100644
--- a/nodes/node-signal/CHANGELOG.md
+++ b/nodes/node-signal/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-signal
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-signal/package.json b/nodes/node-signal/package.json
index 19ec0da9..a0313bad 100644
--- a/nodes/node-signal/package.json
+++ b/nodes/node-signal/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-signal",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Signal",
diff --git a/nodes/node-stereo-panning/CHANGELOG.md b/nodes/node-stereo-panning/CHANGELOG.md
index 5aa58e06..36f6e51d 100644
--- a/nodes/node-stereo-panning/CHANGELOG.md
+++ b/nodes/node-stereo-panning/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-stereo-panning
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-stereo-panning/package.json b/nodes/node-stereo-panning/package.json
index 354ec966..5b8d50e4 100644
--- a/nodes/node-stereo-panning/package.json
+++ b/nodes/node-stereo-panning/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-stereo-panning",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/nodes/node-text/CHANGELOG.md b/nodes/node-text/CHANGELOG.md
index 53687e59..9419ee27 100644
--- a/nodes/node-text/CHANGELOG.md
+++ b/nodes/node-text/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-text
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-text/package.json b/nodes/node-text/package.json
index deb6edd0..d9a3b9b6 100644
--- a/nodes/node-text/package.json
+++ b/nodes/node-text/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-text",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Text",
diff --git a/nodes/node-tile-offset/CHANGELOG.md b/nodes/node-tile-offset/CHANGELOG.md
index 6807ffa9..573e9014 100644
--- a/nodes/node-tile-offset/CHANGELOG.md
+++ b/nodes/node-tile-offset/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-tile-offset
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-tile-offset/package.json b/nodes/node-tile-offset/package.json
index d79af3c2..e03f6f4e 100644
--- a/nodes/node-tile-offset/package.json
+++ b/nodes/node-tile-offset/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-tile-offset",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "TileOffset",
diff --git a/nodes/node-toon/CHANGELOG.md b/nodes/node-toon/CHANGELOG.md
index 1c205c1f..b3603b49 100644
--- a/nodes/node-toon/CHANGELOG.md
+++ b/nodes/node-toon/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-toon
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-toon/package.json b/nodes/node-toon/package.json
index 438af830..3a06a776 100644
--- a/nodes/node-toon/package.json
+++ b/nodes/node-toon/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-toon",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Toon",
diff --git a/nodes/node-trails/CHANGELOG.md b/nodes/node-trails/CHANGELOG.md
index 4d1ffa8a..28bc7594 100644
--- a/nodes/node-trails/CHANGELOG.md
+++ b/nodes/node-trails/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-trails
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-trails/package.json b/nodes/node-trails/package.json
index 815885e6..8e2f6966 100644
--- a/nodes/node-trails/package.json
+++ b/nodes/node-trails/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-trails",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Trails",
diff --git a/nodes/node-unsharp-mask/CHANGELOG.md b/nodes/node-unsharp-mask/CHANGELOG.md
index 241d2b36..3cab1002 100644
--- a/nodes/node-unsharp-mask/CHANGELOG.md
+++ b/nodes/node-unsharp-mask/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-unsharp-mask
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-unsharp-mask/package.json b/nodes/node-unsharp-mask/package.json
index 79505ee4..93a1b18c 100644
--- a/nodes/node-unsharp-mask/package.json
+++ b/nodes/node-unsharp-mask/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-unsharp-mask",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "UnsharpMask",
diff --git a/nodes/node-vignette/CHANGELOG.md b/nodes/node-vignette/CHANGELOG.md
index 5c809c46..4ddaf4cd 100644
--- a/nodes/node-vignette/CHANGELOG.md
+++ b/nodes/node-vignette/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-vignette
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-vignette/package.json b/nodes/node-vignette/package.json
index c658abd2..0946fa29 100644
--- a/nodes/node-vignette/package.json
+++ b/nodes/node-vignette/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-vignette",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"framefields": {
"type": "Vignette",
diff --git a/nodes/node-vision/CHANGELOG.md b/nodes/node-vision/CHANGELOG.md
index f4461281..a57e4c4e 100644
--- a/nodes/node-vision/CHANGELOG.md
+++ b/nodes/node-vision/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-vision
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/nodes/node-vision/package.json b/nodes/node-vision/package.json
index f0b14dcb..0100b0ae 100644
--- a/nodes/node-vision/package.json
+++ b/nodes/node-vision/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-vision",
- "version": "2.9.3",
+ "version": "2.11.0",
"license": "Apache-2.0",
"private": false,
"framefields": {
diff --git a/package.json b/package.json
index 399ff64f..44209dde 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"private": true,
- "version": "2.9.3",
+ "version": "2.11.0",
"scripts": {
"build": "turbo run build",
"dev": "turbo run dev",
diff --git a/packages/ai/CHANGELOG.md b/packages/ai/CHANGELOG.md
index 2ee6ad48..7a36414f 100644
--- a/packages/ai/CHANGELOG.md
+++ b/packages/ai/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/ai
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/ai/package.json b/packages/ai/package.json
index 5053cf5e..988745a7 100644
--- a/packages/ai/package.json
+++ b/packages/ai/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/ai",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"description": "AI models (images, video, music, voice) through the framefields AI gateway, authenticated with a FRAMEFIELDS_API_KEY.",
"publishConfig": {
diff --git a/packages/client-utils/CHANGELOG.md b/packages/client-utils/CHANGELOG.md
index 34fd626b..01a93dc1 100644
--- a/packages/client-utils/CHANGELOG.md
+++ b/packages/client-utils/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/client-utils
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/client-utils/package.json b/packages/client-utils/package.json
index ca41ce6e..b0f59268 100644
--- a/packages/client-utils/package.json
+++ b/packages/client-utils/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/client-utils",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"publishConfig": {
"access": "public"
diff --git a/packages/components/CHANGELOG.md b/packages/components/CHANGELOG.md
index b2a4336e..ff8dff82 100644
--- a/packages/components/CHANGELOG.md
+++ b/packages/components/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/components
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/components/package.json b/packages/components/package.json
index fd9c693d..4119d972 100644
--- a/packages/components/package.json
+++ b/packages/components/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/components",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"description": "Parametric motion components for framefields. Internal: the toolkit ships in `framefields/components`; components ship as a registry that `framefields add` copies into projects.",
"type": "module",
diff --git a/packages/compositions/CHANGELOG.md b/packages/compositions/CHANGELOG.md
index 14971ed7..6c37e9cd 100644
--- a/packages/compositions/CHANGELOG.md
+++ b/packages/compositions/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/compositions
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/compositions/package.json b/packages/compositions/package.json
index c564cb18..5a9c0474 100644
--- a/packages/compositions/package.json
+++ b/packages/compositions/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/compositions",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"type": "module",
"sideEffects": false,
diff --git a/packages/core/CHANGELOG.md b/packages/core/CHANGELOG.md
index cabd0b8b..8d5e2183 100644
--- a/packages/core/CHANGELOG.md
+++ b/packages/core/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/core
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/core/package.json b/packages/core/package.json
index 385308eb..b5cf1b31 100644
--- a/packages/core/package.json
+++ b/packages/core/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/core",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"publishConfig": {
"access": "public"
diff --git a/packages/framefields/CHANGELOG.md b/packages/framefields/CHANGELOG.md
index 930d7683..ce0b21f7 100644
--- a/packages/framefields/CHANGELOG.md
+++ b/packages/framefields/CHANGELOG.md
@@ -1,5 +1,16 @@
# framefields
+## 2.11.0
+
+### Minor Changes
+
+- `AudioSignal.pitch(source, { fps, minHz, maxHz })` tracks the fundamental frequency of audio or video on the GPU: YIN period candidates per frame, then a Viterbi decode that rejects octave and subharmonic errors. It returns `hz`, `confidence` and `voiced` signals; `hz` is interpolated across unvoiced gaps. Use it for engine RPM, melody lines or any pitched source.
+- Split a long render across GPUs. `renderVideo({ frames: { start, end }, audio: false })` renders one chunk of frames: it starts on a keyframe, its timestamps start at 0, and trails, datamosh and particles rebuild their state before `start`. `stitchChunks({ chunks, audio, outputPath })` from `framefields/renderer` joins the chunks and a separate audio-only render (`codec: "aac"`) into one file without re-encoding or a GPU. It refuses chunks encoded with a different codec, size or encoder parameters. `planChunks({ totalFrames, samples, workers: "auto" })` decides how many chunks a render should use (more only while each chunk stays at least 85% efficient against the per-worker start-up) and cuts them at equal render cost; `renderer.probeRender(comp)` measures the frame costs it needs.
+
+ `framefields render <entry> --cloud [--workers auto] [--plan]` renders the pushed commit of the project's git repo on api.framefields.dev and downloads the video; `--plan` prints the workers, time and estimated cost without rendering. The same from code: `renderCloud()` from the new `framefields/cloud` entry, which needs no GPU. `framefields init` adds a `render:cloud` script.
+
+- ab1cc06: FocusZoom takes `drift`: a slow push while each shot holds (zoom gained per second, e.g. `0.012`), so the picture never stops between camera moves. Defaults to 0, which holds still as before.
+
## 2.9.3
### Patch Changes
diff --git a/packages/framefields/package.json b/packages/framefields/package.json
index bc15fc29..4574fc37 100644
--- a/packages/framefields/package.json
+++ b/packages/framefields/package.json
@@ -1,6 +1,6 @@
{
"name": "framefields",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"description": "High-performance WebGPU rendering SDK for video composition, Photoshop-grade 2D VFX, audio DSP, and reactive signals. Built for AI Agents.",
"license": "Apache-2.0",
@@ -81,6 +81,12 @@
"import": "./dist/preview/index.mjs",
"default": "./dist/preview/index.mjs"
},
+ "./cloud": {
+ "types": "./dist/cloud/index.d.mts",
+ "development": "./src/cloud/index.ts",
+ "import": "./dist/cloud/index.mjs",
+ "default": "./dist/cloud/index.mjs"
+ },
"./sam": {
"types": "./dist/sam/index.d.mts",
"development": "./src/sam/index.ts",
diff --git a/packages/framefields/src/cli.ts b/packages/framefields/src/cli.ts
index e6a55506..07222f48 100644
--- a/packages/framefields/src/cli.ts
+++ b/packages/framefields/src/cli.ts
@@ -7,6 +7,7 @@
* npx framefields list the components you can add
* npx framefields fonts add <role...> install fonts (emoji)
* npx framefields fonts list the fonts you can install
+ * npx framefields render <entry> --cloud render on api.framefields.dev
*/
import { readFileSync } from "node:fs";
import { relative, resolve } from "node:path";
@@ -14,6 +15,7 @@ import { fileURLToPath } from "node:url";
import { parseArgs } from "node:util";
import { type AddReport, add } from "./cli/add.js";
import { runCheck } from "./cli/check.js";
+import { runCloudRender } from "./cli/cloud.js";
import { readConfig } from "./cli/config.js";
import {
FONTS,
@@ -51,6 +53,10 @@ Commands:
Install fonts (pinned, hash-checked, with their license)
and register them in the generated <components>/fonts.ts.
fonts list List the fonts you can install.
+ render <entry> --cloud
+ Render on api.framefields.dev from the pushed commit of
+ this git repo, and download the video. Needs
+ FRAMEFIELDS_API_KEY. --export defaults to buildFilm.
Options for check:
--export <name> Composition or builder export (default buildFilm).
@@ -60,6 +66,20 @@ Options for check:
--warnings-as-errors Fail on lint warnings.
--json Machine-readable check, component list, or font list result.
+Options for render --cloud:
+ --workers <n|auto> 1 (default) renders on one GPU. auto measures the film
+ first and splits it across as many GPUs as stay
+ efficient; a number splits it into that many.
+ --max-workers <n> Upper bound for auto.
+ --plan Measure and plan only: prints workers, time and the
+ estimated cost, renders nothing.
+ --output <file> Where the video goes (default output/<entry>.mp4).
+ --codec <codec> h264 (default) or vp9.
+ --quality <q> very_low, low, high or very_high (default).
+ --select <name> When the export returns several compositions.
+ --json Print the final status as JSON.
+ Ctrl-C cancels the render; GPU time already used is charged.
+
Options for init:
--name <name> Package name (default: the folder's name).
--pm <manager> npm, pnpm, yarn or bun (default: the one running this).
@@ -87,6 +107,7 @@ Examples:
npx framefields init my-film --install
npx framefields add mouse-pointer focus-zoom
npx framefields fonts add emoji
+ npx framefields render src/film.ts --cloud --workers auto --plan
`;
async function main(argv: string[]): Promise<number> {
@@ -112,6 +133,13 @@ async function main(argv: string[]): Promise<number> {
output: { type: "string" },
timeout: { type: "string" },
"warnings-as-errors": { type: "boolean" },
+ cloud: { type: "boolean" },
+ workers: { type: "string" },
+ "max-workers": { type: "string" },
+ plan: { type: "boolean" },
+ codec: { type: "string" },
+ quality: { type: "string" },
+ select: { type: "string" },
help: { type: "boolean", short: "h" },
},
});
@@ -164,6 +192,29 @@ async function main(argv: string[]): Promise<number> {
}
return report.status === "passed" ? 0 : 1;
}
+ if (command === "render") {
+ if (names.length !== 1 || !values.cloud) {
+ process.stderr.write(
+ "Usage: framefields render <entry> --cloud [--workers auto] [--plan]\nLocal renders run your project's render script (`npm run render`).\n",
+ );
+ return 2;
+ }
+ return runCloudRender(
+ {
+ entry: names[0],
+ export: values.export,
+ select: values.select,
+ workers: values.workers,
+ maxWorkers: values["max-workers"],
+ codec: values.codec,
+ quality: values.quality,
+ output: values.output,
+ plan: values.plan,
+ json: values.json,
+ },
+ log,
+ );
+ }
if (command === "fonts") return fonts(names, cwd, values, log);
if (command === "init") {
if (names.length > 1) {
diff --git a/packages/framefields/src/cli/cloud.ts b/packages/framefields/src/cli/cloud.ts
new file mode 100644
index 00000000..5a7fca68
--- /dev/null
+++ b/packages/framefields/src/cli/cloud.ts
@@ -0,0 +1,184 @@
+/**
+ * `framefields render <entry> --cloud`: renders the project on
+ * api.framefields.dev and downloads the video. Prints one plain line per
+ * phase, and a progress line at most every 10 seconds while frames render.
+ */
+import path from "node:path";
+import {
+ CloudRenderError,
+ type CloudRenderOptions,
+ type CloudRenderStatus,
+ renderCloud,
+} from "../cloud/index.js";
+
+export interface CloudCliOptions {
+ entry: string;
+ export?: string;
+ select?: string;
+ workers?: string;
+ maxWorkers?: string;
+ codec?: string;
+ quality?: string;
+ output?: string;
+ plan?: boolean;
+ json?: boolean;
+}
+
+const PROGRESS_EVERY_MS = 10_000;
+
+export async function runCloudRender(
+ o: CloudCliOptions,
+ log: (line: string) => void,
+ render: (
+ options: CloudRenderOptions,
+ ) => Promise<CloudRenderStatus> = renderCloud,
+): Promise<number> {
+ const workers = parseWorkers(o.workers);
+ if (workers === undefined) {
+ log(
+ `--workers must be "auto" or a positive whole number, not "${o.workers}".`,
+ );
+ return 2;
+ }
+ if (o.codec && o.codec !== "h264" && o.codec !== "vp9") {
+ log(`--codec must be h264 or vp9, not "${o.codec}".`);
+ return 2;
+ }
+ const codec = (o.codec ?? "h264") as "h264" | "vp9";
+ const outputPath =
+ o.output ??
+ path.join(
+ "output",
+ `${path.basename(o.entry).replace(/\.[^.]+$/, "")}.${codec === "vp9" ? "webm" : "mp4"}`,
+ );
+ const controller = new AbortController();
+ const onSigint = () => {
+ log("Cancelling the cloud render…");
+ controller.abort(new Error("cancelled"));
+ };
+ process.once("SIGINT", onSigint);
+ const started = Date.now();
+ const narrate = narrator(o.json ? () => {} : log, Date.now);
+ try {
+ const result = await render({
+ entry: o.entry,
+ export: o.export,
+ select: o.select,
+ workers: o.plan && workers === 1 ? "auto" : workers,
+ maxWorkers: o.maxWorkers ? Number(o.maxWorkers) : undefined,
+ codec,
+ quality: o.quality,
+ dryRun: o.plan,
+ outputPath: o.plan ? undefined : outputPath,
+ signal: controller.signal,
+ onProgress: narrate,
+ onWarning: (m) => log(`warning: ${m}`),
+ });
+ if (o.json) {
+ log(JSON.stringify(result));
+ return 0;
+ }
+ if (result.status === "planned") {
+ log(
+ `Render it: framefields render ${o.entry} --cloud --workers ${o.workers ?? "auto"}`,
+ );
+ return 0;
+ }
+ const bytes = result.output?.bytes;
+ log(
+ `Done in ${duration((Date.now() - started) / 1000)}: ${outputPath}${bytes ? ` (${(bytes / 1e6).toFixed(1)} MB)` : ""}, ${Math.round(result.gpuSeconds)} GPU seconds, ${usd(result.costUsd)}.`,
+ );
+ return 0;
+ } catch (err) {
+ if (err instanceof CloudRenderError) {
+ log(err.message);
+ if (err.status && err.status.costUsd > 0)
+ log(`Charged ${usd(err.status.costUsd)} for the GPU time used.`);
+ if (err.code === "insufficient_credits")
+ log("Top up at https://framefields.dev/dashboard/billing");
+ return 1;
+ }
+ throw err;
+ } finally {
+ process.removeListener("SIGINT", onSigint);
+ }
+}
+
+function parseWorkers(value: string | undefined): number | "auto" | undefined {
+ if (value === undefined) return 1;
+ if (value === "auto") return "auto";
+ const n = Number(value);
+ return Number.isInteger(n) && n >= 1 ? n : undefined;
+}
+
+/** Turns status polls into log lines: one per phase, throttled frame progress. */
+export function narrator(
+ log: (line: string) => void,
+ now: () => number,
+): (s: CloudRenderStatus) => void {
+ let phase: string | undefined;
+ let planned = false;
+ let lastProgress = 0;
+ return (s) => {
+ if (!phase) log(`Cloud render ${s.id} started.`);
+ if (s.plan && !planned) {
+ planned = true;
+ log(describePlan(s));
+ }
+ if (s.status !== phase) {
+ phase = s.status;
+ const line = phaseLine(s);
+ if (line) log(line);
+ lastProgress = now();
+ return;
+ }
+ if (s.status === "rendering" && now() - lastProgress >= PROGRESS_EVERY_MS) {
+ lastProgress = now();
+ log(progressLine(s));
+ }
+ };
+}
+
+export function describePlan(s: CloudRenderStatus): string {
+ const plan = s.plan;
+ if (!plan) return "";
+ const est =
+ s.estimateUsd !== undefined ? `, estimated ${usd(s.estimateUsd)}` : "";
+ return `Plan: ${plan.workers} ${plan.workers === 1 ? "worker" : "workers"}, ${duration(plan.workSec)} of render work, ready in about ${duration(plan.etaSec)}${est}.`;
+}
+
+function phaseLine(s: CloudRenderStatus): string | undefined {
+ switch (s.status) {
+ case "probing":
+ return "Measuring frame costs on one worker to plan the split…";
+ case "rendering":
+ return s.workers > 1
+ ? `Rendering ${s.workers} chunks and the audio in parallel…`
+ : "Rendering on one worker…";
+ case "stitching":
+ return `Joining ${s.workers} chunks…`;
+ default:
+ return undefined;
+ }
+}
+
+export function progressLine(s: CloudRenderStatus): string {
+ const { frames, totalFrames, etaSec } = s.progress;
+ const share = totalFrames
+ ? `${Math.floor((frames / totalFrames) * 100)}% (${frames}/${totalFrames} frames)`
+ : `${frames} frames`;
+ const eta = etaSec ? `, about ${duration(etaSec)} left` : "";
+ return `Rendering ${share}${eta}, ${usd(s.costUsd)} so far.`;
+}
+
+export function duration(sec: number): string {
+ const s = Math.max(0, Math.round(sec));
+ if (s < 90) return `${s} s`;
+ const m = Math.round(s / 60);
+ if (m < 90) return `${m} min`;
+ return `${Math.floor(m / 60)} h ${m % 60} min`;
+}
+
+export function usd(value: number): string {
+ return value > 0 && value < 0.01 ? "<$0.01" : `$${value.toFixed(2)}`;
+}
diff --git a/packages/framefields/src/cli/init.test.ts b/packages/framefields/src/cli/init.test.ts
index 35df5f82..1644e082 100644
--- a/packages/framefields/src/cli/init.test.ts
+++ b/packages/framefields/src/cli/init.test.ts
@@ -127,6 +127,7 @@ describe("framefields init", () => {
"grid",
"preview",
"render",
+ "render:cloud",
]);
expect(Object.keys(pkg.devDependencies).sort()).toEqual([
"@types/node",
diff --git a/packages/framefields/src/cli/templates.ts b/packages/framefields/src/cli/templates.ts
index 1d2c5853..744d3d75 100644
--- a/packages/framefields/src/cli/templates.ts
+++ b/packages/framefields/src/cli/templates.ts
@@ -34,6 +34,7 @@ export const SCRIPTS: Record<string, string> = {
grid: "tsx src/render.ts grid",
preview: "tsx src/render.ts preview",
render: "tsx src/render.ts",
+ "render:cloud": "framefields render src/film.ts --cloud --workers auto",
};
/**
@@ -250,6 +251,7 @@ ${run("frames")} -- 0 30 60 # PNGs in output/
${run("grid")} # a contact sheet of the motion
${run("preview")} # play it in the browser
${run("render")} # output/output.mp4
+${run("render:cloud")} # on cloud GPUs, from the pushed commit
\`\`\`
- \`src/film.ts\` builds the composition, \`src/theme.ts\` holds sizes, colors and fonts,
@@ -258,6 +260,9 @@ ${run("render")} # output/output.mp4
one into \`src/components/\`.
- Optional: the AI gateway generates images, video, music, voice and 3D from code. Copy
\`.env.example\` to \`.env\` and set \`FRAMEFIELDS_API_KEY\` (keys at ${AI_DASHBOARD}).
+- \`render:cloud\` renders on framefields' GPUs with the same key, and splits a long film
+ across several (\`-- --plan\` shows workers, time and cost first). It renders the
+ commit you pushed, not uncommitted changes.
- Footage, audio and fonts live in \`assets/\` and are tracked with Git LFS
(\`git lfs install\` once per machine, \`git lfs pull\` after cloning).
`;
diff --git a/packages/framefields/src/cloud/cloud.test.ts b/packages/framefields/src/cloud/cloud.test.ts
new file mode 100644
index 00000000..f1e387d5
--- /dev/null
+++ b/packages/framefields/src/cloud/cloud.test.ts
@@ -0,0 +1,366 @@
+import { execFileSync } from "node:child_process";
+import fs from "node:fs/promises";
+import http from "node:http";
+import type { AddressInfo } from "node:net";
+import os from "node:os";
+import path from "node:path";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { duration, narrator, progressLine, usd } from "../cli/cloud.js";
+import {
+ CloudRenderError,
+ type CloudRenderStatus,
+ renderCloud,
+ resolveGitSource,
+ toHttps,
+} from "./index.js";
+
+describe("toHttps", () => {
+ it("turns git remotes into URLs a cloud worker can fetch", () => {
+ expect(toHttps("[email protected]:acme/films.git")).toBe(
+ "https://github.com/acme/films.git",
+ );
+ expect(toHttps("ssh://[email protected]/acme/films.git")).toBe(
+ "https://gitlab.com/acme/films.git",
+ );
+ expect(toHttps("https://[email protected]/acme/films.git")).toBe(
+ "https://github.com/acme/films.git",
+ );
+ expect(toHttps("https://github.com/acme/films")).toBe(
+ "https://github.com/acme/films",
+ );
+ });
+});
+
+describe("resolveGitSource", () => {
+ let dir: string;
+ let work: string;
+ const git = (cwd: string, ...args: string[]) =>
+ execFileSync("git", args, {
+ cwd,
+ encoding: "utf8",
+ stdio: ["ignore", "pipe", "pipe"],
+ }).trim();
+
+ beforeEach(async () => {
+ dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-cloud-git-"));
+ const remote = path.join(dir, "remote.git");
+ work = path.join(dir, "work");
+ git(dir, "init", "--bare", "-b", "main", remote);
+ git(dir, "clone", remote, work);
+ for (const [k, v] of [
+ ["user.email", "[email protected]"],
+ ["user.name", "t"],
+ ["commit.gpgsign", "false"],
+ ])
+ git(work, "config", k, v);
+ await fs.mkdir(path.join(work, "films/launch/src"), { recursive: true });
+ await fs.writeFile(
+ path.join(work, "films/launch/src/film.ts"),
+ "export const buildFilm = () => null;\n",
+ );
+ git(work, "add", ".");
+ git(work, "commit", "-m", "film");
+ git(work, "push", "-u", "origin", "main");
+ git(work, "remote", "set-url", "origin", "[email protected]:acme/films.git");
+ });
+ afterEach(() => fs.rm(dir, { recursive: true, force: true }));
+
+ it("reads repo, branch, pushed commit and the project folder", () => {
+ const project = path.join(work, "films/launch");
+ const sha = git(work, "rev-parse", "HEAD");
+ expect(resolveGitSource(project, "src/film.ts")).toEqual({
+ source: {
+ repo: "https://github.com/acme/films.git",
+ ref: "main",
+ commit: sha,
+ workdir: "films/launch",
+ },
+ entry: "src/film.ts",
+ warnings: [],
+ });
+ });
+
+ it("warns that uncommitted changes are not rendered", async () => {
+ await fs.writeFile(
+ path.join(work, "films/launch/src/film.ts"),
+ "export const buildFilm = () => 1;\n",
+ );
+ const { warnings } = resolveGitSource(
+ path.join(work, "films/launch"),
+ "src/film.ts",
+ );
+ expect(warnings[0]).toMatch(/Uncommitted changes are not rendered/);
+ });
+
+ it("refuses a commit that is not pushed", async () => {
+ await fs.writeFile(path.join(work, "x.txt"), "x");
+ git(work, "add", ".");
+ git(work, "commit", "-m", "local only");
+ expect(() => resolveGitSource(work, "films/launch/src/film.ts")).toThrow(
+ /is not on origin\. Push it first/,
+ );
+ });
+
+ it("refuses an entry outside the project", () => {
+ expect(() =>
+ resolveGitSource(path.join(work, "films/launch"), "../other.ts"),
+ ).toThrow(/outside/);
+ });
+});
+
+describe("renderCloud", () => {
+ let server: http.Server;
+ let url: string;
+ let requests: { method: string; path: string; body?: unknown }[];
+ /** Statuses the fake API returns, one per GET. */
+ let script: Partial<Record<string, unknown>>[];
+
+ const status = (over: Record<string, unknown>) => ({
+ id: "r1",
+ status: "rendering",
+ workers: 3,
+ progress: { frames: 0, total_frames: 3000, eta_sec: null },
+ cost_usd: 0,
+ gpu_seconds: 0,
+ ...over,
+ });
+
+ beforeEach(async () => {
+ requests = [];
+ script = [];
+ server = http.createServer(async (req, res) => {
+ let raw = "";
+ for await (const chunk of req) raw += chunk;
+ requests.push({
+ method: req.method ?? "",
+ path: req.url ?? "",
+ ...(raw && { body: JSON.parse(raw) }),
+ });
+ if (req.url === "/files/render.mp4") {
+ res.end("video-bytes");
+ return;
+ }
+ if (req.headers.authorization !== "Bearer ff_test_key") {
+ res.writeHead(401).end(
+ JSON.stringify({
+ error: { type: "invalid_api_key", message: "API key is invalid" },
+ }),
+ );
+ return;
+ }
+ const body =
+ req.method === "POST"
+ ? status({ status: "probing", workers: 1 })
+ : req.method === "DELETE"
+ ? status({ status: "cancelled", cost_usd: 0.12 })
+ : (script.shift() ?? status({}));
+ res
+ .writeHead(req.method === "POST" ? 202 : 200, {
+ "content-type": "application/json",
+ })
+ .end(JSON.stringify(body));
+ });
+ await new Promise<void>((r) => server.listen(0, "127.0.0.1", r));
+ url = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
+ });
+ afterEach(() => new Promise<void>((r) => server.close(() => r())));
+
+ const source = {
+ repo: "https://github.com/acme/films",
+ commit: "abc1234",
+ workdir: ".",
+ };
+
+ it("submits, polls until done and downloads the video", async () => {
+ const out = path.join(
+ await fs.mkdtemp(path.join(os.tmpdir(), "ff-cloud-")),
+ "out/film.mp4",
+ );
+ script = [
+ status({
+ plan: { workers: 3, work_sec: 1800, eta_sec: 700, chunks: [] },
+ estimate_usd: 1.9,
+ }),
+ status({ status: "stitching" }),
+ status({
+ status: "completed",
+ gpu_seconds: 2100,
+ cost_usd: 1.86,
+ output: { url: `${url}/files/render.mp4`, bytes: 11 },
+ }),
+ ];
+ const seen: CloudRenderStatus[] = [];
+ const result = await renderCloud({
+ entry: "src/film.ts",
+ workers: "auto",
+ source,
+ outputPath: out,
+ apiKey: "ff_test_key",
+ apiUrl: url,
+ pollMs: 1,
+ onProgress: (s) => seen.push(s),
+ });
+ expect(requests[0]).toEqual({
+ method: "POST",
+ path: "/v1/render",
+ body: {
+ source,
+ target: { entry: "src/film.ts", export: "buildFilm" },
+ render: { codec: "h264" },
+ workers: "auto",
+ dry_run: false,
+ },
+ });
+ expect(seen.map((s) => s.status)).toEqual([
+ "probing",
+ "rendering",
+ "stitching",
+ "completed",
+ ]);
+ expect(seen[1].plan).toEqual({
+ workers: 3,
+ workSec: 1800,
+ etaSec: 700,
+ chunks: [],
+ });
+ expect(result).toMatchObject({
+ status: "completed",
+ costUsd: 1.86,
+ gpuSeconds: 2100,
+ filePath: out,
+ });
+ expect(await fs.readFile(out, "utf8")).toBe("video-bytes");
+ });
+
+ it("throws with the render's error when it fails", async () => {
+ script = [
+ status({
+ status: "failed",
+ error: {
+ code: "ASSET_NOT_FOUND",
+ message: "chunk-1: missing logo.png",
+ },
+ cost_usd: 0.3,
+ }),
+ ];
+ const failed = renderCloud({
+ entry: "src/film.ts",
+ source,
+ apiKey: "ff_test_key",
+ apiUrl: url,
+ pollMs: 1,
+ });
+ await expect(failed).rejects.toThrow(
+ "Cloud render failed: chunk-1: missing logo.png (ASSET_NOT_FOUND)",
+ );
+ await expect(failed).rejects.toBeInstanceOf(CloudRenderError);
+ });
+
+ it("cancels the cloud render when the signal aborts", async () => {
+ const controller = new AbortController();
+ const render = renderCloud({
+ entry: "src/film.ts",
+ source,
+ apiKey: "ff_test_key",
+ apiUrl: url,
+ pollMs: 60_000,
+ signal: controller.signal,
+ });
+ setTimeout(() => controller.abort(new Error("stop")), 50);
+ const err = (await render.catch((e) => e)) as CloudRenderError;
+ expect(err.code).toBe("cancelled");
+ expect(err.status?.costUsd).toBe(0.12);
+ expect(requests.at(-1)).toMatchObject({
+ method: "DELETE",
+ path: "/v1/render/r1",
+ });
+ });
+
+ it("reports the API's refusal", async () => {
+ await expect(
+ renderCloud({
+ entry: "src/film.ts",
+ source,
+ apiKey: "ff_test_bad",
+ apiUrl: url,
+ }),
+ ).rejects.toMatchObject({
+ code: "invalid_api_key",
+ message: "API key is invalid",
+ });
+ });
+
+ it("needs an API key", async () => {
+ const saved = process.env.FRAMEFIELDS_API_KEY;
+ delete process.env.FRAMEFIELDS_API_KEY;
+ try {
+ await expect(
+ renderCloud({ entry: "src/film.ts", source, apiUrl: url }),
+ ).rejects.toThrow(/FRAMEFIELDS_API_KEY/);
+ } finally {
+ if (saved !== undefined) process.env.FRAMEFIELDS_API_KEY = saved;
+ }
+ });
+});
+
+describe("render --cloud output", () => {
+ const s = (over: Partial<CloudRenderStatus>): CloudRenderStatus => ({
+ id: "r1",
+ status: "rendering",
+ workers: 10,
+ progress: { frames: 0, totalFrames: 324_000, etaSec: null },
+ costUsd: 0,
+ gpuSeconds: 0,
+ ...over,
+ });
+
+ it("prints each phase once and throttles frame progress", () => {
+ const lines: string[] = [];
+ let clock = 0;
+ const narrate = narrator(
+ (l) => lines.push(l),
+ () => clock,
+ );
+ narrate(s({ status: "probing", workers: 1 }));
+ narrate(s({ status: "probing", workers: 1 }));
+ narrate(
+ s({
+ plan: { workers: 10, workSec: 10_800, etaSec: 1170, chunks: [] },
+ estimateUsd: 4.2,
+ }),
+ );
+ clock = 5_000;
+ narrate(
+ s({ progress: { frames: 1000, totalFrames: 324_000, etaSec: 1100 } }),
+ );
+ clock = 12_000;
+ narrate(
+ s({
+ progress: { frames: 162_000, totalFrames: 324_000, etaSec: 540 },
+ costUsd: 2.1,
+ }),
+ );
+ narrate(s({ status: "stitching" }));
+ expect(lines).toEqual([
+ "Cloud render r1 started.",
+ "Measuring frame costs on one worker to plan the split…",
+ "Plan: 10 workers, 3 h 0 min of render work, ready in about 20 min, estimated $4.20.",
+ "Rendering 10 chunks and the audio in parallel…",
+ "Rendering 50% (162000/324000 frames), about 9 min left, $2.10 so far.",
+ "Joining 10 chunks…",
+ ]);
+ });
+
+ it("formats durations and money", () => {
+ expect(duration(40)).toBe("40 s");
+ expect(duration(14 * 60)).toBe("14 min");
+ expect(duration(6120)).toBe("1 h 42 min");
+ expect(usd(0.004)).toBe("<$0.01");
+ expect(usd(1.861)).toBe("$1.86");
+ expect(
+ progressLine(
+ s({ progress: { frames: 5, totalFrames: null, etaSec: null } }),
+ ),
+ ).toBe("Rendering 5 frames, $0.00 so far.");
+ });
+});
diff --git a/packages/framefields/src/cloud/git-source.ts b/packages/framefields/src/cloud/git-source.ts
new file mode 100644
index 00000000..222d9e3c
--- /dev/null
+++ b/packages/framefields/src/cloud/git-source.ts
@@ -0,0 +1,116 @@
+/**
+ * Where a cloud render gets the project from: the git repo the working
+ * directory is in, at the commit checked out now. Cloud workers fetch from
+ * the remote, so that commit has to be pushed; uncommitted changes are not
+ * rendered.
+ */
+import { execFileSync } from "node:child_process";
+import { realpathSync } from "node:fs";
+import path from "node:path";
+
+export interface CloudSource {
+ repo: string;
+ ref?: string;
+ commit?: string;
+ /** The project's folder inside the repo (monorepos), `.` at the root. */
+ workdir?: string;
+}
+
+export interface ResolvedSource {
+ source: CloudSource;
+ /** The entry relative to `source.workdir`, as the worker loads it. */
+ entry: string;
+ /** Things that render differently than the user may expect (uncommitted changes). */
+ warnings: string[];
+}
+
+/** Reads repo, branch, commit and folder from git, and checks the commit is pushed. */
+export function resolveGitSource(cwd: string, entry: string): ResolvedSource {
+ const git = (...args: string[]) =>
+ execFileSync("git", args, {
+ cwd,
+ encoding: "utf8",
+ stdio: ["ignore", "pipe", "pipe"],
+ }).trim();
+ let root: string;
+ try {
+ root = git("rev-parse", "--show-toplevel");
+ } catch {
+ throw new Error(
+ `${cwd} is not in a git repository. Cloud renders fetch the project from git: commit it and push it to a remote first.`,
+ );
+ }
+ // git reports the root with symlinks resolved (macOS /var → /private/var).
+ const workdir =
+ path
+ .relative(realpathSync(root), realpathSync(cwd))
+ .split(path.sep)
+ .join("/") || ".";
+ const relEntry = path
+ .relative(cwd, path.resolve(cwd, entry))
+ .split(path.sep)
+ .join("/");
+ if (relEntry.startsWith("../") || path.isAbsolute(relEntry))
+ throw new Error(
+ `${entry} is outside ${cwd}; run from the project folder that contains it.`,
+ );
+
+ const branch = git("rev-parse", "--abbrev-ref", "HEAD");
+ const commit = git("rev-parse", "HEAD");
+ const remoteName = upstreamRemote(git, branch) ?? "origin";
+ let remoteUrl: string;
+ try {
+ remoteUrl = git("remote", "get-url", remoteName);
+ } catch {
+ throw new Error(
+ `The repository has no "${remoteName}" remote. Push the project to a git host first.`,
+ );
+ }
+ const pushed = git("branch", "-r", "--contains", commit);
+ if (!pushed)
+ throw new Error(
+ `Commit ${commit.slice(0, 7)} is not on ${remoteName}. Push it first (\`git push\`): cloud workers fetch the project from the remote.`,
+ );
+ const dirty = git("status", "--porcelain", "--untracked-files=no", "--", ".");
+ return {
+ source: {
+ repo: toHttps(remoteUrl),
+ ...(branch !== "HEAD" && { ref: branch }),
+ commit,
+ workdir,
+ },
+ entry: relEntry,
+ warnings: dirty
+ ? [
+ `Uncommitted changes are not rendered: the cloud renders commit ${commit.slice(0, 7)}.`,
+ ]
+ : [],
+ };
+}
+
+function upstreamRemote(
+ git: (...args: string[]) => string,
+ branch: string,
+): string | undefined {
+ if (branch === "HEAD") return undefined;
+ try {
+ return git("config", `branch.${branch}.remote`) || undefined;
+ } catch {
+ return undefined;
+ }
+}
+
+/** `[email protected]:acme/films.git` → `https://github.com/acme/films.git`; credentials dropped. */
+export function toHttps(url: string): string {
+ const scp = /^[\w.-]+@([^:/]+):(.+)$/.exec(url);
+ if (scp) return `https://${scp[1]}/${scp[2]}`;
+ const parsed = URL.canParse(url) ? new URL(url) : undefined;
+ if (parsed && (parsed.protocol === "ssh:" || parsed.protocol === "git:"))
+ return `https://${parsed.hostname}${parsed.pathname}`;
+ if (parsed?.protocol === "https:" || parsed?.protocol === "http:") {
+ parsed.username = "";
+ parsed.password = "";
+ return parsed.toString();
+ }
+ return url;
+}
diff --git a/packages/framefields/src/cloud/index.ts b/packages/framefields/src/cloud/index.ts
new file mode 100644
index 00000000..7acb084d
--- /dev/null
+++ b/packages/framefields/src/cloud/index.ts
@@ -0,0 +1,320 @@
+/**
+ * Cloud renders on api.framefields.dev: the project renders from git on
+ * framefields' GPU workers, billed to the account that owns the API key.
+ *
+ * ```ts
+ * import { renderCloud } from "framefields/cloud";
+ *
+ * const result = await renderCloud({
+ * entry: "src/film.ts",
+ * workers: "auto", // split a long film across GPUs; 1 (default) renders on one
+ * outputPath: "output/film.mp4",
+ * onProgress: (s) => console.log(s.status, s.progress.frames),
+ * });
+ * ```
+ *
+ * `workers: "auto"` probes the film on one worker first, then splits it into
+ * as many chunks as stay efficient: short films stay on one worker, long ones
+ * spread out up to `maxWorkers`. `dryRun: true` returns that plan with the
+ * estimated cost and renders nothing. Aborting `signal` cancels the render;
+ * GPU time already used is charged.
+ */
+import { createWriteStream } from "node:fs";
+import fs from "node:fs/promises";
+import path from "node:path";
+import { Readable } from "node:stream";
+import { pipeline } from "node:stream/promises";
+import type { ReadableStream as WebReadableStream } from "node:stream/web";
+import { type CloudSource, resolveGitSource } from "./git-source.js";
+
+export { type CloudSource, resolveGitSource, toHttps } from "./git-source.js";
+
+export const FRAMEFIELDS_API_URL = "https://api.framefields.dev";
+
+export interface CloudRenderOptions {
+ /** The module that builds the composition, relative to `cwd`. */
+ entry: string;
+ /** Its export: a Composition, or a function returning one. Default `buildFilm`. */
+ export?: string;
+ /** Passed to the export when it is a function. */
+ args?: unknown;
+ /** When the export returns several compositions, the one to render. */
+ select?: string;
+ /** 1 (default) renders on one worker; a number or "auto" splits the film. */
+ workers?: number | "auto";
+ /** Upper bound for "auto" (the API caps it too). */
+ maxWorkers?: number;
+ codec?: "h264" | "vp9";
+ quality?: string;
+ /** Probe and plan only: returns the plan and estimated cost, renders nothing. */
+ dryRun?: boolean;
+ /** Download the finished video here. */
+ outputPath?: string;
+ /** Project folder. Default `process.cwd()`. */
+ cwd?: string;
+ /** Instead of reading repo, branch and commit from git. */
+ source?: CloudSource;
+ /** Default `process.env.FRAMEFIELDS_API_KEY`. */
+ apiKey?: string;
+ /** Default `process.env.FRAMEFIELDS_API_URL`, else https://api.framefields.dev. */
+ apiUrl?: string;
+ /** Called with every status poll. */
+ onProgress?: (status: CloudRenderStatus) => void;
+ /** Called for each warning (uncommitted changes). Default: stderr. */
+ onWarning?: (message: string) => void;
+ /** Aborting cancels the cloud render. */
+ signal?: AbortSignal;
+ /** Status poll interval, ms. Default 3000. */
+ pollMs?: number;
+}
+
+export type CloudRenderPhase =
+ | "probing"
+ | "holding"
+ | "rendering"
+ | "stitching"
+ | "planned"
+ | "completed"
+ | "failed"
+ | "cancelled";
+
+export interface CloudRenderPlan {
+ workers: number;
+ /** Render time of the whole film on one GPU, seconds. */
+ workSec: number;
+ /** Expected wall time with this plan, seconds. */
+ etaSec: number;
+ chunks: { start: number; end: number; workSec: number }[];
+}
+
+export interface CloudRenderStatus {
+ id: string;
+ status: CloudRenderPhase;
+ workers: number;
+ plan?: CloudRenderPlan;
+ progress: {
+ frames: number;
+ totalFrames: number | null;
+ etaSec: number | null;
+ };
+ /** Charged so far (final once the render ended), USD. */
+ costUsd: number;
+ gpuSeconds: number;
+ /** The planned render's estimated charge, USD. */
+ estimateUsd?: number;
+ /** The commit rendered. */
+ sha?: string;
+ output?: { url: string; bytes?: number };
+ error?: { code: string; message: string };
+}
+
+export interface CloudRenderResult extends CloudRenderStatus {
+ /** Where the video was downloaded (`outputPath`). */
+ filePath?: string;
+}
+
+/** A cloud render that failed, was cancelled, or was refused by the API. */
+export class CloudRenderError extends Error {
+ constructor(
+ message: string,
+ readonly code: string,
+ readonly status?: CloudRenderStatus,
+ ) {
+ super(message);
+ this.name = "CloudRenderError";
+ }
+}
+
+const DONE = new Set<CloudRenderPhase>([
+ "planned",
+ "completed",
+ "failed",
+ "cancelled",
+]);
+
+/** Starts a cloud render and waits for it; resolves when it completed (or was planned). */
+export async function renderCloud(
+ options: CloudRenderOptions,
+): Promise<CloudRenderResult> {
+ const api = cloudClient(options);
+ const cwd = options.cwd ?? process.cwd();
+ const resolved = options.source
+ ? { source: options.source, entry: options.entry, warnings: [] }
+ : resolveGitSource(cwd, options.entry);
+ const warn =
+ options.onWarning ??
+ ((m: string) => process.stderr.write(`framefields: ${m}\n`));
+ for (const w of resolved.warnings) warn(w);
+
+ let status = await api.start({
+ source: resolved.source,
+ target: {
+ entry: resolved.entry,
+ export: options.export ?? "buildFilm",
+ ...(options.args !== undefined && { args: options.args }),
+ ...(options.select && { select: [options.select] }),
+ },
+ render: {
+ codec: options.codec ?? "h264",
+ ...(options.quality && { quality: options.quality }),
+ },
+ workers: options.workers ?? 1,
+ ...(options.maxWorkers && { max_workers: options.maxWorkers }),
+ dry_run: options.dryRun ?? false,
+ });
+ options.onProgress?.(status);
+
+ try {
+ while (!DONE.has(status.status)) {
+ await sleep(options.pollMs ?? 3000, options.signal);
+ status = await api.status(status.id);
+ options.onProgress?.(status);
+ }
+ } catch (err) {
+ if (options.signal?.aborted) {
+ const final = await api.cancel(status.id).catch(() => status);
+ throw new CloudRenderError(
+ "The cloud render was cancelled.",
+ "cancelled",
+ final,
+ );
+ }
+ throw err;
+ }
+
+ if (status.status === "failed" || status.status === "cancelled")
+ throw new CloudRenderError(
+ status.error
+ ? `Cloud render ${status.status}: ${status.error.message} (${status.error.code})`
+ : `Cloud render ${status.status}.`,
+ status.error?.code ?? status.status,
+ status,
+ );
+ if (status.status === "completed" && options.outputPath && status.output) {
+ await download(status.output.url, options.outputPath, options.signal);
+ return { ...status, filePath: options.outputPath };
+ }
+ return status;
+}
+
+/** The API calls behind `renderCloud`, for callers who track renders themselves. */
+export function cloudClient(
+ options: Pick<CloudRenderOptions, "apiKey" | "apiUrl">,
+) {
+ const key = options.apiKey ?? process.env.FRAMEFIELDS_API_KEY;
+ if (!key)
+ throw new CloudRenderError(
+ "No API key. Set FRAMEFIELDS_API_KEY (keys at https://framefields.dev/dashboard/keys) or pass apiKey.",
+ "invalid_api_key",
+ );
+ const base = (
+ options.apiUrl ??
+ process.env.FRAMEFIELDS_API_URL ??
+ FRAMEFIELDS_API_URL
+ ).replace(/\/+$/, "");
+ const call = async (method: string, route: string, body?: unknown) => {
+ const res = await fetch(`${base}/v1/render${route}`, {
+ method,
+ headers: {
+ Authorization: `Bearer ${key}`,
+ "Content-Type": "application/json",
+ },
+ ...(body !== undefined && { body: JSON.stringify(body) }),
+ });
+ const json = (await res.json().catch(() => ({}))) as ApiStatus & {
+ error?: { type?: string; message?: string };
+ };
+ if (!res.ok)
+ throw new CloudRenderError(
+ json.error?.message ??
+ `api.framefields.dev answered HTTP ${res.status}`,
+ json.error?.type ?? `http_${res.status}`,
+ );
+ return fromApi(json);
+ };
+ return {
+ start: (body: unknown) => call("POST", "", body),
+ status: (id: string) => call("GET", `/${encodeURIComponent(id)}`),
+ cancel: (id: string) => call("DELETE", `/${encodeURIComponent(id)}`),
+ };
+}
+
+/** The API's snake_case status. */
+interface ApiStatus {
+ id: string;
+ status: CloudRenderPhase;
+ workers?: number;
+ plan?: {
+ workers: number;
+ work_sec: number;
+ eta_sec: number;
+ chunks: CloudRenderPlan["chunks"];
+ };
+ progress?: {
+ frames?: number;
+ total_frames?: number | null;
+ eta_sec?: number | null;
+ };
+ cost_usd?: number;
+ gpu_seconds?: number;
+ estimate_usd?: number;
+ sha?: string;
+ output?: { url: string; bytes?: number };
+ error?: { code: string; message: string };
+}
+
+function fromApi(s: ApiStatus): CloudRenderStatus {
+ return {
+ id: s.id,
+ status: s.status,
+ workers: s.workers ?? 1,
+ ...(s.plan && {
+ plan: {
+ workers: s.plan.workers,
+ workSec: s.plan.work_sec,
+ etaSec: s.plan.eta_sec,
+ chunks: s.plan.chunks,
+ },
+ }),
+ progress: {
+ frames: s.progress?.frames ?? 0,
+ totalFrames: s.progress?.total_frames ?? null,
+ etaSec: s.progress?.eta_sec ?? null,
+ },
+ costUsd: s.cost_usd ?? 0,
+ gpuSeconds: s.gpu_seconds ?? 0,
+ ...(s.estimate_usd !== undefined && { estimateUsd: s.estimate_usd }),
+ ...(s.sha && { sha: s.sha }),
+ ...(s.output && { output: s.output }),
+ ...(s.error && { error: s.error }),
+ };
+}
+
+async function download(url: string, file: string, signal?: AbortSignal) {
+ const res = await fetch(url, { signal });
+ if (!res.ok || !res.body)
+ throw new CloudRenderError(
+ `Downloading the render failed: HTTP ${res.status}`,
+ "download_failed",
+ );
+ await fs.mkdir(path.dirname(path.resolve(file)), { recursive: true });
+ await pipeline(
+ Readable.fromWeb(res.body as WebReadableStream),
+ createWriteStream(file),
+ );
+}
+
+function sleep(ms: number, signal?: AbortSignal): Promise<void> {
+ return new Promise((resolve, reject) => {
+ if (signal?.aborted) return reject(signal.reason);
+ const timer = setTimeout(resolve, ms);
+ signal?.addEventListener(
+ "abort",
+ () => {
+ clearTimeout(timer);
+ reject(signal.reason);
+ },
+ { once: true },
+ );
+ });
+}
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 1d70687a..f10ce004 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -12,6 +12,8 @@ export {
ValidationError,
} from "@framefields/ai";
export {
+ type AudioPitchOptions,
+ type AudioPitchSignals,
AudioSignal,
type AudioSignalOptions,
computed,
@@ -1179,6 +1181,19 @@ export interface RenderVideoOptions {
* (GIF only) and "done". The engine logs the same phases on its own.
*/
onProgress?: (progress: RenderProgress) => void;
+ /**
+ * Renders only frames `start` (inclusive) to `end` (exclusive): one chunk
+ * of a render split across workers. Chunks start on a keyframe with
+ * timestamps from 0; join them with `stitchChunks` from
+ * `framefields/renderer`. Effects that carry state between frames
+ * (trails, particles, datamosh) rebuild it before `start`.
+ */
+ frames?: { start: number; end: number };
+ /**
+ * `false` writes no audio track, as for chunks: render the timeline's
+ * audio once with `codec: "aac"` and pass it to `stitchChunks`.
+ */
+ audio?: boolean;
}
export interface RenderVideoResult {
@@ -3511,8 +3526,11 @@ export {
} from "./preview/index.js";
export {
type AudioQaStats,
+ type ChunkPlan,
+ type ChunkPlanOptions,
computeGridLayout,
describeGpu,
+ type FrameCostSample,
type FrameGridOptions,
type FrameGridRenderable,
type FrameSamplePoint,
@@ -3520,6 +3538,8 @@ export {
type GpuAdapterDescription,
HeadlessMediaRenderer,
HeadlessWebGPURenderer,
+ type PlannedChunk,
+ planChunks,
type QaIssue,
type QaSegment,
type RenderProgress,
@@ -3527,6 +3547,9 @@ export {
renderSemaphore,
resetRenderCaches,
resolveSamplePoints,
+ type StitchOptions,
+ type StitchResult,
+ stitchChunks,
type VideoQaOptions,
type VideoQaReport,
type VideoQaStats,
diff --git a/packages/framefields/src/render-chunks.test.ts b/packages/framefields/src/render-chunks.test.ts
new file mode 100644
index 00000000..7ddb2b59
--- /dev/null
+++ b/packages/framefields/src/render-chunks.test.ts
@@ -0,0 +1,245 @@
+/**
+ * A render split into chunks (`renderVideo({ frames, audio: false })`) and
+ * joined by `stitchChunks` matches the same render done in one go: every
+ * frame, including the first one after a seam (where particles and trails
+ * rebuild their state), and the soundtrack.
+ */
+import { execFileSync } from "node:child_process";
+import fs from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import {
+ ALL_FORMATS,
+ EncodedPacketSink,
+ FilePathSource,
+ Input,
+} from "mediabunny";
+import { describe, expect, it } from "vitest";
+import {
+ Composition,
+ encodeStereoWav,
+ Layer,
+ LayerAnimation,
+ Trails,
+} from "./index.js";
+import { stitchChunks } from "./renderer/index.js";
+
+const hasFfmpeg = (() => {
+ try {
+ execFileSync("ffmpeg", ["-version"], { stdio: "ignore" });
+ return true;
+ } catch {
+ return false;
+ }
+})();
+
+const W = 160;
+const H = 96;
+const FPS = 30;
+const FRAMES = 30;
+const SEAM = 13;
+
+function tone(seconds: number, sampleRate: number): Float32Array {
+ const out = new Float32Array(Math.round(seconds * sampleRate));
+ for (let i = 0; i < out.length; i++)
+ out[i] = 0.3 * Math.sin((2 * Math.PI * 440 * i) / sampleRate);
+ return out;
+}
+
+async function film(dir: string, width = W): Promise<Composition> {
+ const wav = path.join(dir, "tone.wav");
+ const source = tone(FRAMES / FPS, 48_000);
+ await fs.writeFile(
+ wav,
+ encodeStereoWav([source, source], { sampleRate: 48_000 }),
+ );
+ const comp = new Composition({
+ width,
+ height: H,
+ fps: FPS,
+ durationFrames: FRAMES,
+ backgroundColor: "#000000",
+ });
+ comp.add(
+ Layer.particles({
+ width,
+ height: H,
+ max: 1000,
+ emitter: { shape: "point", x: width / 2, y: H - 8, rate: 300 },
+ direction: -90,
+ spread: 30,
+ speed: [60, 90],
+ lifetime: [2, 2],
+ size: [4, 4],
+ color: "#ffffff",
+ }),
+ );
+ comp.add(
+ Layer.shape("rect", {
+ fillColor: "#ff4040",
+ width: 20,
+ height: 20,
+ x: 0,
+ y: 10,
+ })
+ .animate(
+ LayerAnimation.create().fromTo("x", 0, width - 20, {
+ start: 0,
+ end: FRAMES,
+ }),
+ )
+ .withEffect(new Trails({ decay: 0.8 })),
+ );
+ comp.add(Layer.audio(wav, { volume: 1, durationFrames: FRAMES }));
+ return comp;
+}
+
+function decodeFrames(file: string): Buffer[] {
+ const raw = execFileSync(
+ "ffmpeg",
+ [
+ "-v",
+ "error",
+ "-i",
+ file,
+ "-f",
+ "rawvideo",
+ "-pix_fmt",
+ "rgb24",
+ "pipe:1",
+ ],
+ { maxBuffer: 1 << 28 },
+ );
+ const size = W * H * 3;
+ return Array.from({ length: raw.length / size }, (_, i) =>
+ raw.subarray(i * size, (i + 1) * size),
+ );
+}
+
+function decodeAudio(file: string): Float32Array {
+ const raw = execFileSync(
+ "ffmpeg",
+ ["-v", "error", "-i", file, "-vn", "-ac", "1", "-f", "f32le", "pipe:1"],
+ { maxBuffer: 1 << 28 },
+ );
+ return new Float32Array(raw.buffer, raw.byteOffset, raw.byteLength / 4);
+}
+
+function meanAbsDiff(a: Buffer, b: Buffer): number {
+ let sum = 0;
+ for (let i = 0; i < a.length; i++) sum += Math.abs(a[i] - b[i]);
+ return sum / a.length;
+}
+
+async function packetCount(
+ file: string,
+): Promise<{ video: number; audio: number }> {
+ const input = new Input({
+ source: new FilePathSource(file),
+ formats: ALL_FORMATS,
+ });
+ try {
+ const count = async (
+ track: Parameters<typeof EncodedPacketSink>[0] | null,
+ ) => {
+ let n = 0;
+ if (track)
+ for await (const _ of new EncodedPacketSink(track).packets()) n++;
+ return n;
+ };
+ return {
+ video: await count(await input.getPrimaryVideoTrack()),
+ audio: await count(await input.getPrimaryAudioTrack()),
+ };
+ } finally {
+ input.dispose();
+ }
+}
+
+describe("chunked render + stitchChunks", () => {
+ it.skipIf(!hasFfmpeg)(
+ "matches the render done in one go, frame for frame and in audio",
+ async () => {
+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-chunks-"));
+ const comp = await film(dir);
+ const full = path.join(dir, "full.mp4");
+ await comp.renderVideo({ outputPath: full });
+
+ const chunks = [path.join(dir, "a.mp4"), path.join(dir, "b.mp4")];
+ // Each chunk on its own composition, as a worker builds it.
+ await (await film(dir)).renderVideo({
+ outputPath: chunks[0],
+ frames: { start: 0, end: SEAM },
+ audio: false,
+ });
+ await (await film(dir)).renderVideo({
+ outputPath: chunks[1],
+ frames: { start: SEAM, end: FRAMES },
+ audio: false,
+ });
+ const audio = path.join(dir, "audio.mp4");
+ await (await film(dir)).renderVideo({ outputPath: audio, codec: "aac" });
+
+ expect(await packetCount(chunks[0])).toEqual({ video: SEAM, audio: 0 });
+ expect((await packetCount(chunks[1])).video).toBe(FRAMES - SEAM);
+
+ const stitched = path.join(dir, "stitched.mp4");
+ const result = await stitchChunks({
+ chunks,
+ audio,
+ outputPath: stitched,
+ });
+ expect(result.frames).toBe(FRAMES);
+ expect(result.durationSec).toBeCloseTo(FRAMES / FPS, 3);
+ expect((await packetCount(stitched)).video).toBe(FRAMES);
+
+ const want = decodeFrames(full);
+ const got = decodeFrames(stitched);
+ expect(got).toHaveLength(FRAMES);
+ // The scene really moves: consecutive frames differ.
+ expect(meanAbsDiff(want[SEAM - 1], want[SEAM])).toBeGreaterThan(1);
+ // Same pictures; a chunk's first frame is a fresh keyframe, so allow
+ // for the codec, not for wrong content.
+ for (let f = 0; f < FRAMES; f++)
+ expect(meanAbsDiff(got[f], want[f]), `frame ${f}`).toBeLessThan(0.5);
+
+ const a = decodeAudio(full);
+ const b = decodeAudio(stitched);
+ expect(b.length).toBe(a.length);
+ expect(meanAbsDiff(Buffer.from(a.buffer), Buffer.from(b.buffer))).toBe(0);
+ },
+ 120_000,
+ );
+
+ it("rejects a frame range outside the composition", async () => {
+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-chunks-"));
+ const comp = await film(dir);
+ await expect(
+ comp.renderVideo({ frames: { start: 10, end: FRAMES + 1 } }),
+ ).rejects.toThrow(/Invalid frames/);
+ await expect(
+ comp.renderVideo({ frames: { start: 5, end: 5 } }),
+ ).rejects.toThrow(/Invalid frames/);
+ });
+
+ it("refuses to stitch chunks encoded differently", async () => {
+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-chunks-"));
+ const a = path.join(dir, "a.mp4");
+ const b = path.join(dir, "b.mp4");
+ await (await film(dir)).renderVideo({
+ outputPath: a,
+ frames: { start: 0, end: 4 },
+ audio: false,
+ });
+ await (await film(dir, W + 32)).renderVideo({
+ outputPath: b,
+ frames: { start: 4, end: 8 },
+ audio: false,
+ });
+ const out = path.join(dir, "out.mp4");
+ await expect(
+ stitchChunks({ chunks: [a, b], outputPath: out }),
+ ).rejects.toThrow(/size/);
+ await expect(fs.stat(out)).rejects.toThrow();
+ }, 60_000);
+});
diff --git a/packages/framefields/src/renderer/index.ts b/packages/framefields/src/renderer/index.ts
index c333bb1d..379cfdcf 100644
--- a/packages/framefields/src/renderer/index.ts
+++ b/packages/framefields/src/renderer/index.ts
@@ -1,13 +1,22 @@
export {
type AudioQaStats,
+ type ChunkPlan,
+ type ChunkPlanOptions,
+ type FrameCostSample,
+ type FrameRange,
formatQaReport,
HeadlessMediaRenderer,
HeadlessWebGPURenderer,
+ type PlannedChunk,
+ planChunks,
type QaIssue,
type QaSegment,
type RenderProgress,
renderSemaphore,
resetRenderCaches,
+ type StitchOptions,
+ type StitchResult,
+ stitchChunks,
type VideoQaOptions,
type VideoQaReport,
type VideoQaStats,
diff --git a/packages/framefields/src/signals/audio.ts b/packages/framefields/src/signals/audio.ts
index d5391d4b..49891d09 100644
--- a/packages/framefields/src/signals/audio.ts
+++ b/packages/framefields/src/signals/audio.ts
@@ -5,9 +5,11 @@ import {
type GPUTextureBinding,
} from "@framefields/core";
import {
+ type AudioPitchTrackConfig,
type AudioSignalComputeConfig,
AudioSignalComputePipeline,
ensureDevice,
+ trackAudioPitch,
} from "@framefields/webgpu-renderers";
export interface AudioSignalOptions extends AudioSignalComputeConfig {
@@ -33,6 +35,32 @@ export interface ExtractedAudioSignalsBundle {
seekFrame(frame: number): void;
}
+export interface AudioPitchOptions extends AudioPitchTrackConfig {
+ fps?: number;
+}
+
+/** Fundamental frequency per frame, e.g. an engine note (RPM) or a melody line. */
+export interface AudioPitchSignals {
+ /** Pitch in Hz; interpolated across unvoiced gaps so it never drops to 0 mid-track. */
+ hz: FrameSignal<number>;
+ /** 0..1 periodicity of the measured frame, 0 where unvoiced. */
+ confidence: FrameSignal<number>;
+ /** 1 where a pitch was measured, 0 where `hz` is interpolated. */
+ voiced: FrameSignal<number>;
+ samples: { hz: Float32Array; confidence: Float32Array; voiced: Float32Array };
+ fps: number;
+ numFrames: number;
+ durationSec: number;
+}
+
+async function decodeOrThrow(source: string, purpose: string) {
+ const decoded = await decodeAudioSource(source);
+ if (!decoded || decoded.channels.length === 0) {
+ throw new Error(`Failed to decode audio source for ${purpose}: ${source}`);
+ }
+ return decoded;
+}
+
export const AudioSignal = {
async extractFromChannels(
channels: Float32Array[],
@@ -124,16 +152,54 @@ export const AudioSignal = {
source: string,
options: AudioSignalOptions = {},
): Promise<ExtractedAudioSignalsBundle> {
- const decoded = await decodeAudioSource(source);
- if (!decoded || decoded.channels.length === 0) {
- throw new Error(
- `Failed to decode audio source for feature extraction: ${source}`,
- );
- }
+ const decoded = await decodeOrThrow(source, "feature extraction");
return AudioSignal.extractFromChannels(
decoded.channels,
decoded.sampleRate,
options,
);
},
+
+ /** Tracks the fundamental frequency of the downmixed channels on the GPU. */
+ async pitchFromChannels(
+ channels: Float32Array[],
+ sampleRate: number,
+ options: AudioPitchOptions = {},
+ ): Promise<AudioPitchSignals> {
+ const device = await ensureDevice();
+ const fps = options.fps ?? 24;
+ const track = await trackAudioPitch(
+ device,
+ channels,
+ sampleRate,
+ fps,
+ options,
+ );
+ return {
+ hz: frameArraySignal(track.hz, fps),
+ confidence: frameArraySignal(track.confidence, fps),
+ voiced: frameArraySignal(track.voiced, fps),
+ samples: {
+ hz: track.hz,
+ confidence: track.confidence,
+ voiced: track.voiced,
+ },
+ fps,
+ numFrames: track.numFrames,
+ durationSec: track.durationSec,
+ };
+ },
+
+ /** Decodes `source` (audio or video file) and tracks its pitch. */
+ async pitch(
+ source: string,
+ options: AudioPitchOptions = {},
+ ): Promise<AudioPitchSignals> {
+ const decoded = await decodeOrThrow(source, "pitch tracking");
+ return AudioSignal.pitchFromChannels(
+ decoded.channels,
+ decoded.sampleRate,
+ options,
+ );
+ },
};
diff --git a/packages/framefields/src/signals/index.ts b/packages/framefields/src/signals/index.ts
index b0388b9a..783d0284 100644
--- a/packages/framefields/src/signals/index.ts
+++ b/packages/framefields/src/signals/index.ts
@@ -24,6 +24,8 @@ export {
signalRegistry,
} from "@framefields/webgpu-renderers";
export {
+ type AudioPitchOptions,
+ type AudioPitchSignals,
AudioSignal,
type AudioSignalOptions,
type ExtractedAudioSignalsBundle,
diff --git a/packages/framefields/tsdown.config.ts b/packages/framefields/tsdown.config.ts
index c11c3374..68c0716a 100644
--- a/packages/framefields/tsdown.config.ts
+++ b/packages/framefields/tsdown.config.ts
@@ -15,6 +15,7 @@ export default defineConfig({
"src/preview/index.ts",
"src/components/index.ts",
"src/sam/index.ts",
+ "src/cloud/index.ts",
"src/cli.ts",
"src/cli/check-worker.ts",
],
diff --git a/packages/node-sdk/CHANGELOG.md b/packages/node-sdk/CHANGELOG.md
index d1b86f0b..55f6a438 100644
--- a/packages/node-sdk/CHANGELOG.md
+++ b/packages/node-sdk/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/node-sdk
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/node-sdk/package.json b/packages/node-sdk/package.json
index fbd66eb1..deab4791 100644
--- a/packages/node-sdk/package.json
+++ b/packages/node-sdk/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/node-sdk",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"publishConfig": {
"access": "public"
diff --git a/packages/renderer/CHANGELOG.md b/packages/renderer/CHANGELOG.md
index bcb6fd04..5852411f 100644
--- a/packages/renderer/CHANGELOG.md
+++ b/packages/renderer/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/renderer
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/renderer/package.json b/packages/renderer/package.json
index 393c345a..f851b764 100644
--- a/packages/renderer/package.json
+++ b/packages/renderer/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/renderer",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"type": "module",
"sideEffects": false,
diff --git a/packages/renderer/src/chunk-plan.ts b/packages/renderer/src/chunk-plan.ts
new file mode 100644
index 00000000..3c00cece
--- /dev/null
+++ b/packages/renderer/src/chunk-plan.ts
@@ -0,0 +1,174 @@
+/**
+ * Plans a render split across workers: how many chunks, and where each one
+ * starts and ends. Pure arithmetic over measured frame costs, so it runs
+ * anywhere (a worker's probe job, a coordinator, a local script).
+ *
+ * Every chunk pays a fixed cost before its first frame (worker start, fetch,
+ * install, composition build). `workers: "auto"` adds workers only while each
+ * chunk still spends at least `efficiency` of its time rendering, so a short
+ * film stays on one worker and a long one spreads out until `maxWorkers`.
+ * Chunks are cut at equal render cost, not equal frame counts, so a heavy
+ * stretch (3D, vision) gets shorter chunks.
+ */
+
+/** A measured render time for one frame. */
+export interface FrameCostSample {
+ frame: number;
+ ms: number;
+}
+
+export interface ChunkPlanOptions {
+ /** Frames in the composition. */
+ totalFrames: number;
+ /** Measured frame times; between samples the cost is interpolated linearly. */
+ samples: readonly FrameCostSample[];
+ /** A worker count, or `"auto"` to pick one from the cost. Default 1. */
+ workers?: number | "auto";
+ /** Upper bound for `"auto"` (and for a number). Default 10. */
+ maxWorkers?: number;
+ /** Fixed cost each chunk pays before rendering, in seconds. Default 90 (a cold start). */
+ overheadSec?: number;
+ /**
+ * `"auto"` only: the smallest share of a chunk's time spent rendering,
+ * between 0 and 1. Default 0.85: each chunk renders at least
+ * `overheadSec × 0.85 / 0.15` (about 5.7× the overhead).
+ */
+ efficiency?: number;
+ /** No chunk is shorter than this many frames. Default 60. */
+ minChunkFrames?: number;
+}
+
+export interface PlannedChunk {
+ /** First frame (inclusive). */
+ start: number;
+ /** Last frame (exclusive). */
+ end: number;
+ /** Estimated render time of its frames, in seconds. */
+ workSec: number;
+}
+
+export interface ChunkPlan {
+ workers: number;
+ /** Estimated render time of the whole composition on one worker, without overhead. */
+ workSec: number;
+ /** Estimated wall time with this plan: overhead plus the longest chunk. */
+ etaSec: number;
+ chunks: PlannedChunk[];
+}
+
+export function planChunks(options: ChunkPlanOptions): ChunkPlan {
+ const total = options.totalFrames;
+ if (!Number.isInteger(total) || total < 1)
+ throw new Error(
+ `planChunks: totalFrames must be a positive integer, got ${total}.`,
+ );
+ const overheadSec = options.overheadSec ?? 90;
+ const maxWorkers = Math.max(1, Math.floor(options.maxWorkers ?? 10));
+ const minChunkFrames = Math.max(1, Math.floor(options.minChunkFrames ?? 60));
+ const cost = frameCosts(total, options.samples);
+ const workSec = cost.reduce((sum, ms) => sum + ms, 0) / 1000;
+
+ const wanted = workerCount(
+ options.workers ?? 1,
+ workSec,
+ overheadSec,
+ options.efficiency ?? 0.85,
+ );
+ const workers = Math.max(
+ 1,
+ Math.min(wanted, maxWorkers, Math.floor(total / minChunkFrames)),
+ );
+ const chunks = splitByCost(cost, workers, minChunkFrames);
+ const longest = Math.max(...chunks.map((c) => c.workSec));
+ return {
+ workers: chunks.length,
+ workSec,
+ etaSec: overheadSec + longest,
+ chunks,
+ };
+}
+
+function workerCount(
+ workers: number | "auto",
+ workSec: number,
+ overheadSec: number,
+ efficiency: number,
+): number {
+ if (workers !== "auto") {
+ if (!Number.isInteger(workers) || workers < 1)
+ throw new Error(
+ `planChunks: workers must be a positive integer or "auto", got ${workers}.`,
+ );
+ return workers;
+ }
+ if (!(efficiency > 0 && efficiency < 1))
+ throw new Error(
+ `planChunks: efficiency must be between 0 and 1, got ${efficiency}.`,
+ );
+ const minChunkSec = (overheadSec * efficiency) / (1 - efficiency);
+ return minChunkSec > 0
+ ? Math.max(1, Math.floor(workSec / minChunkSec))
+ : Number.POSITIVE_INFINITY;
+}
+
+/** Per-frame cost in ms: linear between samples, flat beyond the first and last. */
+function frameCosts(
+ total: number,
+ samples: readonly FrameCostSample[],
+): Float64Array {
+ const sorted = samples
+ .filter(
+ (s) => Number.isFinite(s.frame) && Number.isFinite(s.ms) && s.ms >= 0,
+ )
+ .toSorted((a, b) => a.frame - b.frame);
+ if (sorted.length === 0)
+ throw new Error("planChunks: needs at least one frame cost sample.");
+ const cost = new Float64Array(total);
+ let s = 0;
+ for (let f = 0; f < total; f++) {
+ while (s + 1 < sorted.length && sorted[s + 1].frame <= f) s++;
+ const a = sorted[s];
+ const b = sorted[s + 1];
+ cost[f] =
+ !b || f <= a.frame
+ ? a.ms
+ : a.ms + ((b.ms - a.ms) * (f - a.frame)) / (b.frame - a.frame);
+ }
+ return cost;
+}
+
+/**
+ * Cuts the frames into `count` contiguous chunks of about equal total cost,
+ * none shorter than `minFrames` (`count × minFrames` must fit).
+ */
+function splitByCost(
+ cost: Float64Array,
+ count: number,
+ minFrames: number,
+): PlannedChunk[] {
+ const total = cost.reduce((sum, ms) => sum + ms, 0);
+ const chunks: PlannedChunk[] = [];
+ let start = 0;
+ let acc = 0;
+ let chunkMs = 0;
+ for (let f = 0; f < cost.length; f++) {
+ acc += cost[f];
+ chunkMs += cost[f];
+ const cut = (total * (chunks.length + 1)) / count;
+ const framesLeft = cost.length - (f + 1);
+ const chunksLeft = count - chunks.length - 1;
+ const long = f + 1 - start >= minFrames;
+ const roomAfter = framesLeft >= minFrames * chunksLeft;
+ if (
+ chunksLeft > 0 &&
+ long &&
+ ((acc >= cut && roomAfter) || framesLeft === minFrames * chunksLeft)
+ ) {
+ chunks.push({ start, end: f + 1, workSec: chunkMs / 1000 });
+ start = f + 1;
+ chunkMs = 0;
+ }
+ }
+ chunks.push({ start, end: cost.length, workSec: chunkMs / 1000 });
+ return chunks;
+}
diff --git a/packages/renderer/src/headless-webgpu-renderer.ts b/packages/renderer/src/headless-webgpu-renderer.ts
index f7c0a979..6a79219b 100644
--- a/packages/renderer/src/headless-webgpu-renderer.ts
+++ b/packages/renderer/src/headless-webgpu-renderer.ts
@@ -68,6 +68,7 @@ import { type RenderProgress, RenderReporter } from "./render-reporter.js";
);
};
+import type { FrameCostSample } from "./chunk-plan.js";
import {
type AudioQaStats,
analyzeAudio,
@@ -359,6 +360,112 @@ async function fileSize(file: string): Promise<number | undefined> {
}
}
+/** Composition frames `start` (inclusive) to `end` (exclusive). */
+export interface FrameRange {
+ start: number;
+ end: number;
+}
+
+/** The frames a render covers, checked against the composition's length. */
+function resolveFrameRange(
+ frames: FrameRange | undefined,
+ total: number,
+): FrameRange & { total: number } {
+ if (!frames) return { start: 0, end: total, total };
+ const { start, end } = frames;
+ if (
+ !Number.isInteger(start) ||
+ !Number.isInteger(end) ||
+ start < 0 ||
+ end <= start ||
+ end > total
+ )
+ throw new Error(
+ `Invalid frames { start: ${start}, end: ${end} }: need integers with 0 <= start < end <= ${total} (the composition's frame count).`,
+ );
+ return { start, end, total };
+}
+
+/**
+ * Mixes the composition's whole audio timeline and encodes it into `source`
+ * in 100 ms samples. Returns its QA stats when `qaOptions` is set.
+ */
+async function encodeAudioTrack(job: {
+ virtualMedia: VirtualMediaData;
+ fps: number;
+ device: GPUDevice;
+ renderId: string;
+ source: AudioSampleSource;
+ codec: string;
+ reporter: RenderReporter;
+ qaOptions?: VideoQaOptions;
+}): Promise<AudioQaStats | undefined> {
+ const { reporter } = job;
+ reporter.stage("audio", "Mixing audio");
+ const { channels, sampleRate } = await mixAudioTracks(
+ job.virtualMedia,
+ job.fps,
+ 48_000,
+ job.device,
+ job.renderId,
+ {
+ onProgress: ({ done, total, source }) =>
+ reporter.update(
+ total > 0
+ ? `${done}/${total} clips${source ? `: ${source}` : ""}`
+ : "no audio clips",
+ total > 0 ? done / total : undefined,
+ ),
+ },
+ );
+
+ const numChannels = channels.length;
+ if (numChannels === 0) throw new Error("mixAudioTracks returned no channels");
+
+ const totalSamples = channels[0].length;
+ const chunkSize = Math.round(0.1 * sampleRate);
+ reporter.stage(
+ "audio-encode",
+ `Encoding ${(totalSamples / sampleRate).toFixed(1)}s of audio (${job.codec})`,
+ );
+
+ for (let offset = 0; offset < totalSamples; offset += chunkSize) {
+ const currentChunk = Math.min(chunkSize, totalSamples - offset);
+ // A fresh buffer per chunk: the encoder may read a sample's data after
+ // add() resolves, so a buffer reused across chunks gets overwritten by
+ // the next chunk before it is encoded (audible as 100 ms garbling).
+ const interleaved = new Float32Array(currentChunk * numChannels);
+
+ for (let i = 0; i < currentChunk; i++) {
+ for (let ch = 0; ch < numChannels; ch++) {
+ interleaved[i * numChannels + ch] = channels[ch]?.[offset + i] ?? 0;
+ }
+ }
+
+ const sample = new AudioSample({
+ data: interleaved,
+ format: "f32",
+ numberOfChannels: numChannels,
+ sampleRate,
+ timestamp: offset / sampleRate,
+ });
+ try {
+ await job.source.add(sample);
+ } finally {
+ sample.close();
+ }
+ const written = offset + currentChunk;
+ reporter.update(
+ `${(written / sampleRate).toFixed(1)}s/${(totalSamples / sampleRate).toFixed(1)}s`,
+ written / totalSamples,
+ );
+ }
+
+ return job.qaOptions
+ ? analyzeAudio(channels, sampleRate, job.qaOptions)
+ : undefined;
+}
+
function toEvenDimension(n: number): number {
const v = Math.round(n);
return v % 2 === 0 ? v : v + 1;
@@ -794,6 +901,71 @@ export class HeadlessWebGPURenderer {
});
}
+ /**
+ * Measures what frames cost to render across the timeline, for planning a
+ * split render (`planChunks`). At `samples` evenly spaced points (default
+ * 12) it renders three consecutive frames and times the last two, so
+ * effects that carry state (trails, particles) are caught up and their
+ * replay is not counted. Encoding is not included.
+ */
+ public async probeRender(
+ source: VirtualMediaData | { toVirtualMedia(): VirtualMediaData },
+ options: {
+ samples?: number;
+ signal?: AbortSignal;
+ /** Called after each sample point with how many are done. */
+ onSample?: (done: number, total: number) => void;
+ } = {},
+ ): Promise<{
+ totalFrames: number;
+ fps: number;
+ durationMs: number;
+ samples: FrameCostSample[];
+ }> {
+ const vm =
+ "toVirtualMedia" in source
+ ? (source as { toVirtualMedia(): VirtualMediaData }).toVirtualMedia()
+ : source;
+ const fps = vm.metadata.fps;
+ if (!fps) throw new Error("probeRender: the composition has no fps.");
+ let durationMs = vm.metadata.durationMs;
+ if (!durationMs || durationMs <= 0) {
+ durationMs = await this.resolveMediaDuration(vm, fps);
+ vm.metadata.durationMs = durationMs;
+ }
+ const totalFrames = frameCount(durationMs, fps);
+ const count = Math.max(
+ 1,
+ Math.min(Math.floor(options.samples ?? 12), totalFrames),
+ );
+ const last = Math.max(0, totalFrames - 3);
+ const renderId = `probe-${randomUUID()}`;
+ const render = (frame: number) =>
+ this.renderImage(vm, Math.min(frame, totalFrames - 1), fps, {
+ renderId,
+ format: "rgba",
+ });
+ const samples: FrameCostSample[] = [];
+ try {
+ for (let i = 0; i < count; i++) {
+ options.signal?.throwIfAborted();
+ const at = count === 1 ? 0 : Math.round((i * last) / (count - 1));
+ await render(at);
+ const started = performance.now();
+ await render(at + 1);
+ await render(at + 2);
+ samples.push({
+ frame: Math.min(at + 1, totalFrames - 1),
+ ms: (performance.now() - started) / 2,
+ });
+ options.onSample?.(i + 1, count);
+ }
+ } finally {
+ mediaDecoderCache.clearNode(renderId);
+ }
+ return { totalFrames, fps, durationMs, samples };
+ }
+
/**
* Mixes the composition's full audio timeline (clips, effects, synthesized
* tracks) to stereo PCM, exactly as `renderVideo` muxes it.
@@ -1060,6 +1232,19 @@ export class HeadlessWebGPURenderer {
* "mux", "convert" (GIF only) and "done".
*/
onProgress?: (progress: RenderProgress) => void;
+ /**
+ * Renders only composition frames `start` (inclusive) to `end`
+ * (exclusive), as one chunk of a render split across workers. The
+ * chunk's timestamps start at 0 and its first frame is a keyframe,
+ * so `stitchChunks` joins chunks without re-encoding. Effects that
+ * carry state between frames rebuild it before `start`.
+ */
+ frames?: FrameRange;
+ /**
+ * `false` writes no audio track, as for chunks: the timeline's audio
+ * is exported once with `codec: "aac"` and added by `stitchChunks`.
+ */
+ audio?: boolean;
},
): Promise<{
filePath: string;
@@ -1110,7 +1295,15 @@ export class HeadlessWebGPURenderer {
durationMs = await this.resolveMediaDuration(virtualMedia, fps);
virtualMedia.metadata.durationMs = durationMs;
}
- const totalFrames = frameCount(durationMs, fps);
+ const range = resolveFrameRange(
+ options?.frames,
+ frameCount(durationMs, fps),
+ );
+ if (isAudioOnly && (options?.frames || options?.audio === false))
+ throw new Error(
+ `An audio-only render (codec "${codecOption}") takes neither \`frames\` nor \`audio: false\`.`,
+ );
+ const totalFrames = range.end - range.start;
const renderId = `vid-${randomUUID()}`;
const qaOptions: VideoQaOptions | undefined =
@@ -1232,11 +1425,14 @@ export class HeadlessWebGPURenderer {
} as any);
output.addVideoTrack(videoSource);
}
- const audioSource = new AudioSampleSource({
- codec: audioCodec,
- bitrate: quality,
- });
- output.addAudioTrack(audioSource);
+ let audioSource: AudioSampleSource | undefined;
+ if (options?.audio !== false) {
+ audioSource = new AudioSampleSource({
+ codec: audioCodec,
+ bitrate: quality,
+ });
+ output.addAudioTrack(audioSource);
+ }
let outputStarted = false;
let outputFinalized = false;
@@ -1309,18 +1505,25 @@ export class HeadlessWebGPURenderer {
device.queue.submit([preEncoder.finish()]);
await lutStore.awaitAllPending(device);
};
- await prePass(0);
-
- reporter.stage("render", `Rendering ${totalFrames} frames`);
+ updateClockSignals(range.start, fps, durationMs);
+ await prePass(range.start);
+
+ reporter.stage(
+ "render",
+ range.start === 0 && totalFrames === range.total
+ ? `Rendering ${totalFrames} frames`
+ : `Rendering frames ${range.start}–${range.end - 1} of ${range.total}`,
+ );
// Render and encode frames using pipelined double-buffered DMA staging ring
- for (let i = 0; i < totalFrames; i++) {
+ for (let i = range.start; i < range.end; i++) {
options?.signal?.throwIfAborted();
if (ctx && surface && renderer && pipeline) {
// Assets first used on a later frame (a LUT, a glyph) get the
- // same pre-pass as frame 0, as renderImage and the preview do.
+ // same pre-pass as the first frame, as renderImage and the
+ // preview do.
if (
- i > 0 &&
+ i > range.start &&
(lutStore.hasPending(device) || SlugFontCache.hasPending())
) {
updateClockSignals(i, fps, durationMs);
@@ -1366,8 +1569,10 @@ export class HeadlessWebGPURenderer {
},
);
- await pipeline.enqueueFrame(i, targetTexture, encoder);
- if (i + 1 < totalFrames) reporter.frameDone(i + 1);
+ // Chunk-local index: the pipeline timestamps frames from it.
+ const done = i - range.start + 1;
+ await pipeline.enqueueFrame(done - 1, targetTexture, encoder);
+ if (done < totalFrames) reporter.frameDone(done);
} else {
throw new Error("Invalid render context state");
}
@@ -1396,68 +1601,18 @@ export class HeadlessWebGPURenderer {
}
// ── Audio ──────────────────────────────────────────────────────────────
- options?.signal?.throwIfAborted();
- reporter.stage("audio", "Mixing audio");
- const { channels, sampleRate } = await mixAudioTracks(
- virtualMedia,
- fps,
- 48_000,
- device,
- renderId,
- {
- onProgress: ({ done, total, source }) =>
- reporter.update(
- total > 0
- ? `${done}/${total} clips${source ? `: ${source}` : ""}`
- : "no audio clips",
- total > 0 ? done / total : undefined,
- ),
- },
- );
-
- const numChannels = channels.length;
- if (numChannels === 0)
- throw new Error("mixAudioTracks returned no channels");
-
- if (qaOptions) audioQa = analyzeAudio(channels, sampleRate, qaOptions);
-
- const totalSamples = channels[0].length;
- const chunkSize = Math.round(0.1 * sampleRate);
- reporter.stage(
- "audio-encode",
- `Encoding ${(totalSamples / sampleRate).toFixed(1)}s of audio (${audioCodec})`,
- );
-
- for (let offset = 0; offset < totalSamples; offset += chunkSize) {
- const currentChunk = Math.min(chunkSize, totalSamples - offset);
- // A fresh buffer per chunk: the encoder may read a sample's data after
- // add() resolves, so a buffer reused across chunks gets overwritten by
- // the next chunk before it is encoded (audible as 100 ms garbling).
- const interleaved = new Float32Array(currentChunk * numChannels);
-
- for (let i = 0; i < currentChunk; i++) {
- for (let ch = 0; ch < numChannels; ch++) {
- interleaved[i * numChannels + ch] = channels[ch]?.[offset + i] ?? 0;
- }
- }
-
- const sample = new AudioSample({
- data: interleaved,
- format: "f32",
- numberOfChannels: numChannels,
- sampleRate,
- timestamp: offset / sampleRate,
+ if (audioSource) {
+ options?.signal?.throwIfAborted();
+ audioQa = await encodeAudioTrack({
+ virtualMedia,
+ fps,
+ device,
+ renderId,
+ source: audioSource,
+ codec: audioCodec,
+ reporter,
+ qaOptions,
});
- try {
- await audioSource.add(sample);
- } finally {
- sample.close();
- }
- const written = offset + currentChunk;
- reporter.update(
- `${(written / sampleRate).toFixed(1)}s/${(totalSamples / sampleRate).toFixed(1)}s`,
- written / totalSamples,
- );
}
options?.signal?.throwIfAborted();
diff --git a/packages/renderer/src/index.ts b/packages/renderer/src/index.ts
index 44c33604..44188693 100644
--- a/packages/renderer/src/index.ts
+++ b/packages/renderer/src/index.ts
@@ -1,4 +1,5 @@
export * from "./asset-preloader.js";
+export * from "./chunk-plan.js";
export * from "./dma-staging-ring.js";
export * from "./dynamic-node-discovery.js";
export * from "./headless-webgpu-renderer.js";
@@ -6,5 +7,6 @@ export * from "./job-reset.js";
export * from "./logger.js";
export * from "./node-manifest.js";
export type { RenderPhase, RenderProgress } from "./render-reporter.js";
+export * from "./stitch.js";
export * from "./video-qa.js";
export * from "./zero-copy-webcodecs-pipeline.js";
diff --git a/packages/renderer/src/stitch.ts b/packages/renderer/src/stitch.ts
new file mode 100644
index 00000000..b975a847
--- /dev/null
+++ b/packages/renderer/src/stitch.ts
@@ -0,0 +1,245 @@
+/**
+ * Joins the chunks of one render split across workers. Each chunk is made
+ * with `renderVideo({ frames, audio: false })`, starts on a keyframe and
+ * timestamps from 0; the stitch copies their encoded frames into one track,
+ * shifted to follow each other, and copies in the audio track of a separate
+ * audio-only render. Nothing is decoded or re-encoded, so it needs no GPU.
+ */
+import fs from "node:fs/promises";
+import path from "node:path";
+import {
+ ALL_FORMATS,
+ EncodedAudioPacketSource,
+ type EncodedPacket,
+ EncodedPacketSink,
+ EncodedVideoPacketSource,
+ FilePathSource,
+ FilePathTarget,
+ Input,
+ type InputAudioTrack,
+ type InputVideoTrack,
+ Mp4OutputFormat,
+ Output,
+ type VideoCodec,
+ WebMOutputFormat,
+} from "mediabunny";
+
+export interface StitchOptions {
+ /** Chunk files in timeline order, each a `renderVideo({ frames, audio: false })` output. */
+ chunks: readonly string[];
+ /**
+ * An audio-only render of the whole timeline (`renderVideo({ codec: "aac" })`,
+ * or `"opus"` for WebM chunks), copied in as the audio track.
+ */
+ audio?: string;
+ outputPath: string;
+ /** Stops between packets; the promise rejects and no file is left behind. */
+ signal?: AbortSignal;
+}
+
+export interface StitchResult {
+ filePath: string;
+ /** Video frames written, across all chunks. */
+ frames: number;
+ /** End of the video track, in seconds. */
+ durationSec: number;
+}
+
+type VideoConfig = NonNullable<
+ Awaited<ReturnType<InputVideoTrack["getDecoderConfig"]>>
+>;
+type AudioConfig = NonNullable<
+ Awaited<ReturnType<InputAudioTrack["getDecoderConfig"]>>
+>;
+
+interface VideoChunk {
+ file: string;
+ input: Input;
+ track: InputVideoTrack;
+ codec: VideoCodec;
+ config: VideoConfig;
+}
+
+export async function stitchChunks(
+ options: StitchOptions,
+): Promise<StitchResult> {
+ if (options.chunks.length === 0)
+ throw new Error("stitchChunks needs at least one chunk.");
+ const inputs: Input[] = [];
+ const open = (file: string) => {
+ const input = new Input({
+ source: new FilePathSource(file),
+ formats: ALL_FORMATS,
+ });
+ inputs.push(input);
+ return input;
+ };
+ let output: Output | undefined;
+ try {
+ const chunks = await Promise.all(
+ options.chunks.map((file) => openVideoChunk(file, open(file))),
+ );
+ const [reference] = chunks as [VideoChunk, ...VideoChunk[]];
+ for (const chunk of chunks.slice(1)) assertSameEncoding(reference, chunk);
+
+ const webm = (await reference.input.getMimeType()).includes("webm");
+ await fs.mkdir(path.dirname(path.resolve(options.outputPath)), {
+ recursive: true,
+ });
+ output = new Output({
+ format: webm ? new WebMOutputFormat() : new Mp4OutputFormat(),
+ target: new FilePathTarget(options.outputPath),
+ });
+ const video = new EncodedVideoPacketSource(reference.codec);
+ output.addVideoTrack(video);
+ const audio = options.audio
+ ? await AudioCopier.open(options.audio, open(options.audio), output)
+ : undefined;
+
+ await output.start();
+ const written = await copyVideo(chunks, video, audio, options.signal);
+ await audio?.writeUntil(Number.POSITIVE_INFINITY, options.signal);
+ await output.finalize();
+ output = undefined;
+ return { filePath: options.outputPath, ...written };
+ } catch (error) {
+ if (output) {
+ await output.cancel().catch(() => {});
+ await fs.rm(options.outputPath, { force: true });
+ }
+ throw error;
+ } finally {
+ for (const input of inputs) input.dispose();
+ }
+}
+
+async function openVideoChunk(file: string, input: Input): Promise<VideoChunk> {
+ const track = await input.getPrimaryVideoTrack();
+ if (!track) throw new Error(`Chunk ${file} has no video track.`);
+ const codec = await track.getCodec();
+ const config = await track.getDecoderConfig();
+ if (!codec || !config)
+ throw new Error(`Chunk ${file}: its video codec cannot be read.`);
+ return { file, input, track, codec, config };
+}
+
+/**
+ * Copied frames are decoded with the first chunk's codec configuration, so
+ * every chunk must have been encoded with the same one.
+ */
+function assertSameEncoding(reference: VideoChunk, chunk: VideoChunk): void {
+ const a = reference.config;
+ const b = chunk.config;
+ const differs =
+ a.codec !== b.codec
+ ? `codec ${b.codec}, not ${a.codec}`
+ : a.codedWidth !== b.codedWidth || a.codedHeight !== b.codedHeight
+ ? `size ${b.codedWidth}x${b.codedHeight}, not ${a.codedWidth}x${a.codedHeight}`
+ : !sameBytes(a.description, b.description)
+ ? "different encoder parameters (SPS/PPS)"
+ : undefined;
+ if (differs)
+ throw new Error(
+ `Chunk ${chunk.file} has ${differs} than ${reference.file}. Render every chunk with the same framefields version, codec and quality, on the same GPU type.`,
+ );
+}
+
+function sameBytes(
+ a: VideoConfig["description"],
+ b: VideoConfig["description"],
+): boolean {
+ if (!a || !b) return a === b;
+ const x = toBytes(a);
+ const y = toBytes(b);
+ return x.length === y.length && x.every((v, i) => v === y[i]);
+}
+
+function toBytes(data: NonNullable<VideoConfig["description"]>): Uint8Array {
+ return ArrayBuffer.isView(data)
+ ? new Uint8Array(data.buffer, data.byteOffset, data.byteLength)
+ : new Uint8Array(data);
+}
+
+/** Writes every chunk's frames, each chunk starting where the last one ended. */
+async function copyVideo(
+ chunks: readonly VideoChunk[],
+ video: EncodedVideoPacketSource,
+ audio: AudioCopier | undefined,
+ signal: AbortSignal | undefined,
+): Promise<{ frames: number; durationSec: number }> {
+ let offset = 0;
+ let frames = 0;
+ for (const chunk of chunks) {
+ let first: number | undefined;
+ let end = offset;
+ for await (const packet of new EncodedPacketSink(chunk.track).packets()) {
+ signal?.throwIfAborted();
+ if (first === undefined) {
+ if (packet.type !== "key")
+ throw new Error(`Chunk ${chunk.file} does not start on a keyframe.`);
+ first = packet.timestamp;
+ }
+ const timestamp = offset + packet.timestamp - first;
+ // Interleaved: audio up to this frame goes in first.
+ await audio?.writeUntil(timestamp, signal);
+ await video.add(
+ packet.clone({ timestamp }),
+ frames === 0 ? { decoderConfig: chunk.config } : undefined,
+ );
+ frames++;
+ end = Math.max(end, timestamp + packet.duration);
+ }
+ if (first === undefined)
+ throw new Error(`Chunk ${chunk.file} has no video frames.`);
+ offset = end;
+ }
+ return { frames, durationSec: offset };
+}
+
+/** Copies an audio track's packets in timestamp order, as far as asked. */
+class AudioCopier {
+ private readonly packets: AsyncGenerator<EncodedPacket, void>;
+ /** The next packet to write; `undefined` until read, `null` at the end. */
+ private next: EncodedPacket | null | undefined;
+ private written = 0;
+
+ private constructor(
+ track: InputAudioTrack,
+ private readonly config: AudioConfig,
+ private readonly source: EncodedAudioPacketSource,
+ ) {
+ this.packets = new EncodedPacketSink(track).packets();
+ }
+
+ static async open(
+ file: string,
+ input: Input,
+ output: Output,
+ ): Promise<AudioCopier> {
+ const track = await input.getPrimaryAudioTrack();
+ if (!track) throw new Error(`Audio file ${file} has no audio track.`);
+ const config = await track.getDecoderConfig();
+ const codec = await track.getCodec();
+ if (!config || !codec)
+ throw new Error(`Audio file ${file}: its codec cannot be read.`);
+ const source = new EncodedAudioPacketSource(codec);
+ output.addAudioTrack(source);
+ return new AudioCopier(track, config, source);
+ }
+
+ async writeUntil(timestamp: number, signal?: AbortSignal): Promise<void> {
+ for (;;) {
+ if (this.next === undefined)
+ this.next = (await this.packets.next()).value ?? null;
+ const packet = this.next;
+ if (!packet || packet.timestamp > timestamp) return;
+ signal?.throwIfAborted();
+ await this.source.add(
+ packet,
+ this.written === 0 ? { decoderConfig: this.config } : undefined,
+ );
+ this.written++;
+ this.next = undefined;
+ }
+ }
+}
diff --git a/packages/renderer/src/tests/chunk-plan.test.ts b/packages/renderer/src/tests/chunk-plan.test.ts
new file mode 100644
index 00000000..917d6205
--- /dev/null
+++ b/packages/renderer/src/tests/chunk-plan.test.ts
@@ -0,0 +1,120 @@
+import { describe, expect, it } from "vitest";
+import { type ChunkPlan, planChunks } from "../chunk-plan.js";
+
+const FPS = 30;
+/** Renders as fast as it plays: 33.3 ms per frame. */
+const realtime = [{ frame: 0, ms: 1000 / FPS }];
+
+function expectCovers(plan: ChunkPlan, total: number) {
+ expect(plan.chunks[0].start).toBe(0);
+ expect(plan.chunks.at(-1)?.end).toBe(total);
+ for (let i = 1; i < plan.chunks.length; i++)
+ expect(plan.chunks[i].start).toBe(plan.chunks[i - 1].end);
+ expect(plan.chunks).toHaveLength(plan.workers);
+}
+
+describe("planChunks", () => {
+ it("keeps one worker by default", () => {
+ const total = 3 * 3600 * FPS;
+ const plan = planChunks({ totalFrames: total, samples: realtime });
+ expect(plan.workers).toBe(1);
+ expectCovers(plan, total);
+ expect(plan.workSec).toBeCloseTo(3 * 3600, 0);
+ });
+
+ it("auto: a five-minute film stays on one worker", () => {
+ const plan = planChunks({
+ totalFrames: 5 * 60 * FPS,
+ samples: realtime,
+ workers: "auto",
+ });
+ expect(plan.workers).toBe(1);
+ });
+
+ it("auto: a three-hour film spreads to maxWorkers", () => {
+ const total = 3 * 3600 * FPS;
+ const plan = planChunks({
+ totalFrames: total,
+ samples: realtime,
+ workers: "auto",
+ });
+ // 10 800 s of work over chunks of at least 510 s would be 21; capped at 10.
+ expect(plan.workers).toBe(10);
+ expectCovers(plan, total);
+ expect(plan.etaSec).toBeCloseTo(90 + 1080, 0);
+ });
+
+ it("auto: adds workers only while each chunk stays efficient", () => {
+ // 30 min of work; 510 s minimum per chunk at 85% → 3 workers.
+ const plan = planChunks({
+ totalFrames: 30 * 60 * FPS,
+ samples: realtime,
+ workers: "auto",
+ maxWorkers: 50,
+ });
+ expect(plan.workers).toBe(3);
+ for (const chunk of plan.chunks)
+ expect(90 / (90 + chunk.workSec)).toBeLessThanOrEqual(0.15 + 1e-9);
+ });
+
+ it("cuts at equal cost, so a heavy stretch gets a shorter chunk", () => {
+ const total = 1000;
+ const plan = planChunks({
+ totalFrames: total,
+ samples: [
+ { frame: 0, ms: 10 },
+ { frame: 499, ms: 10 },
+ { frame: 500, ms: 30 },
+ { frame: 999, ms: 30 },
+ ],
+ workers: 2,
+ minChunkFrames: 1,
+ });
+ expectCovers(plan, total);
+ // Half the cost (10 s of 20 s) is reached at frame ~667.
+ expect(plan.chunks[0].end).toBeGreaterThan(650);
+ expect(plan.chunks[0].end).toBeLessThan(680);
+ expect(plan.chunks[0].workSec).toBeCloseTo(plan.chunks[1].workSec, 0);
+ });
+
+ it("never makes a chunk shorter than minChunkFrames", () => {
+ const plan = planChunks({
+ totalFrames: 250,
+ samples: [
+ { frame: 0, ms: 1000 },
+ { frame: 10, ms: 1 },
+ ],
+ workers: 4,
+ minChunkFrames: 60,
+ });
+ expect(plan.workers).toBe(4);
+ expectCovers(plan, 250);
+ for (const chunk of plan.chunks)
+ expect(chunk.end - chunk.start).toBeGreaterThanOrEqual(60);
+ });
+
+ it("never plans more chunks than frames allow", () => {
+ const plan = planChunks({ totalFrames: 90, samples: realtime, workers: 8 });
+ expect(plan.workers).toBe(1);
+ });
+
+ it("rejects bad input", () => {
+ expect(() => planChunks({ totalFrames: 0, samples: realtime })).toThrow(
+ /totalFrames/,
+ );
+ expect(() => planChunks({ totalFrames: 100, samples: [] })).toThrow(
+ /sample/,
+ );
+ expect(() =>
+ planChunks({ totalFrames: 100, samples: realtime, workers: 0 }),
+ ).toThrow(/workers/);
+ expect(() =>
+ planChunks({
+ totalFrames: 100,
+ samples: realtime,
+ workers: "auto",
+ efficiency: 1,
+ }),
+ ).toThrow(/efficiency/);
+ });
+});
diff --git a/packages/renderers/CHANGELOG.md b/packages/renderers/CHANGELOG.md
index 662b48a4..e3849bdf 100644
--- a/packages/renderers/CHANGELOG.md
+++ b/packages/renderers/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/renderers
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/renderers/package.json b/packages/renderers/package.json
index 3bb46776..eee645a0 100644
--- a/packages/renderers/package.json
+++ b/packages/renderers/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/renderers",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"type": "module",
"sideEffects": false,
diff --git a/packages/tensor-webgpu/CHANGELOG.md b/packages/tensor-webgpu/CHANGELOG.md
index d51b583d..37bcabc7 100644
--- a/packages/tensor-webgpu/CHANGELOG.md
+++ b/packages/tensor-webgpu/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/tensor-webgpu
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/tensor-webgpu/package.json b/packages/tensor-webgpu/package.json
index f2578766..eaf217e7 100644
--- a/packages/tensor-webgpu/package.json
+++ b/packages/tensor-webgpu/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/tensor-webgpu",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"publishConfig": {
"access": "public"
diff --git a/packages/tsconfig/CHANGELOG.md b/packages/tsconfig/CHANGELOG.md
index 2af3965a..3a214e18 100644
--- a/packages/tsconfig/CHANGELOG.md
+++ b/packages/tsconfig/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/tsconfig
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/tsconfig/package.json b/packages/tsconfig/package.json
index 58aaf137..28e7e2d3 100644
--- a/packages/tsconfig/package.json
+++ b/packages/tsconfig/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/tsconfig",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"license": "Apache-2.0",
"publishConfig": {
diff --git a/packages/vision-sam/CHANGELOG.md b/packages/vision-sam/CHANGELOG.md
index fa44b878..8ff755b3 100644
--- a/packages/vision-sam/CHANGELOG.md
+++ b/packages/vision-sam/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/vision-sam
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/vision-sam/package.json b/packages/vision-sam/package.json
index 8bec0a64..7a7e9965 100644
--- a/packages/vision-sam/package.json
+++ b/packages/vision-sam/package.json
@@ -1,7 +1,7 @@
{
"name": "@framefields/vision-sam",
"description": "Prompted video tracking for framefields: SAM 2.1 Tiny / Small and SAM 3 (text selection, several objects). SAM 3's model weights, downloaded at runtime, are under Meta's SAM License.",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": true,
"type": "module",
"sideEffects": false,
diff --git a/packages/vision/CHANGELOG.md b/packages/vision/CHANGELOG.md
index c7281d1e..8157cd40 100644
--- a/packages/vision/CHANGELOG.md
+++ b/packages/vision/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/vision
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/vision/package.json b/packages/vision/package.json
index 1b751324..a62a83f7 100644
--- a/packages/vision/package.json
+++ b/packages/vision/package.json
@@ -1,7 +1,7 @@
{
"name": "@framefields/vision",
"description": "On-device vision for framefields: RTMDet-Ins detection + instance masks, RTMO pose, and Selfie Segmenter mattes over ONNX Runtime.",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"publishConfig": {
"access": "public"
diff --git a/packages/webgpu-renderers/CHANGELOG.md b/packages/webgpu-renderers/CHANGELOG.md
index a215284a..56c5932a 100644
--- a/packages/webgpu-renderers/CHANGELOG.md
+++ b/packages/webgpu-renderers/CHANGELOG.md
@@ -1,5 +1,7 @@
# @framefields/webgpu-renderers
+## 2.11.0
+
## 2.9.3
## 2.9.2
diff --git a/packages/webgpu-renderers/package.json b/packages/webgpu-renderers/package.json
index b2b5839c..749b0a78 100644
--- a/packages/webgpu-renderers/package.json
+++ b/packages/webgpu-renderers/package.json
@@ -1,6 +1,6 @@
{
"name": "@framefields/webgpu-renderers",
- "version": "2.9.3",
+ "version": "2.11.0",
"private": false,
"publishConfig": {
"access": "public"
diff --git a/packages/webgpu-renderers/src/signals/audio-pitch-compute.test.ts b/packages/webgpu-renderers/src/signals/audio-pitch-compute.test.ts
new file mode 100644
index 00000000..72e297e8
--- /dev/null
+++ b/packages/webgpu-renderers/src/signals/audio-pitch-compute.test.ts
@@ -0,0 +1,131 @@
+import { describe, expect, it } from "vitest";
+import { ensureDevice } from "../device.js";
+import {
+ decodePitchTrack,
+ PITCH_CANDIDATES,
+ trackAudioPitch,
+} from "./audio-pitch-compute.js";
+
+const SR = 44100;
+const FPS = 50;
+
+/** Candidates for one frame as [lag, yinValue] pairs, padded with empty slots. */
+function frame(...pairs: [number, number][]): number[] {
+ const out: number[] = [];
+ for (let k = 0; k < PITCH_CANDIDATES; k++) {
+ const [lag, val] = pairs[k] ?? [0, 2];
+ out.push(lag, val);
+ }
+ return out;
+}
+
+/** Pitch of the synthetic engine at time t: two upshift ramps, then a held note. */
+function engineHz(t: number): number {
+ if (t < 1) return 85 + 15 * t;
+ if (t < 2) return 88 + 12 * (t - 1);
+ return 100;
+}
+
+/** Harmonic-rich tone with a weak fundamental and seeded noise; silent after 3 s. */
+function syntheticEngine(seconds: number): Float32Array {
+ const out = new Float32Array(Math.round(seconds * SR));
+ let phase = 0;
+ let seed = 1;
+ const noise = () => {
+ seed = (seed * 1664525 + 1013904223) >>> 0;
+ return seed / 2 ** 32 - 0.5;
+ };
+ for (let i = 0; i < out.length; i++) {
+ const t = i / SR;
+ phase += engineHz(t) / SR;
+ if (t >= 3) continue;
+ let v = 0;
+ for (let h = 1; h <= 12; h++) {
+ const amp = h % 3 === 0 ? 1 : 0.25 / h;
+ v += amp * Math.sin(2 * Math.PI * h * phase);
+ }
+ out[i] = 0.1 * v + 0.05 * noise();
+ }
+ return out;
+}
+
+describe("decodePitchTrack", () => {
+ it("rejects a one-frame octave error and fills unvoiced gaps", () => {
+ const lag100 = SR / 100;
+ const data = new Float32Array([
+ ...frame([lag100, 0.2], [lag100 * 2, 0.25]),
+ // The deepest dip is an octave up; continuity should keep 100 Hz.
+ ...frame([lag100 / 2, 0.15], [lag100, 0.2]),
+ ...frame([lag100, 0.2]),
+ ...frame(),
+ ...frame([SR / 110, 0.1]),
+ ...frame([SR / 110, 0.1]),
+ ]);
+ const track = decodePitchTrack({ data, numFrames: 6, sampleRate: SR }, {});
+ expect(Array.from(track.voiced)).toEqual([1, 1, 1, 0, 1, 1]);
+ expect(track.hz[1]).toBeCloseTo(100, 3);
+ expect(track.hz[3]).toBeCloseTo(105, 3);
+ expect(track.confidence[3]).toBe(0);
+ expect(track.confidence[0]).toBeCloseTo(0.8, 5);
+ });
+
+ it("prefers the period over a slightly deeper double-period echo", () => {
+ // Real engine audio at top speed: 101 Hz dips to 0.41, its 2x echo to 0.38.
+ const lag = SR / 101;
+ const echo = Array.from({ length: 10 }, () =>
+ frame([lag * 2, 0.38], [lag, 0.41], [SR / 138, 0.8]),
+ );
+ const track = decodePitchTrack(
+ { data: new Float32Array(echo.flat()), numFrames: 10, sampleRate: SR },
+ {},
+ );
+ for (const hz of track.hz) expect(hz).toBeCloseTo(101, 3);
+ });
+});
+
+describe("trackAudioPitch", () => {
+ it("tracks a harmonic sweep through a shift and goes unvoiced in silence", async () => {
+ let device: GPUDevice;
+ try {
+ device = await ensureDevice();
+ } catch {
+ console.warn("Skipping due to no WebGPU device");
+ return;
+ }
+
+ const track = await trackAudioPitch(device, [syntheticEngine(4)], SR, FPS, {
+ minHz: 50,
+ maxHz: 1000,
+ });
+ expect(track.numFrames).toBe(4 * FPS);
+
+ let worst = 0;
+ for (let f = 2; f < 3 * FPS - 2; f++) {
+ const t = f / FPS;
+ // Skip the frames whose window straddles the shift discontinuity.
+ if (Math.abs(t - 1) < 0.06) continue;
+ expect(track.voiced[f]).toBe(1);
+ worst = Math.max(worst, Math.abs(track.hz[f] / engineHz(t) - 1));
+ }
+ expect(worst).toBeLessThan(0.01);
+
+ for (let f = 3.2 * FPS; f < 4 * FPS; f++) {
+ expect(track.voiced[f]).toBe(0);
+ expect(track.hz[f]).toBeCloseTo(track.hz[3 * FPS - 3], 0);
+ }
+ });
+
+ it("refuses a minHz whose period exceeds the lag limit", async () => {
+ let device: GPUDevice;
+ try {
+ device = await ensureDevice();
+ } catch {
+ return;
+ }
+ await expect(
+ trackAudioPitch(device, [new Float32Array(SR)], SR, FPS, {
+ minHz: 10,
+ }),
+ ).rejects.toThrow(/minHz >= 22/);
+ });
+});
diff --git a/packages/webgpu-renderers/src/signals/audio-pitch-compute.ts b/packages/webgpu-renderers/src/signals/audio-pitch-compute.ts
new file mode 100644
index 00000000..4c2f800c
--- /dev/null
+++ b/packages/webgpu-renderers/src/signals/audio-pitch-compute.ts
@@ -0,0 +1,488 @@
+/// <reference types="webgpu" />
+
+import { downmixToMono } from "./audio-signal-compute.js";
+
+/**
+ * Fundamental-frequency (f0) tracking: YIN period candidates on the GPU, then
+ * a Viterbi decode on the CPU (in the spirit of pYIN) that keeps the track
+ * continuous across octave errors and marks frames with no clear period as
+ * unvoiced.
+ */
+export interface AudioPitchTrackConfig {
+ /** Lowest pitch to search, in Hz. The longest period must fit in `MAX_PITCH_LAG` samples. */
+ minHz?: number;
+ /** Highest pitch to search, in Hz. */
+ maxHz?: number;
+ /** Analysis window per frame, in milliseconds. Never shorter than the longest period. */
+ windowMs?: number;
+ /** YIN dip depth (0..1) above which a frame is treated as unvoiced. Higher accepts noisier sources. */
+ voicingThreshold?: number;
+ /** Cost per octave of pitch change between neighboring frames. Higher gives a smoother track. */
+ octaveJumpCost?: number;
+}
+
+export interface AudioPitchTrack {
+ /** Pitch in Hz per frame; unvoiced gaps are interpolated between voiced neighbors. */
+ hz: Float32Array;
+ /** 1 minus the YIN dip depth of the chosen period, 0 on unvoiced frames. */
+ confidence: Float32Array;
+ /** 1 where a period was measured, 0 where `hz` is interpolated. */
+ voiced: Float32Array;
+ numFrames: number;
+ durationSec: number;
+}
+
+/** Period candidates as `[lagSamples, yinValue]` pairs, `PITCH_CANDIDATES` per frame. */
+export interface PitchCandidates {
+ data: Float32Array;
+ numFrames: number;
+ sampleRate: number;
+}
+
+export const MAX_PITCH_LAG = 2048;
+export const PITCH_CANDIDATES = 8;
+const EMPTY_CANDIDATE = 2;
+/** Cost of switching between voiced and unvoiced from one frame to the next. */
+const VOICING_SWITCH_COST = 0.6;
+/** Extra cost of a dip that echoes a shorter candidate's period. */
+const SUBHARMONIC_COST = 0.5;
+/** How many times shallower the shorter dip may be and still explain a longer one. */
+const SUBHARMONIC_DEPTH_RATIO = 1.25;
+/** Relative lag error allowed when matching a period multiple. */
+const PERIOD_MULTIPLE_TOLERANCE = 0.03;
+/** Relative lag distance under which two local minima are one noisy dip. */
+const RIPPLE_MERGE = 0.03;
+const WORKGROUP_SIZE = 128;
+const MAX_GRID_X = 65535;
+
+export const AUDIO_PITCH_CANDIDATES_SHADER = `
+struct PitchUniforms {
+ sampleRate : f32,
+ numSamples : u32,
+ numFrames : u32,
+ windowSize : u32,
+ lagMin : u32,
+ lagMax : u32,
+ gridX : u32,
+ fps : f32,
+};
+
+const K : u32 = ${PITCH_CANDIDATES}u;
+const WG : u32 = ${WORKGROUP_SIZE}u;
+const EMPTY : f32 = ${EMPTY_CANDIDATE}.0f;
+const MERGE : f32 = ${RIPPLE_MERGE}f;
+
+@group(0) @binding(0) var<uniform> u : PitchUniforms;
+@group(0) @binding(1) var<storage, read> pcm : array<f32>;
+@group(0) @binding(2) var<storage, read_write> candidates : array<vec2f>;
+
+var<workgroup> yin : array<f32, ${MAX_PITCH_LAG + 1}>;
+
+fn sampleAt(i : u32) -> f32 {
+ if (i >= u.numSamples) { return 0.0f; }
+ return pcm[i];
+}
+
+// Keeps the K deepest dips sorted by value, shallowest last. A dip within
+// MERGE of an existing candidate's lag is the same broad dip: only the
+// deeper of the two survives.
+fn insertCandidate(lags : ptr<function, array<f32, K>>, vals : ptr<function, array<f32, K>>, lag : f32, val : f32) {
+ var slot = K - 1u;
+ for (var k = 0u; k < K; k = k + 1u) {
+ if ((*vals)[k] < EMPTY && abs((*lags)[k] - lag) <= MERGE * lag) {
+ if ((*vals)[k] <= val) { return; }
+ slot = k;
+ break;
+ }
+ }
+ if ((*vals)[slot] <= val) { return; }
+ var i = slot;
+ loop {
+ if (i == 0u || (*vals)[i - 1u] <= val) { break; }
+ (*vals)[i] = (*vals)[i - 1u];
+ (*lags)[i] = (*lags)[i - 1u];
+ i = i - 1u;
+ }
+ (*vals)[i] = val;
+ (*lags)[i] = lag;
+}
+
+@compute @workgroup_size(${WORKGROUP_SIZE})
+fn main(@builtin(workgroup_id) wid : vec3<u32>, @builtin(local_invocation_index) lid : u32) {
+ let frame = wid.x + wid.y * u.gridX;
+ if (frame >= u.numFrames) { return; }
+
+ // Center the analysed span (window + longest lag) on the frame time.
+ let span = u.windowSize + u.lagMax;
+ let center = i32(round(f32(frame) * u.sampleRate / u.fps));
+ let maxStart = max(i32(u.numSamples) - i32(span), 0);
+ let start = u32(clamp(center - i32(span / 2u), 0, maxStart));
+
+ // YIN difference function, one lag per invocation per stride.
+ for (var tau = lid; tau <= u.lagMax; tau = tau + WG) {
+ var acc = 0.0f;
+ for (var j = 0u; j < u.windowSize; j = j + 1u) {
+ let d = sampleAt(start + j) - sampleAt(start + j + tau);
+ acc = acc + d * d;
+ }
+ yin[tau] = acc;
+ }
+ workgroupBarrier();
+ if (lid != 0u) { return; }
+
+ // Cumulative mean normalized difference, in place.
+ var running = 0.0f;
+ yin[0] = 1.0f;
+ for (var tau = 1u; tau <= u.lagMax; tau = tau + 1u) {
+ running = running + yin[tau];
+ yin[tau] = select(1.0f, yin[tau] * f32(tau) / running, running > 0.0f);
+ }
+
+ var lags : array<f32, K>;
+ var vals : array<f32, K>;
+ for (var k = 0u; k < K; k = k + 1u) {
+ lags[k] = 0.0f;
+ vals[k] = EMPTY;
+ }
+ for (var tau = max(u.lagMin, 1u); tau < u.lagMax; tau = tau + 1u) {
+ let prev = yin[tau - 1u];
+ let cur = yin[tau];
+ let next = yin[tau + 1u];
+ if (cur < prev && cur <= next) {
+ // Parabolic interpolation of the dip for sub-sample period.
+ let den = prev - 2.0f * cur + next;
+ let offset = select(0.0f, 0.5f * (prev - next) / den, den > 0.0f);
+ insertCandidate(&lags, &vals, f32(tau) + offset, cur);
+ }
+ }
+
+ let base = frame * K;
+ for (var k = 0u; k < K; k = k + 1u) {
+ candidates[base + k] = vec2f(lags[k], vals[k]);
+ }
+}
+`;
+
+interface ResolvedPitchConfig {
+ lagMin: number;
+ lagMax: number;
+ windowSize: number;
+ voicingThreshold: number;
+ octaveJumpCost: number;
+}
+
+function resolvePitchConfig(
+ sampleRate: number,
+ config: AudioPitchTrackConfig,
+): ResolvedPitchConfig {
+ const minHz = config.minHz ?? 50;
+ const maxHz = config.maxHz ?? 1000;
+ if (!(minHz > 0 && maxHz > minHz)) {
+ throw new Error(
+ `Pitch range needs 0 < minHz < maxHz (got ${minHz}..${maxHz}).`,
+ );
+ }
+ const lagMax = Math.ceil(sampleRate / minHz) + 1;
+ if (lagMax > MAX_PITCH_LAG) {
+ throw new Error(
+ `minHz ${minHz} needs a ${lagMax}-sample period at ${sampleRate} Hz; the limit is ${MAX_PITCH_LAG} (minHz >= ${Math.ceil(sampleRate / (MAX_PITCH_LAG - 1))}).`,
+ );
+ }
+ const lagMin = Math.max(2, Math.floor(sampleRate / maxHz));
+ const windowMs = config.windowMs ?? 45;
+ return {
+ lagMin,
+ lagMax,
+ windowSize: Math.max(lagMax, Math.round((windowMs / 1000) * sampleRate)),
+ voicingThreshold: config.voicingThreshold ?? 0.8,
+ octaveJumpCost: config.octaveJumpCost ?? 8,
+ };
+}
+
+function transitionCost(
+ prevLag: number,
+ lag: number,
+ octaveJumpCost: number,
+): number {
+ const prevVoiced = prevLag > 0;
+ const voiced = lag > 0;
+ if (prevVoiced !== voiced) return VOICING_SWITCH_COST;
+ if (!voiced) return 0;
+ return octaveJumpCost * Math.abs(Math.log2(lag / prevLag));
+}
+
+/**
+ * True when `lag` is an integer multiple (2x, 3x, ...) of a shorter candidate
+ * that is about as deep. YIN's normalized difference dips at every multiple
+ * of the true period, often slightly deeper, so such a dip is an echo.
+ */
+function isPeriodMultiple(
+ data: Float32Array,
+ base: number,
+ lag: number,
+ depth: number,
+): boolean {
+ for (let j = 0; j < PITCH_CANDIDATES; j++) {
+ const shorter = data[(base + j) * 2];
+ const shorterDepth = data[(base + j) * 2 + 1];
+ if (shorterDepth >= EMPTY_CANDIDATE || shorter <= 0 || shorter >= lag)
+ continue;
+ const m = Math.round(lag / shorter);
+ if (
+ m >= 2 &&
+ Math.abs(lag - m * shorter) <= PERIOD_MULTIPLE_TOLERANCE * lag &&
+ shorterDepth <= depth * SUBHARMONIC_DEPTH_RATIO
+ ) {
+ return true;
+ }
+ }
+ return false;
+}
+
+/**
+ * Writes each state's lag and emission cost for frame `f`. A candidate costs
+ * its YIN dip depth, plus `SUBHARMONIC_COST` when it is a period multiple;
+ * the unvoiced state (index K, lag 0) costs the voicing threshold.
+ */
+function observeFrame(
+ data: Float32Array,
+ f: number,
+ voicingThreshold: number,
+ lags: Float64Array,
+ costs: Float64Array,
+): void {
+ const K = PITCH_CANDIDATES;
+ const base = f * K;
+ const row = f * (K + 1);
+ for (let k = 0; k < K; k++) {
+ const lag = data[(base + k) * 2];
+ const depth = data[(base + k) * 2 + 1];
+ const valid = depth < EMPTY_CANDIDATE && lag > 0;
+ lags[row + k] = valid ? lag : 0;
+ costs[row + k] = !valid
+ ? Number.POSITIVE_INFINITY
+ : depth +
+ (isPeriodMultiple(data, base, lag, depth) ? SUBHARMONIC_COST : 0);
+ }
+ lags[row + K] = 0;
+ costs[row + K] = voicingThreshold;
+}
+
+/** Most likely state per frame given per-state lags and -log emission costs. */
+function viterbi(
+ lags: Float64Array,
+ costs: Float64Array,
+ numFrames: number,
+ octaveJumpCost: number,
+): Uint8Array {
+ const S = PITCH_CANDIDATES + 1;
+ const back = new Uint8Array(numFrames * S);
+ let total = costs.slice(0, S);
+ for (let f = 1; f < numFrames; f++) {
+ const next = new Float64Array(S);
+ for (let s = 0; s < S; s++) {
+ let best = Number.POSITIVE_INFINITY;
+ for (let p = 0; p < S; p++) {
+ const c =
+ total[p] +
+ transitionCost(
+ lags[(f - 1) * S + p],
+ lags[f * S + s],
+ octaveJumpCost,
+ );
+ if (c < best) {
+ best = c;
+ back[f * S + s] = p;
+ }
+ }
+ next[s] = best + costs[f * S + s];
+ }
+ total = next;
+ }
+ const path = new Uint8Array(numFrames);
+ let state = total.indexOf(Math.min(...total));
+ for (let f = numFrames - 1; f >= 0; f--) {
+ path[f] = state;
+ state = back[f * S + state];
+ }
+ return path;
+}
+
+/**
+ * Viterbi decode over each frame's period candidates plus one unvoiced state,
+ * emissions from `observeFrame`, transitions costing `octaveJumpCost` per
+ * octave and a fixed voicing switch.
+ */
+export function decodePitchTrack(
+ candidates: PitchCandidates,
+ config: Pick<AudioPitchTrackConfig, "voicingThreshold" | "octaveJumpCost">,
+): Omit<AudioPitchTrack, "durationSec"> {
+ const { data, numFrames, sampleRate } = candidates;
+ const K = PITCH_CANDIDATES;
+ const S = K + 1;
+ const lags = new Float64Array(numFrames * S);
+ const costs = new Float64Array(numFrames * S);
+ for (let f = 0; f < numFrames; f++) {
+ observeFrame(data, f, config.voicingThreshold ?? 0.8, lags, costs);
+ }
+ const path = viterbi(lags, costs, numFrames, config.octaveJumpCost ?? 8);
+
+ const hz = new Float32Array(numFrames);
+ const confidence = new Float32Array(numFrames);
+ const voiced = new Float32Array(numFrames);
+ for (let f = 0; f < numFrames; f++) {
+ const s = path[f];
+ if (lags[f * S + s] === 0) continue;
+ hz[f] = sampleRate / lags[f * S + s];
+ confidence[f] = Math.min(1, Math.max(0, 1 - data[(f * K + s) * 2 + 1]));
+ voiced[f] = 1;
+ }
+ fillUnvoiced(hz, voiced);
+ return { hz, confidence, voiced, numFrames };
+}
+
+/** Linearly interpolates `hz` across unvoiced runs; holds the nearest voiced value at the ends. */
+function fillUnvoiced(hz: Float32Array, voiced: Float32Array): void {
+ let prev = -1;
+ for (let f = 0; f <= hz.length; f++) {
+ if (f < hz.length && voiced[f] === 0) continue;
+ const from = prev >= 0 ? hz[prev] : f < hz.length ? hz[f] : 0;
+ const to = f < hz.length ? hz[f] : from;
+ for (let g = prev + 1; g < f; g++) {
+ const t = prev >= 0 && f < hz.length ? (g - prev) / (f - prev) : 0;
+ hz[g] = from + (to - from) * t;
+ }
+ prev = f;
+ }
+}
+
+const pipelines = new WeakMap<GPUDevice, GPUComputePipeline>();
+
+function getPipeline(device: GPUDevice): GPUComputePipeline {
+ let pipeline = pipelines.get(device);
+ if (!pipeline) {
+ pipeline = device.createComputePipeline({
+ label: "AudioPitchCandidatesPipeline",
+ layout: "auto",
+ compute: {
+ module: device.createShaderModule({
+ label: "AudioPitchCandidatesPipeline.wgsl",
+ code: AUDIO_PITCH_CANDIDATES_SHADER,
+ }),
+ entryPoint: "main",
+ },
+ });
+ pipelines.set(device, pipeline);
+ }
+ return pipeline;
+}
+
+/** Runs the YIN candidate pass on the GPU: `PITCH_CANDIDATES` periods per frame. */
+export async function extractPitchCandidates(
+ device: GPUDevice,
+ audioChannels: Float32Array[],
+ sampleRate: number,
+ fps: number,
+ config: AudioPitchTrackConfig = {},
+): Promise<PitchCandidates> {
+ if (audioChannels.length === 0 || audioChannels[0].length === 0) {
+ throw new Error("Empty audio buffer supplied to the pitch tracker.");
+ }
+ const { lagMin, lagMax, windowSize } = resolvePitchConfig(sampleRate, config);
+ const mono = downmixToMono(audioChannels);
+ const numFrames = Math.max(1, Math.ceil((mono.length / sampleRate) * fps));
+ const gridX = Math.min(numFrames, MAX_GRID_X);
+ const outBytes = numFrames * PITCH_CANDIDATES * 2 * 4;
+
+ const uniforms = new ArrayBuffer(8 * 4);
+ const f32 = new Float32Array(uniforms);
+ const u32 = new Uint32Array(uniforms);
+ f32[0] = sampleRate;
+ u32[1] = mono.length;
+ u32[2] = numFrames;
+ u32[3] = windowSize;
+ u32[4] = lagMin;
+ u32[5] = lagMax;
+ u32[6] = gridX;
+ f32[7] = fps;
+
+ const uniformBuffer = device.createBuffer({
+ size: uniforms.byteLength,
+ usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
+ label: "audio_pitch_uniforms",
+ });
+ device.queue.writeBuffer(uniformBuffer, 0, uniforms);
+ const pcmBuffer = device.createBuffer({
+ size: mono.byteLength,
+ usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST,
+ label: "audio_pitch_pcm",
+ });
+ device.queue.writeBuffer(
+ pcmBuffer,
+ 0,
+ mono as unknown as GPUAllowSharedBufferSource,
+ );
+ const candidateBuffer = device.createBuffer({
+ size: outBytes,
+ usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_SRC,
+ label: "audio_pitch_candidates",
+ });
+ const readback = device.createBuffer({
+ size: outBytes,
+ usage: GPUBufferUsage.MAP_READ | GPUBufferUsage.COPY_DST,
+ label: "audio_pitch_candidates_readback",
+ });
+
+ const pipeline = getPipeline(device);
+ const encoder = device.createCommandEncoder({
+ label: "audio_pitch_encoder",
+ });
+ const pass = encoder.beginComputePass({
+ label: "audio_pitch_candidates_pass",
+ });
+ pass.setPipeline(pipeline);
+ pass.setBindGroup(
+ 0,
+ device.createBindGroup({
+ layout: pipeline.getBindGroupLayout(0),
+ entries: [
+ { binding: 0, resource: { buffer: uniformBuffer } },
+ { binding: 1, resource: { buffer: pcmBuffer } },
+ { binding: 2, resource: { buffer: candidateBuffer } },
+ ],
+ }),
+ );
+ pass.dispatchWorkgroups(gridX, Math.ceil(numFrames / gridX));
+ pass.end();
+ encoder.copyBufferToBuffer(candidateBuffer, 0, readback, 0, outBytes);
+ device.queue.submit([encoder.finish()]);
+
+ await readback.mapAsync(GPUMapMode.READ);
+ const data = new Float32Array(readback.getMappedRange().slice(0));
+ readback.unmap();
+ for (const b of [readback, candidateBuffer, pcmBuffer, uniformBuffer]) {
+ b.destroy();
+ }
+ return { data, numFrames, sampleRate };
+}
+
+/** Tracks the fundamental frequency of the downmixed channels at `fps` frames per second. */
+export async function trackAudioPitch(
+ device: GPUDevice,
+ audioChannels: Float32Array[],
+ sampleRate: number,
+ fps: number,
+ config: AudioPitchTrackConfig = {},
+): Promise<AudioPitchTrack> {
+ const candidates = await extractPitchCandidates(
+ device,
+ audioChannels,
+ sampleRate,
+ fps,
+ config,
+ );
+ return {
+ ...decodePitchTrack(candidates, config),
+ durationSec: audioChannels[0].length / sampleRate,
+ };
+}
diff --git a/packages/webgpu-renderers/src/signals/audio-signal-compute.ts b/packages/webgpu-renderers/src/signals/audio-signal-compute.ts
index 2a71d7a7..f3bbc9ed 100644
--- a/packages/webgpu-renderers/src/signals/audio-signal-compute.ts
+++ b/packages/webgpu-renderers/src/signals/audio-signal-compute.ts
@@ -340,6 +340,17 @@ fn main(@builtin(global_invocation_id) gid : vec3<u32>) {
// Audio Signal Compute Engine Class
// ---------------------------------------------------------------------------
+/** Averages every channel into one mono buffer for storage buffer upload. */
+export function downmixToMono(channels: Float32Array[]): Float32Array {
+ const mono = new Float32Array(channels[0]?.length ?? 0);
+ for (const ch of channels) {
+ for (let i = 0; i < mono.length; i++) {
+ mono[i] += ch[i] / channels.length;
+ }
+ }
+ return mono;
+}
+
function calculateBiquadBandpassCoeffs(
sampleRate: number,
centerFreq: number,
@@ -457,14 +468,7 @@ export class AudioSignalComputePipeline {
const hopSize = Math.max(1, Math.round(sampleRate / fps));
const windowSize = Math.max(32, Math.round(hopSize * 2));
- // Downmix channels to mono on CPU for storage buffer upload
- const monoPcm = new Float32Array(numAudioSamples);
- for (let c = 0; c < numChannels; c++) {
- const ch = audioChannels[c];
- for (let i = 0; i < numAudioSamples; i++) {
- monoPcm[i] += ch[i] / numChannels;
- }
- }
+ const monoPcm = downmixToMono(audioChannels);
// 1. Allocate GPU Buffers
const rawAudioBuffer = device.createBuffer({
diff --git a/packages/webgpu-renderers/src/signals/index.ts b/packages/webgpu-renderers/src/signals/index.ts
index 7d72694e..3a1803b9 100644
--- a/packages/webgpu-renderers/src/signals/index.ts
+++ b/packages/webgpu-renderers/src/signals/index.ts
@@ -1,6 +1,6 @@
+export * from "./audio-pitch-compute.js";
export * from "./audio-signal-compute.js";
export * from "./signal-registry.js";
export * from "./types.js";
export * from "./video-resolver.js";
export * from "./wgsl.js";
-
diff --git a/plugins/framefields/.claude-plugin/plugin.json b/plugins/framefields/.claude-plugin/plugin.json
index 81f6b06c..9dd703b1 100644
--- a/plugins/framefields/.claude-plugin/plugin.json
+++ b/plugins/framefields/.claude-plugin/plugin.json
@@ -2,7 +2,7 @@
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "framefields",
"displayName": "Framefields",
- "version": "2.9.3",
+ "version": "2.11.0",
"description": "Author, render and verify video compositions with the framefields WebGPU SDK.",
"author": {
"name": "Framefields",
diff --git a/plugins/framefields/plugin.json b/plugins/framefields/plugin.json
index 1cf2d2f1..66a1e6d0 100644
--- a/plugins/framefields/plugin.json
+++ b/plugins/framefields/plugin.json
@@ -1,7 +1,7 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "framefields",
- "version": "2.9.3",
+ "version": "2.11.0",
"description": "Author, render and verify video compositions with the framefields WebGPU SDK.",
"author": {
"name": "Gatewai Studio",
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index 819e3187..3ef05b54 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -273,6 +273,18 @@ object is carried on its track and registered with the program automatically (un
Anything else (a number, `undefined`) throws. Signals are sampled at the **composition** frame
(`ctx.frame`), not the layer's local frame.
+To pull signals out of audio, use `AudioSignal`, which runs on the GPU and accepts audio or video files.
+`AudioSignal.extract(src, { fps })` gives `primary`, `beat`, `bass` and `energy` envelopes.
+`AudioSignal.pitch(src, { fps, minHz, maxHz })` tracks the fundamental frequency, such as an engine note or a melody line.
+It returns `hz`, `confidence` and `voiced` signals plus raw `samples`. `hz` is interpolated across unvoiced gaps,
+so a gauge never drops to 0. Use `voiced` to find where the pitch was actually measured: short gaps mark gear
+shifts and note onsets. For a 4-stroke engine, the full-cycle pitch is RPM / 120.
+```typescript
+const engine = await AudioSignal.pitch("assets/onboard.mp4", { fps, minHz: 50, maxHz: 400 });
+const rpm = engine.samples.hz.map((hz) => hz * 120);
+const needle = LayerAnimation.create().signal("rotation", engine.hz, { multiplier: 1.2, offset: -120 });
+```
+
---
## Color, Gradients & Text Metrics
diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.md
index 94f8f21e..6eefe70f 100644
--- a/plugins/framefields/skills/framefields-render/SKILL.md
+++ b/plugins/framefields/skills/framefields-render/SKILL.md
@@ -168,6 +168,40 @@ await result.cleanup?.();
Pass `qa: true` to have the engine check the output while it renders (see **Built-in Video QA** below) and return the findings as `result.qa`.
+### Splitting a long render into chunks
+To render on several GPUs at once, have each process render a range of frames without audio, render the audio once, then join the files without re-encoding:
+
+```typescript
+import { stitchChunks } from "framefields/renderer";
+
+// On each worker (its own process and composition): frames start..end-1.
+await comp.renderVideo({ outputPath: "out/chunk-0.mp4", frames: { start: 0, end: 900 }, audio: false });
+// Once, anywhere: the whole timeline's audio.
+await comp.renderVideo({ outputPath: "out/audio.mp4", codec: "aac" });
+// On any machine (no GPU needed): chunks in timeline order.
+await stitchChunks({ chunks: ["out/chunk-0.mp4", "out/chunk-1.mp4"], audio: "out/audio.mp4", outputPath: "out/final.mp4" });
+```
+
+- Trails, datamosh and particles rebuild their state before `start` by replaying earlier frames, so a chunk starting late costs extra time but renders the same picture.
+- Render every chunk with the same framefields version, codec and quality on the same GPU type. `stitchChunks` refuses chunks with a different codec, size or encoder parameters.
+- SAM 2 / SAM 3 tracks run in order from their first prompted frame. Run the vision analysis once and point every worker at the shared cache (`FRAMEFIELDS_VISION_CACHE_DIR`), so no chunk re-tracks the frames before it.
+- `qa` checks each chunk on its own. Check the stitched file with ffmpeg (below).
+- `planChunks({ totalFrames, samples, workers: "auto" })` picks the chunk count and cuts: it adds workers only while each chunk spends at least 85% of its time rendering (about 5.7× the per-worker start-up), and splits at equal render cost. `renderer.probeRender(comp)` measures the `samples`.
+
+## Rendering in the Cloud (`framefields render --cloud`)
+
+api.framefields.dev renders the project on framefields' GPUs and splits a long film across several. It fetches the project from git, so it renders the **pushed commit**: commit and push first; uncommitted changes are not rendered (the CLI warns), and an unpushed commit is refused. It bills the account behind `FRAMEFIELDS_API_KEY`; if authentication is required, ask the user to set it in the process environment and never print or embed it.
+
+```bash
+npx framefields render src/film.ts --cloud --workers auto --plan # workers, time and estimated cost; renders nothing
+npx framefields render src/film.ts --cloud --workers auto # render, then download to output/film.mp4
+```
+
+- `--workers` defaults to 1 (one GPU, no planning step). `auto` measures frame costs on one worker, then uses as many workers as stay efficient: a 5-minute film stays on one, a 3-hour film spreads to `--max-workers` (API cap 10). A number splits into that many.
+- Show the user `--plan` before a long or expensive render: it prints the estimated cost.
+- Ctrl-C cancels the render; GPU time already used is charged. Failures print the worker's error code (e.g. `ASSET_NOT_FOUND` for an asset left out of LFS).
+- From code: `renderCloud({ entry, workers: "auto", outputPath, onProgress })` from `framefields/cloud` (no GPU needed locally). The HTTP API is `POST /v1/render` (`workers`, `max_workers`, `dry_run`), `GET` and `DELETE /v1/render/:id`.
+
---
## Rendering Without a GPU
diff --git a/plugins/framefields/skills/framefields/SKILL.md b/plugins/framefields/skills/framefields/SKILL.md
index 231c16ba..52d28ebd 100644
--- a/plugins/framefields/skills/framefields/SKILL.md
+++ b/plugins/framefields/skills/framefields/SKILL.md
@@ -225,10 +225,13 @@ Add scripts to `package.json`:
"frames": "tsx src/render.ts frames",
"grid": "tsx src/render.ts grid",
"preview": "tsx src/render.ts preview",
- "render": "tsx src/render.ts"
+ "render": "tsx src/render.ts",
+ "render:cloud": "framefields render src/film.ts --cloud --workers auto"
}
```
+`render:cloud` renders on framefields' GPUs (api.framefields.dev) from the pushed commit, with the same `FRAMEFIELDS_API_KEY` as the AI gateway; `framefields-render` covers it.
+
### Video assets & Git LFS
Video inputs (`Layer.video("assets/footage/clip.mp4")`), Lottie animations (`Layer.lottie(...)`), audio tracks (`Layer.audio(...)`), LUTs, and custom fonts belong in **Git LFS**, while render outputs in `output/` belong in `.gitignore`.