Code-first video & image, rendered natively on WebGPU.
feat: split renders across GPU workers, and render in the cloud
okanasla029c9583454
diff --git a/.changeset/chunked-render-stitch.md b/.changeset/chunked-render-stitch.mdnew file mode 100644index 00000000..f525ecaf--- /dev/null+++ b/.changeset/chunked-render-stitch.md@@ -0,0 +1,7 @@+---+"framefields": minor+---++Split a long render across GPUs. `renderVideo({ frames: { start, end }, audio: false })` renders one chunk of frames: it starts on a keyframe, its timestamps start at 0, and trails, datamosh and particles rebuild their state before `start`. `stitchChunks({ chunks, audio, outputPath })` from `framefields/renderer` joins the chunks and a separate audio-only render (`codec: "aac"`) into one file without re-encoding or a GPU. It refuses chunks encoded with a different codec, size or encoder parameters. `planChunks({ totalFrames, samples, workers: "auto" })` decides how many chunks a render should use (more only while each chunk stays at least 85% efficient against the per-worker start-up) and cuts them at equal render cost; `renderer.probeRender(comp)` measures the frame costs it needs.++`framefields render <entry> --cloud [--workers auto] [--plan]` renders the pushed commit of the project's git repo on api.framefields.dev and downloads the video; `--plan` prints the workers, time and estimated cost without rendering. The same from code: `renderCloud()` from the new `framefields/cloud` entry, which needs no GPU. `framefields init` adds a `render:cloud` script.diff --git a/CLAUDE.md b/CLAUDE.mdindex 47622f5e..833bf4cb 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -13,7 +13,7 @@ Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turb- Keep this file current when structure or dev patterns change but do not bloat it.## Repo map-- `packages/framefields`: public SDK. `Composition`, `Layer.*`, `LayerAnimation`, `Signal`, effects (`src/effects/classes.ts`), `FontManager`, `renderFrameGrid`. `src/index.ts` is the export surface. CLI in `src/cli/` (`init`, `add`, `list`, `fonts add|list`); `init` never prompts or overwrites, and its file templates are strings in `templates.ts` (a test parses them).+- `packages/framefields`: public SDK. `Composition`, `Layer.*`, `LayerAnimation`, `Signal`, effects (`src/effects/classes.ts`), `FontManager`, `renderFrameGrid`. `src/index.ts` is the export surface. CLI in `src/cli/` (`init`, `add`, `list`, `fonts add|list`, `check`, `render --cloud`); `framefields/cloud` (`renderCloud`) is the GPU-free client for api.framefields.dev; `init` never prompts or overwrites, and its file templates are strings in `templates.ts` (a test parses them).- `packages/components`: the component registry `framefields add` copies into user projects (shadcn-style). `defineComponent`/`bind` contract in `src/core/contract.ts`, shared helpers in `src/core/` (keys, color, layout, surface, theme). Components import only `framefields` and `../core`. Registry items declare font needs (e.g. emoji); CLI fonts are pinned by URL, size and SHA-256 in `cli/fonts.ts`. Prop docs in `components.json` are generated (drift test). Tree-shake size budgets are tested.- `packages/compositions`: Yoga layout (`utils/layout/resolve-layout.ts`), program schema (`program/schema.ts`), lint (`program/lint.ts`), animation/evaluator.- `packages/webgpu-renderers`: WGSL shaders, 2D/3D pipelines, Slug text (`src/slug/`), audio DSP.@@ -22,7 +22,7 @@ Code-first video engine on native WebGPU (Dawn) in Node, no browser. pnpm + turb- `packages/core`: shared types, `Effect` base, signals, `emojiPresentation` (the one UTS #51 rule used by layout, preload and lint).- Others: `renderers`, `media`, `node-sdk`, `server-utils`, `client-utils`, `tensor-webgpu`, `vision`, `tsconfig`.- `nodes/node-*`: one VFX/audio/layout node each (`src/shared/`, `src/renderers/`, `src/index.ts`). Only `node-compositor` has a `SKILL.md`; update it when Compositor behavior changes (guarded by `skillmd-check.test.ts`). Effect renderers declare `reach` and `maxPixelScale` (node-sdk) when they can: without `maxPixelScale` a scaled-up leaf using the effect stays at layout resolution.-- `apps/renderer-service`: HTTP render service (Docker, RunPod).+- `apps/renderer-service`: HTTP render service (Docker, RunPod). It renders single jobs, including the pieces of a split render (`probe`, `range` chunks, `stitch`); the API repo (`/Users/okanaslankan/framefields-dev/api`, `src/render/`) plans, dispatches and bills them.- `examples/NN_*`: reference films. Study `19_framefields_film` (structure), `22_framefields_launch` (lyric sync), `28_physics_dancer` (physics, vision, GPU particles) before writing one.- `plugins/framefields/skills/framefields{,-compose,-effects,-render,-vision}`: agent skills, symlinked from `.agents/skills` and `.claude/skills`.- `framefields` is the only published package: it bundles every `@framefields/*` workspace package, and its build fails if `dist` still imports one (`scripts/check-bundled-imports.mjs`).diff --git a/apps/renderer-service/PLAN.md b/apps/renderer-service/PLAN.mdindex c6185e9c..55b7c4ff 100644--- a/apps/renderer-service/PLAN.md+++ b/apps/renderer-service/PLAN.md@@ -23,7 +23,7 @@ A job tells the worker which **git repo + ref** holds a framefields project. The### Non-goals (for this iteration)-- Splitting one render across several workers by frame range. Section 13 covers this as future work.+- Deciding how to split one render across several workers. The worker renders the pieces (`probe`, a `video` job with `range`, `audio`, `stitch`; see the README's "Split renders"), and the caller coordinates them: api.framefields.dev does it in its render jobs.- A queue or scheduler of our own. RunPod or the caller does the queueing.- Repo encryption filters such as `git-crypt`. These are detected and rejected with a clear error.- Running untrusted third-party repos on a shared multi-tenant worker. The trust model is in section 9.@@ -512,6 +512,7 @@ Every failure maps to a stable `code`. `retryable` is what RunPod or the caller| install | `LOCKFILE_MISSING` / `INSTALL_FAILED` / `INSTALL_TIMEOUT` | no / no / yes | The tail of the install log is attached. || install | `ENGINE_NOT_FOUND` | no | The project doesn't depend on, or didn't install, framefields. || install / render | `ENGINE_SWITCH` | yes (fresh worker) | The project needs a different framefields version than the worker bound, or its dependencies changed after the worker loaded them. The worker recycles. |+| render | `ENGINE_UNSUPPORTED` | no | A `probe`, `stitch` or `range` job on a framefields version without split renders. || load | `ENTRY_NOT_FOUND` / `ENTRY_COMPILE_FAILED` / `EXPORT_NOT_FOUND` / `EXPORT_INVALID` / `EXPORT_SIDE_EFFECT` / `COMPOSITION_BUILD_FAILED` | no | The user's stack trace is attached and trimmed to the project's files. || render | `ASSET_NOT_FOUND` | no | A referenced file is missing. Often an LFS include that is too narrow, so the hint says so. || render | `RENDER_FAILED` / `RENDER_STALLED` / `RENDER_TIMEOUT` | no / yes / no | |@@ -640,7 +641,7 @@ Each phase is its own PR-sized step with tests.- Pinned SSH host keys, the RunPod templates and `create-template.sh`, `docker-compose.yml` with MinIO, `env.example`, the release workflow, and the README guide as listed above.- **Still to do:** an amd64 build (CI), the nightly host smoke tests (6.5), and a page on the docs website.9. **(Later)**- - Chunked distributed rendering: split a frame range across N jobs and concat with a stream copy.+ - ✅ Chunked distributed rendering: the worker side is done (`probe`, `range` + `audio: false`, `stitch`). Splitting, retries and backups of slow chunks live in the caller (api.framefields.dev `/v1/render`).- Webhooks on completion (`output.webhook` with an HMAC signature).## 14. Test plandiff --git a/apps/renderer-service/README.md b/apps/renderer-service/README.mdindex 26aefd5f..472d5f94 100644--- a/apps/renderer-service/README.md+++ b/apps/renderer-service/README.md@@ -276,7 +276,18 @@ Versions of `framefields` released before worker support (2.1.3 and older) still- `audio` (`aac`, `mp3`, `opus`);- `frames` (`frames: [0, 120]` → PNGs);- `frameGrid` (a contact sheet);-- `lut`.+- `lut`;+- `probe` and `stitch`, for split renders (below).++### Split renders++The worker renders one piece of a split render per job. A coordinator decides the split and sends the jobs; for api.framefields.dev that is the API's render jobs (`POST /v1/render` with `workers: "auto"`).++1. **Probe:** `{ "kind": "probe", "probe": { "workers": "auto", "maxWorkers": 10, "overheadSec": 90 } }` times frames across the timeline and plans the split with the project's own `planChunks`. Send it with an `inline` output: the result is one JSON file, `{ totalFrames, fps, samples, plan: { workers, chunks: [{ start, end, workSec }], workSec, etaSec } }`, and `source.sha` is the commit to pin the other jobs to.+2. **Chunks and audio:** one `video` job per chunk with `"range": { "from": start, "to": end }, "audio": false`, plus one `audio` job (`aac`, or `opus` for VP9). Each writes to the output bucket.+3. **Stitch:** `{ "kind": "stitch", "stitch": { "chunks": ["…/chunk-000.mp4", …], "audio": "…/audio.m4a" } }` downloads the files from the output bucket (`stitch.bucket` defaults to `STORAGE_BUCKET`; either must be in `FF_OUTPUT_BUCKETS`), joins them without re-encoding, and uploads the result to the job's output.++These kinds need a framefields version with split renders; older ones fail with `ENGINE_UNSUPPORTED` rather than render the whole film.### Output targets@@ -350,6 +361,7 @@ A failed job returns:| `RENDER_TIMEOUT` / `RENDER_STALLED` | The render took too long, or stopped making progress. || `ENGINE_NOT_FOUND` | The project doesn't depend on `framefields`, so there is nothing to render with. || `ENGINE_SWITCH` | Retry. The project needs another framefields version, or new dependencies; the worker restarts to load them. |+| `ENGINE_UNSUPPORTED` | A `probe`, `stitch` or `range` job on a framefields version without split renders. Update framefields in the project. || `GPU_DEVICE_LOST` / `RENDER_OOM` | Retry. The worker replaces itself. || `GPU_UNAVAILABLE` | The machine has no usable GPU (`FF_GPU=require`), or has one that WebGPU couldn't use. Retry on another worker; on RunPod this happens automatically. Check the GPU's Vulkan driver with `vulkaninfo --summary`. || `QA_FAILED` | `render.qa.failOn` matched. The outputs are still uploaded and listed. |diff --git a/apps/renderer-service/src/engine.gpu.test.ts b/apps/renderer-service/src/engine.gpu.test.tsindex f7c08197..2881131e 100644--- a/apps/renderer-service/src/engine.gpu.test.ts+++ b/apps/renderer-service/src/engine.gpu.test.ts@@ -173,6 +173,21 @@ gpu("in-process render step (real Node loader)", () => {expect(r.cancel).toMatchObject({ ok: false, code: "CANCELLED" });expect(r.after.ok).toBe(true);expect(r.switch).toMatchObject({ ok: false, code: "ENGINE_SWITCH" });+ expect(r.probe).toMatchObject({+ ok: true,+ files: [{ name: "main", ext: "json" }],+ });+ expect(r.probePlan).toMatchObject({+ totalFrames: 24,+ fps: 24,+ plan: { workers: 2 },+ });+ expect(r.probePlan.samples).toHaveLength(4);+ expect(r.probePlan.plan.chunks.at(-1).end).toBe(24);+ expect(r.chunk).toMatchObject({+ ok: true,+ files: [{ name: "main", ext: "mp4" }],+ });for (const k of ["video1","video2",diff --git a/apps/renderer-service/src/errors.ts b/apps/renderer-service/src/errors.tsindex e008736f..41db8b6b 100644--- a/apps/renderer-service/src/errors.ts+++ b/apps/renderer-service/src/errors.ts@@ -56,6 +56,7 @@ const CODES = {INSTALL_TIMEOUT: { phase: "install", retryable: true },ENGINE_NOT_FOUND: { phase: "install", retryable: false },ENGINE_SWITCH: { phase: "render", retryable: true },+ ENGINE_UNSUPPORTED: { phase: "render", retryable: false },ENTRY_NOT_FOUND: { phase: "load", retryable: false },ENTRY_COMPILE_FAILED: { phase: "load", retryable: false },diff --git a/apps/renderer-service/src/jobs/schema.ts b/apps/renderer-service/src/jobs/schema.tsindex 7bcae544..9fe1c908 100644--- a/apps/renderer-service/src/jobs/schema.ts+++ b/apps/renderer-service/src/jobs/schema.ts@@ -63,18 +63,47 @@ export const RENDER_KINDS = ["frameGrid","lut","audio",+ "probe",+ "stitch",] as const;+/** An object key the worker reads from storage: relative, no "..". */+const storageKey = z.string().min(1).max(1024).refine(isSafeRelativePath, {+ message: 'keys must be relative and must not contain ".."',+});+export const renderSchema = z.object({kind: z.enum(RENDER_KINDS).default("video"),codec: z.string().max(32).optional(),audioCodec: z.string().max(32).optional(),quality: z.string().max(32).optional(),frames: z.array(z.number().int().min(0)).min(1).max(10_000).optional(),+ /** kind=video: render only frames `from` (inclusive) to `to` (exclusive), as one chunk of a split render. */range: z.object({ from: z.number().int().min(0), to: z.number().int().min(1) }).refine((r) => r.to > r.from, { message: "range.to must be > range.from" }).optional(),+ /** kind=video: `false` writes no audio track (chunks; the audio is its own kind=audio job). */+ audio: z.boolean().optional(),+ /** kind=probe: measure frame costs and plan a split render (`planChunks`). */+ probe: z+ .object({+ samples: z.number().int().min(1).max(64).optional(),+ workers: z.union([z.number().int().min(1), z.literal("auto")]).optional(),+ maxWorkers: z.number().int().min(1).max(256).optional(),+ overheadSec: z.number().min(0).max(3600).optional(),+ efficiency: z.number().gt(0).lt(1).optional(),+ minChunkFrames: z.number().int().min(1).optional(),+ })+ .optional(),+ /** kind=stitch: chunk files (and the audio file) in the output bucket, joined in order. */+ stitch: z+ .object({+ bucket: z.string().min(1).max(255).optional(),+ chunks: z.array(storageKey).min(1).max(256),+ audio: storageKey.optional(),+ })+ .optional(),grid: z.object({from: z.number().int().min(0).optional(),@@ -146,6 +175,20 @@ export const jobSchema = zmessage: 'required when render.kind is "frames"',});}+ if (job.render.kind === "stitch" && !job.render.stitch) {+ ctx.addIssue({+ code: "custom",+ path: ["render", "stitch"],+ message: 'required when render.kind is "stitch"',+ });+ }+ if (job.render.range && job.render.kind !== "video") {+ ctx.addIssue({+ code: "custom",+ path: ["render", "range"],+ message: 'only applies to render.kind "video"',+ });+ }});export type JobInput = z.infer<typeof jobSchema>;diff --git a/apps/renderer-service/src/jobs/validate.ts b/apps/renderer-service/src/jobs/validate.tsindex 402111c6..af3d0f5c 100644--- a/apps/renderer-service/src/jobs/validate.ts+++ b/apps/renderer-service/src/jobs/validate.ts@@ -79,6 +79,7 @@ export function validateJob(}validateOutput(input, config);+ validateStitchInputs(input, config);for (const name of Object.keys(input.env)) {if (RESERVED_JOB_ENV.some((re) => re.test(name))) {@@ -137,6 +138,25 @@ function validateOutput(input: JobInput, config: RendererConfig): void {}}+/** A stitch job reads its chunks with the worker's own storage credentials. */+function validateStitchInputs(input: JobInput, config: RendererConfig): void {+ const stitch = input.render.stitch;+ if (input.render.kind !== "stitch" || !stitch) return;+ if (config.storage.driver !== "s3") {+ throw new RenderError(+ "JOB_INVALID",+ 'render.kind "stitch" needs STORAGE_DRIVER=s3 on the worker.',+ );+ }+ const bucket = stitch.bucket ?? config.storage.bucket;+ if (!bucket || !config.storage.outputBuckets.includes(bucket)) {+ throw new RenderError(+ "BUCKET_NOT_ALLOWED",+ `Bucket "${bucket}" is not in FF_OUTPUT_BUCKETS.`,+ );+ }+}+function globMatch(pattern: string, value: string): boolean {const re = new RegExp(`^${pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*")}$`,diff --git a/apps/renderer-service/src/output/sinks.ts b/apps/renderer-service/src/output/sinks.tsindex 3d4be55a..41ff0bb6 100644--- a/apps/renderer-service/src/output/sinks.ts+++ b/apps/renderer-service/src/output/sinks.ts@@ -1,6 +1,8 @@-import { createReadStream } from "node:fs";+import { createReadStream, createWriteStream } from "node:fs";import fs from "node:fs/promises";import path from "node:path";+import type { Readable } from "node:stream";+import { pipeline } from "node:stream/promises";import {GetObjectCommand,HeadObjectCommand,@@ -87,6 +89,41 @@ export function s3Client(config: RendererConfig): S3Client {return client;}+/** Downloads an object to `file`: a stitch job's chunks, read with the worker's own credentials. */+export async function downloadS3(+ config: RendererConfig,+ bucket: string,+ key: string,+ file: string,+ signal: AbortSignal,+): Promise<void> {+ try {+ await withRetries(signal, async () => {+ const res = await s3Client(config).send(+ new GetObjectCommand({ Bucket: bucket, Key: key }),+ { abortSignal: signal },+ );+ if (!res.Body) throw new Error(`s3://${bucket}/${key} has no body`);+ await pipeline(res.Body as Readable, createWriteStream(file), {+ signal,+ });+ });+ } catch (err) {+ const missing =+ err instanceof S3ServiceException && err.name === "NoSuchKey";+ if (+ missing ||+ (err instanceof RenderError && /NoSuchKey/.test(err.message))+ )+ throw new RenderError(+ "ASSET_NOT_FOUND",+ `Stitch input s3://${bucket}/${key} does not exist.`,+ { phase: "render" },+ );+ throw err;+ }+}+export function publicUrl(config: RendererConfig,key: string,diff --git a/apps/renderer-service/src/render/engine.ts b/apps/renderer-service/src/render/engine.tsindex cea764f1..4e166d46 100644--- a/apps/renderer-service/src/render/engine.ts+++ b/apps/renderer-service/src/render/engine.ts@@ -26,10 +26,32 @@ export interface EngineModule {resetRenderCaches?: () => void;/** The adapter the engine renders on (newer framefields versions). */describeGpu?: () => Promise<GpuAdapter>;+ /** Split renders (framefields with `renderVideo({ frames })`). */+ planChunks?: (options: Record<string, unknown>) => unknown;+ stitchChunks?: (options: {+ chunks: readonly string[];+ audio?: string;+ outputPath: string;+ signal?: AbortSignal;+ }) => Promise<{ filePath: string; frames: number; durationSec: number }>;}export interface EngineRenderer {renderLut(vm: never, frame?: number): Promise<string | Buffer>;+ /** Present with `planChunks`: frame costs across the timeline. */+ probeRender?(+ vm: never,+ options: {+ samples?: number;+ signal?: AbortSignal;+ onSample?: (done: number, total: number) => void;+ },+ ): Promise<{+ totalFrames: number;+ fps: number;+ durationMs: number;+ samples: { frame: number; ms: number }[];+ }>;}export interface BoundEngine {diff --git a/apps/renderer-service/src/render/render-step.ts b/apps/renderer-service/src/render/render-step.tsindex 4769f42a..8fc322fb 100644--- a/apps/renderer-service/src/render/render-step.ts+++ b/apps/renderer-service/src/render/render-step.ts@@ -10,9 +10,16 @@ import type {import type { JobInput, RenderSettings } from "../jobs/schema.js";import type { JobProgress } from "../jobs/store.js";import { rendererLogger } from "../logger.js";+import { downloadS3 } from "../output/sinks.js";import { gpuStatus } from "../worker/gpu.js";import { workerHealth } from "../worker/health.js";-import { type BoundEngine, bindEngine, markDepsLoaded } from "./engine.js";+import {+ type BoundEngine,+ bindEngine,+ type EngineModule,+ type EngineRenderer,+ markDepsLoaded,+} from "./engine.js";import { clearWorkspaceRequireCache } from "./engine-alias.js";import { buildPlan, importEntry, type PlanOutput } from "./load.js";import { openSandbox } from "./sandbox.js";@@ -62,6 +69,11 @@ export const render: Pipeline["render"] = async (checkout, input, ctx) => {let dispose: (() => Promise<void>) | undefined;try {const work = (async () => {+ // A stitch joins files already rendered: no composition to load.+ if (input.render.kind === "stitch")+ return [+ await stitchOutput(input.render, outDir, ctx, controller, engine),+ ];const files: RenderedFile[] = [];const plan = await withTimeout(loadPlan(checkout, input, ctx.jobId, (d) => {@@ -192,8 +204,11 @@ async function renderOutput(let lastEmit = 0;let lastPhase: string | undefined;let expectedFrames: number | undefined;+ const chunk =+ settings.kind === "video" ? chunkOptions(settings, engine) : {};const result = await output.target.renderVideo({outputPath: `${base}.${ext}`,+ ...chunk,codec,audioCodec: settings.audioCodec,quality: settings.quality,@@ -232,6 +247,25 @@ async function renderOutput(},];}+ case "probe":+ return [+ await probeOutput(+ output,+ settings,+ base,+ ctx,+ controller,+ engine,+ renderer,+ stall.touch,+ count > 1 ? index : undefined,+ ),+ ];+ case "stitch":+ throw new RenderError(+ "INTERNAL",+ "Stitch jobs are handled before the plan loads.",+ );case "frames": {const files: RenderedFile[] = [];const frames = settings.frames ?? [0];@@ -316,6 +350,176 @@ async function renderOutput(}}+/** A function the project's framefields must export for this job, or ENGINE_UNSUPPORTED. */+function requireEngine<K extends "planChunks" | "stitchChunks">(+ engine: BoundEngine,+ name: K,+ what: string,+): NonNullable<EngineModule[K]> {+ const fn = engine.mod[name];+ if (typeof fn !== "function") {+ throw new RenderError(+ "ENGINE_UNSUPPORTED",+ `framefields ${engine.version} can't run ${what}: it has no \`${name}\`.`,+ {+ phase: "render",+ hint: "Update framefields in the project to a version with split renders.",+ },+ );+ }+ return fn as NonNullable<EngineModule[K]>;+}++/**+ * `renderVideo` options for one chunk of a split render. An engine without+ * them would ignore `frames` and render the whole film, so it is refused.+ */+function chunkOptions(+ settings: RenderSettings,+ engine: BoundEngine,+): { frames?: { start: number; end: number }; audio?: false } {+ if (!settings.range && settings.audio !== false) return {};+ requireEngine(engine, "stitchChunks", "chunked renders");+ return {+ ...(settings.range && {+ frames: { start: settings.range.from, end: settings.range.to },+ }),+ ...(settings.audio === false && { audio: false }),+ };+}++/**+ * kind=probe: times frames across the timeline and plans the split render+ * with the project's own `planChunks`, written as one JSON file.+ */+async function probeOutput(+ output: PlanOutput,+ settings: RenderSettings,+ base: string,+ ctx: StepContext,+ controller: AbortController,+ engine: BoundEngine,+ renderer: Pick<EngineRenderer, "probeRender">,+ touch: () => void,+ outputIndex: number | undefined,+): Promise<RenderedFile> {+ const planChunks = requireEngine(engine, "planChunks", "probe jobs");+ const vm = output.target.toVirtualMedia?.() ?? output.target.virtualMedia;+ if (!renderer.probeRender || !vm) {+ throw new RenderError(+ "EXPORT_INVALID",+ `Output ${output.name} can't be probed.`,+ { phase: "render" },+ );+ }+ const options = settings.probe ?? {};+ const measured = await renderer.probeRender(vm as never, {+ samples: options.samples,+ signal: controller.signal,+ onSample: (done, total) => {+ touch();+ ctx.progress({+ phase: "rendering",+ output: outputIndex,+ frame: done,+ total,+ message: "probing frame costs",+ });+ },+ });+ const plan = planChunks({+ totalFrames: measured.totalFrames,+ samples: measured.samples,+ workers: options.workers,+ maxWorkers: options.maxWorkers,+ overheadSec: options.overheadSec,+ efficiency: options.efficiency,+ minChunkFrames: options.minChunkFrames,+ });+ const file = `${base}-probe.json`;+ await fs.writeFile(file, JSON.stringify({ ...measured, plan }));+ return {+ name: output.name,+ path: file,+ ext: "json",+ contentType: "application/json",+ };+}++/**+ * kind=stitch: downloads the chunks (and audio) from the output bucket and+ * joins them with the project's `stitchChunks`, without re-encoding.+ */+async function stitchOutput(+ settings: RenderSettings,+ outDir: string,+ ctx: StepContext,+ controller: AbortController,+ engine: BoundEngine,+): Promise<RenderedFile> {+ const stitch = settings.stitch;+ const bucket = stitch?.bucket ?? ctx.config.storage.bucket;+ if (!stitch || !bucket)+ throw new RenderError(+ "JOB_INVALID",+ "render.stitch needs chunks and a bucket.",+ );+ const stitchChunks = requireEngine(engine, "stitchChunks", "stitch jobs");+ const inDir = path.join(ctx.jobDir, "in");+ await fs.mkdir(inDir, { recursive: true });+ const keys = [...stitch.chunks, ...(stitch.audio ? [stitch.audio] : [])];+ let downloaded = 0;+ const local = await mapLimit(keys, 4, async (key, i) => {+ const file = path.join(inDir, `${i}-${path.basename(key)}`);+ await downloadS3(ctx.config, bucket, key, file, controller.signal);+ ctx.progress({+ phase: "rendering",+ frame: ++downloaded,+ total: keys.length,+ message: "downloading chunks",+ });+ return file;+ });+ const ext = path.extname(stitch.chunks[0]).slice(1) || "mp4";+ const { contentType } = containerFor(ext === "webm" ? "vp9" : "h264");+ const outputPath = path.join(outDir, `0-main.${ext}`);+ ctx.progress({+ phase: "encoding",+ message: `stitching ${stitch.chunks.length} chunks`,+ });+ const result = await stitchChunks({+ chunks: local.slice(0, stitch.chunks.length),+ audio: stitch.audio ? local.at(-1) : undefined,+ outputPath,+ signal: controller.signal,+ });+ return {+ name: "main",+ path: outputPath,+ ext,+ contentType,+ expectedFrames: result.frames,+ };+}++/** `Promise.all` over `items`, at most `limit` at a time, results in order. */+async function mapLimit<T, R>(+ items: readonly T[],+ limit: number,+ fn: (item: T, index: number) => Promise<R>,+): Promise<R[]> {+ const results = new Array<R>(items.length);+ let next = 0;+ const run = async () => {+ while (next < items.length) {+ const i = next++;+ results[i] = await fn(items[i], i);+ }+ };+ await Promise.all(Array.from({ length: Math.min(limit, items.length) }, run));+ return results;+}+/*** What `renderVideo({ onProgress })` reports. Engines before the phase* breakdown send only "render" and "audio", without the optional fields.diff --git a/apps/renderer-service/test/render-harness.ts b/apps/renderer-service/test/render-harness.tsindex 6b9df1e5..ef41ae18 100644--- a/apps/renderer-service/test/render-harness.ts+++ b/apps/renderer-service/test/render-harness.ts@@ -202,6 +202,21 @@ results.after = await job("after", {entry: "src/film.ts",export: "buildFilm",});+// Split renders: plan with the project's engine, then render one chunk.+results.probe = await job(+ "probe",+ { entry: "src/film.ts", export: "buildFilm" },+ { kind: "probe", probe: { samples: 4, workers: 2, minChunkFrames: 6 } },+);+const probeFile = (results.probe as { files?: { path: string }[] }).files?.[0];+results.probePlan = probeFile+ ? JSON.parse(await fs.readFile(probeFile.path, "utf8"))+ : undefined;+results.chunk = await job(+ "chunk",+ { entry: "src/film.ts", export: "buildFilm" },+ { range: { from: 12, to: 24 }, audio: false },+);// Last: a repo pinned to another framefields version can't share this process.results.switch = await job("switch",diff --git a/packages/framefields/package.json b/packages/framefields/package.jsonindex bc15fc29..993dec25 100644--- a/packages/framefields/package.json+++ b/packages/framefields/package.json@@ -81,6 +81,12 @@"import": "./dist/preview/index.mjs","default": "./dist/preview/index.mjs"},+ "./cloud": {+ "types": "./dist/cloud/index.d.mts",+ "development": "./src/cloud/index.ts",+ "import": "./dist/cloud/index.mjs",+ "default": "./dist/cloud/index.mjs"+ },"./sam": {"types": "./dist/sam/index.d.mts","development": "./src/sam/index.ts",diff --git a/packages/framefields/src/cli.ts b/packages/framefields/src/cli.tsindex e6a55506..07222f48 100644--- a/packages/framefields/src/cli.ts+++ b/packages/framefields/src/cli.ts@@ -7,6 +7,7 @@* npx framefields list the components you can add* npx framefields fonts add <role...> install fonts (emoji)* npx framefields fonts list the fonts you can install+ * npx framefields render <entry> --cloud render on api.framefields.dev*/import { readFileSync } from "node:fs";import { relative, resolve } from "node:path";@@ -14,6 +15,7 @@ import { fileURLToPath } from "node:url";import { parseArgs } from "node:util";import { type AddReport, add } from "./cli/add.js";import { runCheck } from "./cli/check.js";+import { runCloudRender } from "./cli/cloud.js";import { readConfig } from "./cli/config.js";import {FONTS,@@ -51,6 +53,10 @@ Commands:Install fonts (pinned, hash-checked, with their license)and register them in the generated <components>/fonts.ts.fonts list List the fonts you can install.+ render <entry> --cloud+ Render on api.framefields.dev from the pushed commit of+ this git repo, and download the video. Needs+ FRAMEFIELDS_API_KEY. --export defaults to buildFilm.Options for check:--export <name> Composition or builder export (default buildFilm).@@ -60,6 +66,20 @@ Options for check:--warnings-as-errors Fail on lint warnings.--json Machine-readable check, component list, or font list result.+Options for render --cloud:+ --workers <n|auto> 1 (default) renders on one GPU. auto measures the film+ first and splits it across as many GPUs as stay+ efficient; a number splits it into that many.+ --max-workers <n> Upper bound for auto.+ --plan Measure and plan only: prints workers, time and the+ estimated cost, renders nothing.+ --output <file> Where the video goes (default output/<entry>.mp4).+ --codec <codec> h264 (default) or vp9.+ --quality <q> very_low, low, high or very_high (default).+ --select <name> When the export returns several compositions.+ --json Print the final status as JSON.+ Ctrl-C cancels the render; GPU time already used is charged.+Options for init:--name <name> Package name (default: the folder's name).--pm <manager> npm, pnpm, yarn or bun (default: the one running this).@@ -87,6 +107,7 @@ Examples:npx framefields init my-film --installnpx framefields add mouse-pointer focus-zoomnpx framefields fonts add emoji+ npx framefields render src/film.ts --cloud --workers auto --plan`;async function main(argv: string[]): Promise<number> {@@ -112,6 +133,13 @@ async function main(argv: string[]): Promise<number> {output: { type: "string" },timeout: { type: "string" },"warnings-as-errors": { type: "boolean" },+ cloud: { type: "boolean" },+ workers: { type: "string" },+ "max-workers": { type: "string" },+ plan: { type: "boolean" },+ codec: { type: "string" },+ quality: { type: "string" },+ select: { type: "string" },help: { type: "boolean", short: "h" },},});@@ -164,6 +192,29 @@ async function main(argv: string[]): Promise<number> {}return report.status === "passed" ? 0 : 1;}+ if (command === "render") {+ if (names.length !== 1 || !values.cloud) {+ process.stderr.write(+ "Usage: framefields render <entry> --cloud [--workers auto] [--plan]\nLocal renders run your project's render script (`npm run render`).\n",+ );+ return 2;+ }+ return runCloudRender(+ {+ entry: names[0],+ export: values.export,+ select: values.select,+ workers: values.workers,+ maxWorkers: values["max-workers"],+ codec: values.codec,+ quality: values.quality,+ output: values.output,+ plan: values.plan,+ json: values.json,+ },+ log,+ );+ }if (command === "fonts") return fonts(names, cwd, values, log);if (command === "init") {if (names.length > 1) {diff --git a/packages/framefields/src/cli/cloud.ts b/packages/framefields/src/cli/cloud.tsnew file mode 100644index 00000000..5a7fca68--- /dev/null+++ b/packages/framefields/src/cli/cloud.ts@@ -0,0 +1,184 @@+/**+ * `framefields render <entry> --cloud`: renders the project on+ * api.framefields.dev and downloads the video. Prints one plain line per+ * phase, and a progress line at most every 10 seconds while frames render.+ */+import path from "node:path";+import {+ CloudRenderError,+ type CloudRenderOptions,+ type CloudRenderStatus,+ renderCloud,+} from "../cloud/index.js";++export interface CloudCliOptions {+ entry: string;+ export?: string;+ select?: string;+ workers?: string;+ maxWorkers?: string;+ codec?: string;+ quality?: string;+ output?: string;+ plan?: boolean;+ json?: boolean;+}++const PROGRESS_EVERY_MS = 10_000;++export async function runCloudRender(+ o: CloudCliOptions,+ log: (line: string) => void,+ render: (+ options: CloudRenderOptions,+ ) => Promise<CloudRenderStatus> = renderCloud,+): Promise<number> {+ const workers = parseWorkers(o.workers);+ if (workers === undefined) {+ log(+ `--workers must be "auto" or a positive whole number, not "${o.workers}".`,+ );+ return 2;+ }+ if (o.codec && o.codec !== "h264" && o.codec !== "vp9") {+ log(`--codec must be h264 or vp9, not "${o.codec}".`);+ return 2;+ }+ const codec = (o.codec ?? "h264") as "h264" | "vp9";+ const outputPath =+ o.output ??+ path.join(+ "output",+ `${path.basename(o.entry).replace(/\.[^.]+$/, "")}.${codec === "vp9" ? "webm" : "mp4"}`,+ );+ const controller = new AbortController();+ const onSigint = () => {+ log("Cancelling the cloud render…");+ controller.abort(new Error("cancelled"));+ };+ process.once("SIGINT", onSigint);+ const started = Date.now();+ const narrate = narrator(o.json ? () => {} : log, Date.now);+ try {+ const result = await render({+ entry: o.entry,+ export: o.export,+ select: o.select,+ workers: o.plan && workers === 1 ? "auto" : workers,+ maxWorkers: o.maxWorkers ? Number(o.maxWorkers) : undefined,+ codec,+ quality: o.quality,+ dryRun: o.plan,+ outputPath: o.plan ? undefined : outputPath,+ signal: controller.signal,+ onProgress: narrate,+ onWarning: (m) => log(`warning: ${m}`),+ });+ if (o.json) {+ log(JSON.stringify(result));+ return 0;+ }+ if (result.status === "planned") {+ log(+ `Render it: framefields render ${o.entry} --cloud --workers ${o.workers ?? "auto"}`,+ );+ return 0;+ }+ const bytes = result.output?.bytes;+ log(+ `Done in ${duration((Date.now() - started) / 1000)}: ${outputPath}${bytes ? ` (${(bytes / 1e6).toFixed(1)} MB)` : ""}, ${Math.round(result.gpuSeconds)} GPU seconds, ${usd(result.costUsd)}.`,+ );+ return 0;+ } catch (err) {+ if (err instanceof CloudRenderError) {+ log(err.message);+ if (err.status && err.status.costUsd > 0)+ log(`Charged ${usd(err.status.costUsd)} for the GPU time used.`);+ if (err.code === "insufficient_credits")+ log("Top up at https://framefields.dev/dashboard/billing");+ return 1;+ }+ throw err;+ } finally {+ process.removeListener("SIGINT", onSigint);+ }+}++function parseWorkers(value: string | undefined): number | "auto" | undefined {+ if (value === undefined) return 1;+ if (value === "auto") return "auto";+ const n = Number(value);+ return Number.isInteger(n) && n >= 1 ? n : undefined;+}++/** Turns status polls into log lines: one per phase, throttled frame progress. */+export function narrator(+ log: (line: string) => void,+ now: () => number,+): (s: CloudRenderStatus) => void {+ let phase: string | undefined;+ let planned = false;+ let lastProgress = 0;+ return (s) => {+ if (!phase) log(`Cloud render ${s.id} started.`);+ if (s.plan && !planned) {+ planned = true;+ log(describePlan(s));+ }+ if (s.status !== phase) {+ phase = s.status;+ const line = phaseLine(s);+ if (line) log(line);+ lastProgress = now();+ return;+ }+ if (s.status === "rendering" && now() - lastProgress >= PROGRESS_EVERY_MS) {+ lastProgress = now();+ log(progressLine(s));+ }+ };+}++export function describePlan(s: CloudRenderStatus): string {+ const plan = s.plan;+ if (!plan) return "";+ const est =+ s.estimateUsd !== undefined ? `, estimated ${usd(s.estimateUsd)}` : "";+ return `Plan: ${plan.workers} ${plan.workers === 1 ? "worker" : "workers"}, ${duration(plan.workSec)} of render work, ready in about ${duration(plan.etaSec)}${est}.`;+}++function phaseLine(s: CloudRenderStatus): string | undefined {+ switch (s.status) {+ case "probing":+ return "Measuring frame costs on one worker to plan the split…";+ case "rendering":+ return s.workers > 1+ ? `Rendering ${s.workers} chunks and the audio in parallel…`+ : "Rendering on one worker…";+ case "stitching":+ return `Joining ${s.workers} chunks…`;+ default:+ return undefined;+ }+}++export function progressLine(s: CloudRenderStatus): string {+ const { frames, totalFrames, etaSec } = s.progress;+ const share = totalFrames+ ? `${Math.floor((frames / totalFrames) * 100)}% (${frames}/${totalFrames} frames)`+ : `${frames} frames`;+ const eta = etaSec ? `, about ${duration(etaSec)} left` : "";+ return `Rendering ${share}${eta}, ${usd(s.costUsd)} so far.`;+}++export function duration(sec: number): string {+ const s = Math.max(0, Math.round(sec));+ if (s < 90) return `${s} s`;+ const m = Math.round(s / 60);+ if (m < 90) return `${m} min`;+ return `${Math.floor(m / 60)} h ${m % 60} min`;+}++export function usd(value: number): string {+ return value > 0 && value < 0.01 ? "<$0.01" : `$${value.toFixed(2)}`;+}diff --git a/packages/framefields/src/cli/init.test.ts b/packages/framefields/src/cli/init.test.tsindex 35df5f82..1644e082 100644--- a/packages/framefields/src/cli/init.test.ts+++ b/packages/framefields/src/cli/init.test.ts@@ -127,6 +127,7 @@ describe("framefields init", () => {"grid","preview","render",+ "render:cloud",]);expect(Object.keys(pkg.devDependencies).sort()).toEqual(["@types/node",diff --git a/packages/framefields/src/cli/templates.ts b/packages/framefields/src/cli/templates.tsindex 1d2c5853..744d3d75 100644--- a/packages/framefields/src/cli/templates.ts+++ b/packages/framefields/src/cli/templates.ts@@ -34,6 +34,7 @@ export const SCRIPTS: Record<string, string> = {grid: "tsx src/render.ts grid",preview: "tsx src/render.ts preview",render: "tsx src/render.ts",+ "render:cloud": "framefields render src/film.ts --cloud --workers auto",};/**@@ -250,6 +251,7 @@ ${run("frames")} -- 0 30 60 # PNGs in output/${run("grid")} # a contact sheet of the motion${run("preview")} # play it in the browser${run("render")} # output/output.mp4+${run("render:cloud")} # on cloud GPUs, from the pushed commit\`\`\`- \`src/film.ts\` builds the composition, \`src/theme.ts\` holds sizes, colors and fonts,@@ -258,6 +260,9 @@ ${run("render")} # output/output.mp4one into \`src/components/\`.- Optional: the AI gateway generates images, video, music, voice and 3D from code. Copy\`.env.example\` to \`.env\` and set \`FRAMEFIELDS_API_KEY\` (keys at ${AI_DASHBOARD}).+- \`render:cloud\` renders on framefields' GPUs with the same key, and splits a long film+ across several (\`-- --plan\` shows workers, time and cost first). It renders the+ commit you pushed, not uncommitted changes.- Footage, audio and fonts live in \`assets/\` and are tracked with Git LFS(\`git lfs install\` once per machine, \`git lfs pull\` after cloning).`;diff --git a/packages/framefields/src/cloud/cloud.test.ts b/packages/framefields/src/cloud/cloud.test.tsnew file mode 100644index 00000000..f1e387d5--- /dev/null+++ b/packages/framefields/src/cloud/cloud.test.ts@@ -0,0 +1,366 @@+import { execFileSync } from "node:child_process";+import fs from "node:fs/promises";+import http from "node:http";+import type { AddressInfo } from "node:net";+import os from "node:os";+import path from "node:path";+import { afterEach, beforeEach, describe, expect, it } from "vitest";+import { duration, narrator, progressLine, usd } from "../cli/cloud.js";+import {+ CloudRenderError,+ type CloudRenderStatus,+ renderCloud,+ resolveGitSource,+ toHttps,+} from "./index.js";++describe("toHttps", () => {+ it("turns git remotes into URLs a cloud worker can fetch", () => {+ expect(toHttps("[email protected]:acme/films.git")).toBe(+ "https://github.com/acme/films.git",+ );+ expect(toHttps("ssh://[email protected]/acme/films.git")).toBe(+ "https://gitlab.com/acme/films.git",+ );+ expect(toHttps("https://[email protected]/acme/films.git")).toBe(+ "https://github.com/acme/films.git",+ );+ expect(toHttps("https://github.com/acme/films")).toBe(+ "https://github.com/acme/films",+ );+ });+});++describe("resolveGitSource", () => {+ let dir: string;+ let work: string;+ const git = (cwd: string, ...args: string[]) =>+ execFileSync("git", args, {+ cwd,+ encoding: "utf8",+ stdio: ["ignore", "pipe", "pipe"],+ }).trim();++ beforeEach(async () => {+ dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-cloud-git-"));+ const remote = path.join(dir, "remote.git");+ work = path.join(dir, "work");+ git(dir, "init", "--bare", "-b", "main", remote);+ git(dir, "clone", remote, work);+ for (const [k, v] of [+ ["user.email", "[email protected]"],+ ["user.name", "t"],+ ["commit.gpgsign", "false"],+ ])+ git(work, "config", k, v);+ await fs.mkdir(path.join(work, "films/launch/src"), { recursive: true });+ await fs.writeFile(+ path.join(work, "films/launch/src/film.ts"),+ "export const buildFilm = () => null;\n",+ );+ git(work, "add", ".");+ git(work, "commit", "-m", "film");+ git(work, "push", "-u", "origin", "main");+ git(work, "remote", "set-url", "origin", "[email protected]:acme/films.git");+ });+ afterEach(() => fs.rm(dir, { recursive: true, force: true }));++ it("reads repo, branch, pushed commit and the project folder", () => {+ const project = path.join(work, "films/launch");+ const sha = git(work, "rev-parse", "HEAD");+ expect(resolveGitSource(project, "src/film.ts")).toEqual({+ source: {+ repo: "https://github.com/acme/films.git",+ ref: "main",+ commit: sha,+ workdir: "films/launch",+ },+ entry: "src/film.ts",+ warnings: [],+ });+ });++ it("warns that uncommitted changes are not rendered", async () => {+ await fs.writeFile(+ path.join(work, "films/launch/src/film.ts"),+ "export const buildFilm = () => 1;\n",+ );+ const { warnings } = resolveGitSource(+ path.join(work, "films/launch"),+ "src/film.ts",+ );+ expect(warnings[0]).toMatch(/Uncommitted changes are not rendered/);+ });++ it("refuses a commit that is not pushed", async () => {+ await fs.writeFile(path.join(work, "x.txt"), "x");+ git(work, "add", ".");+ git(work, "commit", "-m", "local only");+ expect(() => resolveGitSource(work, "films/launch/src/film.ts")).toThrow(+ /is not on origin\. Push it first/,+ );+ });++ it("refuses an entry outside the project", () => {+ expect(() =>+ resolveGitSource(path.join(work, "films/launch"), "../other.ts"),+ ).toThrow(/outside/);+ });+});++describe("renderCloud", () => {+ let server: http.Server;+ let url: string;+ let requests: { method: string; path: string; body?: unknown }[];+ /** Statuses the fake API returns, one per GET. */+ let script: Partial<Record<string, unknown>>[];++ const status = (over: Record<string, unknown>) => ({+ id: "r1",+ status: "rendering",+ workers: 3,+ progress: { frames: 0, total_frames: 3000, eta_sec: null },+ cost_usd: 0,+ gpu_seconds: 0,+ ...over,+ });++ beforeEach(async () => {+ requests = [];+ script = [];+ server = http.createServer(async (req, res) => {+ let raw = "";+ for await (const chunk of req) raw += chunk;+ requests.push({+ method: req.method ?? "",+ path: req.url ?? "",+ ...(raw && { body: JSON.parse(raw) }),+ });+ if (req.url === "/files/render.mp4") {+ res.end("video-bytes");+ return;+ }+ if (req.headers.authorization !== "Bearer ff_test_key") {+ res.writeHead(401).end(+ JSON.stringify({+ error: { type: "invalid_api_key", message: "API key is invalid" },+ }),+ );+ return;+ }+ const body =+ req.method === "POST"+ ? status({ status: "probing", workers: 1 })+ : req.method === "DELETE"+ ? status({ status: "cancelled", cost_usd: 0.12 })+ : (script.shift() ?? status({}));+ res+ .writeHead(req.method === "POST" ? 202 : 200, {+ "content-type": "application/json",+ })+ .end(JSON.stringify(body));+ });+ await new Promise<void>((r) => server.listen(0, "127.0.0.1", r));+ url = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;+ });+ afterEach(() => new Promise<void>((r) => server.close(() => r())));++ const source = {+ repo: "https://github.com/acme/films",+ commit: "abc1234",+ workdir: ".",+ };++ it("submits, polls until done and downloads the video", async () => {+ const out = path.join(+ await fs.mkdtemp(path.join(os.tmpdir(), "ff-cloud-")),+ "out/film.mp4",+ );+ script = [+ status({+ plan: { workers: 3, work_sec: 1800, eta_sec: 700, chunks: [] },+ estimate_usd: 1.9,+ }),+ status({ status: "stitching" }),+ status({+ status: "completed",+ gpu_seconds: 2100,+ cost_usd: 1.86,+ output: { url: `${url}/files/render.mp4`, bytes: 11 },+ }),+ ];+ const seen: CloudRenderStatus[] = [];+ const result = await renderCloud({+ entry: "src/film.ts",+ workers: "auto",+ source,+ outputPath: out,+ apiKey: "ff_test_key",+ apiUrl: url,+ pollMs: 1,+ onProgress: (s) => seen.push(s),+ });+ expect(requests[0]).toEqual({+ method: "POST",+ path: "/v1/render",+ body: {+ source,+ target: { entry: "src/film.ts", export: "buildFilm" },+ render: { codec: "h264" },+ workers: "auto",+ dry_run: false,+ },+ });+ expect(seen.map((s) => s.status)).toEqual([+ "probing",+ "rendering",+ "stitching",+ "completed",+ ]);+ expect(seen[1].plan).toEqual({+ workers: 3,+ workSec: 1800,+ etaSec: 700,+ chunks: [],+ });+ expect(result).toMatchObject({+ status: "completed",+ costUsd: 1.86,+ gpuSeconds: 2100,+ filePath: out,+ });+ expect(await fs.readFile(out, "utf8")).toBe("video-bytes");+ });++ it("throws with the render's error when it fails", async () => {+ script = [+ status({+ status: "failed",+ error: {+ code: "ASSET_NOT_FOUND",+ message: "chunk-1: missing logo.png",+ },+ cost_usd: 0.3,+ }),+ ];+ const failed = renderCloud({+ entry: "src/film.ts",+ source,+ apiKey: "ff_test_key",+ apiUrl: url,+ pollMs: 1,+ });+ await expect(failed).rejects.toThrow(+ "Cloud render failed: chunk-1: missing logo.png (ASSET_NOT_FOUND)",+ );+ await expect(failed).rejects.toBeInstanceOf(CloudRenderError);+ });++ it("cancels the cloud render when the signal aborts", async () => {+ const controller = new AbortController();+ const render = renderCloud({+ entry: "src/film.ts",+ source,+ apiKey: "ff_test_key",+ apiUrl: url,+ pollMs: 60_000,+ signal: controller.signal,+ });+ setTimeout(() => controller.abort(new Error("stop")), 50);+ const err = (await render.catch((e) => e)) as CloudRenderError;+ expect(err.code).toBe("cancelled");+ expect(err.status?.costUsd).toBe(0.12);+ expect(requests.at(-1)).toMatchObject({+ method: "DELETE",+ path: "/v1/render/r1",+ });+ });++ it("reports the API's refusal", async () => {+ await expect(+ renderCloud({+ entry: "src/film.ts",+ source,+ apiKey: "ff_test_bad",+ apiUrl: url,+ }),+ ).rejects.toMatchObject({+ code: "invalid_api_key",+ message: "API key is invalid",+ });+ });++ it("needs an API key", async () => {+ const saved = process.env.FRAMEFIELDS_API_KEY;+ delete process.env.FRAMEFIELDS_API_KEY;+ try {+ await expect(+ renderCloud({ entry: "src/film.ts", source, apiUrl: url }),+ ).rejects.toThrow(/FRAMEFIELDS_API_KEY/);+ } finally {+ if (saved !== undefined) process.env.FRAMEFIELDS_API_KEY = saved;+ }+ });+});++describe("render --cloud output", () => {+ const s = (over: Partial<CloudRenderStatus>): CloudRenderStatus => ({+ id: "r1",+ status: "rendering",+ workers: 10,+ progress: { frames: 0, totalFrames: 324_000, etaSec: null },+ costUsd: 0,+ gpuSeconds: 0,+ ...over,+ });++ it("prints each phase once and throttles frame progress", () => {+ const lines: string[] = [];+ let clock = 0;+ const narrate = narrator(+ (l) => lines.push(l),+ () => clock,+ );+ narrate(s({ status: "probing", workers: 1 }));+ narrate(s({ status: "probing", workers: 1 }));+ narrate(+ s({+ plan: { workers: 10, workSec: 10_800, etaSec: 1170, chunks: [] },+ estimateUsd: 4.2,+ }),+ );+ clock = 5_000;+ narrate(+ s({ progress: { frames: 1000, totalFrames: 324_000, etaSec: 1100 } }),+ );+ clock = 12_000;+ narrate(+ s({+ progress: { frames: 162_000, totalFrames: 324_000, etaSec: 540 },+ costUsd: 2.1,+ }),+ );+ narrate(s({ status: "stitching" }));+ expect(lines).toEqual([+ "Cloud render r1 started.",+ "Measuring frame costs on one worker to plan the split…",+ "Plan: 10 workers, 3 h 0 min of render work, ready in about 20 min, estimated $4.20.",+ "Rendering 10 chunks and the audio in parallel…",+ "Rendering 50% (162000/324000 frames), about 9 min left, $2.10 so far.",+ "Joining 10 chunks…",+ ]);+ });++ it("formats durations and money", () => {+ expect(duration(40)).toBe("40 s");+ expect(duration(14 * 60)).toBe("14 min");+ expect(duration(6120)).toBe("1 h 42 min");+ expect(usd(0.004)).toBe("<$0.01");+ expect(usd(1.861)).toBe("$1.86");+ expect(+ progressLine(+ s({ progress: { frames: 5, totalFrames: null, etaSec: null } }),+ ),+ ).toBe("Rendering 5 frames, $0.00 so far.");+ });+});diff --git a/packages/framefields/src/cloud/git-source.ts b/packages/framefields/src/cloud/git-source.tsnew file mode 100644index 00000000..222d9e3c--- /dev/null+++ b/packages/framefields/src/cloud/git-source.ts@@ -0,0 +1,116 @@+/**+ * Where a cloud render gets the project from: the git repo the working+ * directory is in, at the commit checked out now. Cloud workers fetch from+ * the remote, so that commit has to be pushed; uncommitted changes are not+ * rendered.+ */+import { execFileSync } from "node:child_process";+import { realpathSync } from "node:fs";+import path from "node:path";++export interface CloudSource {+ repo: string;+ ref?: string;+ commit?: string;+ /** The project's folder inside the repo (monorepos), `.` at the root. */+ workdir?: string;+}++export interface ResolvedSource {+ source: CloudSource;+ /** The entry relative to `source.workdir`, as the worker loads it. */+ entry: string;+ /** Things that render differently than the user may expect (uncommitted changes). */+ warnings: string[];+}++/** Reads repo, branch, commit and folder from git, and checks the commit is pushed. */+export function resolveGitSource(cwd: string, entry: string): ResolvedSource {+ const git = (...args: string[]) =>+ execFileSync("git", args, {+ cwd,+ encoding: "utf8",+ stdio: ["ignore", "pipe", "pipe"],+ }).trim();+ let root: string;+ try {+ root = git("rev-parse", "--show-toplevel");+ } catch {+ throw new Error(+ `${cwd} is not in a git repository. Cloud renders fetch the project from git: commit it and push it to a remote first.`,+ );+ }+ // git reports the root with symlinks resolved (macOS /var → /private/var).+ const workdir =+ path+ .relative(realpathSync(root), realpathSync(cwd))+ .split(path.sep)+ .join("/") || ".";+ const relEntry = path+ .relative(cwd, path.resolve(cwd, entry))+ .split(path.sep)+ .join("/");+ if (relEntry.startsWith("../") || path.isAbsolute(relEntry))+ throw new Error(+ `${entry} is outside ${cwd}; run from the project folder that contains it.`,+ );++ const branch = git("rev-parse", "--abbrev-ref", "HEAD");+ const commit = git("rev-parse", "HEAD");+ const remoteName = upstreamRemote(git, branch) ?? "origin";+ let remoteUrl: string;+ try {+ remoteUrl = git("remote", "get-url", remoteName);+ } catch {+ throw new Error(+ `The repository has no "${remoteName}" remote. Push the project to a git host first.`,+ );+ }+ const pushed = git("branch", "-r", "--contains", commit);+ if (!pushed)+ throw new Error(+ `Commit ${commit.slice(0, 7)} is not on ${remoteName}. Push it first (\`git push\`): cloud workers fetch the project from the remote.`,+ );+ const dirty = git("status", "--porcelain", "--untracked-files=no", "--", ".");+ return {+ source: {+ repo: toHttps(remoteUrl),+ ...(branch !== "HEAD" && { ref: branch }),+ commit,+ workdir,+ },+ entry: relEntry,+ warnings: dirty+ ? [+ `Uncommitted changes are not rendered: the cloud renders commit ${commit.slice(0, 7)}.`,+ ]+ : [],+ };+}++function upstreamRemote(+ git: (...args: string[]) => string,+ branch: string,+): string | undefined {+ if (branch === "HEAD") return undefined;+ try {+ return git("config", `branch.${branch}.remote`) || undefined;+ } catch {+ return undefined;+ }+}++/** `[email protected]:acme/films.git` → `https://github.com/acme/films.git`; credentials dropped. */+export function toHttps(url: string): string {+ const scp = /^[\w.-]+@([^:/]+):(.+)$/.exec(url);+ if (scp) return `https://${scp[1]}/${scp[2]}`;+ const parsed = URL.canParse(url) ? new URL(url) : undefined;+ if (parsed && (parsed.protocol === "ssh:" || parsed.protocol === "git:"))+ return `https://${parsed.hostname}${parsed.pathname}`;+ if (parsed?.protocol === "https:" || parsed?.protocol === "http:") {+ parsed.username = "";+ parsed.password = "";+ return parsed.toString();+ }+ return url;+}diff --git a/packages/framefields/src/cloud/index.ts b/packages/framefields/src/cloud/index.tsnew file mode 100644index 00000000..7acb084d--- /dev/null+++ b/packages/framefields/src/cloud/index.ts@@ -0,0 +1,320 @@+/**+ * Cloud renders on api.framefields.dev: the project renders from git on+ * framefields' GPU workers, billed to the account that owns the API key.+ *+ * ```ts+ * import { renderCloud } from "framefields/cloud";+ *+ * const result = await renderCloud({+ * entry: "src/film.ts",+ * workers: "auto", // split a long film across GPUs; 1 (default) renders on one+ * outputPath: "output/film.mp4",+ * onProgress: (s) => console.log(s.status, s.progress.frames),+ * });+ * ```+ *+ * `workers: "auto"` probes the film on one worker first, then splits it into+ * as many chunks as stay efficient: short films stay on one worker, long ones+ * spread out up to `maxWorkers`. `dryRun: true` returns that plan with the+ * estimated cost and renders nothing. Aborting `signal` cancels the render;+ * GPU time already used is charged.+ */+import { createWriteStream } from "node:fs";+import fs from "node:fs/promises";+import path from "node:path";+import { Readable } from "node:stream";+import { pipeline } from "node:stream/promises";+import type { ReadableStream as WebReadableStream } from "node:stream/web";+import { type CloudSource, resolveGitSource } from "./git-source.js";++export { type CloudSource, resolveGitSource, toHttps } from "./git-source.js";++export const FRAMEFIELDS_API_URL = "https://api.framefields.dev";++export interface CloudRenderOptions {+ /** The module that builds the composition, relative to `cwd`. */+ entry: string;+ /** Its export: a Composition, or a function returning one. Default `buildFilm`. */+ export?: string;+ /** Passed to the export when it is a function. */+ args?: unknown;+ /** When the export returns several compositions, the one to render. */+ select?: string;+ /** 1 (default) renders on one worker; a number or "auto" splits the film. */+ workers?: number | "auto";+ /** Upper bound for "auto" (the API caps it too). */+ maxWorkers?: number;+ codec?: "h264" | "vp9";+ quality?: string;+ /** Probe and plan only: returns the plan and estimated cost, renders nothing. */+ dryRun?: boolean;+ /** Download the finished video here. */+ outputPath?: string;+ /** Project folder. Default `process.cwd()`. */+ cwd?: string;+ /** Instead of reading repo, branch and commit from git. */+ source?: CloudSource;+ /** Default `process.env.FRAMEFIELDS_API_KEY`. */+ apiKey?: string;+ /** Default `process.env.FRAMEFIELDS_API_URL`, else https://api.framefields.dev. */+ apiUrl?: string;+ /** Called with every status poll. */+ onProgress?: (status: CloudRenderStatus) => void;+ /** Called for each warning (uncommitted changes). Default: stderr. */+ onWarning?: (message: string) => void;+ /** Aborting cancels the cloud render. */+ signal?: AbortSignal;+ /** Status poll interval, ms. Default 3000. */+ pollMs?: number;+}++export type CloudRenderPhase =+ | "probing"+ | "holding"+ | "rendering"+ | "stitching"+ | "planned"+ | "completed"+ | "failed"+ | "cancelled";++export interface CloudRenderPlan {+ workers: number;+ /** Render time of the whole film on one GPU, seconds. */+ workSec: number;+ /** Expected wall time with this plan, seconds. */+ etaSec: number;+ chunks: { start: number; end: number; workSec: number }[];+}++export interface CloudRenderStatus {+ id: string;+ status: CloudRenderPhase;+ workers: number;+ plan?: CloudRenderPlan;+ progress: {+ frames: number;+ totalFrames: number | null;+ etaSec: number | null;+ };+ /** Charged so far (final once the render ended), USD. */+ costUsd: number;+ gpuSeconds: number;+ /** The planned render's estimated charge, USD. */+ estimateUsd?: number;+ /** The commit rendered. */+ sha?: string;+ output?: { url: string; bytes?: number };+ error?: { code: string; message: string };+}++export interface CloudRenderResult extends CloudRenderStatus {+ /** Where the video was downloaded (`outputPath`). */+ filePath?: string;+}++/** A cloud render that failed, was cancelled, or was refused by the API. */+export class CloudRenderError extends Error {+ constructor(+ message: string,+ readonly code: string,+ readonly status?: CloudRenderStatus,+ ) {+ super(message);+ this.name = "CloudRenderError";+ }+}++const DONE = new Set<CloudRenderPhase>([+ "planned",+ "completed",+ "failed",+ "cancelled",+]);++/** Starts a cloud render and waits for it; resolves when it completed (or was planned). */+export async function renderCloud(+ options: CloudRenderOptions,+): Promise<CloudRenderResult> {+ const api = cloudClient(options);+ const cwd = options.cwd ?? process.cwd();+ const resolved = options.source+ ? { source: options.source, entry: options.entry, warnings: [] }+ : resolveGitSource(cwd, options.entry);+ const warn =+ options.onWarning ??+ ((m: string) => process.stderr.write(`framefields: ${m}\n`));+ for (const w of resolved.warnings) warn(w);++ let status = await api.start({+ source: resolved.source,+ target: {+ entry: resolved.entry,+ export: options.export ?? "buildFilm",+ ...(options.args !== undefined && { args: options.args }),+ ...(options.select && { select: [options.select] }),+ },+ render: {+ codec: options.codec ?? "h264",+ ...(options.quality && { quality: options.quality }),+ },+ workers: options.workers ?? 1,+ ...(options.maxWorkers && { max_workers: options.maxWorkers }),+ dry_run: options.dryRun ?? false,+ });+ options.onProgress?.(status);++ try {+ while (!DONE.has(status.status)) {+ await sleep(options.pollMs ?? 3000, options.signal);+ status = await api.status(status.id);+ options.onProgress?.(status);+ }+ } catch (err) {+ if (options.signal?.aborted) {+ const final = await api.cancel(status.id).catch(() => status);+ throw new CloudRenderError(+ "The cloud render was cancelled.",+ "cancelled",+ final,+ );+ }+ throw err;+ }++ if (status.status === "failed" || status.status === "cancelled")+ throw new CloudRenderError(+ status.error+ ? `Cloud render ${status.status}: ${status.error.message} (${status.error.code})`+ : `Cloud render ${status.status}.`,+ status.error?.code ?? status.status,+ status,+ );+ if (status.status === "completed" && options.outputPath && status.output) {+ await download(status.output.url, options.outputPath, options.signal);+ return { ...status, filePath: options.outputPath };+ }+ return status;+}++/** The API calls behind `renderCloud`, for callers who track renders themselves. */+export function cloudClient(+ options: Pick<CloudRenderOptions, "apiKey" | "apiUrl">,+) {+ const key = options.apiKey ?? process.env.FRAMEFIELDS_API_KEY;+ if (!key)+ throw new CloudRenderError(+ "No API key. Set FRAMEFIELDS_API_KEY (keys at https://framefields.dev/dashboard/keys) or pass apiKey.",+ "invalid_api_key",+ );+ const base = (+ options.apiUrl ??+ process.env.FRAMEFIELDS_API_URL ??+ FRAMEFIELDS_API_URL+ ).replace(/\/+$/, "");+ const call = async (method: string, route: string, body?: unknown) => {+ const res = await fetch(`${base}/v1/render${route}`, {+ method,+ headers: {+ Authorization: `Bearer ${key}`,+ "Content-Type": "application/json",+ },+ ...(body !== undefined && { body: JSON.stringify(body) }),+ });+ const json = (await res.json().catch(() => ({}))) as ApiStatus & {+ error?: { type?: string; message?: string };+ };+ if (!res.ok)+ throw new CloudRenderError(+ json.error?.message ??+ `api.framefields.dev answered HTTP ${res.status}`,+ json.error?.type ?? `http_${res.status}`,+ );+ return fromApi(json);+ };+ return {+ start: (body: unknown) => call("POST", "", body),+ status: (id: string) => call("GET", `/${encodeURIComponent(id)}`),+ cancel: (id: string) => call("DELETE", `/${encodeURIComponent(id)}`),+ };+}++/** The API's snake_case status. */+interface ApiStatus {+ id: string;+ status: CloudRenderPhase;+ workers?: number;+ plan?: {+ workers: number;+ work_sec: number;+ eta_sec: number;+ chunks: CloudRenderPlan["chunks"];+ };+ progress?: {+ frames?: number;+ total_frames?: number | null;+ eta_sec?: number | null;+ };+ cost_usd?: number;+ gpu_seconds?: number;+ estimate_usd?: number;+ sha?: string;+ output?: { url: string; bytes?: number };+ error?: { code: string; message: string };+}++function fromApi(s: ApiStatus): CloudRenderStatus {+ return {+ id: s.id,+ status: s.status,+ workers: s.workers ?? 1,+ ...(s.plan && {+ plan: {+ workers: s.plan.workers,+ workSec: s.plan.work_sec,+ etaSec: s.plan.eta_sec,+ chunks: s.plan.chunks,+ },+ }),+ progress: {+ frames: s.progress?.frames ?? 0,+ totalFrames: s.progress?.total_frames ?? null,+ etaSec: s.progress?.eta_sec ?? null,+ },+ costUsd: s.cost_usd ?? 0,+ gpuSeconds: s.gpu_seconds ?? 0,+ ...(s.estimate_usd !== undefined && { estimateUsd: s.estimate_usd }),+ ...(s.sha && { sha: s.sha }),+ ...(s.output && { output: s.output }),+ ...(s.error && { error: s.error }),+ };+}++async function download(url: string, file: string, signal?: AbortSignal) {+ const res = await fetch(url, { signal });+ if (!res.ok || !res.body)+ throw new CloudRenderError(+ `Downloading the render failed: HTTP ${res.status}`,+ "download_failed",+ );+ await fs.mkdir(path.dirname(path.resolve(file)), { recursive: true });+ await pipeline(+ Readable.fromWeb(res.body as WebReadableStream),+ createWriteStream(file),+ );+}++function sleep(ms: number, signal?: AbortSignal): Promise<void> {+ return new Promise((resolve, reject) => {+ if (signal?.aborted) return reject(signal.reason);+ const timer = setTimeout(resolve, ms);+ signal?.addEventListener(+ "abort",+ () => {+ clearTimeout(timer);+ reject(signal.reason);+ },+ { once: true },+ );+ });+}diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.tsindex 647f38ee..29258344 100644--- a/packages/framefields/src/index.ts+++ b/packages/framefields/src/index.ts@@ -1179,6 +1179,19 @@ export interface RenderVideoOptions {* (GIF only) and "done". The engine logs the same phases on its own.*/onProgress?: (progress: RenderProgress) => void;+ /**+ * Renders only frames `start` (inclusive) to `end` (exclusive): one chunk+ * of a render split across workers. Chunks start on a keyframe with+ * timestamps from 0; join them with `stitchChunks` from+ * `framefields/renderer`. Effects that carry state between frames+ * (trails, particles, datamosh) rebuild it before `start`.+ */+ frames?: { start: number; end: number };+ /**+ * `false` writes no audio track, as for chunks: render the timeline's+ * audio once with `codec: "aac"` and pass it to `stitchChunks`.+ */+ audio?: boolean;}export interface RenderVideoResult {@@ -3589,8 +3602,11 @@ export {} from "./preview/index.js";export {type AudioQaStats,+ type ChunkPlan,+ type ChunkPlanOptions,computeGridLayout,describeGpu,+ type FrameCostSample,type FrameGridOptions,type FrameGridRenderable,type FrameSamplePoint,@@ -3598,6 +3614,8 @@ export {type GpuAdapterDescription,HeadlessMediaRenderer,HeadlessWebGPURenderer,+ type PlannedChunk,+ planChunks,type QaIssue,type QaSegment,type RenderProgress,@@ -3605,6 +3623,9 @@ export {renderSemaphore,resetRenderCaches,resolveSamplePoints,+ type StitchOptions,+ type StitchResult,+ stitchChunks,type VideoQaOptions,type VideoQaReport,type VideoQaStats,diff --git a/packages/framefields/src/render-chunks.test.ts b/packages/framefields/src/render-chunks.test.tsnew file mode 100644index 00000000..7ddb2b59--- /dev/null+++ b/packages/framefields/src/render-chunks.test.ts@@ -0,0 +1,245 @@+/**+ * A render split into chunks (`renderVideo({ frames, audio: false })`) and+ * joined by `stitchChunks` matches the same render done in one go: every+ * frame, including the first one after a seam (where particles and trails+ * rebuild their state), and the soundtrack.+ */+import { execFileSync } from "node:child_process";+import fs from "node:fs/promises";+import os from "node:os";+import path from "node:path";+import {+ ALL_FORMATS,+ EncodedPacketSink,+ FilePathSource,+ Input,+} from "mediabunny";+import { describe, expect, it } from "vitest";+import {+ Composition,+ encodeStereoWav,+ Layer,+ LayerAnimation,+ Trails,+} from "./index.js";+import { stitchChunks } from "./renderer/index.js";++const hasFfmpeg = (() => {+ try {+ execFileSync("ffmpeg", ["-version"], { stdio: "ignore" });+ return true;+ } catch {+ return false;+ }+})();++const W = 160;+const H = 96;+const FPS = 30;+const FRAMES = 30;+const SEAM = 13;++function tone(seconds: number, sampleRate: number): Float32Array {+ const out = new Float32Array(Math.round(seconds * sampleRate));+ for (let i = 0; i < out.length; i++)+ out[i] = 0.3 * Math.sin((2 * Math.PI * 440 * i) / sampleRate);+ return out;+}++async function film(dir: string, width = W): Promise<Composition> {+ const wav = path.join(dir, "tone.wav");+ const source = tone(FRAMES / FPS, 48_000);+ await fs.writeFile(+ wav,+ encodeStereoWav([source, source], { sampleRate: 48_000 }),+ );+ const comp = new Composition({+ width,+ height: H,+ fps: FPS,+ durationFrames: FRAMES,+ backgroundColor: "#000000",+ });+ comp.add(+ Layer.particles({+ width,+ height: H,+ max: 1000,+ emitter: { shape: "point", x: width / 2, y: H - 8, rate: 300 },+ direction: -90,+ spread: 30,+ speed: [60, 90],+ lifetime: [2, 2],+ size: [4, 4],+ color: "#ffffff",+ }),+ );+ comp.add(+ Layer.shape("rect", {+ fillColor: "#ff4040",+ width: 20,+ height: 20,+ x: 0,+ y: 10,+ })+ .animate(+ LayerAnimation.create().fromTo("x", 0, width - 20, {+ start: 0,+ end: FRAMES,+ }),+ )+ .withEffect(new Trails({ decay: 0.8 })),+ );+ comp.add(Layer.audio(wav, { volume: 1, durationFrames: FRAMES }));+ return comp;+}++function decodeFrames(file: string): Buffer[] {+ const raw = execFileSync(+ "ffmpeg",+ [+ "-v",+ "error",+ "-i",+ file,+ "-f",+ "rawvideo",+ "-pix_fmt",+ "rgb24",+ "pipe:1",+ ],+ { maxBuffer: 1 << 28 },+ );+ const size = W * H * 3;+ return Array.from({ length: raw.length / size }, (_, i) =>+ raw.subarray(i * size, (i + 1) * size),+ );+}++function decodeAudio(file: string): Float32Array {+ const raw = execFileSync(+ "ffmpeg",+ ["-v", "error", "-i", file, "-vn", "-ac", "1", "-f", "f32le", "pipe:1"],+ { maxBuffer: 1 << 28 },+ );+ return new Float32Array(raw.buffer, raw.byteOffset, raw.byteLength / 4);+}++function meanAbsDiff(a: Buffer, b: Buffer): number {+ let sum = 0;+ for (let i = 0; i < a.length; i++) sum += Math.abs(a[i] - b[i]);+ return sum / a.length;+}++async function packetCount(+ file: string,+): Promise<{ video: number; audio: number }> {+ const input = new Input({+ source: new FilePathSource(file),+ formats: ALL_FORMATS,+ });+ try {+ const count = async (+ track: Parameters<typeof EncodedPacketSink>[0] | null,+ ) => {+ let n = 0;+ if (track)+ for await (const _ of new EncodedPacketSink(track).packets()) n++;+ return n;+ };+ return {+ video: await count(await input.getPrimaryVideoTrack()),+ audio: await count(await input.getPrimaryAudioTrack()),+ };+ } finally {+ input.dispose();+ }+}++describe("chunked render + stitchChunks", () => {+ it.skipIf(!hasFfmpeg)(+ "matches the render done in one go, frame for frame and in audio",+ async () => {+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-chunks-"));+ const comp = await film(dir);+ const full = path.join(dir, "full.mp4");+ await comp.renderVideo({ outputPath: full });++ const chunks = [path.join(dir, "a.mp4"), path.join(dir, "b.mp4")];+ // Each chunk on its own composition, as a worker builds it.+ await (await film(dir)).renderVideo({+ outputPath: chunks[0],+ frames: { start: 0, end: SEAM },+ audio: false,+ });+ await (await film(dir)).renderVideo({+ outputPath: chunks[1],+ frames: { start: SEAM, end: FRAMES },+ audio: false,+ });+ const audio = path.join(dir, "audio.mp4");+ await (await film(dir)).renderVideo({ outputPath: audio, codec: "aac" });++ expect(await packetCount(chunks[0])).toEqual({ video: SEAM, audio: 0 });+ expect((await packetCount(chunks[1])).video).toBe(FRAMES - SEAM);++ const stitched = path.join(dir, "stitched.mp4");+ const result = await stitchChunks({+ chunks,+ audio,+ outputPath: stitched,+ });+ expect(result.frames).toBe(FRAMES);+ expect(result.durationSec).toBeCloseTo(FRAMES / FPS, 3);+ expect((await packetCount(stitched)).video).toBe(FRAMES);++ const want = decodeFrames(full);+ const got = decodeFrames(stitched);+ expect(got).toHaveLength(FRAMES);+ // The scene really moves: consecutive frames differ.+ expect(meanAbsDiff(want[SEAM - 1], want[SEAM])).toBeGreaterThan(1);+ // Same pictures; a chunk's first frame is a fresh keyframe, so allow+ // for the codec, not for wrong content.+ for (let f = 0; f < FRAMES; f++)+ expect(meanAbsDiff(got[f], want[f]), `frame ${f}`).toBeLessThan(0.5);++ const a = decodeAudio(full);+ const b = decodeAudio(stitched);+ expect(b.length).toBe(a.length);+ expect(meanAbsDiff(Buffer.from(a.buffer), Buffer.from(b.buffer))).toBe(0);+ },+ 120_000,+ );++ it("rejects a frame range outside the composition", async () => {+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-chunks-"));+ const comp = await film(dir);+ await expect(+ comp.renderVideo({ frames: { start: 10, end: FRAMES + 1 } }),+ ).rejects.toThrow(/Invalid frames/);+ await expect(+ comp.renderVideo({ frames: { start: 5, end: 5 } }),+ ).rejects.toThrow(/Invalid frames/);+ });++ it("refuses to stitch chunks encoded differently", async () => {+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "ff-chunks-"));+ const a = path.join(dir, "a.mp4");+ const b = path.join(dir, "b.mp4");+ await (await film(dir)).renderVideo({+ outputPath: a,+ frames: { start: 0, end: 4 },+ audio: false,+ });+ await (await film(dir, W + 32)).renderVideo({+ outputPath: b,+ frames: { start: 4, end: 8 },+ audio: false,+ });+ const out = path.join(dir, "out.mp4");+ await expect(+ stitchChunks({ chunks: [a, b], outputPath: out }),+ ).rejects.toThrow(/size/);+ await expect(fs.stat(out)).rejects.toThrow();+ }, 60_000);+});diff --git a/packages/framefields/src/renderer/index.ts b/packages/framefields/src/renderer/index.tsindex c333bb1d..379cfdcf 100644--- a/packages/framefields/src/renderer/index.ts+++ b/packages/framefields/src/renderer/index.ts@@ -1,13 +1,22 @@export {type AudioQaStats,+ type ChunkPlan,+ type ChunkPlanOptions,+ type FrameCostSample,+ type FrameRange,formatQaReport,HeadlessMediaRenderer,HeadlessWebGPURenderer,+ type PlannedChunk,+ planChunks,type QaIssue,type QaSegment,type RenderProgress,renderSemaphore,resetRenderCaches,+ type StitchOptions,+ type StitchResult,+ stitchChunks,type VideoQaOptions,type VideoQaReport,type VideoQaStats,diff --git a/packages/framefields/tsdown.config.ts b/packages/framefields/tsdown.config.tsindex c11c3374..68c0716a 100644--- a/packages/framefields/tsdown.config.ts+++ b/packages/framefields/tsdown.config.ts@@ -15,6 +15,7 @@ export default defineConfig({"src/preview/index.ts","src/components/index.ts","src/sam/index.ts",+ "src/cloud/index.ts","src/cli.ts","src/cli/check-worker.ts",],diff --git a/packages/renderer/src/chunk-plan.ts b/packages/renderer/src/chunk-plan.tsnew file mode 100644index 00000000..3c00cece--- /dev/null+++ b/packages/renderer/src/chunk-plan.ts@@ -0,0 +1,174 @@+/**+ * Plans a render split across workers: how many chunks, and where each one+ * starts and ends. Pure arithmetic over measured frame costs, so it runs+ * anywhere (a worker's probe job, a coordinator, a local script).+ *+ * Every chunk pays a fixed cost before its first frame (worker start, fetch,+ * install, composition build). `workers: "auto"` adds workers only while each+ * chunk still spends at least `efficiency` of its time rendering, so a short+ * film stays on one worker and a long one spreads out until `maxWorkers`.+ * Chunks are cut at equal render cost, not equal frame counts, so a heavy+ * stretch (3D, vision) gets shorter chunks.+ */++/** A measured render time for one frame. */+export interface FrameCostSample {+ frame: number;+ ms: number;+}++export interface ChunkPlanOptions {+ /** Frames in the composition. */+ totalFrames: number;+ /** Measured frame times; between samples the cost is interpolated linearly. */+ samples: readonly FrameCostSample[];+ /** A worker count, or `"auto"` to pick one from the cost. Default 1. */+ workers?: number | "auto";+ /** Upper bound for `"auto"` (and for a number). Default 10. */+ maxWorkers?: number;+ /** Fixed cost each chunk pays before rendering, in seconds. Default 90 (a cold start). */+ overheadSec?: number;+ /**+ * `"auto"` only: the smallest share of a chunk's time spent rendering,+ * between 0 and 1. Default 0.85: each chunk renders at least+ * `overheadSec × 0.85 / 0.15` (about 5.7× the overhead).+ */+ efficiency?: number;+ /** No chunk is shorter than this many frames. Default 60. */+ minChunkFrames?: number;+}++export interface PlannedChunk {+ /** First frame (inclusive). */+ start: number;+ /** Last frame (exclusive). */+ end: number;+ /** Estimated render time of its frames, in seconds. */+ workSec: number;+}++export interface ChunkPlan {+ workers: number;+ /** Estimated render time of the whole composition on one worker, without overhead. */+ workSec: number;+ /** Estimated wall time with this plan: overhead plus the longest chunk. */+ etaSec: number;+ chunks: PlannedChunk[];+}++export function planChunks(options: ChunkPlanOptions): ChunkPlan {+ const total = options.totalFrames;+ if (!Number.isInteger(total) || total < 1)+ throw new Error(+ `planChunks: totalFrames must be a positive integer, got ${total}.`,+ );+ const overheadSec = options.overheadSec ?? 90;+ const maxWorkers = Math.max(1, Math.floor(options.maxWorkers ?? 10));+ const minChunkFrames = Math.max(1, Math.floor(options.minChunkFrames ?? 60));+ const cost = frameCosts(total, options.samples);+ const workSec = cost.reduce((sum, ms) => sum + ms, 0) / 1000;++ const wanted = workerCount(+ options.workers ?? 1,+ workSec,+ overheadSec,+ options.efficiency ?? 0.85,+ );+ const workers = Math.max(+ 1,+ Math.min(wanted, maxWorkers, Math.floor(total / minChunkFrames)),+ );+ const chunks = splitByCost(cost, workers, minChunkFrames);+ const longest = Math.max(...chunks.map((c) => c.workSec));+ return {+ workers: chunks.length,+ workSec,+ etaSec: overheadSec + longest,+ chunks,+ };+}++function workerCount(+ workers: number | "auto",+ workSec: number,+ overheadSec: number,+ efficiency: number,+): number {+ if (workers !== "auto") {+ if (!Number.isInteger(workers) || workers < 1)+ throw new Error(+ `planChunks: workers must be a positive integer or "auto", got ${workers}.`,+ );+ return workers;+ }+ if (!(efficiency > 0 && efficiency < 1))+ throw new Error(+ `planChunks: efficiency must be between 0 and 1, got ${efficiency}.`,+ );+ const minChunkSec = (overheadSec * efficiency) / (1 - efficiency);+ return minChunkSec > 0+ ? Math.max(1, Math.floor(workSec / minChunkSec))+ : Number.POSITIVE_INFINITY;+}++/** Per-frame cost in ms: linear between samples, flat beyond the first and last. */+function frameCosts(+ total: number,+ samples: readonly FrameCostSample[],+): Float64Array {+ const sorted = samples+ .filter(+ (s) => Number.isFinite(s.frame) && Number.isFinite(s.ms) && s.ms >= 0,+ )+ .toSorted((a, b) => a.frame - b.frame);+ if (sorted.length === 0)+ throw new Error("planChunks: needs at least one frame cost sample.");+ const cost = new Float64Array(total);+ let s = 0;+ for (let f = 0; f < total; f++) {+ while (s + 1 < sorted.length && sorted[s + 1].frame <= f) s++;+ const a = sorted[s];+ const b = sorted[s + 1];+ cost[f] =+ !b || f <= a.frame+ ? a.ms+ : a.ms + ((b.ms - a.ms) * (f - a.frame)) / (b.frame - a.frame);+ }+ return cost;+}++/**+ * Cuts the frames into `count` contiguous chunks of about equal total cost,+ * none shorter than `minFrames` (`count × minFrames` must fit).+ */+function splitByCost(+ cost: Float64Array,+ count: number,+ minFrames: number,+): PlannedChunk[] {+ const total = cost.reduce((sum, ms) => sum + ms, 0);+ const chunks: PlannedChunk[] = [];+ let start = 0;+ let acc = 0;+ let chunkMs = 0;+ for (let f = 0; f < cost.length; f++) {+ acc += cost[f];+ chunkMs += cost[f];+ const cut = (total * (chunks.length + 1)) / count;+ const framesLeft = cost.length - (f + 1);+ const chunksLeft = count - chunks.length - 1;+ const long = f + 1 - start >= minFrames;+ const roomAfter = framesLeft >= minFrames * chunksLeft;+ if (+ chunksLeft > 0 &&+ long &&+ ((acc >= cut && roomAfter) || framesLeft === minFrames * chunksLeft)+ ) {+ chunks.push({ start, end: f + 1, workSec: chunkMs / 1000 });+ start = f + 1;+ chunkMs = 0;+ }+ }+ chunks.push({ start, end: cost.length, workSec: chunkMs / 1000 });+ return chunks;+}diff --git a/packages/renderer/src/headless-webgpu-renderer.ts b/packages/renderer/src/headless-webgpu-renderer.tsindex f7c0a979..6a79219b 100644--- a/packages/renderer/src/headless-webgpu-renderer.ts+++ b/packages/renderer/src/headless-webgpu-renderer.ts@@ -68,6 +68,7 @@ import { type RenderProgress, RenderReporter } from "./render-reporter.js";);};+import type { FrameCostSample } from "./chunk-plan.js";import {type AudioQaStats,analyzeAudio,@@ -359,6 +360,112 @@ async function fileSize(file: string): Promise<number | undefined> {}}+/** Composition frames `start` (inclusive) to `end` (exclusive). */+export interface FrameRange {+ start: number;+ end: number;+}++/** The frames a render covers, checked against the composition's length. */+function resolveFrameRange(+ frames: FrameRange | undefined,+ total: number,+): FrameRange & { total: number } {+ if (!frames) return { start: 0, end: total, total };+ const { start, end } = frames;+ if (+ !Number.isInteger(start) ||+ !Number.isInteger(end) ||+ start < 0 ||+ end <= start ||+ end > total+ )+ throw new Error(+ `Invalid frames { start: ${start}, end: ${end} }: need integers with 0 <= start < end <= ${total} (the composition's frame count).`,+ );+ return { start, end, total };+}++/**+ * Mixes the composition's whole audio timeline and encodes it into `source`+ * in 100 ms samples. Returns its QA stats when `qaOptions` is set.+ */+async function encodeAudioTrack(job: {+ virtualMedia: VirtualMediaData;+ fps: number;+ device: GPUDevice;+ renderId: string;+ source: AudioSampleSource;+ codec: string;+ reporter: RenderReporter;+ qaOptions?: VideoQaOptions;+}): Promise<AudioQaStats | undefined> {+ const { reporter } = job;+ reporter.stage("audio", "Mixing audio");+ const { channels, sampleRate } = await mixAudioTracks(+ job.virtualMedia,+ job.fps,+ 48_000,+ job.device,+ job.renderId,+ {+ onProgress: ({ done, total, source }) =>+ reporter.update(+ total > 0+ ? `${done}/${total} clips${source ? `: ${source}` : ""}`+ : "no audio clips",+ total > 0 ? done / total : undefined,+ ),+ },+ );++ const numChannels = channels.length;+ if (numChannels === 0) throw new Error("mixAudioTracks returned no channels");++ const totalSamples = channels[0].length;+ const chunkSize = Math.round(0.1 * sampleRate);+ reporter.stage(+ "audio-encode",+ `Encoding ${(totalSamples / sampleRate).toFixed(1)}s of audio (${job.codec})`,+ );++ for (let offset = 0; offset < totalSamples; offset += chunkSize) {+ const currentChunk = Math.min(chunkSize, totalSamples - offset);+ // A fresh buffer per chunk: the encoder may read a sample's data after+ // add() resolves, so a buffer reused across chunks gets overwritten by+ // the next chunk before it is encoded (audible as 100 ms garbling).+ const interleaved = new Float32Array(currentChunk * numChannels);++ for (let i = 0; i < currentChunk; i++) {+ for (let ch = 0; ch < numChannels; ch++) {+ interleaved[i * numChannels + ch] = channels[ch]?.[offset + i] ?? 0;+ }+ }++ const sample = new AudioSample({+ data: interleaved,+ format: "f32",+ numberOfChannels: numChannels,+ sampleRate,+ timestamp: offset / sampleRate,+ });+ try {+ await job.source.add(sample);+ } finally {+ sample.close();+ }+ const written = offset + currentChunk;+ reporter.update(+ `${(written / sampleRate).toFixed(1)}s/${(totalSamples / sampleRate).toFixed(1)}s`,+ written / totalSamples,+ );+ }++ return job.qaOptions+ ? analyzeAudio(channels, sampleRate, job.qaOptions)+ : undefined;+}+function toEvenDimension(n: number): number {const v = Math.round(n);return v % 2 === 0 ? v : v + 1;@@ -794,6 +901,71 @@ export class HeadlessWebGPURenderer {});}+ /**+ * Measures what frames cost to render across the timeline, for planning a+ * split render (`planChunks`). At `samples` evenly spaced points (default+ * 12) it renders three consecutive frames and times the last two, so+ * effects that carry state (trails, particles) are caught up and their+ * replay is not counted. Encoding is not included.+ */+ public async probeRender(+ source: VirtualMediaData | { toVirtualMedia(): VirtualMediaData },+ options: {+ samples?: number;+ signal?: AbortSignal;+ /** Called after each sample point with how many are done. */+ onSample?: (done: number, total: number) => void;+ } = {},+ ): Promise<{+ totalFrames: number;+ fps: number;+ durationMs: number;+ samples: FrameCostSample[];+ }> {+ const vm =+ "toVirtualMedia" in source+ ? (source as { toVirtualMedia(): VirtualMediaData }).toVirtualMedia()+ : source;+ const fps = vm.metadata.fps;+ if (!fps) throw new Error("probeRender: the composition has no fps.");+ let durationMs = vm.metadata.durationMs;+ if (!durationMs || durationMs <= 0) {+ durationMs = await this.resolveMediaDuration(vm, fps);+ vm.metadata.durationMs = durationMs;+ }+ const totalFrames = frameCount(durationMs, fps);+ const count = Math.max(+ 1,+ Math.min(Math.floor(options.samples ?? 12), totalFrames),+ );+ const last = Math.max(0, totalFrames - 3);+ const renderId = `probe-${randomUUID()}`;+ const render = (frame: number) =>+ this.renderImage(vm, Math.min(frame, totalFrames - 1), fps, {+ renderId,+ format: "rgba",+ });+ const samples: FrameCostSample[] = [];+ try {+ for (let i = 0; i < count; i++) {+ options.signal?.throwIfAborted();+ const at = count === 1 ? 0 : Math.round((i * last) / (count - 1));+ await render(at);+ const started = performance.now();+ await render(at + 1);+ await render(at + 2);+ samples.push({+ frame: Math.min(at + 1, totalFrames - 1),+ ms: (performance.now() - started) / 2,+ });+ options.onSample?.(i + 1, count);+ }+ } finally {+ mediaDecoderCache.clearNode(renderId);+ }+ return { totalFrames, fps, durationMs, samples };+ }+/*** Mixes the composition's full audio timeline (clips, effects, synthesized* tracks) to stereo PCM, exactly as `renderVideo` muxes it.@@ -1060,6 +1232,19 @@ export class HeadlessWebGPURenderer {* "mux", "convert" (GIF only) and "done".*/onProgress?: (progress: RenderProgress) => void;+ /**+ * Renders only composition frames `start` (inclusive) to `end`+ * (exclusive), as one chunk of a render split across workers. The+ * chunk's timestamps start at 0 and its first frame is a keyframe,+ * so `stitchChunks` joins chunks without re-encoding. Effects that+ * carry state between frames rebuild it before `start`.+ */+ frames?: FrameRange;+ /**+ * `false` writes no audio track, as for chunks: the timeline's audio+ * is exported once with `codec: "aac"` and added by `stitchChunks`.+ */+ audio?: boolean;},): Promise<{filePath: string;@@ -1110,7 +1295,15 @@ export class HeadlessWebGPURenderer {durationMs = await this.resolveMediaDuration(virtualMedia, fps);virtualMedia.metadata.durationMs = durationMs;}- const totalFrames = frameCount(durationMs, fps);+ const range = resolveFrameRange(+ options?.frames,+ frameCount(durationMs, fps),+ );+ if (isAudioOnly && (options?.frames || options?.audio === false))+ throw new Error(+ `An audio-only render (codec "${codecOption}") takes neither \`frames\` nor \`audio: false\`.`,+ );+ const totalFrames = range.end - range.start;const renderId = `vid-${randomUUID()}`;const qaOptions: VideoQaOptions | undefined =@@ -1232,11 +1425,14 @@ export class HeadlessWebGPURenderer {} as any);output.addVideoTrack(videoSource);}- const audioSource = new AudioSampleSource({- codec: audioCodec,- bitrate: quality,- });- output.addAudioTrack(audioSource);+ let audioSource: AudioSampleSource | undefined;+ if (options?.audio !== false) {+ audioSource = new AudioSampleSource({+ codec: audioCodec,+ bitrate: quality,+ });+ output.addAudioTrack(audioSource);+ }let outputStarted = false;let outputFinalized = false;@@ -1309,18 +1505,25 @@ export class HeadlessWebGPURenderer {device.queue.submit([preEncoder.finish()]);await lutStore.awaitAllPending(device);};- await prePass(0);-- reporter.stage("render", `Rendering ${totalFrames} frames`);+ updateClockSignals(range.start, fps, durationMs);+ await prePass(range.start);++ reporter.stage(+ "render",+ range.start === 0 && totalFrames === range.total+ ? `Rendering ${totalFrames} frames`+ : `Rendering frames ${range.start}–${range.end - 1} of ${range.total}`,+ );// Render and encode frames using pipelined double-buffered DMA staging ring- for (let i = 0; i < totalFrames; i++) {+ for (let i = range.start; i < range.end; i++) {options?.signal?.throwIfAborted();if (ctx && surface && renderer && pipeline) {// Assets first used on a later frame (a LUT, a glyph) get the- // same pre-pass as frame 0, as renderImage and the preview do.+ // same pre-pass as the first frame, as renderImage and the+ // preview do.if (- i > 0 &&+ i > range.start &&(lutStore.hasPending(device) || SlugFontCache.hasPending())) {updateClockSignals(i, fps, durationMs);@@ -1366,8 +1569,10 @@ export class HeadlessWebGPURenderer {},);- await pipeline.enqueueFrame(i, targetTexture, encoder);- if (i + 1 < totalFrames) reporter.frameDone(i + 1);+ // Chunk-local index: the pipeline timestamps frames from it.+ const done = i - range.start + 1;+ await pipeline.enqueueFrame(done - 1, targetTexture, encoder);+ if (done < totalFrames) reporter.frameDone(done);} else {throw new Error("Invalid render context state");}@@ -1396,68 +1601,18 @@ export class HeadlessWebGPURenderer {}// ── Audio ──────────────────────────────────────────────────────────────- options?.signal?.throwIfAborted();- reporter.stage("audio", "Mixing audio");- const { channels, sampleRate } = await mixAudioTracks(- virtualMedia,- fps,- 48_000,- device,- renderId,- {- onProgress: ({ done, total, source }) =>- reporter.update(- total > 0- ? `${done}/${total} clips${source ? `: ${source}` : ""}`- : "no audio clips",- total > 0 ? done / total : undefined,- ),- },- );-- const numChannels = channels.length;- if (numChannels === 0)- throw new Error("mixAudioTracks returned no channels");-- if (qaOptions) audioQa = analyzeAudio(channels, sampleRate, qaOptions);-- const totalSamples = channels[0].length;- const chunkSize = Math.round(0.1 * sampleRate);- reporter.stage(- "audio-encode",- `Encoding ${(totalSamples / sampleRate).toFixed(1)}s of audio (${audioCodec})`,- );-- for (let offset = 0; offset < totalSamples; offset += chunkSize) {- const currentChunk = Math.min(chunkSize, totalSamples - offset);- // A fresh buffer per chunk: the encoder may read a sample's data after- // add() resolves, so a buffer reused across chunks gets overwritten by- // the next chunk before it is encoded (audible as 100 ms garbling).- const interleaved = new Float32Array(currentChunk * numChannels);-- for (let i = 0; i < currentChunk; i++) {- for (let ch = 0; ch < numChannels; ch++) {- interleaved[i * numChannels + ch] = channels[ch]?.[offset + i] ?? 0;- }- }-- const sample = new AudioSample({- data: interleaved,- format: "f32",- numberOfChannels: numChannels,- sampleRate,- timestamp: offset / sampleRate,+ if (audioSource) {+ options?.signal?.throwIfAborted();+ audioQa = await encodeAudioTrack({+ virtualMedia,+ fps,+ device,+ renderId,+ source: audioSource,+ codec: audioCodec,+ reporter,+ qaOptions,});- try {- await audioSource.add(sample);- } finally {- sample.close();- }- const written = offset + currentChunk;- reporter.update(- `${(written / sampleRate).toFixed(1)}s/${(totalSamples / sampleRate).toFixed(1)}s`,- written / totalSamples,- );}options?.signal?.throwIfAborted();diff --git a/packages/renderer/src/index.ts b/packages/renderer/src/index.tsindex 44c33604..44188693 100644--- a/packages/renderer/src/index.ts+++ b/packages/renderer/src/index.ts@@ -1,4 +1,5 @@export * from "./asset-preloader.js";+export * from "./chunk-plan.js";export * from "./dma-staging-ring.js";export * from "./dynamic-node-discovery.js";export * from "./headless-webgpu-renderer.js";@@ -6,5 +7,6 @@ export * from "./job-reset.js";export * from "./logger.js";export * from "./node-manifest.js";export type { RenderPhase, RenderProgress } from "./render-reporter.js";+export * from "./stitch.js";export * from "./video-qa.js";export * from "./zero-copy-webcodecs-pipeline.js";diff --git a/packages/renderer/src/stitch.ts b/packages/renderer/src/stitch.tsnew file mode 100644index 00000000..b975a847--- /dev/null+++ b/packages/renderer/src/stitch.ts@@ -0,0 +1,245 @@+/**+ * Joins the chunks of one render split across workers. Each chunk is made+ * with `renderVideo({ frames, audio: false })`, starts on a keyframe and+ * timestamps from 0; the stitch copies their encoded frames into one track,+ * shifted to follow each other, and copies in the audio track of a separate+ * audio-only render. Nothing is decoded or re-encoded, so it needs no GPU.+ */+import fs from "node:fs/promises";+import path from "node:path";+import {+ ALL_FORMATS,+ EncodedAudioPacketSource,+ type EncodedPacket,+ EncodedPacketSink,+ EncodedVideoPacketSource,+ FilePathSource,+ FilePathTarget,+ Input,+ type InputAudioTrack,+ type InputVideoTrack,+ Mp4OutputFormat,+ Output,+ type VideoCodec,+ WebMOutputFormat,+} from "mediabunny";++export interface StitchOptions {+ /** Chunk files in timeline order, each a `renderVideo({ frames, audio: false })` output. */+ chunks: readonly string[];+ /**+ * An audio-only render of the whole timeline (`renderVideo({ codec: "aac" })`,+ * or `"opus"` for WebM chunks), copied in as the audio track.+ */+ audio?: string;+ outputPath: string;+ /** Stops between packets; the promise rejects and no file is left behind. */+ signal?: AbortSignal;+}++export interface StitchResult {+ filePath: string;+ /** Video frames written, across all chunks. */+ frames: number;+ /** End of the video track, in seconds. */+ durationSec: number;+}++type VideoConfig = NonNullable<+ Awaited<ReturnType<InputVideoTrack["getDecoderConfig"]>>+>;+type AudioConfig = NonNullable<+ Awaited<ReturnType<InputAudioTrack["getDecoderConfig"]>>+>;++interface VideoChunk {+ file: string;+ input: Input;+ track: InputVideoTrack;+ codec: VideoCodec;+ config: VideoConfig;+}++export async function stitchChunks(+ options: StitchOptions,+): Promise<StitchResult> {+ if (options.chunks.length === 0)+ throw new Error("stitchChunks needs at least one chunk.");+ const inputs: Input[] = [];+ const open = (file: string) => {+ const input = new Input({+ source: new FilePathSource(file),+ formats: ALL_FORMATS,+ });+ inputs.push(input);+ return input;+ };+ let output: Output | undefined;+ try {+ const chunks = await Promise.all(+ options.chunks.map((file) => openVideoChunk(file, open(file))),+ );+ const [reference] = chunks as [VideoChunk, ...VideoChunk[]];+ for (const chunk of chunks.slice(1)) assertSameEncoding(reference, chunk);++ const webm = (await reference.input.getMimeType()).includes("webm");+ await fs.mkdir(path.dirname(path.resolve(options.outputPath)), {+ recursive: true,+ });+ output = new Output({+ format: webm ? new WebMOutputFormat() : new Mp4OutputFormat(),+ target: new FilePathTarget(options.outputPath),+ });+ const video = new EncodedVideoPacketSource(reference.codec);+ output.addVideoTrack(video);+ const audio = options.audio+ ? await AudioCopier.open(options.audio, open(options.audio), output)+ : undefined;++ await output.start();+ const written = await copyVideo(chunks, video, audio, options.signal);+ await audio?.writeUntil(Number.POSITIVE_INFINITY, options.signal);+ await output.finalize();+ output = undefined;+ return { filePath: options.outputPath, ...written };+ } catch (error) {+ if (output) {+ await output.cancel().catch(() => {});+ await fs.rm(options.outputPath, { force: true });+ }+ throw error;+ } finally {+ for (const input of inputs) input.dispose();+ }+}++async function openVideoChunk(file: string, input: Input): Promise<VideoChunk> {+ const track = await input.getPrimaryVideoTrack();+ if (!track) throw new Error(`Chunk ${file} has no video track.`);+ const codec = await track.getCodec();+ const config = await track.getDecoderConfig();+ if (!codec || !config)+ throw new Error(`Chunk ${file}: its video codec cannot be read.`);+ return { file, input, track, codec, config };+}++/**+ * Copied frames are decoded with the first chunk's codec configuration, so+ * every chunk must have been encoded with the same one.+ */+function assertSameEncoding(reference: VideoChunk, chunk: VideoChunk): void {+ const a = reference.config;+ const b = chunk.config;+ const differs =+ a.codec !== b.codec+ ? `codec ${b.codec}, not ${a.codec}`+ : a.codedWidth !== b.codedWidth || a.codedHeight !== b.codedHeight+ ? `size ${b.codedWidth}x${b.codedHeight}, not ${a.codedWidth}x${a.codedHeight}`+ : !sameBytes(a.description, b.description)+ ? "different encoder parameters (SPS/PPS)"+ : undefined;+ if (differs)+ throw new Error(+ `Chunk ${chunk.file} has ${differs} than ${reference.file}. Render every chunk with the same framefields version, codec and quality, on the same GPU type.`,+ );+}++function sameBytes(+ a: VideoConfig["description"],+ b: VideoConfig["description"],+): boolean {+ if (!a || !b) return a === b;+ const x = toBytes(a);+ const y = toBytes(b);+ return x.length === y.length && x.every((v, i) => v === y[i]);+}++function toBytes(data: NonNullable<VideoConfig["description"]>): Uint8Array {+ return ArrayBuffer.isView(data)+ ? new Uint8Array(data.buffer, data.byteOffset, data.byteLength)+ : new Uint8Array(data);+}++/** Writes every chunk's frames, each chunk starting where the last one ended. */+async function copyVideo(+ chunks: readonly VideoChunk[],+ video: EncodedVideoPacketSource,+ audio: AudioCopier | undefined,+ signal: AbortSignal | undefined,+): Promise<{ frames: number; durationSec: number }> {+ let offset = 0;+ let frames = 0;+ for (const chunk of chunks) {+ let first: number | undefined;+ let end = offset;+ for await (const packet of new EncodedPacketSink(chunk.track).packets()) {+ signal?.throwIfAborted();+ if (first === undefined) {+ if (packet.type !== "key")+ throw new Error(`Chunk ${chunk.file} does not start on a keyframe.`);+ first = packet.timestamp;+ }+ const timestamp = offset + packet.timestamp - first;+ // Interleaved: audio up to this frame goes in first.+ await audio?.writeUntil(timestamp, signal);+ await video.add(+ packet.clone({ timestamp }),+ frames === 0 ? { decoderConfig: chunk.config } : undefined,+ );+ frames++;+ end = Math.max(end, timestamp + packet.duration);+ }+ if (first === undefined)+ throw new Error(`Chunk ${chunk.file} has no video frames.`);+ offset = end;+ }+ return { frames, durationSec: offset };+}++/** Copies an audio track's packets in timestamp order, as far as asked. */+class AudioCopier {+ private readonly packets: AsyncGenerator<EncodedPacket, void>;+ /** The next packet to write; `undefined` until read, `null` at the end. */+ private next: EncodedPacket | null | undefined;+ private written = 0;++ private constructor(+ track: InputAudioTrack,+ private readonly config: AudioConfig,+ private readonly source: EncodedAudioPacketSource,+ ) {+ this.packets = new EncodedPacketSink(track).packets();+ }++ static async open(+ file: string,+ input: Input,+ output: Output,+ ): Promise<AudioCopier> {+ const track = await input.getPrimaryAudioTrack();+ if (!track) throw new Error(`Audio file ${file} has no audio track.`);+ const config = await track.getDecoderConfig();+ const codec = await track.getCodec();+ if (!config || !codec)+ throw new Error(`Audio file ${file}: its codec cannot be read.`);+ const source = new EncodedAudioPacketSource(codec);+ output.addAudioTrack(source);+ return new AudioCopier(track, config, source);+ }++ async writeUntil(timestamp: number, signal?: AbortSignal): Promise<void> {+ for (;;) {+ if (this.next === undefined)+ this.next = (await this.packets.next()).value ?? null;+ const packet = this.next;+ if (!packet || packet.timestamp > timestamp) return;+ signal?.throwIfAborted();+ await this.source.add(+ packet,+ this.written === 0 ? { decoderConfig: this.config } : undefined,+ );+ this.written++;+ this.next = undefined;+ }+ }+}diff --git a/packages/renderer/src/tests/chunk-plan.test.ts b/packages/renderer/src/tests/chunk-plan.test.tsnew file mode 100644index 00000000..917d6205--- /dev/null+++ b/packages/renderer/src/tests/chunk-plan.test.ts@@ -0,0 +1,120 @@+import { describe, expect, it } from "vitest";+import { type ChunkPlan, planChunks } from "../chunk-plan.js";++const FPS = 30;+/** Renders as fast as it plays: 33.3 ms per frame. */+const realtime = [{ frame: 0, ms: 1000 / FPS }];++function expectCovers(plan: ChunkPlan, total: number) {+ expect(plan.chunks[0].start).toBe(0);+ expect(plan.chunks.at(-1)?.end).toBe(total);+ for (let i = 1; i < plan.chunks.length; i++)+ expect(plan.chunks[i].start).toBe(plan.chunks[i - 1].end);+ expect(plan.chunks).toHaveLength(plan.workers);+}++describe("planChunks", () => {+ it("keeps one worker by default", () => {+ const total = 3 * 3600 * FPS;+ const plan = planChunks({ totalFrames: total, samples: realtime });+ expect(plan.workers).toBe(1);+ expectCovers(plan, total);+ expect(plan.workSec).toBeCloseTo(3 * 3600, 0);+ });++ it("auto: a five-minute film stays on one worker", () => {+ const plan = planChunks({+ totalFrames: 5 * 60 * FPS,+ samples: realtime,+ workers: "auto",+ });+ expect(plan.workers).toBe(1);+ });++ it("auto: a three-hour film spreads to maxWorkers", () => {+ const total = 3 * 3600 * FPS;+ const plan = planChunks({+ totalFrames: total,+ samples: realtime,+ workers: "auto",+ });+ // 10 800 s of work over chunks of at least 510 s would be 21; capped at 10.+ expect(plan.workers).toBe(10);+ expectCovers(plan, total);+ expect(plan.etaSec).toBeCloseTo(90 + 1080, 0);+ });++ it("auto: adds workers only while each chunk stays efficient", () => {+ // 30 min of work; 510 s minimum per chunk at 85% → 3 workers.+ const plan = planChunks({+ totalFrames: 30 * 60 * FPS,+ samples: realtime,+ workers: "auto",+ maxWorkers: 50,+ });+ expect(plan.workers).toBe(3);+ for (const chunk of plan.chunks)+ expect(90 / (90 + chunk.workSec)).toBeLessThanOrEqual(0.15 + 1e-9);+ });++ it("cuts at equal cost, so a heavy stretch gets a shorter chunk", () => {+ const total = 1000;+ const plan = planChunks({+ totalFrames: total,+ samples: [+ { frame: 0, ms: 10 },+ { frame: 499, ms: 10 },+ { frame: 500, ms: 30 },+ { frame: 999, ms: 30 },+ ],+ workers: 2,+ minChunkFrames: 1,+ });+ expectCovers(plan, total);+ // Half the cost (10 s of 20 s) is reached at frame ~667.+ expect(plan.chunks[0].end).toBeGreaterThan(650);+ expect(plan.chunks[0].end).toBeLessThan(680);+ expect(plan.chunks[0].workSec).toBeCloseTo(plan.chunks[1].workSec, 0);+ });++ it("never makes a chunk shorter than minChunkFrames", () => {+ const plan = planChunks({+ totalFrames: 250,+ samples: [+ { frame: 0, ms: 1000 },+ { frame: 10, ms: 1 },+ ],+ workers: 4,+ minChunkFrames: 60,+ });+ expect(plan.workers).toBe(4);+ expectCovers(plan, 250);+ for (const chunk of plan.chunks)+ expect(chunk.end - chunk.start).toBeGreaterThanOrEqual(60);+ });++ it("never plans more chunks than frames allow", () => {+ const plan = planChunks({ totalFrames: 90, samples: realtime, workers: 8 });+ expect(plan.workers).toBe(1);+ });++ it("rejects bad input", () => {+ expect(() => planChunks({ totalFrames: 0, samples: realtime })).toThrow(+ /totalFrames/,+ );+ expect(() => planChunks({ totalFrames: 100, samples: [] })).toThrow(+ /sample/,+ );+ expect(() =>+ planChunks({ totalFrames: 100, samples: realtime, workers: 0 }),+ ).toThrow(/workers/);+ expect(() =>+ planChunks({+ totalFrames: 100,+ samples: realtime,+ workers: "auto",+ efficiency: 1,+ }),+ ).toThrow(/efficiency/);+ });+});diff --git a/plugins/framefields/skills/framefields-render/SKILL.md b/plugins/framefields/skills/framefields-render/SKILL.mdindex 94f8f21e..6eefe70f 100644--- a/plugins/framefields/skills/framefields-render/SKILL.md+++ b/plugins/framefields/skills/framefields-render/SKILL.md@@ -168,6 +168,40 @@ await result.cleanup?.();Pass `qa: true` to have the engine check the output while it renders (see **Built-in Video QA** below) and return the findings as `result.qa`.+### Splitting a long render into chunks+To render on several GPUs at once, have each process render a range of frames without audio, render the audio once, then join the files without re-encoding:++```typescript+import { stitchChunks } from "framefields/renderer";++// On each worker (its own process and composition): frames start..end-1.+await comp.renderVideo({ outputPath: "out/chunk-0.mp4", frames: { start: 0, end: 900 }, audio: false });+// Once, anywhere: the whole timeline's audio.+await comp.renderVideo({ outputPath: "out/audio.mp4", codec: "aac" });+// On any machine (no GPU needed): chunks in timeline order.+await stitchChunks({ chunks: ["out/chunk-0.mp4", "out/chunk-1.mp4"], audio: "out/audio.mp4", outputPath: "out/final.mp4" });+```++- Trails, datamosh and particles rebuild their state before `start` by replaying earlier frames, so a chunk starting late costs extra time but renders the same picture.+- Render every chunk with the same framefields version, codec and quality on the same GPU type. `stitchChunks` refuses chunks with a different codec, size or encoder parameters.+- SAM 2 / SAM 3 tracks run in order from their first prompted frame. Run the vision analysis once and point every worker at the shared cache (`FRAMEFIELDS_VISION_CACHE_DIR`), so no chunk re-tracks the frames before it.+- `qa` checks each chunk on its own. Check the stitched file with ffmpeg (below).+- `planChunks({ totalFrames, samples, workers: "auto" })` picks the chunk count and cuts: it adds workers only while each chunk spends at least 85% of its time rendering (about 5.7× the per-worker start-up), and splits at equal render cost. `renderer.probeRender(comp)` measures the `samples`.++## Rendering in the Cloud (`framefields render --cloud`)++api.framefields.dev renders the project on framefields' GPUs and splits a long film across several. It fetches the project from git, so it renders the **pushed commit**: commit and push first; uncommitted changes are not rendered (the CLI warns), and an unpushed commit is refused. It bills the account behind `FRAMEFIELDS_API_KEY`; if authentication is required, ask the user to set it in the process environment and never print or embed it.++```bash+npx framefields render src/film.ts --cloud --workers auto --plan # workers, time and estimated cost; renders nothing+npx framefields render src/film.ts --cloud --workers auto # render, then download to output/film.mp4+```++- `--workers` defaults to 1 (one GPU, no planning step). `auto` measures frame costs on one worker, then uses as many workers as stay efficient: a 5-minute film stays on one, a 3-hour film spreads to `--max-workers` (API cap 10). A number splits into that many.+- Show the user `--plan` before a long or expensive render: it prints the estimated cost.+- Ctrl-C cancels the render; GPU time already used is charged. Failures print the worker's error code (e.g. `ASSET_NOT_FOUND` for an asset left out of LFS).+- From code: `renderCloud({ entry, workers: "auto", outputPath, onProgress })` from `framefields/cloud` (no GPU needed locally). The HTTP API is `POST /v1/render` (`workers`, `max_workers`, `dry_run`), `GET` and `DELETE /v1/render/:id`.+---## Rendering Without a GPUdiff --git a/plugins/framefields/skills/framefields/SKILL.md b/plugins/framefields/skills/framefields/SKILL.mdindex 231c16ba..52d28ebd 100644--- a/plugins/framefields/skills/framefields/SKILL.md+++ b/plugins/framefields/skills/framefields/SKILL.md@@ -225,10 +225,13 @@ Add scripts to `package.json`:"frames": "tsx src/render.ts frames","grid": "tsx src/render.ts grid","preview": "tsx src/render.ts preview",- "render": "tsx src/render.ts"+ "render": "tsx src/render.ts",+ "render:cloud": "framefields render src/film.ts --cloud --workers auto"}```+`render:cloud` renders on framefields' GPUs (api.framefields.dev) from the pushed commit, with the same `FRAMEFIELDS_API_KEY` as the AI gateway; `framefields-render` covers it.+### Video assets & Git LFSVideo inputs (`Layer.video("assets/footage/clip.mp4")`), Lottie animations (`Layer.lottie(...)`), audio tracks (`Layer.audio(...)`), LUTs, and custom fonts belong in **Git LFS**, while render outputs in `output/` belong in `.gitignore`.