Code-first video & image, rendered natively on WebGPU.
Ship the preview engine with the package; opening a browser is opt-in
okanasl603e73940a41
diff --git a/README.md b/README.mdindex 194ffb91..1b1a8adb 100644--- a/README.md+++ b/README.md@@ -263,6 +263,7 @@ Project normalized landmarks to screen space with a configurable camera FOV, the- **Runs your composition, not a video** — `startPreview({ entry, export })` serves a localhost WebGPU player that loads the composition's own module and renders every frame live in the browser. Nothing is streamed: the server only hands over the bundle, the project's assets, and the soundtrack mixed by the export engine.- **Timeline, waveform & frame stepping** — play/pause, scrub, step frame by frame, and read resolution, FPS, duration, and audio status at a glance.- **One stable URL per project** — the port is derived from the working directory, so re-running the preview replaces the running server and any open tab reloads into the new version by itself. Close the tab and the server shuts down about five seconds later.+- **Shown where you are** — `startPreview` serves the page and returns its URL instead of opening a browser, so an agent can show it in its own pane (Claude Code, Codex); pass `open: true` to open the system browser.```typescriptimport { startPreview } from "gitframes";diff --git a/examples/19_gitframes_film/src/render.ts b/examples/19_gitframes_film/src/render.tsindex bcbefeb2..53e4994e 100644--- a/examples/19_gitframes_film/src/render.ts+++ b/examples/19_gitframes_film/src/render.ts@@ -1,6 +1,6 @@/*** pnpm render → output/gitframes-film.mp4 (30 s, 1080p30, with score)- * pnpm render preview → live player on localhost; stops when its tab closes+ * pnpm render preview → live player on localhost; add --open for the system browser* pnpm render frames 0 45 90 → output/frames/f0000.png … for quick visual checks*/import fs from "node:fs/promises";@@ -16,7 +16,7 @@ if (mode === "preview") {// The page runs film.ts itself and renders with WebGPU in the browser.const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },- { title: "gitframes film" },+ { title: "gitframes film", open: args.includes("--open") },);console.log(`Preview at ${session.url}`);// Serves until its tab closes, or the next `pnpm render preview` takes over.diff --git a/examples/21_full_circle/src/render.ts b/examples/21_full_circle/src/render.tsindex db083579..49ba8cca 100644--- a/examples/21_full_circle/src/render.ts+++ b/examples/21_full_circle/src/render.ts@@ -1,6 +1,6 @@/*** pnpm render → output/full-circle.mp4 (with score)- * pnpm render preview → live player on localhost; stops when its tab closes+ * pnpm render preview → live player on localhost; add --open for the system browser* pnpm render frames 0 140 637 → output/frames/f0000.png … for visual checks* pnpm render sheet 0 1080 24 → output/frames/ every 24th frame from 0 to 1080*/@@ -16,7 +16,7 @@ if (mode === "preview") {// The page runs film.ts itself and renders with WebGPU in the browser.const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },- { title: "Full circle" },+ { title: "Full circle", open: args.includes("--open") },);console.log(`Preview at ${session.url}`);// Serves until its tab closes, or the next `pnpm render preview` takes over.diff --git a/examples/22_gitframes_launch/src/render.ts b/examples/22_gitframes_launch/src/render.tsindex b929a882..750bff82 100644--- a/examples/22_gitframes_launch/src/render.ts+++ b/examples/22_gitframes_launch/src/render.ts@@ -1,6 +1,6 @@/*** pnpm render → output/gitframes-launch.mp4 (with the score) + QA report; exits 1 if QA fails- * pnpm render preview → live player on localhost; stops when its tab closes, re-running replaces it+ * pnpm render preview → live player on localhost; add --open for the system browser* pnpm render frames 0 80 320 → output/frames/f0000.png … for visual checks* pnpm render sheet 0 320 20 → every 20th frame from 0 to 320, plus a contact sheet* pnpm render:src … → any of the above, run on the engine's source (no package builds)@@ -17,7 +17,7 @@ if (mode === "preview") {// The page runs film.ts itself and renders with WebGPU in the browser.const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },- { title: "gitframes launch" },+ { title: "gitframes launch", open: args.includes("--open") },);console.log(`Preview at ${session.url}`);// Serves until its tab closes, or the next `pnpm render preview` takes over.diff --git a/examples/23_data_story/src/render.ts b/examples/23_data_story/src/render.tsindex 46e9baf6..e079ab38 100644--- a/examples/23_data_story/src/render.ts+++ b/examples/23_data_story/src/render.ts@@ -2,7 +2,7 @@* pnpm render → output/data-story.mp4* pnpm render frames 60 180 → output/frames/f0060.png …* pnpm render grid → output/grid.png, a contact sheet of the whole film- * pnpm render preview → live player on localhost; stops when its tab closes, re-running replaces it+ * pnpm render preview → live player on localhost; add --open for the system browser*/import fs from "node:fs/promises";import path from "node:path";@@ -17,7 +17,7 @@ if (mode === "preview") {// The page runs film.ts itself and renders with WebGPU in the browser.const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },- { title: "Data story" },+ { title: "Data story", open: args.includes("--open") },);console.log(`Preview at ${session.url}`);// Serves until its tab closes, or the next `pnpm render preview` takes over.diff --git a/packages/gitframes/README.md b/packages/gitframes/README.mdindex 3745b464..a2661bf2 100644--- a/packages/gitframes/README.md+++ b/packages/gitframes/README.md@@ -69,7 +69,7 @@ comp.add(- **Unified 3D Scene Graph**: Position 2D surfaces in 3D coordinate space with 3D cameras, lights, and OBJ/FBX/glTF models.- **Audio DSP & SFX**: Procedural sound effects (impact, whoosh, riser), audio stem analysis, and audio-reactive signals.- **Headless GPU Video Rendering**: Direct hardware encoding via `@mediabunny/server` / WebCodecs without headless browser overhead.-- **Live Preview**: `startPreview({ entry, export })` opens a localhost player (soundtrack, waveform timeline, frame stepping). The page runs the composition's own code and renders it with WebGPU in the browser, at or near full frame rate. Re-running it replaces the running preview, and an open tab reloads into the new version.+- **Live Preview**: `startPreview({ entry, export })` opens a localhost player (soundtrack, waveform timeline, frame stepping). The page runs the composition's own code and renders it with WebGPU in the browser, at or near full frame rate. It serves the page and returns the URL, so an agent can show it in its own browser pane (`open: true` opens the system browser instead). Re-running it replaces the running preview, and an open tab reloads into the new version.---diff --git a/packages/gitframes/src/preview/bundle.ts b/packages/gitframes/src/preview/bundle.tsindex 62cacae4..9ddf6b41 100644--- a/packages/gitframes/src/preview/bundle.ts+++ b/packages/gitframes/src/preview/bundle.ts@@ -1,29 +1,56 @@/**- * Bundles a project's composition module, the engine and the browser player- * into one script, so the page runs the composition code itself and renders- * with WebGPU. One bundle, so the user's code and the engine share the same- * signal and clock instances.+ * The preview page runs the project's composition code and renders it with+ * WebGPU. Two bundles make that work:*- * Node APIs that compositions commonly use at build time are shimmed: `path`,- * `url`, `import.meta.url` (each file keeps its own `file://` URL), and the- * synchronous `fs` reads, which fetch the file from the preview server.- * Absolute file paths double as URLs on that server, so assets referenced by- * path load unchanged. Node-only packages become stubs that throw when used.+ * - The engine (`buildEngine`): gitframes and its browser player, built once+ * for the browser. Published packages ship it in `dist/preview-engine`;+ * running from source, it is built on first use.+ * - The project (`bundleProject`): only the user's own files, built on each+ * preview. Its `gitframes` imports point at the engine's files, so the+ * composition and the player share one engine (signals, clock, caches).+ *+ * Node APIs that compositions use at build time are shimmed: `path`, `url`,+ * `os`, `import.meta.url` (each file keeps its own `file://` URL), and `fs`+ * reads, which fetch the file from the preview server. Absolute file paths+ * double as URLs there, so assets referenced by path load unchanged.+ * Node-only packages become stubs that throw when used.*/import fs from "node:fs";-import { createRequire } from "node:module";+import { createRequire, isBuiltin } from "node:module";import os from "node:os";import path from "node:path";import { fileURLToPath, pathToFileURL } from "node:url";-import type { Plugin } from "esbuild";+import type { BuildOptions, PluginBuild } from "esbuild";const HERE = path.dirname(fileURLToPath(import.meta.url));// Spelled in two parts so this file survives its own rewrite.const META_URL = ["import", "meta", "url"].join(".");const require = createRequire(import.meta.url);+/** URL prefix the server serves the engine's files under. */+export const ENGINE_URL = "/@gitframes/engine";++/** Public entry points a composition may import, and their sources. */+const ENTRIES: Record<string, string> = {+ gitframes: "index.ts",+ "gitframes/effects": "effects/index.ts",+ "gitframes/audio": "audio/index.ts",+ "gitframes/signals": "signals/index.ts",+ "gitframes/fonts": "fonts/index.ts",+};++/** Node built-ins with a browser version (shim file names); others are stubbed. */+const SHIMS: Record<string, string> = {+ path: "path",+ "path/posix": "path",+ url: "url",+ os: "os",+ fs: "fs",+ "fs/promises": "fs-promises",+};+/** Packages that only work in Node; the browser never needs them to draw. */-const NODE_ONLY = [+const NODE_ONLY = new Set(["sharp","skia-canvas","node-av",@@ -36,10 +63,12 @@ const NODE_ONLY = ["pino","pino-pretty","dotenv",- "@gitframes/renderer","esbuild",+ "@gitframes/renderer","@gitframes/server-utils",-];+ "gitframes/renderer",+ "gitframes/preview",+]);// Reported as an ES module so named imports (`import { homedir } from "os"`)// read through the proxy instead of being copied off it.@@ -52,124 +81,109 @@ const stub = () => new Proxy(function () {}, {module.exports = stub();`;-const SHIMS: Record<string, string> = {- path: "path.ts",- "path/posix": "path.ts",- url: "url.ts",- os: "os.ts",- fs: "fs.ts",- "fs/promises": "fs-promises.ts",-};+/**+ * Builds gitframes for the browser into `outDir`: one module per public entry,+ * the player and the shims, with shared code split into chunks (node renderers+ * load only when a composition uses them).+ */+export async function buildEngine(outDir: string): Promise<void> {+ const { build } = await import("esbuild");+ const src = path.resolve(HERE, "..");+ const shims = path.join(HERE, "shims");+ const entryPoints: Record<string, string> = {+ player: path.join(HERE, "player.ts"),+ };+ for (const [spec, file] of Object.entries(ENTRIES)) {+ entryPoints[engineName(spec)] = path.join(src, file);+ }+ for (const name of new Set(Object.values(SHIMS))) {+ entryPoints[`shims/${name}`] = path.join(shims, `${name}.ts`);+ }-export interface BundleResult {+ await fs.promises.rm(outDir, { recursive: true, force: true });+ await build({+ ...BROWSER,+ entryPoints,+ outdir: outDir,+ outExtension: { ".js": ".mjs" },+ splitting: true,+ minify: true,+ conditions: ["development", "browser", "import"],+ define: { "process.env.NODE_ENV": '"production"', global: "globalThis" },+ inject: [path.join(shims, "process.ts")],+ plugins: [+ {+ name: "gitframes-engine",+ setup(b) {+ b.onResolve({ filter: /^gitframes-preview:renderers$/ }, () => ({+ path: builtinRenderersModule(),+ }));+ resolveNode(b, (name) => ({ path: path.join(shims, `${name}.ts`) }));+ },+ },+ ],+ });+}++export interface ProjectBundle {code: string;- /** Source files the bundle was built from, to watch for changes. */+ /** The project's source files, to watch for changes. */inputs: string[];}-export async function bundlePlayer(options: {+/** Bundles the module that builds the composition, and what it imports. */+export async function bundleProject(options: {/** Module that exports the composition factory. */entry: string;/** Name of the export; a function returning a Composition (or one itself). */exportName: string;-}): Promise<BundleResult> {+ /** Where cached vision models are, and where onnxruntime-web loads from. */+ modelsDir: string;+ ortBase: string;+}): Promise<ProjectBundle> {const { build } = await import("esbuild");const entry = path.resolve(options.entry);- const player = resolveOwn("player");- const renderers = builtinRenderersModule();- const shimDir = path.join(path.dirname(player), "shims");- const vision = visionAssets();-- const plugin: Plugin = {- name: "gitframes-preview",- setup(b) {- b.onResolve({ filter: /^gitframes-preview:entry$/ }, () => ({- path: "entry",- namespace: "gitframes-preview",- }));- b.onLoad({ filter: /.*/, namespace: "gitframes-preview" }, () => ({- contents: [- `import * as mod from ${JSON.stringify(entry)};`,- `import { startPlayer } from ${JSON.stringify(player)};`,- `import { BUILTIN_NODE_RENDERERS } from ${JSON.stringify(renderers)};`,- `startPlayer(mod, ${JSON.stringify(options.exportName)}, {`,- " renderers: BUILTIN_NODE_RENDERERS,",- // Vision nodes run on onnxruntime-web; its .wasm files come from the server.- " loadOrt: async () => {",- ' const ort = await import("onnxruntime-web/webgpu");',- ` ort.env.wasm.wasmPaths = ${JSON.stringify(`/@fs${vision.ortDir}/`)};`,- ' ort.env.logLevel = "error";',- " return ort;",- " },",- "});",- ].join("\n"),- resolveDir: path.dirname(entry),- loader: "ts",- }));-- // onnxruntime-web is the vision package's (optional) dependency.- b.onResolve({ filter: /^onnxruntime-web(\/|$)/ }, (args) => {- if (args.pluginData?.fromVision) return undefined;- return b.resolve(args.path, {- kind: args.kind,- resolveDir: vision.packageDir,- pluginData: { fromVision: true },- });- });-- b.onResolve({ filter: /^(node:)?[a-z_/]+$/ }, (args) => {- const name = args.path.replace(/^node:/, "");- if (SHIMS[name]) return { path: resolveShim(shimDir, SHIMS[name]) };- if (args.path.startsWith("node:") || isBuiltin(name))- return { path: name, namespace: "gitframes-stub" };- return undefined;- });- b.onResolve({ filter: /.*/ }, (args) => {- const pkg = packageName(args.path);- if (pkg && NODE_ONLY.includes(pkg))- return { path: args.path, namespace: "gitframes-stub" };- return undefined;- });- b.onLoad({ filter: /.*/, namespace: "gitframes-stub" }, () => ({- contents: STUB,- loader: "js",- }));-- // Each file keeps its own URL, as `fileURLToPath(import.meta.url)`- // expects: the expression becomes a constant declared per file.- b.onLoad({ filter: /\.[cm]?[jt]sx?$/ }, async (args) => {- if (args.path.includes(`${path.sep}node_modules${path.sep}`))- return undefined;- const source = await fs.promises.readFile(args.path, "utf8");- if (!source.includes(META_URL)) return undefined;- return {- contents: `const __gitframesFileUrl = ${JSON.stringify(pathToFileURL(args.path).href)};\n${source.replaceAll(META_URL, "__gitframesFileUrl")}`,- loader: loaderFor(args.path),- };- });- },- };+ const fromEngine = (file: string) => ({+ path: `${ENGINE_URL}/${file}.mjs`,+ external: true,+ });+ const settings = { modelsDir: options.modelsDir, ortBase: options.ortBase };const result = await build({+ ...BROWSER,entryPoints: ["gitframes-preview:entry"],- bundle: true,write: false,- format: "esm",- platform: "browser",- target: "es2022",- conditions: ["development", "browser", "import"],- mainFields: ["browser", "module", "main"],metafile: true,sourcemap: "inline",- logLevel: "silent",- define: {- "process.env.NODE_ENV": '"development"',- // Cached models load from the server, which serves this folder.- "process.env.GITFRAMES_MODELS_DIR": JSON.stringify(vision.modelsDir),- global: "globalThis",- },- inject: [resolveShim(shimDir, "process.ts")],- plugins: [plugin],+ plugins: [+ {+ name: "gitframes-project",+ setup(b) {+ b.onResolve({ filter: /^gitframes-preview:entry$/ }, () => ({+ path: "entry",+ namespace: "gitframes-preview",+ }));+ b.onLoad({ filter: /.*/, namespace: "gitframes-preview" }, () => ({+ contents: [+ `import { startPlayer } from ${JSON.stringify(`${ENGINE_URL}/player.mjs`)};`,+ `import * as mod from ${JSON.stringify(entry)};`,+ `startPlayer(mod, ${JSON.stringify(options.exportName)}, ${JSON.stringify(settings)});`,+ ].join("\n"),+ resolveDir: path.dirname(entry),+ loader: "js",+ }));+ b.onResolve({ filter: /^\/@gitframes\/engine\// }, (args) => ({+ path: args.path,+ external: true,+ }));+ b.onResolve({ filter: /^gitframes(\/|$)/ }, (args) =>+ ENTRIES[args.path] ? fromEngine(engineName(args.path)) : undefined,+ );+ resolveNode(b, (name) => fromEngine(`shims/${name}`));+ rewriteMetaUrl(b);+ },+ },+ ],});const inputs = Object.keys(result.metafile?.inputs ?? {})@@ -178,45 +192,129 @@ export async function bundlePlayer(options: {return { code: result.outputFiles[0].text, inputs };}-function resolveOwn(name: string): string {- // The player is bundled from source: next to this file in development,- // and from the package's src/ when this runs from dist/.- for (const dir of [HERE, path.join(HERE, "..", "src", "preview")]) {- const file = path.join(dir, `${name}.ts`);- if (fs.existsSync(file)) return file;- }- throw new Error(- `[gitframes] The browser preview needs the gitframes sources (src/preview/${name}.ts), which this install does not include.`,- );+let engineBuild: Promise<string> | undefined;++/**+ * The prebuilt engine shipped beside a compiled bundle, if there is one. The+ * bundle lands either at the package's dist root or beside dist/preview,+ * depending on how the bundler splits it, so both are checked.+ */+export function prebuiltEngineDir(here: string): string | undefined {+ return [here, path.join(here, "..")]+ .map((base) => path.join(base, "preview-engine"))+ .find((dir) => fs.existsSync(path.join(dir, "player.mjs")));}/**- * Where vision models are cached (as `@gitframes/vision` resolves it), and the- * onnxruntime-web build that runs them in the browser.+ * The engine to serve: prebuilt in the package's dist, or, running from+ * source, built once per process (so engine edits show without a rebuild).*/-export function visionAssets(): {- modelsDir: string;- packageDir: string;- ortDir: string;-} {- const modelsDir = process.env.GITFRAMES_MODELS_DIR+export function engineDir(): Promise<string> {+ // From source HERE is src/preview, so this misses and the engine is built.+ const prebuilt = prebuiltEngineDir(HERE);+ if (prebuilt) {+ return Promise.resolve(prebuilt);+ }+ engineBuild ??= (async () => {+ const dir = path.join(+ os.tmpdir(),+ `gitframes-preview-engine-${process.pid}`,+ );+ await buildEngine(dir);+ return dir;+ })();+ engineBuild.catch(() => {+ engineBuild = undefined;+ });+ return engineBuild;+}++/** Where vision models are cached, as `@gitframes/vision` resolves it. */+export function visionModelsDir(): string {+ return process.env.GITFRAMES_MODELS_DIR? path.resolve(process.env.GITFRAMES_MODELS_DIR): path.join(os.homedir(), ".cache", "gitframes", "models");- const packageDir = path.dirname(- path.dirname(require.resolve("@gitframes/vision")),- );- let ortDir = "";- try {- ortDir = path.dirname(- createRequire(path.join(packageDir, "package.json")).resolve(+}++const ORT_WEB_VERSION = "1.30.0";++/**+ * onnxruntime-web, for vision effects: the project's own install when it has+ * one (served from disk), otherwise a pinned CDN build. Loaded only when a+ * vision effect runs, so gitframes doesn't ship its WebAssembly.+ */+export function onnxRuntimeWeb(fromDir: string): {+ base: string;+ dir?: string;+} {+ for (const from of [fromDir, HERE]) {+ try {+ const main = createRequire(path.join(from, "noop.js")).resolve("onnxruntime-web",- ),- );- } catch {}- return { modelsDir, packageDir, ortDir };+ );+ const dir = path.dirname(main);+ return { base: `/@fs${dir}/`, dir };+ } catch {}+ }+ return {+ base: `https://cdn.jsdelivr.net/npm/onnxruntime-web@${ORT_WEB_VERSION}/dist/`,+ };}-/** The generated list of node renderers, imported by package name. */+// ── shared ──++const BROWSER: BuildOptions = {+ bundle: true,+ format: "esm",+ platform: "browser",+ target: "es2022",+ mainFields: ["browser", "module", "main"],+ logLevel: "silent",+};++function engineName(spec: string): string {+ return spec === "gitframes" ? "gitframes" : spec.slice("gitframes/".length);+}++/** Node built-ins to shims, the rest of Node to stubs. */+function resolveNode(+ b: PluginBuild,+ shim: (name: string) => { path: string; external?: boolean },+): void {+ b.onResolve({ filter: /^(node:)?[a-z_/]+$/ }, (args) => {+ const name = args.path.replace(/^node:/, "");+ if (SHIMS[name]) return shim(SHIMS[name]);+ if (args.path.startsWith("node:") || isBuiltin(name))+ return { path: name, namespace: "gitframes-stub" };+ return undefined;+ });+ b.onResolve({ filter: /.*/ }, (args) => {+ const pkg = packageName(args.path);+ if (pkg && (NODE_ONLY.has(pkg) || NODE_ONLY.has(args.path)))+ return { path: args.path, namespace: "gitframes-stub" };+ return undefined;+ });+ b.onLoad({ filter: /.*/, namespace: "gitframes-stub" }, () => ({+ contents: STUB,+ loader: "js",+ }));+}++/** Each project file keeps its own URL, as `fileURLToPath(import.meta.url)` expects. */+function rewriteMetaUrl(b: PluginBuild): void {+ b.onLoad({ filter: /\.[cm]?[jt]sx?$/ }, async (args) => {+ if (args.path.includes(`${path.sep}node_modules${path.sep}`))+ return undefined;+ const source = await fs.promises.readFile(args.path, "utf8");+ if (!source.includes(META_URL)) return undefined;+ return {+ contents: `const __gitframesFileUrl = ${JSON.stringify(pathToFileURL(args.path).href)};\n${source.replaceAll(META_URL, "__gitframesFileUrl")}`,+ loader: loaderFor(args.path),+ };+ });+}++/** The generated list of node renderers (needed only to build the engine). */function builtinRenderersModule(): string {const dist = require.resolve("@gitframes/renderer");const file = path.join(@@ -231,20 +329,6 @@ function builtinRenderersModule(): string {return file;}-function resolveShim(dir: string, file: string): string {- const ts = path.join(dir, file);- if (fs.existsSync(ts)) return ts;- return ts.replace(/\.ts$/, ".mjs");-}--function isBuiltin(name: string): boolean {- try {- return require("node:module").isBuiltin(name);- } catch {- return false;- }-}-function packageName(spec: string): string | undefined {if (spec.startsWith(".") || spec.startsWith("/")) return undefined;const parts = spec.split("/");diff --git a/packages/gitframes/src/preview/index.ts b/packages/gitframes/src/preview/index.tsindex b5e506ea..7b80bdcb 100644--- a/packages/gitframes/src/preview/index.ts+++ b/packages/gitframes/src/preview/index.ts@@ -8,7 +8,13 @@ import { fileURLToPath, pathToFileURL } from "node:url";import type { VirtualMediaData } from "@gitframes/core";import { encodeStereoWav } from "../audio/index.js";import { HeadlessMediaRenderer } from "../renderer/index.js";-import { bundlePlayer, visionAssets } from "./bundle.js";+import {+ bundleProject,+ ENGINE_URL,+ engineDir,+ onnxRuntimeWeb,+ visionModelsDir,+} from "./bundle.js";import { previewPage } from "./page.js";/** Where the composition comes from: the module that builds it. */@@ -43,8 +49,9 @@ export interface PreviewOptions {/** Interface to bind. Defaults to "127.0.0.1" so the preview never leaves the machine. */host?: string;/**- * Open the page in the default browser. Defaults to true unless `CI` is set.- * Skipped when a tab from an earlier preview reconnects on its own.+ * Also open the page in the system's default browser. Off by default: the+ * caller shows the URL where it suits, e.g. an agent app's own browser+ * pane. Skipped when a tab from an earlier preview reconnects on its own.*/open?: boolean;/** Title shown in the page header. */@@ -114,9 +121,11 @@ export async function startPreview(: path.resolve(String(source.entry));const exportName = source.export ?? "default";const root = path.resolve(options.root ?? repoRoot(path.dirname(entry)));- // Vision nodes load their cached models and onnxruntime-web's .wasm files.- const vision = visionAssets();- const roots = [root, vision.modelsDir, vision.ortDir].filter(Boolean);+ // Vision nodes load their cached models, and onnxruntime-web's files when+ // the project has it installed.+ const modelsDir = visionModelsDir();+ const ort = onnxRuntimeWeb(path.dirname(entry));+ const roots = [root, modelsDir, ort.dir].filter((r): r is string => !!r);const sseClients = new Set<http.ServerResponse>();const idleCloseMs = options.idleCloseMs ?? 5000;let idleTimer: NodeJS.Timeout | undefined;@@ -125,14 +134,27 @@ export async function startPreview(const target = await loadTarget(entry, exportName);const meta = await resolveMeta(target, title, session);+ // The project's own code, bundled against the browser engine; the engine+ // itself is prebuilt (or built once, running from source).let bundle: Promise<string> | undefined;const playerBundle = () => {- bundle ??= bundlePlayer({ entry, exportName }).then((b) => b.code);- bundle.catch(() => {- bundle = undefined;- });+ if (!bundle) {+ const built = bundleProject({+ entry,+ exportName,+ modelsDir,+ ortBase: ort.base,+ }).then((b) => b.code);+ bundle = built;+ built.catch(() => {+ // Let a later request try again, unless it started its own build.+ if (bundle === built) bundle = undefined;+ });+ }return bundle;};+ const engine = engineDir();+ engine.catch(() => {});// ── Audio: mixed once, by the export engine ──let audio: AudioStatus = {@@ -227,9 +249,23 @@ export async function startPreview(sendJson(res, 200, { ...meta, audio });return;}+ if (req.method === "GET" && url.pathname.startsWith(`${ENGINE_URL}/`)) {+ const dir = await engine;+ const file = path.resolve(+ dir,+ `.${url.pathname.slice(ENGINE_URL.length)}`,+ );+ if (isInside(dir, file) && (await isFile(file))) {+ await sendFile(req, res, file);+ return;+ }+ sendJson(res, 404, { error: "not found" });+ return;+ }if (req.method === "GET" && url.pathname === PLAYER_PATH) {let code: string;try {+ await engine;code = await playerBundle();} catch (err) {// Shown on the page: usually an error in the composition's code.@@ -367,7 +403,7 @@ export async function startPreview(// Build the page's bundle and mix the soundtrack while the browser opens.void playerBundle().catch(() => {});startAudio();- if (options.open ?? !process.env.CI) {+ if (options.open === true) {// A tab from the last preview reconnects within ~1 s and reloads itself;// only open a new one when nobody comes back.const reconnected = await new Promise<boolean>((resolve) => {diff --git a/packages/gitframes/src/preview/player.ts b/packages/gitframes/src/preview/player.tsindex ecf23ae4..c4e46ccd 100644--- a/packages/gitframes/src/preview/player.ts+++ b/packages/gitframes/src/preview/player.ts@@ -2,9 +2,11 @@/// <reference types="webgpu" />/*** The browser side of the preview: runs the project's composition code and- * renders its frames with WebGPU straight onto the page's canvas. Bundled- * together with the composition module by bundle.ts.+ * renders its frames with WebGPU straight onto the page's canvas. Part of the+ * browser engine (bundle.ts); the project's bundle calls `startPlayer`.*/++import { BUILTIN_NODE_RENDERERS } from "gitframes-preview:renderers";import { GetFontAssetUrl } from "@gitframes/client-utils";import {compositionStateStore,@@ -90,28 +92,41 @@ declare global {}}-export interface PlayerOptions {- renderers: readonly BuiltinRenderer[];- /** Loads onnxruntime-web for vision nodes. */- loadOrt?: () => Promise<unknown>;+export interface PlayerSettings {+ /** Where cached vision models are (served by the preview server). */+ modelsDir: string;+ /** URL of an onnxruntime-web `dist/` folder, for vision nodes. */+ ortBase: string;}export function startPlayer(mod: Record<string, unknown>,exportName: string,- options: PlayerOptions,+ settings: PlayerSettings,): void {- const { loadOrt } = options;- // Vision nodes create their runners themselves; here they run on WebGPU.- if (loadOrt) {- setDefaultSessionProvider(- () =>- new WebGPUProvider({- loader: () => loadOrt() as Promise<OrtWebModule>,- }),- );- }- window.gitframesPlayer = createPlayer(mod, exportName, options.renderers);+ process.env.GITFRAMES_MODELS_DIR = settings.modelsDir;+ // Vision nodes create their runners themselves; here they run on WebGPU,+ // with onnxruntime-web loaded on first use.+ setDefaultSessionProvider(+ () =>+ new WebGPUProvider({+ loader: async () => {+ const ort = await import(+ /* @vite-ignore */ `${settings.ortBase}ort.webgpu.bundle.min.mjs`+ );+ ort.env.wasm.wasmPaths = settings.ortBase;+ ort.env.logLevel = "error";+ // Threads need the wasm on this origin; a CDN build runs on one.+ if (!settings.ortBase.startsWith("/")) ort.env.wasm.numThreads = 1;+ return ort as OrtWebModule;+ },+ }),+ );+ window.gitframesPlayer = createPlayer(+ mod,+ exportName,+ BUILTIN_NODE_RENDERERS,+ );// The page listens for this; a failure is shown there.window.dispatchEvent(new Event("gitframes:player"));}diff --git a/packages/gitframes/src/preview/preview.test.ts b/packages/gitframes/src/preview/preview.test.tsindex 3d36c7e0..3f3a5a1d 100644--- a/packages/gitframes/src/preview/preview.test.ts+++ b/packages/gitframes/src/preview/preview.test.ts@@ -1,6 +1,10 @@+import fs from "node:fs";import http from "node:http";+import os from "node:os";+import path from "node:path";import { fileURLToPath } from "node:url";import { afterEach, describe, expect, it } from "vitest";+import { prebuiltEngineDir } from "./bundle.js";import { type PreviewSession, startPreview } from "./index.js";const entry = new URL("./fixtures/comp.ts", import.meta.url);@@ -42,6 +46,23 @@ describe("startPreview", () => {expect(code).not.toContain("Could not build the preview");}, 60_000);+ it("serves the browser engine the project bundle imports", async () => {+ const session = await start({ entry }, { open: false, port: 0 });++ const player = await fetch(`${session.url}@gitframes/engine/player.mjs`);+ expect(player.status).toBe(200);+ expect(player.headers.get("content-type")).toContain("javascript");+ expect(await player.text()).toContain("startPlayer");++ const shim = await fetch(`${session.url}@gitframes/engine/shims/path.mjs`);+ expect(shim.status).toBe(200);+ expect(await shim.text()).toContain("export");++ expect(+ (await fetch(`${session.url}@gitframes/engine/missing.mjs`)).status,+ ).toBe(404);+ }, 60_000);+it("serves project files by absolute path, with byte ranges", async () => {const session = await start({ entry }, { open: false, port: 0 });const file = fileURLToPath(entry);@@ -109,3 +130,25 @@ describe("startPreview", () => {expect((await fetch(`${second.url}meta`)).status).toBe(200);});});++describe("prebuiltEngineDir", () => {+ it("finds the engine at the dist root or beside dist/preview", async () => {+ const root = await fs.promises.mkdtemp(+ path.join(os.tmpdir(), "gf-engine-"),+ );+ try {+ const engine = path.join(root, "preview-engine");+ expect(prebuiltEngineDir(root)).toBeUndefined();++ await fs.promises.mkdir(engine, { recursive: true });+ await fs.promises.writeFile(path.join(engine, "player.mjs"), "export {}");+ expect(prebuiltEngineDir(root)).toBe(engine);++ const nested = path.join(root, "preview");+ await fs.promises.mkdir(nested, { recursive: true });+ expect(prebuiltEngineDir(nested)).toBe(engine);+ } finally {+ await fs.promises.rm(root, { recursive: true, force: true });+ }+ });+});diff --git a/packages/gitframes/src/preview/shims/process.ts b/packages/gitframes/src/preview/shims/process.tsindex 1c395286..d0a886d2 100644--- a/packages/gitframes/src/preview/shims/process.ts+++ b/packages/gitframes/src/preview/shims/process.ts@@ -1,13 +1,23 @@-// A minimal `process` for the browser preview bundle. It has no-// `versions.node`, so code that checks for Node takes its browser path.-export const process = {- env: { NODE_ENV: "development" } as Record<string, string | undefined>,- argv: [] as string[],- platform: "browser",- browser: true,- cwd: () => "/",- nextTick: (fn: (...a: unknown[]) => void, ...args: unknown[]) =>- queueMicrotask(() => fn(...args)),- on: () => process,- emitWarning: () => undefined,-};+// A minimal `process` for the browser preview. It has no `versions.node`, so+// code that checks for Node takes its browser path. Also set as the global, for+// the project's own code, which is bundled separately from the engine.+type Process = ReturnType<typeof createProcess>;++function createProcess() {+ const p = {+ env: { NODE_ENV: "production" } as Record<string, string | undefined>,+ argv: [] as string[],+ platform: "browser",+ browser: true,+ cwd: () => "/",+ nextTick: (fn: (...a: unknown[]) => void, ...args: unknown[]) =>+ queueMicrotask(() => fn(...args)),+ on: (): unknown => p,+ emitWarning: () => undefined,+ };+ return p;+}++const g = globalThis as unknown as { process?: Process };+export const process: Process = g.process ?? createProcess();+g.process ??= process;diff --git a/packages/gitframes/src/preview/virtual.d.ts b/packages/gitframes/src/preview/virtual.d.tsnew file mode 100644index 00000000..2cb83fd9--- /dev/null+++ b/packages/gitframes/src/preview/virtual.d.ts@@ -0,0 +1,6 @@+// Resolved when the browser engine is built (bundle.ts): the generated list of+// built-in node renderers from @gitframes/renderer.+declare module "gitframes-preview:renderers" {+ import type { BuiltinRenderer } from "./player.js";+ export const BUILTIN_NODE_RENDERERS: readonly BuiltinRenderer[];+}diff --git a/packages/gitframes/tsdown.config.ts b/packages/gitframes/tsdown.config.tsindex 8671a100..78cce7b3 100644--- a/packages/gitframes/tsdown.config.ts+++ b/packages/gitframes/tsdown.config.ts@@ -1,4 +1,5 @@import { defineConfig } from "tsdown";+import { buildEngine } from "./src/preview/bundle.ts";export default defineConfig({entry: [@@ -12,6 +13,8 @@ export default defineConfig({],format: ["esm"],clean: true,+ // The live preview's browser engine, shipped prebuilt in dist/preview-engine.+ onSuccess: () => buildEngine("dist/preview-engine"),dts: { resolve: [/^@gitframes\//] },target: "es2022",noExternal: [/^@gitframes\//],diff --git a/plugins/gitframes/skills/gitframes-render/SKILL.md b/plugins/gitframes/skills/gitframes-render/SKILL.mdindex 5cc4e496..6deb67d8 100644--- a/plugins/gitframes/skills/gitframes-render/SKILL.md+++ b/plugins/gitframes/skills/gitframes-render/SKILL.md@@ -128,7 +128,7 @@ await session.closed; // keep the process alive while the user watches- **The export is a function returning the `Composition`** (it may be async), or a `Composition` itself. The browser calls it, so build-time Node code in it must stay simple: `node:path`, `fileURLToPath(import.meta.url)` and synchronous `fs` reads (JSON, text) work through shims; writes are skipped. Assets referenced by absolute path load from the server, limited to the repository that holds the entry (`root` overrides it).- **Run it in the background.** The preview serves until it is stopped. Start the script as a background command with the longest timeout your harness allows, tell the user the URL, and ask for feedback in the chat. Feedback comes from the user, not from the page.- **One tab for the whole loop.** The port is fixed per project directory. Running the preview again (after you've made changes) takes over from the one already running, and a tab left open reconnects and reloads into the new version by itself. Don't ask the user to reopen anything. When the user closes the tab, the preview stops by itself about 5 s later (a reload doesn't stop it). If the preview was stopped (tab closed, time limit, or you killed it), just start it again.-- The browser opens automatically (`open: false`, or `CI`, skips that) unless an open tab reconnects first. It needs WebGPU (current Chrome, Edge or Safari).+- **Show it where the user is.** `startPreview` only serves the page and returns the URL. If your app has a built-in browser (Claude Code's browser pane, Codex's), open the URL there. Otherwise give the user the link, or pass `open: true` to open their default browser (skipped when an open tab reconnects on its own). The page needs WebGPU (current Chrome, Edge or Safari).- Audio is the same mix `renderVideo` muxes (`audio: false` skips it). Playback uses the soundtrack as its clock; when a frame takes long (a video seek, a heavy shot), the clock and the sound hold until the picture catches up.---diff --git a/plugins/gitframes/skills/gitframes/SKILL.md b/plugins/gitframes/skills/gitframes/SKILL.mdindex a1bd75e9..f21eab3c 100644--- a/plugins/gitframes/skills/gitframes/SKILL.md+++ b/plugins/gitframes/skills/gitframes/SKILL.md@@ -87,7 +87,7 @@ Keep the composition separate from the script that renders it, so tests and thesrc/theme.ts # canvas size, fps, colors, fonts: the only place for magic numbersfilm.ts # buildFilm(): assembles the Composition- render.ts # renders frames for review, or the full video+ render.ts # renders frames for review, a live preview, or the full videoassets/fonts/ # .ttf / .otf files the film usesoutput/ # renders (add to .gitignore)@@ -158,12 +158,20 @@ export async function buildFilm(): Promise<Composition> {// src/render.tsimport fs from "node:fs/promises";import path from "node:path";-import { HeadlessMediaRenderer } from "gitframes";+import { HeadlessMediaRenderer, startPreview } from "gitframes";import { buildFilm } from "./film.js";const OUT = path.resolve(import.meta.dirname, "../output");const [mode = "video", ...args] = process.argv.slice(2);+if (mode === "preview") {+ // The page runs film.ts itself and renders it with WebGPU in the browser.+ const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });+ console.log(`Preview at ${session.url}`);+ await session.closed; // until its tab closes or the next preview replaces it+ process.exit(0);+}+const film = await buildFilm();await fs.mkdir(OUT, { recursive: true });@@ -201,6 +209,7 @@ Add scripts to `package.json`:"typecheck": "tsc --noEmit -p .","frames": "tsx src/render.ts frames","grid": "tsx src/render.ts grid",+ "preview": "tsx src/render.ts preview","render": "tsx src/render.ts"}```@@ -212,10 +221,20 @@ Never go straight to a full video render. Each step catches problems in seconds1. `npm run typecheck`2. `npm run frames -- 0 30 60` and look at the PNGs in `output/`. A black frame usually means a missing font, a layer outside the canvas, or opacity stuck at 0.3. `npm run grid` and check the motion reads smoothly across the contact sheet.-4. `npm run render` once frames and motion look right.+4. `npm run preview` when a person is watching: they play the film with sound before you export (see below).+5. `npm run render` once frames and motion look right.Show the user the frame PNGs or the grid before the full render. `gitframes-render` covers pixel-probe tests, render QA (`renderVideo({ qa })`), and diagnosing blank frames.+### Live preview++`npm run preview` serves a localhost player (picture, sound, timeline) and prints its URL. The page runs `film.ts` itself and renders with WebGPU, so the user sees the real film at full frame rate.++- Run it as a background command; it serves until its tab closes.+- It doesn't open anything. If your app has a built-in browser (Claude Code, Codex), open the URL there; otherwise give the user the link. `startPreview(..., { open: true })` opens their default browser.+- After changing the film, run it again. It takes over the same URL, and the open tab reloads by itself.+- Take feedback in chat. `gitframes-render` has the details.+## 5. Grow the filmOnce the first render works, move to the specialist skills: `gitframes-compose` for splitting the film into scene modules on a beat grid, `gitframes-effects` for grading, grain, and vision-driven effects, and `gitframes-render` for tests and delivery checks. Keep `theme.ts` as the single source of sizes, colors, fonts, and timing as the project grows.