Code-first video & image, rendered natively on WebGPU.
rm local test mock refs
okanasl3b7dcab6f860
diff --git a/packages/cli/ROADMAP.md b/packages/cli/ROADMAP.mdnew file mode 100644index 00000000..3d2c70ee--- /dev/null+++ b/packages/cli/ROADMAP.md@@ -0,0 +1,79 @@+# `ff` roadmap++Status: **Draft** · Date: 2026-10-10 · Companion: `framefields-cloud/docs/ROADMAP.md`++## Direction++The cloud is moving to GitHub-hosted repositories with media in Git LFS (see the cloud roadmap).+That splits `ff` in two:++- **The local half is the product, and it stays.** `ls`, `check`, `frames`, `grid`, `preview`, `render --local` and `init` are the verification ladder agents and humans use on every change. They don't touch the cloud, and they're open source.+- **The cloud half shrinks to what Framefields uniquely does:** cloud renders and preview links. Git, auth for git, assets, issues and PRs belong to `git`, `git lfs` and `gh`.++Nothing in the cloud flow may *require* `ff`. A user with only `git` and a browser must be able to+connect, push, get PR previews and render. `ff` makes it faster, and makes it scriptable for agents.++| Command | Fate |+|---|---|+| `ls`, `check`, `frames`, `grid`, `preview`, `render --local`, `config` | Keep |+| `init` | Keep; also writes `.gitattributes` for LFS (phase 2) |+| `auth login/status/logout` | Keep (API key); add browser login (phase 1) |+| `auth setup-git`, hidden `git-credential` | **Remove** (GitHub credentials do this) |+| `repo create`, `repo fork`, `repo delete` | **Remove** → `repo connect` / `repo disconnect` |+| `repo view`, `repo list` | Keep, re-pointed at connected repos |+| `clone` | **Remove** (`git clone` + `git lfs` do it) |+| `push`, `pull` (`sync.ts`) | **Remove** (plain `git`) |+| `asset sync/pull/status`, `assets.ts`, `framefields.assets.json` | **Remove**; replaced by `lfs` checks (phase 2) |+| `issue *`, `pr *` | **Remove** (`gh`) |+| `render` (cloud), `render status/list/download/cancel` | Keep and extend |+| `api` | Keep |++Roughly 2,000 of the CLI's ~5,100 source lines go away (`issue.ts`, `pr.ts`, `sync.ts`,+`clone.ts`, `assets.ts`, `asset.ts`, most of `repo.ts`, the credential parts of `auth.ts` and+`git/index.ts`), with the matching parts of `test/cloud.test.ts` and `SPEC.md` §cloud.++---++## Phase 1 — Align with GitHub-hosted repos (ships with cloud phase 1)++- [ ] `ff repo connect`: resolve the GitHub remote of the current directory; if the App isn't installed on it, open the install/connect page and wait for it to finish; then write `git config framefields.repo`. `--json` and non-TTY paths print the URL to open instead of prompting.+- [ ] `ff repo disconnect`, and `ff repo view`/`list` against the new repo shape (GitHub full name, default branch, last push).+- [ ] `ff auth login` without `--with-token` opens a browser device-code flow (sign in with GitHub on the web, the CLI receives an API key). `FF_TOKEN` stays the CI/agent path.+- [ ] Remove `auth setup-git`, `git-credential`, `repo create/fork/delete`, `clone`, `push`, `pull`, `issue`, `pr`. In this release they print a one-line pointer ("use `git push`", "use `gh pr create`") and exit 2; delete them one release later.+- [ ] `render` (cloud) checks the commit is on GitHub (`git ls-remote`) instead of on the Framefields remote.+- [ ] Bump to 3.0.0: it's a breaking change for anyone scripting the cloud commands.++## Phase 2 — Assets through Git LFS (ships with cloud phase 2)++- [ ] `ff init` writes `.gitattributes` with LFS rules for the manifest's `assets.directory` and common media extensions (`*.mp4 *.mov *.wav *.mp3 *.png *.jpg *.exr *.ttf *.otf *.glb *.onnx`), and warns if `git lfs` isn't installed.+- [ ] `ff check` adds an **assets** step:+ - media files over a size threshold that aren't LFS-tracked → error, with the `git lfs track`/`git lfs migrate` command to run;+ - LFS objects referenced at `HEAD` but missing locally (not pulled) → error naming `git lfs pull`.+- [ ] `ff lfs setup` for existing projects: write the rules, and print (not run) the `git lfs migrate import` command, since rewriting history is the user's call.+- [ ] Remove `asset *`, `assets.ts` and every read/write of `framefields.assets.json`. Local rendering already reads files from disk, so nothing in the engine changes.++## Phase 3 — Cloud rendering (ships with cloud phase 3)++- [ ] `ff render --cloud` options matching the plans: `--resolution 4k`, `--fps`, `--format mp4|prores|webm-alpha`, `--chunks auto`.+- [ ] Clear errors for plan limits (exit 3 with the upgrade URL as the hint) and for compositions that fail on the server ("passes `ff check` locally?" hint).+- [ ] `ff render --wait` streams the server's progress events (already in the API) to a progress bar on stderr.+- [ ] Parity test in CI: one example rendered locally and in the cloud must match frame hashes (or a PSNR threshold for encoded output).++## Phase 4 — Preview links (ships with cloud phase 4)++- [ ] `ff open [<composition>]`: open the share page for the current branch's latest preview render; `--json` prints the URL. Agents use it to hand a human a link.+- [ ] `ff previews` (or `render list --previews`): preview renders for the current branch/PR with their check status.+- [ ] Engine-side: add an optional `previews` section to `framefields.json` (which compositions and quality to render on push/PR). The format lives in `framefields/project` (SPEC §3.4) so `ff`, the cloud and the engine agree; `ff check` validates it.++## Phase 5 — Agent-native (any time after phase 1; local parts can start now)++- [ ] `ff mcp`: an MCP server over stdio exposing `ls`, `check`, `frames`, `grid`, `render` (local and cloud), `open`. Frames and grids come back as images, so an agent can *see* its change and iterate. This is the differentiator in the pitch, and the local half needs no cloud at all.+- [ ] Ship a `framefields-render` skill/instructions file that teaches agents the verification ladder through `ff`, replacing per-project `render.ts` notes.++## Not planned++- Re-implementing `gh` (issues, PRs, reviews) or git transport. If a cloud feature needs a git operation, `ff` shells out to `git`.+- Any asset storage protocol of our own. If Framefields-hosted LFS ships (cloud phase 6), it's a standard LFS endpoint configured in `.lfsconfig`; `ff` only helps write that file.++## Related cleanup spotted while planning+- [x] `packages/webgpu-renderers/src/slug/slug-font-cache.ts`: removed hardcoded machine-specific font paths in favor of resolving against project font directories and upward traversal.diff --git a/packages/webgpu-renderers/src/slug/slug-font-cache.ts b/packages/webgpu-renderers/src/slug/slug-font-cache.tsindex 88049e79..ae5c677d 100644--- a/packages/webgpu-renderers/src/slug/slug-font-cache.ts+++ b/packages/webgpu-renderers/src/slug/slug-font-cache.ts@@ -712,10 +712,37 @@ export class SlugFontCache {if (isNode && !isHttp) {// Node.js local file environmentconst fs = await import(- /* webpackIgnore: true */ "fs/promises"+ /* webpackIgnore: true */ "node:fs/promises");- const path = await import(/* webpackIgnore: true */ "path");+ const path = await import(+ /* webpackIgnore: true */ "node:path"+ );+ const fontDirs: string[] = [];+ if (process.env.FONTS_DIR) {+ fontDirs.push(path.resolve(process.env.FONTS_DIR));+ }+ if (process.env.FRAMEFIELDS_FONTS_DIR) {+ fontDirs.push(+ path.resolve(process.env.FRAMEFIELDS_FONTS_DIR),+ );+ }++ // Upward traversal to locate assets/fonts in parent monorepo roots+ let currentDir = process.cwd();+ for (let i = 0; i < 5; i++) {+ fontDirs.push(path.resolve(currentDir, "assets/fonts"));+ const parent = path.dirname(currentDir);+ if (parent === currentDir) break;+ currentDir = parent;+ }+const candidates: string[] = [];+ const addCandidate = (candidatePath: string) => {+ if (!candidates.includes(candidatePath)) {+ candidates.push(candidatePath);+ }+ };+if (fontUrl && !fontUrl.startsWith("//api/")) {let fontPath = fontUrl;if (fontUrl.startsWith("file://")) {@@ -725,8 +752,11 @@ export class SlugFontCache {fontPath = fontUrl.replace(/^file:\/\//, "");}}- candidates.push(fontPath);- candidates.push(path.resolve(process.cwd(), fontPath));+ addCandidate(fontPath);+ addCandidate(path.resolve(process.cwd(), fontPath));+ for (const dir of fontDirs) {+ addCandidate(path.resolve(dir, path.basename(fontPath)));+ }}const cleanFamily = fontFamily.replace(/['"]/g, "").trim();const variations = [@@ -734,29 +764,16 @@ export class SlugFontCache {cleanFamily.replace(/ /g, ""),cleanFamily.replace(/ /g, "_"),];- for (const v of variations) {- candidates.push(- path.resolve(process.cwd(), "assets/fonts", v + ".ttf"),- );- candidates.push(- path.resolve(process.cwd(), "assets/fonts", v + ".otf"),- );- candidates.push(- `/Users/okanaslankan/framefields/assets/fonts/${v}.ttf`,- );- candidates.push(- `/Users/okanaslankan/Gatewai/apps/gatewai-artifex/assets/fonts/${v}.ttf`,- );- candidates.push(- `/Users/okanaslankan/Gatewai/apps/gatewai-backend/src/assets/fonts/${v}/font_file.ttf`,- );+ for (const dir of fontDirs) {+ for (const v of variations) {+ addCandidate(path.resolve(dir, `${v}.ttf`));+ addCandidate(path.resolve(dir, `${v}.otf`));+ }+ }+ for (const dir of fontDirs) {+ addCandidate(path.resolve(dir, "Inter.ttf"));+ addCandidate(path.resolve(dir, "Inter.otf"));}- candidates.push(- "/Users/okanaslankan/framefields/assets/fonts/Inter.ttf",- );- candidates.push(- "/Users/okanaslankan/Gatewai/apps/gatewai-artifex/assets/fonts/Inter.ttf",- );let nodeBuffer: Buffer | null = null;for (const c of candidates) {diff --git a/scripts/render-3d-demo.mjs b/scripts/render-3d-demo.mjsindex b31356a6..dc235c64 100644--- a/scripts/render-3d-demo.mjs+++ b/scripts/render-3d-demo.mjs@@ -1,17 +1,18 @@+import fs from "node:fs/promises";+import path from "node:path";import {CameraAnimation,Composition,FontManager,Layer,} from "../packages/framefields/dist/index.mjs";-import fs from "node:fs/promises";-import path from "node:path";async function main() {console.log("=== WebGPU 3D Camera & Multiplane Render Demonstration ===");- const artifactDir = "/Users/okanaslankan/.gemini/antigravity-ide/brain/18faf2d5-3c9b-408e-8ba7-bcbf855a7dc5";- const assetsDir = "/Users/okanaslankan/framefields/assets/renders-3d";+ const assetsDir =+ process.env.ASSETS_DIR || path.resolve(process.cwd(), "assets/renders-3d");+ const artifactDir = process.env.ARTIFACT_DIR || assetsDir;await fs.mkdir(artifactDir, { recursive: true });await fs.mkdir(assetsDir, { recursive: true });@@ -99,7 +100,10 @@ async function main() {);const dollyBuffer = await compDolly.renderFrame({ frame: 20 });- await fs.writeFile(path.join(artifactDir, "3d_camera_dolly.png"), dollyBuffer);+ await fs.writeFile(+ path.join(artifactDir, "3d_camera_dolly.png"),+ dollyBuffer,+ );await fs.writeFile(path.join(assetsDir, "3d_camera_dolly.png"), dollyBuffer);console.log(" Saved: 3d_camera_dolly.png");@@ -150,7 +154,10 @@ async function main() {);const orbitBuffer = await compOrbit.renderFrame({ frame: 22 });- await fs.writeFile(path.join(artifactDir, "3d_camera_orbit.png"), orbitBuffer);+ await fs.writeFile(+ path.join(artifactDir, "3d_camera_orbit.png"),+ orbitBuffer,+ );await fs.writeFile(path.join(assetsDir, "3d_camera_orbit.png"), orbitBuffer);console.log(" Saved: 3d_camera_orbit.png");@@ -232,7 +239,9 @@ async function main() {await fs.writeFile(path.join(assetsDir, "3d_camera_dof.png"), dofBuffer);console.log(" Saved: 3d_camera_dof.png");- console.log("=== All 3D Camera Render Demonstrations Generated Successfully ===");+ console.log(+ "=== All 3D Camera Render Demonstrations Generated Successfully ===",+ );}main().catch((err) => {diff --git a/scripts/render-3d-text-demo.mjs b/scripts/render-3d-text-demo.mjsindex 32dab420..59b4fa49 100644--- a/scripts/render-3d-text-demo.mjs+++ b/scripts/render-3d-text-demo.mjs@@ -1,3 +1,5 @@+import fs from "node:fs/promises";+import path from "node:path";import {CameraAnimation,Composition,@@ -6,14 +8,14 @@ import {Layer3D,LayerAnimation,} from "../packages/framefields/dist/index.mjs";-import fs from "node:fs/promises";-import path from "node:path";async function main() {console.log("=== WebGPU 3D Typography Render Demonstration ===");- const artifactDir = "/Users/okanaslankan/.gemini/antigravity-ide/brain/18faf2d5-3c9b-408e-8ba7-bcbf855a7dc5";- const outputDir = "/Users/okanaslankan/framefields/examples/output/3d_text_demo";+ const outputDir =+ process.env.OUTPUT_DIR ||+ path.resolve(process.cwd(), "examples/output/3d_text_demo");+ const artifactDir = process.env.ARTIFACT_DIR || outputDir;await fs.mkdir(artifactDir, { recursive: true });await fs.mkdir(outputDir, { recursive: true });diff --git a/scripts/render-demo.mjs b/scripts/render-demo.mjsindex fd9fd2b3..97a7b41e 100644--- a/scripts/render-demo.mjs+++ b/scripts/render-demo.mjs@@ -1,321 +1,339 @@-import {- Composition,- FontManager,- Layer,- LayerAnimation,- Media,- Vignette,- Blur,- ColorBalance,- Signal,- signal,- computed,- AudioSignal,- AudioSignalExtractor-} from "/Users/okanaslankan/framefields/packages/framefields/dist/index.mjs";-import { HeadlessMediaRenderer } from "/Users/okanaslankan/framefields/packages/framefields/dist/renderer/index.mjs";import fs from "node:fs/promises";+import path from "node:path";+import {+ AudioSignal,+ AudioSignalExtractor,+ Blur,+ ColorBalance,+ Composition,+ computed,+ FontManager,+ Layer,+ LayerAnimation,+ Media,+ Signal,+ signal,+ Vignette,+} from "../packages/framefields/dist/index.mjs";+import { HeadlessMediaRenderer } from "../packages/framefields/dist/renderer/index.mjs";async function main() {- console.log("=== Framefields Signal-Driven Animation Pipeline ===");- const inputVideo = "/Users/okanaslankan/framefields/export (5).mp4";- const outputPng = "/Users/okanaslankan/framefields/rendered_preview.png";- const outputMp4 = "/Users/okanaslankan/framefields/rendered_composition.mp4";-- console.log("1. Registering TrueType fonts via FontManager singleton...");- const font = await FontManager.register("assets/fonts/Inter.ttf");- console.log(` FontManager registered: "${font.family}" (${font.postscriptName}), unitsPerEm: ${font.unitsPerEm}`);-- console.log("2. Initializing AudioSignalExtractor node & extracting WebGPU audio features...");- const audioExtractor = new AudioSignalExtractor({- nodeId: "speech_extractor",- extractionMode: "rms_envelope",- attackMs: 12,- releaseMs: 140,- sensitivity: 1.25,- noiseFloorDb: -42,- dynamicRangeDb: 40,- smoothing: 0.18,- curve: "smoothstep"- });-- const audio = await audioExtractor.extract(inputVideo, { fps: 24 });- console.log(` Extracted WebGPU audio speech signals (${audio.numFrames} frames) via AudioSignalExtractor`);-- console.log("3. Initializing composition and defining reactive signals...");- const comp = new Composition({- width: 1280,- height: 720,- fps: 24,- backgroundColor: "#0d1117"- });-- // Define reactive signals- // 1. Rhythmic breathing pulse signal (1.4 Hz sinusoidal LFO)- comp.addSignal("pulse_lfo", (frame, fps) => {- const t = frame / fps;- const wave = Math.sin(t * Math.PI * 2 * 1.4);- return (wave + 1) * 0.5; // normalized 0..1- });-- // 2. Real-time speech energy signal extracted via AudioSignalExtractor- comp.addSignal("energy_signal", (frame) => {- audio.seekFrame(frame);- return audio.energy.value;- });-- // Base video layer- comp.add(Layer.video(inputVideo, {- x: 0,- y: 0,- width: 1280,- height: 720,- fit: "cover"- }));-- // Audio track (mixed directly from video source)- comp.addAudio(Layer.audio(inputVideo, {- volume: 1.0- }));-- // 1. Frosted dark card container behind typography- comp.add(- Layer.shape("rect", {- id: "card-container",- x: 70,- y: 490,- width: 1140,- height: 165,- borderRadius: 18,- fillColor: "rgba(15, 23, 42, 0.88)"- }).animate(- new LayerAnimation()- .slideInY(590, 490, 0, 22, "power3.out")- .fadeIn(0, 16)- )- );-- // 2. Pulsing glowing indicator circle (scale & color modulated by pulse_lfo signal)- comp.add(- Layer.shape("circle", {- id: "pulse-dot",- x: 100,- y: 520,- width: 14,- height: 14,- borderRadius: 7,- fillColor: "#38bdf8"- }).animate(- new LayerAnimation()- .signal("scale", "pulse_lfo", { multiplier: 0.35, offset: 0.85 })- .colorSignal("fillColor", "pulse_lfo", { colorA: "#38bdf8", colorB: "#ec4899" })- .fadeIn(0, 18)- )- );-- // 3. Accent pill badge (scale modulated by signal, organic rotation wiggle)- comp.add(- Layer.shape("rect", {- id: "vfx-badge",- x: 126,- y: 514,- width: 185,- height: 26,- borderRadius: 6,- fillColor: "#38bdf8"- }).animate(- new LayerAnimation()- .signal("scale", "pulse_lfo", { multiplier: 0.08, offset: 1.0 })- .wiggle("rotation", { frequency: 1.2, amplitude: 3.0, seed: 101 })- .slideInY(536, 514, 6, 22)- .fadeIn(6, 18)- )- );-- // 4. Badge label- comp.add(- Layer.text("AUDIO SIGNAL EXTRACTOR", {- id: "badge-label",- x: 136,- y: 520,- fontSize: 11,- fill: "#0f172a",- fontFamily: "Inter",- fontWeight: "bold"- }).animate(- new LayerAnimation()- .slideInY(542, 520, 6, 22)- .fadeIn(6, 18)- )- );-- // 5. Signal reactive level track (meter background)- comp.add(- Layer.shape("rect", {- id: "meter-bg",- x: 330,- y: 524,- width: 220,- height: 6,- borderRadius: 3,- fillColor: "rgba(255, 255, 255, 0.12)"- }).animate(- new LayerAnimation()- .fadeIn(10, 24)- )- );-- // 6. Active signal meter bar (width & color dynamically modulated by speech energy_signal)- comp.add(- Layer.shape("rect", {- id: "meter-active",- x: 330,- y: 524,- width: 60,- height: 6,- borderRadius: 3,- fillColor: "#38bdf8"- }).animate(- new LayerAnimation()- .signal("width", "energy_signal", { multiplier: 190, offset: 25 })- .colorSignal("fillColor", "energy_signal", { colorA: "#38bdf8", colorB: "#f43f5e" })- .fadeIn(10, 24)- )- );-- // 7. Signal meter label- comp.add(- Layer.text("SIGNAL MODULATION", {- id: "meter-label",- x: 566,- y: 522,- fontSize: 11,- fill: "#94a3b8",- fontFamily: "Inter",- fontWeight: "bold"- }).animate(- new LayerAnimation()- .fadeIn(12, 26)- )- );-- // 8. Main title (entrance slide-up & fade-in)- comp.add(- Layer.text("Framefields Render Engine", {- id: "main-title",- x: 100,- y: 554,- fontSize: 32,- fill: "#ffffff",- fontFamily: "Inter"- }).animate(- new LayerAnimation()- .slideInY(580, 554, 8, 26)- .fadeIn(8, 22)- )- );-- // 9. Subtitle (entrance slide-up & fade-in)- comp.add(- Layer.text("node-audio-signal-extractor • WebGPU Audio DSP • Reactive Speech Blur", {- id: "subtitle",- x: 100,- y: 605,- fontSize: 15,- fill: "#94a3b8",- fontFamily: "Inter"- }).animate(- new LayerAnimation()- .slideInY(625, 605, 14, 32)- .fadeIn(14, 28)- )- );-- console.log("3. Dynamically computing composition duration & configuring reactive effects...");- const durationMs = await comp.computeDuration();- console.log(` Dynamically computed duration: ${durationMs}ms (${(durationMs / 1000).toFixed(2)}s)`);-- console.log("4. Instantiating reactive effect classes with game-engine per-frame lifecycle hooks...");-- // 1. First-class ColorBalance effect- const colorBalance = new ColorBalance({- shadows: { cyanRed: 12, magentaGreen: -4, yellowBlue: -12 },- midtones: { cyanRed: 4, magentaGreen: 6, yellowBlue: 2 },- highlights: { cyanRed: -4, magentaGreen: 0, yellowBlue: 8 },- preserveLuminosity: true- });-- // 2. Procedural breathing LFO for subtle vignette modulation- const vignetteBreathing = Signal.builder({- type: "sine",- frequency: 0.8,- amplitude: 8.0,- offset: 50.0- });-- // 3. First-class Vignette effect bound to reactive signals with per-frame mutation- const vignette = new Vignette({- strength: vignetteBreathing,- radius: 0.85,- softness: 0.55- });-- // Per-frame game engine hook: dynamically adjust radius over frames- vignette.onRequestFrame(({ frame }) => {- if (frame < 50) {- vignette.radius = 0.75 + frame * 0.002;- }- });-- // 4. Reactive Speech Blur: Screen blurs dynamically when the man is talking!- // When quiet/silent: 0 blur (crystal clear).- // When speech detected: blur smoothly scales with voice energy (up to 20px).- const speechBlurStrength = computed(() => {- const energy = audio.energy.value;- if (energy < 0.15) return 0;- return Math.min(22, Math.max(0, (energy - 0.15) * 32 + 2));- });-- const blur = new Blur({- blurType: "Gaussian",- strength: speechBlurStrength- });-- blur.onRequestFrame(({ frame }) => {- audio.seekFrame(frame);- });-- const compVM = await comp.toVirtualMediaAsync();- const styledMedia = new Media(compVM)- .apply(colorBalance)- .apply(vignette)- .apply(blur);-- const finalVM = styledMedia.toVirtualMedia();- const renderer = new HeadlessMediaRenderer();-- console.log("5. Rendering high-quality preview frame at frame 48 (2.0s)...");- const t0 = Date.now();- const pngBuffer = await renderer.renderImage(finalVM, 48, 24);- await fs.writeFile(outputPng, pngBuffer);- console.log(` Saved preview PNG (${(pngBuffer.length / 1024).toFixed(1)} KB) in ${Date.now() - t0}ms`);-- console.log(`6. Rendering and encoding full-duration MP4 video (${(durationMs / 1000).toFixed(2)}s, ${Math.round(durationMs / 1000 * 24)} frames @ 24fps)...`);- const t1 = Date.now();- const { filePath: tempVideoPath, cleanup } = await renderer.renderVideo(finalVM, {- codec: "h264",- quality: "medium"- });-- await fs.copyFile(tempVideoPath, outputMp4);- await cleanup();- const stat = await fs.stat(outputMp4);- console.log(` Saved full-length MP4 video (${(stat.size / 1024 / 1024).toFixed(2)} MB) in ${Date.now() - t1}ms`);-- console.log("=== Render Completed Successfully ===");+ console.log("=== Framefields Signal-Driven Animation Pipeline ===");+ const inputVideo =+ process.env.INPUT_VIDEO || path.resolve(process.cwd(), "export.mp4");+ const outputPng = path.resolve(process.cwd(), "rendered_preview.png");+ const outputMp4 = path.resolve(process.cwd(), "rendered_composition.mp4");++ console.log("1. Registering TrueType fonts via FontManager singleton...");+ const font = await FontManager.register("assets/fonts/Inter.ttf");+ console.log(+ ` FontManager registered: "${font.family}" (${font.postscriptName}), unitsPerEm: ${font.unitsPerEm}`,+ );++ console.log(+ "2. Initializing AudioSignalExtractor node & extracting WebGPU audio features...",+ );+ const audioExtractor = new AudioSignalExtractor({+ nodeId: "speech_extractor",+ extractionMode: "rms_envelope",+ attackMs: 12,+ releaseMs: 140,+ sensitivity: 1.25,+ noiseFloorDb: -42,+ dynamicRangeDb: 40,+ smoothing: 0.18,+ curve: "smoothstep",+ });++ const audio = await audioExtractor.extract(inputVideo, { fps: 24 });+ console.log(+ ` Extracted WebGPU audio speech signals (${audio.numFrames} frames) via AudioSignalExtractor`,+ );++ console.log("3. Initializing composition and defining reactive signals...");+ const comp = new Composition({+ width: 1280,+ height: 720,+ fps: 24,+ backgroundColor: "#0d1117",+ });++ // Define reactive signals+ // 1. Rhythmic breathing pulse signal (1.4 Hz sinusoidal LFO)+ comp.addSignal("pulse_lfo", (frame, fps) => {+ const t = frame / fps;+ const wave = Math.sin(t * Math.PI * 2 * 1.4);+ return (wave + 1) * 0.5; // normalized 0..1+ });++ // 2. Real-time speech energy signal extracted via AudioSignalExtractor+ comp.addSignal("energy_signal", (frame) => {+ audio.seekFrame(frame);+ return audio.energy.value;+ });++ // Base video layer+ comp.add(+ Layer.video(inputVideo, {+ x: 0,+ y: 0,+ width: 1280,+ height: 720,+ fit: "cover",+ }),+ );++ // Audio track (mixed directly from video source)+ comp.addAudio(+ Layer.audio(inputVideo, {+ volume: 1.0,+ }),+ );++ // 1. Frosted dark card container behind typography+ comp.add(+ Layer.shape("rect", {+ id: "card-container",+ x: 70,+ y: 490,+ width: 1140,+ height: 165,+ borderRadius: 18,+ fillColor: "rgba(15, 23, 42, 0.88)",+ }).animate(+ new LayerAnimation()+ .slideInY(590, 490, 0, 22, "power3.out")+ .fadeIn(0, 16),+ ),+ );++ // 2. Pulsing glowing indicator circle (scale & color modulated by pulse_lfo signal)+ comp.add(+ Layer.shape("circle", {+ id: "pulse-dot",+ x: 100,+ y: 520,+ width: 14,+ height: 14,+ borderRadius: 7,+ fillColor: "#38bdf8",+ }).animate(+ new LayerAnimation()+ .signal("scale", "pulse_lfo", { multiplier: 0.35, offset: 0.85 })+ .colorSignal("fillColor", "pulse_lfo", {+ colorA: "#38bdf8",+ colorB: "#ec4899",+ })+ .fadeIn(0, 18),+ ),+ );++ // 3. Accent pill badge (scale modulated by signal, organic rotation wiggle)+ comp.add(+ Layer.shape("rect", {+ id: "vfx-badge",+ x: 126,+ y: 514,+ width: 185,+ height: 26,+ borderRadius: 6,+ fillColor: "#38bdf8",+ }).animate(+ new LayerAnimation()+ .signal("scale", "pulse_lfo", { multiplier: 0.08, offset: 1.0 })+ .wiggle("rotation", { frequency: 1.2, amplitude: 3.0, seed: 101 })+ .slideInY(536, 514, 6, 22)+ .fadeIn(6, 18),+ ),+ );++ // 4. Badge label+ comp.add(+ Layer.text("AUDIO SIGNAL EXTRACTOR", {+ id: "badge-label",+ x: 136,+ y: 520,+ fontSize: 11,+ fill: "#0f172a",+ fontFamily: "Inter",+ fontWeight: "bold",+ }).animate(new LayerAnimation().slideInY(542, 520, 6, 22).fadeIn(6, 18)),+ );++ // 5. Signal reactive level track (meter background)+ comp.add(+ Layer.shape("rect", {+ id: "meter-bg",+ x: 330,+ y: 524,+ width: 220,+ height: 6,+ borderRadius: 3,+ fillColor: "rgba(255, 255, 255, 0.12)",+ }).animate(new LayerAnimation().fadeIn(10, 24)),+ );++ // 6. Active signal meter bar (width & color dynamically modulated by speech energy_signal)+ comp.add(+ Layer.shape("rect", {+ id: "meter-active",+ x: 330,+ y: 524,+ width: 60,+ height: 6,+ borderRadius: 3,+ fillColor: "#38bdf8",+ }).animate(+ new LayerAnimation()+ .signal("width", "energy_signal", { multiplier: 190, offset: 25 })+ .colorSignal("fillColor", "energy_signal", {+ colorA: "#38bdf8",+ colorB: "#f43f5e",+ })+ .fadeIn(10, 24),+ ),+ );++ // 7. Signal meter label+ comp.add(+ Layer.text("SIGNAL MODULATION", {+ id: "meter-label",+ x: 566,+ y: 522,+ fontSize: 11,+ fill: "#94a3b8",+ fontFamily: "Inter",+ fontWeight: "bold",+ }).animate(new LayerAnimation().fadeIn(12, 26)),+ );++ // 8. Main title (entrance slide-up & fade-in)+ comp.add(+ Layer.text("Framefields Render Engine", {+ id: "main-title",+ x: 100,+ y: 554,+ fontSize: 32,+ fill: "#ffffff",+ fontFamily: "Inter",+ }).animate(new LayerAnimation().slideInY(580, 554, 8, 26).fadeIn(8, 22)),+ );++ // 9. Subtitle (entrance slide-up & fade-in)+ comp.add(+ Layer.text(+ "node-audio-signal-extractor • WebGPU Audio DSP • Reactive Speech Blur",+ {+ id: "subtitle",+ x: 100,+ y: 605,+ fontSize: 15,+ fill: "#94a3b8",+ fontFamily: "Inter",+ },+ ).animate(new LayerAnimation().slideInY(625, 605, 14, 32).fadeIn(14, 28)),+ );++ console.log(+ "3. Dynamically computing composition duration & configuring reactive effects...",+ );+ const durationMs = await comp.computeDuration();+ console.log(+ ` Dynamically computed duration: ${durationMs}ms (${(durationMs / 1000).toFixed(2)}s)`,+ );++ console.log(+ "4. Instantiating reactive effect classes with game-engine per-frame lifecycle hooks...",+ );++ // 1. First-class ColorBalance effect+ const colorBalance = new ColorBalance({+ shadows: { cyanRed: 12, magentaGreen: -4, yellowBlue: -12 },+ midtones: { cyanRed: 4, magentaGreen: 6, yellowBlue: 2 },+ highlights: { cyanRed: -4, magentaGreen: 0, yellowBlue: 8 },+ preserveLuminosity: true,+ });++ // 2. Procedural breathing LFO for subtle vignette modulation+ const vignetteBreathing = Signal.builder({+ type: "sine",+ frequency: 0.8,+ amplitude: 8.0,+ offset: 50.0,+ });++ // 3. First-class Vignette effect bound to reactive signals with per-frame mutation+ const vignette = new Vignette({+ strength: vignetteBreathing,+ radius: 0.85,+ softness: 0.55,+ });++ // Per-frame game engine hook: dynamically adjust radius over frames+ vignette.onRequestFrame(({ frame }) => {+ if (frame < 50) {+ vignette.radius = 0.75 + frame * 0.002;+ }+ });++ // 4. Reactive Speech Blur: Screen blurs dynamically when the man is talking!+ // When quiet/silent: 0 blur (crystal clear).+ // When speech detected: blur smoothly scales with voice energy (up to 20px).+ const speechBlurStrength = computed(() => {+ const energy = audio.energy.value;+ if (energy < 0.15) return 0;+ return Math.min(22, Math.max(0, (energy - 0.15) * 32 + 2));+ });++ const blur = new Blur({+ blurType: "Gaussian",+ strength: speechBlurStrength,+ });++ blur.onRequestFrame(({ frame }) => {+ audio.seekFrame(frame);+ });++ const compVM = await comp.toVirtualMediaAsync();+ const styledMedia = new Media(compVM)+ .apply(colorBalance)+ .apply(vignette)+ .apply(blur);++ const finalVM = styledMedia.toVirtualMedia();+ const renderer = new HeadlessMediaRenderer();++ console.log("5. Rendering high-quality preview frame at frame 48 (2.0s)...");+ const t0 = Date.now();+ const pngBuffer = await renderer.renderImage(finalVM, 48, 24);+ await fs.writeFile(outputPng, pngBuffer);+ console.log(+ ` Saved preview PNG (${(pngBuffer.length / 1024).toFixed(1)} KB) in ${Date.now() - t0}ms`,+ );++ console.log(+ `6. Rendering and encoding full-duration MP4 video (${(durationMs / 1000).toFixed(2)}s, ${Math.round((durationMs / 1000) * 24)} frames @ 24fps)...`,+ );+ const t1 = Date.now();+ const { filePath: tempVideoPath, cleanup } = await renderer.renderVideo(+ finalVM,+ {+ codec: "h264",+ quality: "medium",+ },+ );++ await fs.copyFile(tempVideoPath, outputMp4);+ await cleanup();+ const stat = await fs.stat(outputMp4);+ console.log(+ ` Saved full-length MP4 video (${(stat.size / 1024 / 1024).toFixed(2)} MB) in ${Date.now() - t1}ms`,+ );++ console.log("=== Render Completed Successfully ===");}-main().catch(err => {- console.error("Pipeline execution failed:", err);- process.exit(1);+main().catch((err) => {+ console.error("Pipeline execution failed:", err);+ process.exit(1);});diff --git a/scripts/render-text-2d-3d.mjs b/scripts/render-text-2d-3d.mjsindex b0372909..882a58ea 100644--- a/scripts/render-text-2d-3d.mjs+++ b/scripts/render-text-2d-3d.mjs@@ -1,3 +1,6 @@+import fs from "node:fs/promises";+import path from "node:path";+import { Canvas, loadImage } from "skia-canvas";import {CameraAnimation,Composition,@@ -6,16 +9,14 @@ import {Layer3D,LayerAnimation,} from "../packages/framefields/dist/index.mjs";-import { loadImage, Canvas } from "skia-canvas";-import fs from "node:fs/promises";-import path from "node:path";async function main() {console.log("=== WebGPU 2D & 3D Typography Render Engine ===");- const artifactDir =- "/Users/okanaslankan/.gemini/antigravity-ide/brain/b888d40e-44c5-4686-bebd-a131288bd97e";- const outputDir = "/Users/okanaslankan/framefields/examples/output/text_2d_3d";+ const outputDir =+ process.env.OUTPUT_DIR ||+ path.resolve(process.cwd(), "examples/output/text_2d_3d");+ const artifactDir = process.env.ARTIFACT_DIR || outputDir;await fs.mkdir(artifactDir, { recursive: true });await fs.mkdir(outputDir, { recursive: true });@@ -250,13 +251,16 @@ async function main() {fill: "#ffffff",letterSpacing: 0, // Tests pair kerning: AV, VA, AT, AR}),- Layer.text("Zero-error pair kerning verified against Skia reference rasters (Δ = 0px).", {- id: "card-sub-2d",- fontSize: 13,- fontWeight: 400,- fill: "#94a3b8",- lineHeight: 1.4,- }),+ Layer.text(+ "Zero-error pair kerning verified against Skia reference rasters (Δ = 0px).",+ {+ id: "card-sub-2d",+ fontSize: 13,+ fontWeight: 400,+ fill: "#94a3b8",+ lineHeight: 1.4,+ },+ ),],}),);@@ -278,13 +282,16 @@ async function main() {borderWidth: 1,borderColor: "rgba(30, 41, 59, 0.7)",children: [- Layer.text("WEBGPU PIPELINE: 0 CHURN / 0 PIPELINE REBUILDS / 120 FPS ZERO-CPU-RE-RASTER", {- id: "footer-left",- fontSize: 12,- fontWeight: "bold",- fill: "#10b981",- letterSpacing: 2,- }),+ Layer.text(+ "WEBGPU PIPELINE: 0 CHURN / 0 PIPELINE REBUILDS / 120 FPS ZERO-CPU-RE-RASTER",+ {+ id: "footer-left",+ fontSize: 12,+ fontWeight: "bold",+ fill: "#10b981",+ letterSpacing: 2,+ },+ ),Layer.text("FRAMEFIELDS CORE ENGINE", {id: "footer-right",fontSize: 12,@@ -343,7 +350,11 @@ async function main() {const text3DData = ctx.getImageData(width / 2 - 200, 420, 400, 100).data;let has3DTextPixels = false;for (let i = 0; i < text3DData.length; i += 4) {- if (text3DData[i] > 150 || text3DData[i + 1] > 120 || text3DData[i + 2] > 180) {+ if (+ text3DData[i] > 150 ||+ text3DData[i + 1] > 120 ||+ text3DData[i + 2] > 180+ ) {has3DTextPixels = true;break;}@@ -353,7 +364,9 @@ async function main() {console.log(`3D Extruded Text Rasterization Verified: ${has3DTextPixels}`);if (!has2DTextPixels || !has3DTextPixels) {- throw new Error("Validation failed: text pixels not detected in expected regions.");+ throw new Error(+ "Validation failed: text pixels not detected in expected regions.",+ );}console.log("✅ Successfully rendered 2D & 3D text!");