Code-first video & image, rendered natively on WebGPU.
Work in progress across packages: renderer video QA, render diagnostics, loader/renderer3d fixes, launch example updates
okanasl2ceb93e17b2a
diff --git a/README.md b/README.mdindex 011c115e..b3cda2f2 100644--- a/README.md+++ b/README.md@@ -101,22 +101,6 @@ Developers generating video programmatically commonly weigh **Remotion** (React/| **Headless Verification** | **FrameGrid contact sheets**, single-frame snapshots, Skia MSE pixel-invariant assertions | Playwright/Puppeteer visual snapshots | Manual frame inspection / canvas diffing || **Docker / Cloud Portability** | **Compact** (~500 MB slim image with native GPU/Vulkan drivers) | **Heavy** (~2–3 GB with Chromium, fonts, X11/Mesa) | Moderate container size |-### Why Gitframes? Deep Architectural Analysis--#### 1. Why not Remotion for high-performance pipelines?-Remotion pioneered the "video in React" paradigm and is a great fit for marketing templates built from existing web UI. But because it is coupled to **headless Chromium**:-- **Scalability ceiling** — Chromium is an interactive document browser, not a real-time compositor. Every frame pays DOM style recalculation, layout reflow, paint traversal, and cross-process buffer copies over CDP.-- **GPU inefficiency** — WebGL inside headless Chromium is sandboxed and cannot use compute shaders, storage buffers, or native WebGPU multi-render targets (MRT) efficiently.-- **Infrastructure cost** — Scaling in production needs high-memory CPU instances to survive Chromium OOM crashes.--**Gitframes eliminates the browser entirely.** Running directly on the GPU via native WebGPU, it behaves like a game engine or DAW and delivers order-of-magnitude throughput gains.--#### 2. Why not Hyperframes or Canvas2D?-Canvas engines avoid DOM overhead but:-- **CPU-bound rasterization** — `ctx.arc` / `ctx.bezierCurveTo` execute largely on the CPU; hundreds of animated type layers stall quickly.-- **Shallow post-processing** — no multi-pass fragment pipelines, so pro grading (3D LUTs, selective color, bilateral SSAO, bokeh DoF) is effectively impossible.-- **Disconnected audio** — Canvas frameworks treat audio as an external soundtrack; Gitframes unifies synthesis and animation through frame-accurate signals.----## Key Features & Capabilitiesdiff --git a/examples/22_gitframes_launch/package.json b/examples/22_gitframes_launch/package.jsonindex a647b528..381f826a 100644--- a/examples/22_gitframes_launch/package.json+++ b/examples/22_gitframes_launch/package.json@@ -9,6 +9,7 @@"images": "tsx src/generate-images.ts","precomp": "tsx src/precomp.ts","render": "tsx src/render.ts",+ "render:src": "tsx --tsconfig tsconfig.dev.json --conditions=development src/render.ts","test": "vitest run","typecheck": "tsc --noEmit -p ."},diff --git a/examples/22_gitframes_launch/src/render.ts b/examples/22_gitframes_launch/src/render.tsindex d07cae14..16cfbb20 100644--- a/examples/22_gitframes_launch/src/render.ts+++ b/examples/22_gitframes_launch/src/render.ts@@ -1,11 +1,12 @@/**- * pnpm render → output/gitframes-launch.mp4 (with the score)+ * pnpm render → output/gitframes-launch.mp4 (with the score) + QA report; exits 1 if QA fails* pnpm render frames 0 80 320 → output/frames/f0000.png … for visual checks* pnpm render sheet 0 320 20 → every 20th frame from 0 to 320, plus a contact sheet+ * pnpm render:src … → any of the above, run on the engine's source (no package builds)*/import fs from "node:fs/promises";import path from "node:path";-import { HeadlessMediaRenderer } from "gitframes";+import { formatQaReport, HeadlessMediaRenderer } from "gitframes";import { buildFilm } from "./film.js";import { OUTPUT } from "./paths.js";@@ -39,10 +40,15 @@ if (mode === "frames" || mode === "sheet") {const result = await film.renderVideo({outputPath: path.join(OUTPUT, "gitframes-launch.mp4"),quality: "high",+ qa: true,});await result.cleanup?.().catch(() => {});console.log(`${path.join(OUTPUT, "gitframes-launch.mp4")} (${((Date.now() - started) / 1000).toFixed(1)} s)`,);+ if (result.qa) {+ console.log(formatQaReport(result.qa));+ if (!result.qa.passed) process.exitCode = 1;+ }}-process.exit(0);+process.exit(process.exitCode ?? 0);diff --git a/examples/22_gitframes_launch/src/scenes/logo.ts b/examples/22_gitframes_launch/src/scenes/logo.tsindex 2ea01a32..67219805 100644--- a/examples/22_gitframes_launch/src/scenes/logo.ts+++ b/examples/22_gitframes_launch/src/scenes/logo.ts@@ -85,10 +85,8 @@ export function logoScene() {fontFamily: DISPLAY,fontWeight: 900,fontSize: 210,- fill: FG,- bevelColor: ACCENT,+ fill: ACCENT,depth: 90,- slices: 24,x: CX,y: CY,z: 0,diff --git a/examples/22_gitframes_launch/src/scenes/not.ts b/examples/22_gitframes_launch/src/scenes/not.tsindex 1672586a..fa33f847 100644--- a/examples/22_gitframes_launch/src/scenes/not.ts+++ b/examples/22_gitframes_launch/src/scenes/not.ts@@ -53,7 +53,9 @@ function denials(): Denial[] {function denial(d: Denial, i: number, from: number) {const at = Math.min(d.no.at, d.thing.at) - from;const until = d.until - from;- const strikeAt = d.thing.at - from + BEAT;+ // The bar lands a beat after the word, but never so late that its 8-frame+ // sweep would run past the clip (the last word is followed quickly).+ const strikeAt = Math.min(d.thing.at - from + BEAT, until - 9);const step = until - at;return [word({diff --git a/examples/22_gitframes_launch/src/scenes/outro.ts b/examples/22_gitframes_launch/src/scenes/outro.tsindex a324f56d..21906b04 100644--- a/examples/22_gitframes_launch/src/scenes/outro.ts+++ b/examples/22_gitframes_launch/src/scenes/outro.ts@@ -103,10 +103,8 @@ function lockup() {fontFamily: DISPLAY,fontWeight: 900,fontSize: 200,- fill: FG,- bevelColor: ACCENT,+ fill: ACCENT,depth: 80,- slices: 24,x: CX,y: CY - 40,z: 0,diff --git a/examples/22_gitframes_launch/src/scenes/vfx.ts b/examples/22_gitframes_launch/src/scenes/vfx.tsindex f75048c9..c74576b7 100644--- a/examples/22_gitframes_launch/src/scenes/vfx.ts+++ b/examples/22_gitframes_launch/src/scenes/vfx.ts@@ -388,6 +388,7 @@ function passShot(p: Pass, sceneFrom: number) {id: `${p.id}-words`,words: p.words,from: p.verb.at,+ until: len,y: (H - 210) / 2,size: 170,color: p.id === "vfx-loud" ? ACCENT : FG,diff --git a/examples/22_gitframes_launch/src/scenes/world.ts b/examples/22_gitframes_launch/src/scenes/world.tsindex d40b4455..67784eef 100644--- a/examples/22_gitframes_launch/src/scenes/world.ts+++ b/examples/22_gitframes_launch/src/scenes/world.ts@@ -232,10 +232,8 @@ function depth(probeAt: number, lightAt: number, depthAt: number, len: number) {fontFamily: DISPLAY,fontWeight: 900,fontSize: 300,- fill: ACCENT,- bevelColor: ACCENT_DEEP,+ fill: ACCENT_DEEP,depth: 140,- slices: 28,x: CX,y: CY + 40,z: DEPTH_Z,diff --git a/examples/22_gitframes_launch/src/theme.ts b/examples/22_gitframes_launch/src/theme.tsindex 13911e64..1221838e 100644--- a/examples/22_gitframes_launch/src/theme.ts+++ b/examples/22_gitframes_launch/src/theme.ts@@ -82,9 +82,7 @@ export function keys(list: Key[],anim: Anim = LayerAnimation.create(),): Anim {- for (const [frame, value, ease] of list)- anim.keyframe(prop, Math.round(frame), value, ease);- return anim;+ return anim.keys(prop, list);}export interface SceneOptions {@@ -280,6 +278,8 @@ export interface SungLineOptions {justify?: "start" | "center" | "end";/** Frame (scene-local) the whole line leaves. */outAt?: number;+ /** Frame (relative to `from`) the line's clip ends: entrances are shortened to finish by then. */+ until?: number;enter?: "pop" | "rise";/** Stand the line in the 3D scene at this depth. */z?: number;@@ -302,18 +302,27 @@ export function sungLine(o: SungLineOptions): Node {const at = Math.max(0, w.at - o.from);const text = o.upper === false ? bare(w) : bare(w).toUpperCase();const anim = LayerAnimation.create();+ // A word sung just before the cut still lands at rest on its last frame.+ const fit = (frames: number) =>+ o.until === undefined+ ? frames+ : Math.max(1, Math.min(frames, o.until - 1 - at));if ((o.enter ?? "pop") === "pop") {anim- .fromTo("scale", 1.35, 1, { start: at, end: at + 8, ease: EASE_OUT })- .fromTo("opacity", 0, 1, { start: at, end: at + 2, ease: "none" });+ .fromTo("scale", 1.35, 1, {+ start: at,+ end: at + fit(8),+ ease: EASE_OUT,+ })+ .fromTo("opacity", 0, 1, { start: at, end: at + fit(2), ease: "none" });} else {anim.fromTo("y", size * 0.45, 0, {start: at,- end: at + 10,+ end: at + fit(10),ease: EASE_OUT,})- .fromTo("opacity", 0, 1, { start: at, end: at + 3, ease: "none" });+ .fromTo("opacity", 0, 1, { start: at, end: at + fit(3), ease: "none" });}return Layer.text(text, {id: `${o.id}-${i}`,diff --git a/examples/22_gitframes_launch/tsconfig.dev.json b/examples/22_gitframes_launch/tsconfig.dev.jsonnew file mode 100644index 00000000..8c05246f--- /dev/null+++ b/examples/22_gitframes_launch/tsconfig.dev.json@@ -0,0 +1,9 @@+{+ "extends": "./tsconfig.json",+ "include": [+ "src/**/*",+ "../../nodes/**/*",+ "../../packages/**/*",+ "../../node_modules/@gitframes/**/*"+ ]+}diff --git a/nodes/node-compositor/src/renderers/webgpu-renderer.ts b/nodes/node-compositor/src/renderers/webgpu-renderer.tsindex 8b472047..bca37a21 100644--- a/nodes/node-compositor/src/renderers/webgpu-renderer.ts+++ b/nodes/node-compositor/src/renderers/webgpu-renderer.ts@@ -18,7 +18,6 @@ import type { WebGPUNodeRenderer } from "@gitframes/node-sdk";import { measureText } from "@gitframes/renderers";import { TemporalDeflickerPipeline } from "@gitframes/tensor-webgpu";import {- AudioLatentEngine,AudioLatentTrackCache,type BlendMode,Camera3D,@@ -204,16 +203,24 @@ function getHomographyResources(}const audioTrackLatentCache = new Map<string, AudioLatentTrackCache>();-const deviceRenderer3DCache = new WeakMap<GPUDevice, Renderer3D>();+const deviceRenderer3DCache = new WeakMap<GPUDevice, Map<string, Renderer3D>>();+/** One Renderer3D per device, target format and MSAA sample count. */function getRenderer3D(device: GPUDevice,format: GPUTextureFormat,+ sampleCount: number,): Renderer3D {- let r3d = deviceRenderer3DCache.get(device);- if (!r3d || r3d.format !== format) {- r3d = new Renderer3D(device, format);- deviceRenderer3DCache.set(device, r3d);+ let byConfig = deviceRenderer3DCache.get(device);+ if (!byConfig) {+ byConfig = new Map();+ deviceRenderer3DCache.set(device, byConfig);+ }+ const key = `${format}|${sampleCount}`;+ let r3d = byConfig.get(key);+ if (!r3d) {+ r3d = new Renderer3D(device, format, sampleCount);+ byConfig.set(key, r3d);}return r3d;}@@ -582,6 +589,8 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {if (op.op === "Compositor") {const vop = op as CompositorOperation;+ // 4x MSAA in the 3D pass unless the document opts out (antialias3d: false).+ const sampleCount3d = vop.antialias3d === false ? 1 : 4;const containerWidth = props.containerWidth ?? ctx.surface.width;const containerHeight = props.containerHeight ?? ctx.surface.height;@@ -622,7 +631,11 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {);tl.seek(shiftedFrame / (fps ?? 24));- const r3d = getRenderer3D(ctx.device, ctx.renderer.format ?? "rgba8unorm");+ const r3d = getRenderer3D(+ ctx.device,+ ctx.renderer.format ?? "rgba8unorm",+ sampleCount3d,+ );r3d.resetPools();const relightPipe = getScreenSpaceRelightPipeline(@@ -990,6 +1003,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {const r3d = getRenderer3D(ctx.device,ctx.renderer.format ?? "rgba8unorm",+ sampleCount3d,);// Execute audio mesh deformation compute passes before beginning render pass@@ -1253,11 +1267,16 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {const r3d = getRenderer3D(ctx.device,ctx.renderer.format ?? "rgba8unorm",+ sampleCount3d,);let modelSrc = lop.modelData ?? lop.src;if (!modelSrc && lop.text3dOptions) {- const tOpts = lop.text3dOptions as ExtrudedTextGeometryOptions;+ // The node's Layer3D.extrudedText options: geometry options plus a family name.+ const tOpts = lop.text3dOptions as Omit<+ ExtrudedTextGeometryOptions,+ "font"+ > & { fontFamily?: string };const font =(tOpts.fontFamily? SlugFontCache.getParsed(tOpts.fontFamily)@@ -1268,7 +1287,8 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {if (font) {const res = generateExtrudedTextGeometry({...tOpts,- font,+ // SlugFontCache holds fontkit fonts under a narrower type.+ font: font as unknown as ExtrudedTextGeometryOptions["font"],});modelSrc = res.modelData;lop.modelData = modelSrc;@@ -2628,6 +2648,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {const r3dInstance = getRenderer3D(ctx.device,ctx.renderer.format ?? "rgba8unorm",+ sampleCount3d,);const activeLights = synthesizeRelightLights(@@ -2635,7 +2656,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {sceneLights,rect,shiftedFrame,- fps,+ fps ?? 24,);const lightsBuffer =@@ -2668,7 +2689,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.roughness,0.35,shiftedFrame,- fps,+ fps ?? 24,),specularStrength: resolveVal(target?.specularStrength ??@@ -2676,7 +2697,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.specularStrength,0.7,shiftedFrame,- fps,+ fps ?? 24,),metallic: resolveVal(target?.metallic ??@@ -2684,7 +2705,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.metallic,0.0,shiftedFrame,- fps,+ fps ?? 24,),ambientIntensity: resolveVal(target?.ambientIntensity ??@@ -2692,7 +2713,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.ambientIntensity,0.15,shiftedFrame,- fps,+ fps ?? 24,),depthScale: resolveVal(target?.depthScale ??@@ -2700,7 +2721,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.depthScale,1.0,shiftedFrame,- fps,+ fps ?? 24,),opacity: 1.0,},@@ -3311,13 +3332,14 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {const r3dInstance = getRenderer3D(ctx.device,ctx.renderer.format ?? "rgba8unorm",+ sampleCount3d,);const activeLights = synthesizeRelightLights(relightOpts as Record<string, unknown>,sceneLights,{ x: sampledX, y: sampledY, width: texW, height: texH },shiftedFrame,- fps,+ fps ?? 24,);const lightsBuffer = r3dInstance.getOrCreateLightsBuffer(activeLights);@@ -3349,7 +3371,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.roughness,0.35,shiftedFrame,- fps,+ fps ?? 24,),specularStrength: resolveVal(target?.specularStrength ??@@ -3357,7 +3379,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.specularStrength,0.7,shiftedFrame,- fps,+ fps ?? 24,),metallic: resolveVal(target?.metallic ??@@ -3365,7 +3387,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.metallic,0.0,shiftedFrame,- fps,+ fps ?? 24,),ambientIntensity: resolveVal(target?.ambientIntensity ??@@ -3373,7 +3395,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.ambientIntensity,0.15,shiftedFrame,- fps,+ fps ?? 24,),depthScale: resolveVal(target?.depthScale ??@@ -3381,7 +3403,7 @@ export const CompositorWebGPURenderer: WebGPUNodeRenderer = async (args) => {relightOpts.depthScale,1.0,shiftedFrame,- fps,+ fps ?? 24,),opacity: 1.0,},diff --git a/nodes/node-compositor/src/shared/compiler.test.ts b/nodes/node-compositor/src/shared/compiler.test.tsindex d37f584c..c011fda8 100644--- a/nodes/node-compositor/src/shared/compiler.test.ts+++ b/nodes/node-compositor/src/shared/compiler.test.ts@@ -913,6 +913,21 @@ describe("keyframe tail truncation (review M8)", () => {expect(warn).toHaveBeenCalledTimes(2);});+ it("warns once across single-frame renders of the same doc (frame-by-frame probes)", () => {+ const warn = spyWarn();+ const input = vmWithTrack(+ [+ { frame: 0, value: 0 },+ { frame: 31, value: 100 },+ ],+ 24,+ );+ for (const id of ["img-probe-1", "img-probe-2", "img-probe-3"]) {+ compileTimeline(id, input, { fps: 24, durationSec: 5 });+ }+ expect(warn).toHaveBeenCalledTimes(1);+ });+it("compiles and evaluates text reveal animation tracks accurately", () => {const virtualMedia = {operation: { op: "Compositor", width: 1920, height: 1080 },diff --git a/nodes/node-compositor/src/shared/compiler.ts b/nodes/node-compositor/src/shared/compiler.tsindex c5871896..d692699f 100644--- a/nodes/node-compositor/src/shared/compiler.ts+++ b/nodes/node-compositor/src/shared/compiler.ts@@ -4,11 +4,12 @@ import {collectNodeOps,type EaseRef,evaluateTrackAtFrame,- type LayerAnimation,+ type LayerAnimationSpec,} from "@gitframes/compositions/program";import {DEFAULT_DURATION_MS,getActiveMediaMetadata,+ reportRenderDiagnostic,type VirtualMediaData,} from "@gitframes/core";import gsapModule from "gsap";@@ -462,7 +463,12 @@ export interface TruncationInfo {maxAuthoredFrame: number;}-/** Dedupe per compiled cache key — a fresh render of the same broken doc SHOULD re-warn. */+/**+ * Dedupe per compiled cache key — a fresh render of the same broken doc SHOULD+ * re-warn. Single-frame renders are the exception: each `renderFrame` call has+ * its own `img-` render id, so probing a doc frame by frame would repeat the+ * warning on every frame; those dedupe per document instead.+ */const truncationWarnedKeys = new Set<string>();const TRUNCATION_WARNED_CAP = 100;@@ -789,7 +795,20 @@ export function compileTimeline(}});- if (truncations.length > 0) warnTruncationOnce(cacheKey, truncations);+ if (truncations.length > 0) {+ warnTruncationOnce(+ renderId.startsWith("img-") ? `frame_${irHash}` : cacheKey,+ truncations,+ );+ for (const t of truncations) {+ reportRenderDiagnostic(renderId, {+ severity: "warning",+ code: "animation_truncated",+ layerId: t.layerId,+ message: `'${t.prop}' on '${t.layerId}': ${t.dropped} keyframe(s) past the clip window (last authored frame ${t.maxAuthoredFrame}) never play`,+ });+ }+ }tl.seek(1e-6);tl.seek(0);@@ -818,7 +837,7 @@ const layerTimelineCache = new Map<export function compileLayerTimeline(layerId: string,- animation: LayerAnimation | undefined,+ animation: LayerAnimationSpec | undefined,baseVolume: number,baseMuted: boolean,fps: number,diff --git a/packages/compositions/package.json b/packages/compositions/package.jsonindex be4f4ed8..0426f3d2 100644--- a/packages/compositions/package.json+++ b/packages/compositions/package.json@@ -42,6 +42,7 @@"@gitframes/server-utils": "workspace:*","@gitframes/tsconfig": "workspace:*","@types/react": "^19.2.4",+ "@webgpu/types": "^0.1.70","dotenv": "^16.4.5","tsdown": "^0.19.0","tsx": "^4.20.3",diff --git a/packages/compositions/src/program/animation.test.ts b/packages/compositions/src/program/animation.test.tsindex eea72839..de421029 100644--- a/packages/compositions/src/program/animation.test.ts+++ b/packages/compositions/src/program/animation.test.ts@@ -127,3 +127,32 @@ describe("LayerAnimation builder & keyframe anchoring", () => {expect(cf25.perspective).toBe(1200);});});++describe("LayerAnimation.keys", () => {+ it("adds a keyframe per tuple, rounding frames and parsing eases", () => {+ const anim = LayerAnimation.keys("scale", [+ [0, 0],+ [11.6, 1.1, "back.out"],+ [20, 1],+ ]);+ expect(anim.tracks).toHaveLength(1);+ const [a, b, c] = anim.tracks[0]!.keyframes;+ expect([a!.frame, b!.frame, c!.frame]).toEqual([0, 12, 20]);+ expect(b!.value).toBe(1.1);+ expect(b!.ease).toEqual({ name: "back", dir: "out" });+ expect(a!.ease).toBeUndefined();+ expect(evaluateAnimationAtFrame(anim, 20).scale).toBe(1);+ });++ it("chains onto an existing animation and its track for the same prop", () => {+ const anim = LayerAnimation.create()+ .keyframe("opacity", 0, 0)+ .keys("opacity", [[10, 1]])+ .keys("y", [+ [0, 40],+ [10, 0],+ ]);+ expect(anim.tracks.map((t) => t.prop)).toEqual(["opacity", "y"]);+ expect(anim.tracks[0]!.keyframes.map((k) => k.frame)).toEqual([0, 10]);+ });+});diff --git a/packages/compositions/src/program/animation.ts b/packages/compositions/src/program/animation.tsindex 06d0d0ad..d263b0df 100644--- a/packages/compositions/src/program/animation.ts+++ b/packages/compositions/src/program/animation.ts@@ -6,6 +6,16 @@ import type {TrackSource,} from "./schema.js";+/**+ * One keyframe as a tuple: `[frame, value]` or `[frame, value, ease]`. The+ * ease shapes the motion arriving at this keyframe from the one before.+ */+export type KeyframeTuple = readonly [+ frame: number,+ value: number | string | boolean,+ ease?: string | EaseRef,+];+export interface FromToOptions {from?: number;to?: number;@@ -271,6 +281,14 @@ export class LayerAnimation implements LayerAnimationSpec {return new LayerAnimation().fromTo(prop, fromValue, toValue, options);}+ /** A track from a list of `[frame, value, ease?]` keyframes; see {@link LayerAnimation.keys}. */+ public static keys(+ prop: AnimatableProp,+ keyframes: readonly KeyframeTuple[],+ ): LayerAnimation {+ return new LayerAnimation().keys(prop, keyframes);+ }+public static signal(prop: AnimatableProp,signalOrHandleId: unknown,@@ -499,6 +517,18 @@ export class LayerAnimation implements LayerAnimationSpec {return this;}+ /**+ * Adds several keyframes to prop's track at once:+ * `.keys("scale", [[0, 0], [12, 1.1, "back.out"], [20, 1]])`.+ * Frames are rounded to whole frames, so beat math can be passed straight in.+ */+ public keys(prop: AnimatableProp, keyframes: readonly KeyframeTuple[]): this {+ for (const [frame, value, ease] of keyframes) {+ this.keyframe(prop, Math.round(frame), value, ease);+ }+ return this;+ }+/*** Animates opacity from 0 to 1 between startFrame and endFrame.*/diff --git a/packages/compositions/src/program/evaluator.test.ts b/packages/compositions/src/program/evaluator.test.tsindex 3f38432a..df04fac5 100644--- a/packages/compositions/src/program/evaluator.test.ts+++ b/packages/compositions/src/program/evaluator.test.ts@@ -137,6 +137,7 @@ describe("Evaluator — Signal Binding", () => {inputHandleId: "sine_osc",multiplier: 1.0,offset: 1.0,+ smoothingWindowFrames: 0,},keyframes: [],},@@ -396,7 +397,9 @@ describe("Evaluator — Master evaluateTrackAtFrame & evaluateAnimationAtFrame",inputHandleId: "beat_pulse",multiplier: 0.5,offset: 1.0,+ smoothingWindowFrames: 0,},+ keyframes: [],},],};diff --git a/packages/compositions/src/program/schema.test.ts b/packages/compositions/src/program/schema.test.tsindex f6e78cac..7c14629f 100644--- a/packages/compositions/src/program/schema.test.ts+++ b/packages/compositions/src/program/schema.test.ts@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";import {CompositorProgramSchema,EaseRefSchema,+ type FlexNode,type LayoutNode,} from "./schema.js";import {@@ -87,7 +88,7 @@ describe("CompositorProgramSchema (document v2)", () => {const doc = parsed.data;expect(doc.width).toBe(1920);expect(doc.layout).toHaveLength(1);- const hero = doc.layout[0];+ const hero = doc.layout[0] as FlexNode;expect(hero.kind).toBe("flex");expect(hero.position).toBe("relative"); // defaultexpect(hero.zIndex).toBe(0);diff --git a/packages/compositions/src/program/schema.ts b/packages/compositions/src/program/schema.tsindex 482b53cf..b62861f8 100644--- a/packages/compositions/src/program/schema.ts+++ b/packages/compositions/src/program/schema.ts@@ -981,7 +981,8 @@ export const CameraDoFSchema = z}).strict();-export type CameraDoF = z.infer<typeof CameraDoFSchema>;+/** Depth of field as authored: fields with schema defaults may be left out. */+export type CameraDoF = z.input<typeof CameraDoFSchema>;export const CameraShakeSchema = z.object({@@ -1555,7 +1556,8 @@ export const ChartGPUOptionsSchema = zsamples: z.union([z.literal(1), z.literal(4)]).default(4),}).strict();-export type ChartGPUOptions = z.infer<typeof ChartGPUOptionsSchema>;+/** Chart options as authored: fields with schema defaults may be left out. */+export type ChartGPUOptions = z.input<typeof ChartGPUOptionsSchema>;export const ChartNodeSchema: z.ZodType<ChartNode> = z.object({@@ -1631,6 +1633,11 @@ export const CompositorProgramSchema = zvolume: z.number().min(0).max(1).default(1),fps: z.number().int().min(1).max(120).default(24),mode: z.enum(["Video", "Image"]).default("Video"),+ /**+ * Multisample the 3D pass (4 samples per pixel) so mesh edges are smooth.+ * Default true; false saves the extra target memory (~60-125 MB at 1080p).+ */+ antialias3d: z.boolean().optional(),layout: z.array(LayoutNodeSchema).default([]),fonts: z.array(z.string()).optional(),signals: z.record(z.string(), z.unknown()).optional(),diff --git a/packages/compositions/src/program/sequence.ts b/packages/compositions/src/program/sequence.tsindex 24b98fff..86404381 100644--- a/packages/compositions/src/program/sequence.ts+++ b/packages/compositions/src/program/sequence.ts@@ -88,7 +88,7 @@ export const SceneDefinitionSchema = zcarrier: CarrierHandoffConfigSchema.optional(),whipDirection: z.enum(["left", "right", "up", "down"]).default("left"),})- .default({ type: "cut", durationFrames: 0 }),+ .default({ type: "cut", durationFrames: 0, whipDirection: "left" }),}).passthrough();diff --git a/packages/compositions/src/program/text-animations/advanced-types.ts b/packages/compositions/src/program/text-animations/advanced-types.tsindex e75850cf..aeee67c2 100644--- a/packages/compositions/src/program/text-animations/advanced-types.ts+++ b/packages/compositions/src/program/text-animations/advanced-types.ts@@ -340,6 +340,8 @@ export const ExtendedGlyphPropertiesSchema = z.object({rotationX: z.number().optional(),rotationY: z.number().optional(),rotationZ: z.number().optional(),+ /** Volumetric 3D arrangement (set by `volumetricFormation()`) */+ formation: VolumetricFormationConfigSchema.optional(),skew: z.number().optional(),skewAxis: z.number().optional(),diff --git a/packages/compositions/src/program/to-virtual-media.ts b/packages/compositions/src/program/to-virtual-media.tsindex 166caa15..de358868 100644--- a/packages/compositions/src/program/to-virtual-media.ts+++ b/packages/compositions/src/program/to-virtual-media.ts@@ -202,7 +202,8 @@ function nodeToVirtualMedia(// A container without its own timing keeps its parent's clock — its keyframes// are authored parent-relative and its background shows before its first child// arrives — and stays active until its last child ends.- if (node.durationFrames === undefined) {+ // Also when its own durationFrames is unusable (0, NaN): `dur` is unset then.+ if (dur === undefined) {const childEnds = children.map((c) =>((c.operation as any)?.startFrame ?? 0) +@@ -275,6 +276,9 @@ export function compositorToProgram(fps,mode: config.mode ?? "Video",backgroundColor: config.backgroundColor,+ ...(config.antialias3d !== undefined && {+ antialias3d: config.antialias3d,+ }),// Master gain — consumed by the compositor's audio processor when// the extractor dispatches the root op (audio-processor.ts).volume: config.volume ?? 1,diff --git a/packages/compositions/src/utils/audio-extractor.ts b/packages/compositions/src/utils/audio-extractor.tsindex aeb0e4c5..696d6a2d 100644--- a/packages/compositions/src/utils/audio-extractor.ts+++ b/packages/compositions/src/utils/audio-extractor.ts@@ -9,9 +9,15 @@ import { AudioSampleSink } from "mediabunny";import { computeRenderParams } from "./apply-operations.js";import { normalizeTimeline } from "./normalization.js";+// Progress chatter (every decode step) is debug output: shown only with LOG_LEVEL=debug or trace.+const verbose =+ typeof process !== "undefined" &&+ /^(debug|trace)$/i.test(process.env?.LOG_LEVEL ?? "");+// Fallback logger for browser environment. In Node.js environment, we dynamically load @gitframes/server-utils to avoid bundling it on the frontend.let mediaLogger = {info: (msg: unknown, ...args: unknown[]) => {+ if (!verbose) return;if (typeof msg === "object" && msg !== null) {console.info("[AudioExtractor]", msg, ...args);} else {diff --git a/packages/compositions/src/utils/layout/resolve-layout.ts b/packages/compositions/src/utils/layout/resolve-layout.tsindex 98a02948..54158186 100644--- a/packages/compositions/src/utils/layout/resolve-layout.ts+++ b/packages/compositions/src/utils/layout/resolve-layout.ts@@ -25,7 +25,18 @@ import type { Yoga, Node as YogaNode } from "yoga-layout/load";export type { Yoga };import { loadYoga } from "yoga-layout/load";-import type { LayoutNode, SizeSpec } from "../../program/schema.js";+import type {+ CompositionNodeBase,+ LayoutNode,+ SizeSpec,+} from "../../program/schema.js";++/**+ * A node as layout reads it. Camera, light and model nodes carry only some+ * of the base layout fields; the rest read as undefined, which layout handles.+ */+type LaidOutNode = LayoutNode & Partial<CompositionNodeBase>;+type YogaPosition = number | `${number}%`;export interface Rect {x: number;@@ -222,7 +233,8 @@ function containerStyle(node: LayoutNode): {};}-function isAbsolutePositioned(node: LayoutNode): boolean {+function isAbsolutePositioned(layoutNode: LayoutNode): boolean {+ const node = layoutNode as LaidOutNode;return (node.position === "absolute" ||(node.position === undefined &&@@ -259,11 +271,12 @@ interface NodeMapping {function buildYogaNode(yoga: Yoga,- node: LayoutNode,+ layoutNode: LayoutNode,parentExtent: { width?: number; height?: number } | undefined,measure: MeasureFn | undefined,parentDir: "row" | "column" | null,): NodeMapping {+ const node = layoutNode as LaidOutNode;const yn = yoga.Node.create();const dir = dirOf(node);const isContainer = dir !== null;@@ -276,8 +289,9 @@ function buildYogaNode(yn.setPositionType(yoga.POSITION_TYPE_ABSOLUTE);const posX = toYogaValue(node.x);const posY = toYogaValue(node.y);- if (posX !== undefined) yn.setPosition(yoga.EDGE_LEFT, posX);- if (posY !== undefined) yn.setPosition(yoga.EDGE_TOP, posY);+ if (posX !== undefined)+ yn.setPosition(yoga.EDGE_LEFT, posX as YogaPosition);+ if (posY !== undefined) yn.setPosition(yoga.EDGE_TOP, posY as YogaPosition);} else {yn.setPositionType(yoga.POSITION_TYPE_RELATIVE);// Do not set EDGE_LEFT/EDGE_TOP offsets on the Yoga node itself;@@ -572,8 +586,8 @@ export function computeLayoutSync(const rootLayout = mapping.yogaNode.getComputedLayout();const isRootAbsolute =- root.position === "absolute" ||- (root.position === undefined &&+ (root as LaidOutNode).position === "absolute" ||+ ((root as LaidOutNode).position === undefined &&(root.x !== undefined || root.y !== undefined));const rootX = isRootAbsolute ? rootLayout.left : Number(root.x ?? 0);const rootY = isRootAbsolute ? rootLayout.top : Number(root.y ?? 0);diff --git a/packages/core/package.json b/packages/core/package.jsonindex b325e2aa..e15b3184 100644--- a/packages/core/package.json+++ b/packages/core/package.json@@ -29,8 +29,10 @@},"devDependencies": {"@gitframes/tsconfig": "workspace:*",+ "@webgpu/types": "^0.1.70","tsdown": "^0.19.0",- "typescript": "^5.8.3"+ "typescript": "^5.8.3",+ "vitest": "^4.1.5"},"dependencies": {"nanoid": "^5.1.6",diff --git a/packages/core/src/index.ts b/packages/core/src/index.tsindex c052f8ea..6ca21cc8 100644--- a/packages/core/src/index.ts+++ b/packages/core/src/index.ts@@ -2,6 +2,7 @@ export * from "./effects/index.js";export * from "./errors.js";export * from "./id-gen.js";export * from "./node-metadata.js";+export * from "./render-diagnostics.js";export * from "./signals/index.js";export * from "./types/index.js";export * from "./utils/is-equal.js";diff --git a/packages/core/src/render-diagnostics.ts b/packages/core/src/render-diagnostics.tsnew file mode 100644index 00000000..9f54d49b--- /dev/null+++ b/packages/core/src/render-diagnostics.ts@@ -0,0 +1,56 @@+/**+ * Problems a render noticed about its document (as opposed to its output),+ * collected per render id so the renderer can put them in its QA report.+ * Producers report; the renderer takes them when the render ends.+ */+export interface RenderDiagnostic {+ readonly severity: "error" | "warning" | "info";+ /** Stable machine-readable kind, e.g. "animation_truncated" */+ readonly code: string;+ readonly message: string;+ readonly layerId?: string;+}++// Kept on globalThis: bundles can each carry their own copy of this module+// (the compositor reports through one, the renderer collects through another),+// and they must all see one store.+const STORE_KEY = Symbol.for("gitframes.renderDiagnostics");+const diagnosticsByRender: Map<string, Map<string, RenderDiagnostic>> =+ ((globalThis as Record<symbol, unknown>)[STORE_KEY] as+ | Map<string, Map<string, RenderDiagnostic>>+ | undefined) ??+ ((globalThis as Record<symbol, unknown>)[STORE_KEY] = new Map());+// A render that never collects (a crashed or frame-only render) must not grow this forever.+const MAX_TRACKED_RENDERS = 256;++/** Record a diagnostic for a render; repeats (same code, layer and message) are kept once. */+export function reportRenderDiagnostic(+ renderId: string,+ diagnostic: RenderDiagnostic,+): void {+ let forRender = diagnosticsByRender.get(renderId);+ if (!forRender) {+ if (diagnosticsByRender.size >= MAX_TRACKED_RENDERS) {+ const oldest = diagnosticsByRender.keys().next().value;+ if (oldest !== undefined) diagnosticsByRender.delete(oldest);+ }+ forRender = new Map();+ diagnosticsByRender.set(renderId, forRender);+ }+ const key = `${diagnostic.code}|${diagnostic.layerId ?? ""}|${diagnostic.message}`;+ if (!forRender.has(key)) forRender.set(key, diagnostic);+}++/**+ * Everything reported for a render, including its nested compositions (they+ * render under derived ids, `${renderId}-…`), removing it from the store.+ */+export function takeRenderDiagnostics(renderId: string): RenderDiagnostic[] {+ const collected = new Map<string, RenderDiagnostic>();+ for (const [id, forRender] of diagnosticsByRender) {+ if (id !== renderId && !id.startsWith(`${renderId}-`)) continue;+ for (const [key, diagnostic] of forRender) collected.set(key, diagnostic);+ diagnosticsByRender.delete(id);+ }+ return [...collected.values()];+}diff --git a/packages/core/src/signals/programmatic.ts b/packages/core/src/signals/programmatic.tsindex 19565faf..3eb6f4ad 100644--- a/packages/core/src/signals/programmatic.ts+++ b/packages/core/src/signals/programmatic.ts@@ -35,7 +35,9 @@ export class ProgrammaticSignal implements FrameSignal<number> {private _cachedFrame = -1;private _cachedValue = 0;private _seekedFrame = -1;- private _subscribers = new Set<{ markDirty(): void }>();+ // Holds the computeds that read this signal (registerSignalDependency adds them).+ private _subscribers: Parameters<typeof registerSignalDependency>[0] =+ new Set();private _children = new Set<ProgrammaticSignal>();public get fps(): number {diff --git a/packages/core/src/signals/signals.test.ts b/packages/core/src/signals/signals.test.tsindex f458aff6..15d796d4 100644--- a/packages/core/src/signals/signals.test.ts+++ b/packages/core/src/signals/signals.test.ts@@ -110,6 +110,8 @@ describe("Signals Reactive Core", () => {});it("should support primitive valueOf and arithmetic coercion without NaN", () => {+ // Frame-driven signals read the global clock; earlier tests leave it advanced.+ updateClockSignals(0, 24);const stateSig = signal(15);const compSig = computed(() => stateSig.value * 2);const arraySig = frameArraySignal([10, 20, 30], 24);diff --git a/packages/core/src/types/tensor.ts b/packages/core/src/types/tensor.tsindex e69b7a20..23b7b77c 100644--- a/packages/core/src/types/tensor.ts+++ b/packages/core/src/types/tensor.ts@@ -1,3 +1,4 @@+/// <reference types="@webgpu/types" />import type { Signal } from "../signals/index.js";import type { NormalizedLandmarkList } from "./vision.js";diff --git a/packages/gitframes/src/chartgpu-conformance.test.ts b/packages/gitframes/src/chartgpu-conformance.test.tsindex bdde8145..9aeccbbd 100644--- a/packages/gitframes/src/chartgpu-conformance.test.ts+++ b/packages/gitframes/src/chartgpu-conformance.test.ts@@ -61,7 +61,8 @@ describe("ChartGPU Conformance Test Suite (Headless WebGPU & Compositor Integratheight: 200,});- const testOptions: ChartGPUOptions = {+ // The bridge takes chartgpu's own option type, not the document schema's.+ const testOptions: Parameters<ChartGPUEngineBridge["initialize"]>[0] = {theme: "dark",series: [{diff --git a/packages/gitframes/src/font-manager.test.ts b/packages/gitframes/src/font-manager.test.tsindex cc30ac39..c95b9b60 100644--- a/packages/gitframes/src/font-manager.test.ts+++ b/packages/gitframes/src/font-manager.test.ts@@ -1,39 +1,78 @@+import { existsSync } from "node:fs";import fs from "node:fs/promises";+import os from "node:os";import path from "node:path";-import { beforeEach, describe, expect, it } from "vitest";+import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";import { Composition, FontManager, Layer } from "./index.js";+// Fonts are kept on disk, not in git (.gitignore: *.ttf). Every test that+// needs a real font uses the repo's Inter and is skipped where it is absent.+const INTER = "assets/fonts/Inter.ttf";+const hasInter = existsSync(+ path.resolve(import.meta.dirname, "../../..", INTER),+);+if (!hasInter) {+ console.warn(+ `[font-manager.test] ${INTER} not found at the repo root; skipping tests that need a font file.`,+ );+}+describe("FontManager Singleton & Typography Binding", () => {- beforeEach(() => {- FontManager.clear();+ // A second, distinct font file: a copy of Inter registered under its own family.+ let secondFont = "";++ beforeAll(async () => {+ if (!hasInter) return;+ const dir = await fs.mkdtemp(path.join(os.tmpdir(), "font-manager-"));+ secondFont = path.join(dir, "SecondFace.ttf");+ await fs.copyFile(+ path.resolve(import.meta.dirname, "../../..", INTER),+ secondFont,+ );});- it("1. registers a valid TTF font and parses metadata", async () => {- const font = await FontManager.register("assets/fonts/Inter.ttf");- expect(font).toBeDefined();- expect(font.family).toBe("Inter");- expect(font.format).toBe("truetype");- expect(font.unitsPerEm).toBe(2048);- expect(font.filePath).toBeDefined();-- expect(FontManager.has("Inter")).toBe(true);- expect(FontManager.get("Inter")?.family).toBe("Inter");+ afterAll(async () => {+ if (secondFont) {+ await fs.rm(path.dirname(secondFont), { recursive: true, force: true });+ }});- it("2. registers a font with custom family name and options object", async () => {- const font = await FontManager.register({- family: "CinzelDisplay",- source: "assets/fonts/Cinzel.ttf",- weight: 700,- style: "normal",- });-- expect(font.family).toBe("CinzelDisplay");- expect(font.format).toBe("truetype");- expect(font.weight).toBe(700);- expect(FontManager.has("CinzelDisplay")).toBe(true);+ beforeEach(() => {+ FontManager.clear();});+ it.skipIf(!hasInter)(+ "1. registers a valid TTF font and parses metadata",+ async () => {+ const font = await FontManager.register(INTER);+ expect(font).toBeDefined();+ expect(font.family).toBe("Inter");+ expect(font.format).toBe("truetype");+ expect(font.unitsPerEm).toBe(2048);+ expect(font.filePath).toBeDefined();++ expect(FontManager.has("Inter")).toBe(true);+ expect(FontManager.get("Inter")?.family).toBe("Inter");+ },+ );++ it.skipIf(!hasInter)(+ "2. registers a font with custom family name and options object",+ async () => {+ const font = await FontManager.register({+ family: "InterDisplay",+ source: INTER,+ weight: 700,+ style: "normal",+ });++ expect(font.family).toBe("InterDisplay");+ expect(font.format).toBe("truetype");+ expect(font.weight).toBe(700);+ expect(FontManager.has("InterDisplay")).toBe(true);+ },+ );+it("3. validates font binary and rejects non-font files", async () => {const tmpBadFile = path.resolve(process.cwd(), "scratch_not_a_font.txt");await fs.writeFile(@@ -56,51 +95,65 @@ describe("FontManager Singleton & Typography Binding", () => {);});- it("5. automatically provides registered font paths to Composition specs", async () => {- await FontManager.register("assets/fonts/Inter.ttf");- await FontManager.register("assets/fonts/SpaceGrotesk.ttf");-- const comp = new Composition({- width: 640,- height: 360,- fps: 30,- });-- comp.add(Layer.text("Hello World", { fontFamily: "Inter", fontSize: 32 }));- comp.add(- Layer.caption("captions.vtt", {- fontFamily: "SpaceGrotesk",- fontSize: 24,- }),- );+ it.skipIf(!hasInter)(+ "5. automatically provides registered font paths to Composition specs",+ async () => {+ await FontManager.register(INTER);+ await FontManager.register({ family: "SecondFace", source: secondFont });- const spec = comp.toSpec();- expect(spec.fonts).toBeDefined();- expect(spec.fonts?.length).toBeGreaterThanOrEqual(2);- expect(spec.fonts?.some((p) => p.includes("Inter.ttf"))).toBe(true);- expect(spec.fonts?.some((p) => p.includes("SpaceGrotesk.ttf"))).toBe(true);- });-- it("6. comp.registerFont delegates directly to FontManager", async () => {- const comp = new Composition({- width: 640,- height: 360,- });-- await comp.registerFont("MontserratCustom", "assets/fonts/Montserrat.ttf");- expect(FontManager.has("MontserratCustom")).toBe(true);-- const spec = comp.toSpec();- expect(spec.fonts?.some((p) => p.includes("Montserrat.ttf"))).toBe(true);- });+ const comp = new Composition({+ width: 640,+ height: 360,+ fps: 30,+ });- it("7. Layer.text and Layer.caption automatically use registered font family when omitted", async () => {- const font = await FontManager.register("assets/fonts/SpaceGrotesk.ttf");-- const textLayer = Layer.text("Auto Font Text");- const captionLayer = Layer.caption("subtitles.vtt");+ comp.add(+ Layer.text("Hello World", { fontFamily: "Inter", fontSize: 32 }),+ );+ comp.add(+ Layer.caption("captions.vtt", {+ fontFamily: "SecondFace",+ fontSize: 24,+ }),+ );- expect(textLayer.fontFamily).toBe(font.family);- expect(captionLayer.fontFamily).toBe(font.family);- });+ const spec = comp.toSpec();+ expect(spec.fonts).toBeDefined();+ expect(spec.fonts?.length).toBeGreaterThanOrEqual(2);+ expect(spec.fonts?.some((p) => p.includes("Inter.ttf"))).toBe(true);+ expect(spec.fonts?.some((p) => p.includes("SecondFace.ttf"))).toBe(true);+ },+ );++ it.skipIf(!hasInter)(+ "6. comp.registerFont delegates directly to FontManager",+ async () => {+ const comp = new Composition({+ width: 640,+ height: 360,+ });++ await comp.registerFont("InterCustom", INTER);+ expect(FontManager.has("InterCustom")).toBe(true);++ const spec = comp.toSpec();+ expect(spec.fonts?.some((p) => p.includes("Inter.ttf"))).toBe(true);+ },+ );++ it.skipIf(!hasInter)(+ "7. Layer.text and Layer.caption automatically use registered font family when omitted",+ async () => {+ const font = await FontManager.register({+ family: "SecondFace",+ source: secondFont,+ });++ const textLayer = Layer.text("Auto Font Text");+ const captionLayer = Layer.caption("subtitles.vtt");++ expect(textLayer.fontFamily).toBe(font.family);+ expect(captionLayer.fontFamily).toBe(font.family);+ },+ );});diff --git a/packages/gitframes/src/fonts/manager.ts b/packages/gitframes/src/fonts/manager.tsindex f6316ad4..13eb8341 100644--- a/packages/gitframes/src/fonts/manager.ts+++ b/packages/gitframes/src/fonts/manager.ts@@ -302,7 +302,7 @@ export class FontManager {if (filePath) {await registerHeadlessFont(resolvedFamily, filePath);}- SlugFontCache.parsedFontCache.set(resolvedFamily, parsedFontkit);+ SlugFontCache.setParsed(resolvedFamily, parsedFontkit);return registered;}diff --git a/packages/gitframes/src/http-assets.test.ts b/packages/gitframes/src/http-assets.test.tsindex 4af43895..cc9454a3 100644--- a/packages/gitframes/src/http-assets.test.ts+++ b/packages/gitframes/src/http-assets.test.ts@@ -214,7 +214,7 @@ describe("Web HTTP Asset Loading Pipeline", () => {Layer.text("Cloud Typography", {fontFamily: "CloudInter",fontSize: 48,- color: "#ffffff",+ fill: "#ffffff",}),);diff --git a/packages/gitframes/src/ids.ts b/packages/gitframes/src/ids.tsnew file mode 100644index 00000000..b1949938--- /dev/null+++ b/packages/gitframes/src/ids.ts@@ -0,0 +1,12 @@+/**+ * Default ids for nodes created without one. A counter per prefix rather than+ * a random suffix: the same composition script builds the same ids on every+ * run and in every render worker, so specs, caches and diffs stay stable.+ */+const counters = new Map<string, number>();++export function autoId(prefix: string): string {+ const n = (counters.get(prefix) ?? 0) + 1;+ counters.set(prefix, n);+ return `${prefix}-auto-${n}`;+}diff --git a/packages/gitframes/src/index.ts b/packages/gitframes/src/index.tsindex ac19c6a6..499d5e3e 100644--- a/packages/gitframes/src/index.ts+++ b/packages/gitframes/src/index.ts@@ -20,6 +20,7 @@ import {type RegisteredFont,type RegisterFontOptions,} from "./fonts/index.js";+import { autoId } from "./ids.js";import { normalizeTextSpans, type TextSpan } from "./rich-text.js";export {@@ -149,6 +150,7 @@ import {import {analyzeSequence as analyzeVisionFrames,type LandmarkCoordinateSignals,+ type MaskTrackSignals,type ObjectTrackSignals,type PinToLandmarkOptions,type PinToObjectOptions,@@ -227,6 +229,8 @@ import {type FrameGridOptions,HeadlessMediaRenderer,renderFrameGrid,+ type VideoQaOptions,+ type VideoQaReport,} from "./renderer/index.js";import {type Carousel3DOptions,@@ -260,14 +264,31 @@ export interface SectionOptions {export type SectionNode = AnimatableNode<BoxNode> & {readonly innerMedia: AnimatableNode<MediaNode>;- readonly effects: Effect[];- addEffect(effect: Effect): SectionNode;- withEffect(effect: Effect): SectionNode;- withEffects(effects: Effect[]): SectionNode;+ // Any effect, whatever its config shape (Blur, FilmGrain, …).+ readonly effects: Effect<object>[];+ addEffect(effect: Effect<object>): SectionNode;+ withEffect(effect: Effect<object>): SectionNode;+ withEffects(effects: Effect<object>[]): SectionNode;};export type TrackedRegionOptions = SectionOptions;+/** Editing helpers attached to every tracked object (see decorateTrackSignals). */+export interface TrackEditing {+ /** A section of the source that follows this track */+ section(+ sourceOrOptions?: string | MediaNode | SectionOptions,+ options?: SectionOptions,+ ): SectionNode;+ /** A blur confined to this track's region */+ blurEffect(options?: Partial<BlurProps>): Blur;+ /** The tracked region of `source`, cut out as a section */+ isolate(source: string | MediaNode, options?: TrackedRegionOptions): SectionNode;+}++/** A tracked object with its editing helpers, as vision hands it out. */+export type EditableObjectTrack = ObjectTrackSignals & TrackEditing;+function decorateTrackSignals<T extends ObjectTrackSignals>(track: T,source?: string | MediaNode,@@ -471,6 +492,7 @@ export {getScramblePool,getShuffledOrder,type Keyframe,+ type KeyframeTuple,type KineticMarqueeConfig,KineticMarqueeConfigSchema,LayerAnimation,@@ -703,6 +725,11 @@ export interface CompositionOptions {backgroundColor?: string;fonts?: string[];signals?: Record<string, unknown>;+ /**+ * Smooth the edges of 3D meshes (extruded text, models) with 4x MSAA.+ * Default true; set false to save ~60-125 MB of GPU memory at 1080p.+ */+ antialias3d?: boolean;}/**@@ -819,25 +846,9 @@ export class Media {* Renders this media pipeline into an MP4/WebM video file with audio.*/public async renderVideo(- outputPathOrOptions:- | string- | {- codec?: string;- audioCodec?: string;- quality?: string;- concurrency?: number;- outputPath?: string;- renderer?: HeadlessMediaRenderer;- } = {},- options: {- codec?: string;- audioCodec?: string;- quality?: string;- concurrency?: number;- outputPath?: string;- renderer?: HeadlessMediaRenderer;- } = {},- ): Promise<{ filePath: string; cleanup: () => Promise<void> }> {+ outputPathOrOptions: string | RenderVideoOptions = {},+ options: RenderVideoOptions = {},+ ): Promise<RenderVideoResult> {const opts =typeof outputPathOrOptions === "string"? { ...options, outputPath: outputPathOrOptions }@@ -852,6 +863,29 @@ export class Media {}}+export interface RenderVideoOptions {+ codec?: string;+ audioCodec?: string;+ quality?: string;+ concurrency?: number;+ outputPath?: string;+ renderer?: HeadlessMediaRenderer;+ /**+ * Check the output while it renders and return the findings as `qa`:+ * dropped, black or frozen frames, loudness (LUFS), clipping, silence and+ * document warnings such as truncated animations. `true` uses defaults;+ * pass thresholds (e.g. `{ targetLufs: -14 }`) to tighten them.+ */+ qa?: boolean | VideoQaOptions;+}++export interface RenderVideoResult {+ filePath: string;+ cleanup: () => Promise<void>;+ /** Present when rendered with `qa`; print it with `formatQaReport` */+ qa?: VideoQaReport;+}+export interface SubjectSandwichOptions {readonly source: string | MediaNode;readonly subject?: MaskTrackSignals | ObjectTrackSignals;@@ -898,6 +932,7 @@ export class Composition {public fps: number;public durationMs?: number;public backgroundColor: string;+ public antialias3d?: boolean;public fonts: string[];public children: LayoutNode[] = [];public audioTracks: MediaNode[] = [];@@ -1046,6 +1081,7 @@ export class Composition {? (options.durationFrames / this.fps) * 1000: undefined);this.backgroundColor = options.backgroundColor ?? "#000000";+ this.antialias3d = options.antialias3d;this.fonts = options.fonts ?? [];this.signals = options.signals ?? {};}@@ -1148,7 +1184,7 @@ export class Composition {const src =typeof options.source === "string"? options.source- : (options.source.src ?? "");+ : (options.source.inputHandleId ?? "");const fit = options.fit ?? "cover";const feather = options.feather ?? 4;const featherVal = feather > 1 ? feather / 100 : feather;@@ -1205,7 +1241,7 @@ export class Composition {const src =typeof options.source === "string"? options.source- : (options.source.src ?? "");+ : (options.source.inputHandleId ?? "");const damping = Math.max(0, Math.min(0.99, options.damping ?? 0.8));const headroom = options.leadHeadroom ?? 0.18;const compW = this.width;@@ -1325,7 +1361,7 @@ export class Composition {volume: options.muted ? 0 : (options.volume ?? 1),});- container.children.push(videoPlate);+ (container.children ??= []).push(videoPlate);this.add(container);return this;}@@ -1353,7 +1389,7 @@ export class Composition {const src =typeof options.source === "string"? options.source- : (options.source?.src ?? "");+ : (options.source?.inputHandleId ?? "");const outlineLayer = Layer.box({width: compW,@@ -1378,7 +1414,7 @@ export class Composition {mode: "mask",enableSegmentation: true,});- outlineLayer.children.push(maskVideo);+ (outlineLayer.children ??= []).push(maskVideo);}outlineLayer.apply(@@ -1547,6 +1583,7 @@ export class Composition {fps: this.fps,mode: "Video",backgroundColor: this.backgroundColor,+ ...(this.antialias3d !== undefined && { antialias3d: this.antialias3d }),layout: layoutItems,fonts: mergedFonts,signals: this.signals,@@ -1652,25 +1689,9 @@ export class Composition {* Renders this composition into an MP4/WebM video file with audio.*/public async renderVideo(- outputPathOrOptions:- | string- | {- codec?: string;- audioCodec?: string;- quality?: string;- concurrency?: number;- outputPath?: string;- renderer?: HeadlessMediaRenderer;- } = {},- options: {- codec?: string;- audioCodec?: string;- quality?: string;- concurrency?: number;- outputPath?: string;- renderer?: HeadlessMediaRenderer;- } = {},- ): Promise<{ filePath: string; cleanup: () => Promise<void> }> {+ outputPathOrOptions: string | RenderVideoOptions = {},+ options: RenderVideoOptions = {},+ ): Promise<RenderVideoResult> {const opts =typeof outputPathOrOptions === "string"? { ...options, outputPath: outputPathOrOptions }@@ -2039,7 +2060,7 @@ export const Layer = {options: Partial<MediaNode> = {},): AnimatableNode<MediaNode> =>withAnimation({- id: options.id ?? "video-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("video"),kind: "media",inputHandleId: src,dataType: "Video",@@ -2051,7 +2072,7 @@ export const Layer = {options: Partial<MediaNode> = {},): AnimatableNode<MediaNode> =>withAnimation({- id: options.id ?? "img-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("img"),kind: "media",inputHandleId: src,dataType: "Image",@@ -2063,7 +2084,7 @@ export const Layer = {options: Partial<MediaNode> = {},): AnimatableNode<MediaNode> =>withAnimation({- id: options.id ?? "svg-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("svg"),kind: "media",inputHandleId: src,dataType: "SVG",@@ -2075,7 +2096,7 @@ export const Layer = {options: Partial<MediaNode> = {},): AnimatableNode<MediaNode> =>withAnimation({- id: options.id ?? "lottie-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("lottie"),kind: "media",inputHandleId: src,dataType: "Lottie",@@ -2087,7 +2108,7 @@ export const Layer = {options: Partial<MediaNode> = {},): AnimatableNode<MediaNode> =>withAnimation({- id: options.id ?? "audio-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("audio"),kind: "media",inputHandleId: src,dataType: "Audio",@@ -2135,7 +2156,7 @@ export const Layer = {}return withAnimation({- id: options.id ?? "text-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("text"),kind: "text",fontFamily:options.fontFamily ?? FontManager.getAll()[0]?.family ?? "Inter",@@ -2151,7 +2172,7 @@ export const Layer = {options: Partial<MediaNode> = {},): AnimatableNode<MediaNode> =>withAnimation({- id: options.id ?? "caption-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("caption"),kind: "media",inputHandleId: src,dataType: options.dataType ?? "Caption",@@ -2173,7 +2194,7 @@ export const Layer = {options: Partial<ShapeNode> = {},): AnimatableNode<ShapeNode> =>withAnimation({- id: options.id ?? "shape-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("shape"),kind: "shape",shapeType: options.shapeType ?? shapeType,fillColor: options.fillColor ?? "#3b82f6",@@ -2181,7 +2202,7 @@ export const Layer = {}),flex: (options: Partial<FlexNode> = {}): AnimatableNode<FlexNode> =>withAnimation({- id: options.id ?? "flex-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("flex"),kind: "flex",dir: options.dir ?? "column",children: options.children ?? [],@@ -2189,7 +2210,7 @@ export const Layer = {}),box: (options: Partial<BoxNode> = {}): AnimatableNode<BoxNode> =>withAnimation({- id: options.id ?? "box-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("box"),kind: "box",background: options.background ?? "transparent",children: options.children ?? [],@@ -2312,9 +2333,7 @@ export const Layer = {const boxId =opts.id ??- (target- ? `section-track-${target.trackId}-${Math.random().toString(36).slice(2, 8)}`- : `section-${Math.random().toString(36).slice(2, 8)}`);+ (target ? autoId(`section-track-${target.trackId}`) : autoId("section"));let box = Layer.box({id: boxId,@@ -2417,14 +2436,14 @@ export const Layer = {props: Partial<ChartNode> = {},): AnimatableNode<ChartNode> =>withAnimation({- id: props.id ?? "chart-" + Math.random().toString(36).slice(2, 9),+ id: props.id ?? autoId("chart"),kind: "chart",chartOptions: options,...props,}),camera: (options: Partial<CameraNode> = {}): AnimatableNode<CameraNode> =>withAnimation<CameraNode>({- id: options.id ?? "camera-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("camera"),kind: "camera",mode: options.mode ?? "lookAt",pitch: options.pitch ?? 0,@@ -2434,7 +2453,7 @@ export const Layer = {}),light: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>withAnimation<LightNode>({- id: options.id ?? "light-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("light"),kind: "light",lightType: options.lightType ?? "point",color: options.color ?? "#ffffff",@@ -2463,7 +2482,7 @@ export const Layer = {Layer3D.prism(options),model: (options: Partial<Model3DNode> = {}): AnimatableNode<Model3DNode> =>withAnimation({- id: options.id ?? "model3d-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("model3d"),kind: "model3d",is3D: options.is3D ?? true,modelFormat: options.modelFormat ?? "auto",@@ -2567,8 +2586,7 @@ export const Light = {options: Partial<LightNode> = {},): AnimatableNode<LightNode> =>withAnimation({- id:- options.id ?? "light-ambient-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("light-ambient"),kind: "light",lightType: "ambient",color,@@ -2577,9 +2595,7 @@ export const Light = {}),directional: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>withAnimation({- id:- options.id ??- "light-directional-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("light-directional"),kind: "light",lightType: "directional",color: options.color ?? "#ffffff",@@ -2594,7 +2610,7 @@ export const Light = {}),point: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>withAnimation({- id: options.id ?? "light-point-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("light-point"),kind: "light",lightType: "point",color: options.color ?? "#ffffff",@@ -2608,7 +2624,7 @@ export const Light = {}),spot: (options: Partial<LightNode> = {}): AnimatableNode<LightNode> =>withAnimation({- id: options.id ?? "light-spot-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("light-spot"),kind: "light",lightType: "spot",color: options.color ?? "#ffffff",@@ -2632,11 +2648,18 @@ export {type FrameGridOptions,type FrameGridRenderable,type FrameSamplePoint,+ formatQaReport,HeadlessMediaRenderer,HeadlessWebGPURenderer,+ type QaIssue,+ type QaSegment,renderFrameGrid,renderSemaphore,resolveSamplePoints,+ type AudioQaStats,+ type VideoQaOptions,+ type VideoQaReport,+ type VideoQaStats,} from "./renderer/index.js";export function section(diff --git a/packages/gitframes/src/lights-3d-render.test.ts b/packages/gitframes/src/lights-3d-render.test.tsindex 506bee14..c4213b54 100644--- a/packages/gitframes/src/lights-3d-render.test.ts+++ b/packages/gitframes/src/lights-3d-render.test.ts@@ -1,6 +1,6 @@import { Canvas, loadImage } from "skia-canvas";import { describe, expect, it } from "vitest";-import { Composition, Layer, LayerAnimation, Light } from "./index.js";+import { Composition, Layer, LayerAnimation } from "./index.js";interface DecodedImage {width: number;diff --git a/packages/gitframes/src/model3d-render.test.ts b/packages/gitframes/src/model3d-render.test.tsindex dddd6ecc..a6556b55 100644--- a/packages/gitframes/src/model3d-render.test.ts+++ b/packages/gitframes/src/model3d-render.test.ts@@ -1,7 +1,7 @@import fs from "node:fs/promises";import os from "node:os";import path from "node:path";-import { Signal, signal } from "@gitframes/core";+import { Signal } from "@gitframes/core";import { Canvas, loadImage } from "skia-canvas";import { describe, expect, it } from "vitest";import { Composition, Layer, Layer3D, LayerAnimation, Light } from "./index.js";diff --git a/packages/gitframes/src/render-conformance.test.ts b/packages/gitframes/src/render-conformance.test.tsindex 1df57c3a..af09714f 100644--- a/packages/gitframes/src/render-conformance.test.ts+++ b/packages/gitframes/src/render-conformance.test.ts@@ -1,4 +1,3 @@-import fs from "node:fs/promises";import type { VirtualMediaData } from "@gitframes/core";import { Canvas, loadImage } from "skia-canvas";import { describe, expect, it } from "vitest";diff --git a/packages/gitframes/src/renderer/index.ts b/packages/gitframes/src/renderer/index.tsindex fbd89382..fa53f35d 100644--- a/packages/gitframes/src/renderer/index.ts+++ b/packages/gitframes/src/renderer/index.ts@@ -1,7 +1,14 @@export {+ type AudioQaStats,+ formatQaReport,HeadlessMediaRenderer,HeadlessWebGPURenderer,+ type QaIssue,+ type QaSegment,renderSemaphore,+ type VideoQaOptions,+ type VideoQaReport,+ type VideoQaStats,} from "@gitframes/renderer";export { initHeadlessWebGPU } from "@gitframes/webgpu-renderers";diff --git a/packages/gitframes/src/scene-3d-regressions.test.ts b/packages/gitframes/src/scene-3d-regressions.test.tsindex 5196db8a..db41deeb 100644--- a/packages/gitframes/src/scene-3d-regressions.test.ts+++ b/packages/gitframes/src/scene-3d-regressions.test.ts@@ -3,7 +3,13 @@ import os from "node:os";import path from "node:path";import { Canvas, loadImage } from "skia-canvas";import { describe, expect, it } from "vitest";-import { Composition, GradientMap, Layer, Layer3D } from "./index.js";+import {+ type BoxNode,+ Composition,+ GradientMap,+ Layer,+ Layer3D,+} from "./index.js";const W = 800;const H = 600;@@ -34,7 +40,7 @@ describe("3D scene regressions", () => {divisions: 10,lineWidth: 4,});- const lines = grid.children ?? [];+ const lines = (grid.children ?? []) as BoxNode[];// 11 lines each way; long lines are split into segments of at most 4096 px.expect(lines.length).toBe(11 * Math.ceil(14000 / 4096) + 11 * Math.ceil(9000 / 4096),@@ -52,7 +58,7 @@ describe("3D scene regressions", () => {Layer.box({ id: `c${i}`, width: 100, height: 60 }),);const ring = Layer3D.carousel({ radius: 500, items });- const [front, right, back, left] = ring.items ?? [];+ const [front, right, back, left] = (ring.items ?? []) as BoxNode[];// The default camera looks down +z from -z: the front item sits at -radius, unrotated.expect(front.z).toBeCloseTo(-500);expect(front.rotateY).toBeCloseTo(0);diff --git a/packages/gitframes/src/screen-space-relight.test.ts b/packages/gitframes/src/screen-space-relight.test.tsindex cc6fdc16..de96b7fa 100644--- a/packages/gitframes/src/screen-space-relight.test.ts+++ b/packages/gitframes/src/screen-space-relight.test.ts@@ -165,8 +165,10 @@ describe("Screen-Space 3D Normal Relighting Conformance", () => {z: 50,radius: 200,decay: 1.0,+ // Sweep across over the first second (30 fps).animation: LayerAnimation.create().fromTo("x", 80, 320, {- duration: 1.0,+ start: 0,+ end: 30,}),}),);diff --git a/packages/gitframes/src/section-effects.test.ts b/packages/gitframes/src/section-effects.test.tsindex ed7902a6..c0401153 100644--- a/packages/gitframes/src/section-effects.test.ts+++ b/packages/gitframes/src/section-effects.test.ts@@ -72,7 +72,7 @@ describe("Unified Composition & Layer Section Architecture Suite", () => {borderRadius: 16,borderColor: "#38bdf8",borderWidth: 2,- effects: [new Blur({ strength: 24 }), new FilmGrain({ intensity: 0.35 })],+ effects: [new Blur({ strength: 24 }), new FilmGrain({ strength: 35 })],});expect(section.kind).toBe("box");@@ -148,8 +148,8 @@ describe("Unified Composition & Layer Section Architecture Suite", () => {height: 400,borderRadius: 20,effects: [- new Vignette({ radius: 0.7, amount: 0.8 }),- new FilmGrain({ intensity: 0.4 }),+ new Vignette({ radius: 0.7, strength: 80 }),+ new FilmGrain({ strength: 40 }),],});@@ -226,7 +226,7 @@ describe("Unified Composition & Layer Section Architecture Suite", () => {// Chain multiple effectssection.withEffects([- new FilmGrain({ intensity: 0.2 }),+ new FilmGrain({ strength: 20 }),new Vignette({ radius: 0.9 }),]);expect(section.effects).toHaveLength(3);@@ -246,7 +246,7 @@ describe("Unified Composition & Layer Section Architecture Suite", () => {comp.add(video);const sec = comp.addSection(targetPerson, {- effects: [new Blur({ strength: 20 }), new FilmGrain({ intensity: 0.3 })],+ effects: [new Blur({ strength: 20 }), new FilmGrain({ strength: 30 })],});const vm = comp.toVirtualMedia();@@ -299,7 +299,7 @@ describe("Unified Composition & Layer Section Architecture Suite", () => {});expect(sec).toBeDefined();- const png = await comp.renderFrame(0);+ const png = await comp.renderFrame({ frame: 0 });expect(Buffer.isBuffer(png)).toBe(true);expect(png.length).toBeGreaterThan(100);// Validate PNG magic bytesdiff --git a/packages/gitframes/src/shapes-3d-render.test.ts b/packages/gitframes/src/shapes-3d-render.test.tsindex dd2aaad9..0652990a 100644--- a/packages/gitframes/src/shapes-3d-render.test.ts+++ b/packages/gitframes/src/shapes-3d-render.test.ts@@ -1,7 +1,6 @@import { Canvas, loadImage } from "skia-canvas";import { describe, expect, it } from "vitest";import {- CameraAnimation,Composition,Layer,Layer3D,@@ -289,33 +288,23 @@ describe("WebGPU 3D Shapes & Path Follower Headless Conformance", () => {const img15 = await decodePngPixels(f15Buf);const img30 = await decodePngPixels(f30Buf);- // Find where orange pixels are in img0- let found0: { x: number; y: number } | null = null;- for (let y = 0; y < img0.height; y++) {- for (let x = 0; x < img0.width; x++) {- const pix = getPixel(img0, x, y);- if (pix.r > 200 && pix.g > 100 && pix.b < 50) {- found0 = { x, y };- break;+ // First orange (follower) pixel in a frame, scanning row by row.+ const findOrange = (img: typeof img0) => {+ for (let y = 0; y < img.height; y++) {+ for (let x = 0; x < img.width; x++) {+ const pix = getPixel(img, x, y);+ if (pix.r > 200 && pix.g > 100 && pix.b < 50) return { x, y };}}- if (found0) break;- }-- // Find where orange pixels are in img30- let found30: { x: number; y: number } | null = null;- for (let y = 0; y < img30.height; y++) {- for (let x = 0; x < img30.width; x++) {- const pix = getPixel(img30, x, y);- if (pix.r > 200 && pix.g > 100 && pix.b < 50) {- found30 = { x, y };- break;- }- }- if (found30) break;- }+ return null;+ };+ const found0 = findOrange(img0);+ const found15 = findOrange(img15);+ const found30 = findOrange(img30);expect(found0).not.toBeNull();+ // Halfway along the path the follower is still on screen.+ expect(found15).not.toBeNull();expect(found30).not.toBeNull();const p0 = getPixel(img0, found0!.x, found0!.y);diff --git a/packages/gitframes/src/shapes3d.ts b/packages/gitframes/src/shapes3d.tsindex 4869348a..049af630 100644--- a/packages/gitframes/src/shapes3d.ts+++ b/packages/gitframes/src/shapes3d.ts@@ -3,13 +3,13 @@ import type {LayoutNode,MaterialType,Model3DNode,- TextNode,} from "@gitframes/compositions/program";import {generateExtrudedTextGeometry,type Model3DData,SlugFontCache,} from "@gitframes/webgpu-renderers";+import { autoId } from "./ids.js";import { type AnimatableNode, Layer } from "./index.js";export interface CubeFaceConfig {@@ -169,6 +169,10 @@ export interface Text3DOptions {rotateX?: number;rotateY?: number;rotateZ?: number;+ scale?: number;+ scaleX?: number;+ scaleY?: number;+ scaleZ?: number;align?: "start" | "center" | "end" | "left" | "right";verticalAlign?: "top" | "middle" | "bottom";letterSpacing?: number;@@ -193,14 +197,23 @@ export interface Text3DOptions {stroke?: string;strokeWidth?: number;extrusionDepth?: number;+ /** @deprecated No effect: extruded text is one solid mesh, not a stack of slices. */extrusionSteps?: number;+ /**+ * Extruded text only: color of the side walls. `fill` colors the letter+ * faces (front and back). Defaults to `fill`.+ */bevelColor?: string;}export interface ExtrudedText3DOptions extends Text3DOptions {depth: number;+ /** @deprecated No effect: extruded text is one solid mesh, not a stack of slices. */slices?: number;+ /** Fixed straight segments per curve; by default curves split as finely as their on-screen size needs. */curveSegments?: number;+ /** Most a flattened curve strays from the outline, in on-screen px at scale 1 (default 0.1). */+ curveTolerance?: number;}export type Shape3DContainer = AnimatableNode<BoxNode> & {@@ -209,6 +222,16 @@ export type Shape3DContainer = AnimatableNode<BoxNode> & {[Symbol.iterator](): Iterator<LayoutNode>;};+/** Largest initial scale of an extruded text node along its outline's axes. */+function extrudedTextScale(options: ExtrudedText3DOptions): number {+ const scale = options.scale ?? 1;+ const s = Math.max(+ Math.abs(scale * (options.scaleX ?? 1)),+ Math.abs(scale * (options.scaleY ?? 1)),+ );+ return s > 0 ? s : 1;+}+export const Layer3D = {/*** Creates a true 3D volumetric cube with 6 textured or colored faces.@@ -345,7 +368,7 @@ export const Layer3D = {];const container = Layer.box({- id: options.id ?? "cube-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("cube"),position: "absolute",x: options.x ?? 0,y: options.y ?? 0,@@ -401,7 +424,7 @@ export const Layer3D = {});const container = Layer.box({- id: options.id ?? "carousel-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("carousel"),position: "absolute",x: options.x ?? 0,y: options.y ?? 0,@@ -445,7 +468,8 @@ export const Layer3D = {const color =typeof cfg === "string"? cfg- : (cfg?.background ?? `hsl(${(i * 360) / n}, 70%, 50%)`);+ : ((cfg as { background?: string } | undefined)?.background ??+ `hsl(${(i * 360) / n}, 70%, 50%)`);panels.push(Layer.box({@@ -465,7 +489,7 @@ export const Layer3D = {}const container = Layer.box({- id: options.id ?? "prism-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("prism"),position: "absolute",x: options.x ?? 0,y: options.y ?? 0,@@ -495,7 +519,7 @@ export const Layer3D = {*/plane(options: Plane3DOptions): AnimatableNode<BoxNode> {return Layer.box({- id: options.id ?? "plane-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("plane"),position: "absolute",width: options.width,height: options.height,@@ -520,7 +544,7 @@ export const Layer3D = {*/grid(options: Grid3DOptions): AnimatableNode<BoxNode> {const { width, height } = options;- const id = options.id ?? "grid-" + Math.random().toString(36).slice(2, 9);+ const id = options.id ?? autoId("grid");const divisions = Math.max(1, Math.round(options.divisions ?? 10));const lineWidth = options.lineWidth ?? 2;const color = options.color ?? "rgba(59, 130, 246, 0.5)";@@ -583,19 +607,18 @@ export const Layer3D = {},/**- * Creates a standalone planar 3D text node or multi-slice extruded 3D text.+ * Creates a standalone planar 3D text node, or solid extruded 3D text when `extrusionDepth` is set.*/text(options: Text3DOptions): AnimatableNode<LayoutNode> {if (options.extrusionDepth && options.extrusionDepth > 0) {return Layer3D.extrudedText({...options,depth: options.extrusionDepth,- slices: options.extrusionSteps,});}return Layer.text(options.text, {- id: options.id ?? "text3d-" + Math.random().toString(36).slice(2, 9),+ id: options.id ?? autoId("text3d"),position: "absolute",x: options.x ?? 0,y: options.y ?? 0,@@ -630,12 +653,16 @@ export const Layer3D = {},/**- * Creates multi-slice volumetric extruded 3D typography with depth shading.+ * Creates solid extruded 3D typography: `fill` colors the letter faces,+ * `bevelColor` the side walls; curves are tessellated to `curveTolerance`.*/extrudedText(options: ExtrudedText3DOptions): Shape3DContainer {const { depth, text, fill = "#ffffff", bevelColor } = options;let modelData: Model3DData | undefined;+ // Tolerance is in on-screen px: a node scaled up 3x needs 3x finer geometry.+ const curveTolerance =+ (options.curveTolerance ?? 0.1) / extrudedTextScale(options);const parsedFont =(options.fontFamily? SlugFontCache.getParsed(options.fontFamily)@@ -647,28 +674,36 @@ export const Layer3D = {if (parsedFont) {const res = generateExtrudedTextGeometry({text,- font: parsedFont,+ // SlugFontCache holds fontkit fonts under a narrower type.+ font: parsedFont as unknown as Parameters<+ typeof generateExtrudedTextGeometry+ >[0]["font"],fontSize: options.fontSize ?? 48,depth,fill,bevelColor,- curveSegments: options.curveSegments ?? 5,+ curveSegments: options.curveSegments,+ curveTolerance,align: options.align,verticalAlign: options.verticalAlign,letterSpacing: options.letterSpacing,- material: options.material,+ // The mesh's own shading only knows these; glass and the like are+ // layer materials, applied by the model node below.+ material:+ options.material === "lit" ||+ options.material === "unlit" ||+ options.material === "toon"+ ? options.material+ : undefined,shininess: options.shininess,roughness: options.roughness,- specularIntensity: options.specularIntensity,- ambientIntensity: options.ambientIntensity,metallic: options.metallic,});modelData = res.modelData;}const modelNode = Layer.model({- id:- options.id ?? `extruded-text-${Math.random().toString(36).slice(2, 9)}`,+ id: options.id ?? autoId("extruded-text"),modelData,is3D: true,x: options.x ?? 0,@@ -689,11 +724,13 @@ export const Layer3D = {ambientIntensity: options.ambientIntensity ?? 1.0,metallic: options.metallic,twoSided: options.twoSided ?? true,- text3dOptions: options,+ // What the compositor rebuilds the mesh from when modelData did not+ // survive (a spec sent as JSON): the same tolerance as here.+ text3dOptions: { ...options, curveTolerance },});return Object.assign(modelNode, {- faces: [modelNode] as AnimatableNode<BoxNode>[],+ faces: [modelNode] as unknown as AnimatableNode<BoxNode>[],items: [modelNode],[Symbol.iterator]: function* () {yield modelNode;diff --git a/packages/gitframes/src/tracked-region-blur.test.ts b/packages/gitframes/src/tracked-region-blur.test.tsindex 72df46b0..f36449a4 100644--- a/packages/gitframes/src/tracked-region-blur.test.ts+++ b/packages/gitframes/src/tracked-region-blur.test.ts@@ -3,15 +3,15 @@ import {Blur,Composition,Effect,+ type EditableObjectTrack,Layer,- type ObjectTrackSignals,VisionNode,} from "./index.js";describe("Tracked Section Editing & Blur Engine Suite", () => {function setupVisionWithSyntheticTrack(): {vision: ReturnType<typeof VisionNode.attach>;- targetPerson: ObjectTrackSignals;+ targetPerson: EditableObjectTrack;} {const video = Layer.video("test.mp4");const vision = VisionNode.attach(video, {@@ -48,7 +48,10 @@ describe("Tracked Section Editing & Blur Engine Suite", () => {},]);- const targetPerson = vision.objects.byCategory("person");+ // Vision decorates every track with its editing helpers.+ const targetPerson = vision.objects.byCategory(+ "person",+ ) as EditableObjectTrack;return { vision, targetPerson };}@@ -137,7 +140,7 @@ describe("Tracked Section Editing & Blur Engine Suite", () => {// Inside the box, a nested video plate counter-offsets by -screenX + paddingexpect(trackedBox.children).toHaveLength(1);- const innerVideo = trackedBox.children[0] as unknown as {+ const innerVideo = trackedBox.children?.[0] as unknown as {kind: string;x: { get: (ctx: unknown) => number };y: { get: (ctx: unknown) => number };@@ -254,7 +257,7 @@ describe("Tracked Section Editing & Blur Engine Suite", () => {});comp.add(bgCard);- const png = await comp.renderFrame(0);+ const png = await comp.renderFrame({ frame: 0 });expect(Buffer.isBuffer(png)).toBe(true);expect(png.length).toBeGreaterThan(100);// Check PNG signature: 0x89 'P' 'N' 'G'diff --git a/packages/node-sdk/package.json b/packages/node-sdk/package.jsonindex 8c6aa21e..dae58f8f 100644--- a/packages/node-sdk/package.json+++ b/packages/node-sdk/package.json@@ -36,6 +36,7 @@"devDependencies": {"@gitframes/tsconfig": "workspace:*","@types/node": "^24.1.0",+ "@webgpu/types": "^0.1.70","tsdown": "^0.19.0","typescript": "^5.8.3","webgpu": "^0.4.0"diff --git a/packages/node-sdk/renderer/index.ts b/packages/node-sdk/renderer/index.tsindex 38a363f9..d3421ee8 100644--- a/packages/node-sdk/renderer/index.ts+++ b/packages/node-sdk/renderer/index.ts@@ -1,13 +1,10 @@+/// <reference types="@webgpu/types" />import type { VirtualMediaData } from "@gitframes/core";import type { RenderContextValue } from "@gitframes/webgpu-renderers";-// @ts-expect-errorexport type GPUCommandEncoder = globalThis.GPUCommandEncoder;-// @ts-expect-errorexport type GPURenderPassEncoder = globalThis.GPURenderPassEncoder;-// @ts-expect-errorexport type GPUTextureView = globalThis.GPUTextureView;-// @ts-expect-errorexport type GPUTexture = globalThis.GPUTexture;export interface NodeRenderProps {@@ -33,6 +30,8 @@ export interface NodeRenderProps {isVideoMode?: boolean;isPlaying?: boolean;excludeTextures?: GPUTexture[];+ /** Signal values that override the node's own (text animators read them). */+ signals?: Record<string, unknown>;renderChild?: (child: VirtualMediaData,overrides?: Partial<NodeRenderProps>,diff --git a/packages/renderer/src/dma-staging-ring.ts b/packages/renderer/src/dma-staging-ring.tsindex 181f5727..702a682a 100644--- a/packages/renderer/src/dma-staging-ring.ts+++ b/packages/renderer/src/dma-staging-ring.ts@@ -216,6 +216,8 @@ export class DmaStagingRing {frameIndex: number,timestampUs: number,colorSpace?: VideoColorSpaceConfig,+ /** Sees the tightly packed RGBA pixels while they are mapped; must not keep them. */+ inspect?: (rgba: Uint8Array) => void,): Promise<VideoFrame> {this.checkDestroyed();const slot = this.slots[frameIndex % this.capacity];@@ -251,6 +253,8 @@ export class DmaStagingRing {}}+ inspect?.(pixelData);+const vf = new VideoFrame(pixelData, {format: "RGBA",codedWidth: this.width,diff --git a/packages/renderer/src/headless-webgpu-renderer.ts b/packages/renderer/src/headless-webgpu-renderer.tsindex 3a77193e..7c978ed5 100644--- a/packages/renderer/src/headless-webgpu-renderer.ts+++ b/packages/renderer/src/headless-webgpu-renderer.ts@@ -47,9 +47,15 @@ import {drawCompositionTree,mixAudioTracks,} from "@gitframes/compositions";-import { getMediaType, updateClockSignals, type VirtualMediaData } from "@gitframes/core";+import {+ getMediaType,+ takeRenderDiagnostics,+ updateClockSignals,+ type VirtualMediaData,+} from "@gitframes/core";import { rendererLogger } from "@gitframes/server-utils";import {+ clearAllVideoCache,ensureDevice,getRenderer2D,initHeadlessWebGPU,@@ -82,6 +88,15 @@ import sharp from "sharp";import { Canvas as SkiaCanvas, Image as SkiaImage } from "skia-canvas";import { preloadFonts } from "./asset-preloader.js";import { discoverAndRegisterNodeRenderers } from "./dynamic-node-discovery.js";+import {+ type AudioQaStats,+ analyzeAudio,+ buildQaReport,+ FrameInspector,+ type VideoQaOptions,+ type VideoQaReport,+ type VideoQaStats,+} from "./video-qa.js";import { ZeroCopyWebCodecsPipeline } from "./zero-copy-webcodecs-pipeline.js";// ─── Bootstrap ─────────────────────────────────────────────────────────────────@@ -471,7 +486,11 @@ export class HeadlessWebGPURenderer {surface = new NodeSurfaceProvider(device, width, height);renderer = getRenderer2D(device, surface.colorFormat);const ctx: RenderContextValue = { device, renderer, surface };- updateClockSignals(frame, fps, virtualMedia.metadata.durationMs);+ updateClockSignals(+ frame,+ fps,+ virtualMedia.metadata.durationMs ?? undefined,+ );const needsPrePass =frame === 0 ||@@ -604,7 +623,9 @@ export class HeadlessWebGPURenderer {options: { atMs?: number; frame?: number; fps?: number } = {},): Promise<Buffer> {const vm =- typeof source === "object" && source !== null && "toVirtualMedia" in source+ typeof source === "object" &&+ source !== null &&+ "toVirtualMedia" in source? (source as { toVirtualMedia(): VirtualMediaData }).toVirtualMedia(): (source as VirtualMediaData);const fps = options.fps ?? (vm.metadata?.fps || 30);@@ -828,8 +849,18 @@ export class HeadlessWebGPURenderer {audioCodec?: string;quality?: string;concurrency?: number;+ /**+ * Measure the output while it renders (black and frozen frames,+ * dropped frames, loudness, clipping, silence, document warnings)+ * and return the findings as `qa`. `true` uses default thresholds.+ */+ qa?: boolean | VideoQaOptions;},- ): Promise<{ filePath: string; cleanup: () => Promise<void> }> {+ ): Promise<{+ filePath: string;+ cleanup: () => Promise<void>;+ qa?: VideoQaReport;+ }> {await HeadlessWebGPURenderer.initialize();const codecOption = options?.codec ?? "h264";@@ -876,6 +907,11 @@ export class HeadlessWebGPURenderer {const totalFrames = Math.max(1, Math.round((durationMs / 1000) * fps));const renderId = `vid-${randomUUID()}`;+ const qaOptions: VideoQaOptions | undefined =+ options?.qa === true ? {} : options?.qa ? options.qa : undefined;+ const inspector = qaOptions ? new FrameInspector(qaOptions) : undefined;+ let videoQa: VideoQaStats | undefined;+ let audioQa: AudioQaStats | undefined;const videoCodec: "avc" | "vp9" | "vp8" =codecOption === "vp9" ? "vp9" : codecOption === "vp8" ? "vp8" : "avc";const audioCodec: "aac" | "opus" | "mp3" =@@ -1014,6 +1050,9 @@ export class HeadlessWebGPURenderer {fps,videoSource,ringCapacity: 2,+ onFrame: inspector+ ? (rgba) => inspector.inspect(rgba, width, height)+ : undefined,});const ctx: RenderContextValue | null =@@ -1114,6 +1153,16 @@ export class HeadlessWebGPURenderer {if (pipeline) {await pipeline.drain();const pipelineStats = pipeline.getStats();+ if (inspector && qaOptions) {+ videoQa = inspector.stats(+ totalFrames,+ pipelineStats.totalFramesEncoded,+ fps,+ width,+ height,+ qaOptions,+ );+ }rendererLogger.debug({ renderId, totalFrames, ...pipelineStats },`[HeadlessWebGPURenderer] All frames rendered and encoded (avg cycle: ${pipelineStats.averageCycleTimeMs.toFixed(2)}ms, throughput: ${pipelineStats.throughputFps.toFixed(1)} fps)`,@@ -1134,6 +1183,8 @@ export class HeadlessWebGPURenderer {if (numChannels === 0)throw new Error("mixAudioTracks returned no channels");+ if (qaOptions) audioQa = analyzeAudio(channels, sampleRate, qaOptions);+const totalSamples = channels[0].length;const chunkSize = Math.round(0.1 * sampleRate);@@ -1167,6 +1218,15 @@ export class HeadlessWebGPURenderer {await output.finalize();outputFinalized = true;+ const qa = qaOptions+ ? buildQaReport({+ video: videoQa,+ audio: audioQa,+ diagnostics: takeRenderDiagnostics(renderId),+ options: qaOptions,+ })+ : undefined;+const cleanup = async () => {try {await fs.unlink(tempFilePath).catch(() => {});@@ -1181,10 +1241,10 @@ export class HeadlessWebGPURenderer {"[HeadlessWebGPURenderer] Converting rendered MP4 to high-quality GIF...",);await convertMp4ToGif(tempFilePath, tempGifPath);- return { filePath: tempGifPath, cleanup };+ return { filePath: tempGifPath, cleanup, ...(qa && { qa }) };}- return { filePath: tempFilePath, cleanup };+ return { filePath: tempFilePath, cleanup, ...(qa && { qa }) };} catch (error) {if (outputStarted && !outputFinalized) {try {@@ -1227,6 +1287,8 @@ export class HeadlessWebGPURenderer {try {shaderStore.clear(renderId);} catch {}+ // Drop anything reported for this render that QA did not collect.+ takeRenderDiagnostics(renderId);renderSemaphore.release();rendererLogger.debug({ renderId, ...renderSemaphore.stats },diff --git a/packages/renderer/src/index.ts b/packages/renderer/src/index.tsindex 606869d4..0de6cc97 100644--- a/packages/renderer/src/index.ts+++ b/packages/renderer/src/index.ts@@ -3,4 +3,5 @@ export * from "./dma-staging-ring.js";export * from "./dynamic-node-discovery.js";export * from "./headless-webgpu-renderer.js";export * from "./node-manifest.js";+export * from "./video-qa.js";export * from "./zero-copy-webcodecs-pipeline.js";diff --git a/packages/renderer/src/offscreen-canvas.ts b/packages/renderer/src/offscreen-canvas.tsindex 577c583f..18219784 100644--- a/packages/renderer/src/offscreen-canvas.ts+++ b/packages/renderer/src/offscreen-canvas.ts@@ -1,7 +1,9 @@import { Canvas as SkiaCanvas } from "skia-canvas";-let GL: any = null;-function getGL() { return null; }+// No WebGL backend is bundled; the WebGL paths below stay inert.+function getGL(): any {+ return null;+}// Only polyfill WebGLRenderingContext if not provided by the runtime (e.g. standard Node without Electron)if (typeof globalThis.WebGLRenderingContext === "undefined") {diff --git a/packages/renderer/src/tests/video-qa.test.ts b/packages/renderer/src/tests/video-qa.test.tsnew file mode 100644index 00000000..5106e8da--- /dev/null+++ b/packages/renderer/src/tests/video-qa.test.ts@@ -0,0 +1,140 @@+import { reportRenderDiagnostic, takeRenderDiagnostics } from "@gitframes/core";+import { describe, expect, it } from "vitest";+import {+ analyzeAudio,+ buildQaReport,+ FrameInspector,+ formatQaReport,+ integratedLoudness,+} from "../video-qa.js";++const RATE = 48_000;++function sine(amplitude: number, seconds: number, hz = 997): Float32Array {+ const out = new Float32Array(Math.round(seconds * RATE));+ for (let i = 0; i < out.length; i++) {+ out[i] = amplitude * Math.sin((2 * Math.PI * hz * i) / RATE);+ }+ return out;+}++function frame(width: number, height: number, rgb: number): Uint8Array {+ const px = new Uint8Array(width * height * 4);+ for (let i = 0; i < px.length; i += 4) {+ px[i] = rgb;+ px[i + 1] = rgb;+ px[i + 2] = rgb;+ px[i + 3] = 255;+ }+ return px;+}++describe("integratedLoudness (ITU-R BS.1770)", () => {+ it("reads a -20 dBFS 997 Hz stereo sine as -20 LUFS", () => {+ const ch = sine(0.1, 5);+ expect(integratedLoudness([ch, ch], RATE)).toBeCloseTo(-20, 1);+ });++ it("reads the same sine on one channel 3 LU quieter", () => {+ const ch = sine(0.1, 5);+ const lufs = integratedLoudness([ch, new Float32Array(ch.length)], RATE)!;+ expect(lufs).toBeCloseTo(-23.01, 1);+ });++ it("gates out silence instead of averaging it in", () => {+ const loud = sine(0.1, 3);+ const ch = new Float32Array(loud.length * 3);+ ch.set(loud); // 3 s of tone, 6 s of silence+ expect(integratedLoudness([ch, ch], RATE)).toBeCloseTo(-20, 0);+ });+});++describe("analyzeAudio", () => {+ it("measures peak, clipping and silent stretches", () => {+ const ch = new Float32Array(RATE * 6);+ ch.set(sine(1.2, 2)); // 2 s overdriven, then 4 s silence+ const stats = analyzeAudio([ch, ch], RATE);+ expect(stats.peakDbfs).toBeCloseTo(20 * Math.log10(1.2), 1);+ expect(stats.clippedSamples).toBeGreaterThan(0);+ expect(stats.silentSegments).toHaveLength(1);+ expect(stats.silentSegments[0]!.startSec).toBeCloseTo(2, 1);+ expect(stats.silentSegments[0]!.endSec).toBeCloseTo(6, 1);+ });++ it("reports a silent track with no loudness", () => {+ const stats = analyzeAudio([new Float32Array(RATE * 3)], RATE);+ expect(stats.integratedLufs).toBeNull();+ });+});++describe("FrameInspector", () => {+ it("finds black and frozen stretches", () => {+ const fps = 10;+ const inspector = new FrameInspector();+ // 1 s of moving gray, 1 s of black, 3 s holding one gray+ for (let i = 0; i < 10; i++) inspector.inspect(frame(64, 36, 100 + i * 10), 64, 36);+ for (let i = 0; i < 10; i++) inspector.inspect(frame(64, 36, 0), 64, 36);+ for (let i = 0; i < 30; i++) inspector.inspect(frame(64, 36, 128), 64, 36);++ const stats = inspector.stats(50, 50, fps, 64, 36);+ expect(stats.blackSegments).toEqual([{ startSec: 1, endSec: 2 }]);+ // The black second also holds still, but is shorter than the 2 s minimum.+ expect(stats.frozenSegments).toEqual([{ startSec: 2, endSec: 5 }]);+ });+});++describe("buildQaReport", () => {+ it("fails on warnings and errors, passes with only info", () => {+ const inspector = new FrameInspector();+ for (let i = 0; i < 4; i++) inspector.inspect(frame(8, 8, 200), 8, 8);+ const video = inspector.stats(5, 4, 30, 8, 8);+ const ch = sine(0.1, 2);+ const audio = analyzeAudio([ch, ch], RATE);++ const failing = buildQaReport({+ video,+ audio,+ diagnostics: [+ {+ severity: "warning",+ code: "animation_truncated",+ message: "tail never plays",+ layerId: "title",+ },+ ],+ options: { targetLufs: -14 },+ });+ expect(failing.passed).toBe(false);+ expect(failing.issues.map((i) => i.code)).toEqual([+ "frames_missing",+ "loudness_off_target",+ "animation_truncated",+ ]);+ expect(formatQaReport(failing)).toContain("QA FAILED");++ const passing = buildQaReport({+ video: inspector.stats(4, 4, 30, 8, 8),+ audio,+ });+ expect(passing.passed).toBe(true);+ });+});++describe("render diagnostics", () => {+ it("collects a render's own and its nested compositions' reports once", () => {+ const d = {+ severity: "warning" as const,+ code: "animation_truncated",+ message: "tail never plays",+ layerId: "a",+ };+ reportRenderDiagnostic("vid-1", d);+ reportRenderDiagnostic("vid-1", d); // repeat: kept once+ reportRenderDiagnostic("vid-1-c3-Compositor", { ...d, layerId: "b" });+ reportRenderDiagnostic("vid-10", { ...d, layerId: "other render" });++ expect(takeRenderDiagnostics("vid-1").map((x) => x.layerId)).toEqual(["a", "b"]);+ expect(takeRenderDiagnostics("vid-1")).toEqual([]);+ expect(takeRenderDiagnostics("vid-10")).toHaveLength(1);+ });+});diff --git a/packages/renderer/src/video-qa.ts b/packages/renderer/src/video-qa.tsnew file mode 100644index 00000000..e145e4af--- /dev/null+++ b/packages/renderer/src/video-qa.ts@@ -0,0 +1,466 @@+import type { RenderDiagnostic } from "@gitframes/core";++/**+ * Quality checks run on a video while it renders: frames are measured as+ * they are read back for the encoder and audio as it is mixed, so the report+ * describes exactly what was encoded without decoding the file again.+ */+export interface VideoQaOptions {+ /** Luma (0-1) at or below which a pixel counts as black. Default 0.1. */+ readonly blackPixelThreshold?: number;+ /** Share of black pixels that makes a frame black. Default 0.98. */+ readonly blackFrameRatio?: number;+ /** Shortest run of black frames reported. Default 0.5 s. */+ readonly minBlackSec?: number;+ /** Shortest run of unchanging frames reported. Default 2 s. */+ readonly minFrozenSec?: number;+ /** Level (dBFS, 100 ms windows) below which audio counts as silent. Default -60. */+ readonly silenceDbfs?: number;+ /** Shortest silent stretch reported. Default 2 s. */+ readonly minSilenceSec?: number;+ /** Integrated loudness to hit, e.g. -14 for most streaming platforms. Unchecked when unset. */+ readonly targetLufs?: number;+ /** Allowed distance from `targetLufs`, in LU. Default 1. */+ readonly lufsTolerance?: number;+}++export interface QaSegment {+ readonly startSec: number;+ readonly endSec: number;+}++export interface QaIssue {+ readonly severity: "error" | "warning" | "info";+ /** Stable machine-readable kind, e.g. "black_frames", "audio_clipping" */+ readonly code: string;+ readonly message: string;+ readonly startSec?: number;+ readonly endSec?: number;+ readonly layerId?: string;+}++export interface VideoQaStats {+ readonly expectedFrames: number;+ readonly encodedFrames: number;+ readonly fps: number;+ readonly width: number;+ readonly height: number;+ /** Average luma over the whole video, 0-1 */+ readonly meanLuma: number;+ readonly blackSegments: readonly QaSegment[];+ readonly frozenSegments: readonly QaSegment[];+}++export interface AudioQaStats {+ readonly sampleRate: number;+ readonly channels: number;+ readonly durationSec: number;+ /** Highest sample, dBFS (sample peak, not true peak) */+ readonly peakDbfs: number;+ /** Mean level over the whole track, dBFS */+ readonly rmsDbfs: number;+ /** ITU-R BS.1770 integrated loudness; null when the track is silent */+ readonly integratedLufs: number | null;+ /** Samples at or beyond full scale */+ readonly clippedSamples: number;+ readonly silentSegments: readonly QaSegment[];+}++export interface VideoQaReport {+ /** No errors or warnings (info issues do not fail a render) */+ readonly passed: boolean;+ readonly video?: VideoQaStats;+ readonly audio?: AudioQaStats;+ readonly issues: readonly QaIssue[];+}++const DEFAULTS = {+ blackPixelThreshold: 0.1,+ blackFrameRatio: 0.98,+ minBlackSec: 0.5,+ minFrozenSec: 2,+ silenceDbfs: -60,+ minSilenceSec: 2,+ lufsTolerance: 1,+} as const;++// Every 4th pixel of every 4th row: 1/16 of the frame keeps inspection+// around a millisecond at 1080p while still seeing anything frame-sized.+const SAMPLE_STEP = 4;+// Mean absolute luma change (0-255) under which two frames are the same picture.+const FROZEN_DIFF = 0.25;++/** Measures frames handed to it in order, as RGBA bytes. */+export class FrameInspector {+ private readonly blackLuma: number;+ private readonly blackRatio: number;+ private readonly black: boolean[] = [];+ private readonly sameAsPrevious: boolean[] = [];+ private lumaSum = 0;+ private previous: Uint8Array | null = null;++ constructor(options: VideoQaOptions = {}) {+ this.blackLuma =+ (options.blackPixelThreshold ?? DEFAULTS.blackPixelThreshold) * 255;+ this.blackRatio = options.blackFrameRatio ?? DEFAULTS.blackFrameRatio;+ }++ get frameCount(): number {+ return this.black.length;+ }++ inspect(rgba: Uint8Array, width: number, height: number): void {+ const cols = Math.ceil(width / SAMPLE_STEP);+ const rows = Math.ceil(height / SAMPLE_STEP);+ const luma = new Uint8Array(cols * rows);+ let blackCount = 0;+ let sum = 0;+ let k = 0;+ for (let y = 0; y < height; y += SAMPLE_STEP) {+ let p = y * width * 4;+ for (let x = 0; x < width; x += SAMPLE_STEP, p += SAMPLE_STEP * 4) {+ // BT.709 luma from 8-bit RGB+ const l =+ 0.2126 * rgba[p]! + 0.7152 * rgba[p + 1]! + 0.0722 * rgba[p + 2]!;+ luma[k++] = l;+ sum += l;+ if (l <= this.blackLuma) blackCount++;+ }+ }+ const n = luma.length;+ this.lumaSum += sum / n / 255;+ this.black.push(blackCount / n >= this.blackRatio);++ let same = false;+ if (this.previous && this.previous.length === n) {+ let diff = 0;+ for (let i = 0; i < n; i++)+ diff += Math.abs(luma[i]! - this.previous[i]!);+ same = diff / n < FROZEN_DIFF;+ }+ this.sameAsPrevious.push(same);+ this.previous = luma;+ }++ stats(+ expectedFrames: number,+ encodedFrames: number,+ fps: number,+ width: number,+ height: number,+ options: VideoQaOptions = {},+ ): VideoQaStats {+ const minBlack = Math.max(+ 1,+ Math.round((options.minBlackSec ?? DEFAULTS.minBlackSec) * fps),+ );+ const minFrozen = Math.max(+ 2,+ Math.round((options.minFrozenSec ?? DEFAULTS.minFrozenSec) * fps),+ );+ const blackSegments = runs(this.black, minBlack).map(([a, b]) => ({+ startSec: a / fps,+ endSec: b / fps,+ }));+ // A frozen run starts on the frame before its first repeat.+ const frozenSegments = runs(this.sameAsPrevious, minFrozen - 1).map(+ ([a, b]) => ({ startSec: (a - 1) / fps, endSec: b / fps }),+ );+ return {+ expectedFrames,+ encodedFrames,+ fps,+ width,+ height,+ meanLuma: this.frameCount > 0 ? this.lumaSum / this.frameCount : 0,+ blackSegments,+ frozenSegments,+ };+ }+}++/** [start, end) index ranges where flags hold for at least minLength in a row. */+function runs(+ flags: readonly boolean[],+ minLength: number,+): [number, number][] {+ const out: [number, number][] = [];+ let start = -1;+ for (let i = 0; i <= flags.length; i++) {+ if (i < flags.length && flags[i]) {+ if (start < 0) start = i;+ } else if (start >= 0) {+ if (i - start >= minLength) out.push([start, i]);+ start = -1;+ }+ }+ return out;+}++const toDb = (power: number) =>+ power > 0 ? 10 * Math.log10(power) : -Infinity;++/** Measures a mixed audio track (one Float32Array per channel). */+export function analyzeAudio(+ channels: readonly Float32Array[],+ sampleRate: number,+ options: VideoQaOptions = {},+): AudioQaStats {+ const length = channels[0]?.length ?? 0;+ let peak = 0;+ let sumSquares = 0;+ let clipped = 0;+ for (const ch of channels) {+ for (let i = 0; i < length; i++) {+ const v = Math.abs(ch[i]!);+ if (v > peak) peak = v;+ if (v >= 0.9999) clipped++;+ sumSquares += v * v;+ }+ }+ const total = length * Math.max(1, channels.length);++ const silenceDbfs = options.silenceDbfs ?? DEFAULTS.silenceDbfs;+ const window = Math.max(1, Math.round(sampleRate * 0.1));+ const silent: boolean[] = [];+ for (let start = 0; start < length; start += window) {+ const end = Math.min(length, start + window);+ let s = 0;+ for (const ch of channels)+ for (let i = start; i < end; i++) s += ch[i]! * ch[i]!;+ silent.push(toDb(s / ((end - start) * channels.length)) < silenceDbfs);+ }+ const minSilence = Math.max(+ 1,+ Math.round((options.minSilenceSec ?? DEFAULTS.minSilenceSec) / 0.1),+ );+ const silentSegments = runs(silent, minSilence).map(([a, b]) => ({+ startSec: (a * window) / sampleRate,+ endSec: Math.min(length, b * window) / sampleRate,+ }));++ const peakDbfs = 20 * Math.log10(Math.max(peak, 1e-12));+ return {+ sampleRate,+ channels: channels.length,+ durationSec: length / sampleRate,+ peakDbfs,+ rmsDbfs: toDb(sumSquares / Math.max(1, total)),+ integratedLufs:+ peakDbfs < silenceDbfs ? null : integratedLoudness(channels, sampleRate),+ clippedSamples: clipped,+ silentSegments,+ };+}++interface Biquad {+ b0: number;+ b1: number;+ b2: number;+ a1: number;+ a2: number;+}++/** BS.1770 K-weighting stage 1: +4 dB high shelf around 1.5 kHz (the head). */+function highShelf(sampleRate: number): Biquad {+ const A = 10 ** (4 / 40);+ const w0 = (2 * Math.PI * 1500) / sampleRate;+ const cos = Math.cos(w0);+ const alpha = Math.sin(w0) / (2 / Math.SQRT2);+ const sq = 2 * Math.sqrt(A) * alpha;+ const a0 = A + 1 - (A - 1) * cos + sq;+ return {+ b0: (A * (A + 1 + (A - 1) * cos + sq)) / a0,+ b1: (-2 * A * (A - 1 + (A + 1) * cos)) / a0,+ b2: (A * (A + 1 + (A - 1) * cos - sq)) / a0,+ a1: (2 * (A - 1 - (A + 1) * cos)) / a0,+ a2: (A + 1 - (A - 1) * cos - sq) / a0,+ };+}++/** BS.1770 K-weighting stage 2: 38 Hz high pass, Q 0.5 (the RLB curve). */+function highPass(sampleRate: number): Biquad {+ const w0 = (2 * Math.PI * 38) / sampleRate;+ const cos = Math.cos(w0);+ const alpha = Math.sin(w0) / (2 * 0.5);+ const a0 = 1 + alpha;+ return {+ b0: (1 + cos) / 2 / a0,+ b1: -(1 + cos) / a0,+ b2: (1 + cos) / 2 / a0,+ a1: (-2 * cos) / a0,+ a2: (1 - alpha) / a0,+ };+}++function filter(input: Float32Array, f: Biquad): Float32Array {+ const out = new Float32Array(input.length);+ let x1 = 0;+ let x2 = 0;+ let y1 = 0;+ let y2 = 0;+ for (let i = 0; i < input.length; i++) {+ const x = input[i]!;+ const y = f.b0 * x + f.b1 * x1 + f.b2 * x2 - f.a1 * y1 - f.a2 * y2;+ out[i] = y;+ x2 = x1;+ x1 = x;+ y2 = y1;+ y1 = y;+ }+ return out;+}++/**+ * ITU-R BS.1770-4 integrated loudness (LUFS): K-weighted mean square in+ * 400 ms blocks with 75% overlap, gated at -70 LUFS and then at 10 LU below+ * the mean of what passed. Channels weigh 1 (stereo and mono layouts).+ */+export function integratedLoudness(+ channels: readonly Float32Array[],+ sampleRate: number,+): number | null {+ const shelf = highShelf(sampleRate);+ const hp = highPass(sampleRate);+ const weighted = channels.map((ch) => filter(filter(ch, shelf), hp));+ const length = weighted[0]?.length ?? 0;+ const block = Math.round(0.4 * sampleRate);+ const hop = Math.round(0.1 * sampleRate);+ if (length < block) return null;++ const blockPower: number[] = [];+ for (let start = 0; start + block <= length; start += hop) {+ let power = 0;+ for (const ch of weighted) {+ let s = 0;+ for (let i = start; i < start + block; i++) s += ch[i]! * ch[i]!;+ power += s / block;+ }+ blockPower.push(power);+ }+ const lufs = (power: number) => -0.691 + toDb(power);+ const mean = (xs: number[]) => xs.reduce((a, b) => a + b, 0) / xs.length;++ const absolute = blockPower.filter((p) => lufs(p) > -70);+ if (absolute.length === 0) return null;+ const relativeGate = lufs(mean(absolute)) - 10;+ const gated = absolute.filter((p) => lufs(p) > relativeGate);+ return gated.length > 0 ? lufs(mean(gated)) : null;+}++/** Turn measurements and document diagnostics into a report with issues. */+export function buildQaReport(parts: {+ video?: VideoQaStats;+ audio?: AudioQaStats;+ diagnostics?: readonly RenderDiagnostic[];+ options?: VideoQaOptions;+}): VideoQaReport {+ const { video, audio, options = {} } = parts;+ const issues: QaIssue[] = [];+ const at = (s: QaSegment) =>+ `${s.startSec.toFixed(2)}–${s.endSec.toFixed(2)} s`;++ if (video) {+ if (video.encodedFrames !== video.expectedFrames) {+ issues.push({+ severity: "error",+ code: "frames_missing",+ message: `${video.encodedFrames} of ${video.expectedFrames} frames were encoded`,+ });+ }+ for (const s of video.blackSegments) {+ issues.push({+ severity: "warning",+ code: "black_frames",+ message: `Black frames at ${at(s)}`,+ ...s,+ });+ }+ for (const s of video.frozenSegments) {+ issues.push({+ severity: "info",+ code: "frozen_video",+ message: `Picture does not change at ${at(s)}`,+ ...s,+ });+ }+ }++ if (audio) {+ if (audio.integratedLufs === null) {+ issues.push({+ severity: "info",+ code: "audio_silent",+ message: "The audio track is silent",+ });+ } else {+ if (audio.clippedSamples > 0) {+ issues.push({+ severity: "warning",+ code: "audio_clipping",+ message: `${audio.clippedSamples} audio sample(s) at or beyond full scale (peak ${audio.peakDbfs.toFixed(1)} dBFS)`,+ });+ }+ for (const s of audio.silentSegments) {+ issues.push({+ severity: "info",+ code: "audio_silence",+ message: `Silence at ${at(s)}`,+ ...s,+ });+ }+ if (options.targetLufs !== undefined) {+ const off = audio.integratedLufs - options.targetLufs;+ const tolerance = options.lufsTolerance ?? DEFAULTS.lufsTolerance;+ if (Math.abs(off) > tolerance) {+ issues.push({+ severity: "warning",+ code: "loudness_off_target",+ message: `Integrated loudness ${audio.integratedLufs.toFixed(1)} LUFS is ${Math.abs(off).toFixed(1)} LU ${off > 0 ? "above" : "below"} the ${options.targetLufs} LUFS target`,+ });+ }+ }+ }+ }++ for (const d of parts.diagnostics ?? []) {+ issues.push({+ severity: d.severity,+ code: d.code,+ message: d.message,+ ...(d.layerId !== undefined && { layerId: d.layerId }),+ });+ }++ return {+ passed: !issues.some((i) => i.severity !== "info"),+ ...(video && { video }),+ ...(audio && { audio }),+ issues,+ };+}++/** A short human- and agent-readable summary of a report. */+export function formatQaReport(report: VideoQaReport): string {+ const lines = [`QA ${report.passed ? "passed" : "FAILED"}`];+ const v = report.video;+ if (v) {+ lines.push(+ ` video: ${v.encodedFrames}/${v.expectedFrames} frames, ${v.width}x${v.height} @ ${v.fps} fps, mean luma ${v.meanLuma.toFixed(2)}`,+ );+ }+ const a = report.audio;+ if (a) {+ const lufs =+ a.integratedLufs === null+ ? "silent"+ : `${a.integratedLufs.toFixed(1)} LUFS`;+ lines.push(+ ` audio: ${a.durationSec.toFixed(2)} s, ${lufs}, peak ${a.peakDbfs.toFixed(1)} dBFS, rms ${a.rmsDbfs.toFixed(1)} dBFS`,+ );+ }+ for (const i of report.issues) {+ lines.push(` ${i.severity.padEnd(7)} ${i.code}: ${i.message}`);+ }+ return lines.join("\n");+}diff --git a/packages/renderer/src/zero-copy-webcodecs-pipeline.ts b/packages/renderer/src/zero-copy-webcodecs-pipeline.tsindex b0826a43..c6a3c784 100644--- a/packages/renderer/src/zero-copy-webcodecs-pipeline.ts+++ b/packages/renderer/src/zero-copy-webcodecs-pipeline.ts@@ -12,6 +12,11 @@ export interface ZeroCopyWebCodecsPipelineOptions {readonly ringCapacity?: number;readonly colorSpace?: VideoColorSpaceConfig;readonly directCanvasSource?: unknown;+ /**+ * Called with each frame's RGBA pixels, in frame order, just before it is+ * encoded (staging-ring path only). The buffer is reused: read, don't keep.+ */+ readonly onFrame?: (rgba: Uint8Array, frameIndex: number) => void;}export interface WebCodecsPipelineStats {@@ -46,6 +51,7 @@ export class ZeroCopyWebCodecsPipeline {private readonly stagingRing: DmaStagingRing;private readonly colorSpace?: VideoColorSpaceConfig;private readonly directCanvasSource?: unknown;+ private readonly onFrame?: (rgba: Uint8Array, frameIndex: number) => void;private readonly inFlightFrames: number[] = [];private readonly cycleStartTimes = new Map<number, number>();@@ -68,6 +74,7 @@ export class ZeroCopyWebCodecsPipeline {this.ringCapacity = Math.max(2, options.ringCapacity ?? 2);this.colorSpace = options.colorSpace;this.directCanvasSource = options.directCanvasSource;+ this.onFrame = options.onFrame;this.stagingRing = new DmaStagingRing(device, {width: this.width,@@ -209,13 +216,17 @@ export class ZeroCopyWebCodecsPipeline {sample.close();}} else {+ const onFrame = this.onFrame;const vf = await this.stagingRing.readFrameToVideoFrame(frameIndex,timestampUs,this.colorSpace,+ onFrame && ((rgba) => onFrame(rgba, frameIndex)),);try {- const sample = new VideoSample(vf, {+ // bootstrap-mediabunny installs @napi-rs/webcodecs as the global+ // VideoFrame, so this is the class mediabunny expects at runtime.+ const sample = new VideoSample(vf as unknown as globalThis.VideoFrame, {duration: 1 / this.fps,timestamp: frameIndex / this.fps,});diff --git a/packages/webgpu-renderers/src/loaders/3ds-loader.ts b/packages/webgpu-renderers/src/loaders/3ds-loader.tsindex 0bc0e506..eb991e6f 100644--- a/packages/webgpu-renderers/src/loaders/3ds-loader.ts+++ b/packages/webgpu-renderers/src/loaders/3ds-loader.ts@@ -127,7 +127,7 @@ export class ThreeDSLoader {startOffset: number,endOffset: number,mesh: Raw3DSMesh,- materials: Record<string, Material3D>,+ _materials: Record<string, Material3D>,): void {let offset = startOffset;diff --git a/packages/webgpu-renderers/src/loaders/extended-loaders.test.ts b/packages/webgpu-renderers/src/loaders/extended-loaders.test.tsindex d811c0bc..e027b712 100644--- a/packages/webgpu-renderers/src/loaders/extended-loaders.test.ts+++ b/packages/webgpu-renderers/src/loaders/extended-loaders.test.ts@@ -1,10 +1,10 @@import { describe, expect, it } from "vitest";-import { parseSTL, StlLoader } from "./stl-loader.js";-import { parsePLY, PlyLoader } from "./ply-loader.js";+import { parseSTL } from "./stl-loader.js";+import { parsePLY } from "./ply-loader.js";import { parseGLTF, parseGLB, GltfLoader } from "./gltf-loader.js";import { parseVOX, VoxLoader } from "./vox-loader.js";import { parse3DS, ThreeDSLoader } from "./3ds-loader.js";-import { parseOFF, OffLoader } from "./off-loader.js";+import { parseOFF } from "./off-loader.js";import { loadModel3D } from "../renderer3d/renderer3d.js";describe("Extended 3D Model Loaders (STL, PLY, glTF/GLB, VOX, 3DS, OFF)", () => {diff --git a/packages/webgpu-renderers/src/loaders/gltf-loader.ts b/packages/webgpu-renderers/src/loaders/gltf-loader.tsindex f142c008..ce8a8e21 100644--- a/packages/webgpu-renderers/src/loaders/gltf-loader.ts+++ b/packages/webgpu-renderers/src/loaders/gltf-loader.ts@@ -59,12 +59,6 @@ function componentReader(}}-interface GltfHeader {- magic: number;- version: number;- length: number;-}-interface GltfAccessor {bufferView?: number;byteOffset?: number;@@ -278,7 +272,6 @@ export class GltfLoader {private static unpackGlb(bytes: Uint8Array): { doc: GltfDocument; binBuffer?: Uint8Array } {const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);const magic = view.getUint32(0, true);- const version = view.getUint32(4, true);const totalLength = view.getUint32(8, true);if (magic !== GLB_MAGIC) {diff --git a/packages/webgpu-renderers/src/loaders/loaders.test.ts b/packages/webgpu-renderers/src/loaders/loaders.test.tsindex 8a1d9443..d5ac6a8e 100644--- a/packages/webgpu-renderers/src/loaders/loaders.test.ts+++ b/packages/webgpu-renderers/src/loaders/loaders.test.ts@@ -34,7 +34,7 @@ vt 1.0 1.0vt 0.0 1.0f 1/1/1 2/2/1 3/3/1 4/4/1`;- const model = parseOBJ(objText, { mtlText });+ const model = parseOBJ(objText, mtlText);expect(model.meshes.length).toBe(1);const mesh = model.meshes[0];expect(mesh.materialName).toBe("GoldMaterial");diff --git a/packages/webgpu-renderers/src/loaders/obj-loader.ts b/packages/webgpu-renderers/src/loaders/obj-loader.tsindex e70ff060..ee4dba9c 100644--- a/packages/webgpu-renderers/src/loaders/obj-loader.ts+++ b/packages/webgpu-renderers/src/loaders/obj-loader.ts@@ -1,6 +1,6 @@import fs from "node:fs";import path from "node:path";-import { Matrix4Math, type Vec3, Vector3Math } from "../math3d/index.js";+import { type Vec3, Vector3Math } from "../math3d/index.js";import type {LoadModelOptions,Material3D,@@ -8,6 +8,8 @@ import type {Model3DData,} from "./types.js";+type Vec2 = [number, number];+export class ObjLoader {/*** Parses Wavefront .obj text content (and optional .mtl text) into Model3DData.diff --git a/packages/webgpu-renderers/src/loaders/types.ts b/packages/webgpu-renderers/src/loaders/types.tsindex 389ee99a..6fe7538c 100644--- a/packages/webgpu-renderers/src/loaders/types.ts+++ b/packages/webgpu-renderers/src/loaders/types.ts@@ -25,6 +25,11 @@ export interface Material3D {specularColor?: [number, number, number];ambientColor?: [number, number, number];shininess?: number;+ /**+ * Specular strength parsed from the file (e.g. OBJ `Ks`). The mesh pass+ * lights with the layer's `specularIntensity`; this is not applied yet.+ */+ specularIntensity?: number;roughness?: number;metallic?: number;opacity?: number;diff --git a/packages/webgpu-renderers/src/loaders/vox-loader.ts b/packages/webgpu-renderers/src/loaders/vox-loader.tsindex c98a5373..47e360db 100644--- a/packages/webgpu-renderers/src/loaders/vox-loader.ts+++ b/packages/webgpu-renderers/src/loaders/vox-loader.ts@@ -51,7 +51,6 @@ export class VoxLoader {while (offset + 12 <= bytes.byteLength) {const chunkId = new TextDecoder("ascii").decode(bytes.subarray(offset, offset + 4));const contentSize = view.getUint32(offset + 4, true);- const childrenSize = view.getUint32(offset + 8, true);offset += 12;const chunkDataOffset = offset;@@ -95,9 +94,9 @@ export class VoxLoader {private static buildModel(voxels: Voxel[],palette: Array<[number, number, number, number]>,- sizeX: number,- sizeY: number,- sizeZ: number,+ _sizeX: number,+ _sizeY: number,+ _sizeZ: number,options: LoadModelOptions,): Model3DData {const voxelSet = new Set<string>();diff --git a/packages/webgpu-renderers/src/math3d/camera3d.ts b/packages/webgpu-renderers/src/math3d/camera3d.tsindex 13811a64..b7a7c4d6 100644--- a/packages/webgpu-renderers/src/math3d/camera3d.ts+++ b/packages/webgpu-renderers/src/math3d/camera3d.ts@@ -51,13 +51,6 @@ export interface FrustumPlanes {far: FrustumPlane;}-/** Deterministic pseudo-random gradient noise generator */-function pseudoNoise(t: number, seed: number): number {- const n = Math.sin(t * 12.9898 + seed * 78.233) * 43758.5453123;- const frac = n - Math.floor(n);- return frac * 2.0 - 1.0;-}-/** Smooth multi-harmonic procedural continuous noise */function smoothHarmonicNoise(t: number, seed: number, octaves = 3): number {let total = 0;diff --git a/packages/webgpu-renderers/src/math3d/path3d.ts b/packages/webgpu-renderers/src/math3d/path3d.tsindex 031a5bd0..e2534f4b 100644--- a/packages/webgpu-renderers/src/math3d/path3d.ts+++ b/packages/webgpu-renderers/src/math3d/path3d.ts@@ -160,7 +160,7 @@ export class Path3D {/*** Extracts pitch, yaw, roll Euler angles in degrees from the orthonormal frame.*/- private eulerFromFrame(tangent: Vec3, normal: Vec3, binormal: Vec3): Vec3 {+ private eulerFromFrame(tangent: Vec3, normal: Vec3, _binormal: Vec3): Vec3 {// Tangent is the forward viewing axis Z_fwd or X_fwdconst pitch = Math.asin(Math.max(-1, Math.min(1, -tangent[1]))) * (180 / Math.PI);const yaw = Math.atan2(tangent[0], tangent[2]) * (180 / Math.PI);diff --git a/packages/webgpu-renderers/src/nodes/paragraph.ts b/packages/webgpu-renderers/src/nodes/paragraph.tsindex fc2cc9f8..7292e5ec 100644--- a/packages/webgpu-renderers/src/nodes/paragraph.ts+++ b/packages/webgpu-renderers/src/nodes/paragraph.ts@@ -4,7 +4,11 @@ import type { RenderContextValue } from "../render-context.js";import type { Color } from "../renderer2d/index.js";import { signalRegistry } from "../signals/signal-registry.js";import { parseFontWeight, SlugFontCache } from "../slug/slug-font-cache.js";-import { SlugGeometry } from "../slug/slug-geometry.js";+import {+ SlugGeometry,+ type SlugGlyphLayout,+ type SlugLayoutResult,+} from "../slug/slug-geometry.js";import type { SlugFont } from "../slug/slug-loader.js";import {type AdvancedTextAnimator,@@ -632,7 +636,9 @@ export function applyGlyphAnimators(for (const animator of animators) {if ("selectors" in animator && Array.isArray(animator.selectors)) {const aeAnim = animator as AETextAnimator | AdvancedTextAnimator;- const p = aeAnim.props ?? {};+ // Basic and advanced animators share one props bag; read it as both.+ const p = (aeAnim.props ?? {}) as ExtendedGlyphProperties &+ NonNullable<AETextAnimator["props"]>;const selectorWeight = evaluateAEAnimatorWeight(aeAnim,glyph,@@ -897,14 +903,15 @@ export function drawParagraphNode(const strokeWidthInFontUnits =effectiveStrokeWidth * (slugFont.unitsPerEm / fontSize);- strokeFont = SlugFontCache.getFont(- fontFamily,- props.fontWeight,- ctx.device,- strokeWidthInFontUnits,- fontSize,- props.variableAxes,- );+ strokeFont =+ SlugFontCache.getFont(+ fontFamily,+ props.fontWeight,+ ctx.device,+ strokeWidthInFontUnits,+ fontSize,+ props.variableAxes,+ ) ?? null;}if (!slugFont?.curvesTex || (hasStroke && !strokeFont?.curvesTex)) {@@ -1075,7 +1082,6 @@ export function drawParagraphNode(}}}- const passes = 1;const instanceCount = layout.glyphs.length;const elapsed =diff --git a/packages/webgpu-renderers/src/nodes/signal-integration.test.ts b/packages/webgpu-renderers/src/nodes/signal-integration.test.tsindex fbb1f987..11d009b1 100644--- a/packages/webgpu-renderers/src/nodes/signal-integration.test.ts+++ b/packages/webgpu-renderers/src/nodes/signal-integration.test.ts@@ -2,7 +2,6 @@ import fs from "node:fs";import path from "node:path";import { describe, expect, it } from "vitest";import { ensureDevice } from "../device.js";-import { signalRegistry } from "../signals/signal-registry.js";import { drawSignalNode } from "./signal.js";describe("Signal Node Audio Integration", () => {@@ -84,7 +83,12 @@ describe("Signal Node Audio Integration", () => {height: 256,};- await drawSignalNode(mockCtx as any, encoder, pass, props);+ await drawSignalNode(+ mockCtx as any,+ encoder,+ pass,+ props as unknown as Parameters<typeof drawSignalNode>[3],+ );pass.end();// Copy renderTarget to readback bufferdiff --git a/packages/webgpu-renderers/src/nodes/signal.test.ts b/packages/webgpu-renderers/src/nodes/signal.test.tsindex 40d26443..cda91d4e 100644--- a/packages/webgpu-renderers/src/nodes/signal.test.ts+++ b/packages/webgpu-renderers/src/nodes/signal.test.ts@@ -89,7 +89,12 @@ describe("Signal Node", () => {height: 256,};- await drawSignalNode(mockCtx, mockEncoder, mockPass, props);+ await drawSignalNode(+ mockCtx,+ mockEncoder,+ mockPass,+ props as unknown as Parameters<typeof drawSignalNode>[3],+ );expect(signalRegistry.getOrCreate2DTextureView).toHaveBeenCalledWith(mockDevice,@@ -128,7 +133,12 @@ describe("Signal Node", () => {height: 100,};- await drawSignalNode(mockCtx, mockEncoder, mockPass, props);+ await drawSignalNode(+ mockCtx,+ mockEncoder,+ mockPass,+ props as unknown as Parameters<typeof drawSignalNode>[3],+ );expect(signalRegistry.getOrCreate2DTextureView).not.toHaveBeenCalled();expect(mockPass.draw).not.toHaveBeenCalled();@@ -155,7 +165,12 @@ describe("Signal Node", () => {height: 256,};- await drawSignalNode(mockCtx, mockEncoder, mockPass, props);+ await drawSignalNode(+ mockCtx,+ mockEncoder,+ mockPass,+ props as unknown as Parameters<typeof drawSignalNode>[3],+ );expect(signalRegistry.extractFromAudioSource).toHaveBeenCalledWith(mockDevice,@@ -209,7 +224,12 @@ describe("Signal Node", () => {height: 256,};- await drawSignalNode(mockCtx, mockEncoder, mockPass, math2Props);+ await drawSignalNode(+ mockCtx,+ mockEncoder,+ mockPass,+ math2Props as unknown as Parameters<typeof drawSignalNode>[3],+ );// Audio should ONLY be extracted for audio-extractor-root, NEVER for math-node-1 or math-node-2expect(signalRegistry.extractFromAudioSource).toHaveBeenCalledTimes(1);diff --git a/packages/webgpu-renderers/src/nodes/types.ts b/packages/webgpu-renderers/src/nodes/types.tsindex fdcd1fb1..e53eb052 100644--- a/packages/webgpu-renderers/src/nodes/types.ts+++ b/packages/webgpu-renderers/src/nodes/types.ts@@ -1,4 +1,4 @@-import type { SignalData } from "@gitframes/core";+import type { SignalData, TextSpan } from "@gitframes/core";import type { ChartGPUOptions } from "chartgpu";import type { Rect } from "../renderer2d/index.js";@@ -128,7 +128,8 @@ export interface ParagraphNodeProps {opacity?: number;matrix?: DOMMatrix;lineHeight?: number;- align?: TextAlign;+ /** "start"/"end" follow the line direction (left/right for LTR text) */+ align?: TextAlign | "start" | "end";fontWeight?: number | string;fontStyle?: "normal" | "italic";letterSpacing?: number;@@ -173,6 +174,10 @@ export interface ParagraphNodeProps {durationMs?: number;elapsedMs?: number;isCaption?: boolean;+ /** Rich-text runs, each with its own font, size or color */+ spans?: TextSpan[];+ /** Variable-font axis values, e.g. `{ wght: 650 }` */+ variableAxes?: Record<string, number>;isVideoMode?: boolean;renderId?: string;}diff --git a/packages/webgpu-renderers/src/renderer2d/motion-graphics.test.ts b/packages/webgpu-renderers/src/renderer2d/motion-graphics.test.tsindex 5e8c6cd3..320bebf7 100644--- a/packages/webgpu-renderers/src/renderer2d/motion-graphics.test.ts+++ b/packages/webgpu-renderers/src/renderer2d/motion-graphics.test.ts@@ -1,9 +1,8 @@-import { beforeAll, beforeEach, describe, expect, it, vi } from "vitest";+import { beforeAll, beforeEach, describe, expect, it } from "vitest";import { Renderer2D } from "./index.js";import {createMockCommandEncoder,createMockDevice,- createMockRenderPassEncoder,createMockTexture,ensureDOMGlobals,} from "./test-helpers.js";diff --git a/packages/webgpu-renderers/src/renderer2d/screen-space-relight-pipeline.ts b/packages/webgpu-renderers/src/renderer2d/screen-space-relight-pipeline.tsindex f31d369d..e25a651c 100644--- a/packages/webgpu-renderers/src/renderer2d/screen-space-relight-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer2d/screen-space-relight-pipeline.ts@@ -5,7 +5,6 @@* and surface normal maps in real-time.*/-import type { NormalRelightingOptions } from "@gitframes/core";import { screenSpaceRelightWgsl } from "../shaders/screen-space-relight.js";import { BufferPool } from "./buffer-pool.js";@@ -205,11 +204,7 @@ export class ScreenSpaceRelightPipeline {matData[14] = 0.0;matData[15] = 0.0;- const matBuffer = this.materialPool.getBuffer(- this.device,- matData,- "relight_mat_buffer",- );+ const matBuffer = this.materialPool.getBuffer(this.device, matData);const matBindGroup = this.device.createBindGroup({layout: this.materialLayout,diff --git a/packages/webgpu-renderers/src/renderer2d/test-helpers.ts b/packages/webgpu-renderers/src/renderer2d/test-helpers.tsindex 91a90254..8af1ec31 100644--- a/packages/webgpu-renderers/src/renderer2d/test-helpers.ts+++ b/packages/webgpu-renderers/src/renderer2d/test-helpers.ts@@ -13,6 +13,8 @@ export interface MockBuffer {size: number;usage: number;destroy: any;+ getMappedRange: any;+ unmap: any;}export function createMockTexture(diff --git a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.tsindex 3e1ba988..ca809556 100644--- a/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/mesh3d-pipeline.ts@@ -1,4 +1,3 @@-import type { Mat4 } from "../math3d/index.js";import { BufferPool } from "../renderer2d/buffer-pool.js";import { mesh3dWgsl } from "../shaders/mesh3d.js";@@ -52,6 +51,8 @@ export class Mesh3DPipeline {depthFormat: GPUTextureFormat = "depth24plus",cameraLayout?: GPUBindGroupLayout,lightsLayout?: GPUBindGroupLayout,+ /** Samples per pixel of the pass this draws into (MSAA). */+ sampleCount = 1,) {this.device = device;this.modelPool = new BufferPool(@@ -188,6 +189,7 @@ export class Mesh3DPipeline {topology: "triangle-list",cullMode: "none",},+ multisample: { count: sampleCount },});// MRT pipeline (Color + Linear Depth)@@ -228,6 +230,7 @@ export class Mesh3DPipeline {topology: "triangle-list",cullMode: "none",},+ multisample: { count: sampleCount },});// 1x1 White dummy texturediff --git a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.tsindex d7e973fc..67d52c33 100644--- a/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/quad3d-pipeline.ts@@ -67,6 +67,8 @@ export class Quad3DPipeline {device: GPUDevice,format: GPUTextureFormat,depthFormat: GPUTextureFormat = "depth24plus",+ /** Samples per pixel of the pass this draws into (MSAA). */+ sampleCount = 1,) {this.device = device;this.modelPool = new BufferPool(@@ -187,6 +189,7 @@ export class Quad3DPipeline {topology: "triangle-list",cullMode: "none",},+ multisample: { count: sampleCount },});// 2. MRT pipeline (Color + Linear Depth)@@ -227,6 +230,7 @@ export class Quad3DPipeline {topology: "triangle-list",cullMode: "none",},+ multisample: { count: sampleCount },});// Unit quad centered at origin: [-0.5, -0.5] to [0.5, 0.5] with normal [0, 0, -1]diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.test.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.test.tsindex 94cf4c81..1a9408b7 100644--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.test.ts+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.test.ts@@ -1,4 +1,4 @@-import { beforeAll, beforeEach, describe, expect, it, vi } from "vitest";+import { beforeAll, beforeEach, describe, expect, it } from "vitest";import { Camera3D } from "../math3d/index.js";import { Renderer3D } from "./renderer3d.js";import {@@ -115,8 +115,8 @@ describe("Renderer3D Subsystem", () => {const { ShadowPipeline } = await import("./shadow-pipeline.js");const { GlassPipeline } = await import("./glass-pipeline.js");- const motionBlur = new MotionBlurPipeline(mockDevice, "rgba8unorm");- const ssao = new SSAOPipeline(mockDevice, "rgba8unorm");+ const motionBlur = new MotionBlurPipeline(mockDevice);+ const ssao = new SSAOPipeline(mockDevice);const shadow = new ShadowPipeline(mockDevice, 1024);const glass = new GlassPipeline(mockDevice, "rgba8unorm");diff --git a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts b/packages/webgpu-renderers/src/renderer3d/renderer3d.tsindex a7913570..2bb0587a 100644--- a/packages/webgpu-renderers/src/renderer3d/renderer3d.ts+++ b/packages/webgpu-renderers/src/renderer3d/renderer3d.ts@@ -17,15 +17,13 @@ import {GltfLoader,load3DS,loadFBX,+ loadGLB,loadGLTF,loadOBJ,loadOFF,loadPLY,loadSTL,loadVOX,- ObjLoader,- OffLoader,- PlyLoader,StlLoader,ThreeDSLoader,VoxLoader,@@ -37,11 +35,7 @@ import type {Model3DData,} from "../loaders/types.js";import { type Color, parseColor } from "../color.js";-import {- type GPUMeshBuffers,- Mesh3DPipeline,- type MeshUniformData,-} from "./mesh3d-pipeline.js";+import { type GPUMeshBuffers, Mesh3DPipeline } from "./mesh3d-pipeline.js";import { Quad3DPipeline } from "./quad3d-pipeline.js";import { Slug3DPipeline } from "./slug3d-pipeline.js";import type { SlugGlyphBatch } from "../slug/slug-pipeline.js";@@ -173,6 +167,9 @@ export class Renderer3D {private samplerCache = new SamplerCache();private depthTexture: GPUTexture | null = null;private linearDepthTexture: GPUTexture | null = null;+ // Multisampled targets the pass draws into; they resolve into the caller's views.+ private msaaColorTexture: GPUTexture | null = null;+ private msaaLinearDepthTexture: GPUTexture | null = null;private cameraUniformData = new Float32Array(80); // 320 bytes = 80 floatsprivate meshBuffersCache = new WeakMap<Mesh3DData, GPUMeshBuffers>();private materialTextures = new WeakMap<@@ -184,16 +181,30 @@ export class Renderer3D {private lastCameraPose: Camera3DPose | null = null;private lastCameraBuffer: GPUBuffer | null = null;- constructor(device: GPUDevice, format: GPUTextureFormat) {+ /**+ * Samples per pixel in the 3D pass. Mesh silhouettes (extruded text,+ * models) are rasterized triangles: without MSAA every edge is a hard+ * stair-step. 4 is the count every WebGPU device supports; 1 turns it off.+ */+ public readonly sampleCount: number;++ constructor(device: GPUDevice, format: GPUTextureFormat, sampleCount = 4) {this.device = device;this.format = format;- this.quad3dPipeline = new Quad3DPipeline(device, format);+ this.sampleCount = sampleCount;+ this.quad3dPipeline = new Quad3DPipeline(+ device,+ format,+ "depth24plus",+ sampleCount,+ );this.slug3dPipeline = new Slug3DPipeline(device,format,"depth24plus",this.quad3dPipeline.cameraLayout,this.quad3dPipeline.lightsLayout,+ sampleCount,);this.mesh3dPipeline = new Mesh3DPipeline(device,@@ -201,6 +212,7 @@ export class Renderer3D {"depth24plus",this.quad3dPipeline.cameraLayout,this.quad3dPipeline.lightsLayout,+ sampleCount,);this.audioMeshDeformPipeline = new AudioMeshDeformPipeline(device);@@ -286,12 +298,33 @@ export class Renderer3D {label: "Renderer3DDepthTexture",size: { width, height },format: "depth24plus",+ sampleCount: this.sampleCount,usage: GPUTextureUsage.RENDER_ATTACHMENT,});}return this.depthTexture;}+ private getOrCreateMsaaTexture(+ current: GPUTexture | null,+ label: string,+ format: GPUTextureFormat,+ width: number,+ height: number,+ ): GPUTexture {+ if (current && current.width === width && current.height === height) {+ return current;+ }+ current?.destroy();+ return this.device.createTexture({+ label,+ size: { width, height },+ format,+ sampleCount: this.sampleCount,+ usage: GPUTextureUsage.RENDER_ATTACHMENT,+ });+ }+getOrCreateLinearDepthTexture(width: number, height: number): GPUTexture {if (!this.linearDepthTexture ||@@ -430,22 +463,68 @@ export class Renderer3D {lightsBuffer,);+ const msaa = this.sampleCount > 1;+ if (msaa && loadOp === "load") {+ // The multisampled target starts empty: it cannot carry the view's contents in.+ throw new Error(+ "Renderer3D.beginPass: loadOp 'load' needs a Renderer3D built with sampleCount 1",+ );+ }++ // With MSAA each attachment draws into its multisampled twin and+ // resolves into the caller's view when the pass ends.+ const attach = (+ view: GPUTextureView,+ msaaTex: GPUTexture | null,+ clearValue: GPUColor,+ attachmentLoadOp: GPULoadOp,+ ): GPURenderPassColorAttachment =>+ msaaTex+ ? {+ view: msaaTex.createView(),+ resolveTarget: view,+ loadOp: "clear",+ storeOp: "discard",+ clearValue,+ }+ : { view, loadOp: attachmentLoadOp, storeOp: "store", clearValue };++ if (msaa) {+ this.msaaColorTexture = this.getOrCreateMsaaTexture(+ this.msaaColorTexture,+ "Renderer3DMsaaColor",+ this.format,+ surfaceWidth,+ surfaceHeight,+ );+ }const colorAttachments: GPURenderPassColorAttachment[] = [- {- view: colorTargetView,+ attach(+ colorTargetView,+ msaa ? this.msaaColorTexture : null,+ clearColor,loadOp,- storeOp: "store",- clearValue: clearColor,- },+ ),];if (linearDepthTargetView) {- colorAttachments.push({- view: linearDepthTargetView,- loadOp: "clear",- storeOp: "store",- clearValue: { r: camera.far, g: 0, b: 0, a: 1 },- });+ if (msaa) {+ this.msaaLinearDepthTexture = this.getOrCreateMsaaTexture(+ this.msaaLinearDepthTexture,+ "Renderer3DMsaaLinearDepth",+ "rgba16float",+ surfaceWidth,+ surfaceHeight,+ );+ }+ colorAttachments.push(+ attach(+ linearDepthTargetView,+ msaa ? this.msaaLinearDepthTexture : null,+ { r: camera.far, g: 0, b: 0, a: 1 },+ "clear",+ ),+ );}const pass = encoder.beginRenderPass({@@ -454,7 +533,8 @@ export class Renderer3D {view: depthTex.createView(),depthClearValue: 1.0,depthLoadOp: "clear",- depthStoreOp: "store",+ // Nothing reads depth after the pass; the MSAA copy is never resolved.+ depthStoreOp: msaa ? "discard" : "store",},});@@ -975,6 +1055,8 @@ export class Renderer3D {destroy(): void {this.depthTexture?.destroy();this.linearDepthTexture?.destroy();+ this.msaaColorTexture?.destroy();+ this.msaaLinearDepthTexture?.destroy();this.quad3dPipeline.destroy();this.slug3dPipeline.destroy();this.mesh3dPipeline.destroy();diff --git a/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.tsindex db978bda..fe756b66 100644--- a/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/shadow-pipeline.ts@@ -3,7 +3,7 @@* Directional and Point Light Shadow Map Generator with Percentage Closer Filtering (PCF).*/-import { type Mat4, Matrix4Math, type Vec3, Vector3Math } from "../math3d/index.js";+import { type Mat4, Matrix4Math, type Vec3 } from "../math3d/index.js";export interface ShadowMapOptions {resolution?: number;@@ -14,14 +14,12 @@ export interface ShadowMapOptions {}export class ShadowPipeline {- private device: GPUDevice;public shadowDepthTexture: GPUTexture;public shadowDepthView: GPUTextureView;public shadowComparisonSampler: GPUSampler;public lightViewProjMatrix: Mat4;constructor(device: GPUDevice, resolution = 2048) {- this.device = device;this.lightViewProjMatrix = Matrix4Math.identity();this.shadowDepthTexture = device.createTexture({@@ -52,7 +50,7 @@ export class ShadowPipeline {): Mat4 {const viewMatrix = Matrix4Math.lookAt(lightPos, targetPos, [0, 1, 0]);const half = orthoSize * 0.5;- const projMatrix = Matrix4Math.ortho(-half, half, -half, half, near, far);+ const projMatrix = Matrix4Math.orthographic(-half, half, -half, half, near, far);this.lightViewProjMatrix = Matrix4Math.multiply(projMatrix, viewMatrix);return this.lightViewProjMatrix;}diff --git a/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.tsindex 6ff2a3d0..5a62b3c5 100644--- a/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/slug3d-pipeline.ts@@ -40,6 +40,8 @@ export class Slug3DPipeline {depthFormat: GPUTextureFormat,cameraLayout: GPUBindGroupLayout,lightsLayout: GPUBindGroupLayout,+ /** Samples per pixel of the pass this draws into (MSAA). */+ sampleCount = 1,) {this.device = device;this.modelPool = new BufferPool(@@ -132,6 +134,7 @@ export class Slug3DPipeline {topology: "triangle-list",cullMode: "none",};+ const multisample: GPUMultisampleState = { count: sampleCount };this.pipelineSingle = device.createRenderPipeline({label: "Slug3DPipelineSingle",@@ -144,6 +147,7 @@ export class Slug3DPipeline {},depthStencil,primitive,+ multisample,});this.pipelineMrt = device.createRenderPipeline({@@ -157,6 +161,7 @@ export class Slug3DPipeline {},depthStencil,primitive,+ multisample,});}diff --git a/packages/webgpu-renderers/src/renderer3d/ssao-pipeline.ts b/packages/webgpu-renderers/src/renderer3d/ssao-pipeline.tsindex 784120c2..c2d056ac 100644--- a/packages/webgpu-renderers/src/renderer3d/ssao-pipeline.ts+++ b/packages/webgpu-renderers/src/renderer3d/ssao-pipeline.ts@@ -168,7 +168,11 @@ export class SSAOPipeline {finalAOView: GPUTextureView,options: SSAOPassOptions,): void {- const invProj = options.invProjMatrix ?? Matrix4Math.invert(options.projMatrix);+ // A singular projection has no inverse; identity keeps the pass running.+ const invProj =+ options.invProjMatrix ??+ Matrix4Math.invert(options.projMatrix) ??+ Matrix4Math.identity();// Pack SSAO uniformsconst uBuf = new Float32Array(64);diff --git a/packages/webgpu-renderers/src/renderer3d/text-extruder.test.ts b/packages/webgpu-renderers/src/renderer3d/text-extruder.test.tsindex ecc93a41..449c14ea 100644--- a/packages/webgpu-renderers/src/renderer3d/text-extruder.test.ts+++ b/packages/webgpu-renderers/src/renderer3d/text-extruder.test.ts@@ -1,6 +1,7 @@import fs from "node:fs";import path from "node:path";import * as fontkit from "fontkit";+import { Canvas, Path2D } from "skia-canvas";import { describe, expect, it } from "vitest";import { generateExtrudedTextGeometry } from "./text-extruder.js";@@ -11,7 +12,7 @@ describe("Extruded 3D Text Polygonal Mesh Generator", () => {"../../examples/22_gitframes_launch/assets/fonts/Unbounded.ttf",);const fontBuf = fs.readFileSync(fontPath);- const font = fontkit.create(fontBuf);+ const font = fontkit.create(fontBuf) as fontkit.Font;const result = generateExtrudedTextGeometry({text: "GITFRAMES",@@ -63,7 +64,7 @@ describe("Extruded 3D Text Polygonal Mesh Generator", () => {"../../examples/22_gitframes_launch/assets/fonts/Inter.ttf",);const fontBuf = fs.readFileSync(fontPath);- const font = fontkit.create(fontBuf);+ const font = fontkit.create(fontBuf) as fontkit.Font;const result = generateExtrudedTextGeometry({text: "GITFRAMES",@@ -88,7 +89,7 @@ describe("Extruded 3D Text Polygonal Mesh Generator", () => {"../../examples/22_gitframes_launch/assets/fonts/Unbounded.ttf",);const fontBuf = fs.readFileSync(fontPath);- const font = fontkit.create(fontBuf);+ const font = fontkit.create(fontBuf) as fontkit.Font;const result = generateExtrudedTextGeometry({text: "TFMA",@@ -108,4 +109,98 @@ describe("Extruded 3D Text Polygonal Mesh Generator", () => {expect(result.bounds.width).toBeGreaterThan(250);expect(result.bounds.height).toBeGreaterThan(60);});++ const loadInter = () =>+ fontkit.create(+ fs.readFileSync(+ path.resolve(+ process.cwd(),+ "../../examples/22_gitframes_launch/assets/fonts/Inter.ttf",+ ),+ ),+ ) as fontkit.Font;++ /** Area of the front cap's triangles (vertices at -z, facing the camera). */+ const frontCapArea = (+ result: ReturnType<typeof generateExtrudedTextGeometry>,+ ) => {+ const front = result.modelData.meshes.find((m) => m.name === "front")!;+ const p = front.positions;+ const ix = front.indices;+ let area = 0;+ for (let i = 0; i < ix.length; i += 3) {+ const [a, b, c] = [ix[i]!, ix[i + 1]!, ix[i + 2]!];+ if (p[a * 3 + 2]! >= 0) continue;+ area +=+ Math.abs(+ (p[b * 3]! - p[a * 3]!) * (p[c * 3 + 1]! - p[a * 3 + 1]!) -+ (p[c * 3]! - p[a * 3]!) * (p[b * 3 + 1]! - p[a * 3 + 1]!),+ ) / 2;+ }+ return area;+ };++ it("fills shapes sitting inside a glyph's counter (the R of ®, the C of ©)", () => {+ const font = loadInter();+ const fontSize = 200;+ for (const text of ["O", "®", "©", "@"]) {+ const glyph = font.layout(text).glyphs[0]!;+ const scale = fontSize / font.unitsPerEm;+ // Reference: the glyph filled by skia, measured in covered pixels.+ const canvas = new Canvas(400, 400);+ const ctx = canvas.getContext("2d");+ ctx.translate(100, 300);+ ctx.scale(scale, -scale);+ ctx.fill(new Path2D(glyph.path.toSVG()));+ const data = ctx.getImageData(0, 0, 400, 400).data;+ let trueArea = 0;+ for (let i = 3; i < data.length; i += 4) trueArea += data[i]! / 255;++ const result = generateExtrudedTextGeometry({+ text,+ font,+ fontSize,+ depth: 10,+ });+ expect(frontCapArea(result) / trueArea, text).toBeCloseTo(1, 1);+ }+ });++ it("maps front UVs across the aligned glyphs", () => {+ const result = generateExtrudedTextGeometry({+ text: "GIT",+ font: loadInter(),+ fontSize: 120,+ depth: 20,+ });+ const front = result.modelData.meshes.find((m) => m.name === "front")!;+ let [minU, maxU, minV, maxV] = [1, 0, 1, 0];+ for (let i = 0; i < front.uvs.length; i += 2) {+ minU = Math.min(minU, front.uvs[i]!);+ maxU = Math.max(maxU, front.uvs[i]!);+ minV = Math.min(minV, front.uvs[i + 1]!);+ maxV = Math.max(maxV, front.uvs[i + 1]!);+ }+ expect(minU).toBeCloseTo(0, 5);+ expect(maxU).toBeCloseTo(1, 5);+ expect(minV).toBeCloseTo(0, 5);+ expect(maxV).toBeCloseTo(1, 5);+ // Centered text: bounds straddle the origin.+ expect(result.bounds.minX).toBeCloseTo(-result.bounds.width / 2, 5);+ });++ it("gives the back cap the fill material, not the bevel color", () => {+ const result = generateExtrudedTextGeometry({+ text: "O",+ font: loadInter(),+ fontSize: 100,+ depth: 20,+ });+ const front = result.modelData.meshes.find((m) => m.name === "front")!;+ const side = result.modelData.meshes.find((m) => m.name === "side")!;+ const zNormals = (n: ArrayLike<number>) =>+ Array.from({ length: n.length / 3 }, (_, i) => n[i * 3 + 2]!);+ expect(zNormals(front.normals)).toContain(-1);+ expect(zNormals(side.normals).every((z) => z === 0)).toBe(true);+ });});diff --git a/packages/webgpu-renderers/src/renderer3d/text-extruder.ts b/packages/webgpu-renderers/src/renderer3d/text-extruder.tsindex cb77f9cc..4a183a42 100644--- a/packages/webgpu-renderers/src/renderer3d/text-extruder.ts+++ b/packages/webgpu-renderers/src/renderer3d/text-extruder.ts@@ -15,17 +15,25 @@ export interface ExtrudedTextGeometryOptions {font: fontkit.Font;fontSize?: number;depth?: number;+ /** Fixed straight segments per curve; overrides `curveTolerance`. */curveSegments?: number;- align?: "left" | "center" | "right";+ /**+ * Most a flattened curve may stray from the true outline, in px at+ * `fontSize`. Each curve gets as many segments as its bend needs, so big+ * text and tight bowls stay round while straight stems stay cheap.+ */+ curveTolerance?: number;+ /** "start"/"end" read as left/right (the text runs left to right) */+ align?: "left" | "center" | "right" | "start" | "end";verticalAlign?: "top" | "middle" | "bottom";letterSpacing?: number;+ /** Letter faces: the front cap (toward a default camera, -z) and the back cap */fill?: string;+ /** Side walls; defaults to `fill` */bevelColor?: string;material?: "lit" | "unlit" | "toon";shininess?: number;roughness?: number;- specularIntensity?: number;- ambientIntensity?: number;metallic?: number;}@@ -108,13 +116,43 @@ export function isPointInsidePolygon(p: Point2D, poly: Contour): boolean {return inside;}+export interface CurveFlattening {+ /** Fixed segment count per curve, if set */+ segments?: number;+ /** Max distance between curve and polyline, in output units */+ tolerance: number;+}++const DEFAULT_CURVE_TOLERANCE = 0.1;+const MAX_CURVE_SEGMENTS = 64;++/**+ * Segments needed for a polyline through n uniform steps of a curve to stay+ * within `tolerance`: the chord error is at most max|B''| / (8 n^2).+ */+function curveSegmentCount(+ maxSecondDerivative: number,+ flattening: CurveFlattening,+): number {+ if (flattening.segments !== undefined) return flattening.segments;+ const n = Math.ceil(+ Math.sqrt(maxSecondDerivative / (8 * flattening.tolerance)),+ );+ return Math.min(MAX_CURVE_SEGMENTS, Math.max(1, n));+}+function processCurveQuadratic(p0: Point2D,cp: Point2D,end: Point2D,- segments: number,+ flattening: CurveFlattening,out: Contour,): void {+ // B'' = 2 (p0 - 2 cp + end), constant along the curve.+ const segments = curveSegmentCount(+ 2 * Math.hypot(p0.x - 2 * cp.x + end.x, p0.y - 2 * cp.y + end.y),+ flattening,+ );for (let s = 1; s <= segments; s++) {const t = s / segments;const mt = 1 - t;@@ -130,9 +168,18 @@ function processCurveCubic(cp1: Point2D,cp2: Point2D,end: Point2D,- segments: number,+ flattening: CurveFlattening,out: Contour,): void {+ // B'' is linear in t, so its largest magnitude is at an end: 6 x the larger second difference.+ const segments = curveSegmentCount(+ 6 *+ Math.max(+ Math.hypot(p0.x - 2 * cp1.x + cp2.x, p0.y - 2 * cp1.y + cp2.y),+ Math.hypot(cp1.x - 2 * cp2.x + end.x, cp1.y - 2 * cp2.y + end.y),+ ),+ flattening,+ );for (let s = 1; s <= segments; s++) {const t = s / segments;const mt = 1 - t;@@ -156,7 +203,7 @@ function extractGlyphContours(glyphX: number,glyphY: number,scale: number,- curveSegments: number,+ flattening: CurveFlattening,): Contour[] {const svg = glyph.path?.toSVG();if (!svg) return [];@@ -189,7 +236,8 @@ function extractGlyphContours(for (const edge of edges) {const name = edge[0];- const args = edge.slice(1);+ // skia's edges are [verb, ...coordinates]+ const args = edge.slice(1) as number[];if (name === "moveTo") {finishCurrent();@@ -213,7 +261,7 @@ function extractGlyphContours(x: (glyphX + args[2]!) * scale,y: -(glyphY + args[3]!) * scale,};- processCurveQuadratic(curr, cp, end, curveSegments, current);+ processCurveQuadratic(curr, cp, end, flattening, current);curr = end;} else if (name === "bezierCurveTo") {const cp1: Point2D = {@@ -228,7 +276,7 @@ function extractGlyphContours(x: (glyphX + args[4]!) * scale,y: -(glyphY + args[5]!) * scale,};- processCurveCubic(curr, cp1, cp2, end, curveSegments, current);+ processCurveCubic(curr, cp1, cp2, end, flattening, current);curr = end;} else if (name === "closePath") {finishCurrent();@@ -238,53 +286,103 @@ function extractGlyphContours(return contours;}+/** Side-wall segments meeting at more than this angle (30°) keep a hard edge. */+const SIDE_CREASE_COS = Math.cos((30 * Math.PI) / 180);+interface PolygonHierarchy {outer: Contour;holes: Contour[];}+/**+ * Group a glyph's contours into filled outlines and their holes by nesting+ * depth: a contour inside an even number of others is an outline (the ring+ * of "O", or the "R" sitting in the counter of "®"), one inside an odd number+ * is a hole of the smallest outline around it.+ */function organizeGlyphPolygons(glyphContours: Contour[]): PolygonHierarchy[] {- if (glyphContours.length === 0) return [];-- const sorted = [...glyphContours].sort(- (a, b) =>- Math.abs(computeContourSignedArea(b)) -- Math.abs(computeContourSignedArea(a)),- );-- const outers: PolygonHierarchy[] = [];-- for (const contour of sorted) {- const area = Math.abs(computeContourSignedArea(contour));- let parent: PolygonHierarchy | null = null;-- for (const out of outers) {- const parentArea = Math.abs(computeContourSignedArea(out.outer));- if (area < parentArea) {- // Check sample point or mid-point- const sample = contour[0]!;- const mid = {- x: (contour[0]!.x + contour[Math.floor(contour.length / 2)]!.x) / 2,- y: (contour[0]!.y + contour[Math.floor(contour.length / 2)]!.y) / 2,- };- if (- isPointInsidePolygon(sample, out.outer) ||- isPointInsidePolygon(mid, out.outer)- ) {- parent = out;- break;- }+ const contours = glyphContours.map((contour) => ({+ contour,+ area: Math.abs(computeContourSignedArea(contour)),+ }));+ // Outermost first, so every contour's container is handled before it.+ contours.sort((a, b) => b.area - a.area);++ const containers: number[][] = contours.map(() => []);+ for (let i = 0; i < contours.length; i++) {+ const inner = contours[i]!.contour;+ const mid = inner[Math.floor(inner.length / 2)]!;+ for (let j = 0; j < i; j++) {+ if (contours[j]!.area <= contours[i]!.area) continue;+ const outer = contours[j]!.contour;+ // Contours do not cross (the path is simplified), so a vertex+ // decides; a second guards a vertex lying on the other outline.+ if (+ isPointInsidePolygon(inner[0]!, outer) ||+ isPointInsidePolygon(mid, outer)+ ) {+ containers[i]!.push(j);}}+ }- if (parent) {- parent.holes.push(contour);+ const polygons: PolygonHierarchy[] = [];+ const polygonOf = new Map<number, PolygonHierarchy>();+ for (let i = 0; i < contours.length; i++) {+ const around = containers[i]!;+ if (around.length % 2 === 0) {+ const poly = { outer: contours[i]!.contour, holes: [] };+ polygons.push(poly);+ polygonOf.set(i, poly);} else {- outers.push({ outer: contour, holes: [] });+ // The innermost container is the latest (smallest) one.+ polygonOf+ .get(around[around.length - 1]!)+ ?.holes.push(contours[i]!.contour);}}+ return polygons;+}- return outers;+/**+ * A glyph's outlines are flattened and sorted into outlines and holes once per+ * font, size and flattening, at the glyph origin; each use gets a translated+ * copy (callers reorder and move the points they are given).+ */+const glyphOutlineCache = new WeakMap<+ object,+ Map<string, PolygonHierarchy[]>+>();++function glyphPolygonsAt(+ font: fontkit.Font,+ glyph: fontkit.Glyph,+ glyphX: number,+ glyphY: number,+ scale: number,+ flattening: CurveFlattening,+): PolygonHierarchy[] {+ let byGlyph = glyphOutlineCache.get(font);+ if (!byGlyph) {+ byGlyph = new Map();+ glyphOutlineCache.set(font, byGlyph);+ }+ const key = `${glyph.id}|${scale}|${flattening.segments ?? ""}|${flattening.tolerance}`;+ let atOrigin = byGlyph.get(key);+ if (!atOrigin) {+ atOrigin = organizeGlyphPolygons(+ extractGlyphContours(glyph, 0, 0, scale, flattening),+ );+ byGlyph.set(key, atOrigin);+ }+ const dx = glyphX * scale;+ const dy = -glyphY * scale;+ const moved = (c: Contour): Contour =>+ c.map((p) => ({ x: p.x + dx, y: p.y + dy }));+ return atOrigin.map((poly) => ({+ outer: moved(poly.outer),+ holes: poly.holes.map(moved),+ }));}export function generateExtrudedTextGeometry(@@ -294,7 +392,16 @@ export function generateExtrudedTextGeometry(const fontSize = options.fontSize ?? 48;const depth = options.depth ?? 20;const halfDepth = depth / 2;- const curveSegments = Math.max(2, options.curveSegments ?? 5);+ const flattening: CurveFlattening = {+ segments:+ options.curveSegments !== undefined+ ? Math.max(2, options.curveSegments)+ : undefined,+ tolerance: Math.max(+ 1e-3,+ options.curveTolerance ?? DEFAULT_CURVE_TOLERANCE,+ ),+ };const unitsPerEm = font.unitsPerEm || 1000;const scale = fontSize / unitsPerEm;@@ -308,15 +415,14 @@ export function generateExtrudedTextGeometry(const glyphX = cursorX + (pos.xOffset ?? 0);const glyphY = pos.yOffset ?? 0;- const glyphContours = extractGlyphContours(+ const glyphPolys = glyphPolygonsAt(+ font,glyph,glyphX,glyphY,scale,- curveSegments,+ flattening,);-- const glyphPolys = organizeGlyphPolygons(glyphContours);for (const gp of glyphPolys) {polygons.push(gp);}@@ -359,8 +465,9 @@ export function generateExtrudedTextGeometry(const h = maxY - minY;let alignOffsetX = (minX + maxX) / 2;- if (options.align === "left") alignOffsetX = minX;- if (options.align === "right") alignOffsetX = maxX;+ if (options.align === "left" || options.align === "start")+ alignOffsetX = minX;+ if (options.align === "right" || options.align === "end") alignOffsetX = maxX;let alignOffsetY = (minY + maxY) / 2;if (options.verticalAlign === "top") alignOffsetY = minY;@@ -378,6 +485,11 @@ export function generateExtrudedTextGeometry(}}}+ // Bounds (and the UVs built from them) describe the aligned geometry.+ minX -= alignOffsetX;+ maxX -= alignOffsetX;+ minY -= alignOffsetY;+ maxY -= alignOffsetY;const frontPositions: number[] = [];const frontNormals: number[] = [];@@ -389,35 +501,70 @@ export function generateExtrudedTextGeometry(const sideUvs: number[] = [];const sideIndices: number[] = [];- const addSideQuad = (p0: Point2D, p1: Point2D): void => {- const dx = p1.x - p0.x;- const dy = p1.y - p0.y;- const len = Math.hypot(dx, dy);- if (len < 1e-6) return;-- const nx = dy / len;- const ny = -dx / len;-- const baseIdx = sidePositions.length / 3;-- sidePositions.push(p0.x, p0.y, halfDepth);- sideNormals.push(nx, ny, 0);- sideUvs.push(0, 0);-- sidePositions.push(p1.x, p1.y, halfDepth);- sideNormals.push(nx, ny, 0);- sideUvs.push(1, 0);-- sidePositions.push(p1.x, p1.y, -halfDepth);- sideNormals.push(nx, ny, 0);- sideUvs.push(1, 1);+ // Side walls are columns of vertex pairs (top at +z, bottom at -z) joined+ // by quads. Along a smooth run neighbouring quads share their column, so+ // a flattened curve shades as one surface (averaged normals) at half the+ // vertices; a corner sharper than the crease angle gets one column per+ // side and stays crisp. u runs along the outline, v front to back.+ const addColumn = (p: Point2D, n: Point2D, u: number): number => {+ const index = sidePositions.length / 3;+ sidePositions.push(p.x, p.y, halfDepth, p.x, p.y, -halfDepth);+ sideNormals.push(n.x, n.y, 0, n.x, n.y, 0);+ sideUvs.push(u, 0, u, 1);+ return index;+ };- sidePositions.push(p0.x, p0.y, -halfDepth);- sideNormals.push(nx, ny, 0);- sideUvs.push(0, 1);+ const addSideWall = (contour: Contour): void => {+ const pts = contour.filter((p, k) => {+ const next = contour[(k + 1) % contour.length]!;+ return Math.hypot(next.x - p.x, next.y - p.y) >= 1e-6;+ });+ const count = pts.length;+ if (count < 2) return;+ const edgeLengths: number[] = [];+ const faceNormals = pts.map((p, k) => {+ const next = pts[(k + 1) % count]!;+ const len = Math.hypot(next.x - p.x, next.y - p.y);+ edgeLengths.push(len);+ return { x: (next.y - p.y) / len, y: -(next.x - p.x) / len };+ });+ const perimeter = edgeLengths.reduce((a, b) => a + b, 0);++ // Column each edge starts from, and column the previous edge ends on.+ const startColumn: number[] = [];+ const endColumn: number[] = [];+ let along = 0;+ for (let k = 0; k < count; k++) {+ const incoming = faceNormals[(k - 1 + count) % count]!;+ const outgoing = faceNormals[k]!;+ const u = along / perimeter;+ const sx = incoming.x + outgoing.x;+ const sy = incoming.y + outgoing.y;+ const smooth =+ incoming.x * outgoing.x + incoming.y * outgoing.y >= SIDE_CREASE_COS;+ const averaged = smooth+ ? { x: sx / Math.hypot(sx, sy), y: sy / Math.hypot(sx, sy) }+ : null;+ if (averaged && k > 0) {+ startColumn[k] = endColumn[k] = addColumn(pts[k]!, averaged, u);+ } else {+ // Vertex 0 always splits, smooth or not: its end column closes the outline at u = 1.+ endColumn[k] = addColumn(+ pts[k]!,+ averaged ?? incoming,+ k === 0 ? 1 : u,+ );+ startColumn[k] = addColumn(pts[k]!, averaged ?? outgoing, u);+ }+ along += edgeLengths[k]!;+ }- sideIndices.push(baseIdx, baseIdx + 2, baseIdx + 1);- sideIndices.push(baseIdx, baseIdx + 3, baseIdx + 2);+ for (let k = 0; k < count; k++) {+ const a = startColumn[k]!; // top; a + 1 is its bottom+ const b = endColumn[(k + 1) % count]!;+ sideIndices.push(a, b + 1, b);+ sideIndices.push(a, a + 1, b + 1);+ }};for (const poly of polygons) {@@ -447,48 +594,45 @@ export function generateExtrudedTextGeometry(const triangles = earcut(earcutCoords, holeIndices, 2);const vertCount = earcutCoords.length / 2;+ // The front cap faces the default camera, which looks along +z (as+ // planar layers do, normal -z).const frontBaseIdx = frontPositions.length / 3;for (let v = 0; v < vertCount; v++) {const vx = earcutCoords[v * 2]!;const vy = earcutCoords[v * 2 + 1]!;- frontPositions.push(vx, vy, halfDepth);- frontNormals.push(0, 0, 1);+ frontPositions.push(vx, vy, -halfDepth);+ frontNormals.push(0, 0, -1);frontUvs.push((vx - minX) / (w || 1), (vy - minY) / (h || 1));}for (let t = 0; t < triangles.length; t += 3) {frontIndices.push(frontBaseIdx + triangles[t]!,- frontBaseIdx + triangles[t + 1]!,frontBaseIdx + triangles[t + 2]!,+ frontBaseIdx + triangles[t + 1]!,);}- const backBaseIdx = sidePositions.length / 3;+ // The back cap is a face of the letter like the front: it takes the+ // fill, so text seen from behind is not painted in the bevel color.+ const backBaseIdx = frontPositions.length / 3;for (let v = 0; v < vertCount; v++) {const vx = earcutCoords[v * 2]!;const vy = earcutCoords[v * 2 + 1]!;- sidePositions.push(vx, vy, -halfDepth);- sideNormals.push(0, 0, -1);- sideUvs.push((vx - minX) / (w || 1), (vy - minY) / (h || 1));+ frontPositions.push(vx, vy, halfDepth);+ frontNormals.push(0, 0, 1);+ frontUvs.push((vx - minX) / (w || 1), (vy - minY) / (h || 1));}for (let t = 0; t < triangles.length; t += 3) {- sideIndices.push(+ frontIndices.push(backBaseIdx + triangles[t]!,- backBaseIdx + triangles[t + 2]!,backBaseIdx + triangles[t + 1]!,+ backBaseIdx + triangles[t + 2]!,);}- for (let k = 0; k < outer.length; k++) {- const next = (k + 1) % outer.length;- addSideQuad(outer[k]!, outer[next]!);- }-+ addSideWall(outer);for (const hole of poly.holes) {- for (let k = 0; k < hole.length; k++) {- const next = (k + 1) % hole.length;- addSideQuad(hole[k]!, hole[next]!);- }+ addSideWall(hole);}}@@ -503,8 +647,6 @@ export function generateExtrudedTextGeometry(shading: options.material ?? "lit",shininess: options.shininess ?? 32,roughness: options.roughness,- specularIntensity: options.specularIntensity ?? 0.8,- ambientIntensity: options.ambientIntensity ?? 1.0,metallic: options.metallic,twoSided: true,};@@ -515,8 +657,6 @@ export function generateExtrudedTextGeometry(shading: options.material ?? "lit",shininess: options.shininess ?? 32,roughness: options.roughness,- specularIntensity: options.specularIntensity ?? 0.8,- ambientIntensity: options.ambientIntensity ?? 1.0,metallic: options.metallic,twoSided: true,};@@ -561,6 +701,13 @@ export function generateExtrudedTextGeometry("side-material": sideMaterial,},animations: [],+ bounds: {+ min: [minX, minY, -halfDepth],+ max: [maxX, maxY, halfDepth],+ center: [(minX + maxX) / 2, (minY + maxY) / 2, 0],+ size: [w, h, depth],+ boundingSphereRadius: Math.hypot(w, h, depth) / 2,+ },};return {diff --git a/packages/webgpu-renderers/src/signals/signal-registry.ts b/packages/webgpu-renderers/src/signals/signal-registry.tsindex 6ee38430..dd0b4a11 100644--- a/packages/webgpu-renderers/src/signals/signal-registry.ts+++ b/packages/webgpu-renderers/src/signals/signal-registry.ts@@ -782,7 +782,7 @@ export class SignalRegistry {*/public getOrCreate1DBuffer(device: GPUDevice,- _encoder: GPUCommandEncoder,+ _encoder: GPUCommandEncoder | null,nodeId: string,elapsedTime: number,duration: number,@@ -1061,7 +1061,7 @@ export class SignalRegistry {*/public getOrCreate2DTextureView(device: GPUDevice,- _encoder: GPUCommandEncoder,+ _encoder: GPUCommandEncoder | null,nodeId: string,elapsedTime: number,duration: number,diff --git a/packages/webgpu-renderers/src/slug/slug-font-cache.ts b/packages/webgpu-renderers/src/slug/slug-font-cache.tsindex 2fc2cb99..83a917f8 100644--- a/packages/webgpu-renderers/src/slug/slug-font-cache.ts+++ b/packages/webgpu-renderers/src/slug/slug-font-cache.ts@@ -3,6 +3,11 @@ import type { FontkitFont } from "./slug-generator.js";import { SlugGenerator } from "./slug-generator.js";import type { SlugFont } from "./slug-loader.js";+/** Per-font load timings are debug output: shown only with LOG_LEVEL=debug or trace. */+const VERBOSE_FONT_LOGS =+ typeof process !== "undefined" &&+ /^(debug|trace)$/i.test(process.env?.LOG_LEVEL ?? "");+export type FontLoadListener = (fontFamily: string) => void;export function parseFontWeight(fontWeight?: string | number): number {@@ -136,6 +141,11 @@ export class SlugFontCache {return g[GLOBAL_PARSED_KEY];}+ /** Register an already parsed font (e.g. by FontManager) under a family name. */+ public static setParsed(fontFamily: string, font: FontkitFont): void {+ SlugFontCache.parsedFontCache.set(fontFamily, font);+ }+public static getParsed(fontFamily: string): FontkitFont | null {return SlugFontCache.parsedFontCache.get(fontFamily) || null;}@@ -700,7 +710,7 @@ export class SlugFontCache {buffer = nodeBuffer.buffer.slice(nodeBuffer.byteOffset,nodeBuffer.byteOffset + nodeBuffer.byteLength,- );+ ) as ArrayBuffer;} else {// Browser/Web or remote URL environmentconst response = await fetch(fontUrl);@@ -726,12 +736,13 @@ export class SlugFontCache {const generated = generator.generate(device, font);const generateTime = performance.now();- console.log(- `[SlugFontCache] Preloaded and parsed font "${fontFamily}":\n` +- ` - fontkit parse: ${(parseTime - startTime).toFixed(2)}ms\n` +- ` - slug generation: ${(generateTime - parseTime).toFixed(2)}ms\n` +- ` - total: ${(generateTime - startTime).toFixed(2)}ms`,- );+ if (VERBOSE_FONT_LOGS)+ console.log(+ `[SlugFontCache] Preloaded and parsed font "${fontFamily}":\n` ++ ` - fontkit parse: ${(parseTime - startTime).toFixed(2)}ms\n` ++ ` - slug generation: ${(generateTime - parseTime).toFixed(2)}ms\n` ++ ` - total: ${(generateTime - startTime).toFixed(2)}ms`,+ );generated.fontFamily = fontFamily;SlugFontCache.cache.set(fontFamily, generated);diff --git a/packages/webgpu-renderers/src/slug/slug-generator.ts b/packages/webgpu-renderers/src/slug/slug-generator.tsindex b56e2cd7..53d7ce93 100644--- a/packages/webgpu-renderers/src/slug/slug-generator.ts+++ b/packages/webgpu-renderers/src/slug/slug-generator.ts@@ -33,6 +33,11 @@ export interface FontkitFont {unitsPerEm: number;variationAxes?: Record<string, unknown>;getVariation?(settings: Record<string, number>): FontkitFont;+ /** Shapes a string (kerning, ligatures); fontkit fonts always have it. */+ layout?(text: string): {+ glyphs: unknown[];+ positions: Array<{ xAdvance: number }>;+ };}const TEXTURE_WIDTH = 4096;diff --git a/packages/webgpu-renderers/src/slug/slug-geometry.ts b/packages/webgpu-renderers/src/slug/slug-geometry.tsindex 8eb9b3ab..5d907c06 100644--- a/packages/webgpu-renderers/src/slug/slug-geometry.ts+++ b/packages/webgpu-renderers/src/slug/slug-geometry.ts@@ -99,7 +99,7 @@ function resolveParsedFont(font: SlugFont): FontkitFont | null {const parsed = SlugFontCache.getParsed(font.fontFamily);if (parsed) return parsed;}- const globalParsed = (globalThis as Record<string, unknown>)[+ const globalParsed = (globalThis as Record<symbol, unknown>)[Symbol.for("gatewai.slugFontCache.parsed")] as Map<string, FontkitFont> | undefined;if (globalParsed) {@@ -124,13 +124,14 @@ export function getKernOffset(font: SlugFont, a: number, b: number): number {if (src) {try {const s = String.fromCodePoint(a) + String.fromCodePoint(b);- const laid = src.layout(s);+ const laid = src.layout?.(s);+ if (!laid) throw new Error("font cannot shape text");const total = laid.positions.reduce((acc: number, p: { xAdvance: number }) => acc + p.xAdvance,0,);- const advA = src.glyphForCodePoint(a)?.advanceWidth ?? 0;- const advB = src.glyphForCodePoint(b)?.advanceWidth ?? 0;+ const advA = src.glyphForCodePoint?.(a)?.advanceWidth ?? 0;+ const advB = src.glyphForCodePoint?.(b)?.advanceWidth ?? 0;v = laid.glyphs.length === 2 ? total - (advA + advB) : 0;} catch {v = 0;diff --git a/packages/webgpu-renderers/src/text-animations/advanced-types.ts b/packages/webgpu-renderers/src/text-animations/advanced-types.tsindex 08c26a2e..9ec78fc4 100644--- a/packages/webgpu-renderers/src/text-animations/advanced-types.ts+++ b/packages/webgpu-renderers/src/text-animations/advanced-types.ts@@ -340,6 +340,8 @@ export const ExtendedGlyphPropertiesSchema = z.object({rotationX: z.number().optional(),rotationY: z.number().optional(),rotationZ: z.number().optional(),+ /** Volumetric 3D arrangement (set by `volumetricFormation()`) */+ formation: VolumetricFormationConfigSchema.optional(),skew: z.number().optional(),skewAxis: z.number().optional(),diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex faf36bc7..d2c2a597 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -2086,6 +2086,9 @@ importers:'@types/react':specifier: ^19.2.4version: 19.2.14+ '@webgpu/types':+ specifier: ^0.1.70+ version: 0.1.70dotenv:specifier: ^16.4.5version: 16.6.1@@ -2117,12 +2120,18 @@ importers:'@gitframes/tsconfig':specifier: workspace:*version: link:../tsconfig+ '@webgpu/types':+ specifier: ^0.1.70+ version: 0.1.70tsdown:specifier: ^0.19.0version: 0.19.0([email protected])([email protected])typescript:specifier: ^5.8.3version: 5.8.3+ vitest:+ specifier: ^4.1.5+ version: 4.1.5(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected](@noble/[email protected])([email protected]))([email protected](@types/[email protected])([email protected])([email protected])([email protected])([email protected])([email protected]))packages/gitframes:dependencies:@@ -2420,6 +2429,9 @@ importers:'@types/node':specifier: ^24.1.0version: 24.10.14+ '@webgpu/types':+ specifier: ^0.1.70+ version: 0.1.70tsdown:specifier: ^0.19.0version: 0.19.0([email protected])([email protected])diff --git a/skills/gitframes-compose/SKILL.md b/skills/gitframes-compose/SKILL.mdindex f4b8a32e..fb936c30 100644--- a/skills/gitframes-compose/SKILL.md+++ b/skills/gitframes-compose/SKILL.md@@ -116,6 +116,17 @@ const anim = LayerAnimation.create()// Directions: in, out, inOut (e.g. "expo.out", "back.out(1.7)", "power2.inOut")```+For more than two keyframes on one prop, list them with `keys`: each tuple is `[frame, value, ease?]`, where the ease shapes the motion *into* that keyframe. Frames are rounded, so beat math (`at + BEAT / 2`) can go straight in:++```typescript+const pop = LayerAnimation.create()+ .keys("scale", [[0, 0], [10, 1.12, "back.out(1.6)"], [16, 1, "power2.out"]])+ .keys("opacity", [[0, 0], [6, 1]]);+// Static form: LayerAnimation.keys("rotation", [[0, -8], [20, 0, "expo.out"]])+```++Keyframes past the layer's `durationFrames` never play; the render warns (`animation_truncated`) and `renderVideo({ qa: true })` reports it.+### 2. Kinetic Typography (`TextAnimator`)Text animators operate at the GPU glyph instancing level, cascading offsets across characters, words, or lines:diff --git a/skills/gitframes-render/SKILL.md b/skills/gitframes-render/SKILL.mdindex d4acb7fe..dde67739 100644--- a/skills/gitframes-render/SKILL.md+++ b/skills/gitframes-render/SKILL.md@@ -127,6 +127,8 @@ const result = await comp.renderVideo({await result.cleanup?.();```+Pass `qa: true` to have the engine check the output while it renders (see **Built-in Video QA** below) and return the findings as `result.qa`.+---## Production Render Script Pattern@@ -169,6 +171,8 @@ if (mode === "frames") {process.exit(0);```+To run against the engine's TypeScript source instead of built `dist` (while changing the engine itself), give tsx a `tsconfig.dev.json` that extends the project's tsconfig and adds `"../../packages/**/*"` and `"../../nodes/**/*"` to `include` (tsx only applies compiler options, such as the decorators `server-utils` needs, to included files), then run `tsx --tsconfig tsconfig.dev.json --conditions=development src/render.ts`. `examples/22_gitframes_launch` has it as `pnpm render:src`.+CLI Usage:```bash# Render specific keyframes for instant review:@@ -180,9 +184,39 @@ tsx src/render.ts----## Video QA & Automated Inspection+## Built-in Video QA (`renderVideo({ qa })`)++The engine measures the video while it renders: every frame as it is read back for the encoder, and the audio as it is mixed. No second decode pass, and the numbers describe exactly what was encoded.++```typescript+import { formatQaReport } from "gitframes";++const result = await comp.renderVideo({+ outputPath: "output/final-video.mp4",+ qa: { targetLufs: -14 }, // or `qa: true` for defaults (no loudness target)+});+console.log(formatQaReport(result.qa!));+if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loop+```++`result.qa` is structured, so branch on it instead of parsing logs:++| `issues[].code` | Severity | Meaning |+|---|---|---|+| `frames_missing` | error | Fewer frames encoded than the composition has |+| `black_frames` | warning | ≥ `minBlackSec` (0.5 s) of black frames; has `startSec`/`endSec` |+| `audio_clipping` | warning | Samples at or beyond full scale |+| `loudness_off_target` | warning | Integrated loudness more than `lufsTolerance` (1 LU) from `targetLufs` |+| `animation_truncated` | warning | Keyframes past a layer's clip window never play; has `layerId` |+| `frozen_video` | info | Picture unchanged for ≥ `minFrozenSec` (2 s); fine for held title cards |+| `audio_silence` | info | ≥ `minSilenceSec` (2 s) below `silenceDbfs` (-60) |+| `audio_silent` | info | The whole track is silent |++`passed` is false when any error or warning is present. `qa.audio.integratedLufs` is ITU-R BS.1770 loudness (streaming platforms normalize to about -14 LUFS); `qa.audio.peakDbfs` is the sample peak.++## Inspecting a Finished File with ffmpeg-After rendering, inspect the output with `ffmpeg`/`ffprobe` to verify audio presence, duration, and absence of black frames:+For files rendered elsewhere, or to compare against a previous render, inspect with `ffmpeg`/`ffprobe`:```bash# 1. Check video and audio stream properties