Skip to content

Code-first video & image, rendered natively on WebGPU.

fix(render): match preview to export; emoji font is provided by the user

Claude616da992140f
diff --git a/nodes/node-compositor/SKILL.md b/nodes/node-compositor/SKILL.md
index ed1b3921..41d9c863 100644
--- a/nodes/node-compositor/SKILL.md
+++ b/nodes/node-compositor/SKILL.md
@@ -209,7 +209,7 @@ Each track represents animatable property modifications:
- **glTF scenes:** a glTF/GLB `model3d` keeps its node hierarchy, every skin and its animation clips (translation, rotation, scale; linear, step and cubic-spline). Skinned meshes are posed per frame from the clip (`animationName`, `animationTime`/`animationProgress`, `loop`); base-colour textures (with mipmaps), alpha `MASK`/`BLEND` and VRM MToon materials (`material: "toon"`, picked automatically when `material` is unset) render as authored. `center`/`normalizeSize` apply above the scene graph, so skinning stays intact.
## Don't Forget
-- If text rendering required, you must include a font in spec (CLI TOOL only). By default emoji font is not loaded. NotoColorEmoji seems to be working well. Try to use NotoColorEmoji as default font for emojis.
+- If text rendering required, you must include a font in spec (CLI TOOL only). No emoji font ships with framefields: to draw emoji, provide a color emoji font yourself with `FontManager.registerEmojiFont(path)` (a CBDT/CBLC bitmap font such as Noto Color Emoji).
- The `layout` tree IS the composition — there is no other layer model. Every **media** node
needs a valid `inputHandleId` matching a connected input, and every node needs a `kind`.
- `type` is an input DataType — never put it on layout nodes; use `kind`.
diff --git a/packages/framefields/src/fonts/manager.ts b/packages/framefields/src/fonts/manager.ts
index fb79250b..78c17ad7 100644
--- a/packages/framefields/src/fonts/manager.ts
+++ b/packages/framefields/src/fonts/manager.ts
@@ -186,6 +186,22 @@ export class FontManager {
return FontManager.getInstance().register(optionsOrFamilyOrSource, source);
}
+ /**
+ * Sets the color emoji font text draws emoji with, for the preview and the
+ * export alike: a CBDT/CBLC bitmap font such as Noto Color Emoji, as a file
+ * path, URL or bytes. framefields ships none; without one, exports draw
+ * nothing for emoji.
+ */
+ public static async registerEmojiFont(
+ source: string | Uint8Array,
+ ): Promise<void> {
+ const { buffer, filePath } = await resolveSourceBuffer(source);
+ const label =
+ filePath ?? (typeof source === "string" ? source : "Uint8Array");
+ validateFontBinaryHeader(buffer, label);
+ SlugFontCache.setEmojiFont(buffer, filePath);
+ }
+
/**
* Retrieves registered font metadata by family name.
*/
diff --git a/packages/framefields/src/index.ts b/packages/framefields/src/index.ts
index 8890c714..678e00f5 100644
--- a/packages/framefields/src/index.ts
+++ b/packages/framefields/src/index.ts
@@ -1576,6 +1576,12 @@ export class Composition {
return this;
}
+ /** Sets the color emoji font; see {@link FontManager.registerEmojiFont}. */
+ public async registerEmojiFont(source: string | Uint8Array): Promise<this> {
+ await FontManager.registerEmojiFont(source);
+ return this;
+ }
+
public toSpec(): CompositorProgramConfig {
const layoutItems: LayoutNode[] = [...this.children, ...this.audioTracks];
const registeredPaths = FontManager.getRegisteredFontPaths();
diff --git a/packages/framefields/src/preview/player.ts b/packages/framefields/src/preview/player.ts
index c043e645..923a0dc1 100644
--- a/packages/framefields/src/preview/player.ts
+++ b/packages/framefields/src/preview/player.ts
@@ -7,7 +7,6 @@
*/
import { BUILTIN_NODE_RENDERERS } from "framefields-preview:renderers";
-import { GetFontAssetUrl } from "@framefields/client-utils";
import {
compositionStateStore,
drawCompositionTree,
@@ -18,7 +17,7 @@ import {
type NodeRendererPlugin,
registerWebGPURenderer,
} from "@framefields/node-sdk";
-import { preloadFont } from "@framefields/renderers";
+import { preloadCompositionFonts } from "@framefields/renderers";
import {
type OrtWebModule,
setDefaultSessionProvider,
@@ -27,7 +26,6 @@ import {
import {
acquireDevice,
BrowserSurfaceProvider,
- getHeadlessFontPath,
getRenderer2D,
lutStore,
type RenderContextValue,
@@ -175,6 +173,8 @@ async function createPlayer(
frameCount: Math.max(1, Math.round((durationMs / 1000) * fps)),
};
const hasVision = containsOp(vm as VirtualMediaData, "Vision");
+ // As the export decides it, so text entrances play the same.
+ const isVideoMode = vm.operation?.dataType === "Video";
const device = await acquireDevice();
const canvas =
@@ -183,7 +183,7 @@ async function createPlayer(
canvas.width = width % 2 ? width + 1 : width;
canvas.height = height % 2 ? height + 1 : height;
const surface = new BrowserSurfaceProvider(device, canvas);
- await preloadFonts(vm, device);
+ await preloadCompositionFonts(vm, device);
const renderId = `preview-${Math.random().toString(36).slice(2)}`;
let first = true;
@@ -269,7 +269,10 @@ async function createPlayer(
virtualMedia: vm as VirtualMediaData,
containerWidth: w,
containerHeight: h,
- isVideoMode: true,
+ isVideoMode,
+ // Every drawn frame is cached or buffered, so it waits for its LUTs
+ // and exact video frame as the export does, never a stand-in.
+ forceWait: true,
};
updateClockSignals(frame, fps, durationMs || undefined);
@@ -597,35 +600,3 @@ async function registerRenderers(renderers: readonly BuiltinRenderer[]) {
}),
);
}
-
-/** Loads every font the program's text uses, for layout and for drawing. */
-async function preloadFonts(vm: VirtualMediaData, device: GPUDevice) {
- const families = new Set<string>();
- (function walk(node: VirtualMediaData) {
- const op = node.operation as Record<string, unknown> | undefined;
- if (op) {
- const isText =
- op.op === "text" ||
- op.kind === "text" ||
- op.op === "caption" ||
- op.kind === "caption" ||
- typeof op.text === "string";
- const family = (op.fontFamily as string) || (isText ? "Inter" : "");
- if (family) families.add(family);
- }
- for (const child of node.children ?? []) walk(child);
- })(vm);
- await Promise.all(
- [...families].map(async (family) => {
- const url = getHeadlessFontPath(family) ?? GetFontAssetUrl(family);
- await Promise.all([
- preloadFont(family, url).catch((err: unknown) =>
- console.warn(`[framefields] font "${family}"`, err),
- ),
- SlugFontCache.preloadSlugFont(device, family, url).catch(
- (err: unknown) => console.warn(`[framefields] font "${family}"`, err),
- ),
- ]);
- }),
- );
-}
diff --git a/packages/renderer/src/asset-preloader.ts b/packages/renderer/src/asset-preloader.ts
index a5058779..d96526aa 100644
--- a/packages/renderer/src/asset-preloader.ts
+++ b/packages/renderer/src/asset-preloader.ts
@@ -1,120 +1,23 @@
-import {
- GetFontAssetUrl,
- R2_CUSTOM_DOMAIN,
-} from "@framefields/client-utils";
import type { VirtualMediaData } from "@framefields/core";
-import { preloadFont } from "@framefields/renderers";
+import { preloadCompositionFonts } from "@framefields/renderers";
import { rendererLogger } from "@framefields/server-utils";
-import {
- getHeadlessFontPath,
- SlugFontCache,
-} from "@framefields/webgpu-renderers";
import type {} from "webgpu";
/**
- * Recursively scans and preloads only the required font assets inside a VirtualMediaData tree.
- * This ensures typography renders correctly on Node.js without pre-loading or memory bloating from large media assets.
+ * Preloads only the fonts (and emoji) a VirtualMediaData tree's text uses, so
+ * typography renders correctly on Node.js without loading its media. Shared
+ * with the browser preview; emoji need a font set with
+ * `FontManager.registerEmojiFont()`.
*/
export async function preloadFonts(
media: VirtualMediaData,
device: GPUDevice | null = null,
): Promise<void> {
if (!media) return;
-
- const promises: Promise<unknown>[] = [];
- const op = media.operation;
-
- // Configure emoji font source: prefer R2 CDN, fallback to local file
- if (R2_CUSTOM_DOMAIN) {
- SlugFontCache.emojiFontUrl = `https://${R2_CUSTOM_DOMAIN}/static/NotoColorEmoji.ttf`;
- }
-
- // Local fallback for environments without R2 (e.g., local dev)
- if (!SlugFontCache.emojiFontPath) {
- try {
- const isDev = import.meta.url.endsWith(".ts");
- const emojiLocalUrl = new URL(
- isDev
- ? "./assets/NotoColorEmoji.ttf"
- : "../src/assets/NotoColorEmoji.ttf",
- import.meta.url,
- ).href;
-
- let fontPath = emojiLocalUrl;
- if (emojiLocalUrl.startsWith("file://")) {
- try {
- fontPath = new URL(emojiLocalUrl).pathname;
- } catch {
- fontPath = emojiLocalUrl.replace(/^file:\/\//, "");
- }
- }
- SlugFontCache.emojiFontPath = fontPath;
- } catch {
- // Local font not available — rely on R2 URL
- }
- }
-
- if (op) {
- const opText = op as Record<string, unknown>;
- const isTextNode =
- opText.op === "text" ||
- opText.kind === "text" ||
- opText.op === "caption" ||
- opText.kind === "caption" ||
- typeof opText.text === "string";
- const fontFamily =
- (opText.fontFamily as string) || (isTextNode ? "Inter" : undefined);
- if (fontFamily) {
- // Fonts registered through FontManager carry their real file path; the
- // asset URL is only a naming convention and would miss e.g. "Foo-Regular.ttf".
- const fontUrl =
- getHeadlessFontPath(fontFamily) ?? GetFontAssetUrl(fontFamily);
- promises.push(
- preloadFont(fontFamily, fontUrl).catch((err) => {
- rendererLogger.error(
- err,
- `[HeadlessMediaRenderer] Failed to preload font "${fontFamily}" from ${fontUrl}`,
- );
- }),
- );
- if (device) {
- promises.push(
- SlugFontCache.preloadSlugFont(device, fontFamily, fontUrl).catch(
- (err) => {
- rendererLogger.error(
- err,
- `[HeadlessMediaRenderer] Failed to preload Slug font "${fontFamily}" from ${fontUrl}`,
- );
- },
- ),
- );
- }
- }
- if (opText.op === "text" && opText.text && device) {
- const EMOJI_REGEX = /\p{Extended_Pictographic}/gu;
- const matches = (opText.text as string).match(EMOJI_REGEX);
- if (matches) {
- const fontSize = (opText.fontSize as number) ?? 48;
- for (const char of matches) {
- promises.push(
- SlugFontCache.preloadEmoji(device, char, fontSize).catch((err) => {
- rendererLogger.error(
- err,
- `[HeadlessMediaRenderer] Failed to preload emoji texture for "${char}"`,
- );
- }),
- );
- }
- }
- }
- }
-
- // Recurse into nested children
- if (media.children && media.children.length > 0) {
- for (const child of media.children) {
- promises.push(preloadFonts(child, device));
- }
- }
-
- await Promise.all(promises);
+ await preloadCompositionFonts(media, device, (err, what) =>
+ rendererLogger.error(
+ err,
+ `[HeadlessMediaRenderer] Failed to preload ${what}`,
+ ),
+ );
}
diff --git a/packages/renderer/src/headless-webgpu-renderer.ts b/packages/renderer/src/headless-webgpu-renderer.ts
index dd8d14ef..92348248 100644
--- a/packages/renderer/src/headless-webgpu-renderer.ts
+++ b/packages/renderer/src/headless-webgpu-renderer.ts
@@ -1219,44 +1219,46 @@ export class HeadlessWebGPURenderer {
const ctx: RenderContextValue | null =
surface && renderer ? { device, renderer, surface } : null;
- if (ctx && renderer) {
- // Pre-pass: trigger and await all LUT extractions
- {
- const preEncoder = device.createCommandEncoder();
- const dummyTex = renderer.getTemporaryTexture(width, height);
- const dummyView = dummyTex.createView();
- const preClear = renderer.beginFrame(
- preEncoder,
- dummyView,
- { r: 0, g: 0, b: 0, a: 0 },
- width,
- height,
- "clear",
- );
- preClear.end();
- await drawCompositionTree(
- ctx,
- preEncoder,
- dummyView,
- dummyTex,
- width,
- height,
+ // Pre-pass: draws a frame whose output is dropped so the LUT
+ // extractions and glyph uploads it starts finish before the frame
+ // is drawn for real.
+ const prePass = async (frame: number) => {
+ if (!ctx || !renderer) return;
+ const preEncoder = device.createCommandEncoder();
+ const dummyTex = renderer.getTemporaryTexture(width, height);
+ const dummyView = dummyTex.createView();
+ const preClear = renderer.beginFrame(
+ preEncoder,
+ dummyView,
+ { r: 0, g: 0, b: 0, a: 0 },
+ width,
+ height,
+ "clear",
+ );
+ preClear.end();
+ await drawCompositionTree(
+ ctx,
+ preEncoder,
+ dummyView,
+ dummyTex,
+ width,
+ height,
+ virtualMedia,
+ {
+ frame,
+ fps,
+ isHeadless: true,
+ renderId,
virtualMedia,
- {
- frame: 0,
- fps,
- isHeadless: true,
- renderId,
- virtualMedia,
- containerWidth: width,
- containerHeight: height,
- isVideoMode: virtualMedia.operation?.dataType === "Video",
- },
- );
- device.queue.submit([preEncoder.finish()]);
- await lutStore.awaitAllPending(device);
- }
- }
+ containerWidth: width,
+ containerHeight: height,
+ isVideoMode: virtualMedia.operation?.dataType === "Video",
+ },
+ );
+ device.queue.submit([preEncoder.finish()]);
+ await lutStore.awaitAllPending(device);
+ };
+ await prePass(0);
// Render and encode frames using pipelined double-buffered DMA staging ring
for (let i = 0; i < totalFrames; i++) {
@@ -1268,6 +1270,15 @@ export class HeadlessWebGPURenderer {
}
if (ctx && surface && renderer && pipeline) {
+ // Assets first used on a later frame (a LUT, a glyph) get the
+ // same pre-pass as frame 0, as renderImage and the preview do.
+ if (
+ i > 0 &&
+ (lutStore.hasPending(device) || SlugFontCache.hasPending())
+ ) {
+ updateClockSignals(i, fps, durationMs);
+ await prePass(i);
+ }
compositionStateStore.setState(renderId, i, fps, true);
updateClockSignals(i, fps, durationMs);
diff --git a/packages/renderer/src/test-runner.ts b/packages/renderer/src/test-runner.ts
index 48f0dc3c..fdd8bdc8 100644
--- a/packages/renderer/src/test-runner.ts
+++ b/packages/renderer/src/test-runner.ts
@@ -10,11 +10,7 @@ async function runTests() {
globalThis.fetch = async (input: RequestInfo | URL, init?: RequestInit) => {
const url =
typeof input === "string" ? input : (input as { url?: string }).url || "";
- if (
- url.includes("fonts/load") ||
- url.includes("NotoColorEmoji") ||
- url.includes("example.com")
- ) {
+ if (url.includes("fonts/load") || url.includes("example.com")) {
throw new Error("Network requests are disabled in unit tests");
}
return originalFetch(input, init);
diff --git a/packages/renderers/src/index.ts b/packages/renderers/src/index.ts
index c294c165..77904394 100644
--- a/packages/renderers/src/index.ts
+++ b/packages/renderers/src/index.ts
@@ -2,6 +2,7 @@ export * from "./blend.js";
export * from "./geometry.js";
export * from "./interpolate.js";
export * from "./math3d/index.js";
+export * from "./preload-fonts.js";
export * from "./spring.js";
export * from "./text-measurer.js";
export * from "./utils.js";
diff --git a/packages/renderers/src/preload-fonts.test.ts b/packages/renderers/src/preload-fonts.test.ts
new file mode 100644
index 00000000..dec8c0a4
--- /dev/null
+++ b/packages/renderers/src/preload-fonts.test.ts
@@ -0,0 +1,57 @@
+import type { VirtualMediaData } from "@framefields/core";
+import { SlugFontCache } from "@framefields/webgpu-renderers";
+import { afterEach, describe, expect, it, vi } from "vitest";
+import { preloadCompositionFonts } from "./preload-fonts.js";
+
+type Device = Parameters<typeof preloadCompositionFonts>[1];
+
+vi.mock("./utils.js", () => ({ preloadFont: vi.fn(async () => {}) }));
+
+const text = (value: string, fontFamily?: string): VirtualMediaData =>
+ ({
+ operation: { op: "text", text: value, fontFamily, fontSize: 32 },
+ }) as unknown as VirtualMediaData;
+
+const tree = (...children: VirtualMediaData[]): VirtualMediaData =>
+ ({ operation: { op: "compose" }, children }) as unknown as VirtualMediaData;
+
+describe("preloadCompositionFonts", () => {
+ afterEach(() => vi.restoreAllMocks());
+
+ it("loads each family and each emoji once", async () => {
+ const slug = vi
+ .spyOn(SlugFontCache, "preloadSlugFont")
+ .mockResolvedValue({} as never);
+ vi.spyOn(SlugFontCache, "hasEmojiFont").mockReturnValue(true);
+ const emoji = vi
+ .spyOn(SlugFontCache, "preloadEmoji")
+ .mockResolvedValue({} as never);
+
+ await preloadCompositionFonts(
+ tree(text("hi 😀", "Serif"), text("😀 🎉", "Serif"), text("plain")),
+ {} as Device,
+ );
+
+ expect(slug.mock.calls.map((c) => c[1]).sort()).toEqual(["Inter", "Serif"]);
+ expect(emoji.mock.calls.map((c) => c[1]).sort()).toEqual(["🎉", "😀"]);
+ });
+
+ it("warns once, and does not fail, when emoji have no font", async () => {
+ vi.spyOn(SlugFontCache, "preloadSlugFont").mockResolvedValue({} as never);
+ vi.spyOn(SlugFontCache, "hasEmojiFont").mockReturnValue(false);
+ vi.spyOn(SlugFontCache, "preloadEmoji").mockRejectedValue(
+ new Error("no emoji font"),
+ );
+ const warn = vi.spyOn(console, "warn").mockImplementation(() => {});
+ const onError = vi.fn();
+
+ await preloadCompositionFonts(tree(text("😀")), {} as Device, onError);
+ await preloadCompositionFonts(tree(text("🎉")), {} as Device, onError);
+
+ const notices = warn.mock.calls.filter((c) =>
+ String(c[0]).includes("registerEmojiFont"),
+ );
+ expect(notices).toHaveLength(1);
+ expect(onError).not.toHaveBeenCalled();
+ });
+});
diff --git a/packages/renderers/src/preload-fonts.ts b/packages/renderers/src/preload-fonts.ts
new file mode 100644
index 00000000..3f415b8f
--- /dev/null
+++ b/packages/renderers/src/preload-fonts.ts
@@ -0,0 +1,85 @@
+import { GetFontAssetUrl } from "@framefields/client-utils";
+import type { VirtualMediaData } from "@framefields/core";
+import {
+ getHeadlessFontPath,
+ SlugFontCache,
+} from "@framefields/webgpu-renderers";
+import { preloadFont } from "./utils.js";
+
+const EMOJI_REGEX = /\p{Extended_Pictographic}/gu;
+
+type Device = Parameters<typeof SlugFontCache.preloadSlugFont>[0];
+
+let warnedNoEmojiFont = false;
+
+/**
+ * Loads every font a composition's text uses (for layout and for drawing) and
+ * the emoji it draws. The browser preview and the export both call this, so
+ * they start each render from the same glyphs.
+ */
+export async function preloadCompositionFonts(
+ media: VirtualMediaData,
+ device: Device | null,
+ onError: (err: unknown, what: string) => void = (err, what) =>
+ console.warn(`[framefields] ${what}`, err),
+): Promise<void> {
+ const families = new Set<string>();
+ const emoji = new Map<string, number>();
+ (function walk(node: VirtualMediaData) {
+ const op = node.operation as Record<string, unknown> | undefined;
+ if (op) {
+ const isText =
+ op.op === "text" ||
+ op.kind === "text" ||
+ op.op === "caption" ||
+ op.kind === "caption" ||
+ typeof op.text === "string";
+ const family = (op.fontFamily as string) || (isText ? "Inter" : "");
+ if (family) families.add(family);
+ if (op.op === "text" && typeof op.text === "string") {
+ const fontSize = (op.fontSize as number) ?? 48;
+ for (const char of op.text.match(EMOJI_REGEX) ?? []) {
+ emoji.set(`${char}-${fontSize}`, fontSize);
+ }
+ }
+ }
+ for (const child of node.children ?? []) walk(child);
+ })(media);
+
+ const tasks: Promise<unknown>[] = [];
+ for (const family of families) {
+ // Fonts registered through FontManager carry their real file path; the
+ // asset URL is only a naming convention and would miss e.g. "Foo-Regular.ttf".
+ const url = getHeadlessFontPath(family) ?? GetFontAssetUrl(family);
+ tasks.push(
+ preloadFont(family, url).catch((err) =>
+ onError(err, `font "${family}" from ${url}`),
+ ),
+ );
+ if (device) {
+ tasks.push(
+ SlugFontCache.preloadSlugFont(device, family, url).catch((err) =>
+ onError(err, `Slug font "${family}" from ${url}`),
+ ),
+ );
+ }
+ }
+ if (device && emoji.size > 0) {
+ if (!SlugFontCache.hasEmojiFont() && !warnedNoEmojiFont) {
+ warnedNoEmojiFont = true;
+ console.warn(
+ "[framefields] Text uses emoji but no emoji font is registered, so exports draw nothing for them. Register a color emoji font (e.g. Noto Color Emoji) with FontManager.registerEmojiFont(path).",
+ );
+ }
+ for (const [key, fontSize] of emoji) {
+ const char = key.slice(0, key.lastIndexOf("-"));
+ tasks.push(
+ SlugFontCache.preloadEmoji(device, char, fontSize).catch((err) => {
+ // Already reported above when no font is set.
+ if (SlugFontCache.hasEmojiFont()) onError(err, `emoji "${char}"`);
+ }),
+ );
+ }
+ }
+ await Promise.all(tasks);
+}
diff --git a/packages/webgpu-renderers/src/nodes/paragraph.ts b/packages/webgpu-renderers/src/nodes/paragraph.ts
index 1d4c1cbc..88130522 100644
--- a/packages/webgpu-renderers/src/nodes/paragraph.ts
+++ b/packages/webgpu-renderers/src/nodes/paragraph.ts
@@ -155,7 +155,7 @@ function getOrCreateEmojiTexture(
const canvas = new OffscreenCanvas(size, size);
const c2d = canvas.getContext("2d");
if (c2d) {
- c2d.font = `${fontSize * resolutionScale}px "Apple Color Emoji", "Segoe UI Emoji", "Noto Color Emoji", "NotoColorEmoji", sans-serif`;
+ c2d.font = `${fontSize * resolutionScale}px "Apple Color Emoji", "Segoe UI Emoji", "Noto Color Emoji", sans-serif`;
c2d.textBaseline = "middle";
c2d.textAlign = "center";
c2d.fillText(char, size / 2, size / 2);
diff --git a/packages/webgpu-renderers/src/slug/emoji-font.test.ts b/packages/webgpu-renderers/src/slug/emoji-font.test.ts
new file mode 100644
index 00000000..6b9a3fb3
--- /dev/null
+++ b/packages/webgpu-renderers/src/slug/emoji-font.test.ts
@@ -0,0 +1,61 @@
+import { afterEach, describe, expect, it } from "vitest";
+import { SlugFontCache } from "./slug-font-cache.js";
+
+/** An SFNT whose table directory lists `tags` (the tables themselves are empty). */
+function sfnt(tags: string[]): Uint8Array {
+ const bytes = new Uint8Array(12 + tags.length * 16 + 16);
+ const view = new DataView(bytes.buffer);
+ view.setUint32(0, 0x00010000);
+ view.setUint16(4, tags.length);
+ tags.forEach((tag, i) => {
+ const rec = 12 + i * 16;
+ for (let c = 0; c < 4; c++) view.setUint8(rec + c, tag.charCodeAt(c));
+ view.setUint32(rec + 8, 12 + tags.length * 16);
+ });
+ return bytes;
+}
+
+function reset() {
+ SlugFontCache.destroy();
+ SlugFontCache.emojiFontPath = null;
+ SlugFontCache.emojiFontUrl = null;
+}
+
+describe("emoji font", () => {
+ afterEach(reset);
+
+ it("ships with none", () => {
+ reset();
+ expect(SlugFontCache.hasEmojiFont()).toBe(false);
+ });
+
+ it("takes a color bitmap font and drops emoji drawn before it", () => {
+ let destroyed = false;
+ SlugFontCache.emojiTextureCache.set("😀-48", {
+ destroy: () => {
+ destroyed = true;
+ },
+ } as unknown as GPUTexture);
+
+ SlugFontCache.setEmojiFont(sfnt(["cmap", "CBDT", "CBLC"]), "/x/emoji.ttf");
+
+ expect(SlugFontCache.hasEmojiFont()).toBe(true);
+ expect(SlugFontCache.emojiFontPath).toBe("/x/emoji.ttf");
+ expect(SlugFontCache.emojiTextureCache.size).toBe(0);
+ expect(destroyed).toBe(true);
+ });
+
+ it("rejects a font without color bitmaps", () => {
+ SlugFontCache.setEmojiFont(sfnt(["cmap", "CBDT", "CBLC"]));
+ expect(() => SlugFontCache.setEmojiFont(sfnt(["cmap", "glyf"]))).toThrow(
+ /CBDT\/CBLC/,
+ );
+ expect(SlugFontCache.hasEmojiFont()).toBe(false);
+ });
+
+ it("says how to set one when an emoji is drawn without it", async () => {
+ await expect(
+ SlugFontCache.preloadEmoji({} as GPUDevice, "😀", 48),
+ ).rejects.toThrow(/registerEmojiFont/);
+ });
+});
diff --git a/packages/webgpu-renderers/src/slug/slug-font-cache.ts b/packages/webgpu-renderers/src/slug/slug-font-cache.ts
index 30a37ea1..88049e79 100644
--- a/packages/webgpu-renderers/src/slug/slug-font-cache.ts
+++ b/packages/webgpu-renderers/src/slug/slug-font-cache.ts
@@ -181,12 +181,44 @@ export class SlugFontCache {
private static cblcOffset = 0;
private static cbdtOffset = 0;
+ /**
+ * Sets the color emoji font text draws emoji with: a CBDT/CBLC bitmap font
+ * such as Noto Color Emoji. framefields ships none, so without one emoji
+ * draw nothing in exports (the browser preview falls back to the system's).
+ */
+ public static setEmojiFont(bytes: Uint8Array, filePath?: string): void {
+ SlugFontCache.parseTableDirectory(bytes);
+ if (!SlugFontCache.cblcOffset || !SlugFontCache.cbdtOffset) {
+ SlugFontCache.emojiFontBuffer = null;
+ throw new Error(
+ `Emoji font ${filePath ? `"${filePath}" ` : ""}has no CBDT/CBLC color bitmaps. Use a bitmap color emoji font such as Noto Color Emoji.`,
+ );
+ }
+ SlugFontCache.emojiFontPath = filePath ?? null;
+ SlugFontCache.emojiFontLoadPromise = null;
+ // Emoji drawn before the font was set are stale.
+ for (const tex of SlugFontCache.emojiTextureCache.values()) tex.destroy();
+ SlugFontCache.emojiTextureCache.clear();
+ }
+
+ /** True once an emoji font is set or a path / URL to load one from is. */
+ public static hasEmojiFont(): boolean {
+ return !!(
+ SlugFontCache.emojiFontBuffer ||
+ SlugFontCache.emojiFontPath ||
+ SlugFontCache.emojiFontUrl
+ );
+ }
+
/**
* Parse the SFNT table directory from a raw font buffer and cache
* the offsets for cmap, CBLC, and CBDT tables.
*/
private static parseTableDirectory(uint8: Uint8Array): void {
SlugFontCache.emojiFontBuffer = uint8;
+ SlugFontCache.cmapOffset = 0;
+ SlugFontCache.cblcOffset = 0;
+ SlugFontCache.cbdtOffset = 0;
const view = new DataView(uint8.buffer, uint8.byteOffset, uint8.byteLength);
const numTables = view.getUint16(4);
let offset = 12;
@@ -373,23 +405,28 @@ export class SlugFontCache {
!!(globalThis as Record<string, unknown>).__IS_HEADLESS_RENDERER__ ||
typeof window === "undefined" ||
typeof globalThis.document === "undefined";
+ const codePoint = char.codePointAt(0);
+ if (!codePoint) throw new Error("No codepoint");
+ const resolutionScale = 4.0;
+ const targetSize = Math.max(
+ 16,
+ Math.ceil(fontSize * 1.5 * resolutionScale),
+ );
+ // Preview and export draw the same glyph from the set emoji font.
+ const pngInfo = SlugFontCache.hasEmojiFont()
+ ? await SlugFontCache.getEmojiPngBuffer(codePoint)
+ : null;
if (isNode) {
- const codePoint = char.codePointAt(0);
- if (!codePoint) throw new Error("No codepoint");
-
- const pngInfo = await SlugFontCache.getEmojiPngBuffer(codePoint);
if (!pngInfo) {
- throw new Error(`Emoji glyph not found in font for ${char}`);
+ throw new Error(
+ SlugFontCache.hasEmojiFont()
+ ? `Emoji glyph not found in font for ${char}`
+ : `No emoji font is set for ${char}; register one with FontManager.registerEmojiFont()`,
+ );
}
const sharp = (await import(/* webpackIgnore: true */ "sharp")).default;
- const resolutionScale = 4.0;
- const targetSize = Math.max(
- 16,
- Math.ceil(fontSize * 1.5 * resolutionScale),
- );
-
const { data, info } = await sharp(Buffer.from(pngInfo.buffer))
.ensureAlpha()
.resize(targetSize, targetSize, {
@@ -425,18 +462,48 @@ export class SlugFontCache {
{ bytesPerRow: info.width * 4 },
[info.width, info.height],
);
- } else {
- const resolutionScale = 4.0;
- const emojiSize = Math.max(
- 16,
- Math.ceil(fontSize * 1.5 * resolutionScale),
+ } else if (pngInfo) {
+ // The glyph's own bitmap, fitted like the export's resize.
+ const bitmap = await createImageBitmap(
+ new Blob([pngInfo.buffer as Uint8Array<ArrayBuffer>], {
+ type: "image/png",
+ }),
);
- const size = Math.ceil(emojiSize / 4) * 4;
+ const canvas = new OffscreenCanvas(targetSize, targetSize);
+ const c2d = canvas.getContext("2d");
+ if (c2d) {
+ const fit = Math.min(
+ targetSize / bitmap.width,
+ targetSize / bitmap.height,
+ );
+ const w = bitmap.width * fit;
+ const h = bitmap.height * fit;
+ c2d.imageSmoothingQuality = "high";
+ c2d.drawImage(bitmap, (targetSize - w) / 2, (targetSize - h) / 2, w, h);
+ }
+ bitmap.close();
+
+ tex = device.createTexture({
+ label: `Emoji-${char}`,
+ size: [targetSize, targetSize],
+ format: "rgba8unorm",
+ usage:
+ GPUTextureUsage.TEXTURE_BINDING |
+ GPUTextureUsage.COPY_DST |
+ GPUTextureUsage.RENDER_ATTACHMENT,
+ });
+ device.queue.copyExternalImageToTexture(
+ { source: canvas },
+ { texture: tex, premultipliedAlpha: true },
+ [targetSize, targetSize],
+ );
+ } else {
+ const size = Math.ceil(targetSize / 4) * 4;
const canvas = new OffscreenCanvas(size, size);
const c2d = canvas.getContext("2d");
if (c2d) {
- c2d.font = `${fontSize * resolutionScale}px "NotoColorEmoji", "Emoji", "Apple Color Emoji", "Segoe UI Emoji", sans-serif`;
+ c2d.font = `${fontSize * resolutionScale}px "Apple Color Emoji", "Segoe UI Emoji", sans-serif`;
c2d.textBaseline = "middle";
c2d.textAlign = "center";
c2d.fillText(char, size / 2, size / 2);
@@ -836,7 +903,7 @@ export class SlugFontCache {
SlugFontCache.parseTableDirectory(new Uint8Array(arrayBuffer));
// Register as a FontFace so OffscreenCanvas/canvas 2D can use it
- const fontFace = new FontFace("NotoColorEmoji", arrayBuffer);
+ const fontFace = new FontFace("FramefieldsEmoji", arrayBuffer);
const loadedFace = await fontFace.load();
document.fonts.add(loadedFace);
await document.fonts.ready;
@@ -848,7 +915,7 @@ export class SlugFontCache {
// Notify listeners that the emoji font is ready so that canvas compositions redraw
SlugFontCache.listeners.forEach((l) => {
try {
- l("NotoColorEmoji");
+ l("FramefieldsEmoji");
} catch (e) {
console.error("[SlugFontCache] Error in listener:", e);
}
diff --git a/plugins/framefields/skills/framefields-compose/SKILL.md b/plugins/framefields/skills/framefields-compose/SKILL.md
index 4ba42fa1..bb50b02d 100644
--- a/plugins/framefields/skills/framefields-compose/SKILL.md
+++ b/plugins/framefields/skills/framefields-compose/SKILL.md
@@ -235,7 +235,7 @@ Anything else (a number, `undefined`) throws. Signals are sampled at the **compo
1. **Local Keyframe Coordinates:** Keyframes are strictly relative to the node's own timeline window (`0` = node appearance). Never offset keyframe frames by the parent's `startFrame`.
2. **Coordinate Overrides:** Animating `y` overrides the node's layout `y`. If a node is placed at `y: 400`, animate `fromTo("y", 440, 400)`, NOT `fromTo("y", 40, 0)`.
3. **No String Styles for Borders:** Use `borderColor` and `borderWidth`. CSS `border: "1px solid red"` is silently ignored.
-4. **Font Registration is Mandatory:** In headless WebGPU, system fonts do not exist. Every font must be registered via `FontManager.register()` and listed in `comp.fonts`.
+4. **Font Registration is Mandatory:** In headless WebGPU, system fonts do not exist. Every font must be registered via `FontManager.register()` and listed in `comp.fonts`. Emoji need a color emoji font too: framefields ships none, so register one with `FontManager.registerEmojiFont("assets/fonts/NotoColorEmoji.ttf")` (a CBDT/CBLC bitmap font such as Noto Color Emoji). Without it exports draw nothing for emoji.
5. **Integer Frames Only:** Keyframe frames, start frames, and durations must be integers. Decimal frames cause schema validation errors.
6. **Relative vs. Absolute:** Providing `x` or `y` without `position: "relative"` turns the node into an absolute-positioned layer removed from flex flow.
7. **Audio Layer Duration:** Every `Layer.audio()` must be assigned `durationFrames` to prevent audio truncation in headless rendering.