Code-first video & image, rendered natively on WebGPU.
fix(renderer-service): clean up memory and disk after every job
okanasledfe48321dda
diff --git a/.changeset/render-caches-fonts.md b/.changeset/render-caches-fonts.mdnew file mode 100644index 00000000..9c81bf6a--- /dev/null+++ b/.changeset/render-caches-fonts.md@@ -0,0 +1,7 @@+---+"framefields": patch+"@framefields/renderer": patch+"@framefields/webgpu-renderers": patch+---++`resetRenderCaches()` also forgets registered font files and deletes fonts downloaded from URLs (`clearHeadlessFonts()`). Before, the first file registered for a family stayed in use for the life of the process. In a long-lived worker, the next project's "Inter" then loaded the previous project's file, and failed once that checkout was evicted. Downloaded fonts now go to a fresh temp folder per session instead of a shared `gatewai-fonts`.diff --git a/apps/renderer-service/PLAN.md b/apps/renderer-service/PLAN.mdindex a64723db..ef0b47b1 100644--- a/apps/renderer-service/PLAN.md+++ b/apps/renderer-service/PLAN.md@@ -197,7 +197,7 @@ The worker emits phase events:The v1 routes `/run` and `/status/:id` are removed.-Finished jobs are evicted from memory after `FF_JOB_TTL_SEC` (default 1h). At most `FF_JOB_HISTORY_MAX` jobs are kept.+Finished jobs are evicted from memory after `FF_JOB_TTL_SEC` (default 1h). At most `FF_JOB_HISTORY_MAX` jobs are kept. Inline outputs are dropped from finished records after `FF_INLINE_TTL_SEC` (default 10 min), so the history doesn't keep every rendered video in memory: a later read returns the output with `base64Expired: true`. After each job the worker also calls the engine's `resetRenderCaches()`, so an idle worker holds no decoders, textures or fonts from the last job.**Concurrency.** There is exactly one active job per worker (section 3). Extra jobs queue in-process up to `FF_QUEUE_MAX`, and anything beyond that gets `429 BUSY`. To scale, run more workers.@@ -225,6 +225,7 @@ $FF_CACHE_DIR/$FF_WORK_DIR/ (local container disk, default /tmp/framefields)ws/<repo-hash>/<slot>/ # warm checkout + node_modules, leased one job at a timejobs/<jobId>/{out,logs}+ tmp/ # TMPDIR for the engine and installs; emptied at boot and after each jobinflight.json # the running job, so a crash can be reported after restart (8.3)```@@ -295,7 +296,8 @@ Free disk space decides (`src/worker/janitor.ts`); there is no fixed size budget- the workspace the bound engine was loaded from (always protected, also on the emergency paths);- pinned vision models;- another worker's mirror on a shared volume, until it has been idle for over an hour.-- **Failed-job dirs** kept by `FF_KEEP_FAILED_JOBS` are deleted after a day.+- **Failed-job dirs** kept by `FF_KEEP_FAILED_JOBS` are deleted after a day. Without that flag, `jobs/` is emptied at boot, which clears what a process that died mid-job left behind.+- **Temp files:** the worker points `TMPDIR` at `$FF_WORK_DIR/tmp`, so the engine's render temp files and install homes count against the work disk. It empties that dir at boot and after every job. A render killed mid-file can't leave gigabytes in the system temp dir.- **Optional hard caps** `FF_WORK_MAX_GB` / `FF_CACHE_MAX_GB` (default 0 = off) still apply on top. They walk the trees, counting each hard-linked file once.- **Container disks:** inside Docker, `statfs` on the overlay root may report the host's disk rather than the container's allowance, depending on the storage driver's quota support. Where that happens (to be checked on RunPod), set `FF_WORK_MAX_GB` below the container disk size.- Every eviction is logged ("Cache eviction") with the paths and the disks' free space.@@ -536,7 +538,7 @@ On boot the service validates **all** keys and fails with the complete list of p| Install | `FF_INSTALL_SCRIPTS` (true), `FF_ALLOW_UNLOCKED_INSTALL` (false), `FF_NPMRC` || Storage | `STORAGE_DRIVER` (`s3`\|`local`\|`none`), `STORAGE_ENDPOINT`, `STORAGE_REGION` (`auto`), `STORAGE_ACCESS_KEY_ID`, `STORAGE_SECRET_ACCESS_KEY`, `STORAGE_BUCKET`, `STORAGE_PUBLIC_BASE_URL`, `STORAGE_FORCE_PATH_STYLE` (MinIO), `FF_OUTPUT_BUCKETS`, `FF_OUTPUT_DIR`, `FF_INLINE_MAX_BYTES`, `FF_OUTPUT_MAX_BYTES`. The `R2_*` names are gone, with no aliases. || Paths & cache | `FF_CACHE_DIR`, `FF_WORK_DIR`, `FF_DISK_MIN_FREE_PCT` (15), `FF_DISK_MIN_FREE_GB` (10), `FF_DISK_TARGET_FREE_PCT` (25), `FF_SHARED_MODULES` (true), optional caps `FF_CACHE_MAX_GB` / `FF_WORK_MAX_GB` (0 = off), `FRAMEFIELDS_MODELS_DIR` (`$FF_CACHE_DIR/models`), `FRAMEFIELDS_FONTS_DIR` |-| Limits | `FF_QUEUE_MAX` (8), `FF_JOB_TIMEOUT_MAX_SEC` (7200), `FF_TIMEOUT_{GIT,LFS,INSTALL,LOAD,RENDER}_SEC`, `FF_STALL_TIMEOUT_SEC` (300), `FF_KILL_GRACE_SEC` (20), `FF_JOB_TTL_SEC` (3600), `FF_JOB_HISTORY_MAX` (500) |+| Limits | `FF_QUEUE_MAX` (8), `FF_JOB_TIMEOUT_MAX_SEC` (7200), `FF_TIMEOUT_{GIT,LFS,INSTALL,LOAD,RENDER}_SEC`, `FF_STALL_TIMEOUT_SEC` (300), `FF_KILL_GRACE_SEC` (20), `FF_JOB_TTL_SEC` (3600), `FF_JOB_HISTORY_MAX` (500), `FF_INLINE_TTL_SEC` (600) || Recycling | `FF_MAX_JOBS_PER_WORKER` (25; 0 = unlimited), `FF_RECYCLE_HEAP_MB` (4096), `FF_RECYCLE_RSS_MB` (0 = off), `FF_EVENT_LOOP_BLOCK_SEC` (120), `FF_STRAY_ERROR_LIMIT` (3), `FF_LEAKED_HANDLE_LIMIT` (50). `NODE_OPTIONS=--max-old-space-size` is set by the image. || GPU | `FF_GPU` (`auto`\|`require`\|`cpu`; default `auto`, RunPod templates `require`). The engine reads `FRAMEFIELDS_HW_CODECS=off` to force software video encoding. || Job env | `FF_JOB_ENV_ALLOW` (`*` minus a reserved denylist), `FF_KEEP_FAILED_JOBS` (false), `FF_QA_BLOCK_UPLOAD` (false) |diff --git a/apps/renderer-service/README.md b/apps/renderer-service/README.mdindex fac4856d..52072561 100644--- a/apps/renderer-service/README.md+++ b/apps/renderer-service/README.md@@ -211,6 +211,8 @@ Shared files are read-only, because changing one would change it for every projeSome filesystems report sizes that can't be true, such as Docker Desktop host mounts (around 118 TB) or a network volume reporting a whole storage cluster. Above 64 TiB a disk is marked `measurable: false` and the worker doesn't evict by free space there. Set `FF_WORK_MAX_GB` / `FF_CACHE_MAX_GB` if such a disk needs a limit.+**After each job** the worker deletes the job's folder and empties its temp dir (`$FF_WORK_DIR/tmp`, where the engine writes videos before they're copied out), and frees the engine's caches. At startup it empties both again, which clears what a crashed job left behind (keep failed jobs with `FF_KEEP_FAILED_JOBS=true`).+`FF_WORK_MAX_GB` / `FF_CACHE_MAX_GB` add hard caps if you want them. On RunPod the container disk reports its real size (no cap needed), but a network volume reports 2.3 PB, so set `FF_CACHE_MAX_GB` for it.## Jobs@@ -288,7 +290,7 @@ Versions of `framefields` released before worker support (2.1.3 and older) still| `s3` | `key` (template), `bucket` | The bucket must be in `FF_OUTPUT_BUCKETS` (defaults to `STORAGE_BUCKET`). Any S3-compatible store works: R2, MinIO, AWS. || `presigned` | `urls` (one per output), `headers` | The worker needs no storage credentials. Sign with an expiry longer than the job timeout. || `local` | `path` (template) | Written under `FF_OUTPUT_DIR`. |-| `inline` | — | Base64 in the result, up to `FF_INLINE_MAX_BYTES`. |+| `inline` | — | Base64 in the result, up to `FF_INLINE_MAX_BYTES`. Read it within `FF_INLINE_TTL_SEC` (10 min): after that the record keeps the output's details with `base64Expired: true`, to free memory. |Key and path templates can use `{jobId}`, `{sha}`, `{shortSha}`, `{ref}`, `{date}`, `{name}`, `{index}` and `{ext}`.@@ -325,6 +327,7 @@ Everything is set by environment variables. The worker checks them all at startu| `FF_CACHE_MAX_GB`, `FF_WORK_MAX_GB` | Optional hard caps on top of the free-space rule (default off). || `FF_JOB_TIMEOUT_MAX_SEC`, `FF_STALL_TIMEOUT_SEC` | Time limits. || `FF_MAX_JOBS_PER_WORKER` | Recycle after N jobs (default 25). |+| `FF_INLINE_TTL_SEC` | How long a finished job keeps its inline (base64) outputs in memory (default 600). || `FF_GPU` | `auto` (default), `require` or `cpu`. See [GPU and CPU rendering](#gpu-and-cpu-rendering). || `FRAMEFIELDS_HW_CODECS` | `off` encodes video in software even when a hardware encoder works. || `FF_NPMRC` | `.npmrc` contents, for private registries. |diff --git a/apps/renderer-service/src/config/schema.ts b/apps/renderer-service/src/config/schema.tsindex 02709f21..ea800ff3 100644--- a/apps/renderer-service/src/config/schema.ts+++ b/apps/renderer-service/src/config/schema.ts@@ -132,6 +132,7 @@ export const envSchema = z.object({FF_KILL_GRACE_SEC: int(20),FF_JOB_TTL_SEC: int(3600),FF_JOB_HISTORY_MAX: int(500),+ FF_INLINE_TTL_SEC: int(600),// GPU: auto = the GPU when the machine has one, else the CPU.FF_GPU: z.enum(["auto", "require", "cpu"]).default("auto"),@@ -374,6 +375,7 @@ export function buildConfig(killGraceSec: e.FF_KILL_GRACE_SEC,jobTtlSec: e.FF_JOB_TTL_SEC,jobHistoryMax: e.FF_JOB_HISTORY_MAX,+ inlineTtlSec: e.FF_INLINE_TTL_SEC,},gpu: e.FF_GPU,recycle: {diff --git a/apps/renderer-service/src/jobs/jobs.test.ts b/apps/renderer-service/src/jobs/jobs.test.tsindex 943b0736..291bbfc4 100644--- a/apps/renderer-service/src/jobs/jobs.test.ts+++ b/apps/renderer-service/src/jobs/jobs.test.ts@@ -228,6 +228,24 @@ describe("JobStore", () => {store.update("a", { status: "COMPLETED" });expect(store.get("a")?.status).toBe("CANCELLED");});++ it("drops inline payloads after the inline TTL but keeps the record", () => {+ let now = 0;+ const store = new JobStore(60_000, 10, () => now, 1000);+ store.create("a");+ store.update("a", {+ status: "COMPLETED",+ result: { outputs: [{ name: "film", base64: "AAAA" }] },+ });+ now = 500;+ expect(store.get("a")?.result?.outputs).toEqual([+ { name: "film", base64: "AAAA" },+ ]);+ now = 2000;+ expect(store.get("a")?.result?.outputs).toEqual([+ { name: "film", base64Expired: true },+ ]);+ });});describe("JobQueue", () => {diff --git a/apps/renderer-service/src/jobs/pipeline.ts b/apps/renderer-service/src/jobs/pipeline.tsindex f7d8cbe8..9fccdcd7 100644--- a/apps/renderer-service/src/jobs/pipeline.ts+++ b/apps/renderer-service/src/jobs/pipeline.ts@@ -53,6 +53,8 @@ export interface DeliveredFile {url?: string;path?: string;base64?: string;+ /** The inline payload was dropped after FF_INLINE_TTL_SEC. */+ base64Expired?: boolean;durationMs?: number;frames?: number;/** QA report, when render.qa was set. */diff --git a/apps/renderer-service/src/jobs/runner.ts b/apps/renderer-service/src/jobs/runner.tsindex e8285b4e..4d9c29bb 100644--- a/apps/renderer-service/src/jobs/runner.ts+++ b/apps/renderer-service/src/jobs/runner.ts@@ -33,7 +33,11 @@ export class JobRunner {this.store = new JobStore(config.limits.jobTtlSec * 1000,config.limits.jobHistoryMax,+ Date.now,+ config.limits.inlineTtlSec * 1000,);+ // Frees expired records and inline payloads while the worker is idle.+ setInterval(() => this.store.prune(), 60_000).unref();this.queue = new JobQueue(config.limits.queueMax);}diff --git a/apps/renderer-service/src/jobs/store.ts b/apps/renderer-service/src/jobs/store.tsindex 7a2b2176..2d86c486 100644--- a/apps/renderer-service/src/jobs/store.ts+++ b/apps/renderer-service/src/jobs/store.ts@@ -57,7 +57,9 @@ export function isTerminal(status: JobStatus): boolean {/*** In-memory job records with TTL and size-bounded eviction. Emits `update`- * with the record on every change (SSE progress).+ * with the record on every change (SSE progress). Inline (base64) outputs are+ * dropped from finished records after `inlineTtlMs`, so a busy worker doesn't+ * hold every rendered video in memory for the whole TTL.*/export class JobStore extends EventEmitter<{ update: [JobRecord] }> {private readonly jobs = new Map<string, JobRecord>();@@ -66,6 +68,7 @@ export class JobStore extends EventEmitter<{ update: [JobRecord] }> {private readonly ttlMs: number,private readonly maxHistory: number,private readonly now: () => number = Date.now,+ private readonly inlineTtlMs = Number.POSITIVE_INFINITY,) {super();this.setMaxListeners(0);@@ -88,7 +91,9 @@ export class JobStore extends EventEmitter<{ update: [JobRecord] }> {}get(id: string): JobRecord | undefined {- return this.jobs.get(id);+ const record = this.jobs.get(id);+ if (record) this.expireInline(record, this.now() - this.inlineTtlMs);+ return record;}update(@@ -117,9 +122,12 @@ export class JobStore extends EventEmitter<{ update: [JobRecord] }> {prune(): void {const cutoff = this.now() - this.ttlMs;+ const inlineCutoff = this.now() - this.inlineTtlMs;for (const [id, job] of this.jobs) {if (isTerminal(job.status) && Date.parse(job.updatedAt) < cutoff) {this.jobs.delete(id);+ } else {+ this.expireInline(job, inlineCutoff);}}// Oldest finished jobs go first; active jobs are never evicted.@@ -131,6 +139,19 @@ export class JobStore extends EventEmitter<{ update: [JobRecord] }> {}}+ /** Replaces old inline payloads with `base64Expired: true`. */+ private expireInline(job: JobRecord, cutoff: number): void {+ if (!isTerminal(job.status) || Date.parse(job.updatedAt) >= cutoff) return;+ const outputs = job.result?.outputs;+ if (!Array.isArray(outputs)) return;+ for (const output of outputs as Record<string, unknown>[]) {+ if (typeof output?.base64 === "string") {+ delete output.base64;+ output.base64Expired = true;+ }+ }+ }+get size(): number {return this.jobs.size;}diff --git a/apps/renderer-service/src/render/render-step.ts b/apps/renderer-service/src/render/render-step.tsindex 9dd4cda6..00d5900c 100644--- a/apps/renderer-service/src/render/render-step.ts+++ b/apps/renderer-service/src/render/render-step.ts@@ -8,6 +8,7 @@ import type {StepContext,} from "../jobs/pipeline.js";import type { JobInput, RenderSettings } from "../jobs/schema.js";+import { rendererLogger } from "../logger.js";import { gpuStatus } from "../worker/gpu.js";import { workerHealth } from "../worker/health.js";import { type BoundEngine, bindEngine, markDepsLoaded } from "./engine.js";@@ -126,6 +127,16 @@ export const render: Pipeline["render"] = async (checkout, input, ctx) => {sandbox.restore();clearWorkspaceRequireCache();await dispose?.().catch(() => {});+ if (engine.workerHooks) {+ // Free this job's decoders, textures and fonts now rather than when+ // the next job starts: an idle worker shouldn't hold them.+ try {+ engine.mod.resetRenderCaches?.();+ engine.mod.FontManager?.clear();+ } catch (err) {+ rendererLogger.warn({ err: String(err) }, "Render cache reset failed");+ }+ }}};diff --git a/apps/renderer-service/src/worker/boot.ts b/apps/renderer-service/src/worker/boot.tsindex cb7084de..8f2ef90f 100644--- a/apps/renderer-service/src/worker/boot.ts+++ b/apps/renderer-service/src/worker/boot.ts@@ -1,4 +1,4 @@-import { mkdirSync, realpathSync } from "node:fs";+import { mkdirSync, readdirSync, realpathSync, rmSync } from "node:fs";import path from "node:path";import {describeConfig,@@ -70,6 +70,23 @@ export function bootWorker(: "Rendering on the GPU",);+ // Temp files (the engine renders video there before copying it out,+ // installs put their HOME there) live under the work dir, where the+ // janitor sees their disk use and a crash's leftovers get cleared.+ // os.tmpdir() reads TMPDIR on every call.+ const tmpDir = path.join(config.paths.workDir, "tmp");+ rmSync(tmpDir, { recursive: true, force: true });+ mkdirSync(tmpDir, { recursive: true });+ process.env.TMPDIR = tmpDir;+ // Job dirs left by a process that died mid-job; the crash journal reports+ // the job, so only kept failures (FF_KEEP_FAILED_JOBS) are worth keeping.+ if (!config.keepFailedJobs) {+ rmSync(path.join(config.paths.workDir, "jobs"), {+ recursive: true,+ force: true,+ });+ }+// Workspaces live here; repo code under it gets the engine alias (§7).const workspaces = path.join(config.paths.workDir, "ws");mkdirSync(workspaces, { recursive: true });@@ -118,6 +135,7 @@ export function bootWorker(journal,beforeJob: (job) => janitor.maybeRun([repoHash(job.repo)]),afterJob: (completed) => {+ emptyTmp(tmpDir);const engine = boundEngine();if (engine && boundVersion !== engine.version) {// The engine's own timers and GPU handles aren't leaks.@@ -207,3 +225,22 @@ export function bootWorker(stop: stopWatchdog,};}++/**+ * Engine versions up to 2.2.0-beta.3 cache downloaded fonts by URL for the+ * life of the process, in `gatewai-fonts`; deleting them would break the next+ * job that uses the same URL.+ */+const KEEP_IN_TMP = new Set(["gatewai-fonts"]);++/** Removes what the job left in the temp dir (jobs run one at a time). */+function emptyTmp(dir: string): void {+ try {+ for (const name of readdirSync(dir)) {+ if (KEEP_IN_TMP.has(name)) continue;+ rmSync(path.join(dir, name), { recursive: true, force: true });+ }+ } catch (err) {+ rendererLogger.warn({ dir, err: String(err) }, "Temp dir cleanup failed");+ }+}diff --git a/packages/renderer/src/job-reset.ts b/packages/renderer/src/job-reset.tsindex 4e9706d1..ff165211 100644--- a/packages/renderer/src/job-reset.ts+++ b/packages/renderer/src/job-reset.ts@@ -1,5 +1,6 @@import {clearAllVideoCache,+ clearHeadlessFonts,inputStore,lutStore,mediaDecoderCache,@@ -9,7 +10,7 @@ import {/*** Drops every cache keyed by a file path or URL: media inputs and decoders,- * textures, LUTs and fonts. A long-lived worker calls this between jobs,+ * textures, LUTs, fonts and downloaded font files. A long-lived worker calls this between jobs,* because the next checkout may hold different bytes at the same path. The* GPU device and compiled pipelines stay warm.*/@@ -20,4 +21,5 @@ export function resetRenderCaches(): void {inputStore.clear();lutStore.destroy();SlugFontCache.destroy();+ void clearHeadlessFonts();}diff --git a/packages/webgpu-renderers/src/headless-font-loader.test.ts b/packages/webgpu-renderers/src/headless-font-loader.test.tsnew file mode 100644index 00000000..8c06000a--- /dev/null+++ b/packages/webgpu-renderers/src/headless-font-loader.test.ts@@ -0,0 +1,44 @@+import { existsSync } from "node:fs";+import path from "node:path";+import { afterEach, describe, expect, it, vi } from "vitest";+import {+ clearHeadlessFonts,+ getHeadlessFontPath,+ registerHeadlessFont,+} from "./headless-font-loader.js";++describe("clearHeadlessFonts", () => {+ afterEach(async () => {+ vi.unstubAllGlobals();+ await clearHeadlessFonts();+ });++ it("lets the next job register a family from another file", async () => {+ expect(await registerHeadlessFont("Inter", "file:///a/Inter.ttf")).toBe(+ "/a/Inter.ttf",+ );+ // Without a reset, the first path wins for the life of the process.+ expect(await registerHeadlessFont("Inter", "file:///b/Inter.ttf")).toBe(+ "/a/Inter.ttf",+ );+ await clearHeadlessFonts();+ expect(getHeadlessFontPath("Inter")).toBeUndefined();+ expect(await registerHeadlessFont("Inter", "file:///b/Inter.ttf")).toBe(+ "/b/Inter.ttf",+ );+ });++ it("deletes fonts downloaded from URLs", async () => {+ vi.stubGlobal(+ "fetch",+ vi.fn(async () => new Response(new Uint8Array([0, 1, 0, 0]))),+ );+ const file = await registerHeadlessFont(+ "Remote",+ "https://fonts.example/Remote.ttf",+ );+ expect(file && existsSync(file)).toBe(true);+ await clearHeadlessFonts();+ expect(existsSync(path.dirname(file as string))).toBe(false);+ });+});diff --git a/packages/webgpu-renderers/src/headless-font-loader.ts b/packages/webgpu-renderers/src/headless-font-loader.tsindex c2d5aebf..991ea899 100644--- a/packages/webgpu-renderers/src/headless-font-loader.ts+++ b/packages/webgpu-renderers/src/headless-font-loader.ts@@ -1,7 +1,7 @@import { BASE_URL } from "@framefields/client-utils";import { debugLog } from "./debug-log.js";-let tempDir: string | null = null;+let tempDir: Promise<string> | null = null;const fontPathCache = new Map<string, string>();const registeredFonts = new Set<string>();const fontPaths = new Map<string, string>();@@ -54,10 +54,15 @@ async function getLocalFontPath(family: string, url: string): Promise<string> {const path = await import(/* webpackIgnore: true */ "path");const os = await import(/* webpackIgnore: true */ "os");- if (!tempDir) {- tempDir = path.join(os.tmpdir(), "gatewai-fonts");- await fs.mkdir(tempDir, { recursive: true }).catch(() => {});- }+ // A fresh dir per session, so clearHeadlessFonts() can delete the last+ // one without racing new downloads.+ tempDir ??= fs+ .mkdtemp(path.join(os.tmpdir(), "gatewai-fonts-"))+ .catch((err) => {+ tempDir = null;+ throw err;+ });+ const dir = await tempDir;const safeFamily = family.replace(/[^a-zA-Z0-9_-]/g, "_");const urlObj = new URL(resolvedUrl);@@ -65,7 +70,7 @@ async function getLocalFontPath(family: string, url: string): Promise<string> {if (!ext || ext.length > 5) {ext = ".ttf";}- const destPath = path.join(tempDir, `${safeFamily}-${Date.now()}${ext}`);+ const destPath = path.join(dir, `${safeFamily}-${Date.now()}${ext}`);const response = await fetch(resolvedUrl);if (!response.ok) {@@ -80,6 +85,26 @@ async function getLocalFontPath(family: string, url: string): Promise<string> {return destPath;}+/**+ * Forgets every registered font and deletes the fonts downloaded from URLs.+ * A long-lived process calls it between jobs (via `resetRenderCaches`): the+ * next job may register the same family from another file, or fetch a URL+ * whose font changed.+ */+export async function clearHeadlessFonts(): Promise<void> {+ fontPathCache.clear();+ fontPaths.clear();+ fontDataCache.clear();+ registeredFonts.clear();+ const pending = tempDir;+ tempDir = null;+ if (pending) {+ const fs = await import(/* webpackIgnore: true */ "fs/promises");+ const dir = await pending.catch(() => undefined);+ if (dir) await fs.rm(dir, { recursive: true, force: true }).catch(() => {});+ }+}+export async function registerHeadlessFont(family: string,url: string,