The VS Code of video editing, built for agents.
TypeScript
Add `useFile` hook for resolving file sources in effects
konstantin-paulus8b9c8ef0ccff
diff --git a/apps/cli/reference/jsx/lifecycle.md b/apps/cli/reference/jsx/lifecycle.mdindex bb318476..50ed64c5 100644--- a/apps/cli/reference/jsx/lifecycle.md+++ b/apps/cli/reference/jsx/lifecycle.md@@ -51,3 +51,29 @@ Call it in a component body. It returns accessors for the playhead of the scene| `playing()` | Whether the scene is playing |The values respect play, pause, scrubbing, looping, and playback speed, which wall-clock timers do not. Each accessor only propagates when its value changes, so a paused scene re-runs nothing and `frame()` consumers update at most once per frame. In a one-shot mount the first value renders and then freezes; the hook is only useful with `--live`.++## `useFile`++Resolves a [`src`](./media.md) (path, asset id, URL, or a `generate.*` ref) to its `File`, so effects can read the raw bytes: draw a library image onto a [`<canvas>`](./canvas-paint.md), parse a data file, decode audio. The module is sandboxed and can't fetch a path or asset id itself, so resolution goes through the host, exactly as `src` does.++```tsx+import { createSignal, createEffect } from "solid-js";+import { useFile } from "@diffusionstudio/jsx";++function Logo() {+ const [canvas, setCanvas] = createSignal<HTMLCanvasElement>();+ const [file] = useFile("/assets/logo.png"); // path, asset id, URL, or AssetRef++ createEffect(async () => {+ const el = canvas();+ const f = file(); // undefined until it resolves+ if (!el || !f) return;+ const ctx = el.getContext("2d")!;+ ctx.drawImage(await createImageBitmap(f), 0, 0, el.width, el.height);+ });++ return <canvas ref={setCanvas} width={640} height={360} />;+}+```++Returns Solid's `createResource` tuple unchanged: `[file, { mutate, refetch }]`. The `file` accessor reads `undefined` until resolution completes, then the `File`; `file.loading` and `file.error` report progress and failure. Resolution is async (fetch a URL, read a path or library asset, await a `generate.*` ref) and only runs in a live graph; in a one-shot mount there is no effect to consume the result.diff --git a/apps/cli/reference/jsx/media.md b/apps/cli/reference/jsx/media.mdindex fd6cb2a8..06a827c0 100644--- a/apps/cli/reference/jsx/media.md+++ b/apps/cli/reference/jsx/media.md@@ -8,3 +8,5 @@- **`AssetRef`**: the value returned by a `generate.*` declaration (see [generate.md](./generate.md)). The node is inserted immediately as a placeholder and its paint is attached once the asset has generated.In [`dapi node patch`](../node/patch.md), `src` takes a path, URL, or asset id; `generate.*` declarations are JSX-only.++To read a source's raw bytes inside an effect (rather than mount it as a node), pass the same input to [`useFile`](./lifecycle.md#usefile), which resolves it to a `File`.diff --git a/apps/web/src/utils/jsx.ts b/apps/web/src/utils/jsx.tsindex 7051d799..55aba7f8 100644--- a/apps/web/src/utils/jsx.ts+++ b/apps/web/src/utils/jsx.ts@@ -45,7 +45,7 @@ import { ASPECT_RATIO_DIMENSIONS, findEmptyPlacement } from "@/utils/genai";import { resolveAsset, resolveGeneratedAsset } from "@/utils/jsx-generation";import { assert, assertAllSettled, parseColor } from "@/utils";-import type { AssetRef, AssetSpecInput, ProjectDocument, ProjectTick } from "@diffusionstudio/jsx";+import type { AssetInput, AssetRef, AssetSpecInput, ProjectDocument, ProjectTick } from "@diffusionstudio/jsx";import type { Engine, EngineWorld } from "@/components/engine";import type { GenerationMemo } from "@/utils/jsx-generation";@@ -431,6 +431,19 @@ export class WorldDocument implements ProjectDocument<DocumentNode> {return this.ticker[0]();}+ /**+ * Backs `useFile`: resolves a source (path, URL, asset id, or `AssetRef`) to+ * its `File`, the same input `src` accepts. A `generate.*` ref generates on+ * demand (content-cached in the library, so it collapses with the mount's own+ * generation of the same spec).+ */+ public async loadFile(input: AssetInput): Promise<File> {+ const asset = isAssetRef(input)+ ? await resolveGeneratedAsset(this.world, input, new Map())+ : await resolveAsset(this.world, input);+ return await asset.handle.getFile();+ }+/*** Reads the playhead of the nearest Playback-carrying ancestor of the* mount's root (the root itself for a mounted scene) into the tickerdiff --git a/docs/composition.md b/docs/composition.mdindex e5348040..7bee956d 100644--- a/docs/composition.md+++ b/docs/composition.md@@ -88,6 +88,25 @@ These same props are what `dapi node patch` assigns on existing nodes.- an **asset id**: an imported library asset (see `dapi asset tree`)- an **`AssetRef`**: the value returned by a `generate.*` declaration+To read a source's raw bytes inside an effect instead of mounting it as a node, pass the same input to `useFile`, which resolves it to a `File`. It returns Solid's `createResource` tuple (`[file, { mutate, refetch }]`): the `file` accessor reads `undefined` until it resolves, then the `File`. Useful for drawing a library image onto a `<canvas>` or parsing a data file:++```tsx+import { createSignal, createEffect } from "solid-js";+import { useFile } from "@diffusionstudio/jsx";++const [canvas, setCanvas] = createSignal<HTMLCanvasElement>();+const [file] = useFile("/assets/logo.png");++createEffect(async () => {+ const el = canvas();+ const f = file();+ if (!el || !f) return;+ el.getContext("2d")!.drawImage(await createImageBitmap(f), 0, 0);+});++// <canvas ref={setCanvas} width={640} height={360} />+```+## Timing`start`/`end` place a clip on the parent timeline; `sourceIn`/`sourceOut` select which part of its source plays. The two are independent. Values take any [time format](cli.md#time-values) (seconds, `"45f"` frames, `"MM:SS"`).diff --git a/packages/jsx/src/document.ts b/packages/jsx/src/document.tsindex f0244a81..a0e7b924 100644--- a/packages/jsx/src/document.ts+++ b/packages/jsx/src/document.ts@@ -10,6 +10,8 @@* opaque to the renderer.*/+import type { AssetInput } from "./generate";+/*** One reading of the host's timeline clock, local to the mounted project:* the playhead of the scene the mount's root lives in (or is). Values only@@ -41,4 +43,5 @@ export interface ProjectDocument<N = unknown> {getNextSibling(node: N): N | undefined;tick?(): ProjectTick;applyRef?(node: N, ref: (target: unknown) => void): void;+ loadFile?(input: AssetInput): Promise<File>;}diff --git a/packages/jsx/src/index.ts b/packages/jsx/src/index.tsindex f56fae2e..097ddd61 100644--- a/packages/jsx/src/index.ts+++ b/packages/jsx/src/index.ts@@ -15,7 +15,7 @@ export type {} from "./generate";export { PATCH_PROP_KEYS } from "./types";export { parseTime, TIME_FPS } from "./time";-export { useTicker } from "./renderer";+export { useTicker, useFile } from "./renderer";export type { Ticker } from "./renderer";export type {Animatable,diff --git a/packages/jsx/src/renderer.ts b/packages/jsx/src/renderer.tsindex 5b657656..e57f6784 100644--- a/packages/jsx/src/renderer.ts+++ b/packages/jsx/src/renderer.ts@@ -2,10 +2,11 @@* License, v. 2.0. If a copy of the MPL was not distributed with this* file, You can obtain one at http://mozilla.org/MPL/2.0/. */-import { createContext, createMemo, useContext } from "solid-js";+import { createContext, createMemo, createResource, useContext } from "solid-js";import { createRenderer } from "solid-js/universal";-import type { Accessor, JSX } from "solid-js";+import type { Accessor, JSX, ResourceReturn } from "solid-js";+import type { AssetInput } from "./generate";import type { ProjectDocument } from "./document";// Compiled project modules call the static runtime exports below@@ -114,6 +115,37 @@ export function useTicker(): Ticker {};}+/**+ * Resolves a source (path, asset id, URL, or a `generate.*` ref) to its `File`,+ * for reading raw bytes inside effects. Returns Solid's `createResource` tuple+ * unchanged: the resource accessor reads `undefined` until it resolves, then+ * the `File` (with `.loading` / `.error`), plus `mutate` / `refetch`.+ * Resolution is async (fetch a URL, read a path or library asset, await a+ * `generate.*` ref) and, like `src`, needs the host — a sandboxed module can't+ * fetch a path or asset id itself.+ *+ * ```tsx+ * const [canvas, setCanvas] = createSignal<HTMLCanvasElement>();+ * const [file] = useFile("/assets/logo.png");+ * createEffect(async () => {+ * const el = canvas();+ * const f = file();+ * if (!el || !f) return;+ * el.getContext("2d")!.drawImage(await createImageBitmap(f), 0, 0);+ * });+ * // <canvas ref={setCanvas} width={640} height={360} />+ * ```+ */+export function useFile(src: AssetInput): ResourceReturn<File> {+ const document = doc();+ if (!document.loadFile) {+ throw new Error("useFile: this host does not provide file resolution");+ }++ const load = document.loadFile.bind(document);+ return createResource(() => src, (input) => load(input));+}+/*** Host entry point: renders a project component directly into `document`.* Mounting is synchronous — the document is fully written when this returns.