Skip to content

The VS Code of video editing, built for agents.

TypeScript

Add `useFile` hook for resolving file sources in effects

konstantin-paulus8b9c8ef0ccff
diff --git a/apps/cli/reference/jsx/lifecycle.md b/apps/cli/reference/jsx/lifecycle.md
index bb318476..50ed64c5 100644
--- a/apps/cli/reference/jsx/lifecycle.md
+++ b/apps/cli/reference/jsx/lifecycle.md
@@ -51,3 +51,29 @@ Call it in a component body. It returns accessors for the playhead of the scene
| `playing()` | Whether the scene is playing |
The values respect play, pause, scrubbing, looping, and playback speed, which wall-clock timers do not. Each accessor only propagates when its value changes, so a paused scene re-runs nothing and `frame()` consumers update at most once per frame. In a one-shot mount the first value renders and then freezes; the hook is only useful with `--live`.
+
+## `useFile`
+
+Resolves a [`src`](./media.md) (path, asset id, URL, or a `generate.*` ref) to its `File`, so effects can read the raw bytes: draw a library image onto a [`<canvas>`](./canvas-paint.md), parse a data file, decode audio. The module is sandboxed and can't fetch a path or asset id itself, so resolution goes through the host, exactly as `src` does.
+
+```tsx
+import { createSignal, createEffect } from "solid-js";
+import { useFile } from "@diffusionstudio/jsx";
+
+function Logo() {
+ const [canvas, setCanvas] = createSignal<HTMLCanvasElement>();
+ const [file] = useFile("/assets/logo.png"); // path, asset id, URL, or AssetRef
+
+ createEffect(async () => {
+ const el = canvas();
+ const f = file(); // undefined until it resolves
+ if (!el || !f) return;
+ const ctx = el.getContext("2d")!;
+ ctx.drawImage(await createImageBitmap(f), 0, 0, el.width, el.height);
+ });
+
+ return <canvas ref={setCanvas} width={640} height={360} />;
+}
+```
+
+Returns Solid's `createResource` tuple unchanged: `[file, { mutate, refetch }]`. The `file` accessor reads `undefined` until resolution completes, then the `File`; `file.loading` and `file.error` report progress and failure. Resolution is async (fetch a URL, read a path or library asset, await a `generate.*` ref) and only runs in a live graph; in a one-shot mount there is no effect to consume the result.
diff --git a/apps/cli/reference/jsx/media.md b/apps/cli/reference/jsx/media.md
index fd6cb2a8..06a827c0 100644
--- a/apps/cli/reference/jsx/media.md
+++ b/apps/cli/reference/jsx/media.md
@@ -8,3 +8,5 @@
- **`AssetRef`**: the value returned by a `generate.*` declaration (see [generate.md](./generate.md)). The node is inserted immediately as a placeholder and its paint is attached once the asset has generated.
In [`dapi node patch`](../node/patch.md), `src` takes a path, URL, or asset id; `generate.*` declarations are JSX-only.
+
+To read a source's raw bytes inside an effect (rather than mount it as a node), pass the same input to [`useFile`](./lifecycle.md#usefile), which resolves it to a `File`.
diff --git a/apps/web/src/utils/jsx.ts b/apps/web/src/utils/jsx.ts
index 7051d799..55aba7f8 100644
--- a/apps/web/src/utils/jsx.ts
+++ b/apps/web/src/utils/jsx.ts
@@ -45,7 +45,7 @@ import { ASPECT_RATIO_DIMENSIONS, findEmptyPlacement } from "@/utils/genai";
import { resolveAsset, resolveGeneratedAsset } from "@/utils/jsx-generation";
import { assert, assertAllSettled, parseColor } from "@/utils";
-import type { AssetRef, AssetSpecInput, ProjectDocument, ProjectTick } from "@diffusionstudio/jsx";
+import type { AssetInput, AssetRef, AssetSpecInput, ProjectDocument, ProjectTick } from "@diffusionstudio/jsx";
import type { Engine, EngineWorld } from "@/components/engine";
import type { GenerationMemo } from "@/utils/jsx-generation";
@@ -431,6 +431,19 @@ export class WorldDocument implements ProjectDocument<DocumentNode> {
return this.ticker[0]();
}
+ /**
+ * Backs `useFile`: resolves a source (path, URL, asset id, or `AssetRef`) to
+ * its `File`, the same input `src` accepts. A `generate.*` ref generates on
+ * demand (content-cached in the library, so it collapses with the mount's own
+ * generation of the same spec).
+ */
+ public async loadFile(input: AssetInput): Promise<File> {
+ const asset = isAssetRef(input)
+ ? await resolveGeneratedAsset(this.world, input, new Map())
+ : await resolveAsset(this.world, input);
+ return await asset.handle.getFile();
+ }
+
/**
* Reads the playhead of the nearest Playback-carrying ancestor of the
* mount's root (the root itself for a mounted scene) into the ticker
diff --git a/docs/composition.md b/docs/composition.md
index e5348040..7bee956d 100644
--- a/docs/composition.md
+++ b/docs/composition.md
@@ -88,6 +88,25 @@ These same props are what `dapi node patch` assigns on existing nodes.
- an **asset id**: an imported library asset (see `dapi asset tree`)
- an **`AssetRef`**: the value returned by a `generate.*` declaration
+To read a source's raw bytes inside an effect instead of mounting it as a node, pass the same input to `useFile`, which resolves it to a `File`. It returns Solid's `createResource` tuple (`[file, { mutate, refetch }]`): the `file` accessor reads `undefined` until it resolves, then the `File`. Useful for drawing a library image onto a `<canvas>` or parsing a data file:
+
+```tsx
+import { createSignal, createEffect } from "solid-js";
+import { useFile } from "@diffusionstudio/jsx";
+
+const [canvas, setCanvas] = createSignal<HTMLCanvasElement>();
+const [file] = useFile("/assets/logo.png");
+
+createEffect(async () => {
+ const el = canvas();
+ const f = file();
+ if (!el || !f) return;
+ el.getContext("2d")!.drawImage(await createImageBitmap(f), 0, 0);
+});
+
+// <canvas ref={setCanvas} width={640} height={360} />
+```
+
## Timing
`start`/`end` place a clip on the parent timeline; `sourceIn`/`sourceOut` select which part of its source plays. The two are independent. Values take any [time format](cli.md#time-values) (seconds, `"45f"` frames, `"MM:SS"`).
diff --git a/packages/jsx/src/document.ts b/packages/jsx/src/document.ts
index f0244a81..a0e7b924 100644
--- a/packages/jsx/src/document.ts
+++ b/packages/jsx/src/document.ts
@@ -10,6 +10,8 @@
* opaque to the renderer.
*/
+import type { AssetInput } from "./generate";
+
/**
* One reading of the host's timeline clock, local to the mounted project:
* the playhead of the scene the mount's root lives in (or is). Values only
@@ -41,4 +43,5 @@ export interface ProjectDocument<N = unknown> {
getNextSibling(node: N): N | undefined;
tick?(): ProjectTick;
applyRef?(node: N, ref: (target: unknown) => void): void;
+ loadFile?(input: AssetInput): Promise<File>;
}
diff --git a/packages/jsx/src/index.ts b/packages/jsx/src/index.ts
index f56fae2e..097ddd61 100644
--- a/packages/jsx/src/index.ts
+++ b/packages/jsx/src/index.ts
@@ -15,7 +15,7 @@ export type {
} from "./generate";
export { PATCH_PROP_KEYS } from "./types";
export { parseTime, TIME_FPS } from "./time";
-export { useTicker } from "./renderer";
+export { useTicker, useFile } from "./renderer";
export type { Ticker } from "./renderer";
export type {
Animatable,
diff --git a/packages/jsx/src/renderer.ts b/packages/jsx/src/renderer.ts
index 5b657656..e57f6784 100644
--- a/packages/jsx/src/renderer.ts
+++ b/packages/jsx/src/renderer.ts
@@ -2,10 +2,11 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
-import { createContext, createMemo, useContext } from "solid-js";
+import { createContext, createMemo, createResource, useContext } from "solid-js";
import { createRenderer } from "solid-js/universal";
-import type { Accessor, JSX } from "solid-js";
+import type { Accessor, JSX, ResourceReturn } from "solid-js";
+import type { AssetInput } from "./generate";
import type { ProjectDocument } from "./document";
// Compiled project modules call the static runtime exports below
@@ -114,6 +115,37 @@ export function useTicker(): Ticker {
};
}
+/**
+ * Resolves a source (path, asset id, URL, or a `generate.*` ref) to its `File`,
+ * for reading raw bytes inside effects. Returns Solid's `createResource` tuple
+ * unchanged: the resource accessor reads `undefined` until it resolves, then
+ * the `File` (with `.loading` / `.error`), plus `mutate` / `refetch`.
+ * Resolution is async (fetch a URL, read a path or library asset, await a
+ * `generate.*` ref) and, like `src`, needs the host — a sandboxed module can't
+ * fetch a path or asset id itself.
+ *
+ * ```tsx
+ * const [canvas, setCanvas] = createSignal<HTMLCanvasElement>();
+ * const [file] = useFile("/assets/logo.png");
+ * createEffect(async () => {
+ * const el = canvas();
+ * const f = file();
+ * if (!el || !f) return;
+ * el.getContext("2d")!.drawImage(await createImageBitmap(f), 0, 0);
+ * });
+ * // <canvas ref={setCanvas} width={640} height={360} />
+ * ```
+ */
+export function useFile(src: AssetInput): ResourceReturn<File> {
+ const document = doc();
+ if (!document.loadFile) {
+ throw new Error("useFile: this host does not provide file resolution");
+ }
+
+ const load = document.loadFile.bind(document);
+ return createResource(() => src, (input) => load(input));
+}
+
/**
* Host entry point: renders a project component directly into `document`.
* Mounting is synchronous — the document is fully written when this returns.