Skip to content

The VS Code of video editing, built for agents.

TypeScript

Enhance documentation and implement live mode features for mounting

konstantin-paulus86b06ac30079
diff --git a/apps/cli/reference/jsx/README.md b/apps/cli/reference/jsx/README.md
index 0a0b562c..7f1dfd3b 100644
--- a/apps/cli/reference/jsx/README.md
+++ b/apps/cli/reference/jsx/README.md
@@ -26,7 +26,7 @@ The pipeline is driven by two commands: [`dapi mount`](../mount.md) renders the
| [audio-sync.md](./audio-sync.md) | `syncTo` audio alignment |
| [captions.md](./captions.md) | `<captions>` and style presets |
| [generate.md](./generate.md) | Declarative AI asset generation (`generate.*`) |
-| [lifecycle.md](./lifecycle.md) | One-shot render lifecycle |
+| [lifecycle.md](./lifecycle.md) | Render lifecycle: one-shot vs `--live`, `useTicker` |
| [errors.md](./errors.md) | Where each pipeline stage fails and with what effect |
## Pipeline
diff --git a/apps/cli/reference/jsx/lifecycle.md b/apps/cli/reference/jsx/lifecycle.md
index 83f50f15..bb318476 100644
--- a/apps/cli/reference/jsx/lifecycle.md
+++ b/apps/cli/reference/jsx/lifecycle.md
@@ -1,9 +1,53 @@
# Lifecycle
+## One-shot render (default)
+
Mounting is a **one-shot render**: the reactive graph exists to *compose* the document (loops, conditionals, derived values, component props), not to drive it afterward.
1. The component tree renders synchronously into the staging root; `onMount` callbacks flush once.
2. The subtree commits and the reactive root is **disposed**. Signal changes after commit do not affect the document.
3. Ownership transfers to the document: every materialized node is a fully editable composition node. Asset generation is owned by the engine, not the reactive graph, so it proceeds normally after disposal.
-The update path is **re-running the project**, which rebuilds the mount root in place, like refreshing a webpage; unchanged asset specs hit the generation cache. A future *live mode* may keep the reactive root mounted so signals animate the document; the one-shot contract above is deliberately forward-compatible with it.
+The update path is **re-running the project**, which rebuilds the mount root in place, like refreshing a webpage; unchanged asset specs hit the generation cache.
+
+## Live mode (`dapi mount --live`)
+
+`--live` skips the disposal in step 2: the reactive root stays mounted after the command returns, so signals, effects, and timers keep driving the composition. Everything else is unchanged; the nodes are ordinary editable entities and generation blocks the command as usual.
+
+A live run ends when:
+
+- a later `mount` (live or not) claims one of its root keys: the keyed replacement swaps the entities and disposes the old graph (`onCleanup` callbacks flush), or
+- the project or app closes.
+
+Inside a live graph, updates behave as in any Solid app: prop writes, conditional inserts and removals (`<Show>`, `<For>`), and text updates land in the document immediately. They are not recorded as undo steps. A reactive `src` swap works, including `generate.*` refs (the node shows its generating state until the new asset lands). Two things stay mount-only and throw if changed after commit: `syncTo` and creating new `<captions>`; re-mount for those.
+
+`node insert` is always one-shot; only `mount` supports `--live`.
+
+## `useTicker`
+
+Live graphs can subscribe to the project's timeline instead of reaching for wall-clock timers:
+
+```tsx
+import { useTicker } from "@diffusionstudio/jsx";
+
+export default function Project() {
+ const { time, frame } = useTicker();
+ return (
+ <scene key="hud" width={1920} height={1080}>
+ <text fontSize={80}>{`frame ${frame()}`}</text>
+ <rect x={860 + Math.sin(time() * 4) * 200} y={490} width={100} height={100} fill="#f43" />
+ </scene>
+ );
+}
+```
+
+Call it in a component body. It returns accessors for the playhead of the scene the mount's root lives in (or is):
+
+| Accessor | Value |
+| -------- | ----- |
+| `time()` | Playhead in seconds (sub-frame precision while playing) |
+| `frame()` | Playhead in frames (30 fps) |
+| `delta()` | Seconds advanced since the previous engine tick: 0 while paused, negative on a backward scrub or loop |
+| `playing()` | Whether the scene is playing |
+
+The values respect play, pause, scrubbing, looping, and playback speed, which wall-clock timers do not. Each accessor only propagates when its value changes, so a paused scene re-runs nothing and `frame()` consumers update at most once per frame. In a one-shot mount the first value renders and then freezes; the hook is only useful with `--live`.
diff --git a/apps/cli/reference/mount.md b/apps/cli/reference/mount.md
index 917e0ec5..dd27b438 100644
--- a/apps/cli/reference/mount.md
+++ b/apps/cli/reference/mount.md
@@ -11,6 +11,10 @@ Exactly one of:
- `<path>`: path to a `.tsx` / `.jsx` / `.ts` / `.js` entry module.
- `--code <str>`: inline module source, compiled identically. The `export default () =>` wrapper is optional; a bare JSX expression (no `export default` in the source) is wrapped into a component module automatically.
+Optionally:
+
+- `--live`: keep the reactive graph alive after mounting, so signals, effects, timers, and [`useTicker`](./jsx/lifecycle.md#useticker) keep driving the mounted entities. Without it the graph is disposed once the mount lands (the entities stay, reactivity stops). A live run ends when a later mount claims one of its root keys or the project closes (see [jsx/lifecycle.md](./jsx/lifecycle.md)).
+
## Output
None. The command exits `0` once the tree is committed and every declared asset has generated; inspect the result with [`dapi context`](./context.md), [`dapi node ls`](./node/ls.md), or [`dapi node tree`](./node/tree.md).
diff --git a/apps/web/src/components/engine/systems/playback.ts b/apps/web/src/components/engine/systems/playback.ts
index 2bd05986..7f106c82 100644
--- a/apps/web/src/components/engine/systems/playback.ts
+++ b/apps/web/src/components/engine/systems/playback.ts
@@ -409,4 +409,6 @@ export function playbackSystem(world: EngineWorld): void {
}
}
}
+
+ world.liveMounts.advance();
}
diff --git a/apps/web/src/components/engine/utils/live-mounts.ts b/apps/web/src/components/engine/utils/live-mounts.ts
index 4c86635a..8ff079ae 100644
--- a/apps/web/src/components/engine/utils/live-mounts.ts
+++ b/apps/web/src/components/engine/utils/live-mounts.ts
@@ -11,13 +11,22 @@
* disposed with them. Root keys are read live rather than snapshotted, so
* roots a running graph adds or removes later still reconcile.
*/
-export type LiveMount = { rootKeys: () => string[]; dispose: () => void };
+export type LiveMount = {
+ rootKeys: () => string[];
+ advance: () => void;
+ dispose: () => void;
+};
export type LiveMounts = ReturnType<typeof createLiveMounts>;
export function createLiveMounts() {
const mounts = new Set<LiveMount>();
return {
+ advance(): void {
+ for (const mount of mounts) {
+ mount.advance();
+ }
+ },
supersede(keys: string[]): void {
for (const mount of mounts) {
if (keys.some((key) => mount.rootKeys().includes(key))) {
diff --git a/apps/web/src/context/editor-api/mount.ts b/apps/web/src/context/editor-api/mount.ts
index 3d35f3eb..eabb92a2 100644
--- a/apps/web/src/context/editor-api/mount.ts
+++ b/apps/web/src/context/editor-api/mount.ts
@@ -65,7 +65,11 @@ export function handleMount(engine: Accessor<Engine>) {
}
if (live) {
- world.liveMounts.register({ rootKeys: () => documentRootKeys(world, document), dispose });
+ world.liveMounts.register({
+ rootKeys: () => documentRootKeys(world, document),
+ advance: () => document.advanceTicker(),
+ dispose,
+ });
persisted = true;
}
diff --git a/apps/web/src/utils/jsx.ts b/apps/web/src/utils/jsx.ts
index dabd3ff5..6929041b 100644
--- a/apps/web/src/utils/jsx.ts
+++ b/apps/web/src/utils/jsx.ts
@@ -43,7 +43,7 @@ import { ASPECT_RATIO_DIMENSIONS, findEmptyPlacement } from "@/utils/genai";
import { resolveAsset, resolveGeneratedAsset } from "@/utils/jsx-generation";
import { assert, assertAllSettled, parseColor } from "@/utils";
-import type { AssetRef, AssetSpecInput, ProjectDocument } from "@diffusionstudio/jsx";
+import type { AssetRef, AssetSpecInput, ProjectDocument, ProjectTick } from "@diffusionstudio/jsx";
import type { Engine, EngineWorld } from "@/components/engine";
import type { GenerationMemo } from "@/utils/jsx-generation";
@@ -392,6 +392,50 @@ export class WorldDocument implements ProjectDocument<DocumentNode> {
return this.engine.world;
}
+ private ticker = solid.createSignal<ProjectTick>(
+ { time: 0, frame: 0, delta: 0, playing: false },
+ { equals: (a, b) => a.time === b.time && a.frame === b.frame && a.delta === b.delta && a.playing === b.playing },
+ );
+ private lastTickTime: number | null = null;
+
+ public tick(): ProjectTick {
+ return this.ticker[0]();
+ }
+
+ /**
+ * Reads the playhead of the nearest Playback-carrying ancestor of the
+ * mount's root (the root itself for a mounted scene) into the ticker
+ * signal. Called by the playback system once per tick.
+ */
+ public advanceTicker(): void {
+ const c = this.world.components;
+ const sid = this.playbackRoot();
+ const time = sid !== null ? (c.Computed.localTimeInSeconds[sid] ?? 0) : 0;
+ const delta = this.lastTickTime === null ? 0 : time - this.lastTickTime;
+ this.lastTickTime = time;
+ this.ticker[1]({
+ time,
+ frame: sid !== null ? (c.Computed.localTime[sid] ?? 0) : 0,
+ delta,
+ playing: sid !== null && c.Playback.playing[sid] === 1,
+ });
+ }
+
+ private playbackRoot(): number | null {
+ // Resolved per tick rather than cached: the root lands after component
+ // bodies run (so it is null during the first render), and reparenting or
+ // keyed replacement can change the answer later.
+
+ const world = this.world;
+ const c = world.components;
+ let current = this.rootEid ?? (this.stage.kind === "root" ? this.stage.eid : undefined) ?? null;
+ while (current !== null && !hasComponent(world, current, c.Deleted)) {
+ if (hasComponent(world, current, c.Playback)) return current;
+ current = getParentEntity(world, current);
+ }
+ return null;
+ }
+
/**
* Resolves a generated asset and attaches it: during commit via the genai
* queue, or immediately when a live mount sets an asset ref after commit.
@@ -423,6 +467,7 @@ export class WorldDocument implements ProjectDocument<DocumentNode> {
setComponent(world, eid, c.Geometry, GeometryType.RECT);
addComponent(world, eid, c.Scene);
addComponent(world, eid, c.ClipsContent);
+ setComponent(world, eid, c.Playback, {});
setComponent(world, eid, c.Name, getNextName(world, "Scene"));
return node;
} case "group": {
diff --git a/packages/jsx/src/document.ts b/packages/jsx/src/document.ts
index 620ec306..6768ca52 100644
--- a/packages/jsx/src/document.ts
+++ b/packages/jsx/src/document.ts
@@ -10,6 +10,23 @@
* opaque to the renderer.
*/
+/**
+ * One reading of the host's timeline clock, local to the mounted project:
+ * the playhead of the scene the mount's root lives in (or is). Values only
+ * move for live mounts; the host pushes a fresh reading once per playback
+ * tick.
+ */
+export type ProjectTick = {
+ /** Playhead in seconds (sub-frame precision while playing). */
+ time: number;
+ /** Playhead in frames at the host frame rate. */
+ frame: number;
+ /** Seconds advanced since the previous tick: 0 while paused, negative on a backward scrub or loop. */
+ delta: number;
+ /** Whether the scene is currently playing. */
+ playing: boolean;
+};
+
export interface ProjectDocument<N = unknown> {
stage: N;
createElement(tag: string): N;
@@ -22,4 +39,5 @@ export interface ProjectDocument<N = unknown> {
getParentNode(node: N): N | undefined;
getFirstChild(node: N): N | undefined;
getNextSibling(node: N): N | undefined;
+ tick?(): ProjectTick;
}
diff --git a/packages/jsx/src/index.ts b/packages/jsx/src/index.ts
index 804b62b5..f7d1cea8 100644
--- a/packages/jsx/src/index.ts
+++ b/packages/jsx/src/index.ts
@@ -15,6 +15,8 @@ export type {
} from "./generate";
export { PATCH_PROP_KEYS } from "./types";
export { parseTime, TIME_FPS } from "./time";
+export { useTicker } from "./renderer";
+export type { Ticker } from "./renderer";
export type {
Animatable,
AnimationSpec,
@@ -59,4 +61,4 @@ export {
renderProject,
} from "./renderer";
-export type { ProjectDocument } from "./document";
+export type { ProjectDocument, ProjectTick } from "./document";
diff --git a/packages/jsx/src/renderer.ts b/packages/jsx/src/renderer.ts
index 7117005f..6d52721b 100644
--- a/packages/jsx/src/renderer.ts
+++ b/packages/jsx/src/renderer.ts
@@ -2,10 +2,10 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
-import { createContext, useContext } from "solid-js";
+import { createContext, createMemo, useContext } from "solid-js";
import { createRenderer } from "solid-js/universal";
-import type { JSX } from "solid-js";
+import type { Accessor, JSX } from "solid-js";
import type { ProjectDocument } from "./document";
// Compiled project modules call the static runtime exports below
@@ -71,6 +71,35 @@ export const {
use,
} = renderer;
+export type Ticker = {
+ time: Accessor<number>;
+ frame: Accessor<number>;
+ delta: Accessor<number>;
+ playing: Accessor<boolean>;
+};
+
+/**
+ * Subscribes to the project's timeline: the playhead of the scene the mount's
+ * root lives in (or is), pushed by the host once per playback tick. Each
+ * accessor only propagates when its value changes, so a paused scene re-runs
+ * nothing and `frame()` consumers update at most once per frame. Values only
+ * move while the reactive graph is alive, i.e. under `dapi mount --live`.
+ */
+export function useTicker(): Ticker {
+ const document = doc();
+ if (!document.tick) {
+ throw new Error("useTicker: this host does not provide a timeline clock");
+ }
+
+ const tick = document.tick.bind(document);
+ return {
+ time: createMemo(() => tick().time),
+ frame: createMemo(() => tick().frame),
+ delta: createMemo(() => tick().delta),
+ playing: createMemo(() => tick().playing),
+ };
+}
+
/**
* Host entry point: renders a project component directly into `document`.
* Mounting is synchronous — the document is fully written when this returns.