The VS Code of video editing, built for agents.
TypeScript
Add mask support to `<rect>` and enhance entity management
konstantin-paulusc88f7ff25fe8
diff --git a/packages/jsx/src/types.ts b/packages/jsx/src/types.tsindex 4b711332..86f07daa 100644--- a/packages/jsx/src/types.ts+++ b/packages/jsx/src/types.ts@@ -384,6 +384,17 @@ export type GroupProps = CommonProps & FillProps & {};export type RectProps = CommonProps & FillProps & {+ /**+ * Makes the rect a mask of its parent: it clips the parent (its fills,+ * strokes and children show only inside the rect's box) instead of drawing.+ * The rect keeps its transform, `cornerRadius` and timing — a keyframed+ * mask sliding across a text is a wipe, one that ends early lets go — and+ * several masks under one parent intersect. A mask is never rendered or+ * hit, so its `fill`, `opacity`, `blendMode` and paint children have no+ * effect. Without `width`/`height` a mask is 500×500, and without `end` it+ * clips for the parent's whole window.+ */+ mask?: boolean;/*** Paint children (`<SolidPaint>`, `<LinearGradientPaint>`,* `<RadialGradientPaint>`), plus `<Stroke>`, `<Shadow>`, `<Effect>`,diff --git a/packages/reconciler/src/document.ts b/packages/reconciler/src/document.tsindex e603022a..0cb41b71 100644--- a/packages/reconciler/src/document.ts+++ b/packages/reconciler/src/document.ts@@ -35,6 +35,7 @@ import {getParentEntity,getParentNode,Hidden,+ IsMask,isText,ItemIndex,Keyframe,@@ -767,6 +768,14 @@ export class RuntimeDocument implements ProjectDocument<HostNode> {entity.set(BlendMode, { value: mode });return;}+ case 'mask': {+ if (value === true && entity !== this.stage.entity) {+ entity.add(IsMask);+ } else {+ entity.remove(IsMask);+ }+ return;+ }case 'hidden': {if (value === true && entity !== this.stage.entity) {entity.add(Hidden);diff --git a/packages/runtime/MIGRATION.md b/packages/runtime/MIGRATION.mdindex c3d3cb8a..f2ef18b5 100644--- a/packages/runtime/MIGRATION.md+++ b/packages/runtime/MIGRATION.md@@ -159,6 +159,8 @@ apps/web now depends on `@diffusionstudio/runtime` and `@diffusionstudio/koota-s**Text is fully hosted: the remaining `TextStyle` fields and `<textRange>`.** `TextProps` gained `letterSpacing` (px, `TextStyle.letterSpacing`), `textCase` (`original`/`upper`/`lower`, the new `TextCase` type in `@diffusionstudio/jsx`, mapped onto the runtime enum; the glyphs are cased when drawn, `Chars` stays as written), `leading` (line height as a multiple of the natural line height, `TextStyle.leading`; non-positive/unset removes it), `textBaseline="alphabetic"` (`TextBaseline.ALPHABETIC`, which `shapeTokens` already laid out) and `fontStyle="oblique"` (was mapped, not typed). The font fields (`fontFamily`/`fontSize`/`fontWeight`/`fontStyle`/`letterSpacing`/`textCase`) moved into a shared `FontProps` group because `<textRange>` takes the same ones. `<textRange start end>` (`TextRangeProps`, `textRange` in `IntrinsicElements` and `COMPOSITION_TAGS`, `TextRange` component in the reconciler) is the element form of the runtime's `TextRange` sub-entity — a style override over a run of the parent `<text>`'s glyphs, addressed by character index into the text as written (`end` unset is `null`, to the end), stacking in document order with later ones winning where they overlap, exactly what the caption presets build by hand. Its recipe is `TextRange` + a sparse `TextStyle` (an unset field inherits the text's, which is how `utils/text.ts`'s per-range accessors already read it), plus `Color` when `color` is set, and it takes paint, `<stroke>`, `<shadow>` and `<keyframeTrack>` (a `color` track) children, whose caches `rebuildCaches` already builds per parent. `setProperty('start'|'end')` branches on `TextRange` first, since on every other element those names are timing props. One runtime change: `renderTokens` resolved the intrinsic glyph fill once per text (`entity.has(Color)`), so a range could recolor glyphs only through a solid paint child; it now resolves it per word through `getIntrinsicColor`, the last range in the word's ranges carrying `Color` winning over the text's (via `Computed.color`, so a keyframed range color animates), which is what makes `<textRange color>` a plain prop. Layout props (`textAlign`, `textBaseline`, `leading`) stay per text: the shaper reads them from the node only. Not exposed: the motion system's `chars` keyframe path (a `Keyframe.value` is a number, so nothing meaningful can be authored on it) and `Opacity` on a range (nothing reads it; a range's translucency is its paints').+ **`mask` is a `<rect>` prop** (`RectProps.mask?: boolean` in `@diffusionstudio/jsx`; no new tag). In the engine a mask is nothing but a geometry with the `IsMask` tag, `ChildOf` the node it clips: `rebuildCaches` files it under `Cache.masks` instead of `Cache.children`, `renderNode` skips drawing it and uses `drawRectPath` of it as a `clip()` (several intersect), hit-testing and the timeline index treat it apart, and the transform, visibility and motion systems see an ordinary geometry — so a keyframed mask `width` is a wipe with nothing new. The prop mirrors that one to one: `setProperty('mask')` adds/removes `IsMask` on the rect's entity, and every other prop is the rect's (`fill`/`opacity`/`blendMode`/paint children are accepted by the type but do nothing on a mask, as they do nothing in the engine). A tag (`<mask>`) was tried first and dropped: the recipe was `<rect>` minus a default paint, and a tag would have made "make this rect a mask" a retag instead of a prop flip. Host behaviour: a `<rect mask>` with no `width`/`height` is 500×500 instead of the rect's 100×100 (`setProperty('mask')` resizes only the dimensions not yet authored, so prop order does not matter); sizing to the parent, which the old editor's "Add mask" did, was tried and dropped for a plain default. Runtime changes: `recomputeEntityTimeRange` gives a mask with no `End`/`SourceOut` of its own its parent's `Computed.end` instead of the sourceless 16 s default (a mask silently letting go at 16 s on a 30 s clip is a surprise nobody authored; `Ignorable` gained `IsMask` so a removal can recompute as if the tag were gone), `bubbleTimeRangeUp` re-propagates into a fitted container's masks after recomputing it (they are not in `Cache.children`, so they would otherwise keep the stale end), the transform system's group-bounds union skips `IsMask` children (a mask clips a group's box, it does not extend it), and `world/observers.ts` watches `IsMask` add/remove on an attached geometry — `refileMask` (new in `actions/cache.ts`) moves it between the parent's `children` and `masks` by hand, since koota fires onRemove with the tag still on and the queries cannot be trusted, then the time range is re-derived and bubbled — so `mask` toggles live like any other prop. Not exposed: `hidden` on a mask does not switch it off (`renderNode` skips masks by role before it looks at `Hidden`, and `computed.visibility` is timing only); text-shaped masks are not a runtime feature either (`drawRectPath` is the only mask path).+**Left stale on purpose** (deferred to a docs/examples pass): all eight `examples/*.tsx` still open with `<rect scene="...">` and no `<stage>`, so `tsc -p examples --noEmit` fails on them; `reference/jsx/{scene,roots}.md` still describes the pre-`<stage>` model where a mount grafts roots into an existing document; and `reference/jsx/{audio-sync,video,audio,elements}.md` still document `key` as the `syncTo` target. Deliberately not half-migrated: `reference/jsx` is stale as a whole, and patching the `key` rows alone would hide that.## Remaining work (suggested order)diff --git a/packages/runtime/src/actions/cache.ts b/packages/runtime/src/actions/cache.tsindex a0ce2ab5..e51bb10c 100644--- a/packages/runtime/src/actions/cache.ts+++ b/packages/runtime/src/actions/cache.ts@@ -97,6 +97,28 @@ export function rebuildCaches(world: World, entity: Entity, parent: Entity | nul}}+/**+ * Move an attached geometry between its parent's `children` and `masks` when+ * its `IsMask` tag is toggled in place. `rebuildCaches` files by trait on+ * attach; here the queries cannot be trusted (koota fires onRemove before the+ * tag is cleared), so the two lists are edited by hand.+ */+export function refileMask(world: World, entity: Entity, parent: Entity | null, mask: boolean): void {+ if (parent === null || isStage(parent)) return;+ if (!parent.has(Cache)) parent.add(Cache);++ const cache = store(world, Cache);+ const pid = parent.id();+ const without = (list: Entity[] | undefined) => (list ?? []).filter((e) => e !== entity);++ const masks = without(cache.masks[pid]);+ const children = without(cache.children[pid]);+ (mask ? masks : children).push(entity);++ cache.masks[pid] = masks.sort(sortByItemIndex);+ cache.children[pid] = children.sort(sortByItemIndex);+}+function isNodeEntity(entity: Entity): boolean {return entity.has(Geometry) || entity.has(Group) || entity.has(AdjustmentLayer);}diff --git a/packages/runtime/src/actions/timing.ts b/packages/runtime/src/actions/timing.tsindex 81fa6490..4924be7d 100644--- a/packages/runtime/src/actions/timing.ts+++ b/packages/runtime/src/actions/timing.ts@@ -9,7 +9,7 @@import { store } from '../world/store';import { DEFAULT_DURATION_FRAMES } from '../constants';import {- Geometry, Group, Paint, Caption, Cache, Computed, AssetId,+ Geometry, Group, Paint, Caption, Cache, Computed, AssetId, IsMask,Start, End, SourceIn, SourceOut, PlaybackRate, CaptionDecoderHandle,} from '../traits';import { isGroupLike } from '../queries/predicates';@@ -28,7 +28,7 @@ type TimeTrait = typeof Start;* A trait a recompute may read as absent: one of the authored time traits, or* the node's own AssetId (its intrinsic media, another source of a length).*/-type Ignorable = TimeTrait | typeof AssetId;+type Ignorable = TimeTrait | typeof AssetId | typeof IsMask;/*** The authored value, or undefined when the node doesn't carry it. `ignore`diff --git a/packages/runtime/src/systems/transform.ts b/packages/runtime/src/systems/transform.tsindex fe819451..bda0bfb0 100644--- a/packages/runtime/src/systems/transform.ts+++ b/packages/runtime/src/systems/transform.ts@@ -6,7 +6,7 @@ import { Not, Or } from 'koota';import { store } from '../world/store';import {- ChildOf, Geometry, Group, Hidden, Sequential, AdjustmentLayer,+ ChildOf, Geometry, Group, Hidden, IsMask, Sequential, AdjustmentLayer,Culled, Flip, Anchor, Computed, Cache, LocalTransform, WorldTransform,WorldBounds, RenderSurface,Root,@@ -209,7 +209,7 @@ export function computeGroupBounds(world: World, entity: Entity): void {let maxY = -Infinity;let any = false;- for (const child of world.query(Or(Geometry, Group), ChildOf(entity), Not(Hidden))) {+ for (const child of world.query(Or(Geometry, Group), ChildOf(entity), Not(Hidden), Not(IsMask))) {const cid = child.id();const mat: Mat2D = {a: localStore.a[cid] ?? 1, b: localStore.b[cid] ?? 0,diff --git a/packages/runtime/src/world/observers.ts b/packages/runtime/src/world/observers.tsindex 33c8993c..171ef65b 100644--- a/packages/runtime/src/world/observers.ts+++ b/packages/runtime/src/world/observers.ts@@ -22,13 +22,13 @@ import {Start, End, SourceIn, SourceOut, PlaybackRate, Keyframe, ItemIndex,Position, Offset, Rotation, Scale, UniformScale, Skew, Anchor, Flip,Opacity, Color, Blur, Volume, Effect, CornerRadius, MixedCornerRadius,- ColorStop, StrokeStyle, Size, Computed, Active, Stage,+ ColorStop, StrokeStyle, Size, Computed, Active, Stage, IsMask,ImageDecoderHandle, VideoDecoderHandle, SequenceDecoderHandle,AudioDecoderHandle, CaptionDecoderHandle,HtmlHostHandle, SurfaceHostHandle, ShaderHostHandle, AudioBusHandle,} from '../traits';import { getParentEntity } from '../queries/hierarchy';-import { rebuildCaches } from '../actions/cache';+import { rebuildCaches, refileMask } from '../actions/cache';import { disposeDecoders, disconnectAudioBus } from '../media/dispose';import {reactToChildAttached, reactToChildDetached, reactToAssetChange,@@ -105,6 +105,14 @@ export function observeWorld(world: World): () => void {entity.remove(Position, Offset, Rotation, Scale, Skew, Anchor, Flip);}));+ const refile = (mask: boolean) => (entity: Entity) => {+ const parent = getParentEntity(entity);+ if (parent === null || !world.has(parent)) return;+ refileMask(world, entity, parent, mask);+ };+ subs.push(world.onAdd(IsMask, refile(true)));+ subs.push(world.onRemove(IsMask, refile(false)));+// One active entity per world, and only a root.subs.push(world.onAdd(Active, (entity) => {for (const other of [...world.query(Active)]) {diff --git a/reference/jsx/elements.md b/reference/jsx/elements.mdindex 86ea944f..ae8a294c 100644--- a/reference/jsx/elements.md+++ b/reference/jsx/elements.md@@ -5,7 +5,7 @@ camelCase composition elements map 1:1 onto internal node types. Lowercase DOM v| Element | Internal node | Notes || ------- | ------------- | ----- || [`<group>`](./group.md) | **Container with Group tag** | Container with a transform; auto-fits its size from its children. Takes no `fill` or explicit size. |-| [`<rect>`](./rect.md) | **Geometry with Solid paint** | A filled rectangle; takes only paint children. |+| [`<rect>`](./rect.md) | **Geometry with Solid paint** | A filled rectangle; takes only paint children. With `mask` it clips its parent instead of drawing. || [`<video>`](./video.md) | **Geometry with Video paint** | `src` resolves to a video asset. || [`<image>`](./image.md) | **Geometry with Image paint** | `src` resolves to an image asset. || [`<audio>`](./audio.md) | **Geometry with Audio component and a hidden Waveform paint** | No visual output; carries volume. |diff --git a/reference/jsx/rect.md b/reference/jsx/rect.mdindex 919a4014..298d130b 100644--- a/reference/jsx/rect.md+++ b/reference/jsx/rect.md@@ -1,6 +1,6 @@# `<rect>`-A filled rectangle. Takes only [paint children](./paints.md); use `cornerRadius` for rounded corners.+A filled rectangle. Takes [paint children](./paints.md), plus `<stroke>`, `<shadow>`, `<effect>`, `<animation>` and `<keyframeTrack>`; use `cornerRadius` for rounded corners.```tsx<rect x={40} y={40} width={640} height={360} cornerRadius={24} fill="#FF0055" />@@ -13,3 +13,27 @@ All [common props](./elements.md#common-props), plus:| Prop | Type | Default | Meaning || ---- | ---- | ------- | ------- || `fill` | `string` | light gray | Any CSS color; alpha is ignored (use `opacity`). Shorthand for a solid paint child. |+| `mask` | `boolean` | `false` | Makes the rect a mask of its parent instead of a drawn shape — see below. |++## Masks++A `<rect mask>` clips the element holding it instead of drawing: the parent's fills, strokes, shadows and children show only inside the rect's box. It is still a rect — same coordinates, `cornerRadius`, transform and timing — so it can be moved, rotated, keyframed and animated independently of what it clips: a mask whose `width` is keyframed across a text is a wipe, a mask that starts later than its parent is a reveal.++```tsx+<text fontSize={120} color="#FFFFFF" textAlign="center" textBaseline="middle">+ Hello World+ <rect mask>+ <keyframeTrack property="width">+ <keyframe time={0} value={0} />+ <keyframe time={1} value={1920} easing="easeOut" />+ </keyframeTrack>+ </rect>+</text>+```++- Any node can hold masks (`<rect>`, `<text>`, `<video>`, `<image>`, `<group>`, `<scene>`); the mask clips that element as a whole.+- Several masks under one element **intersect**: content shows only where every mask covers it.+- Outside the mask's own time window it does not clip and the parent shows in full. Without `end` (or `sourceOut`) a mask clips for the parent's whole window, not the usual 16 s default.+- Never rendered and never hit on the canvas, so `fill`, `opacity`, `blendMode` and paint / stroke / shadow / effect children have no effect on it.+- Without `width`/`height` a mask is 500×500 (a plain rect is 100×100).+- `hidden` on a mask does not switch it off (it is skipped by role before hiding is looked at); use timing, or set `mask={false}`.