Skip to content

Open source CapCut alternative.

TypeScript

feat(site): elements ride the clip beneath them [rebuild]

David Le92a7548d2d3e
diff --git a/docs/guides/cut/README.md b/docs/guides/cut/README.md
index d6afaa09..56fbbb5d 100644
--- a/docs/guides/cut/README.md
+++ b/docs/guides/cut/README.md
@@ -28,7 +28,7 @@ browser (the editor)
The client probes the engine's health endpoint (dedicated port first, dev server second) and remembers the winner. Browsers permission-gate a hosted page's calls to the local machine, so the page probes only when that raises no prompt — it is served by the engine, or the permission is already granted — and starts in the cloud otherwise. The engine grants the hosted origin cross-origin access, and only that origin. Nothing walls the app, because cloud editing always works. Closing the Donkey app does not hide the work made in it either: a browser that has reached this Mac's engine before keeps listing that shelf from the listing it stored, so those projects stay on the home beside the cloud ones, each badged with where it lives. They are read-only while the app is away — no renames, moves, or deletes — and opening one says that it lives on this Mac rather than guessing at a cloud copy that was never there. That is where the warning belongs, so that is the only place it appears: a banner over the thing that will not open, which the user cannot dismiss and which opens the recovery steps in a dialog with the button that fires the browser's permission prompt. Probing continues behind it, so starting the app clears the banner and loads the project. Engine updates ride the Donkey app's own auto-updater, so the client never prompts to update.
-**A cut needs no footage.** The frame starts as a color the project owns, and everything else — clips, titles, shapes, stickers — draws over it. So a video made entirely of text and graphics is an ordinary project: it plays, exports, shares, and renders on the worker like any other, and the same color letterboxes a fitted clip and fills a gap on track 0. Elements stack by row: two on one row slide clear of each other in time, so a title that sits over a card lives a row above it. Type stays inside the picture on its own: a line wider than the room its anchor leaves is measured against the face it will actually be drawn in and broken onto another line, in the preview and in the export alike, so whoever writes the words — a person, the assistant — picks the size the design wants and lets the frame do the breaking.
+**A cut needs no footage.** The frame starts as a color the project owns, and everything else — clips, titles, shapes, stickers — draws over it. So a video made entirely of text and graphics is an ordinary project: it plays, exports, shares, and renders on the worker like any other, and the same color letterboxes a fitted clip and fills a gap on track 0. Elements stack by row: two on one row slide clear of each other in time, so a title that sits over a card lives a row above it. An element rides the frames of the clip under it: its ends are remembered as that clip's source seconds, so moving the clip carries the element, a speed change stretches it with the picture, a trim leaves it on the frames it covered, a split hands it to the half that plays them, and deleting the clip leaves it in place over whatever is there now. The store's one write path keeps that true for every edit, the assistant's included, and the Inspector frees an element that should stay put. Type stays inside the picture on its own: a line wider than the room its anchor leaves is measured against the face it will actually be drawn in and broken onto another line, in the preview and in the export alike, so whoever writes the words — a person, the assistant — picks the size the design wants and lets the frame do the breaking.
**Framing is per clip.** Every video clip meets its box — the whole frame, or the region it was given — through one geometry: fitted with margins or covering with a crop, zoomed in further, panned to choose what the crop keeps, turned, faded, cornered, masked, and shadowed. One function computes where the picture lands, the preview draws that rectangle, and the export graph scales and crops to the same numbers, so a reframed clip looks the same on every surface, freeze frames and headless renders included. On the stage the selected clip wears two outlines: a blue box for its layout — a drag anywhere on it moves the clip, its grips resize it, the button above turns it — and a grey dashed one standing where the source really lands, so the part a crop discards stays visible: its corners zoom the picture and dragging the discarded part slides the crop. The shadow is cast by the shape the clip actually shows, mask included, which is what makes a circle-masked camera bubble throw a circular shadow.
diff --git a/site/packages/effects-kit/src/types.ts b/site/packages/effects-kit/src/types.ts
index 1e409d8f..d875d11c 100644
--- a/site/packages/effects-kit/src/types.ts
+++ b/site/packages/effects-kit/src/types.ts
@@ -29,6 +29,11 @@ export interface OverlayBase {
hidden?: boolean;
/** Group membership (select/move as one); assigned by the host's grouping. */
groupId?: string;
+ /** The video clip whose frames this element rides: the editor keeps it over
+ * the same footage when that clip moves, retimes or splits, and leaves it
+ * where it is when the clip goes. Absent = the editor picks the clip under
+ * the element whenever it is placed; null = the element stays free. */
+ hostClipId?: string | null;
/** Preset In / Out / Loop animation (see anim.ts); absent = static. */
anim?: OverlayAnim;
/** Keyframed pose track, seconds from the element's start (see keys.ts).
diff --git a/site/src/cut/components/Inspector.tools.ts b/site/src/cut/components/Inspector.tools.ts
index 9957e7e4..239bec10 100644
--- a/site/src/cut/components/Inspector.tools.ts
+++ b/site/src/cut/components/Inspector.tools.ts
@@ -72,6 +72,9 @@ export const INSPECTOR_TOOLS = [
inputSchema: obj({
id: str("Overlay element id"),
text: str("New text (titles)"),
+ follows_clip: bool(
+ "Whether the element rides the clip under it (on by default): true homes it to that clip, false frees it where it is"
+ ),
start: num("Start s"),
end: num("End s"),
x: num("Center x 0..1"),
diff --git a/site/src/cut/components/Inspector.tsx b/site/src/cut/components/Inspector.tsx
index 23c73dc2..f7cf48d4 100644
--- a/site/src/cut/components/Inspector.tsx
+++ b/site/src/cut/components/Inspector.tsx
@@ -2035,6 +2035,7 @@ function TextPanel({ overlay: o }: { overlay: TextOverlay }) {
</Section>
<OverlayMaskSection overlay={o} />
<TransformRows overlay={o} />
+ <FollowRow overlay={o} />
<GroupRow overlay={o} />
<AnimationSection
overlay={o}
@@ -2702,6 +2703,22 @@ function AnimationToolbar({ overlay: o, slot }: { overlay: Overlay; slot: AnimSl
/** Group / Ungroup, shown when the selection can form one or the element is
* in one. Grouping is shallow by design: select-one-selects-all, and gestures
* apply member-relative deltas. */
+/** Whether the element rides the clip under it. On, the store homes it to
+ * that clip and carries it with the clip's frames; off, it stays free. */
+function FollowRow({ overlay: o }: { overlay: Overlay }) {
+ return (
+ <Row
+ label="Follows clip"
+ info="The element moves and retimes with the frames of the video clip under it. Off keeps it where it is whatever the clip does."
+ >
+ <Switch
+ checked={o.hostClipId !== null}
+ onCheckedChange={(v) => useEditor.getState().setOverlayFollows(o.id, v)}
+ />
+ </Row>
+ );
+}
+
function GroupRow({ overlay: o }: { overlay: Overlay }) {
const selectedOverlays = useEditor(
(s) => s.multiSelection.filter((x) => x?.kind === "overlay").length
@@ -3829,6 +3846,7 @@ function ShapePanel({ overlay: o }: { overlay: ShapeOverlay }) {
)}
<OverlayMaskSection overlay={o} />
<TransformRows overlay={o} />
+ <FollowRow overlay={o} />
<GroupRow overlay={o} />
<AnimationSection
overlay={o}
@@ -3876,6 +3894,7 @@ function EffectPanel({ overlay: o }: { overlay: EffectOverlay }) {
</Row>
)}
<HiddenRow overlay={o} />
+ <FollowRow overlay={o} />
<GroupRow overlay={o} />
</div>
</>
@@ -4055,6 +4074,7 @@ function StickerPanel({ overlay: o }: { overlay: StickerOverlay }) {
</Row>
<OverlayMaskSection overlay={o} />
<TransformRows overlay={o} />
+ <FollowRow overlay={o} />
<GroupRow overlay={o} />
<AnimationSection
overlay={o}
diff --git a/site/src/cut/lib/aiContext.ts b/site/src/cut/lib/aiContext.ts
index 89ca43b2..ff3f3ea4 100644
--- a/site/src/cut/lib/aiContext.ts
+++ b/site/src/cut/lib/aiContext.ts
@@ -538,6 +538,7 @@ function describeOverlay(o: Overlay) {
...(o.mask ? { mask: o.mask } : {}),
...(o.lane ? { lane: o.lane } : {}),
...(o.hidden ? { hidden: true } : {}),
+ ...(o.hostClipId ? { host: o.hostClipId } : o.hostClipId === null ? { free: true } : {}),
};
if (o.kind === "shape") {
return {
diff --git a/site/src/cut/lib/aiTools.ts b/site/src/cut/lib/aiTools.ts
index 2e7b2602..51c37a21 100644
--- a/site/src/cut/lib/aiTools.ts
+++ b/site/src/cut/lib/aiTools.ts
@@ -2054,6 +2054,7 @@ const toolRuns: Record<BrowserToolName, ToolRun> = {
update_overlay: (s, input) => {
const o = requireItem(s.overlays, input.id, "overlay element");
s.updateOverlay(o.id, overlayPatch(input, o.kind ?? "text"));
+ if (typeof input.follows_clip === "boolean") s.setOverlayFollows(o.id, input.follows_clip);
const next = useEditor.getState().overlays.find((x) => x.id === o.id)!;
return {
id: next.id,
@@ -2061,6 +2062,7 @@ const toolRuns: Record<BrowserToolName, ToolRun> = {
...(isTextOverlay(next) ? { text: next.text, color: next.color, size: next.size } : {}),
start: round2(next.start),
end: round2(next.end),
+ ...(next.hostClipId ? { host: next.hostClipId } : {}),
};
},
diff --git a/site/src/cut/lib/overlayHosts.test.ts b/site/src/cut/lib/overlayHosts.test.ts
new file mode 100644
index 00000000..99bbafab
--- /dev/null
+++ b/site/src/cut/lib/overlayHosts.test.ts
@@ -0,0 +1,88 @@
+import { describe, expect, test } from "bun:test";
+import type { Overlay } from "@donkeycut/effects-kit";
+import type { VideoClip } from "./types";
+import { adoptOverlayHosts, hostClipFor, settleOverlayHosts } from "./overlayHosts";
+
+const clip = (p: Partial<VideoClip> & { id: string }): VideoClip =>
+ ({ track: 0, assetId: "a", start: 0, in: 0, out: 10, muted: false, ...p }) as VideoClip;
+const title = (p: Partial<Overlay> & { id: string }): Overlay =>
+ ({ kind: "text", text: "hi", x: 0.5, y: 0.5, start: 2, end: 4, ...p }) as Overlay;
+
+const c = clip({ id: "c1", start: 10, in: 0, out: 10 });
+const t = title({ id: "t1", start: 12, end: 14, hostClipId: "c1" });
+
+describe("element hosts", () => {
+ test("a placed element homes to the clip under its middle, highest track first", () => {
+ const low = clip({ id: "lo", start: 0, out: 20, track: 0 });
+ const high = clip({ id: "hi", start: 5, out: 10, track: 1 });
+ expect(hostClipFor([low, high], { start: 6, end: 8 })?.id).toBe("hi");
+ expect(hostClipFor([low, high], { start: 1, end: 3 })?.id).toBe("lo");
+ expect(hostClipFor([low, high], { start: 30, end: 32 })).toBeUndefined();
+ });
+
+ test("a moved host carries its element by the same distance", () => {
+ const moved = { ...c, start: 40 };
+ const r = settleOverlayHosts([c], [moved], [t], [t])!;
+ expect(r.overlays[0].start).toBe(42);
+ expect(r.overlays[0].end).toBe(44);
+ expect(r.carried.has("t1")).toBe(true);
+ });
+
+ test("a retimed host stretches the element with the picture, keys included", () => {
+ const keyed = { ...t, kf: [{ t: 1, x: 0.2, y: 0.2 }] } as Overlay;
+ const slow = { ...c, speed: 0.5 };
+ const r = settleOverlayHosts([c], [slow], [keyed], [keyed])!;
+ expect(r.overlays[0].start).toBeCloseTo(14, 6);
+ expect(r.overlays[0].end).toBeCloseTo(18, 6);
+ expect(r.overlays[0].kf![0].t).toBeCloseTo(2, 6);
+ });
+
+ test("a head trim leaves the element on the frames it covered", () => {
+ const trimmed = { ...c, start: 11, in: 1 };
+ expect(settleOverlayHosts([c], [trimmed], [t], [t])).toBeNull();
+ });
+
+ test("a split hands the element to the half that plays its frames", () => {
+ const left = { ...c, out: 1.5 };
+ const right = clip({ id: "c2", start: 11.5, in: 1.5, out: 10 });
+ const r = settleOverlayHosts([c], [left, right], [t], [t])!;
+ expect(r.overlays[0].start).toBe(12);
+ expect(r.overlays[0].hostClipId).toBe("c2");
+ });
+
+ test("a ripple that already moved the element with its footage changes nothing", () => {
+ const moved = { ...c, start: 15 };
+ const rippled = { ...t, start: 17, end: 19 };
+ expect(settleOverlayHosts([c], [moved], [t], [rippled])).toBeNull();
+ });
+
+ test("an element dragged by hand homes where it lands", () => {
+ const other = clip({ id: "c2", start: 30, out: 10 });
+ const dragged = { ...t, start: 33, end: 35 };
+ const r = settleOverlayHosts([c, other], [c, other], [t], [dragged])!;
+ expect(r.overlays[0].hostClipId).toBe("c2");
+ });
+
+ test("a deleted host leaves the element in place, homed to what is there now", () => {
+ const next = clip({ id: "c2", start: 10, out: 10 });
+ const r = settleOverlayHosts([c], [next], [t], [t])!;
+ expect(r.overlays[0].start).toBe(12);
+ expect(r.overlays[0].hostClipId).toBe("c2");
+ const gone = settleOverlayHosts([c], [], [t], [t])!;
+ expect(gone.overlays[0].hostClipId).toBeUndefined();
+ });
+
+ test("a freed element stays free through every write", () => {
+ const free = { ...t, hostClipId: null } as Overlay;
+ expect(settleOverlayHosts([c], [{ ...c, start: 40 }], [free], [free])).toBeNull();
+ const dragged = { ...free, start: 20, end: 22 };
+ expect(settleOverlayHosts([c], [c], [free], [dragged])).toBeNull();
+ });
+
+ test("an older document's elements adopt the clips under them on load", () => {
+ const bare = title({ id: "t2", start: 12, end: 14 });
+ expect(adoptOverlayHosts([c], [bare])[0].hostClipId).toBe("c1");
+ const off = title({ id: "t3", start: 50, end: 52 });
+ expect(adoptOverlayHosts([c], [off])[0].hostClipId).toBeUndefined();
+ });
+});
diff --git a/site/src/cut/lib/overlayHosts.ts b/site/src/cut/lib/overlayHosts.ts
new file mode 100644
index 00000000..2382e718
--- /dev/null
+++ b/site/src/cut/lib/overlayHosts.ts
@@ -0,0 +1,139 @@
+import { retimeOf, type Overlay } from "@donkeycut/effects-kit";
+import type { VideoClip } from "./types";
+
+// An element rides the frames of the clip under it. Its start and end are
+// remembered as source seconds of that clip, so wherever those frames go —
+// the clip moved, retimed, trimmed, split — the element goes over the same
+// footage: a move carries it, a speed change stretches it with the picture,
+// a trim leaves it on the frames it covered, and a split re-homes it to the
+// half that now plays them. The store's write wrapper runs this on every
+// write, transient drag frames included, so no edit path can leave a hosted
+// element behind.
+
+const EPS = 1e-6;
+const NEAR = 1e-3;
+const MIN_LEN = 0.1;
+
+const spanEnd = (c: VideoClip) => c.start + retimeOf(c).len;
+
+/** The clip whose frames sit under the element's middle; the highest track
+ * wins, since that is the picture the element is drawn over. */
+export function hostClipFor(
+ clips: readonly VideoClip[],
+ o: Pick<Overlay, "start" | "end">
+): VideoClip | undefined {
+ const mid = (o.start + o.end) / 2;
+ let best: VideoClip | undefined;
+ for (const c of clips) {
+ if (mid < c.start - EPS || mid >= spanEnd(c) + EPS) continue;
+ if (!best || c.track > best.track) best = c;
+ }
+ return best;
+}
+
+const spanChanged = (a: VideoClip, b: VideoClip) =>
+ a.start !== b.start ||
+ a.in !== b.in ||
+ a.out !== b.out ||
+ a.speed !== b.speed ||
+ a.speedCurve !== b.speedCurve;
+
+/** The element over the same source seconds of `host` that it covered of
+ * `was`. The curve maps linearly past either end, so an element leading into
+ * or trailing out of the clip keeps that lead. A stretched element scales its
+ * keyed poses with it, so a key stays on the frame it was set on. */
+function ride(o: Overlay, was: VideoClip, host: VideoClip): Overlay {
+ const from = retimeOf(was);
+ const to = retimeOf(host);
+ let start = host.start + to.tAt(from.srcAt(o.start - was.start));
+ let end = host.start + to.tAt(from.srcAt(o.end - was.start));
+ if (end - start < MIN_LEN) end = start + MIN_LEN;
+ if (start < 0) {
+ end -= start;
+ start = 0;
+ }
+ const scale = (end - start) / Math.max(EPS, o.end - o.start);
+ const next: Overlay = { ...o, start, end };
+ if (Math.abs(scale - 1) > EPS) {
+ if (o.kf?.length) next.kf = o.kf.map((k) => ({ ...k, t: k.t * scale }));
+ if (o.mask?.kf?.length)
+ next.mask = { ...o.mask, kf: o.mask.kf.map((k) => ({ ...k, t: k.t * scale })) };
+ }
+ return next;
+}
+
+const near = (a: Pick<Overlay, "start" | "end">, b: Pick<Overlay, "start" | "end">) =>
+ Math.abs(a.start - b.start) < NEAR && Math.abs(a.end - b.end) < NEAR;
+
+/** Every element after a write: hosted ones carried with their host's frames,
+ * placed ones and ones whose host went re-homed to the clip under them.
+ * Null when the write changes nothing here. `carried` names the elements
+ * this moved, for the caller to part the others clear of. */
+export function settleOverlayHosts(
+ prevClips: readonly VideoClip[],
+ nextClips: readonly VideoClip[],
+ prevOverlays: readonly Overlay[],
+ nextOverlays: readonly Overlay[]
+): { overlays: Overlay[]; carried: Set<string> } | null {
+ if (prevClips === nextClips && prevOverlays === nextOverlays) return null;
+ const prevClip = new Map(prevClips.map((c) => [c.id, c]));
+ const nextClip = new Map(nextClips.map((c) => [c.id, c]));
+ const prevOv = prevOverlays === nextOverlays ? null : new Map(prevOverlays.map((o) => [o.id, o]));
+ const carried = new Set<string>();
+ let changed = false;
+ const overlays = nextOverlays.map((o) => {
+ if (o.hostClipId === null) return o;
+ const before = prevOv ? prevOv.get(o.id) : o;
+ // The write positioned this element itself: it was created, or its span moved.
+ const placed = !before || before.start !== o.start || before.end !== o.end;
+ const host = o.hostClipId ? nextClip.get(o.hostClipId) : undefined;
+ if (host && before) {
+ const was = prevClip.get(host.id);
+ if (was && spanChanged(was, host)) {
+ const expected = ride(before, was, host);
+ if (!placed) {
+ // Its frames stayed put through a trim: nothing to carry.
+ const landed = near(expected, before) ? o : expected;
+ if (landed !== o) {
+ carried.add(o.id);
+ changed = true;
+ }
+ const mid = (landed.start + landed.end) / 2;
+ if (mid >= host.start - EPS && mid < spanEnd(host) + EPS) return landed;
+ // Its frames left the host — cut away, or split into another clip —
+ // so it homes to the clip that plays them now.
+ const id = hostClipFor(nextClips, landed)?.id;
+ if (id === landed.hostClipId) return landed;
+ changed = true;
+ return { ...landed, hostClipId: id };
+ }
+ // A ripple moved the element with its footage already.
+ if (near(expected, o)) return o;
+ // Placed by hand while its host changed: it homes where it landed.
+ } else if (!placed) {
+ return o;
+ }
+ } else if (!placed && !o.hostClipId) {
+ return o;
+ }
+ const id = hostClipFor(nextClips, o)?.id;
+ if (id === o.hostClipId || (id === undefined && o.hostClipId === undefined)) return o;
+ changed = true;
+ return { ...o, hostClipId: id };
+ });
+ return changed ? { overlays, carried } : null;
+}
+
+/** A loaded document's elements homed to the clips under them, for docs from
+ * before elements had hosts. Elements that already say stay as they say. */
+export function adoptOverlayHosts(clips: readonly VideoClip[], overlays: readonly Overlay[]): Overlay[] {
+ let changed = false;
+ const next = overlays.map((o) => {
+ if (o.hostClipId !== undefined) return o;
+ const id = hostClipFor(clips, o)?.id;
+ if (!id) return o;
+ changed = true;
+ return { ...o, hostClipId: id };
+ });
+ return changed ? next : (overlays as Overlay[]);
+}
diff --git a/site/src/cut/lib/store.ts b/site/src/cut/lib/store.ts
index 1f5826f6..90d41477 100644
--- a/site/src/cut/lib/store.ts
+++ b/site/src/cut/lib/store.ts
@@ -48,6 +48,7 @@ import type {
VideoClip,
} from "./types";
import type { VideoProject } from "./genvideo/types";
+import { adoptOverlayHosts, hostClipFor, settleOverlayHosts } from "./overlayHosts";
import { fillSlot } from "./genvideo/fillSlot";
import { apiFetch, apiJson, getBackend, hasLocalCompute } from "./backend";
import { fetchSignedMediaUrls, pinDocBase } from "./backend/cloud";
@@ -726,6 +727,9 @@ export interface EditorState {
groupSelectedOverlays: () => void;
/** Dissolve a group; the members stay, ungrouped. */
ungroupOverlays: (groupId: string) => void;
+ /** Whether an element rides the clip under it: on homes it to that clip,
+ * off frees it where it is. */
+ setOverlayFollows: (id: string, on: boolean) => void;
/** ⌘/⇧-click: add the item to the selection (or remove it if already in),
* making it the new primary. */
toggleSelect: (sel: NonNullable<Selection>) => void;
@@ -974,6 +978,13 @@ export function moveOverlayGroup(anchor: Overlay, delta: number) {
* elements being pushed keep their own order and spacing. */
function pushOverlaysClearOf(ids: Set<string>) {
const overlays = useEditor.getState().overlays;
+ const cleared = overlaysClearOf(overlays, ids);
+ if (cleared !== overlays) useEditor.setState({ overlays: cleared });
+}
+
+/** `overlays` with everything outside `ids` slid clear of the set, as a new
+ * array; the same array back when nothing had to move. */
+function overlaysClearOf(overlays: Overlay[], ids: Set<string>): Overlay[] {
const moved = overlays.filter((o) => ids.has(o.id));
const shifted = new Map<string, number>();
for (const lane of new Set(moved.map((o) => o.lane ?? 0))) {
@@ -994,13 +1005,11 @@ function pushOverlaysClearOf(ids: Set<string>) {
floor = start + len;
}
}
- if (shifted.size === 0) return;
- useEditor.setState((s) => ({
- overlays: s.overlays.map((o) => {
- const by = shifted.get(o.id);
- return by ? { ...o, start: o.start + by, end: o.end + by } : o;
- }),
- }));
+ if (shifted.size === 0) return overlays;
+ return overlays.map((o) => {
+ const by = shifted.get(o.id);
+ return by ? { ...o, start: o.start + by, end: o.end + by } : o;
+ });
}
/** Resize a clip's footprint to `newLen` (a trim or speed change) and carry
@@ -1235,6 +1244,9 @@ const DOC_KEYS = [
"renders",
] as const;
let hydrating = false;
+// An undo restore writes a document that was consistent when it was taken;
+// homing elements against it would only re-derive what it already says.
+let restoring = false;
/**
* The invariants every write to the store passes through, wherever it comes
@@ -1260,6 +1272,26 @@ function normalizeWrite(prev: EditorState, incoming: Partial<EditorState>): Part
const derived = deriveTransitionFields(clips, next.transitions ?? prev.transitions);
if (derived !== clips) next = { ...next, clips: derived };
}
+ // Elements ride the frames under them: a write that moved or retimed
+ // clips carries every hosted element over the same footage, and one that
+ // placed an element homes it to the clip beneath. Carried elements keep
+ // their exact place; the rest of their row parts clear of them.
+ if (!hydrating && !restoring && (next.clips || next.overlays)) {
+ const settled = settleOverlayHosts(
+ prev.clips,
+ next.clips ?? prev.clips,
+ prev.overlays,
+ next.overlays ?? prev.overlays
+ );
+ if (settled) {
+ next = {
+ ...next,
+ overlays: settled.carried.size
+ ? overlaysClearOf(settled.overlays, settled.carried)
+ : settled.overlays,
+ };
+ }
+ }
// The playhead cannot outlive the timeline. Deleting the last of a long
// row shortens the project under a playhead standing past the new end,
// which leaves the readout ahead of the total and — since the playhead is
@@ -1454,14 +1486,19 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {
}
};
const nextMulti = multiSelection.filter(survives);
- set({
- ...doc,
- clips: nextClips,
- audioClips: nextAudio,
- ...(beatsMoved ? { assets: withBeats } : {}),
- selection: survives(selection) ? selection : nextMulti[nextMulti.length - 1] ?? null,
- multiSelection: nextMulti,
- });
+ restoring = true;
+ try {
+ set({
+ ...doc,
+ clips: nextClips,
+ audioClips: nextAudio,
+ ...(beatsMoved ? { assets: withBeats } : {}),
+ selection: survives(selection) ? selection : nextMulti[nextMulti.length - 1] ?? null,
+ multiSelection: nextMulti,
+ });
+ } finally {
+ restoring = false;
+ }
};
/** Seal the deferred checkpoint: commit it to history only if the doc
@@ -2005,7 +2042,8 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {
sanitizeTransitions(doc.transitions)
),
audioClips: merged.audioClips,
- overlays: withLooks.overlays,
+ // Elements from before they had hosts home to the clips under them.
+ overlays: adoptOverlayHosts(withLooks.clips, withLooks.overlays),
templates: doc.templates ?? [],
mediaFolders: doc.mediaFolders ?? [],
aspect: normalizeAspect(doc.aspect) ?? lastChosenAspect() ?? "9:16",
@@ -3904,6 +3942,8 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {
start: o.start + shift,
end: o.end + shift,
...regroup(o),
+ // A template's element homes to whatever sits under where it lands.
+ ...(typeof o.hostClipId === "string" ? { hostClipId: undefined } : {}),
}));
const newCues: SubtitleCue[] = template.cues.map((c) => ({
...c,
@@ -3950,6 +3990,15 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {
get().updateOverlaysTransient(ids.map((id) => ({ id, patch: { groupId } })));
},
+ setOverlayFollows: (id, on) => {
+ if (!get().overlays.some((o) => o.id === id)) return;
+ push();
+ set((s) => ({
+ overlays: s.overlays.map((o) =>
+ o.id === id ? { ...o, hostClipId: on ? hostClipFor(s.clips, o)?.id : null } : o
+ ),
+ }));
+ },
ungroupOverlays: (groupId) => {
const members = get().overlays.filter((o) => o.groupId === groupId);
if (members.length === 0) return;
@@ -4866,6 +4915,8 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {
start,
end: start + len,
...regroup(cb.item),
+ // A copy homes to whatever sits under where it lands.
+ ...(typeof cb.item.hostClipId === "string" ? { hostClipId: undefined } : {}),
};
overlays = [...overlays, item];
newSel.push({ kind: "overlay", id: item.id });
diff --git a/site/src/cut/server/ai/catalog.ts b/site/src/cut/server/ai/catalog.ts
index dea42a15..fbd3f25b 100644
--- a/site/src/cut/server/ai/catalog.ts
+++ b/site/src/cut/server/ai/catalog.ts
@@ -196,6 +196,7 @@ What the user sees on the transitions row: one badge per bar, and nothing else
"graphics": `# Graphics: titles, shapes & stickers
The title lanes hold three overlay element kinds, all sharing timing (start/end), center position (x/y fractions 0..1), rotation (degrees), opacity, and a lane; \`overlays\` in editor_state lists them with their \`kind\`. All burn into the export exactly as previewed.
Rows are how elements stack (\`lane\`, 0 = the front row; a higher row draws further back). Two elements on ONE row never overlap in time — placing a title where a shape already sits slides it clear instead of laying it on top — so anything meant to sit over something else names a LOWER row: title on lane 0, the background shape behind it on lane 1. Every add tool and update_overlay take \`lane\`.
+An element rides the frames of the video clip under it (\`host\` in editor_state, the clip on the highest track beneath its middle): moving, retiming or splitting that clip carries the element over the same footage, trimming leaves it on the frames it covered, and deleting the clip leaves the element in place, homed to whatever now sits under it. Placing an element homes it automatically; update_overlay follows_clip:false frees one (\`free\` in editor_state) and true homes it again.
The frame's own color is set_background, not a full-frame rectangle: it fills the frame behind every clip and element, holds for the whole cut, letterboxes fitted footage, and shows through gaps on track 0. "Make the background white", a colored card behind a title sequence, a text-and-graphics video with no footage at all — all of them start with set_background. A rectangle element is for a panel or a band inside the frame, not for the frame itself.
A whole run of text — lines timed to speech, a lyric video, a kinetic-typography passage, a quote sequence — is add_text_sequence, one call. Design is opt-in: called with no look it lays a plain run — one readable face, one place in the frame, a short fade, no cards, no moves, the frame color untouched — which is what a bare "add the text" asks for. Name a look when the user asked for a designed run, and then the look is the design of the whole run, not one line: the frame color, the cards, and the ensemble the run walks across its length — faces, entrances, where successive lines land, and what each holds. Read the text-videos skill for the looks and the flow, and text-creativity for the fifty devices that keep a long run from reading flat. A designed run is composed in the call, never laid identical and then repaired element by element. A video needs no footage. Titles, shapes and stickers over a background color are a complete cut on their own: it plays, exports, and shares like any other. Build one by setting the background, then placing elements on the rows the stack needs — reach for footage only when the user asks for it.
Titles (add_title / update_overlay): text, size (frame px; the design short side is 1080), font (${FONTS.map((f) => `${f.id}=${f.label}`).join(", ")}), weight (400/700), color, shadow, plate (translucent dark backdrop). Type never leaves the picture: a line too wide for the room its anchor leaves is broken onto more lines as it draws, in the preview and in the export alike, so write the words at the size the design wants and leave the fitting to the renderer.