Open source CapCut alternative.
TypeScript
fix(timeline): insert rows through shared drag placement [rebuild]
David Le0436f70521b6
diff --git a/docs/guides/cut/README.md b/docs/guides/cut/README.mdindex a63c594b..99e00384 100644--- a/docs/guides/cut/README.md+++ b/docs/guides/cut/README.md@@ -46,7 +46,7 @@ Browser previews use the browser compositor and replace one cached proxy in OPFSCloud source media, generated media, and retained exports count toward storage. Derived previews, share cards, and HLS ladders are quota-exempt; staging inputs are temporary. An export is weighed twice — the size it is heading for before a frame is drawn, and what actually landed before the account is charged — so one render cannot carry an account past its cap. Transports expose structured storage, credit, and authentication failures. Each host chooses their presentation; the website mounts its storage dialog separately.-Timeline and preview share selection: ⌘/Ctrl-click or Shift-click toggles items, and dragging moves them together with one undo step. Items move independently until grouped, and groups save with the project. On the preview the selection wears one frame whose grips scale the set and whose button turns it, while a drag on any member moves them all. Selecting two or more items opens the group panel: it shows the fields every selected item has, reads a field as Mixed where they differ, and writes each edit to all of them in one undo step. A selection of one kind keeps that kind's own rows and its Color and Animation views, and an effect brings only its amount and Hidden.+Timeline and preview share selection: ⌘/Ctrl-click or Shift-click toggles items; dragging moves them with one undo step. Groups persist. Their preview frame scales and rotates the selection; the group panel edits shared fields, showing Mixed where values differ. Text side grips change wrapping width and corners scale proportionally; height follows content across preview, export and chat. Dropping between timeline rows inserts a row through shared text, video and audio placement.Guides draw over the preview from the button beside the timeline zoom: thirds, center, safe margins, the short-form keep-out zone where TikTok, Reels and Shorts draw their own UI or crop the sides of a 9:16 frame, and custom lines dragged into place on the preview. They save with the project, snap a dragged element to their edges, never export, and the assistant reads the safe area they leave when it places graphics.diff --git a/site/src/cut/components/Timeline.tools.ts b/site/src/cut/components/Timeline.tools.tsindex 5c50a908..76fc9026 100644--- a/site/src/cut/components/Timeline.tools.ts+++ b/site/src/cut/components/Timeline.tools.ts@@ -7,7 +7,7 @@*/import { bool, ids, num, obj, str, type AiToolDef } from "@/cut/lib/aiToolDef";-import { ITEM_KIND_IDS, canSplitItem } from "@/cut/lib/itemKinds";+import { ITEM_KIND_IDS, ROW_ITEM_KINDS, canSplitItem } from "@/cut/lib/itemKinds";import { TIMELINE_ITEM_KINDS } from "@/cut/lib/timelineGroups";import {TEXT_EMPHASIS_IDS,@@ -44,6 +44,15 @@ export const TIMELINE_TOOLS = [description: "Move selected timeline items and their group members together by delta seconds, preserving relative timing and caption word timings. Negative moves stop at the timeline start; occupied rows move the set to the next free position. Use this for a grouped selection, including captions and transitions.",inputSchema: obj({ delta: num("Timeline shift in seconds") }, ["delta"]),},+ {+ name: "insert_item_row",+ description: "Move an item into a new row before a displayed row in its own timeline band, like dropping between rows. Display rows count from the top starting at 0; the row count inserts after the last row. Empty source rows collapse. Timing stays fixed and the edit is undoable.",+ inputSchema: obj({+ kind: { type: "string", enum: ROW_ITEM_KINDS, description: "Timeline item kind" },+ id: str("Timeline item id"),+ before_row: { type: "integer", minimum: 0, description: "Display row to insert before" },+ }, ["kind", "id", "before_row"]),+ },{name: "split_at",description:diff --git a/site/src/cut/components/Timeline.tsx b/site/src/cut/components/Timeline.tsxindex 65f5668c..024e1a3a 100644--- a/site/src/cut/components/Timeline.tsx+++ b/site/src/cut/components/Timeline.tsx@@ -803,6 +803,7 @@ export function Timeline() {if (row < 0)return { kind: "insert", level: Math.max(0, ...overlayLayers(st.clips).map((c) => c.track)) + 1 };if (row >= rows.length) return { kind: "insert", level: 0 };+ if (!Number.isInteger(row)) return { kind: "insert", level: rows[Math.floor(row)].track };return { kind: "track", track: rows[row].track };}, []);@@ -1859,6 +1860,18 @@ export function Timeline() {/>) : null;+ const insertionLine = (top: number, key?: number) => (+ <div key={key} data-tl-insertion="" className="pointer-events-none absolute inset-x-0 z-30 h-0.5 bg-[#0a84ff]" style={{ top }} />+ );+ const insertionMarks = (slots: LaneDrag[], h: number, shift: number, external?: number) =>+ [...new Set([...slots.map((slot) => slot.targetRow), ...(external === undefined ? [] : [external])])]+ .filter((row) => !Number.isInteger(row))+ .map((row) => insertionLine(Math.ceil(row) * h + shift - 1, row));+ const videoInsertion = (row: number, track: number) => row > 0 && (+ videoSlots.some((slot) => slot.targetRow === row - 0.5) ||+ (overlayDrop?.target.kind === "insert" && overlayDrop.target.level === track + 1)+ ) ? insertionLine(-4) : null;+// The would-be new video track, one row past the stack's edge — the same// grown-row experience as the audio and title lanes. It mounts only while// the drag is out there, and it grows away from the stack (the top one@@ -2310,6 +2323,7 @@ export function Timeline() {(_, r) => laneRail((r + 1) * TEXT_H - 4, r))}{rowMarks(overlayLanes.count, TEXT_H, topRowShift)}+ {insertionMarks(overlaySlots, TEXT_H, topRowShift, elementDrop?.row)}{overlayLanes.used.map((lane, r) => (<Fragment key={`tgap-${lane}`}>{gapHighlight(@@ -2570,6 +2584,7 @@ export function Timeline() {onContextMenu={openGapMenu({ kind: "video", index: track })}{...overlayDropHandlers}>+ {videoInsertion(row, track)}{laneRail(rowH - 2 + railH)}{gapHighlight({ kind: "video", index: track }, rowH - 4)}{videoSlots.filter((d) => d.targetRow === row).map((d) => (@@ -2625,6 +2640,7 @@ export function Timeline() {onPointerDown={deselectIfSelf}onContextMenu={openGapMenu({ kind: "video", index: 0 })}>+ {videoInsertion(aboveTracks.length, 0)}{spans.length > 0 && laneRail(rowH0 - 2 + rail0)}{gapHighlight({ kind: "video", index: 0 }, rowH0 - 4)}{trackSlot(TRACK_ZERO, rowH0 - 4)}@@ -2699,6 +2715,7 @@ export function Timeline() {(_, r) => laneRail((r + 1) * AUDIO_H - 2, r))}{rowMarks(audioLanes.count, AUDIO_H, audioTopShift)}+ {insertionMarks(audioSlots, AUDIO_H, audioTopShift, audioDrop?.row)}{audioLanes.used.map((lane, r) => (<Fragment key={`agap-${lane}`}>{gapHighlight(diff --git a/site/src/cut/lib/aiTools.ts b/site/src/cut/lib/aiTools.tsindex 7341b4df..0274bf24 100644--- a/site/src/cut/lib/aiTools.ts+++ b/site/src/cut/lib/aiTools.ts@@ -13,7 +13,8 @@ import { projectOperation, type ProjectOperation } from "./projectOperation";import { captureRenderSnapshot, renderDoc } from "./renderSnapshot";import { submitPreviewSnapshot } from "./exportClient";-import { ITEM_KIND_IDS, ITEM_KINDS, type ItemKind } from "./itemKinds";+import { ITEM_KIND_IDS, ITEM_KINDS, ROW_ITEM_KINDS, type ItemKind } from "./itemKinds";+import { insertItemRow } from "./laneTracks";import { timelineItem } from "./timelineItems";import { movePreviewSelection, previewSelectionSnapshot } from "@/cut/lib/previewSelection";@@ -1648,6 +1649,12 @@ const toolRuns: Record<BrowserToolName, ToolRun> = {return { delta: s.moveTimelineSelection(input.delta), selection: useEditor.getState().multiSelection };},+ insert_item_row: (_s, input) => {+ if (!ROW_ITEM_KINDS.includes(input.kind as ItemKind) || typeof input.id !== "string" || !isNum(input.before_row)) throw new ToolError("kind, id and before_row are required.");+ insertItemRow(input.kind as ItemKind, input.id, input.before_row);+ return { ...tracksAfter() };+ },+split_at: (s, input) => {const before = ITEM_KIND_IDS.reduce((n, kind) => n + ITEM_KINDS[kind].list(s).length, 0);s.splitAtPlayhead(isNum(input.t) ? input.t : undefined);diff --git a/site/src/cut/lib/itemKinds.ts b/site/src/cut/lib/itemKinds.tsindex 5d4338c5..13cbe1e0 100644--- a/site/src/cut/lib/itemKinds.ts+++ b/site/src/cut/lib/itemKinds.ts@@ -52,6 +52,8 @@ export interface ItemLists {}export interface ItemKindDef<K extends ItemKind> {+ /** Items can move between rows or open a row between existing ones. */+ multiLane?: boolean;/** The list this kind lives in. */list(s: ItemLists): ItemOf[K][];/** Store the list while preserving the document's other collections. */@@ -86,6 +88,7 @@ function splitMedia<T extends VideoClip | AudioClip>(item: T, at: number): [T, Texport const ITEM_KINDS: { [K in ItemKind]: ItemKindDef<K> } = {clip: {+ multiLane: true,list: (s) => s.clips,withList: (s, clips) => ({ ...s, clips }),duration: (c) => retimeOf(c).len,@@ -125,6 +128,7 @@ export const ITEM_KINDS: { [K in ItemKind]: ItemKindDef<K> } = {},},audio: {+ multiLane: true,list: (s) => s.audioClips,withList: (s, audioClips) => ({ ...s, audioClips }),duration: (a) => retimeOf(a).len,@@ -137,6 +141,7 @@ export const ITEM_KINDS: { [K in ItemKind]: ItemKindDef<K> } = {crossProject: (a) => a,},overlay: {+ multiLane: true,list: (s) => s.overlays,withList: (s, overlays) => ({ ...s, overlays }),duration: (o) => o.end - o.start,@@ -199,6 +204,7 @@ export const ITEM_KINDS: { [K in ItemKind]: ItemKindDef<K> } = {};export const ITEM_KIND_IDS = Object.keys(ITEM_KINDS) as ItemKind[];+export const ROW_ITEM_KINDS = ITEM_KIND_IDS.filter((kind) => ITEM_KINDS[kind].multiLane);/** The item a selection names, as a clipboard item, or null when gone. */export function clipboardItemFor(s: ItemLists, sel: NonNullable<Selection>): TimelineClipboardItem | null {diff --git a/site/src/cut/lib/laneTracks.test.ts b/site/src/cut/lib/laneTracks.test.tsindex 4e4c4ee4..98ff3bcc 100644--- a/site/src/cut/lib/laneTracks.test.ts+++ b/site/src/cut/lib/laneTracks.test.ts@@ -1,5 +1,6 @@import { beforeEach, describe, expect, test } from "bun:test";import { commitRow, groupTrimEdges, landOnRow, NEW_ROW_PX, partAround, resolveRow } from "./laneTracks";+import { runAiTool } from "./aiTools";import { clipLen, useEditor } from "./store";import { emptySubtitles } from "./types";import type { AudioClip, VideoClip } from "./types";@@ -63,12 +64,14 @@ describe("resolveRow", () => {expect(resolveRow(rows, 219, null, open)).toBe(2);});- test("a pointer in the gap between rows takes the row below", () => {+ test("a pointer between rows opens an insertion", () => {const gapped = [{ top: 100, h: 40 },{ top: 146, h: 40 },];- expect(resolveRow(gapped, 143, null, open)).toBe(1);+ expect(resolveRow(gapped, 143, null, open)).toBe(0.5);+ expect(resolveRow(gapped, 153, 0.5, open)).toBe(0.5);+ expect(resolveRow(gapped, 160, 0.5, open)).toBe(1);});test("a new row opens only past the reach beyond either edge", () => {@@ -240,17 +243,17 @@ describe("element rows use the shared drop and move placement", () => {effect: (lane: number) => s().addEffect("blur", { at: 4, lane }),};for (const [kind, place] of Object.entries(add)) {- for (const row of [-1, 0, 1, 2]) {+ for (const row of [-1, 0, 0.5, 1, 2]) {test(`${kind} drops on row ${row} and undoes as one edit`, () => {s().addOverlay({ at: 0, lane: 0 });s().addShape("rect", { at: 0, lane: 1 });const before = s().overlays;landOnRow("overlay", row, place);const landed = s().overlays.find((o) => o.id === s().selection?.id)!;- expect(landed.lane).toBe(Math.max(0, row));+ expect(landed.lane).toBe(Math.max(0, Math.ceil(row)));expect(landed.start).toBe(4);expect(s().overlays.find((o) => o.id === before[0].id)?.lane).toBe(row < 0 ? 1 : 0);- expect(s().overlays.find((o) => o.id === before[1].id)?.lane).toBe(row < 0 ? 2 : 1);+ expect(s().overlays.find((o) => o.id === before[1].id)?.lane).toBe(row < 0 || row === 0.5 ? 2 : 1);s().undo();expect(s().overlays).toEqual(before);});@@ -260,6 +263,22 @@ describe("element rows use the shared drop and move placement", () => {expect(s().overlays).toHaveLength(1);expect(s().overlays[0].lane).toBe(0);});+ test(`${kind} can move between occupied rows and undo`, () => {+ s().addOverlay({ at: 0, lane: 0 });+ const a = s().selection!.id;+ s().addShape("rect", { at: 0, lane: 1 });+ const b = s().selection!.id;+ place(0);+ const id = s().selection!.id;+ const before = s().overlays;+ s().pushHistory();+ commitRow("overlay", id, 0.5);+ expect(s().overlays.find((o) => o.id === id)?.lane).toBe(1);+ expect(s().overlays.find((o) => o.id === a)?.lane).toBe(0);+ expect(s().overlays.find((o) => o.id === b)?.lane).toBe(2);+ s().undo();+ expect(s().overlays).toEqual(before);+ });test(`${kind} can move to a fresh row at either end`, () => {s().addOverlay({ at: 0, lane: 0 });place(0);@@ -316,3 +335,31 @@ describe("groupTrimEdges", () => {expect(groupTrimEdges(edges, 2, 5)).toEqual({ travel: 5, at: [14, 13, 13] });});});++for (const kind of ["video", "audio"] as const) {+ test(`${kind} inserts between occupied rows through the shared resolver`, () => {+ const a = kind === "video" ? vclip({ track: 0 }) : aclip({ lane: 0 });+ const b = kind === "video" ? vclip({ track: 1 }) : aclip({ lane: 1 });+ const mover = kind === "video" ? vclip({ track: 0, start: 3 }) : aclip({ lane: 0, start: 3 });+ useEditor.setState(kind === "video" ? { clips: [a, b, mover] as VideoClip[] } : { audioClips: [a, b, mover] as AudioClip[] });+ const row = resolveRow([{ top: 0, h: 40 }, { top: 40, h: 40 }], 40, null, { top: true, bottom: true });+ commitRow(kind, mover.id, row);+ const lane = (id: string) => kind === "video" ? clipById(id).track : audioById(id).lane ?? 0;+ expect(lane(mover.id)).toBe(1);+ expect(lane(a.id)).toBe(0);+ expect(lane(b.id)).toBe(2);+ });+}++test("chat inserts an element row through the drag coordinator and undoes it", async () => {+ s().addOverlay({ at: 0, lane: 0 });+ s().addShape("rect", { at: 0, lane: 1 });+ s().addOverlay({ at: 5, lane: 0 });+ const id = s().selection!.id;+ const before = s().overlays;+ await runAiTool("insert_item_row", { kind: "overlay", id, before_row: 1 });+ expect(s().overlays.map((o) => o.lane)).toEqual([0, 2, 1]);+ expect(s().overlays[2].start).toBe(5);+ s().undo();+ expect(s().overlays).toEqual(before);+});diff --git a/site/src/cut/lib/laneTracks.ts b/site/src/cut/lib/laneTracks.tsindex 48aec0fa..5b482aa3 100644--- a/site/src/cut/lib/laneTracks.ts+++ b/site/src/cut/lib/laneTracks.ts@@ -77,7 +77,7 @@ export interface RowBox {}/** The display row a drag aims at from the pointer's screen y: the row under- * the pointer (a pointer in the gap between two rows takes the row below),+ * the pointer, a half-index for insertion between neighboring rows,* `-1` past the top of the band, `rows.length` past the bottom. Reaching past* an edge takes `NEW_ROW_PX` of travel; holding it takes none, since by then* the new row's own box fills the space the pointer is in — `held` is the row@@ -97,6 +97,11 @@ export function resolveRow(const bottom = rows[n - 1];if (edges.top && y < top.top - (held === -1 ? 0 : NEW_ROW_PX)) return -1;if (edges.bottom && y > bottom.top + bottom.h + (held === n ? 0 : NEW_ROW_PX)) return n;+ for (let i = 0; i < n - 1; i++) {+ const seam = (rows[i].top + rows[i].h + rows[i + 1].top) / 2;+ const reach = Math.min(rows[i].h, rows[i + 1].h, NEW_ROW_PX) / (held === i + 0.5 ? 2 : 4);+ if (Math.abs(y - seam) <= reach) return i + 0.5;+ }const i = rows.findIndex((r) => y <= r.top + r.h);return i < 0 ? n - 1 : Math.max(0, i);}@@ -229,7 +234,7 @@ function videoMaxLen(s: S, c: VideoClip): number {const videoAdapter: LaneAdapter<VideoClip> = {minLen: 0.15,- multiLane: true,+ multiLane: !!ITEM_KINDS.clip.multiLane,rowsDescend: true,raws: (s) => s.clips,view: (c) => ({ id: c.id, start: c.start, len: ITEM_KINDS.clip.duration(c), lane: ITEM_KINDS.clip.lane(c) }),@@ -257,7 +262,7 @@ const videoAdapter: LaneAdapter<VideoClip> = {const audioAdapter: LaneAdapter<AudioClip> = {minLen: 0.15,- multiLane: true,+ multiLane: !!ITEM_KINDS.audio.multiLane,raws: (s) => s.audioClips,view: (a) => ({ id: a.id, start: a.start, len: ITEM_KINDS.audio.duration(a), lane: ITEM_KINDS.audio.lane(a) }),apply: (patches) => useEditor.getState().updateAudiosTransient(patches),@@ -283,7 +288,7 @@ const audioAdapter: LaneAdapter<AudioClip> = {const textAdapter: LaneAdapter<Overlay> = {minLen: 0.2,- multiLane: true,+ multiLane: !!ITEM_KINDS.overlay.multiLane,raws: (s) => s.overlays,view: (o) => ({ id: o.id, start: o.start, len: ITEM_KINDS.overlay.duration(o), lane: ITEM_KINDS.overlay.lane(o) }),apply: (patches) => useEditor.getState().updateOverlaysTransient(patches),@@ -1004,7 +1009,7 @@ export function startLaneMove(e: React.PointerEvent, kind: LaneKind, id: string,? targetRow < 0? -Infinity: targetRow < usedLanes.length- ? usedLanes[targetRow]+ ? usedLanes[targetRow] ?? Infinity: Infinity: self.lane;@@ -1083,6 +1088,9 @@ function laneOrder(kind: LaneKind, s: S, lanes: number[]): number[] {* after the move folds back into 0..n-1. `beyond` counts rows past the end* for a group whose members straddle the edge. */function laneAtRow(kind: LaneKind, used: number[], row: number): number {+ if (!Number.isInteger(row)) {+ return (laneAtRow(kind, used, Math.floor(row)) + laneAtRow(kind, used, Math.ceil(row))) / 2;+ }if (row >= 0 && row < used.length) return used[row];const lo = Math.min(0, ...used) - 1;const hi = Math.max(-1, ...used) + 1;@@ -1112,6 +1120,20 @@ export function commitRow(kind: LaneKind, id: string, targetRow: number) {ad.apply(raws.map((r, i) => ad.lanePatch!(r, remap.get(moved[i]) ?? 0)));}+/** Insert an item before a display row, using the drag's placement and undo. */+export function insertItemRow(kind: ItemKind, id: string, beforeRow: number): void {+ if (!ITEM_KINDS[kind].multiLane) throw new Error("This item has no movable rows.");+ const band = kind === "clip" ? "video" : kind;+ const ad = ADAPTERS[band];+ const state = useEditor.getState();+ const views = ad.raws(state).map((raw) => ad.view(raw));+ if (!views.some((view) => view.id === id)) throw new Error("Timeline item not found.");+ const rows = laneOrder(band, state, views.map((view) => view.lane));+ if (!Number.isInteger(beforeRow) || beforeRow < 0 || beforeRow > rows.length) throw new Error("Row is outside this timeline band.");+ state.pushHistory();+ commitRow(band, id, beforeRow === 0 ? -1 : beforeRow === rows.length ? rows.length : beforeRow - 0.5);+}+/*** Land a new item on a display row of a multi-lane band — the row under a* drop, or one past either edge — by the lane arithmetic a lane drag commits@@ -1133,7 +1155,11 @@ export function landOnRow(kind: LaneKind, row: number, add: (lane: number) => volet lane = laneAtRow(kind, used, row);s.beginHistoryBatch();try {- if (lane < 0) {+ if (!Number.isInteger(row)) {+ const next = [...new Set([...used, lane])].sort((a, b) => a - b);+ ad.apply(raws.map((r, i) => ad.lanePatch!(r, next.indexOf(views[i].lane))));+ lane = next.indexOf(lane);+ } else if (lane < 0) {const lift = -lane;ad.apply(raws.map((r, i) => ad.lanePatch!(r, views[i].lane + lift)));lane = 0;diff --git a/site/src/cut/server/ai/catalog.ts b/site/src/cut/server/ai/catalog.tsindex 1269519c..f802cd18 100644--- a/site/src/cut/server/ai/catalog.ts+++ b/site/src/cut/server/ai/catalog.ts@@ -353,7 +353,7 @@ What the user sees on the transitions row: one badge per bar, and nothing else"graphics": `# Graphics: titles, shapes & stickersThe title lanes hold three overlay element kinds, all sharing a name (what the person calls it, or what it is — a title's text, "Rectangle"; \`name\` on update_overlay renames it, and the timeline chip, the inspector and @-references show that name, which is how a cut full of look-alike shapes stays legible), timing (start/end), center position (x/y fractions 0..1), rotation (degrees), opacity, and a lane; \`overlays\` in editor_state lists them with their \`kind\` and \`name\`. All burn into the export exactly as previewed.Several elements change together through one update_overlay call with \`ids\` — a group, every title in a run — the way the group panel edits a multi-selection; the fields a kind lacks are skipped on that element.-Rows are how elements stack (\`lane\`, 0 = the front row; a higher row draws further back). Two elements on ONE row never overlap in time — placing a title where a shape already sits slides it clear instead of laying it on top — so anything meant to sit over something else names a LOWER row: title on lane 0, the background shape behind it on lane 1. Every add tool and update_overlay take \`lane\`.+Rows are how elements stack (\`lane\`, 0 = the front row; a higher row draws further back). Two elements on ONE row never overlap in time — placing a title where a shape already sits slides it clear instead of laying it on top — so anything meant to sit over something else names a LOWER row: title on lane 0, the background shape behind it on lane 1. Every add tool and update_overlay take \`lane\`. Use insert_item_row to open a row between occupied rows for text, video or audio; before_row counts displayed rows from the top of that band.Use copy_selection and paste_selection for any timeline type, including effects and transitions. Copies preserve properties and relative timing. Transitions cannot be split.Timeline items start ungrouped. Use group_items to group any selection of clips, audio, elements, captions and transitions, and ungroup_items to separate them. Selecting a member selects the whole group; moving the group preserves its relative timing.The frame's own color is set_background, not a full-frame rectangle: it fills the frame behind every clip and element, holds for the whole cut, letterboxes fitted footage, and shows through gaps on track 0. "Make the background white", a colored card behind a title sequence, a text-and-graphics video with no footage at all — all of them start with set_background. A rectangle element is for a panel or a band inside the frame, not for the frame itself.