Open source CapCut alternative.
TypeScript
feat(site): teach the chat the smooth flag on set_speed and set_speed_curve
David Le2ae78dae223f
diff --git a/site/src/cut/components/Inspector.tools.ts b/site/src/cut/components/Inspector.tools.tsindex 41b49c98..8b4914a9 100644--- a/site/src/cut/components/Inspector.tools.ts+++ b/site/src/cut/components/Inspector.tools.ts@@ -283,13 +283,13 @@ export const INSPECTOR_TOOLS = [{name: "set_speed",description:- "Set a video clip's playback speed, one rate across the whole clip, and/or play it backward. Faster shortens the clip on the timeline; slower stretches it. Later clips, titles, captions, and soundtrack shift to stay in sync. A clip carrying a speed curve loses it when speed is passed: the rate becomes uniform again (set_speed_curve is the tool for a rate that changes through the footage). `reverse: true` plays the clip's trim backward, picture and sound, at its rate — the clip keeps its length and its curve, and its head now shows source `out`; `reverse: false` turns it forward again.",- inputSchema: obj({ clipId: str("Video clip id"), speed: num("Playback rate (1 = normal, no upper limit)"), reverse: bool("Play the footage backward (true) or forward (false)") }, ["clipId"]),+ "Set a video clip's playback speed, one rate across the whole clip, and/or play it backward. Faster shortens the clip on the timeline; slower stretches it. Later clips, titles, captions, and soundtrack shift to stay in sync. A clip carrying a speed curve loses it when speed is passed: the rate becomes uniform again (set_speed_curve is the tool for a rate that changes through the footage). `reverse: true` plays the clip's trim backward, picture and sound, at its rate — the clip keeps its length and its curve, and its head now shows source `out`; `reverse: false` turns it forward again. `smooth: true` smooths the clip's slow motion: wherever its rate is under 1×, the frames between source frames are synthesized (the export estimates the motion between them) so the slow stretch moves without stepping; at 1× and above nothing changes. Turn it on for real slow motion from ordinary footage; `smooth: false` shows the source frames as they are.",+ inputSchema: obj({ clipId: str("Video clip id"), speed: num("Playback rate (1 = normal, no upper limit)"), reverse: bool("Play the footage backward (true) or forward (false)"), smooth: bool("Smooth slow motion: synthesize the frames between source frames wherever the rate is under 1×") }, ["clipId"]),},{name: "set_speed_curve",description:- `Give a video clip (any track) a speed curve: its rate changes through the footage instead of being one number, so one clip can race through a walk, land on a beat, and hold on a face. Nodes are {at, speed}: \`at\` in SOURCE seconds inside the clip's trim — the unit detect_beats, trim_clip and watch_video speak, so beat times drop straight in — and speed ${SPEED_CURVE_MIN}–${SPEED_CURVE_MAX}×. The rate curves smoothly between nodes and holds flat past the outermost ones; the whole list replaces the clip's curve, and an empty list clears it (the clip keeps its length at one uniform rate). Or pass a preset to lay a named ramp over the trim: ${speedCurvePresetCatalogText()}. Technique: give a peak a beat of normal speed on either side or it reads as a glitch; below about 0.5× ordinary footage shows its own frame rate, so real slow motion wants high-frame-rate source (say so instead of shipping judder); sound stretches with its pitch kept, which holds up on speech to roughly 0.5–2×. The clip's timeline length becomes the integral of the curve and later items shift like a trim. Returns the resulting nodes, the clip's length before and after, and the rows that moved. In the UI the Speed Curve row opens the same graph over the timeline.`,+ `Give a video clip (any track) a speed curve: its rate changes through the footage instead of being one number, so one clip can race through a walk, land on a beat, and hold on a face. Nodes are {at, speed}: \`at\` in SOURCE seconds inside the clip's trim — the unit detect_beats, trim_clip and watch_video speak, so beat times drop straight in — and speed ${SPEED_CURVE_MIN}–${SPEED_CURVE_MAX}×. The rate curves smoothly between nodes and holds flat past the outermost ones; the whole list replaces the clip's curve, and an empty list clears it (the clip keeps its length at one uniform rate). Or pass a preset to lay a named ramp over the trim: ${speedCurvePresetCatalogText()}. Technique: give a peak a beat of normal speed on either side or it reads as a glitch; below about 0.5× ordinary footage shows its own frame rate, so pass \`smooth: true\` with the curve — it synthesizes the frames between source frames wherever the rate is under 1× and leaves the faster stretches alone — unless the source is high-frame-rate already; sound stretches with its pitch kept, which holds up on speech to roughly 0.5–2×. The clip's timeline length becomes the integral of the curve and later items shift like a trim. Returns the resulting nodes, the clip's length before and after, and the rows that moved. In the UI the Speed Curve row opens the same graph over the timeline.`,inputSchema: obj({clipId: str("Video clip id"),@@ -309,6 +309,7 @@ export const INSPECTOR_TOOLS = [enum: SPEED_CURVE_PRESET_IDS,description: "A named ramp laid over the clip's trim, instead of nodes",},+ smooth: bool("Smooth slow motion: synthesize the frames between source frames wherever the curve is under 1×"),},["clipId"]),diff --git a/site/src/cut/lib/aiContext.ts b/site/src/cut/lib/aiContext.tsindex 3f911e68..3b62b51f 100644--- a/site/src/cut/lib/aiContext.ts+++ b/site/src/cut/lib/aiContext.ts@@ -482,9 +482,9 @@ export function buildAiContext(opts?: { fullCues?: boolean; chatId?: string | nu/** A clip's rate as the model reads it: one number, or the average plus the* curve's nodes as [sourceSeconds, rate] when the rate changes through the* footage. */-function describeRate(c: { speed?: number; speedCurve?: SpeedNode[]; reverse?: boolean; in: number; out: number }) {+function describeRate(c: { speed?: number; speedCurve?: SpeedNode[]; reverse?: boolean; smoothSlow?: boolean; in: number; out: number }) {const nodes = speedCurveOf(c);- const reverse = c.reverse ? { reverse: true } : {};+ const reverse = { ...(c.reverse ? { reverse: true } : {}), ...(c.smoothSlow ? { smooth: true } : {}) };if (!nodes) return { speed: r(c.speed ?? 1), ...reverse };return {speed: r(retimeOf(c).rate),@@ -541,7 +541,7 @@ function describeOverlayClip(c: VideoClip, assets: Map<string, { name: string }>: {}),...(c.rotation ? { rotation: c.rotation } : {}),...((c.opacity ?? 1) < 1 ? { opacity: r(c.opacity ?? 1) } : {}),- ...(retimeOf(c).rate !== 1 || c.speedCurve || c.reverse ? describeRate(c) : {}),+ ...(retimeOf(c).rate !== 1 || c.speedCurve || c.reverse || c.smoothSlow ? describeRate(c) : {}),...(c.grade ? { colorGrade: c.grade } : {}),...(c.mask ? { mask: c.mask } : {}),...(c.boxStyle ? { boxStyle: c.boxStyle } : {}),diff --git a/site/src/cut/lib/aiTools.ts b/site/src/cut/lib/aiTools.tsindex 054bff3b..31f865d2 100644--- a/site/src/cut/lib/aiTools.ts+++ b/site/src/cut/lib/aiTools.ts@@ -955,6 +955,7 @@ const toolRuns: Record<BrowserToolName, ToolRun> = {out: round2(clip.out),speed: round2(speed),...(clip.reverse ? { reverse: true } : {}),+ ...(clip.smoothSlow ? { smooth: true } : {}),...(speedCurveOf(clip)? {speedCurve: speedCurveOf(clip)!.map(([at, rate]) => [round2(at), round2(rate)]),@@ -3585,16 +3586,19 @@ const toolRuns: Record<BrowserToolName, ToolRun> = {set_speed: (s, input) => {const clip = requireItem(s.clips, input.clipId, "video clip");const reverse = typeof input.reverse === "boolean" ? input.reverse : undefined;- if (!isNum(input.speed) && reverse === undefined)- throw new ToolError("speed (e.g. 1.5) or reverse is required.");+ const smooth = typeof input.smooth === "boolean" ? input.smooth : undefined;+ if (!isNum(input.speed) && reverse === undefined && smooth === undefined)+ throw new ToolError("speed (e.g. 1.5), reverse, or smooth is required.");const before = clipLen(clip);if (isNum(input.speed)) s.setClipSpeed(clip.id, input.speed);if (reverse !== undefined) s.setClipReverse(clip.id, reverse);+ if (smooth !== undefined) s.setClipSmoothSlow(clip.id, smooth);const next = useEditor.getState().clips.find((c) => c.id === clip.id)!;return {id: next.id,speed: next.speed ?? 1,...(next.reverse ? { reverse: true } : {}),+ ...(next.smoothSlow ? { smooth: true } : {}),lenBefore: round2(before),lenAfter: round2(clipLen(next)),...tracksAfter(),@@ -3633,11 +3637,14 @@ const toolRuns: Record<BrowserToolName, ToolRun> = {s.setClipSpeedCurve(clip.id, nodes);}}+ // After the curve, so a rejected curve leaves the flag alone too.+ if (typeof input.smooth === "boolean") s.setClipSmoothSlow(clip.id, input.smooth);const next = useEditor.getState().clips.find((c) => c.id === clip.id)!;const curve = speedCurveOf(next);return {id: next.id,speed: round2(retimeOf(next).rate),+ ...(next.smoothSlow ? { smooth: true } : {}),nodes: curve ? curve.map(([at, speed]) => ({ at: round2(at), speed: round2(speed) })) : [],lenBefore: round2(before),lenAfter: round2(clipLen(next)),@@ -4140,7 +4147,7 @@ function resolveWatchTarget(input: Record<string, unknown>): {asset: MediaAsset;- clip: { id: string; start: number; in: number; out: number; speed?: number; speedCurve?: SpeedNode[]; reverse?: boolean } | null;+ clip: { id: string; start: number; in: number; out: number; speed?: number; speedCurve?: SpeedNode[]; reverse?: boolean; smoothSlow?: boolean } | null;} {if (input.clip_id !== undefined && input.clip_id !== null) {const id = String(input.clip_id);@@ -4167,7 +4174,7 @@ function resolveWatchRange(): {projectId: string;asset: MediaAsset;- clip: { id: string; start: number; in: number; out: number; speed?: number; speedCurve?: SpeedNode[]; reverse?: boolean } | null;+ clip: { id: string; start: number; in: number; out: number; speed?: number; speedCurve?: SpeedNode[]; reverse?: boolean; smoothSlow?: boolean } | null;speed: number;from: number;to: number | undefined;diff --git a/site/src/cut/server/ai/catalog.ts b/site/src/cut/server/ai/catalog.tsindex 979a3cce..fa7a6d25 100644--- a/site/src/cut/server/ai/catalog.ts+++ b/site/src/cut/server/ai/catalog.ts@@ -100,7 +100,7 @@ Times are in seconds on the shared timeline. The playhead is currentTime; a skim- A clip's timeline length is (out-in)/speed — or, for a clip with a speed curve, the integral of its curve over the trim (editor_state reports \`len\` either way, and lists a curved clip's \`speedCurve\` nodes as [sourceSeconds, rate]). A clip with \`reverse: true\` plays its trim backward at that same rate — same length, head showing source \`out\`, sound running backward too. Total duration runs to the last clip's end, gaps included. Transitions blend at cuts and never change layout or length.- trim_clip changes in/out inside the source media. in >= 0, out <= source duration, out-in >= 0.1.- Resizes ripple the same way deletes do: while track 0 is the only video track, a track-0 trim or speed change slides everything past the clip's tail — clips, titles, captions, soundtrack — with the moved edge, in both directions, every existing gap keeping its width. With overlay video tracks present, growth pushes the track-0 run and a shrink leaves a gap.-- set_speed sets a clip's playback rate; it changes the clip's timeline length and carries the timeline behind it like a trim. set_speed with \`reverse: true\` plays the clip backward (a rewind, a drop that rises, a pour that un-pours); it changes nothing about length or the curve. set_speed_curve makes the rate change through the footage — nodes of {at: source seconds, speed} or a named preset ramp — for speed ramps, whips, holds, and hits on the beat: detect_beats first, then place nodes at the beats it returns (same unit, no arithmetic). set_speed on a curved clip makes it uniform again.+- set_speed sets a clip's playback rate; it changes the clip's timeline length and carries the timeline behind it like a trim. set_speed with \`reverse: true\` plays the clip backward (a rewind, a drop that rises, a pour that un-pours); it changes nothing about length or the curve. set_speed_curve makes the rate change through the footage — nodes of {at: source seconds, speed} or a named preset ramp — for speed ramps, whips, holds, and hits on the beat: detect_beats first, then place nodes at the beats it returns (same unit, no arithmetic). set_speed on a curved clip makes it uniform again. Either tool takes \`smooth: true\` — smooth slow motion — which synthesizes the frames between source frames wherever the clip's rate is under 1× (the export estimates the motion between them) and leaves the rest of the clip alone; editor_state shows it as \`smooth\`. Ordinary footage slowed below about 0.5× wants it on.- set_transition joins a clip into the next one (0–2s, ${TRANSITION_STYLE_IDS.length} styles); set_animation animates one clip's own entrance/exit — read the transitions-and-fades skill before styling cuts. Splitting or deleting clears the affected transition. A transition is a bar on its own row at an absolute time, so an edit can leave one lining up with nothing: every timeline mutation reports those as \`parkedTransitions\`, and clearing them (remove_transition) is part of finishing the edit. Grading a clip's picture is the color tools (set_color_preset, set_color_grade, set_color_curves, set_color_wheels, set_color_hsl — read the color-grading skill); a stylized post treatment (grain, VHS, halation…) is add_effect, dropped over the stretch of timeline it covers — as is a treatment on the sound (echo, reverb, telephone…).- Cutting a clip's background away — the AI person matte, ink around the silhouette, a fill behind the subject — is the cutout suite (set_removal, set_removal_stroke, set_removal_background); read the background-removal skill.- split_at cuts the track-0 clip under that time into two clips at the exact frame. With a soundtrack or overlay clip selected it splits that instead.