Open source CapCut alternative.
TypeScript
feat(site): the cross dissolve crosses the cut [rebuild]
David Led5f5fa218129
diff --git a/docs/guides/cut/README.md b/docs/guides/cut/README.mdindex fc771487..c1d5ac39 100644--- a/docs/guides/cut/README.md+++ b/docs/guides/cut/README.md@@ -38,7 +38,9 @@ The client probes the engine's health endpoint (dedicated port first, dev server**An effect treats what runs under it.** An effect is a stretch of the timeline rather than a property of a clip, so it can cover part of a shot, a whole one, or a run of them, and it is trimmed and moved like any other element. Some of them treat the picture — a grade, a grain, a shake, a push in — and some treat the sound: everything audible under the window, clip audio, upper tracks and soundtrack, mixed and then filtered as one. The sound treatments are described as data — a dry level, delay taps, filter bands, a tremolo, a bit crush — and three renderers read that one description: the Web Audio graph the preview plays, the same graph folded offline for an in-tab export, and an ffmpeg chain for the engine and the cloud worker. That is what keeps the export sounding like the preview; the ffmpeg side cuts the window out of the mix, runs the chain over it, and pads the result back to the length it went in at, so a chain that rings past its window cannot walk the rest of the sound late. The window's edges cross over rather than switch, in both mixes, so an effect coming in cannot click. A picture effect can be shown on its tile; a sound treatment has to be heard, so the tile plays — a line of recorded speech run through the same chain the mix would build. The line is one of the voice personas already shipped with the site, fetched once and kept decoded for the session, so the shelf demonstrates itself in a project with no footage in it yet.-**A handover is on the picture or on the sound.** A transition is a blend at the cut rather than an overlap: the clips still abut, the layout never moves, and the incoming clip starts exactly at the cut. Most styles blend the picture across the outgoing clip's last seconds. One hands over on the sound instead — the picture cuts, the outgoing clip's sound fades out into the cut and the incoming clip's fades in out of it. The two windows sit on separate fields, so everything that draws the picture reads a plain cut and only the mix ramps: the preview, the in-tab export, and the ffmpeg graph the engine and the worker build all follow from that one split. A sound handover needs a second clip's sound to cross into, so a bar of it parked on a clip's open head or tail plays nothing. It is picked where the other work on the sound is picked — the Effects tab's Sound family — while the styles that blend the picture are picked from the Transitions tab.+**A handover is on the picture or on the sound.** A transition is a blend at the cut rather than an overlap: the clips still abut, the layout never moves, and the incoming clip starts exactly at the cut. Most styles blend the picture across the outgoing clip's last seconds. The cross dissolve hands over on the sound instead — the picture cuts while the two clips ramp past each other, half the handover either side of it. Its bar sits across the cut for that reason, where every other bar ends on the boundary it plays. The two windows sit on separate fields, so everything that draws the picture reads a plain cut and only the mix ramps: the preview, the in-tab export, and the ffmpeg graph the engine and the worker build all follow from that one split. A sound handover needs a second clip's sound to cross into, so a bar of it parked on a clip's open head or tail plays nothing.++**A crossing needs handle, and holds its loudness.** Clips never overlap, so for the length of a cross dissolve each one plays into the other's time out of its handle — the media its trim left behind, the outgoing clip past its out point and the incoming clip before its in point. That is what makes both sides really audible over a cut their pictures hard-join, and it is why the ramps are equal-power (sine and cosine) rather than straight lines: two signals meeting at half level sum to the loudness they each had, so the mix never dips through the join. A clip trimmed to the end of its source has no handle to reach into; its side of the crossing stops at the cut, and the handover reads as a dip instead of a crossing. Every renderer works from the same two numbers on a span — how far the crossing runs, and how far each side reaches — so the preview, the browser fold and the ffmpeg graph land on one sound. It opens at 0.3 seconds, shorter than a picture blend's half second: that covers the jump in room tone at a cut while staying under the length where the ear starts hearing two voices, and it asks less of the trim each clip has left. It is picked where the other work on the sound is picked — the Effects tab's Sound family — while the styles that blend the picture are picked from the Transitions tab.## The previewdiff --git a/site/src/cut/components/EffectsPanel.tsx b/site/src/cut/components/EffectsPanel.tsxindex 4d452a91..611749fa 100644--- a/site/src/cut/components/EffectsPanel.tsx+++ b/site/src/cut/components/EffectsPanel.tsx@@ -56,7 +56,7 @@ type EffectGroup = "moving" | "filters" | "audio";/** The transition that hands over on the sound alone, picked from this shelf* rather than the Transitions tab: what it does, it does to the sound. */-const SOUND_DISSOLVE = "audiocross" as const;+const CROSS_DISSOLVE = "audiocross" as const;const GROUPS = [{ id: "moving", label: "Moving" },@@ -110,7 +110,7 @@ export function EffectsPanel() {{/* The handover on the sound sits with the treatments on thesound. It is a transition — it drags to a cut, where thepicture keeps cutting and the sound crosses over. */}- <SoundDissolveTile index={AUDIO_EFFECT_IDS.length} />+ <CrossDissolveTile index={AUDIO_EFFECT_IDS.length} /></>) : (ALL_EFFECT_IDS.filter((id) => groupOf(id) === group).map((id) => (@@ -242,21 +242,21 @@ function SoundTile({* the timeline marks the joint it would take — but it belongs on this shelf,* because what it does is done to the sound.*/-function SoundDissolveTile({ index }: { index: number }) {- const { picked, pick } = useAssetPick(`transition:${SOUND_DISSOLVE}`);+function CrossDissolveTile({ index }: { index: number }) {+ const { picked, pick } = useAssetPick(`transition:${CROSS_DISSOLVE}`);const t = useSoundClock(index);return (<SoundCard- pickId={`transition:${SOUND_DISSOLVE}`}- label={TRANSITION_STYLE_LABELS[SOUND_DISSOLVE]}+ pickId={`transition:${CROSS_DISSOLVE}`}+ label={TRANSITION_STYLE_LABELS[CROSS_DISSOLVE]}marked={picked}onChoose={pick}onDragStart={(e) => {- setElementDragData(e, { kind: "transition", style: SOUND_DISSOLVE });+ setElementDragData(e, { kind: "transition", style: CROSS_DISSOLVE });setObjectDragImage(e);}}figure={<CrossStrip t={t} />}- audition={SOUND_DISSOLVE}+ audition={CROSS_DISSOLVE}/>);}@@ -500,7 +500,7 @@ function useSoundClock(index: number): number {return (now + phase) * rate;}-/** The sound dissolve's figure: one sound going as the other arrives, with the+/** The cross dissolve's figure: one sound going as the other arrives, with the* cut standing where the picture changes.** Each side is scaled by its own ramp rather than only faded, because that is@@ -508,9 +508,11 @@ function useSoundClock(index: number): number {* cut, the incoming one grows out of it — and a level is what reads at tile* size. */function CrossStrip({ t }: { t: number }) {- // The two ramps meet at zero on the cut, which is where the sound really is- // at the middle of a dissolve — so the figure pinches there.- const ramp = (i: number, n: number) => Math.max(0, Math.min(1, i / (n - 1) - 0.5) * 2);+ // The ramps are the equal-power pair the crossing really uses: both sides+ // are still up at the cut, each at about seven tenths, so the figure crosses+ // in the middle rather than pinching to nothing there.+ const going = (i: number, n: number) => Math.cos(((i / (n - 1)) * Math.PI) / 2);+ const coming = (i: number, n: number) => Math.sin(((i / (n - 1)) * Math.PI) / 2);const scaled = (bars: number[], at: (i: number, n: number) => number) =>bars.map((v, i) => v * at(i, bars.length));const outgoing = audioFxBars(null, t, STRIP_BARS);@@ -518,13 +520,13 @@ function CrossStrip({ t }: { t: number }) {return (<span className="relative block h-7"><BarStrip- bars={scaled(outgoing, (i, n) => 1 - Math.min(1, (i / (n - 1)) * 2))}- alphaAt={(i, n) => 0.35 + 0.65 * (1 - Math.min(1, (i / (n - 1)) * 2))}+ bars={scaled(outgoing, going)}+ alphaAt={(i, n) => 0.35 + 0.65 * going(i, n)}className="absolute inset-0"/><BarStrip- bars={scaled(incoming, ramp)}- alphaAt={(i, n) => 0.35 + 0.65 * ramp(i, n)}+ bars={scaled(incoming, coming)}+ alphaAt={(i, n) => 0.35 + 0.65 * coming(i, n)}className="absolute inset-0"/><span className="absolute inset-y-0 left-1/2 w-px -translate-x-1/2 bg-foreground/40" />diff --git a/site/src/cut/components/Timeline.tsx b/site/src/cut/components/Timeline.tsxindex 6b0ccf37..421fd7fc 100644--- a/site/src/cut/components/Timeline.tsx+++ b/site/src/cut/components/Timeline.tsx@@ -51,7 +51,7 @@ import { playheadAt, setSkim, skimAt, subscribePlayhead, usePlayhead, useSkim }import { laneHidden, subtitleLaneCount } from "@/cut/lib/subtitles";import { formatTime, formatTimecode } from "@/cut/lib/time";import { EFFECT_LABELS, isAudioEffect, type EffectId } from "@donkeycut/effects-kit";-import { emptySubtitles, IMAGE_CLIP_SECONDS, SHAPE_LABELS, TRANSITION_DEFAULT_SECONDS, TRANSITION_MAX, TRANSITION_STYLE_LABELS, type ShapeKind } from "@/cut/lib/types";+import { emptySubtitles, IMAGE_CLIP_SECONDS, isAudioTransition, SHAPE_LABELS, TRANSITION_MAX, TRANSITION_STYLE_LABELS, transitionBarStart, transitionDefaultSeconds, type ShapeKind } from "@/cut/lib/types";import type { AudioClip, ClipSpan, ColorGrade, MediaAsset, Overlay, Selection, StickerOverlay, SubtitleCue, TimelineTransition, TransitionStyle, VideoClip } from "@/cut/lib/types";import { isLottieAsset } from "@/cut/lib/lottieAssets";import { gradeCssApprox } from "@donkeycut/effects-kit";@@ -257,6 +257,9 @@ type OverlayFamily = "text" | "element" | "effect";* `at` is the boundary itself (the cut, the head, the tail) and `len` the room* a transition takes there.*/+/** `len` is what already plays at this boundary, 0 when nothing does — the+ * length a landing bar takes is that or the dropped style's own default+ * (`anchorLen`), since a cross dissolve wants a shorter one than a blend. */type Anchor = { kind: "cut" | "in" | "out"; clipId: string; at: number; len: number };/** One transition bar drawn on the row: the store object itself, the place it@@ -1109,7 +1112,7 @@ export function Timeline() {kind: "in",clipId: sp.clip.id,at: sp.start,- len: sp.clip.animIn?.seconds || TRANSITION_DEFAULT_SECONDS,+ len: sp.clip.animIn?.seconds ?? 0,});}if (touchesNext) {@@ -1117,14 +1120,14 @@ export function Timeline() {kind: "cut",clipId: sp.clip.id,at: end,- len: sp.clip.transition || TRANSITION_DEFAULT_SECONDS,+ len: sp.clip.transition ?? 0,});} else {out.push({kind: "out",clipId: sp.clip.id,at: end,- len: sp.clip.animOut?.seconds || TRANSITION_DEFAULT_SECONDS,+ len: sp.clip.animOut?.seconds ?? 0,});}});@@ -1164,9 +1167,16 @@ export function Timeline() {[bars, barRoles]);+ /** The room a bar of `style` takes at this anchor: whatever already plays+ * there, or that style's own default length. */+ const anchorLen = (a: Anchor, style: TransitionStyle) =>+ a.len || transitionDefaultSeconds(style);+/** Where a bar of `len` seconds sits when it lands on this anchor: an- * entrance starts at the clip's head, everything else ends on its edge. */- const anchorBarStart = (a: Anchor, len: number) => (a.kind === "in" ? a.at : a.at - len);+ * entrance starts at the clip's head, a cross dissolve straddles the cut,+ * everything else ends on the boundary. */+ const anchorBarStart = (a: Anchor, len: number, style: TransitionStyle) =>+ transitionBarStart(style, a.kind, a.at, len);/** The bar already playing this anchor's boundary, if any — through any of* its roles, so a bar serving several same-instant boundaries is the@@ -1182,12 +1192,12 @@ export function Timeline() {const st = useEditor.getState();if (st.readOnly) return;const a = nearestAnchor(time);- const len = a?.len ?? TRANSITION_DEFAULT_SECONDS;+ const len = a ? anchorLen(a, style) : transitionDefaultSeconds(style);st.beginHistoryBatch();const incumbent = a ? barAt(a) : null;if (incumbent) st.removeTransition(incumbent.id);const id = st.addTransition({- start: a ? anchorBarStart(a, len) : time,+ start: a ? anchorBarStart(a, len, style) : time,seconds: len,style,});@@ -1211,12 +1221,12 @@ export function Timeline() {onMove: (dx) => {const free = Math.max(-x.t.seconds + 0.1, x.t.start + dx / pps);const near = anchors.reduce<{ a: Anchor; gap: number } | null>((best, a) => {- const gap = Math.abs(anchorBarStart(a, x.t.seconds) - free);+ const gap = Math.abs(anchorBarStart(a, x.t.seconds, x.t.style) - free);return !best || gap < best.gap ? { a, gap } : best;}, null);const snapped = near && near.gap <= XBAR_MAGNET_PX / pps ? near.a : null;landing = snapped- ? { start: anchorBarStart(snapped, x.t.seconds), anchor: snapped }+ ? { start: anchorBarStart(snapped, x.t.seconds, x.t.style), anchor: snapped }: { start: free, anchor: null };setSnapX(snapped ? snapped.at * pps : null);setJointDrop(snapped ? { ...snapped, len: x.t.seconds } : null);@@ -1249,15 +1259,23 @@ export function Timeline() {const st0 = useEditor.getState();if (st0.readOnly) return;const growsForward = x.role?.kind === "in";- const fixed = growsForward ? x.t.start : x.t.start + x.t.seconds;+ // A cross dissolve sits across its cut, so retiming holds the middle and+ // both halves grow together; every other bar holds the boundary end.+ const straddles = isAudioTransition(x.t.style);+ const fixed = growsForward+ ? x.t.start+ : straddles+ ? x.t.start + x.t.seconds / 2+ : x.t.start + x.t.seconds;st0.beginHistoryBatch();startDrag(e, {onMove: (dx) => {const loose = (growsForward ? x.t.start + x.t.seconds : x.t.start) + dx / pps;- const seconds = Math.max(0.1, Math.min(TRANSITION_MAX, Math.abs(loose - fixed)));+ const reach = Math.abs(loose - fixed) * (straddles ? 2 : 1);+ const seconds = Math.max(0.1, Math.min(TRANSITION_MAX, reach));useEditor.getState().updateTransitionTransient(x.t.id, {seconds,- start: growsForward ? fixed : fixed - seconds,+ start: growsForward ? fixed : fixed - (straddles ? seconds / 2 : seconds),});},onUp: () => useEditor.getState().endHistoryBatch(),@@ -1276,9 +1294,10 @@ export function Timeline() {// The place a dragged transition would land on, marked with the footprint it// would take while it is in flight.const [jointDrop, setJointDrop] = useDropPreview<Anchor | null>(null);- // A transition tile from the panel is over the timeline: the row lights- // every place a drop could play, the same as a bar drag does.- const [xTileDrag, setXTileDrag] = useState(false);+ // A transition tile from the panel is over the timeline, and which style it+ // carries: the row lights every place a drop could play, the same as a bar+ // drag does, each zone sized and placed the way that style lands.+ const [xTileDrag, setXTileDrag] = useState<TransitionStyle | null>(null);// A transition bar mid-drag, drawn where the pointer has it.const [transitionDrag, setTransitionDrag] = useState<XBar | null>(null);// A drag past the top edge opens a row there, pushing the stack down by one@@ -1812,7 +1831,7 @@ export function Timeline() {}: null);- setXTileDrag(element?.kind === "transition");+ setXTileDrag(element?.kind === "transition" ? element.style : null);setJointDrop(element?.kind === "transition"? nearestAnchor(Math.max(0, timeAt(e.clientX)))@@ -1822,7 +1841,7 @@ export function Timeline() {}setElementDrop(null);setJointDrop(null);- setXTileDrag(false);+ setXTileDrag(null);// Preview where a video would land; audio drops free-form. Library and// stock drags carry their own shape since they aren't in the project yet.let type: "video" | "audio" | "image" | undefined;@@ -1893,7 +1912,7 @@ export function Timeline() {setDropType(null);setElementDrop(null);setJointDrop(null);- setXTileDrag(false);+ setXTileDrag(null);}}}onDrop={(e) => {@@ -1902,7 +1921,7 @@ export function Timeline() {const elementLane = overlayLaneAt(e.clientY);setElementDrop(null);setJointDrop(null);- setXTileDrag(false);+ setXTileDrag(null);const videoPlace = resolveDropTrack(e.clientX, e.clientY);setAssetDrop(null);endCrossDrag();@@ -2381,13 +2400,14 @@ export function Timeline() {// per cut and open edge, each the size the bar would take// there.anchors.map((a) => {- const len = transitionDrag ? transitionDrag.t.seconds : a.len;+ const style = transitionDrag?.t.style ?? xTileDrag ?? "crossfade";+ const len = transitionDrag ? transitionDrag.t.seconds : anchorLen(a, style);return (<divkey={`xzone-${a.kind}-${a.clipId}`}className="pointer-events-none absolute rounded-md bg-[#0a84ff]/10 shadow-[inset_0_0_0_1.5px_rgba(10,132,255,0.4)]"style={{- left: anchorBarStart(a, len) * pps,+ left: anchorBarStart(a, len, style) * pps,top: 2,width: Math.max(14, len * pps - CLIP_GAP),height: TEXT_H - 6,@@ -2395,18 +2415,23 @@ export function Timeline() {/>);})}- {jointDrop && (- // The zone the drop is aimed at burns brighter than the rest.- <div- className="pointer-events-none absolute rounded-md bg-[#0a84ff]/25 shadow-[inset_0_0_0_1.5px_rgba(10,132,255,0.85)]"- style={{- left: anchorBarStart(jointDrop, jointDrop.len) * pps,- top: 2,- width: Math.max(14, jointDrop.len * pps - CLIP_GAP),- height: TEXT_H - 6,- }}- />- )}+ {jointDrop &&+ (() => {+ // The zone the drop is aimed at burns brighter than the rest.+ const style = transitionDrag?.t.style ?? xTileDrag ?? "crossfade";+ const len = anchorLen(jointDrop, style);+ return (+ <div+ className="pointer-events-none absolute rounded-md bg-[#0a84ff]/25 shadow-[inset_0_0_0_1.5px_rgba(10,132,255,0.85)]"+ style={{+ left: anchorBarStart(jointDrop, len, style) * pps,+ top: 2,+ width: Math.max(14, len * pps - CLIP_GAP),+ height: TEXT_H - 6,+ }}+ />+ );+ })()}{transitions.map((live) => {const x = transitionDrag?.t.id === live.t.id ? transitionDrag : live;const Icon = TRANSITION_ICONS[x.t.style];diff --git a/site/src/cut/components/TransitionsPanel.tools.ts b/site/src/cut/components/TransitionsPanel.tools.tsindex f7342652..d9f0a722 100644--- a/site/src/cut/components/TransitionsPanel.tools.ts+++ b/site/src/cut/components/TransitionsPanel.tools.ts@@ -14,7 +14,7 @@ export const TRANSITIONS_TOOLS = [{name: "set_transition",description:- "Set the transition from this clip into the next clip on its track (any video track), in seconds (0 clears it, max 2). A transition is a blend across the cut — it never moves, trims or overlaps clips, and it plays only while the pair touches. On upper tracks every style blends as an alpha dissolve (the tracks beneath show through). Only valid when a next same-track clip exists. Each edge holds one effect: setting a transition clears the animations adjacent to its joint (and set_animation on that edge replaces the transition). Read the transitions-and-fades skill before styling cuts.",+ "Set the transition from this clip into the next clip on its track (any video track), in seconds (0 clears it, max 2). 0.5s suits a blend on the picture; the cross dissolve (audiocross) wants about 0.3s, since it is centered on the cut and both clips sound through it. A transition is a blend across the cut — it never moves, trims or overlaps clips, and it plays only while the pair touches. On upper tracks every style blends as an alpha dissolve (the tracks beneath show through). Only valid when a next same-track clip exists. Each edge holds one effect: setting a transition clears the animations adjacent to its joint (and set_animation on that edge replaces the transition). Read the transitions-and-fades skill before styling cuts.",inputSchema: obj({clipId: str("Video clip id (the clip the transition starts from)"),seconds: num("Transition length in seconds, 0–2 (0 = hard cut)"),diff --git a/site/src/cut/hooks/usePlayback.ts b/site/src/cut/hooks/usePlayback.tsindex bd707b4e..c3a8d39e 100644--- a/site/src/cut/hooks/usePlayback.ts+++ b/site/src/cut/hooks/usePlayback.ts@@ -9,11 +9,11 @@ import {useEditor,} from "@/cut/lib/store";import { playheadAt, previewAt, setPlayhead, subscribePlayhead } from "@/cut/lib/playhead";-import { clipCovers, projectFadeSeconds, rectOf } from "@/cut/lib/types";+import { assetIsSilent, clipCovers, projectFadeSeconds, rectOf } from "@/cut/lib/types";import type { ClipSpan, MediaAsset, VideoClip } from "@/cut/lib/types";import { SubjectMaskCompositor } from "@/cut/lib/behindPass";import { FrameCompositor, MISSING_FRAME, PENDING_FRAME, type Frame } from "@/cut/lib/composite";-import { duckGainAt, overlayPlan, soundCrossGain, trackZeroPlan } from "@/cut/lib/framePlan";+import { crossHandles, duckGainAt, overlayPlan, soundCrossGain, trackZeroPlan } from "@/cut/lib/framePlan";import { type ClipFrameSource, FrameSourcePool, mappingKey, walkCostMs } from "@/cut/lib/frameSource";import { liveAudioFxAt } from "@/cut/lib/audioEffects";import { PreviewMixer, type Voice } from "@/cut/lib/previewMixer";@@ -88,6 +88,27 @@ export function engineLog(msg: string): void {if (log.length > 500) log.splice(0, log.length - 500);}+/**+ * The source span a clip's voice plays, and where it lands.+ *+ * A clip normally plays exactly what it is trimmed to. A cross dissolve widens+ * that: the clip reaches into the handle on either side — source it was+ * trimmed away from — so it is still sounding after the picture has cut, or+ * already sounding before it. The widened span rides the voice for the clip's+ * whole life rather than only inside the handover, so the mixer decodes and+ * schedules one stable voice and the frame-by-frame gain is all that moves.+ */+const voiceSpan = (sp: ClipSpan) => {+ const speed = clipSpeed(sp.clip);+ return {+ url: sp.asset.url,+ start: sp.start - sp.soundBack,+ in: sp.clip.in - sp.soundBack * speed,+ out: sp.clip.out + sp.soundAhead * speed,+ speed,+ };+};+class Engine {/** Which engine this is, for the perf eval's pool dump. */readonly serial = ++engineSerial;@@ -284,6 +305,15 @@ class Engine {);}+ /** Each upper track's spans, for the sound side — a cross dissolve up there+ * crosses the same way track 0's does. */+ private overlaySpans(): ClipSpan[][] {+ const s = useEditor.getState();+ return [...new Set(overlayLayers(s.clips).map((c) => c.track))].map((track) =>+ getClipSpans(s.clips, s.assets, track)+ );+ }+/** Whole-video fade gain at `t`: ramps 0→1 over the project fade-in and 1→0* over the fade-out at the end of the cut. */private projectFadeGain(t: number, total: number) {@@ -301,40 +331,49 @@ class Engine {** The same ramps that dim the picture dim the sound: a clip fading out of a* dissolve takes its audio with it, an upper-track clip's transition carries- * its own, and a live voiceover ducks the rest. A sound dissolve is the- * ramp on its own — the picture cuts and only these gains move.+ * its own, and a live voiceover ducks the rest. A cross dissolve is the+ * ramp on its own — the picture cuts and only these gains move — and it is+ * the one thing here that makes a clip audible outside its own footprint,+ * so both sides of the crossing are really playing at once.*/private voicesAt(t: number, spans: ClipSpan[], master: ClipSpan | undefined): Voice[] {const s = useEditor.getState();const out: Voice[] = [];const duck = duckGainAt(s.audioClips, t);- if (master && !master.clip.muted && !master.clip.hidden) {+ if (master && !master.clip.muted && !master.clip.hidden && !assetIsSilent(master.asset)) {const plan = trackZeroPlan(master, spans, t);out.push({id: master.clip.id,- url: master.asset.url,- start: master.start,- in: master.clip.in,- out: master.clip.out,- speed: clipSpeed(master.clip),+ ...voiceSpan(master),gain: plan.gain * soundCrossGain(spans, master, t) * duck * (master.clip.volume ?? 1),});}- for (const { clip, asset, gain } of this.liveOverlays(t)) {- if (clip.muted || clip.hidden || asset.type === "image") continue;+ // The other half of a crossing: a neighbour playing into its handle, past+ // its own footprint, carried by the cross ramp alone.+ for (const spans0 of [spans, ...this.overlaySpans()]) {+ for (const { span, gain } of crossHandles(spans0, t)) {+ if (assetIsSilent(span.asset)) continue;+ // The clip's own id, not a second voice: the handle is the same voice+ // carrying on past the cut, and a new id would stop and re-open it+ // there — a decode gap in the middle of the crossing.+ out.push({+ id: span.clip.id,+ ...voiceSpan(span),+ gain: gain * (span.clip.volume ?? 1) * duck,+ });+ }+ }+ for (const { span, clip, asset, gain } of this.liveOverlays(t)) {+ if (clip.muted || clip.hidden || assetIsSilent(asset)) continue;out.push({id: clip.id,- url: asset.url,- start: clip.start,- in: clip.in,- out: clip.out,- speed: clipSpeed(clip),+ ...voiceSpan(span),gain: gain * (clip.volume ?? 1) * duck,});}for (const a of s.audioClips) {const asset = s.assets.find((x) => x.id === a.assetId);- if (!asset || a.hidden) continue;+ if (!asset || a.hidden || assetIsSilent(asset)) continue;const speed = a.speed && a.speed > 0 ? a.speed : 1;const len = Math.max(0.1, (a.out - a.in) / speed);if (t < a.start || t >= a.start + len) continue;@@ -419,15 +458,7 @@ class Engine {/** Draw the overlay tracks over track 0, further-back first. */private drawOverlays(t: number, playing: boolean) {- for (const { clip, asset, alpha, zoom } of this.liveOverlays(t)) {- const span: ClipSpan = {- clip,- asset,- start: clip.start,- len: Math.max(0.1, (clip.out - clip.in) / clipSpeed(clip)),- transitionOut: 0,- soundOut: 0,- };+ for (const { span, clip, alpha, zoom } of this.liveOverlays(t)) {const frame = this.frameFor(span, t, playing);if (frame.kind !== "ready") continue;this.comp.drawIntoRect(frame, rectOf(clip), clipCovers(clip), alpha, t, zoom, clip);diff --git a/site/src/cut/lib/audioFxAudition.ts b/site/src/cut/lib/audioFxAudition.tsindex c4c14382..f53ec222 100644--- a/site/src/cut/lib/audioFxAudition.ts+++ b/site/src/cut/lib/audioFxAudition.ts@@ -10,8 +10,9 @@* with the site, so the tiles play the same way in a project with no footage* in it yet — a picker is used before there is anything to use it on.*- * The sound dissolve auditions as what it is: a man's line handing over to a- * woman's across a cut, with the ramps the render would apply.+ * The cross dissolve auditions as what it is: a man's line handing over to a+ * woman's across a cut — both speaking through the join, on the equal-power+ * ramps the render applies.** A clip is fetched once and kept decoded for the session, so the first press* loads it and every press after plays it straight away. One audition at a@@ -21,11 +22,12 @@import { audioFxRecipe, buildAudioFx, type AudioFxNodes } from "@donkeycut/effects-kit";import { create } from "zustand";+import { scheduleCross } from "./audioMix";import { registerPreviewSilencer, usePreviewAudio } from "./previewAudio";import { voiceSampleUrl } from "./voices";/** The id of the sound handover, auditioned as the handover itself. */-export const SOUND_DISSOLVE_AUDITION = "audiocross";+export const CROSS_DISSOLVE_AUDITION = "audiocross";/*** The two lines the shelf plays.@@ -46,7 +48,8 @@ const LINES = {/** Where the sentences sit in those clips, from the silence between them. */const FIRST_SENTENCE = { at: 0.2, len: 1.55 };const SECOND_SENTENCE = { at: 2.0, len: 2.5 };-/** How much of the handover's two sides cross over. */+/** Half the handover: the two lines overlap for twice this, with the cut in+ * the middle of it. */const CROSS_WINDOW = 0.6;interface AuditionState {@@ -130,17 +133,13 @@ export const useFxAudition = create<AuditionState>((set, get) => ({part: { at: number; len: number },when: number,into: AudioNode,- fade?: "in" | "out"+ cross?: { from: number; rising: boolean }) => {const node = target.createBufferSource();node.buffer = buffer;const gain = target.createGain();- if (fade === "in") {- gain.gain.setValueAtTime(0, when);- gain.gain.linearRampToValueAtTime(1, when + CROSS_WINDOW);- } else if (fade === "out") {- gain.gain.setValueAtTime(1, when + Math.max(0, part.len - CROSS_WINDOW));- gain.gain.linearRampToValueAtTime(0, when + part.len);+ if (cross) {+ scheduleCross(gain.gain, 1, cross.from, cross.from + 2 * CROSS_WINDOW, cross.rising);}node.connect(gain);gain.connect(into);@@ -149,22 +148,20 @@ export const useFxAudition = create<AuditionState>((set, get) => ({return node;};- if (effect === SOUND_DISSOLVE_AUDITION) {+ if (effect === CROSS_DISSOLVE_AUDITION) {void Promise.all([clip(target, LINES.man), clip(target, LINES.woman)]).then(([man, woman]) => {if (token !== mine) return;const at = target.currentTime + 0.05;- // What the render does: the outgoing line ramps down into the cut,- // the incoming one ramps up out of it, and the picture cuts where- // the two meet.- play(man, FIRST_SENTENCE, at, target.destination, "out");- const second = play(- woman,- SECOND_SENTENCE,- at + FIRST_SENTENCE.len,- target.destination,- "in"- );+ // What the render does: both lines are speaking across the join,+ // ramping past each other equal-power, with the picture's cut in+ // the middle of the window where they are equally loud.+ const from = at + FIRST_SENTENCE.len - 2 * CROSS_WINDOW;+ play(man, FIRST_SENTENCE, at, target.destination, { from, rising: false });+ const second = play(woman, SECOND_SENTENCE, from, target.destination, {+ from,+ rising: true,+ });second.onended = done;}).catch(done);diff --git a/site/src/cut/lib/audioMix.test.ts b/site/src/cut/lib/audioMix.test.tsindex fdaab38f..e2cb9d1d 100644--- a/site/src/cut/lib/audioMix.test.ts+++ b/site/src/cut/lib/audioMix.test.ts@@ -34,18 +34,22 @@ describe("foldClips", () => {expect(geo[1].fadeIn).toBe(0);});- test("a sound dissolve crosses the cut: one clip out, the next one in", () => {+ test("a cross dissolve crosses the cut: one clip out, the next one in", () => {const geo = foldClips([clip({ soundCross: 1 }), clip()]);- expect(geo[0].fadeOut).toBe(1);- expect(geo[1].fadeIn).toBe(1);+ // A crossing is not a fade — it rides its own pair of ramps, which reach+ // half level on the cut instead of silence.+ expect(geo[0].crossOut).toBe(1);+ expect(geo[1].crossIn).toBe(1);+ expect(geo[0].fadeOut).toBe(0);+ expect(geo[1].fadeIn).toBe(0);// The picture cut claims no layout here either.expect(geo[1].at).toBe(4);});- test("a sound dissolve is clamped by both clips it crosses", () => {+ test("a cross dissolve is clamped by both clips it crosses", () => {const geo = foldClips([clip({ out: 1, soundCross: 3 }), clip({ out: 2 })]);- expect(geo[0].fadeOut).toBeCloseTo(0.9, 5);- expect(geo[1].fadeIn).toBeCloseTo(0.9, 5);+ expect(geo[0].crossOut).toBeCloseTo(0.9, 5);+ expect(geo[1].crossIn).toBeCloseTo(0.9, 5);});test("scales a blend down so it cannot swallow its clip", () => {diff --git a/site/src/cut/lib/audioMix.ts b/site/src/cut/lib/audioMix.tsindex fb002b0a..b2d8c519 100644--- a/site/src/cut/lib/audioMix.ts+++ b/site/src/cut/lib/audioMix.ts@@ -33,10 +33,16 @@ export interface MixClip {volume?: number;/** Cross-dissolve overlap into the next clip, timeline seconds. */transition?: number;- /** Sound dissolve into the next clip, timeline seconds: this clip fades out- * over the window before the cut and the next one fades in over the window- * after it. Set instead of `transition` when the handover is on the sound. */+ /** Half the cross dissolve into the next clip, timeline seconds: the two+ * clips ramp equal-power past each other over that long either side of the+ * cut. Set instead of `transition` when the handover is on the sound. */soundCross?: number;+ /** Source seconds past `out` this clip keeps sounding, into the crossing at+ * its tail, and source seconds before `in` it starts sounding, for the+ * crossing at its head. Both are handle — media the trim left behind — and+ * are what make the two clips audible at once over a hard picture cut. */+ soundAhead?: number;+ soundBack?: number;}/** A clip placed at an absolute time: an upper-track video's audio, a@@ -48,6 +54,13 @@ export interface MixItem {start: number;volume: number;speed?: number;+ /** Half the cross dissolve at this item's head and tail, with the handle+ * each side reaches into — an upper-track cut crosses the same way track+ * 0's does. */+ crossIn?: number;+ crossOut?: number;+ soundAhead?: number;+ soundBack?: number;muted?: boolean;fadeIn?: number;fadeOut?: number;@@ -99,11 +112,21 @@ const itemDur = (a: MixItem) => Math.max(0, (a.out - a.in) / speedOf(a.speed));* sound is the outgoing clip fading across the blend window while the next* enters clean at the cut, exactly like the engine's tail-fade + hard join.*- * A sound dissolve is the same fade out with the next clip's fade in against- * it: the picture cuts and the sound crosses over the cut.+ * A cross dissolve is the other shape: the two clips ramp past each other+ * across the cut, each reaching into its handle so both are really sounding+ * while the picture hard-joins. Those ramps are equal-power and are kept apart+ * from the fades, which are linear and end at silence.*/export function foldClips(clips: MixClip[]) {- const geo = clips.map((clip) => ({ clip, at: 0, dur: clipDur(clip), fadeIn: 0, fadeOut: 0 }));+ const geo = clips.map((clip) => ({+ clip,+ at: 0,+ dur: clipDur(clip),+ fadeIn: 0,+ fadeOut: 0,+ crossIn: 0,+ crossOut: 0,+ }));let acc = 0;geo.forEach((g, j) => {g.at = acc;@@ -114,13 +137,33 @@ export function foldClips(clips: MixClip[]) {if (fade > 0.01) g.fadeOut = fade;const cross = Math.min(g.clip.soundCross ?? 0, g.dur * 0.9, next.dur * 0.9);if (cross > 0.01) {- g.fadeOut = Math.max(g.fadeOut, cross);- next.fadeIn = Math.max(next.fadeIn, cross);+ g.crossOut = cross;+ next.crossIn = cross;}});return geo;}+/** How far each side of a clip reaches into its handle for a crossing, capped+ * by the ramp that crossing actually got. */+const handles = (g: { clip: MixClip; crossIn: number; crossOut: number }) => ({+ // Never past the head of the source: a handle is what the trim left behind.+ back: Math.min(g.clip.soundBack ?? 0, g.crossIn, g.clip.in / speedOf(g.clip.speed)),+ ahead: Math.min(g.clip.soundAhead ?? 0, g.crossOut),+});++/** The source span an entry decodes: its own trim, widened by the handle each+ * crossing at its edges reaches into. Timeline seconds scale by speed on the+ * way into the source. */+const crossSpan = (+ c: { file: string; in: number; out: number; speed?: number },+ back = 0,+ ahead = 0+) => {+ const speed = speedOf(c.speed);+ return { file: c.file, in: c.in - back * speed, out: c.out + ahead * speed };+};+/*** The windows during which a ducking item is audible, merged into a single* envelope: each entry is a stretch of timeline where everything that is not@@ -149,6 +192,34 @@ export function duckWindows(items: MixItem[]): { start: number; end: number; gaireturn merged;}+/**+ * Schedule one side of a cross dissolve: an equal-power ramp between silence+ * and `level` across `[from, to]`, rising for the incoming clip and falling+ * for the outgoing one. The two meet at 0.707 in the middle of the window —+ * the cut — which is what keeps the mix at one loudness through the join.+ *+ * Written as short linear steps rather than `setValueCurveAtTime`, which is+ * not carried by every context this fold runs in (the headless renderer's+ * among them).+ */+const CROSS_STEPS = 24;++export function scheduleCross(+ param: AudioParam,+ level: number,+ from: number,+ to: number,+ rising: boolean+) {+ for (let i = 0; i <= CROSS_STEPS; i++) {+ const p = i / CROSS_STEPS;+ const v = level * (rising ? Math.sin((p * Math.PI) / 2) : Math.cos((p * Math.PI) / 2));+ const at = Math.max(0, from + (to - from) * p);+ if (i === 0) param.setValueAtTime(v, at);+ else param.linearRampToValueAtTime(v, at);+ }+}+/** Schedule a duck envelope onto a gain node: 1 everywhere except inside the* windows, where it steps down and back. The steps are ramps a few* milliseconds long so the drop doesn't click. */@@ -181,16 +252,25 @@ export async function renderMix(spec: MixSpec, opts: MixOptions): Promise<AudioB// ten-second trim of a ninety-minute recording pulling the entire recording// into memory — around 1.4 GB an hour at 48 kHz stereo — and several of them// at once.+ // The track-0 fold, with the source span each clip really decodes: a+ // crossing at either edge widens it into the handle it reaches for.+ const folded = foldClips(spec.clips).map((g) => {+ const { back, ahead } = handles(g);+ return { ...g, back, ahead, span: crossSpan(g.clip, back, ahead) };+ });+const wanted = new Map<string, { file: string; from: number; to: number }>();const spanKey = (c: { file: string; in: number; out: number }) =>`${c.file}|${c.in.toFixed(3)}|${c.out.toFixed(3)}`;- for (const c of spec.clips) {- if (!c.file || c.muted) continue;- wanted.set(spanKey(c), { file: c.file, from: Math.max(0, c.in), to: c.out });+ for (const c of folded) {+ if (!c.clip.file || c.clip.muted) continue;+ const span = c.span;+ wanted.set(spanKey(span), { file: span.file, from: Math.max(0, span.in), to: span.out });}for (const a of spec.items) {if (!a.file || a.muted) continue;- wanted.set(spanKey(a), { file: a.file, from: Math.max(0, a.in), to: a.out });+ const span = crossSpan(a, a.soundBack, a.soundAhead);+ wanted.set(spanKey(span), { file: span.file, from: Math.max(0, span.in), to: span.out });}const buffers = new Map<string, AudioBuffer>();@@ -313,43 +393,60 @@ export async function renderMix(spec: MixSpec, opts: MixOptions): Promise<AudioBsrc.start(Math.max(0, at), 0, Math.max(0, duration));};- for (const g of foldClips(spec.clips)) {+ for (const g of folded) {const c = g.clip;- const buf = !c.muted && c.file ? buffers.get(spanKey(c)) : undefined;+ const key = spanKey(g.span);+ const buf = !c.muted && c.file ? buffers.get(key) : undefined;if (!buf) continue; // muted, silent, or a gap spacer: only shapes time- play(fitted(spanKey(c), buf, speedOf(c.speed)), g.at, g.dur, ducked, (gain) => {+ const from = g.at - g.back;+ const end = g.at + g.dur;+ play(fitted(key, buf, speedOf(c.speed)), from, g.dur + g.back + g.ahead, ducked, (gain) => {const level = Math.max(0, c.volume ?? 1);gain.gain.value = level;- if (g.fadeIn > 0) {+ // A crossing and a fade never share an edge — the cut carries one+ // handover — so each side takes whichever it has.+ if (g.crossIn > 0) {+ scheduleCross(gain.gain, level, g.at - g.crossIn, g.at + g.crossIn, true);+ } else if (g.fadeIn > 0) {gain.gain.setValueAtTime(0, g.at);gain.gain.linearRampToValueAtTime(level, g.at + g.fadeIn);}- if (g.fadeOut > 0) {- gain.gain.setValueAtTime(level, g.at + g.dur - g.fadeOut);- gain.gain.linearRampToValueAtTime(0, g.at + g.dur);+ if (g.crossOut > 0) {+ scheduleCross(gain.gain, level, end - g.crossOut, end + g.crossOut, false);+ } else if (g.fadeOut > 0) {+ gain.gain.setValueAtTime(level, end - g.fadeOut);+ gain.gain.linearRampToValueAtTime(0, end);}});}for (const a of spec.items) {- const buf = a.muted ? undefined : buffers.get(spanKey(a));+ const back = Math.min(a.soundBack ?? 0, a.crossIn ?? 0);+ const ahead = Math.min(a.soundAhead ?? 0, a.crossOut ?? 0);+ const key = spanKey(crossSpan(a, back, ahead));+ const buf = a.muted ? undefined : buffers.get(key);if (!buf) continue;const dur = itemDur(a);+ const end = a.start + dur;// A voiceover that ducks everything else is not itself ducked; it still// lands on the bus, so the effect elements treat it with the rest.const into = a.duck !== undefined && a.duck < 1 ? bus : ducked;- play(fitted(spanKey(a), buf, speedOf(a.speed)), a.start, dur, into, (gain) => {+ play(fitted(key, buf, speedOf(a.speed)), a.start - back, dur + back + ahead, into, (gain) => {const level = Math.max(0, a.volume);gain.gain.value = level;const fIn = Math.min(a.fadeIn ?? 0, dur);const fOut = Math.min(a.fadeOut ?? 0, dur);- if (fIn > 0) {+ if (a.crossIn) {+ scheduleCross(gain.gain, level, a.start - a.crossIn, a.start + a.crossIn, true);+ } else if (fIn > 0) {gain.gain.setValueAtTime(0, a.start);gain.gain.linearRampToValueAtTime(level, a.start + fIn);}- if (fOut > 0) {- gain.gain.setValueAtTime(level, a.start + dur - fOut);- gain.gain.linearRampToValueAtTime(0, a.start + dur);+ if (a.crossOut) {+ scheduleCross(gain.gain, level, end - a.crossOut, end + a.crossOut, false);+ } else if (fOut > 0) {+ gain.gain.setValueAtTime(level, end - fOut);+ gain.gain.linearRampToValueAtTime(0, end);}});}diff --git a/site/src/cut/lib/audioMixCross.test.ts b/site/src/cut/lib/audioMixCross.test.tsnew file mode 100644index 00000000..7d5a6e8d--- /dev/null+++ b/site/src/cut/lib/audioMixCross.test.ts@@ -0,0 +1,89 @@+import { describe, expect, mock, test } from "bun:test";+import { AudioBuffer, OfflineAudioContext } from "node-web-audio-api";++// The offline fold's side of a cross dissolve, on the Web Audio the headless+// runtime installs. Each clip reads back a steady tone of its own frequency,+// so the two are uncorrelated the way two shots are and the mix's level is+// the only thing the crossing can move.+const RATE = 44100;++const globals = globalThis as Record<string, unknown>;+globals.AudioBuffer ??= AudioBuffer;+globals.OfflineAudioContext ??= OfflineAudioContext;++const tone = (seconds: number, hz: number) => {+ const buf = new AudioBuffer({+ length: Math.max(1, Math.round(seconds * RATE)),+ numberOfChannels: 1,+ sampleRate: RATE,+ });+ const ch = buf.getChannelData(0);+ for (let i = 0; i < ch.length; i++) ch[i] = Math.sin((2 * Math.PI * hz * i) / RATE);+ return buf;+};++mock.module("./mediaRead", () => ({+ decodeAudioSpan: async (file: string, from: number, to: number) =>+ tone(to - from, file === "a.mp4" ? 440 : 880),+}));++const { renderMix } = await import("./audioMix");++const CUT = 4;++/** The mix of two four-second clips joined by a crossing that reaches+ * `handle` seconds into each clip's trimmed-away media. */+const crossed = async (half: number, handle: number) => {+ const buffer = await renderMix(+ {+ duration: 8,+ clips: [+ {+ file: "a.mp4",+ in: 1,+ out: 5,+ muted: false,+ soundCross: half,+ soundAhead: handle,+ },+ { file: "b.mp4", in: 1, out: 5, muted: false, soundBack: handle },+ ],+ items: [],+ },+ { sampleRate: RATE, channels: 1, resolve: (f) => f }+ );+ expect(buffer).not.toBeNull();+ return Float32Array.from(buffer!.getChannelData(0));+};++const level = (data: Float32Array, at: number) => {+ const a = Math.floor((at - 0.01) * RATE);+ const b = Math.floor((at + 0.01) * RATE);+ let sum = 0;+ for (let i = a; i < b; i++) sum += data[i] * data[i];+ return Math.sqrt(sum / Math.max(1, b - a));+};++describe("a cross dissolve in the offline fold", () => {+ test("holds one level across the cut when both clips have handle", async () => {+ const mix = await crossed(0.4, 0.4);+ const steady = level(mix, 1);+ // Through the whole crossing the mix stays where it was: the two clips+ // are both sounding, and equal-power ramps keep their sum flat.+ for (const t of [CUT - 0.3, CUT - 0.1, CUT, CUT + 0.1, CUT + 0.3]) {+ expect(level(mix, t) / steady).toBeGreaterThan(0.9);+ expect(level(mix, t) / steady).toBeLessThan(1.1);+ }+ expect(level(mix, 7) / steady).toBeGreaterThan(0.9);+ });++ test("dips rather than empties when neither clip has handle", async () => {+ const mix = await crossed(0.4, 0);+ const steady = level(mix, 1);+ // Nothing to cross with, so the cut is the bottom of the ramps — but they+ // meet at equal power's half, not at silence.+ const middle = level(mix, CUT - 0.005) / steady;+ expect(middle).toBeGreaterThan(0.6);+ expect(middle).toBeLessThan(0.8);+ });+});diff --git a/site/src/cut/lib/exportClient.ts b/site/src/cut/lib/exportClient.tsindex ad3f432b..fdc1fe01 100644--- a/site/src/cut/lib/exportClient.ts+++ b/site/src/cut/lib/exportClient.ts@@ -491,9 +491,12 @@ async function buildExportPayload(frame: sp.clip.frame,speed: clipSpeed(sp.clip),transition: sp.transitionOut,- // A sound dissolve carries its own window: the picture cuts, so the- // server's blend must not see one.+ // A cross dissolve carries its own window: the picture cuts, so the+ // server's blend must not see one. The handles are what the crossing+ // reaches into so both clips are audible over that cut.soundCross: sp.soundOut,+ soundAhead: sp.soundAhead,+ soundBack: sp.soundBack,// The style rides along with the overlap; the server resolves it to an// xfade name (and the cross-zoom ramps) itself, so the spec carries only// the id.@@ -610,6 +613,8 @@ async function buildExportPayload(// so its windows ride apart from the alpha fades.headSound: 0,tailSound: 0,+ soundAhead: 0,+ soundBack: 0,}));trackSpans.forEach((sp, i) => {const r = ramps[i];@@ -638,6 +643,8 @@ async function buildExportPayload(r.tailSound = Math.max(r.tailSound, sp.soundOut);ramps[i + 1].headSound = Math.max(ramps[i + 1].headSound, sp.soundOut);}+ r.soundAhead = sp.soundAhead;+ r.soundBack = sp.soundBack;});return trackSpans.map((sp, i) => ({ c: sp.clip, ramp: ramps[i] }))diff --git a/site/src/cut/lib/exportRender.ts b/site/src/cut/lib/exportRender.tsindex 22c7d0ca..cd277acf 100644--- a/site/src/cut/lib/exportRender.ts+++ b/site/src/cut/lib/exportRender.ts@@ -38,7 +38,7 @@ import { applyEffectToCanvas, evalOverlayFrame, grainTile, isAudioEffect, isMaskimport { hasSubjectOverlays, SubjectMaskCompositor } from "./behindPass";import { createRasterCanvas, type RasterSurface } from "./raster";import { renderElementPng } from "./textRender";-import { behindSubjectOverlay, clipCovers, frameOf, frontSubjectOverlay, isEffectOverlay, isTextOverlay, laneOf, overlayAnimStyle, projectBackground, projectFadeSeconds, rectOf } from "./types";+import { assetIsSilent, behindSubjectOverlay, clipCovers, frameOf, frontSubjectOverlay, isEffectOverlay, isTextOverlay, laneOf, overlayAnimStyle, projectBackground, projectFadeSeconds, rectOf } from "./types";import type { ClipSpan, EffectOverlay, MediaAsset, Overlay, StickerOverlay } from "./types";import type { ExportDoc, ExportSettings } from "./exportClient";@@ -436,8 +436,10 @@ class StampCache {* animation, or the transition that owns that edge. Mirrors the ramps the* export spec carries and the gain the preview applies, so a picture that* fades up takes its sound with it. */-function overlayRamps(spans: ClipSpan[]): { head: number; tail: number }[] {- const ramps = spans.map(() => ({ head: 0, tail: 0 }));+function overlayRamps(+ spans: ClipSpan[]+): { head: number; tail: number; crossIn: number; crossOut: number }[] {+ const ramps = spans.map(() => ({ head: 0, tail: 0, crossIn: 0, crossOut: 0 }));spans.forEach((sp, i) => {const apply = (anim: typeof sp.clip.animIn, side: "head" | "tail") => {if (!anim) return;@@ -452,10 +454,12 @@ function overlayRamps(spans: ClipSpan[]): { head: number; tail: number }[] {ramps[i].tail = Math.max(ramps[i].tail, sp.transitionOut);ramps[i + 1].head = Math.max(ramps[i + 1].head, sp.transitionOut);}- // A sound dissolve is these ramps and nothing else: the picture cuts.+ // A cross dissolve is these ramps and nothing else: the picture cuts.+ // They stay off `head`/`tail` — a fade ends at silence, a crossing ramps+ // equal-power past the clip on the other side of the cut.if (sp.soundOut > 0 && spans[i + 1]) {- ramps[i].tail = Math.max(ramps[i].tail, sp.soundOut);- ramps[i + 1].head = Math.max(ramps[i + 1].head, sp.soundOut);+ ramps[i].crossOut = sp.soundOut;+ ramps[i + 1].crossIn = sp.soundOut;}});return ramps;@@ -474,10 +478,6 @@ export function mixSpecFor(doc: ExportDoc, resolve: (asset: MediaAsset) => strinconst duration = projectDuration(doc);const spans = getClipSpans(doc.clips, doc.assets, 0);const byId = new Map(doc.assets.map((a) => [a.id, a]));- // A still has no sound, and handing its URL to the audio reader would fail on- // a file that is not a media container at all.- const silent = (asset: MediaAsset) => asset.type === "image";-const items: MixItem[] = [];for (const track of new Set(overlayLayers(doc.clips).map((c) => c.track))) {const trackSpans = getClipSpans(doc.clips, doc.assets, track);@@ -491,15 +491,19 @@ export function mixSpecFor(doc: ExportDoc, resolve: (asset: MediaAsset) => strinstart: sp.start,volume: sp.clip.volume ?? 1,speed: sp.clip.speed,- muted: sp.clip.muted || silent(sp.asset),+ muted: sp.clip.muted || assetIsSilent(sp.asset),fadeIn: ramps[i].head,fadeOut: ramps[i].tail,+ crossIn: ramps[i].crossIn,+ crossOut: ramps[i].crossOut,+ soundBack: sp.soundBack,+ soundAhead: sp.soundAhead,});});}for (const a of doc.audioClips) {const asset = byId.get(a.assetId);- if (!asset || a.hidden || a.start >= duration) continue;+ if (!asset || a.hidden || a.start >= duration || assetIsSilent(asset)) continue;items.push({file: resolve(asset),in: a.in,@@ -523,11 +527,13 @@ export function mixSpecFor(doc: ExportDoc, resolve: (asset: MediaAsset) => strinfile: resolve(sp.asset),in: sp.clip.in,out: sp.clip.out,- muted: sp.clip.muted || !!sp.clip.hidden || silent(sp.asset),+ muted: sp.clip.muted || !!sp.clip.hidden || assetIsSilent(sp.asset),speed: sp.clip.speed,volume: sp.clip.volume,transition: sp.transitionOut,soundCross: sp.soundOut,+ soundBack: sp.soundBack,+ soundAhead: sp.soundAhead,},]);diff --git a/site/src/cut/lib/framePlan.test.ts b/site/src/cut/lib/framePlan.test.tsindex 90c5974d..9b15d278 100644--- a/site/src/cut/lib/framePlan.test.ts+++ b/site/src/cut/lib/framePlan.test.ts@@ -1,6 +1,7 @@import { describe, expect, test } from "bun:test";import {clipAnimFx,+ crossHandles,duckGainAt,overlayTransitionFx,prerollLead,@@ -45,39 +46,86 @@ function spansOf(count: number, len = 4, transitions: number[] = []): ClipSpan[]len,transitionOut,soundOut: 0,+ soundAhead: 0,+ soundBack: 0,});at += len; // clips abut — a transition is a blend at the cut, never overlap}return spans;}-/** The same run, handing over on the sound: the picture cuts, the window- * lands on `soundOut`. */+/** The same run, handing over on the sound: the picture cuts, and each cut's+ * crossing runs `crosses[i]` either side of it. Both clips are trimmed out of+ * a longer source, so each has the handle its side of a crossing reaches+ * into. */function soundSpansOf(count: number, len = 4, crosses: number[] = []): ClipSpan[] {const spans = spansOf(count, len);- return spans.map((sp, i) => ({ ...sp, soundOut: crosses[i] ?? 0 }));+ return spans.map((sp, i) => ({+ ...sp,+ soundOut: crosses[i] ?? 0,+ soundAhead: crosses[i] ?? 0,+ soundBack: crosses[i - 1] ?? 0,+ }));}describe("soundCrossGain", () => {test("crosses the level over the cut while the picture stays a cut", () => {const spans = soundSpansOf(2, 4, [2]);const cut = spans[1].start;- // Nothing moves until the window opens.+ // Nothing moves until the window opens, and it is done when it closes.expect(soundCrossGain(spans, spans[0], cut - 2.5)).toBeCloseTo(1, 5);- // The outgoing clip ramps down into the cut…- expect(soundCrossGain(spans, spans[0], cut - 1)).toBeCloseTo(0.5, 5);- expect(soundCrossGain(spans, spans[0], cut - 0.001)).toBeCloseTo(0, 2);- // …and the incoming one ramps up out of it.- expect(soundCrossGain(spans, spans[1], cut)).toBeCloseTo(0, 5);- expect(soundCrossGain(spans, spans[1], cut + 1)).toBeCloseTo(0.5, 5);- expect(soundCrossGain(spans, spans[1], cut + 2)).toBeCloseTo(1, 5);+ expect(soundCrossGain(spans, spans[1], cut - 2)).toBeCloseTo(0, 5);+ expect(soundCrossGain(spans, spans[0], cut + 2)).toBeCloseTo(0, 5);+ expect(soundCrossGain(spans, spans[1], cut + 2.5)).toBeCloseTo(1, 5);+ // The two sides meet on the cut itself, each at equal power's half.+ expect(soundCrossGain(spans, spans[0], cut)).toBeCloseTo(Math.SQRT1_2, 5);+ expect(soundCrossGain(spans, spans[1], cut)).toBeCloseTo(Math.SQRT1_2, 5);// The picture never blends: no overlap, no incoming picture.expect(trackZeroPlan(spans[0], spans, cut - 1).p).toBe(0);expect(trackZeroPlan(spans[0], spans, cut - 1).incoming).toBe(null);expect(trackZeroPlan(spans[0], spans, cut - 1).masterAlpha).toBe(1);});- test("leaves a run with no sound dissolve at full level", () => {+ test("holds one loudness across the crossing", () => {+ const spans = soundSpansOf(2, 4, [2]);+ const cut = spans[1].start;+ // Equal power: the two gains square to one everywhere in the window, so+ // the mix neither dips nor swells through the join.+ for (const t of [cut - 2, cut - 1.3, cut - 0.4, cut, cut + 0.6, cut + 1.7, cut + 2]) {+ const going = soundCrossGain(spans, spans[0], t);+ const coming = soundCrossGain(spans, spans[1], t);+ expect(going * going + coming * coming).toBeCloseTo(1, 5);+ }+ });++ test("both sides of a crossing sound past their own footprint", () => {+ const spans = soundSpansOf(2, 4, [2]);+ const cut = spans[1].start;+ // The outgoing clip is still playing — on its handle — after the picture+ // has cut, and the incoming one before it.+ const after = crossHandles(spans, cut + 1);+ expect(after.map((h) => h.span.clip.id)).toEqual([spans[0].clip.id]);+ expect(after[0].gain).toBeCloseTo(Math.cos((0.75 * Math.PI) / 2), 5);+ const before = crossHandles(spans, cut - 1);+ expect(before.map((h) => h.span.clip.id)).toEqual([spans[1].clip.id]);+ expect(before[0].gain).toBeCloseTo(Math.sin((0.25 * Math.PI) / 2), 5);+ // Outside the crossing nothing sounds past its own footprint.+ expect(crossHandles(spans, cut + 2.5)).toEqual([]);+ });++ test("a clip with no handle keeps its side of the crossing inside itself", () => {+ // Both clips run to the ends of their sources: nothing to reach into, so+ // the ramps stay put and only the ramps play.+ const spans = soundSpansOf(2, 4, [2]).map((sp) => ({+ ...sp,+ soundAhead: 0,+ soundBack: 0,+ }));+ expect(crossHandles(spans, spans[1].start + 1)).toEqual([]);+ expect(soundCrossGain(spans, spans[0], spans[1].start)).toBeCloseTo(Math.SQRT1_2, 5);+ });++ test("leaves a run with no cross dissolve at full level", () => {const spans = soundSpansOf(2, 4);expect(soundCrossGain(spans, spans[0], 3.9)).toBe(1);expect(soundCrossGain(spans, spans[1], 4.1)).toBe(1);@@ -85,14 +133,14 @@ describe("soundCrossGain", () => {});describe("overlayTransitionFx", () => {- test("an upper-track sound dissolve moves the level and not the picture", () => {+ test("an upper-track cross dissolve moves the level and not the picture", () => {const spans = soundSpansOf(2, 4, [2]);const cut = spans[1].start;const out = overlayTransitionFx(spans[0], undefined, spans[1], cut - 1);- expect(out.gain).toBeCloseTo(0.5, 5);+ expect(out.gain).toBeCloseTo(Math.cos((0.25 * Math.PI) / 2), 5);expect(out.alpha).toBe(1);const inc = overlayTransitionFx(spans[1], spans[0], undefined, cut + 1);- expect(inc.gain).toBeCloseTo(0.5, 5);+ expect(inc.gain).toBeCloseTo(Math.sin((0.75 * Math.PI) / 2), 5);expect(inc.alpha).toBe(1);});});diff --git a/site/src/cut/lib/framePlan.ts b/site/src/cut/lib/framePlan.tsindex b16ff6d2..3dc8d171 100644--- a/site/src/cut/lib/framePlan.ts+++ b/site/src/cut/lib/framePlan.ts@@ -67,15 +67,11 @@ export function overlayTransitionFx(zoom = 1 + (TRANSITION_ZOOM - 1) * p;}}- // A sound dissolve at either edge: the picture cuts and only the level+ // A cross dissolve at either edge: the picture cuts and only the level// moves, so the clip arrives at full opacity with its sound coming up.- if (prev && prev.soundOut > 0) {- const rel = t - span.start;- if (rel < prev.soundOut) gain = Math.min(gain, Math.max(0, rel / prev.soundOut));- }+ if (prev && prev.soundOut > 0) gain = Math.min(gain, crossIn(t, span.start, prev.soundOut));if (next && span.soundOut > 0) {- const left = span.start + span.len - t;- if (left < span.soundOut) gain = Math.min(gain, Math.max(0, left / span.soundOut));+ gain = Math.min(gain, crossOut(t, span.start + span.len, span.soundOut));}// The clip's own entrance/exit animations. A transitioned joint owns its// edges: the transition plays there and the adjacent animation is held@@ -206,30 +202,74 @@ export function duckGainAt(audioClips: AudioClip[], t: number): number {}/**- * The gain a sound dissolve puts on the clip playing at `t`.+ * How far a cross dissolve at `cut` has travelled at `t`, 0 before it starts+ * and 1 when it is done. The handover runs `half` seconds either side of the+ * cut, so the crossing point — where both clips are equally loud — is the cut+ * itself.+ */+const crossP = (t: number, cut: number, half: number) =>+ Math.max(0, Math.min(1, (t - (cut - half)) / (2 * half)));++/**+ * The two sides of a cross dissolve, as equal-power gains.*- * The picture cuts and the sound crosses it: the outgoing clip ramps down over- * the window before the cut, and the clip after it ramps up over the window- * after it. The preview reads this per frame; the offline fold schedules the- * same two ramps.+ * Sine and cosine rather than straight lines: two ramps that meet at half+ * level sum to constant power, so the mix holds its loudness through the join.+ * Linear ramps would cross at half amplitude, which is 3 dB down — an audible+ * scoop in the middle of every dissolve.+ */+const crossOut = (t: number, cut: number, half: number) =>+ Math.cos((crossP(t, cut, half) * Math.PI) / 2);+const crossIn = (t: number, cut: number, half: number) =>+ Math.sin((crossP(t, cut, half) * Math.PI) / 2);++/**+ * The gain a cross dissolve puts on a clip at `t`.+ *+ * The picture cuts and the sound crosses it: the outgoing clip ramps down+ * across the cut while the clip after it ramps up across the same window, both+ * equal-power, so the two are audible together and the level never dips. Each+ * side reaches past its own footprint by the handle it has (`soundAhead` and+ * `soundBack`), which is what gives the crossing two real signals; where a+ * clip has no handle its side simply stops at the cut. The preview reads this+ * per frame, the offline fold schedules the same curves, and the ffmpeg graph+ * writes them as volume expressions.*/export function soundCrossGain(spans: ClipSpan[], master: ClipSpan, t: number): number {const i = spans.indexOf(master);if (i < 0) return 1;let g = 1;- const out = master.soundOut;- if (out > 0.01) {- const from = master.start + master.len - out;- if (t > from) g = Math.min(g, Math.max(0, (master.start + master.len - t) / out));- }- const prev = spans[i - 1];- const inc = prev?.soundOut ?? 0;- if (inc > 0.01 && t < master.start + inc) {- g = Math.min(g, Math.max(0, (t - master.start) / inc));+ if (master.soundOut > 0.01) {+ g = Math.min(g, crossOut(t, master.start + master.len, master.soundOut));}+ const inc = spans[i - 1]?.soundOut ?? 0;+ if (inc > 0.01) g = Math.min(g, crossIn(t, master.start, inc));return g;}+/**+ * The clips whose sound is live at `t` past their own footprint: a neighbour+ * still crossing a cut on its handle.+ *+ * A cross dissolve needs both clips audible over a cut their pictures hard-+ * join, so for the length of the handover each side plays into the other's+ * time — the outgoing clip past its out point, the incoming one before its in+ * point. Those stretches are outside the clip's footprint, so nothing else on+ * the timeline reaches them and only the cross ramp shapes them.+ */+export function crossHandles(spans: ClipSpan[], t: number): { span: ClipSpan; gain: number }[] {+ const out: { span: ClipSpan; gain: number }[] = [];+ for (const sp of spans) {+ if (sp.clip.hidden || sp.clip.muted) continue;+ if (t >= sp.start && t < sp.start + sp.len) continue;+ const from = sp.start - sp.soundBack;+ const to = sp.start + sp.len + sp.soundAhead;+ if (t < from || t >= to) continue;+ out.push({ span: sp, gain: soundCrossGain(spans, sp, t) });+ }+ return out;+}+/** How much of a clip's animation window is still running at `rel`, for the* sides that show a neighbour's frame behind them. */function animLive(a: ClipAnim | undefined, side: "in" | "out", rel: number, len: number) {@@ -371,6 +411,7 @@ export function trackZeroPlan(master: ClipSpan, spans: ClipSpan[], t: number): T/** One upper-track clip on screen, with its ramps resolved. */export interface OverlayPlanItem {+ span: ClipSpan;clip: VideoClip;asset: MediaAsset;alpha: number;@@ -399,6 +440,7 @@ export function overlayPlan(if (sp.clip.hidden) continue;if (t < sp.start || t >= sp.start + sp.len) continue;live.push({+ span: sp,clip: sp.clip,asset: sp.asset,...overlayTransitionFx(sp, spans[i - 1], spans[i + 1], t),diff --git a/site/src/cut/lib/store.test.ts b/site/src/cut/lib/store.test.tsindex bb99a7a7..2d4835bb 100644--- a/site/src/cut/lib/store.test.ts+++ b/site/src/cut/lib/store.test.ts@@ -1,6 +1,6 @@import { beforeEach, describe, expect, test } from "bun:test";import { groupRemap } from "@donkeycut/effects-kit";-import { adoptTransitionFields, clipLen, deriveTransitionFields, docOverlays, getClipSpans, liftClipLooks, moveOverlayGroup, overlayLaneOrder, normalizeElementLanes, parkedTransitions, placeInRun, projectDuration, reanchorTransitions, separateOverlaps, serializeDoc, useEditor } from "./store";+import { adoptTransitionFields, clipLen, cutTranscribeSpec, deriveTransitionFields, docOverlays, getClipSpans, liftClipLooks, moveOverlayGroup, overlayLaneOrder, normalizeElementLanes, parkedTransitions, placeInRun, projectDuration, separateOverlaps, serializeDoc, useEditor } from "./store";import { playheadAt, setPlayhead, setSkim } from "./playhead";import { emptySubtitles } from "./types";import type { AudioClip, MediaAsset, SubtitleCue, TextOverlay, VideoClip } from "./types";@@ -974,7 +974,7 @@ describe("bars and clip edges", () => {expect(clipById(b.id).start).toBeCloseTo(4);});- test("a sound dissolve hands over on the audio and leaves the picture cut", () => {+ test("a cross dissolve hands over on the audio and leaves the picture cut", () => {const av = asset(4);const a = vclip({ track: 0, start: 0, out: 4, assetId: av.id });const b = vclip({ track: 0, start: 4, out: 4, assetId: av.id });@@ -982,9 +982,10 @@ describe("bars and clip edges", () => {s().setClipTransition(a.id, 0.8, "audiocross");const spans = getClipSpans(s().clips, s().assets, 0);// The window rides the sound field; every reader of the picture blend- // sees a plain cut.+ // sees a plain cut. The handover crosses the cut, so each side ramps over+ // half the declared length.expect(spans[0].transitionOut).toBe(0);- expect(spans[0].soundOut).toBeCloseTo(0.8);+ expect(spans[0].soundOut).toBeCloseTo(0.4);// Switching the same bar to a picture style moves the window back.s().setClipTransition(a.id, 0.8, "crossfade");const after = getClipSpans(s().clips, s().assets, 0);@@ -992,7 +993,65 @@ describe("bars and clip edges", () => {expect(after[0].soundOut).toBe(0);});- test("a sound dissolve on an open edge plays nothing", () => {+ test("a cross dissolve's bar sits across the cut it plays", () => {+ const av = asset(8);+ const a = vclip({ track: 0, start: 0, out: 4, assetId: av.id });+ const b = vclip({ track: 0, start: 4, out: 4, assetId: av.id });+ useEditor.setState({ assets: [av], clips: [a, b] });+ s().setClipTransition(a.id, 0.8, "audiocross");+ // Half the handover either side of the cut, which is where the sound+ // crosses — a picture blend ends on the cut instead.+ expect(s().transitions[0].start).toBeCloseTo(3.6);+ expect(parkedTransitions(s().clips, s().transitions)).toEqual([]);+ s().setClipTransition(a.id, 0.8, "crossfade");+ expect(s().transitions[0].start).toBeCloseTo(3.2);+ expect(parkedTransitions(s().clips, s().transitions)).toEqual([]);+ });++ test("a cross dissolve reaches into the media each clip was trimmed from", () => {+ const av = asset(8);+ // Both clips are cut out of the middle of an eight-second source, so each+ // has media on either side of its trim for the crossing to reach into.+ const a = vclip({ track: 0, start: 0, in: 1, out: 5, assetId: av.id });+ const b = vclip({ track: 0, start: 4, in: 1, out: 5, assetId: av.id });+ useEditor.setState({ assets: [av], clips: [a, b] });+ s().setClipTransition(a.id, 0.8, "audiocross");+ const spans = getClipSpans(s().clips, s().assets, 0);+ expect(spans[0].soundOut).toBeCloseTo(0.4);+ expect(spans[0].soundAhead).toBeCloseTo(0.4);+ expect(spans[1].soundBack).toBeCloseTo(0.4);+ // Nothing reaches past a cut with no crossing on it.+ expect(spans[0].soundBack).toBe(0);+ expect(spans[1].soundAhead).toBe(0);+ });++ test("a clip trimmed to its source's edge has no handle to cross with", () => {+ const av = asset(4);+ const a = vclip({ track: 0, start: 0, out: 4, assetId: av.id });+ const b = vclip({ track: 0, start: 4, out: 4, assetId: av.id });+ useEditor.setState({ assets: [av], clips: [a, b] });+ s().setClipTransition(a.id, 0.8, "audiocross");+ const spans = getClipSpans(s().clips, s().assets, 0);+ expect(spans[0].soundOut).toBeCloseTo(0.4);+ expect(spans[0].soundAhead).toBe(0);+ expect(spans[1].soundBack).toBe(0);+ });++ test("a sped-up clip's handle is measured in timeline seconds", () => {+ const av = asset(8);+ const a = vclip({ track: 0, start: 0, in: 0, out: 4, speed: 2, assetId: av.id });+ const b = vclip({ track: 0, start: 2, in: 4, out: 8, speed: 2, assetId: av.id });+ useEditor.setState({ assets: [av], clips: [a, b] });+ s().setClipTransition(a.id, 0.8, "audiocross");+ const spans = getClipSpans(s().clips, s().assets, 0);+ // Four source seconds are left past a's out point, which at 2× is two+ // timeline seconds — more than the crossing wants, so it takes its half.+ expect(spans[0].soundAhead).toBeCloseTo(0.4);+ // b starts four source seconds in: two timeline seconds of handle.+ expect(spans[1].soundBack).toBeCloseTo(0.4);+ });++ test("a cross dissolve on an open edge plays nothing", () => {const av = asset(4);const a = vclip({ track: 0, start: 0, out: 4, assetId: av.id });useEditor.setState({ assets: [av], clips: [a] });@@ -1119,14 +1178,14 @@ describe("bars and clip edges", () => {expect(clipById(a.id).animOut).toEqual({ style: "fade", seconds: 0.8 });});- test("a sound dissolve on an open edge reads as parked, not as a live bar", () => {+ test("a cross dissolve on an open edge reads as parked, not as a live bar", () => {const av = asset(8);const a = vclip({ track: 0, start: 0, out: 4, assetId: av.id });const b = vclip({ track: 0, start: 6, out: 4, assetId: av.id });useEditor.setState({ assets: [av], clips: [a, b] });- // A sound dissolve dropped on a's open tail. It has no picture ramp and+ // A cross dissolve straddling a's open tail. It has no picture ramp and// nothing to blend into, so it plays nothing and has to say so.- const id = s().addTransition({ start: 3.2, seconds: 0.8, style: "audiocross" });+ const id = s().addTransition({ start: 3.6, seconds: 0.8, style: "audiocross" });expect(parkedTransitions(s().clips, s().transitions).map((t) => t.id)).toEqual([id]);expect(clipById(a.id).animOut).toBeUndefined();// The same bar on a real cut plays it, and stops reading as parked:@@ -1137,11 +1196,11 @@ describe("bars and clip edges", () => {expect(clipById(a.id).transitionStyle).toBe("audiocross");});- test("a sound dissolve on an open edge leaves the boundary for a bar that plays it", () => {+ test("a cross dissolve on an open edge leaves the boundary for a bar that plays it", () => {const av = asset(8);const a = vclip({ track: 0, start: 0, out: 4, assetId: av.id });useEditor.setState({ assets: [av], clips: [a] });- const sound = s().addTransition({ start: 3.2, seconds: 0.8, style: "audiocross" });+ const sound = s().addTransition({ start: 3.6, seconds: 0.8, style: "audiocross" });// A picture bar on the same tail takes the edge the sound bar cannot.s().addTransition({ start: 3.2, seconds: 0.8, style: "crossfade" });expect(clipById(a.id).animOut).toEqual({ style: "fade", seconds: 0.8 });@@ -1494,3 +1553,36 @@ describe("overlay row order", () => {expect(overlayLaneOrder(useEditor.getState().overlays)).toEqual([0, 1]);});});++describe("transcribe spec", () => {+ test("mutes a still on track 0, which has no sound to read", () => {+ const video = asset(6);+ const still = { ...asset(0, "image"), fileName: "frame-0m55s.png" };+ const spec = cutTranscribeSpec({+ clips: [+ vclip({ assetId: still.id, start: 0, in: 0, out: 0.2 }),+ vclip({ assetId: video.id, start: 0.2, in: 0, out: 4 }),+ ],+ assets: [video, still],+ audioClips: [],+ overlays: [],+ })!;+ expect(spec.clips.find((c) => c.file === still.fileName)?.muted).toBe(true);+ expect(spec.clips.find((c) => c.file === video.fileName)?.muted).toBe(false);+ });++ test("leaves a still on an upper track out of the audio items", () => {+ const video = asset(6);+ const still = { ...asset(0, "image"), fileName: "still.png" };+ const spec = cutTranscribeSpec({+ clips: [+ vclip({ assetId: video.id, start: 0, in: 0, out: 4 }),+ vclip({ assetId: still.id, track: 1, start: 0, in: 0, out: 2 }),+ ],+ assets: [video, still],+ audioClips: [],+ overlays: [],+ })!;+ expect(spec.audio.some((a) => a.file === still.fileName)).toBe(false);+ });+});diff --git a/site/src/cut/lib/store.ts b/site/src/cut/lib/store.tsindex 955444be..93ea99ec 100644--- a/site/src/cut/lib/store.ts+++ b/site/src/cut/lib/store.ts@@ -61,7 +61,7 @@ import { useGenNotify } from "./genNotify";import { clampPlayhead, playheadAt, previewAt, setPlayhead, setSkim } from "./playhead";import { engineTranscribeSamples, withEngineStt } from "./localStt";import { trackLocale } from "./subtitles";-import { ANIM_STYLE_IDS, animStyleOfTransition, clipPoseAt, DEFAULT_BACKGROUND, emptySubtitles, frameOf, IMAGE_CLIP_SECONDS, isAudioTransition, isEffectOverlay, isStickerOverlay, MAX_SUBTITLE_LANES, mediaUrl, migrateBehindSubject, migrateLegacyTransitions, normalizeAspect, overlayAnimStyle, projectBackground, SPEED_FLOOR, SPEED_MIN, stampOverlayKinds, stripDefaultOverlayKinds, TRANSITION_MAX, TRANSITION_STYLE_IDS, transitionStyleOfAnim } from "./types";+import { ANIM_STYLE_IDS, animStyleOfTransition, assetIsSilent, clipPoseAt, DEFAULT_BACKGROUND, emptySubtitles, frameOf, IMAGE_CLIP_SECONDS, isAudioTransition, isEffectOverlay, isStickerOverlay, MAX_SUBTITLE_LANES, mediaUrl, migrateBehindSubject, migrateLegacyTransitions, normalizeAspect, overlayAnimStyle, projectBackground, SPEED_FLOOR, SPEED_MIN, stampOverlayKinds, stripDefaultOverlayKinds, TRANSITION_MAX, TRANSITION_STYLE_IDS, transitionBarAt, transitionBarStart, transitionStyleOfAnim, type TransitionBoundaryKind } from "./types";import { liftMoveTracks } from "./textMotion";import { readTextStyle } from "./textStyle";import { loadUiState, saveUiState, type ProjectUiState } from "./uiState";@@ -1189,7 +1189,7 @@ function normalizeWrite(prev: EditorState, incoming: Partial<EditorState>): Part* always checked against the same audio it was written from. Null when the* timeline has nothing audible.*/-function cutTranscribeSpec(+export function cutTranscribeSpec(s: Pick<EditorState, "clips" | "assets" | "audioClips" | "overlays">,locale?: string): CloudTranscribeSpec | null {@@ -1197,7 +1197,13 @@ function cutTranscribeSpec(const duration = projectDuration(s);const assetById = new Map(s.assets.map((a) => [a.id, a]));const audio = s.audioClips- .filter((a) => !a.hidden && a.start < duration && assetById.has(a.assetId))+ .filter(+ (a) =>+ !a.hidden &&+ a.start < duration &&+ assetById.has(a.assetId) &&+ !assetIsSilent(assetById.get(a.assetId)!)+ ).map((a) => ({file: assetById.get(a.assetId)!.fileName,in: a.in,@@ -1208,7 +1214,14 @@ function cutTranscribeSpec(})).concat(overlayLayers(s.clips)- .filter((c) => !c.hidden && !c.muted && c.start < duration && assetById.has(c.assetId))+ .filter(+ (c) =>+ !c.hidden &&+ !c.muted &&+ c.start < duration &&+ assetById.has(c.assetId) &&+ !assetIsSilent(assetById.get(c.assetId)!)+ ).map((c) => ({file: assetById.get(c.assetId)!.fileName,in: c.in,@@ -1235,7 +1248,7 @@ function cutTranscribeSpec(file: sp.asset.fileName,in: sp.clip.in,out: sp.clip.out,- muted: sp.clip.muted,+ muted: sp.clip.muted || assetIsSilent(sp.asset),speed: clipSpeed(sp.clip),// The clamped cross-dissolve overlap, so the mix overlaps clip// audio the same way the timeline does and cues stay in sync.@@ -2380,17 +2393,18 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {(roles.get(t.id) ?? []).some((r) => r.kind !== "in" && r.clipId === id)) ??s.transitions.find(- (t) => !roles.has(t.id) && Math.abs(t.start + t.seconds - tail) <= TOUCH_EPS+ (t) => !roles.has(t.id) && Math.abs(transitionBarAt(t, "cut") - tail) <= TOUCH_EPS);const value = Math.max(0, Math.min(TRANSITION_MAX, seconds));if (value <= 0) {if (existing) get().removeTransition(existing.id);return;}+ const barStyle = style ?? existing?.style ?? "crossfade";const bar = {- start: tail - value,+ start: transitionBarStart(barStyle, "cut", tail, value),seconds: value,- style: style ?? existing?.style ?? "crossfade",+ style: barStyle,};push();set((s2) => ({@@ -2418,17 +2432,18 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {s.transitions.find((t) =>!roles.has(t.id) &&- Math.abs((which === "in" ? t.start : t.start + t.seconds) - edgeAt) <= TOUCH_EPS+ Math.abs(transitionBarAt(t, which === "in" ? "in" : "cut") - edgeAt) <= TOUCH_EPS);if (!anim || !ANIM_STYLE_IDS.includes(anim.style)) {if (existing) get().removeTransition(existing.id);return;}const seconds = Math.max(0.1, Math.min(TRANSITION_MAX, anim.seconds));+ const animStyle = transitionStyleOfAnim(anim.style);const bar = {- start: which === "in" ? edgeAt : edgeAt - seconds,+ start: transitionBarStart(animStyle, which === "in" ? "in" : "cut", edgeAt, seconds),seconds,- style: transitionStyleOfAnim(anim.style),+ style: animStyle,};push();set((s2) => ({@@ -3005,7 +3020,7 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {const clip = clips.find((c) => c.id === selection.id);if (!clip || clip.muted) return; // no sound to detachconst span = clipWindow(clips, assets, clip.id);- if (!span) return;+ if (!span || assetIsSilent(span.asset)) return; // a still has none eitherpush();const audio: AudioClip = {id: uid(),@@ -3249,7 +3264,7 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {if (barIds.has(t.id)) return true;const rs = roles.get(t.id);if (rs && rs.length > 0) return rs.every((r) => clipIds.has(r.clipId));- return onEdge(t.start) || onEdge(t.start + t.seconds);+ return onEdge(t.start) || onEdge(transitionBarAt(t, "cut"));};const kept = s.transitions.some(dropped)? s.transitions.filter((t) => !dropped(t))@@ -3708,6 +3723,7 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {const projectId = s.projectId;const sp = clipWindow(s.clips, s.assets, clipId);if (!sp) throw new Error("The clip is no longer on the timeline.");+ if (assetIsSilent(sp.asset)) throw new Error("This clip is a still — it has no sound.");const lane = s.subtitleLane;const epoch = laneEpoch;// The clip's own sound, deliberately unmuted: this transcribes what the@@ -4452,7 +4468,8 @@ export const useEditor = create<EditorState>((baseSet, get, api) => {);if (near && Math.abs(near.at - t) <= BAR_PASTE_REACH) {delta =- (near.kind === "in" ? near.at : near.at - first.seconds) - first.start;+ transitionBarStart(first.style, near.kind, near.at, first.seconds) -+ first.start;// The landed bar takes over the boundary; the bar that played// it leaves with it, the way a drop replaces the incumbent.const roles = resolveTransitions(clips, transitions);@@ -4680,7 +4697,7 @@ const clampBarSeconds = (s: number) => Math.max(0.1, Math.min(TRANSITION_MAX, s)/** The place a transition bar is playing at: the cut it ends on (`clipId` is* the outgoing clip), or the open head/tail it rides. */-export type TransitionRole = { kind: "cut" | "in" | "out"; clipId: string };+export type TransitionRole = { kind: TransitionBoundaryKind; clipId: string };type TransitionBoundary = TransitionRole & { at: number };@@ -4708,7 +4725,7 @@ function transitionBoundaries(clips: VideoClip[]): TransitionBoundary[] {/** Whether a bar's style has anything to play at this kind of boundary. A* cut hands one clip's sound and picture to the next, so every style plays* one. An open head or tail has only the single clip's edge to ramp, which is- * a picture ramp — a sound dissolve has nothing to blend there, so it never+ * a picture ramp — a cross dissolve has nothing to blend there, so it never* claims the boundary: it reads as parked, which is what it is, and the AI's* debris report names it. */const playsBoundary = (style: TransitionStyle, kind: TransitionRole["kind"]): boolean =>@@ -4740,7 +4757,7 @@ export function resolveTransitions((b) =>!taken.has(b) &&playsBoundary(t.style, b.kind) &&- Math.abs((b.kind === "in" ? t.start : t.start + t.seconds) - b.at) <= TOUCH_EPS+ Math.abs(transitionBarAt(t, b.kind) - b.at) <= TOUCH_EPS).sort((a, b) => rank[a.kind] - rank[b.kind]);if (fits.length === 0) continue;@@ -4796,7 +4813,7 @@ export function reanchorTransitions(const clip = role && byId.get(role.clipId);if (!clip) return t;const at = role.kind === "in" ? clip.start : clip.start + clipLen(clip);- const start = role.kind === "in" ? at : at - t.seconds;+ const start = transitionBarStart(t.style, role.kind, at, t.seconds);if (Math.abs(start - t.start) <= TOUCH_EPS) return t;changed = true;return { ...t, start };@@ -4914,19 +4931,22 @@ export function adoptTransitionFields(const d = Math.min(c.transition ?? 0, TRANSITION_MAX);const tailFree = !claimed.has(`cut:${c.id}`) && !claimed.has(`out:${c.id}`);if (d > 0 && tailFree) {+ const style = TRANSITION_STYLE_IDS.includes(c.transitionStyle as TransitionStyle)+ ? (c.transitionStyle as TransitionStyle)+ : "crossfade";+ const seconds = clampBarSeconds(d);add(`cut:${c.id}`, {- start: end - d,- seconds: clampBarSeconds(d),- style: TRANSITION_STYLE_IDS.includes(c.transitionStyle as TransitionStyle)- ? (c.transitionStyle as TransitionStyle)- : "crossfade",+ start: transitionBarStart(style, "cut", end, seconds),+ seconds,+ style,});} else if (c.animOut && tailFree) {const seconds = clampBarSeconds(c.animOut.seconds);+ const style = transitionStyleOfAnim(c.animOut.style);add(`cut:${c.id}`, {- start: end - seconds,+ start: transitionBarStart(style, "cut", end, seconds),seconds,- style: transitionStyleOfAnim(c.animOut.style),+ style,});}}@@ -4940,7 +4960,11 @@ export function adoptTransitionFields(} else if (!claimed.has(`cut:${joint.id}`) && !claimed.has(`out:${joint.id}`)) {// The entrance played against the clip before it; at a joint that is the// blend at the cut, and it keeps the length and look it was saved with.- add(`cut:${joint.id}`, { start: c.start - seconds, seconds, style });+ add(`cut:${joint.id}`, {+ start: transitionBarStart(style, "cut", c.start, seconds),+ seconds,+ style,+ });}}return out.length === transitions.length ? transitions : out;@@ -5070,21 +5094,39 @@ function buildClipSpans(clips: VideoClip[], assets: MediaAsset[], track: number)// The blend into the next clip, live only at a cut the pair actually// makes; clips dragged apart dissolve into nothing. A sound style hands// over on the audio alone, so its window lands on the other field and- // every reader of the picture blend sees a plain cut.+ // every reader of the picture blend sees a plain cut. It also crosses the+ // cut instead of ending on it, so the declared handover splits in half:+ // one half before the cut and one after.const overlap = transitionOverlap(clip, next);const onSound = isAudioTransition(clip.transitionStyle);+ const soundOut = onSound ? overlap / 2 : 0;+ // What each side can reach into to make that a real crossing: the source+ // this clip still has past its out point, and the source the next clip+ // has before its in point. Both are trimmed-away media — the handles —+ // and they are what lets the two clips be audible at once over a cut+ // their pictures hard-join.+ const prevSound = spans[spans.length - 1]?.soundOut ?? 0;spans.push({clip,asset,start: clip.start,len,transitionOut: onSound ? 0 : overlap,- soundOut: onSound ? overlap : 0,+ soundOut,+ soundAhead: handleSeconds(soundOut, asset.duration - clip.out, clip),+ soundBack: handleSeconds(prevSound, clip.in, clip),});}return spans;}+/** How far a cross dissolve reaches into a clip's handle: the ramp half it+ * wants, capped by the source seconds actually there, in timeline seconds. */+function handleSeconds(want: number, sourceLeft: number, clip: VideoClip): number {+ if (want <= 0.01) return 0;+ return Math.max(0, Math.min(want, Math.max(0, sourceLeft) / clipSpeed(clip)));+}+/** One row of the timeline, whichever kind of thing sits on it. Video rows are* `track`; audio and title rows are the item's `lane`. */export type LaneRef =diff --git a/site/src/cut/lib/types.ts b/site/src/cut/lib/types.tsindex 0c7d592f..00d3212d 100644--- a/site/src/cut/lib/types.ts+++ b/site/src/cut/lib/types.ts@@ -192,6 +192,14 @@ export interface StoredAsset {chatId?: string;}+/** True for an asset that carries no sound. Only video and audio files hold an+ * audio track; a still (a freeze frame, an imported image) is not a media+ * container at all, and handing its URL to the audio reader fails the read+ * rather than returning silence. Every path that asks a source for audio —+ * the mix, the transcribe spec, the preview's voices — asks this first. */+export const assetIsSilent = (asset: Pick<StoredAsset, "type">): boolean =>+ asset.type !== "video" && asset.type !== "audio";+/** An import whose bytes are still on their way to storage. While this is set* the asset plays from a local object URL. Whether it may join the saved* document depends on where the bytes live: an upload holding only tab-scoped@@ -543,12 +551,30 @@ export const SPEED_FLOOR = 0.05;export const TRANSITION_MAX = 2;/** What a transition runs for when one is placed on a bare cut. */export const TRANSITION_DEFAULT_SECONDS = 0.5;+/**+ * What a cross dissolve runs for instead — shorter, because the ear is less+ * forgiving than the eye.+ *+ * The crossing is centered on the cut, so both clips are speaking for its+ * whole length. Long enough and the outgoing line is still audible under the+ * incoming one, which reads as a mistake; this is the length that covers the+ * jump in room tone at a cut without letting two voices be heard at once. It+ * also asks less of the handles either side, so the handover is a real+ * crossing more often.+ */+export const CROSS_DISSOLVE_DEFAULT_SECONDS = 0.3;++/** How long a bar of this style runs when it lands on a bare cut. */+export const transitionDefaultSeconds = (style: TransitionStyle): number =>+ isAudioTransition(style) ? CROSS_DISSOLVE_DEFAULT_SECONDS : TRANSITION_DEFAULT_SECONDS;/** A transition as a timeline object of its own: a bar on the transitions row* at an absolute time, belonging to no clip. It plays when its window lines up* with a place that has a handover to make — a cut it ends on, an open head it- * starts on, an open tail it ends on — and sits inert anywhere else. Clips- * moving or deleting leave it exactly where it is. */+ * starts on, an open tail it ends on, a cut it straddles when it hands over on+ * the sound — and sits inert anywhere else. Clips moving or deleting leave it+ * exactly where it is. `transitionBarStart` is where the window falls against+ * the boundary. */export interface TimelineTransition {id: string;/** Bar start on the timeline, seconds; the window is [start, start+seconds]. */@@ -629,13 +655,14 @@ export const TRANSITION_XFADE: Record<TransitionStyle, string> = {circleclose: "circleclose",splitopen: "vertopen",splitclose: "vertclose",- // The sound dissolve never reaches xfade — the picture cuts and the renders+ // The cross dissolve never reaches xfade — the picture cuts and the renders// branch before this map. The entry keeps the record total.audiocross: "fade",};-/** The styles that hand over on the sound alone: the picture cuts, and the- * outgoing fades out into the cut while the incoming fades in out of it. They+/** The styles that hand over on the sound alone: the picture cuts while the+ * two clips ramp equal-power past each other across it, each reaching into+ * the media its trim left behind so both are audible through the join. They* take no picture blend, so a clip's own entrance and exit animations still* play across a cut one of them sits on. */export const AUDIO_TRANSITION_STYLES: TransitionStyle[] = ["audiocross"];@@ -643,6 +670,38 @@ export const AUDIO_TRANSITION_STYLES: TransitionStyle[] = ["audiocross"];export const isAudioTransition = (style: string | undefined): boolean =>!!style && (AUDIO_TRANSITION_STYLES as string[]).includes(style);+/** The kinds of place a bar can play: an open head, a cut, an open tail. */+export type TransitionBoundaryKind = "in" | "cut" | "out";++/**+ * Where a bar's window falls against the boundary at `at`.+ *+ * A picture blend runs across the outgoing clip's last seconds and hands over+ * at the cut, so the bar ends on the boundary; an entrance runs forward from+ * the clip's head. A cross dissolve crosses the cut itself — the outgoing+ * sound fades out into it while the incoming fades in out of it — so its bar+ * straddles the boundary, half its length either side.+ */+export const transitionBarStart = (+ style: TransitionStyle,+ kind: TransitionBoundaryKind,+ at: number,+ seconds: number+): number =>+ kind === "in" ? at : isAudioTransition(style) ? at - seconds / 2 : at - seconds;++/** The instant a bar claims for `kind` — the inverse of `transitionBarStart`,+ * and what every alignment test measures against a boundary. */+export const transitionBarAt = (+ t: Pick<TimelineTransition, "start" | "seconds" | "style">,+ kind: TransitionBoundaryKind+): number =>+ kind === "in"+ ? t.start+ : isAudioTransition(t.style)+ ? t.start + t.seconds / 2+ : t.start + t.seconds;+/** Picker layout for the Transitions tab: the styles that blend the picture,* grouped by family, in display order. The sound styles are picked from the* Effects tab's Sound family, beside the other treatments on the sound. */@@ -678,7 +737,7 @@ export const TRANSITION_STYLE_LABELS: Record<TransitionStyle, string> = {circleclose: "Circle close",splitopen: "Split open",splitclose: "Split close",- audiocross: "Sound dissolve",+ audiocross: "Cross dissolve",};/** Peak scale the zoom transitions push into (preview and export). */@@ -1186,13 +1245,22 @@ export interface ClipSpan {* over this clip's live tail. Spans never intersect — the next one starts* exactly at this one's end, and plays from its head there. */transitionOut: number;- /** Sound dissolve into the next span, in timeline seconds: this clip's- * audio fades out over the window before the cut and the next clip's fades- * in over the window after it, while the picture cuts. Zero unless the- * transition at the cut is one of the sound styles, and never set at the- * same time as `transitionOut` — a handover blends the picture or the- * sound. */+ /** Half the cross dissolve at this span's tail, in timeline seconds: the+ * handover runs this long either side of the cut, ramping equal-power from+ * this clip to the next while the picture cuts. Zero unless the transition+ * at the cut is one of the sound styles, and never set at the same time as+ * `transitionOut` — a handover blends the picture or the sound. */soundOut: number;+ /** Seconds of source past this span's out point that its sound keeps+ * playing, so the outgoing half of the cross dissolve at its tail has+ * something to cross with. Zero when the clip is trimmed to the end of its+ * source: with no handle to reach into, its side of the handover simply+ * stops at the cut. */+ soundAhead: number;+ /** Seconds of source before this span's in point that its sound starts, for+ * the incoming half of the cross dissolve at the cut behind it. Zero when+ * the clip starts at its source's head. */+ soundBack: number;}/** The document persisted as project.json inside each project folder. */diff --git a/site/src/cut/server/ai/catalog.ts b/site/src/cut/server/ai/catalog.tsindex 5c761e3f..8d28b8b3 100644--- a/site/src/cut/server/ai/catalog.ts+++ b/site/src/cut/server/ai/catalog.ts@@ -148,18 +148,18 @@ Cutting speech well: pace is part of the message. Keep a beat of the speaker's o"transitions-and-fades": `# Transitions, animations, looks & fadesRoute the ask to the right feature:-- set_transition: a styled join between one video clip and the next, blended across the cut. It never moves clips or shortens anything, and it plays only while the pair touches. One style, audiocross, blends the sound alone — the picture cuts, the outgoing clip's sound fades out into the cut and the incoming clip's fades in out of it (it plays at cuts only, so it does nothing on a clip's open head or tail). In the UI it is picked from the Effects tab's Sound family, beside the treatments on the sound; every style that blends the picture is picked from the Transitions tab.+- set_transition: a styled join between one video clip and the next, blended across the cut. It never moves clips or shortens anything, and it plays only while the pair touches. One style, audiocross — the cross dissolve — blends the sound alone: the picture cuts while the two clips ramp equal-power past each other, half the length either side of the cut, each playing into the other's time out of the media its trim left behind, so the mix holds its loudness through the join (it plays at cuts only, so it does nothing on a clip's open head or tail; a clip trimmed to the end of its source has nothing to cross with and its side stops at the cut). In the UI it is picked from the Effects tab's Sound family, beside the treatments on the sound, and its bar sits across the cut rather than before it; every style that blends the picture is picked from the Transitions tab.- set_animation: one clip's own entrance ("in") or exit ("out") — the clip fades/zooms/pops/slides on its own edge. Never moves neighbors. For "fade this clip in", "make it slide in", use this. A wipe is a transition, not an animation.- set_project_fade: the whole video fades in from black at the start and/or out to black at the end — picture and full mix (titles, captions, soundtrack). Survives clip reordering. For "fade in the video", "fade to black at the end", use this.- set_background: the color of the frame itself, behind everything, for the whole cut. For "make the background white", or any cut whose picture is titles and shapes rather than footage. Covered in the graphics skill.- update_audio fadeIn/fadeOut: audio-only ramps on one soundtrack clip ("fade the music out").- An ask beyond every preset here — a reference template, a grid or mosaic reveal, a multi-panel build — is a construction job: the graphics skill's composite-builds section shows how overlay video tracks, masks, and keyframes compose into it.-set_transition styles (clipId = the leading clip; 0 clears, max 2s): crossfade blends; crosszoom adds a zoom punch; dipblack/dipwhite dip through a solid color; blur defocuses through the cut; pushleft/right/up/down shove the old frame out; wipeleft/right/up/down reveal with a hard traveling edge; circleopen/close and splitopen/close are shape reveals; audiocross cuts the picture and dissolves only the sound across the cut, for a handover the eye takes as a cut and the ear takes as a blend. Directional names describe the motion.+set_transition styles (clipId = the leading clip; 0 clears, max 2s): crossfade blends; crosszoom adds a zoom punch; dipblack/dipwhite dip through a solid color; blur defocuses through the cut; pushleft/right/up/down shove the old frame out; wipeleft/right/up/down reveal with a hard traveling edge; circleopen/close and splitopen/close are shape reveals; audiocross — the cross dissolve — cuts the picture and dissolves only the sound across the cut, for a handover the eye takes as a cut and the ear takes as a blend. Directional names describe the motion.set_animation styles: fade (audio follows), zoom, pop, slideleft/right/up/down — the slide names are the motion direction (slideleft enters from the right edge moving left, or exits off the left). Each edge holds one effect and the last pick wins: set_animation on a transitioned edge replaces that transition, set_transition clears the animations adjacent to its joint.Bars and the debris they leave: a transition lives on the transitions row as its own object at an absolute time, listed in editor_state \`transitions\` with the boundary it \`plays\`. It plays whatever cut or clip edge its window lines up with; a bar lining up with nothing reads \`parked: true\` and plays nothing — the badge is on the row, the blend is gone. Moving, retiming, or replacing clips is what parks a bar, so every timeline mutation returns \`parkedTransitions\` when its edit left one. Own that debris in the same turn you make it: a bar you parked and know the intent of, reattach with set_transition/set_animation on the edge it belongs to, or clear with remove_transition. When you can't tell what a parked bar was for — it predates your edits, or the user's ask doesn't say where the blend should end up — do not guess and do not leave it silent: name what's there ("there's a 0.5s dip to black at 29.4s that isn't on a cut any more") and ask whether to remove it or move it to a specific joint. The same holds for anything else on the timeline that reads as leftover from your own edits: say what you see and confirm before deleting the user's work.-Picking for a vibe: "smooth" → crossfade; "let the sound carry over a hard cut", "audio crossfade", "dissolve just the audio" → audiocross; "punchy/energetic" → crosszoom or a push; "dramatic scene change" → dipblack 0.5–0.8s; "dreamy" → blur (or the dreamy look); "retro" → vhs or vintage look. Between clips 0.4–0.8s reads well; 1s+ is slow and cuts total duration. The editing-taste skill covers when a joint should stay a hard cut and how to keep transition styling consistent across the video.+Picking for a vibe: "smooth" → crossfade; "let the sound carry over a hard cut", "audio crossfade", "cross dissolve", "dissolve just the audio" → audiocross; "punchy/energetic" → crosszoom or a push; "dramatic scene change" → dipblack 0.5–0.8s; "dreamy" → blur (or the dreamy look); "retro" → vhs or vintage look. Between clips 0.4–0.8s reads well on the picture; 1s+ is slow and cuts total duration. The cross dissolve is shorter — 0.3s covers the jump in room tone without letting two voices be heard at once, and it asks less of the trim each clip has left. The editing-taste skill covers when a joint should stay a hard cut and how to keep transition styling consistent across the video.UI: select a clip → Inspector "Effects" opens the panel (Transition / In / Out / Looks tabs). Preview and export render the same treatment.What the user sees on the transitions row: one badge per bar, and nothing else — the row holds no buttons, no placeholders, and no add-affordances. A playing bar is blue; a parked one is the same badge in grey. So a user reporting "greyed out transitions", "grey circles", or "dead/lingering transitions" on that row is describing parked bars, which are real objects saved in their project — never dismiss them as editor chrome or claim they won't export, because the badge is the transition itself. Read \`transitions\` in editor_state and answer from the parked entries there. Grey elsewhere means something different: a clip, element, or track shown grayed is hidden (the \`hidden\` flag), which does drop out of playback and export.`,diff --git a/site/src/cut/server/exportPipeline.test.ts b/site/src/cut/server/exportPipeline.test.tsindex 2b03104b..4b1f3f09 100644--- a/site/src/cut/server/exportPipeline.test.ts+++ b/site/src/cut/server/exportPipeline.test.ts@@ -186,25 +186,33 @@ describe("export filtergraph timebases", () => {expect(xfadeMismatches(g)).toEqual([]);});- test("a sound dissolve cuts the picture and crosses the sound", async () => {+ test("a cross dissolve cuts the picture and crosses the sound", async () => {const g = await graphFor({clips: [- clip("a.mp4", { soundCross: 0.5, transitionStyle: "audiocross" }),- clip("b.mp4"),+ clip("a.mp4", { in: 1, out: 5, soundCross: 0.5, soundAhead: 0.5, transitionStyle: "audiocross" }),+ clip("b.mp4", { in: 1, out: 5, soundBack: 0.5 }),],});const graph = g.join(";");// No blend on the picture at all — the join is a plain concat.expect(graph).not.toContain("xfade=");expect(graph).not.toContain("tpad=start_duration");- // The outgoing clip's sound fades into the cut, the incoming clip's out- // of it, and the pieces concat back to the same length.- expect(g).toContain("[a0]afade=t=out:st=3.500:d=0.500[ah1]");- expect(g).toContain("[a1]afade=t=in:st=0:d=0.500[ai1]");- expect(g).toContain("[ah1][ai1]concat=n=2:v=0:a=1[aj1]");+ // The two segments carry equal-power ramps into and out of the cut, not+ // fades to silence.+ expect(graph).not.toContain("afade=t=out:st=3.500");+ expect(graph).toContain("cos(clip((t-(3.500))/1.000,0,1)*PI/2)");+ expect(graph).toContain("sin(clip((t-(-0.500))/1.000,0,1)*PI/2)");+ // …and each clip plays its handle across the cut, delayed onto it, so+ // both are really sounding while the picture has already changed.+ expect(graph).toContain("atrim=5.000:5.500");+ expect(graph).toContain("atrim=0.500:1.000");+ expect(graph).toContain("adelay=4000:all=1[xha1]");+ expect(graph).toContain("adelay=3500:all=1[xhb1]");+ expect(graph).toContain("[xha1]");+ expect(graph).toContain("[xhb1]");});- test("a sound dissolve joined into a picture transition keeps the streams matched", async () => {+ test("a cross dissolve joined into a picture transition keeps the streams matched", async () => {const g = await graphFor({clips: [clip("a.mp4", { soundCross: 0.5, transitionStyle: "audiocross" }),@@ -215,18 +223,39 @@ describe("export filtergraph timebases", () => {expect(xfadeMismatches(g)).toEqual([]);});- test("an upper-track sound dissolve ramps the sound and never the picture", async () => {+ test("an upper-track cross dissolve ramps the sound and never the picture", async () => {const g = await graphFor({clips: [clip("a.mp4", { out: 8 })],overlayVideos: [- { file: "u1.mp4", in: 0, out: 4, start: 0, track: 1, muted: false, tailSound: 0.5 },- { file: "u2.mp4", in: 0, out: 4, start: 4, track: 1, muted: false, headSound: 0.5 },+ {+ file: "u1.mp4",+ in: 1,+ out: 5,+ start: 0,+ track: 1,+ muted: false,+ tailSound: 0.5,+ soundAhead: 0.5,+ },+ {+ file: "u2.mp4",+ in: 1,+ out: 5,+ start: 4,+ track: 1,+ muted: false,+ headSound: 0.5,+ soundBack: 0.5,+ },],});const graph = g.join(";");- // The sound crosses at the cut between the two upper-track clips…- expect(graph).toContain("afade=t=out:st=3.500:d=0.500");- expect(graph).toContain("afade=t=in:st=0:d=0.500");+ // The sound crosses at the cut between the two upper-track clips, each+ // reaching into its handle so both are audible there…+ expect(graph).toContain("cos(clip((t-(3.500))/1.000,0,1)*PI/2)");+ expect(graph).toContain("sin(clip((t-(0.000))/1.000,0,1)*PI/2)");+ expect(graph).toContain("atrim=1.000:5.500");+ expect(graph).toContain("atrim=0.500:5.000");// …and the picture keeps its opacity throughout: no alpha ramp anywhere.expect(graph).not.toContain("fade=t=in:st=0:d=0.500:alpha=1");expect(graph).not.toContain("alpha=1");diff --git a/site/src/cut/server/exportPipeline.ts b/site/src/cut/server/exportPipeline.tsindex 41e97219..8eda1b33 100644--- a/site/src/cut/server/exportPipeline.ts+++ b/site/src/cut/server/exportPipeline.ts@@ -10,6 +10,36 @@ import { audioFxFilters, buildGradeLut, effectFilterLines, gradeKey, gradeNeedsL// engine's job registry (jobs.ts) and the cloud render worker, which stage// media differently but must produce byte-identical renders.+/**+ * One side of a cross dissolve as a volume expression in a stream's own time.+ *+ * The crossing's middle sits on the cut, `half` seconds either side, and the+ * two sides are sine and cosine of the same travel: they meet at 0.707 on the+ * cut, which is where equal power puts a handover that must not dip. Outside+ * the window the expression is 1, so it can ride a stream that is doing other+ * things at its far end. Empty when there is no crossing there.+ */+function crossExpr(cut: number, half: number, rising: boolean): string {+ if (!(half > 0.01)) return "";+ // Parenthesized: a crossing that opens before its stream starts puts a+ // negative number here, and `t--0.5` is not something to hand a parser.+ const p = `clip((t-(${num(cut - half)}))/${num(2 * half)},0,1)`;+ return `${rising ? "sin" : "cos"}(${p}*PI/2)`;+}++/** The expressions folded into one `volume` filter, quoted so the graph+ * parser carries their commas whole.+ *+ * `volume` re-evaluates once a frame, and a default frame is 23 ms — coarse+ * enough for a half-second ramp to move in audible steps. Short frames first+ * (unpadded, so the stream keeps its exact length) put the steps under 6 ms,+ * where the ramp reads as a ramp. */+function crossFilters(exprs: string[]): string {+ const live = exprs.filter(Boolean);+ if (live.length === 0) return "";+ return `,asetnsamples=n=256:p=0,volume=volume=${fexpr(live.join("*"))}:eval=frame`;+}+export interface ExportSpec {projectId: string;/** Where the render lands instead of a stamped file in exports/: "hls" is the@@ -50,12 +80,17 @@ export interface ExportSpec {speed?: number; // playback rate, default 1/** Transition into the next clip, in timeline seconds (overlap). */transition?: number;- /** Sound dissolve into the next clip, in timeline seconds. The picture- * cuts; this clip's sound fades out over the window before the cut and- * the next clip's fades in over the window after it. Carried apart from- * `transition` so a join that blends only the sound is not read as a- * picture blend anywhere. */+ /** Half the cross dissolve into the next clip, in timeline seconds: the+ * picture cuts while the two clips ramp equal-power past each other over+ * that long either side of it. Carried apart from `transition` so a join+ * that blends only the sound is not read as a picture blend anywhere. */soundCross?: number;+ /** Source seconds past `out` this clip keeps sounding into the crossing+ * at its tail, and source seconds before `in` it starts sounding for the+ * crossing at its head — the handles that let both clips be audible over+ * a cut their pictures hard-join. */+ soundAhead?: number;+ soundBack?: number;/** Transition look id, resolved to an xfade name through the* TRANSITION_XFADE allowlist (unknown ids render as a plain fade). Cross* zoom renders as the fade plus zoom ramps on both segments' overlap@@ -133,10 +168,14 @@ export interface ExportSpec {tailFade?: number;headZoom?: number;tailZoom?: number;- /** Sound dissolve ramps, timeline seconds from this overlay's head/tail:- * the level crosses at the cut and the picture is left alone. */+ /** Half the cross dissolve at this overlay's head/tail, timeline seconds:+ * the level crosses equal-power at the cut and the picture is left alone.+ * `soundBack`/`soundAhead` are the handle each side reaches into so both+ * clips are really sounding through the crossing. */headSound?: number;tailSound?: number;+ soundBack?: number;+ soundAhead?: number;/** A still image: looped for the clip's length instead of trimmed. */image?: boolean;/** Manual color adjustments, baked into this overlay's segment. */@@ -993,6 +1032,23 @@ export async function runExport(const needFirstFreeze = spec.clips.map((_, j) => j > 0 && tailBd[j - 1]);const needLastFreeze = spec.clips.map((_, j) => j < spec.clips.length - 1 && headBd[j + 1]);+ // Where every clip starts on the timeline. Clips abut and a transition+ // claims no layout, so the running sum is the cut each join happens at.+ const clipAt: number[] = [];+ spec.clips.reduce((acc, c, j) => {+ clipAt[j] = acc;+ return acc + clipDur(c);+ }, 0);+ // Half the cross dissolve at each cut, clamped the way the join clamps its+ // overlap: index j is the crossing between clip j-1 and clip j.+ const crossHalf: number[] = spec.clips.map((_, j) => {+ const prevC = spec.clips[j - 1];+ if (!prevC) return 0;+ const want = prevC.soundCross ?? 0;+ if (want <= 0.01) return 0;+ return Math.min(want, clipAt[j] * 0.9, clipDur(spec.clips[j]) * 0.9);+ });+// Per-clip normalized video + audio segments for the join below.spec.clips.forEach((c, j) => {const idx = c.image ? imageClipInput.get(j)! : inputIndex.get(c.file)!;@@ -1194,7 +1250,14 @@ export async function runExport(// The picture's fade edges carry the sound with them; zoom edges don't.const afades =(ahf > 0.01 ? `,afade=t=in:st=0:d=${num(ahf)}` : "") +- (atf > 0.01 ? `,afade=t=out:st=${num(Math.max(0, dur - atf))}:d=${num(atf)}` : "");+ (atf > 0.01 ? `,afade=t=out:st=${num(Math.max(0, dur - atf))}:d=${num(atf)}` : "") ++ // A cross dissolve is not a fade: it ramps equal-power past the clip+ // on the other side of the cut, reaching half level there rather than+ // silence, which is why it is a volume expression and not afade.+ crossFilters([+ crossExpr(0, crossHalf[j], true),+ crossExpr(dur, crossHalf[j + 1] ?? 0, false),+ ]);filters.push(`[${idx}:a]atrim=${num(c.in)}:${num(c.out)},asetpts=PTS-STARTPTS,${tempo}` +`aresample=44100,aformat=sample_fmts=fltp:channel_layouts=stereo,${vol}` +@@ -1279,13 +1342,56 @@ export async function runExport(}});+ // The handles a cross dissolve reaches into. A concat cannot have two+ // segments sounding at once, and the whole point of a crossing is that they+ // do: so the stretch each clip plays past its own footprint — the outgoing+ // one after the cut, the incoming one before it — is lifted out as its own+ // delayed stream and mixed in beside the joined clip audio. Its ramp+ // carries on from where the segment's left off, at half level on the cut.+ const crossHandleLabels: string[] = [];+ const handleStream = (+ src: (typeof spec.clips)[number],+ key: string,+ from: number,+ seconds: number,+ at: number,+ expr: string+ ) => {+ if (seconds <= 0.01 || src.muted || src.hidden || src.image) return;+ if (!audioPresence.get(src.file)) return;+ const rate = clipRate(src);+ const lab = `xh${key}`;+ filters.push(+ `[${inputIndex.get(src.file)!}:a]atrim=${num(from)}:${num(from + seconds * rate)},` ++ `asetpts=PTS-STARTPTS,${rate !== 1 ? `${atempoChain(rate)},` : ""}` ++ `aresample=44100,aformat=sample_fmts=fltp:channel_layouts=stereo` ++ `${(src.volume ?? 1) !== 1 ? `,volume=${num(src.volume ?? 1)}` : ""}${crossFilters([expr])},` ++ `adelay=${Math.max(0, Math.round(at * 1000))}:all=1[${lab}]`+ );+ crossHandleLabels.push(lab);+ };+ spec.clips.forEach((c, j) => {+ const half = crossHalf[j];+ const prevC = spec.clips[j - 1];+ if (half <= 0.01 || !prevC) return;+ const cut = clipAt[j];+ const ahead = Math.min(prevC.soundAhead ?? 0, half);+ // Never reach past the head of the source: a handle is what the trim left+ // behind, and there is none before zero.+ const back = Math.min(c.soundBack ?? 0, half, c.in / clipRate(c));+ // The outgoing clip, still sounding past its out point.+ handleStream(prevC, `a${j}`, prevC.out, ahead, cut, crossExpr(0, half, false));+ // The incoming clip, already sounding before its in point.+ handleStream(c, `b${j}`, c.in - back * clipRate(c), back, cut - back, crossExpr(back, half, true));+ });+// Join the segments. Clips abut — a transition claims no layout — so every// join keeps the accumulator's full length. A transitioned join pads the// incoming segment's head with its cloned first frame and runs the xfade// across the outgoing clip's last blend-window seconds: the held frame// arrives over the live tail, and the real segment starts exactly at the// cut. Its sound is a tail fade on the outgoing side and a hard join. A- // sound dissolve cuts the picture and crosses the sound instead — a tail+ // cross dissolve cuts the picture and crosses the sound instead — a tail// fade against the incoming clip's head fade. The rest hard-cut (concat).// Fold left so mixed sequences chain correctly.let vAcc = segLabel[0];@@ -1301,13 +1407,11 @@ export async function runExport(const aOut = `aj${j}`;if (cross > 0.01) {// The picture cuts: concat, re-stamped for a later xfade the same way- // a plain join is. The sound crosses the cut.+ // a plain join is. The sound crosses the cut — its ramps are already on+ // the two segments, and the halves that reach past the cut ride their+ // own streams into the mix, so the join here is a plain one.filters.push(`[${vAcc}][${segLabel[j]}]concat=n=2:v=1:a=0,fps=${fps}[${vOut}]`);- filters.push(- `[${aAcc}]afade=t=out:st=${num(Math.max(0, acc - cross))}:d=${num(cross)}[ah${j}]`- );- filters.push(`[a${j}]afade=t=in:st=0:d=${num(cross)}[ai${j}]`);- filters.push(`[ah${j}][ai${j}]concat=n=2:v=0:a=1[${aOut}]`);+ filters.push(`[${aAcc}][a${j}]concat=n=2:v=0:a=1[${aOut}]`);} else if (d > 0.01) {const offset = Math.max(0, acc - d);// The style id resolves through the allowlist map; anything unknown@@ -1353,7 +1457,7 @@ export async function runExport(const tz = Math.max(0, Math.min(oc.tailZoom ?? 0, olen - hz));const hf = Math.max(0, Math.min(oc.headFade ?? 0, olen));const tf = Math.max(0, Math.min(oc.tailFade ?? 0, olen - hf));- // A sound dissolve's own ramps: the sound crosses at the cut, the picture+ // A cross dissolve's own ramps: the sound crosses at the cut, the picture// cuts with it.const hs = Math.max(0, Math.min(oc.headSound ?? 0, olen));const ts = Math.max(0, Math.min(oc.tailSound ?? 0, olen - hs));@@ -1518,18 +1622,30 @@ export async function runExport(if (!oc.muted && audioPresence.get(oc.file)) {const tempo = ospeed !== 1 ? `${atempoChain(ospeed)},` : "";const vol = (oc.volume ?? 1) !== 1 ? `volume=${num(oc.volume ?? 1)},` : "";+ // A cross dissolve reaches into the handle either side of its cut, so+ // this stream starts before the clip's own head and runs past its tail;+ // everything timed inside it shifts by that head reach.+ const back = Math.min(oc.soundBack ?? 0, hs, oc.in / ospeed);+ const ahead = Math.min(oc.soundAhead ?? 0, ts);// The picture's fade edges carry the sound with them; zoom edges don't.- // A sound dissolve's ramps are here and nowhere else.+ // A cross dissolve's ramps are equal-power and reach half level on the+ // cut, so they are a volume expression rather than a fade to silence.const afades =- (hf > 0.01 ? `afade=t=in:st=0:d=${num(hf)},` : "") +- (tf > 0.01 ? `afade=t=out:st=${num(Math.max(0, olen - tf))}:d=${num(tf)},` : "") +- (hs > 0.01 ? `afade=t=in:st=0:d=${num(hs)},` : "") +- (ts > 0.01 ? `afade=t=out:st=${num(Math.max(0, olen - ts))}:d=${num(ts)},` : "");- const delayMs = Math.max(0, Math.round(oc.start * 1000));+ (hf > 0.01 ? `afade=t=in:st=${num(back)}:d=${num(hf)},` : "") ++ (tf > 0.01+ ? `afade=t=out:st=${num(Math.max(0, back + olen - tf))}:d=${num(tf)},`+ : "");+ const crosses = crossFilters([+ crossExpr(back, hs, true),+ crossExpr(back + olen, ts, false),+ ]);+ const delayMs = Math.max(0, Math.round((oc.start - back) * 1000));const lab = `ovs${k}`;filters.push(- `[${idx}:a]atrim=${num(oc.in)}:${num(oc.out)},asetpts=PTS-STARTPTS,${tempo}${vol}${afades}` +- `aresample=44100,aformat=sample_fmts=fltp:channel_layouts=stereo,adelay=${delayMs}:all=1[${lab}]`+ `[${idx}:a]atrim=${num(oc.in - back * ospeed)}:${num(oc.out + ahead * ospeed)},` ++ `asetpts=PTS-STARTPTS,${tempo}${vol}${afades}` ++ `aresample=44100,aformat=sample_fmts=fltp:channel_layouts=stereo` ++ `${crosses},adelay=${delayMs}:all=1[${lab}]`);overlaySoundLabels.push(lab);}@@ -1702,7 +1818,11 @@ export async function runExport(// The joined clip audio and overlay-video audio duck too.let aLabel = duckOthers(aAcc);- const extraSound = [...soundLabels, ...overlaySoundLabels.map(duckOthers)];+ const extraSound = [+ ...soundLabels,+ ...overlaySoundLabels.map(duckOthers),+ ...crossHandleLabels.map(duckOthers),+ ];if (extraSound.length > 0) {const mixIn = [aLabel, ...extraSound].map((l) => `[${l}]`).join("");filters.push(diff --git a/site/src/cut/server/transcribe.ts b/site/src/cut/server/transcribe.tsindex f3f0f4e6..0956ae6d 100644--- a/site/src/cut/server/transcribe.ts+++ b/site/src/cut/server/transcribe.ts@@ -363,7 +363,7 @@ async function runTranscribe(job: TranscribeJob, spec: TranscribeSpec) {// transition fades the outgoing clip's tail while the next enters clean at// the cut, so the chain is a flat concat with tail fades — the mix runs// exactly the timeline's length and every cue lands where it plays. A- // sound dissolve's own fades stay out of it: they change no cue's time,+ // cross dissolve's own fades stay out of it: they change no cue's time,// and words are transcribed best at the level they were spoken.// Skipped entirely for an audio-only cut (no clips) — there is no clip-audio// chain to build, only the soundtrack, mixed below. (`spec.clips[0]` would be