🎥 Make videos programmatically with React
Skills: Fix caption timing guidance for edited clips
Jonny Burgerf46b453284d7
diff --git a/packages/skills/skills/remotion-captions/display-captions.md b/packages/skills/skills/remotion-captions/display-captions.mdindex 1bee82282b..0e4d1cb365 100644--- a/packages/skills/skills/remotion-captions/display-captions.md+++ b/packages/skills/skills/remotion-captions/display-captions.md@@ -33,8 +33,27 @@ Fetch its source code from https://www.remotion.dev/elements/captions/basic-capt## Displaying captions alongside video content-Put the captions next to the video so they stay in sync. The Caption editor-writes back to the array literal on the selected `<BasicCaptions>` source node,+Keep each clip's video and captions inside the same `<Sequence>`,+`<Series.Sequence>`, or `<TransitionSeries.Sequence>`. Put the clip's+`trimBefore`, `durationInFrames`, and `playbackRate` on that shared parent so+trimming or changing the speed affects both. For independently positioned+clips, put `from` on the shared `<Sequence>` too. Do not repeat the shared trim+or playback rate on the video or captions, which would apply it twice.++Keep caption timestamps in milliseconds relative to the original media file.+Do not add the clip's start time in the main composition or subtract its trim+from the caption data. The shared parent maps that source time to the edited+timeline. Give every captioned clip its own group and inline caption array;+do not combine captions from multiple clips into one overlay outside the+scene timeline. Moving, reordering, trimming, or deleting the group then+affects its video and captions together.++Putting captions inside a scene is insufficient if `trimBefore` or+`playbackRate` is still applied only to `<Video>`: its sibling captions do not+inherit those props. Apply them to the shared parent instead.++The Caption editor writes back to the array literal on the selected+`<BasicCaptions>` source node,so inline the captions directly in its `captions` prop. Passing a variable orcomponent prop such as `captions={captions}` leaves no source array for theeditor to update. If the captions were transcribed to a JSON file, copy its@@ -46,7 +65,7 @@ Give the caption area a width and position it over the video. In this```tsximport { Video } from "@remotion/media";-import { Composition, staticFile } from "remotion";+import { Composition, Series, staticFile, useVideoConfig } from "remotion";import { BasicCaptions } from "./basic-captions";export const MyComposition: React.FC = () => {@@ -54,7 +73,7 @@ export const MyComposition: React.FC = () => {<Compositionid="MyComposition"component={MyComponent}- durationInFrames={150}+ durationInFrames={750}fps={30}width={1920}height={1080}@@ -63,38 +82,105 @@ export const MyComposition: React.FC = () => {};export const MyComponent: React.FC = () => {+ const { fps } = useVideoConfig();+return (- <>- <Video name="Camera footage" src={staticFile("video.mp4")} />- <BasicCaptions- width={900}- style={{- position: "absolute",- bottom: 120,- translate: "510px 0px",- }}- captions={[- {- text: "Hello",- startMs: 0,- endMs: 400,- timestampMs: 200,- confidence: null,- },- {- text: " world",- startMs: 400,- endMs: 900,- timestampMs: 650,- confidence: null,- },- ]}- />- </>+ <Series>+ <Series.Sequence+ name="Opening"+ trimBefore={4 * fps}+ durationInFrames={20 * fps}+ premountFor={fps}+ >+ <Video+ name="Opening footage"+ src={staticFile("opening.mp4")}+ premountFor={fps}+ objectFit="cover"+ style={{ width: "100%", height: "100%" }}+ />+ <BasicCaptions+ name="Opening captions"+ premountFor={fps}+ width={900}+ style={{+ position: "absolute",+ bottom: 120,+ translate: "510px 0px",+ }}+ captions={[+ {+ text: "Hello",+ startMs: 5000,+ endMs: 5400,+ timestampMs: 5200,+ confidence: null,+ },+ {+ text: " world",+ startMs: 5400,+ endMs: 5900,+ timestampMs: 5650,+ confidence: null,+ },+ ]}+ />+ </Series.Sequence>+ <Series.Sequence+ name="Next clip"+ trimBefore={1 * fps}+ durationInFrames={5 * fps}+ premountFor={fps}+ >+ <Video+ name="Next footage"+ src={staticFile("next.mp4")}+ premountFor={fps}+ objectFit="cover"+ style={{ width: "100%", height: "100%" }}+ />+ <BasicCaptions+ name="Next captions"+ premountFor={fps}+ width={900}+ style={{+ position: "absolute",+ bottom: 120,+ translate: "510px 0px",+ }}+ captions={[+ {+ text: "Next",+ startMs: 2000,+ endMs: 2400,+ timestampMs: 2200,+ confidence: null,+ },+ {+ text: " clip",+ startMs: 2400,+ endMs: 2900,+ timestampMs: 2650,+ confidence: null,+ },+ ]}+ />+ </Series.Sequence>+ </Series>);};```+The opening skips four seconds of source footage, so its caption at `5000`+milliseconds appears one second into the scene. The next clip's timestamps+refer to `next.mp4`, not the main composition. Changing the opening's duration+moves the next clip and its captions together.++To change a captioned clip's speed, set `playbackRate` on its shared sequence.+`durationInFrames` selects a range of source frames; the group occupies+`durationInFrames / playbackRate` frames in its parent timeline. Update the+composition duration when changing the total timeline length.+If a caption has `pageBreakAfter: true`, the current page ends after that caption and the next caption starts a new page.## White-space preservationdiff --git a/packages/skills/skills/remotion-captions/transcribe-captions.md b/packages/skills/skills/remotion-captions/transcribe-captions.mdindex 49ef11795c..1751da46d1 100644--- a/packages/skills/skills/remotion-captions/transcribe-captions.md+++ b/packages/skills/skills/remotion-captions/transcribe-captions.md@@ -165,6 +165,11 @@ export const transcribeFile = async (file: File) => {};```-Transcribe each clip individually.+Transcribe each original source video or audio file individually. Keep the+resulting caption timestamps relative to the beginning of that source file,+including portions currently trimmed out of the edit. Do not add composition+offsets or flatten multiple transcripts into composition-wide timestamps.+Apply editing offsets, trims, and playback rates through each clip's shared+timing group when displaying its captions.See [Displaying captions](display-captions.md) for how to display the captions in Remotion.diff --git a/packages/skills/skills/remotion-markup/video-editing.md b/packages/skills/skills/remotion-markup/video-editing.mdindex af8eab9f92..36aa1f2e28 100644--- a/packages/skills/skills/remotion-markup/video-editing.md+++ b/packages/skills/skills/remotion-markup/video-editing.md@@ -28,10 +28,31 @@ directly in each clip's JSX, for example `trimBefore={4 * fps}`. Do not use aseparate fixed `FPS` constant: Studio must be able to update each clip's sourceexpression when the user trims or splits it.+## Clips with synchronized captions or overlays++For a clip with synchronized captions or overlays, make a shared timing group+the editable timeline item. Keep the video and its synchronized layers inside+one `<Sequence>` for independently positioned clips, or inside the same+`<Series.Sequence>` or `<TransitionSeries.Sequence>` for consecutive clips.++Put the clip's `trimBefore`, `durationInFrames`, and `playbackRate` on that+shared parent, plus `from` when using an independently positioned `<Sequence>`.+Apply the shared timing once, rather than repeating it on the children.+Moving, trimming, changing the speed, reordering, or deleting the group then+affects all its layers together. A shared wrapper has a purpose here even+though the children accept timing props themselves.++Keep each clip's captions in an inline array on its own `<BasicCaptions>` node,+with timestamps relative to the original media file. Do not put a combined+caption overlay outside the clip groups. See+[Displaying captions alongside video content](../remotion-captions/display-captions.md#displaying-captions-alongside-video-content)+for the complete example.+## Independently positioned clips-Place every `<Video>` directly in the composition and write its timing props-on the JSX node. Use literal frame counts when they are already known;+For clips without synchronized captions or overlays, place each `<Video>`+directly in the composition and write its timing props on the JSX node.+Use literal frame counts when they are already known;`from={0}` may be omitted:```tsx@@ -72,7 +93,9 @@ overlaps are therefore allowed.## Consecutive clips with `Series`Use `<Series>` when the clips should remain adjacent and do not need-transitions:+transitions. The example below has no synchronized captions; for captioned+clips, put the captions inside each sequence and move the video's trim to+that shared sequence as described above:```tsx<Series>@@ -116,7 +139,9 @@ Keep every sequence as a separate JSX node with a hardcoded `name` and## Consecutive clips with transitionsUse `<TransitionSeries>` when the timeline needs transitions or overlays.-Preserve the same one-source-node-per-clip structure:+Preserve the same one-source-node-per-clip structure. As with `<Series>`, put+a clip's synchronized captions inside its sequence and apply shared timing+to that parent:```tsx<TransitionSeries name="Video timeline">