🎥 Make videos programmatically with React
Document WebMCP tool schemas
JonnyBurger94c0ac93ad59
diff --git a/packages/docs/docs/ai/webmcp.mdx b/packages/docs/docs/ai/webmcp.mdxindex 48525b6430..00631ace57 100644--- a/packages/docs/docs/ai/webmcp.mdx+++ b/packages/docs/docs/ai/webmcp.mdx@@ -9,6 +9,20 @@ sidebar_label: WebMCPRemotion Studio exposes WebMCP tools that allow an AI agent to inspect and control the open composition.+## Capabilities++Agents can:++- See which items you currently have selected+- See selectable items and select them+- See available compositions and navigate+- Get the current HTML of your composition+- See current outlines of elements and their geometry+- See your current playhead position and playback state+- Play, pause and seek+- Set timeline zoom, playback rate, and mute state+- See, add and remove guides+## RequirementsStart the [Remotion Studio](/docs/studio) and open it in a browser with an AI agent that supports [WebMCP](https://webmcp.dev/).@@ -19,36 +33,440 @@ Out of the major harnesses, only ChatGPT Codex supports WebMCP. We hope more age## Available tools-| Tool | Input | Description |-| -------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |-| `get_compositions` | None | Returns mounted compositions as a nested tree of folders and compositions. |-| `select_composition` | `compositionName`: string | Opens a registered composition in Studio. |-| `get_sequences` | None | Returns mounted timeline sequences with IDs, hierarchy, timing, type and source context. |-| `select_sequence` | `sequenceId`: string | Selects and reveals a sequence returned by `get_sequences`. |-| `get_composition` | None | Returns composition metadata and the current frame, or `null` fields for other canvas content. |-| `get_canvas_html` | None | Returns the rendered composition HTML at the current frame, capped at 100,000 characters. |-| `get_outlines` | None | Returns active component outlines with source context and composition-space geometry. |-| `get_playback_state` | None | Returns the current playback, audio, looping and timeline zoom state. |-| `get_selection` | None | Returns the selection type, owning sequence metadata and source-code context. |-| `get_guides` | None | Returns the horizontal and vertical guides drawn over the canvas. |-| `set_guides_visible` | `visible`: boolean | Shows or hides all guides in the current composition. |-| `add_guide` | `orientation`: `horizontal` or `vertical`; `position`: number | Adds and shows a guide in the current composition. |-| `remove_guide` | `guideId`: string | Removes a guide from the current composition. |-| `seek_to_frame` | `frame`: a non-negative integer | Seeks the timeline. Values beyond the duration are clamped to the final frame. |-| `play` | None | Starts playback from the current frame. |-| `pause` | None | Pauses playback at the current frame. |-| `set_timeline_zoom` | `zoom`: a positive number | Sets the timeline zoom. Values outside the supported range are clamped. |-| `set_playback_rate` | `playbackRate`: `-4`, `-2`, `-1`, `-0.5`, `-0.25`, `0.25`, `0.5`, `1`, `1.5`, `2` or `4` | Sets the playback speed. Negative values play backwards. |-| `mute` | None | Mutes audio playback. |-| `unmute` | None | Unmutes audio playback. |--Tools that change playback require a composition to be open. `play` is unavailable for still compositions.--`get_selection` returns source-code context when exactly one sequence or property is selected. Its `selectedSequence` uses the same descriptor and `sequenceId` as `get_sequences`, including when a property, effect or keyframe inside that sequence is selected. Source context is `null` when there is no selection, multiple items are selected or a guide is selected.--`get_canvas_html` returns only the composition canvas markup, not the surrounding Studio interface. It reports the full `htmlLength` and whether the returned HTML was `truncated`. Canvas and WebGL pixels are not represented in HTML.--`get_outlines` returns the selectable component outlines active at the current frame. Each outline includes the `sequenceId` accepted by `select_sequence`, a node-path-based `outlineId`, source-code context, and a four-point polygon with its bounding box in composition pixels. Components without measurable DOM geometry are omitted.+Tools that modify the timeline, playback or guides require a composition to be open.++### `get_compositions`++Returns the mounted compositions in the same nested folder hierarchy shown in the Studio sidebar.++```ts title="Input"+{}+```++```ts title="Output"+type CompositionTreeItem =+ | {+ type: 'composition';+ compositionName: string;+ }+ | {+ type: 'folder';+ folderName: string;+ children: CompositionTreeItem[];+ };++{+ compositions: CompositionTreeItem[];+}+```++### `select_composition`++Opens a registered composition by name.++```ts title="Input"+{+ compositionName: string;+}+```++```ts title="Output"+{+ currentComposition: string;+}+```++### `get_sequences`++Returns the timeline sequences mounted for the current composition, including their hierarchy, timing, source stack and selection support.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string | null;+ sequences: Array<{+ sequenceId: string;+ name: string | null;+ type: 'sequence' | 'audio' | 'video' | 'image';+ parentSequenceId: string | null;+ depth: number;+ startFrame: number;+ endFrame: number;+ durationInFrames: number;+ stack: string | null;+ selectable: boolean;+ }>;+}+```++### `select_sequence`++Selects and reveals a sequence using a `sequenceId` returned by `get_sequences`.++```ts title="Input"+{+ sequenceId: string;+}+```++```ts title="Output"+{+ currentComposition: string;+ selectedSequence: {+ sequenceId: string;+ name: string | null;+ type: 'sequence' | 'audio' | 'video' | 'image';+ parentSequenceId: string | null;+ depth: number;+ startFrame: number;+ endFrame: number;+ durationInFrames: number;+ stack: string | null;+ selectable: boolean;+ };+}+```++### `get_composition`++Returns metadata and the current frame for the open composition. All fields are `null` when the canvas is showing an asset or other non-composition content.++```ts title="Input"+{}+```++```ts title="Output"+{+ compositionName: string | null;+ stack: string | null;+ durationInFrames: number | null;+ height: number | null;+ width: number | null;+ fps: number | null;+ currentFrame: number | null;+}+```++### `get_canvas_html`++Returns the rendered composition HTML at the current frame without the surrounding Studio interface. The returned HTML is capped at 100,000 characters. Canvas and WebGL pixels are not represented.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string | null;+ currentFrame: number | null;+ html: string | null;+ htmlLength: number | null;+ truncated: boolean;+}+```++### `get_outlines`++Returns the measurable selectable component outlines active at the current frame. Coordinates use composition pixels and do not change with the Studio canvas zoom.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string | null;+ currentFrame: number | null;+ outlines: Array<{+ outlineId: string;+ sequenceId: string;+ name: string | null;+ depth: number;+ context: string | null;+ geometry: {+ points: [+ {x: number; y: number},+ {x: number; y: number},+ {x: number; y: number},+ {x: number; y: number},+ ];+ boundingBox: {+ x: number;+ y: number;+ width: number;+ height: number;+ };+ };+ }>;+}+```++### `get_playback_state`++Returns the current playhead, playback, audio, looping and timeline zoom state. Playback fields are `null` when the canvas is not showing a composition.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string | null;+ currentFrame: number | null;+ playing: boolean | null;+ muted: boolean | null;+ volume: number | null;+ playbackRate: number | null;+ looping: boolean | null;+ timelineZoom: number | null;+}+```++### `get_selection`++Returns source-code context when exactly one supported timeline item is selected. `currentSelection` is `null` when there is no selection, multiple items are selected, a guide is selected or the source location cannot be resolved. `selectedSequence` identifies the owning sequence when a property, effect or keyframe is selected.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentFrame: number;+ currentComposition: string | null;+ currentSelection: string | null;+ selectionType:+ | 'guide'+ | 'sequence'+ | 'sequence-prop'+ | 'sequence-all-effects'+ | 'sequence-effect'+ | 'sequence-effect-prop'+ | 'keyframe'+ | 'easing'+ | null;+ selectedSequence: {+ sequenceId: string;+ name: string | null;+ type: 'sequence' | 'audio' | 'video' | 'image';+ parentSequenceId: string | null;+ depth: number;+ startFrame: number;+ endFrame: number;+ durationInFrames: number;+ stack: string | null;+ selectable: boolean;+ } | null;+}+```++### `get_guides`++Returns guides for the current composition. Vertical positions are x-coordinates from the left edge; horizontal positions are y-coordinates from the top edge.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string | null;+ guidesVisible: boolean;+ guides: Array<{+ id: string;+ orientation: 'horizontal' | 'vertical';+ position: number;+ visible: boolean;+ }>;+}+```++### `set_guides_visible`++Shows or hides every guide in the current composition.++```ts title="Input"+{+ visible: boolean;+}+```++```ts title="Output"+{+ currentComposition: string;+ guidesVisible: boolean;+}+```++### `add_guide`++Adds and shows a guide in the current composition. Positions use composition pixels.++```ts title="Input"+{+ orientation: 'horizontal' | 'vertical';+ position: number;+}+```++```ts title="Output"+{+ currentComposition: string;+ guide: {+ id: string;+ orientation: 'horizontal' | 'vertical';+ position: number;+ visible: true;+ };+}+```++### `remove_guide`++Removes a guide using an ID returned by `get_guides` or `add_guide`.++```ts title="Input"+{+ guideId: string;+}+```++```ts title="Output"+{+ currentComposition: string;+ guideId: string;+ removed: true;+}+```++### `play`++Starts playback from the current frame. Still compositions cannot be played.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string;+ playing: true;+}+```++### `pause`++Pauses playback at the current frame.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string;+ playing: false;+}+```++### `mute`++Mutes audio playback.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string;+ muted: true;+}+```++### `unmute`++Unmutes audio playback.++```ts title="Input"+{}+```++```ts title="Output"+{+ currentComposition: string;+ muted: false;+}+```++### `set_timeline_zoom`++Sets the timeline zoom factor. The requested value is clamped to the range supported by the composition duration. Still compositions do not have timeline zoom.++```ts title="Input"+{+ zoom: number;+}+```++`zoom` must be a positive finite number.++```ts title="Output"+{+ currentComposition: string;+ timelineZoom: number;+}+```++### `set_playback_rate`++Sets the playback multiplier. Negative values play backwards.++```ts title="Input"+{+ playbackRate:+ | -4+ | -2+ | -1+ | -0.5+ | -0.25+ | 0.25+ | 0.5+ | 1+ | 1.5+ | 2+ | 4;+}+```++```ts title="Output"+{+ currentComposition: string;+ playbackRate: number;+}+```++### `seek_to_frame`++Seeks to a zero-based frame. Values beyond the duration are clamped to the final frame.++```ts title="Input"+{+ frame: number;+}+```++`frame` must be a non-negative integer.++```ts title="Output"+{+ currentComposition: string;+ currentFrame: number;+}+```## Example prompts