Skip to content

🎥 Make videos programmatically with React

Document WebMCP tool schemas

JonnyBurger94c0ac93ad59
diff --git a/packages/docs/docs/ai/webmcp.mdx b/packages/docs/docs/ai/webmcp.mdx
index 48525b6430..00631ace57 100644
--- a/packages/docs/docs/ai/webmcp.mdx
+++ b/packages/docs/docs/ai/webmcp.mdx
@@ -9,6 +9,20 @@ sidebar_label: WebMCP
Remotion Studio exposes WebMCP tools that allow an AI agent to inspect and control the open composition.
+## Capabilities
+
+Agents can:
+
+- See which items you currently have selected
+- See selectable items and select them
+- See available compositions and navigate
+- Get the current HTML of your composition
+- See current outlines of elements and their geometry
+- See your current playhead position and playback state
+- Play, pause and seek
+- Set timeline zoom, playback rate, and mute state
+- See, add and remove guides
+
## Requirements
Start the [Remotion Studio](/docs/studio) and open it in a browser with an AI agent that supports [WebMCP](https://webmcp.dev/).
@@ -19,36 +33,440 @@ Out of the major harnesses, only ChatGPT Codex supports WebMCP. We hope more age
## Available tools
-| Tool | Input | Description |
-| -------------------- | ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
-| `get_compositions` | None | Returns mounted compositions as a nested tree of folders and compositions. |
-| `select_composition` | `compositionName`: string | Opens a registered composition in Studio. |
-| `get_sequences` | None | Returns mounted timeline sequences with IDs, hierarchy, timing, type and source context. |
-| `select_sequence` | `sequenceId`: string | Selects and reveals a sequence returned by `get_sequences`. |
-| `get_composition` | None | Returns composition metadata and the current frame, or `null` fields for other canvas content. |
-| `get_canvas_html` | None | Returns the rendered composition HTML at the current frame, capped at 100,000 characters. |
-| `get_outlines` | None | Returns active component outlines with source context and composition-space geometry. |
-| `get_playback_state` | None | Returns the current playback, audio, looping and timeline zoom state. |
-| `get_selection` | None | Returns the selection type, owning sequence metadata and source-code context. |
-| `get_guides` | None | Returns the horizontal and vertical guides drawn over the canvas. |
-| `set_guides_visible` | `visible`: boolean | Shows or hides all guides in the current composition. |
-| `add_guide` | `orientation`: `horizontal` or `vertical`; `position`: number | Adds and shows a guide in the current composition. |
-| `remove_guide` | `guideId`: string | Removes a guide from the current composition. |
-| `seek_to_frame` | `frame`: a non-negative integer | Seeks the timeline. Values beyond the duration are clamped to the final frame. |
-| `play` | None | Starts playback from the current frame. |
-| `pause` | None | Pauses playback at the current frame. |
-| `set_timeline_zoom` | `zoom`: a positive number | Sets the timeline zoom. Values outside the supported range are clamped. |
-| `set_playback_rate` | `playbackRate`: `-4`, `-2`, `-1`, `-0.5`, `-0.25`, `0.25`, `0.5`, `1`, `1.5`, `2` or `4` | Sets the playback speed. Negative values play backwards. |
-| `mute` | None | Mutes audio playback. |
-| `unmute` | None | Unmutes audio playback. |
-
-Tools that change playback require a composition to be open. `play` is unavailable for still compositions.
-
-`get_selection` returns source-code context when exactly one sequence or property is selected. Its `selectedSequence` uses the same descriptor and `sequenceId` as `get_sequences`, including when a property, effect or keyframe inside that sequence is selected. Source context is `null` when there is no selection, multiple items are selected or a guide is selected.
-
-`get_canvas_html` returns only the composition canvas markup, not the surrounding Studio interface. It reports the full `htmlLength` and whether the returned HTML was `truncated`. Canvas and WebGL pixels are not represented in HTML.
-
-`get_outlines` returns the selectable component outlines active at the current frame. Each outline includes the `sequenceId` accepted by `select_sequence`, a node-path-based `outlineId`, source-code context, and a four-point polygon with its bounding box in composition pixels. Components without measurable DOM geometry are omitted.
+Tools that modify the timeline, playback or guides require a composition to be open.
+
+### `get_compositions`
+
+Returns the mounted compositions in the same nested folder hierarchy shown in the Studio sidebar.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+type CompositionTreeItem =
+ | {
+ type: 'composition';
+ compositionName: string;
+ }
+ | {
+ type: 'folder';
+ folderName: string;
+ children: CompositionTreeItem[];
+ };
+
+{
+ compositions: CompositionTreeItem[];
+}
+```
+
+### `select_composition`
+
+Opens a registered composition by name.
+
+```ts title="Input"
+{
+ compositionName: string;
+}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+}
+```
+
+### `get_sequences`
+
+Returns the timeline sequences mounted for the current composition, including their hierarchy, timing, source stack and selection support.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string | null;
+ sequences: Array<{
+ sequenceId: string;
+ name: string | null;
+ type: 'sequence' | 'audio' | 'video' | 'image';
+ parentSequenceId: string | null;
+ depth: number;
+ startFrame: number;
+ endFrame: number;
+ durationInFrames: number;
+ stack: string | null;
+ selectable: boolean;
+ }>;
+}
+```
+
+### `select_sequence`
+
+Selects and reveals a sequence using a `sequenceId` returned by `get_sequences`.
+
+```ts title="Input"
+{
+ sequenceId: string;
+}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ selectedSequence: {
+ sequenceId: string;
+ name: string | null;
+ type: 'sequence' | 'audio' | 'video' | 'image';
+ parentSequenceId: string | null;
+ depth: number;
+ startFrame: number;
+ endFrame: number;
+ durationInFrames: number;
+ stack: string | null;
+ selectable: boolean;
+ };
+}
+```
+
+### `get_composition`
+
+Returns metadata and the current frame for the open composition. All fields are `null` when the canvas is showing an asset or other non-composition content.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ compositionName: string | null;
+ stack: string | null;
+ durationInFrames: number | null;
+ height: number | null;
+ width: number | null;
+ fps: number | null;
+ currentFrame: number | null;
+}
+```
+
+### `get_canvas_html`
+
+Returns the rendered composition HTML at the current frame without the surrounding Studio interface. The returned HTML is capped at 100,000 characters. Canvas and WebGL pixels are not represented.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string | null;
+ currentFrame: number | null;
+ html: string | null;
+ htmlLength: number | null;
+ truncated: boolean;
+}
+```
+
+### `get_outlines`
+
+Returns the measurable selectable component outlines active at the current frame. Coordinates use composition pixels and do not change with the Studio canvas zoom.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string | null;
+ currentFrame: number | null;
+ outlines: Array<{
+ outlineId: string;
+ sequenceId: string;
+ name: string | null;
+ depth: number;
+ context: string | null;
+ geometry: {
+ points: [
+ {x: number; y: number},
+ {x: number; y: number},
+ {x: number; y: number},
+ {x: number; y: number},
+ ];
+ boundingBox: {
+ x: number;
+ y: number;
+ width: number;
+ height: number;
+ };
+ };
+ }>;
+}
+```
+
+### `get_playback_state`
+
+Returns the current playhead, playback, audio, looping and timeline zoom state. Playback fields are `null` when the canvas is not showing a composition.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string | null;
+ currentFrame: number | null;
+ playing: boolean | null;
+ muted: boolean | null;
+ volume: number | null;
+ playbackRate: number | null;
+ looping: boolean | null;
+ timelineZoom: number | null;
+}
+```
+
+### `get_selection`
+
+Returns source-code context when exactly one supported timeline item is selected. `currentSelection` is `null` when there is no selection, multiple items are selected, a guide is selected or the source location cannot be resolved. `selectedSequence` identifies the owning sequence when a property, effect or keyframe is selected.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentFrame: number;
+ currentComposition: string | null;
+ currentSelection: string | null;
+ selectionType:
+ | 'guide'
+ | 'sequence'
+ | 'sequence-prop'
+ | 'sequence-all-effects'
+ | 'sequence-effect'
+ | 'sequence-effect-prop'
+ | 'keyframe'
+ | 'easing'
+ | null;
+ selectedSequence: {
+ sequenceId: string;
+ name: string | null;
+ type: 'sequence' | 'audio' | 'video' | 'image';
+ parentSequenceId: string | null;
+ depth: number;
+ startFrame: number;
+ endFrame: number;
+ durationInFrames: number;
+ stack: string | null;
+ selectable: boolean;
+ } | null;
+}
+```
+
+### `get_guides`
+
+Returns guides for the current composition. Vertical positions are x-coordinates from the left edge; horizontal positions are y-coordinates from the top edge.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string | null;
+ guidesVisible: boolean;
+ guides: Array<{
+ id: string;
+ orientation: 'horizontal' | 'vertical';
+ position: number;
+ visible: boolean;
+ }>;
+}
+```
+
+### `set_guides_visible`
+
+Shows or hides every guide in the current composition.
+
+```ts title="Input"
+{
+ visible: boolean;
+}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ guidesVisible: boolean;
+}
+```
+
+### `add_guide`
+
+Adds and shows a guide in the current composition. Positions use composition pixels.
+
+```ts title="Input"
+{
+ orientation: 'horizontal' | 'vertical';
+ position: number;
+}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ guide: {
+ id: string;
+ orientation: 'horizontal' | 'vertical';
+ position: number;
+ visible: true;
+ };
+}
+```
+
+### `remove_guide`
+
+Removes a guide using an ID returned by `get_guides` or `add_guide`.
+
+```ts title="Input"
+{
+ guideId: string;
+}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ guideId: string;
+ removed: true;
+}
+```
+
+### `play`
+
+Starts playback from the current frame. Still compositions cannot be played.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ playing: true;
+}
+```
+
+### `pause`
+
+Pauses playback at the current frame.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ playing: false;
+}
+```
+
+### `mute`
+
+Mutes audio playback.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ muted: true;
+}
+```
+
+### `unmute`
+
+Unmutes audio playback.
+
+```ts title="Input"
+{}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ muted: false;
+}
+```
+
+### `set_timeline_zoom`
+
+Sets the timeline zoom factor. The requested value is clamped to the range supported by the composition duration. Still compositions do not have timeline zoom.
+
+```ts title="Input"
+{
+ zoom: number;
+}
+```
+
+`zoom` must be a positive finite number.
+
+```ts title="Output"
+{
+ currentComposition: string;
+ timelineZoom: number;
+}
+```
+
+### `set_playback_rate`
+
+Sets the playback multiplier. Negative values play backwards.
+
+```ts title="Input"
+{
+ playbackRate:
+ | -4
+ | -2
+ | -1
+ | -0.5
+ | -0.25
+ | 0.25
+ | 0.5
+ | 1
+ | 1.5
+ | 2
+ | 4;
+}
+```
+
+```ts title="Output"
+{
+ currentComposition: string;
+ playbackRate: number;
+}
+```
+
+### `seek_to_frame`
+
+Seeks to a zero-based frame. Values beyond the duration are clamped to the final frame.
+
+```ts title="Input"
+{
+ frame: number;
+}
+```
+
+`frame` must be a non-negative integer.
+
+```ts title="Output"
+{
+ currentComposition: string;
+ currentFrame: number;
+}
+```
## Example prompts