Skip to content

🎥 Make videos programmatically with React

`@remotion/canvas-capture-extension`: Add `setup_shot` for agentic framing

Jonny Burger2c9de7ac5dce
diff --git a/.agents/skills/canvas-capture-extension/SKILL.md b/.agents/skills/canvas-capture-extension/SKILL.md
index 15d11faf6b..cca1fc48b1 100644
--- a/.agents/skills/canvas-capture-extension/SKILL.md
+++ b/.agents/skills/canvas-capture-extension/SKILL.md
@@ -53,7 +53,7 @@ To select another Chrome 157.0.8080.0 or newer browser, set
## Reload in Chrome
- Launch Chrome 157.0.8080.0 or newer with the
- dedicated Canvas Capture profile, then open `chrome://extensions` manually.
+ dedicated Canvas Capture profile, then open `chrome://extensions`.
- Enable **Developer mode** on `chrome://extensions`.
- If **Remotion Canvas Capture** already points to the durable directory, click
**Reload**.
@@ -74,5 +74,16 @@ To select another Chrome 157.0.8080.0 or newer browser, set
experimental HTML-in-canvas API is unavailable.
Do not edit Chrome's `Preferences` or `Secure Preferences` files to move or
-reload the extension, and do not automate the Chrome UI. Let the user use
-Chrome's extensions page so its integrity metadata stays valid.
+reload the extension. Use Chrome's extensions page so its integrity metadata
+stays valid. Browser UI automation is allowed for loading, reloading, and
+activating Canvas Capture; do not require the user to perform those clicks.
+
+## Prepare a shot
+
+Open the requested webpage in the compatible recording browser and activate
+Canvas Capture through its toolbar icon or the Extensions menu. This exposes
+the `setup_shot`, `get_shot`, and `save_recording` WebMCP tools in that tab.
+
+Prepare the page and viewport, call `setup_shot`, and inspect a still preview
+with `get_shot`. Leave the browser open with the shot ready for the user to
+record. Start and stop recording manually so the user controls cursor movement.
diff --git a/packages/canvas-capture-extension/README.md b/packages/canvas-capture-extension/README.md
index fe4af11c93..3ae4555c20 100644
--- a/packages/canvas-capture-extension/README.md
+++ b/packages/canvas-capture-extension/README.md
@@ -66,6 +66,12 @@ extension, so its path and extension ID remain stable across worktrees.
## Build and install
+The extension exposes `setup_shot`, `get_shot`, and `save_recording` to WebMCP-aware
+agents after its icon is clicked. Agents can frame the viewport, an element, or an
+exact rectangle, inspect a PNG still preview, and save a completed recording.
+Recording starts and stops through the manual controls. See
+[Agentic shot setup](https://www.remotion.dev/docs/canvas-capture/agentic-shot-setup).
+
1. From the repository root, run `.agents/skills/canvas-capture-extension/scripts/rebuild-extension.sh --repo "$PWD"`. This creates the production WXT bundle and installs the complete unpacked extension outside the worktree. Building does not require an installed browser.
2. In Chrome 157.0.8080.0 or newer, open `chrome://extensions`, enable **Developer mode**, and choose **Load unpacked**.
3. Select `/Users/jonathanburger/Applications/Remotion Canvas Capture Extension`.
diff --git a/packages/canvas-capture-extension/src/background.ts b/packages/canvas-capture-extension/src/background.ts
index 9a39769edd..6dc844345d 100644
--- a/packages/canvas-capture-extension/src/background.ts
+++ b/packages/canvas-capture-extension/src/background.ts
@@ -30,6 +30,12 @@ export const startBackground = () => {
await browser.tabs.sendMessage(tab.id, request);
}
+ await browser.scripting.executeScript({
+ target: {tabId: tab.id},
+ files: ['/shot-tools.js'],
+ world: 'MAIN',
+ });
+
await browser.action.setBadgeText({tabId: tab.id, text: ''});
await browser.action.setTitle({
tabId: tab.id,
diff --git a/packages/canvas-capture-extension/src/capture.ts b/packages/canvas-capture-extension/src/capture.ts
index 36f768f386..4f6081bddd 100644
--- a/packages/canvas-capture-extension/src/capture.ts
+++ b/packages/canvas-capture-extension/src/capture.ts
@@ -342,6 +342,30 @@ export class PageCapture {
this.#requestPaint();
};
+ preview = async () => {
+ // Wait for the drawable subtree to paint before reading its pixels.
+ await new Promise<void>((resolve, reject) => {
+ let timeout = 0;
+ const onPaint = () => {
+ clearTimeout(timeout);
+ resolve();
+ };
+
+ timeout = window.setTimeout(() => {
+ this.#wrapped.canvas.removeEventListener('paint', onPaint);
+ reject(new Error('The shot preview did not paint in time.'));
+ }, 5000);
+ this.#wrapped.canvas.addEventListener('paint', onPaint, {once: true});
+ this.#requestPaint();
+ });
+ if (this.#paintError) {
+ throw this.#paintError;
+ }
+
+ this.#draw();
+ return this.#captureCanvas.convertToBlob({type: 'image/png'});
+ };
+
stop = async () => {
try {
if (this.#paintError) {
diff --git a/packages/canvas-capture-extension/src/content.ts b/packages/canvas-capture-extension/src/content.ts
index b15a27d185..68e0b1a20f 100644
--- a/packages/canvas-capture-extension/src/content.ts
+++ b/packages/canvas-capture-extension/src/content.ts
@@ -22,6 +22,12 @@ import {
type SelectionHandle,
type SelectionRectangle,
} from './selection';
+import {
+ shotRequestEvent,
+ shotResponseEvent,
+ shotTools,
+ type ShotToolName,
+} from './shot-tools';
type ExtensionController = {
readonly handleRequest: (
@@ -648,6 +654,7 @@ export const startContent = () => {
readonly offsetY: number;
} | null = null;
let encoderSupportCheckId = 0;
+ let requestedFormat: CaptureFormat | null = null;
let encoderSupportKey: string | null = null;
let recordingStartedAt: number | null = null;
let recordingDurationInterval: number | null = null;
@@ -779,6 +786,25 @@ export const startContent = () => {
return;
}
+ if (requestedFormat !== null) {
+ format = requestedFormat;
+ encoderSupport = 'checking';
+ const canEncode = await canEncodeCapture(format, outputSize);
+ if (checkId !== encoderSupportCheckId) {
+ return;
+ }
+
+ encoderSupport = canEncode ? 'supported' : 'unsupported';
+ encoderSupportKey = supportKey;
+ setStatus(
+ canEncode
+ ? `Ready to record ${getFormatLabel(format)} at ${outputSize.width}×${outputSize.height}.`
+ : `${getFormatLabel(format)} is unavailable at ${outputSize.width}×${outputSize.height}.`,
+ !canEncode,
+ );
+ return;
+ }
+
encoderSupport = 'checking';
setStatus(
`Checking H.264 MP4 support at ${outputSize.width}×${outputSize.height}…`,
@@ -927,6 +953,8 @@ export const startContent = () => {
};
const getState = (): CaptureControllerState => ({
+ crop: resolveCaptureTarget()?.crop ?? null,
+ viewport: {width: window.innerWidth, height: window.innerHeight},
supported,
selecting,
hasTarget: selectedTarget !== null,
@@ -1032,6 +1060,7 @@ export const startContent = () => {
startingRecording ||
completedRecording ||
finalizing ||
+ controlsBusy ||
selectionDrag
) {
return;
@@ -1477,7 +1506,7 @@ export const startContent = () => {
};
controlsClose.addEventListener('click', () => {
- if (capture || startingRecording || finalizing) {
+ if (capture || startingRecording || finalizing || controlsBusy) {
return;
}
@@ -1660,6 +1689,331 @@ export const startContent = () => {
updateControls();
});
});
+ const handleShotRequest = async (
+ name: ShotToolName,
+ input: Record<string, unknown>,
+ ) => {
+ if (name === 'get_shot') {
+ if (input.preview !== undefined && typeof input.preview !== 'boolean') {
+ throw new Error('preview must be a boolean.');
+ }
+
+ await refreshEncoderSupport();
+ updateControls();
+ const state = getState();
+ if (!input.preview) {
+ return state;
+ }
+
+ if (
+ capture ||
+ startingRecording ||
+ finalizing ||
+ controlsBusy ||
+ selecting ||
+ selectionDrag
+ ) {
+ throw new Error(
+ 'Finish the current recording or selection before previewing the shot.',
+ );
+ }
+
+ const previewTarget = resolveCaptureTarget();
+ if (!previewTarget) {
+ throw new Error('Set up a shot before requesting a preview.');
+ }
+
+ controlsBusy = true;
+ updateControls();
+ let previewCapture: PageCapture | null = null;
+ try {
+ previewCapture = new PageCapture({
+ crop: previewTarget.crop,
+ scale,
+ format,
+ });
+ const blob = await previewCapture.preview();
+ const dataUrl = await new Promise<string>((resolve, reject) => {
+ const reader = new FileReader();
+ reader.onload = () => resolve(String(reader.result));
+ reader.onerror = () =>
+ reject(new Error('Could not read the shot preview.'));
+ reader.readAsDataURL(blob);
+ });
+ return {
+ content: [
+ {type: 'text', text: JSON.stringify(state)},
+ {
+ type: 'image',
+ mimeType: 'image/png',
+ data: dataUrl.slice(dataUrl.indexOf(',') + 1),
+ },
+ ],
+ };
+ } finally {
+ previewCapture?.restore();
+ controlsBusy = false;
+ updateControls();
+ }
+ }
+
+ if (capture || startingRecording || finalizing || controlsBusy) {
+ throw new Error('Wait until recording and finalization have finished.');
+ }
+
+ if (name === 'save_recording') {
+ const {destination} = input;
+ if (
+ destination !== 'download' &&
+ destination !== 'new' &&
+ destination !== 'convert'
+ ) {
+ throw new Error('destination must be download, new, or convert.');
+ }
+
+ if (!completedRecording) {
+ throw new Error(
+ 'No completed recording is available. Stop recording manually before saving.',
+ );
+ }
+
+ await handleCompletedRecording(destination);
+ const state = getState();
+ if (state.error) {
+ throw new Error(state.status);
+ }
+
+ return state;
+ }
+
+ if (completedRecording) {
+ throw new Error(
+ 'Save or discard the completed recording before setting up another shot.',
+ );
+ }
+
+ const {target, resolution, format: nextFormat} = input;
+ if (
+ typeof target !== 'object' ||
+ target === null ||
+ !('type' in target)
+ ) {
+ throw new Error(
+ 'target must describe a viewport, element, or rectangle.',
+ );
+ }
+
+ let crop: CaptureCrop;
+ if (target.type === 'viewport') {
+ crop = {
+ left: 0,
+ top: 0,
+ width: window.innerWidth,
+ height: window.innerHeight,
+ };
+ } else if (target.type === 'element') {
+ if (
+ !('selector' in target) ||
+ typeof target.selector !== 'string' ||
+ !target.selector.trim()
+ ) {
+ throw new Error('An element target requires a CSS selector.');
+ }
+
+ const padding = 'padding' in target ? target.padding : 0;
+ if (
+ typeof padding !== 'number' ||
+ !Number.isFinite(padding) ||
+ padding < 0
+ ) {
+ throw new Error('padding must be a finite non-negative number.');
+ }
+
+ const elements = document.querySelectorAll(target.selector);
+ if (elements.length !== 1) {
+ throw new Error(
+ `The selector matched ${elements.length} elements. Choose a selector matching exactly one element.`,
+ );
+ }
+
+ const element = elements[0];
+ if (!element) {
+ throw new Error('The selected element is unavailable.');
+ }
+
+ const rect = element.getBoundingClientRect();
+ const elementStyle = window.getComputedStyle(element);
+ if (
+ rect.width <= 0 ||
+ rect.height <= 0 ||
+ elementStyle.visibility === 'hidden' ||
+ elementStyle.visibility === 'collapse' ||
+ Number(elementStyle.opacity) === 0 ||
+ rect.left < 0 ||
+ rect.top < 0 ||
+ rect.right > window.innerWidth ||
+ rect.bottom > window.innerHeight
+ ) {
+ throw new Error(
+ 'The element must be visible and fully inside the viewport. Scroll or prepare the page before setting up the shot.',
+ );
+ }
+
+ const left = Math.max(0, rect.left - padding);
+ const top = Math.max(0, rect.top - padding);
+ crop = {
+ left,
+ top,
+ width: Math.min(window.innerWidth, rect.right + padding) - left,
+ height: Math.min(window.innerHeight, rect.bottom + padding) - top,
+ };
+ } else if (target.type === 'rectangle') {
+ if (
+ !('left' in target) ||
+ !('top' in target) ||
+ !('width' in target) ||
+ !('height' in target) ||
+ typeof target.left !== 'number' ||
+ typeof target.top !== 'number' ||
+ typeof target.width !== 'number' ||
+ typeof target.height !== 'number'
+ ) {
+ throw new Error(
+ 'A rectangle requires numeric left, top, width, and height.',
+ );
+ }
+
+ crop = {
+ left: target.left,
+ top: target.top,
+ width: target.width,
+ height: target.height,
+ };
+ } else {
+ throw new Error('target.type must be viewport, element, or rectangle.');
+ }
+
+ if (
+ !Object.values(crop).every(Number.isFinite) ||
+ crop.left < 0 ||
+ crop.top < 0 ||
+ crop.width < minimumCaptureArea.width ||
+ crop.height < minimumCaptureArea.height ||
+ crop.left + crop.width > window.innerWidth ||
+ crop.top + crop.height > window.innerHeight
+ ) {
+ throw new Error(
+ 'The crop must be at least 10×10 CSS pixels and entirely inside the viewport.',
+ );
+ }
+
+ let outputBounds = defaultResolution;
+ if (resolution !== undefined) {
+ if (
+ typeof resolution !== 'object' ||
+ resolution === null ||
+ !('width' in resolution) ||
+ !('height' in resolution) ||
+ typeof resolution.width !== 'number' ||
+ typeof resolution.height !== 'number' ||
+ !Number.isInteger(resolution.width) ||
+ !Number.isInteger(resolution.height) ||
+ resolution.width < 2 ||
+ resolution.height < 2
+ ) {
+ throw new Error(
+ 'resolution requires integer width and height of at least 2 pixels.',
+ );
+ }
+
+ outputBounds = {width: resolution.width, height: resolution.height};
+ }
+
+ if (
+ nextFormat !== undefined &&
+ nextFormat !== 'mp4' &&
+ nextFormat !== 'webm'
+ ) {
+ throw new Error('format must be mp4 or webm.');
+ }
+
+ const nextScale = Math.min(
+ outputBounds.width / crop.width,
+ outputBounds.height / crop.height,
+ maxZoom,
+ );
+ getCapturePreflight({crop, scale: nextScale});
+ if (selectionDrag) {
+ highlight.releasePointerCapture(selectionDrag.pointerId);
+ selectionDrag = null;
+ }
+
+ if (selecting) {
+ cancelSelection();
+ }
+
+ selectedTarget = {type: 'page-crop', crop};
+ requestedFormat = nextFormat ?? null;
+ useDefaultResolution = false;
+ scale = nextScale;
+ encoderSupportKey = null;
+ controlsDismissed = false;
+ controlsBusy = true;
+ updateHighlight();
+ updateControls();
+ try {
+ await refreshEncoderSupport();
+ return getState();
+ } finally {
+ controlsBusy = false;
+ updateControls();
+ }
+ };
+
+ // Only shot preparation, inspection, and saving cross into the page world.
+ // Recording commands stay in the extension's manual controls.
+ document.addEventListener(shotRequestEvent, (event) => {
+ if (!(event instanceof CustomEvent) || typeof event.detail !== 'string') {
+ return;
+ }
+
+ let request;
+ try {
+ request = JSON.parse(event.detail);
+ } catch {
+ return;
+ }
+
+ if (
+ typeof request?.id !== 'string' ||
+ !shotTools.some((tool) => tool.name === request.name) ||
+ typeof request.input !== 'object' ||
+ request.input === null ||
+ Array.isArray(request.input)
+ ) {
+ return;
+ }
+
+ handleShotRequest(request.name, request.input)
+ .then((result) =>
+ document.dispatchEvent(
+ new CustomEvent(shotResponseEvent, {
+ detail: JSON.stringify({id: request.id, result}),
+ }),
+ ),
+ )
+ .catch((error) =>
+ document.dispatchEvent(
+ new CustomEvent(shotResponseEvent, {
+ detail: JSON.stringify({
+ id: request.id,
+ error: error instanceof Error ? error.message : String(error),
+ }),
+ }),
+ ),
+ );
+ });
+
return {
handleRequest,
};
diff --git a/packages/canvas-capture-extension/src/entrypoints/shot-tools.ts b/packages/canvas-capture-extension/src/entrypoints/shot-tools.ts
new file mode 100644
index 0000000000..e9a5156be4
--- /dev/null
+++ b/packages/canvas-capture-extension/src/entrypoints/shot-tools.ts
@@ -0,0 +1,95 @@
+import {defineUnlistedScript} from 'wxt/utils/define-unlisted-script';
+import {shotRequestEvent, shotResponseEvent, shotTools} from '../shot-tools';
+
+type ModelContext = {
+ readonly registerTool: (
+ tool: (typeof shotTools)[number] & {
+ readonly execute: (input: Record<string, unknown>) => Promise<unknown>;
+ },
+ options: {readonly signal: AbortSignal},
+ ) => Promise<void>;
+};
+
+export default defineUnlistedScript(() => {
+ const {modelContext} = document as Document & {
+ readonly modelContext: ModelContext | undefined;
+ };
+ if (typeof modelContext?.registerTool !== 'function') {
+ return;
+ }
+
+ const toolsWindow = window as unknown as Window & {
+ __remotionCanvasCaptureTools: AbortController | undefined;
+ };
+ if (toolsWindow.__remotionCanvasCaptureTools) {
+ return;
+ }
+
+ const controller = new AbortController();
+ toolsWindow.__remotionCanvasCaptureTools = controller;
+ Promise.all(
+ shotTools.map((tool) =>
+ modelContext.registerTool(
+ {
+ ...tool,
+ execute: (input) =>
+ new Promise((resolve, reject) => {
+ const id = crypto.randomUUID();
+ let timeout = 0;
+ const onResponse = (event: Event) => {
+ if (
+ !(event instanceof CustomEvent) ||
+ typeof event.detail !== 'string'
+ ) {
+ return;
+ }
+
+ let response;
+ try {
+ response = JSON.parse(event.detail);
+ } catch {
+ return;
+ }
+
+ if (response?.id !== id) {
+ return;
+ }
+
+ clearTimeout(timeout);
+ document.removeEventListener(shotResponseEvent, onResponse);
+ if (typeof response.error === 'string') {
+ reject(new Error(response.error));
+ } else {
+ resolve(response.result);
+ }
+ };
+
+ timeout = window.setTimeout(() => {
+ document.removeEventListener(shotResponseEvent, onResponse);
+ reject(
+ new Error(
+ 'Canvas Capture did not respond. Reopen the extension controls and retry.',
+ ),
+ );
+ }, 30_000);
+ document.addEventListener(shotResponseEvent, onResponse);
+ document.dispatchEvent(
+ new CustomEvent(shotRequestEvent, {
+ detail: JSON.stringify({id, name: tool.name, input}),
+ }),
+ );
+ }),
+ },
+ {signal: controller.signal},
+ ),
+ ),
+ ).catch((error) => {
+ controller.abort();
+ toolsWindow.__remotionCanvasCaptureTools = undefined;
+ // eslint-disable-next-line no-console -- Surface tool registration failures in DevTools.
+ console.error(
+ '[Remotion Canvas Capture] Could not register shot tools',
+ error,
+ );
+ });
+});
diff --git a/packages/canvas-capture-extension/src/messages.ts b/packages/canvas-capture-extension/src/messages.ts
index 35f91c7eb5..373110faa4 100644
--- a/packages/canvas-capture-extension/src/messages.ts
+++ b/packages/canvas-capture-extension/src/messages.ts
@@ -4,6 +4,13 @@ export const captureControllerMessageType =
export type CaptureFormat = 'mp4' | 'webm';
export type CaptureControllerState = {
+ readonly crop: {
+ readonly left: number;
+ readonly top: number;
+ readonly width: number;
+ readonly height: number;
+ } | null;
+ readonly viewport: {readonly width: number; readonly height: number};
readonly supported: boolean;
readonly selecting: boolean;
readonly hasTarget: boolean;
diff --git a/packages/canvas-capture-extension/src/shot-tools.ts b/packages/canvas-capture-extension/src/shot-tools.ts
new file mode 100644
index 0000000000..1843c52cd8
--- /dev/null
+++ b/packages/canvas-capture-extension/src/shot-tools.ts
@@ -0,0 +1,117 @@
+export const shotRequestEvent = 'remotion-canvas-capture-shot-request';
+export const shotResponseEvent = 'remotion-canvas-capture-shot-response';
+
+export type ShotToolName = 'setup_shot' | 'get_shot' | 'save_recording';
+
+export const shotTools: {
+ readonly name: ShotToolName;
+ readonly title: string;
+ readonly description: string;
+ readonly inputSchema: Record<string, unknown>;
+ readonly annotations: {readonly readOnlyHint: boolean};
+}[] = [
+ {
+ name: 'setup_shot',
+ title: 'Set up Canvas Capture shot',
+ description:
+ 'Frame a Canvas Capture shot without recording or moving the cursor. Prepare the page and scroll first. Element bounds are resolved once; the crop stays fixed in viewport CSS pixels. The user starts and stops recording manually.',
+ inputSchema: {
+ type: 'object',
+ properties: {
+ target: {
+ oneOf: [
+ {
+ type: 'object',
+ properties: {type: {const: 'viewport'}},
+ required: ['type'],
+ additionalProperties: false,
+ },
+ {
+ type: 'object',
+ properties: {
+ type: {const: 'element'},
+ selector: {
+ type: 'string',
+ description:
+ 'A CSS selector matching exactly one visible element.',
+ },
+ padding: {
+ type: 'number',
+ minimum: 0,
+ description:
+ 'Padding in CSS pixels. Defaults to 0. Clipped to the viewport.',
+ },
+ },
+ required: ['type', 'selector'],
+ additionalProperties: false,
+ },
+ {
+ type: 'object',
+ properties: {
+ type: {const: 'rectangle'},
+ left: {type: 'number', minimum: 0},
+ top: {type: 'number', minimum: 0},
+ width: {type: 'number', minimum: 10},
+ height: {type: 'number', minimum: 10},
+ },
+ required: ['type', 'left', 'top', 'width', 'height'],
+ additionalProperties: false,
+ },
+ ],
+ },
+ resolution: {
+ type: 'object',
+ description:
+ 'Output bounding box in pixels. Fits the crop inside it, preserving aspect ratio. Defaults to 2560×1440.',
+ properties: {
+ width: {type: 'integer', minimum: 2},
+ height: {type: 'integer', minimum: 2},
+ },
+ required: ['width', 'height'],
+ additionalProperties: false,
+ },
+ format: {
+ type: 'string',
+ enum: ['mp4', 'webm'],
+ description:
+ 'Omit to prefer MP4 with WebM fallback. An explicit format never falls back.',
+ },
+ },
+ required: ['target'],
+ additionalProperties: false,
+ },
+ annotations: {readOnlyHint: false},
+ },
+ {
+ name: 'get_shot',
+ title: 'Inspect Canvas Capture shot',
+ description:
+ 'Read the crop, viewport, output size and recording readiness. Optionally return a PNG still preview without recording. Preview is unavailable during recording or finalization.',
+ inputSchema: {
+ type: 'object',
+ properties: {
+ preview: {
+ type: 'boolean',
+ description: 'Include a PNG still preview. Defaults to false.',
+ },
+ },
+ additionalProperties: false,
+ },
+ annotations: {readOnlyHint: true},
+ },
+ {
+ name: 'save_recording',
+ title: 'Save completed Canvas Capture recording',
+ description:
+ 'Download a completed recording or open it in Remotion Browser Studio or Convert. The user must have stopped recording manually first. This tool cannot start or stop recording.',
+ inputSchema: {
+ type: 'object',
+ properties: {
+ destination: {type: 'string', enum: ['download', 'new', 'convert']},
+ },
+ required: ['destination'],
+ additionalProperties: false,
+ },
+ annotations: {readOnlyHint: false},
+ },
+];
diff --git a/packages/docs/docs/canvas-capture/TableOfContents.tsx b/packages/docs/docs/canvas-capture/TableOfContents.tsx
index e4de48813c..4f436024a6 100644
--- a/packages/docs/docs/canvas-capture/TableOfContents.tsx
+++ b/packages/docs/docs/canvas-capture/TableOfContents.tsx
@@ -10,6 +10,10 @@ export const TableOfContents: React.FC = () => {
<strong>Installation</strong>
<div>Install the browser and Canvas Capture extension</div>
</TOCItem>
+ <TOCItem link="/docs/canvas-capture/agentic-shot-setup">
+ <strong>Agentic shot setup</strong>
+ <div>Let an agent frame and preview your shot</div>
+ </TOCItem>
</Grid>
</div>
);
diff --git a/packages/docs/docs/canvas-capture/agentic-shot-setup.mdx b/packages/docs/docs/canvas-capture/agentic-shot-setup.mdx
new file mode 100644
index 0000000000..1e5702d27b
--- /dev/null
+++ b/packages/docs/docs/canvas-capture/agentic-shot-setup.mdx
@@ -0,0 +1,88 @@
+---
+image: /generated/articles-docs-canvas-capture-agentic-shot-setup.png
+title: Agentic shot setup
+---
+
+# Agentic shot setup
+
+Canvas Capture exposes [WebMCP tools](/docs/ai/webmcp) for framing a shot, checking a still preview, and saving a completed recording.
+You start and stop recording manually and perform the interaction yourself, keeping cursor movement natural.
+
+## Set up a shot
+
+1. [Install Canvas Capture](/docs/canvas-capture/installation) and open the webpage in a browser with an agent that supports WebMCP.
+2. Click the Canvas Capture extension icon to expose the shot tools in that tab.
+3. Ask the agent to prepare the page, scroll to the subject, frame the shot, and inspect a preview.
+4. Click Record, perform the interaction, then click Stop recording.
+5. Download the recording yourself or ask the agent to save it.
+
+For example:
+
+> Prepare a shot of the pricing card with 40px padding and a 1920×1080 output bounding box. Show me a still preview. Leave it ready for me to record opening the dropdown.
+
+The agent sets the browser viewport and prepares the page through its browser controls. Canvas Capture handles the crop and output resolution.
+After reloading or navigating to another page, click the extension icon again to expose the tools.
+
+## Tools
+
+### `setup_shot`
+
+Sets the capture area, output bounding resolution, and format. The crop is resolved once in viewport CSS pixels and stays fixed during recording. It does not follow the selected element.
+
+#### `target`
+
+Use `{type: "viewport"}` for the current viewport, `{type: "element", selector: "#pricing-card", padding: 40}` for an element, or `{type: "rectangle", left: 100, top: 80, width: 800, height: 450}` for an exact rectangle.
+
+An element selector must match exactly one visible element fully inside the viewport. Scroll into position before setting up the shot. Padding defaults to `0` and is clipped at the viewport edges.
+Rectangles must be at least 10×10 CSS pixels and entirely inside the viewport.
+
+#### `resolution?`
+
+An output bounding box in pixels, defaulting to `{width: 2560, height: 1440}`. The crop fits inside this box while preserving its aspect ratio, with scale capped at 30×. Encoded dimensions are rounded down to even numbers.
+For example, a square crop with a 1920×1080 bounding box produces a 1080×1080 recording.
+
+#### `format?`
+
+Use `"mp4"` for H.264 MP4 or `"webm"` for VP9 WebM. When omitted, Canvas Capture prefers MP4 and falls back to WebM if necessary.
+An explicit format never falls back; check encoder readiness before recording.
+
+```json title="Example setup_shot input"
+{
+ "target": {
+ "type": "element",
+ "selector": "#pricing-card",
+ "padding": 40
+ },
+ "resolution": {"width": 1920, "height": 1080},
+ "format": "mp4"
+}
+```
+
+### `get_shot`
+
+Returns the crop, viewport size, output dimensions, format, scale, encoder readiness, and recording state.
+Recording is ready when `supported` is `true`, `encoderSupport` is `"supported"`, and a target is selected.
+
+#### `preview?`
+
+Set to `true` to include a PNG still preview. Defaults to `false`.
+The preview uses the capture renderer and excludes the extension controls. It temporarily wraps the page contents for rendering and restores them afterward, without creating a recording.
+Finish an active recording or selection before requesting a preview.
+
+### `save_recording`
+
+Saves a completed recording. Stop recording manually and wait for finalization before calling it.
+
+#### `destination`
+
+Use `"download"` to save the original file, `"new"` to open it in [Remotion Browser Studio](https://www.remotion.dev/new), or `"convert"` to inspect it in [Remotion Convert](https://www.remotion.dev/convert).
+
+## Browser support
+
+Shot tools require a WebMCP-aware agent and a browser exposing the WebMCP imperative API. See [Chrome's WebMCP documentation](https://developer.chrome.com/docs/ai/agents) for current support.
+Manual capture controls remain available when WebMCP is unavailable.
+
+## See also
+
+- [Install Canvas Capture](/docs/canvas-capture/installation)
+- [WebMCP](/docs/ai/webmcp)
diff --git a/packages/docs/sidebars.ts b/packages/docs/sidebars.ts
index d001b416a7..1f70708ed1 100644
--- a/packages/docs/sidebars.ts
+++ b/packages/docs/sidebars.ts
@@ -2045,7 +2045,10 @@ const sidebars: SidebarsConfig = {
type: 'doc',
id: 'canvas-capture/index',
},
- items: ['canvas-capture/installation'],
+ items: [
+ 'canvas-capture/installation',
+ 'canvas-capture/agentic-shot-setup',
+ ],
},
],
browserBundlerSidebar: [
diff --git a/packages/docs/src/data/articles.ts b/packages/docs/src/data/articles.ts
index 2f3b8ef849..bc6090a5e6 100644
--- a/packages/docs/src/data/articles.ts
+++ b/packages/docs/src/data/articles.ts
@@ -612,6 +612,15 @@ export const articles = [
noAi: false,
slug: 'cancel-render',
},
+ {
+ id: 'canvas-capture/agentic-shot-setup',
+ title: 'Agentic shot setup',
+ relativePath: 'docs/canvas-capture/agentic-shot-setup.mdx',
+ compId: 'articles-docs-canvas-capture-agentic-shot-setup',
+ crumb: null,
+ noAi: false,
+ slug: 'canvas-capture/agentic-shot-setup',
+ },
{
id: 'canvas-capture/index',
title: 'Canvas Capture',
diff --git a/packages/docs/static/generated/articles-docs-canvas-capture-agentic-shot-setup.png b/packages/docs/static/generated/articles-docs-canvas-capture-agentic-shot-setup.png
new file mode 100644
index 0000000000..9d3844d552
Binary files /dev/null and b/packages/docs/static/generated/articles-docs-canvas-capture-agentic-shot-setup.png differ