🎥 Make videos programmatically with React
`@remotion/canvas-capture-extension`: Add `setup_shot` for agentic framing
Jonny Burger2c9de7ac5dce
diff --git a/.agents/skills/canvas-capture-extension/SKILL.md b/.agents/skills/canvas-capture-extension/SKILL.mdindex 15d11faf6b..cca1fc48b1 100644--- a/.agents/skills/canvas-capture-extension/SKILL.md+++ b/.agents/skills/canvas-capture-extension/SKILL.md@@ -53,7 +53,7 @@ To select another Chrome 157.0.8080.0 or newer browser, set## Reload in Chrome- Launch Chrome 157.0.8080.0 or newer with the- dedicated Canvas Capture profile, then open `chrome://extensions` manually.+ dedicated Canvas Capture profile, then open `chrome://extensions`.- Enable **Developer mode** on `chrome://extensions`.- If **Remotion Canvas Capture** already points to the durable directory, click**Reload**.@@ -74,5 +74,16 @@ To select another Chrome 157.0.8080.0 or newer browser, setexperimental HTML-in-canvas API is unavailable.Do not edit Chrome's `Preferences` or `Secure Preferences` files to move or-reload the extension, and do not automate the Chrome UI. Let the user use-Chrome's extensions page so its integrity metadata stays valid.+reload the extension. Use Chrome's extensions page so its integrity metadata+stays valid. Browser UI automation is allowed for loading, reloading, and+activating Canvas Capture; do not require the user to perform those clicks.++## Prepare a shot++Open the requested webpage in the compatible recording browser and activate+Canvas Capture through its toolbar icon or the Extensions menu. This exposes+the `setup_shot`, `get_shot`, and `save_recording` WebMCP tools in that tab.++Prepare the page and viewport, call `setup_shot`, and inspect a still preview+with `get_shot`. Leave the browser open with the shot ready for the user to+record. Start and stop recording manually so the user controls cursor movement.diff --git a/packages/canvas-capture-extension/README.md b/packages/canvas-capture-extension/README.mdindex fe4af11c93..3ae4555c20 100644--- a/packages/canvas-capture-extension/README.md+++ b/packages/canvas-capture-extension/README.md@@ -66,6 +66,12 @@ extension, so its path and extension ID remain stable across worktrees.## Build and install+The extension exposes `setup_shot`, `get_shot`, and `save_recording` to WebMCP-aware+agents after its icon is clicked. Agents can frame the viewport, an element, or an+exact rectangle, inspect a PNG still preview, and save a completed recording.+Recording starts and stops through the manual controls. See+[Agentic shot setup](https://www.remotion.dev/docs/canvas-capture/agentic-shot-setup).+1. From the repository root, run `.agents/skills/canvas-capture-extension/scripts/rebuild-extension.sh --repo "$PWD"`. This creates the production WXT bundle and installs the complete unpacked extension outside the worktree. Building does not require an installed browser.2. In Chrome 157.0.8080.0 or newer, open `chrome://extensions`, enable **Developer mode**, and choose **Load unpacked**.3. Select `/Users/jonathanburger/Applications/Remotion Canvas Capture Extension`.diff --git a/packages/canvas-capture-extension/src/background.ts b/packages/canvas-capture-extension/src/background.tsindex 9a39769edd..6dc844345d 100644--- a/packages/canvas-capture-extension/src/background.ts+++ b/packages/canvas-capture-extension/src/background.ts@@ -30,6 +30,12 @@ export const startBackground = () => {await browser.tabs.sendMessage(tab.id, request);}+ await browser.scripting.executeScript({+ target: {tabId: tab.id},+ files: ['/shot-tools.js'],+ world: 'MAIN',+ });+await browser.action.setBadgeText({tabId: tab.id, text: ''});await browser.action.setTitle({tabId: tab.id,diff --git a/packages/canvas-capture-extension/src/capture.ts b/packages/canvas-capture-extension/src/capture.tsindex 36f768f386..4f6081bddd 100644--- a/packages/canvas-capture-extension/src/capture.ts+++ b/packages/canvas-capture-extension/src/capture.ts@@ -342,6 +342,30 @@ export class PageCapture {this.#requestPaint();};+ preview = async () => {+ // Wait for the drawable subtree to paint before reading its pixels.+ await new Promise<void>((resolve, reject) => {+ let timeout = 0;+ const onPaint = () => {+ clearTimeout(timeout);+ resolve();+ };++ timeout = window.setTimeout(() => {+ this.#wrapped.canvas.removeEventListener('paint', onPaint);+ reject(new Error('The shot preview did not paint in time.'));+ }, 5000);+ this.#wrapped.canvas.addEventListener('paint', onPaint, {once: true});+ this.#requestPaint();+ });+ if (this.#paintError) {+ throw this.#paintError;+ }++ this.#draw();+ return this.#captureCanvas.convertToBlob({type: 'image/png'});+ };+stop = async () => {try {if (this.#paintError) {diff --git a/packages/canvas-capture-extension/src/content.ts b/packages/canvas-capture-extension/src/content.tsindex b15a27d185..68e0b1a20f 100644--- a/packages/canvas-capture-extension/src/content.ts+++ b/packages/canvas-capture-extension/src/content.ts@@ -22,6 +22,12 @@ import {type SelectionHandle,type SelectionRectangle,} from './selection';+import {+ shotRequestEvent,+ shotResponseEvent,+ shotTools,+ type ShotToolName,+} from './shot-tools';type ExtensionController = {readonly handleRequest: (@@ -648,6 +654,7 @@ export const startContent = () => {readonly offsetY: number;} | null = null;let encoderSupportCheckId = 0;+ let requestedFormat: CaptureFormat | null = null;let encoderSupportKey: string | null = null;let recordingStartedAt: number | null = null;let recordingDurationInterval: number | null = null;@@ -779,6 +786,25 @@ export const startContent = () => {return;}+ if (requestedFormat !== null) {+ format = requestedFormat;+ encoderSupport = 'checking';+ const canEncode = await canEncodeCapture(format, outputSize);+ if (checkId !== encoderSupportCheckId) {+ return;+ }++ encoderSupport = canEncode ? 'supported' : 'unsupported';+ encoderSupportKey = supportKey;+ setStatus(+ canEncode+ ? `Ready to record ${getFormatLabel(format)} at ${outputSize.width}×${outputSize.height}.`+ : `${getFormatLabel(format)} is unavailable at ${outputSize.width}×${outputSize.height}.`,+ !canEncode,+ );+ return;+ }+encoderSupport = 'checking';setStatus(`Checking H.264 MP4 support at ${outputSize.width}×${outputSize.height}…`,@@ -927,6 +953,8 @@ export const startContent = () => {};const getState = (): CaptureControllerState => ({+ crop: resolveCaptureTarget()?.crop ?? null,+ viewport: {width: window.innerWidth, height: window.innerHeight},supported,selecting,hasTarget: selectedTarget !== null,@@ -1032,6 +1060,7 @@ export const startContent = () => {startingRecording ||completedRecording ||finalizing ||+ controlsBusy ||selectionDrag) {return;@@ -1477,7 +1506,7 @@ export const startContent = () => {};controlsClose.addEventListener('click', () => {- if (capture || startingRecording || finalizing) {+ if (capture || startingRecording || finalizing || controlsBusy) {return;}@@ -1660,6 +1689,331 @@ export const startContent = () => {updateControls();});});+ const handleShotRequest = async (+ name: ShotToolName,+ input: Record<string, unknown>,+ ) => {+ if (name === 'get_shot') {+ if (input.preview !== undefined && typeof input.preview !== 'boolean') {+ throw new Error('preview must be a boolean.');+ }++ await refreshEncoderSupport();+ updateControls();+ const state = getState();+ if (!input.preview) {+ return state;+ }++ if (+ capture ||+ startingRecording ||+ finalizing ||+ controlsBusy ||+ selecting ||+ selectionDrag+ ) {+ throw new Error(+ 'Finish the current recording or selection before previewing the shot.',+ );+ }++ const previewTarget = resolveCaptureTarget();+ if (!previewTarget) {+ throw new Error('Set up a shot before requesting a preview.');+ }++ controlsBusy = true;+ updateControls();+ let previewCapture: PageCapture | null = null;+ try {+ previewCapture = new PageCapture({+ crop: previewTarget.crop,+ scale,+ format,+ });+ const blob = await previewCapture.preview();+ const dataUrl = await new Promise<string>((resolve, reject) => {+ const reader = new FileReader();+ reader.onload = () => resolve(String(reader.result));+ reader.onerror = () =>+ reject(new Error('Could not read the shot preview.'));+ reader.readAsDataURL(blob);+ });+ return {+ content: [+ {type: 'text', text: JSON.stringify(state)},+ {+ type: 'image',+ mimeType: 'image/png',+ data: dataUrl.slice(dataUrl.indexOf(',') + 1),+ },+ ],+ };+ } finally {+ previewCapture?.restore();+ controlsBusy = false;+ updateControls();+ }+ }++ if (capture || startingRecording || finalizing || controlsBusy) {+ throw new Error('Wait until recording and finalization have finished.');+ }++ if (name === 'save_recording') {+ const {destination} = input;+ if (+ destination !== 'download' &&+ destination !== 'new' &&+ destination !== 'convert'+ ) {+ throw new Error('destination must be download, new, or convert.');+ }++ if (!completedRecording) {+ throw new Error(+ 'No completed recording is available. Stop recording manually before saving.',+ );+ }++ await handleCompletedRecording(destination);+ const state = getState();+ if (state.error) {+ throw new Error(state.status);+ }++ return state;+ }++ if (completedRecording) {+ throw new Error(+ 'Save or discard the completed recording before setting up another shot.',+ );+ }++ const {target, resolution, format: nextFormat} = input;+ if (+ typeof target !== 'object' ||+ target === null ||+ !('type' in target)+ ) {+ throw new Error(+ 'target must describe a viewport, element, or rectangle.',+ );+ }++ let crop: CaptureCrop;+ if (target.type === 'viewport') {+ crop = {+ left: 0,+ top: 0,+ width: window.innerWidth,+ height: window.innerHeight,+ };+ } else if (target.type === 'element') {+ if (+ !('selector' in target) ||+ typeof target.selector !== 'string' ||+ !target.selector.trim()+ ) {+ throw new Error('An element target requires a CSS selector.');+ }++ const padding = 'padding' in target ? target.padding : 0;+ if (+ typeof padding !== 'number' ||+ !Number.isFinite(padding) ||+ padding < 0+ ) {+ throw new Error('padding must be a finite non-negative number.');+ }++ const elements = document.querySelectorAll(target.selector);+ if (elements.length !== 1) {+ throw new Error(+ `The selector matched ${elements.length} elements. Choose a selector matching exactly one element.`,+ );+ }++ const element = elements[0];+ if (!element) {+ throw new Error('The selected element is unavailable.');+ }++ const rect = element.getBoundingClientRect();+ const elementStyle = window.getComputedStyle(element);+ if (+ rect.width <= 0 ||+ rect.height <= 0 ||+ elementStyle.visibility === 'hidden' ||+ elementStyle.visibility === 'collapse' ||+ Number(elementStyle.opacity) === 0 ||+ rect.left < 0 ||+ rect.top < 0 ||+ rect.right > window.innerWidth ||+ rect.bottom > window.innerHeight+ ) {+ throw new Error(+ 'The element must be visible and fully inside the viewport. Scroll or prepare the page before setting up the shot.',+ );+ }++ const left = Math.max(0, rect.left - padding);+ const top = Math.max(0, rect.top - padding);+ crop = {+ left,+ top,+ width: Math.min(window.innerWidth, rect.right + padding) - left,+ height: Math.min(window.innerHeight, rect.bottom + padding) - top,+ };+ } else if (target.type === 'rectangle') {+ if (+ !('left' in target) ||+ !('top' in target) ||+ !('width' in target) ||+ !('height' in target) ||+ typeof target.left !== 'number' ||+ typeof target.top !== 'number' ||+ typeof target.width !== 'number' ||+ typeof target.height !== 'number'+ ) {+ throw new Error(+ 'A rectangle requires numeric left, top, width, and height.',+ );+ }++ crop = {+ left: target.left,+ top: target.top,+ width: target.width,+ height: target.height,+ };+ } else {+ throw new Error('target.type must be viewport, element, or rectangle.');+ }++ if (+ !Object.values(crop).every(Number.isFinite) ||+ crop.left < 0 ||+ crop.top < 0 ||+ crop.width < minimumCaptureArea.width ||+ crop.height < minimumCaptureArea.height ||+ crop.left + crop.width > window.innerWidth ||+ crop.top + crop.height > window.innerHeight+ ) {+ throw new Error(+ 'The crop must be at least 10×10 CSS pixels and entirely inside the viewport.',+ );+ }++ let outputBounds = defaultResolution;+ if (resolution !== undefined) {+ if (+ typeof resolution !== 'object' ||+ resolution === null ||+ !('width' in resolution) ||+ !('height' in resolution) ||+ typeof resolution.width !== 'number' ||+ typeof resolution.height !== 'number' ||+ !Number.isInteger(resolution.width) ||+ !Number.isInteger(resolution.height) ||+ resolution.width < 2 ||+ resolution.height < 2+ ) {+ throw new Error(+ 'resolution requires integer width and height of at least 2 pixels.',+ );+ }++ outputBounds = {width: resolution.width, height: resolution.height};+ }++ if (+ nextFormat !== undefined &&+ nextFormat !== 'mp4' &&+ nextFormat !== 'webm'+ ) {+ throw new Error('format must be mp4 or webm.');+ }++ const nextScale = Math.min(+ outputBounds.width / crop.width,+ outputBounds.height / crop.height,+ maxZoom,+ );+ getCapturePreflight({crop, scale: nextScale});+ if (selectionDrag) {+ highlight.releasePointerCapture(selectionDrag.pointerId);+ selectionDrag = null;+ }++ if (selecting) {+ cancelSelection();+ }++ selectedTarget = {type: 'page-crop', crop};+ requestedFormat = nextFormat ?? null;+ useDefaultResolution = false;+ scale = nextScale;+ encoderSupportKey = null;+ controlsDismissed = false;+ controlsBusy = true;+ updateHighlight();+ updateControls();+ try {+ await refreshEncoderSupport();+ return getState();+ } finally {+ controlsBusy = false;+ updateControls();+ }+ };++ // Only shot preparation, inspection, and saving cross into the page world.+ // Recording commands stay in the extension's manual controls.+ document.addEventListener(shotRequestEvent, (event) => {+ if (!(event instanceof CustomEvent) || typeof event.detail !== 'string') {+ return;+ }++ let request;+ try {+ request = JSON.parse(event.detail);+ } catch {+ return;+ }++ if (+ typeof request?.id !== 'string' ||+ !shotTools.some((tool) => tool.name === request.name) ||+ typeof request.input !== 'object' ||+ request.input === null ||+ Array.isArray(request.input)+ ) {+ return;+ }++ handleShotRequest(request.name, request.input)+ .then((result) =>+ document.dispatchEvent(+ new CustomEvent(shotResponseEvent, {+ detail: JSON.stringify({id: request.id, result}),+ }),+ ),+ )+ .catch((error) =>+ document.dispatchEvent(+ new CustomEvent(shotResponseEvent, {+ detail: JSON.stringify({+ id: request.id,+ error: error instanceof Error ? error.message : String(error),+ }),+ }),+ ),+ );+ });+return {handleRequest,};diff --git a/packages/canvas-capture-extension/src/entrypoints/shot-tools.ts b/packages/canvas-capture-extension/src/entrypoints/shot-tools.tsnew file mode 100644index 0000000000..e9a5156be4--- /dev/null+++ b/packages/canvas-capture-extension/src/entrypoints/shot-tools.ts@@ -0,0 +1,95 @@+import {defineUnlistedScript} from 'wxt/utils/define-unlisted-script';+import {shotRequestEvent, shotResponseEvent, shotTools} from '../shot-tools';++type ModelContext = {+ readonly registerTool: (+ tool: (typeof shotTools)[number] & {+ readonly execute: (input: Record<string, unknown>) => Promise<unknown>;+ },+ options: {readonly signal: AbortSignal},+ ) => Promise<void>;+};++export default defineUnlistedScript(() => {+ const {modelContext} = document as Document & {+ readonly modelContext: ModelContext | undefined;+ };+ if (typeof modelContext?.registerTool !== 'function') {+ return;+ }++ const toolsWindow = window as unknown as Window & {+ __remotionCanvasCaptureTools: AbortController | undefined;+ };+ if (toolsWindow.__remotionCanvasCaptureTools) {+ return;+ }++ const controller = new AbortController();+ toolsWindow.__remotionCanvasCaptureTools = controller;+ Promise.all(+ shotTools.map((tool) =>+ modelContext.registerTool(+ {+ ...tool,+ execute: (input) =>+ new Promise((resolve, reject) => {+ const id = crypto.randomUUID();+ let timeout = 0;+ const onResponse = (event: Event) => {+ if (+ !(event instanceof CustomEvent) ||+ typeof event.detail !== 'string'+ ) {+ return;+ }++ let response;+ try {+ response = JSON.parse(event.detail);+ } catch {+ return;+ }++ if (response?.id !== id) {+ return;+ }++ clearTimeout(timeout);+ document.removeEventListener(shotResponseEvent, onResponse);+ if (typeof response.error === 'string') {+ reject(new Error(response.error));+ } else {+ resolve(response.result);+ }+ };++ timeout = window.setTimeout(() => {+ document.removeEventListener(shotResponseEvent, onResponse);+ reject(+ new Error(+ 'Canvas Capture did not respond. Reopen the extension controls and retry.',+ ),+ );+ }, 30_000);+ document.addEventListener(shotResponseEvent, onResponse);+ document.dispatchEvent(+ new CustomEvent(shotRequestEvent, {+ detail: JSON.stringify({id, name: tool.name, input}),+ }),+ );+ }),+ },+ {signal: controller.signal},+ ),+ ),+ ).catch((error) => {+ controller.abort();+ toolsWindow.__remotionCanvasCaptureTools = undefined;+ // eslint-disable-next-line no-console -- Surface tool registration failures in DevTools.+ console.error(+ '[Remotion Canvas Capture] Could not register shot tools',+ error,+ );+ });+});diff --git a/packages/canvas-capture-extension/src/messages.ts b/packages/canvas-capture-extension/src/messages.tsindex 35f91c7eb5..373110faa4 100644--- a/packages/canvas-capture-extension/src/messages.ts+++ b/packages/canvas-capture-extension/src/messages.ts@@ -4,6 +4,13 @@ export const captureControllerMessageType =export type CaptureFormat = 'mp4' | 'webm';export type CaptureControllerState = {+ readonly crop: {+ readonly left: number;+ readonly top: number;+ readonly width: number;+ readonly height: number;+ } | null;+ readonly viewport: {readonly width: number; readonly height: number};readonly supported: boolean;readonly selecting: boolean;readonly hasTarget: boolean;diff --git a/packages/canvas-capture-extension/src/shot-tools.ts b/packages/canvas-capture-extension/src/shot-tools.tsnew file mode 100644index 0000000000..1843c52cd8--- /dev/null+++ b/packages/canvas-capture-extension/src/shot-tools.ts@@ -0,0 +1,117 @@+export const shotRequestEvent = 'remotion-canvas-capture-shot-request';+export const shotResponseEvent = 'remotion-canvas-capture-shot-response';++export type ShotToolName = 'setup_shot' | 'get_shot' | 'save_recording';++export const shotTools: {+ readonly name: ShotToolName;+ readonly title: string;+ readonly description: string;+ readonly inputSchema: Record<string, unknown>;+ readonly annotations: {readonly readOnlyHint: boolean};+}[] = [+ {+ name: 'setup_shot',+ title: 'Set up Canvas Capture shot',+ description:+ 'Frame a Canvas Capture shot without recording or moving the cursor. Prepare the page and scroll first. Element bounds are resolved once; the crop stays fixed in viewport CSS pixels. The user starts and stops recording manually.',+ inputSchema: {+ type: 'object',+ properties: {+ target: {+ oneOf: [+ {+ type: 'object',+ properties: {type: {const: 'viewport'}},+ required: ['type'],+ additionalProperties: false,+ },+ {+ type: 'object',+ properties: {+ type: {const: 'element'},+ selector: {+ type: 'string',+ description:+ 'A CSS selector matching exactly one visible element.',+ },+ padding: {+ type: 'number',+ minimum: 0,+ description:+ 'Padding in CSS pixels. Defaults to 0. Clipped to the viewport.',+ },+ },+ required: ['type', 'selector'],+ additionalProperties: false,+ },+ {+ type: 'object',+ properties: {+ type: {const: 'rectangle'},+ left: {type: 'number', minimum: 0},+ top: {type: 'number', minimum: 0},+ width: {type: 'number', minimum: 10},+ height: {type: 'number', minimum: 10},+ },+ required: ['type', 'left', 'top', 'width', 'height'],+ additionalProperties: false,+ },+ ],+ },+ resolution: {+ type: 'object',+ description:+ 'Output bounding box in pixels. Fits the crop inside it, preserving aspect ratio. Defaults to 2560×1440.',+ properties: {+ width: {type: 'integer', minimum: 2},+ height: {type: 'integer', minimum: 2},+ },+ required: ['width', 'height'],+ additionalProperties: false,+ },+ format: {+ type: 'string',+ enum: ['mp4', 'webm'],+ description:+ 'Omit to prefer MP4 with WebM fallback. An explicit format never falls back.',+ },+ },+ required: ['target'],+ additionalProperties: false,+ },+ annotations: {readOnlyHint: false},+ },+ {+ name: 'get_shot',+ title: 'Inspect Canvas Capture shot',+ description:+ 'Read the crop, viewport, output size and recording readiness. Optionally return a PNG still preview without recording. Preview is unavailable during recording or finalization.',+ inputSchema: {+ type: 'object',+ properties: {+ preview: {+ type: 'boolean',+ description: 'Include a PNG still preview. Defaults to false.',+ },+ },+ additionalProperties: false,+ },+ annotations: {readOnlyHint: true},+ },+ {+ name: 'save_recording',+ title: 'Save completed Canvas Capture recording',+ description:+ 'Download a completed recording or open it in Remotion Browser Studio or Convert. The user must have stopped recording manually first. This tool cannot start or stop recording.',+ inputSchema: {+ type: 'object',+ properties: {+ destination: {type: 'string', enum: ['download', 'new', 'convert']},+ },+ required: ['destination'],+ additionalProperties: false,+ },+ annotations: {readOnlyHint: false},+ },+];diff --git a/packages/docs/docs/canvas-capture/TableOfContents.tsx b/packages/docs/docs/canvas-capture/TableOfContents.tsxindex e4de48813c..4f436024a6 100644--- a/packages/docs/docs/canvas-capture/TableOfContents.tsx+++ b/packages/docs/docs/canvas-capture/TableOfContents.tsx@@ -10,6 +10,10 @@ export const TableOfContents: React.FC = () => {<strong>Installation</strong><div>Install the browser and Canvas Capture extension</div></TOCItem>+ <TOCItem link="/docs/canvas-capture/agentic-shot-setup">+ <strong>Agentic shot setup</strong>+ <div>Let an agent frame and preview your shot</div>+ </TOCItem></Grid></div>);diff --git a/packages/docs/docs/canvas-capture/agentic-shot-setup.mdx b/packages/docs/docs/canvas-capture/agentic-shot-setup.mdxnew file mode 100644index 0000000000..1e5702d27b--- /dev/null+++ b/packages/docs/docs/canvas-capture/agentic-shot-setup.mdx@@ -0,0 +1,88 @@+---+image: /generated/articles-docs-canvas-capture-agentic-shot-setup.png+title: Agentic shot setup+---++# Agentic shot setup++Canvas Capture exposes [WebMCP tools](/docs/ai/webmcp) for framing a shot, checking a still preview, and saving a completed recording.+You start and stop recording manually and perform the interaction yourself, keeping cursor movement natural.++## Set up a shot++1. [Install Canvas Capture](/docs/canvas-capture/installation) and open the webpage in a browser with an agent that supports WebMCP.+2. Click the Canvas Capture extension icon to expose the shot tools in that tab.+3. Ask the agent to prepare the page, scroll to the subject, frame the shot, and inspect a preview.+4. Click Record, perform the interaction, then click Stop recording.+5. Download the recording yourself or ask the agent to save it.++For example:++> Prepare a shot of the pricing card with 40px padding and a 1920×1080 output bounding box. Show me a still preview. Leave it ready for me to record opening the dropdown.++The agent sets the browser viewport and prepares the page through its browser controls. Canvas Capture handles the crop and output resolution.+After reloading or navigating to another page, click the extension icon again to expose the tools.++## Tools++### `setup_shot`++Sets the capture area, output bounding resolution, and format. The crop is resolved once in viewport CSS pixels and stays fixed during recording. It does not follow the selected element.++#### `target`++Use `{type: "viewport"}` for the current viewport, `{type: "element", selector: "#pricing-card", padding: 40}` for an element, or `{type: "rectangle", left: 100, top: 80, width: 800, height: 450}` for an exact rectangle.++An element selector must match exactly one visible element fully inside the viewport. Scroll into position before setting up the shot. Padding defaults to `0` and is clipped at the viewport edges.+Rectangles must be at least 10×10 CSS pixels and entirely inside the viewport.++#### `resolution?`++An output bounding box in pixels, defaulting to `{width: 2560, height: 1440}`. The crop fits inside this box while preserving its aspect ratio, with scale capped at 30×. Encoded dimensions are rounded down to even numbers.+For example, a square crop with a 1920×1080 bounding box produces a 1080×1080 recording.++#### `format?`++Use `"mp4"` for H.264 MP4 or `"webm"` for VP9 WebM. When omitted, Canvas Capture prefers MP4 and falls back to WebM if necessary.+An explicit format never falls back; check encoder readiness before recording.++```json title="Example setup_shot input"+{+ "target": {+ "type": "element",+ "selector": "#pricing-card",+ "padding": 40+ },+ "resolution": {"width": 1920, "height": 1080},+ "format": "mp4"+}+```++### `get_shot`++Returns the crop, viewport size, output dimensions, format, scale, encoder readiness, and recording state.+Recording is ready when `supported` is `true`, `encoderSupport` is `"supported"`, and a target is selected.++#### `preview?`++Set to `true` to include a PNG still preview. Defaults to `false`.+The preview uses the capture renderer and excludes the extension controls. It temporarily wraps the page contents for rendering and restores them afterward, without creating a recording.+Finish an active recording or selection before requesting a preview.++### `save_recording`++Saves a completed recording. Stop recording manually and wait for finalization before calling it.++#### `destination`++Use `"download"` to save the original file, `"new"` to open it in [Remotion Browser Studio](https://www.remotion.dev/new), or `"convert"` to inspect it in [Remotion Convert](https://www.remotion.dev/convert).++## Browser support++Shot tools require a WebMCP-aware agent and a browser exposing the WebMCP imperative API. See [Chrome's WebMCP documentation](https://developer.chrome.com/docs/ai/agents) for current support.+Manual capture controls remain available when WebMCP is unavailable.++## See also++- [Install Canvas Capture](/docs/canvas-capture/installation)+- [WebMCP](/docs/ai/webmcp)diff --git a/packages/docs/sidebars.ts b/packages/docs/sidebars.tsindex d001b416a7..1f70708ed1 100644--- a/packages/docs/sidebars.ts+++ b/packages/docs/sidebars.ts@@ -2045,7 +2045,10 @@ const sidebars: SidebarsConfig = {type: 'doc',id: 'canvas-capture/index',},- items: ['canvas-capture/installation'],+ items: [+ 'canvas-capture/installation',+ 'canvas-capture/agentic-shot-setup',+ ],},],browserBundlerSidebar: [diff --git a/packages/docs/src/data/articles.ts b/packages/docs/src/data/articles.tsindex 2f3b8ef849..bc6090a5e6 100644--- a/packages/docs/src/data/articles.ts+++ b/packages/docs/src/data/articles.ts@@ -612,6 +612,15 @@ export const articles = [noAi: false,slug: 'cancel-render',},+ {+ id: 'canvas-capture/agentic-shot-setup',+ title: 'Agentic shot setup',+ relativePath: 'docs/canvas-capture/agentic-shot-setup.mdx',+ compId: 'articles-docs-canvas-capture-agentic-shot-setup',+ crumb: null,+ noAi: false,+ slug: 'canvas-capture/agentic-shot-setup',+ },{id: 'canvas-capture/index',title: 'Canvas Capture',diff --git a/packages/docs/static/generated/articles-docs-canvas-capture-agentic-shot-setup.png b/packages/docs/static/generated/articles-docs-canvas-capture-agentic-shot-setup.pngnew file mode 100644index 0000000000..9d3844d552Binary files /dev/null and b/packages/docs/static/generated/articles-docs-canvas-capture-agentic-shot-setup.png differ