Skip to content

Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.

docs(mcp): say clicks share a zoom only when they are close on screen too

EtienneLescot201a913cad63
diff --git a/electron/mcp/media-tools.ts b/electron/mcp/media-tools.ts
index 49388f593..4c7fc2bb0 100644
--- a/electron/mcp/media-tools.ts
+++ b/electron/mcp/media-tools.ts
@@ -146,7 +146,7 @@ const READ_ONLY = { readOnlyHint: true, destructiveHint: false, openWorldHint: f
export const MEDIA_PREAMBLE = [
"OpenScreen records the screen, turns each take into a project you edit with the document tools, and exports it as a video.",
"A demo goes: listSources, then startRecording, then drive the app, then stopRecording. It returns the projectId and a checkpointId. Edit with the document tools, always passing that projectId: every edit is saved to the project straight away, and restoreCheckpoint with the checkpointId takes the project back to the raw take in one step. Then exportVideo, and getFrames with the projectId to check the export before you hand it over: it takes the same ruler seconds as the document tools, so a trim or a speed region never shifts what you look at.",
- "Drive the app with the real mouse when you can: OpenScreen records the pointer, draws a clean cursor and zooms on the clicks as the editor does: clicks close together share one zoom, and the first seconds and the end stay on the whole screen. If you drive it from a script instead (Playwright, CDP), the screen pointer never moves: log each action as the script runs it, then hand the log to setPointerTrack after stopRecording.",
+ "Drive the app with the real mouse when you can: OpenScreen records the pointer, draws a clean cursor and zooms on the clicks as the editor does: clicks close in time and on screen share one zoom, and the first seconds and the end stay on the whole screen. If you drive it from a script instead (Playwright, CDP), the screen pointer never moves: log each action as the script runs it, then hand the log to setPointerTrack after stopRecording.",
"These projects are in the OpenScreen app's project list, so the user can open one in the editor. Do not edit a project while the user has it open there: the editor would keep its own copy.",
"",
].join("\n");
@@ -253,7 +253,7 @@ export function registerMediaTools(server: McpServer, deps: MediaToolDeps): void
"startRecording",
{
description:
- "Start recording one window (preferred: nothing else on the desktop is captured) or one display. Returns once capture has started; then drive the app and call stopRecording. The pointer is recorded as data, so the export draws a clean cursor and zooms on the clicks. On Linux the desktop's own picker asks the user which window to record.",
+ "Start recording one window (preferred: nothing else on the desktop is captured) or one display. Returns once capture has started; then drive the app and call stopRecording. The pointer is recorded as data, so the export draws a clean cursor and the zooms planned from the clicks. On Linux the desktop's own picker asks the user which window to record.",
inputSchema: z.object({
window: z
.string()
@@ -318,7 +318,7 @@ export function registerMediaTools(server: McpServer, deps: MediaToolDeps): void
"stopRecording",
{
description:
- "Stop the recording and turn the take into a new project, laid out as the editor lays out a fresh recording: one clip, plus the zooms planned from the recorded clicks. Not one per click: clicks close together share one zoom, a click in the first seconds or the one that stopped the take gets none, and autoZooms says how many there are. Returns the projectId to edit, a checkpointId that restores the raw take, and the take's videoPath.",
+ "Stop the recording and turn the take into a new project, laid out as the editor lays out a fresh recording: one clip, plus the zooms planned from the recorded clicks. Not one per click: clicks close in time and on screen share one zoom, a click in the first seconds or the one that stopped the take gets none, and autoZooms says how many there are. Returns the projectId to edit, a checkpointId that restores the raw take, and the take's videoPath.",
inputSchema: z.object({
title: z.string().optional().describe("The project's title in OpenScreen."),
}),
diff --git a/skills/openscreen-demo/SKILL.md b/skills/openscreen-demo/SKILL.md
index b02b99f78..6c4b59f07 100644
--- a/skills/openscreen-demo/SKILL.md
+++ b/skills/openscreen-demo/SKILL.md
@@ -80,7 +80,7 @@ Use the document tools, always with the `projectId`. Read the project first with
- **Format first:** `setAspectRatio` for where the video will be shown, `16:9` for X. Do it before placing text.
- **Cut dead time:** the setup at the start, the stop at the end, any wait. Use `addTrims`, or `addSpeed` to fast-forward a wait instead of cutting it.
-- **Zoom on the action:** clicks close together share one zoom, and the first 2.5 s stay wide. Check them, then add or move zooms with `addZooms` and `setZoom` where a result deserves its own. Hold a zoom at least 1.5 s.
+- **Zoom on the action:** clicks close in time and on screen share one zoom, and the first 2.5 s stay wide. Check them, then add or move zooms with `addZooms` and `setZoom` where a result deserves its own. Hold a zoom at least 1.5 s.
- **Callouts:** one `addAnnotation` per beat, short, placed clear of the action. Text sits on the output frame and stays put when the picture zooms. Use its look for hierarchy: a large title with `animation: "typewriter"`, step callouts with `animation: "pop"`. `setAnnotation` moves or restyles one.
- **Undo:** `restoreCheckpoint` with the `checkpointId` takes the project back to the raw take in one step.
diff --git a/website/docs/ai-editing.md b/website/docs/ai-editing.md
index bc39321c9..9a02d3947 100644
--- a/website/docs/ai-editing.md
+++ b/website/docs/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
The executable on each system is listed in [Running the CLI](./cli.md#running-the-cli).
- **No port, no token, no setting.** The agent starts the process itself, with your rights, as it would run `openscreen record`. So it can record your screen and edit without the **Project edits** switch.
-- **Each take becomes a new project** in your project list, zoomed on its clicks as a recording made in the app is: clicks close together share one zoom, and the first seconds and the end stay on the whole screen. Do not edit it in the editor while the agent is still working on it: the editor keeps its own copy.
+- **Each take becomes a new project** in your project list, zoomed on its clicks as a recording made in the app is: clicks close in time and on screen share one zoom, and the first seconds and the end stay on the whole screen. Do not edit it in the editor while the agent is still working on it: the editor keeps its own copy.
- **A take driven by a script** (Playwright, for example) never moves the screen pointer. The agent hands OpenScreen its own log of the actions with `setPointerTrack`, and the video gets a cursor and its zooms all the same.
- **The `openscreen-demo` skill** teaches an agent the whole job, from the storyboard to the finished video. It then attaches the video to a pull request, so reviewers watch the change in place, or drafts a post on X that it publishes only once you say yes. Add it with `npx skills add getopenscreen/openscreen --skill openscreen-demo`. The recipe is [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md).
diff --git a/website/i18n/de/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/de/docusaurus-plugin-content-docs/current/ai-editing.md
index 348bca6da..1b00bbf75 100644
--- a/website/i18n/de/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/de/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
Die ausführbare Datei für jedes System steht unter [Die CLI ausführen](./cli.md#running-the-cli).
- **Kein Port, kein Token, keine Einstellung.** Der Agent startet den Prozess selbst, mit deinen Rechten, so wie er `openscreen record` ausführen würde. Deshalb kann er deinen Bildschirm aufnehmen und bearbeiten, ohne den Schalter **Project edits**.
-- **Jeder Take wird zu einem neuen Projekt** in deiner Projektliste, mit Zooms auf seine Klicks, wie eine in der App gemachte Aufnahme: Nah beieinanderliegende Klicks teilen sich einen Zoom, und die ersten Sekunden und das Ende bleiben auf dem ganzen Bildschirm. Bearbeite es nicht im Editor, solange der Agent noch daran arbeitet: Der Editor hält seine eigene Kopie.
+- **Jeder Take wird zu einem neuen Projekt** in deiner Projektliste, mit Zooms auf seine Klicks, wie eine in der App gemachte Aufnahme: Zeitlich und räumlich nah beieinanderliegende Klicks teilen sich einen Zoom, und die ersten Sekunden und das Ende bleiben auf dem ganzen Bildschirm. Bearbeite es nicht im Editor, solange der Agent noch daran arbeitet: Der Editor hält seine eigene Kopie.
- **Ein per Skript gesteuerter Take** (zum Beispiel mit Playwright) bewegt nie den Zeiger auf dem Bildschirm. Der Agent übergibt OpenScreen mit `setPointerTrack` sein eigenes Protokoll der Aktionen, und das Video bekommt trotzdem einen Cursor und dessen Zooms.
- **Der Skill `openscreen-demo`** bringt einem Agenten die ganze Arbeit bei, vom Storyboard bis zum fertigen Video. Danach hängt er das Video an einen Pull Request, damit die Reviewer die Änderung direkt dort sehen, oder entwirft einen Post auf X, den er erst veröffentlicht, wenn du Ja sagst. Füge ihn mit `npx skills add getopenscreen/openscreen --skill openscreen-demo` hinzu. Das Rezept ist [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md).
diff --git a/website/i18n/es/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/es/docusaurus-plugin-content-docs/current/ai-editing.md
index 4899617d6..c43ae2eef 100644
--- a/website/i18n/es/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/es/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
El ejecutable de cada sistema aparece en [Ejecutar la CLI](./cli.md#running-the-cli).
- **Sin puerto, sin token, sin ajuste.** El agente inicia el proceso él mismo, con tus permisos, como ejecutaría `openscreen record`. Así puede grabar tu pantalla y editar sin el interruptor **Ediciones del proyecto**.
-- **Cada toma se convierte en un proyecto nuevo** en tu lista de proyectos, con zooms en sus clics, como una grabación hecha en la app: los clics cercanos comparten un zoom, y los primeros segundos y el final se quedan en la pantalla completa. No lo edites en el editor mientras el agente siga trabajando en él: el editor conserva su propia copia.
+- **Cada toma se convierte en un proyecto nuevo** en tu lista de proyectos, con zooms en sus clics, como una grabación hecha en la app: los clics cercanos en el tiempo y en la pantalla comparten un zoom, y los primeros segundos y el final se quedan en la pantalla completa. No lo edites en el editor mientras el agente siga trabajando en él: el editor conserva su propia copia.
- **Una toma controlada por un script** (Playwright, por ejemplo) nunca mueve el puntero de la pantalla. El agente le pasa a OpenScreen su propio registro de las acciones con `setPointerTrack`, y el video obtiene igualmente un cursor y sus zooms.
- **La skill `openscreen-demo`** le enseña a un agente todo el trabajo, desde el storyboard hasta el video terminado. Después adjunta el video a un pull request, para que los revisores vean el cambio ahí mismo, o redacta un post en X que solo publica cuando dices que sí. Agrégala con `npx skills add getopenscreen/openscreen --skill openscreen-demo`. La receta es [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md).
diff --git a/website/i18n/fr/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/fr/docusaurus-plugin-content-docs/current/ai-editing.md
index 70592899c..ee65529d0 100644
--- a/website/i18n/fr/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/fr/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
L'exécutable de chaque système est indiqué dans [Lancer la CLI](./cli.md#running-the-cli).
- **Ni port, ni jeton, ni réglage.** L'agent lance lui-même le processus, avec vos droits, comme il lancerait `openscreen record`. Il peut donc enregistrer votre écran et faire des modifications sans l'interrupteur **Modifications du projet**.
-- **Chaque prise devient un nouveau projet** dans votre liste de projets, zoomé sur ses clics comme un enregistrement fait dans l'application : des clics rapprochés partagent un zoom, et les premières secondes comme la fin restent sur l'écran entier. Ne le modifiez pas dans l'éditeur tant que l'agent y travaille encore : l'éditeur garde sa propre copie.
+- **Chaque prise devient un nouveau projet** dans votre liste de projets, zoomé sur ses clics comme un enregistrement fait dans l'application : des clics rapprochés dans le temps et à l'écran partagent un zoom, et les premières secondes comme la fin restent sur l'écran entier. Ne le modifiez pas dans l'éditeur tant que l'agent y travaille encore : l'éditeur garde sa propre copie.
- **Une prise pilotée par un script** (Playwright, par exemple) ne déplace jamais le pointeur de l'écran. L'agent transmet à OpenScreen son propre journal des actions avec `setPointerTrack`, et la vidéo a malgré tout un curseur et ses zooms.
- **La skill `openscreen-demo`** enseigne à un agent tout le travail, du storyboard jusqu'à la vidéo finie. Il joint ensuite la vidéo à une pull request, pour que les relecteurs voient le changement sur place, ou rédige un post sur X qu'il ne publie qu'une fois que vous avez dit oui. Ajoutez-la avec `npx skills add getopenscreen/openscreen --skill openscreen-demo`. La recette est [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md).
diff --git a/website/i18n/ja/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/ja/docusaurus-plugin-content-docs/current/ai-editing.md
index 3ca2ae429..d8ee21745 100644
--- a/website/i18n/ja/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/ja/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
各システムでの実行ファイルは、[CLI の実行](./cli.md#running-the-cli)に記載しています。
- **ポートも、トークンも、設定も不要。** エージェントは、`openscreen record` を実行するときと同じように、あなたの権限でプロセスを自分で起動します。そのため、**プロジェクトの編集**スイッチなしで、画面を録画し、編集できます。
-- **テイクはそれぞれ新しいプロジェクトになり**、プロジェクト一覧に表示されます。アプリで作った録画と同じく、記録されたクリックにズームが付きます。近いクリックは1つのズームにまとめられ、最初の数秒と最後は画面全体のままです。エージェントがまだ作業している間は、エディターで編集しないでください。エディターは独自のコピーを保持しています。
+- **テイクはそれぞれ新しいプロジェクトになり**、プロジェクト一覧に表示されます。アプリで作った録画と同じく、記録されたクリックにズームが付きます。時間的にも画面上でも近いクリックは1つのズームにまとめられ、最初の数秒と最後は画面全体のままです。エージェントがまだ作業している間は、エディターで編集しないでください。エディターは独自のコピーを保持しています。
- **スクリプトで操作するテイク**(たとえば Playwright)では、画面のポインターはまったく動きません。エージェントが `setPointerTrack` で自分の操作ログを OpenScreen に渡せば、動画には同じようにカーソルとそのズームが付きます。
- **`openscreen-demo` スキル**は、絵コンテから完成した動画まで、作業全体をエージェントに教えます。そのあとエージェントは、レビュアーがその場で変更を確認できるよう動画をプルリクエストに添付するか、X への投稿を下書きします。投稿は、あなたが了承してはじめて公開されます。`npx skills add getopenscreen/openscreen --skill openscreen-demo` で追加します。手順は [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md) にあります。
diff --git a/website/i18n/pt-BR/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/pt-BR/docusaurus-plugin-content-docs/current/ai-editing.md
index 8c5355ece..3de3c038b 100644
--- a/website/i18n/pt-BR/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/pt-BR/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
O executável de cada sistema está listado em [Como executar a CLI](./cli.md#running-the-cli).
- **Sem porta, sem token, sem configuração.** O agente inicia o processo por conta própria, com as suas permissões, como faria ao rodar `openscreen record`. Por isso ele pode gravar a sua tela e editar sem o interruptor **Edições do projeto**.
-- **Cada tomada vira um novo projeto** na sua lista de projetos, com zooms nos seus cliques, como uma gravação feita no app: cliques próximos dividem um zoom, e os primeiros segundos e o final ficam na tela inteira. Não o edite no editor enquanto o agente ainda estiver trabalhando nele: o editor mantém a própria cópia.
+- **Cada tomada vira um novo projeto** na sua lista de projetos, com zooms nos seus cliques, como uma gravação feita no app: cliques próximos no tempo e na tela dividem um zoom, e os primeiros segundos e o final ficam na tela inteira. Não o edite no editor enquanto o agente ainda estiver trabalhando nele: o editor mantém a própria cópia.
- **Uma tomada conduzida por um script** (o Playwright, por exemplo) nunca move o ponteiro da tela. O agente passa ao OpenScreen o próprio registro das ações com `setPointerTrack`, e o vídeo ganha um cursor e seus zooms do mesmo jeito.
- **A skill `openscreen-demo`** ensina a um agente o trabalho inteiro, do storyboard até o vídeo pronto. Depois ele anexa o vídeo a um pull request, para que os revisores vejam a mudança ali mesmo, ou redige um post no X que só publica depois que você disser sim. Adicione-a com `npx skills add getopenscreen/openscreen --skill openscreen-demo`. A receita é [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md).
diff --git a/website/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai-editing.md
index cedab1a99..abbf8f33c 100644
--- a/website/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/zh-CN/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
各系统上的可执行文件列在[运行 CLI](./cli.md#running-the-cli)中。
- **无需端口、令牌或设置**。AI 代理以你的权限自行启动进程,就像它运行 `openscreen record` 一样。因此,它无需**项目编辑**开关,就能录制你的屏幕并进行编辑。
-- **每次录制都会成为一个新项目**,出现在你的项目列表中,并且与在应用中录制的内容一样,在录制下来的点击处添加缩放:相近的点击共用一个缩放,开头几秒和结尾保持全屏。在 AI 代理仍在处理它时,不要在编辑器中编辑它:编辑器会保留自己的副本。
+- **每次录制都会成为一个新项目**,出现在你的项目列表中,并且与在应用中录制的内容一样,在录制下来的点击处添加缩放:时间和位置都相近的点击共用一个缩放,开头几秒和结尾保持全屏。在 AI 代理仍在处理它时,不要在编辑器中编辑它:编辑器会保留自己的副本。
- **由脚本驱动的录制**(例如 Playwright)永远不会移动屏幕指针。AI 代理通过 `setPointerTrack` 把它自己的操作日志交给 OpenScreen,视频照样会有光标及其缩放。
- **`openscreen-demo` 技能**会教 AI 代理完成整个工作,从分镜脚本到成品视频。之后,它会把视频附加到拉取请求中,让审阅者直接在那里看到改动;或者起草一条 X 帖子,而且只有在你同意之后,它才会发布这条帖子。使用 `npx skills add getopenscreen/openscreen --skill openscreen-demo` 添加它。具体步骤见 [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md)。
diff --git a/website/i18n/zh-TW/docusaurus-plugin-content-docs/current/ai-editing.md b/website/i18n/zh-TW/docusaurus-plugin-content-docs/current/ai-editing.md
index 63a32b1d3..0c2bd4672 100644
--- a/website/i18n/zh-TW/docusaurus-plugin-content-docs/current/ai-editing.md
+++ b/website/i18n/zh-TW/docusaurus-plugin-content-docs/current/ai-editing.md
@@ -89,6 +89,6 @@ codex mcp add openscreen -- "/Applications/Openscreen.app/Contents/MacOS/Openscr
各系統上的執行檔列在[執行 CLI](./cli.md#running-the-cli)中。
- **不需要連接埠、權杖或設定。** 代理會以你的權限自行啟動程序,就像它執行 `openscreen record` 一樣。因此,它不需要**專案編輯**開關,就能錄製你的螢幕並進行剪輯。
-- **每段錄影都會成為一個新專案**,出現在你的專案清單中,並和在應用程式中錄製的影片一樣,在錄下的點擊處加上縮放:相近的點擊共用一個縮放,開頭幾秒與結尾保持全螢幕。代理仍在處理時,請不要在編輯器中剪輯它:編輯器會保留自己的副本。
+- **每段錄影都會成為一個新專案**,出現在你的專案清單中,並和在應用程式中錄製的影片一樣,在錄下的點擊處加上縮放:時間與位置都相近的點擊共用一個縮放,開頭幾秒與結尾保持全螢幕。代理仍在處理時,請不要在編輯器中剪輯它:編輯器會保留自己的副本。
- **由腳本驅動的錄影**(例如 Playwright)永遠不會移動螢幕上的指標。代理會用 `setPointerTrack` 把它自己的操作記錄交給 OpenScreen,影片照樣會有游標與其縮放。
- **`openscreen-demo` 技能**會教代理完成整個工作,從分鏡腳本到完成的影片。之後,它會把影片附加到拉取請求中,讓審閱者直接在那裡看到變更;或者草擬一則 X 貼文,而且只有在你同意之後,它才會發布這則貼文。使用 `npx skills add getopenscreen/openscreen --skill openscreen-demo` 加入它。完整步驟請見 [`skills/openscreen-demo/SKILL.md`](https://github.com/getopenscreen/openscreen/blob/main/skills/openscreen-demo/SKILL.md)。