Desktop app that records your on-screen work session and uses the GitHub Copilot SDK to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Autopilots like Scout, Microsoft Copilot Cowork, or Copilot Studio.
TypeScript agent-skillsai-agentsautomationcopilotcopilot-clicopilot-coworkcopilot-studioelectronmicrosoft-scoutscreen-recording
Rewrite README to be friendlier and user-facing
adilei744d0488d39a
diff --git a/README.md b/README.mdindex cb7b2e9..bea2a0a 100644--- a/README.md+++ b/README.md@@ -1,10 +1,19 @@# Skill Recorder-Record a real work session on your screen, and turn it into a reusable **skill** or-**automation** for an AI agent. Skill Recorder captures what you did — screen video-plus activity signals — then uses the GitHub Copilot CLI to reconstruct it into a-clear **intent + ordered steps**, which you can generalize into something an agent-can repeat.+**Record yourself doing a task once — turn it into a skill your AI agent can repeat.**++Skill Recorder captures a real work session on your screen — the clicks, the app and+window switches, the pages you visit, and (if you want) your spoken narration — then uses+the **GitHub Copilot CLI** to reconstruct *what you actually did* as a clear **intent plus+an ordered list of steps**. From there, one step turns that single run into something an+agent can reuse:++- a **Skill** — a `SKILL.md` procedure an agent runs on demand, or+- an **Automation** — the same procedure on a schedule or trigger.++Both prefer the agent's **native tools** (like the `gh` CLI or `web_fetch`) over replaying+UI clicks, and generalize from your one example — so recording yourself submitting *one*+form can teach the agent to submit *all* of them.<p align="center"><img src="docs/images/recorder.png" alt="Skill Recorder capture window: a record button, timer, optional narration toggle, and readiness checks" width="420">@@ -12,108 +21,114 @@ can repeat.<img src="docs/images/library.png" alt="Skill Recorder library: recorded sessions on the left, the reconstructed intent and ordered steps on the right" width="520"></p>-## What it does--1. **Record** — hit record (or `⌘⇧R` / `Ctrl+Shift+R` from anywhere) and do your task.- Skill Recorder captures the screen and your activity in the background.-2. **Control** — while recording, a movable always-on-top bar shows capture and- microphone state. Use its split microphone control to mute, unmute, or change- inputs, then finish or discard the recording; discard always asks for confirmation.-3. **Analyze** — when you choose Analyze, relevant recording data is sent to GitHub's- cloud service and processed by GitHub Copilot to reconstruct *what you did*: one- overall intent plus an ordered list of steps. You can review and edit the result.-4. **Create** — from an approved analysis, generalize the one run into a:- - **Skill** — a `SKILL.md` procedure an agent runs on demand, and- - **Automation** — the same procedure on a schedule/trigger.-- Both prefer the agent's **native tools** (e.g. the `gh` CLI, `web_fetch`) over- replaying UI clicks, and generalize from your single example (e.g. "submit one form- per row" for *all* N rows, not the 3 you happened to record).--## What's captured--Recording, storage, frame extraction, and optional narration transcription happen-**locally**, and nothing leaves your computer while you record. When you choose-Analyze, the event timeline (including window/document titles, URLs, and clipboard-previews), extracted screen images, narration text, and other content you provide are-sent to GitHub's cloud service and processed by GitHub Copilot.--Do not record, type, paste, display, copy, or narrate passwords, access tokens, API-keys, credentials, secrets, or other sensitive or confidential information. The app-shows a reminder before recording and its "Records your screen and activity" panel-explains exactly what's collected:--- **Window tracking** — active-app / window switches (Koffi/Win32 on Windows,- `get-windows` elsewhere).-- **Browser URLs** — the page you're on (macOS, via AppleScript).-- **Screen video** — recorded by Chromium; low-rate snapshots are captured alongside- it and retained only when the screen changes or a heartbeat is due.-- **Clipboard** — short previews of copied text that tie steps together.-- **Narration** *(optional)* — turn on **Narrate** before capture or toggle the- microphone from the floating recording bar. Narrate shows the active input and- remembers a selected microphone, with an explicit **System default** fallback.- During capture, the bar can switch inputs without interrupting screen recording;- each microphone-on interval is timestamped on the video timeline, saved locally,- and can be transcribed **on-device** in any of Whisper's 99 supported languages- (via transformers.js), preserving the selected language. The first transcription- uses an explicit, one-time ~252 MB model download. The multilingual q8 checkpoint- is only about 0.6 MB larger than the previous English-only model and uses the same- `small` architecture, so runtime memory and transcription speed are expected to- remain effectively unchanged.--## Requirements--- **macOS** (primary target). Windows 11 x64 and ARM64 are also supported; see- [`WINDOWS-VALIDATION.md`](WINDOWS-VALIDATION.md).-- **Node.js 22+**.-- **GitHub Copilot CLI** installed and signed in — the `copilot` command must be on- your `PATH`. This powers the analysis and the skill/automation builders.-- No system media tools are required. Chromium handles screen snapshots and narration- audio decoding. A system `ffmpeg` is used only when opening a recording created by- an older Skill Recorder version that has no snapshot manifest.+## How it works-On first launch macOS will prompt for **Screen Recording** (required). Enabling-Narrate requests **Microphone** permission immediately so named inputs can be selected-before recording; the permission-check stream is released without saving audio.+1. 🔴 **Record** — Hit record (or `⌘⇧R` / `Ctrl+Shift+R` from anywhere) and just do your+ task. Skill Recorder captures your screen and activity locally, in the background.+2. 🎛️ **Control** — While recording, a small always-on-top bar shows capture and+ microphone state. Mute, unmute, or switch mics on the fly, then finish — or discard+ (with a confirmation) if the take didn't go to plan.+3. 🧠 **Analyze** — Click Analyze and GitHub Copilot reconstructs one overall intent and+ an ordered list of steps. Review and edit until it reads right.+4. ✨ **Create** — From an approved analysis, generate a reusable **Skill** and/or a+ scheduled **Automation**.-## Quick start (download & run)+## Get started-Don't want to clone and manage a source checkout? A one-line installer downloads Skill-Recorder into `~/.skill-recorder`, installs its dependencies, builds it, and launches-it. It's **safe to re-run**: the same command updates an existing install to the latest-code and only reinstalls or rebuilds when something actually changed.+The quickest path: one command downloads Skill Recorder, sets it up, builds it, and+launches it. It's **safe to re-run** — the same command updates your install to the latest+version and only does work when something actually changed.-**macOS / Linux:**+**macOS / Linux**```bashcurl -fsSL https://raw.githubusercontent.com/adilei/skill-recorder/master/install.sh | bash```-**Windows (PowerShell):**+**Windows (PowerShell)**```powershellirm https://raw.githubusercontent.com/adilei/skill-recorder/master/install.ps1 | iex```-You still need **Node.js 22+** and, for analysis, the **GitHub Copilot CLI** (see-[Requirements](#requirements)). The installer prefers `git` and falls back to a source-tarball/zip download when `git` isn't available.+You'll need **Node.js 22+** and the **GitHub Copilot CLI** signed in first (see+[Requirements](#requirements)). On first launch, macOS asks for **Screen Recording**+permission — grant it and you're ready to record.++> ⚠️ **Keep secrets out of your recordings.** Don't record, type, paste, or narrate+> passwords, tokens, API keys, or other confidential info — choosing *Analyze* sends+> recording data to GitHub's cloud. Skill Recorder reminds you before every recording.+> Details in [What gets captured](#what-gets-captured).-Prefix either command with these optional overrides to change its behavior:+<details>+<summary>Install options & updating</summary>-| Variable | Default | Purpose |+Prefix either command with any of these:++| Variable | Default | What it does || --- | --- | --- || `SKILL_RECORDER_HOME` | `~/.skill-recorder` | where the app is installed || `SKILL_RECORDER_REF` | `master` | branch, tag, or commit to install |-| `SKILL_RECORDER_NO_RUN` | *(unset)* | install & build only, don't launch |+| `SKILL_RECORDER_NO_RUN` | *(unset)* | set up without launching |++Re-run the one-liner any time to update and relaunch. To relaunch without re-downloading,+run `npm start` from the install directory. The installer uses `git` when available and+falls back to a source tarball/zip download otherwise.+</details>-To update and relaunch later, just run the same one-liner again; to launch without-re-downloading, run `npm start` from the install directory.+---++*The rest of this README is for people who want the details — or want to hack on the code.*++## Requirements++- **macOS** (primary target). Windows 11 x64 and ARM64 are also supported — see+ [`WINDOWS-VALIDATION.md`](WINDOWS-VALIDATION.md).+- **Node.js 22+**.+- **GitHub Copilot CLI**, installed and signed in, with `copilot` on your `PATH`. It+ powers the analysis and the skill/automation builders.+- No system media tools required — Chromium handles screen snapshots and narration audio+ decoding. A system `ffmpeg` is only used to open recordings made by an older Skill+ Recorder version that has no snapshot manifest.++On first launch macOS prompts for **Screen Recording** (required). Turning on **Narrate**+requests **Microphone** permission immediately so you can pick a named input before+recording; that permission-check stream is released without saving any audio.++## What gets captured++Recording, storage, frame extraction, and optional narration transcription all happen+**on your computer** — nothing leaves while you record. Only when you choose **Analyze**+does Skill Recorder send the event timeline (window/document titles, URLs, and clipboard+previews), extracted screen images, narration text, and anything else you provide to+GitHub's cloud service for Copilot to process.++> ⚠️ **Please don't capture secrets.** Passwords, access tokens, API keys, credentials, and+> other confidential information should never be recorded, typed, pasted, shown, copied,+> or narrated during a session.++The in-app "Records your screen and activity" panel spells out exactly what's collected:++- **Window tracking** — active-app / window switches (Koffi/Win32 on Windows,+ `get-windows` elsewhere).+- **Browser URLs** — the page you're on (macOS, via AppleScript).+- **Screen video** — recorded by Chromium; low-rate snapshots are captured alongside it+ and retained only when the screen changes or a heartbeat is due.+- **Clipboard** — short previews of copied text that tie steps together.+- **Narration** *(optional)* — turn on **Narrate** before capture or toggle the microphone+ from the floating recording bar. Narrate shows the active input and remembers a selected+ microphone, with an explicit **System default** fallback. During capture, the bar can+ switch inputs without interrupting screen recording; each microphone-on interval is+ timestamped on the video timeline, saved locally, and can be transcribed **on-device** in+ any of Whisper's 99 supported languages (via transformers.js), preserving the selected+ language. The first transcription uses an explicit, one-time ~252 MB model download. The+ multilingual q8 checkpoint is only about 0.6 MB larger than the previous English-only+ model and uses the same `small` architecture, so runtime memory and transcription speed+ are expected to remain effectively unchanged.-## Run it (development)+## Run from source (development)-> Prefer the one-line installer above for a quick run. To hack on the code, run it from-> source instead:+Prefer to hack on the code instead of using the installer? Clone the repo and:```bashnpm install@@ -122,8 +137,8 @@ npm run dev`npm run dev` starts Vite and launches the Electron app with hot-reload. The app alsolives in the menu-bar tray; `⌘⇧R` (macOS) / `Ctrl+Shift+R` (Windows) toggles recording-from anywhere. On Windows, press `F12` while the recorder or Sessions window is focused-to toggle DevTools during development.+from anywhere. On Windows, press `F12` while the recorder or Sessions window is focused to+toggle DevTools during development.Other useful scripts: