The VS Code of video editing, built for agents.
TypeScript
Refactor media command structure and update documentation for clarity
konstantin-paulus4a92a1913b8a
diff --git a/README.md b/README.mdindex a8d76819..7c89961f 100644--- a/README.md+++ b/README.md@@ -60,11 +60,11 @@ Everything a mount produces stays a first-class editor node, so a person can picCutting footage requires looking at it. The CLI ships the inspection tools an agent needs to work with media it cannot watch:```sh-dapi asset probe clip.mp4 # container + codec metadata, like ffprobe-dapi asset frame clip.mp4 -t 0 12 45 # decode frames to PNGs-dapi asset visualize track.mp3 # waveform / filmstrip previews-dapi asset transcribe interview.wav # timed, word-level transcript-dapi asset analyze b-roll.mp4 -p "when is the product shown?" # ask a vision model+dapi media probe clip.mp4 # container + codec metadata, like ffprobe+dapi media grab clip.mp4 -t 0 12 45 # decode frames to PNGs+dapi media visualize track.mp3 # waveform / filmstrip previews+dapi media transcribe interview.wav # timed, word-level transcript+dapi media analyze b-roll.mp4 -p "when is the product shown?" # ask a vision modeldapi node screenshot # see the canvas itself```@@ -76,7 +76,8 @@ dapi node screenshot # see the canvas itself| `dapi context` | Summary of the open project: scenes, playhead, fonts || `dapi mount` | Compile and mount a JSX composition || `dapi node …` | The scene graph: `ls`, `tree`, `grep`, `patch`, `insert`, `cp`, `rm`, `screenshot`, `export` |-| `dapi asset …` | The media library: `add`, `ls`, `probe`, `frame`, `visualize`, `analyze`, `transcribe`, `export` |+| `dapi asset …` | The media library: `add`, `ls`, `tree`, `mv`, `rm`, `export` |+| `dapi media …` | Inspect a file by id or path: `probe`, `grab`, `visualize`, `transcribe`, `analyze` || `dapi project …` / `dapi folder …` / `dapi selection …` | Projects, library folders, canvas selection || `dapi models` / `dapi voices` / `dapi fonts` | Discover generation models, speech voices, local fonts |diff --git a/apps/cli/reference/README.md b/apps/cli/reference/README.mdindex ca5e709f..78e646c3 100644--- a/apps/cli/reference/README.md+++ b/apps/cli/reference/README.md@@ -70,11 +70,14 @@ How the surface is divided:- [`dapi asset rm`](./asset/rm.md): delete assets- [`dapi asset mv`](./asset/mv.md): move assets into a folder- [`dapi asset export`](./asset/export.md): write assets' original bytes to disk-- [`dapi asset probe`](./asset/probe.md): container and track metadata-- [`dapi asset transcribe`](./asset/transcribe.md): timed speech transcript-- [`dapi asset grab`](./asset/grab.md): decode video frames to PNGs-- [`dapi asset visualize`](./asset/visualize.md): waveform / filmstrip / thumbnail PNG-- [`dapi asset analyze`](./asset/analyze.md): AI description of an asset++### Media++- [`dapi media probe`](./media/probe.md): container and track metadata+- [`dapi media transcribe`](./media/transcribe.md): timed speech transcript+- [`dapi media grab`](./media/grab.md): decode video frames to PNGs+- [`dapi media visualize`](./media/visualize.md): waveform / filmstrip / thumbnail PNG+- [`dapi media analyze`](./media/analyze.md): AI description of a media file### Foldersdiff --git a/apps/cli/reference/jsx/timing.md b/apps/cli/reference/jsx/timing.mdindex 2f092072..bc621483 100644--- a/apps/cli/reference/jsx/timing.md+++ b/apps/cli/reference/jsx/timing.md@@ -31,4 +31,4 @@ The canonical internal unit is frames at **30 fps**; all formats are converted o| `"MM:SS"` | `"02:30"` | Minutes and seconds. || `"HH:MM:SS"` | `"01:02:30"` | Hours, minutes, seconds. |-CLI flags documented as taking a `Time` value (e.g. `--time` on [`node screenshot`](../node/screenshot.md) and [`asset grab`](../asset/grab.md)) accept the same formats.+CLI flags documented as taking a `Time` value (e.g. `--time` on [`node screenshot`](../node/screenshot.md) and [`media grab`](../media/grab.md)) accept the same formats.diff --git a/apps/cli/reference/asset/analyze.md b/apps/cli/reference/media/analyze.mdsimilarity index 97%rename from apps/cli/reference/asset/analyze.mdrename to apps/cli/reference/media/analyze.mdindex 6077628d..5e161d65 100644--- a/apps/cli/reference/asset/analyze.md+++ b/apps/cli/reference/media/analyze.md@@ -1,4 +1,4 @@-# `dapi asset analyze <id|path>`+# `dapi media analyze <id|path>`Analyzes an image, video, or audio asset with a multimodal model. With no prompt it returns a general description of the asset; with `--prompt` it answers that question about the asset (e.g. "what's the dominant color?", "summarize what happens").diff --git a/apps/cli/reference/asset/grab.md b/apps/cli/reference/media/grab.mdsimilarity index 97%rename from apps/cli/reference/asset/grab.mdrename to apps/cli/reference/media/grab.mdindex 078179a1..bb0b843f 100644--- a/apps/cli/reference/asset/grab.md+++ b/apps/cli/reference/media/grab.md@@ -1,4 +1,4 @@-# `dapi asset grab <id|path>`+# `dapi media grab <id|path>`Decodes one or more frames of a video asset at the given times and writes each to a PNG file; like [`node screenshot`](../node/screenshot.md), but grabs the asset's own pixels at full resolution (unlike `node screenshot`, which captures the composited canvas). Renders locally; no credits.diff --git a/apps/cli/reference/asset/probe.md b/apps/cli/reference/media/probe.mdsimilarity index 96%rename from apps/cli/reference/asset/probe.mdrename to apps/cli/reference/media/probe.mdindex 9650106e..7bd30247 100644--- a/apps/cli/reference/asset/probe.md+++ b/apps/cli/reference/media/probe.md@@ -1,4 +1,4 @@-# `dapi asset probe <id|path>`+# `dapi media probe <id|path>`Reads the container and per-track technical metadata of an asset; like `ffprobe`, but demuxed locally with mediabunny. Reports the container format, duration, metadata tags, and every track's codec parameters without decoding any media. Reads locally; no credits.diff --git a/apps/cli/reference/asset/transcribe.md b/apps/cli/reference/media/transcribe.mdsimilarity index 96%rename from apps/cli/reference/asset/transcribe.mdrename to apps/cli/reference/media/transcribe.mdindex 36d88bed..0a0b28ee 100644--- a/apps/cli/reference/asset/transcribe.md+++ b/apps/cli/reference/media/transcribe.md@@ -1,4 +1,4 @@-# `dapi asset transcribe [options] <id|path>`+# `dapi media transcribe [options] <id|path>`Transcribes the speech in a video or audio asset and returns the timed transcript. Word-level start/end times are in **seconds** (source/content time).diff --git a/apps/cli/reference/asset/visualize.md b/apps/cli/reference/media/visualize.mdsimilarity index 98%rename from apps/cli/reference/asset/visualize.mdrename to apps/cli/reference/media/visualize.mdindex 961dbd8c..fcf20a2f 100644--- a/apps/cli/reference/asset/visualize.md+++ b/apps/cli/reference/media/visualize.md@@ -1,4 +1,4 @@-# `dapi asset visualize <id|path>`+# `dapi media visualize <id|path>`Renders a visual preview of an asset to a PNG, written to a file in the system temp directory. The visualization is chosen by the asset's media type, so a single command covers the whole library. Renders locally; no credits. Alias: `viz`.diff --git a/apps/cli/reference/node/screenshot.md b/apps/cli/reference/node/screenshot.mdindex e34631fc..c6f9df02 100644--- a/apps/cli/reference/node/screenshot.md+++ b/apps/cli/reference/node/screenshot.md@@ -1,6 +1,6 @@# `dapi node screenshot [id] [-t, --time <time>]`-Focuses the node on the canvas and captures a screenshot, written to a PNG file in the system temp directory. If `[id]` is omitted, captures the canvas (the active scene). To grab a video asset's own pixels at full resolution instead of the composited canvas, use [`asset grab`](../asset/grab.md).+Focuses the node on the canvas and captures a screenshot, written to a PNG file in the system temp directory. If `[id]` is omitted, captures the canvas (the active scene). To grab a video asset's own pixels at full resolution instead of the composited canvas, use [`media grab`](../media/grab.md).## Inputdiff --git a/apps/cli/src/index.ts b/apps/cli/src/index.tsindex 32016f99..0e0a4d61 100644--- a/apps/cli/src/index.ts+++ b/apps/cli/src/index.ts@@ -1019,7 +1019,7 @@ const asset = program.command("asset").alias("a").description(- "Manage, analyze, and generate assets in the open project — probe and export files, transcribe speech, decode frames, render visual previews, and analyze with VLMs.",+ "Manage the asset library of the open project — add local files, list and organize records, and export original bytes.",);asset@@ -1065,33 +1065,40 @@ asset.option("-o, --output <path>", "directory to write into, or an exact file path for a single id (default: system temp dir)").action((ids: string[], opts: AssetExportOptions) => exportAssets(ids, opts));-asset+const media = program+ .command("media")+ .alias("m")+ .description(+ "Inspect a media file by asset id or local path, without adding it to the project — probe metadata, transcribe speech, grab frames, render visual previews, and analyze with multimodal models.",+ );++media.command("probe")- .description(`Read the container and per-track technical metadata of an asset. Commonly useful for a quick, free technical read (container, duration, per-track codec params), e.g. checking codec compatibility or duration before cutting${docs("asset/probe")}`)+ .description(`Read the container and per-track technical metadata of a media file. Commonly useful for a quick, free technical read (container, duration, per-track codec params), e.g. checking codec compatibility or duration before cutting${docs("media/probe")}`).argument("<id|path>", "asset id, or a local file").action((ref: string) => assetProbe(ref));-asset+media.command("transcribe")- .description(`Transcribe the speech in a video or audio asset and print the timed transcript. Commonly useful for footage with speakers (talking head, interview), where the word-level times let you cut on a line. Note a transcript marks only speech; the gaps are not necessarily silent (music, score, applause)${docs("asset/transcribe")}`)+ .description(`Transcribe the speech in a video or audio file and print the timed transcript. Commonly useful for footage with speakers (talking head, interview), where the word-level times let you cut on a line. Note a transcript marks only speech; the gaps are not necessarily silent (music, score, applause)${docs("media/transcribe")}`).argument("<id|path>", "video or audio asset id, or a local file").option("-s, --start <time>", `start of the range to print — seconds, "45f" frames, or "MM:SS" (default: 0)`).option("-e, --end <time>", `end of the range to print — seconds, "45f" frames, or "MM:SS" (default: asset duration)`).action((ref: string, opts: AssetTranscribeOptions) => assetTranscribe(ref, opts));-asset+media.command("grab")- .description(`Decode one or more frames of a video asset and write them as high-res PNGs, each stamped in the top-left with its HH:MM:SS:FF timestamp. Best for inspecting individual frames in detail where a filmstrip is too coarse: seeks to the exact requested time with frame-level precision${docs("asset/grab")}`)+ .description(`Decode one or more frames of a video file and write them as high-res PNGs, each stamped in the top-left with its HH:MM:SS:FF timestamp. Best for inspecting individual frames in detail where a filmstrip is too coarse: seeks to the exact requested time with frame-level precision${docs("media/grab")}`).argument("<id|path>", "video asset id, or a local video file to grab frames from").option("-t, --time <time...>", `one or more timestamps to grab — seconds ("1.5"), frames ("45f"), or "MM:SS" (default: 0)`).option("-r, --resolution <pixels>", "cap each frame to this many total pixels, preserving aspect ratio; 0 for native (default: 147456, i.e. 384x384)").option("-o, --output <dir>", "directory to write the PNGs into (default: system temp dir)").action((ref: string, opts: AssetFrameOptions) => assetFrame(ref, opts));-asset+media.command("visualize").alias("viz")- .description(`Render a compact visual preview of an asset to a PNG: a waveform for audio, a filmstrip plus waveform for video, or a thumbnail for an image. The primary tool for understanding a video; you can narrow the window to zoom into a region of interest. Fast and token-efficient. The waveform shows loudness over time and marks the silent stretches${docs("asset/visualize")}`)+ .description(`Render a compact visual preview of a media file to a PNG: a waveform for audio, a filmstrip plus waveform for video, or a thumbnail for an image. The primary tool for understanding a video; you can narrow the window to zoom into a region of interest. Fast and token-efficient. The waveform shows loudness over time and marks the silent stretches${docs("media/visualize")}`).argument("<id|path>", "image, audio, or video asset id, or a local file to visualize").option("-s, --start <time>", `start of the window to visualize — seconds, "45f" frames, or "MM:SS" (default: 0)`).option("-e, --end <time>", `end of the window to visualize — seconds, "45f" frames, or "MM:SS" (default: asset duration)`)@@ -1099,9 +1106,9 @@ asset.option("-o, --output <path>", "write the PNG here instead of a temp file").action((ref: string, opts: AssetVisualizeOptions) => assetVisualize(ref, opts));-asset+media.command("analyze")- .description(`Prompt a multimodal model for a semantic analysis of an asset and print its answer. Handles images, audio, and video, but shines on the semantics of audio: what it actually contains (the name of the music playing, who is speaking, the spoken content with second-granularity timestamps)${docs("asset/analyze")}`)+ .description(`Prompt a multimodal model for a semantic analysis of a media file and print its answer. Handles images, audio, and video, but shines on the semantics of audio: what it actually contains (the name of the music playing, who is speaking, the spoken content with second-granularity timestamps)${docs("media/analyze")}`).argument("<id|path>", "image, audio, or video asset id, or a local file to analyze").option("-p, --prompt <str>", "question or instruction to guide the analysis").option("-s, --start <time>", `start of the segment to analyze — seconds, "45f" frames, or "MM:SS" (default: 0); timestamps in the analysis are relative to this point`)diff --git a/docs/assets.md b/docs/assets.mdindex da28e488..57029109 100644--- a/docs/assets.md+++ b/docs/assets.md@@ -1,8 +1,8 @@# Working with media-The asset library holds a project's media. Beyond storage, the CLI ships an inspection toolchain (probe, decode, visualize, transcribe, analyze) so an agent can *look at* footage before cutting it. All of it runs locally except `analyze` and `transcribe`, which use hosted models.+The asset library holds a project's media. Beyond storage, the CLI ships an inspection toolchain under `dapi media` (probe, grab, visualize, transcribe, analyze) so an agent can *look at* footage before cutting it. All of it runs locally except `analyze` and `transcribe`, which use hosted models.-**Id or path:** every inspection command takes either an asset id or a local file path. A path is imported into the library first, then inspected, so `dapi asset probe clip.mp4` works on a file that isn't in the project yet.+**Id or path:** every inspection command takes either an asset id or a local file path. A path is read directly, so `dapi media probe clip.mp4` works on a file that isn't in the project yet.## The library@@ -30,42 +30,44 @@ Omitting a folder id anywhere always means the library root. (Opening a folder w## Inspecting media-### `asset probe`+Inspection lives under its own group, `dapi media`, because none of these commands touch the library: they read a media file by asset id or by local path and never mutate the project.++### `media probe````sh-dapi asset probe <id|path>+dapi media probe <id|path>```Container and per-track technical metadata (format, duration, codec parameters), demuxed locally without decoding, like `ffprobe`. Packet stats (frame rate, bitrate) are estimated from a leading sample, so they're fast but approximate.-### `asset frame`+### `media grab````sh-dapi asset frame <id|path> [-t <time...>] [-o <dir>]+dapi media grab <id|path> [-t <time...>] [-o <dir>]```Decodes video frames at the given timestamps to PNGs, at the asset's full resolution. Prints `{ time, path }` per frame. (For the *composited* canvas instead of the raw asset, use `dapi node screenshot`.)-### `asset visualize`+### `media visualize````sh-dapi asset visualize <id|path> [-s <start>] [-e <end>] [-x <scale>] [-o <path>]+dapi media visualize <id|path> [-s <start>] [-e <end>] [-x <scale>] [-o <path>]```Renders a one-image preview chosen by media type: audio gets an amplitude **waveform** with a time axis, video gets a **filmstrip** of evenly sampled frames with the audio waveform beneath it, and images get a thumbnail. `-s`/`-e` window the preview; `-x` trades thumbnail size against grid density. Prints `{ path }`.-### `asset transcribe`+### `media transcribe````sh-dapi asset transcribe <id|path>+dapi media transcribe <id|path>```-Transcribes speech in a video or audio asset. Returns segments with **word-level start/end times in seconds**: the raw material for caption timing, jump cuts, and content search.+Transcribes speech in a video or audio file. Returns segments with **word-level start/end times in seconds**: the raw material for caption timing, jump cuts, and content search.-### `asset analyze`+### `media analyze````sh-dapi asset analyze <id|path> [-p <prompt>] [-s <start>] [-e <end>]+dapi media analyze <id|path> [-p <prompt>] [-s <start>] [-e <end>]```Puts a multimodal model in front of an image, video, or audio asset. Without a prompt, returns a general description; with one, answers it ("what's the dominant color?", "when is the product shown?"). `-s`/`-e` scope the analyzed segment; timestamps in the answer are relative to `-s`.@@ -76,10 +78,10 @@ Puts a multimodal model in front of an image, video, or audio asset. Without a p```shdapi open -b ./shoot # project from footagedapi asset tree --depth 1 # what's here?-dapi asset probe gbHJ # what format is it?-dapi asset visualize gbHJ # what does it look like over time?-dapi asset transcribe gbHJ # what is said, and when?-dapi asset analyze gbHJ -p "best moment for a thumbnail?"+dapi media probe gbHJ # what format is it?+dapi media visualize gbHJ # what does it look like over time?+dapi media transcribe gbHJ # what is said, and when?+dapi media analyze gbHJ -p "best moment for a thumbnail?"dapi mount cut.tsx # compose the editdapi node screenshot # verify visuallydapi node render -o cut.mp4 # ship it