Skip to content

The VS Code of video editing, built for agents.

TypeScript

Refactor media command structure and update documentation

konstantin-paulus55e1f1189ff3
diff --git a/README.md b/README.md
index 50e2f6be..04c8d7e2 100644
--- a/README.md
+++ b/README.md
@@ -180,16 +180,16 @@ Everything a mount produces stays a first-class editor node, so a person can pic
Cutting footage requires understanding it. The app exposes the inspection tools an agent needs to work with media it cannot watch — as MCP tools, and as the same commands in a shell:
```sh
-diffusion media probe clip.mp4 # container + codec metadata, like ffprobe
-diffusion media grab clip.mp4 -t 0 12 45 # decode frames to PNGs
-diffusion media filmstrip clip.mp4 # grid of video frames
-diffusion media waveform track.mp3 # audio waveform, silence flagged
-diffusion media transcribe interview.wav # timed, word-level transcript
-diffusion media listen interview.mp4 -p "what is said in the intro?" # ask a multimodal model
-diffusion capture intro -t 0 2 4 # the frames a render would produce, by scene id
+diffusion probe clip.mp4 # container + codec metadata, like ffprobe
+diffusion grab clip.mp4 -t 0 12 45 # decode frames to PNGs
+diffusion filmstrip clip.mp4 # grid of video frames
+diffusion waveform track.mp3 # audio waveform, silence flagged
+diffusion transcribe interview.wav # timed, word-level transcript
+diffusion listen interview.mp4 -p "what is said in the intro?" # ask a multimodal model
+diffusion capture intro -t 0 2 4 # the frames a render would produce, by scene id
```
-Each command is the MCP tool of the same name: `diffusion media grab` is `media_grab`, `--per-sheet` is `perSheet`.
+Each command is the MCP tool of the same name: `diffusion grab` is `grab`, `--per-sheet` is `perSheet`.
| Command | Purpose |
| --- | --- |
@@ -197,7 +197,7 @@ Each command is the MCP tool of the same name: `diffusion media grab` is `media_
| `diffusion context` | Summary of app state |
| `diffusion capture` | Render frames of a scene, as an export would, to a labelled contact sheet or one PNG per position |
| `diffusion check` | Check a node's subtree for structural mistakes (black-frame gaps, never-visible nodes, failed sources) and report subtree stats |
-| `diffusion media …` | Inspect a file by id or path: `probe`, `grab`, `filmstrip`, `waveform`, `transcribe`, `listen` |
+| `diffusion probe` / `grab` / `filmstrip` / `waveform` / `transcribe` / `listen` | Inspect a file by id or path |
| `diffusion fonts` | Discover Google Fonts and local font families |
| `diffusion screenshot` / `diffusion logs` | The app itself: capture the window, read recent console output |
| `diffusion report` | Report a bug in the tools or the app: diagnostics bundled, filed as a GitHub issue via `gh` |
diff --git a/apps/cli/src/cli-client.ts b/apps/cli/src/cli-client.ts
index 53e07ad4..cae1aafc 100644
--- a/apps/cli/src/cli-client.ts
+++ b/apps/cli/src/cli-client.ts
@@ -16,8 +16,8 @@ export const APP_NAME = "Diffusion Studio";
const TIMEOUTS: Record<string, number> = {
export: 3_600_000,
capture: 600_000,
- media_transcribe: 600_000,
- media_listen: 600_000,
+ transcribe: 600_000,
+ listen: 600_000,
generate: 600_000,
job: 600_000,
};
diff --git a/apps/cli/src/index.ts b/apps/cli/src/index.ts
index a62111f4..212612ff 100644
--- a/apps/cli/src/index.ts
+++ b/apps/cli/src/index.ts
@@ -107,7 +107,7 @@ Use for any media analysis or video editing task. No ffmpeg needed.`)
program
.command("open")
.description(
- `Launch ${APP_NAME} (or surface the running instance) and, given a path, open that folder as a project, creating the project files if the folder is not one yet. Prints the project's id, display name, and folder. Run this once before commands that need an open project (capture, check, export, context, and library paths in media commands).`,
+ `Launch ${APP_NAME} (or surface the running instance) and, given a path, open that folder as a project, creating the project files if the folder is not one yet. Prints the project's id, display name, and folder. Run this once before commands that need an open project.`,
)
.argument("[path]", `${field("open", "dir")} (default: none — just launch the app)`)
.option("-b, --background", "launch or keep the app in the background, without raising a window")
@@ -161,81 +161,74 @@ program
if (output.issues.some((issue) => issue.severity === "error")) process.exitCode = 1;
});
-const media = program
- .command("media")
- .alias("m")
- .description(
- "Inspect a media file by path, without adding it to the project: probe metadata, transcribe speech, grab frames, render visual previews, and analyze with multimodal models. Local files work with or without an open project; library paths need one.",
- );
-
-media
+program
.command("probe")
- .description(describe("media_probe"))
- .argument("<path>", field("media_probe", "path"))
- .action((ref: string) => run("media_probe", { path: assetPath(ref) }));
+ .description(describe("probe"))
+ .argument("<path>", field("probe", "path"))
+ .action((ref: string) => run("probe", { path: assetPath(ref) }));
-media
+program
.command("transcribe")
- .description(describe("media_transcribe"))
- .argument("<path>", field("media_transcribe", "path"))
- .option("-o, --output <path>", field("media_transcribe", "output"))
- .action((ref: string, opts: Omit<ToolInput<"media_transcribe">, "path">) =>
- run("media_transcribe", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
+ .description(describe("transcribe"))
+ .argument("<path>", field("transcribe", "path"))
+ .option("-o, --output <path>", field("transcribe", "output"))
+ .action((ref: string, opts: Omit<ToolInput<"transcribe">, "path">) =>
+ run("transcribe", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
);
-media
+program
.command("grab")
.alias("sample")
- .description(describe("media_grab"))
- .argument("<path>", field("media_grab", "path"))
- .option("-t, --times <time...>", field("media_grab", "times"))
- .option("-c, --count <n>", field("media_grab", "count"), numeric)
- .option("-a, --auto", field("media_grab", "auto"))
- .option("-s, --start <time>", field("media_grab", "start"))
- .option("-e, --end <time>", field("media_grab", "end"))
- .option("-q, --quality <preset>", field("media_grab", "quality"))
- .option("-S, --separate", field("media_grab", "separate"))
- .option("--per-sheet <n>", field("media_grab", "perSheet"), numeric)
- .option("--uncapped", field("media_grab", "uncapped"))
- .option("-o, --output <dir>", field("media_grab", "output"))
- .action((ref: string, opts: Omit<ToolInput<"media_grab">, "path">) =>
- run("media_grab", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
+ .description(describe("grab"))
+ .argument("<path>", field("grab", "path"))
+ .option("-t, --times <time...>", field("grab", "times"))
+ .option("-c, --count <n>", field("grab", "count"), numeric)
+ .option("-a, --auto", field("grab", "auto"))
+ .option("-s, --start <time>", field("grab", "start"))
+ .option("-e, --end <time>", field("grab", "end"))
+ .option("-q, --quality <preset>", field("grab", "quality"))
+ .option("-S, --separate", field("grab", "separate"))
+ .option("--per-sheet <n>", field("grab", "perSheet"), numeric)
+ .option("--uncapped", field("grab", "uncapped"))
+ .option("-o, --output <dir>", field("grab", "output"))
+ .action((ref: string, opts: Omit<ToolInput<"grab">, "path">) =>
+ run("grab", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
);
-media
+program
.command("filmstrip")
.alias("film")
- .description(describe("media_filmstrip"))
- .argument("<path>", field("media_filmstrip", "path"))
- .option("-s, --start <time>", field("media_filmstrip", "start"))
- .option("-e, --end <time>", field("media_filmstrip", "end"))
- .option("-x, --scale <factor>", field("media_filmstrip", "scale"), numeric)
- .option("-o, --output <path>", field("media_filmstrip", "output"))
- .action((ref: string, opts: Omit<ToolInput<"media_filmstrip">, "path">) =>
- run("media_filmstrip", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
+ .description(describe("filmstrip"))
+ .argument("<path>", field("filmstrip", "path"))
+ .option("-s, --start <time>", field("filmstrip", "start"))
+ .option("-e, --end <time>", field("filmstrip", "end"))
+ .option("-x, --scale <factor>", field("filmstrip", "scale"), numeric)
+ .option("-o, --output <path>", field("filmstrip", "output"))
+ .action((ref: string, opts: Omit<ToolInput<"filmstrip">, "path">) =>
+ run("filmstrip", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
);
-media
+program
.command("waveform")
.alias("wave")
- .description(describe("media_waveform"))
- .argument("<path>", field("media_waveform", "path"))
- .option("-s, --start <time>", field("media_waveform", "start"))
- .option("-e, --end <time>", field("media_waveform", "end"))
- .option("-x, --scale <factor>", field("media_waveform", "scale"), numeric)
- .option("-o, --output <path>", field("media_waveform", "output"))
- .action((ref: string, opts: Omit<ToolInput<"media_waveform">, "path">) =>
- run("media_waveform", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
+ .description(describe("waveform"))
+ .argument("<path>", field("waveform", "path"))
+ .option("-s, --start <time>", field("waveform", "start"))
+ .option("-e, --end <time>", field("waveform", "end"))
+ .option("-x, --scale <factor>", field("waveform", "scale"), numeric)
+ .option("-o, --output <path>", field("waveform", "output"))
+ .action((ref: string, opts: Omit<ToolInput<"waveform">, "path">) =>
+ run("waveform", { path: assetPath(ref), ...opts, output: opts.output && resolve(opts.output) }),
);
-media
+program
.command("listen")
- .description(describe("media_listen"))
- .argument("<path>", field("media_listen", "path"))
- .option("-p, --prompt <str>", field("media_listen", "prompt"))
- .option("-s, --start <time>", field("media_listen", "start"))
- .option("-e, --end <time>", field("media_listen", "end"))
- .action((ref: string, opts: Omit<ToolInput<"media_listen">, "path">) => run("media_listen", { path: assetPath(ref), ...opts }));
+ .description(describe("listen"))
+ .argument("<path>", field("listen", "path"))
+ .option("-p, --prompt <str>", field("listen", "prompt"))
+ .option("-s, --start <time>", field("listen", "start"))
+ .option("-e, --end <time>", field("listen", "end"))
+ .action((ref: string, opts: Omit<ToolInput<"listen">, "path">) => run("listen", { path: assetPath(ref), ...opts }));
program
.command("generate")
@@ -326,4 +319,8 @@ program
// Explicit argv convention: the packaged wrapper runs this bundle on
// Electron in ELECTRON_RUN_AS_NODE mode, where commander would otherwise
// detect Electron and drop the script path from argv.
-program.parse(process.argv, { from: "node" });
+// The media commands used to sit under a `media` (`m`) group; drop that word so
+// scripts written against it (older projects' npm scripts) keep working.
+const argv = ["media", "m"].includes(process.argv[2] ?? "") ? process.argv.toSpliced(2, 1) : process.argv;
+
+program.parse(argv, { from: "node" });
diff --git a/apps/desktop/src/dapi/present.test.ts b/apps/desktop/src/dapi/present.test.ts
index 1ad4da8f..bbbea7b0 100644
--- a/apps/desktop/src/dapi/present.test.ts
+++ b/apps/desktop/src/dapi/present.test.ts
@@ -25,7 +25,7 @@ describe("present", () => {
});
it("picks a fresh temp directory when none is given", async () => {
- const presented = await present("media_grab", { path: "/c.mp4" }, [{ timecode: "0f", png: png(3) }]);
+ const presented = await present("grab", { path: "/c.mp4" }, [{ timecode: "0f", png: png(3) }]);
const { path } = (presented.output as { images: Array<{ path: string }> }).images[0]!;
expect(path).toMatch(/dapi-grab-.*[\\/]0f\.png$/);
rmSync(join(path, ".."), { recursive: true, force: true });
@@ -33,20 +33,20 @@ describe("present", () => {
it("keeps a preview's other fields next to the path", async () => {
const file = join(dir, "wave.png");
- const presented = await present("media_waveform", { path: "/c.mp4", output: file }, { png: png(4), silences: [{ start: 0, end: 1 }] });
+ const presented = await present("waveform", { path: "/c.mp4", output: file }, { png: png(4), silences: [{ start: 0, end: 1 }] });
expect(presented.output).toEqual({ path: file, silences: [{ start: 0, end: 1 }] });
});
it("writes a preview or transcript into an output that names an existing directory", async () => {
const out = join(dir, "into");
mkdirSync(out, { recursive: true });
- const preview = await present("media_waveform", { path: "/c.mp4", output: out }, { png: png(7), silences: [] });
+ const preview = await present("waveform", { path: "/c.mp4", output: out }, { png: png(7), silences: [] });
const previewPath = (preview.output as { path: string }).path;
expect(dirname(previewPath)).toBe(out);
expect(basename(previewPath)).toMatch(/^dapi-waveform-.*\.png$/);
expect(readFileSync(previewPath)).toEqual(Buffer.from(png(7)));
- const transcript = await present("media_transcribe", { path: "/c.mp4", output: out }, { segments: [] });
+ const transcript = await present("transcribe", { path: "/c.mp4", output: out }, { segments: [] });
const transcriptPath = (transcript.output as { path: string }).path;
expect(transcriptPath).toMatch(/dapi-transcript-.*\.json$/);
expect(JSON.parse(readFileSync(transcriptPath, "utf8"))).toEqual({ segments: [] });
@@ -64,7 +64,7 @@ describe("present", () => {
it("writes a transcript to a file, unchanged, and returns its path and size", async () => {
const file = join(dir, "talk.json");
const segments = [{ text: "Hi there", words: [{ text: "Hi", start: 0, end: 0.2 }, { text: "there", start: 0.3, end: 0.6 }] }];
- const presented = await present("media_transcribe", { path: "/c.mp4", output: file }, { segments });
+ const presented = await present("transcribe", { path: "/c.mp4", output: file }, { segments });
expect(presented).toEqual({ output: { path: file, segments: 1, words: 2 }, images: [] });
expect(JSON.parse(readFileSync(file, "utf8"))).toEqual({ segments });
});
diff --git a/apps/desktop/src/dapi/present.ts b/apps/desktop/src/dapi/present.ts
index daad6cbf..10fa3e3e 100644
--- a/apps/desktop/src/dapi/present.ts
+++ b/apps/desktop/src/dapi/present.ts
@@ -40,16 +40,16 @@ export async function present(name: ToolName, args: unknown, result: unknown): P
switch (name) {
case "capture":
return presentImages(result as ToolResult<"capture">, (args as ToolArgs<"capture">).output, "capture");
- case "media_grab":
- return presentImages(result as ToolResult<"media_grab">, (args as ToolArgs<"media_grab">).output, "grab");
- case "media_filmstrip":
- return presentPreview(result as ToolResult<"media_filmstrip">, (args as ToolArgs<"media_filmstrip">).output, "filmstrip");
- case "media_waveform":
- return presentPreview(result as ToolResult<"media_waveform">, (args as ToolArgs<"media_waveform">).output, "waveform");
+ case "grab":
+ return presentImages(result as ToolResult<"grab">, (args as ToolArgs<"grab">).output, "grab");
+ case "filmstrip":
+ return presentPreview(result as ToolResult<"filmstrip">, (args as ToolArgs<"filmstrip">).output, "filmstrip");
+ case "waveform":
+ return presentPreview(result as ToolResult<"waveform">, (args as ToolArgs<"waveform">).output, "waveform");
case "screenshot":
return presentScreenshot(result as ToolResult<"screenshot">, (args as ToolArgs<"screenshot">).output);
- case "media_transcribe":
- return presentTranscript(result as ToolResult<"media_transcribe">, (args as ToolArgs<"media_transcribe">).output);
+ case "transcribe":
+ return presentTranscript(result as ToolResult<"transcribe">, (args as ToolArgs<"transcribe">).output);
case "generate":
if ((args as ToolArgs<"generate">).estimate) return { output: result, images: [] };
return presentJob(result as GenerationResult);
@@ -113,11 +113,11 @@ async function presentScreenshot(result: ToolResult<"screenshot">, output: strin
return { output: presented, images: [{ path, png: result.png }] };
}
-async function presentTranscript(transcript: ToolResult<"media_transcribe">, output: string | undefined): Promise<Presented> {
+async function presentTranscript(transcript: ToolResult<"transcribe">, output: string | undefined): Promise<Presented> {
const path = await singleFilePath(output, `dapi-transcript-${randomUUID()}.json`);
await writeFile(path, JSON.stringify(transcript, null, 2));
const words = transcript.segments.reduce((sum, segment) => sum + segment.words.length, 0);
- const presented: ToolOutput<"media_transcribe"> = { path, segments: transcript.segments.length, words };
+ const presented: ToolOutput<"transcribe"> = { path, segments: transcript.segments.length, words };
return { output: presented, images: [] };
}
diff --git a/apps/desktop/src/projects.ts b/apps/desktop/src/projects.ts
index a787d4c7..bd5ca0f4 100644
--- a/apps/desktop/src/projects.ts
+++ b/apps/desktop/src/projects.ts
@@ -452,20 +452,19 @@ const SOLID_VERSION = "^1.9.10";
/**
* The diffusion surface as npm scripts: the CLI is how a project is inspected and
* cut, so its commands belong in the record of the project they act on —
- * `npm run` prints the menu, `npm run <name> -- <args>` runs one. Named after
- * the command rather than its path (`grab`, not `media:grab`): the `media`
- * subcommands have no top-level namesakes to collide with.
+ * `npm run` prints the menu, `npm run <name> -- <args>` runs one, named after
+ * the command.
*/
const SCRIPTS: Record<string, string> = {
open: "diffusion open .",
context: "diffusion context",
capture: "diffusion capture",
- probe: "diffusion media probe",
- transcribe: "diffusion media transcribe",
- grab: "diffusion media grab",
- filmstrip: "diffusion media filmstrip",
- waveform: "diffusion media waveform",
- listen: "diffusion media listen",
+ probe: "diffusion probe",
+ transcribe: "diffusion transcribe",
+ grab: "diffusion grab",
+ filmstrip: "diffusion filmstrip",
+ waveform: "diffusion waveform",
+ listen: "diffusion listen",
fonts: "diffusion fonts",
logs: "diffusion logs",
screenshot: "diffusion screenshot",
@@ -620,12 +619,12 @@ All of them talk to the running app, except \`fonts\`.
| \`open\` | \`diffusion open .\` | Launch the app with this project open. |
| \`context\` | \`diffusion context\` | Which project the app has open, where its playhead sits, its fonts, its background mask tracks. |
| \`capture\` | \`diffusion capture <id>\` | Render frames of a scene, as an export would, to labelled PNG contact sheets. |
-| \`probe\` | \`diffusion media probe <id\\|path>\` | Container and per-track metadata, without decoding. |
-| \`transcribe\` | \`diffusion media transcribe <id\\|path>\` | Timed speech transcript, word by word. |
-| \`grab\` | \`diffusion media grab <id\\|path>\` | Decode frames of a video to labelled PNG contact sheets. |
-| \`filmstrip\` | \`diffusion media filmstrip <id\\|path>\` | Thumbnail grid across a window of a video. |
-| \`waveform\` | \`diffusion media waveform <id\\|path>\` | Loudness over time, with the silences marked. |
-| \`listen\` | \`diffusion media listen <id\\|path>\` | Ask a multimodal model what is in an audio track. |
+| \`probe\` | \`diffusion probe <id\\|path>\` | Container and per-track metadata, without decoding. |
+| \`transcribe\` | \`diffusion transcribe <id\\|path>\` | Timed speech transcript, word by word. |
+| \`grab\` | \`diffusion grab <id\\|path>\` | Decode frames of a video to labelled PNG contact sheets. |
+| \`filmstrip\` | \`diffusion filmstrip <id\\|path>\` | Thumbnail grid across a window of a video. |
+| \`waveform\` | \`diffusion waveform <id\\|path>\` | Loudness over time, with the silences marked. |
+| \`listen\` | \`diffusion listen <id\\|path>\` | Ask a multimodal model what is in an audio track. |
| \`fonts\` | \`diffusion fonts\` | Google Fonts and local font families, valid as \`fontFamily\`. |
| \`logs\` | \`diffusion logs\` | Recent console output from the app. |
| \`screenshot\` | \`diffusion screenshot\` | The whole app window as a PNG. |
diff --git a/apps/web/src/dapi/handlers/media-filmstrip.ts b/apps/web/src/dapi/handlers/filmstrip.ts
similarity index 84%
rename from apps/web/src/dapi/handlers/media-filmstrip.ts
rename to apps/web/src/dapi/handlers/filmstrip.ts
index 7902b328..5c987c41 100644
--- a/apps/web/src/dapi/handlers/media-filmstrip.ts
+++ b/apps/web/src/dapi/handlers/filmstrip.ts
@@ -8,7 +8,7 @@ import { dataUrlToBytes } from "../lib/png";
import type { ToolHandler } from "../handler";
-export const mediaFilmstrip: ToolHandler<"media_filmstrip"> = async ({ path, start, end, scale }, ctx) => {
+export const filmstrip: ToolHandler<"filmstrip"> = async ({ path, start, end, scale }, ctx) => {
const asset = await resolveAsset(ctx, path);
const { dataUrl, ...rest } = await filmstripAsset(asset, { start, end, scale });
return { png: dataUrlToBytes(dataUrl), ...rest };
diff --git a/apps/web/src/dapi/handlers/media-grab.ts b/apps/web/src/dapi/handlers/grab.ts
similarity index 98%
rename from apps/web/src/dapi/handlers/media-grab.ts
rename to apps/web/src/dapi/handlers/grab.ts
index 7212d4f1..86158f3c 100644
--- a/apps/web/src/dapi/handlers/media-grab.ts
+++ b/apps/web/src/dapi/handlers/grab.ts
@@ -26,7 +26,7 @@ const FRAME_QUALITY_BUDGETS: Record<FrameQuality, number> = {
// Default cap on frames returned by auto selection when `count` is not given.
const AUTO_MAX_FRAMES = 30;
-export const mediaGrab: ToolHandler<"media_grab"> = async (args, ctx) => {
+export const grab: ToolHandler<"grab"> = async (args, ctx) => {
const { times, count, start, end, quality, auto, separate, perSheet } = args;
const asset = await resolveAsset(ctx, args.path);
requireAssetType(asset, ["VIDEO"], "a video");
diff --git a/apps/web/src/dapi/handlers/index.ts b/apps/web/src/dapi/handlers/index.ts
index 284d1ac4..fc03dc38 100644
--- a/apps/web/src/dapi/handlers/index.ts
+++ b/apps/web/src/dapi/handlers/index.ts
@@ -9,12 +9,12 @@ import { check } from "./check";
import { exportScene } from "./export";
import { fonts } from "./fonts";
import { screenshot } from "./screenshot";
-import { mediaProbe } from "./media-probe";
-import { mediaGrab } from "./media-grab";
-import { mediaTranscribe } from "./media-transcribe";
-import { mediaFilmstrip } from "./media-filmstrip";
-import { mediaWaveform } from "./media-waveform";
-import { mediaListen } from "./media-listen";
+import { probe } from "./probe";
+import { grab } from "./grab";
+import { transcribe } from "./transcribe";
+import { filmstrip } from "./filmstrip";
+import { waveform } from "./waveform";
+import { listen } from "./listen";
import { generate, job } from "./generate";
import type { Handlers } from "../handler";
@@ -28,12 +28,12 @@ export const handlers: Handlers = {
export: exportScene,
fonts,
screenshot,
- media_probe: mediaProbe,
- media_grab: mediaGrab,
- media_transcribe: mediaTranscribe,
- media_filmstrip: mediaFilmstrip,
- media_waveform: mediaWaveform,
- media_listen: mediaListen,
+ probe,
+ grab,
+ transcribe,
+ filmstrip,
+ waveform,
+ listen,
generate,
job,
};
diff --git a/apps/web/src/dapi/handlers/media-listen.ts b/apps/web/src/dapi/handlers/listen.ts
similarity index 94%
rename from apps/web/src/dapi/handlers/media-listen.ts
rename to apps/web/src/dapi/handlers/listen.ts
index fdf1e39c..6663ce2c 100644
--- a/apps/web/src/dapi/handlers/media-listen.ts
+++ b/apps/web/src/dapi/handlers/listen.ts
@@ -13,7 +13,7 @@ import { apiError, requireSignIn } from "../lib/remote";
import type { ToolHandler } from "../handler";
/** The asset's audio, or a window of it, analyzed on the API; times in the answer count from `start`, echoed with `end` when a window was given. */
-export const mediaListen: ToolHandler<"media_listen"> = async ({ path, prompt, start, end }, ctx) => {
+export const listen: ToolHandler<"listen"> = async ({ path, prompt, start, end }, ctx) => {
const asset = await resolveAsset(ctx, path);
requireAssetType(asset, ["AUDIO", "VIDEO"], "a video or audio asset");
diff --git a/apps/web/src/dapi/handlers/media-probe.ts b/apps/web/src/dapi/handlers/probe.ts
similarity index 96%
rename from apps/web/src/dapi/handlers/media-probe.ts
rename to apps/web/src/dapi/handlers/probe.ts
index 525ddff5..322f82b1 100644
--- a/apps/web/src/dapi/handlers/media-probe.ts
+++ b/apps/web/src/dapi/handlers/probe.ts
@@ -11,7 +11,7 @@ import type { ToolHandler } from "../handler";
const PROBE_SAMPLE_PACKETS = 200;
-export const mediaProbe: ToolHandler<"media_probe"> = async ({ path }, ctx) => {
+export const probe: ToolHandler<"probe"> = async ({ path }, ctx) => {
const asset = await resolveAsset(ctx, path);
const blob = await getAssetFile(asset);
const base = {
diff --git a/apps/web/src/dapi/handlers/media-transcribe.ts b/apps/web/src/dapi/handlers/transcribe.ts
similarity index 90%
rename from apps/web/src/dapi/handlers/media-transcribe.ts
rename to apps/web/src/dapi/handlers/transcribe.ts
index a47dd344..94a8e959 100644
--- a/apps/web/src/dapi/handlers/media-transcribe.ts
+++ b/apps/web/src/dapi/handlers/transcribe.ts
@@ -15,7 +15,7 @@ import type { ToolHandler } from "../handler";
const transcripts = new Map<string, Promise<Transcript>>();
/** The asset's audio, transcribed on the API; the main process writes the transcript to its file. */
-export const mediaTranscribe: ToolHandler<"media_transcribe"> = async ({ path }, ctx) => {
+export const transcribe: ToolHandler<"transcribe"> = async ({ path }, ctx) => {
const asset = await resolveAsset(ctx, path);
requireAssetType(asset, ["AUDIO", "VIDEO"], "a video or audio asset");
await requireSignIn();
@@ -24,7 +24,7 @@ export const mediaTranscribe: ToolHandler<"media_transcribe"> = async ({ path },
const key = await sha256(audio);
let transcript = transcripts.get(key);
if (!transcript) {
- transcript = transcribe(audio, ctx.signal);
+ transcript = transcribeOnApi(audio, ctx.signal);
transcripts.set(key, transcript);
transcript.catch(() => transcripts.delete(key));
}
@@ -37,7 +37,7 @@ export const mediaTranscribe: ToolHandler<"media_transcribe"> = async ({ path },
};
/** `audio` uploaded and transcribed; aborting `signal` cancels the job. */
-async function transcribe(audio: Blob, signal: AbortSignal): Promise<Transcript> {
+async function transcribeOnApi(audio: Blob, signal: AbortSignal): Promise<Transcript> {
try {
const job = await runJob({ model: "universal-3.5-pro", audio: await uploadFile(audio) }, () => {}, signal);
if (job.status !== "succeeded") throw new Error(jobFailure(job));
diff --git a/apps/web/src/dapi/handlers/media-waveform.ts b/apps/web/src/dapi/handlers/waveform.ts
similarity index 84%
rename from apps/web/src/dapi/handlers/media-waveform.ts
rename to apps/web/src/dapi/handlers/waveform.ts
index e15591c8..61c35698 100644
--- a/apps/web/src/dapi/handlers/media-waveform.ts
+++ b/apps/web/src/dapi/handlers/waveform.ts
@@ -8,7 +8,7 @@ import { dataUrlToBytes } from "../lib/png";
import type { ToolHandler } from "../handler";
-export const mediaWaveform: ToolHandler<"media_waveform"> = async ({ path, start, end, scale }, ctx) => {
+export const waveform: ToolHandler<"waveform"> = async ({ path, start, end, scale }, ctx) => {
const asset = await resolveAsset(ctx, path);
const { dataUrl, ...rest } = await waveformAsset(asset, { start, end, scale });
return { png: dataUrlToBytes(dataUrl), ...rest };
diff --git a/docs/guides/prompts/media-listen.md b/docs/guides/prompts/media-listen.md
index 75ce57cc..ac35f5fa 100644
--- a/docs/guides/prompts/media-listen.md
+++ b/docs/guides/prompts/media-listen.md
@@ -1,6 +1,6 @@
-# Writing prompts for `media_listen`
+# Writing prompts for `listen`
-[`media_listen`](../../reference/tools/media/listen.md) puts a multimodal model in front of an audio track and answers a prompt about it. A good prompt is specific about what you want and asks for timestamps so the answer lines up with the timeline. With no prompt it returns a general description; the examples below shape that into something more useful.
+[`listen`](../../reference/tools/media/listen.md) puts a multimodal model in front of an audio track and answers a prompt about it. A good prompt is specific about what you want and asks for timestamps so the answer lines up with the timeline. With no prompt it returns a general description; the examples below shape that into something more useful.
## Examples
When you refer to a specific moment inside a prompt, use `MM:SS` (e.g. `01:15` for one minute fifteen seconds).
diff --git a/docs/guides/walkthroughs/podcast-clip.md b/docs/guides/walkthroughs/podcast-clip.md
index 6c499dca..bcaf4c46 100644
--- a/docs/guides/walkthroughs/podcast-clip.md
+++ b/docs/guides/walkthroughs/podcast-clip.md
@@ -16,15 +16,15 @@ yt-dlp -x --audio-format m4a -o podcast.m4a <url>
## 3. Segment the audio
-A three-hour track is too long for one analysis pass, so split it into segments and analyze each. Segments are **windows**, not files: `media_listen` takes the range as `start`/`end`, and the timestamps it returns are relative to `start`. Pick one of three ways to choose the boundaries:
+A three-hour track is too long for one analysis pass, so split it into segments and analyze each. Segments are **windows**, not files: `listen` takes the range as `start`/`end`, and the timestamps it returns are relative to `start`. Pick one of three ways to choose the boundaries:
-1. **Waveform.** `media_waveform` renders loudness over the whole track and returns the silent spans as second ranges. Cut segments at those spans — the boundaries land between thoughts instead of inside them.
+1. **Waveform.** `waveform` renders loudness over the whole track and returns the silent spans as second ranges. Cut segments at those spans — the boundaries land between thoughts instead of inside them.
2. **Naive.** Fixed 15–30 minute segments. Nothing to parse, and good enough because the analysis reports its own timestamps; a moment straddling a boundary is the only loss.
3. **Source transcript.** Many platforms publish one (YouTube captions, show notes with chapter markers). Parse its timestamps and use chapters or topic shifts as boundaries — the cheapest option when it exists, since it needs no decoding at all.
## 4. Find the clip-worthy moments
-Run `media_listen` on each segment (`start` 15:00, `end` 45:00, and so on) with a prompt that spells out the criteria and demands timestamps:
+Run `listen` on each segment (`start` 15:00, `end` 45:00, and so on) with a prompt that spells out the criteria and demands timestamps:
```text
This is a podcast. Find the 3 most clip-worthy self-contained moments in this segment for a vertical social short. Criteria: a complete thought, punchline, or surprising revelation that works with NO prior context; roughly 25-35 seconds long; opens on a strong hook line and lands on a clean button. For each give: exact start and end timestamp (MM:SS, relative to this segment), who is speaking, a one-line summary of what is said, and a hook-strength rating 1-10. Be strict about self-containment.
@@ -36,8 +36,8 @@ Asking for a rating and a one-line summary is what makes the candidates comparab
The analysis gives you seconds; a clip needs the frame. Tighten both ends against the real audio:
-- `media_transcribe` writes word-level start/end times to a JSON file (use ffmpeg to shorten); search it for the hook line. Put the in-point on the first word of the hook line and the out-point after the last word of the button.
-- `media_waveform` with `start`/`end` around the cut shows the breaths around those words, so you can open the in-point a beat early and let the out-point land on the silence after the line instead of clipping its tail.
+- `transcribe` writes word-level start/end times to a JSON file (use ffmpeg to shorten); search it for the hook line. Put the in-point on the first word of the hook line and the out-point after the last word of the button.
+- `waveform` with `start`/`end` around the cut shows the breaths around those words, so you can open the in-point a beat early and let the out-point land on the silence after the line instead of clipping its tail.
## 6. Download the segment and lay it out
@@ -48,7 +48,7 @@ yt-dlp -f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]/bv*+ba/b" --merge-output-format m
--download-sections "*41:08-41:48" -o assets/a-roll/clip-raw.mp4 <url>
```
-`media_probe` on the download gives the source dimensions (a podcast is almost always 1920×1080) and confirms where the padded range actually starts, since a keyframe-aligned download can begin slightly early.
+`probe` on the download gives the source dimensions (a podcast is almost always 1920×1080) and confirms where the padded range actually starts, since a keyframe-aligned download can begin slightly early.
Give the node the **source's own aspect ratio**, scaled to the scene height, rather than the scene's box: the node is then wider than the scene, and the scene crops it. That geometry is what makes the framing in the next step possible.
diff --git a/docs/guides/walkthroughs/talking-head.md b/docs/guides/walkthroughs/talking-head.md
index f1e30648..5b3c73f3 100644
--- a/docs/guides/walkthroughs/talking-head.md
+++ b/docs/guides/walkthroughs/talking-head.md
@@ -10,9 +10,9 @@ Write the brief first. It records the source files, the target duration (or "kee
The transcript is the spine, so transcribe everything with speech.
-- **Transcribe every video and any external audio.** `media_transcribe` writes word-level start/end times to a JSON file — the times you cut on. Run it on each camera take and on any separate recording (a lav or interface track you will sync to).
-- **Render the waveform to read the gaps.** `media_waveform` marks silence in red and returns it as second ranges. A transcript marks only speech, so a gap in it can be silence *or* something to keep — a laugh, a breath. The waveform tells them apart: red is dead air to cut; a gap that still shows signal is performance to keep.
-- **Resolve double takes visually.** The transcript shows repeated lines; usually the **last** take is the keeper. When takes read equally well, `media_grab` the first word of each and keep the sharpest — motion blur means a bad take.
+- **Transcribe every video and any external audio.** `transcribe` writes word-level start/end times to a JSON file — the times you cut on. Run it on each camera take and on any separate recording (a lav or interface track you will sync to).
+- **Render the waveform to read the gaps.** `waveform` marks silence in red and returns it as second ranges. A transcript marks only speech, so a gap in it can be silence *or* something to keep — a laugh, a breath. The waveform tells them apart: red is dead air to cut; a gap that still shows signal is performance to keep.
+- **Resolve double takes visually.** The transcript shows repeated lines; usually the **last** take is the keeper. When takes read equally well, `grab` the first word of each and keep the sharpest — motion blur means a bad take.
## 3. Lay out the A-roll
diff --git a/docs/reference/tools/README.md b/docs/reference/tools/README.md
index 75a77b37..23a84079 100644
--- a/docs/reference/tools/README.md
+++ b/docs/reference/tools/README.md
@@ -13,14 +13,14 @@ The JSX code syntax specified in [jsx/](../jsx/README.md) is **pseudo-SVG**, mir
A tool is named as MCP lists it, and the CLI spelling follows from the name:
-- `_` in a tool name is a space on the command line: `media_grab` is `diffusion media grab`. The `media` group is also `m`.
+- The tool name is the command: `grab` is `diffusion grab`.
- A tool's first field is the positional argument: `capture`'s `id` is `diffusion capture <id>`.
- Every other field is an option in kebab-case: `perSheet` is `--per-sheet`, `separate` is `--separate`. Short forms are listed on each page.
- Times are written the same way everywhere: seconds (`1.5`), frames at the project's rate (`45f`), or a clock string (`1:30`, `00:01:30`). Times in **results** are plain seconds.
## Results
-Every tool returns one JSON object, its *structured content*. Over MCP that is the result's `structuredContent`, repeated as a text block for clients that ignore structured content; the CLI prints it to stdout, unchanged. Tools that render images (`capture`, `media_grab`, `media_filmstrip`, `media_waveform`, `screenshot`) write PNGs to disk and return their paths; over MCP a result of at most four images, none over a megabyte, also carries them inline as image content, so a contact sheet arrives in context without opening anything.
+Every tool returns one JSON object, its *structured content*. Over MCP that is the result's `structuredContent`, repeated as a text block for clients that ignore structured content; the CLI prints it to stdout, unchanged. Tools that render images (`capture`, `grab`, `filmstrip`, `waveform`, `screenshot`) write PNGs to disk and return their paths; over MCP a result of at most four images, none over a megabyte, also carries them inline as image content, so a contact sheet arrives in context without opening anything.
## Errors
@@ -37,12 +37,12 @@ Every tool runs inside the app, so the app has to be running. Over MCP that is a
| [`capture`](./capture.md) | `diffusion capture` | Capture frames |
| [`check`](./check.md) | `diffusion check` | Check structure |
| [`export`](./export.md) | `diffusion export` | Export scene |
-| [`media_probe`](./media/probe.md) | `diffusion media probe` | Probe media |
-| [`media_grab`](./media/grab.md) | `diffusion media grab` | Grab frames |
-| [`media_transcribe`](./media/transcribe.md) | `diffusion media transcribe` | Transcribe speech |
-| [`media_filmstrip`](./media/filmstrip.md) | `diffusion media filmstrip` | Filmstrip preview |
-| [`media_waveform`](./media/waveform.md) | `diffusion media waveform` | Waveform preview |
-| [`media_listen`](./media/listen.md) | `diffusion media listen` | Listen to audio |
+| [`probe`](./media/probe.md) | `diffusion probe` | Probe media |
+| [`grab`](./media/grab.md) | `diffusion grab` | Grab frames |
+| [`transcribe`](./media/transcribe.md) | `diffusion transcribe` | Transcribe speech |
+| [`filmstrip`](./media/filmstrip.md) | `diffusion filmstrip` | Filmstrip preview |
+| [`waveform`](./media/waveform.md) | `diffusion waveform` | Waveform preview |
+| [`listen`](./media/listen.md) | `diffusion listen` | Listen to audio |
| [`generate`](./generate.md) | `diffusion generate` | Generate media |
| [`job`](./job.md) | `diffusion job` | Generation job |
| [`logs`](./logs.md) | `diffusion logs` | App logs |
@@ -54,7 +54,7 @@ Every tool runs inside the app, so the app has to be running. Over MCP that is a
How the surface is divided:
- **The project loop.** [`open`](./open.md) a folder, edit its JSX, [`context`](./context.md) for what the source cannot say, [`capture`](./capture.md) and [`check`](./check.md) to verify, [`export`](./export.md) when asked.
-- **Media inspection** (`media_*`): a file by path, without adding it to the project. Absolute paths and URLs work with or without an open project; library paths (`b-roll/clip.mp4`) need one.
+- **Media inspection** ([`probe`](./media/probe.md), [`grab`](./media/grab.md), [`transcribe`](./media/transcribe.md), [`filmstrip`](./media/filmstrip.md), [`waveform`](./media/waveform.md), [`listen`](./media/listen.md)): a file by path, without adding it to the project. Absolute paths and URLs work with or without an open project; library paths (`b-roll/clip.mp4`) need one.
- **Generation.** [`generate`](./generate.md) runs a model of the Diffusion Studio API (images, video, music, speech, and tools such as upscaling) on the account's credits, or a model on this machine for free (with `estimate`, it only prices the request) — [`sam-2.1`](./media/segment.md) segments and tracks an object in footage, making the mask file a `<mask src>` names; poll [`job`](./job.md) until it has ended, and its files are saved into the library or a directory. The models and their fields are in [models.md](../models.md).
- **What a declaration may name.** [`fonts`](./fonts.md).
- **The app and the machine.** [`logs`](./logs.md), [`screenshot`](./screenshot.md), [`window`](./window.md), [`report`](./report.md).
diff --git a/docs/reference/tools/capture.md b/docs/reference/tools/capture.md
index e164db3e..c1473b4c 100644
--- a/docs/reference/tools/capture.md
+++ b/docs/reference/tools/capture.md
@@ -1,6 +1,6 @@
# capture
-Render single frames of a scene to PNGs — each frame is the frame an export of that scene would encode, drawn offscreen at the scene's own size. By default the positions are merged into contact sheets: up to 12 per image, each cell labelled with its timecode (`08s10f`, zero segments dropped) and rendered as large as fits, so a few positions arrive as one high-resolution picture instead of a directory to open one by one (separate: true writes a PNG per position, at 720p height). The tool for checking composition ("what plays at time T": layout, overlaps, text, timing) and for verifying frames before an export. Scenes only — a single element renders inside its scene, so capture the scene at the times it plays. For a video asset's own full-resolution pixels use media_grab.
+Render single frames of a scene to PNGs — each frame is the frame an export of that scene would encode, drawn offscreen at the scene's own size. By default the positions are merged into contact sheets: up to 12 per image, each cell labelled with its timecode (`08s10f`, zero segments dropped) and rendered as large as fits, so a few positions arrive as one high-resolution picture instead of a directory to open one by one (separate: true writes a PNG per position, at 720p height). The tool for checking composition ("what plays at time T": layout, overlaps, text, timing) and for verifying frames before an export. Scenes only — a single element renders inside its scene, so capture the scene at the times it plays. For a video asset's own full-resolution pixels use grab.
| | |
| --- | --- |
@@ -21,7 +21,7 @@ Render single frames of a scene to PNGs — each frame is the frame an export of
Each frame is **the frame an export of that scene would encode**: the scene is re-rendered from a fresh mount at its own size, position `0` is the workarea's first frame, and the requested positions are evaluated in timeline order, forward only — the way an export advances — so a composition whose look depends on having played (an `<html>` node's own animation state, for instance) captures exactly as it exports. A position past the workarea's end is not an error: the scene keeps playing past it, and the frame is what plays there — just not one an export would include.
-Scenes only: a single element renders inside its scene, so capture the scene at the times the element plays. To grab a video asset's own pixels instead of a composited frame, use [`media_grab`](./media/grab.md).
+Scenes only: a single element renders inside its scene, so capture the scene at the times the element plays. To grab a video asset's own pixels instead of a composited frame, use [`grab`](./media/grab.md).
With `separate`, each position is rendered at 720p height and named after its timecode (e.g. `01s12f.png`). Without an `output` directory the images land in a fresh `dapi-capture-*` directory under the system temp directory, so runs never overwrite each other. Writing into the same directory twice overwrites images whose name matches; with `separate`, requested times that land on the same frame share one file.
diff --git a/docs/reference/tools/generate.md b/docs/reference/tools/generate.md
index 42c0b7c9..bb01267a 100644
--- a/docs/reference/tools/generate.md
+++ b/docs/reference/tools/generate.md
@@ -63,7 +63,7 @@ Files (`{ "path": … }`) are uploaded first, as for a job, since a tool such as
- **Several files are numbered.** A job that makes more than one (an image `count` above 1) saves `fox-1.png`, `fox-2.png`, …; a single file is saved at the path itself.
- **A library path** (`b-roll/fox.png`, needs an open project) puts the file into the project's library, under `assets/`, named by `src` like any other asset, and notes the job it came from, as the app's own generations do. A file the project made already at that path is replaced, and everything naming it shows the new one; a file linked from elsewhere on disk is never written over, and the result gets a free name next to it instead.
- **An absolute path** saves the file there, replacing a file already there.
-- **Left out**, the file goes into the open project's library at its root under the API's name (`red-fox-at-dawn.png`, numbered if taken), or where a local model files its own (`sam-2.1`: `masks/<video>/Tracking <n>.mask`); with no project open, into a fresh directory of the job's own under the system temp dir, as [`media_transcribe`](./media/transcribe.md) and the other inspection tools do.
+- **Left out**, the file goes into the open project's library at its root under the API's name (`red-fox-at-dawn.png`, numbered if taken), or where a local model files its own (`sam-2.1`: `masks/<video>/Tracking <n>.mask`); with no project open, into a fresh directory of the job's own under the system temp dir, as [`transcribe`](./media/transcribe.md) and the other inspection tools do.
The files are saved by the first [`job`](./job.md) call that sees the job succeed (a local model's library files as soon as it does); later calls return the same paths. A file an API model made that went into the library records the job in `assets.yml` (its `job` field), so what made it can be looked up later.
diff --git a/docs/reference/tools/media/filmstrip.md b/docs/reference/tools/media/filmstrip.md
index 66360bab..3c31ed67 100644
--- a/docs/reference/tools/media/filmstrip.md
+++ b/docs/reference/tools/media/filmstrip.md
@@ -1,12 +1,12 @@
-# media_filmstrip
+# filmstrip
-Render a grid of thumbnails sampled across the timeline to a PNG (local render, no credits), each row stamped with an HH:MM:SS:FF ruler. A fast, token-efficient video track preview; narrow the window to zoom into a region of interest. Video only (use media_waveform for audio).
+Render a grid of thumbnails sampled across the timeline to a PNG (local render, no credits), each row stamped with an HH:MM:SS:FF ruler. A fast, token-efficient video track preview; narrow the window to zoom into a region of interest. Video only (use waveform for audio).
| | |
| --- | --- |
-| MCP tool | `media_filmstrip` |
-| CLI | `diffusion media filmstrip <path> [options]` |
-| CLI aliases | `diffusion media film` |
+| MCP tool | `filmstrip` |
+| CLI | `diffusion filmstrip <path> [options]` |
+| CLI aliases | `diffusion film` |
## Input
@@ -18,7 +18,7 @@ Render a grid of thumbnails sampled across the timeline to a PNG (local render,
| `output` | `string` | `-o, --output <path>` | absolute path to write the PNG to (default: a fresh file under the system temp dir) |
| `scale` | `number` | `-x, --scale <factor>` | scale factor for the thumbnails; smaller fits more rows and columns, larger fits fewer (default: 1) |
-Frames are sampled at even intervals across the window. An `end` past the asset's duration is clamped to it; a `start` at or past the end is an error, since the window would be empty. Tick labels use `HH:MM:SS:FF` timecode (hours, minutes, seconds, frame within the second) at every zoom level, so labels stay comparable regardless of the window's span; frames count against the video's frame rate. Video only; use [`media_waveform`](./waveform.md) to inspect the audio track.
+Frames are sampled at even intervals across the window. An `end` past the asset's duration is clamped to it; a `start` at or past the end is an error, since the window would be empty. Tick labels use `HH:MM:SS:FF` timecode (hours, minutes, seconds, frame within the second) at every zoom level, so labels stay comparable regardless of the window's span; frames count against the video's frame rate. Video only; use [`waveform`](./waveform.md) to inspect the audio track.
The overall canvas size stays fixed, so a smaller `scale` (clamped to `0.25`–`4`) fits **more rows and columns** — a denser grid sampling more moments — and a larger one fits fewer but shows more detail each. Without `output` the PNG lands in a fresh file under the system temp directory; an `output` naming an existing directory gets that fresh file inside it.
diff --git a/docs/reference/tools/media/grab.md b/docs/reference/tools/media/grab.md
index be8677cc..cefc5093 100644
--- a/docs/reference/tools/media/grab.md
+++ b/docs/reference/tools/media/grab.md
@@ -1,12 +1,12 @@
-# media_grab
+# grab
-Decode frames of a video file and write them as PNGs (local render, no credits). By default the frames are merged into contact sheets: up to 12 per image, each cell labelled with its timecode (`08s10f`, zero segments dropped) and drawn as large as fits, so a handful of frames arrives as one high-resolution picture instead of a directory to open one by one (separate: true writes a PNG per frame). Grabs the asset's own pixels, unlike capture which renders the composited node. The recommended tool for understanding a video at the frame level; past ~12 frames prefer media_filmstrip.
+Decode frames of a video file and write them as PNGs (local render, no credits). By default the frames are merged into contact sheets: up to 12 per image, each cell labelled with its timecode (`08s10f`, zero segments dropped) and drawn as large as fits, so a handful of frames arrives as one high-resolution picture instead of a directory to open one by one (separate: true writes a PNG per frame). Grabs the asset's own pixels, unlike capture which renders the composited node. The recommended tool for understanding a video at the frame level; past ~12 frames prefer filmstrip.
| | |
| --- | --- |
-| MCP tool | `media_grab` |
-| CLI | `diffusion media grab <path> [options]` |
-| CLI aliases | `diffusion media sample` |
+| MCP tool | `grab` |
+| CLI | `diffusion grab <path> [options]` |
+| CLI aliases | `diffusion sample` |
## Input
@@ -32,7 +32,7 @@ Three ways to say which frames, mutually exclusive:
- `count`: that many frames evenly spaced across the clip, or across the `start`/`end` window, at a fixed interval of `window / count`, starting at the window start.
- `auto`: a scan at 2 fps that keeps a frame each time the picture settles into a new visual state, dropping near-duplicates and waiting out transitions so picks stay sharp. Returns at most `count` frames (default cap 30); static footage such as a screen recording returns far fewer. Needs WebGPU.
-`start` and `end` only apply with `count` or `auto`. An `end` past the asset's duration is clamped to it; a `start` at or past the end is an error, since the window would be empty. Like [`capture`](../capture.md), but this grabs the asset's own pixels; `capture` renders the composited node. Renders locally; no credits. Past ~12 frames, [`media_filmstrip`](./filmstrip.md) is the cheaper way to scan a clip.
+`start` and `end` only apply with `count` or `auto`. An `end` past the asset's duration is clamped to it; a `start` at or past the end is an error, since the window would be empty. Like [`capture`](../capture.md), but this grabs the asset's own pixels; `capture` renders the composited node. Renders locally; no credits. Past ~12 frames, [`filmstrip`](./filmstrip.md) is the cheaper way to scan a clip.
`quality` is a pixel budget rather than a box: a frame is scaled down, aspect ratio kept, until it holds no more pixels than the preset's square (`small` is 384², so a 16:9 frame becomes 512x288; `medium` 768² gives 1024x576; `large` 1536² gives 2048x1152), and never enlarged, so `large` on 1080p footage is the native 1920x1080.
@@ -40,7 +40,7 @@ With `separate`, frames keep their own resolution and alpha, and each file is na
## Timecodes
-Cell labels, the `timecode` field, and the filenames all use the same stamp, which drops its zero segments: `08s10f` is 8 seconds and 10 frames, `01m05s` is 65 seconds, and the first frame is `0f`. Each segment carries its unit, so nothing is ambiguous once the empty ones are gone. (The rulers [`media_filmstrip`](./filmstrip.md) and [`media_waveform`](./waveform.md) draw stay on fixed-width `HH:MM:SS:FF`, so their ticks line up.)
+Cell labels, the `timecode` field, and the filenames all use the same stamp, which drops its zero segments: `08s10f` is 8 seconds and 10 frames, `01m05s` is 65 seconds, and the first frame is `0f`. Each segment carries its unit, so nothing is ambiguous once the empty ones are gone. (The rulers [`filmstrip`](./filmstrip.md) and [`waveform`](./waveform.md) draw stay on fixed-width `HH:MM:SS:FF`, so their ticks line up.)
## Layout
diff --git a/docs/reference/tools/media/listen.md b/docs/reference/tools/media/listen.md
index 6277a883..48b8cc36 100644
--- a/docs/reference/tools/media/listen.md
+++ b/docs/reference/tools/media/listen.md
@@ -1,11 +1,11 @@
-# media_listen
+# listen
Prompt a multimodal model for a semantic analysis of an audio track and return its answer. Shines on audio semantics (the name of the music playing, who is speaking, the spoken content with second-granularity timestamps). Accepts an audio file or a video; of a video only the audio track is analyzed.
| | |
| --- | --- |
-| MCP tool | `media_listen` |
-| CLI | `diffusion media listen <path> [options]` |
+| MCP tool | `listen` |
+| CLI | `diffusion listen <path> [options]` |
## Input
diff --git a/docs/reference/tools/media/probe.md b/docs/reference/tools/media/probe.md
index 9c22744e..c027afc2 100644
--- a/docs/reference/tools/media/probe.md
+++ b/docs/reference/tools/media/probe.md
@@ -1,11 +1,11 @@
-# media_probe
+# probe
Read the container and per-track technical metadata of a media file (local read, no credits): container format, duration, tags, and each track's codec params, without decoding. Commonly useful for a quick technical read, e.g. checking codec compatibility or duration before cutting. Packet stats (fps, bitrate) are estimated from a leading sample; images and transcripts report file-level info only.
| | |
| --- | --- |
-| MCP tool | `media_probe` |
-| CLI | `diffusion media probe <path>` |
+| MCP tool | `probe` |
+| CLI | `diffusion probe <path>` |
## Input
diff --git a/docs/reference/tools/media/segment.md b/docs/reference/tools/media/segment.md
index 92965ba8..f4b5d3b8 100644
--- a/docs/reference/tools/media/segment.md
+++ b/docs/reference/tools/media/segment.md
@@ -32,7 +32,7 @@ SAM 2.1 segments an object in a video and tracks it through the footage, making
## Prompting
-A prompt needs at least one point on the object or a box around it; `exclude` points only take things away. Coordinates are fractions of the frame as it displays (rotation applied, x right, y down), the same space [`media_grab`](./grab.md) frames are drawn in. A box is usually the most reliable first prompt: draw it around the object as seen in a grabbed frame. When the mask spills onto something next to the object, add an `exclude` point on that thing; when it misses part of the object, add a point on the missing part.
+A prompt needs at least one point on the object or a box around it; `exclude` points only take things away. Coordinates are fractions of the frame as it displays (rotation applied, x right, y down), the same space [`grab`](./grab.md) frames are drawn in. A box is usually the most reliable first prompt: draw it around the object as seen in a grabbed frame. When the mask spills onto something next to the object, add an `exclude` point on that thing; when it misses part of the object, add a point on the missing part.
With `preview: true`, `generate` segments that frame alone and answers with the job already succeeded: no file, and as `details.image` the frame with everything outside the mask washed magenta, its edge outlined, the prompts drawn (`+n` on the object, `−n` off it, the box dashed) and a 0.1 grid to read coordinates off. The first preview of a frame encodes it, which takes a moment; further previews on the same frame, file and size run the mask decoder alone and return at once. A preview makes no mask, so it takes neither a span nor an `output`.
@@ -76,7 +76,7 @@ details: {
}
```
-`lost` covers occlusion and the object leaving the frame as well as a track that slipped; look at those moments with [`media_grab`](./grab.md) before trusting or re-prompting them.
+`lost` covers occlusion and the object leaving the frame as well as a track that slipped; look at those moments with [`grab`](./grab.md) before trusting or re-prompting them.
## Errors
diff --git a/docs/reference/tools/media/transcribe.md b/docs/reference/tools/media/transcribe.md
index bf82478f..8eeafcdf 100644
--- a/docs/reference/tools/media/transcribe.md
+++ b/docs/reference/tools/media/transcribe.md
@@ -1,11 +1,11 @@
-# media_transcribe
+# transcribe
Transcribe the speech in a video or audio file and write the timed transcript to a JSON file, with word-level start/end times in seconds and speaker labels; returns the file's path and its segment and word counts. Commonly useful for footage with speakers (talking head, interview), where the word times let you cut on a line. A transcript marks only speech; the gaps are not necessarily silent (music, score, applause).
| | |
| --- | --- |
-| MCP tool | `media_transcribe` |
-| CLI | `diffusion media transcribe <path>` |
+| MCP tool | `transcribe` |
+| CLI | `diffusion transcribe <path>` |
## Input
diff --git a/docs/reference/tools/media/waveform.md b/docs/reference/tools/media/waveform.md
index 9f0374e3..d7a9d28f 100644
--- a/docs/reference/tools/media/waveform.md
+++ b/docs/reference/tools/media/waveform.md
@@ -1,12 +1,12 @@
-# media_waveform
+# waveform
Render the audio track of a video or audio file as a waveform PNG (local render, no credits) with a timestamp ruler: loudness over time, with silent stretches highlighted in red. A fast, token-efficient audio track preview; the silent spans are also returned as second ranges.
| | |
| --- | --- |
-| MCP tool | `media_waveform` |
-| CLI | `diffusion media waveform <path> [options]` |
-| CLI aliases | `diffusion media wave` |
+| MCP tool | `waveform` |
+| CLI | `diffusion waveform <path> [options]` |
+| CLI aliases | `diffusion wave` |
## Input
diff --git a/docs/skills/editor.md b/docs/skills/editor.md
index 372f4d6f..ac68433b 100644
--- a/docs/skills/editor.md
+++ b/docs/skills/editor.md
@@ -2,11 +2,11 @@
How to understand source material before editing it. Inspect only the modalities the decision turns on — speech, action, music, graphics, or atmosphere may lead, so there is no fixed priority. Sample the picture against what the audio tells you.
-- **Always probe first.** `media_probe` reports the container and its tracks, telling you up front whether the file has a video track, an audio track, or both. Everything after branches on that.
-- **Get the lay of the land.** Render a `media_waveform` (audio) and a `media_filmstrip` (video) for a fast, cheap overview of where the loud and quiet stretches fall, and where the visual scene changes are. A filmstrip shows coarse structure and scene state, not crop, framing, readability, or an exact cut frame.
-- **Listen to the audio.** Run `media_listen` with a prompt tailored to the context (what you actually need to know), and explicitly ask the model to include timestamps in its answer. See [media-listen.md](../guides/prompts/media-listen.md) for prompt patterns.
-- **Transcribe speech.** For speech, `media_transcribe` writes the full transcript with word-level start/end times to a JSON file and returns its path — search the file for the line you need and read its times from there, rather than loading it whole.
-- **Sample the video against the audio.** Use `media_grab` to pull frames. When the audio has already pointed you at specific moments, feed those timestamps straight in from the transcript or listen output as `times` (e.g. `00:32`, `00:45`). When you need a visual pass without such cues, reach for `auto`: it scans the footage and keeps only the frames where the picture settles into a new visual state, dropping near-duplicates.
+- **Always probe first.** `probe` reports the container and its tracks, telling you up front whether the file has a video track, an audio track, or both. Everything after branches on that.
+- **Get the lay of the land.** Render a `waveform` (audio) and a `filmstrip` (video) for a fast, cheap overview of where the loud and quiet stretches fall, and where the visual scene changes are. A filmstrip shows coarse structure and scene state, not crop, framing, readability, or an exact cut frame.
+- **Listen to the audio.** Run `listen` with a prompt tailored to the context (what you actually need to know), and explicitly ask the model to include timestamps in its answer. See [media-listen.md](../guides/prompts/media-listen.md) for prompt patterns.
+- **Transcribe speech.** For speech, `transcribe` writes the full transcript with word-level start/end times to a JSON file and returns its path — search the file for the line you need and read its times from there, rather than loading it whole.
+- **Sample the video against the audio.** Use `grab` to pull frames. When the audio has already pointed you at specific moments, feed those timestamps straight in from the transcript or listen output as `times` (e.g. `00:32`, `00:45`). When you need a visual pass without such cues, reach for `auto`: it scans the footage and keeps only the frames where the picture settles into a new visual state, dropping near-duplicates.
# The editing loop
@@ -74,4 +74,4 @@ Read worked example(s) that match your context.
## Prompts
-- [Writing prompts for `media_listen`](../guides/prompts/media-listen.md)
+- [Writing prompts for `listen`](../guides/prompts/media-listen.md)
diff --git a/docs/skills/watch.md b/docs/skills/watch.md
index c83f6ab3..1fb63d20 100644
--- a/docs/skills/watch.md
+++ b/docs/skills/watch.md
@@ -2,22 +2,22 @@
How to understand source material. Inspect only the modalities the question turns on — speech, action, music, graphics, or atmosphere may lead, so there is no fixed priority. Sample the picture against what the audio tells you.
-- **Always probe first.** `media_probe` reports the container and its tracks, telling you up front whether the file has a video track, an audio track, or both. Everything after branches on that.
-- **Get the lay of the land.** Render a `media_waveform` (audio) and a `media_filmstrip` (video) for a fast, cheap overview of where the loud and quiet stretches fall, and where the visual scene changes are. A filmstrip shows coarse structure and scene state, not crop, framing, readability, or an exact cut frame.
-- **Listen to the audio.** Run `media_listen` with a prompt tailored to the context (what you actually need to know), and explicitly ask the model to include timestamps in its answer. See [media-listen.md](../guides/prompts/media-listen.md) for prompt patterns.
-- **Transcribe speech.** For speech, `media_transcribe` writes the full transcript with word-level start/end times to a JSON file and returns its path — search the file for the line you need and read its times from there, rather than loading it whole.
-- **Sample the video against the audio.** Use `media_grab` to pull frames. When the audio has already pointed you at specific moments, feed those timestamps straight in from the transcript or listen output as `times` (e.g. `00:32`, `00:45`). When you need a visual pass without such cues, reach for `auto`: it scans the footage and keeps only the frames where the picture settles into a new visual state, dropping near-duplicates.
+- **Always probe first.** `probe` reports the container and its tracks, telling you up front whether the file has a video track, an audio track, or both. Everything after branches on that.
+- **Get the lay of the land.** Render a `waveform` (audio) and a `filmstrip` (video) for a fast, cheap overview of where the loud and quiet stretches fall, and where the visual scene changes are. A filmstrip shows coarse structure and scene state, not crop, framing, readability, or an exact cut frame.
+- **Listen to the audio.** Run `listen` with a prompt tailored to the context (what you actually need to know), and explicitly ask the model to include timestamps in its answer. See [media-listen.md](../guides/prompts/media-listen.md) for prompt patterns.
+- **Transcribe speech.** For speech, `transcribe` writes the full transcript with word-level start/end times to a JSON file and returns its path — search the file for the line you need and read its times from there, rather than loading it whole.
+- **Sample the video against the audio.** Use `grab` to pull frames. When the audio has already pointed you at specific moments, feed those timestamps straight in from the transcript or listen output as `times` (e.g. `00:32`, `00:45`). When you need a visual pass without such cues, reach for `auto`: it scans the footage and keeps only the frames where the picture settles into a new visual state, dropping near-duplicates.
-The `media_*` tools take a path or a direct media URL. Footage behind a page URL (YouTube, TikTok, Instagram, …) is downloaded first with `yt-dlp` from a shell — audio only (`yt-dlp -x --audio-format m4a -o talk.m4a <url>`) unless the question needs frames; see the [tool reference](https://github.com/yt-dlp/yt-dlp).
+The media tools (`probe`, `grab`, `transcribe`, `filmstrip`, `waveform`, `listen`) take a path or a direct media URL. Footage behind a page URL (YouTube, TikTok, Instagram, …) is downloaded first with `yt-dlp` from a shell — audio only (`yt-dlp -x --audio-format m4a -o talk.m4a <url>`) unless the question needs frames; see the [tool reference](https://github.com/yt-dlp/yt-dlp).
# Matching depth to the question
Read only as much of the footage as the answer requires — each pass costs time.
-- A duration or format question ends at `media_probe`.
+- A duration or format question ends at `probe`.
- "Where is the quiet part" or "how is it paced" is usually answered by the waveform and filmstrip alone.
-- A question about what was said resolves fastest through `media_transcribe`; search the transcript file for the passage and quote it with its times.
-- Questions about non-speech audio (music, tone, sound events, speaker identity) are what `media_listen` is for.
+- A question about what was said resolves fastest through `transcribe`; search the transcript file for the passage and quote it with its times.
+- Questions about non-speech audio (music, tone, sound events, speaker identity) are what `listen` is for.
- Only questions about what is *seen* need frames — and the audio pass usually tells you which moments to grab, so grab those instead of scanning blind.
- For open-ended questions ("summarize this", "what happens here"), combine passes: structure from waveform + filmstrip, content from transcript or listen, then frames at the salient moments to confirm what the picture shows.
diff --git a/packages/dapi/README.md b/packages/dapi/README.md
index dc1ece32..0a67a253 100644
--- a/packages/dapi/README.md
+++ b/packages/dapi/README.md
@@ -6,8 +6,8 @@ The tool catalog behind `dapi`: one entry per tool with its name, description, a
import { catalog, toolByName, Time } from "@diffusionstudio/dapi";
import type { ToolArgs, ToolResult } from "@diffusionstudio/dapi";
-const grab = toolByName("media_grab");
-const args: ToolArgs<"media_grab"> = grab.input.parse({ path: "/clip.mp4", times: ["45f", "1:10"] });
+const grab = toolByName("grab");
+const args: ToolArgs<"grab"> = grab.input.parse({ path: "/clip.mp4", times: ["45f", "1:10"] });
```
The package is free of Node built-ins so the renderer can import it. The MCP URL the app serves on (`MCP_URL`) is exported too, for the CLI and the docs the app writes.
diff --git a/packages/dapi/src/catalog.test.ts b/packages/dapi/src/catalog.test.ts
index 75749849..b0fb7294 100644
--- a/packages/dapi/src/catalog.test.ts
+++ b/packages/dapi/src/catalog.test.ts
@@ -50,7 +50,7 @@ describe("catalog", () => {
expect(toolByName("fonts").environment).toBe("renderer");
expect(toolByName("logs").environment).toBe("main");
expect(toolByName("window").environment).toBe("main");
- expect(isToolName("media_grab")).toBe(true);
+ expect(isToolName("grab")).toBe(true);
expect(isToolName("media.frame")).toBe(false);
});
});