Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
refactor(video-cli): remove hypit speak; spoken A-roll is a graph output
kashorin0192ef03d28a
diff --git a/packages/video-cli/README.md b/packages/video-cli/README.mdindex 93cd2719..cf9f65cd 100644--- a/packages/video-cli/README.md+++ b/packages/video-cli/README.md@@ -36,17 +36,18 @@ file the caller chose:```bashhypit observe <media…> --instruction <text|file> --prompt <text|file> --to notes/frame.mdhypit transcribe reference.mp4 --to notes/reference.transcript.json [--language en|zh|es]-hypit speak --text line.txt --voice "warm, unhurried" --to voice/intro.mp3hypit measure main.svml --segment hook --language en --pace normal --min 4 --max 15 --rounding round```-`observe` uses the Profile's Gemini Endpoint, `transcribe` its `whisperx-alignment` Endpoint (after-extracting 16 kHz mono speech audio with ffmpeg), `speak` its MiMo VoiceDesign Endpoint and reports-the audio's duration so the author can write it as a literal. `measure` counts a Segment's pronunciation-units at a delivery policy and prints the seconds to write as the literal `duration`; it opens no-Profile and spends nothing. `--runtime <profile>` names the Profile;-otherwise the project's `hypit runtime use` selection is read. Slow paid generation — pictures and-clips — is not a creation-time tool: declare it in the Source and go through `plan` and `build`.+`observe` uses the Profile's Gemini Endpoint and `transcribe` its `whisperx-alignment` Endpoint (after+extracting 16 kHz mono speech audio with ffmpeg). `measure` counts a Segment's pronunciation units at a+delivery policy and prints the seconds to write as the literal `duration`; it opens no Profile and+spends nothing. `--runtime <profile>` names the Profile; otherwise the project's `hypit runtime use`+selection is read. Anything the Author Graph declares as an output is a Build, however quickly it+comes back: pictures, clips and the spoken A-roll (`@hypit/mimo-tts`) carry the identity of the Source+that produced them, so they are declared in the Source and go through `plan` and `build`. To hear a+voice or learn a passage's real length before authoring the rest, build a Run whose target is that+speech output and reuse it as a Candidate.Two more families are local, stateless and spend nothing. `hypit media` prepares what the eyes willlook at when a whole video is too long or too dense to hand over at once, and `hypit vocabulary`@@ -54,21 +55,26 @@ prints what a Source may write:```bashhypit media probe reference.mp4-hypit media cut reference.mp4 --start 12 --end 19.5 --to notes/hook.mp4-hypit media frames reference.mp4 --at 12.4,13.1 --to notes/hook-frames+hypit media cut reference.mp4 --start 12 --end 19.5 --label-time --to notes/hook.mp4+hypit media frames reference.mp4 --at 12.4,13.1 --label-time --to notes/hook-frameshypit media tile reference.mp4 --start 12 --end 19.5 --to notes/hook-grid.jpg-hypit media shots reference.mp4+hypit media tile reference.mp4 --at 12.4,13.1,14.8 --columns 3 --to notes/exact-grid.jpg+hypit media tiles reference.mp4 --ranges notes/ranges.json --to notes/grids+hypit media boundaries reference.mp4hypit media fetch https://… --to reference/source.mp4hypit vocabularyhypit vocabulary @hypit/media-pipeline --tag StillVideohypit vocabulary --visual text```-`cut` seeks to the exact frame; `frames` writes one JPEG per named second; `tile` draws four to nine-frames of a stretch in one grid, never wider than the source, so small type stays legible for an-observer that reads pictures; `shots` lists where the picture jumps, which is a starting point when-nothing is known yet and not a shot list; `fetch` turns a link into a file with the pinned yt-dlp.-Every command writes only what `--to` names and refuses to overwrite. `vocabulary` reads the installed+`cut` seeks to the exact frame and can visibly overlay absolute source time on the evidence copy;+`frames` writes one JPEG per named second and can add the same visible label. `tile` always draws the+absolute source time below every cell. It accepts either one range sampled evenly or exact `--at`+times; `tiles` repeats that operation for an ordinary JSON array of `{ start, end, id?, frames? }`+ranges. Columns, cell width and sample count remain caller choices. `boundaries` reports adjacent-frame+change candidates and their measured scores; it does not suppress short changes or call them shots.+`fetch` turns a link into a file with the pinned yt-dlp. Commands that create evidence write only+what `--to` names and refuse to overwrite. `vocabulary` reads the installedmanifests: every package with its tags and models, or one package's Surfaces with their attributes,children and example, or the value shapes a drawing Producer must emit.diff --git a/packages/video-cli/package.json b/packages/video-cli/package.jsonindex 9dd2ad52..0b943ad7 100644--- a/packages/video-cli/package.json+++ b/packages/video-cli/package.json@@ -18,7 +18,6 @@"@hypit/gemini": "workspace:*","@hypit/generation": "workspace:*","@hypit/markup": "workspace:*",- "@hypit/mimo-tts": "workspace:*","@hypit/model-kit": "workspace:*","@hypit/package-loader-node": "workspace:*","@hypit/protocol": "workspace:*",diff --git a/packages/video-cli/src/creation.ts b/packages/video-cli/src/creation.tsindex 7d09e135..1d8ce6fa 100644--- a/packages/video-cli/src/creation.ts+++ b/packages/video-cli/src/creation.ts@@ -13,9 +13,6 @@ import {} from "@hypit/estimate";import { geminiCapabilities, geminiModels, geminiTypes, sealGeminiRequest, verifyVisualObservation } from "@hypit/gemini";import type { GeminiMediaPart, GeminiModel, VisualObservation } from "@hypit/gemini";-import { verifyGeneratedAudioSet } from "@hypit/generation";-import type { GeneratedAudioSet } from "@hypit/generation";-import { mimoTtsEndpoints, sealMimoTtsRequest } from "@hypit/mimo-tts";import type { CanonicalValue, CapabilityRef, Need, StoredValue } from "@hypit/protocol";import type { ResourceStore } from "@hypit/runtime";import type { RuntimeHostCapabilityProvider, RuntimeHostProviderQuery } from "@hypit/runtime-host-node";@@ -30,15 +27,17 @@ import { videoCliDistribution } from "./distribution.js";import { runProcess } from "./process.js";/**- * Creation-time tools: see a picture, hear a recording, speak a line.+ * Creation-time tools: see a picture, hear a recording, measure a script.** Each is one immediate Need executed through the selected Runtime Profile, exactly as a Build would* resolve it — same Endpoint, same credential, same Provider — without a Build, Result or state. The* command names the Endpoint and its price page before it spends anything, writes one file the caller- * chose, and nothing else. Slow paid generation (pictures, clips) is not here: it is a Build.+ * chose, and nothing else. Everything the Author Graph declares as an output — pictures, clips, the+ * spoken A-roll — is a Build, however fast it comes back: an output carries the identity of the Source+ * that produced it, and a command's file does not.*/-export const creationCommands = ["observe", "transcribe", "speak", "measure"] as const;+export const creationCommands = ["observe", "transcribe", "measure"] as const;export type CreationCommand = typeof creationCommands[number];/** The slice of the Runtime host these commands use; tests hand in a fake. */@@ -273,20 +272,6 @@ async function speechEvidenceBytes(path: string): Promise<{ readonly bytes: Uint}}-/** Seconds of audio in a file: read from a WAV header, otherwise asked of ffprobe when it is present. */-async function audioSeconds(path: string, bytes: Uint8Array): Promise<number | undefined> {- const shape = wavShape(bytes);- if (shape !== undefined && shape.codec === 1 && shape.channels > 0 && shape.sampleRate > 0 && shape.bits > 0) {- return round(shape.dataBytes / (shape.channels * (shape.bits / 8)) / shape.sampleRate);- }- try {- const seconds = Number((await run("ffprobe", ["-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", path])).trim());- return Number.isFinite(seconds) && seconds > 0 ? round(seconds) : undefined;- } catch {- return undefined;- }-}-function round(value: number): number { return Number(value.toFixed(3)); }function seconds(sample: number | undefined): number | undefined {return sample === undefined ? undefined : round(sample / EVIDENCE_SAMPLE_RATE);@@ -422,56 +407,6 @@ async function transcribe(argv: readonly string[], io: CliIo, environment: Createlse io.write(`✓ Transcript written\n\n ${words} words in ${passages.length} passage${passages.length === 1 ? "" : "s"} over ${file.audio_seconds}s\n ${to}\n`);}-// ----------------------------------------------------------------------------------------------------// speak--async function speak(argv: readonly string[], io: CliIo, environment: CreationEnvironment): Promise<void> {- const parsed = parseArguments(argv, ["--text", "--voice", "--to", "--runtime", "--workspace"]);- assert(parsed.positionals.length === 0, `speak takes no positional arguments; received ${parsed.positionals.join(" ")}`);- const text = await textOrFile(required(parsed, "--text", "the words to speak, or a file holding them"), "--text", environment.cwd);- const voice = await textOrFile(required(parsed, "--voice", "a description of the voice, or a file holding it"), "--voice", environment.cwd);- const to = await destination(parsed, environment.cwd);- const endpoint = mimoTtsEndpoints.voiceDesign;- assert(endpoint !== undefined, "@hypit/mimo-tts declares no voice-design endpoint");- const { profile, host } = await environment.openHost(- parsed.options.get("--runtime"),- parsed.options.get("--workspace"),- );- const resources = new MemoryResourceStore();- const need: Need = {- id: "need:hypit-speak",- capability: endpoint.capability,- returns: endpoint.returns,- constraints: sealMimoTtsRequest("mimo-v2.5-tts-voicedesign", { text: [text], voiceDescription: [voice] }) as unknown as CanonicalValue,- result: "record:hypit-speak",- };- const provider = await selectedProvider(host, need, profile);- if (!parsed.json) io.write(`Speaking with ${endpoint.ports.model} through ${providerLine(provider)}\n`);- const fulfillment = await host.invoke(need, resources);- assert(fulfillment.value.kind === "inline", "the speech came back by reference");- verifyGeneratedAudioSet(fulfillment.value.value);- const [audio] = (fulfillment.value.value as unknown as GeneratedAudioSet).audios;- assert(audio !== undefined, "the model returned no audio");- const bytes = await resources.get(audio.resource);- assert(bytes !== undefined, `generated audio ${audio.resource} was not stored`);- await writeNew(to, bytes);- const durationSeconds = await audioSeconds(to, bytes);- const view = {- format: "hypit.video-cli-speak@1",- model: endpoint.ports.model,- ...providerView(provider),- mediaType: audio.mediaType,- bytes: bytes.byteLength,- ...(durationSeconds === undefined ? {} : { duration_seconds: durationSeconds }),- path: to,- };- if (parsed.json) io.write(`${JSON.stringify(view, null, 2)}\n`);- else {- io.write(`✓ Speech written\n\n ${audio.mediaType} · ${bytes.byteLength} bytes${durationSeconds === undefined ? "" : ` · ${durationSeconds}s`}\n ${to}\n`);- if (durationSeconds === undefined) io.write(" (duration unknown: ffprobe is unavailable; measure the file before writing a literal)\n");- }-}-// ---------------------------------------------------------------------------------------------------// measure@@ -579,16 +514,6 @@ export function writeCreationHelp(io: CliIo, topic?: CreationCommand): void {"Extracts 16 kHz mono speech audio with ffmpeg and writes every word with its start and end in","seconds. One immediate request; no Build, Result or state.",],- speak: [- "hypit speak",- "Speak a line with the MiMo VoiceDesign Endpoint of the selected Runtime Profile.",- "",- " hypit speak --text <text|file> --voice <description|file> --to <audio file> [--runtime <profile>] [--workspace <project>]",- "",- "Writes the audio and reports its duration, so the author can write that duration as a literal.",- "For a voice-over video this is the A-roll: speak first, measure, then author B-roll that covers",- "every passage. Where a real presenter is the A-roll, do not replace them with this.",- ],measure: ["hypit measure","Measure a script before writing a duration. Pure local work; no Runtime Profile, no request.",@@ -604,7 +529,7 @@ export function writeCreationHelp(io: CliIo, topic?: CreationCommand): void {};const chosen = topic === undefined ? creationCommands : [topic];io.write(`${chosen.map((item) => sections[item].join("\n")).join("\n\n")}\n\n`- + "observe, transcribe and speak name the Endpoint and its price page before they run; the Profile\n"+ + "observe and transcribe name the Endpoint and its price page before they run; the Profile\n"+ "comes from --runtime or the project's `hypit runtime use` selection. measure spends nothing.\n");}@@ -614,6 +539,5 @@ export async function runCreationCli(argv: readonly string[], io: CliIo, environif (argv.includes("--help")) { writeCreationHelp(io, command); return; }if (command === "observe") await observe(argv, io, environment);else if (command === "transcribe") await transcribe(argv, io, environment);- else if (command === "speak") await speak(argv, io, environment);else await measure(argv, io, environment);}diff --git a/packages/video-cli/test/creation.test.ts b/packages/video-cli/test/creation.test.tsindex 07b457da..a70745ee 100644--- a/packages/video-cli/test/creation.test.ts+++ b/packages/video-cli/test/creation.test.ts@@ -53,10 +53,6 @@ function host(seen: Need[]): CreationHost {}, request.sampleFrames),})) } };}- if (need.capability.name === "mimo-v2.5-tts-voicedesign") {- const artifact = await resources.put(wav(24_000), "audio/wav");- return { value: { kind: "inline" as const, value: canonicalize({ audios: [artifact] }) } };- }const request = need.constraints as { readonly prompt: string; readonly media: readonly unknown[] };return { value: { kind: "inline" as const, value: canonicalize(sealVisualObservation(`saw ${request.media.length} media; asked: ${request.prompt}`)) } };},@@ -122,28 +118,6 @@ test("transcribe writes every word in seconds from the Profile's alignment Endpo}});-test("speak writes the audio and its duration so the author can write a literal", async () => {- const root = await mkdtemp(join(tmpdir(), "hypit-speak-"));- const seen: Need[] = [];- try {- await writeFile(join(root, "line.txt"), "Welcome back to the channel.\n", "utf8");- const out = capture();- await runCreationCli([- "speak", "--text", "line.txt", "--voice", "warm, unhurried, mid-thirties", "--to", "voice/intro.wav", "--json",- ], out.io, environment(root, seen));- const view = JSON.parse(out.text()) as Record<string, unknown>;- assert.equal(view.model, "mimo-v2.5-tts-voicedesign");- assert.equal(view.mediaType, "audio/wav");- assert.equal(view.duration_seconds, 1.5);- assert.equal((await readFile(join(root, "voice", "intro.wav"))).byteLength, 44 + 24_000 * 2);- const ports = seen[0]!.constraints as { readonly ports?: Record<string, unknown> };- assert.equal(JSON.stringify(ports).includes("Welcome back to the channel."), true);- assert.equal(JSON.stringify(ports).includes("warm, unhurried"), true);- } finally {- await rm(root, { recursive: true, force: true });- }-});-test("measure prints the seconds a line takes so the author can write the literal, without any Profile", async () => {const root = await mkdtemp(join(tmpdir(), "hypit-measure-"));try {diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex 865d8c13..898ca485 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -3240,9 +3240,6 @@ importers:'@hypit/markup':specifier: workspace:*version: link:../markup- '@hypit/mimo-tts':- specifier: workspace:*- version: link:../mimo-tts'@hypit/model-kit':specifier: workspace:*version: link:../model-kit