Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

refactor(video-cli): remove hypit speak; spoken A-roll is a graph output

kashorin0192ef03d28a
diff --git a/packages/video-cli/README.md b/packages/video-cli/README.md
index 93cd2719..cf9f65cd 100644
--- a/packages/video-cli/README.md
+++ b/packages/video-cli/README.md
@@ -36,17 +36,18 @@ file the caller chose:
```bash
hypit observe <media…> --instruction <text|file> --prompt <text|file> --to notes/frame.md
hypit transcribe reference.mp4 --to notes/reference.transcript.json [--language en|zh|es]
-hypit speak --text line.txt --voice "warm, unhurried" --to voice/intro.mp3
hypit measure main.svml --segment hook --language en --pace normal --min 4 --max 15 --rounding round
```
-`observe` uses the Profile's Gemini Endpoint, `transcribe` its `whisperx-alignment` Endpoint (after
-extracting 16 kHz mono speech audio with ffmpeg), `speak` its MiMo VoiceDesign Endpoint and reports
-the audio's duration so the author can write it as a literal. `measure` counts a Segment's pronunciation
-units at a delivery policy and prints the seconds to write as the literal `duration`; it opens no
-Profile and spends nothing. `--runtime <profile>` names the Profile;
-otherwise the project's `hypit runtime use` selection is read. Slow paid generation — pictures and
-clips — is not a creation-time tool: declare it in the Source and go through `plan` and `build`.
+`observe` uses the Profile's Gemini Endpoint and `transcribe` its `whisperx-alignment` Endpoint (after
+extracting 16 kHz mono speech audio with ffmpeg). `measure` counts a Segment's pronunciation units at a
+delivery policy and prints the seconds to write as the literal `duration`; it opens no Profile and
+spends nothing. `--runtime <profile>` names the Profile; otherwise the project's `hypit runtime use`
+selection is read. Anything the Author Graph declares as an output is a Build, however quickly it
+comes back: pictures, clips and the spoken A-roll (`@hypit/mimo-tts`) carry the identity of the Source
+that produced them, so they are declared in the Source and go through `plan` and `build`. To hear a
+voice or learn a passage's real length before authoring the rest, build a Run whose target is that
+speech output and reuse it as a Candidate.
Two more families are local, stateless and spend nothing. `hypit media` prepares what the eyes will
look at when a whole video is too long or too dense to hand over at once, and `hypit vocabulary`
@@ -54,21 +55,26 @@ prints what a Source may write:
```bash
hypit media probe reference.mp4
-hypit media cut reference.mp4 --start 12 --end 19.5 --to notes/hook.mp4
-hypit media frames reference.mp4 --at 12.4,13.1 --to notes/hook-frames
+hypit media cut reference.mp4 --start 12 --end 19.5 --label-time --to notes/hook.mp4
+hypit media frames reference.mp4 --at 12.4,13.1 --label-time --to notes/hook-frames
hypit media tile reference.mp4 --start 12 --end 19.5 --to notes/hook-grid.jpg
-hypit media shots reference.mp4
+hypit media tile reference.mp4 --at 12.4,13.1,14.8 --columns 3 --to notes/exact-grid.jpg
+hypit media tiles reference.mp4 --ranges notes/ranges.json --to notes/grids
+hypit media boundaries reference.mp4
hypit media fetch https://… --to reference/source.mp4
hypit vocabulary
hypit vocabulary @hypit/media-pipeline --tag StillVideo
hypit vocabulary --visual text
```
-`cut` seeks to the exact frame; `frames` writes one JPEG per named second; `tile` draws four to nine
-frames of a stretch in one grid, never wider than the source, so small type stays legible for an
-observer that reads pictures; `shots` lists where the picture jumps, which is a starting point when
-nothing is known yet and not a shot list; `fetch` turns a link into a file with the pinned yt-dlp.
-Every command writes only what `--to` names and refuses to overwrite. `vocabulary` reads the installed
+`cut` seeks to the exact frame and can visibly overlay absolute source time on the evidence copy;
+`frames` writes one JPEG per named second and can add the same visible label. `tile` always draws the
+absolute source time below every cell. It accepts either one range sampled evenly or exact `--at`
+times; `tiles` repeats that operation for an ordinary JSON array of `{ start, end, id?, frames? }`
+ranges. Columns, cell width and sample count remain caller choices. `boundaries` reports adjacent-frame
+change candidates and their measured scores; it does not suppress short changes or call them shots.
+`fetch` turns a link into a file with the pinned yt-dlp. Commands that create evidence write only
+what `--to` names and refuse to overwrite. `vocabulary` reads the installed
manifests: every package with its tags and models, or one package's Surfaces with their attributes,
children and example, or the value shapes a drawing Producer must emit.
diff --git a/packages/video-cli/package.json b/packages/video-cli/package.json
index 9dd2ad52..0b943ad7 100644
--- a/packages/video-cli/package.json
+++ b/packages/video-cli/package.json
@@ -18,7 +18,6 @@
"@hypit/gemini": "workspace:*",
"@hypit/generation": "workspace:*",
"@hypit/markup": "workspace:*",
- "@hypit/mimo-tts": "workspace:*",
"@hypit/model-kit": "workspace:*",
"@hypit/package-loader-node": "workspace:*",
"@hypit/protocol": "workspace:*",
diff --git a/packages/video-cli/src/creation.ts b/packages/video-cli/src/creation.ts
index 7d09e135..1d8ce6fa 100644
--- a/packages/video-cli/src/creation.ts
+++ b/packages/video-cli/src/creation.ts
@@ -13,9 +13,6 @@ import {
} from "@hypit/estimate";
import { geminiCapabilities, geminiModels, geminiTypes, sealGeminiRequest, verifyVisualObservation } from "@hypit/gemini";
import type { GeminiMediaPart, GeminiModel, VisualObservation } from "@hypit/gemini";
-import { verifyGeneratedAudioSet } from "@hypit/generation";
-import type { GeneratedAudioSet } from "@hypit/generation";
-import { mimoTtsEndpoints, sealMimoTtsRequest } from "@hypit/mimo-tts";
import type { CanonicalValue, CapabilityRef, Need, StoredValue } from "@hypit/protocol";
import type { ResourceStore } from "@hypit/runtime";
import type { RuntimeHostCapabilityProvider, RuntimeHostProviderQuery } from "@hypit/runtime-host-node";
@@ -30,15 +27,17 @@ import { videoCliDistribution } from "./distribution.js";
import { runProcess } from "./process.js";
/**
- * Creation-time tools: see a picture, hear a recording, speak a line.
+ * Creation-time tools: see a picture, hear a recording, measure a script.
*
* Each is one immediate Need executed through the selected Runtime Profile, exactly as a Build would
* resolve it — same Endpoint, same credential, same Provider — without a Build, Result or state. The
* command names the Endpoint and its price page before it spends anything, writes one file the caller
- * chose, and nothing else. Slow paid generation (pictures, clips) is not here: it is a Build.
+ * chose, and nothing else. Everything the Author Graph declares as an output — pictures, clips, the
+ * spoken A-roll — is a Build, however fast it comes back: an output carries the identity of the Source
+ * that produced it, and a command's file does not.
*/
-export const creationCommands = ["observe", "transcribe", "speak", "measure"] as const;
+export const creationCommands = ["observe", "transcribe", "measure"] as const;
export type CreationCommand = typeof creationCommands[number];
/** The slice of the Runtime host these commands use; tests hand in a fake. */
@@ -273,20 +272,6 @@ async function speechEvidenceBytes(path: string): Promise<{ readonly bytes: Uint
}
}
-/** Seconds of audio in a file: read from a WAV header, otherwise asked of ffprobe when it is present. */
-async function audioSeconds(path: string, bytes: Uint8Array): Promise<number | undefined> {
- const shape = wavShape(bytes);
- if (shape !== undefined && shape.codec === 1 && shape.channels > 0 && shape.sampleRate > 0 && shape.bits > 0) {
- return round(shape.dataBytes / (shape.channels * (shape.bits / 8)) / shape.sampleRate);
- }
- try {
- const seconds = Number((await run("ffprobe", ["-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", path])).trim());
- return Number.isFinite(seconds) && seconds > 0 ? round(seconds) : undefined;
- } catch {
- return undefined;
- }
-}
-
function round(value: number): number { return Number(value.toFixed(3)); }
function seconds(sample: number | undefined): number | undefined {
return sample === undefined ? undefined : round(sample / EVIDENCE_SAMPLE_RATE);
@@ -422,56 +407,6 @@ async function transcribe(argv: readonly string[], io: CliIo, environment: Creat
else io.write(`✓ Transcript written\n\n ${words} words in ${passages.length} passage${passages.length === 1 ? "" : "s"} over ${file.audio_seconds}s\n ${to}\n`);
}
-// ---------------------------------------------------------------------------------------------------
-// speak
-
-async function speak(argv: readonly string[], io: CliIo, environment: CreationEnvironment): Promise<void> {
- const parsed = parseArguments(argv, ["--text", "--voice", "--to", "--runtime", "--workspace"]);
- assert(parsed.positionals.length === 0, `speak takes no positional arguments; received ${parsed.positionals.join(" ")}`);
- const text = await textOrFile(required(parsed, "--text", "the words to speak, or a file holding them"), "--text", environment.cwd);
- const voice = await textOrFile(required(parsed, "--voice", "a description of the voice, or a file holding it"), "--voice", environment.cwd);
- const to = await destination(parsed, environment.cwd);
- const endpoint = mimoTtsEndpoints.voiceDesign;
- assert(endpoint !== undefined, "@hypit/mimo-tts declares no voice-design endpoint");
- const { profile, host } = await environment.openHost(
- parsed.options.get("--runtime"),
- parsed.options.get("--workspace"),
- );
- const resources = new MemoryResourceStore();
- const need: Need = {
- id: "need:hypit-speak",
- capability: endpoint.capability,
- returns: endpoint.returns,
- constraints: sealMimoTtsRequest("mimo-v2.5-tts-voicedesign", { text: [text], voiceDescription: [voice] }) as unknown as CanonicalValue,
- result: "record:hypit-speak",
- };
- const provider = await selectedProvider(host, need, profile);
- if (!parsed.json) io.write(`Speaking with ${endpoint.ports.model} through ${providerLine(provider)}\n`);
- const fulfillment = await host.invoke(need, resources);
- assert(fulfillment.value.kind === "inline", "the speech came back by reference");
- verifyGeneratedAudioSet(fulfillment.value.value);
- const [audio] = (fulfillment.value.value as unknown as GeneratedAudioSet).audios;
- assert(audio !== undefined, "the model returned no audio");
- const bytes = await resources.get(audio.resource);
- assert(bytes !== undefined, `generated audio ${audio.resource} was not stored`);
- await writeNew(to, bytes);
- const durationSeconds = await audioSeconds(to, bytes);
- const view = {
- format: "hypit.video-cli-speak@1",
- model: endpoint.ports.model,
- ...providerView(provider),
- mediaType: audio.mediaType,
- bytes: bytes.byteLength,
- ...(durationSeconds === undefined ? {} : { duration_seconds: durationSeconds }),
- path: to,
- };
- if (parsed.json) io.write(`${JSON.stringify(view, null, 2)}\n`);
- else {
- io.write(`✓ Speech written\n\n ${audio.mediaType} · ${bytes.byteLength} bytes${durationSeconds === undefined ? "" : ` · ${durationSeconds}s`}\n ${to}\n`);
- if (durationSeconds === undefined) io.write(" (duration unknown: ffprobe is unavailable; measure the file before writing a literal)\n");
- }
-}
-
// ---------------------------------------------------------------------------------------------------
// measure
@@ -579,16 +514,6 @@ export function writeCreationHelp(io: CliIo, topic?: CreationCommand): void {
"Extracts 16 kHz mono speech audio with ffmpeg and writes every word with its start and end in",
"seconds. One immediate request; no Build, Result or state.",
],
- speak: [
- "hypit speak",
- "Speak a line with the MiMo VoiceDesign Endpoint of the selected Runtime Profile.",
- "",
- " hypit speak --text <text|file> --voice <description|file> --to <audio file> [--runtime <profile>] [--workspace <project>]",
- "",
- "Writes the audio and reports its duration, so the author can write that duration as a literal.",
- "For a voice-over video this is the A-roll: speak first, measure, then author B-roll that covers",
- "every passage. Where a real presenter is the A-roll, do not replace them with this.",
- ],
measure: [
"hypit measure",
"Measure a script before writing a duration. Pure local work; no Runtime Profile, no request.",
@@ -604,7 +529,7 @@ export function writeCreationHelp(io: CliIo, topic?: CreationCommand): void {
};
const chosen = topic === undefined ? creationCommands : [topic];
io.write(`${chosen.map((item) => sections[item].join("\n")).join("\n\n")}\n\n`
- + "observe, transcribe and speak name the Endpoint and its price page before they run; the Profile\n"
+ + "observe and transcribe name the Endpoint and its price page before they run; the Profile\n"
+ "comes from --runtime or the project's `hypit runtime use` selection. measure spends nothing.\n");
}
@@ -614,6 +539,5 @@ export async function runCreationCli(argv: readonly string[], io: CliIo, environ
if (argv.includes("--help")) { writeCreationHelp(io, command); return; }
if (command === "observe") await observe(argv, io, environment);
else if (command === "transcribe") await transcribe(argv, io, environment);
- else if (command === "speak") await speak(argv, io, environment);
else await measure(argv, io, environment);
}
diff --git a/packages/video-cli/test/creation.test.ts b/packages/video-cli/test/creation.test.ts
index 07b457da..a70745ee 100644
--- a/packages/video-cli/test/creation.test.ts
+++ b/packages/video-cli/test/creation.test.ts
@@ -53,10 +53,6 @@ function host(seen: Need[]): CreationHost {
}, request.sampleFrames),
})) } };
}
- if (need.capability.name === "mimo-v2.5-tts-voicedesign") {
- const artifact = await resources.put(wav(24_000), "audio/wav");
- return { value: { kind: "inline" as const, value: canonicalize({ audios: [artifact] }) } };
- }
const request = need.constraints as { readonly prompt: string; readonly media: readonly unknown[] };
return { value: { kind: "inline" as const, value: canonicalize(sealVisualObservation(`saw ${request.media.length} media; asked: ${request.prompt}`)) } };
},
@@ -122,28 +118,6 @@ test("transcribe writes every word in seconds from the Profile's alignment Endpo
}
});
-test("speak writes the audio and its duration so the author can write a literal", async () => {
- const root = await mkdtemp(join(tmpdir(), "hypit-speak-"));
- const seen: Need[] = [];
- try {
- await writeFile(join(root, "line.txt"), "Welcome back to the channel.\n", "utf8");
- const out = capture();
- await runCreationCli([
- "speak", "--text", "line.txt", "--voice", "warm, unhurried, mid-thirties", "--to", "voice/intro.wav", "--json",
- ], out.io, environment(root, seen));
- const view = JSON.parse(out.text()) as Record<string, unknown>;
- assert.equal(view.model, "mimo-v2.5-tts-voicedesign");
- assert.equal(view.mediaType, "audio/wav");
- assert.equal(view.duration_seconds, 1.5);
- assert.equal((await readFile(join(root, "voice", "intro.wav"))).byteLength, 44 + 24_000 * 2);
- const ports = seen[0]!.constraints as { readonly ports?: Record<string, unknown> };
- assert.equal(JSON.stringify(ports).includes("Welcome back to the channel."), true);
- assert.equal(JSON.stringify(ports).includes("warm, unhurried"), true);
- } finally {
- await rm(root, { recursive: true, force: true });
- }
-});
-
test("measure prints the seconds a line takes so the author can write the literal, without any Profile", async () => {
const root = await mkdtemp(join(tmpdir(), "hypit-measure-"));
try {
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 865d8c13..898ca485 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -3240,9 +3240,6 @@ importers:
'@hypit/markup':
specifier: workspace:*
version: link:../markup
- '@hypit/mimo-tts':
- specifier: workspace:*
- version: link:../mimo-tts
'@hypit/model-kit':
specifier: workspace:*
version: link:../model-kit