Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.
TypeScript agentic-aiaiai-agentsai-videocompilerdslffmpeggenerative-aillmmarkup-languagemonorepoplugin-systemprogramming-languagetext-to-videotypescriptvideovideo-automationvideo-clonevideo-editingvideo-generation
fix(estimate): clarify delivery pace and mixed-language speech measurement
kashorin98127b817e1a
diff --git a/packages/estimate/README.md b/packages/estimate/README.mdindex 61abb23d..18064dfd 100644--- a/packages/estimate/README.md+++ b/packages/estimate/README.md@@ -14,9 +14,46 @@ hypit measure main.svml --segment opening --language en --pace normalhypit measure --text "Video editing begins with meaning." --language en --pace normal --rounding ceil```-The English pace presets are `slow = 4.2`, `normal = 4.6` and `fast = 5.0` syllables per second; a-positive numeric `rate` replaces `pace`. `round` selects the nearest whole second, `ceil` a-never-shorter one, `none` keeps the fraction.+## Units and delivery++English counts pronunciation syllables, not words or letters. Mandarin Chinese counts each Han+character as one approximate syllable, with English syllables for embedded English words.+Traditional and simplified Han use the same counting rule; punctuation separates words without+adding units. This remains an approximation: connected English speech can reduce syllables,+and Mandarin erhua can combine written characters into one syllable. An explicit `--language zh`+keeps a Chinese passage with many English names under its intended delivery policy.++Measure the spoken wording. `--segment` reads the Script's pronunciation side of Dual Text;+`--text` reads literal speech, so spell numbers and letter names as they will be said. A count+cannot resolve an unspecified pronunciation.++| Language | slow | normal | fast | Unit |+| --- | ---: | ---: | ---: | --- |+| English (`en`) | 4.2 | 4.6 | 5.6 | syllables/s |+| Mandarin Chinese (`zh`) | 4.2 | 5.25 | 6.5625 | approximate syllables/s |++These are authoring starting points for whole-passage delivery, including ordinary speech pauses.+`normal` suits conversational explanation; `fast` gives brisk, tightly delivered copy less time.+The English fast preset is deliberately separated from normal so that choosing brisk delivery+meaningfully changes the estimate. The Chinese normal preset is retained: available research+does not establish that it is systematically too slow for whole passages. Neither table is a+calibration of a particular video or voice model. For a specific delivery, a positive numeric+`--rate` replaces the preset, including rates outside this table.++```bash+hypit measure main.svml --segment opening --language zh --pace fast --rounding round+hypit measure main.svml --segment opening --language zh --rate 6 --rounding none+```++`seconds = pronunciation units / rate + padding`, followed by rounding. Add `--padding` for extra+time deliberately reserved for a reaction, demonstration or held pause. Ordinary phrasing is+already part of the chosen density; adding a generic pause allowance on top lengthens the passage+again. `round` selects the nearest whole second, `ceil` rounds upward, and `none` keeps the fraction.+The CLI shows the resolved rate and padding; JSON retains full numeric precision.++For 60 Mandarin units, normal gives 11.43 seconds and fast gives 9.14 seconds before rounding.+For 60 English syllables, normal gives 13.04 seconds and fast gives 10.71 seconds. Equal unit counts+do not imply equal information, and 60 English words need not contain 60 syllables.The estimate reports the time the wording needs at that delivery. Use it to decide whether a passageneeds merging, fuller wording, tightening, or splitting, then choose the request duration with the@@ -30,3 +67,19 @@ performed media's semantic preparation.```tsimport { estimateSpeechDuration, sealSpeechEstimatePolicy, speechEstimatePolicyFromRecipe } from "@hypit/estimate";```++## Research basis and limits++[Burchfield and Bradlow (2014)](https://doi.org/10.1121/1.4874357) measured Mandarin and English+across clear reading, plain reading and spontaneous speech. Plain-reading means were 5.97 and+5.39 canonical syllables/s respectively. Their articulation measure excludes errors, silence+and non-speech, so those means cannot directly size an entire performance containing pauses.+[Coupé et al. (2019)](https://doi.org/10.1126/sciadv.aaw2594) likewise counted canonical syllables+and excluded pauses longer than 150 ms in their cross-language reading study. These studies+support distinguishing language, pronunciation units and timing denominator; they do not+establish one ideal pace for social video or validate these preset values.++When an already understood reference passage provides useful timing, its spoken-unit count+divided by its first-to-last-word span offers a starting density, including internal pauses.+Choose how much of that rhythm belongs in the new performance. Requested clip length alone+is not measured speaking time, and delivery density does not describe emphasis or expression.diff --git a/packages/estimate/src/program.ts b/packages/estimate/src/program.tsindex 3bb3ed5e..de58d6d0 100644--- a/packages/estimate/src/program.ts+++ b/packages/estimate/src/program.ts@@ -14,9 +14,9 @@ const PACE_RATE: Readonly<Record<ResolvedSpeechEstimateLanguage,Readonly<Record<SpeechEstimatePace, number>>>> = {- // English is calibrated as delivery density, not pause-free articulation rate.- en: { slow: 4.2, normal: 4.6, fast: 5 },- // Preserve the existing non-English behavior until those languages are audited.+ // Authoring delivery densities, including ordinary pauses, not measured model+ // guarantees or pause-free articulation rates. See README for units and rationale.+ en: { slow: 4.2, normal: 4.6, fast: 5.6 },zh: { slow: 4.2, normal: 5.25, fast: 6.5625 },ja: { slow: 6, normal: 7.5, fast: 9.375 },es: { slow: 4.72, normal: 5.9, fast: 7.375 },@@ -35,7 +35,7 @@ function looksSpanish(text: string): boolean {}export function detectSpeechEstimateLanguage(text: string): ResolvedSpeechEstimateLanguage {- const han = (text.match(/[\u3400-\u9fff]/gu) ?? []).length;+ const han = (text.match(/\p{Script=Han}/gu) ?? []).length;const kana = (text.match(/[\u3040-\u30ff]/gu) ?? []).length;const ascii = (text.match(/[A-Za-z]/gu) ?? []).length;if (kana > Math.max(han, ascii / 4)) return "ja";@@ -117,10 +117,9 @@ export function countSpeechEstimateUnits(language: ResolvedSpeechEstimateLanguage,): number {if (language === "zh" || language === "ja") {- const cjk = (text.match(/[\u3400-\u9fff\u3040-\u30ff]/gu) ?? []).length;- const latin = text- .replace(/[\u3400-\u9fff\u3040-\u30ff]/gu, " ")- .split(/\s+/u)+ const characters = /[\p{Script=Han}\u3040-\u30ff]/gu;+ const cjk = (text.match(characters) ?? []).length;+ const latin = words(text.replace(characters, " ")).filter((word) => /[A-Za-z]/u.test(word));return cjk + latin.reduce((sum, word) => sum + englishSyllables(word), 0);}diff --git a/packages/estimate/test/estimate.test.ts b/packages/estimate/test/estimate.test.tsindex 6dfaccd6..a8d39b9b 100644--- a/packages/estimate/test/estimate.test.ts+++ b/packages/estimate/test/estimate.test.ts@@ -4,6 +4,7 @@ import { sealText } from "@hypit/text";import {countSpeechEstimateUnits,+ detectSpeechEstimateLanguage,estimateSpeechDuration,sealSpeechEstimatePolicy,speechEstimatePolicyFromAttributes,@@ -32,7 +33,7 @@ test("normal English speech estimate follows the delivery-density policy", () =>assert.equal(result, 4);});-test("English pace presets occupy adjacent integer durations", () => {+test("English pace choices give a passage room for deliberate or brisk delivery", () => {const source = sealText(Array.from({ length: 49 }, () => "day").join(" "));const estimate = (pace: "slow" | "normal" | "fast") => estimateSpeechDuration(source,@@ -42,7 +43,7 @@ test("English pace presets occupy adjacent integer durations", () => {rounding: normal.rounding,}),);- assert.deepEqual([estimate("slow"), estimate("normal"), estimate("fast")], [12, 11, 10]);+ assert.deepEqual([estimate("slow"), estimate("normal"), estimate("fast")], [12, 11, 9]);});test("a numeric rate gives SVS a continuous author-controlled pace", () => {@@ -123,3 +124,21 @@ test("an SVS Recipe configures one reusable estimate policy without becoming exeproperties: { provider: "unknown-provider" },}), /unknown property/u);});++test("Chinese estimates count Han pronunciation units, including names outside the basic Unicode plane", () => {+ assert.equal(detectSpeechEstimateLanguage("𠮷"), "zh");+ assert.equal(countSpeechEstimateUnits("這段中文有節奏。", "zh"), 7);+ assert.equal(countSpeechEstimateUnits("𠮷野家", "zh"), 3);+ assert.equal(countSpeechEstimateUnits("二零二六年", "zh"), 5);+ assert.equal(countSpeechEstimateUnits("用 video 做视频", "zh"), 7);+});++test("Chinese mixed-language punctuation preserves English word boundaries", () => {+ for (const separator of [" ", ",", ",", "/", "。", ":"]) {+ assert.equal(countSpeechEstimateUnits(`用 idea${separator}video 做视频`, "zh"), 10);+ }+ const source = sealText("用 idea,video 做视频");+ assert.equal(estimateSpeechDuration(source, {+ language: "zh", rate: 5, rounding: "none", paddingSec: 0.5,+ }), 2.5);+});diff --git a/packages/video-cli/README.md b/packages/video-cli/README.mdindex 5832ac21..ce490493 100644--- a/packages/video-cli/README.md+++ b/packages/video-cli/README.md@@ -37,14 +37,23 @@ Result or state. It names the Endpoint and its price page before it runs, and wrto the chosen file. `measure` estimates a passage locally:```bash-hypit transcribe reference.mp4 --to notes/reference.transcript.json [--language en|zh|es]+hypit transcribe reference.mp4 --to notes/reference.transcript.json --language enhypit measure main.svml --segment hook --language en --pace normal --rounding round```+For `transcribe`, set `--language` to the spoken language: `en`, `zh` or `es`. Chinese speech uses `zh`, including+Chinese speech containing English names; the requested language selects the recognition/alignment+model, independently of the eventual caption font or script's simplified/traditional characters.+`transcribe` uses the Profile's `whisperx-alignment` Endpoint (afterextracting 16 kHz mono speech audio with ffmpeg). `measure` counts a Segment's pronunciation units at a-delivery policy and prints the seconds to write as the literal `duration`; it opens no Profile and-spends nothing. `--runtime <profile>` names the Profile; otherwise the project's `hypit runtime use`+delivery policy and prints estimated seconds, the resolved rate, padding and rounding. Choose the+literal `duration` from that estimate and the intended performance. `measure` opens no Profile and+spends nothing. It accepts `--pace slow|normal|fast` or `--rate <units/s>`; JSON includes the resolved+`rate` even when using a named pace. [Estimate](../estimate/README.md#units-and-delivery) explains+Chinese/English units and choosing a whole-passage density.++For `transcribe`, `--runtime <profile>` names the Profile; otherwise the project's `hypit runtime use`selection is read. Anything the Author Graph declares as an output is a Build, however quickly itcomes back: pictures, clips and accepted voice references (`@hypit/mimo-speech`) carry the identity of the Sourcethat produced them, so they are declared in the Source and go through `plan` and `build`. To hear adiff --git a/packages/video-cli/src/creation.ts b/packages/video-cli/src/creation.tsindex c65eee5d..535da6a4 100644--- a/packages/video-cli/src/creation.ts+++ b/packages/video-cli/src/creation.ts@@ -9,6 +9,7 @@ import {countSpeechEstimateUnits,estimateSpeechDuration,resolveSpeechEstimateLanguage,+ resolveSpeechEstimateRate,speechEstimatePolicyFromAttributes,} from "@hypit/estimate";import type { CanonicalValue, CapabilityRef, Need, StoredValue } from "@hypit/protocol";@@ -275,8 +276,9 @@ async function transcribe(argv: readonly string[], io: CliIo, environment: Creatconst parsed = parseArguments(argv, ["--language", "--to", "--runtime", "--workspace"]);assert(parsed.positionals.length === 1, "transcribe takes exactly one audio or video file");const source = resolve(environment.cwd, parsed.positionals[0]!);- const language = parsed.options.get("--language") ?? "en";- assert(language === "en" || language === "zh" || language === "es", "--language must be en, zh or es");+ const language = parsed.options.get("--language");+ assert(language === "en" || language === "zh" || language === "es",+ "transcribe requires --language en|zh|es for the spoken language (use --language zh for Chinese)");const to = await destination(parsed, environment.cwd);const { profile, host } = await environment.openHost(parsed.options.get("--runtime"),@@ -382,6 +384,7 @@ async function measure(argv: readonly string[], io: CliIo, environment: Creation}const pace = parsed.options.get("--pace");const rate = parsed.options.get("--rate");+ assert(pace === undefined || rate === undefined, "measure takes either --pace or --rate, not both");const policy = speechEstimatePolicyFromAttributes({language: parsed.options.get("--language") ?? "auto",...(rate === undefined ? { pace: pace ?? "normal" } : { rate }),@@ -391,6 +394,7 @@ async function measure(argv: readonly string[], io: CliIo, environment: Creationconst language = resolveSpeechEstimateLanguage(text, policy.language);const units = countSpeechEstimateUnits(text, language);const seconds = estimateSpeechDuration(sealText(text), policy);+ const resolvedRate = resolveSpeechEstimateRate(policy, language);const view = {format: "hypit.video-cli-measure@1",...where,@@ -398,11 +402,13 @@ async function measure(argv: readonly string[], io: CliIo, environment: Creationunits,language,policy,+ rate: resolvedRate,seconds,};if (parsed.json) { io.write(`${JSON.stringify(view, null, 2)}\n`); return; }- const delivery = policy.rate === undefined ? `${policy.pace} pace` : `${policy.rate} units/s`;- io.write(`${seconds}s\n\n ${units} pronunciation units · ${language} · ${delivery} · rounding ${policy.rounding}\n`+ const delivery = `${resolvedRate} units/s${policy.pace === undefined ? "" : ` (${policy.pace})`}`;+ io.write(`${Number(seconds.toFixed(3))}s\n\n ${units} pronunciation units · ${language} · ${delivery}\n`+ + ` padding ${policy.paddingSec ?? 0}s · rounding ${policy.rounding}\n`+ ` ${"segment" in where ? `${where.segment} in ${where.source}` : `${text.length} characters`}\n`+ " Choose the request duration from this estimate, the intended performance, and the selected model's supported values.\n");}@@ -416,7 +422,7 @@ export function writeCreationHelp(io: CliIo, topic?: CreationCommand): void {"hypit transcribe","Establish word times with the whisperx-alignment Endpoint of the selected Runtime Profile.","",- " hypit transcribe <audio|video> --to <transcript.json> [--language en|zh|es] [--runtime <profile>] [--workspace <project>]",+ " hypit transcribe <audio|video> --to <transcript.json> --language en|zh|es [--runtime <profile>] [--workspace <project>]","","Extracts 16 kHz mono speech audio with ffmpeg and writes every word with its start and end in","seconds. One immediate request; no Build, Result or state.",diff --git a/packages/video-cli/test/creation.test.ts b/packages/video-cli/test/creation.test.tsindex 47493e9c..79c3b230 100644--- a/packages/video-cli/test/creation.test.ts+++ b/packages/video-cli/test/creation.test.ts@@ -96,15 +96,24 @@ test("measure prints the seconds a line takes so the author can write the litera"measure", "--text", "Video editing begins with meaning, not a pile of clips on a timeline.","--language", "en", "--pace", "normal", "--rounding", "round", "--json",], out.io, noHost);- const view = JSON.parse(out.text()) as { readonly seconds: number; readonly units: number; readonly language: string };+ const view = JSON.parse(out.text()) as { readonly seconds: number; readonly units: number; readonly language: string; readonly rate: number };assert.equal(view.units, 20);assert.equal(view.language, "en");assert.equal(view.seconds, 4);+ assert.equal(view.rate, 4.6);const human = capture();await runCreationCli(["measure", "--text", "hello world"], human.io, noHost);assert.match(human.text(), /^\d+(\.\d+)?s\n/u);assert.match(human.text(), /Choose the request duration/u);+ assert.match(human.text(), /4\.6 units\/s \(normal\)/u);++ const chinese = capture();+ await runCreationCli(["measure", "--text", "这段中文有节奏", "--language", "zh", "--rate", "6", "--padding", "0.5"], chinese.io, noHost);+ assert.match(chinese.text(), /^1\.667s/u);+ assert.match(chinese.text(), /7 pronunciation units · zh · 6 units\/s/u);+ assert.match(chinese.text(), /padding 0\.5s/u);+ await assert.rejects(runCreationCli(["measure", "--text", "Hello", "--pace", "fast", "--rate", "6"], capture().io, noHost), /either --pace or --rate/u);for (const [text, seconds] of [["Hello.", 2 / 4.6], [Array.from({ length: 460 }, () => "day").join(" "), 100]] as const) {const measured = capture();@@ -135,7 +144,7 @@ test("a Profile that does not serve the capability stops before anything is spen} }),};await assert.rejects(- runCreationCli(["transcribe", "speech.wav", "--to", "speech.json"], capture().io, unserved),+ runCreationCli(["transcribe", "speech.wav", "--to", "speech.json", "--language", "en"], capture().io, unserved),/No Endpoint in .*hypit\.runtime\.json serves @hypit\/whisperx@1#whisperx-alignment; hypit plan --runtime/u,);assert.equal(invoked, false);@@ -143,3 +152,34 @@ test("a Profile that does not serve the capability stops before anything is spenawait rm(root, { recursive: true, force: true });}});++test("transcribe requires a spoken language before opening a host", async () => {+ const noHost: CreationEnvironment = { cwd: "/tmp", openHost: async () => { throw new Error("host must not open"); } };+ await assert.rejects(runCreationCli(["transcribe", "speech.wav", "--to", "speech.json"], capture().io, noHost),+ /requires --language en\|zh\|es/u);+});++test("Chinese transcription explicitly sends zh and retains individual character windows", async () => {+ const root = await mkdtemp(join(tmpdir(), "hypit-transcribe-zh-"));+ try {+ await writeFile(join(root, "speech.wav"), wav(32_000));+ const selected = host([]);+ const env: CreationEnvironment = { cwd: root, openHost: async () => ({ profile: join(root, "runtime.json"), host: {+ ...selected,+ invoke: async (need) => {+ const request = need.constraints as unknown as WhisperXAlignmentRequest;+ assert.equal(request.language, "zh");+ return { value: { kind: "inline", value: canonicalize({ passages: interpretWhisperXTranscript({+ segments: [{ words: [{ word: "你", start: 0.1, end: 0.24 }, { word: "好!", start: 0.5, end: 1.1 }] }],+ }, request.sampleFrames) }) } };+ },+ } }) };+ await runCreationCli(["transcribe", "speech.wav", "--language", "zh", "--to", "speech.json"], capture().io, env);+ const result = JSON.parse(await readFile(join(root, "speech.json"), "utf8"));+ assert.equal(result.language, "zh");+ assert.deepEqual(result.passages[0].words, [+ { text: "你", start_seconds: 0.1, end_seconds: 0.24 },+ { text: "好!", start_seconds: 0.5, end_seconds: 1.1 },+ ]);+ } finally { await rm(root, { recursive: true, force: true }); }+});diff --git a/skills/hypit/references/creation/reference-video.md b/skills/hypit/references/creation/reference-video.mdindex 59d66c57..ab96f12b 100644--- a/skills/hypit/references/creation/reference-video.md+++ b/skills/hypit/references/creation/reference-video.md@@ -48,6 +48,11 @@ reveals and emphasis. Keep that transcript beside the reference and interpret itsurrounding argument, visible names and supplied context. Small transcription errors can coexistwith a clear understanding of what the passage means.+Pass the reference's spoken language explicitly: `--language zh` for Chinese, `en` for English,+or `es` for Spanish. A Chinese passage can contain English brands and names while still using `zh`.+Its timed Chinese characters help locate a phrase precisely; group those characters into meaningful+phrases when describing the reference and writing the new Script.+[Environment selection](../environment/profile.md#choose-the-practical-capability-path-with-the-user)explains assessing local preparation and choosing the local or hosted path for this transcription.@@ -78,7 +83,7 @@ their changes and handoffs.```bashhypit transcribe references/ad/source.mp4 \- --to references/ad/transcript.json+ --language en --to references/ad/transcript.jsonhypit media tile references/ad/source.mp4 --start 0 --end 12 --every 1 \--to references/ad/evidence/opening.jpgdiff --git a/skills/hypit/references/creation/script-and-time.md b/skills/hypit/references/creation/script-and-time.mdindex 60d5dd2c..c61bcc97 100644--- a/skills/hypit/references/creation/script-and-time.md+++ b/skills/hypit/references/creation/script-and-time.md@@ -126,6 +126,14 @@ name, including other Segments, so separately generated performances receive theThe right-hand wording reaches the model through the Segment's `.dialogue`; Caption keeps thedisplay spelling. Settle these readings before measuring the Script and requesting its performance.+The same distinction helps Chinese copy express numbers and names clearly. For example,+`今年<2026|二零二六>年` displays the year compactly while specifying how it is said;+`只要<¥19.9|十九块九>` chooses a conversational price reading. Choose the spoken form for this+sentence's meaning and delivery. Measuring the Segment with `--language zh` then uses that spoken+wording, including the syllables hidden behind its compact numeric display. Caption retains the+authored simplified or traditional characters; transcription supplies timing rather than rewriting+the displayed Script.+## Use the Script's deliberate projectionsOne Script publishes the full Narrative and the narrow views needed by the rest of the work:@@ -219,10 +227,18 @@ estimates how many seconds they need at the chosen language and pace, using loca```bashhypit measure path/to/source.svml --segment opening --language en --pace normalhypit measure --text "You expect me to type every coffee?" --language en --pace normal --rounding ceil+hypit measure path/to/source.svml --segment opening --language zh --pace fast --rounding round```-Use `--pace slow|normal|fast`, or `--rate` for pronunciation units per second; `--padding` adds time-for the intended pause or action. Keep related performances at a coherent delivery density. Initial+Choose `--pace slow|normal|fast` from the intended delivery: conversational explanation may suit+`normal`, while brisk, tightly cut social delivery often suits `fast`. The CLI prints the actual+units-per-second rate; `--rate` lets you choose it directly. Mandarin counts Han characters as+approximate syllables and embedded English by syllable; English counts syllables rather than words.+Set `--language zh` for Chinese copy, including copy with English product names, and measure the+pronunciation side of Dual Text with `--segment`.++The density includes ordinary phrasing pauses. `--padding` reserves additional time for an intended+reaction, demonstration or held pause. Keep related performances at a coherent delivery density. Initialestimates can retain fractions with `--rounding none`. `ceil` rounds upward to a whole second, while`round` chooses the nearest one. Most video requests use whole seconds; choose the final literal withthe selected model's supported values and the intended performance in mind.