Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

fix(estimate): clarify delivery pace and mixed-language speech measurement

kashorin98127b817e1a
diff --git a/packages/estimate/README.md b/packages/estimate/README.md
index 61abb23d..18064dfd 100644
--- a/packages/estimate/README.md
+++ b/packages/estimate/README.md
@@ -14,9 +14,46 @@ hypit measure main.svml --segment opening --language en --pace normal
hypit measure --text "Video editing begins with meaning." --language en --pace normal --rounding ceil
```
-The English pace presets are `slow = 4.2`, `normal = 4.6` and `fast = 5.0` syllables per second; a
-positive numeric `rate` replaces `pace`. `round` selects the nearest whole second, `ceil` a
-never-shorter one, `none` keeps the fraction.
+## Units and delivery
+
+English counts pronunciation syllables, not words or letters. Mandarin Chinese counts each Han
+character as one approximate syllable, with English syllables for embedded English words.
+Traditional and simplified Han use the same counting rule; punctuation separates words without
+adding units. This remains an approximation: connected English speech can reduce syllables,
+and Mandarin erhua can combine written characters into one syllable. An explicit `--language zh`
+keeps a Chinese passage with many English names under its intended delivery policy.
+
+Measure the spoken wording. `--segment` reads the Script's pronunciation side of Dual Text;
+`--text` reads literal speech, so spell numbers and letter names as they will be said. A count
+cannot resolve an unspecified pronunciation.
+
+| Language | slow | normal | fast | Unit |
+| --- | ---: | ---: | ---: | --- |
+| English (`en`) | 4.2 | 4.6 | 5.6 | syllables/s |
+| Mandarin Chinese (`zh`) | 4.2 | 5.25 | 6.5625 | approximate syllables/s |
+
+These are authoring starting points for whole-passage delivery, including ordinary speech pauses.
+`normal` suits conversational explanation; `fast` gives brisk, tightly delivered copy less time.
+The English fast preset is deliberately separated from normal so that choosing brisk delivery
+meaningfully changes the estimate. The Chinese normal preset is retained: available research
+does not establish that it is systematically too slow for whole passages. Neither table is a
+calibration of a particular video or voice model. For a specific delivery, a positive numeric
+`--rate` replaces the preset, including rates outside this table.
+
+```bash
+hypit measure main.svml --segment opening --language zh --pace fast --rounding round
+hypit measure main.svml --segment opening --language zh --rate 6 --rounding none
+```
+
+`seconds = pronunciation units / rate + padding`, followed by rounding. Add `--padding` for extra
+time deliberately reserved for a reaction, demonstration or held pause. Ordinary phrasing is
+already part of the chosen density; adding a generic pause allowance on top lengthens the passage
+again. `round` selects the nearest whole second, `ceil` rounds upward, and `none` keeps the fraction.
+The CLI shows the resolved rate and padding; JSON retains full numeric precision.
+
+For 60 Mandarin units, normal gives 11.43 seconds and fast gives 9.14 seconds before rounding.
+For 60 English syllables, normal gives 13.04 seconds and fast gives 10.71 seconds. Equal unit counts
+do not imply equal information, and 60 English words need not contain 60 syllables.
The estimate reports the time the wording needs at that delivery. Use it to decide whether a passage
needs merging, fuller wording, tightening, or splitting, then choose the request duration with the
@@ -30,3 +67,19 @@ performed media's semantic preparation.
```ts
import { estimateSpeechDuration, sealSpeechEstimatePolicy, speechEstimatePolicyFromRecipe } from "@hypit/estimate";
```
+
+## Research basis and limits
+
+[Burchfield and Bradlow (2014)](https://doi.org/10.1121/1.4874357) measured Mandarin and English
+across clear reading, plain reading and spontaneous speech. Plain-reading means were 5.97 and
+5.39 canonical syllables/s respectively. Their articulation measure excludes errors, silence
+and non-speech, so those means cannot directly size an entire performance containing pauses.
+[Coupé et al. (2019)](https://doi.org/10.1126/sciadv.aaw2594) likewise counted canonical syllables
+and excluded pauses longer than 150 ms in their cross-language reading study. These studies
+support distinguishing language, pronunciation units and timing denominator; they do not
+establish one ideal pace for social video or validate these preset values.
+
+When an already understood reference passage provides useful timing, its spoken-unit count
+divided by its first-to-last-word span offers a starting density, including internal pauses.
+Choose how much of that rhythm belongs in the new performance. Requested clip length alone
+is not measured speaking time, and delivery density does not describe emphasis or expression.
diff --git a/packages/estimate/src/program.ts b/packages/estimate/src/program.ts
index 3bb3ed5e..de58d6d0 100644
--- a/packages/estimate/src/program.ts
+++ b/packages/estimate/src/program.ts
@@ -14,9 +14,9 @@ const PACE_RATE: Readonly<Record<
ResolvedSpeechEstimateLanguage,
Readonly<Record<SpeechEstimatePace, number>>
>> = {
- // English is calibrated as delivery density, not pause-free articulation rate.
- en: { slow: 4.2, normal: 4.6, fast: 5 },
- // Preserve the existing non-English behavior until those languages are audited.
+ // Authoring delivery densities, including ordinary pauses, not measured model
+ // guarantees or pause-free articulation rates. See README for units and rationale.
+ en: { slow: 4.2, normal: 4.6, fast: 5.6 },
zh: { slow: 4.2, normal: 5.25, fast: 6.5625 },
ja: { slow: 6, normal: 7.5, fast: 9.375 },
es: { slow: 4.72, normal: 5.9, fast: 7.375 },
@@ -35,7 +35,7 @@ function looksSpanish(text: string): boolean {
}
export function detectSpeechEstimateLanguage(text: string): ResolvedSpeechEstimateLanguage {
- const han = (text.match(/[\u3400-\u9fff]/gu) ?? []).length;
+ const han = (text.match(/\p{Script=Han}/gu) ?? []).length;
const kana = (text.match(/[\u3040-\u30ff]/gu) ?? []).length;
const ascii = (text.match(/[A-Za-z]/gu) ?? []).length;
if (kana > Math.max(han, ascii / 4)) return "ja";
@@ -117,10 +117,9 @@ export function countSpeechEstimateUnits(
language: ResolvedSpeechEstimateLanguage,
): number {
if (language === "zh" || language === "ja") {
- const cjk = (text.match(/[\u3400-\u9fff\u3040-\u30ff]/gu) ?? []).length;
- const latin = text
- .replace(/[\u3400-\u9fff\u3040-\u30ff]/gu, " ")
- .split(/\s+/u)
+ const characters = /[\p{Script=Han}\u3040-\u30ff]/gu;
+ const cjk = (text.match(characters) ?? []).length;
+ const latin = words(text.replace(characters, " "))
.filter((word) => /[A-Za-z]/u.test(word));
return cjk + latin.reduce((sum, word) => sum + englishSyllables(word), 0);
}
diff --git a/packages/estimate/test/estimate.test.ts b/packages/estimate/test/estimate.test.ts
index 6dfaccd6..a8d39b9b 100644
--- a/packages/estimate/test/estimate.test.ts
+++ b/packages/estimate/test/estimate.test.ts
@@ -4,6 +4,7 @@ import { sealText } from "@hypit/text";
import {
countSpeechEstimateUnits,
+ detectSpeechEstimateLanguage,
estimateSpeechDuration,
sealSpeechEstimatePolicy,
speechEstimatePolicyFromAttributes,
@@ -32,7 +33,7 @@ test("normal English speech estimate follows the delivery-density policy", () =>
assert.equal(result, 4);
});
-test("English pace presets occupy adjacent integer durations", () => {
+test("English pace choices give a passage room for deliberate or brisk delivery", () => {
const source = sealText(Array.from({ length: 49 }, () => "day").join(" "));
const estimate = (pace: "slow" | "normal" | "fast") => estimateSpeechDuration(
source,
@@ -42,7 +43,7 @@ test("English pace presets occupy adjacent integer durations", () => {
rounding: normal.rounding,
}),
);
- assert.deepEqual([estimate("slow"), estimate("normal"), estimate("fast")], [12, 11, 10]);
+ assert.deepEqual([estimate("slow"), estimate("normal"), estimate("fast")], [12, 11, 9]);
});
test("a numeric rate gives SVS a continuous author-controlled pace", () => {
@@ -123,3 +124,21 @@ test("an SVS Recipe configures one reusable estimate policy without becoming exe
properties: { provider: "unknown-provider" },
}), /unknown property/u);
});
+
+test("Chinese estimates count Han pronunciation units, including names outside the basic Unicode plane", () => {
+ assert.equal(detectSpeechEstimateLanguage("𠮷"), "zh");
+ assert.equal(countSpeechEstimateUnits("這段中文有節奏。", "zh"), 7);
+ assert.equal(countSpeechEstimateUnits("𠮷野家", "zh"), 3);
+ assert.equal(countSpeechEstimateUnits("二零二六年", "zh"), 5);
+ assert.equal(countSpeechEstimateUnits("用 video 做视频", "zh"), 7);
+});
+
+test("Chinese mixed-language punctuation preserves English word boundaries", () => {
+ for (const separator of [" ", ",", ",", "/", "。", ":"]) {
+ assert.equal(countSpeechEstimateUnits(`用 idea${separator}video 做视频`, "zh"), 10);
+ }
+ const source = sealText("用 idea,video 做视频");
+ assert.equal(estimateSpeechDuration(source, {
+ language: "zh", rate: 5, rounding: "none", paddingSec: 0.5,
+ }), 2.5);
+});
diff --git a/packages/video-cli/README.md b/packages/video-cli/README.md
index 5832ac21..ce490493 100644
--- a/packages/video-cli/README.md
+++ b/packages/video-cli/README.md
@@ -37,14 +37,23 @@ Result or state. It names the Endpoint and its price page before it runs, and wr
to the chosen file. `measure` estimates a passage locally:
```bash
-hypit transcribe reference.mp4 --to notes/reference.transcript.json [--language en|zh|es]
+hypit transcribe reference.mp4 --to notes/reference.transcript.json --language en
hypit measure main.svml --segment hook --language en --pace normal --rounding round
```
+For `transcribe`, set `--language` to the spoken language: `en`, `zh` or `es`. Chinese speech uses `zh`, including
+Chinese speech containing English names; the requested language selects the recognition/alignment
+model, independently of the eventual caption font or script's simplified/traditional characters.
+
`transcribe` uses the Profile's `whisperx-alignment` Endpoint (after
extracting 16 kHz mono speech audio with ffmpeg). `measure` counts a Segment's pronunciation units at a
-delivery policy and prints the seconds to write as the literal `duration`; it opens no Profile and
-spends nothing. `--runtime <profile>` names the Profile; otherwise the project's `hypit runtime use`
+delivery policy and prints estimated seconds, the resolved rate, padding and rounding. Choose the
+literal `duration` from that estimate and the intended performance. `measure` opens no Profile and
+spends nothing. It accepts `--pace slow|normal|fast` or `--rate <units/s>`; JSON includes the resolved
+`rate` even when using a named pace. [Estimate](../estimate/README.md#units-and-delivery) explains
+Chinese/English units and choosing a whole-passage density.
+
+For `transcribe`, `--runtime <profile>` names the Profile; otherwise the project's `hypit runtime use`
selection is read. Anything the Author Graph declares as an output is a Build, however quickly it
comes back: pictures, clips and accepted voice references (`@hypit/mimo-speech`) carry the identity of the Source
that produced them, so they are declared in the Source and go through `plan` and `build`. To hear a
diff --git a/packages/video-cli/src/creation.ts b/packages/video-cli/src/creation.ts
index c65eee5d..535da6a4 100644
--- a/packages/video-cli/src/creation.ts
+++ b/packages/video-cli/src/creation.ts
@@ -9,6 +9,7 @@ import {
countSpeechEstimateUnits,
estimateSpeechDuration,
resolveSpeechEstimateLanguage,
+ resolveSpeechEstimateRate,
speechEstimatePolicyFromAttributes,
} from "@hypit/estimate";
import type { CanonicalValue, CapabilityRef, Need, StoredValue } from "@hypit/protocol";
@@ -275,8 +276,9 @@ async function transcribe(argv: readonly string[], io: CliIo, environment: Creat
const parsed = parseArguments(argv, ["--language", "--to", "--runtime", "--workspace"]);
assert(parsed.positionals.length === 1, "transcribe takes exactly one audio or video file");
const source = resolve(environment.cwd, parsed.positionals[0]!);
- const language = parsed.options.get("--language") ?? "en";
- assert(language === "en" || language === "zh" || language === "es", "--language must be en, zh or es");
+ const language = parsed.options.get("--language");
+ assert(language === "en" || language === "zh" || language === "es",
+ "transcribe requires --language en|zh|es for the spoken language (use --language zh for Chinese)");
const to = await destination(parsed, environment.cwd);
const { profile, host } = await environment.openHost(
parsed.options.get("--runtime"),
@@ -382,6 +384,7 @@ async function measure(argv: readonly string[], io: CliIo, environment: Creation
}
const pace = parsed.options.get("--pace");
const rate = parsed.options.get("--rate");
+ assert(pace === undefined || rate === undefined, "measure takes either --pace or --rate, not both");
const policy = speechEstimatePolicyFromAttributes({
language: parsed.options.get("--language") ?? "auto",
...(rate === undefined ? { pace: pace ?? "normal" } : { rate }),
@@ -391,6 +394,7 @@ async function measure(argv: readonly string[], io: CliIo, environment: Creation
const language = resolveSpeechEstimateLanguage(text, policy.language);
const units = countSpeechEstimateUnits(text, language);
const seconds = estimateSpeechDuration(sealText(text), policy);
+ const resolvedRate = resolveSpeechEstimateRate(policy, language);
const view = {
format: "hypit.video-cli-measure@1",
...where,
@@ -398,11 +402,13 @@ async function measure(argv: readonly string[], io: CliIo, environment: Creation
units,
language,
policy,
+ rate: resolvedRate,
seconds,
};
if (parsed.json) { io.write(`${JSON.stringify(view, null, 2)}\n`); return; }
- const delivery = policy.rate === undefined ? `${policy.pace} pace` : `${policy.rate} units/s`;
- io.write(`${seconds}s\n\n ${units} pronunciation units · ${language} · ${delivery} · rounding ${policy.rounding}\n`
+ const delivery = `${resolvedRate} units/s${policy.pace === undefined ? "" : ` (${policy.pace})`}`;
+ io.write(`${Number(seconds.toFixed(3))}s\n\n ${units} pronunciation units · ${language} · ${delivery}\n`
+ + ` padding ${policy.paddingSec ?? 0}s · rounding ${policy.rounding}\n`
+ ` ${"segment" in where ? `${where.segment} in ${where.source}` : `${text.length} characters`}\n`
+ " Choose the request duration from this estimate, the intended performance, and the selected model's supported values.\n");
}
@@ -416,7 +422,7 @@ export function writeCreationHelp(io: CliIo, topic?: CreationCommand): void {
"hypit transcribe",
"Establish word times with the whisperx-alignment Endpoint of the selected Runtime Profile.",
"",
- " hypit transcribe <audio|video> --to <transcript.json> [--language en|zh|es] [--runtime <profile>] [--workspace <project>]",
+ " hypit transcribe <audio|video> --to <transcript.json> --language en|zh|es [--runtime <profile>] [--workspace <project>]",
"",
"Extracts 16 kHz mono speech audio with ffmpeg and writes every word with its start and end in",
"seconds. One immediate request; no Build, Result or state.",
diff --git a/packages/video-cli/test/creation.test.ts b/packages/video-cli/test/creation.test.ts
index 47493e9c..79c3b230 100644
--- a/packages/video-cli/test/creation.test.ts
+++ b/packages/video-cli/test/creation.test.ts
@@ -96,15 +96,24 @@ test("measure prints the seconds a line takes so the author can write the litera
"measure", "--text", "Video editing begins with meaning, not a pile of clips on a timeline.",
"--language", "en", "--pace", "normal", "--rounding", "round", "--json",
], out.io, noHost);
- const view = JSON.parse(out.text()) as { readonly seconds: number; readonly units: number; readonly language: string };
+ const view = JSON.parse(out.text()) as { readonly seconds: number; readonly units: number; readonly language: string; readonly rate: number };
assert.equal(view.units, 20);
assert.equal(view.language, "en");
assert.equal(view.seconds, 4);
+ assert.equal(view.rate, 4.6);
const human = capture();
await runCreationCli(["measure", "--text", "hello world"], human.io, noHost);
assert.match(human.text(), /^\d+(\.\d+)?s\n/u);
assert.match(human.text(), /Choose the request duration/u);
+ assert.match(human.text(), /4\.6 units\/s \(normal\)/u);
+
+ const chinese = capture();
+ await runCreationCli(["measure", "--text", "这段中文有节奏", "--language", "zh", "--rate", "6", "--padding", "0.5"], chinese.io, noHost);
+ assert.match(chinese.text(), /^1\.667s/u);
+ assert.match(chinese.text(), /7 pronunciation units · zh · 6 units\/s/u);
+ assert.match(chinese.text(), /padding 0\.5s/u);
+ await assert.rejects(runCreationCli(["measure", "--text", "Hello", "--pace", "fast", "--rate", "6"], capture().io, noHost), /either --pace or --rate/u);
for (const [text, seconds] of [["Hello.", 2 / 4.6], [Array.from({ length: 460 }, () => "day").join(" "), 100]] as const) {
const measured = capture();
@@ -135,7 +144,7 @@ test("a Profile that does not serve the capability stops before anything is spen
} }),
};
await assert.rejects(
- runCreationCli(["transcribe", "speech.wav", "--to", "speech.json"], capture().io, unserved),
+ runCreationCli(["transcribe", "speech.wav", "--to", "speech.json", "--language", "en"], capture().io, unserved),
/No Endpoint in .*hypit\.runtime\.json serves @hypit\/whisperx@1#whisperx-alignment; hypit plan --runtime/u,
);
assert.equal(invoked, false);
@@ -143,3 +152,34 @@ test("a Profile that does not serve the capability stops before anything is spen
await rm(root, { recursive: true, force: true });
}
});
+
+test("transcribe requires a spoken language before opening a host", async () => {
+ const noHost: CreationEnvironment = { cwd: "/tmp", openHost: async () => { throw new Error("host must not open"); } };
+ await assert.rejects(runCreationCli(["transcribe", "speech.wav", "--to", "speech.json"], capture().io, noHost),
+ /requires --language en\|zh\|es/u);
+});
+
+test("Chinese transcription explicitly sends zh and retains individual character windows", async () => {
+ const root = await mkdtemp(join(tmpdir(), "hypit-transcribe-zh-"));
+ try {
+ await writeFile(join(root, "speech.wav"), wav(32_000));
+ const selected = host([]);
+ const env: CreationEnvironment = { cwd: root, openHost: async () => ({ profile: join(root, "runtime.json"), host: {
+ ...selected,
+ invoke: async (need) => {
+ const request = need.constraints as unknown as WhisperXAlignmentRequest;
+ assert.equal(request.language, "zh");
+ return { value: { kind: "inline", value: canonicalize({ passages: interpretWhisperXTranscript({
+ segments: [{ words: [{ word: "你", start: 0.1, end: 0.24 }, { word: "好!", start: 0.5, end: 1.1 }] }],
+ }, request.sampleFrames) }) } };
+ },
+ } }) };
+ await runCreationCli(["transcribe", "speech.wav", "--language", "zh", "--to", "speech.json"], capture().io, env);
+ const result = JSON.parse(await readFile(join(root, "speech.json"), "utf8"));
+ assert.equal(result.language, "zh");
+ assert.deepEqual(result.passages[0].words, [
+ { text: "你", start_seconds: 0.1, end_seconds: 0.24 },
+ { text: "好!", start_seconds: 0.5, end_seconds: 1.1 },
+ ]);
+ } finally { await rm(root, { recursive: true, force: true }); }
+});
diff --git a/skills/hypit/references/creation/reference-video.md b/skills/hypit/references/creation/reference-video.md
index 59d66c57..ab96f12b 100644
--- a/skills/hypit/references/creation/reference-video.md
+++ b/skills/hypit/references/creation/reference-video.md
@@ -48,6 +48,11 @@ reveals and emphasis. Keep that transcript beside the reference and interpret it
surrounding argument, visible names and supplied context. Small transcription errors can coexist
with a clear understanding of what the passage means.
+Pass the reference's spoken language explicitly: `--language zh` for Chinese, `en` for English,
+or `es` for Spanish. A Chinese passage can contain English brands and names while still using `zh`.
+Its timed Chinese characters help locate a phrase precisely; group those characters into meaningful
+phrases when describing the reference and writing the new Script.
+
[Environment selection](../environment/profile.md#choose-the-practical-capability-path-with-the-user)
explains assessing local preparation and choosing the local or hosted path for this transcription.
@@ -78,7 +83,7 @@ their changes and handoffs.
```bash
hypit transcribe references/ad/source.mp4 \
- --to references/ad/transcript.json
+ --language en --to references/ad/transcript.json
hypit media tile references/ad/source.mp4 --start 0 --end 12 --every 1 \
--to references/ad/evidence/opening.jpg
diff --git a/skills/hypit/references/creation/script-and-time.md b/skills/hypit/references/creation/script-and-time.md
index 60d5dd2c..c61bcc97 100644
--- a/skills/hypit/references/creation/script-and-time.md
+++ b/skills/hypit/references/creation/script-and-time.md
@@ -126,6 +126,14 @@ name, including other Segments, so separately generated performances receive the
The right-hand wording reaches the model through the Segment's `.dialogue`; Caption keeps the
display spelling. Settle these readings before measuring the Script and requesting its performance.
+The same distinction helps Chinese copy express numbers and names clearly. For example,
+`今年<2026|二零二六>年` displays the year compactly while specifying how it is said;
+`只要<¥19.9|十九块九>` chooses a conversational price reading. Choose the spoken form for this
+sentence's meaning and delivery. Measuring the Segment with `--language zh` then uses that spoken
+wording, including the syllables hidden behind its compact numeric display. Caption retains the
+authored simplified or traditional characters; transcription supplies timing rather than rewriting
+the displayed Script.
+
## Use the Script's deliberate projections
One Script publishes the full Narrative and the narrow views needed by the rest of the work:
@@ -219,10 +227,18 @@ estimates how many seconds they need at the chosen language and pace, using loca
```bash
hypit measure path/to/source.svml --segment opening --language en --pace normal
hypit measure --text "You expect me to type every coffee?" --language en --pace normal --rounding ceil
+hypit measure path/to/source.svml --segment opening --language zh --pace fast --rounding round
```
-Use `--pace slow|normal|fast`, or `--rate` for pronunciation units per second; `--padding` adds time
-for the intended pause or action. Keep related performances at a coherent delivery density. Initial
+Choose `--pace slow|normal|fast` from the intended delivery: conversational explanation may suit
+`normal`, while brisk, tightly cut social delivery often suits `fast`. The CLI prints the actual
+units-per-second rate; `--rate` lets you choose it directly. Mandarin counts Han characters as
+approximate syllables and embedded English by syllable; English counts syllables rather than words.
+Set `--language zh` for Chinese copy, including copy with English product names, and measure the
+pronunciation side of Dual Text with `--segment`.
+
+The density includes ordinary phrasing pauses. `--padding` reserves additional time for an intended
+reaction, demonstration or held pause. Keep related performances at a coherent delivery density. Initial
estimates can retain fractions with `--rounding none`. `ceil` rounds upward to a whole second, while
`round` chooses the nearest one. Most video requests use whole seconds; choose the final literal with
the selected model's supported values and the intended performance in mind.