Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

fix(script): separate Latin words from adjacent CJK characters

kashorin32574cb6e876
diff --git a/packages/script/README.md b/packages/script/README.md
index 762b833b..bac9d1c5 100644
--- a/packages/script/README.md
+++ b/packages/script/README.md
@@ -99,6 +99,12 @@ Dual Text preserves display spelling while supplying an explicit pronunciation.
Unit is indivisible for Caption timing and authored Cue Breaks. Roles are lexical speaking cues;
the Source's model references and action direction bind them to the intended performers.
+English words and numbers normally form lexical units; Han characters form individual units, as do
+Hiragana and Katakana characters. A Latin name adjacent to Han text remains separate from the
+following characters. Display punctuation attaches to neighboring words without adding timing units.
+These units support precise timing and highlighting. A Caption Cue can hold a whole phrase of them;
+`||` chooses its handoff independently of character counts or visual line wrapping.
+
## Selection and Moment affinity
A marker selects the adjacent semantic anchor; it does not write a timecode. Inside a spoken
diff --git a/packages/script/src/lexical.ts b/packages/script/src/lexical.ts
index 89c8711d..73642677 100644
--- a/packages/script/src/lexical.ts
+++ b/packages/script/src/lexical.ts
@@ -3,8 +3,14 @@ export type LexicalUnit = {
readonly index: number;
};
-const LEXICAL_UNIT =
- /(?:\p{N}{1,3}(?:[,,]\p{N}{3})+|\p{N}+)(?:[..]\p{N}+)?(?:-\p{N}+(?:[..]\p{N}+)?)*(?!\p{N}|-[\p{L}\p{M}])|[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}]\p{M}*|[\p{L}\p{M}\p{N}]+(?:['’.-][\p{L}\p{M}\p{N}]+)*/gu;
+const CHARACTER_UNIT = String.raw`[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}]`;
+// A Latin name beside Han prose ends at the script boundary, even without an authored space.
+const WORD_CHARACTER = String.raw`(?:(?!${CHARACTER_UNIT})[\p{L}\p{M}\p{N}])`;
+const LEXICAL_UNIT = new RegExp([
+ String.raw`(?:\p{N}{1,3}(?:[,,]\p{N}{3})+|\p{N}+)(?:[..]\p{N}+)?(?:-\p{N}+(?:[..]\p{N}+)?)*(?!\p{N}|-[\p{L}\p{M}])`,
+ String.raw`${CHARACTER_UNIT}\p{M}*`,
+ String.raw`${WORD_CHARACTER}+(?:['’.-]${WORD_CHARACTER}+)*`,
+].join("|"), "gu");
const OPENING_PUNCTUATION = new Set([
"(", "[", "{", "(", "【", "《", "「", "『", "〔", "〈", "“", "‘",
diff --git a/packages/script/test/script.test.ts b/packages/script/test/script.test.ts
index 3873dc65..fab497ca 100644
--- a/packages/script/test/script.test.ts
+++ b/packages/script/test/script.test.ts
@@ -130,6 +130,18 @@ test("Caption punctuation assigns ASCII quotes to the enclosed display words", (
]);
});
+test("Mixed-script brand names preserve following character units and authored Cue breaks", () => {
+ const parsed = parseScript("mixed-script.svml", "<line>用Hypit生成视频,||我的AI助手做3个视频。</line>");
+ const document = captionDocument(parsed, "story.caption", "story");
+ assert.deepEqual(parsed.tokens.map((token) => token.text), [
+ "用", "Hypit", "生", "成", "视", "频", "我", "的", "AI", "助", "手", "做", "3", "个", "视", "频",
+ ]);
+ assert.deepEqual(document.words.slice(0, 6).map((word) => word.text), [
+ "用", "Hypit", "生", "成", "视", "频,",
+ ]);
+ assert.equal(document.cueBreaks[0]?.afterUnitId, document.units[5]?.id);
+});
+
test("Script keeps ordinary compounds and formatted numbers lexical", () => {
const parsed = parseScript(
"punctuation-compounds.svml",