Explainer videos and product demos made by your AI agent. Free and open source: a local voice (Kokoro), word timing (Whisper) and a canvas renderer turn a script into a narrated MP4.
JavaScript ai-agentsanimationcanvasclaude-codecodexexplainer-videoffmpegjavascriptkokoroproduct-demotext-to-speechvideo-generationwhisper
Narrate in Spanish, French, Italian, Portuguese and Hindi (#14)
Vincent Schmalbach23f2c0283e67
diff --git a/AGENTS.md b/AGENTS.mdindex 1d8fed1..5b973f7 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -34,10 +34,11 @@ npm installnode bin/explainroo.js doctor --fetch```-`doctor` checks Node, ffmpeg and Chrome. It also downloads the speech models,-about 400 MB, the first time only. If the user ran `npm link`, you can write-`explainroo` instead of `node bin/explainroo.js` in the commands below. Every-command takes the project folder first and accepts `--json`.+`doctor` checks Node, ffmpeg, Chrome and espeak-ng. It also downloads the+English speech models, about 400 MB, the first time only. Other languages+fetch their speech check model on first use. If the user ran `npm link`, you+can write `explainroo` instead of `node bin/explainroo.js` in the commands+below. Every command takes the project folder first and accepts `--json`.### When to ask the user@@ -295,9 +296,58 @@ speech check reports a mismatch, make script.md say what was actually said.Replacing a scene recording makes that scene again. Replacing a whole takemakes all its scenes again.-ElevenLabs narration and recordings must be in English for now. The word-timing model understands English only. Run `voice`, `check` and `render`-as usual with any of the three sources.+ElevenLabs narration and recordings work in every supported language.+Run `voice`, `check` and `render` as usual with any of the three sources.++## Videos in other languages++Set `language` in `video.json` to `en`, `es`, `fr`, `it`, `pt-br` or `hi`.+English is the default. `pt-br` means Brazilian Portuguese. You can also+start with `explainroo init videos/my-video --language es`.++When `voice` is left out, the language picks its Kokoro voice:++| Language | Default voice |+|---|---|+| English (`en`) | `af_heart` |+| Spanish (`es`) | `ef_dora` |+| French (`fr`) | `ff_siwis` |+| Italian (`it`) | `if_sara` |+| Brazilian Portuguese (`pt-br`) | `pf_dora` |+| Hindi (`hi`) | `hf_alpha` |++Run `explainroo voices --language es` to see voices for one language.+A Kokoro voice must match the video's language. If a project already has+`voice` set, change it too, or remove it to use the default.++Write the narration in that language as a person would say it. Do not+translate English word for word. The writing rules above apply in spirit.+Cue words and `[#markers]` work the same. `at: 'qué'` matches "Qué".+Hindi sentences can end with `।`. Captions and subtitle files keep the+written text. The looks include a Hindi font, so no system font is needed.++Kokoro needs espeak-ng once for these five languages. Install it on Linux+with `sudo apt install espeak-ng`, or on macOS with `brew install espeak-ng`.+On Windows, use the installer from the+[espeak-ng releases page](https://github.com/espeak-ng/espeak-ng/releases).+If the program is in another place, set `EXPLAINROO_ESPEAK` to its binary.+English, ElevenLabs and recordings do not need it.++The first speech check in another language downloads multilingual Whisper+small. `doctor --fetch` still fetches only the English models. `verify`+uses the video's language to check the finished sound too.++If a symbol, short form or word sounds wrong, use `{shown|spoken}` in+`script.md` and run `voice` again. Put the pronunciation in the same+language. The speech check accepts digits for numbers. It does not try to+match spelled numbers to digits outside English. Write the digits, or use+an override whose spoken text has digits when that helps the check.+Japanese and Mandarin are not supported.++In Hindi, the speech check confirms fewer words than in the other languages,+because the speech model knows Hindi less well. A few unconfirmed words are+normal there. Fix a word when it stays unconfirmed in several scenes, or+when the user hears it wrong.## Images@@ -575,7 +625,7 @@ project.```videos/<name>/- video.json look, size, voice, music, captions, watermark+ video.json look, size, language, voice, music, captions, watermarkscript.md the narration, one "## scene-id" per scenescenes.js one drawing function per sceneassets/ images: screenshots, logos, generated illustrations@@ -591,7 +641,8 @@ videos/<name>/| `theme` | `paper` | a built-in look, or a relative path to a `.json` look file (see Custom looks) || `size` | `16:9` | a platform (`youtube`, `shorts`, `tiktok`, `reels`, `vertical`, `instagram`, `linkedin`, `square`), `16:9`, `9:16`, `1:1`, `4:5` or `WIDTHxHEIGHT` || `fps` | 30 | 24, 25, 30, 50 or 60 |-| `voice` | `af_heart` | a Kokoro id or a provider object for Kokoro, ElevenLabs or a recording; see Voices |+| `language` | `en` | narration language: `en`, `es`, `fr`, `it`, `pt-br` or `hi`; see Videos in other languages |+| `voice` | the language's voice | a Kokoro id or a provider object for Kokoro, ElevenLabs or a recording; English uses `af_heart`; see Voices || `speed` | 0.9 for Kokoro, 1 otherwise | voice speed only, 0.6 to 1.6; ElevenLabs limits speed times pace to 0.7 through 1.2 || `pace` | 1 | speed of the whole video (voice, pauses, animations), 0.7 to 1.6 || `music` | `true` | `true` (the look's style), `warm`, `upbeat`, `calm`, `tech`, `playful`, `{ "style", "volume" }` or `false` |@@ -667,7 +718,7 @@ when `ink` or `muted` has too little contrast against `bg`.| Command | What it does ||---|---|-| `init <dir>` | creates a project (`--theme`, `--size`, `--pace`, `--voice`, `--title`) |+| `init <dir>` | creates a project (`--theme`, `--size`, `--pace`, `--language`, `--voice`, `--title`) || `voice [project]` | makes the narration and the word times, saved per scene so only changed scenes are made again || `preview [project]` | a live preview in the browser that reloads when you save || `still [project] [times]` | PNG pictures at `12.5`, `scene`, `[email protected]` or `scene@end` (`--transparent` keeps alpha) |@@ -678,10 +729,10 @@ when `ink` or `muted` has too little contrast against `bg`.| `capture [project] <url>` | saves screenshots, wording, colors, fonts and logos from a live site (`--mobile`, `--json`) || `image [project] <name> "<prompt>"` | makes an illustration with OpenRouter || `images [project]` | lists the images and what they cost |-| `voices`, `say "text"` | lists the 28 Kokoro voices, or makes a sample |+| `voices`, `say "text"` | lists Kokoro voices with their languages (`--language es` filters voices), or makes a sample in the chosen voice's language || `voices [project] --elevenlabs` | lists the names, ids and labels available to the key (`--json` also works) || `themes`, `formats`, `icons <word>` | lists the looks and the sizes, searches the 1,854 icons |-| `doctor` | checks the setup (`--fetch` downloads the speech models) |+| `doctor` | checks Node, ffmpeg, Chrome, espeak-ng and models (`--fetch` downloads only English models); missing espeak-ng is allowed |The extra files share the video's name, also with `--out`: `video.srt`,`video.vtt`, `video.cover.jpg` and `video.chapters.txt`. A GIF has no sound,diff --git a/engine/captions.js b/engine/captions.jsindex ae71278..6ed71af 100644--- a/engine/captions.js+++ b/engine/captions.js@@ -21,7 +21,7 @@ export function buildPhrases(timeline, maxChars) {if (!cur) cur = { words: [], start: g.start };cur.words.push(g);cur.end = g.end;- if (/[.!?;:]["'’)]*$/.test(g.text) || (/,$/.test(g.text) && cur.words.length >= 4)) flush();+ if (/[.!?;:।]["'’)]*$/.test(g.text) || (/,$/.test(g.text) && cur.words.length >= 4)) flush();});flush();}diff --git a/engine/runtime.js b/engine/runtime.jsindex 0a84f3b..5b276f1 100644--- a/engine/runtime.js+++ b/engine/runtime.js@@ -26,6 +26,7 @@ async function loadFonts(theme, config) {// The UI kit (s.ui) draws app screens in Inter and headlines in Instrument Serif.wanted.add('Inter');wanted.add('Instrument Serif');+ wanted.add('Noto Sans Devanagari');const faces = [];const add = (family, url, o) => {const face = new FontFace(family, `url(${url})`, { weight: String(o.weight ?? 400), style: o.style ?? 'normal', unicodeRange: o.unicodeRange, display: 'block' });diff --git a/engine/themes.js b/engine/themes.jsindex a98aca8..72532d4 100644--- a/engine/themes.js+++ b/engine/themes.js@@ -199,7 +199,7 @@ export function fontString(theme, role = 'body', size = 48, weight) {const f = theme.fonts[role] || theme.fonts.body;const w = weight ?? f.weight;const fallback = role === 'mono' ? 'monospace' : 'sans-serif';- return `${w} ${Math.round(size * 100) / 100}px "${f.family}", ${fallback}`;+ return `${w} ${Math.round(size * 100) / 100}px "${f.family}", "Noto Sans Devanagari", ${fallback}`;}// Background textures are rendered once and reused on every frame.diff --git a/engine/ui.js b/engine/ui.jsindex ddeb25f..0094420 100644--- a/engine/ui.js+++ b/engine/ui.js@@ -121,7 +121,7 @@ export class UI {}fontString(size, weight = 400, family = this.font, italic = false) {- return `${italic ? 'italic ' : ''}${weight} ${size}px "${family}", Inter, sans-serif`;+ return `${italic ? 'italic ' : ''}${weight} ${size}px "${family}", Inter, "Noto Sans Devanagari", sans-serif`;}_family(f) {diff --git a/engine/util.js b/engine/util.jsindex a0d4eca..2f9525a 100644--- a/engine/util.js+++ b/engine/util.js@@ -197,6 +197,6 @@ export function normWord(w) {.normalize('NFKD').replace(/[̀-ͯ]/g, '').replace(/['’]/g, '')- .replace(/[^a-z0-9%$€£+#@.]/g, '')+ .replace(/[^\p{L}\p{M}\p{N}%$€£+#@.]/gu, '').replace(/^\.+|\.+$/g, '');}diff --git a/fonts/NotoSansDevanagari-400.ttf b/fonts/NotoSansDevanagari-400.ttfnew file mode 100644index 0000000..cb4957aBinary files /dev/null and b/fonts/NotoSansDevanagari-400.ttf differdiff --git a/fonts/NotoSansDevanagari-700.ttf b/fonts/NotoSansDevanagari-700.ttfnew file mode 100644index 0000000..669f369Binary files /dev/null and b/fonts/NotoSansDevanagari-700.ttf differdiff --git a/fonts/fonts.json b/fonts/fonts.jsonindex 88c62c6..21b46f6 100644--- a/fonts/fonts.json+++ b/fonts/fonts.json@@ -198,5 +198,19 @@"subset": "latin","file": "InstrumentSerif-400-italic-latin.woff2","unicodeRange": "U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD"+ },+ {+ "family": "Noto Sans Devanagari",+ "weight": 400,+ "subset": "devanagari",+ "file": "NotoSansDevanagari-400.ttf",+ "unicodeRange": "U+0900-097F,U+1CD0-1CFF,U+200C-200D,U+20B9,U+25CC,U+A8E0-A8FF"+ },+ {+ "family": "Noto Sans Devanagari",+ "weight": 700,+ "subset": "devanagari",+ "file": "NotoSansDevanagari-700.ttf",+ "unicodeRange": "U+0900-097F,U+1CD0-1CFF,U+200C-200D,U+20B9,U+25CC,U+A8E0-A8FF"}]diff --git a/fonts/licenses/NotoSansDevanagari-OFL.txt b/fonts/licenses/NotoSansDevanagari-OFL.txtnew file mode 100644index 0000000..63426f0--- /dev/null+++ b/fonts/licenses/NotoSansDevanagari-OFL.txt@@ -0,0 +1,118 @@+Copyright 2010, 2012-2020 Google Inc.+Copyright 2015-2020 Google LLC.++SIL OPEN FONT LICENSE Version 1.1++PREAMBLE++The goals of the Open Font License (OFL) are to stimulate+worldwide development of collaborative font projects,+to support the font creation efforts+of academic and linguistic communities,+and to provide a free and open framework+in which fonts may be shared and improved+in partnership with others.++The OFL allows the licensed fonts to be used, studied,+modified and redistributed freely+as long as they are not sold by themselves.+The fonts, including any derivative works,+can be bundled, embedded,+redistributed and/or sold with any software+provided that any reserved names are not used+by derivative works.+The fonts and derivatives, however,+cannot be released under any other type of license.+The requirement for fonts to remain under this license+does not apply to any document+created using the fonts or their derivatives.++DEFINITIONS++"Font Software" refers to the set of files+released by the Copyright Holder(s) under this license+and clearly marked as such.+This may include source files, build scripts and documentation.++"Reserved Font Name" refers to any names specified as such+after the copyright statement(s).++"Original Version" refers to the collection+of Font Software components+as distributed by the Copyright Holder(s).++"Modified Version" refers to any derivative+made by adding to, deleting, or substituting --+in part or in whole --+any of the components of the Original Version,+by changing formats+or by porting the Font Software to a new environment.++"Author" refers to any designer, engineer, programmer,+technical writer or other person+who contributed to the Font Software.++PERMISSION & CONDITIONS++Permission is hereby granted, free of charge,+to any person obtaining a copy of the Font Software,+to use, study, copy, merge, embed, modify, redistribute,+and sell modified and unmodified copies of the Font Software,+subject to the following conditions:++1) Neither the Font Software nor any of its individual components,+in Original or Modified Versions,+may be sold by itself.++2) Original or Modified Versions of the Font Software+may be bundled, redistributed and/or sold with any software,+provided that each copy contains the above copyright notice+and this license.+These can be included either as stand-alone text files,+human-readable headers or in the appropriate+machine-readable metadata fields within text or binary files+as long as those fields can be easily viewed by the user.++3) No Modified Version of the Font Software may use+the Reserved Font Name(s)+unless explicit written permission is granted+by the corresponding Copyright Holder.+This restriction only applies to the primary font name+as presented to the users.++4) The name(s) of the Copyright Holder(s)+or the Author(s) of the Font Software+shall not be used to promote,+endorse or advertise any Modified Version,+except to acknowledge the contribution(s)+of the Copyright Holder(s) and the Author(s)+or with their explicit written permission.++5) The Font Software,+modified or unmodified, in part or in whole,+must be distributed entirely under this license,+and must not be distributed under any other license.+The requirement for fonts to remain under this license+does not apply to any document+created using the Font Software.++TERMINATION++This license becomes null and void+if any of the above conditions are not met.++DISCLAIMER++THE FONT SOFTWARE IS PROVIDED "AS IS",+WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,+INCLUDING BUT NOT LIMITED TO+ANY WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT+OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT.+IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE+FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,+INCLUDING ANY GENERAL, SPECIAL, INDIRECT,+INCIDENTAL, OR CONSEQUENTIAL DAMAGES,+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,+ARISING FROM, OUT OF THE USE OR INABILITY TO USE+THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE.diff --git a/scripts/fetch-fonts.mjs b/scripts/fetch-fonts.mjsindex 0c80bf8..0ea8b12 100644--- a/scripts/fetch-fonts.mjs+++ b/scripts/fetch-fonts.mjs@@ -48,11 +48,18 @@ for (const [family, weights] of FAMILIES) {console.log(`${file} ${(buf.length / 1024).toFixed(1)} KB`);}}+for (const [weight, style] of [[400, 'Regular'], [700, 'Bold']]) {+ const file = `NotoSansDevanagari-${weight}.ttf`;+ const res = await fetch(`https://raw.githubusercontent.com/notofonts/noto-fonts/main/hinted/ttf/NotoSansDevanagari/NotoSansDevanagari-${style}.ttf`);+ if (!res.ok) throw new Error(`no ${file} (${res.status})`);+ fs.writeFileSync(path.join(dir, file), Buffer.from(await res.arrayBuffer()));+ manifest.push({ family: 'Noto Sans Devanagari', weight, subset: 'devanagari', file, unicodeRange: 'U+0900-097F,U+1CD0-1CFF,U+200C-200D,U+20B9,U+25CC,U+A8E0-A8FF' });+}fs.writeFileSync(path.join(dir, 'fonts.json'), JSON.stringify(manifest, null, 2) + '\n');// SIL Open Font License texts, one per family.fs.mkdirSync(path.join(dir, 'licenses'), { recursive: true });-for (const [family] of FAMILIES) {+for (const [family] of [...FAMILIES, ['Noto Sans Devanagari']]) {const slug = family.toLowerCase().replace(/\s+/g, '');const res = await fetch(`https://raw.githubusercontent.com/google/fonts/main/ofl/${slug}/OFL.txt`);if (!res.ok) throw new Error(`no OFL.txt for ${family} (${res.status})`);diff --git a/src/align.js b/src/align.jsindex e6cadd2..e27b6c2 100644--- a/src/align.js+++ b/src/align.js@@ -35,7 +35,12 @@ export function intWords(n) {// Whisper writes "$1,050" and "5%" where the voice said "one thousand and// fifty dollars" and "five percent", and "example.com" or "1.1.1.1" where it// said words joined by "dot". Both sides are expanded the same way.-export function expand(word) {+export function expand(word, language = 'en') {+ if (language !== 'en') {+ const n = normWord(word).replace(/[०-९]/g, d => String(d.charCodeAt(0) - 0x0966));+ const digits = /^[\p{N}%$€£+.]+$/u.test(n) ? n.replace(/[^\p{N}]/gu, '') : '';+ return n ? [digits || n] : [];+ }const raw = String(word).trim().toLowerCase();// Whisper splits "1.1.1.1" into "1", ".1", ".1", ".1": a leading dot is a spoken "dot".const lead = /^\.[a-z0-9]/i.test(raw) ? ['dot'] : [];@@ -90,14 +95,16 @@ export function similar(a, b) {// words: [{ spoken }] in order. heard: [{ text, start, end }] from Whisper.// Returns per-word { start, end, matched } and the share of spoken tokens matched.-export function alignWords(words, heard, duration) {+export function alignWords(words, heard, duration, language = 'en') {+ const partsOf = word => expand(word, language);+ const matches = (a, b) => language !== 'en' && (/\p{N}/u.test(a) || /\p{N}/u.test(b)) ? a === b : similar(a, b);const tokens = [];words.forEach((w, wi) => {- const parts = String(w.spoken).split(/\s+/).flatMap(expand);+ const parts = String(w.spoken).split(/\s+/).flatMap(partsOf);(parts.length ? parts : ['']).forEach((p, k) => {// "and" inside a spoken number ("one thousand and fifty") is optional.const prev = tokens[tokens.length - 1];- const optional = p === 'and' && prev && isNumberToken(prev.norm);+ const optional = language === 'en' && p === 'and' && prev && isNumberToken(prev.norm);tokens.push({ wi, norm: p, first: k === 0, last: k === parts.length - 1 || !parts.length, optional });});});@@ -117,7 +124,7 @@ export function alignWords(words, heard, duration) {}const H = [];for (const h of joined) {- const parts = expand(h.text);+ const parts = partsOf(h.text);const step = (h.end - h.start) / Math.max(1, parts.length);parts.forEach((p, k) => H.push({ text: p, norm: p, start: h.start + k * step, end: h.start + (k + 1) * step }));}@@ -144,16 +151,16 @@ export function alignWords(words, heard, duration) {if (i < T && c + (tokens[i].optional ? 0 : 1) < cost[i + 1][j]) { cost[i + 1][j] = c + (tokens[i].optional ? 0 : 1); move[i + 1][j] = 2; }if (j < N && c + 1 < cost[i][j + 1]) { cost[i][j + 1] = c + 1; move[i][j + 1] = 3; }if (i < T && j < N) {- const ok = similar(tokens[i].norm, H[j].norm);+ const ok = matches(tokens[i].norm, H[j].norm);const v = c + (ok ? 0 : 1.6);if (v < cost[i + 1][j + 1]) { cost[i + 1][j + 1] = v; move[i + 1][j + 1] = ok ? 1 : 4; }}if (i < T && j < N && tokens[i].first && groupEnd[i] > i) {const e = groupEnd[i];const joined = tokens.slice(i, e + 1).map((t) => t.norm).join('');- if (similar(joined, H[j].norm) && c < cost[e + 1][j + 1]) { cost[e + 1][j + 1] = c; move[e + 1][j + 1] = 5; }+ if (matches(joined, H[j].norm) && c < cost[e + 1][j + 1]) { cost[e + 1][j + 1] = c; move[e + 1][j + 1] = 5; }}- if (i < T && j + 1 < N && similar(tokens[i].norm, H[j].norm + H[j + 1].norm) && c < cost[i + 1][j + 2]) {+ if (i < T && j + 1 < N && matches(tokens[i].norm, H[j].norm + H[j + 1].norm) && c < cost[i + 1][j + 2]) {cost[i + 1][j + 2] = c;move[i + 1][j + 2] = 6;}diff --git a/src/cli.js b/src/cli.jsindex 4ae5102..e25721d 100644--- a/src/cli.js+++ b/src/cli.js@@ -8,7 +8,8 @@ import { prepare, makeState } from './pipeline.js';import { render } from './render.js';import { check, stills, sheet, verify } from './qa.js';import { startServer, ROOT } from './server.js';-import { VOICES, loadTTS, loadASR, cacheRoot, ttsDtype, TTS_SAMPLE_RATE } from './models.js';+import { VOICES, LANGUAGES, voiceLanguage, loadTTS, loadASR, cacheRoot, ttsDtype, TTS_SAMPLE_RATE, MULTILINGUAL_ASR_MODEL } from './models.js';+import { espeakVersion, phonemize, generateSpeech } from './phonemes.js';import { writeWav } from './wav.js';import { ffmpegVersion } from './ffmpeg.js';import { findChrome } from './browser.js';@@ -23,7 +24,7 @@ const HELP = `explainroo ${VERSION}: narrated explainer videos from codeUsage: explainroo <command> [project] [options]Make a video- init <dir> create a project (--theme, --size, --pace, --voice, --title)+ init <dir> create a project (--theme, --size, --pace, --language es, --voice, --title)voice [project] make the narration (Kokoro, ElevenLabs or a recording) and word timingspreview [project] live preview in your browser, reloads on save (--port)still [project] [t…] PNG stills: "12.5", "scene", "[email protected]" (--transparent, default: end of every scene)@@ -42,12 +43,12 @@ Images (optional, needs an OpenRouter API key)images [project] list generated images and what they costReference- voices [project] list Kokoro voices (--elevenlabs lists the voices of your ElevenLabs key)+ voices [project] list Kokoro voices (--language es for one language, --elevenlabs for your ElevenLabs key)say "text" hear a Kokoro voice (--voice am_michael, --speed 1, --out sample.wav)themes list looksformats list sizes for YouTube, Shorts, TikTok, Reels, Instagram, LinkedInicons <word…> search the 1,800+ built-in icons- doctor check ffmpeg, Chrome and the speech models (--fetch downloads them)+ doctor check ffmpeg, Chrome, espeak-ng and models (--fetch downloads English models)Options: --json for machine-readable output. The project defaults to the current folder.Docs: AGENTS.md (for coding agents) and https://www.explainroo.com/docs/`;@@ -191,11 +192,14 @@ export async function main(argv) {return 0;}case 'voices': {+ const language = textFlag(flags, 'language');+ if (language !== undefined) normalizeConfig({ language });if (flags.elevenlabs) {const voices = await elevenVoices(path.resolve(pos[0] || '.'));print(voices.map((v) => `${v.name} ${v.id} ${Object.entries(v.labels).map(([k, x]) => `${k}: ${x}`).join(', ')}`).join('\n'), { voices });} else {- print(Object.entries(VOICES).map(([k, v]) => `${k.padEnd(12)} ${v}`).join('\n') + '\nUse voices --elevenlabs for ElevenLabs, or set voice to { "provider": "recording", "src": "assets/narration.wav" } in video.json.', { voices: VOICES });+ const voices = Object.fromEntries(Object.entries(VOICES).filter(([id]) => language === undefined || voiceLanguage(id) === language));+ print(Object.entries(voices).map(([k, v]) => `${k.padEnd(12)} ${voiceLanguage(k).padEnd(6)} ${v}`).join('\n') + '\nUse voices --elevenlabs for ElevenLabs, or set voice to { "provider": "recording", "src": "assets/narration.wav" } in video.json.', { voices, languages: Object.fromEntries(Object.keys(voices).map(id => [id, voiceLanguage(id)])) });}return 0;}@@ -250,18 +254,21 @@ function textFlag(flags, name) {}function init(dir, flags, print) {- if (!dir) throw new ProjectError('usage: explainroo init <dir> [--theme paper] [--size youtube|tiktok|linkedin|...] [--pace 1.2] [--voice af_heart] [--title "..."]');+ if (!dir) throw new ProjectError('usage: explainroo init <dir> [--theme paper] [--size youtube|tiktok|linkedin|...] [--pace 1.2] [--language es] [--voice af_heart] [--title "..."]');const target = path.resolve(dir);if (fs.existsSync(path.join(target, 'video.json'))) throw new ProjectError(`${target} already has a video.json`);const theme = textFlag(flags, 'theme') || 'paper';if (!THEMES.includes(theme)) throw new ProjectError(`theme must be one of ${THEMES.join(', ')}`);- const voice = textFlag(flags, 'voice') || 'af_heart';+ const language = textFlag(flags, 'language') ?? 'en';+ normalizeConfig({ language });+ const voice = textFlag(flags, 'voice') || LANGUAGES[language];if (!VOICES[voice]) throw new ProjectError(`voice "${voice}" does not exist. Run "explainroo voices".`);const title = textFlag(flags, 'title') || path.basename(target).replace(/[-_]+/g, ' ').replace(/^\w/, (c) => c.toUpperCase());const tpl = path.join(ROOT, 'templates', 'starter');const size = textFlag(flags, 'size') || '16:9';resolveSize(size);const config = { title, theme, size, voice, music: true, captions: 'auto' };+ if (flags.language !== undefined) config.language = language;if (flags.pace !== undefined) config.pace = num(flags.pace, 'pace');normalizeConfig(config);// Create the folder only after every setting checks out, so a bad flag@@ -329,7 +336,9 @@ async function say(text, flags, log, print) {const voice = textFlag(flags, 'voice') || 'af_heart';if (!VOICES[voice]) throw new ProjectError(`voice "${voice}" does not exist. Run "explainroo voices".`);const tts = await loadTTS({ log });- const audio = await tts.generate(text, { voice, speed: num(flags.speed, 'speed') ?? 1 });+ const language = voiceLanguage(voice);+ const phonemes = language === 'en' ? undefined : (await phonemize([text], language))[0];+ const audio = await generateSpeech(tts, text, { language, voice, speed: num(flags.speed, 'speed') ?? 1, phonemes });const out = path.resolve(textFlag(flags, 'out') || `${voice}.wav`);writeWav(out, audio.audio, audio.sampling_rate || TTS_SAMPLE_RATE);print(out, { file: out, voice });@@ -373,6 +382,8 @@ async function doctor(flags, log, print) {add('node', nodeOk, `v${process.versions.node}${nodeOk ? '' : ' (need 20.11 or newer)'}`);const ff = ffmpegVersion();add('ffmpeg', !!ff, ff || 'not found; install ffmpeg or set EXPLAINROO_FFMPEG');+ const espeak = espeakVersion();+ add('espeak-ng', true, espeak || 'not found; only needed for other languages');try {add('chrome', true, findChrome());} catch (e) {@@ -387,6 +398,7 @@ async function doctor(flags, log, print) {}add('voice model', has('Kokoro-82M-v1.0-ONNX'), has('Kokoro-82M-v1.0-ONNX') ? `Kokoro 82M (${ttsDtype()}) in ${models}` : 'not downloaded yet; "explainroo doctor --fetch" or the first voice run fetches it');add('speech check model', has('whisper-base.en_timestamped'), has('whisper-base.en_timestamped') ? 'Whisper base.en' : 'not downloaded yet; fetched on first use');+ add('language model', has(MULTILINGUAL_ASR_MODEL.split('/')[1]), 'Whisper small for es, fr, it, pt-br and hi; downloaded on first use, not by --fetch');const ok = rows.every((r) => r.ok || r.name.includes('model'));print(rows.map((r) => `${r.ok ? 'ok ' : 'FAIL'} ${r.name.padEnd(20)} ${r.detail}`).join('\n'), { ok, checks: rows });return ok ? 0 : 1;diff --git a/src/licenses/misaki-Apache-2.0.txt b/src/licenses/misaki-Apache-2.0.txtnew file mode 100644index 0000000..261eeb9--- /dev/null+++ b/src/licenses/misaki-Apache-2.0.txt@@ -0,0 +1,201 @@+ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work+ (an example is provided in the Appendix below).++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONS++ APPENDIX: How to apply the Apache License to your work.++ To apply the Apache License to your work, attach the following+ boilerplate notice, with the fields enclosed by brackets "[]"+ replaced with your own identifying information. (Don't include+ the brackets!) The text should be enclosed in the appropriate+ comment syntax for the file format. We also recommend that a+ file or class name and description of purpose be included on the+ same "printed page" as the copyright notice for easier+ identification within third-party archives.++ Copyright [yyyy] [name of copyright owner]++ Licensed under the Apache License, Version 2.0 (the "License");+ you may not use this file except in compliance with the License.+ You may obtain a copy of the License at++ http://www.apache.org/licenses/LICENSE-2.0++ Unless required by applicable law or agreed to in writing, software+ distributed under the License is distributed on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ See the License for the specific language governing permissions and+ limitations under the License.diff --git a/src/models.js b/src/models.jsindex bd9cbf6..edd1d38 100644--- a/src/models.js+++ b/src/models.js@@ -6,8 +6,21 @@ import path from 'node:path';export const TTS_MODEL = 'onnx-community/Kokoro-82M-v1.0-ONNX';export const ASR_MODEL = 'onnx-community/whisper-base.en_timestamped';+export const MULTILINGUAL_ASR_MODEL = 'onnx-community/whisper-small_timestamped';export const TTS_SAMPLE_RATE = 24000;+export const LANGUAGES = {+ en: 'af_heart', es: 'ef_dora', fr: 'ff_siwis', it: 'if_sara', 'pt-br': 'pf_dora', hi: 'hf_alpha',+};++export function voiceLanguage(id) {+ return { a: 'en', b: 'en', e: 'es', f: 'fr', i: 'it', p: 'pt-br', h: 'hi' }[id[0]];+}++export function asrModel(language = 'en') {+ return language === 'en' ? ASR_MODEL : MULTILINGUAL_ASR_MODEL;+}+export function cacheRoot() {return process.env.EXPLAINROO_CACHE || path.join(os.homedir(), '.cache', 'explainroo');}@@ -62,31 +75,34 @@ export function loadTTS({ log } = {}) {return ttsPromise;}-let asrPromise = null;-export function loadASR({ log } = {}) {- if (!asrPromise) {- asrPromise = (async () => {+const asrPromises = new Map();+export function loadASR({ log, language = 'en' } = {}) {+ const model = asrModel(language);+ if (!asrPromises.has(model)) {+ asrPromises.set(model, (async () => {const { pipeline } = await lib();- return pipeline('automatic-speech-recognition', ASR_MODEL, {+ return pipeline('automatic-speech-recognition', model, {dtype: 'q8',progress_callback: progressLogger('speech check model', log),});- })();+ })());}- return asrPromise;+ return asrPromises.get(model);}// samples: Float32Array at 16 kHz. Returns [{ text, start, end }].-export async function transcribeWords(samples, { log } = {}) {- const asr = await loadASR({ log });- const r = await asr(samples, { return_timestamps: 'word', chunk_length_s: 30, stride_length_s: 5 });+export async function transcribeWords(samples, { log, language = 'en' } = {}) {+ const asr = await loadASR({ log, language });+ const r = await asr(samples, { return_timestamps: 'word', chunk_length_s: 30, stride_length_s: 5,+ ...(language === 'en' ? {} : { language: language === 'pt-br' ? 'pt' : language, task: 'transcribe' }),+ });return {text: (r.text || '').trim(),words: (r.chunks || []).map((c) => ({ text: c.text.trim(), start: c.timestamp[0] ?? 0, end: c.timestamp[1] ?? c.timestamp[0] ?? 0 })),};}-// Kokoro v1.0 English voices.+// Kokoro v1.0 voices for the supported languages.export const VOICES = {af_heart: 'American English, female (default)',af_bella: 'American English, female',@@ -116,4 +132,17 @@ export const VOICES = {bm_fable: 'British English, male',bm_lewis: 'British English, male',bm_daniel: 'British English, male',+ ef_dora: 'Spanish, female (default)',+ em_alex: 'Spanish, male',+ em_santa: 'Spanish, male',+ ff_siwis: 'French, female (default)',+ if_sara: 'Italian, female (default)',+ im_nicola: 'Italian, male',+ pf_dora: 'Brazilian Portuguese, female (default)',+ pm_alex: 'Brazilian Portuguese, male',+ pm_santa: 'Brazilian Portuguese, male',+ hf_alpha: 'Hindi, female (default)',+ hf_beta: 'Hindi, female',+ hm_omega: 'Hindi, male',+ hm_psi: 'Hindi, male',};diff --git a/src/phonemes.js b/src/phonemes.jsnew file mode 100644index 0000000..1526070--- /dev/null+++ b/src/phonemes.js@@ -0,0 +1,68 @@+import { spawn, spawnSync } from 'node:child_process';++const ESPEAK_LANGUAGES = { es: 'es', fr: 'fr-fr', it: 'it', 'pt-br': 'pt-br', hi: 'hi' };+// Adapted from misaki's EspeakG2P (Apache-2.0, see licenses/).+// https://github.com/hexgrad/misaki/blob/main/misaki/espeak.py+// Kokoro v1.0 keeps nasal vowels, unlike misaki's v2.0 mapping.+const E2M = {+ 'a^ɪ': 'I', 'a^ʊ': 'W', 'd^z': 'ʣ', 'd^ʒ': 'ʤ', 'e^ɪ': 'A',+ 'o^ʊ': 'O', 'ə^ʊ': 'Q', 's^s': 'S', 't^s': 'ʦ', 't^ʃ': 'ʧ', 'ɔ^ɪ': 'Y',+};++export function mapPhonemes(ipa) {+ let ps = ipa.trim();+ for (const [old, replacement] of Object.entries(E2M).sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0)) ps = ps.replaceAll(old, replacement);+ return ps.replaceAll('^', '').replaceAll('-', '').replaceAll('«', '(').replaceAll('»', ')');+}++function binary() {+ return process.env.EXPLAINROO_ESPEAK || 'espeak-ng';+}++const INSTALL = 'espeak-ng is needed for this language. Linux: sudo apt install espeak-ng. macOS: brew install espeak-ng. Windows: use the installer at https://github.com/espeak-ng/espeak-ng/releases. Or set EXPLAINROO_ESPEAK to its binary.';++export function espeakVersion() {+ const r = spawnSync(binary(), ['--version'], { encoding: 'utf8', timeout: 5000 });+ return !r.error && r.status === 0 ? r.stdout.trim().split('\n')[0] : null;+}++export async function phonemize(texts, language) {+ const lang = ESPEAK_LANGUAGES[language];+ if (!lang) throw new Error('espeak-ng supports es, fr, it, pt-br and hi here.');+ const input = [];+ const parts = texts.map(text => {+ // Preserve punctuation outside espeak, as misaki's phonemizer backend does.+ const prepared = text.replaceAll('«', '“').replaceAll('»', '”').replaceAll('(', '«').replaceAll(')', '»');+ return prepared.split(/([;:,.!?¡¿—…"«»“”{}\[\]।]+|\s*\n\s*)/u).map((part, i) => {+ if (i % 2 || !part.trim()) return { text: part };+ const index = input.length;+ input.push(part.trim());+ return { index, before: /^\s*/.exec(part)[0], after: /\s*$/.exec(part)[0] };+ });+ });+ if (!input.length) return parts.map(p => mapPhonemes(p.map(x => x.text).join('')));+ const output = await new Promise((resolve, reject) => {+ // Without --stdin, espeak processes each input line separately.+ const child = spawn(binary(), ['-q', '-b', '1', '-v', lang, '--ipa=1', '--tie=^'], { stdio: ['pipe', 'pipe', 'pipe'] });+ let stdout = '';+ let stderr = '';+ child.stdout.setEncoding('utf8').on('data', data => { stdout += data; });+ child.stderr.setEncoding('utf8').on('data', data => { stderr += data; });+ child.on('error', e => reject(new Error(['ENOENT', 'EACCES'].includes(e.code) ? INSTALL : 'espeak-ng could not start. Check EXPLAINROO_ESPEAK.')));+ child.stdin.on('error', () => {});+ child.on('close', code => {+ if (code !== 0) return reject(new Error('espeak-ng could not read the text. Check its installation and EXPLAINROO_ESPEAK.'));+ const lines = stdout.trimEnd().split(/\r?\n/);+ if (lines.length !== input.length || lines.some(line => !line.trim()) || stderr.trim()) return reject(new Error('espeak-ng returned incomplete phonemes. Check its installation and EXPLAINROO_ESPEAK.'));+ resolve(lines.map(line => line.replace(/\([a-z-]+\)/g, '').trim()));+ });+ child.stdin.end(input.join('\n') + '\n');+ });+ return parts.map(p => mapPhonemes(p.map(x => x.index === undefined ? x.text : x.before + output[x.index] + x.after).join('')));+}++export function generateSpeech(tts, text, { language = 'en', voice, speed, phonemes } = {}) {+ if (language === 'en') return tts.generate(text, { voice, speed });+ const { input_ids } = tts.tokenizer(phonemes, { truncation: true });+ return tts.generate_from_ids(input_ids, { voice, speed });+}diff --git a/src/project.js b/src/project.jsindex e8fabbd..61c5342 100644--- a/src/project.js+++ b/src/project.js@@ -6,7 +6,7 @@import fs from 'node:fs';import path from 'node:path';import { parseScript } from './script.js';-import { VOICES } from './models.js';+import { VOICES, LANGUAGES, voiceLanguage } from './models.js';import { THEMES as LOOKS } from '../engine/themes.js';import { parseColor } from '../engine/util.js';@@ -36,6 +36,7 @@ export const DEFAULTS = {theme: 'paper',size: '16:9',fps: 30,+ language: 'en',voice: 'af_heart',speed: 0.9,pace: 1,@@ -159,6 +160,8 @@ export function resolveSize(size) {export function normalizeConfig(raw, dir = '.') {const cfg = { ...DEFAULTS, ...raw };+ if (typeof cfg.language !== 'string' || !Object.hasOwn(LANGUAGES, cfg.language)) fail(`language must be one of ${Object.keys(LANGUAGES).join(', ')}`);+ if (raw.voice === undefined) cfg.voice = LANGUAGES[cfg.language];const unknown = Object.keys(raw).filter((k) => !(k in DEFAULTS) && !k.startsWith('_'));if (unknown.length) fail(`video.json has unknown setting(s): ${unknown.join(', ')}. Known: ${Object.keys(DEFAULTS).join(', ')}`);if (!THEMES.includes(cfg.theme) && !lookPath(cfg.theme)) fail(`theme must be one of ${THEMES.join(', ')} or a relative path to a .json look file, not "${cfg.theme}"`);@@ -182,6 +185,7 @@ export function normalizeConfig(raw, dir = '.') {}if (typeof cfg.voice === 'string') {if (!Object.hasOwn(VOICES, cfg.voice)) fail(`voice "${cfg.voice}" does not exist. Run "explainroo voices" for the list.`);+ if (voiceLanguage(cfg.voice) !== cfg.language) fail(`voice "${cfg.voice}" does not speak ${cfg.language}. Use ${Object.keys(VOICES).filter(id => voiceLanguage(id) === cfg.language).join(', ')}.`);} else if (!cfg.voice || typeof cfg.voice !== 'object') fail('voice must be a Kokoro id or an object with provider');if (typeof cfg.voice !== 'string' && raw.speed === undefined) cfg.speed = 1;if (!(cfg.speed >= 0.6 && cfg.speed <= 1.6)) fail('speed must be between 0.6 and 1.6');diff --git a/src/qa.js b/src/qa.jsindex 3bee7ef..f4f28a6 100644--- a/src/qa.js+++ b/src/qa.js@@ -15,7 +15,7 @@ export async function check(project, { log = () => {}, step = 0.25, viewWidth =const issues = chapterHints(buildChapters(timeline, project.script.title || project.config.title));for (const sc of project.script.scenes) {const v = voices[sc.id];- const suspicious = v ? v.unmatched.filter((w) => w.replace(/[^\p{L}\p{N}]/gu, '').length > 3) : [];+ const suspicious = v ? v.unmatched.filter((w) => w.replace(/[^\p{L}\p{M}\p{N}]/gu, '').length > 3) : [];if (v && v.duration && (v.matchRate < 0.9 || suspicious.length)) {issues.push({ level: 'warn', scene: sc.id, t: null, message: `the voice check could not confirm ${suspicious.length ? suspicious.slice(0, 8).map((w) => `"${w}"`).join(', ') : `${Math.round((1 - v.matchRate) * 100)}% of the words`}. It heard: "${v.transcript.slice(0, 200)}". ${project.config.voice.provider === 'recording' ? 'The recording does not match script.md here. Make script.md say what was actually said.' : 'If a word is mispronounced, write it as {shown|spoken} in script.md.'}` });}@@ -148,9 +148,9 @@ export async function verify(project, { file = null, log = () => {} } = {}) {if (!words.length) continue;const from = Math.max(0, Math.floor(t.start * 16000));const to = Math.min(samples.length, Math.ceil((t.start + t.dur) * 16000));- const heard = await transcribeWords(samples.subarray(from, to), { log });+ const heard = await transcribeWords(samples.subarray(from, to), { log, language: project.config.language });texts.push(heard.text);- const rate = alignWords(words, heard.words, t.dur).matchRate;+ const rate = alignWords(words, heard.words, t.dur, project.config.language).matchRate;scenes.push({ id: t.id, matchRate: Math.round(rate * 1000) / 1000 });total += words.length;matched += rate * words.length;@@ -159,9 +159,9 @@ export async function verify(project, { file = null, log = () => {} } = {}) {const rate = total ? matched / total : 1;speech = { matchRate: Math.round(rate * 1000) / 1000, transcript: texts.join(' '), scenes };} else {- const heard = await transcribeWords(samples, { log });+ const heard = await transcribeWords(samples, { log, language: project.config.language });const script = project.script.scenes.flatMap(spokenWords);- speech = { matchRate: Math.round(alignWords(script, heard.words, duration).matchRate * 1000) / 1000, transcript: heard.text };+ speech = { matchRate: Math.round(alignWords(script, heard.words, duration, project.config.language).matchRate * 1000) / 1000, transcript: heard.text };}if (speech.matchRate < 0.85) issues.push({ level: 'warn', message: `only ${Math.round(speech.matchRate * 100)}% of the script was understood in the final mix; lower music.volume or check pronunciation` });fs.rmSync(wav, { force: true });diff --git a/src/recordings.js b/src/recordings.jsindex 61da60d..0dcd978 100644--- a/src/recordings.js+++ b/src/recordings.js@@ -72,9 +72,9 @@ export async function saveRecording(file, samples, sampleRate) {}}-export function recordingCuts(scenes, heard, duration) {+export function recordingCuts(scenes, heard, duration, language = 'en') {const spoken = scenes.map((scene) => ({ scene, words: scene.units.filter((u) => u.type === 'word') })).filter((s) => s.words.length);- const aligned = alignWords(spoken.flatMap((s) => s.words), heard, duration).words;+ const aligned = alignWords(spoken.flatMap((s) => s.words), heard, duration, language).words;let offset = 0;const spans = spoken.map(({ scene, words }) => {const times = aligned.slice(offset, offset + words.length);diff --git a/src/script.js b/src/script.jsindex ff067f8..53b35ed 100644--- a/src/script.js+++ b/src/script.js@@ -22,7 +22,7 @@ export function normWord(w) {.normalize('NFKD').replace(/[̀-ͯ]/g, '').replace(/['’]/g, '')- .replace(/[^a-z0-9%$€£+#@.]/g, '')+ .replace(/[^\p{L}\p{M}\p{N}%$€£+#@.]/gu, '').replace(/^\.+|\.+$/g, '');}@@ -61,7 +61,7 @@ export function tokenizeNarration(text, line = 0) {const n = text.length;const pushWord = (display, spoken) => {const prev = units[units.length - 1];- if (!pendingSpace && prev && prev.type === 'word' && /^[.,;:!?)\]"'’…]+$/.test(display)) {+ if (!pendingSpace && prev && prev.type === 'word' && /^[.,;:!?।)\]"'’…]+$/.test(display)) {prev.display += display;prev.spoken += spoken;return;@@ -223,7 +223,7 @@ export function speechChunks(units, { sentenceGap = 0.3, paragraphGap = 0.55, pa}cur.text += (cur.words.length ? ' ' : '') + u.spoken;cur.words.push({ unitIndex: index, display: u.display, spoken: u.spoken });- if (/[.!?…]["'’)\]]*$/.test(u.spoken)) {+ if (/[.!?…।]["'’)\]]*$/.test(u.spoken)) {close();gapBefore = sentenceGap / pace;}diff --git a/src/subtitles.js b/src/subtitles.jsindex cfdc881..3a3ee72 100644--- a/src/subtitles.js+++ b/src/subtitles.js@@ -1,6 +1,6 @@// Subtitle cues use the shown words and their times from the voice timeline.-const sentenceEnd = /[.!?…]["'’”)\]]*$/;-const punctuation = /[,;:.!?…]["'’”)\]]*$/;+const sentenceEnd = /[.!?…।]["'’”)\]]*$/;+const punctuation = /[,;:.!?…।]["'’”)\]]*$/;function lines(words) {const text = words.map((w) => w.text).join(' ');diff --git a/src/voice.js b/src/voice.jsindex 1f54bc3..f50c017 100644--- a/src/voice.js+++ b/src/voice.js@@ -6,7 +6,8 @@ import path from 'node:path';import crypto from 'node:crypto';import { speechChunks, resolveMarks } from './script.js';import { alignWords } from './align.js';-import { loadTTS, transcribeWords, TTS_MODEL, ASR_MODEL, TTS_SAMPLE_RATE, ttsDtype } from './models.js';+import { loadTTS, transcribeWords, TTS_MODEL, TTS_SAMPLE_RATE, ttsDtype, asrModel } from './models.js';+import { phonemize, generateSpeech } from './phonemes.js';import { writeWav, readWav, resample, trimSilence } from './wav.js';import { elevenSpeech } from './elevenlabs.js';import { recordingFiles, decodeRecording, saveRecording, recordingCuts } from './recordings.js';@@ -19,7 +20,7 @@ export function sceneHash(scene, chunks, config, recording = null) {.update(JSON.stringify({PIPELINE,TTS_MODEL,- ASR_MODEL,+ ASR_MODEL: asrModel(config.language),dtype: ttsDtype(),voice: config.voice,speed: config.speed * config.pace,@@ -29,6 +30,7 @@ export function sceneHash(scene, chunks, config, recording = null) {layout: (recording ? scene.units.filter((u) => u.type !== 'pause') : scene.units).map((u) => (u.type === 'mark' ? `#${u.name}` : u.type === 'word' ? u.display : '')).join('|'),// Added last and only for recordings, so Kokoro scenes keep their cache....(recording ? { recording } : {}),+ ...(config.language && config.language !== 'en' ? { language: config.language } : {}),})).digest('hex').slice(0, 16);@@ -84,8 +86,8 @@ export async function synthesize(project, { force = false, log = () => {}, onlyif (needsSpeech && recordings && !recordings.folder) {log('transcribing the recording and finding scene boundaries');take = await decodeRecording(Object.values(recordings.files)[0].file, project.paths.build, wanted);- heardTake = await transcribeWords(resample(take.samples, take.sampleRate, 16000), { log });- cuts = recordingCuts(project.script.scenes, heardTake.words, take.samples.length / take.sampleRate);+ heardTake = await transcribeWords(resample(take.samples, take.sampleRate, 16000), { log, language: config.language });+ cuts = recordingCuts(project.script.scenes, heardTake.words, take.samples.length / take.sampleRate, config.language);}for (const { scene, chunks, hash } of todo) {const t0 = Date.now();@@ -122,6 +124,8 @@ async function synthesizeScene(project, scene, chunks, hash, tts, log) {let matchedTokens = 0;let totalTokens = 0;+ const phonemes = config.voice.provider !== 'elevenlabs' && config.language !== 'en'+ ? await phonemize(chunks.map(c => c.text), config.language) : null;for (let ci = 0; ci < chunks.length; ci++) {const chunk = chunks[ci];const gap = ci === 0 ? 0 : chunk.gapBefore;@@ -131,16 +135,16 @@ async function synthesizeScene(project, scene, chunks, hash, tts, log) {}const audio = config.voice.provider === 'elevenlabs'? await elevenSpeech(project, scene.id, chunk.text, { previous: chunks[ci - 1]?.text, next: chunks[ci + 1]?.text })- : await tts.generate(chunk.text, { voice: config.voice, speed: config.speed * config.pace });+ : await generateSpeech(tts, chunk.text, { language: config.language, voice: config.voice, speed: config.speed * config.pace, phonemes: phonemes?.[ci] });const samples = trimSilence(audio.audio, audio.sampling_rate || sr);const start = cursor / sr;const dur = samples.length / sr;pieces.push(samples);cursor += samples.length;- const heard = await transcribeWords(resample(samples, sr, 16000), { log });+ const heard = await transcribeWords(resample(samples, sr, 16000), { log, language: config.language });transcripts.push(heard.text);- const aligned = alignWords(chunk.words, heard.words, dur);+ const aligned = alignWords(chunk.words, heard.words, dur, config.language);chunk.words.forEach((w, k) => {const a = aligned.words[k];wordTimes[w.unitIndex] = { start: start + a.start, end: start + a.end, matched: a.matched };@@ -181,6 +185,7 @@ async function synthesizeScene(project, scene, chunks, hash, tts, log) {}async function recordingScene(project, scene, hash, source, take, cut, heardTake, log) {+ const { config } = project;const audio = take || await decodeRecording(source.file, project.paths.build, project.config.speed * project.config.pace);const sr = audio.sampleRate;let from = cut ? Math.floor(cut.start * sr) : 0;@@ -202,9 +207,9 @@ async function recordingScene(project, scene, hash, source, take, cut, heardTake};const heard = heardTake ? {words: heardTake.words.filter((w) => w.start >= cut.start && w.start < cut.end).map(local),- } : await transcribeWords(resample(samples, sr, 16000), { log });+ } : await transcribeWords(resample(samples, sr, 16000), { log, language: config.language });const units = scene.units.filter((u) => u.type === 'word');- const aligned = alignWords(units, heard.words, duration);+ const aligned = alignWords(units, heard.words, duration, config.language);const times = cut ? cut.words.map(local) : aligned.words;const wordTimes = [];let wi = 0;diff --git a/test/languages-browser.test.js b/test/languages-browser.test.jsnew file mode 100644index 0000000..2889004--- /dev/null+++ b/test/languages-browser.test.js@@ -0,0 +1,78 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import fs from 'node:fs';+import os from 'node:os';+import path from 'node:path';+import { findChrome } from '../src/browser.js';+import { loadProject, THEMES } from '../src/project.js';+import { buildTimeline } from '../src/timeline.js';+import { withEngine } from '../src/pipeline.js';++let chrome;+try { chrome = findChrome(); } catch {}++const sentences = [+ '¿Qué pasó? ¡El niño llegó al café!',+ 'La sœur prépare une leçon en français.',+ 'La città è bella e il caffè è pronto.',+ 'Não há açúcar, mas os pães são bons.',+ 'हिन्दी में शब्दों और तस्वीरों से जानकारी समझाएँ।',+];++test('every look uses bundled fonts for all five languages, including Devanagari in captions and UI', { skip: !chrome }, async t => {+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-language-fonts-'));+ t.after(() => fs.rmSync(dir, { recursive: true, force: true }));+ fs.writeFileSync(path.join(dir, 'script.md'), '## demo {min=3}');+ fs.writeFileSync(path.join(dir, 'scenes.js'), `export default { demo(s) { s.title('Qué', { at: 0 }); } };`);+ for (const theme of THEMES) {+ fs.writeFileSync(path.join(dir, 'video.json'), JSON.stringify({ theme, captions: true, sfx: false }));+ const project = loadProject(dir);+ await withEngine(project, buildTimeline(project, {}), {}, async page => {+ const cdp = await page.context().newCDPSession(page);+ await cdp.send('DOM.enable');+ await cdp.send('CSS.enable');+ const { root } = await cdp.send('DOM.getDocument');+ await page.evaluate(async ([sentences, theme]) => {+ const { getTheme, fontString } = await import('/engine/themes.js');+ const { UI } = await import('/engine/ui.js');+ const fonts = ['display', 'body', 'hand', 'mono'].map(role => fontString(getTheme(theme), role, 48));+ fonts.push(UI.prototype.fontString.call({ font: 'Inter' }, 48));+ let i = 0;+ for (const font of fonts) for (const sentence of sentences) {+ const span = document.createElement('div');+ span.id = 'glyph' + i++;+ span.style.font = font;+ span.textContent = sentence;+ document.body.append(span);+ }+ await new Promise(resolve => requestAnimationFrame(resolve));+ }, [sentences, theme]);+ for (let i = 0; i < 25; i++) {+ const { nodeId } = await cdp.send('DOM.querySelector', { nodeId: root.nodeId, selector: '#glyph' + i });+ const { fonts } = await cdp.send('CSS.getPlatformFontsForNode', { nodeId });+ assert.ok(fonts.length, `${theme} sentence ${i}`);+ assert.ok(fonts.every(f => f.isCustomFont), `${theme} sentence ${i}: ${JSON.stringify(fonts)}`);+ if (i % 5 === 4) assert.ok(fonts.some(f => f.familyName === 'Noto Sans Devanagari'), `${theme} Hindi fallback`);+ }+ });+ }+});++test('accented and Hindi cue words resolve to their spoken times in the page', { skip: !chrome }, async t => {+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-language-cues-'));+ t.after(() => fs.rmSync(dir, { recursive: true, force: true }));+ fs.writeFileSync(path.join(dir, 'video.json'), '{}');+ fs.writeFileSync(path.join(dir, 'script.md'), '## demo\nQué cœur pão città फ़ाइलें');+ fs.writeFileSync(path.join(dir, 'scenes.js'), `export default { demo(s) {+ s.text('Qué', { y: 200, at: 'qué' }); s.text('cœur', { y: 300, at: 'CŒUR' });+ s.text('pão', { y: 400, at: 'PÃO' }); s.text('città', { y: 500, at: 'CITTÀ' });+ s.text('फ़ाइलें', { y: 600, at: 'फ़ाइलें' });+ } };`);+ const project = loadProject(dir);+ const words = project.script.scenes[0].units.map((u, i) => ({ text: u.display, start: i * 0.5, end: (i + 1) * 0.5 }));+ const timeline = buildTimeline(project, { demo: { duration: 2.5, words, marks: {} } });+ await withEngine(project, timeline, {}, async page => {+ const result = await page.evaluate(() => window.explainroo.check());+ assert.deepEqual(result.filter(i => i.level === 'error'), []);+ });+});diff --git a/test/languages.test.js b/test/languages.test.jsnew file mode 100644index 0000000..081bc2a--- /dev/null+++ b/test/languages.test.js@@ -0,0 +1,133 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import fs from 'node:fs';+import os from 'node:os';+import path from 'node:path';+import crypto from 'node:crypto';+import { normWord, parseScript, speechChunks, tokenizeNarration } from '../src/script.js';+import { normWord as browserNorm } from '../engine/util.js';+import { alignWords, expand } from '../src/align.js';+import { normalizeConfig, ProjectError } from '../src/project.js';+import { VOICES, LANGUAGES, voiceLanguage, asrModel, ASR_MODEL, MULTILINGUAL_ASR_MODEL, TTS_MODEL, ttsDtype } from '../src/models.js';+import { sceneHash } from '../src/voice.js';+import { recordingCuts } from '../src/recordings.js';+import { subtitleCues, toSrt, toVtt } from '../src/subtitles.js';+import { buildPhrases } from '../engine/captions.js';+import { main } from '../src/cli.js';++test('word normalization keeps letters, marks and numbers across scripts, with the existing accent rules', () => {+ for (const [word, expected] of [+ ['Qué', 'que'], ['¿NIÑO?', 'nino'], ['français', 'francais'], ['cœur', 'cœur'],+ ['ação', 'acao'], ['pão', 'pao'], ['città', 'citta'], ['हिन्दी।', 'हिन्दी'],+ ['फ़ाइलें', 'फ़ाइलें'], ['१२३', '१२३'], ['किंतु', 'किंतु'], ['देवनागरी', 'देवनागरी'],+ ["Visitor’s", 'visitors'], ['example.com.', 'example.com'], ['$1,050.50', '$1050.50'], ['API+#@', 'api+#@'],+ ]) {+ assert.equal(normWord(word), expected, word);+ assert.equal(browserNorm(word), expected, word);+ assert.equal(normWord(word.normalize('NFD')), expected, word);+ }+});++test('sentence boundaries include danda and Spanish question and exclamation marks', () => {+ const words = tokenizeNarration('¿Qué pasa? ¡Todo bien! \"यह अच्छा है।\" फिर चलें।');+ const chunks = speechChunks(words);+ assert.deepEqual(chunks.map(c => c.text), ['¿Qué pasa?', '¡Todo bien!', '\"यह अच्छा है।\"', 'फिर चलें।']);+ assert.deepEqual(chunks.map(c => c.gapBefore), [0, 0.3, 0.3, 0.3]);+ assert.equal(tokenizeNarration('एक {दो|दो}।').length, 2);+});++test('each language picks its default and rejects every voice from a different language', () => {+ assert.equal(normalizeConfig({}).language, 'en');+ for (const [language, id] of Object.entries(LANGUAGES)) {+ assert.equal(normalizeConfig({ language }).voice, id);+ assert.equal(normalizeConfig({ language, voice: { provider: 'kokoro', id } }).voice, id);+ for (const voice of Object.keys(VOICES)) {+ if (voiceLanguage(voice) === language) assert.equal(normalizeConfig({ language, voice }).voice, voice);+ else assert.throws(() => normalizeConfig({ language, voice }), e => e instanceof ProjectError && e.message.includes(id));+ }+ for (const voice of [{ provider: 'elevenlabs', id: 'sample' }, { provider: 'recording', src: 'assets/take.wav' }]) {+ assert.equal(normalizeConfig({ language, voice }).language, language);+ }+ assert.equal(asrModel(language), language === 'en' ? ASR_MODEL : MULTILINGUAL_ASR_MODEL);+ }+ for (const language of ['ja', 'zh', 'pt', '', 'EN', 'toString', null, true, ['es']]) assert.throws(() => normalizeConfig({ language }), /language must be one of/);+});++test('non-English numbers stay digits, including Devanagari, and are never fuzzy matched or spelled out', () => {+ for (const language of ['es', 'fr', 'it', 'pt-br', 'hi']) {+ assert.deepEqual(expand('1,050', language), ['1050']);+ assert.deepEqual(expand('€1.050', language), ['1050']);+ assert.deepEqual(expand('१२%', language), ['12']);+ assert.deepEqual(expand('twenty', language), ['twenty']);+ const r = alignWords([{ spoken: '१२३' }, { spoken: '1050' }], [{ text: '123', start: 0, end: 0.5 }, { text: '1,050', start: 0.5, end: 1 }], 1, language);+ assert.equal(r.matchRate, 1);+ assert.equal(alignWords([{ spoken: '1234' }], [{ text: '1235', start: 0, end: 1 }], 1, language).matchRate, 0);+ assert.equal(alignWords([{ spoken: 'twenty' }], [{ text: '20', start: 0, end: 1 }], 1, language).matchRate, 0);+ }+ assert.deepEqual(expand('twenty'), ['20']);+});++test('Hindi recording cuts use the same language rules as per-scene alignment', () => {+ const scenes = parseScript('## first\n१२३\n## second\nफ़ाइलें').scenes;+ const cuts = recordingCuts(scenes, [{ text: '123', start: 0.2, end: 0.5 }, { text: 'फ़ाइलें', start: 1, end: 1.5 }], 2, 'hi');+ assert.ok(cuts.every(c => c.words[0].matched));+});++test('English cache hashes keep the previous format; other languages are part of the cache', () => {+ const scene = parseScript('## first\n[#a] Hello. Goodbye.').scenes[0];+ const chunks = speechChunks(scene.units);+ const config = normalizeConfig({});+ const previous = crypto.createHash('sha256').update(JSON.stringify({+ PIPELINE: 7, TTS_MODEL, ASR_MODEL, dtype: ttsDtype(), voice: config.voice, speed: config.speed * config.pace,+ chunks: chunks.map(c => [c.gapBefore, c.text]),+ layout: scene.units.map(u => u.type === 'mark' ? `#${u.name}` : u.type === 'word' ? u.display : '').join('|'),+ })).digest('hex').slice(0, 16);+ assert.equal(sceneHash(scene, chunks, config), previous);+ assert.equal(sceneHash(scene, chunks, { ...config, language: undefined }), previous);+ const voice = { provider: 'recording', src: 'assets/take.wav' };+ assert.notEqual(sceneHash(scene, chunks, normalizeConfig({ language: 'fr', voice })), sceneHash(scene, chunks, normalizeConfig({ language: 'es', voice })));+});++test('captions and subtitle files keep native text and split at danda', () => {+ for (const text of ['¿Qué pasó?', 'Le cœur écoute.', 'La città è bella.', 'Não há açúcar.', 'यह अच्छा है। फिर चलें।']) {+ const words = text.split(' ').map((text, i) => ({ text, start: i * 0.2, end: (i + 1) * 0.2 }));+ const timeline = { scenes: [{ start: 0, dur: 10, words }] };+ const cues = subtitleCues(timeline);+ assert.equal(cues.map(c => c.text.replaceAll('\n', ' ')).join(' '), text);+ assert.ok(toSrt(cues).includes(text.split(' ')[0]));+ assert.ok(toVtt(cues).includes(text.split(' ')[0]));+ if (text.includes('।')) {+ assert.equal(cues.length, 2);+ assert.equal(buildPhrases(timeline, 80).length, 2);+ }+ }+});++test('init stores the language and default voice; voices filters and doctor allows missing espeak', async t => {+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-language-cli-'));+ t.after(() => fs.rmSync(dir, { recursive: true, force: true }));+ let output = '';+ let errors = '';+ t.mock.method(process.stdout, 'write', s => { output += s; return true; });+ t.mock.method(process.stderr, 'write', s => { errors += s; return true; });+ assert.equal(await main(['init', path.join(dir, 'video'), '--language', 'es', '--json']), 0);+ const config = JSON.parse(fs.readFileSync(path.join(dir, 'video/video.json')));+ assert.equal(config.language, 'es');+ assert.equal(config.voice, 'ef_dora');+ for (const flags of [['--language', 'ja'], ['--language'], ['--language', 'es', '--voice', 'af_heart']]) {+ assert.equal(await main(['init', path.join(dir, 'bad'), ...flags]), 1);+ assert.equal(fs.existsSync(path.join(dir, 'bad')), false);+ }+ assert.match(errors, /--language needs a value/);+ output = '';+ assert.equal(await main(['voices', '--language', 'es', '--json']), 0);+ const list = JSON.parse(output);+ assert.deepEqual(Object.keys(list.voices), ['ef_dora', 'em_alex', 'em_santa']);+ assert.equal(list.languages.ef_dora, 'es');+ const saved = process.env.EXPLAINROO_ESPEAK;+ process.env.EXPLAINROO_ESPEAK = path.join(dir, 'missing');+ t.after(() => { if (saved === undefined) delete process.env.EXPLAINROO_ESPEAK; else process.env.EXPLAINROO_ESPEAK = saved; });+ output = '';+ assert.equal(await main(['doctor', '--json']), 0);+ assert.match(JSON.parse(output).checks.find(c => c.name === 'espeak-ng').detail, /only needed for other languages/);+});diff --git a/test/phonemes.test.js b/test/phonemes.test.jsnew file mode 100644index 0000000..5d6fde6--- /dev/null+++ b/test/phonemes.test.js@@ -0,0 +1,90 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import fs from 'node:fs';+import os from 'node:os';+import path from 'node:path';+import { mapPhonemes, phonemize, espeakVersion, generateSpeech } from '../src/phonemes.js';++function fake(t, source) {+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-espeak-'));+ const file = path.join(dir, 'fake espeak');+ fs.writeFileSync(file, '#!/usr/bin/env node\n' + source, { mode: 0o755 });+ const previous = process.env.EXPLAINROO_ESPEAK;+ process.env.EXPLAINROO_ESPEAK = file;+ t.after(() => {+ if (previous === undefined) delete process.env.EXPLAINROO_ESPEAK;+ else process.env.EXPLAINROO_ESPEAK = previous;+ fs.rmSync(dir, { recursive: true, force: true });+ });+ return file;+}++test('maps every misaki v1.0 tie sequence and leaves language-specific IPA intact', () => {+ assert.equal(mapPhonemes(' a^ɪ a^ʊ d^z d^ʒ e^ɪ o^ʊ ə^ʊ s^s t^s t^ʃ ɔ^ɪ '), 'I W ʣ ʤ A O Q S ʦ ʧ Y');+ assert.equal(mapPhonemes('«ʧˈit-t^sːa» r x ɬ ʲ œ̃ ɔ̃ ɑ̃ ɛ̃ ɐ̃ ʊ̃ n̩'), '(ʧˈitʦːa) r x ɬ ʲ œ̃ ɔ̃ ɑ̃ ɛ̃ ɐ̃ ʊ̃ n̩');+ assert.equal(mapPhonemes('कˈʌcʰcʰi'), 'कˈʌcʰcʰi');+});++test('batches text through stdin, preserves punctuation and removes language flags', async t => {+ fake(t, `+ if (process.argv.includes('--version')) { console.log('eSpeak NG 1.51'); process.exit(0); }+ const assert = require('node:assert/strict');+ assert.deepEqual(process.argv.slice(2), ['-q', '-b', '1', '-v', 'fr-fr', '--ipa=1', '--tie=^']);+ let text = '';+ process.stdin.on('data', d => text += d);+ process.stdin.on('end', () => {+ assert.equal(text, ['bonjour', 'merci', 'שלום $HOME \`echo bad\`', 'encore'].join('\\n') + '\\n');+ process.stdout.write('(en) t^ʃa\\nd^ʒa\\ne^ɪ\\no^ʊ\\n');+ });+ `);+ assert.equal(espeakVersion(), 'eSpeak NG 1.51');+ const result = await phonemize(['«bonjour», (merci)!', 'שלום $HOME `echo bad`\nencore'], 'fr');+ assert.deepEqual(result, ['“ʧa”, (ʤa)!', 'A\nO']);+});++test('punctuation-only text needs no program', async t => {+ const file = fake(t, 'process.exit(1);');+ fs.rmSync(file);+ assert.deepEqual(await phonemize(['¿?!', '(…)'], 'es'), ['¿?!', '(…)']);+ assert.equal(espeakVersion(), null);+});++test('missing and non-executable programs give install instructions for every platform', async t => {+ const file = fake(t, '');+ for (const missing of [false, true]) {+ if (missing) fs.rmSync(file);+ else fs.chmodSync(file, 0o644);+ await assert.rejects(phonemize(['Hola'], 'es'), e => {+ assert.match(e.message, /sudo apt install espeak-ng/);+ assert.match(e.message, /brew install espeak-ng/);+ assert.match(e.message, /espeak-ng\/espeak-ng\/releases/);+ assert.match(e.message, /EXPLAINROO_ESPEAK/);+ return true;+ });+ }+});++test('failed and incomplete output gives a friendly error', async t => {+ const file = fake(t, 'process.stderr.write("failure"); process.exit(2);');+ await assert.rejects(phonemize(['Hola'], 'es'), /could not read the text/);+ fs.writeFileSync(file, '#!/usr/bin/env node\nconsole.log("one");');+ await assert.rejects(phonemize(['Hola', 'Adiós'], 'es'), /incomplete phonemes/);+ await assert.rejects(phonemize(['こんにちは'], 'ja'), /supports es, fr, it, pt-br and hi/);+});++test('English keeps generate; other languages tokenize mapped phonemes and call generate_from_ids', async () => {+ const calls = [];+ const ids = { dims: [1, 4] };+ const tts = {+ generate(text, options) { calls.push(['english', text, options]); return 'audio'; },+ tokenizer(text, options) { calls.push(['tokens', text, options]); return { input_ids: ids }; },+ generate_from_ids(input, options) { calls.push(['ids', input, options]); return 'audio'; },+ };+ assert.equal(await generateSpeech(tts, 'Hello.', { voice: 'af_heart', speed: 0.9 }), 'audio');+ assert.equal(await generateSpeech(tts, 'Bonjour.', { language: 'fr', voice: 'ff_siwis', speed: 1, phonemes: 'bɔ̃ʒˈuʁ.' }), 'audio');+ assert.deepEqual(calls, [+ ['english', 'Hello.', { voice: 'af_heart', speed: 0.9 }],+ ['tokens', 'bɔ̃ʒˈuʁ.', { truncation: true }],+ ['ids', ids, { voice: 'ff_siwis', speed: 1 }],+ ]);+});