Skip to content

利用 AI 大模型和自动化工作流,根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.

feat: added native support for pauses on the video script

Ismael Camachod3f342c7a57b
diff --git a/README-en.md b/README-en.md
index bc51e9d2..8460f61a 100644
--- a/README-en.md
+++ b/README-en.md
@@ -174,6 +174,7 @@ Thanks to [Kimi](https://platform.kimi.ai?track_id=track-f6b0a640d35c41deb03b247
### Voiceover, Subtitles, and Background Music
- [x] Choose automatic voiceover, uploaded audio, or no voiceover, with voice samples and full narration previews
+- [x] Insert precise custom pauses in scripts using multi-language tags (e.g. `[pause: 2s]`, `[pausa: 2s]`, `[停顿: 2秒]`, `[silence: 1s]`), maintaining synchronized audio, subtitles, and video pacing
- [x] Use **Edge TTS (free, no API key required)**, Azure Speech, SiliconFlow, Google Gemini, Xiaomi MiMo, MiniMax, ElevenLabs, Chatterbox, Fish Audio, and other voice services
- [x] Generate subtitles and configure their font, position, color, size, outline, and background style
- [x] Use random, local, or AI-generated background music with independent volume control
diff --git a/README-ja.md b/README-ja.md
index ea8a75f4..083b0c35 100644
--- a/README-ja.md
+++ b/README-ja.md
@@ -174,6 +174,7 @@
### ナレーション・字幕・BGM
- [x] 自動ナレーション、音声ファイルのアップロード、音声なしの 3 つの方式に対応し、音声サンプルとナレーション全体をプレビュー可能
+- [x] スクリプト内で多言語の一時停止タグ(例:`[一時停止: 2秒]`、`[pause: 2s]`、`[ポーズ: 1.5s]`、`[無音: 1s]`)を使用して任意の無音時間を挿入可能。音声・字幕・映像のタイムラインが自動で正確に同期
- [x] **Edge TTS(無料・API キー不要)**、Azure Speech、SiliconFlow、Google Gemini、Xiaomi MiMo、MiniMax、ElevenLabs、Chatterbox、Fish Audio などの音声サービスに対応
- [x] 字幕を自動生成し、フォント、位置、色、サイズ、縁取り、背景スタイルを調整可能
- [x] ランダム、ローカル、AI 生成の BGM に対応し、音量を個別に調整可能
diff --git a/README.md b/README.md
index 89b40c18..f99bee9e 100644
--- a/README.md
+++ b/README.md
@@ -174,6 +174,7 @@
### 配音、字幕与配乐
- [x] 支持自动配音、上传配音和无配音三种方式,并提供音色试听与完整配音预览
+- [x] 支持在文案中使用多语言停顿标签(如 `[停顿: 2秒]`、`[pause: 2s]`、`[pausa: 2s]`、`[静音: 1s]`)插入精准静音间歇,音频、字幕与画面剪辑自动精确同步
- [x] 集成 **Edge TTS(免费、无需 API Key)**、Azure Speech、SiliconFlow、Google Gemini、小米 MiMo、MiniMax、ElevenLabs、Chatterbox 和 Fish Audio 等配音服务
- [x] 支持自动生成字幕,可调整字体、位置、颜色、大小、描边和背景样式
- [x] 支持随机、本地及 AI 生成背景音乐,并可独立控制音量
diff --git a/app/services/llm.py b/app/services/llm.py
index e29e5f40..90ca5b49 100644
--- a/app/services/llm.py
+++ b/app/services/llm.py
@@ -10,6 +10,7 @@ from openai.types.chat import ChatCompletion
from app.config import config
from app.models.llm_provider import DEFAULT_LLM_PROVIDER_ID, get_llm_provider
+from app.utils import utils
_max_retries = 5
MIN_SCRIPT_PARAGRAPH_NUMBER = 1
@@ -603,6 +604,7 @@ def generate_terms(
match_script_order: bool = False,
app_config=None,
) -> List[str]:
+ video_script = utils.remove_pause_tags(video_script or "").strip()
if match_script_order:
goal = (
f"Generate {amount} chronological stock-video search terms that follow "
diff --git a/app/services/task.py b/app/services/task.py
index 9e4bd98c..776229f0 100644
--- a/app/services/task.py
+++ b/app/services/task.py
@@ -317,7 +317,7 @@ def generate_terms(task_id, params, video_script):
# 无法改善“后面内容的画面提前出现”的问题。
video_terms = llm.generate_terms(
video_subject=params.video_subject,
- video_script=video_script,
+ video_script=utils.remove_pause_tags(video_script),
amount=8 if params.match_materials_to_script else 5,
match_script_order=params.match_materials_to_script,
)
diff --git a/app/services/voice.py b/app/services/voice.py
index 053191c7..d0699dd8 100644
--- a/app/services/voice.py
+++ b/app/services/voice.py
@@ -7,12 +7,13 @@ import math
import os
import queue
import re
+import shutil
import subprocess
import tempfile
import threading
import time
import unicodedata
-from datetime import datetime
+from datetime import datetime, timedelta
from typing import Union
from urllib.parse import urlparse
from xml.sax.saxutils import escape, unescape
@@ -20,6 +21,7 @@ from xml.sax.saxutils import escape, unescape
import edge_tts
import requests
from edge_tts import SubMaker
+from edge_tts.srt_composer import Subtitle
from loguru import logger
from moviepy.video.tools import subtitles
from moviepy.audio.io.AudioFileClip import AudioFileClip
@@ -452,7 +454,7 @@ def generate_silent_audio(duration_seconds: float, output_file: str) -> bool:
return True
-def tts(
+def _single_tts(
text: str,
voice_name: str,
voice_rate: float,
@@ -553,6 +555,181 @@ def tts(
return azure_tts_v1(text, voice_name, voice_rate, voice_file)
+def _concat_audio_files(audio_files: list[str], output_file: str) -> bool:
+ """使用 FFmpeg concat 滤镜拼接多个音频文件,自动统一重采样并避免爆音/漂移。"""
+ if not audio_files:
+ return False
+ ensure_file_path_exists(output_file)
+ if len(audio_files) == 1:
+ if audio_files[0] != output_file:
+ shutil.copyfile(audio_files[0], output_file)
+ return True
+
+ ffmpeg_binary = utils.get_ffmpeg_binary()
+ inputs = []
+ filter_inputs = []
+ for idx, f in enumerate(audio_files):
+ inputs.extend(["-i", f])
+ filter_inputs.append(f"[{idx}:a]")
+
+ filter_complex = f"{''.join(filter_inputs)}concat=n={len(audio_files)}:v=0:a=1[a]"
+ command = [
+ ffmpeg_binary,
+ "-y",
+ *inputs,
+ "-filter_complex",
+ filter_complex,
+ "-map",
+ "[a]",
+ "-codec:a",
+ "libmp3lame",
+ "-q:a",
+ "4",
+ output_file,
+ ]
+ result = subprocess.run(command, capture_output=True, text=True, check=False)
+ if result.returncode != 0:
+ logger.error(
+ "failed to concat audio files with ffmpeg: "
+ f"{(result.stderr or result.stdout or '').strip()}"
+ )
+ return False
+ return os.path.exists(output_file) and os.path.getsize(output_file) > 0
+
+
+def _tts_with_pauses(
+ text: str,
+ voice_name: str,
+ voice_rate: float,
+ voice_file: str,
+ voice_volume: float = 1.0,
+) -> Union[SubMaker, None]:
+ """
+ 处理包含停顿标签(如 [pause: 2s] / [pausa: 1.5s] / [停顿: 3秒])的脚本合成。
+ 分段生成语音和静音音频,通过 FFmpeg 拼接并调整字幕时间轴。
+ """
+ segments = utils.parse_script_with_pauses(text)
+ if not segments:
+ return None
+
+ speech_segments = [s for s in segments if s[0] == "speech"]
+ pause_segments = [s for s in segments if s[0] == "pause"]
+
+ if not pause_segments:
+ return _single_tts(text, voice_name, voice_rate, voice_file, voice_volume)
+
+ if not speech_segments:
+ total_pause_duration = sum(float(s[1]) for s in pause_segments)
+ total_pause_duration = min(
+ total_pause_duration, utils.MAX_PAUSE_DURATION_SECONDS
+ )
+ if not generate_silent_audio(total_pause_duration, voice_file):
+ return None
+ sub_maker = ensure_legacy_submaker_fields(SubMaker())
+ sub_maker.duration = total_pause_duration
+ return populate_legacy_submaker_with_full_text(
+ sub_maker=sub_maker,
+ text=utils.remove_pause_tags(text),
+ audio_duration_seconds=total_pause_duration,
+ )
+
+ with tempfile.TemporaryDirectory() as temp_dir:
+ audio_chunk_files: list[str] = []
+ combined_submaker = ensure_legacy_submaker_fields(SubMaker())
+ current_offset_seconds = 0.0
+
+ for idx, (seg_type, seg_val) in enumerate(segments):
+ if seg_type == "pause":
+ pause_duration = float(seg_val)
+ silence_file = os.path.join(temp_dir, f"silence_{idx}.mp3")
+ if not generate_silent_audio(pause_duration, silence_file):
+ logger.error(
+ f"failed to generate silent audio for pause of {pause_duration}s"
+ )
+ return None
+ actual_duration = get_audio_duration(silence_file)
+ if actual_duration <= 0:
+ actual_duration = pause_duration
+ audio_chunk_files.append(silence_file)
+ current_offset_seconds += actual_duration
+ elif seg_type == "speech":
+ speech_text = str(seg_val).strip()
+ if not speech_text:
+ continue
+ chunk_audio_file = os.path.join(temp_dir, f"speech_{idx}.mp3")
+ chunk_submaker = _single_tts(
+ text=speech_text,
+ voice_name=voice_name,
+ voice_rate=voice_rate,
+ voice_file=chunk_audio_file,
+ voice_volume=voice_volume,
+ )
+ if not chunk_submaker or not os.path.exists(chunk_audio_file):
+ logger.error(
+ f"failed to synthesize speech chunk: {speech_text[:50]}"
+ )
+ return None
+
+ chunk_duration = get_audio_duration(chunk_audio_file)
+ if chunk_duration <= 0:
+ chunk_duration = get_audio_duration(chunk_submaker)
+
+ # 1. 迁移 cues (edge_tts 7.x)
+ if hasattr(chunk_submaker, "cues") and chunk_submaker.cues:
+ offset_td = timedelta(seconds=current_offset_seconds)
+ for cue in chunk_submaker.cues:
+ shifted_cue = Subtitle(
+ index=len(combined_submaker.cues) + 1,
+ start=cue.start + offset_td,
+ end=cue.end + offset_td,
+ content=cue.content,
+ )
+ combined_submaker.cues.append(shifted_cue)
+
+ # 2. 迁移 legacy subs/offset
+ if hasattr(chunk_submaker, "subs") and chunk_submaker.subs:
+ combined_submaker.subs.extend(chunk_submaker.subs)
+ if hasattr(chunk_submaker, "offset") and chunk_submaker.offset:
+ offset_100ns = int(current_offset_seconds * 10000000)
+ for start_ns, end_ns in chunk_submaker.offset:
+ combined_submaker.offset.append(
+ (start_ns + offset_100ns, end_ns + offset_100ns)
+ )
+
+ audio_chunk_files.append(chunk_audio_file)
+ current_offset_seconds += chunk_duration
+
+ if not _concat_audio_files(audio_chunk_files, voice_file):
+ logger.error("failed to concatenate audio chunks with pauses")
+ return None
+
+ return combined_submaker
+
+
+def tts(
+ text: str,
+ voice_name: str,
+ voice_rate: float,
+ voice_file: str,
+ voice_volume: float = 1.0,
+) -> Union[SubMaker, None]:
+ if utils.has_pause_tags(text):
+ return _tts_with_pauses(
+ text=text,
+ voice_name=voice_name,
+ voice_rate=voice_rate,
+ voice_file=voice_file,
+ voice_volume=voice_volume,
+ )
+ return _single_tts(
+ text=text,
+ voice_name=voice_name,
+ voice_rate=voice_rate,
+ voice_file=voice_file,
+ voice_volume=voice_volume,
+ )
+
+
def convert_rate_to_percent(rate: float) -> str:
# edge-tts requires a sign-prefixed percentage (e.g. "+0%", "-20%").
# Rounding can yield 0 for rates near but not equal to 1.0 (e.g. 1.004,
@@ -1897,6 +2074,7 @@ def _format_text(text: str) -> str:
对齐仍保留这些字符,`create_subtitle()` 会一直等待不存在的 cue,
最终导致字幕文件缺失并在 Whisper fallback 校正时补出全 0 时间轴。
"""
+ text = utils.remove_pause_tags(text or "")
text = text.replace("[", " ")
text = text.replace("]", " ")
text = text.replace("(", " ")
@@ -2208,6 +2386,9 @@ def _get_audio_duration_from_submaker(sub_maker: SubMaker):
"""
获取音频时长
"""
+ if hasattr(sub_maker, "duration") and getattr(sub_maker, "duration", 0) > 0:
+ return float(getattr(sub_maker, "duration"))
+
# 优先兼容 edge_tts 7.x 的 cues 结构;
# 如果是项目里其他 TTS 手工填充的旧结构,则继续读取 offset。
if hasattr(sub_maker, "cues") and sub_maker.cues:
diff --git a/app/utils/utils.py b/app/utils/utils.py
index 92869693..aa5ad24d 100644
--- a/app/utils/utils.py
+++ b/app/utils/utils.py
@@ -320,6 +320,97 @@ def split_string_by_punctuations(s):
return result
+PAUSE_TAG_KEYWORDS = (
+ r"pause|pausa|silence|silencio|silêncio|silenzio|stille|"
+ r"пауза|тишина|停顿|暂停|静音|ポーズ|一時停止|無音|일시중지|정지"
+)
+PAUSE_TAG_PATTERN = re.compile(
+ rf"[\[\(]\s*(?:{PAUSE_TAG_KEYWORDS})\s*(?:[::]?\s*(\d+(?:\.\d+)?)\s*(s|sec|secs|second|seconds|ms|msec|msecs|秒|毫秒)?)?\s*[\]\)]",
+ re.IGNORECASE,
+)
+
+
+MAX_PAUSE_DURATION_SECONDS = 10.0
+
+
+def has_pause_tags(text: str) -> bool:
+ """检查文本中是否包含停顿/暂停标签。"""
+ if not text:
+ return False
+ return bool(PAUSE_TAG_PATTERN.search(text))
+
+
+def remove_pause_tags(text: str) -> str:
+ """
+ 移除脚本文本中的停顿标签。
+
+ 在字幕分句、LLM关键词提取或关键词搜索时,需要将此类非发音标记清除,
+ 避免被误当作台词参与对齐或作为视觉搜索词。
+ """
+ if not text:
+ return ""
+ return PAUSE_TAG_PATTERN.sub(" ", text)
+
+
+def parse_script_with_pauses(text: str) -> list[tuple[str, Any]]:
+ """
+ 解析脚本中的文本与停顿标签。
+
+ 连续停顿标签会自动合并为一个停顿段;小于等于 0 的无效时长会被忽略;
+ 超过 MAX_PAUSE_DURATION_SECONDS 的过长停顿会被限制在安全上限内。
+
+ Returns:
+ 有序元组列表,形式为 [("speech", "文案"), ("pause", 2.0), ...]
+ """
+ if not text:
+ return []
+
+ segments: list[tuple[str, Any]] = []
+ last_idx = 0
+
+ for match in PAUSE_TAG_PATTERN.finditer(text):
+ start, end = match.span()
+ if start > last_idx:
+ speech_text = text[last_idx:start].strip()
+ if speech_text:
+ segments.append(("speech", speech_text))
+
+ val_match = match.group(1)
+ val = float(val_match) if val_match is not None else 1.0
+ unit = (match.group(2) or "s").lower()
+ duration = val / 1000.0 if ("ms" in unit or "毫秒" in unit) else val
+
+ if duration <= 0:
+ logger.warning(f"invalid pause duration {duration}s, ignored")
+ last_idx = end
+ continue
+
+ if duration > MAX_PAUSE_DURATION_SECONDS:
+ logger.warning(
+ f"pause duration {duration}s exceeds maximum limit of {MAX_PAUSE_DURATION_SECONDS}s, "
+ f"clamped to {MAX_PAUSE_DURATION_SECONDS}s"
+ )
+ duration = MAX_PAUSE_DURATION_SECONDS
+
+ # 连续出现的停顿标签合并为一个停顿段,避免生成碎片化静音文件
+ if segments and segments[-1][0] == "pause":
+ merged_duration = min(
+ segments[-1][1] + duration, MAX_PAUSE_DURATION_SECONDS
+ )
+ segments[-1] = ("pause", merged_duration)
+ else:
+ segments.append(("pause", duration))
+
+ last_idx = end
+
+ if last_idx < len(text):
+ speech_text = text[last_idx:].strip()
+ if speech_text:
+ segments.append(("speech", speech_text))
+
+ return segments
+
+
def normalize_script_for_subtitle_matching(video_script: str) -> str:
"""
清理字幕匹配前的脚本文本。
@@ -329,7 +420,7 @@ def normalize_script_for_subtitle_matching(video_script: str) -> str:
字幕逐行匹配,脚本行数量会大于真实字幕行数量,最终可能补出
`00:00:00,000 --> 00:00:00,000`,导致剪辑软件无法导入 SRT。
"""
- video_script = video_script or ""
+ video_script = remove_pause_tags(video_script or "")
underscore_count = video_script.count("_")
video_script = video_script.replace("_", "")
cleaned_lines = []
diff --git a/test/services/test_voice.py b/test/services/test_voice.py
index 9b65b3ad..b823eeaf 100644
--- a/test/services/test_voice.py
+++ b/test/services/test_voice.py
@@ -1530,6 +1530,333 @@ class TestElevenLabsVoice(unittest.TestCase):
f"({expected_end_100ns} units = {audio_duration_seconds}s)",
)
+ def test_pause_tag_detection_and_parsing(self):
+ """测试多语言停顿标签的检测、解析与清洗。"""
+ sample_script = (
+ "Hola a todos. [pausa: 2s] "
+ "Welcome back. [pause: 1.5s] "
+ "今天天气很好。[停顿: 3秒] "
+ "잠시 멈춤 [일시중지: 500ms] "
+ "Final sentence."
+ )
+
+ self.assertTrue(utils.has_pause_tags(sample_script))
+ self.assertFalse(utils.has_pause_tags("Plain script with no pause tags."))
+
+ segments = utils.parse_script_with_pauses(sample_script)
+ self.assertEqual(len(segments), 9)
+ self.assertEqual(segments[0], ("speech", "Hola a todos."))
+ self.assertEqual(segments[1], ("pause", 2.0))
+ self.assertEqual(segments[2], ("speech", "Welcome back."))
+ self.assertEqual(segments[3], ("pause", 1.5))
+ self.assertEqual(segments[4], ("speech", "今天天气很好。"))
+ self.assertEqual(segments[5], ("pause", 3.0))
+ self.assertEqual(segments[6], ("speech", "잠시 멈춤"))
+ self.assertEqual(segments[7], ("pause", 0.5))
+ self.assertEqual(segments[8], ("speech", "Final sentence."))
+
+ cleaned = utils.remove_pause_tags(sample_script)
+ self.assertNotIn("[pausa", cleaned)
+ self.assertNotIn("[pause", cleaned)
+ self.assertNotIn("[停顿", cleaned)
+ self.assertNotIn("[일시중지", cleaned)
+
+ normalized = utils.normalize_script_for_subtitle_matching(sample_script)
+ self.assertNotIn("[pausa", normalized)
+ self.assertIn("Hola a todos", normalized)
+ self.assertIn("Final sentence", normalized)
+
+ # Flexible syntax tests: without colon, with parentheses, and with default duration
+ flexible_script = "Intro. [pausa 1s] Middle. (pausa: 2s) Next. [pause] End."
+ self.assertTrue(utils.has_pause_tags(flexible_script))
+ flex_segments = utils.parse_script_with_pauses(flexible_script)
+ self.assertEqual(len(flex_segments), 7)
+ self.assertEqual(flex_segments[0], ("speech", "Intro."))
+ self.assertEqual(flex_segments[1], ("pause", 1.0))
+ self.assertEqual(flex_segments[2], ("speech", "Middle."))
+ self.assertEqual(flex_segments[3], ("pause", 2.0))
+ self.assertEqual(flex_segments[4], ("speech", "Next."))
+ self.assertEqual(flex_segments[5], ("pause", 1.0))
+ self.assertEqual(flex_segments[6], ("speech", "End."))
+
+ flex_cleaned = utils.remove_pause_tags(flexible_script)
+ self.assertNotIn("[pausa", flex_cleaned)
+ self.assertNotIn("(pausa", flex_cleaned)
+ self.assertNotIn("[pause", flex_cleaned)
+
+ def test_tts_with_pauses_shifts_submaker_timeline(self):
+ """测试包含停顿标签时,SubMaker 时间轴和音频拼接正确偏移。"""
+ from edge_tts.srt_composer import Subtitle
+
+ fake_sub1 = vs.ensure_legacy_submaker_fields(vs.SubMaker())
+ fake_sub1.cues = [
+ Subtitle(1, timedelta(seconds=0.1), timedelta(seconds=1.5), "Segment 1"),
+ ]
+ fake_sub1.subs = ["Segment 1"]
+ fake_sub1.offset = [(1000000, 15000000)]
+
+ fake_sub2 = vs.ensure_legacy_submaker_fields(vs.SubMaker())
+ fake_sub2.cues = [
+ Subtitle(1, timedelta(seconds=0.2), timedelta(seconds=1.8), "Segment 2"),
+ ]
+ fake_sub2.subs = ["Segment 2"]
+ fake_sub2.offset = [(2000000, 18000000)]
+
+ def fake_single_tts(text, voice_name, voice_rate, voice_file, voice_volume=1.0):
+ Path(voice_file).touch()
+ if "Segment 1" in text:
+ return fake_sub1
+ return fake_sub2
+
+ def fake_silence(duration, voice_file):
+ Path(voice_file).touch()
+ return True
+
+ with (
+ tempfile.TemporaryDirectory() as tmp_dir,
+ patch.object(vs, "_single_tts", side_effect=fake_single_tts) as mock_single_tts,
+ patch.object(vs, "generate_silent_audio", side_effect=fake_silence) as mock_silence,
+ patch.object(vs, "get_audio_duration", side_effect=[1.5, 2.0, 1.8]),
+ patch.object(vs, "_concat_audio_files", return_value=True) as mock_concat,
+ ):
+ out_file = str(Path(tmp_dir) / "combined.mp3")
+ script = "Segment 1. [pausa: 2s] Segment 2."
+ result_submaker = vs.tts(
+ text=script,
+ voice_name="zh-CN-XiaoxiaoNeural",
+ voice_rate=1.0,
+ voice_file=out_file,
+ )
+
+ self.assertIsNotNone(result_submaker)
+ self.assertEqual(mock_single_tts.call_count, 2)
+ mock_silence.assert_called_once()
+ mock_concat.assert_called_once()
+
+ # 验证第二段 cues 偏移了 1.5s + 2.0s = 3.5s
+ self.assertEqual(len(result_submaker.cues), 2)
+ self.assertAlmostEqual(result_submaker.cues[0].start.total_seconds(), 0.1, places=2)
+ self.assertAlmostEqual(result_submaker.cues[0].end.total_seconds(), 1.5, places=2)
+ self.assertAlmostEqual(result_submaker.cues[1].start.total_seconds(), 3.5 + 0.2, places=2)
+ self.assertAlmostEqual(result_submaker.cues[1].end.total_seconds(), 3.5 + 1.8, places=2)
+
+ # 验证 legacy offset 也正确偏移
+ self.assertEqual(len(result_submaker.offset), 2)
+ self.assertEqual(result_submaker.offset[0], (1000000, 15000000))
+ expected_ns_offset = int(3.5 * 10000000)
+ self.assertEqual(
+ result_submaker.offset[1],
+ (2000000 + expected_ns_offset, 18000000 + expected_ns_offset),
+ )
+
+
+ def test_pause_invalid_and_excessive_durations(self):
+ """测试无效时长(<= 0s)被忽略,以及超长时长被限制在最大上限内。"""
+ # 1. 无效或零时长:不应识别为停顿段
+ zero_script = "Hello [pause: 0s] world. [pause: -2s] Bye."
+ segments = utils.parse_script_with_pauses(zero_script)
+ speech_only = [s for s in segments if s[0] == "speech"]
+ pause_only = [s for s in segments if s[0] == "pause"]
+ self.assertEqual(len(pause_only), 0)
+ self.assertTrue(any("Hello" in s[1] for s in speech_only))
+ self.assertTrue(any("world" in s[1] for s in speech_only))
+
+ # 2. 超长停顿:超过 MAX_PAUSE_DURATION_SECONDS 被 clamp
+ long_script = "Hello [pause: 99s] world."
+ segments_long = utils.parse_script_with_pauses(long_script)
+ pauses = [s for s in segments_long if s[0] == "pause"]
+ self.assertEqual(len(pauses), 1)
+ self.assertEqual(pauses[0][1], utils.MAX_PAUSE_DURATION_SECONDS)
+
+ def test_pause_consecutive_merging(self):
+ """测试连续停顿标签自动合并为一个停顿段,且总时长受上限保护。"""
+ # 两个连续停顿合并为 1s + 2s = 3s
+ script = "First part. [pause: 1s] [pause: 2s] Second part."
+ segments = utils.parse_script_with_pauses(script)
+ pauses = [s for s in segments if s[0] == "pause"]
+ self.assertEqual(len(pauses), 1)
+ self.assertEqual(pauses[0][1], 3.0)
+
+ # 多个停顿叠加超过最大上限时,合并后被截断在 MAX_PAUSE_DURATION_SECONDS
+ over_script = "Start. [pause: 7s] [pause: 8s] End."
+ over_segments = utils.parse_script_with_pauses(over_script)
+ over_pauses = [s for s in over_segments if s[0] == "pause"]
+ self.assertEqual(len(over_pauses), 1)
+ self.assertEqual(over_pauses[0][1], utils.MAX_PAUSE_DURATION_SECONDS)
+
+ def test_pause_leading_and_trailing(self):
+ """测试开头停顿(leading)和结尾停顿(trailing)的音轨和时间轴偏移。"""
+ from edge_tts.srt_composer import Subtitle
+
+ fake_sub = vs.ensure_legacy_submaker_fields(vs.SubMaker())
+ fake_sub.cues = [
+ Subtitle(1, timedelta(seconds=0.1), timedelta(seconds=1.2), "Hello"),
+ ]
+ fake_sub.subs = ["Hello"]
+ fake_sub.offset = [(1000000, 12000000)]
+
+ def fake_single_tts(text, voice_name, voice_rate, voice_file, voice_volume=1.0):
+ Path(voice_file).touch()
+ return fake_sub
+
+ def fake_silence(duration, voice_file):
+ Path(voice_file).touch()
+ return True
+
+ with (
+ tempfile.TemporaryDirectory() as tmp_dir,
+ patch.object(vs, "_single_tts", side_effect=fake_single_tts),
+ patch.object(vs, "generate_silent_audio", side_effect=fake_silence) as mock_silence,
+ patch.object(vs, "get_audio_duration", side_effect=[1.5, 1.2]),
+ patch.object(vs, "_concat_audio_files", return_value=True),
+ ):
+ # Leading pause: [pause: 1.5s] Hello
+ out_file = str(Path(tmp_dir) / "leading.mp3")
+ result = vs.tts(
+ text="[pause: 1.5s] Hello",
+ voice_name="zh-CN-XiaoxiaoNeural",
+ voice_rate=1.0,
+ voice_file=out_file,
+ )
+ self.assertIsNotNone(result)
+ mock_silence.assert_called_with(1.5, mock_silence.call_args[0][1])
+ # 字幕 cue 应该从 1.5s + 0.1s = 1.6s 开始
+ self.assertAlmostEqual(result.cues[0].start.total_seconds(), 1.6, places=2)
+
+ with (
+ tempfile.TemporaryDirectory() as tmp_dir,
+ patch.object(vs, "_single_tts", side_effect=fake_single_tts),
+ patch.object(vs, "generate_silent_audio", side_effect=fake_silence) as mock_silence,
+ patch.object(vs, "get_audio_duration", side_effect=[1.2, 2.0]),
+ patch.object(vs, "_concat_audio_files", return_value=True),
+ ):
+ # Trailing pause: Hello [pause: 2s]
+ out_file = str(Path(tmp_dir) / "trailing.mp3")
+ result = vs.tts(
+ text="Hello [pause: 2s]",
+ voice_name="zh-CN-XiaoxiaoNeural",
+ voice_rate=1.0,
+ voice_file=out_file,
+ )
+ self.assertIsNotNone(result)
+ mock_silence.assert_called_with(2.0, mock_silence.call_args[0][1])
+ # 语音字幕应该保持在原本位置,不受结尾静音后移
+ self.assertAlmostEqual(result.cues[0].start.total_seconds(), 0.1, places=2)
+ self.assertAlmostEqual(result.cues[0].end.total_seconds(), 1.2, places=2)
+
+ def test_pause_script_with_only_pauses(self):
+ """测试脚本只包含停顿标签时的纯静音生成与安全性。"""
+ def fake_silence(duration, voice_file):
+ Path(voice_file).touch()
+ return True
+
+ with (
+ tempfile.TemporaryDirectory() as tmp_dir,
+ patch.object(vs, "generate_silent_audio", side_effect=fake_silence) as mock_silence,
+ patch.object(vs, "_single_tts") as mock_single_tts,
+ ):
+ out_file = str(Path(tmp_dir) / "only_pauses.mp3")
+ result = vs.tts(
+ text="[pause: 2.5s]",
+ voice_name="zh-CN-XiaoxiaoNeural",
+ voice_rate=1.0,
+ voice_file=out_file,
+ )
+ self.assertIsNotNone(result)
+ mock_single_tts.assert_not_called()
+ mock_silence.assert_called_once_with(2.5, out_file)
+ self.assertEqual(vs.get_audio_duration(result), 2.5)
+
+ # 字幕生成应安全处理无台词脚本,不抛出异常
+ srt_path = str(Path(tmp_dir) / "only_pauses.srt")
+ vs.create_subtitle(result, "[pause: 2.5s]", srt_path, word_level=False)
+ vs.create_subtitle(result, "[pause: 2.5s]", srt_path, word_level=True)
+
+ def test_subtitle_sync_sentence_mode_with_pauses(self):
+ """测试句子模式 (sentence) 下包含停顿标签的字幕时间轴与内容完全同步。"""
+ from edge_tts.srt_composer import Subtitle
+
+ fake_sub = vs.ensure_legacy_submaker_fields(vs.SubMaker())
+ # 第一段话在 0.1s - 1.5s,第二段话在停顿 2s 后 (3.5s - 5.0s)
+ fake_sub.cues = [
+ Subtitle(1, timedelta(seconds=0.1), timedelta(seconds=0.7), "Primera"),
+ Subtitle(2, timedelta(seconds=0.8), timedelta(seconds=1.5), "frase."),
+ Subtitle(3, timedelta(seconds=3.6), timedelta(seconds=4.2), "Segunda"),
+ Subtitle(4, timedelta(seconds=4.3), timedelta(seconds=5.0), "frase."),
+ ]
+
+ with tempfile.TemporaryDirectory() as tmp_dir:
+ srt_file = str(Path(tmp_dir) / "sentence_mode.srt")
+ raw_script = "Primera frase. [pausa: 2s] Segunda frase."
+ vs.create_subtitle(
+ sub_maker=fake_sub,
+ text=raw_script,
+ subtitle_file=srt_file,
+ word_level=False,
+ )
+
+ self.assertTrue(os.path.exists(srt_file))
+ with open(srt_file, "r", encoding="utf-8") as f:
+ content = f.read()
+
+ self.assertIn("Primera frase", content)
+ self.assertIn("Segunda frase", content)
+ self.assertNotIn("[pausa", content)
+ # 第一句开始于 0.1s,第二句开始于 3.6s (体现了 2s 停顿)
+ self.assertIn("00:00:00,100 --> 00:00:01,500", content)
+ self.assertIn("00:00:03,600 --> 00:00:05,000", content)
+
+ def test_subtitle_sync_word_mode_with_pauses(self):
+ """测试单字模式 (word_by_word) 下包含停顿标签的字幕时间轴与内容完全同步。"""
+ from edge_tts.srt_composer import Subtitle
+
+ fake_sub = vs.ensure_legacy_submaker_fields(vs.SubMaker())
+ fake_sub.cues = [
+ Subtitle(1, timedelta(seconds=0.1), timedelta(seconds=0.5), "Hello"),
+ Subtitle(2, timedelta(seconds=0.6), timedelta(seconds=1.2), "world."),
+ Subtitle(3, timedelta(seconds=3.3), timedelta(seconds=3.8), "Good"),
+ Subtitle(4, timedelta(seconds=3.9), timedelta(seconds=4.5), "morning."),
+ ]
+
+ with tempfile.TemporaryDirectory() as tmp_dir:
+ srt_file = str(Path(tmp_dir) / "word_mode.srt")
+ raw_script = "Hello world. [pause: 2s] Good morning."
+ vs.create_subtitle(
+ sub_maker=fake_sub,
+ text=raw_script,
+ subtitle_file=srt_file,
+ word_level=True,
+ )
+
+ self.assertTrue(os.path.exists(srt_file))
+ with open(srt_file, "r", encoding="utf-8") as f:
+ content = f.read()
+
+ self.assertIn("Hello", content)
+ self.assertIn("world.", content)
+ self.assertIn("Good", content)
+ self.assertIn("morning.", content)
+ # 第二段词条被正确偏移到了 3.3s 和 3.9s
+ self.assertIn("00:00:00,100 --> 00:00:00,500", content)
+ self.assertIn("00:00:03,300 --> 00:00:03,800", content)
+ self.assertIn("00:00:03,900 --> 00:00:04,500", content)
+
+ def test_tts_without_pauses_calls_single_tts_directly(self):
+ """验证不包含停顿标签时,直接调用 _single_tts,原有行为和性能完全不变。"""
+ with (
+ patch.object(vs, "_single_tts", return_value="normal_submaker") as mock_single_tts,
+ patch.object(vs, "_tts_with_pauses") as mock_pauses,
+ ):
+ res = vs.tts(
+ text="This is regular text without any pauses.",
+ voice_name="zh-CN-XiaoxiaoNeural",
+ voice_rate=1.0,
+ voice_file="any.mp3",
+ )
+ self.assertEqual(res, "normal_submaker")
+ mock_single_tts.assert_called_once()
+ mock_pauses.assert_not_called()
+
if __name__ == "__main__":
# python -m unittest test.services.test_voice.TestVoiceService.test_azure_tts_v1
diff --git a/webui/i18n/de.json b/webui/i18n/de.json
index f0b81ed2..c1cd5af0 100644
--- a/webui/i18n/de.json
+++ b/webui/i18n/de.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Ausgewähltes Skript und Keywords übernommen",
"Auto Detect": "Automatisch erkennen",
"Video Script": "Videoskript (optional)",
- "Video Script Help": "Leer lassen, um es per KI aus dem Videothema zu erstellen. Sinnvolle Zeichensetzung verbessert die Untertitel.",
+ "Video Script Help": "Leer lassen, um es per KI aus dem Videothema zu erstellen. Sinnvolle Zeichensetzung verbessert die Untertitel. Mit [pause: 2s] oder [stille: 1s] können Pausen eingefügt werden.",
"Generate Video Keywords": "Keywords mit KI erstellen",
"Please Enter the Video Subject": "Bitte geben Sie zuerst das Drehbuch an",
"Please Enter the Video Subject First": "Bitte geben Sie zuerst das Videothema ein",
diff --git a/webui/i18n/en.json b/webui/i18n/en.json
index 42e4f0a4..9eec766a 100644
--- a/webui/i18n/en.json
+++ b/webui/i18n/en.json
@@ -51,7 +51,7 @@
"LoomLoom Candidate Applied": "Selected script and keywords applied",
"Auto Detect": "Auto Detect",
"Video Script": "Video Script (Optional)",
- "Video Script Help": "Leave blank to generate it with AI from the video subject. Proper punctuation helps generate subtitles.",
+ "Video Script Help": "Leave blank to generate it with AI from the video subject. Proper punctuation helps generate subtitles. Use [pause: 2s] or [silence: 1s] to add custom pauses.",
"Generate Video Keywords": "Generate Keywords with AI",
"Please Enter the Video Subject": "Please Enter the Video Script First",
"Please Enter the Video Subject First": "Please enter the video subject first",
diff --git a/webui/i18n/es.json b/webui/i18n/es.json
index 0df98f0c..9eff395d 100644
--- a/webui/i18n/es.json
+++ b/webui/i18n/es.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Guion y palabras clave seleccionados aplicados",
"Auto Detect": "Detección automática",
"Video Script": "Guion del vídeo (opcional)",
- "Video Script Help": "Déjalo vacío para generarlo con IA a partir del tema del vídeo. Una puntuación adecuada ayuda a generar los subtítulos.",
+ "Video Script Help": "Déjalo vacío para generarlo con IA a partir del tema del vídeo. Una puntuación adecuada ayuda a generar los subtítulos. Puedes insertar pausas con [pausa: 2s] o [silencio: 1s].",
"Generate Video Keywords": "Generar palabras clave con IA",
"Please Enter the Video Subject": "Introduce primero el guion del vídeo",
"Please Enter the Video Subject First": "Introduce primero el tema del vídeo",
diff --git a/webui/i18n/fr.json b/webui/i18n/fr.json
index e15cd13b..a211353a 100644
--- a/webui/i18n/fr.json
+++ b/webui/i18n/fr.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Script et mots-clés sélectionnés appliqués",
"Auto Detect": "Détection automatique",
"Video Script": "Script de la vidéo (Optionnel)",
- "Video Script Help": "Laissez vide pour le générer par IA à partir du sujet. Une bonne ponctuation aide à la génération des sous-titres.",
+ "Video Script Help": "Laissez vide pour le générer par IA à partir du sujet. Une bonne ponctuation aide à la génération des sous-titres. Utilisez [pause: 2s] ou [silence: 1s] pour insérer des pauses.",
"Generate Video Keywords": "Générer les mots-clés par IA",
"Please Enter the Video Subject": "Veuillez d'abord saisir le script de la vidéo",
"Please Enter the Video Subject First": "Veuillez d'abord saisir le sujet de la vidéo",
diff --git a/webui/i18n/id.json b/webui/i18n/id.json
index 1f0b788a..4d337c66 100644
--- a/webui/i18n/id.json
+++ b/webui/i18n/id.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Skrip dan kata kunci terpilih diterapkan",
"Auto Detect": "Deteksi Otomatis",
"Video Script": "Skrip Video (Opsional)",
- "Video Script Help": "Kosongkan untuk membuatnya dengan AI dari topik video. Tanda baca yang tepat membantu pembuatan subtitle.",
+ "Video Script Help": "Kosongkan untuk membuatnya dengan AI dari topik video. Tanda baca yang tepat membantu pembuatan subtitle. Gunakan [pause: 2s] atau [silence: 1s] untuk jeda kustom.",
"Generate Video Keywords": "Buat Kata Kunci dengan AI",
"Please Enter the Video Subject": "Silakan Masukkan Skrip Video Terlebih Dahulu",
"Please Enter the Video Subject First": "Harap masukkan subjek video terlebih dahulu",
diff --git a/webui/i18n/it.json b/webui/i18n/it.json
index b653b5bd..7ea47397 100644
--- a/webui/i18n/it.json
+++ b/webui/i18n/it.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Testo e parole chiave selezionati applicati",
"Auto Detect": "Rilevamento automatico",
"Video Script": "Testo del video (facoltativo)",
- "Video Script Help": "Lascia vuoto per generarlo con l'IA a partire dall'argomento del video. Una corretta punteggiatura aiuta a generare i sottotitoli.",
+ "Video Script Help": "Lascia vuoto per generarlo con l'IA a partire dall'argomento del video. Una corretta punteggiatura aiuta a generare i sottotitoli. Usa [pausa: 2s] o [silenzio: 1s] per inserire pause.",
"Generate Video Keywords": "Genera parole chiave con l'IA",
"Please Enter the Video Subject": "Inserisci prima il testo del video",
"Please Enter the Video Subject First": "Inserisci prima l'argomento del video",
diff --git a/webui/i18n/ko.json b/webui/i18n/ko.json
index e93ac682..709a1f52 100644
--- a/webui/i18n/ko.json
+++ b/webui/i18n/ko.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "선택한 스크립트와 키워드가 적용되었습니다",
"Auto Detect": "자동 감지",
"Video Script": "영상 스크립트 (선택)",
- "Video Script Help": "비워 두면 영상 주제로부터 AI가 생성합니다. 문장 부호를 잘 사용하면 자막 생성에 도움이 됩니다.",
+ "Video Script Help": "비워 두면 영상 주제로부터 AI가 생성합니다. 문장 부호를 잘 사용하면 자막 생성에 도움이 됩니다. [pause: 2s] 또는 [일시중지: 2s]로 원하는 멈춤 시간을 추가할 수 있습니다.",
"Generate Video Keywords": "AI로 키워드 생성",
"Please Enter the Video Subject": "먼저 영상 스크립트를 입력하세요",
"Please Enter the Video Subject First": "먼저 영상 주제를 입력하세요",
diff --git a/webui/i18n/pt.json b/webui/i18n/pt.json
index 41d55170..d0708113 100644
--- a/webui/i18n/pt.json
+++ b/webui/i18n/pt.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Roteiro e palavras-chave selecionados aplicados",
"Auto Detect": "Detectar Automaticamente",
"Video Script": "Roteiro do vídeo (opcional)",
- "Video Script Help": "Deixe em branco para gerá-lo com IA a partir do tema do vídeo. Uma pontuação adequada ajuda a gerar as legendas.",
+ "Video Script Help": "Deixe em branco para gerá-lo com IA a partir do tema do vídeo. Uma pontuação adequada ajuda a gerar as legendas. Use [pausa: 2s] ou [silêncio: 1s] para inserir pausas.",
"Generate Video Keywords": "Gerar palavras-chave com IA",
"Please Enter the Video Subject": "Por favor, insira o Roteiro do Vídeo primeiro",
"Please Enter the Video Subject First": "Insira primeiro o tema do vídeo",
diff --git a/webui/i18n/ru.json b/webui/i18n/ru.json
index 14070b61..a80339d2 100644
--- a/webui/i18n/ru.json
+++ b/webui/i18n/ru.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Выбранные сценарий и ключевые слова применены",
"Auto Detect": "Определить автоматически",
"Video Script": "Сценарий (необязательно)",
- "Video Script Help": "Оставьте поле пустым, чтобы создать сценарий с помощью ИИ по теме видео. Правильная пунктуация помогает создавать субтитры.",
+ "Video Script Help": "Оставьте поле пустым, чтобы создать сценарий с помощью ИИ по теме видео. Правильная пунктуация помогает создавать субтитры. Используйте [пауза: 2s] или [тишина: 1s] для пауз.",
"Generate Video Keywords": "Создать ключевые слова с ИИ",
"Please Enter the Video Subject": "Сначала введите сценарий видео",
"Please Enter the Video Subject First": "Сначала введите тему видео",
diff --git a/webui/i18n/tr.json b/webui/i18n/tr.json
index f08c6f6e..14cc477f 100644
--- a/webui/i18n/tr.json
+++ b/webui/i18n/tr.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Seçilen senaryo ve anahtar kelimeler uygulandı",
"Auto Detect": "Otomatik Algılama",
"Video Script": "Video Senaryosu (İsteğe Bağlı)",
- "Video Script Help": "Video konusundan AI ile oluşturmak için boş bırakın. Doğru noktalama, altyazı oluşturmayı kolaylaştırır.",
+ "Video Script Help": "Video konusundan AI ile oluşturmak için boş bırakın. Doğru noktalama, altyazı oluşturmayı kolaylaştırır. [pause: 2s] veya [silence: 1s] ile duraklamalar ekleyebilirsiniz.",
"Generate Video Keywords": "AI ile Anahtar Kelime Oluştur",
"Please Enter the Video Subject": "Lütfen Önce Video Senaryosunu Girin",
"Please Enter the Video Subject First": "Lütfen önce video konusunu girin",
diff --git a/webui/i18n/vi.json b/webui/i18n/vi.json
index 8c5b30ce..bc998ed2 100644
--- a/webui/i18n/vi.json
+++ b/webui/i18n/vi.json
@@ -41,7 +41,7 @@
"LoomLoom Candidate Applied": "Đã áp dụng kịch bản và từ khóa đã chọn",
"Auto Detect": "Tự Động Phát Hiện",
"Video Script": "Kịch Bản Video (Không Bắt Buộc)",
- "Video Script Help": "Để trống để AI tạo từ chủ đề video. Dấu câu hợp lý giúp tạo phụ đề chính xác hơn.",
+ "Video Script Help": "Để trống để AI tạo từ chủ đề video. Dấu câu hợp lý giúp tạo phụ đề chính xác hơn. Sử dụng [pause: 2s] hoặc [silence: 1s] để chèn khoảng dừng.",
"Generate Video Keywords": "Tạo từ khóa bằng AI",
"Please Enter the Video Subject": "Vui lòng Nhập Kịch Bản Video Trước",
"Please Enter the Video Subject First": "Vui lòng nhập chủ đề video trước",
diff --git a/webui/i18n/zh.json b/webui/i18n/zh.json
index 8d261b5e..3a6cfec5 100644
--- a/webui/i18n/zh.json
+++ b/webui/i18n/zh.json
@@ -51,7 +51,7 @@
"LoomLoom Candidate Applied": "已应用选中的文案和关键词",
"Auto Detect": "自动检测",
"Video Script": "视频文案(可选)",
- "Video Script Help": "可留空,由 AI 根据视频主题生成。合理使用标点断句有助于生成字幕。",
+ "Video Script Help": "可留空,由 AI 根据视频主题生成。合理使用标点断句有助于生成字幕。支持使用 [停顿: 2s] 或 [pause: 2s] 插入指定秒数的停顿。",
"Generate Video Keywords": "点击使用AI根据**文案**生成视频关键词",
"Please Enter the Video Subject": "请先填写视频文案",
"Please Enter the Video Subject First": "请先填写视频主题",