Skip to content

利用 AI 大模型和自动化工作流,根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.

feat(tts): add VoxCPM voice cloning (#1390)

shinve4d8e2afc2102
diff --git a/README-en.md b/README-en.md
index bf326922..b65bca92 100644
--- a/README-en.md
+++ b/README-en.md
@@ -475,7 +475,7 @@ The summary contains `total`, `succeeded`, `failed`, and `tasks`; each task entr
Select a provider and voice in the WebUI, then follow the on-screen instructions for any required credentials. Edge TTS does not require an API key; [Azure TTS V2](https://portal.azure.com/) and other cloud providers require credentials from their respective platforms. See the available Edge TTS voices in the [voice list](./docs/voice-list.txt).
-ModelBest VoxCPM requires an API key and a model ID with the `speech_synthesis` capability. Its SSE response streams WAV audio, which MoneyPrinterTurbo automatically converts to the MP3 used by the video pipeline. This integration supports standard text-to-speech with the default voice; reference-audio voice cloning is not included yet.
+ModelBest VoxCPM requires an API key and a model ID with the `speech_synthesis` capability. Its SSE response streams WAV audio, which MoneyPrinterTurbo automatically converts to the MP3 used by the video pipeline. In addition to standard text-to-speech, the WebUI can clone speaker identity from optional reference audio. High-fidelity delivery reuses that clip with its exact transcript by default, or accepts a separate performance example, and sends `prompt_audio` plus `prompt_text` to continue its pacing, emotion, and pronunciation. The WebUI can use the local Whisper configuration to create an editable transcript draft. Uploads are limited to 20 MiB and each converted WAV to 5 MiB. Reference material is scoped to the current browser session and task and is not written to configuration, presets, task history, or logs.
### Subtitle Generation
diff --git a/README.md b/README.md
index 45154707..19dbffb0 100644
--- a/README.md
+++ b/README.md
@@ -461,7 +461,7 @@ WebUI 中的 **Azure TTS V1** 基于 **Edge TTS**,免费且无需 API Key。
可直接在 WebUI 中选择 Provider 和音色,并按照界面提示填写所需凭据。Edge TTS 不需要 API Key;[Azure TTS V2](https://portal.azure.com/) 及其他云端服务需要对应平台的凭据。Edge TTS 音色可查看:[音色列表](./docs/voice-list.txt)。
-ModelBest VoxCPM 需要填写 API Key 和支持 `speech_synthesis` 的模型 ID。服务以 SSE 返回 WAV 音频,项目会自动转换为视频流水线使用的 MP3。当前接入支持普通文本合成和默认音色;参考音频复刻尚未包含。
+ModelBest VoxCPM 需要填写 API Key 和支持 `speech_synthesis` 的模型 ID。服务以 SSE 返回 WAV 音频,项目会自动转换为视频流水线使用的 MP3。除了普通文本合成,还可以上传参考音频复刻说话人音色;开启“保留参考音频的节奏与情绪”后,默认复用这段音频并要求填写其准确逐字稿,也可另选一段节奏与情绪示范,通过 `prompt_audio` 和 `prompt_text` 延续演绎。WebUI 可使用本地 Whisper 自动生成可编辑的逐字稿草稿。上传文件最大 20 MiB,转换后的单个 WAV 最大 5 MiB。参考素材仅用于当前浏览器会话和任务,不写入配置、预设、任务历史或日志。
### 字幕生成
diff --git a/app/services/subtitle.py b/app/services/subtitle.py
index 3d9d3d74..04386705 100644
--- a/app/services/subtitle.py
+++ b/app/services/subtitle.py
@@ -1,6 +1,7 @@
import json
import os.path
import re
+import tempfile
from timeit import default_timer as timer
try:
@@ -19,7 +20,12 @@ initial_prompt = config.whisper.get("initial_prompt", "") or None
model = None
-def create(audio_file, subtitle_file: str = "", word_level: bool = False):
+def create(
+ audio_file,
+ subtitle_file: str = "",
+ word_level: bool = False,
+ log_details: bool = True,
+):
global model
if WhisperModel is None:
logger.warning("faster_whisper not available, skipping whisper subtitle generation")
@@ -48,7 +54,8 @@ def create(audio_file, subtitle_file: str = "", word_level: bool = False):
)
return None
- logger.info(f"start, output file: {subtitle_file}")
+ if log_details:
+ logger.info(f"start, output file: {subtitle_file}")
if not subtitle_file:
subtitle_file = f"{audio_file}.srt"
@@ -73,8 +80,9 @@ def create(audio_file, subtitle_file: str = "", word_level: bool = False):
if not seg_text:
return
- msg = "[%.2fs -> %.2fs] %s" % (seg_start, seg_end, seg_text)
- logger.debug(msg)
+ if log_details:
+ msg = "[%.2fs -> %.2fs] %s" % (seg_start, seg_end, seg_text)
+ logger.debug(msg)
subtitles.append(
{"msg": seg_text, "start_time": seg_start, "end_time": seg_end}
@@ -148,7 +156,8 @@ def create(audio_file, subtitle_file: str = "", word_level: bool = False):
sub = "\n".join(lines) + "\n"
with open(subtitle_file, "w", encoding="utf-8") as f:
f.write(sub)
- logger.info(f"subtitle file created: {subtitle_file}")
+ if log_details:
+ logger.info(f"subtitle file created: {subtitle_file}")
def file_to_subtitles(filename):
@@ -180,6 +189,24 @@ def file_to_subtitles(filename):
return times_texts
+def transcribe_audio_bytes(audio_bytes: bytes) -> str:
+ """Transcribe in-memory WAV audio and remove every temporary artifact."""
+ if not isinstance(audio_bytes, bytes) or not audio_bytes:
+ return ""
+
+ with tempfile.TemporaryDirectory(prefix="whisper-transcript-") as temp_dir:
+ audio_file = os.path.join(temp_dir, "reference.wav")
+ subtitle_file = os.path.join(temp_dir, "reference.srt")
+ with open(audio_file, "wb") as output:
+ output.write(audio_bytes)
+
+ create_result = create(audio_file, subtitle_file, log_details=False)
+ if create_result in (None, "") and not os.path.isfile(subtitle_file):
+ return ""
+ subtitle_items = file_to_subtitles(subtitle_file)
+ return " ".join(item[2].strip() for item in subtitle_items if item[2].strip())
+
+
def levenshtein_distance(s1, s2):
if len(s1) < len(s2):
return levenshtein_distance(s2, s1)
diff --git a/app/services/task.py b/app/services/task.py
index 6be13086..93ef3901 100644
--- a/app/services/task.py
+++ b/app/services/task.py
@@ -486,6 +486,9 @@ def generate_audio(
voice_preview=None,
*,
allow_server_file_input: bool = False,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
):
"""
Generate audio for the video script.
@@ -527,12 +530,18 @@ def generate_audio(
logger.info("no custom audio file provided, using TTS to generate audio.")
audio_file = path.join(utils.task_dir(task_id), "audio.mp3")
- sub_maker = voice.tts(
- text=video_script,
- voice_name=voice.parse_voice_name(params.voice_name),
- voice_rate=params.voice_rate,
- voice_file=audio_file,
- )
+ tts_kwargs = {
+ "text": video_script,
+ "voice_name": voice.parse_voice_name(params.voice_name),
+ "voice_rate": params.voice_rate,
+ "voice_file": audio_file,
+ }
+ if voxcpm_reference_audio is not None:
+ tts_kwargs["voxcpm_reference_audio"] = voxcpm_reference_audio
+ if voxcpm_prompt_audio is not None:
+ tts_kwargs["voxcpm_prompt_audio"] = voxcpm_prompt_audio
+ tts_kwargs["voxcpm_prompt_text"] = voxcpm_prompt_text
+ sub_maker = voice.tts(**tts_kwargs)
if sub_maker is None:
_mark_task_failed(
task_id,
@@ -1371,6 +1380,9 @@ def _run_pipeline(
voice_preview: dict | None = None,
loomloom_video_request: loomloom.LoomLoomConfirmedVideoRequest | None = None,
allow_server_file_input: bool = False,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
):
logger.info(f"start task: {task_id}, stop_at: {stop_at}")
sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=5)
@@ -1525,12 +1537,20 @@ def _run_pipeline(
sm.state.update_task(task_id, state=const.TASK_STATE_PROCESSING, progress=20)
# 3. Generate audio
+ generate_audio_kwargs = {
+ "voice_preview": voice_preview,
+ "allow_server_file_input": allow_server_file_input,
+ }
+ if voxcpm_reference_audio is not None:
+ generate_audio_kwargs["voxcpm_reference_audio"] = voxcpm_reference_audio
+ if voxcpm_prompt_audio is not None:
+ generate_audio_kwargs["voxcpm_prompt_audio"] = voxcpm_prompt_audio
+ generate_audio_kwargs["voxcpm_prompt_text"] = voxcpm_prompt_text
audio_file, audio_duration, sub_maker = generate_audio(
task_id,
params,
video_script,
- voice_preview=voice_preview,
- allow_server_file_input=allow_server_file_input,
+ **generate_audio_kwargs,
)
if not audio_file:
return _mark_task_failed(
@@ -1687,6 +1707,9 @@ def start(
voice_preview: dict | None = None,
loomloom_video_request: loomloom.LoomLoomConfirmedVideoRequest | None = None,
allow_server_file_input: bool = False,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
):
"""
执行任务流水线,并确保未预期异常也会转换成可查询的失败状态。
@@ -1702,6 +1725,9 @@ def start(
voice_preview=voice_preview,
loomloom_video_request=loomloom_video_request,
allow_server_file_input=allow_server_file_input,
+ voxcpm_reference_audio=voxcpm_reference_audio,
+ voxcpm_prompt_audio=voxcpm_prompt_audio,
+ voxcpm_prompt_text=voxcpm_prompt_text,
)
except Exception as exc:
logger.exception(
diff --git a/app/services/voice.py b/app/services/voice.py
index 09373a63..7e14895a 100644
--- a/app/services/voice.py
+++ b/app/services/voice.py
@@ -77,6 +77,11 @@ GEMINI_TTS_VOICES = (
_MINIMAX_TTS_MAX_AUDIO_HEX_CHARS = 100 * 1024 * 1024
VOXCPM_DEFAULT_BASE_URL = "https://api.modelbest.cn/v1"
VOXCPM_DEFAULT_VOICE = "default"
+VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES = 20 * 1024 * 1024
+VOXCPM_REFERENCE_AUDIO_MAX_WAV_BYTES = 5 * 1024 * 1024
+VOXCPM_REFERENCE_AUDIO_CONVERSION_TIMEOUT_SECONDS = 15
+VOXCPM_REFERENCE_AUDIO_MAX_DURATION_SECONDS = 120
+VOXCPM_REFERENCE_AUDIO_FILE_TYPES = ("wav", "mp3", "m4a", "aac", "ogg", "flac")
_VOXCPM_NON_RETRYABLE_STATUS_CODES = {400, 401, 403, 404, 422}
_VOXCPM_RETRY_DELAY_SECONDS = (1.0, 2.0)
NO_VOICE_NAME = "no-voice"
@@ -595,6 +600,9 @@ def _single_tts(
voice_rate: float,
voice_file: str,
voice_volume: float = 1.0,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
) -> Union[SubMaker, None]:
if is_no_voice(voice_name):
duration_seconds = estimate_no_voice_duration(text)
@@ -703,7 +711,18 @@ def _single_tts(
elif is_voxcpm_voice(voice_name):
voice_id = voice_name.split(":", 1)[1].strip()
if voice_id:
- return voxcpm_tts(text, voice_id, voice_file, voice_rate, voice_volume)
+ if voxcpm_reference_audio is None:
+ return voxcpm_tts(text, voice_id, voice_file, voice_rate, voice_volume)
+ return voxcpm_tts(
+ text,
+ voice_id,
+ voice_file,
+ voice_rate,
+ voice_volume,
+ reference_audio=voxcpm_reference_audio,
+ prompt_audio=voxcpm_prompt_audio,
+ prompt_text=voxcpm_prompt_text,
+ )
logger.error(f"Invalid VoxCPM voice name format: {voice_name}")
return None
return azure_tts_v1(text, voice_name, voice_rate, voice_file)
@@ -979,6 +998,9 @@ def tts(
voice_rate: float,
voice_file: str,
voice_volume: float = 1.0,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
) -> Union[SubMaker, None]:
# 无停顿标签时,原样直通原始文本,避免无意义的正则处理或空白截断
if not utils.has_pause_tags(text):
@@ -988,6 +1010,9 @@ def tts(
voice_rate=voice_rate,
voice_file=voice_file,
voice_volume=voice_volume,
+ voxcpm_reference_audio=voxcpm_reference_audio,
+ voxcpm_prompt_audio=voxcpm_prompt_audio,
+ voxcpm_prompt_text=voxcpm_prompt_text,
)
# 仅 Azure TTS v1 (Edge TTS) 且脚本包含停顿标签时进入分段合成
@@ -1009,6 +1034,9 @@ def tts(
voice_rate=voice_rate,
voice_file=voice_file,
voice_volume=voice_volume,
+ voxcpm_reference_audio=voxcpm_reference_audio,
+ voxcpm_prompt_audio=voxcpm_prompt_audio,
+ voxcpm_prompt_text=voxcpm_prompt_text,
)
@@ -2457,12 +2485,116 @@ def _iter_voxcpm_sse_events(response):
raise ValueError("VoxCPM returned invalid trailing SSE event data") from exc
+def prepare_voxcpm_reference_audio(uploaded_audio: bytes, suffix: str = "") -> bytes:
+ """Convert an uploaded reference clip into ModelBest's bounded WAV payload.
+
+ The WebUI keeps the returned bytes only in its current session and passes a
+ copy to the request that needs it. Temporary source and converted files are
+ always scoped to this function, including FFmpeg timeouts and failures.
+ """
+ if not isinstance(uploaded_audio, bytes) or not uploaded_audio:
+ raise ValueError("reference audio is empty")
+ if len(uploaded_audio) > VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES:
+ raise ValueError("reference audio upload exceeds 20 MiB")
+
+ normalized_suffix = str(suffix or "").lower()
+ if normalized_suffix and not normalized_suffix.startswith("."):
+ normalized_suffix = f".{normalized_suffix}"
+ if normalized_suffix.removeprefix(".") not in VOXCPM_REFERENCE_AUDIO_FILE_TYPES:
+ raise ValueError("unsupported reference audio format")
+
+ try:
+ with tempfile.TemporaryDirectory(prefix="voxcpm-reference-") as temp_dir:
+ input_path = os.path.join(temp_dir, f"input{normalized_suffix or '.audio'}")
+ output_path = os.path.join(temp_dir, "reference.wav")
+ with open(input_path, "wb") as source:
+ source.write(uploaded_audio)
+
+ result = subprocess.run(
+ [
+ utils.get_ffmpeg_binary(),
+ "-nostdin",
+ "-v",
+ "error",
+ "-xerror",
+ "-t",
+ str(VOXCPM_REFERENCE_AUDIO_MAX_DURATION_SECONDS),
+ "-i",
+ input_path,
+ "-map",
+ "0:a:0",
+ "-ac",
+ "1",
+ "-ar",
+ "16000",
+ "-c:a",
+ "pcm_s16le",
+ "-f",
+ "wav",
+ output_path,
+ ],
+ capture_output=True,
+ timeout=VOXCPM_REFERENCE_AUDIO_CONVERSION_TIMEOUT_SECONDS,
+ check=False,
+ )
+ if result.returncode != 0 or not os.path.isfile(output_path):
+ raise ValueError(
+ "reference audio must contain a decodable audio stream"
+ )
+
+ with open(output_path, "rb") as converted:
+ wav_audio = converted.read()
+ if (
+ not wav_audio
+ or len(wav_audio) > VOXCPM_REFERENCE_AUDIO_MAX_WAV_BYTES
+ ):
+ raise ValueError(
+ "converted reference audio exceeds ModelBest's 5 MiB limit"
+ )
+
+ try:
+ with wave.open(io.BytesIO(wav_audio), "rb") as wav_file:
+ if wav_file.getnframes() <= 0:
+ raise ValueError("reference audio is empty")
+ except wave.Error as exc:
+ raise ValueError(
+ "reference audio conversion did not produce a valid WAV"
+ ) from exc
+ return wav_audio
+ except subprocess.TimeoutExpired as exc:
+ raise ValueError("reference audio conversion timed out") from exc
+ except OSError as exc:
+ raise ValueError("failed to convert reference audio") from exc
+
+
+def _encode_voxcpm_audio_data_uri(
+ audio_bytes: bytes | None,
+ field_name: str,
+) -> str | None:
+ if audio_bytes is None:
+ return None
+ if not isinstance(audio_bytes, bytes) or not audio_bytes:
+ raise ValueError(f"{field_name} audio is empty")
+ if len(audio_bytes) > VOXCPM_REFERENCE_AUDIO_MAX_WAV_BYTES:
+ raise ValueError(f"{field_name} audio exceeds ModelBest's 5 MiB limit")
+ try:
+ with wave.open(io.BytesIO(audio_bytes), "rb") as wav_file:
+ if wav_file.getnframes() <= 0:
+ raise ValueError(f"{field_name} audio is empty")
+ except wave.Error as exc:
+ raise ValueError(f"{field_name} audio must be a valid WAV") from exc
+ return "data:audio/wav;base64," + base64.b64encode(audio_bytes).decode("ascii")
+
+
def voxcpm_tts(
text: str,
voice_id: str,
voice_file: str,
voice_rate: float = 1.0,
voice_volume: float = 1.0,
+ reference_audio: bytes | None = None,
+ prompt_audio: bytes | None = None,
+ prompt_text: str = "",
) -> Union[SubMaker, None]:
"""Generate speech through ModelBest's VoxCPM Audio Speech API.
@@ -2507,6 +2639,27 @@ def voxcpm_tts(
"response_format": "wav",
"stream": True,
}
+ prompt_text = str(prompt_text or "").strip()
+ if bool(prompt_audio) != bool(prompt_text):
+ logger.error("VoxCPM prompt audio and prompt text must be provided together")
+ return None
+ try:
+ encoded_reference_audio = _encode_voxcpm_audio_data_uri(
+ reference_audio,
+ "reference",
+ )
+ encoded_prompt_audio = _encode_voxcpm_audio_data_uri(
+ prompt_audio,
+ "prompt",
+ )
+ except ValueError as exc:
+ logger.error(f"VoxCPM audio prompt is invalid: {exc}")
+ return None
+ if encoded_reference_audio:
+ payload["ref_audio"] = encoded_reference_audio
+ if encoded_prompt_audio:
+ payload["prompt_audio"] = encoded_prompt_audio
+ payload["prompt_text"] = prompt_text
_configure_pydub_ffmpeg(AudioSegment)
for attempt in range(3):
diff --git a/app/services/webui_task.py b/app/services/webui_task.py
index 498ddb4e..f3c10fb4 100644
--- a/app/services/webui_task.py
+++ b/app/services/webui_task.py
@@ -56,6 +56,9 @@ def _run_generation(
capture_logs: bool,
voice_preview: dict | None = None,
loomloom_video_request: LoomLoomConfirmedVideoRequest | None = None,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
) -> dict:
"""
在后台线程中执行现有视频流水线。
@@ -84,6 +87,9 @@ def _run_generation(
params=params,
voice_preview=voice_preview,
loomloom_video_request=loomloom_video_request,
+ voxcpm_reference_audio=voxcpm_reference_audio,
+ voxcpm_prompt_audio=voxcpm_prompt_audio,
+ voxcpm_prompt_text=voxcpm_prompt_text,
)
except Exception as exc:
# tm.start 已负责把流水线异常转换成失败状态;这里额外保护日志 sink、
@@ -125,6 +131,9 @@ def submit_generation(
capture_logs: bool = True,
voice_preview: dict | None = None,
loomloom_video_request: LoomLoomConfirmedVideoRequest | None = None,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
) -> None:
"""
登记并提交 WebUI 视频生成任务,调用后立即返回。
@@ -136,6 +145,18 @@ def submit_generation(
# 预览载荷只包含不可变音频路径、参数快照和只读字幕时间轴。复制外层字典,
# 避免页面后续 rerun 替换缓存字段时影响已经提交到后台队列的任务。
voice_preview_snapshot = dict(voice_preview) if voice_preview else None
+ # Reference audio belongs only to this queued request. It is deliberately
+ # separate from VideoParams so it cannot reach task history, presets, state
+ # persistence, or logs. ``bytes`` is immutable; make an explicit snapshot
+ # before the background worker starts so later WebUI reruns cannot share
+ # mutable upload state with this task.
+ voxcpm_reference_audio_snapshot = (
+ bytes(voxcpm_reference_audio) if voxcpm_reference_audio else None
+ )
+ voxcpm_prompt_audio_snapshot = (
+ bytes(voxcpm_prompt_audio) if voxcpm_prompt_audio else None
+ )
+ voxcpm_prompt_text_snapshot = str(voxcpm_prompt_text or "")
# 已确认请求是冻结的数据对象,只在当前进程内传递。API Key 不会进入
# VideoParams、任务状态、日志或落盘历史,也不会受后续页面 rerun 影响。
loomloom_request_snapshot = loomloom_video_request
@@ -153,6 +174,9 @@ def submit_generation(
capture_logs=capture_logs,
voice_preview=voice_preview_snapshot,
loomloom_video_request=loomloom_request_snapshot,
+ voxcpm_reference_audio=voxcpm_reference_audio_snapshot,
+ voxcpm_prompt_audio=voxcpm_prompt_audio_snapshot,
+ voxcpm_prompt_text=voxcpm_prompt_text_snapshot,
)
except Exception as exc:
# 调度失败与流水线失败一样必须成为可查询状态,避免任务管理器永久显示
diff --git a/test/services/test_subtitle.py b/test/services/test_subtitle.py
index 7b09ed73..7b4f5012 100644
--- a/test/services/test_subtitle.py
+++ b/test/services/test_subtitle.py
@@ -19,6 +19,60 @@ class TestSubtitleService(unittest.TestCase):
missing_file = Path(tmp_dir) / "missing.srt"
self.assertEqual(subtitle.file_to_subtitles(str(missing_file)), [])
+ def test_transcribe_audio_bytes_returns_editable_text_and_cleans_temps(self):
+ temporary_paths = []
+
+ def fake_create(audio_file, subtitle_file, word_level=False, log_details=True):
+ self.assertFalse(word_level)
+ self.assertFalse(log_details)
+ temporary_paths.extend([Path(audio_file), Path(subtitle_file)])
+ self.assertEqual(Path(audio_file).read_bytes(), b"wav-audio")
+ Path(subtitle_file).write_text(
+ "1\n00:00:00,000 --> 00:00:01,000\n第一句\n\n"
+ "2\n00:00:01,000 --> 00:00:02,000\nOpen A I\n\n",
+ encoding="utf-8",
+ )
+
+ with patch.object(subtitle, "create", side_effect=fake_create):
+ transcript = subtitle.transcribe_audio_bytes(b"wav-audio")
+
+ self.assertEqual(transcript, "第一句 Open A I")
+ self.assertTrue(all(not path.exists() for path in temporary_paths))
+
+ def test_transcribe_audio_bytes_returns_empty_when_whisper_fails(self):
+ with patch.object(subtitle, "create", return_value=""):
+ self.assertEqual(subtitle.transcribe_audio_bytes(b"wav-audio"), "")
+
+ def test_create_can_hide_sensitive_paths_and_recognized_text(self):
+ class _FakeWhisperModel:
+ def __init__(self, **_kwargs):
+ pass
+
+ def transcribe(self, _audio_file, **_kwargs):
+ word = SimpleNamespace(start=0.0, end=0.5, word="private transcript.")
+ segment = SimpleNamespace(start=0.0, end=0.5, words=[word])
+ info = SimpleNamespace(language="en", language_probability=0.99)
+ return [segment], info
+
+ with tempfile.TemporaryDirectory() as tmp_dir:
+ subtitle_file = Path(tmp_dir) / "private-reference.srt"
+ with (
+ patch.object(subtitle, "model", None),
+ patch.object(subtitle, "WhisperModel", _FakeWhisperModel),
+ patch.object(subtitle.logger, "info") as info,
+ patch.object(subtitle.logger, "debug") as debug,
+ ):
+ subtitle.create(
+ str(Path(tmp_dir) / "private-reference.wav"),
+ str(subtitle_file),
+ log_details=False,
+ )
+
+ logged = " ".join(str(call) for call in info.call_args_list)
+ self.assertNotIn("private-reference", logged)
+ self.assertNotIn("private transcript", logged)
+ debug.assert_not_called()
+
def test_levenshtein_distance_and_similarity_cover_common_boundaries(self):
"""
字幕校正依赖编辑距离选择是否继续合并相邻字幕,因此覆盖空字符串、
diff --git a/test/services/test_voxcpm.py b/test/services/test_voxcpm.py
index acf26ea1..0c1eec2f 100644
--- a/test/services/test_voxcpm.py
+++ b/test/services/test_voxcpm.py
@@ -1,7 +1,10 @@
"""ModelBest VoxCPM provider tests without external credentials or network access."""
import base64
+import io
import json
+import subprocess
+import wave
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock
@@ -35,6 +38,16 @@ def _sse_lines(*events):
return lines
+def _wav_bytes() -> bytes:
+ buffer = io.BytesIO()
+ with wave.open(buffer, "wb") as wav_file:
+ wav_file.setnchannels(1)
+ wav_file.setsampwidth(2)
+ wav_file.setframerate(16000)
+ wav_file.writeframes(b"\x00\x00" * 160)
+ return buffer.getvalue()
+
+
@pytest.fixture
def voxcpm_config(monkeypatch):
settings = {
@@ -113,6 +126,127 @@ def test_voxcpm_tts_assembles_sse_wav_and_converts_to_mp3(
response.close.assert_called_once()
+def test_voxcpm_tts_sends_identity_and_delivery_references_as_data_uris(
+ monkeypatch, tmp_path, voxcpm_config
+):
+ response = SimpleNamespace(
+ status_code=400,
+ text="invalid request",
+ close=lambda: None,
+ )
+ post = Mock(return_value=response)
+ monkeypatch.setattr(voice.requests, "post", post)
+ reference_audio = _wav_bytes()
+ prompt_audio = _wav_bytes()
+ prompt_text = "准确的演绎示范逐字稿。"
+
+ assert (
+ voice.voxcpm_tts(
+ "Hello",
+ "default",
+ str(tmp_path / "voice.mp3"),
+ reference_audio=reference_audio,
+ prompt_audio=prompt_audio,
+ prompt_text=prompt_text,
+ )
+ is None
+ )
+ assert post.call_args.kwargs["json"]["ref_audio"] == (
+ "data:audio/wav;base64," + base64.b64encode(reference_audio).decode("ascii")
+ )
+ assert post.call_args.kwargs["json"]["prompt_audio"] == (
+ "data:audio/wav;base64," + base64.b64encode(prompt_audio).decode("ascii")
+ )
+ assert post.call_args.kwargs["json"]["prompt_text"] == prompt_text
+
+
+ ("prompt_audio", "prompt_text"),
+ [(_wav_bytes(), ""), (None, "演绎示范")],
+)
+def test_voxcpm_tts_requires_prompt_audio_and_text_together(
+ monkeypatch,
+ tmp_path,
+ voxcpm_config,
+ prompt_audio,
+ prompt_text,
+):
+ post = Mock()
+ monkeypatch.setattr(voice.requests, "post", post)
+
+ assert (
+ voice.voxcpm_tts(
+ "Hello",
+ "default",
+ str(tmp_path / "voice.mp3"),
+ reference_audio=_wav_bytes(),
+ prompt_audio=prompt_audio,
+ prompt_text=prompt_text,
+ )
+ is None
+ )
+ post.assert_not_called()
+
+
+def test_voxcpm_tts_rejects_invalid_reference_audio_without_request(
+ monkeypatch, tmp_path, voxcpm_config
+):
+ post = Mock()
+ monkeypatch.setattr(voice.requests, "post", post)
+
+ assert (
+ voice.voxcpm_tts(
+ "Hello",
+ "default",
+ str(tmp_path / "voice.mp3"),
+ reference_audio=b"not-a-wav",
+ )
+ is None
+ )
+ post.assert_not_called()
+
+
+def test_prepare_voxcpm_reference_audio_bounds_conversion_and_cleans_temps(monkeypatch):
+ source_audio = b"uploaded-reference"
+ temporary_paths = []
+
+ def convert(command, **_kwargs):
+ input_path = Path(command[command.index("-i") + 1])
+ output_path = Path(command[-1])
+ temporary_paths.extend([input_path, output_path])
+ assert input_path.read_bytes() == source_audio
+ with wave.open(str(output_path), "wb") as wav_file:
+ wav_file.setnchannels(1)
+ wav_file.setsampwidth(2)
+ wav_file.setframerate(16000)
+ wav_file.writeframes(b"\x00\x00" * 160)
+ return SimpleNamespace(returncode=0)
+
+ monkeypatch.setattr(voice.subprocess, "run", convert)
+
+ normalized = voice.prepare_voxcpm_reference_audio(source_audio, ".mp3")
+
+ assert normalized == _wav_bytes()
+ assert all(not path.exists() for path in temporary_paths)
+
+
+def test_prepare_voxcpm_reference_audio_rejects_oversized_upload(monkeypatch):
+ monkeypatch.setattr(voice, "VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES", 2)
+
+ with pytest.raises(ValueError, match="exceeds"):
+ voice.prepare_voxcpm_reference_audio(b"123", ".wav")
+
+
+def test_prepare_voxcpm_reference_audio_reports_conversion_timeout(monkeypatch):
+ def timeout(*_args, **_kwargs):
+ raise subprocess.TimeoutExpired("ffmpeg", 15)
+
+ monkeypatch.setattr(voice.subprocess, "run", timeout)
+
+ with pytest.raises(ValueError, match="timed out"):
+ voice.prepare_voxcpm_reference_audio(b"audio", ".wav")
+
+
@pytest.mark.parametrize(
"events",
[
diff --git a/test/services/test_webui_i18n.py b/test/services/test_webui_i18n.py
index 268f6f57..33a28f70 100644
--- a/test/services/test_webui_i18n.py
+++ b/test/services/test_webui_i18n.py
@@ -106,6 +106,33 @@ ENGLISH_FALLBACK_KEYS = frozenset(
"Stop Tracking LoomLoom Run Help",
"Unavailable AI Video Model",
"VoxCPM Speed Not Supported",
+ "VoxCPM Reference Audio",
+ "VoxCPM Reference Audio Help",
+ "VoxCPM Reference Audio Notice",
+ "VoxCPM Reference Audio Empty",
+ "VoxCPM Reference Audio Upload Too Large",
+ "Validating VoxCPM Reference Audio",
+ "VoxCPM Reference Audio Invalid",
+ "VoxCPM High Fidelity Delivery",
+ "VoxCPM High Fidelity Delivery Help",
+ "VoxCPM Separate Prompt Audio",
+ "VoxCPM Separate Prompt Audio Help",
+ "VoxCPM Prompt Audio",
+ "VoxCPM Prompt Audio Help",
+ "VoxCPM Prompt Text",
+ "VoxCPM Prompt Text Help",
+ "Transcribe VoxCPM Prompt Audio",
+ "Transcribe VoxCPM Prompt Audio Help",
+ "Transcribing VoxCPM Prompt Audio",
+ "VoxCPM Prompt Audio Transcribed",
+ "VoxCPM Prompt Audio Transcription Failed",
+ "VoxCPM Prompt Transcript Review",
+ "VoxCPM Prompt Audio Empty",
+ "VoxCPM Prompt Audio Upload Too Large",
+ "Validating VoxCPM Prompt Audio",
+ "VoxCPM Prompt Text Required",
+ "VoxCPM Prompt Pair Required",
+ "VoxCPM Prompt Invalid",
"None (Animation)",
}
)
diff --git a/test/services/test_webui_task.py b/test/services/test_webui_task.py
index 37258e17..aeb32cb1 100644
--- a/test/services/test_webui_task.py
+++ b/test/services/test_webui_task.py
@@ -309,6 +309,36 @@ def test_submit_generation_copies_params_before_starting_worker():
webui_task.sm.state.delete_task("copied-params-test")
+def test_submit_generation_keeps_voxcpm_reference_audio_out_of_params():
+ """参考音频仅属于内存中的当前请求,不能进入可持久化任务参数。"""
+ params = VideoParams(video_subject="task isolation")
+ reference_audio = b"bounded-reference-wav"
+ prompt_audio = b"bounded-prompt-wav"
+ prompt_text = "delivery transcript"
+ with patch.object(webui_task._task_manager, "add_task") as add_task:
+ webui_task.submit_generation(
+ "reference-audio-isolation",
+ params,
+ capture_logs=False,
+ voxcpm_reference_audio=reference_audio,
+ voxcpm_prompt_audio=prompt_audio,
+ voxcpm_prompt_text=prompt_text,
+ )
+
+ submitted_params = add_task.call_args.kwargs["params"]
+ serialized_params = submitted_params.model_dump_json()
+ assert "voxcpm_reference_audio" not in serialized_params
+ assert "voxcpm_prompt_audio" not in serialized_params
+ assert "voxcpm_prompt_text" not in serialized_params
+ assert reference_audio.decode("ascii") not in serialized_params
+ assert prompt_audio.decode("ascii") not in serialized_params
+ assert prompt_text not in serialized_params
+ assert add_task.call_args.kwargs["voxcpm_reference_audio"] == reference_audio
+ assert add_task.call_args.kwargs["voxcpm_prompt_audio"] == prompt_audio
+ assert add_task.call_args.kwargs["voxcpm_prompt_text"] == prompt_text
+ webui_task.sm.state.delete_task("reference-audio-isolation")
+
+
def test_scheduling_failure_is_saved_as_terminal_task_state():
"""队列或线程启动失败时不能让任务管理器永久停留在“生成中”。"""
task_id = "scheduling-failure-test"
@@ -363,6 +393,32 @@ def test_worker_logs_are_available_without_streamlit_session_state():
)
+def test_webui_worker_forwards_reference_audio_to_pipeline():
+ reference_audio = b"task-local-reference-wav"
+ prompt_audio = b"task-local-prompt-wav"
+ prompt_text = "task-local transcript"
+ with (
+ patch.object(webui_task.tm, "start", return_value={"videos": []}) as start,
+ patch.object(
+ webui_task.config,
+ "runtime_config_lock",
+ return_value=nullcontext(),
+ ),
+ ):
+ webui_task._run_generation(
+ "reference-audio-forwarding",
+ VideoParams(video_subject="reference forwarding"),
+ capture_logs=False,
+ voxcpm_reference_audio=reference_audio,
+ voxcpm_prompt_audio=prompt_audio,
+ voxcpm_prompt_text=prompt_text,
+ )
+
+ assert start.call_args.kwargs["voxcpm_reference_audio"] == reference_audio
+ assert start.call_args.kwargs["voxcpm_prompt_audio"] == prompt_audio
+ assert start.call_args.kwargs["voxcpm_prompt_text"] == prompt_text
+
+
def test_log_paths_stay_posix_style_on_every_platform():
"""
调用位置必须始终显示为 ``./app/services/task.py``。
diff --git a/test/services/test_webui_tts_settings.py b/test/services/test_webui_tts_settings.py
index 3cd51fdd..9312b09f 100644
--- a/test/services/test_webui_tts_settings.py
+++ b/test/services/test_webui_tts_settings.py
@@ -138,6 +138,38 @@ def test_voxcpm_settings_render_model_and_endpoint_fields():
assert [str(item.value) for item in app.exception] == []
+def test_voxcpm_settings_render_reference_audio_with_cloud_notice():
+ test_config = dict(
+ config.voxcpm,
+ api_key="",
+ model_id="speech-model",
+ base_url=voice.VOXCPM_DEFAULT_BASE_URL,
+ voice_id=voice.VOXCPM_DEFAULT_VOICE,
+ )
+ test_ui = dict(
+ config.ui,
+ voice_mode="tts",
+ tts_server="voxcpm",
+ voice_name="voxcpm:default",
+ )
+
+ with (
+ patch.object(config, "voxcpm", test_config),
+ patch.object(config, "ui", test_ui),
+ patch.object(config, "try_save_config", return_value=True),
+ ):
+ app = AppTest.from_file(str(WEBUI_MAIN), default_timeout=30)
+ app.session_state["ui_language"] = "en"
+ app.run()
+
+ uploader = _widget_by_key(app.file_uploader, "voxcpm_reference_audio_uploader")
+ assert uploader.label == "VoxCPM Reference Audio (Optional)"
+ assert "short reference clip" in uploader.help
+ assert any("sent to ModelBest" in str(item.value) for item in app.caption)
+ assert "voxcpm_reference_audio" not in test_config
+ assert [str(item.value) for item in app.exception] == []
+
+
def test_voxcpm_reconnect_restores_saved_key_instead_of_clearing_it():
"""Streamlit 重连重放空密码状态时,VoxCPM Key 仍应保留。"""
test_config = dict(config.voxcpm, api_key="saved-voxcpm-key")
diff --git a/test/services/test_webui_voice_preview.py b/test/services/test_webui_voice_preview.py
index 1ff7c614..07bc4464 100644
--- a/test/services/test_webui_voice_preview.py
+++ b/test/services/test_webui_voice_preview.py
@@ -36,7 +36,7 @@ def _load_duration_estimator():
return namespace["_estimate_voiceover_duration_range"]
-def _load_provider_signature(test_config):
+def _load_provider_signature(test_config, session_state=None):
"""加载凭证摘要和 Provider 指纹函数,独立验证缓存失效规则。"""
tree = ast.parse(WEBUI_MAIN.read_text(encoding="utf-8"))
functions = [
@@ -46,15 +46,104 @@ def _load_provider_signature(test_config):
and node.name
in {
"_credential_signature",
+ "_get_voxcpm_reference_audio_digest",
+ "_get_voxcpm_prompt_audio_digest",
+ "_get_voxcpm_prompt_text",
+ "_get_voxcpm_effective_prompt_audio_digest",
"_get_voice_preview_provider_signature",
}
]
module = ast.Module(body=functions, type_ignores=[])
- namespace = {"hashlib": hashlib, "config": test_config}
+ namespace = {
+ "hashlib": hashlib,
+ "config": test_config,
+ "st": SimpleNamespace(
+ session_state={} if session_state is None else session_state
+ ),
+ "VOXCPM_REFERENCE_AUDIO_SESSION_KEY": "voxcpm_reference_audio",
+ "VOXCPM_PROMPT_AUDIO_SESSION_KEY": "voxcpm_prompt_audio",
+ "VOXCPM_PROMPT_TEXT_SESSION_KEY": "voxcpm_prompt_text_input",
+ "VOXCPM_HIGH_FIDELITY_SESSION_KEY": "voxcpm_high_fidelity_enabled",
+ "VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY": (
+ "voxcpm_separate_prompt_audio_enabled"
+ ),
+ }
exec(compile(module, str(WEBUI_MAIN), "exec"), namespace)
return namespace["_get_voice_preview_provider_signature"]
+def _load_voxcpm_state_helpers(session_state):
+ """Load upload-state helpers without executing the full Streamlit app."""
+ tree = ast.parse(WEBUI_MAIN.read_text(encoding="utf-8"))
+ function_names = {
+ "_get_voxcpm_reference_audio",
+ "_get_voxcpm_reference_audio_digest",
+ "_get_voxcpm_prompt_audio",
+ "_get_voxcpm_prompt_audio_digest",
+ "_get_voxcpm_prompt_text",
+ "_clear_voxcpm_separate_prompt_audio",
+ "_clear_voxcpm_prompt_state",
+ "_clear_voxcpm_prompt_transcript",
+ "_sync_voxcpm_prompt_example_mode",
+ "_get_voxcpm_effective_prompt_audio",
+ "_get_voxcpm_effective_prompt_audio_digest",
+ "_get_voxcpm_prompt_validation_error",
+ "_get_voxcpm_preview_validation_error",
+ "_sync_voxcpm_reference_audio",
+ "_sync_voxcpm_prompt_audio",
+ }
+ functions = [
+ node
+ for node in tree.body
+ if isinstance(node, ast.FunctionDef) and node.name in function_names
+ ]
+ module = ast.Module(body=functions, type_ignores=[])
+ namespace = {
+ "Path": Path,
+ "hashlib": hashlib,
+ "st": SimpleNamespace(
+ session_state=session_state,
+ spinner=lambda _label: nullcontext(),
+ ),
+ "tr": lambda key: key,
+ "voice": SimpleNamespace(
+ VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES=1024,
+ prepare_voxcpm_reference_audio=lambda audio, _suffix: audio,
+ is_voxcpm_voice=lambda name: str(name or "").startswith("voxcpm:"),
+ ),
+ "VOXCPM_REFERENCE_AUDIO_SESSION_KEY": "voxcpm_reference_audio",
+ "VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY": "voxcpm_reference_audio_error",
+ "VOXCPM_PROMPT_AUDIO_SESSION_KEY": "voxcpm_prompt_audio",
+ "VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY": "voxcpm_prompt_audio_error",
+ "VOXCPM_PROMPT_TEXT_SESSION_KEY": "voxcpm_prompt_text_input",
+ "VOXCPM_HIGH_FIDELITY_SESSION_KEY": "voxcpm_high_fidelity_enabled",
+ "VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY": (
+ "voxcpm_separate_prompt_audio_enabled"
+ ),
+ "VOXCPM_PROMPT_EXAMPLE_MODE_SESSION_KEY": "voxcpm_prompt_example_mode",
+ }
+ exec(compile(module, str(WEBUI_MAIN), "exec"), namespace)
+ return namespace
+
+
+class _FakeAudioUpload:
+ def __init__(self, name: str, payload: bytes):
+ self.name = name
+ self.size = len(payload)
+ self._payload = payload
+
+ def getvalue(self):
+ return self._payload
+
+
+def _session_audio(payload: bytes) -> dict:
+ return {
+ "audio_bytes": payload,
+ "audio_digest": hashlib.sha256(payload).hexdigest(),
+ "upload_digest": hashlib.sha256(payload).hexdigest(),
+ }
+
+
def _button_by_key(app, key):
return next(
button
@@ -110,6 +199,7 @@ def test_provider_signature_changes_when_api_key_changes():
def test_voxcpm_preview_signature_tracks_endpoint_model_and_credentials():
+ session_state = {}
test_config = SimpleNamespace(
app={}, azure={}, siliconflow={}, elevenlabs={}, chatterbox={}, kokoro={},
voxcpm={
@@ -119,19 +209,95 @@ def test_voxcpm_preview_signature_tracks_endpoint_model_and_credentials():
"voice_id": "default",
},
)
- provider_signature = _load_provider_signature(test_config)
+ provider_signature = _load_provider_signature(test_config, session_state)
original = provider_signature("voxcpm")
test_config.voxcpm["model_id"] = "speech-model-b"
changed_model = provider_signature("voxcpm")
test_config.voxcpm["api_key"] = "new-key"
changed_key = provider_signature("voxcpm")
-
- assert original != changed_model != changed_key
+ session_state["voxcpm_reference_audio"] = {"audio_digest": "new-reference"}
+ changed_reference = provider_signature("voxcpm")
+ session_state["voxcpm_high_fidelity_enabled"] = True
+ session_state["voxcpm_prompt_text_input"] = "delivery transcript"
+ reused_reference_as_prompt = provider_signature("voxcpm")
+ session_state["voxcpm_separate_prompt_audio_enabled"] = True
+ session_state["voxcpm_prompt_audio"] = {"audio_digest": "new-prompt"}
+ changed_prompt = provider_signature("voxcpm")
+
+ assert original != changed_model != changed_key != changed_reference
+ assert changed_reference != reused_reference_as_prompt != changed_prompt
assert "old-key" not in str(original)
assert "new-key" not in str(changed_key)
+def test_replacing_shared_reference_clears_its_transcript():
+ session_state = {
+ "voxcpm_reference_audio": _session_audio(b"clip-a"),
+ "voxcpm_high_fidelity_enabled": True,
+ "voxcpm_separate_prompt_audio_enabled": False,
+ "voxcpm_prompt_text_input": "clip A transcript",
+ }
+ helpers = _load_voxcpm_state_helpers(session_state)
+
+ helpers["_sync_voxcpm_reference_audio"](
+ _FakeAudioUpload("clip-b.wav", b"clip-b")
+ )
+
+ assert "voxcpm_prompt_text_input" not in session_state
+
+
+def test_replacing_separate_delivery_audio_clears_transcript_but_not_identity_change():
+ session_state = {
+ "voxcpm_reference_audio": _session_audio(b"identity-a"),
+ "voxcpm_prompt_audio": _session_audio(b"delivery-a"),
+ "voxcpm_high_fidelity_enabled": True,
+ "voxcpm_separate_prompt_audio_enabled": True,
+ "voxcpm_prompt_text_input": "delivery A transcript",
+ }
+ helpers = _load_voxcpm_state_helpers(session_state)
+
+ helpers["_sync_voxcpm_reference_audio"](
+ _FakeAudioUpload("identity-b.wav", b"identity-b")
+ )
+ assert session_state["voxcpm_prompt_text_input"] == "delivery A transcript"
+
+ helpers["_sync_voxcpm_prompt_audio"](
+ _FakeAudioUpload("delivery-b.wav", b"delivery-b")
+ )
+ assert "voxcpm_prompt_text_input" not in session_state
+
+
+def test_switching_prompt_example_modes_clears_transcript():
+ session_state = {
+ "voxcpm_prompt_example_mode": False,
+ "voxcpm_prompt_text_input": "shared transcript",
+ }
+ helpers = _load_voxcpm_state_helpers(session_state)
+
+ helpers["_sync_voxcpm_prompt_example_mode"](True)
+
+ assert "voxcpm_prompt_text_input" not in session_state
+
+
+def test_invalid_reference_upload_blocks_voxcpm_preview_only():
+ session_state = {"voxcpm_reference_audio_error": "invalid WAV"}
+ helpers = _load_voxcpm_state_helpers(session_state)
+
+ assert (
+ helpers["_get_voxcpm_preview_validation_error"](
+ "voxcpm", "voxcpm:default"
+ )
+ == "invalid WAV"
+ )
+ assert (
+ helpers["_get_voxcpm_preview_validation_error"](
+ "azure-tts-v1", "zh-CN-XiaoxiaoNeural-Female"
+ )
+ == ""
+ )
+
+
def test_full_voiceover_preview_is_disabled_until_script_exists():
"""完整预览必须由用户主动触发,文案为空时不能误调用商业 TTS。"""
test_ui = dict(
diff --git a/webui/Main.py b/webui/Main.py
index 505987b9..8c0be499 100644
--- a/webui/Main.py
+++ b/webui/Main.py
@@ -52,6 +52,7 @@ from app.services import (
metaso_minimax,
muapi,
ofox,
+ subtitle,
video,
volcengine_seedance,
voice,
@@ -104,6 +105,14 @@ DEFAULT_KOKORO_MODEL = "kokoro"
DEFAULT_KOKORO_VOICES: list[str] = []
DEFAULT_VOXCPM_BASE_URL = voice.VOXCPM_DEFAULT_BASE_URL
DEFAULT_VOXCPM_VOICE = voice.VOXCPM_DEFAULT_VOICE
+VOXCPM_REFERENCE_AUDIO_SESSION_KEY = "voxcpm_reference_audio"
+VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY = "voxcpm_reference_audio_error"
+VOXCPM_PROMPT_AUDIO_SESSION_KEY = "voxcpm_prompt_audio"
+VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY = "voxcpm_prompt_audio_error"
+VOXCPM_PROMPT_TEXT_SESSION_KEY = "voxcpm_prompt_text_input"
+VOXCPM_HIGH_FIDELITY_SESSION_KEY = "voxcpm_high_fidelity_enabled"
+VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY = "voxcpm_separate_prompt_audio_enabled"
+VOXCPM_PROMPT_EXAMPLE_MODE_SESSION_KEY = "voxcpm_prompt_example_mode"
ONBOARDING_TOUR_KEY = "mpt-onboarding-v1"
CUSTOM_LLM_ENDPOINT_ID = "custom"
VOICE_MODE_TTS = "tts"
@@ -1564,6 +1573,16 @@ def _apply_restored_params(params):
# 同时清空当前页面已缓存的上传素材,避免恢复后误用另一个任务的文件。
st.session_state["local_video_materials"] = []
st.session_state.pop("custom_audio_file_uploader", None)
+ st.session_state.pop("voxcpm_reference_audio_uploader", None)
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY, None)
+ st.session_state.pop("voxcpm_prompt_audio_uploader", None)
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_PROMPT_TEXT_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_HIGH_FIDELITY_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_PROMPT_EXAMPLE_MODE_SESSION_KEY, None)
st.session_state.pop("custom_bgm_uploader", None)
st.session_state.pop("custom_bgm_validation", None)
st.session_state["task_restore_upload_requirements"] = (
@@ -5590,6 +5609,227 @@ def _credential_signature(value: str) -> str:
return hashlib.sha256(normalized_value.encode("utf-8")).hexdigest()
+def _get_voxcpm_reference_audio() -> bytes | None:
+ """Return the current session's normalized reference audio, if any."""
+ payload = st.session_state.get(VOXCPM_REFERENCE_AUDIO_SESSION_KEY)
+ if not isinstance(payload, dict):
+ return None
+ audio_bytes = payload.get("audio_bytes")
+ return bytes(audio_bytes) if isinstance(audio_bytes, bytes) else None
+
+
+def _get_voxcpm_reference_audio_digest() -> str:
+ payload = st.session_state.get(VOXCPM_REFERENCE_AUDIO_SESSION_KEY)
+ if not isinstance(payload, dict):
+ return ""
+ digest = payload.get("audio_digest")
+ return str(digest) if isinstance(digest, str) else ""
+
+
+def _get_voxcpm_prompt_audio() -> bytes | None:
+ payload = st.session_state.get(VOXCPM_PROMPT_AUDIO_SESSION_KEY)
+ if not isinstance(payload, dict):
+ return None
+ audio_bytes = payload.get("audio_bytes")
+ return bytes(audio_bytes) if isinstance(audio_bytes, bytes) else None
+
+
+def _get_voxcpm_prompt_audio_digest() -> str:
+ payload = st.session_state.get(VOXCPM_PROMPT_AUDIO_SESSION_KEY)
+ if not isinstance(payload, dict):
+ return ""
+ digest = payload.get("audio_digest")
+ return str(digest) if isinstance(digest, str) else ""
+
+
+def _get_voxcpm_prompt_text() -> str:
+ return str(st.session_state.get(VOXCPM_PROMPT_TEXT_SESSION_KEY, "") or "").strip()
+
+
+def _clear_voxcpm_separate_prompt_audio() -> None:
+ st.session_state.pop("voxcpm_prompt_audio_uploader", None)
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY, None)
+
+
+def _clear_voxcpm_prompt_state() -> None:
+ _clear_voxcpm_separate_prompt_audio()
+ _clear_voxcpm_prompt_transcript()
+ st.session_state.pop(VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_PROMPT_EXAMPLE_MODE_SESSION_KEY, None)
+
+
+def _clear_voxcpm_prompt_transcript() -> None:
+ st.session_state.pop(VOXCPM_PROMPT_TEXT_SESSION_KEY, None)
+
+
+def _sync_voxcpm_prompt_example_mode(use_separate_prompt_audio: bool) -> None:
+ """Invalidate the transcript whenever its effective example changes mode."""
+ previous_mode = st.session_state.get(VOXCPM_PROMPT_EXAMPLE_MODE_SESSION_KEY)
+ current_mode = bool(use_separate_prompt_audio)
+ if previous_mode is not None and bool(previous_mode) != current_mode:
+ _clear_voxcpm_prompt_transcript()
+ st.session_state[VOXCPM_PROMPT_EXAMPLE_MODE_SESSION_KEY] = current_mode
+
+
+def _get_voxcpm_effective_prompt_audio() -> bytes | None:
+ if not st.session_state.get(VOXCPM_HIGH_FIDELITY_SESSION_KEY, False):
+ return None
+ if st.session_state.get(VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY, False):
+ return _get_voxcpm_prompt_audio()
+ return _get_voxcpm_reference_audio()
+
+
+def _get_voxcpm_effective_prompt_audio_digest() -> str:
+ if not st.session_state.get(VOXCPM_HIGH_FIDELITY_SESSION_KEY, False):
+ return ""
+ if st.session_state.get(VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY, False):
+ return _get_voxcpm_prompt_audio_digest()
+ return _get_voxcpm_reference_audio_digest()
+
+
+def _get_voxcpm_prompt_validation_error() -> str:
+ if not st.session_state.get(VOXCPM_HIGH_FIDELITY_SESSION_KEY, False):
+ return ""
+ upload_error = st.session_state.get(VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY)
+ if upload_error:
+ return str(upload_error)
+ prompt_text = _get_voxcpm_prompt_text()
+ if not prompt_text:
+ return tr("VoxCPM Prompt Text Required")
+ if (
+ st.session_state.get(VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY, False)
+ and not _get_voxcpm_prompt_audio()
+ ):
+ return tr("VoxCPM Prompt Pair Required")
+ return ""
+
+
+def _get_voxcpm_preview_validation_error(
+ selected_tts_server: str,
+ voice_name: str,
+) -> str:
+ if selected_tts_server != "voxcpm" and not voice.is_voxcpm_voice(voice_name):
+ return ""
+ reference_error = st.session_state.get(VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY)
+ if reference_error:
+ return str(reference_error)
+ return _get_voxcpm_prompt_validation_error()
+
+
+def _sync_voxcpm_reference_audio(uploaded_file) -> bytes | None:
+ """Normalize a new upload once and keep only bounded WAV bytes in session."""
+ if uploaded_file is None:
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY, None)
+ _clear_voxcpm_prompt_state()
+ st.session_state.pop(VOXCPM_HIGH_FIDELITY_SESSION_KEY, None)
+ return None
+
+ upload_size = int(getattr(uploaded_file, "size", 0) or 0)
+ if upload_size <= 0:
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_SESSION_KEY, None)
+ _clear_voxcpm_prompt_state()
+ st.session_state[VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY] = tr(
+ "VoxCPM Reference Audio Empty"
+ )
+ return None
+ if upload_size > voice.VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES:
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_SESSION_KEY, None)
+ _clear_voxcpm_prompt_state()
+ st.session_state[VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY] = tr(
+ "VoxCPM Reference Audio Upload Too Large"
+ ).format(limit=voice.VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES // (1024 * 1024))
+ return None
+
+ previous_digest = _get_voxcpm_reference_audio_digest()
+ raw_audio = uploaded_file.getvalue()
+ upload_digest = hashlib.sha256(raw_audio).hexdigest()
+ cached = st.session_state.get(VOXCPM_REFERENCE_AUDIO_SESSION_KEY)
+ if isinstance(cached, dict) and cached.get("upload_digest") == upload_digest:
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY, None)
+ return _get_voxcpm_reference_audio()
+
+ try:
+ with st.spinner(tr("Validating VoxCPM Reference Audio")):
+ wav_audio = voice.prepare_voxcpm_reference_audio(
+ raw_audio,
+ Path(str(getattr(uploaded_file, "name", ""))).suffix,
+ )
+ except ValueError as exc:
+ # Do not retain a prior clip after the user selects a replacement that
+ # fails validation. Otherwise preview and generation could silently use
+ # a different person's voice than the one currently shown in the UI.
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_SESSION_KEY, None)
+ _clear_voxcpm_prompt_state()
+ st.session_state[VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY] = str(exc)
+ return None
+
+ st.session_state[VOXCPM_REFERENCE_AUDIO_SESSION_KEY] = {
+ "upload_digest": upload_digest,
+ "audio_digest": hashlib.sha256(wav_audio).hexdigest(),
+ "audio_bytes": wav_audio,
+ }
+ if (
+ previous_digest != hashlib.sha256(wav_audio).hexdigest()
+ and not st.session_state.get(VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY, False)
+ ):
+ _clear_voxcpm_prompt_transcript()
+ st.session_state.pop(VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY, None)
+ return bytes(wav_audio)
+
+
+def _sync_voxcpm_prompt_audio(uploaded_file) -> bytes | None:
+ """Normalize the optional delivery example without persisting its content."""
+ if uploaded_file is None:
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY, None)
+ return None
+
+ upload_size = int(getattr(uploaded_file, "size", 0) or 0)
+ if upload_size <= 0:
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state[VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY] = tr(
+ "VoxCPM Prompt Audio Empty"
+ )
+ return None
+ if upload_size > voice.VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES:
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state[VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY] = tr(
+ "VoxCPM Prompt Audio Upload Too Large"
+ ).format(limit=voice.VOXCPM_REFERENCE_AUDIO_MAX_UPLOAD_BYTES // (1024 * 1024))
+ return None
+
+ previous_digest = _get_voxcpm_prompt_audio_digest()
+ raw_audio = uploaded_file.getvalue()
+ upload_digest = hashlib.sha256(raw_audio).hexdigest()
+ cached = st.session_state.get(VOXCPM_PROMPT_AUDIO_SESSION_KEY)
+ if isinstance(cached, dict) and cached.get("upload_digest") == upload_digest:
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY, None)
+ return _get_voxcpm_prompt_audio()
+
+ try:
+ with st.spinner(tr("Validating VoxCPM Prompt Audio")):
+ wav_audio = voice.prepare_voxcpm_reference_audio(
+ raw_audio,
+ Path(str(getattr(uploaded_file, "name", ""))).suffix,
+ )
+ except ValueError as exc:
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_SESSION_KEY, None)
+ st.session_state[VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY] = str(exc)
+ return None
+
+ st.session_state[VOXCPM_PROMPT_AUDIO_SESSION_KEY] = {
+ "upload_digest": upload_digest,
+ "audio_digest": hashlib.sha256(wav_audio).hexdigest(),
+ "audio_bytes": wav_audio,
+ }
+ if previous_digest != hashlib.sha256(wav_audio).hexdigest():
+ _clear_voxcpm_prompt_transcript()
+ st.session_state.pop(VOXCPM_PROMPT_AUDIO_ERROR_SESSION_KEY, None)
+ return bytes(wav_audio)
+
+
def _get_voice_preview_provider_signature(tts_server: str) -> dict:
"""
返回会影响试听结果的非敏感 Provider 配置。
@@ -5643,6 +5883,13 @@ def _get_voice_preview_provider_signature(tts_server: str) -> dict:
"model_id": config.voxcpm.get("model_id", ""),
"voice_id": config.voxcpm.get("voice_id", "default"),
"credential": _credential_signature(config.voxcpm.get("api_key", "")),
+ "reference_audio": _get_voxcpm_reference_audio_digest(),
+ "prompt_audio": _get_voxcpm_effective_prompt_audio_digest(),
+ "prompt_text": _credential_signature(
+ _get_voxcpm_prompt_text()
+ if st.session_state.get(VOXCPM_HIGH_FIDELITY_SESSION_KEY, False)
+ else ""
+ ),
}
return {}
@@ -5655,6 +5902,9 @@ def _synthesize_voice_preview(
voice_name: str,
voice_rate: float,
voice_volume: float,
+ voxcpm_reference_audio: bytes | None = None,
+ voxcpm_prompt_audio: bytes | None = None,
+ voxcpm_prompt_text: str = "",
) -> dict | None:
"""生成一次试听并转为内存缓存,临时文件不会跨会话长期保留。"""
if selected_tts_server == "chatterbox":
@@ -5673,13 +5923,19 @@ def _synthesize_voice_preview(
with config.try_runtime_config_lock() as lock_acquired:
if not lock_acquired:
return {"busy": True}
- sub_maker = voice.tts(
- text=content,
- voice_name=voice_name,
- voice_rate=voice_rate,
- voice_file=audio_file,
- voice_volume=voice_volume,
- )
+ tts_kwargs = {
+ "text": content,
+ "voice_name": voice_name,
+ "voice_rate": voice_rate,
+ "voice_file": audio_file,
+ "voice_volume": voice_volume,
+ }
+ if voxcpm_reference_audio is not None:
+ tts_kwargs["voxcpm_reference_audio"] = voxcpm_reference_audio
+ if voxcpm_prompt_audio is not None:
+ tts_kwargs["voxcpm_prompt_audio"] = voxcpm_prompt_audio
+ tts_kwargs["voxcpm_prompt_text"] = voxcpm_prompt_text
+ sub_maker = voice.tts(**tts_kwargs)
if not sub_maker or not os.path.exists(audio_file):
logger.error(f"{preview_type} voice preview did not produce an audio file")
return None
@@ -5751,12 +6007,17 @@ def _render_voice_preview(params, friendly_names, selected_tts_server, voice_nam
sample_content = _get_voice_preview_sample(voice_name)
provider_signature = _get_voice_preview_provider_signature(selected_tts_server)
+ preview_validation_error = _get_voxcpm_preview_validation_error(
+ selected_tts_server,
+ voice_name,
+ )
preview_columns = st.columns(2)
short_preview_requested = preview_columns[0].button(
tr("Play Voice"),
key="play_voice_button",
icon=":material/graphic_eq:",
use_container_width=True,
+ disabled=bool(preview_validation_error),
)
full_preview_requested = preview_columns[1].button(
tr("Generate Full Voiceover Preview"),
@@ -5764,7 +6025,7 @@ def _render_voice_preview(params, friendly_names, selected_tts_server, voice_nam
icon=":material/article:",
help=tr("Full Voiceover Preview Cost Hint"),
use_container_width=True,
- disabled=not bool(script_content),
+ disabled=not bool(script_content) or bool(preview_validation_error),
)
preview_type = ""
@@ -5817,6 +6078,21 @@ def _render_voice_preview(params, friendly_names, selected_tts_server, voice_nam
voice_name=voice_name,
voice_rate=params.voice_rate,
voice_volume=params.voice_volume,
+ voxcpm_reference_audio=(
+ _get_voxcpm_reference_audio()
+ if selected_tts_server == "voxcpm"
+ else None
+ ),
+ voxcpm_prompt_audio=(
+ _get_voxcpm_effective_prompt_audio()
+ if selected_tts_server == "voxcpm"
+ else None
+ ),
+ voxcpm_prompt_text=(
+ _get_voxcpm_prompt_text()
+ if selected_tts_server == "voxcpm"
+ else ""
+ ),
)
except Exception as exc:
logger.exception(f"failed to generate {preview_type} voice preview")
@@ -6877,6 +7153,92 @@ def _render_audio_settings(panel, params):
(voxcpm_base_url or DEFAULT_VOXCPM_BASE_URL).strip().rstrip("/"),
)
+ uploaded_reference_audio = st.file_uploader(
+ tr("VoxCPM Reference Audio"),
+ type=list(voice.VOXCPM_REFERENCE_AUDIO_FILE_TYPES),
+ key="voxcpm_reference_audio_uploader",
+ help=tr("VoxCPM Reference Audio Help"),
+ )
+ reference_audio = _sync_voxcpm_reference_audio(
+ uploaded_reference_audio
+ )
+ st.caption(tr("VoxCPM Reference Audio Notice"))
+ reference_audio_error = st.session_state.get(
+ VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY
+ )
+ if reference_audio_error:
+ st.error(
+ tr("VoxCPM Reference Audio Invalid").format(
+ error=reference_audio_error
+ )
+ )
+
+ if reference_audio:
+ high_fidelity_enabled = st.toggle(
+ tr("VoxCPM High Fidelity Delivery"),
+ key=VOXCPM_HIGH_FIDELITY_SESSION_KEY,
+ help=tr("VoxCPM High Fidelity Delivery Help"),
+ )
+ if high_fidelity_enabled:
+ use_separate_prompt_audio = st.toggle(
+ tr("VoxCPM Separate Prompt Audio"),
+ key=VOXCPM_SEPARATE_PROMPT_AUDIO_SESSION_KEY,
+ help=tr("VoxCPM Separate Prompt Audio Help"),
+ )
+ _sync_voxcpm_prompt_example_mode(use_separate_prompt_audio)
+ if use_separate_prompt_audio:
+ prompt_audio_file = st.file_uploader(
+ tr("VoxCPM Prompt Audio"),
+ type=list(voice.VOXCPM_REFERENCE_AUDIO_FILE_TYPES),
+ key="voxcpm_prompt_audio_uploader",
+ help=tr("VoxCPM Prompt Audio Help"),
+ )
+ _sync_voxcpm_prompt_audio(prompt_audio_file)
+ else:
+ _clear_voxcpm_separate_prompt_audio()
+ effective_prompt_audio = _get_voxcpm_effective_prompt_audio()
+ if st.button(
+ tr("Transcribe VoxCPM Prompt Audio"),
+ key="transcribe_voxcpm_prompt_audio_button",
+ icon=":material/transcribe:",
+ help=tr("Transcribe VoxCPM Prompt Audio Help"),
+ use_container_width=True,
+ disabled=not bool(effective_prompt_audio),
+ ):
+ try:
+ with st.spinner(tr("Transcribing VoxCPM Prompt Audio")):
+ recognized_text = subtitle.transcribe_audio_bytes(
+ effective_prompt_audio
+ )
+ except Exception:
+ logger.exception(
+ "failed to transcribe VoxCPM prompt audio"
+ )
+ recognized_text = ""
+ if recognized_text:
+ st.session_state[VOXCPM_PROMPT_TEXT_SESSION_KEY] = (
+ recognized_text
+ )
+ st.toast(tr("VoxCPM Prompt Audio Transcribed"))
+ else:
+ st.error(tr("VoxCPM Prompt Audio Transcription Failed"))
+ st.text_area(
+ tr("VoxCPM Prompt Text"),
+ key=VOXCPM_PROMPT_TEXT_SESSION_KEY,
+ help=tr("VoxCPM Prompt Text Help"),
+ height=100,
+ )
+ st.caption(tr("VoxCPM Prompt Transcript Review"))
+ prompt_error = _get_voxcpm_prompt_validation_error()
+ if prompt_error:
+ st.error(
+ tr("VoxCPM Prompt Invalid").format(
+ error=prompt_error
+ )
+ )
+ else:
+ _clear_voxcpm_prompt_state()
+
# Chatterbox API settings section (self-hosted, OpenAI-compatible)
if tts_mode_enabled and (
selected_tts_server == "chatterbox"
@@ -7446,6 +7808,34 @@ def _render_generation_controls(
st.error(tr("Video Script and Subject Cannot Both Be Empty"))
st.stop()
+ voxcpm_reference_audio = None
+ voxcpm_prompt_audio = None
+ voxcpm_prompt_text = ""
+ if voice.is_voxcpm_voice(params.voice_name or ""):
+ reference_audio_error = st.session_state.get(
+ VOXCPM_REFERENCE_AUDIO_ERROR_SESSION_KEY
+ )
+ if reference_audio_error:
+ _remove_active_generation_task(task_id)
+ st.error(
+ tr("VoxCPM Reference Audio Invalid").format(
+ error=reference_audio_error
+ )
+ )
+ st.stop()
+ voxcpm_reference_audio = _get_voxcpm_reference_audio()
+ prompt_validation_error = _get_voxcpm_prompt_validation_error()
+ if prompt_validation_error:
+ _remove_active_generation_task(task_id)
+ st.error(
+ tr("VoxCPM Prompt Invalid").format(
+ error=prompt_validation_error
+ )
+ )
+ st.stop()
+ voxcpm_prompt_audio = _get_voxcpm_effective_prompt_audio()
+ voxcpm_prompt_text = _get_voxcpm_prompt_text()
+
if params.video_source not in [
"pexels",
"pixabay",
@@ -7759,6 +8149,9 @@ def _render_generation_controls(
capture_logs=not config.ui.get("hide_log", False),
voice_preview=reusable_voice_preview,
loomloom_video_request=loomloom_video_request,
+ voxcpm_reference_audio=voxcpm_reference_audio,
+ voxcpm_prompt_audio=voxcpm_prompt_audio,
+ voxcpm_prompt_text=voxcpm_prompt_text,
)
if loomloom_video_request is not None:
# 一个报价只允许提交一次。后台请求自带稳定幂等 ID;提交成功后
diff --git a/webui/i18n/en.json b/webui/i18n/en.json
index 9a976562..11a2d345 100644
--- a/webui/i18n/en.json
+++ b/webui/i18n/en.json
@@ -288,6 +288,33 @@
"VoxCPM Model ID": "VoxCPM Model ID",
"VoxCPM Model ID Placeholder": "Select a speech_synthesis model ID in ModelBest",
"VoxCPM Base URL": "VoxCPM Base URL",
+ "VoxCPM Reference Audio": "VoxCPM Reference Audio (Optional)",
+ "VoxCPM Reference Audio Help": "Upload a short reference clip to clone its voice for this VoxCPM request. Uploads are limited to 20 MiB and the converted WAV to 5 MiB.",
+ "VoxCPM Reference Audio Notice": "This clip is sent to ModelBest for voice cloning. Only use your own voice or a voice you are authorized to use. It is kept only for this browser session and task.",
+ "VoxCPM Reference Audio Empty": "Reference audio is empty.",
+ "VoxCPM Reference Audio Upload Too Large": "Reference audio upload must not exceed {limit} MiB.",
+ "Validating VoxCPM Reference Audio": "Validating reference audio...",
+ "VoxCPM Reference Audio Invalid": "Reference audio cannot be used: {error}",
+ "VoxCPM High Fidelity Delivery": "Use high-fidelity delivery reference",
+ "VoxCPM High Fidelity Delivery Help": "In addition to cloning the voice, continue the example audio's pacing, emotion, and pronunciation using its exact transcript. The voice reference above is reused by default.",
+ "VoxCPM Separate Prompt Audio": "Use a different pacing and emotion example",
+ "VoxCPM Separate Prompt Audio Help": "By default, the voice reference above is also used as the delivery example. Enable this only to follow another clip's pacing and emotion; that clip may also slightly influence accent and timbre.",
+ "VoxCPM Prompt Audio": "Pacing and Emotion Example",
+ "VoxCPM Prompt Audio Help": "Upload an example whose pacing, emotion, and pronunciation should be continued. It does not replace the voice reference above that identifies the speaker. Maximum upload: 20 MiB; converted WAV: 5 MiB.",
+ "VoxCPM Prompt Text": "Example Audio Transcript",
+ "VoxCPM Prompt Text Help": "Enter the exact words spoken in the current pacing and emotion example, not the new video script.",
+ "Transcribe VoxCPM Prompt Audio": "Auto-transcribe Example Audio",
+ "Transcribe VoxCPM Prompt Audio Help": "Use the configured local Whisper model to transcribe the audio currently used as the pacing and emotion example. The model may be downloaded on first use if it is not cached.",
+ "Transcribing VoxCPM Prompt Audio": "Transcribing the example audio locally...",
+ "VoxCPM Prompt Audio Transcribed": "The example audio transcript was generated. Review it before synthesis.",
+ "VoxCPM Prompt Audio Transcription Failed": "The example audio could not be transcribed. Check the Whisper configuration or enter the transcript manually.",
+ "VoxCPM Prompt Transcript Review": "Review names, numbers, punctuation, and mixed-language terms before synthesis. The transcript is not sent until you request a preview or generation.",
+ "VoxCPM Prompt Audio Empty": "Delivery prompt audio is empty.",
+ "VoxCPM Prompt Audio Upload Too Large": "Delivery prompt audio must not exceed {limit} MiB.",
+ "Validating VoxCPM Prompt Audio": "Validating delivery prompt audio...",
+ "VoxCPM Prompt Text Required": "Enter the exact transcript of the current pacing and emotion example.",
+ "VoxCPM Prompt Pair Required": "The pacing and emotion example and its exact transcript must be provided together.",
+ "VoxCPM Prompt Invalid": "Pacing and emotion continuation cannot be used: {error}",
"Chatterbox Base URL": "Chatterbox Base URL",
"Chatterbox API Key": "Chatterbox API Key (Optional; [View Setup](https://github.com/travisvn/chatterbox-tts-api))",
"Chatterbox Model": "Chatterbox Model",
@@ -309,7 +336,7 @@
"tts_provider_tips.chatterbox": "Self-hosted. Run a compatible [Chatterbox TTS API](https://github.com/travisvn/chatterbox-tts-api), then enter a Base URL ending in `/v1`; the API key is usually optional.",
"tts_provider_tips.kokoro": "Self-hosted and free: run a compatible [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI) server (Apache-2.0 model, CPU is enough), then enter a Base URL ending in `/v1`; the voice list is read from the server.",
"tts_provider_tips.fish_audio": "Requires a Fish Audio API key from the [developer portal](https://fish.audio/app/developers). [Browse public voices](https://fish.audio) or configure custom voices in `config.toml`. See [pricing & terms](https://fish.audio/pricing).",
- "tts_provider_tips.voxcpm": "Requires a [ModelBest API Key](https://platform.modelbest.cn/console/keys) and a model ID with the `speech_synthesis` capability. The API streams WAV audio and supports the default voice in this integration. Numeric speed is not supported; describe the desired pace at the start of the script, for example `(slightly faster)`. Reference-audio voice cloning is not included yet.",
+ "tts_provider_tips.voxcpm": "Requires a [ModelBest API Key](https://platform.modelbest.cn/console/keys) and a model ID with the `speech_synthesis` capability. The API streams WAV audio and supports the default voice in this integration. Numeric speed is not supported; describe the desired pace at the start of the script, for example `(slightly faster)`. Optionally upload a reference clip to clone its voice; high-fidelity delivery can reuse that clip with its exact transcript or use a separate performance example.",
"Hide Log": "Hide Log",
"Hide Basic Settings": "Hide Basic Settings\n\nHidden, the basic settings panel will not be displayed on the page.\n\nIf you need to display it again, please set `hide_config = false` in `config.toml`",
"LLM Settings": "**LLM Settings**",
diff --git a/webui/i18n/zh.json b/webui/i18n/zh.json
index f94c0437..a2d3ef8f 100644
--- a/webui/i18n/zh.json
+++ b/webui/i18n/zh.json
@@ -288,6 +288,33 @@
"VoxCPM Model ID": "VoxCPM 模型 ID",
"VoxCPM Model ID Placeholder": "在 ModelBest 中选择支持 speech_synthesis 的模型 ID",
"VoxCPM Base URL": "VoxCPM Base URL",
+ "VoxCPM Reference Audio": "VoxCPM 参考音频(可选)",
+ "VoxCPM Reference Audio Help": "上传一段短参考音频,用于在本次 VoxCPM 请求中复刻音色。上传文件最大 20 MiB,转换后的 WAV 最大 5 MiB。",
+ "VoxCPM Reference Audio Notice": "该音频会发送给 ModelBest 用于音色复刻。请仅使用自己的声音或已获得授权的声音。音频仅保留在当前浏览器会话和任务中。",
+ "VoxCPM Reference Audio Empty": "参考音频为空。",
+ "VoxCPM Reference Audio Upload Too Large": "参考音频上传文件不能超过 {limit} MiB。",
+ "Validating VoxCPM Reference Audio": "正在校验参考音频...",
+ "VoxCPM Reference Audio Invalid": "参考音频无法使用:{error}",
+ "VoxCPM High Fidelity Delivery": "使用高保真演绎参考",
+ "VoxCPM High Fidelity Delivery Help": "开启后,除了复刻音色,还会根据示范音频及其逐字稿延续节奏、情绪与咬字。默认复用上方的音色参考音频。",
+ "VoxCPM Separate Prompt Audio": "改用另一段节奏与情绪示范",
+ "VoxCPM Separate Prompt Audio Help": "默认使用上方的音色参考音频。仅在希望模仿另一段音频的节奏和情绪时开启;演绎示范也可能轻微影响口音和音色表现。",
+ "VoxCPM Prompt Audio": "节奏与情绪示范音频",
+ "VoxCPM Prompt Audio Help": "上传希望延续其节奏、情绪和咬字的示范。它不会取代上方用于锁定说话人身份的音色参考。上传文件最大 20 MiB,转换后的 WAV 最大 5 MiB。",
+ "VoxCPM Prompt Text": "示范音频逐字稿",
+ "VoxCPM Prompt Text Help": "填写当前用作节奏与情绪示范的音频中实际说出的准确文字,不要填写本次要生成的视频文案。",
+ "Transcribe VoxCPM Prompt Audio": "自动识别示范音频",
+ "Transcribe VoxCPM Prompt Audio Help": "使用当前配置的本地 Whisper 识别正在用作节奏与情绪示范的音频,并生成可编辑草稿;模型未缓存时,首次使用可能需要下载。",
+ "Transcribing VoxCPM Prompt Audio": "正在本地识别示范音频...",
+ "VoxCPM Prompt Audio Transcribed": "示范音频逐字稿已生成,请校对后再合成。",
+ "VoxCPM Prompt Audio Transcription Failed": "无法识别示范音频,请检查 Whisper 配置或手动填写逐字稿。",
+ "VoxCPM Prompt Transcript Review": "合成前请校对人名、数字、标点和中英文混读内容。只有点击试听或生成后,逐字稿才会发送给 ModelBest。",
+ "VoxCPM Prompt Audio Empty": "演绎参考音频为空。",
+ "VoxCPM Prompt Audio Upload Too Large": "演绎参考音频不能超过 {limit} MiB。",
+ "Validating VoxCPM Prompt Audio": "正在校验演绎参考音频...",
+ "VoxCPM Prompt Text Required": "请填写当前示范音频中实际说出的准确逐字稿。",
+ "VoxCPM Prompt Pair Required": "节奏与情绪示范音频和准确逐字稿必须同时提供。",
+ "VoxCPM Prompt Invalid": "节奏与情绪延续无法使用:{error}",
"Chatterbox Base URL": "Chatterbox Base URL",
"Chatterbox API Key": "Chatterbox API 密钥(可选,[查看配置](https://github.com/travisvn/chatterbox-tts-api))",
"Chatterbox Model": "Chatterbox 模型",
@@ -309,7 +336,7 @@
"tts_provider_tips.chatterbox": "自托管服务。运行兼容的 [Chatterbox TTS API](https://github.com/travisvn/chatterbox-tts-api),并填写以 `/v1` 结尾的 Base URL;API Key 通常可选。",
"tts_provider_tips.kokoro": "自托管、免费:运行兼容的 [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)(Apache-2.0 模型,CPU 即可),并填写以 `/v1` 结尾的 Base URL;音色列表自动从服务端读取。",
"tts_provider_tips.fish_audio": "需要 Fish Audio API Key,可在[开发者控制台](https://fish.audio/app/developers)获取;可在 [Fish Audio](https://fish.audio) 浏览更多音色或在 `config.toml` 中配置自定义音色;查看[价格与条款](https://fish.audio/pricing)。",
- "tts_provider_tips.voxcpm": "需要 [ModelBest API Key](https://platform.modelbest.cn/console/keys) 和支持 `speech_synthesis` 能力的模型 ID。该 API 以流式 WAV 返回音频,本次接入使用默认音色。接口暂不支持数值语速,可在文案开头加入 `(语速稍快)` 等自然语言描述;参考音频复刻尚未包含。",
+ "tts_provider_tips.voxcpm": "需要 [ModelBest API Key](https://platform.modelbest.cn/console/keys) 和支持 `speech_synthesis` 能力的模型 ID。该 API 以流式 WAV 返回音频,本次接入使用默认音色。接口暂不支持数值语速,可在文案开头加入 `(语速稍快)` 等自然语言描述;可选上传参考音频复刻音色,高保真演绎可复用该音频及其准确逐字稿,也可另选演绎示范。",
"Hide Log": "隐藏日志",
"Hide Basic Settings": "隐藏基础设置\n\n隐藏后,基础设置面板将不会显示在页面中。\n\n如需要再次显示,请在 `config.toml` 中设置 `hide_config = false`",
"LLM Settings": "**大模型设置**",