Skip to content

AI turns documents or topics into real, native PowerPoint decks—with native shapes, transitions and animations, data-backed charts and tables on demand, audio narration from speaker notes, and support for your own .pptx templates. · by Hugo He

fix(narration): calibrate subtitles to exported video audio

hugohe31249d1347cc3
diff --git a/docs/audio-narration.md b/docs/audio-narration.md
index 3773086a..a48e7b03 100644
--- a/docs/audio-narration.md
+++ b/docs/audio-narration.md
@@ -10,7 +10,7 @@ PPT Master can turn the speaker notes into per-slide narration via [`edge-tts`](
- One audio file per slide under `<project_path>/audio/`, named to match the SVG (`01_cover.mp3`, `02_market_landscape.mp3`, …).
- With edge, one matching subtitle file per slide under `<project_path>/notes/subtitles/` (`01_cover.srt`, `02_market_landscape.srt`, …). Each file uses a page-local timeline with a `00:00:00,000` origin and edge's word-boundary timing.
-- With an SVG-to-SRT timing plan, a rebuilt `animations.json` whose click-free entrance animations wait for the relevant subtitle cue, plus a deck-wide `<project_path>/notes/subtitles/total.srt` aligned to the final PowerPoint timeline.
+- With an SVG-to-SRT timing plan, a rebuilt `animations.json` whose click-free entrance animations wait for the relevant subtitle cue, plus a deck-wide `<project_path>/notes/subtitles/total.srt` aligned to the final PPTX timeline. After PowerPoint exports a video, the same command can calibrate the page starts against its audio track for frame-accurate sidecar subtitles.
- Optional re-export: a new PPTX in `exports/` with each `m4a` / `mp3` / `wav` file embedded into the matching slide and slide auto-advance timings set to the audio length, so kiosk/auto-play and video export work without manual timing.
- The original speaker notes are preserved.
@@ -19,7 +19,7 @@ PPT Master can turn the speaker notes into per-slide narration via [`edge-tts`](
1. **Speaker notes are written as pure spoken narration.** PPT Master's notes spec deliberately produces TTS-friendly prose — no bracketed stage markers, no `Key points:` / `Duration:` meta-lines — so what is read aloud is exactly what's on the page.
2. **AI picks the voice for you.** When you ask for narration, the AI checks the deck's primary language (`zh-CN` / `en-US` / `ja-JP` / `ko-KR` / …), pulls the selected provider's voice catalog, and recommends 3–6 candidates with a one-line tone description for each (e.g. "steady male voice for financial reporting"). It also recommends a speaking rate or provider defaults based on notes density.
3. **One question, one answer.** You are asked once — voice, rate, and "embed audio back into PPTX (yes/no)" — all with a recommended default. Reply "ok" to accept everything, or just call out the part you want to change.
-4. **Generation runs.** With edge, the script writes each page's MP3 and SRT from the same stream to `audio/` and `notes/subtitles/`; cloud providers currently write audio only. For Generate PPTX, the AI maps current SVG content groups to numbered SRT cues, rebuilds click-free animations, and re-exports the deck with audio attached. It then merges the local SRT files using timing values read from that final PPTX. Long-audio import and automatic long-audio splitting are not supported.
+4. **Generation runs.** With edge, the script writes each page's MP3 and SRT from the same stream to `audio/` and `notes/subtitles/`; cloud providers currently write audio only. For Generate PPTX, the AI maps current SVG content groups to numbered SRT cues, rebuilds click-free animations, and re-exports the deck with audio attached. It then merges the local SRT files using timing values read from that final PPTX. If a PowerPoint-exported video is available, an optional final pass measures each page narration in the video audio and corrects only the page offsets. Long-audio import and automatic long-audio splitting are not supported.
Subtitles remain external artifacts: PPT Master does not embed them into the PPTX or export MP4 directly. Use PowerPoint's native video export with the generated `total.srt`.
@@ -112,13 +112,21 @@ python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> \
# 6. Merge page-local SRT using the final PowerPoint timings
python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
--pptx <final_narrated_pptx> --force
+
+# 7. Optional after PowerPoint creates the video: calibrate page starts against
+# the exported audio track and write a same-stem sidecar SRT
+python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
+ --pptx <final_narrated_pptx> --video <powerpoint_exported_video> \
+ -o exports/<powerpoint_exported_video_stem>.srt --force
```
For edge, `--voice` is required. Use `--list-voices --locale <locale>` to see what's available.
The edge command creates `audio/<stem>.mp3` and `notes/subtitles/<stem>.srt` from the same streaming request. Sentence-ending punctuation closes a cue. A cue over 20 visible characters first splits at commas, semicolons, or colons, then at the nearest word boundary only if it is still too long. Use `--subtitle-max-chars` to change the limit. Each SRT uses a page-local timebase with a zero origin and preserves edge's `WordBoundary` timing, including any leading silence before the first cue. The cloud-provider commands currently create audio only.
-`narration_timing.json` is deliberately separate from `animations.json`. It records the ordered SRT-set SHA-256, narration padding, ordered SVG group IDs, and optional 1-based cue numbers. `narration_sync.py animations` rejects a stale fingerprint, validates the group IDs against the current SVGs, and replaces the animation sidecar with only supported PowerPoint fields. `narration_sync.py subtitles` reads the final PPTX's actual presentation order plus millisecond slide-advance and transition values, so `total.srt` follows the same timeline PowerPoint uses when creating a video. A relative `--pptx` path is resolved under `<project_path>`.
+`narration_timing.json` is deliberately separate from `animations.json`. It records the ordered SRT-set SHA-256, narration padding, ordered SVG group IDs, and optional 1-based cue numbers. `narration_sync.py animations` rejects a stale fingerprint, validates the group IDs against the current SVGs, and replaces the animation sidecar with only supported PowerPoint fields. `narration_sync.py subtitles` reads the final PPTX's actual presentation order plus millisecond slide-advance and transition values, so `total.srt` follows the native PPTX timeline. A relative `--pptx` path is resolved under `<project_path>`.
+
+PowerPoint's video encoder can quantize each slide/media segment to its output frame clock. Those small per-page differences may accumulate even when the PPTX timing values are correct. Passing the finished `.mp4` / `.wmv` / `.mov` with `--video` uses normalized audio correlation to locate each original page narration in the exported audio track. It changes only the page-level offsets: edge's cue text and page-local `WordBoundary` timing remain untouched. This is a post-export subtitle calibration step; PPT Master still does not create or rewrite the video.
Use `--no-merge` for the final narrated SVG export. Keeping each SVG line in its own text frame preserves the authored coordinates; paragraph merging lets PowerPoint recalculate multiline text geometry and can introduce visible offsets.
@@ -211,6 +219,7 @@ Once the narrated PPTX is in `exports/`, PowerPoint exports it as a video native
2. **File → Export → Create a Video**.
3. Pick a quality (4K / Full HD / HD / Standard) and "Use Recorded Timings and Narrations" — PPT Master has already set both for you.
4. **Create Video** → save as `.mp4` (or `.wmv` on Windows).
+5. For the closest external-subtitle sync, run the optional `narration_sync.py subtitles --video ...` command above and use its same-stem SRT beside the exported video.
**Keynote (Mac)**: open the deck → **File → Export To → Movie…** — Keynote also honors embedded audio and per-slide timings, output `.m4v` / `.mov`.
diff --git a/docs/zh/audio-narration.md b/docs/zh/audio-narration.md
index 357d35ca..3e70ff88 100644
--- a/docs/zh/audio-narration.md
+++ b/docs/zh/audio-narration.md
@@ -10,7 +10,7 @@ PPT Master 可以把演讲者备注转成逐页音频旁白(默认基于 [`edg
- 每页一个音频文件,存放于 `<project_path>/audio/`,文件名与 SVG 对齐(`01_cover.mp3`、`02_market_landscape.mp3` …)。
- 使用 Edge 时,每页还有一个同名字幕文件,存放于 `<project_path>/notes/subtitles/`(`01_cover.srt`、`02_market_landscape.srt` …)。每个文件使用以 `00:00:00,000` 为原点的页内时间轴,时间来自 Edge 的词边界。
-- 提供 SVG 到 SRT 的计时计划后,还会重建 `animations.json`:无点击入场动画会等待相关字幕 cue;同时生成与最终 PowerPoint 时间轴一致的 `<project_path>/notes/subtitles/total.srt`。
+- 提供 SVG 到 SRT 的计时计划后,还会重建 `animations.json`:无点击入场动画会等待相关字幕 cue;同时生成与最终 PPTX 时间轴一致的 `<project_path>/notes/subtitles/total.srt`。PowerPoint 导出视频后,还可用同一命令根据视频音轨校准每页起点,得到帧级对齐的外挂字幕。
- 可选重新导出:在 `exports/` 生成新版 PPTX,每页对应的 `m4a` / `mp3` / `wav` 音频已嵌入到该页,且页面切换时间按音频长度自动设置——无人值守自动播放和视频导出都不用再手动调时间。
- 演讲者备注原样保留。
@@ -19,7 +19,7 @@ PPT Master 可以把演讲者备注转成逐页音频旁白(默认基于 [`edg
1. **备注本身就是为 TTS 写的口播稿**。PPT Master 的 notes 规范刻意产出适合朗读的散文——没有 `[过渡]` / `[停顿]` 这种舞台标记,也没有 `要点:` / `时长:` 这种 meta 行——念出来的内容就是页面上的内容。
2. **AI 替你选音色**。当你提出生成旁白时,AI 根据 deck 的主语言(`zh-CN` / `en-US` / `ja-JP` / `ko-KR` / …)和所选 provider 拉取或解释可用音色,挑出候选并给每个写一句中文调性说明(如"稳重男声·适合财报")。语速/风格也会基于 notes 信息密度给出推荐值。
3. **一次问完,一次回答**。AI 在一条消息里同时问三件事——生成模式、音色、是否把音频嵌入回 PPTX——每项都标了推荐值。回"好"接受全部默认,或者只说要改的部分(如"音色 2,语速 -5%")。
-4. **执行**。使用 Edge 时,脚本从同一次流中把每页 MP3 和 SRT 分别写到 `audio/` 与 `notes/subtitles/`;云端 provider 目前仍只写音频。对于 Generate PPTX,AI 将当前 SVG 内容组映射到编号后的 SRT cue,重建无点击动画,再导出带音频的 PPTX;最后从该 PPTX 读回实际计时并合并逐页 SRT。不支持长音频导入或自动拆分。
+4. **执行**。使用 Edge 时,脚本从同一次流中把每页 MP3 和 SRT 分别写到 `audio/` 与 `notes/subtitles/`;云端 provider 目前仍只写音频。对于 Generate PPTX,AI 将当前 SVG 内容组映射到编号后的 SRT cue,重建无点击动画,再导出带音频的 PPTX;最后从该 PPTX 读回实际计时并合并逐页 SRT。如果已有 PowerPoint 导出的视频,可再根据视频音轨测出每页旁白的真实起点,只修正分页偏移。不支持长音频导入或自动拆分。
字幕保持为外部 SRT 文件:PPT Master 不把字幕嵌入 PPTX,也不直接导出 MP4。请使用 PowerPoint 原生视频导出,并配套使用生成的 `total.srt`。
@@ -111,13 +111,21 @@ python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> \
# 6. 按最终 PowerPoint 计时合并逐页 SRT
python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
--pptx <final_narrated_pptx> --force
+
+# 7. 可选:PowerPoint 创建视频后,根据导出音轨校准每页起点,
+# 生成与视频同名的外挂 SRT
+python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
+ --pptx <final_narrated_pptx> --video <powerpoint_exported_video> \
+ -o exports/<powerpoint_exported_video_stem>.srt --force
```
edge 模式下 `--voice` 是必填项,可用 `--list-voices --locale <locale>` 查看音色。
Edge 命令会从同一次流式请求中生成 `audio/<stem>.mp3` 与 `notes/subtitles/<stem>.srt`。句末标点必定结束一条字幕;单条超过默认 20 个可见字符时,优先在逗号、分号或冒号处拆分,仍然过长才在最近的词边界拆分。可用 `--subtitle-max-chars` 调整上限。每页 SRT 使用从零计时的页内时间基准,并保留 Edge `WordBoundary` 的实际时间(包括首条字幕前的静音);云端 provider 命令目前只生成音频。
-`narration_timing.json` 与 `animations.json` 刻意分离:前者记录整套有序 SRT 的 SHA-256、旁白 padding、有序 SVG 组 ID 和可选的 1-based cue 编号。`narration_sync.py animations` 会拒绝过期的 SRT 指纹,用当前 SVG 校验组 ID,并用 PowerPoint 支持的字段完整替换动画 sidecar。`narration_sync.py subtitles` 从最终 PPTX 读取真实页面关系顺序、毫秒级页面推进与转场时间,因此 `total.srt` 与 PowerPoint“创建视频”使用同一条时间轴。相对 `--pptx` 路径按 `<project_path>` 解析。
+`narration_timing.json` 与 `animations.json` 刻意分离:前者记录整套有序 SRT 的 SHA-256、旁白 padding、有序 SVG 组 ID 和可选的 1-based cue 编号。`narration_sync.py animations` 会拒绝过期的 SRT 指纹,用当前 SVG 校验组 ID,并用 PowerPoint 支持的字段完整替换动画 sidecar。`narration_sync.py subtitles` 从最终 PPTX 读取真实页面关系顺序、毫秒级页面推进与转场时间,因此 `total.srt` 使用原生 PPTX 时间轴。相对 `--pptx` 路径按 `<project_path>` 解析。
+
+PowerPoint 的视频编码器可能把每个页面 / 媒体段落量化到输出帧时钟;即使 PPTX 计时值正确,这些很小的分页误差仍可能逐页累积。把最终 `.mp4` / `.wmv` / `.mov` 通过 `--video` 传入后,脚本会用归一化音频相关性在视频音轨中定位每页原始旁白。它只改页级偏移,Edge 的字幕文本和页内 `WordBoundary` 时间保持不变。这是视频导出后的字幕校准步骤;PPT Master 仍不创建或改写视频。
最终带旁白的 SVG 导出固定使用 `--no-merge`。让每条 SVG 文本行保持独立文本框,可以保留作者坐标;合并段落会让 PowerPoint 重新计算多行文本几何,可能造成肉眼可见的偏移。
@@ -210,6 +218,7 @@ python3 -m pip install edge-tts
2. **文件 → 导出 → 创建视频**。
3. 选清晰度(4K / 全高清 / 高清 / 标准)以及"使用录制的计时和旁白"——PPT Master 已经替你录好了。
4. **创建视频** → 保存为 `.mp4`(Windows 也支持 `.wmv`)。
+5. 若要让外挂字幕最贴合最终视频,再执行上面的 `narration_sync.py subtitles --video ...`,把生成的同名 SRT 与视频放在一起。
**Keynote(Mac)**:打开 deck → **文件 → 导出到 → 影片…** ——Keynote 同样会读取嵌入的音频和分页计时,输出 `.m4v` / `.mov`。
diff --git a/skills/ppt-master/scripts/narration_sync.py b/skills/ppt-master/scripts/narration_sync.py
index 3e852d6b..73546d6f 100644
--- a/skills/ppt-master/scripts/narration_sync.py
+++ b/skills/ppt-master/scripts/narration_sync.py
@@ -10,15 +10,19 @@ Usage:
python3 scripts/narration_sync.py fingerprint <project_path>
python3 scripts/narration_sync.py animations <project_path> --force
python3 scripts/narration_sync.py subtitles <project_path> --pptx <pptx> --force
+ python3 scripts/narration_sync.py subtitles <project_path> --pptx <pptx> \
+ --video <powerpoint_video> --force
Examples:
python3 scripts/narration_sync.py fingerprint projects/demo
python3 scripts/narration_sync.py animations projects/demo --force
python3 scripts/narration_sync.py subtitles projects/demo --pptx exports/demo.pptx --force
+ python3 scripts/narration_sync.py subtitles projects/demo \
+ --pptx exports/demo.pptx --video exports/demo.mp4 --force
Dependencies:
- ffprobe for animation-window validation; otherwise standard library and
- PPT Master's local SVG/PPTX helpers.
+ ffprobe for animation-window validation. Optional exported-video calibration
+ additionally requires ffmpeg and numpy.
"""
from __future__ import annotations
@@ -30,6 +34,8 @@ import math
import os
import posixpath
import re
+import shutil
+import subprocess
import sys
import tempfile
import zipfile
@@ -85,6 +91,14 @@ _DEFAULT_PROFILE = {
"animation_duration": 0.4,
"stagger": 0.25,
}
+_ALIGNMENT_SAMPLE_RATE = 8000
+_ALIGNMENT_FINE_HZ = 1000
+_ALIGNMENT_COARSE_FACTOR = 10
+_ALIGNMENT_SEARCH_WINDOW_MS = 2000
+_ALIGNMENT_REFINE_WINDOW_MS = 100
+_ALIGNMENT_TEMPLATE_MAX_MS = 12_000
+_ALIGNMENT_MIN_CORRELATION = 0.85
+_ALIGNMENT_END_TOLERANCE_MS = 100
@dataclass(frozen=True)
@@ -104,6 +118,18 @@ class TimingPlanEntry:
cue_number: int | None
+@dataclass(frozen=True)
+class SubtitleMergeResult:
+ """Summary of one page-local SRT merge."""
+
+ slide_count: int
+ cue_count: int
+ powerpoint_timeline_ms: int
+ minimum_video_adjustment_ms: int | None = None
+ maximum_video_adjustment_ms: int | None = None
+ minimum_video_correlation: float | None = None
+
+
def _timestamp_to_ms(value: str) -> int:
hours_text, minutes_text, remainder = value.split(":")
seconds_text, milliseconds_text = remainder.split(",")
@@ -715,15 +741,269 @@ def _read_powerpoint_timings(pptx_path: Path, slide_count: int) -> list[tuple[in
return timings
-def merge_subtitles(
+def _powerpoint_audio_starts(
+ timings: list[tuple[int, int]],
+) -> tuple[list[int], int]:
+ """Return theoretical narration starts and the complete PPTX timeline."""
+ audio_starts: list[int] = []
+ timeline_ms = 0
+ for transition_ms, advance_ms in timings:
+ audio_start_ms = timeline_ms + transition_ms
+ audio_starts.append(audio_start_ms)
+ timeline_ms = audio_start_ms + advance_ms
+ return audio_starts, timeline_ms
+
+
+def _require_numpy() -> Any:
+ try:
+ import numpy as np
+ except ImportError as exc:
+ raise RuntimeError(
+ "Exported-video subtitle calibration requires numpy. "
+ "Install it with: python3 -m pip install numpy"
+ ) from exc
+ return np
+
+
+def _decode_audio_envelopes(path: Path, ffmpeg_path: str) -> tuple[Any, Any]:
+ """Decode the first audio stream and return 1 ms and 10 ms RMS envelopes."""
+ np = _require_numpy()
+ command = [
+ ffmpeg_path,
+ "-v",
+ "error",
+ "-i",
+ str(path),
+ "-map",
+ "0:a:0",
+ "-vn",
+ "-ac",
+ "1",
+ "-ar",
+ str(_ALIGNMENT_SAMPLE_RATE),
+ "-f",
+ "s16le",
+ "-",
+ ]
+ result = subprocess.run(command, capture_output=True, check=False)
+ if result.returncode != 0:
+ details = result.stderr.decode("utf-8", errors="replace").strip()
+ raise RuntimeError(f"Unable to decode audio with ffmpeg: {path}\n{details}")
+
+ samples = np.frombuffer(result.stdout, dtype="<i2")
+ samples_per_frame = _ALIGNMENT_SAMPLE_RATE // _ALIGNMENT_FINE_HZ
+ usable_sample_count = len(samples) // samples_per_frame * samples_per_frame
+ if usable_sample_count < samples_per_frame * 500:
+ raise ValueError(f"Audio is too short for subtitle calibration: {path}")
+
+ frames = (
+ samples[:usable_sample_count]
+ .astype(np.float32)
+ .reshape(-1, samples_per_frame)
+ )
+ fine = np.sqrt(np.mean(frames * frames, axis=1) + 1e-12)
+ fine = np.log1p(120 * fine)
+ smooth_width = max(1, _ALIGNMENT_FINE_HZ // 200)
+ fine = np.convolve(
+ fine,
+ np.ones(smooth_width, dtype=np.float64) / smooth_width,
+ mode="same",
+ ).astype(np.float64)
+
+ coarse_count = len(fine) // _ALIGNMENT_COARSE_FACTOR
+ coarse = fine[
+ :coarse_count * _ALIGNMENT_COARSE_FACTOR
+ ].reshape(coarse_count, _ALIGNMENT_COARSE_FACTOR).mean(axis=1)
+ return fine, coarse
+
+
+def _best_correlation(search: Any, template: Any) -> tuple[int, float]:
+ """Return the best normalized-correlation index for one search window."""
+ np = _require_numpy()
+ if len(search) < len(template):
+ raise ValueError("Video alignment search window is shorter than its template")
+
+ centered_template = template - template.mean()
+ template_norm = float(np.linalg.norm(centered_template))
+ if template_norm <= 1e-9:
+ raise ValueError("Narration audio has no usable variation for video alignment")
+
+ numerators = np.correlate(search, centered_template, mode="valid")
+ prefix = np.concatenate(([0.0], np.cumsum(search, dtype=np.float64)))
+ squared_prefix = np.concatenate(
+ ([0.0], np.cumsum(search * search, dtype=np.float64))
+ )
+ width = len(template)
+ window_sums = prefix[width:] - prefix[:-width]
+ window_squared_sums = squared_prefix[width:] - squared_prefix[:-width]
+ window_variances = np.maximum(
+ window_squared_sums - (window_sums * window_sums / width),
+ 1e-18,
+ )
+ scores = numerators / (np.sqrt(window_variances) * template_norm)
+ best_index = int(np.argmax(scores))
+ return best_index, float(scores[best_index])
+
+
+def _alignment_template_bounds(
+ fine_envelope: Any,
+ cues: list[SubtitleCue],
+) -> tuple[int, int]:
+ duration_ms = len(fine_envelope)
+ start_ms = min(cues[0].start_ms, max(0, duration_ms - 500))
+ cue_end_ms = min(cues[-1].end_ms, duration_ms)
+ end_ms = min(duration_ms, start_ms + _ALIGNMENT_TEMPLATE_MAX_MS)
+ end_ms = min(end_ms, max(start_ms + 1000, cue_end_ms))
+ if end_ms - start_ms < 500:
+ raise ValueError("Narration cue range is too short for video alignment")
+ return start_ms, end_ms
+
+
+def _locate_audio_start(
+ video_fine: Any,
+ video_coarse: Any,
+ audio_fine: Any,
+ audio_coarse: Any,
+ cues: list[SubtitleCue],
+ predicted_start_ms: int,
+) -> tuple[int, float]:
+ """Locate one page narration near its predicted exported-video position."""
+ start_ms, end_ms = _alignment_template_bounds(audio_fine, cues)
+ coarse_start = start_ms // _ALIGNMENT_COARSE_FACTOR
+ coarse_end = max(
+ coarse_start + 50,
+ end_ms // _ALIGNMENT_COARSE_FACTOR,
+ )
+ coarse_template = audio_coarse[coarse_start:coarse_end]
+ predicted_coarse = predicted_start_ms // _ALIGNMENT_COARSE_FACTOR
+ coarse_window = _ALIGNMENT_SEARCH_WINDOW_MS // _ALIGNMENT_COARSE_FACTOR
+ search_start = max(
+ 0,
+ predicted_coarse + coarse_start - coarse_window,
+ )
+ search_end = min(
+ len(video_coarse),
+ predicted_coarse + coarse_start + coarse_window + len(coarse_template),
+ )
+ coarse_index, _coarse_score = _best_correlation(
+ video_coarse[search_start:search_end],
+ coarse_template,
+ )
+ coarse_audio_start_ms = (
+ search_start + coarse_index - coarse_start
+ ) * _ALIGNMENT_COARSE_FACTOR
+
+ fine_template = audio_fine[start_ms:end_ms]
+ fine_search_start = max(
+ 0,
+ coarse_audio_start_ms + start_ms - _ALIGNMENT_REFINE_WINDOW_MS,
+ )
+ fine_search_end = min(
+ len(video_fine),
+ coarse_audio_start_ms
+ + start_ms
+ + _ALIGNMENT_REFINE_WINDOW_MS
+ + len(fine_template),
+ )
+ fine_index, fine_score = _best_correlation(
+ video_fine[fine_search_start:fine_search_end],
+ fine_template,
+ )
+ audio_start_ms = fine_search_start + fine_index - start_ms
+ return audio_start_ms, fine_score
+
+
+def _align_audio_starts_to_video(
+ *,
+ slide_names: list[str],
+ local_cues: dict[str, list[SubtitleCue]],
+ theoretical_starts: list[int],
+ audio_dir: Path,
+ video_path: Path,
+) -> tuple[list[int], list[float], list[Path]]:
+ """Align every page narration to the audio track of an exported video."""
+ if not video_path.is_file():
+ raise FileNotFoundError(f"Exported video does not exist: {video_path}")
+ ffmpeg_path = shutil.which("ffmpeg")
+ if ffmpeg_path is None:
+ raise RuntimeError(
+ "Exported-video subtitle calibration requires ffmpeg. "
+ "Install ffmpeg and make it available on PATH."
+ )
+
+ video_fine, video_coarse = _decode_audio_envelopes(video_path, ffmpeg_path)
+ aligned_starts: list[int] = []
+ correlations: list[float] = []
+ audio_paths: list[Path] = []
+
+ for index, slide_name in enumerate(slide_names):
+ audio_path = _find_audio(audio_dir, slide_name)
+ audio_paths.append(audio_path)
+ audio_fine, audio_coarse = _decode_audio_envelopes(audio_path, ffmpeg_path)
+ if (
+ local_cues[slide_name][-1].end_ms
+ > len(audio_fine) + _ALIGNMENT_END_TOLERANCE_MS
+ ):
+ raise ValueError(
+ f"{slide_name}.srt ends after its narration audio: "
+ f"cue end={_seconds_from_ms(local_cues[slide_name][-1].end_ms):.3f}s, "
+ f"decoded audio={_seconds_from_ms(len(audio_fine)):.3f}s"
+ )
+ if index == 0:
+ predicted_start_ms = theoretical_starts[index]
+ else:
+ predicted_start_ms = (
+ aligned_starts[index - 1]
+ + theoretical_starts[index]
+ - theoretical_starts[index - 1]
+ )
+ aligned_start_ms, correlation = _locate_audio_start(
+ video_fine,
+ video_coarse,
+ audio_fine,
+ audio_coarse,
+ local_cues[slide_name],
+ predicted_start_ms,
+ )
+ if correlation < _ALIGNMENT_MIN_CORRELATION:
+ raise ValueError(
+ f"Exported-video audio match is unreliable for {slide_name}: "
+ f"correlation={correlation:.3f}, "
+ f"required>={_ALIGNMENT_MIN_CORRELATION:.2f}"
+ )
+ if aligned_starts and aligned_start_ms <= aligned_starts[-1]:
+ raise ValueError(
+ f"Exported-video audio order is invalid at slide {slide_name}"
+ )
+ final_cue_end_ms = (
+ aligned_start_ms + local_cues[slide_name][-1].end_ms
+ )
+ if (
+ final_cue_end_ms
+ > len(video_fine) + _ALIGNMENT_END_TOLERANCE_MS
+ ):
+ raise ValueError(
+ f"Exported video ends before the final cue on slide {slide_name}: "
+ f"cue end={_seconds_from_ms(final_cue_end_ms):.3f}s, "
+ f"decoded video audio={_seconds_from_ms(len(video_fine)):.3f}s"
+ )
+ aligned_starts.append(aligned_start_ms)
+ correlations.append(correlation)
+
+ return aligned_starts, correlations, audio_paths
+
+
+def _merge_subtitles_result(
project_path: Path,
*,
pptx_path: Path,
subtitle_dir: Path,
output_path: Path,
force: bool,
-) -> tuple[int, int, int]:
- """Merge local SRT files using timing values read from the final PPTX."""
+ audio_dir: Path | None = None,
+ video_path: Path | None = None,
+) -> SubtitleMergeResult:
+ """Merge local SRT files on the PPTX or exported-video timeline."""
targets_by_slide, _anonymous_groups = scan_project_targets(project_path)
slide_names = list(targets_by_slide)
if not slide_names:
@@ -735,35 +1015,126 @@ def merge_subtitles(
]
_reject_output_alias(
output_path,
- [pptx_path, *local_subtitle_paths],
+ [pptx_path, *local_subtitle_paths, *([video_path] if video_path else [])],
label="Merged subtitle",
)
_require_replaceable(output_path, force)
timings = _read_powerpoint_timings(pptx_path, len(slide_names))
+ theoretical_starts, timeline_ms = _powerpoint_audio_starts(timings)
+ local_cues = {
+ slide_name: _parse_srt(subtitle_dir / f"{slide_name}.srt")
+ for slide_name in slide_names
+ }
+ video_adjustments: list[int] = []
+ correlations: list[float] = []
+
+ if video_path is None:
+ audio_starts = theoretical_starts
+ else:
+ resolved_audio_dir = audio_dir or project_path / "audio"
+ audio_starts, correlations, audio_paths = _align_audio_starts_to_video(
+ slide_names=slide_names,
+ local_cues=local_cues,
+ theoretical_starts=theoretical_starts,
+ audio_dir=resolved_audio_dir,
+ video_path=video_path,
+ )
+ _reject_output_alias(
+ output_path,
+ audio_paths,
+ label="Merged subtitle",
+ )
+ video_adjustments = [
+ actual - theoretical
+ for actual, theoretical in zip(audio_starts, theoretical_starts)
+ ]
+
merged_cues: list[SubtitleCue] = []
- timeline_ms = 0
- for slide_name, (transition_ms, advance_ms) in zip(slide_names, timings):
- local_cues = _parse_srt(subtitle_dir / f"{slide_name}.srt")
- if local_cues[-1].end_ms > advance_ms:
+ for slide_name, audio_start_ms, (_transition_ms, advance_ms) in zip(
+ slide_names,
+ audio_starts,
+ timings,
+ ):
+ slide_cues = local_cues[slide_name]
+ if slide_cues[-1].end_ms > advance_ms:
raise ValueError(
f"{slide_name}.srt ends at "
- f"{_seconds_from_ms(local_cues[-1].end_ms):.3f}s, after the "
+ f"{_seconds_from_ms(slide_cues[-1].end_ms):.3f}s, after the "
f"PowerPoint slide advance at {_seconds_from_ms(advance_ms):.3f}s"
)
- audio_start_ms = timeline_ms + transition_ms
- merged_cues.extend(
- SubtitleCue(
+ for cue in slide_cues:
+ merged_cue = SubtitleCue(
cue.start_ms + audio_start_ms,
cue.end_ms + audio_start_ms,
cue.text,
)
- for cue in local_cues
- )
- timeline_ms = audio_start_ms + advance_ms
+ if merged_cues and merged_cue.start_ms < merged_cues[-1].end_ms:
+ raise ValueError(
+ f"Video-calibrated subtitle overlap before slide {slide_name}"
+ )
+ merged_cues.append(merged_cue)
_atomic_write_text(output_path, _format_srt(merged_cues))
- return len(slide_names), len(merged_cues), timeline_ms
+ return SubtitleMergeResult(
+ slide_count=len(slide_names),
+ cue_count=len(merged_cues),
+ powerpoint_timeline_ms=timeline_ms,
+ minimum_video_adjustment_ms=(
+ min(video_adjustments) if video_adjustments else None
+ ),
+ maximum_video_adjustment_ms=(
+ max(video_adjustments) if video_adjustments else None
+ ),
+ minimum_video_correlation=(
+ min(correlations) if correlations else None
+ ),
+ )
+
+
+def merge_subtitles(
+ project_path: Path,
+ *,
+ pptx_path: Path,
+ subtitle_dir: Path,
+ output_path: Path,
+ force: bool,
+) -> tuple[int, int, int]:
+ """Merge local SRT files using timing values read from the final PPTX."""
+ result = _merge_subtitles_result(
+ project_path,
+ pptx_path=pptx_path,
+ subtitle_dir=subtitle_dir,
+ output_path=output_path,
+ force=force,
+ )
+ return (
+ result.slide_count,
+ result.cue_count,
+ result.powerpoint_timeline_ms,
+ )
+
+
+def merge_subtitles_to_video(
+ project_path: Path,
+ *,
+ pptx_path: Path,
+ subtitle_dir: Path,
+ audio_dir: Path,
+ video_path: Path,
+ output_path: Path,
+ force: bool,
+) -> SubtitleMergeResult:
+ """Merge local SRT files after calibrating page starts to an exported video."""
+ return _merge_subtitles_result(
+ project_path,
+ pptx_path=pptx_path,
+ subtitle_dir=subtitle_dir,
+ output_path=output_path,
+ force=force,
+ audio_dir=audio_dir,
+ video_path=video_path,
+ )
def build_parser() -> argparse.ArgumentParser:
@@ -843,6 +1214,22 @@ def build_parser() -> argparse.ArgumentParser:
default=None,
help="Page-local SRT directory; default: <project>/notes/subtitles",
)
+ subtitles.add_argument(
+ "--video",
+ default=None,
+ help=(
+ "PowerPoint-exported video whose audio track calibrates page starts; "
+ "relative paths are resolved under the project"
+ ),
+ )
+ subtitles.add_argument(
+ "--audio-dir",
+ default=None,
+ help=(
+ "Page-local narration audio used with --video; "
+ "default: <project>/audio"
+ ),
+ )
subtitles.add_argument(
"-o",
"--output",
@@ -925,23 +1312,44 @@ def main(argv: list[str] | None = None) -> int:
args.subtitle_dir,
Path("notes/subtitles"),
)
+ audio_dir = _project_path(
+ project_path,
+ args.audio_dir,
+ Path("audio"),
+ )
+ video_path = (
+ _project_input_path(project_path, args.video).resolve()
+ if args.video
+ else None
+ )
output_path = _project_path(
project_path,
args.output,
Path("notes/subtitles/total.srt"),
)
- slide_count, cue_count, timeline_ms = merge_subtitles(
+ result = _merge_subtitles_result(
project_path,
pptx_path=pptx_path,
subtitle_dir=subtitle_dir,
+ audio_dir=audio_dir,
+ video_path=video_path,
output_path=output_path,
force=args.force,
)
print(f"Merged subtitle written: {output_path}")
print(
- f"Slides: {slide_count}; cues: {cue_count}; "
- f"PowerPoint timeline: {_seconds_from_ms(timeline_ms):.3f}s"
+ f"Slides: {result.slide_count}; cues: {result.cue_count}; "
+ "PowerPoint timeline: "
+ f"{_seconds_from_ms(result.powerpoint_timeline_ms):.3f}s"
)
+ if result.minimum_video_correlation is not None:
+ print(
+ "Exported-video calibration: page adjustment "
+ f"{_seconds_from_ms(result.minimum_video_adjustment_ms or 0):+.3f}s "
+ "to "
+ f"{_seconds_from_ms(result.maximum_video_adjustment_ms or 0):+.3f}s; "
+ f"minimum correlation: {result.minimum_video_correlation:.3f}"
+ )
return 0
except (
ET.ParseError,
diff --git a/skills/ppt-master/workflows/stages/generate-audio.md b/skills/ppt-master/workflows/stages/generate-audio.md
index 59357388..939244a1 100644
--- a/skills/ppt-master/workflows/stages/generate-audio.md
+++ b/skills/ppt-master/workflows/stages/generate-audio.md
@@ -15,6 +15,7 @@ This stage is **context-independent**: it reads `notes/*.md` and queries the sel
- The stage is page-level only: with edge, one notes file becomes `audio/<stem>.mp3` plus `notes/subtitles/<stem>.srt`; with a cloud provider, it becomes one audio file. Do not use a single long audio track or attempt automatic long-audio splitting.
- PPT narration assets must be PowerPoint-reliable audio: `m4a` (AAC), `mp3`, or `wav`. The built-in TTS path defaults to `mp3`; provider formats such as `pcm`, `opus`, or `flac` must be transcoded before embedding.
- PowerPoint recorded narration export requires `ffprobe` so slide timings can be written from actual audio duration.
+- Optional post-export video calibration requires `ffmpeg` plus `numpy`; it runs only when the caller supplies the finished PowerPoint video.
- High-quality cloud mode: provider API key is set before use:
- ElevenLabs: `ELEVENLABS_API_KEY`
- MiniMax: `MINIMAX_API_KEY`
@@ -153,6 +154,12 @@ python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> \
# 2C. Merge page-local SRT against timing values read from the final PPTX
python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
--pptx <final_narrated_pptx> --force
+
+# 2D. Optional after PowerPoint exports the video: calibrate page starts against
+# the actual video audio and write a same-stem sidecar SRT
+python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
+ --pptx <final_narrated_pptx> --video <powerpoint_exported_video> \
+ -o exports/<powerpoint_exported_video_stem>.srt --force
```
If `notes_to_audio.py` errors with a missing dependency or missing provider API key, fix the prerequisite and re-run — do NOT swallow the error.
@@ -183,7 +190,9 @@ python3 skills/ppt-master/scripts/narration_sync.py fingerprint <project_path>
}
```
-After the narrated PPTX is exported, `narration_sync.py subtitles` reads the actual presentation relationship order plus integer `advTm` and transition duration from every slide, then writes `<project_path>/notes/subtitles/total.srt`. Relative `--pptx` paths are project-relative. This keeps the deck-wide SRT aligned with PowerPoint's native video timeline, including page transitions and narration padding.
+After the narrated PPTX is exported, `narration_sync.py subtitles` reads the actual presentation relationship order plus integer `advTm` and transition duration from every slide, then writes `<project_path>/notes/subtitles/total.srt`. Relative `--pptx` paths are project-relative. This produces the deck-wide SRT on the native PPTX timeline, including page transitions and narration padding.
+
+PowerPoint video export may quantize each slide/media segment to the video frame clock, creating small page-boundary differences that accumulate across a long deck. When the finished video exists, pass it with `--video`. The command correlates each original page narration with the exported audio track and replaces only the theoretical page offsets; it preserves every page-local cue and rejects weak matches. This optional post-export calibration does not create or rewrite MP4.
This stage keeps subtitles as external SRT files. It does not embed subtitles into PPTX or export MP4 directly.
@@ -191,7 +200,7 @@ This stage keeps subtitles as external SRT files. It does not embed subtitles in
| Caller | After audio generation |
|---|---|
-| Generate PPTX | With Edge SRT, rebuild animations, export with `--recorded-narration audio`, then merge `total.srt` from the final PPTX as shown above |
+| Generate PPTX | With Edge SRT, rebuild animations, export with `--recorded-narration audio`, then merge `total.srt` from the final PPTX. If PowerPoint later exports a video, optionally rerun the merge with `--video` for exact sidecar alignment. |
| Enhance Native PPTX | Return to [`native-enhance-pptx`](../native-enhance-pptx.md) Step 9; its `apply` command owns audio relationships, timings, transitions, and the enhanced export |
For Generate PPTX, `--recorded-narration audio` prepares PowerPoint's recorded timings and narrations: every slide must have a matching supported audio file, every duration must be readable by `ffprobe`, and object animations must not use `--animation-trigger on-click`. Use `after-previous` or `with-previous` for narrated/video export. Narration changes the slide-advance layer only: the resolved page-transition effect remains unchanged, `-t none` remains visually transition-free, and narration advance disables click while using audio duration plus padding. The re-export is saved as `exports/<project_name>_<timestamp>_narrated.pptx`, telling it apart from silent exports.
@@ -207,6 +216,7 @@ Output one summary block listing:
- Number of audio files generated and their location (`<project_path>/audio/*`).
- For edge, number of matching page-local SRT files and their location (`<project_path>/notes/subtitles/*`).
- For Generate PPTX with Edge SRT, rebuilt animation group count and merged SRT path (`<project_path>/notes/subtitles/total.srt`).
+- When `--video` was used, the calibrated sidecar SRT path, page-offset adjustment range, and minimum audio-correlation score.
- The provider, voice, and rate/settings actually used.
- The caller-owned integration result: narrated SVG export path, enhanced native PPTX path, or “audio only”.
- For Generate PPTX when embedding was skipped, one-line hint: `python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> --recorded-narration audio`.