返回 ppt-master
generate-audio.md
根目录 / skills / ppt-master / workflows / stages / generate-audio.md
1 ---
2 description: Shared post-processing stage for narration audio, PPTX embedding, and optional native video export.
3 ---
4
5 # Generate Audio Stage
6
7 > Shared narration stage. Run from the Generate PPTX route after notes/export readiness, or from the Enhance Native PPTX narration module after its notes step. By default, `edge-tts` produces one audio/SRT pair per slide from the same streaming request. Cloud TTS providers (`elevenlabs` / `minimax` / `qwen` / `cosyvoice`) currently produce audio only. The caller owns final PPTX integration.
8
9 This stage is **context-independent**: it reads `notes/*.md` and queries the selected TTS voice catalog, so either owning route may invoke it in a fresh session. It does not choose the top-level route and does not patch slide design.
10
11 **Trigger**: In Generate PPTX, run when the effective `Narration Audio` outcome
12 in `design_spec.md` §I is `enabled`; a later explicit request first updates that
13 outcome and its provenance. Quick Generate instead runs when the request or
14 current agent's active-context decision selects narration. In Enhance Native
15 PPTX, run when its confirmed enhancement plan has `audio.enabled: true`.
16
17 **Hard dependency — speaker notes**: Audio requires complete per-slide speaker
18 notes. Generate PPTX additionally requires its effective `Speaker Notes`
19 outcome to be enabled; Enhance Native PPTX follows its confirmed notes/audio
20 plan, where enabling audio also enables notes. Quick records the same dependency
21 in active context. Do not enter audio generation while the owning route's notes
22 are missing or incomplete; generate and validate those notes first, then resume
23 this stage.
24
25 ## When to Run
26
27 - Per-page narration files exist at `notes/*.md`. In Generate PPTX, split `notes/total.md` during Step 7.1. In Enhance Native PPTX, the notes module writes numeric files such as `001.md`.
28 - Default mode: `edge-tts` is installed (`python3 -m pip install edge-tts`).
29 - The stage is page-level only: with edge, one notes file becomes `audio/<stem>.mp3` plus `notes/subtitles/<stem>.srt`; with a cloud provider, it becomes one audio file. Do not use a single long audio track or attempt automatic long-audio splitting.
30 - PPT narration assets must be PowerPoint-reliable audio: `m4a` (AAC), `mp3`, or `wav`. The built-in TTS path defaults to `mp3`; provider formats such as `pcm`, `opus`, or `flac` must be transcoded before embedding.
31 - PowerPoint recorded narration export requires `ffprobe` so slide timings can be written from actual audio duration.
32 - Optional automatic video export requires Windows PowerPoint 2016+ and runs
33 through `powerpoint_video.py`; the command waits for PowerPoint's native
34 encoder to finish before returning.
35 - macOS PowerPoint may export MP4/MOV manually, but it has no equivalent
36 `CreateVideo` automation contract and its movie export does not preserve
37 animation effects. Do not replace the missing API with UI scripting.
38 - Optional post-export video calibration requires `ffmpeg` plus `numpy`; it runs only after a finished PowerPoint video is supplied or created.
39 - High-quality cloud mode: provider API key is set before use:
40 - ElevenLabs: `ELEVENLABS_API_KEY`
41 - MiniMax: `MINIMAX_API_KEY`
42 - Qwen: `QWEN_API_KEY` or `DASHSCOPE_API_KEY`
43 - CosyVoice: `COSYVOICE_API_KEY` or `DASHSCOPE_API_KEY`
44 - Keys may live in the current process environment or the first `.env` found in this order: current working directory, skill directory (e.g. `~/.agents/skills/ppt-master/.env`), clone repo root, `~/.ppt-master/.env`
45 - The deck is in a single dominant language (mixed-language decks: pick the dominant one — the AI uses judgment, not a heuristic).
46
47 If per-slide notes are missing, recover through the owning route. Generate
48 PPTX returns to its enabled notes branch and then runs
49 `total_md_split.py <project_path>`; Enhance Native PPTX returns to
50 `native-enhance-pptx` Step 6 and writes numeric notes directly. Never run the
51 Generate splitter against a Native Enhance project.
52
53 ---
54
55 ## Step 1: Determine the deck's language
56
57 The AI already knows the deck's language from writing the notes. No detection script needed.
58
59 - Identify the primary language from the notes content: `zh` / `en` / `ja` / `ko` / etc.
60 - For mixed-language decks (e.g. Chinese with English technical terms), pick the language the audience will hear most of.
61 - For Chinese specifically: pick the locale based on context — `zh-CN` (mainland mandarin, default), `zh-TW` (Taiwanese mandarin), or `zh-HK` (Cantonese). Default Generate may ask when context is unclear; Quick chooses the best supported default and continues.
62
63 ---
64
65 ## Step 2: Choose audio backend and pull the voice catalog
66
67 Default to **edge** unless the user explicitly asks for a cloud provider / higher-quality cloud narration / a cloned voice.
68
69 **edge backend**:
70
71 ```bash
72 python3 skills/ppt-master/scripts/notes_to_audio.py --list-voices --locale <locale>
73 ```
74
75 **ElevenLabs backend**:
76
77 ```bash
78 python3 skills/ppt-master/scripts/notes_to_audio.py --provider elevenlabs --list-voices
79 ```
80
81 **Cloud providers using explicit voice IDs/names**:
82
83 ```bash
84 python3 skills/ppt-master/scripts/notes_to_audio.py --provider minimax --list-voices
85 python3 skills/ppt-master/scripts/notes_to_audio.py --provider qwen --list-voices
86 python3 skills/ppt-master/scripts/notes_to_audio.py --provider cosyvoice --list-voices
87 ```
88
89 The output is a flat list of all available voices for the selected provider. From this list, the AI picks **3–6 candidates** to recommend, applying these rules:
90
91 - **Cover both genders** when both exist for the locale.
92 - **For edge**: prefer `COMMON_VOICES`-listed voices (curated set inside `notes_to_audio.py`) when the locale has them — they are battle-tested.
93 - **For ElevenLabs**: prefer voices already present in the user's account; if the user provides a specific `voice_id`, do not override it.
94 - **For MiniMax / Qwen / CosyVoice**: if the user provides a cloned `voice_id`, use it directly. Do not attempt voice cloning inside this narration stage.
95 - **Match the deck's tone** — pick the strongest recommendation based on style:
96 - Consultant / data-driven / 财报 → 稳重男声(如 `zh-CN-YunjianNeural`)or 清晰女声(如 `zh-CN-XiaoxiaoNeural`)
97 - General / 教学 / 产品介绍 → 明亮女声 / 年轻男声(如 `zh-CN-XiaoyiNeural` / `zh-CN-YunxiNeural`)
98 - 发布会 / 播报 → 播报感男声(如 `zh-CN-YunyangNeural`)
99 - English consultant deck → `en-US-GuyNeural` (steady) or `en-US-JennyNeural` (clear)
100 - Japanese / Korean → pick from `ja-JP-*` / `ko-KR-*` neural voices, mark gender + tone
101
102 For each candidate, write a **one-line Chinese description** covering: 性别 · 调性 · 适用场景。For cloud providers, include the voice name/ID exactly as it must be passed to `--voice-id`.
103
104 ---
105
106 ## Step 3: One-shot user interaction (mandatory)
107
108 Send a single message to the user that resolves all five configuration decisions at once and provides a recommended value for each. Before offering automatic video export, run `python3 skills/ppt-master/scripts/powerpoint_video.py --check`; do not present an unavailable local capability as executable. Do NOT split into multiple rounds.
109
110 **Cloned-voice fast path**: if the user mentioned a cloned voice / 克隆音色 / 复刻音色 / "my own voice" along with a `voice_id`, skip the voice-recommendation list — set the provider to whichever the user named (`elevenlabs` / `minimax` / `qwen` / `cosyvoice`), pin the `voice_id` they gave you, and only confirm rate + embed + video.
111
112 **Message template** (Chinese; translate to user's chat language if different). “Embed” means caller-specific integration: SVG re-export for Generate PPTX, or native OOXML application for Enhance Native PPTX.
113
114 > 检测到 notes 主语言为 **<语言>**(locale: `<locale>`)。基于 deck 调性(<风格>),我推荐以下配置:
115 >
116 > **生成模式**:⭐ 推荐 `<edge|elevenlabs|minimax|qwen|cosyvoice>`(理由:<一句话,如"无需配置,稳定生成"或"用户要求高质量云端音色">)。
117 >
118 > **音色**:
119 > - **[1] <ShortName>** — <性别·调性·适用场景> ⭐ **推荐**
120 > - [2] <ShortName> — <性别·调性·适用场景>
121 > - [3] <ShortName> — <性别·调性·适用场景>
122 > - [4] <ShortName> — <性别·调性·适用场景>
123 > - [5] <ShortName> — <性别·调性·适用场景>
124 > - 也可直接输入清单中的其他 ShortName。
125 >
126 > **语速/风格参数**:⭐ 推荐 `<rate or provider defaults>`(理由:<一句话,如"页均 2–3 句,正常语速听感最稳"或"ElevenLabs 默认 voice settings 保留音色原始表现最稳">)。
127 >
128 > **生成完是否重新导出嵌入音频的 PPTX**:⭐ 推荐 **是**(一次到位,自动按音频时长设页面停留)。
129 >
130 > **带音频 PPTX 完成后是否继续导出视频**:⭐ 推荐 **是**(仅在本机 Windows PowerPoint 2016+ 可用时;将等待原生视频导出完成)。
131 >
132 > 直接回"好"用全部推荐值,或告诉我想改的部分(如"音色 2,语速 -5%"或"用 MiniMax 的 voice_id xxx")。
133
134 **Recommended-value rules**:
135 - 生成模式:默认 `edge`;当用户明确追求高质量云端音色或提供 cloud voice ID 时,按用户指定选 `elevenlabs` / `minimax` / `qwen` / `cosyvoice`。
136 - 音色:从 Step 2 候选里挑最贴合 deck 调性的那一个。
137 - 语速:edge 默认 `+0%`;notes 字数密集(页均 >4 句长句)建议 `-5%`;notes 简短紧凑建议 `+5%`;超出此范围需说明理由。Cloud providers 默认用 provider defaults,除非用户明确要调速或改风格。
138 - 嵌入:默认推荐"是";除非用户已有定制 PPTX 不希望覆盖。
139 - 视频:`powerpoint_video.py --check` 成功时默认推荐"是";不可用时说明只能交付带音频 PPTX,不自动改用第三方渲染器。
140
141 ---
142
143 ## Step 4: Execute (no further interaction)
144
145 **Blocking notes preflight**: `notes_to_audio.py` resolves the complete notes
146 roster from `svg_output/*.svg` on Generate projects or
147 `analysis/slide_index.json` on Native Enhance projects. Before any TTS request,
148 every expected note must exist, be readable, and contain spoken text. Exit code
149 `2` returns the caller to its notes-generation step; never continue with partial
150 audio generation.
151
152 Run sequentially — do NOT bundle:
153
154 ```bash
155 # 1A. Generate audio with edge (default)
156 python3 skills/ppt-master/scripts/notes_to_audio.py <project_path> \
157 --voice <chosen-ShortName> --rate <chosen-rate>
158
159 # 1B. Or generate audio with ElevenLabs
160 python3 skills/ppt-master/scripts/notes_to_audio.py <project_path> \
161 --provider elevenlabs --voice-id <chosen-voice-id> \
162 --elevenlabs-model eleven_multilingual_v2
163
164 # 1C. Or generate audio with MiniMax
165 # Defaults to the China endpoint; set MINIMAX_TTS_BASE_URL=https://api.minimax.io/v1/t2a_v2 for overseas access.
166 python3 skills/ppt-master/scripts/notes_to_audio.py <project_path> \
167 --provider minimax --voice-id <chosen-voice-id> \
168 --minimax-model speech-2.8-hd
169
170 # 1D. Or generate audio with Qwen TTS
171 python3 skills/ppt-master/scripts/notes_to_audio.py <project_path> \
172 --provider qwen --voice-id <chosen-voice> \
173 --qwen-model qwen3-tts-flash --qwen-language-type Chinese
174
175 # 1E. Or generate audio with CosyVoice
176 python3 skills/ppt-master/scripts/notes_to_audio.py <project_path> \
177 --provider cosyvoice --voice-id <chosen-voice> \
178 --cosyvoice-model cosyvoice-v3-flash
179
180 # 2A. When animations.json is active, author or refresh narration_timing.json
181 # by matching SVG group semantics to SRT topics, then derive the narrated
182 # sidecar. Reuse current SVG semantics when complete; otherwise read only
183 # the missing or stale svg_output pages.
184 python3 skills/ppt-master/scripts/narration_sync.py animations <project_path> \
185 --narration-padding 0.5 --force
186
187 # 2B. Re-export with audio embedded
188 # Use the base export's [REPORT] path to preserve source-bound deck motion.
189 python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> \
190 --recorded-narration audio --narration-padding 0.5 \
191 --inherit-motion-from "<base_postflight_report>"
192
193 # Optional: use the canonical presentation animation instead
194 python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> \
195 --recorded-narration audio --narration-padding 0.5 \
196 --animation-config animations.json \
197 --inherit-motion-from "<base_postflight_report>"
198
199 # Optional: export narration with no object or page-transition animation
200 python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> \
201 --recorded-narration audio --narration-padding 0.5 \
202 --no-animations
203
204 # 2C. Merge page-local SRT against timing values read from the final PPTX
205 python3 skills/ppt-master/scripts/narration_sync.py subtitles <project_path> \
206 --pptx <final_narrated_pptx> --force
207
208 # 2D. Optional: export through installed Windows PowerPoint and wait for completion
209 python3 skills/ppt-master/scripts/powerpoint_video.py \
210 <final_narrated_pptx> -o <final_video.mp4>
211
212 # 2E. Align the frozen narration text against the finished video's audio track
213 python3 skills/ppt-master/scripts/video_subtitles.py <project_path> \
214 --video <final_video.mp4> --language <language> --force
215 ```
216
217 **Default — bounded Edge concurrency (may override)**: Generate up to three
218 slide-level audio/SRT pairs concurrently. Use `--concurrency <N>` to tune the
219 Edge path or `--concurrency 1` for serial troubleshooting. Cloud providers
220 remain serial.
221
222 If `notes_to_audio.py` errors with a missing dependency or missing provider API key, fix the prerequisite and re-run — do NOT swallow the error.
223
224 The edge command writes each MP3 and its internal page SRT from the same `edge-tts` stream. SRT cues use the service's `WordBoundary` timing: sentence-ending punctuation always closes a cue; text over the default 20-visible-character limit first splits at commas, semicolons, or colons, then at the nearest word boundary. Override the limit with `--subtitle-max-chars`. Adjacent timing overlap up to 100 ms is tolerated by moving the later cue start to the previous cue end; larger overlap fails instead of silently distorting timing. Each SRT uses a page-local timeline whose origin is `00:00:00,000`, including any leading silence before the first cue. Cloud-provider commands currently write audio only.
225
226 **Mandatory when `animations.json` is consumed — semantic animation context**: Before writing or refreshing `<project_path>/narration_timing.json`, determine whether the active context already contains the current top-level SVG group IDs and visible group-content semantics for every affected page. Reuse that context without rereading SVG when it is complete and still matches the current `svg_output/`. If any page is missing, stale, or represented only by group IDs/order without content meaning, read only that page's SVG as a read-only source and extract the missing group semantics. Always combine those semantics with the page SRT topics/timestamps and `animations.json`; group order alone is not a semantic narration mapping.
227
228 > Active `animations.json` requires `narration_timing.json`; explicit `--no-animations` bypasses both. Without a sidecar, `narration_sync.py animations` maps groups **positionally** (group N → cue N) and warns when later objects may reveal during an earlier topic. Treat that warning as required repair: author the semantic plan and re-derive.
229
230 **Narration animation ownership**: When `animations.json` is consumed, it remains read-only. The audio stage deep-copies it to `narration_animations.json`, preserves transitions, effects, durations, order, and explicit `effect: none`, then changes only the derived trigger/delay values needed for click-free narration playback. The authored `narration_timing.json` maps each animated content group—not each effect row—to the SRT cue that speaks about that content. For `effects[]`, the cue anchors the group's first active row; later rows keep global order and their relative delay. The command may still read an affected SVG page to resolve structural group order when a sparse sidecar cannot identify every effective group; this structural fallback does not replace the semantic-context step and never edits SVG, notes, or `animations.json`. Unmatched groups keep their canonical relative delay.
231
232 **Title timing handoff when canonical animation exists**: preserve the title reveal decision already made by the custom-animation pass. Assign a title group to an SRT cue only when the user's request or the active motion plan explicitly chose `narration-cued`; otherwise leave its `cue` omitted in `narration_timing.json` so it keeps the canonical relative delay from `animations.json`. Do not infer `narration-cued` merely because speaker notes mention the title.
233
234 **Narrated export animation selection**:
235
236 | Sidecar state | Behavior |
237 |---|---|
238 | `narration_animations.json` exists | Use it by default |
239 | Only canonical `animations.json` exists | Block until narration synchronization creates the derived sidecar |
240 | Both are absent | Create no sidecar; inherit the base report's deck motion |
241
242 Generate passes the base report through `--inherit-motion-from`: inherited
243 `-a none` preserves explicit objects-off, while final Stage-2 `false` does not.
244 Only explicit all-motion-off uses `--no-animations`. Invalid reports block;
245 audio duration plus padding owns final advance.
246
247 When canonical custom animation is synchronized,
248 `<project_path>/narration_timing.json` is the explicit semantic mapping for
249 narrated object animation. It is fingerprinted to the ordered SRT set; `cue`
250 is the 1-based subtitle cue, and omitted `cue` keeps that group's canonical
251 relative delay. Reuse a complete current mapping when its fingerprint and SVG
252 group semantics remain valid; rebuild only affected pages when either input
253 changed.
254
255 Get the exact fingerprint value with:
256
257 ```bash
258 python3 skills/ppt-master/scripts/narration_sync.py fingerprint <project_path>
259 ```
260
261 ```json
262 {
263 "version": 1,
264 "srt_sha256": "<sha256 of the ordered page-local SRT set>",
265 "narration_padding": 0.5,
266 "slides": {
267 "01_title": {
268 "groups": [
269 { "id": "page-title", "cue": 1 },
270 { "id": "supporting-visual" }
271 ]
272 }
273 }
274 }
275 ```
276
277 `narration_sync.py subtitles` may still write `<project_path>/notes/subtitles/total.srt` as a PPTX-timeline diagnostic. It is not the delivery subtitle for a finished video.
278
279 When video export was selected, `powerpoint_video.py` opens the final narrated PPTX through local Windows PowerPoint, requests its native video encoder with recorded timings and narrations enabled, and polls `CreateVideoStatus` until the MP4 succeeds, fails, or times out. The interface is synchronous to its caller even though PowerPoint performs encoding asynchronously. It preserves PowerPoint's own animation and media behavior rather than re-rendering the deck.
280
281 If native video export fails, keep the narrated PPTX as a successful upstream
282 artifact and report the video failure separately. Do not regenerate audio or
283 the PPTX unless their own validation failed.
284
285 After the MP4 exists, `video_subtitles.py` takes the exact narration text frozen in the page SRT set and force-aligns it against the finished video's actual audio track with `stable-ts`. Long delivery cues may be split for display at this final stage. This writes a same-stem external SRT without changing the MP4, notes, page SRT, or animation files.
286
287 This stage keeps subtitles as external SRT files. It does not burn subtitles into the video. Automatic MP4 export is an optional Windows PowerPoint integration, not an independent renderer; when PowerPoint automation is unavailable, stop after the narrated PPTX instead of claiming a downgraded video.
288
289 **Caller integration**:
290
291 | Caller | After audio generation |
292 |---|---|
293 | Generate PPTX | With Edge SRT and an existing `animations.json`, derive `narration_animations.json`; with no sidecar, inherit the base report's resolved motion, while explicit all-motion-off uses `--no-animations`. Export with `--recorded-narration audio`, optionally continue through `powerpoint_video.py`, then generate the delivery SRT from the finished video. |
294 | Enhance Native PPTX | Return to [`native-enhance-pptx`](../native-enhance-pptx.md) Step 9; its `apply` command owns audio relationships, timings, transitions, and the enhanced export. If video was selected, pass that final PPTX to `powerpoint_video.py`. |
295
296 For Generate PPTX, `--recorded-narration audio` prepares PowerPoint's recorded timings and narrations: every slide must have a matching supported audio file, every duration must be readable by `ffprobe`, and object animations must not use `--animation-trigger on-click`. Use `after-previous` or `with-previous` for narrated/video export. Narration changes the slide-advance layer only: the resolved page-transition effect remains unchanged, `-t none` remains visually transition-free, and narration advance disables click while using audio duration plus padding. The re-export is saved as `exports/<project_name>_<timestamp>_narrated.pptx`, telling it apart from silent exports.
297
298 **Narrated SVG export**: use the default text-flow mode. It keeps authored line breaks in one editable, no-wrap text frame; narration does not require per-line text frames.
299
300 ---
301
302 ## Step 5: Completion report
303
304 Output one summary block listing:
305
306 - Number of audio files generated and their location (`<project_path>/audio/*`).
307 - For edge, number of matching page-local SRT files and their location (`<project_path>/notes/subtitles/*`).
308 - For narrated object animation, whether current SVG semantics were reused or which missing/stale pages were reread, plus semantic mapping coverage and fallback count.
309 - For Generate PPTX with Edge SRT and canonical custom animation, derived narration animation group count and `narration_animations.json` path; otherwise report inherited base motion or explicit all-motion-off.
310 - When video export was selected, the final MP4 path and native PowerPoint export status.
311 - When a finished video exists, the final aligned sidecar SRT path.
312 - The provider, voice, and rate/settings actually used.
313 - The caller-owned integration result: narrated SVG export path, enhanced native PPTX path, or “audio only”.
314 - For Generate PPTX when embedding was skipped, one-line hint: `python3 skills/ppt-master/scripts/svg_to_pptx.py <project_path> --recorded-narration audio`.
315
315 lines MARKDOWN