| 1 | # Conversion Tools |
| 2 | |
| 3 | > **Design boundary**: use native-Python converters for supported formats, |
| 4 | > invoke Pandoc only for explicit fallback formats, and let web conversion use |
| 5 | > `curl_cffi` when available for sites that reject Python's default TLS |
| 6 | > fingerprint. |
| 7 | |
| 8 | Source conversion tools turn PDFs, documents, slide decks, and web pages into Markdown before project creation. |
| 9 | |
| 10 | Default workflow entry: use `source_to_md.py` unless a backend-specific |
| 11 | diagnostic or forced route is needed. |
| 12 | |
| 13 | ## Shared Output Contract |
| 14 | |
| 15 | All `source_to_md` converters keep their existing Markdown output behavior and |
| 16 | now also write a lightweight sidecar profile when conversion succeeds: |
| 17 | |
| 18 | | Output | Convention | |
| 19 | |---|---| |
| 20 | | Markdown | `<stem>.md` beside the local source unless `-o` selects another path | |
| 21 | | Asset directory | `<stem>_files/` when the backend extracts images or media | |
| 22 | | Image manifest | `<stem>_files/image_manifest.json` when image metadata is available | |
| 23 | | Conversion profile | `<stem>.conversion_profile.json` beside the Markdown output | |
| 24 | |
| 25 | The conversion profile is metadata only. It records the converter, source path, |
| 26 | Markdown structure counts, asset directory, image manifest path, and image |
| 27 | count. Downstream PPT workflows still use the Markdown and image manifest as the |
| 28 | content/asset contract; the profile is for inspection and debugging. |
| 29 | |
| 30 | ## `source_to_md.py` |
| 31 | |
| 32 | Unified dispatcher for ad hoc explicit-source conversion. It auto-detects each |
| 33 | listed input file or URL and calls the existing backend converter, so backend |
| 34 | behavior remains the source of truth. |
| 35 | |
| 36 | Routing is centralized in `source_to_md/_dispatcher.py` and reused by |
| 37 | `project_manager.py import-sources`; do not add a second type-to-backend table. |
| 38 | |
| 39 | ```bash |
| 40 | python3 scripts/source_to_md.py paper.pdf |
| 41 | python3 scripts/source_to_md.py paper.pdf report.docx deck.pptx |
| 42 | python3 scripts/source_to_md.py ./sources |
| 43 | python3 scripts/source_to_md.py ./pdfs/*.pdf |
| 44 | python3 scripts/source_to_md.py ./decks/*.pptx |
| 45 | python3 scripts/source_to_md.py report.docx -o report.md |
| 46 | python3 scripts/source_to_md.py ./sources -o ./markdown # explicit separate output directory |
| 47 | python3 scripts/source_to_md.py workbook.xlsx --json |
| 48 | python3 scripts/source_to_md.py deck.pptx |
| 49 | python3 scripts/source_to_md.py https://example.com/article -o article.md |
| 50 | ``` |
| 51 | |
| 52 | Useful options: |
| 53 | - `-t pdf|doc|excel|pptx|web|markdown|text` forces a route when extension |
| 54 | detection is not enough. |
| 55 | - `--json` prints a compact machine-readable result after success when the |
| 56 | output path is known. With multiple inputs, each successful conversion prints |
| 57 | its own JSON line after that source finishes. |
| 58 | - `--images all|filtered|none`, `--no-images`, and `--filter-images` map to the |
| 59 | existing PDF image mode. They are intentionally PDF-only until other backends |
| 60 | expose the same behavior natively. |
| 61 | - Unknown backend-specific flags are passed through to each selected converter. |
| 62 | - `-o/--output` selects one Markdown file for one input, or an output directory |
| 63 | for multiple inputs / directory inputs. |
| 64 | |
| 65 | For multi-source project intake, use `project_manager.py import-sources` with |
| 66 | all source paths / URLs. For local files, the default is to keep generated |
| 67 | Markdown/profile outputs beside the original source. `source_to_md.py` and the |
| 68 | backend converters support single files, explicit multi-file inputs, and |
| 69 | non-recursive directory inputs. |
| 70 | |
| 71 | ## `source_to_md/pdf_to_md.py` |
| 72 | |
| 73 | Recommended first choice for native PDFs. |
| 74 | |
| 75 | ```bash |
| 76 | python3 scripts/source_to_md/pdf_to_md.py book.pdf |
| 77 | python3 scripts/source_to_md/pdf_to_md.py book.pdf -o output.md |
| 78 | python3 scripts/source_to_md/pdf_to_md.py book.pdf appendix.pdf |
| 79 | python3 scripts/source_to_md/pdf_to_md.py ./pdfs |
| 80 | python3 scripts/source_to_md/pdf_to_md.py ./pdfs -o ./markdown # explicit separate output directory |
| 81 | |
| 82 | # Image extraction control (default: filtered) |
| 83 | python3 scripts/source_to_md/pdf_to_md.py book.pdf --images filtered # size/quality filters applied |
| 84 | python3 scripts/source_to_md/pdf_to_md.py book.pdf --images all # extract all images, no filtering |
| 85 | python3 scripts/source_to_md/pdf_to_md.py book.pdf --images none # skip all images (text only) |
| 86 | ``` |
| 87 | |
| 88 | Use cases: |
| 89 | - Native PDFs exported from Word, PowerPoint, LaTeX, or similar tools |
| 90 | - Privacy-sensitive documents that should stay local |
| 91 | - Fast first-pass extraction before falling back to OCR-heavy tools |
| 92 | |
| 93 | Prefer MinerU or another OCR/layout tool when: |
| 94 | - The PDF is scanned or image-based |
| 95 | - Multi-column layout parsing is poor |
| 96 | - Encoding is garbled |
| 97 | |
| 98 | Dependency: |
| 99 | |
| 100 | ```bash |
| 101 | pip install PyMuPDF |
| 102 | ``` |
| 103 | |
| 104 | ## `source_to_md/doc_to_md.py` |
| 105 | |
| 106 | Hybrid converter: pure-Python for the common formats, pandoc fallback for the rest. |
| 107 | |
| 108 | Native path (no external binary required): |
| 109 | - `.docx` — via `mammoth`; text-only tables are preserved as pipe Markdown, and OMML / Office Math equations (Word-native or MathType "Convert to Office Math") are rewritten to inline LaTeX. Classic MathType OLE objects carry no OMML and are kept only as their preview image. |
| 110 | - `.html` / `.htm` — via `markdownify` + `beautifulsoup4` |
| 111 | - `.epub` — via `ebooklib` + `markdownify` |
| 112 | - `.ipynb` — via `nbconvert` |
| 113 | |
| 114 | Pandoc fallback (only if you need these): |
| 115 | - `.doc`, `.odt`, `.rtf`, `.tex`/`.latex`, `.rst`, `.org`, `.typ` |
| 116 | |
| 117 | ```bash |
| 118 | python3 scripts/source_to_md/doc_to_md.py lecture.docx |
| 119 | python3 scripts/source_to_md/doc_to_md.py lecture.docx -o output.md |
| 120 | python3 scripts/source_to_md/doc_to_md.py lecture.docx notes.html |
| 121 | python3 scripts/source_to_md/doc_to_md.py ./docs |
| 122 | python3 scripts/source_to_md/doc_to_md.py ./docs -o ./markdown # explicit separate output directory |
| 123 | python3 scripts/source_to_md/doc_to_md.py notes.epub |
| 124 | python3 scripts/source_to_md/doc_to_md.py paper.tex -o paper.md # uses pandoc |
| 125 | ``` |
| 126 | |
| 127 | Dependencies: |
| 128 | |
| 129 | ```bash |
| 130 | # Native path — always required |
| 131 | pip install mammoth markdownify ebooklib nbconvert beautifulsoup4 |
| 132 | |
| 133 | # Fallback path — only for .doc/.odt/.rtf/.tex/.rst/.org/.typ |
| 134 | # macOS: brew install pandoc |
| 135 | # Ubuntu: sudo apt install pandoc |
| 136 | # Windows: https://pandoc.org/installing.html |
| 137 | ``` |
| 138 | |
| 139 | All paths produce the same output convention: `<input>.md` plus a sibling `<input>_files/` directory containing extracted images with relative references. |
| 140 | On success, a sibling `<input>.conversion_profile.json` is also written. |
| 141 | |
| 142 | ## `source_to_md/excel_to_md.py` |
| 143 | |
| 144 | Excel workbook converter for presentation source intake. |
| 145 | |
| 146 | Supported formats: |
| 147 | - `.xlsx` |
| 148 | - `.xlsm` |
| 149 | |
| 150 | Unsupported by default: |
| 151 | - `.xls` — resave as `.xlsx` first |
| 152 | |
| 153 | ```bash |
| 154 | python3 scripts/source_to_md/excel_to_md.py report.xlsx |
| 155 | python3 scripts/source_to_md/excel_to_md.py report.xlsx -o output.md |
| 156 | python3 scripts/source_to_md/excel_to_md.py report.xlsx budget.xlsm |
| 157 | python3 scripts/source_to_md/excel_to_md.py ./workbooks |
| 158 | python3 scripts/source_to_md/excel_to_md.py ./workbooks -o ./markdown # explicit separate output directory |
| 159 | python3 scripts/source_to_md/excel_to_md.py report.xlsm --max-rows 200 --max-cols 40 |
| 160 | ``` |
| 161 | |
| 162 | Behavior: |
| 163 | - preserves workbook and sheet structure in Markdown |
| 164 | - exports visible sheets only |
| 165 | - trims empty outer rows and columns |
| 166 | - propagates merged-cell labels for readable Markdown tables |
| 167 | - exports formula cells as cached values; it does not recalculate formulas |
| 168 | - writes `<input>.conversion_profile.json` after successful conversion |
| 169 | |
| 170 | Dependency: |
| 171 | |
| 172 | ```bash |
| 173 | pip install openpyxl |
| 174 | ``` |
| 175 | |
| 176 | CSV/TSV files are already plain-text table sources and do not require this converter. |
| 177 | |
| 178 | ## `source_to_md/ppt_to_md.py` |
| 179 | |
| 180 | Structured PowerPoint-to-Markdown converter for Open XML slide decks. |
| 181 | |
| 182 | Supported formats include: |
| 183 | - `.pptx`, `.pptm` |
| 184 | - `.ppsx`, `.ppsm` |
| 185 | - `.potx`, `.potm` |
| 186 | |
| 187 | ```bash |
| 188 | python3 scripts/source_to_md/ppt_to_md.py sales_deck.pptx |
| 189 | python3 scripts/source_to_md/ppt_to_md.py sales_deck.pptx -o output.md |
| 190 | python3 scripts/source_to_md/ppt_to_md.py sales_deck.pptx appendix.pptx |
| 191 | python3 scripts/source_to_md/ppt_to_md.py ./decks |
| 192 | python3 scripts/source_to_md/ppt_to_md.py ./decks -o ./markdown # explicit separate output directory |
| 193 | python3 scripts/source_to_md/ppt_to_md.py template.ppsx -o notes/template.md |
| 194 | ``` |
| 195 | |
| 196 | Behavior: |
| 197 | - extracts slide text in reading order |
| 198 | - converts PowerPoint tables to Markdown tables; cell-internal line breaks become `<br>` so they cannot break the pipe-table row structure |
| 199 | - transcribes category charts as category × series tables and scatter/bubble charts as typed X/Y[/size] point tables |
| 200 | - preserves every readable chart dimension or series and emits `[Chart data warning: <reason>]` for missing caches/count mismatches; `[Chart data unavailable: <reason>]` is reserved for charts with no readable points (and unsupported ChartEx), so XY data is never flattened into a misleading category table |
| 201 | - transcribes SmartArt semantic nodes as hierarchical Markdown; unreadable diagram data emits an explicit placeholder and conversion warning |
| 202 | - exports embedded pictures to a sibling `_files/` directory |
| 203 | - appends speaker notes when present |
| 204 | - writes `<input>.conversion_profile.json` after successful conversion |
| 205 | |
| 206 | Dependency: |
| 207 | |
| 208 | ```bash |
| 209 | pip install python-pptx |
| 210 | ``` |
| 211 | |
| 212 | Legacy `.ppt` is not parsed directly. Resave it as `.pptx` or export it to PDF first. |
| 213 | |
| 214 | ## `pptx_intake.py` |
| 215 | |
| 216 | Standard enrichment layer for PPTX sources. It complements `ppt_to_md.py` rather |
| 217 | than replacing it: Markdown remains the normalized content source, while intake |
| 218 | artifacts provide source facts for Strategist and standalone PPTX workflows. |
| 219 | |
| 220 | ```bash |
| 221 | python3 scripts/pptx_intake.py deck.pptx -o projects/demo/analysis |
| 222 | ``` |
| 223 | |
| 224 | Outputs (per source deck, prefixed by file stem): |
| 225 | - `<stem>.identity.json` — canvas size/aspect, theme palette/fonts, observed colors/fonts |
| 226 | - `<stem>.slide_library.json` — text slots, geometry, native tables, native chart display caches, and SmartArt nodes/connections |
| 227 | - `source_profile.json` — the single multi-deck index: a compact Strategist-facing digest per deck (over identity, tables, charts, SmartArt, and page types) under `decks[]`, with prefixed artifact pointers |
| 228 | |
| 229 | `project_manager.py import-sources` runs this automatically for PPTX/PPTM/PPSX/PPSM/POTX/POTM inputs and stores the bundle directly under `analysis/`. Multi-deck per project: importing several PPTX files gives each its own `<stem>.*` artifacts and a `decks[]` entry in the shared `source_profile.json` index (re-importing the same stem replaces its entry). The beautify profile and Fill Native PPTX route stay single-deck and read one chosen deck's `<stem>.*` artifacts. |
| 230 | |
| 231 | Usage boundary: |
| 232 | - Standard generation uses these fields as facts and recommendation candidates; it does not inherit source slide coordinates or page order by default. |
| 233 | - Beautify promotes selected identity/content fields into locked constraints after confirmation and redraws SmartArt meaning with ordinary editable shapes. |
| 234 | - Template-fill uses the slide library as the native PPTX fill contract; SmartArt is inventory-only and remains unchanged. |
| 235 | |
| 236 | ## `pptx_to_svg.py` |
| 237 | |
| 238 | Reconstruct a PPTX package as editable SVG views by reading OOXML directly. |
| 239 | |
| 240 | ```bash |
| 241 | python3 scripts/pptx_to_svg.py deck.pptx --inheritance-mode both |
| 242 | python3 scripts/pptx_to_svg.py deck.pptx --inheritance-mode layered |
| 243 | python3 scripts/pptx_to_svg.py deck.pptx --inheritance-mode flat |
| 244 | python3 scripts/pptx_to_svg.py deck.pptx --strict |
| 245 | ``` |
| 246 | |
| 247 | | Mode | Output | |
| 248 | |---|---| |
| 249 | | `both` (default) | Layered master/layout/slide SVGs under `svg/`, plus self-contained slides under `svg-flat/` | |
| 250 | | `layered` | Only the layered `svg/` view and inheritance metadata | |
| 251 | | `flat` | One self-contained slide SVG per page under `svg/` | |
| 252 | |
| 253 | For Office pictures that carry both a raster compatibility preview on |
| 254 | `a:blip` and an editable SVG relationship in `asvg:svgBlip`, import resolves |
| 255 | the SVG relationship first. The raster relationship is used only when the SVG |
| 256 | relationship or media part cannot be read. The template manifest uses the same |
| 257 | relationship preference for asset identity; its existing missing-media gate |
| 258 | remains strict rather than silently treating the raster preview as the |
| 259 | template's canonical asset. |
| 260 | |
| 261 | ### Import compatibility and recovery boundary |
| 262 | |
| 263 | Import is tolerant by default because the source deck is user-owned or comes |
| 264 | from third-party authoring tools. Recovery happens at the narrowest safe |
| 265 | boundary: first omit only an unsupported property or feature; if that is not |
| 266 | possible, replace only the affected object with a visible diagnostic |
| 267 | placeholder; omit a background without discarding its page. Corrupt ZIP/XML or |
| 268 | missing required package structure remains fatal because no safe local recovery |
| 269 | exists. Pass `--strict` for parser development or contract verification when |
| 270 | the first unsupported/malformed source construct should stop conversion. |
| 271 | |
| 272 | Every successful run writes `<output>/conversion-report.json`. Its stable |
| 273 | top-level fields are `schemaVersion`, `source`, `mode`, `summary`, and |
| 274 | `diagnostics`. Each diagnostic records a reason `code`, source `message`, chosen |
| 275 | `fallback`, package `part_path`, and—when available—`slide_index`, `shape_id`, |
| 276 | `shape_name`, and `shape_kind`. The command also prints a bounded warning |
| 277 | summary instead of a raw Python traceback. |
| 278 | |
| 279 | In the detailed native-object notes below, “fails closed” or “error” describes |
| 280 | the native replacement claim or strict mode. Default tolerant deck import |
| 281 | retains the usable fallback/object and records the degradation; it does not |
| 282 | discard unrelated shapes, pages, or the entire deck. |
| 283 | |
| 284 | ### Native table and chart import claims |
| 285 | |
| 286 | Supported text-grid tables and conservative classic-chart caches carry a |
| 287 | `data-pptx-replace-with` claim beside their SVG fallback, with the replacement |
| 288 | payload in a child `<metadata type="application/json">`. The parent claim |
| 289 | selects the table or chart schema. Table import requires |
| 290 | exact physical row/grid topology and accepts canonical rectangular merges, |
| 291 | safe solid/no-fill per-side borders, plain multi-paragraph cells, and a closed |
| 292 | run-rich paragraph schema. |
| 293 | Each run requires `text` and may use only `bold`, `italic`, `underline`, |
| 294 | `strike`, `color`, `font_size`, one `font_family`, `lang`, and `alt_lang`. |
| 295 | Presentation-only source run XML without a non-empty `effectLst` / `effectDag` |
| 296 | normalizes. A table-cell run effect disables native replacement and adds a |
| 297 | blocking effect diagnostic. Relationship-bearing text, extensions, |
| 298 | noncanonical/overlapping merges, nonblank merge slaves, unsafe border XML, |
| 299 | non-solid fills, structural line breaks/fields/tabs/bullets, and broken text |
| 300 | topology remain fallback-only. |
| 301 | Markers remain dormant |
| 302 | unless a later export uses `--native-charts-and-tables`. That opt-in is |
| 303 | data-object-first: the default fallback still exports as editable DrawingML |
| 304 | shapes, while the opt-in supplies a data source and PowerPoint's |
| 305 | chart/table-specific object model. |
| 306 | The native-object route may normalize styling or omit marker-local details not represented by the |
| 307 | payload, and export reports that risk without disabling an otherwise supported |
| 308 | active marker. Unsupported tables keep their |
| 309 | rendered SVG table; unsupported charts keep a baked preview when one exists. |
| 310 | For the currently supported parsed classic families (column/bar/line/area, |
| 311 | pie/doughnut, scatter, and bubble), a chart without a baked preview receives a |
| 312 | deterministic readable fallback marked |
| 313 | `data-pptx-fallback-kind="normalized"`. Unknown style XML disables the native |
| 314 | replacement claim or falls back to a diagnostic object in tolerant mode; |
| 315 | common solid/no-fill/line/marker forms and scheme colors are normalized for the |
| 316 | SVG fallback and core payload colors, while native opt-in may still normalize |
| 317 | unmodeled alpha, line, marker, or no-fill details. Common General, decimal, |
| 318 | grouped, percent, and simple currency-prefix data-label formats render |
| 319 | deterministically; an unknown Excel format program keeps the active payload but |
| 320 | does not claim a normalized fallback. Active types outside the current renderer |
| 321 | continue to use an explicit placeholder marked |
| 322 | `data-pptx-fallback-kind="placeholder"`. Validation and export report that |
| 323 | reconstruction-only fallback as a warning. Default export keeps the |
| 324 | placeholder; when the same group has a valid active chart replacement payload, |
| 325 | `--native-charts-and-tables` may still reconstruct the PowerPoint-native chart. |
| 326 | Invalid or contradictory fallback declarations remain errors. Fallback-only |
| 327 | replacement capability uses `data-pptx-replacement-status` and remains a |
| 328 | warning when the SVG fallback itself is complete. Imported table/chart groups |
| 329 | under this contract carry `data-pptx-import-source="pptx"`, whether active or |
| 330 | fallback-only; generated authoring omits this provenance attribute. |
| 331 | |
| 332 | Active imported markers also carry `data-pptx-fallback-sha256`, computed over |
| 333 | their canonical fallback plus reachable document-level SVG fragment definitions. |
| 334 | A later visible edit, reachable definition change, local reference-target |
| 335 | change, or marker transform makes the replacement metadata stale. The mandatory |
| 336 | quality checker reports the mismatch; default export keeps the edited fallback, |
| 337 | while `--native-charts-and-tables` fails before replacement so it cannot discard that edit. |
| 338 | `visibility:hidden` content, marker-local unused definitions, and explicitly |
| 339 | referenced document-level target roots (even when hidden) are included |
| 340 | conservatively; marker-local `display:none` subtrees are excluded, and external |
| 341 | file bytes are not read. |
| 342 | Generated authoring and reusable templates omit import provenance and do not |
| 343 | preseed a static fallback hash; that state is normal and does not warn. A legacy |
| 344 | imported marker that still carries PPTX import provenance but lacks the hash |
| 345 | remains native-compatible and warns in the checker/native route that stale |
| 346 | detection is unavailable. |
| 347 | |
| 348 | Legacy `data-pptx-native*`, `data-pptx-visual-status`, and |
| 349 | `data-pptx-route-status` spellings remain read-compatible. New importer output |
| 350 | and generated SVG use the replacement/fallback names above. The old |
| 351 | `--native-objects` option remains a compatibility alias for |
| 352 | `--native-charts-and-tables`. |
| 353 | |
| 354 | For table style `{5C22544A-7EE6-4342-B048-85BDC9FD1C3A}`, the importer resolves |
| 355 | the normalized `wholeTbl`, `firstRow`, `band1H`/`band2H`, theme color/font, and |
| 356 | direct-format override subset. Other built-in/custom style families remain |
| 357 | outside this guarantee. |
| 358 | |
| 359 | The chart importer also accepts the verified column/line/area combo subset, |
| 360 | canonical four-series OHLC stock charts with shared numeric date caches, area |
| 361 | charts with numeric date axes, and verified scatter/bubble charts with a closed |
| 362 | pair of `axes.x` / `axes.y` value axes. Combo primary/secondary plots may retain |
| 363 | independent category caches and workbook ranges. Both the category/value and XY |
| 364 | contracts read back the supported kind/position/visibility, label-position, |
| 365 | number-format, min/max/major-unit, reverse, and major-gridline fields; the native |
| 366 | writer emits every field in those closed contracts. Scatter style is derived |
| 367 | from uniform effective series line/marker/smooth state. The normalized XY |
| 368 | fallback newly consumes only the two major-gridline flags, not the remaining |
| 369 | axis fields. The importer also accepts radar, safe `of_pie` `serLines`, the |
| 370 | closed axis/title/legend normalization cases, and bar/column `gapWidth` / |
| 371 | `overlap`. `gapWidth` must be one integer in `0..500` and `overlap` one integer |
| 372 | in `-100..100`; both normalize in native output, while malformed, duplicate, or |
| 373 | out-of-range values disable the native replacement claim in tolerant mode and |
| 374 | stop strict import. These additions do not expand the normalized |
| 375 | renderer. |
| 376 | Safe stock series style may pass the structural gate, while stock series, |
| 377 | `hiLowLines`, and up-down bar local styling can still normalize under the |
| 378 | data-object-first contract. |
| 379 | ChartEx import accepts exactly the validated treemap, sunburst, histogram, |
| 380 | pareto, box-whisker, waterfall, and funnel data models. Their supported |
| 381 | hierarchy/category/value/series/subtotal topology round-trips to native output. |
| 382 | Numeric caches must be non-empty and finite, with canonical non-negative counts |
| 383 | and indexes and exact contiguous point topology. Source style, axes, labels, |
| 384 | and binning details may normalize. This is not full `AxisSpec`, arbitrary |
| 385 | ChartEx import or presentation fidelity, arbitrary stock variants, other |
| 386 | date-axis chart families, or unlisted axis semantics. ChartEx native output |
| 387 | still consumes valid payload colors in its color-style part. |
| 388 | |
| 389 | Exporter-canonical charts recover canonical solid series/slice colors and exact |
| 390 | one- or two-paragraph title styling; two paragraphs retain their `title` / |
| 391 | `subtitle` roles. This is not a general source-chart style round-trip guarantee. |
| 392 | |
| 393 | Concrete slide SVGs resolve `<a:fld type="slidenum">` using the presentation's |
| 394 | `firstSlideNum` display numbering. Standalone master/layout SVGs keep the |
| 395 | literal field fallback because one shared part can serve multiple slides. |
| 396 | |
| 397 | ### Maintenance smoke checks |
| 398 | |
| 399 | Run these checks from the repository root after changing `pptx_to_svg/` or its |
| 400 | CLI. They generate every required input under `/tmp`; do not replace them with |
| 401 | a committed `test_*.py` suite. |
| 402 | |
| 403 | #### Healthy generated deck |
| 404 | |
| 405 | ```bash |
| 406 | python3 - <<'PY' |
| 407 | from pptx import Presentation |
| 408 | from pptx.enum.shapes import MSO_SHAPE |
| 409 | from pptx.util import Inches |
| 410 | |
| 411 | presentation = Presentation() |
| 412 | slide = presentation.slides.add_slide(presentation.slide_layouts[6]) |
| 413 | shape = slide.shapes.add_shape( |
| 414 | MSO_SHAPE.RECTANGLE, |
| 415 | Inches(1), |
| 416 | Inches(1), |
| 417 | Inches(3), |
| 418 | Inches(1), |
| 419 | ) |
| 420 | shape.text = "PPTX import smoke check" |
| 421 | presentation.save("/tmp/ppt-master-smoke-healthy.pptx") |
| 422 | PY |
| 423 | |
| 424 | python3 "skills/ppt-master/scripts/pptx_to_svg.py" \ |
| 425 | "/tmp/ppt-master-smoke-healthy.pptx" \ |
| 426 | --inheritance-mode flat \ |
| 427 | -o "/tmp/ppt-master-smoke-healthy" |
| 428 | python3 -c "import json; from pathlib import Path; report = json.loads(Path('/tmp/ppt-master-smoke-healthy/conversion-report.json').read_text()); assert report['summary'] == {'slides': 1, 'warnings': 0}, report['summary']; print('OK: 1 slide, 0 warnings')" |
| 429 | ``` |
| 430 | |
| 431 | Expected: both commands exit `0`; the assertion prints |
| 432 | `OK: 1 slide, 0 warnings`. |
| 433 | |
| 434 | #### Tolerant/strict color-structure probe |
| 435 | |
| 436 | Generate a two-shape PPTX, then add one foreign attribute to the first shape's |
| 437 | valid `a:srgbClr` node: |
| 438 | |
| 439 | ```bash |
| 440 | python3 -c ' |
| 441 | from pathlib import Path |
| 442 | from zipfile import ZIP_DEFLATED, ZipFile |
| 443 | |
| 444 | from pptx import Presentation |
| 445 | from pptx.dml.color import RGBColor |
| 446 | from pptx.enum.shapes import MSO_SHAPE |
| 447 | from pptx.util import Inches |
| 448 | |
| 449 | base = Path("/tmp/ppt-master-color-smoke-base.pptx") |
| 450 | target = Path("/tmp/ppt-master-color-smoke.pptx") |
| 451 | presentation = Presentation() |
| 452 | slide = presentation.slides.add_slide(presentation.slide_layouts[6]) |
| 453 | for left, color in ((1, (0x44, 0x72, 0xC4)), (4, (0xED, 0x7D, 0x31))): |
| 454 | shape = slide.shapes.add_shape( |
| 455 | MSO_SHAPE.RECTANGLE, |
| 456 | Inches(left), |
| 457 | Inches(1), |
| 458 | Inches(2), |
| 459 | Inches(1), |
| 460 | ) |
| 461 | shape.fill.solid() |
| 462 | shape.fill.fore_color.rgb = RGBColor(*color) |
| 463 | presentation.save(base) |
| 464 | |
| 465 | with ZipFile(base) as source, ZipFile(target, "w", ZIP_DEFLATED) as destination: |
| 466 | patched = False |
| 467 | for member in source.infolist(): |
| 468 | payload = source.read(member) |
| 469 | if member.filename == "ppt/slides/slide1.xml": |
| 470 | old = b"<a:srgbClr val=\"4472C4\"/>" |
| 471 | new = b"<a:srgbClr val=\"4472C4\" legacy=\"1\"/>" |
| 472 | if old not in payload: |
| 473 | raise RuntimeError("probe color node was not generated") |
| 474 | payload = payload.replace(old, new, 1) |
| 475 | patched = True |
| 476 | destination.writestr(member, payload) |
| 477 | if not patched: |
| 478 | raise RuntimeError("slide XML was not patched") |
| 479 | print(target) |
| 480 | ' |
| 481 | ``` |
| 482 | |
| 483 | Run tolerant import and verify both the recovery report and the visible SVG: |
| 484 | |
| 485 | ```bash |
| 486 | python3 "skills/ppt-master/scripts/pptx_to_svg.py" \ |
| 487 | "/tmp/ppt-master-color-smoke.pptx" \ |
| 488 | --inheritance-mode flat \ |
| 489 | -o "/tmp/ppt-master-smoke-color-tolerant" |
| 490 | python3 -c ' |
| 491 | import json |
| 492 | from pathlib import Path |
| 493 | |
| 494 | output = Path("/tmp/ppt-master-smoke-color-tolerant") |
| 495 | report = json.loads((output / "conversion-report.json").read_text()) |
| 496 | diagnostics = report["diagnostics"] |
| 497 | svg = (output / "svg" / "slide_01.svg").read_text() |
| 498 | assert report["summary"] == {"slides": 1, "warnings": 1}, report["summary"] |
| 499 | assert len(diagnostics) == 1, diagnostics |
| 500 | assert diagnostics[0]["code"] == "color-structure-normalized", diagnostics[0] |
| 501 | assert diagnostics[0]["fallback"] == "retain recognized color attributes and modifiers", diagnostics[0] |
| 502 | assert diagnostics[0]["slide_index"] == 1, diagnostics[0] |
| 503 | assert diagnostics[0]["shape_name"] == "Rectangle 1", diagnostics[0] |
| 504 | assert "#4472C4" in svg and "#ED7D31" in svg |
| 505 | print("OK: tolerant import recovered #4472C4 and preserved #ED7D31") |
| 506 | ' |
| 507 | ``` |
| 508 | |
| 509 | Expected: both commands exit `0`; the importer reports one |
| 510 | `color-structure-normalized` warning and the assertion prints |
| 511 | `OK: tolerant import recovered #4472C4 and preserved #ED7D31`. |
| 512 | |
| 513 | Run the same probe in strict mode: |
| 514 | |
| 515 | ```bash |
| 516 | python3 "skills/ppt-master/scripts/pptx_to_svg.py" \ |
| 517 | "/tmp/ppt-master-color-smoke.pptx" \ |
| 518 | --inheritance-mode flat \ |
| 519 | --strict \ |
| 520 | -o "/tmp/ppt-master-smoke-color-strict" |
| 521 | ``` |
| 522 | |
| 523 | Expected: exit `1`, no traceback, and one error line: |
| 524 | |
| 525 | ```text |
| 526 | Error: PPTX-to-SVG conversion failed: Invalid DrawingML sRGB color structure |
| 527 | ``` |
| 528 | |
| 529 | ## `source_to_md/web_to_md.py` |
| 530 | |
| 531 | Convert web pages to Markdown and download images locally. |
| 532 | |
| 533 | ```bash |
| 534 | python3 scripts/source_to_md/web_to_md.py https://example.com/article |
| 535 | python3 scripts/source_to_md/web_to_md.py https://url1.com https://url2.com |
| 536 | python3 scripts/source_to_md/web_to_md.py -f urls.txt |
| 537 | python3 scripts/source_to_md/web_to_md.py https://example.com -o output.md |
| 538 | python3 scripts/source_to_md/web_to_md.py https://example.com --emit-result /tmp/result.json |
| 539 | ``` |
| 540 | |
| 541 | When `curl_cffi` is installed (included in `requirements.txt`), this script |
| 542 | automatically impersonates a modern Chrome TLS fingerprint, which lets it |
| 543 | fetch WeChat Official Accounts (`mp.weixin.qq.com`) and other sites that |
| 544 | block Python's default TLS fingerprint. No extra flags needed. If |
| 545 | `curl_cffi` is not available, it falls back to plain `requests`. |
| 546 | |
| 547 | On success, the converter writes `<output>.conversion_profile.json` beside the |
| 548 | Markdown output. |
| 549 | `--emit-result` is for wrapper scripts that need the actual saved Markdown path |
| 550 | when the converter derives a title-based filename. |
| 551 | |
| 552 | |
| 553 | ## `rotate_images.py` |
| 554 | |
| 555 | Fix image EXIF orientation in downloaded or imported assets. |
| 556 | |
| 557 | ```bash |
| 558 | python3 scripts/rotate_images.py auto projects/xxx_files |
| 559 | python3 scripts/rotate_images.py gen projects/xxx_files |
| 560 | python3 scripts/rotate_images.py fix fixes.json |
| 561 | ``` |
| 562 | |
| 563 | Use this when extracted photos appear sideways after conversion or import. |
| 564 |