Skip to content

AI turns documents or topics into real, native PowerPoint decks—with native shapes, transitions and animations, data-backed charts and tables on demand, audio narration from speaker notes, and support for your own .pptx templates. · by Hugo He

feat(doc_to_md): native Python paths for docx/html/epub/ipynb

hugohe3167da49241df
diff --git a/AGENTS.md b/AGENTS.md
index 8d6435bf..4bafc524 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -28,7 +28,7 @@ PPT Master is an AI-driven presentation generation system. Through multi-role co
```bash
# Source content conversion
python3 skills/ppt-master/scripts/source_to_md/pdf_to_md.py <PDF_file>
-python3 skills/ppt-master/scripts/source_to_md/doc_to_md.py <DOCX_or_other_file> # Requires: pandoc (DOCX/EPUB/HTML/LaTeX/RST/etc.)
+python3 skills/ppt-master/scripts/source_to_md/doc_to_md.py <DOCX_or_other_file> # Native: .docx/.html/.epub/.ipynb; pandoc fallback: .doc/.odt/.rtf/.tex/.rst/.org/.typ
python3 skills/ppt-master/scripts/source_to_md/ppt_to_md.py <PPTX_file>
python3 skills/ppt-master/scripts/source_to_md/web_to_md.py <URL>
node skills/ppt-master/scripts/source_to_md/web_to_md.cjs <URL>
diff --git a/CLAUDE.md b/CLAUDE.md
index b8243882..4066f978 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -13,7 +13,7 @@ PPT Master is an AI-driven presentation generation system. Through multi-role co
```bash
# Source content conversion
python3 skills/ppt-master/scripts/source_to_md/pdf_to_md.py <PDF_file>
-python3 skills/ppt-master/scripts/source_to_md/doc_to_md.py <DOCX_or_other_file> # Requires: pandoc (DOCX/EPUB/HTML/LaTeX/RST/etc.)
+python3 skills/ppt-master/scripts/source_to_md/doc_to_md.py <DOCX_or_other_file> # Native: .docx/.html/.epub/.ipynb; pandoc fallback: .doc/.odt/.rtf/.tex/.rst/.org/.typ
python3 skills/ppt-master/scripts/source_to_md/ppt_to_md.py <PPTX_file>
python3 skills/ppt-master/scripts/source_to_md/web_to_md.py <URL>
node skills/ppt-master/scripts/source_to_md/web_to_md.cjs <URL>
diff --git a/README.md b/README.md
index a7e8dcf9..a08ef728 100644
--- a/README.md
+++ b/README.md
@@ -73,7 +73,7 @@ This project is my attempt to bridge the gap between **domain expertise** and **
|------------|:---------:|--------------|----------------|
| [Python](https://www.python.org/downloads/) 3.10+ | ✅ **Yes** | Core runtime — everything depends on this | Nothing works |
| [Node.js](https://nodejs.org/) 18+ | ❌ Optional | Converts WeChat articles and complex web pages to Markdown | You can still use PDF, DOCX, plain text, and simple URLs as source |
-| [Pandoc](https://pandoc.org/) | ❌ Optional | Converts DOCX, EPUB, HTML, LaTeX, RST to Markdown | You can still use PDF, plain text, and URLs as source |
+| [Pandoc](https://pandoc.org/) | ❌ Optional | Converts legacy formats (`.doc`, `.odt`, `.rtf`, `.tex`, `.rst`, `.org`, `.typ`) to Markdown. `.docx`, `.html`, `.epub`, `.ipynb` are handled natively in Python — no pandoc needed. | You can still use PDF, DOCX, HTML, EPUB, Jupyter, plain text, and URLs as source |
> **TL;DR** — Install Python, run `pip install -r requirements.txt`, and you're ready to generate presentations. Install the rest only when you need them.
@@ -95,12 +95,12 @@ Quick version: download Python from [python.org](https://www.python.org/download
# macOS
brew install python
brew install node # optional — for WeChat page conversion
-brew install pandoc # optional — for DOCX/EPUB conversion
+brew install pandoc # optional — only for legacy .doc/.odt/.rtf/.tex/.rst/.org/.typ
# Ubuntu / Debian
sudo apt install python3 python3-pip
sudo apt install nodejs npm # optional
-sudo apt install pandoc # optional
+sudo apt install pandoc # optional — only for legacy .doc/.odt/.rtf/.tex/.rst/.org/.typ
```
</details>
diff --git a/README_CN.md b/README_CN.md
index 5d0599b1..3c20341d 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -73,7 +73,7 @@ PPT Master 源于一个真实的痛点:在投融资和咨询工作中,我每
|--------|:--------:|------|-----------|
| [Python](https://www.python.org/downloads/) 3.10+ | ✅ **必需** | 核心运行时——一切都依赖它 | 什么都跑不了 |
| [Node.js](https://nodejs.org/) 18+ | ❌ 可选 | 将微信公众号文章、复杂网页转换为 Markdown | 仍可用 PDF、DOCX、纯文本、简单 URL 作为素材 |
-| [Pandoc](https://pandoc.org/) | ❌ 可选 | 将 DOCX、EPUB、HTML、LaTeX、RST 转换为 Markdown | 仍可用 PDF、纯文本、URL 作为素材 |
+| [Pandoc](https://pandoc.org/) | ❌ 可选 | 将小众格式(`.doc`/`.odt`/`.rtf`/`.tex`/`.rst`/`.org`/`.typ`)转换为 Markdown。`.docx`、`.html`、`.epub`、`.ipynb` 已由 Python 原生处理,无需 pandoc。 | 仍可用 PDF、DOCX、HTML、EPUB、Jupyter、纯文本、URL 作为素材 |
> **一句话总结** — 装好 Python,跑一行 `pip install -r requirements.txt`,就可以开始生成 PPT 了。其余的用到时再装。
@@ -95,12 +95,12 @@ Windows 需要一些额外步骤(PATH 设置、执行策略等)。我们为
# macOS
brew install python
brew install node # 可选——用于微信公众号等网页转换
-brew install pandoc # 可选——用于 DOCX/EPUB 转换
+brew install pandoc # 可选——仅用于 .doc/.odt/.rtf/.tex/.rst/.org/.typ 等小众格式
# Ubuntu / Debian
sudo apt install python3 python3-pip
sudo apt install nodejs npm # 可选
-sudo apt install pandoc # 可选
+sudo apt install pandoc # 可选——仅用于 .doc/.odt/.rtf/.tex/.rst/.org/.typ 等小众格式
```
</details>
diff --git a/docs/windows-installation.md b/docs/windows-installation.md
index 98ec0f20..b35e07c3 100644
--- a/docs/windows-installation.md
+++ b/docs/windows-installation.md
@@ -85,7 +85,7 @@ If a `.pptx` file appears in `exports/` that opens in PowerPoint — **you're do
| Dependency | How to install | Verify |
|------------|----------------|--------|
| **Node.js** — WeChat articles / complex web pages | Download LTS from [nodejs.org](https://nodejs.org/) | `node --version` → v18+ |
-| **Pandoc** — DOCX / EPUB conversion | Download `.msi` from [pandoc.org](https://pandoc.org/installing.html) | `pandoc --version` |
+| **Pandoc** — legacy formats only (`.doc`, `.odt`, `.rtf`, `.tex`, `.rst`, `.org`, `.typ`); `.docx`/`.html`/`.epub`/`.ipynb` work natively in Python | Download `.msi` from [pandoc.org](https://pandoc.org/installing.html) | `pandoc --version` |
| **CairoSVG** — higher quality PNG fallbacks | Install [GTK3 Runtime](https://github.com/nickvdp/gtk3/releases), then `pip install cairosvg` | `python -c "import cairosvg"` |
> Skip CairoSVG if unsure. PPT Master auto-falls back to `svglib` (already installed via `requirements.txt`), which works fine for most cases.
diff --git a/docs/zh/windows-installation.md b/docs/zh/windows-installation.md
index 0908b46a..5330fa61 100644
--- a/docs/zh/windows-installation.md
+++ b/docs/zh/windows-installation.md
@@ -88,7 +88,7 @@ python -c "import pptx; import fitz; print('All core dependencies OK')"
| 依赖 | 安装方式 | 验证 |
|------|---------|------|
| **Node.js** — 微信公众号 / 复杂网页转换 | [nodejs.org](https://nodejs.org/) 下载 LTS 版安装 | `node --version` → v18+ |
-| **Pandoc** — DOCX / EPUB 转换 | [pandoc.org](https://pandoc.org/installing.html) 下载 `.msi` 安装 | `pandoc --version` |
+| **Pandoc** — 仅用于小众格式(`.doc`/`.odt`/`.rtf`/`.tex`/`.rst`/`.org`/`.typ`);`.docx`/`.html`/`.epub`/`.ipynb` 已由 Python 原生处理 | [pandoc.org](https://pandoc.org/installing.html) 下载 `.msi` 安装 | `pandoc --version` |
| **CairoSVG** — 更高质量 PNG 后备图 | 安装 [GTK3 Runtime](https://github.com/nickvdp/gtk3/releases) 后 `pip install cairosvg` | `python -c "import cairosvg"` |
> CairoSVG 不确定就跳过。PPT Master 会自动用 `svglib`(已随 `requirements.txt` 安装),大多数情况够用。
diff --git a/requirements.txt b/requirements.txt
index 22a65559..e682ce06 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -39,10 +39,23 @@ PyMuPDF>=1.23.0
# ─────────────────────────────────────────────────────────────
# Document to Markdown tool / 文档转 Markdown 工具 (skills/ppt-master/scripts/source_to_md/doc_to_md.py)
# ─────────────────────────────────────────────────────────────
-# Uses Pandoc to convert DOCX/EPUB/HTML/LaTeX/RST etc. to Markdown.
-# 使用 Pandoc 将 DOCX/EPUB/HTML/LaTeX/RST 等格式转换为 Markdown
-# No Python package dependencies. Requires system-level pandoc:
-# 无 Python 包依赖,需要系统安装 pandoc:
+# Native (pure-Python) paths — no external binary required:
+# 原生路径(纯 Python,无需外部程序):
+# .docx → mammoth
+# .html → markdownify (+ beautifulsoup4, listed below)
+# .epub → ebooklib (+ markdownify)
+# .ipynb → nbconvert
+mammoth>=1.6.0
+markdownify>=0.11.6
+ebooklib>=0.18
+nbconvert>=7.0.0
+#
+# Fallback path — only needed for less common formats
+# (.doc .odt .rtf .tex/.latex .rst .org .typ).
+# Not required if you only convert the four native formats above.
+# 备用路径 —— 仅在需要处理以下小众格式时才用到
+# (.doc .odt .rtf .tex/.latex .rst .org .typ)。
+# 如果只处理上述四种原生格式,无需安装。
# macOS: brew install pandoc
# Ubuntu: sudo apt install pandoc
# Windows: https://pandoc.org/installing.html
diff --git a/skills/ppt-master/SKILL.md b/skills/ppt-master/SKILL.md
index d04e690d..56b1c610 100644
--- a/skills/ppt-master/SKILL.md
+++ b/skills/ppt-master/SKILL.md
@@ -45,7 +45,7 @@ description: >
| Script | Purpose |
|--------|---------|
| `${SKILL_DIR}/scripts/source_to_md/pdf_to_md.py` | PDF to Markdown |
-| `${SKILL_DIR}/scripts/source_to_md/doc_to_md.py` | Documents to Markdown via Pandoc (DOCX, EPUB, HTML, LaTeX, RST, etc.) |
+| `${SKILL_DIR}/scripts/source_to_md/doc_to_md.py` | Documents to Markdown — native Python for DOCX/HTML/EPUB/IPYNB, pandoc fallback for legacy formats (.doc/.odt/.rtf/.tex/.rst/.org/.typ) |
| `${SKILL_DIR}/scripts/source_to_md/ppt_to_md.py` | PowerPoint to Markdown |
| `${SKILL_DIR}/scripts/source_to_md/web_to_md.py` | Web page to Markdown |
| `${SKILL_DIR}/scripts/source_to_md/web_to_md.cjs` | WeChat / high-security sites to Markdown |
diff --git a/skills/ppt-master/scripts/docs/conversion.md b/skills/ppt-master/scripts/docs/conversion.md
index e3ff9066..5fe61a1b 100644
--- a/skills/ppt-master/scripts/docs/conversion.md
+++ b/skills/ppt-master/scripts/docs/conversion.md
@@ -31,29 +31,38 @@ pip install PyMuPDF
## `source_to_md/doc_to_md.py`
-Pandoc-based converter for office and markup formats.
+Hybrid converter: pure-Python for the common formats, pandoc fallback for the rest.
-Supported formats include:
-- `.docx`, `.doc`, `.odt`, `.rtf`
-- `.epub`, `.html`, `.tex`, `.rst`, `.org`, `.ipynb`, `.typ`
+Native path (no external binary required):
+- `.docx` — via `mammoth`
+- `.html` / `.htm` — via `markdownify` + `beautifulsoup4`
+- `.epub` — via `ebooklib` + `markdownify`
+- `.ipynb` — via `nbconvert`
+
+Pandoc fallback (only if you need these):
+- `.doc`, `.odt`, `.rtf`, `.tex`/`.latex`, `.rst`, `.org`, `.typ`
```bash
python3 scripts/source_to_md/doc_to_md.py lecture.docx
python3 scripts/source_to_md/doc_to_md.py lecture.docx -o output.md
python3 scripts/source_to_md/doc_to_md.py notes.epub
-python3 scripts/source_to_md/doc_to_md.py paper.tex -o paper.md
+python3 scripts/source_to_md/doc_to_md.py paper.tex -o paper.md # uses pandoc
```
-Dependency:
+Dependencies:
```bash
-# macOS
-brew install pandoc
+# Native path — always required
+pip install mammoth markdownify ebooklib nbconvert beautifulsoup4
-# Ubuntu
-sudo apt install pandoc
+# Fallback path — only for .doc/.odt/.rtf/.tex/.rst/.org/.typ
+# macOS: brew install pandoc
+# Ubuntu: sudo apt install pandoc
+# Windows: https://pandoc.org/installing.html
```
+All paths produce the same output convention: `<input>.md` plus a sibling `<input>_files/` directory containing extracted images with relative references.
+
## `source_to_md/ppt_to_md.py`
Structured PowerPoint-to-Markdown converter for Open XML slide decks.
diff --git a/skills/ppt-master/scripts/source_to_md/doc_to_md.py b/skills/ppt-master/scripts/source_to_md/doc_to_md.py
index c56bee54..8236a976 100644
--- a/skills/ppt-master/scripts/source_to_md/doc_to_md.py
+++ b/skills/ppt-master/scripts/source_to_md/doc_to_md.py
@@ -1,94 +1,417 @@
#!/usr/bin/env python3
"""
-Document to Markdown Converter (Pandoc-based)
+Document to Markdown Converter (hybrid Python + Pandoc fallback)
-Uses Pandoc to convert various document formats to Markdown.
-Primary use case: DOCX lecture notes / manuscripts → Markdown for PPT generation.
+Primary formats (pure Python, no external tools required):
+ .docx → mammoth
+ .html → markdownify + BeautifulSoup
+ .epub → ebooklib + markdownify
+ .ipynb → nbconvert
-Dependency: pandoc must be installed
- macOS: brew install pandoc
- Ubuntu: sudo apt install pandoc
- Windows: https://pandoc.org/installing.html
+Fallback formats (require pandoc installed):
+ .doc .odt .rtf .tex .latex .rst .org .typ
+
+All paths produce the same output convention:
+ <input>.md Markdown file
+ <input>_files/<asset> Extracted media (relative references in MD)
"""
import argparse
+import base64
+import mimetypes
import re
import shutil
import subprocess
import sys
from pathlib import Path
+from urllib.parse import unquote, urlparse
+
+# ─────────────────────────────────────────────────────────────
+# Format registry
+# ─────────────────────────────────────────────────────────────
+
+# Formats handled by pure-Python paths
+NATIVE_FORMATS = {".docx", ".html", ".htm", ".epub", ".ipynb"}
+
+# Formats handled by pandoc fallback: suffix → (pandoc input format, description)
+PANDOC_FORMATS = {
+ ".doc": ("doc", "Microsoft Word 97-2003"),
+ ".odt": ("odt", "OpenDocument Text"),
+ ".rtf": ("rtf", "Rich Text Format"),
+ ".tex": ("latex", "LaTeX"),
+ ".latex": ("latex", "LaTeX"),
+ ".rst": ("rst", "reStructuredText"),
+ ".org": ("org", "Emacs Org-mode"),
+ ".typ": ("typst", "Typst"),
+}
+# Formats pandoc should extract embedded media from
+PANDOC_MEDIA_FORMATS = {".odt"}
-# Supported input formats
-SUPPORTED_FORMATS = {
- # Office documents
- ".docx": ("docx", "Microsoft Word"),
- ".doc": ("doc", "Microsoft Word 97-2003"),
- ".odt": ("odt", "OpenDocument Text"),
- ".rtf": ("rtf", "Rich Text Format"),
- # eBooks
- ".epub": ("epub", "EPUB"),
- # Web
- ".html": ("html", "HTML"),
- ".htm": ("html", "HTML"),
- # Academic / technical
- ".tex": ("latex", "LaTeX"),
- ".latex": ("latex", "LaTeX"),
- ".rst": ("rst", "reStructuredText"),
- ".org": ("org", "Emacs Org-mode"),
- ".ipynb": ("ipynb", "Jupyter Notebook"),
- ".typ": ("typst", "Typst"),
-}
-# Formats that may contain embedded media
-MEDIA_FORMATS = {".docx", ".odt", ".epub", ".html", ".htm"}
+# ─────────────────────────────────────────────────────────────
+# Shared helpers
+# ─────────────────────────────────────────────────────────────
+def _format_size(size: int) -> str:
+ for unit in ("B", "KB", "MB"):
+ if size < 1024:
+ return f"{size:.0f} {unit}"
+ size /= 1024
+ return f"{size:.1f} GB"
-def check_pandoc() -> bool:
- """Check if pandoc is installed."""
- return shutil.which("pandoc") is not None
+def _ensure_media_dir(out_file: Path) -> tuple[Path, str]:
+ """Return (absolute media dir, relative dir name) and create the dir."""
+ rel_media_dir = f"{out_file.stem}_files"
+ media_dir = out_file.parent / rel_media_dir
+ media_dir.mkdir(parents=True, exist_ok=True)
+ return media_dir, rel_media_dir
-def convert_to_markdown(input_path: str, output_path: str | None = None) -> str:
- """
- Convert a document to Markdown using Pandoc.
- Args:
- input_path: Path to the input document.
- output_path: Path to write the output Markdown (default: same dir, .md extension).
+_HTML_IMG_PATTERNS = (
+ re.compile(
+ r'<img\s[^>]*?src="(?P<src>[^"]+)"[^>]*?(?:alt="(?P<alt>[^"]*)")?[^>]*/?\s*>'
+ ),
+ re.compile(
+ r'<img\s[^>]*?alt="(?P<alt>[^"]*)"[^>]*?src="(?P<src>[^"]+)"[^>]*/?\s*>'
+ ),
+)
- Returns:
- The generated Markdown content.
- """
- input_file = Path(input_path)
- if not input_file.exists():
- print(f"[ERROR] File not found: {input_path}")
+def _html_img_to_md(markdown_content: str) -> str:
+ """Convert any leftover <img> HTML tags to ![alt](src) syntax."""
+ def _repl(match: re.Match[str]) -> str:
+ src = match.group("src")
+ alt = match.group("alt") or Path(src).stem
+ return f"![{alt}]({src})"
+
+ for pattern in _HTML_IMG_PATTERNS:
+ markdown_content = pattern.sub(_repl, markdown_content)
+ return markdown_content
+
+
+def _report_result(out_file: Path, media_dir: Path | None) -> None:
+ size = out_file.stat().st_size
+ print(f"[OK] Saved Markdown to: {out_file} ({_format_size(size)})")
+ if media_dir and media_dir.exists():
+ files = [f for f in media_dir.rglob("*") if f.is_file()]
+ if files:
+ print(f" Extracted {len(files)} media file(s) → {media_dir}")
+
+
+# ─────────────────────────────────────────────────────────────
+# DOCX → Markdown (mammoth)
+# ─────────────────────────────────────────────────────────────
+
+def _convert_docx(input_file: Path, out_file: Path) -> str:
+ try:
+ import mammoth
+ except ImportError:
+ print("[ERROR] mammoth not installed. Run: pip install mammoth")
return ""
- suffix = input_file.suffix.lower()
- if suffix not in SUPPORTED_FORMATS:
- supported = ", ".join(sorted(SUPPORTED_FORMATS.keys()))
- print(f"[ERROR] Unsupported format: {suffix}")
- print(f" Supported: {supported}")
+ media_dir, rel_media_dir = _ensure_media_dir(out_file)
+ counter = {"n": 0}
+
+ def _save_image(image):
+ counter["n"] += 1
+ ext = mimetypes.guess_extension(image.content_type) or ".bin"
+ # Normalize common JPEG extension
+ if ext == ".jpe":
+ ext = ".jpg"
+ filename = f"image_{counter['n']:03d}{ext}"
+ with image.open() as stream:
+ (media_dir / filename).write_bytes(stream.read())
+ return {"src": f"{rel_media_dir}/{filename}"}
+
+ with input_file.open("rb") as f:
+ result = mammoth.convert_to_markdown(
+ f,
+ convert_image=mammoth.images.img_element(_save_image),
+ )
+
+ markdown = _html_img_to_md(result.value)
+ out_file.write_text(markdown, encoding="utf-8")
+
+ if not any(media_dir.iterdir()):
+ media_dir.rmdir()
+ media_dir = None # type: ignore[assignment]
+
+ for msg in result.messages:
+ if msg.type == "warning":
+ print(f" [warn] {msg.message}")
+
+ _report_result(out_file, media_dir)
+ return markdown
+
+
+# ─────────────────────────────────────────────────────────────
+# HTML → Markdown (markdownify + BeautifulSoup)
+# ─────────────────────────────────────────────────────────────
+
+def _save_data_uri(data_uri: str, media_dir: Path, index: int) -> str | None:
+ """Decode data:image/...;base64,... into a file; return filename or None."""
+ match = re.match(r"data:(?P<mime>[^;]+);base64,(?P<data>.+)", data_uri)
+ if not match:
+ return None
+ mime = match.group("mime")
+ ext = mimetypes.guess_extension(mime) or ".bin"
+ if ext == ".jpe":
+ ext = ".jpg"
+ filename = f"image_{index:03d}{ext}"
+ try:
+ (media_dir / filename).write_bytes(base64.b64decode(match.group("data")))
+ except Exception:
+ return None
+ return filename
+
+
+def _copy_local_image(src: str, base_dir: Path, media_dir: Path, index: int) -> str | None:
+ """Copy a local image (relative or file://) into media_dir."""
+ parsed = urlparse(src)
+ if parsed.scheme in ("http", "https"):
+ return None
+ path_str = unquote(parsed.path if parsed.scheme == "file" else src)
+ candidate = Path(path_str)
+ if not candidate.is_absolute():
+ candidate = (base_dir / candidate).resolve()
+ if not candidate.is_file():
+ return None
+ ext = candidate.suffix or ".bin"
+ filename = f"image_{index:03d}{ext}"
+ shutil.copy2(candidate, media_dir / filename)
+ return filename
+
+
+def _download_remote_image(url: str, media_dir: Path, index: int) -> str | None:
+ """Best-effort download of a remote image. Silent on failure."""
+ try:
+ import requests
+ except ImportError:
+ return None
+ try:
+ resp = requests.get(url, timeout=10, stream=True)
+ resp.raise_for_status()
+ except Exception:
+ return None
+ content_type = resp.headers.get("Content-Type", "").split(";")[0].strip()
+ ext = mimetypes.guess_extension(content_type) if content_type else None
+ if not ext:
+ ext = Path(urlparse(url).path).suffix or ".bin"
+ if ext == ".jpe":
+ ext = ".jpg"
+ filename = f"image_{index:03d}{ext}"
+ (media_dir / filename).write_bytes(resp.content)
+ return filename
+
+
+def _process_html_images(html: str, base_dir: Path, media_dir: Path, rel_media_dir: str) -> str:
+ """Extract & rewrite all <img> srcs in an HTML string."""
+ try:
+ from bs4 import BeautifulSoup
+ except ImportError:
+ print("[ERROR] beautifulsoup4 not installed. Run: pip install beautifulsoup4")
+ return html
+
+ soup = BeautifulSoup(html, "html.parser")
+ index = 0
+ for img in soup.find_all("img"):
+ src = img.get("src", "")
+ if not src:
+ continue
+ index += 1
+ if src.startswith("data:"):
+ filename = _save_data_uri(src, media_dir, index)
+ elif urlparse(src).scheme in ("http", "https"):
+ filename = _download_remote_image(src, media_dir, index)
+ else:
+ filename = _copy_local_image(src, base_dir, media_dir, index)
+ if filename:
+ img["src"] = f"{rel_media_dir}/{filename}"
+ return str(soup)
+
+
+def _convert_html(input_file: Path, out_file: Path) -> str:
+ try:
+ from markdownify import markdownify
+ except ImportError:
+ print("[ERROR] markdownify not installed. Run: pip install markdownify")
return ""
- input_format, format_desc = SUPPORTED_FORMATS[suffix]
- print(f"[INFO] Converting {format_desc} file: {input_file.name}")
+ try:
+ from bs4 import BeautifulSoup
+ except ImportError:
+ print("[ERROR] beautifulsoup4 not installed. Run: pip install beautifulsoup4")
+ return ""
+
+ media_dir, rel_media_dir = _ensure_media_dir(out_file)
+ raw_html = input_file.read_text(encoding="utf-8", errors="replace")
+
+ # Strip non-content elements (head/style/script) so metadata doesn't leak into MD
+ soup = BeautifulSoup(raw_html, "html.parser")
+ for tag in soup(["head", "style", "script", "noscript"]):
+ tag.decompose()
+ html = str(soup)
+ html = _process_html_images(html, input_file.parent, media_dir, rel_media_dir)
+
+ markdown = markdownify(html, heading_style="ATX", bullets="-")
+ # Collapse 3+ blank lines to 2 for tidier output
+ markdown = re.sub(r"\n{3,}", "\n\n", markdown).strip() + "\n"
+ out_file.write_text(markdown, encoding="utf-8")
+
+ if not any(media_dir.iterdir()):
+ media_dir.rmdir()
+ media_dir = None # type: ignore[assignment]
+
+ _report_result(out_file, media_dir)
+ return markdown
+
- # Determine output path
- if output_path:
- out_file = Path(output_path)
- else:
- out_file = input_file.with_suffix(".md")
+# ─────────────────────────────────────────────────────────────
+# EPUB → Markdown (ebooklib + markdownify)
+# ─────────────────────────────────────────────────────────────
- # Media extraction directory (use name only — pandoc writes relative paths)
+def _convert_epub(input_file: Path, out_file: Path) -> str:
+ try:
+ import ebooklib
+ from ebooklib import epub
+ from markdownify import markdownify
+ from bs4 import BeautifulSoup
+ except ImportError as e:
+ print(f"[ERROR] Missing dependency: {e.name}. "
+ f"Run: pip install ebooklib markdownify beautifulsoup4")
+ return ""
+
+ media_dir, rel_media_dir = _ensure_media_dir(out_file)
+ book = epub.read_epub(str(input_file))
+
+ # Extract images, remembering original path → new filename mapping
+ img_map: dict[str, str] = {}
+ index = 0
+ for item in book.get_items_of_type(ebooklib.ITEM_IMAGE):
+ index += 1
+ ext = Path(item.file_name).suffix or ".bin"
+ filename = f"image_{index:03d}{ext}"
+ (media_dir / filename).write_bytes(item.get_content())
+ # Map both full and basename for robust lookup
+ img_map[item.file_name] = filename
+ img_map[Path(item.file_name).name] = filename
+
+ # Iterate document items in spine order
+ html_parts: list[str] = []
+ spine_ids = [sid for sid, _ in book.spine]
+ id_to_item = {it.get_id(): it for it in book.get_items_of_type(ebooklib.ITEM_DOCUMENT)}
+ for sid in spine_ids:
+ item = id_to_item.get(sid)
+ if item is None:
+ continue
+ soup = BeautifulSoup(item.get_content(), "html.parser")
+ for img in soup.find_all("img"):
+ src = img.get("src", "")
+ if not src:
+ continue
+ # Try exact match, then basename, then normalized path
+ candidates = [src, Path(src).name, unquote(src), Path(unquote(src)).name]
+ resolved = next((img_map[c] for c in candidates if c in img_map), None)
+ if resolved:
+ img["src"] = f"{rel_media_dir}/{resolved}"
+ body = soup.find("body") or soup
+ html_parts.append(str(body))
+
+ combined_html = "\n\n".join(html_parts)
+ markdown = markdownify(combined_html, heading_style="ATX", bullets="-")
+ markdown = re.sub(r"\n{3,}", "\n\n", markdown).strip() + "\n"
+ out_file.write_text(markdown, encoding="utf-8")
+
+ if not any(media_dir.iterdir()):
+ media_dir.rmdir()
+ media_dir = None # type: ignore[assignment]
+
+ _report_result(out_file, media_dir)
+ return markdown
+
+
+# ─────────────────────────────────────────────────────────────
+# IPYNB → Markdown (nbconvert)
+# ─────────────────────────────────────────────────────────────
+
+def _convert_ipynb(input_file: Path, out_file: Path) -> str:
+ try:
+ import nbformat
+ from nbconvert import MarkdownExporter
+ from nbconvert.writers import FilesWriter
+ except ImportError:
+ print("[ERROR] nbconvert not installed. Run: pip install nbconvert")
+ return ""
+
+ # Pre-process cell-level markdown attachments: nbconvert leaves
+ # `attachment:<name>` references intact but doesn't write the files.
+ # Extract them into our outputs dict so FilesWriter picks them up.
+ nb = nbformat.read(str(input_file), as_version=4)
+ extra_outputs: dict[str, bytes] = {}
+ rel_media_dir = f"{out_file.stem}_files"
+
+ attach_counter = 0
+ for cell in nb.cells:
+ if cell.cell_type != "markdown":
+ continue
+ attachments = getattr(cell, "attachments", None) or {}
+ if not attachments:
+ continue
+ for att_name, mime_data in attachments.items():
+ for mime, b64 in mime_data.items():
+ attach_counter += 1
+ ext = mimetypes.guess_extension(mime) or ".bin"
+ if ext == ".jpe":
+ ext = ".jpg"
+ filename = f"attachment_{attach_counter:03d}{ext}"
+ out_path = f"{rel_media_dir}/{filename}"
+ try:
+ extra_outputs[out_path] = base64.b64decode(b64)
+ except Exception:
+ continue
+ # Rewrite source references: attachment:<name> → <rel_path>
+ src = cell.source if isinstance(cell.source, str) else "".join(cell.source)
+ src = src.replace(f"attachment:{att_name}", out_path)
+ cell.source = src
+
+ exporter = MarkdownExporter()
+ body, resources = exporter.from_notebook_node(nb)
+
+ # Merge attachment outputs with whatever nbconvert collected
+ resources.setdefault("outputs", {}).update(extra_outputs)
+ resources["output_extension"] = ".md"
+
+ writer = FilesWriter(build_directory=str(out_file.parent))
+ writer.write(body, resources, notebook_name=out_file.stem)
+
+ markdown = out_file.read_text(encoding="utf-8") if out_file.exists() else body
+ media_dir = out_file.parent / rel_media_dir
+ _report_result(out_file, media_dir if media_dir.exists() else None)
+ return markdown
+
+
+# ─────────────────────────────────────────────────────────────
+# Pandoc fallback
+# ─────────────────────────────────────────────────────────────
+
+def _check_pandoc() -> bool:
+ return shutil.which("pandoc") is not None
+
+
+def _convert_with_pandoc(input_file: Path, out_file: Path, suffix: str) -> str:
+ if not _check_pandoc():
+ print(f"[ERROR] Format '{suffix}' requires pandoc. Install it:")
+ print(" macOS: brew install pandoc")
+ print(" Ubuntu: sudo apt install pandoc")
+ print(" Windows: https://pandoc.org/installing.html")
+ return ""
+
+ input_format, _ = PANDOC_FORMATS[suffix]
rel_media_dir = f"{out_file.stem}_files"
media_dir = out_file.parent / rel_media_dir
- # Build pandoc command — use resolved absolute paths for input/output,
- # but a relative directory name for --extract-media so pandoc writes
- # relative image references in the markdown.
cmd = [
"pandoc",
"-f", input_format,
@@ -98,132 +421,115 @@ def convert_to_markdown(input_path: str, output_path: str | None = None) -> str:
"--wrap", "none",
"--strip-comments",
]
-
- # Extract embedded media for formats that support it
- if suffix in MEDIA_FORMATS:
+ if suffix in PANDOC_MEDIA_FORMATS:
cmd.extend(["--extract-media", rel_media_dir])
- try:
- result = subprocess.run(cmd, capture_output=True, text=True,
- cwd=str(out_file.parent))
- except FileNotFoundError:
- print("[ERROR] pandoc not found! Please install it first:")
- print(" macOS: brew install pandoc")
- print(" Ubuntu: sudo apt install pandoc")
- print(" Windows: https://pandoc.org/installing.html")
- return ""
-
+ result = subprocess.run(cmd, capture_output=True, text=True,
+ cwd=str(out_file.parent))
if result.returncode != 0:
print(f"[ERROR] Pandoc conversion failed:\n{result.stderr}")
return ""
-
- # Read generated markdown
if not out_file.exists():
print("[ERROR] Conversion completed but no output file was generated")
return ""
- markdown_content = out_file.read_text(encoding="utf-8")
+ markdown = out_file.read_text(encoding="utf-8")
- # --- Post-processing: fix media paths ---
-
- # 1. Flatten nested media/ subdirectory that pandoc creates
+ # Flatten nested media/ subdir that pandoc creates
nested_media = media_dir / "media"
if nested_media.exists():
for f in nested_media.iterdir():
if f.is_file():
- target = media_dir / f.name
- shutil.move(str(f), str(target))
+ shutil.move(str(f), str(media_dir / f.name))
try:
nested_media.rmdir()
except OSError:
pass
- # Fix references: {dir}/media/img.png → {dir}/img.png
- markdown_content = markdown_content.replace(
- f"{rel_media_dir}/media/", f"{rel_media_dir}/"
- )
+ markdown = markdown.replace(f"{rel_media_dir}/media/", f"{rel_media_dir}/")
- # 2. Convert any absolute paths to relative (safety net)
- abs_media_str = str(media_dir.resolve()).replace("\\", "/")
- if abs_media_str in markdown_content:
- markdown_content = markdown_content.replace(abs_media_str, rel_media_dir)
- # Also handle backslash variant on Windows
- abs_media_str_bs = str(media_dir.resolve())
- if abs_media_str_bs in markdown_content:
- markdown_content = markdown_content.replace(abs_media_str_bs, rel_media_dir)
-
- # 3. Convert <img> HTML tags to Markdown ![alt](src) syntax
- # Pandoc generates <img> when the source has styling (width/height)
- def _img_to_md(match: re.Match[str]) -> str:
- src = match.group("src")
- alt = match.group("alt") or Path(src).stem
- return f"![{alt}]({src})"
+ # Normalize absolute paths to relative
+ for abs_str in (str(media_dir.resolve()).replace("\\", "/"),
+ str(media_dir.resolve())):
+ if abs_str in markdown:
+ markdown = markdown.replace(abs_str, rel_media_dir)
- markdown_content = re.sub(
- r'<img\s[^>]*?src="(?P<src>[^"]+)"[^>]*?(?:alt="(?P<alt>[^"]*)")?[^>]*/?\s*>',
- _img_to_md,
- markdown_content,
- )
- # Handle alt before src order as well
- markdown_content = re.sub(
- r'<img\s[^>]*?alt="(?P<alt>[^"]*)"[^>]*?src="(?P<src>[^"]+)"[^>]*/?\s*>',
- _img_to_md,
- markdown_content,
- )
+ markdown = _html_img_to_md(markdown)
+ out_file.write_text(markdown, encoding="utf-8")
- out_file.write_text(markdown_content, encoding="utf-8")
+ _report_result(out_file, media_dir if media_dir.exists() else None)
+ return markdown
- # Report results
- size = out_file.stat().st_size
- print(f"[OK] Saved Markdown to: {out_file} ({_format_size(size)})")
- if media_dir.exists():
- media_files = [f for f in media_dir.rglob("*") if f.is_file()]
- if media_files:
- print(f" Extracted {len(media_files)} media file(s) → {media_dir}")
+# ─────────────────────────────────────────────────────────────
+# Dispatcher
+# ─────────────────────────────────────────────────────────────
- return markdown_content
+_FORMAT_DESC = {
+ ".docx": "Microsoft Word (mammoth)",
+ ".html": "HTML (markdownify)",
+ ".htm": "HTML (markdownify)",
+ ".epub": "EPUB (ebooklib)",
+ ".ipynb": "Jupyter Notebook (nbconvert)",
+}
-def _format_size(size: int) -> str:
- """Format file size for display."""
- for unit in ("B", "KB", "MB"):
- if size < 1024:
- return f"{size:.0f} {unit}"
- size /= 1024
- return f"{size:.1f} GB"
+def convert_to_markdown(input_path: str, output_path: str | None = None) -> str:
+ input_file = Path(input_path)
+ if not input_file.exists():
+ print(f"[ERROR] File not found: {input_path}")
+ return ""
+
+ suffix = input_file.suffix.lower()
+ if suffix not in NATIVE_FORMATS and suffix not in PANDOC_FORMATS:
+ supported = ", ".join(sorted(NATIVE_FORMATS | PANDOC_FORMATS.keys()))
+ print(f"[ERROR] Unsupported format: {suffix}")
+ print(f" Supported: {supported}")
+ return ""
+
+ out_file = Path(output_path) if output_path else input_file.with_suffix(".md")
+ out_file.parent.mkdir(parents=True, exist_ok=True)
+
+ if suffix in NATIVE_FORMATS:
+ desc = _FORMAT_DESC[suffix]
+ print(f"[INFO] Converting {desc}: {input_file.name}")
+ if suffix == ".docx":
+ return _convert_docx(input_file, out_file)
+ if suffix in (".html", ".htm"):
+ return _convert_html(input_file, out_file)
+ if suffix == ".epub":
+ return _convert_epub(input_file, out_file)
+ if suffix == ".ipynb":
+ return _convert_ipynb(input_file, out_file)
+
+ _, format_desc = PANDOC_FORMATS[suffix]
+ print(f"[INFO] Converting {format_desc} via pandoc: {input_file.name}")
+ return _convert_with_pandoc(input_file, out_file, suffix)
def main() -> None:
- """Run the CLI entry point."""
parser = argparse.ArgumentParser(
- description="Convert documents to Markdown using Pandoc",
+ description="Convert documents to Markdown "
+ "(pure-Python for common formats, pandoc fallback for the rest)",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
- python doc_to_md.py lecture.docx # Convert Word document
- python doc_to_md.py lecture.docx -o output.md # Specify output path
- python doc_to_md.py notes.odt # Convert OpenDocument
-
-Supported formats:
- Office: .docx .doc .odt .rtf
- eBooks: .epub
- Web: .html .htm
- Academic: .tex .rst .org .ipynb .typ
+ python doc_to_md.py lecture.docx # Word → Markdown (mammoth)
+ python doc_to_md.py article.html # HTML → Markdown (markdownify)
+ python doc_to_md.py book.epub # EPUB → Markdown (ebooklib)
+ python doc_to_md.py notebook.ipynb # Jupyter → Markdown (nbconvert)
+ python doc_to_md.py manuscript.tex # LaTeX → Markdown (pandoc fallback)
+
+Native formats (no pandoc required):
+ .docx .html/.htm .epub .ipynb
+
+Pandoc fallback formats (require system pandoc):
+ .doc .odt .rtf .tex/.latex .rst .org .typ
""",
)
-
parser.add_argument("input", help="Input document file")
parser.add_argument("-o", "--output", help="Output Markdown file path")
-
args = parser.parse_args()
- if not check_pandoc():
- print("[ERROR] pandoc not found! Please install it first:")
- print(" macOS: brew install pandoc")
- print(" Ubuntu: sudo apt install pandoc")
- print(" Windows: https://pandoc.org/installing.html")
- sys.exit(1)
-
result = convert_to_markdown(args.input, args.output)
sys.exit(0 if result else 1)