Python tool for converting files and office documents to Markdown.
feat: Add Azure Content Understanding converter (#1865)
Chien Yuan Changa01d74dda70d
diff --git a/README.md b/README.mdindex 71bcaa20..aa2f58bb 100644--- a/README.md+++ b/README.md@@ -107,6 +107,7 @@ At the moment, the following optional dependencies are available:* `[pdf]` Installs dependencies for PDF files* `[outlook]` Installs dependencies for Outlook messages* `[az-doc-intel]` Installs dependencies for Azure Document Intelligence+* `[az-content-understanding]` Installs dependencies for Azure Content Understanding* `[audio-transcription]` Installs dependencies for audio transcription of wav and mp3 files* `[youtube-transcription]` Installs dependencies for fetching YouTube video transcription@@ -158,6 +159,83 @@ If no `llm_client` is provided the plugin still loads, but OCR is silently skippSee [`packages/markitdown-ocr/README.md`](packages/markitdown-ocr/README.md) for detailed documentation.+### Azure Content Understanding++[Azure Content Understanding](https://learn.microsoft.com/azure/ai-services/content-understanding/) provides higher-quality conversion with structured field extraction (YAML front matter), multi-modal support (documents, images, audio, video), and configurable analyzers.++Install: `pip install 'markitdown[az-content-understanding]'`++#### When to use Content Understanding++Content Understanding is ideal when you need capabilities beyond what built-in or Document Intelligence converters provide:++- **Audio and video files** — CU is the only option for video, and the higher-quality cloud option for audio. Built-in converters have no video support and only basic audio transcription.+- **Structured field extraction** — [Prebuilt](https://learn.microsoft.com/azure/ai-services/content-understanding/concepts/prebuilt-analyzers) or [custom-built](https://learn.microsoft.com/azure/ai-services/content-understanding/how-to/customize-analyzer-content-understanding-studio?tabs=portal) analyzers extract domain-specific fields (invoice amounts, receipt dates, contract clauses) serialized as YAML front matter. Neither built-in nor Doc Intel integration exposes fields.+- **Higher-quality document extraction** — Cloud-based layout analysis and OCR for scanned PDFs, complex tables, and multi-page documents.+- **Single API for all modalities** — One `cu_endpoint` handles documents, images, audio, and video with automatic analyzer routing.++| Capability | Built-in converters | Azure Document Intelligence | Azure Content Understanding |+|------------|---------------------|-----------------------------|-----------------------------|+| Document conversion | Offline, format-specific extraction | Cloud layout extraction | Cloud multimodal extraction |+| Structured fields | Not available | Not exposed by this integration | YAML front matter from analyzer fields |+| Custom analyzers | Not available | Not configurable in this integration | Supported with `cu_analyzer_id` |+| Audio and video | Basic audio, no video | Not supported | Audio and video analyzers |+| Cost | Local compute only | Billable Azure API calls | Billable Azure API calls |++**CLI:**++```bash+markitdown path-to-file.pdf --use-cu --cu-endpoint "<content_understanding_endpoint>"+```++**Python API:**++```python+from markitdown import MarkItDown++# Zero-config — auto-selects analyzer per file type+md = MarkItDown(cu_endpoint="<content_understanding_endpoint>")+result = md.convert("report.pdf") # documents → prebuilt-documentSearch+result = md.convert("meeting.mp4") # video → prebuilt-videoSearch+result = md.convert("call.wav") # audio → prebuilt-audioSearch+print(result.markdown)+```++**With a custom analyzer** (for domain-specific field extraction):++```python+md = MarkItDown(+ cu_endpoint="<content_understanding_endpoint>",+ cu_analyzer_id="my-invoice-analyzer",+)+result = md.convert("invoice.pdf")+print(result.markdown)+# Output includes YAML front matter with extracted fields:+# ---+# contentType: document+# fields:+# VendorName: CONTOSO LTD.+# InvoiceDate: '2019-11-15'+# ---+# <!-- page 1 -->+# ...+```++When `cu_analyzer_id` is set, the converter automatically scopes it to compatible file types based on the analyzer's modality. Incompatible types (e.g., audio files with a document analyzer) auto-route to default prebuilt analyzers.++**Cost note:** Each `convert()` call for a CU-routed format is a billable Azure API call. Use `cu_file_types` to restrict which formats route to CU:++```python+from markitdown.converters import ContentUnderstandingFileType++md = MarkItDown(+ cu_endpoint="<content_understanding_endpoint>",+ cu_file_types=[ContentUnderstandingFileType.PDF], # only PDFs use CU+)+```++More information about Azure Content Understanding can be found [here](https://learn.microsoft.com/azure/ai-services/content-understanding/).+### Azure Document IntelligenceTo use Microsoft Document Intelligence for conversion:diff --git a/packages/markitdown/pyproject.toml b/packages/markitdown/pyproject.tomlindex ac3c8d94..d4c20a40 100644--- a/packages/markitdown/pyproject.toml+++ b/packages/markitdown/pyproject.toml@@ -47,6 +47,7 @@ all = ["SpeechRecognition","youtube-transcript-api~=1.0.0","azure-ai-documentintelligence",+ "azure-ai-contentunderstanding>=1.2.0b1","azure-identity",]pptx = ["python-pptx"]@@ -58,6 +59,8 @@ outlook = ["olefile"]audio-transcription = ["pydub", "SpeechRecognition"]youtube-transcription = ["youtube-transcript-api"]az-doc-intel = ["azure-ai-documentintelligence", "azure-identity"]+# >=1.2.0b1 required for to_llm_input() helper used by ContentUnderstandingConverter+az-content-understanding = ["azure-ai-contentunderstanding>=1.2.0b1", "azure-identity"][project.urls]Documentation = "https://github.com/microsoft/markitdown#readme"diff --git a/packages/markitdown/src/markitdown/__main__.py b/packages/markitdown/src/markitdown/__main__.pyindex 6085ad6b..ccb44b64 100644--- a/packages/markitdown/src/markitdown/__main__.py+++ b/packages/markitdown/src/markitdown/__main__.py@@ -4,6 +4,7 @@import argparseimport sysimport codecs+from typing import Any, Dictfrom textwrap import dedentfrom importlib.metadata import entry_pointsfrom .__about__ import __version__@@ -77,13 +78,22 @@ def main():help="Provide a hint about the file's charset (e.g, UTF-8).",)- parser.add_argument(+ cloud_group = parser.add_mutually_exclusive_group()+ cloud_group.add_argument("-d","--use-docintel",action="store_true",help="Use Document Intelligence to extract text instead of offline conversion. Requires a valid Document Intelligence Endpoint.",)+ cloud_group.add_argument(+ "--use-cu",+ "--use-content-understanding",+ action="store_true",+ dest="use_cu",+ help="Use Azure Content Understanding to extract text. Requires --cu-endpoint.",+ )+parser.add_argument("-e","--endpoint",@@ -91,6 +101,24 @@ def main():help="Document Intelligence Endpoint. Required if using Document Intelligence.",)+ parser.add_argument(+ "--cu-endpoint",+ type=str,+ help="Content Understanding Endpoint. Required if using --use-cu.",+ )++ parser.add_argument(+ "--cu-analyzer",+ type=str,+ help="Content Understanding analyzer ID. If not specified, auto-selects by file type.",+ )++ parser.add_argument(+ "--cu-file-types",+ type=str,+ help="Comma-separated list of file types to route to Content Understanding (e.g., pdf,jpeg,mp4). If omitted, all supported types are routed.",+ )+parser.add_argument("-p","--use-plugins",@@ -183,6 +211,36 @@ def main():markitdown = MarkItDown(enable_plugins=args.use_plugins, docintel_endpoint=args.endpoint)+ elif args.use_cu:+ if args.cu_endpoint is None:+ _exit_with_error(+ "Content Understanding Endpoint (--cu-endpoint) is required when using --use-cu."+ )+ elif args.filename is None:+ _exit_with_error("Filename is required when using Content Understanding.")++ cu_kwargs: Dict[str, Any] = {+ "cu_endpoint": args.cu_endpoint,+ }+ if args.cu_analyzer is not None:+ cu_kwargs["cu_analyzer_id"] = args.cu_analyzer+ if args.cu_file_types is not None:+ # Parse comma-separated file types into ContentUnderstandingFileType list+ from .converters import ContentUnderstandingFileType++ type_names = [+ t.strip().lower() for t in args.cu_file_types.split(",") if t.strip()+ ]+ cu_types = []+ for name in type_names:+ # Try matching by value (e.g., "pdf", "jpeg", "mp4")+ try:+ cu_types.append(ContentUnderstandingFileType(name))+ except ValueError:+ _exit_with_error(f"Unknown file type: {name}")+ cu_kwargs["cu_file_types"] = cu_types++ markitdown = MarkItDown(enable_plugins=args.use_plugins, **cu_kwargs)else:markitdown = MarkItDown(enable_plugins=args.use_plugins)diff --git a/packages/markitdown/src/markitdown/_markitdown.py b/packages/markitdown/src/markitdown/_markitdown.pyindex f342a614..f6aa4df0 100644--- a/packages/markitdown/src/markitdown/_markitdown.py+++ b/packages/markitdown/src/markitdown/_markitdown.py@@ -38,6 +38,7 @@ from .converters import (ZipConverter,EpubConverter,DocumentIntelligenceConverter,+ ContentUnderstandingConverter,CsvConverter,)@@ -225,6 +226,28 @@ class MarkItDown:DocumentIntelligenceConverter(**docintel_args),)+ # Register Content Understanding converter at the top of the stack if endpoint is provided+ cu_endpoint = kwargs.get("cu_endpoint")+ if cu_endpoint is not None:+ cu_args: Dict[str, Any] = {}+ cu_args["endpoint"] = cu_endpoint++ cu_credential = kwargs.get("cu_credential")+ if cu_credential is not None:+ cu_args["credential"] = cu_credential++ cu_analyzer_id = kwargs.get("cu_analyzer_id")+ if cu_analyzer_id is not None:+ cu_args["analyzer_id"] = cu_analyzer_id++ cu_file_types = kwargs.get("cu_file_types")+ if cu_file_types is not None:+ cu_args["file_types"] = cu_file_types++ self.register_converter(+ ContentUnderstandingConverter(**cu_args),+ )+self._builtins_enabled = Trueelse:warn("Built-in converters are already enabled.", RuntimeWarning)diff --git a/packages/markitdown/src/markitdown/converters/__init__.py b/packages/markitdown/src/markitdown/converters/__init__.pyindex e4437a58..77f8b1ac 100644--- a/packages/markitdown/src/markitdown/converters/__init__.py+++ b/packages/markitdown/src/markitdown/converters/__init__.py@@ -21,6 +21,10 @@ from ._doc_intel_converter import (DocumentIntelligenceConverter,DocumentIntelligenceFileType,)+from ._cu_converter import (+ ContentUnderstandingConverter,+ ContentUnderstandingFileType,+)from ._epub_converter import EpubConverterfrom ._csv_converter import CsvConverter@@ -43,6 +47,8 @@ __all__ = ["ZipConverter","DocumentIntelligenceConverter","DocumentIntelligenceFileType",+ "ContentUnderstandingConverter",+ "ContentUnderstandingFileType","EpubConverter","CsvConverter",]diff --git a/packages/markitdown/src/markitdown/converters/_cu_converter.py b/packages/markitdown/src/markitdown/converters/_cu_converter.pynew file mode 100644index 00000000..e4080dda--- /dev/null+++ b/packages/markitdown/src/markitdown/converters/_cu_converter.py@@ -0,0 +1,570 @@+"""Azure Content Understanding converter for MarkItDown.++Converts files using Azure Content Understanding (CU) for high-quality,+multi-modal extraction with structured field output. Supports documents,+images, audio, and video. Fields are serialized as YAML front matter via+the CU SDK's ``to_llm_input()`` helper.++Install dependencies: ``pip install markitdown[az-content-understanding]``+"""++import sys+import os+from typing import BinaryIO, Any, List, Optional, Dict+from enum import Enum++from .._base_converter import DocumentConverter, DocumentConverterResult+from .._stream_info import StreamInfo+from .._exceptions import MissingDependencyException++# Try loading optional dependencies — save error for later+_dependency_exc_info = None+try:+ from azure.ai.contentunderstanding import ContentUnderstandingClient, to_llm_input+ from azure.core.credentials import AzureKeyCredential, TokenCredential+ from azure.core.pipeline.policies import UserAgentPolicy+ from azure.identity import DefaultAzureCredential+except ImportError:+ _dependency_exc_info = sys.exc_info()++ # Stub classes for type hinting+ class AzureKeyCredential: # type: ignore[no-redef]+ pass++ class TokenCredential: # type: ignore[no-redef]+ pass++ class ContentUnderstandingClient: # type: ignore[no-redef]+ pass++ class UserAgentPolicy: # type: ignore[no-redef]+ pass++ class DefaultAzureCredential: # type: ignore[no-redef]+ pass++ def to_llm_input(*args, **kwargs): # type: ignore[no-redef]+ pass+++# ---------------------------------------------------------------------------+# File type enum and routing tables+# ---------------------------------------------------------------------------+++class ContentUnderstandingFileType(str, Enum):+ """Supported file types for Content Understanding conversion."""++ # Documents+ PDF = "pdf"+ DOCX = "docx"+ PPTX = "pptx"+ XLSX = "xlsx"+ HTML = "html"+ TXT = "txt"+ MD = "md"+ RTF = "rtf"+ XML = "xml"++ EML = "eml"+ MSG = "msg"++ # Images (document modality)+ JPEG = "jpeg"+ PNG = "png"+ BMP = "bmp"+ TIFF = "tiff"+ HEIF = "heif"++ # Video+ MP4 = "mp4"+ M4V = "m4v"+ MOV = "mov"+ AVI = "avi"+ MKV = "mkv"+ WEBM = "webm"+ FLV = "flv"+ WMV = "wmv"++ # Audio+ WAV = "wav"+ MP3 = "mp3"+ M4A = "m4a"+ FLAC = "flac"+ OGG = "ogg"+ AAC = "aac"+ WMA = "wma"+++# Extension → file type+_EXTENSION_MAP: Dict[str, ContentUnderstandingFileType] = {+ # Documents+ ".pdf": ContentUnderstandingFileType.PDF,+ ".docx": ContentUnderstandingFileType.DOCX,+ ".pptx": ContentUnderstandingFileType.PPTX,+ ".xlsx": ContentUnderstandingFileType.XLSX,+ ".html": ContentUnderstandingFileType.HTML,+ ".txt": ContentUnderstandingFileType.TXT,+ ".md": ContentUnderstandingFileType.MD,+ ".rtf": ContentUnderstandingFileType.RTF,+ ".xml": ContentUnderstandingFileType.XML,+ ".eml": ContentUnderstandingFileType.EML,+ ".msg": ContentUnderstandingFileType.MSG,+ # Images+ ".jpg": ContentUnderstandingFileType.JPEG,+ ".jpeg": ContentUnderstandingFileType.JPEG,+ ".jpe": ContentUnderstandingFileType.JPEG,+ ".png": ContentUnderstandingFileType.PNG,+ ".bmp": ContentUnderstandingFileType.BMP,+ ".tiff": ContentUnderstandingFileType.TIFF,+ ".heif": ContentUnderstandingFileType.HEIF,+ ".heic": ContentUnderstandingFileType.HEIF,+ # Video+ ".mp4": ContentUnderstandingFileType.MP4,+ ".m4v": ContentUnderstandingFileType.M4V,+ ".mov": ContentUnderstandingFileType.MOV,+ ".avi": ContentUnderstandingFileType.AVI,+ ".mkv": ContentUnderstandingFileType.MKV,+ ".webm": ContentUnderstandingFileType.WEBM,+ ".flv": ContentUnderstandingFileType.FLV,+ ".wmv": ContentUnderstandingFileType.WMV,+ # Audio+ ".wav": ContentUnderstandingFileType.WAV,+ ".mp3": ContentUnderstandingFileType.MP3,+ ".m4a": ContentUnderstandingFileType.M4A,+ ".flac": ContentUnderstandingFileType.FLAC,+ ".ogg": ContentUnderstandingFileType.OGG,+ ".aac": ContentUnderstandingFileType.AAC,+ ".wma": ContentUnderstandingFileType.WMA,+}++# MIME type prefixes for each file type+_MIME_PREFIXES: Dict[ContentUnderstandingFileType, List[str]] = {+ # Documents+ ContentUnderstandingFileType.PDF: ["application/pdf", "application/x-pdf"],+ ContentUnderstandingFileType.DOCX: [+ "application/vnd.openxmlformats-officedocument.wordprocessingml.document"+ ],+ ContentUnderstandingFileType.PPTX: [+ "application/vnd.openxmlformats-officedocument.presentationml"+ ],+ ContentUnderstandingFileType.XLSX: [+ "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"+ ],+ ContentUnderstandingFileType.HTML: ["text/html", "application/xhtml+xml"],+ ContentUnderstandingFileType.TXT: ["text/plain"],+ ContentUnderstandingFileType.MD: ["text/markdown"],+ ContentUnderstandingFileType.RTF: ["text/rtf", "application/rtf"],+ ContentUnderstandingFileType.XML: ["text/xml", "application/xml"],+ ContentUnderstandingFileType.EML: ["message/rfc822"],+ ContentUnderstandingFileType.MSG: ["application/vnd.ms-outlook"],+ # Images+ ContentUnderstandingFileType.JPEG: ["image/jpeg"],+ ContentUnderstandingFileType.PNG: ["image/png"],+ ContentUnderstandingFileType.BMP: ["image/bmp"],+ ContentUnderstandingFileType.TIFF: ["image/tiff"],+ ContentUnderstandingFileType.HEIF: ["image/heif", "image/heic"],+ # Video+ ContentUnderstandingFileType.MP4: ["video/mp4"],+ ContentUnderstandingFileType.M4V: ["video/x-m4v"],+ ContentUnderstandingFileType.MOV: ["video/quicktime"],+ ContentUnderstandingFileType.AVI: ["video/x-msvideo"],+ ContentUnderstandingFileType.MKV: ["video/x-matroska"],+ ContentUnderstandingFileType.WEBM: ["video/webm"],+ ContentUnderstandingFileType.FLV: ["video/x-flv"],+ ContentUnderstandingFileType.WMV: ["video/x-ms-wmv"],+ # Audio+ ContentUnderstandingFileType.WAV: ["audio/wav", "audio/x-wav"],+ ContentUnderstandingFileType.MP3: ["audio/mpeg", "audio/mp3"],+ ContentUnderstandingFileType.M4A: ["audio/mp4", "audio/m4a", "audio/x-m4a"],+ ContentUnderstandingFileType.FLAC: ["audio/flac", "audio/x-flac"],+ ContentUnderstandingFileType.OGG: ["audio/ogg"],+ ContentUnderstandingFileType.AAC: ["audio/aac"],+ ContentUnderstandingFileType.WMA: ["audio/x-ms-wma"],+}++_MIME_ALIASES: Dict[str, str] = {+ "audio/x-wav": "audio/wav",+ "audio/x-flac": "audio/flac",+ "audio/x-m4a": "audio/mp4",+ "video/x-m4v": "video/mp4",+}++# File type → modality category+_DOCUMENT_TYPES = {+ ContentUnderstandingFileType.PDF,+ ContentUnderstandingFileType.DOCX,+ ContentUnderstandingFileType.PPTX,+ ContentUnderstandingFileType.XLSX,+ ContentUnderstandingFileType.HTML,+ ContentUnderstandingFileType.TXT,+ ContentUnderstandingFileType.MD,+ ContentUnderstandingFileType.RTF,+ ContentUnderstandingFileType.XML,+ ContentUnderstandingFileType.EML,+ ContentUnderstandingFileType.MSG,+}++_IMAGE_TYPES = {+ ContentUnderstandingFileType.JPEG,+ ContentUnderstandingFileType.PNG,+ ContentUnderstandingFileType.BMP,+ ContentUnderstandingFileType.TIFF,+ ContentUnderstandingFileType.HEIF,+}++_VIDEO_TYPES = {+ ContentUnderstandingFileType.MP4,+ ContentUnderstandingFileType.M4V,+ ContentUnderstandingFileType.MOV,+ ContentUnderstandingFileType.AVI,+ ContentUnderstandingFileType.MKV,+ ContentUnderstandingFileType.WEBM,+ ContentUnderstandingFileType.FLV,+ ContentUnderstandingFileType.WMV,+}++_AUDIO_TYPES = {+ ContentUnderstandingFileType.WAV,+ ContentUnderstandingFileType.MP3,+ ContentUnderstandingFileType.M4A,+ ContentUnderstandingFileType.FLAC,+ ContentUnderstandingFileType.OGG,+ ContentUnderstandingFileType.AAC,+ ContentUnderstandingFileType.WMA,+}++_PREBUILT_ANALYZERS = {+ "document": "prebuilt-documentSearch",+ "image": "prebuilt-documentSearch",+ "video": "prebuilt-videoSearch",+ "audio": "prebuilt-audioSearch",+}++# All supported file types (default set when file_types is None)+_ALL_FILE_TYPES = list(ContentUnderstandingFileType)+++def _get_modality(file_type: ContentUnderstandingFileType) -> str:+ """Get the modality category for a file type."""+ if file_type in _DOCUMENT_TYPES:+ return "document"+ elif file_type in _IMAGE_TYPES:+ return "image"+ elif file_type in _VIDEO_TYPES:+ return "video"+ elif file_type in _AUDIO_TYPES:+ return "audio"+ raise ValueError(f"Unknown file type: {file_type}")+++def _detect_file_type(+ stream_info: StreamInfo,+ file_types: Optional[List[ContentUnderstandingFileType]] = None,+) -> Optional[ContentUnderstandingFileType]:+ """Detect a supported CU file type from extension or MIME type."""+ allowed = set(file_types) if file_types is not None else None++ extension = (stream_info.extension or "").lower()+ file_type = _EXTENSION_MAP.get(extension)+ if file_type is not None and (allowed is None or file_type in allowed):+ return file_type++ mimetype = _clean_mime_type(stream_info.mimetype)+ if not mimetype:+ return None++ return _detect_file_type_from_mime(mimetype, allowed)+++def _clean_mime_type(mimetype: Optional[str]) -> str:+ return (mimetype or "").split(";", 1)[0].strip().lower()+++def _canonical_mime_type(mimetype: Optional[str]) -> str:+ cleaned = _clean_mime_type(mimetype)+ return _MIME_ALIASES.get(cleaned, cleaned) or "application/octet-stream"+++def _content_type_for(+ file_type: ContentUnderstandingFileType,+ mimetype: Optional[str],+) -> str:+ """Resolve the content type to send to the CU API.++ Uses the resolved ``file_type`` as the source of truth so analyzer+ routing and payload metadata stay consistent. The caller-provided+ ``mimetype`` is only used when it is consistent with ``file_type``+ (e.g., to preserve subtype distinctions like ``image/heic`` vs+ ``image/heif``). When ``mimetype`` disagrees with the resolved+ ``file_type`` (e.g., ``.pdf`` extension with ``audio/mpeg``+ mimetype), the canonical MIME type for ``file_type`` is used.+ """+ prefixes = _MIME_PREFIXES.get(file_type, [])+ canonical = _canonical_mime_type(mimetype)++ # Use caller-provided MIME if it's consistent with the resolved file_type+ if prefixes and canonical != "application/octet-stream":+ for prefix in prefixes:+ if canonical.startswith(prefix):+ return canonical++ # Fallback: derive from the resolved file_type (single source of truth)+ if prefixes:+ return _canonical_mime_type(prefixes[0])++ return canonical+++def _detect_file_type_from_mime(+ mimetype: str,+ allowed: Optional[set[ContentUnderstandingFileType]],+) -> Optional[ContentUnderstandingFileType]:+ for candidate, prefixes in _MIME_PREFIXES.items():+ if allowed is not None and candidate not in allowed:+ continue+ for prefix in prefixes:+ if mimetype.startswith(prefix):+ return candidate+ return None+++# ---------------------------------------------------------------------------+# Smart routing: base_analyzer_id → modality mapping+# ---------------------------------------------------------------------------++_BASE_TO_MODALITY: Dict[str, str] = {+ "prebuilt-document": "document",+ "prebuilt-image": "image",+ "prebuilt-audio": "audio",+ "prebuilt-video": "video",+}++# Cache of known prebuilt analyzer name → modality (avoids API call)+_KNOWN_PREBUILT_MODALITY: Dict[str, str] = {+ # Document-based prebuilts+ "prebuilt-documentSearch": "document",+ "prebuilt-layout": "document",+ "prebuilt-read": "document",+ "prebuilt-document": "document",+ "prebuilt-invoice": "document",+ "prebuilt-receipt": "document",+ "prebuilt-receipt.generic": "document",+ "prebuilt-receipt.hotel": "document",+ "prebuilt-idDocument": "document",+ "prebuilt-idDocument.generic": "document",+ "prebuilt-idDocument.passport": "document",+ "prebuilt-healthInsuranceCard.us": "document",+ "prebuilt-contract": "document",+ "prebuilt-creditCard": "document",+ "prebuilt-creditMemo": "document",+ "prebuilt-bankStatement.us": "document",+ "prebuilt-check.us": "document",+ "prebuilt-purchaseOrder": "document",+ "prebuilt-procurement": "document",+ "prebuilt-payStub.us": "document",+ "prebuilt-utilityBill": "document",+ "prebuilt-marriageCertificate.us": "document",+ "prebuilt-documentFieldSchema": "document",+ "prebuilt-documentFields": "document",+ # Tax prebuilts (all document-based)+ "prebuilt-tax.us": "document",+ "prebuilt-tax.us.w2": "document",+ "prebuilt-tax.us.w4": "document",+ "prebuilt-tax.us.1040": "document",+ # Mortgage prebuilts+ "prebuilt-mortgage.us": "document",+ "prebuilt-mortgage.us.1003": "document",+ "prebuilt-mortgage.us.closingDisclosure": "document",+ # Image-based prebuilts+ "prebuilt-image": "image",+ "prebuilt-imageSearch": "image",+ # Audio-based prebuilts+ "prebuilt-audio": "audio",+ "prebuilt-audioSearch": "audio",+ "prebuilt-callCenter": "audio",+ # Video-based prebuilts+ "prebuilt-video": "video",+ "prebuilt-videoSearch": "video",+ "prebuilt-videoSynopsis": "video",+}+++def _resolve_analyzer_modality(client: Any, analyzer_id: str) -> str:+ """Resolve analyzer modality from cache or via get_analyzer() fallback.++ For known prebuilt-* names, returns the modality from+ ``_KNOWN_PREBUILT_MODALITY`` without an API call. For unknown+ prebuilt-* names or custom analyzers, calls ``get_analyzer()``+ to inspect ``base_analyzer_id``.++ Args:+ client: A ``ContentUnderstandingClient`` instance.+ analyzer_id: The analyzer ID to resolve.++ Returns:+ Modality string ("document", "image", "audio", or "video").++ Raises:+ ValueError: If ``get_analyzer()`` fails.+ """+ # Known prebuilt — use cache, no API call+ if analyzer_id in _KNOWN_PREBUILT_MODALITY:+ return _KNOWN_PREBUILT_MODALITY[analyzer_id]++ # Unknown prebuilt or custom analyzer — call get_analyzer()+ try:+ analyzer_info = client.get_analyzer(analyzer_id)+ except Exception as exc:+ raise ValueError(f"Failed to resolve analyzer '{analyzer_id}': {exc}") from exc++ if analyzer_info.base_analyzer_id:+ return _BASE_TO_MODALITY.get(analyzer_info.base_analyzer_id, "document")+ return "document"+++def _is_analyzer_compatible(file_modality: str, analyzer_modality: str) -> bool:+ """Return True when an analyzer modality can process a file modality."""+ if analyzer_modality == "document":+ return file_modality in {"document", "image"}+ return file_modality == analyzer_modality+++# ---------------------------------------------------------------------------+# Converter+# ---------------------------------------------------------------------------+++class ContentUnderstandingConverter(DocumentConverter):+ """Converts files using Azure Content Understanding.++ Provides high-quality document, image, audio, and video conversion+ with structured field extraction via YAML front matter.+ """++ def __init__(+ self,+ *,+ endpoint: str,+ credential: AzureKeyCredential | TokenCredential | None = None,+ analyzer_id: Optional[str] = None,+ file_types: Optional[List[ContentUnderstandingFileType]] = None,+ ):+ """Initialize the Content Understanding converter.++ Args:+ endpoint: CU resource endpoint URL.+ credential: Explicit credential. If None, falls back to+ AZURE_API_KEY env var, then DefaultAzureCredential.+ analyzer_id: Custom analyzer for compatible file types.+ When set, the converter checks the analyzer's base modality+ (via get_analyzer() at init) and routes only compatible+ file types to it. Incompatible modalities auto-route to+ default prebuilts. If None, auto-selects by extension/MIME.+ file_types: Which file types to handle. If None, uses the+ default set (all supported formats).+ """+ super().__init__()++ # Raise if dependencies are missing+ if _dependency_exc_info is not None:+ raise MissingDependencyException(+ "ContentUnderstandingConverter requires the optional dependency "+ "[az-content-understanding] (or [all]) to be installed. "+ "E.g., `pip install markitdown[az-content-understanding]`"+ ) from _dependency_exc_info[+ 1+ ].with_traceback( # type: ignore[union-attr]+ _dependency_exc_info[2]+ )++ self._file_types = file_types if file_types is not None else _ALL_FILE_TYPES+ self._analyzer_id = analyzer_id+ self._analyzer_modality: Optional[str] = None++ # Resolve credential+ if credential is None:+ api_key = os.environ.get("AZURE_API_KEY")+ if api_key is not None:+ credential = AzureKeyCredential(api_key)+ else:+ credential = DefaultAzureCredential()++ # User agent for telemetry+ try:+ from ..__about__ import __version__+ except ImportError:+ __version__ = "unknown"+ user_agent = f"markitdown-cu/{__version__}"++ # Create CU client+ self._client = ContentUnderstandingClient(+ endpoint=endpoint,+ credential=credential,+ user_agent_policy=UserAgentPolicy(user_agent=user_agent),+ )++ # Smart routing: resolve analyzer modality at init (at most one API call)+ if self._analyzer_id is not None:+ self._analyzer_modality = _resolve_analyzer_modality(+ self._client, self._analyzer_id+ )++ def accepts(+ self,+ file_stream: BinaryIO,+ stream_info: StreamInfo,+ **kwargs: Any,+ ) -> bool:+ """Return True if the file type is in the configured set."""+ return _detect_file_type(stream_info, self._file_types) is not None++ def convert(+ self,+ file_stream: BinaryIO,+ stream_info: StreamInfo,+ **kwargs: Any,+ ) -> DocumentConverterResult:+ """Convert the file using CU and return Markdown with YAML front matter."""++ # 1. Determine analyzer_id (smart routing: check modality)+ file_type = _detect_file_type(stream_info, self._file_types)+ if file_type is None:+ raise ValueError(+ "Unsupported file type for Content Understanding conversion."+ )+ file_modality = _get_modality(file_type)++ if (+ self._analyzer_id is not None+ and self._analyzer_modality is not None+ and _is_analyzer_compatible(file_modality, self._analyzer_modality)+ ):+ analyzer_id = self._analyzer_id+ else:+ analyzer_id = _PREBUILT_ANALYZERS.get(+ file_modality, "prebuilt-documentSearch"+ )++ # 2. Read file bytes and determine MIME type+ file_bytes = file_stream.read()+ content_type = _content_type_for(file_type, stream_info.mimetype)++ # 3. Call CU SDK+ poller = self._client.begin_analyze_binary(+ analyzer_id=analyzer_id,+ binary_input=file_bytes,+ content_type=content_type,+ )++ # 4. Block on result+ result = poller.result()++ # 5. Format output using to_llm_input()+ text = to_llm_input(result)++ # 6. Return+ return DocumentConverterResult(markdown=text)diff --git a/packages/markitdown/tests/test_cu_converter.py b/packages/markitdown/tests/test_cu_converter.pynew file mode 100644index 00000000..760f5fcc--- /dev/null+++ b/packages/markitdown/tests/test_cu_converter.py@@ -0,0 +1,928 @@+"""Tests for ContentUnderstandingConverter.++Tests accepts() routing, smart routing modality logic, and convert() via mocks.+Follows the same pattern as test_docintel_html.py.+"""++import io+import sys+from unittest.mock import MagicMock, patch++import pytest++from markitdown.converters._cu_converter import (+ ContentUnderstandingConverter,+ ContentUnderstandingFileType,+ _resolve_analyzer_modality,+ _get_modality,+ _detect_file_type,+ _canonical_mime_type,+ _content_type_for,+ _EXTENSION_MAP,+)+from markitdown._stream_info import StreamInfo++# ---------------------------------------------------------------------------+# Helper: create a converter with accepts() working but no SDK init+# ---------------------------------------------------------------------------+++def _make_converter(file_types=None, analyzer_id=None, analyzer_modality=None):+ """Create a converter bypassing __init__ (no SDK deps needed)."""+ conv = ContentUnderstandingConverter.__new__(ContentUnderstandingConverter)+ conv._analyzer_id = analyzer_id+ conv._analyzer_modality = analyzer_modality++ # Set accepted file types without running SDK-dependent initialization.+ from markitdown.converters._cu_converter import (+ _ALL_FILE_TYPES,+ )++ types = file_types if file_types is not None else _ALL_FILE_TYPES+ conv._file_types = types++ return conv+++# ---------------------------------------------------------------------------+# accepts() tests — extension-based+# ---------------------------------------------------------------------------+++class TestAcceptsExtension:+ """Test accepts() for supported and unsupported file extensions."""++ @pytest.mark.parametrize(+ "ext",+ [+ ".pdf",+ ".docx",+ ".pptx",+ ".xlsx",+ ".html",+ ".txt",+ ".md",+ ".rtf",+ ".xml",+ ".eml",+ ".msg",+ ".jpg",+ ".jpeg",+ ".jpe",+ ".png",+ ".bmp",+ ".tiff",+ ".heif",+ ".heic",+ ".mp4",+ ".m4v",+ ".mov",+ ".avi",+ ".mkv",+ ".webm",+ ".flv",+ ".wmv",+ ".wav",+ ".mp3",+ ".m4a",+ ".flac",+ ".ogg",+ ".aac",+ ".wma",+ ],+ )+ def test_accepts_supported_extensions(self, ext):+ conv = _make_converter()+ assert conv.accepts(io.BytesIO(b""), StreamInfo(extension=ext))++ @pytest.mark.parametrize(+ "ext",+ [+ ".csv",+ ".json",+ ".zip",+ ".epub",+ ".py",+ ".rs",+ ],+ )+ def test_rejects_unsupported_extensions(self, ext):+ conv = _make_converter()+ assert not conv.accepts(io.BytesIO(b""), StreamInfo(extension=ext))+++# ---------------------------------------------------------------------------+# accepts() tests — MIME-based+# ---------------------------------------------------------------------------+++class TestAcceptsMime:+ """Test accepts() for MIME type matching."""++ @pytest.mark.parametrize(+ "mime",+ [+ "application/pdf",+ "image/jpeg",+ "video/mp4",+ "audio/wav",+ "audio/x-wav",+ "text/html",+ "audio/mpeg",+ "audio/x-m4a",+ "audio/x-flac",+ "video/quicktime",+ "video/webm",+ "video/x-m4v",+ "video/x-flv",+ "video/x-ms-wmv",+ "audio/aac",+ "audio/x-ms-wma",+ ],+ )+ def test_accepts_supported_mimetypes(self, mime):+ conv = _make_converter()+ assert conv.accepts(io.BytesIO(b""), StreamInfo(mimetype=mime))++ @pytest.mark.parametrize(+ "mime",+ [+ "text/csv",+ "application/json",+ "application/zip",+ ],+ )+ def test_rejects_unsupported_mimetypes(self, mime):+ conv = _make_converter()+ assert not conv.accepts(io.BytesIO(b""), StreamInfo(mimetype=mime))+++# ---------------------------------------------------------------------------+# accepts() tests — cu_file_types restriction+# ---------------------------------------------------------------------------+++class TestAcceptsFileTypeRestriction:+ """Test that cu_file_types restricts which formats are accepted."""++ def test_restricted_to_pdf_only(self):+ conv = _make_converter(file_types=[ContentUnderstandingFileType.PDF])+ assert conv.accepts(io.BytesIO(b""), StreamInfo(extension=".pdf"))+ assert not conv.accepts(io.BytesIO(b""), StreamInfo(extension=".mp4"))+ assert not conv.accepts(io.BytesIO(b""), StreamInfo(extension=".wav"))+ assert not conv.accepts(io.BytesIO(b""), StreamInfo(extension=".jpg"))++ def test_restricted_to_audio(self):+ conv = _make_converter(+ file_types=[+ ContentUnderstandingFileType.WAV,+ ContentUnderstandingFileType.MP3,+ ]+ )+ assert conv.accepts(io.BytesIO(b""), StreamInfo(extension=".wav"))+ assert conv.accepts(io.BytesIO(b""), StreamInfo(extension=".mp3"))+ assert not conv.accepts(io.BytesIO(b""), StreamInfo(extension=".pdf"))++ def test_webm_value_matches_cli_input(self):+ assert ContentUnderstandingFileType("webm") == ContentUnderstandingFileType.WEBM++ def test_m4v_value_matches_cli_input(self):+ assert ContentUnderstandingFileType("m4v") == ContentUnderstandingFileType.M4V+++# ---------------------------------------------------------------------------+# file type detection tests+# ---------------------------------------------------------------------------+++class TestDetectFileType:+ """Test extension and MIME based file type detection."""++ def test_detects_video_from_mime_without_extension(self):+ assert (+ _detect_file_type(StreamInfo(mimetype="video/mp4"))+ == ContentUnderstandingFileType.MP4+ )++ def test_detects_audio_from_mime_without_extension(self):+ assert (+ _detect_file_type(StreamInfo(mimetype="audio/mpeg"))+ == ContentUnderstandingFileType.MP3+ )++ def test_detects_audio_alias_from_mime_without_extension(self):+ assert (+ _detect_file_type(StreamInfo(mimetype="audio/x-wav"))+ == ContentUnderstandingFileType.WAV+ )++ def test_detects_video_alias_from_mime_without_extension(self):+ assert (+ _detect_file_type(StreamInfo(mimetype="video/x-m4v"))+ == ContentUnderstandingFileType.M4V+ )++ @pytest.mark.parametrize(+ ("mimetype", "expected"),+ [+ ("audio/x-wav", "audio/wav"),+ ("audio/x-flac", "audio/flac"),+ ("audio/x-m4a", "audio/mp4"),+ ("video/x-m4v", "video/mp4"),+ ("video/mp4", "video/mp4"),+ (None, "application/octet-stream"),+ ],+ )+ def test_canonical_mime_type(self, mimetype, expected):+ assert _canonical_mime_type(mimetype) == expected++ @pytest.mark.parametrize(+ ("file_type", "mimetype", "expected"),+ [+ (ContentUnderstandingFileType.PDF, None, "application/pdf"),+ (ContentUnderstandingFileType.M4V, None, "video/mp4"),+ (ContentUnderstandingFileType.FLAC, "audio/x-flac", "audio/flac"),+ ],+ )+ def test_content_type_for(self, file_type, mimetype, expected):+ assert _content_type_for(file_type, mimetype) == expected++ @pytest.mark.parametrize(+ ("file_type", "mimetype", "expected"),+ [+ # Extension/file_type wins when mimetype disagrees — the+ # resolved file_type is the single source of truth so that+ # analyzer routing and payload metadata stay consistent.+ (ContentUnderstandingFileType.PDF, "audio/mpeg", "application/pdf"),+ (ContentUnderstandingFileType.MP3, "application/pdf", "audio/mpeg"),+ (ContentUnderstandingFileType.MP4, "image/jpeg", "video/mp4"),+ (ContentUnderstandingFileType.JPEG, "video/mp4", "image/jpeg"),+ # Subtype distinctions are preserved when consistent+ # (e.g., HEIC vs HEIF both map to file_type HEIF; if the+ # caller passed image/heic explicitly, keep it).+ (ContentUnderstandingFileType.HEIF, "image/heic", "image/heic"),+ (ContentUnderstandingFileType.HEIF, "image/heif", "image/heif"),+ ],+ )+ def test_content_type_for_resolves_conflicts_to_file_type(+ self, file_type, mimetype, expected+ ):+ """When extension and mimetype disagree, file_type wins."""+ assert _content_type_for(file_type, mimetype) == expected++ def test_conflicting_extension_and_mimetype_in_convert(self):+ """End-to-end: conflicting StreamInfo routes by extension and+ sends a content_type consistent with the resolved file_type."""+ conv = _make_converter()+ conv._client = MagicMock()+ mock_poller = MagicMock()+ mock_poller.result.return_value = MagicMock(contents=[])+ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch(+ "markitdown.converters._cu_converter.to_llm_input",+ return_value="ok",+ ):+ conv.convert(+ io.BytesIO(b"fake"),+ # .pdf extension but bogus audio mimetype+ StreamInfo(extension=".pdf", mimetype="audio/mpeg"),+ )++ call_kwargs = conv._client.begin_analyze_binary.call_args.kwargs+ # Routed by extension: document modality → prebuilt-documentSearch+ assert call_kwargs["analyzer_id"] == "prebuilt-documentSearch"+ # content_type derived from file_type (PDF), not the conflicting mime+ assert call_kwargs["content_type"] == "application/pdf"++ def test_file_type_restriction_applies_to_mime(self):+ assert (+ _detect_file_type(+ StreamInfo(mimetype="video/mp4"),+ [ContentUnderstandingFileType.PDF],+ )+ is None+ )+++# ---------------------------------------------------------------------------+# Smart routing tests+# ---------------------------------------------------------------------------+++class TestSmartRouting:+ """Test modality-aware analyzer routing."""++ def test_document_analyzer_routes_pdf_to_custom(self):+ """Document-based analyzer should be used for PDF."""+ conv = _make_converter(+ analyzer_id="my-doc-analyzer",+ analyzer_modality="document",+ )+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake pdf"),+ StreamInfo(extension=".pdf", mimetype="application/pdf"),+ )++ # Should use the custom analyzer for PDF (document modality)+ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "my-doc-analyzer"++ def test_document_analyzer_routes_mp3_to_prebuilt(self):+ """Document-based analyzer should auto-route MP3 to prebuilt-audioSearch."""+ conv = _make_converter(+ analyzer_id="my-doc-analyzer",+ analyzer_modality="document",+ )+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake audio"),+ StreamInfo(extension=".mp3", mimetype="audio/mpeg"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-audioSearch"++ def test_document_analyzer_routes_mp4_to_prebuilt(self):+ """Document-based analyzer should auto-route MP4 to prebuilt-videoSearch."""+ conv = _make_converter(+ analyzer_id="my-doc-analyzer",+ analyzer_modality="document",+ )+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake video"),+ StreamInfo(extension=".mp4", mimetype="video/mp4"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-videoSearch"++ def test_no_analyzer_id_uses_auto_routing(self):+ """Without analyzer_id, PDF should auto-route to prebuilt-documentSearch."""+ conv = _make_converter(analyzer_id=None, analyzer_modality=None)+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake pdf"),+ StreamInfo(extension=".pdf", mimetype="application/pdf"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-documentSearch"++ def test_no_analyzer_id_routes_image_to_document_search(self):+ """Default image routing should still use prebuilt-documentSearch."""+ conv = _make_converter(analyzer_id=None, analyzer_modality=None)+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake image"),+ StreamInfo(extension=".jpg", mimetype="image/jpeg"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-documentSearch"++ def test_document_analyzer_routes_image_to_custom(self):+ """Document-based analyzers should still handle image documents."""+ conv = _make_converter(+ analyzer_id="my-doc-analyzer",+ analyzer_modality="document",+ )+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake image"),+ StreamInfo(extension=".jpg", mimetype="image/jpeg"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "my-doc-analyzer"++ def test_image_analyzer_routes_jpeg_to_custom(self):+ """Image-based analyzers should be used for image files."""+ conv = _make_converter(+ analyzer_id="my-image-analyzer",+ analyzer_modality="image",+ )+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake image"),+ StreamInfo(extension=".jpg", mimetype="image/jpeg"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "my-image-analyzer"++ def test_image_analyzer_routes_pdf_to_document_prebuilt(self):+ """Image-based analyzers should not claim non-image document files."""+ conv = _make_converter(+ analyzer_id="my-image-analyzer",+ analyzer_modality="image",+ )+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(+ io.BytesIO(b"fake pdf"),+ StreamInfo(extension=".pdf", mimetype="application/pdf"),+ )++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-documentSearch"++ @pytest.mark.parametrize(+ ("mimetype", "expected_analyzer"),+ [+ ("video/mp4", "prebuilt-videoSearch"),+ ("video/x-m4v", "prebuilt-videoSearch"),+ ("audio/mpeg", "prebuilt-audioSearch"),+ ("audio/x-wav", "prebuilt-audioSearch"),+ ],+ )+ def test_mime_only_input_uses_auto_routing(self, mimetype, expected_analyzer):+ """MIME-only streams should route to the matching modality analyzer."""+ conv = _make_converter(analyzer_id=None, analyzer_modality=None)+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(io.BytesIO(b"fake content"), StreamInfo(mimetype=mimetype))++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == expected_analyzer++ def test_mime_alias_input_uses_canonical_content_type(self):+ """Alias MIME types should be sent to CU as canonical content types."""+ conv = _make_converter(analyzer_id=None, analyzer_modality=None)+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(io.BytesIO(b"fake video"), StreamInfo(mimetype="video/x-m4v"))++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-videoSearch"+ assert call_args.kwargs["content_type"] == "video/mp4"++ def test_extension_only_input_uses_file_type_content_type(self):+ """Extension-only inputs should send CU a matching content type."""+ conv = _make_converter(analyzer_id=None, analyzer_modality=None)+ conv._client = MagicMock()+ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result++ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch("markitdown.converters._cu_converter.to_llm_input", return_value=""):+ conv.convert(io.BytesIO(b"fake pdf"), StreamInfo(extension=".pdf"))++ call_args = conv._client.begin_analyze_binary.call_args+ assert call_args.kwargs["analyzer_id"] == "prebuilt-documentSearch"+ assert call_args.kwargs["content_type"] == "application/pdf"+++# ---------------------------------------------------------------------------+# _infer_prebuilt_modality tests+# ---------------------------------------------------------------------------+++class TestResolveAnalyzerModality:+ """Test modality resolution from analyzer IDs."""++ def test_known_document_prebuilts(self):+ client = MagicMock()+ assert (+ _resolve_analyzer_modality(client, "prebuilt-documentSearch") == "document"+ )+ assert _resolve_analyzer_modality(client, "prebuilt-invoice") == "document"+ assert _resolve_analyzer_modality(client, "prebuilt-layout") == "document"+ assert _resolve_analyzer_modality(client, "prebuilt-receipt") == "document"+ assert _resolve_analyzer_modality(client, "prebuilt-tax.us.w2") == "document"+ # Known prebuilts should never call get_analyzer()+ client.get_analyzer.assert_not_called()++ def test_known_audio_prebuilts(self):+ client = MagicMock()+ assert _resolve_analyzer_modality(client, "prebuilt-audioSearch") == "audio"+ assert _resolve_analyzer_modality(client, "prebuilt-callCenter") == "audio"+ client.get_analyzer.assert_not_called()++ def test_known_video_prebuilts(self):+ client = MagicMock()+ assert _resolve_analyzer_modality(client, "prebuilt-videoSearch") == "video"+ assert _resolve_analyzer_modality(client, "prebuilt-videoSynopsis") == "video"+ client.get_analyzer.assert_not_called()++ def test_known_image_prebuilts(self):+ client = MagicMock()+ assert _resolve_analyzer_modality(client, "prebuilt-imageSearch") == "image"+ assert _resolve_analyzer_modality(client, "prebuilt-image") == "image"+ client.get_analyzer.assert_not_called()++ def test_unknown_prebuilt_falls_back_to_get_analyzer(self):+ """Unknown prebuilt-* names should call get_analyzer() for resolution."""+ client = MagicMock()+ mock_analyzer = MagicMock()+ mock_analyzer.base_analyzer_id = "prebuilt-audio"+ client.get_analyzer.return_value = mock_analyzer++ result = _resolve_analyzer_modality(client, "prebuilt-newAnalyzer")+ assert result == "audio"+ client.get_analyzer.assert_called_once_with("prebuilt-newAnalyzer")++ def test_custom_analyzer_calls_get_analyzer(self):+ """Custom analyzers should call get_analyzer() to resolve modality."""+ client = MagicMock()+ mock_analyzer = MagicMock()+ mock_analyzer.base_analyzer_id = "prebuilt-document"+ client.get_analyzer.return_value = mock_analyzer++ result = _resolve_analyzer_modality(client, "my-custom-doc-analyzer")+ assert result == "document"+ client.get_analyzer.assert_called_once_with("my-custom-doc-analyzer")++ def test_custom_analyzer_no_base_defaults_to_document(self):+ """Analyzer with no base_analyzer_id defaults to document."""+ client = MagicMock()+ mock_analyzer = MagicMock()+ mock_analyzer.base_analyzer_id = None+ client.get_analyzer.return_value = mock_analyzer++ result = _resolve_analyzer_modality(client, "my-custom-analyzer")+ assert result == "document"++ def test_get_analyzer_failure_raises_value_error(self):+ """Failed get_analyzer() should raise ValueError."""+ client = MagicMock()+ client.get_analyzer.side_effect = Exception("not found")++ with pytest.raises(ValueError, match="Failed to resolve analyzer 'bad-id'"):+ _resolve_analyzer_modality(client, "bad-id")+++# ---------------------------------------------------------------------------+# _get_modality tests+# ---------------------------------------------------------------------------+++class TestGetModality:+ """Test file type → modality mapping."""++ def test_document_types(self):+ assert _get_modality(ContentUnderstandingFileType.PDF) == "document"+ assert _get_modality(ContentUnderstandingFileType.DOCX) == "document"++ def test_image_types(self):+ assert _get_modality(ContentUnderstandingFileType.JPEG) == "image"+ assert _get_modality(ContentUnderstandingFileType.PNG) == "image"++ def test_video_types(self):+ assert _get_modality(ContentUnderstandingFileType.MP4) == "video"+ assert _get_modality(ContentUnderstandingFileType.MOV) == "video"++ def test_audio_types(self):+ assert _get_modality(ContentUnderstandingFileType.WAV) == "audio"+ assert _get_modality(ContentUnderstandingFileType.MP3) == "audio"+++# ---------------------------------------------------------------------------+# convert() mock tests+# ---------------------------------------------------------------------------+++class TestConvertMock:+ """Test convert() with mocked CU SDK."""++ def _run_convert(self, extension, mimetype, expected_output="mock output"):+ conv = _make_converter()+ conv._client = MagicMock()++ mock_result = MagicMock()+ mock_result.contents = []+ mock_poller = MagicMock()+ mock_poller.result.return_value = mock_result+ conv._client.begin_analyze_binary.return_value = mock_poller++ with patch(+ "markitdown.converters._cu_converter.to_llm_input",+ return_value=expected_output,+ ):+ result = conv.convert(+ io.BytesIO(b"fake content"),+ StreamInfo(extension=extension, mimetype=mimetype),+ )+ return result++ def test_pdf_returns_markdown(self):+ result = self._run_convert(+ ".pdf", "application/pdf", "---\ncontentType: document\n---\n# Test"+ )+ assert "contentType: document" in result.markdown++ def test_mp4_returns_markdown(self):+ result = self._run_convert(+ ".mp4", "video/mp4", "---\ncontentType: audioVisual\n---\nSpeaker 1: Hello"+ )+ assert "contentType: audioVisual" in result.markdown++ def test_wav_returns_markdown(self):+ result = self._run_convert(+ ".wav", "audio/wav", "---\ncontentType: audioVisual\n---\nSpeaker 1: Hi"+ )+ assert "audioVisual" in result.markdown++ def test_empty_result(self):+ result = self._run_convert(".pdf", "application/pdf", "")+ assert result.markdown == ""++ def test_jpeg_returns_markdown(self):+ result = self._run_convert(+ ".jpg", "image/jpeg", "---\ncontentType: document\n---\n# Photo"+ )+ assert "contentType: document" in result.markdown+++# ---------------------------------------------------------------------------+# Init-time get_analyzer() error wrapping+# ---------------------------------------------------------------------------+++class TestGetAnalyzerError:+ """Test that get_analyzer() failures at init produce a clear error."""++ def test_nonexistent_analyzer_raises_value_error(self):+ """A failed get_analyzer() should raise ValueError with analyzer name."""+ with patch(+ "markitdown.converters._cu_converter._dependency_exc_info", None+ ), patch(+ "markitdown.converters._cu_converter.ContentUnderstandingClient"+ ) as MockClient, patch(+ "markitdown.converters._cu_converter.DefaultAzureCredential"+ ):+ mock_client = MagicMock()+ mock_client.get_analyzer.side_effect = Exception("not found")+ MockClient.return_value = mock_client++ with pytest.raises(ValueError, match="Failed to resolve analyzer 'bad-id'"):+ ContentUnderstandingConverter(+ endpoint="https://fake", analyzer_id="bad-id"+ )+++# ---------------------------------------------------------------------------+# Registration priority test+# ---------------------------------------------------------------------------+++class TestRegistrationPriority:+ """Test that CU converter is registered with higher priority than Doc Intel."""++ def test_cu_registered_before_docintel(self):+ """When both endpoints are provided, CU should appear before Doc Intel."""+ with patch(+ "markitdown.converters._cu_converter._dependency_exc_info", None+ ), patch(+ "markitdown.converters._cu_converter.ContentUnderstandingClient"+ ), patch(+ "markitdown.converters._cu_converter.DefaultAzureCredential"+ ), patch(+ "markitdown.converters._doc_intel_converter._dependency_exc_info", None+ ), patch(+ "markitdown.converters._doc_intel_converter.DocumentIntelligenceClient"+ ), patch(+ "markitdown.converters._doc_intel_converter.DefaultAzureCredential"+ ):+ from markitdown import MarkItDown+ from markitdown.converters import (+ ContentUnderstandingConverter,+ DocumentIntelligenceConverter,+ )++ md = MarkItDown(+ cu_endpoint="https://fake-cu",+ docintel_endpoint="https://fake-di",+ )++ converter_types = [type(reg.converter) for reg in md._converters]+ cu_idx = converter_types.index(ContentUnderstandingConverter)+ di_idx = converter_types.index(DocumentIntelligenceConverter)+ assert (+ cu_idx < di_idx+ ), "CU should have higher priority (lower index) than Doc Intel"+++# ---------------------------------------------------------------------------+# CLI argument tests+# ---------------------------------------------------------------------------+++class TestCLIArgs:+ """Test CLI argument parsing for CU flags."""++ def test_use_cu_without_endpoint_exits(self):+ """--use-cu without --cu-endpoint should exit with error."""+ import subprocess++ result = subprocess.run(+ [sys.executable, "-m", "markitdown", "--use-cu", "fake.pdf"],+ capture_output=True,+ text=True,+ )+ assert result.returncode != 0+ assert (+ "cu-endpoint" in result.stderr.lower()+ or "cu-endpoint" in (result.stdout or "").lower()+ )++ def test_use_cu_and_use_docintel_mutually_exclusive(self):+ """--use-cu and --use-docintel cannot be used together."""+ import subprocess++ result = subprocess.run(+ [+ sys.executable,+ "-m",+ "markitdown",+ "--use-cu",+ "--cu-endpoint",+ "https://fake",+ "--use-docintel",+ "-e",+ "https://fake-di",+ "fake.pdf",+ ],+ capture_output=True,+ text=True,+ )+ assert result.returncode != 0++ def test_cu_file_types_parsing(self):+ """--cu-file-types should parse comma-separated values into enum list."""+ from markitdown.converters import ContentUnderstandingFileType++ raw = "pdf,jpeg,mp4"+ type_names = [t.strip().lower() for t in raw.split(",") if t.strip()]+ cu_types = [ContentUnderstandingFileType(name) for name in type_names]++ assert cu_types == [+ ContentUnderstandingFileType.PDF,+ ContentUnderstandingFileType.JPEG,+ ContentUnderstandingFileType.MP4,+ ]++ def test_cu_file_types_invalid_value(self):+ """Unknown file type name should raise ValueError."""+ from markitdown.converters import ContentUnderstandingFileType++ with pytest.raises(ValueError):+ ContentUnderstandingFileType("nonsense")++ def test_cu_file_types_single_value(self):+ """Single file type (no comma) should parse correctly."""+ from markitdown.converters import ContentUnderstandingFileType++ cu_types = [+ ContentUnderstandingFileType(t.strip().lower())+ for t in "wav".split(",")+ if t.strip()+ ]+ assert cu_types == [ContentUnderstandingFileType.WAV]++ def test_use_cu_wires_kwargs_to_markitdown(self, capsys):+ """--use-cu should pass CU options through to MarkItDown."""+ import markitdown.__main__ as markitdown_cli++ markitdown_instance = MagicMock()+ markitdown_instance.convert.return_value.markdown = "converted"+ markitdown_cls = MagicMock(return_value=markitdown_instance)++ with patch.object(+ sys,+ "argv",+ [+ "markitdown",+ "--use-cu",+ "--cu-endpoint",+ "https://fake-cu",+ "--cu-analyzer",+ "custom-analyzer",+ "--cu-file-types",+ "pdf,jpeg,mp4",+ "fake.pdf",+ ],+ ), patch.object(markitdown_cli, "MarkItDown", markitdown_cls):+ markitdown_cli.main()++ markitdown_cls.assert_called_once_with(+ enable_plugins=False,+ cu_endpoint="https://fake-cu",+ cu_analyzer_id="custom-analyzer",+ cu_file_types=[+ ContentUnderstandingFileType.PDF,+ ContentUnderstandingFileType.JPEG,+ ContentUnderstandingFileType.MP4,+ ],+ )+ markitdown_instance.convert.assert_called_once_with(+ "fake.pdf", stream_info=None, keep_data_uris=False+ )+ assert capsys.readouterr().out == "converted\n"+++# ---------------------------------------------------------------------------+# MissingDependencyException test+# ---------------------------------------------------------------------------+++class TestMissingDependency:+ """Test that MissingDependencyException is raised when CU SDK is not installed."""++ def test_missing_deps_message(self):+ """Converter construction should surface the optional install hint."""+ import markitdown.converters._cu_converter as cu_converter_module+ from markitdown._exceptions import MissingDependencyException++ import_error = ImportError("No module named 'azure.ai.contentunderstanding'")+ dependency_exc_info = (ImportError, import_error, None)++ with patch.object(+ cu_converter_module, "_dependency_exc_info", dependency_exc_info+ ), pytest.raises(MissingDependencyException) as exc_info:+ ContentUnderstandingConverter(endpoint="https://fake-cu")++ assert "az-content-understanding" in str(exc_info.value)+ assert exc_info.value.__cause__ is import_error