Skip to content

Python tool for converting files and office documents to Markdown.

feat(doc-intel): expose models, features, query fields, and structured output (#1913)

Chien Yuan Chang7482c092d9c0
diff --git a/README.md b/README.md
index fa33f6b3..13e1ea8b 100644
--- a/README.md
+++ b/README.md
@@ -176,8 +176,8 @@ Content Understanding is ideal when you need capabilities beyond what built-in o
| Capability | Built-in converters | Azure Document Intelligence | Azure Content Understanding |
|------------|---------------------|-----------------------------|-----------------------------|
| Document conversion | Offline, format-specific extraction | Cloud layout extraction | Cloud multimodal extraction |
-| Structured fields | Not available | Not exposed by this integration | YAML front matter from analyzer fields |
-| Custom analyzers | Not available | Not configurable in this integration | Supported with `cu_analyzer_id` |
+| Structured fields | Not available | YAML front matter from model or query fields | YAML front matter from analyzer fields |
+| Custom analyzers/models | Not available | Supported with `docintel_model_id` | Supported with `cu_analyzer_id` |
| Audio and video | Basic audio, no video | Not supported | Audio and video analyzers |
| Cost | Local compute only | Billable Azure API calls | Billable Azure API calls |
@@ -247,6 +247,7 @@ More information about Azure Content Understanding can be found [here](https://l
To use Microsoft Document Intelligence for conversion:
```bash
+export AZURE_API_KEY="<document_intelligence_api_key>"
markitdown path-to-file.pdf -o document.md -d -e "<document_intelligence_endpoint>"
```
@@ -257,6 +258,31 @@ export MARKITDOWN_DOCINTEL_ENDPOINT="<document_intelligence_endpoint>"
markitdown path-to-file.pdf -o document.md -d
```
+If `AZURE_API_KEY` is not set, MarkItDown uses `DefaultAzureCredential`.
+
+Select a model, configure analysis add-ons, or request fields from the CLI:
+
+```bash
+markitdown document.pdf --use-docintel \
+ --endpoint "<document_intelligence_endpoint>" \
+ --docintel-model-id prebuilt-layout \
+ --docintel-features "ocrHighResolution,styleFont" \
+ --docintel-query-fields "Title,Author"
+```
+
+No analysis add-ons are enabled by default. Use `--docintel-features` to opt in,
+or `--docintel-features none` to explicitly request no add-ons. Analysis add-ons
+and query fields apply only to OCR-supported inputs such as PDFs and images.
+
+Available feature values are `barcodes`, `formulas`, `keyValuePairs`,
+`languages`, `ocrHighResolution`, and `styleFont`. The `formulas`,
+`ocrHighResolution`, and `styleFont` features incur additional charges, as does
+the `queryFields` add-on, which is priced differently from the other add-ons.
+See the
+[Document Intelligence add-on documentation](https://learn.microsoft.com/azure/ai-services/document-intelligence/concept/add-on-capabilities?view=doc-intel-4.0.0)
+and [pricing](https://azure.microsoft.com/pricing/details/document-intelligence/)
+for details.
+
More information about how to set up an Azure Document Intelligence Resource can be found [here](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/how-to-guides/create-document-intelligence-resource?view=doc-intel-4.0.0)
### Python API
@@ -271,16 +297,53 @@ result = md.convert("test.xlsx")
print(result.markdown)
```
-Document Intelligence conversion in Python:
+Document Intelligence conversion in Python using an API key:
```python
+from azure.core.credentials import AzureKeyCredential
from markitdown import MarkItDown
-md = MarkItDown(docintel_endpoint="<document_intelligence_endpoint>")
+md = MarkItDown(
+ docintel_endpoint="<document_intelligence_endpoint>",
+ docintel_credential=AzureKeyCredential("<document_intelligence_api_key>"),
+)
result = md.convert("test.pdf")
print(result.markdown)
```
+Omit `docintel_credential` to use `AZURE_API_KEY` when it is set, or
+`DefaultAzureCredential` otherwise.
+
+Configure the model, analysis add-ons, and structured query fields:
+
+```python
+from azure.ai.documentintelligence.models import DocumentAnalysisFeature
+from markitdown import MarkItDown
+
+md = MarkItDown(
+ docintel_endpoint="<document_intelligence_endpoint>",
+ docintel_model_id="prebuilt-layout",
+ docintel_features=[
+ DocumentAnalysisFeature.OCR_HIGH_RESOLUTION,
+ DocumentAnalysisFeature.STYLE_FONT,
+ ],
+ docintel_query_fields=["Title", "Author"],
+)
+result = md.convert("test.pdf")
+print(result.markdown)
+```
+
+Omit `docintel_features` or set `docintel_features=[]` to request no analysis
+add-ons. When query fields are provided, MarkItDown automatically enables the
+required `queryFields` add-on.
+Models that return structured fields prepend them to the Markdown as YAML front
+matter. A single document uses a top-level `fields` mapping. When the service
+returns multiple documents, the front matter instead contains a `documents`
+list in service order, with a separate `fields` mapping for each document.
+Documents without extracted fields retain an empty mapping in that list; no
+front matter is emitted when all documents have empty fields. Selection groups
+are represented as lists of selected values.
+
To use Large Language Models for image descriptions (currently only for pptx and image files), provide `llm_client` and `llm_model`:
```python
diff --git a/packages/markitdown/pyproject.toml b/packages/markitdown/pyproject.toml
index b7456edd..fc6db99c 100644
--- a/packages/markitdown/pyproject.toml
+++ b/packages/markitdown/pyproject.toml
@@ -82,6 +82,7 @@ features = ["all"]
features = ["all"]
extra-dependencies = [
"openai",
+ "pyyaml",
]
[tool.hatch.envs.types]
diff --git a/packages/markitdown/src/markitdown/__main__.py b/packages/markitdown/src/markitdown/__main__.py
index 37334ad9..714eaee6 100644
--- a/packages/markitdown/src/markitdown/__main__.py
+++ b/packages/markitdown/src/markitdown/__main__.py
@@ -121,6 +121,33 @@ def main():
help="Comma-separated list of file types to route to Content Understanding (e.g., pdf,jpeg,mp4). If omitted, all supported types are routed.",
)
+ parser.add_argument(
+ "--docintel-model-id",
+ type=str,
+ default=None,
+ help="Document Intelligence model ID (e.g., 'prebuilt-layout', 'prebuilt-invoice', or a custom model ID). Defaults to 'prebuilt-layout'.",
+ )
+
+ parser.add_argument(
+ "--docintel-query-fields",
+ type=str,
+ default=None,
+ help="Comma-separated list of field names to extract via the billable Document Intelligence queryFields add-on (OCR file types only).",
+ )
+
+ parser.add_argument(
+ "--docintel-features",
+ type=str,
+ default=None,
+ help=(
+ "Comma-separated Document Intelligence analysis add-ons for OCR file types. "
+ "Available values: barcodes, formulas, keyValuePairs, languages, "
+ "ocrHighResolution, and styleFont. The formulas, ocrHighResolution, and "
+ "styleFont incur additional charges. No add-ons are enabled by default; "
+ "use 'none' to explicitly request none."
+ ),
+ )
+
parser.add_argument(
"-p",
"--use-plugins",
@@ -211,9 +238,28 @@ def main():
elif args.filename is None:
_exit_with_error("Filename is required when using Document Intelligence.")
- markitdown = MarkItDown(
- enable_plugins=args.use_plugins, docintel_endpoint=args.endpoint
- )
+ docintel_kwargs: Dict[str, Any] = {
+ "docintel_endpoint": args.endpoint,
+ }
+ if args.docintel_model_id:
+ docintel_kwargs["docintel_model_id"] = args.docintel_model_id
+ if args.docintel_features is not None:
+ if args.docintel_features.strip().lower() == "none":
+ docintel_kwargs["docintel_features"] = []
+ else:
+ docintel_kwargs["docintel_features"] = [
+ feature.strip()
+ for feature in args.docintel_features.split(",")
+ if feature.strip()
+ ]
+ if args.docintel_query_fields:
+ fields = [
+ f.strip() for f in args.docintel_query_fields.split(",") if f.strip()
+ ]
+ if fields:
+ docintel_kwargs["docintel_query_fields"] = fields
+
+ markitdown = MarkItDown(enable_plugins=args.use_plugins, **docintel_kwargs)
elif args.use_cu:
if args.cu_endpoint is None:
_exit_with_error(
diff --git a/packages/markitdown/src/markitdown/_markitdown.py b/packages/markitdown/src/markitdown/_markitdown.py
index 8e2fe6a3..45cef7fb 100644
--- a/packages/markitdown/src/markitdown/_markitdown.py
+++ b/packages/markitdown/src/markitdown/_markitdown.py
@@ -269,6 +269,18 @@ class MarkItDown:
if docintel_version is not None:
docintel_args["api_version"] = docintel_version
+ docintel_model_id = kwargs.get("docintel_model_id")
+ if docintel_model_id is not None:
+ docintel_args["model_id"] = docintel_model_id
+
+ docintel_features = kwargs.get("docintel_features")
+ if docintel_features is not None:
+ docintel_args["features"] = docintel_features
+
+ docintel_query_fields = kwargs.get("docintel_query_fields")
+ if docintel_query_fields is not None:
+ docintel_args["query_fields"] = docintel_query_fields
+
self.register_converter(
DocumentIntelligenceConverter(**docintel_args),
)
diff --git a/packages/markitdown/src/markitdown/converters/_doc_intel_converter.py b/packages/markitdown/src/markitdown/converters/_doc_intel_converter.py
index f8a5c8e8..32180ad3 100644
--- a/packages/markitdown/src/markitdown/converters/_doc_intel_converter.py
+++ b/packages/markitdown/src/markitdown/converters/_doc_intel_converter.py
@@ -1,12 +1,16 @@
import sys
import re
import os
-from typing import BinaryIO, Any, List
+from datetime import date, datetime, time
+from typing import BinaryIO, Any, List, Optional
from enum import Enum
from .._base_converter import DocumentConverter, DocumentConverterResult
from .._stream_info import StreamInfo
from .._exceptions import MissingDependencyException
+from .. import __version__ as _markitdown_version
+
+_USER_AGENT = f"markitdown-docintel/{_markitdown_version}"
# Try loading optional (but in this case, required) dependencies
# Save reporting of any exceptions for later
@@ -127,6 +131,193 @@ def _get_file_extensions(types: List[DocumentIntelligenceFileType]) -> List[str]
return extensions
+def _field_value(field: Any) -> Any:
+ """
+ Extract a serializable Python value from a Document Intelligence DocumentField.
+
+ Returns the most specific typed value when available, falling back to the
+ raw ``content`` string. Returns ``None`` when nothing usable is present.
+ """
+ if field is None:
+ return None
+
+ # Typed scalar values (in rough order of specificity).
+ for attr in (
+ "value_string",
+ "value_boolean",
+ "value_integer",
+ "value_number",
+ "value_date",
+ "value_time",
+ "value_phone_number",
+ "value_country_region",
+ "value_selection_mark",
+ "value_signature",
+ ):
+ v = getattr(field, attr, None)
+ if v is not None:
+ if isinstance(v, (date, datetime, time)):
+ return v.isoformat()
+ return v
+
+ # Currency: { amount, currencySymbol, currencyCode }
+ cur = getattr(field, "value_currency", None)
+ if cur is not None:
+ amount = getattr(cur, "amount", None)
+ code = getattr(cur, "currency_code", None) or getattr(
+ cur, "currency_symbol", None
+ )
+ if amount is not None and code:
+ return f"{amount} {code}"
+ if amount is not None:
+ return amount
+
+ # Address: serialize to its content/string form.
+ addr = getattr(field, "value_address", None)
+ if addr is not None:
+ return getattr(field, "content", None) or str(addr)
+
+ # Selection groups contain the selected labels, not DocumentField objects.
+ selection_group = getattr(field, "value_selection_group", None)
+ if selection_group is not None:
+ return list(selection_group)
+
+ # Array of fields -> list of values.
+ arr = getattr(field, "value_array", None)
+ if arr is not None:
+ return [_field_value(item) for item in arr]
+
+ # Object of fields -> dict of values.
+ obj = getattr(field, "value_object", None)
+ if obj is not None:
+ return {k: _field_value(v) for k, v in obj.items()}
+
+ # Last resort: the raw extracted text.
+ return getattr(field, "content", None)
+
+
+def _yaml_scalar(value: Any) -> str:
+ """Render a scalar value as a YAML string."""
+ if value is None:
+ return "null"
+ if isinstance(value, bool):
+ return "true" if value else "false"
+ if isinstance(value, (int, float)):
+ rendered = repr(value)
+ if isinstance(value, float) and "e" in rendered:
+ mantissa, exponent = rendered.split("e", 1)
+ # YAML 1.1 requires a decimal point in exponential float mantissas.
+ if "." not in mantissa:
+ mantissa += ".0"
+ rendered = f"{mantissa}e{exponent}"
+ return rendered
+ s = str(value)
+ # Quote when necessary: contains special chars, leading/trailing whitespace,
+ # or characters that would confuse a YAML parser.
+ if (
+ not re.fullmatch(r"[A-Za-z_][A-Za-z0-9 _./-]*", s)
+ or s != s.strip()
+ or s.lower() in ("null", "true", "false", "yes", "no", "on", "off", "y", "n")
+ ):
+ # Escape backslashes, double quotes, and common whitespace characters.
+ escaped = (
+ s.replace("\\", "\\\\")
+ .replace('"', '\\"')
+ .replace("\n", "\\n")
+ .replace("\r", "\\r")
+ .replace("\t", "\\t")
+ )
+ # Quoting alone does not make YAML control characters valid. Escape
+ # those too, including Unicode line breaks that YAML would normalize.
+ escaped = re.sub(
+ r"[\x00-\x1f\x7f-\x9f\u2028\u2029\ud800-\udfff\ufffe\uffff]",
+ lambda match: f"\\u{ord(match.group()):04x}",
+ escaped,
+ )
+ return f'"{escaped}"'
+ return s
+
+
+def _yaml_dump(value: Any, indent: int = 0) -> str:
+ """Minimal YAML emitter for scalars, lists, and dicts of scalars/lists/dicts."""
+ pad = " " * indent
+ if isinstance(value, dict):
+ if not value:
+ return f"{pad}{{}}"
+ lines: List[str] = []
+ for k, v in value.items():
+ key = _yaml_scalar(k)
+ if isinstance(v, (dict, list)) and v:
+ lines.append(f"{pad}{key}:")
+ lines.append(_yaml_dump(v, indent + 1))
+ else:
+ lines.append(
+ f"{pad}{key}: {_yaml_scalar(v) if not isinstance(v, (dict, list)) else ('{}' if isinstance(v, dict) else '[]')}"
+ )
+ return "\n".join(lines)
+ if isinstance(value, list):
+ if not value:
+ return f"{pad}[]"
+ lines = []
+ for item in value:
+ if isinstance(item, (dict, list)) and item:
+ lines.append(f"{pad}-")
+ lines.append(_yaml_dump(item, indent + 1))
+ else:
+ lines.append(
+ f"{pad}- {_yaml_scalar(item) if not isinstance(item, (dict, list)) else ('{}' if isinstance(item, dict) else '[]')}"
+ )
+ return "\n".join(lines)
+ return f"{pad}{_yaml_scalar(value)}"
+
+
+def _fields_to_front_matter(documents: Any, model_id: Optional[str] = None) -> str:
+ """
+ Build a YAML front matter block from ``AnalyzeResult.documents[*].fields``.
+
+ Returns an empty string when there are no documents or no non-empty fields.
+ A single document uses a ``fields`` mapping. Multiple documents use an
+ ordered ``documents`` list, each with its own ``fields`` mapping, so values
+ from different documents cannot overwrite or be combined with one another.
+
+ A single document has the following shape:
+
+ ---
+ modelId: prebuilt-invoice
+ fields:
+ VendorName: Contoso Ltd.
+ InvoiceTotal: 1250.0
+ ---
+ """
+ if not documents:
+ return ""
+
+ document_fields: List[dict] = []
+ for doc in documents:
+ extracted: dict = {}
+ fields = getattr(doc, "fields", None) or {}
+ for name, field in fields.items():
+ value = _field_value(field)
+ if value is None or value == "" or value == [] or value == {}:
+ continue
+ extracted[name] = value
+ document_fields.append(extracted)
+
+ if not any(document_fields):
+ return ""
+
+ payload: dict = {}
+ if model_id:
+ payload["modelId"] = model_id
+ if len(document_fields) == 1:
+ payload["fields"] = document_fields[0]
+ else:
+ payload["documents"] = [{"fields": fields} for fields in document_fields]
+
+ body = _yaml_dump(payload)
+ return f"---\n{body}\n---\n\n"
+
+
class DocumentIntelligenceConverter(DocumentConverter):
"""Specialized DocumentConverter that uses Document Intelligence to extract text from documents."""
@@ -136,6 +327,9 @@ class DocumentIntelligenceConverter(DocumentConverter):
endpoint: str,
api_version: str | None = None,
credential: AzureKeyCredential | TokenCredential | None = None,
+ model_id: str = "prebuilt-layout",
+ features: Optional[List[str]] = None,
+ query_fields: Optional[List[str]] = None,
file_types: List[DocumentIntelligenceFileType] = [
DocumentIntelligenceFileType.DOCX,
DocumentIntelligenceFileType.PPTX,
@@ -154,11 +348,21 @@ class DocumentIntelligenceConverter(DocumentConverter):
endpoint (str): The endpoint for the Document Intelligence service.
api_version (str | None): The API version to use. Defaults to None.
credential (AzureKeyCredential | TokenCredential | None): The credential to use for authentication.
+ model_id (str): The Document Intelligence model ID to use (e.g., "prebuilt-layout",
+ "prebuilt-invoice", "prebuilt-receipt", or a custom model ID). Defaults to "prebuilt-layout".
+ features (List[str] | None): Analysis add-on features to request for OCR-supported
+ file types. No add-ons are enabled by default. Query fields automatically add
+ the required ``queryFields`` feature.
+ query_fields (List[str] | None): Optional list of field names to extract via the DI
+ ``queryFields`` add-on. Only applied to OCR-supported file types (PDF/images).
file_types (List[DocumentIntelligenceFileType]): The file types to accept. Defaults to all supported file types.
"""
super().__init__()
self._file_types = file_types
+ self._model_id = model_id
+ self._features = list(features) if features is not None else None
+ self._query_fields = list(query_fields) if query_fields else None
# Raise an error if the dependencies are not available.
# This is different than other converters since this one isn't even instantiated
@@ -180,10 +384,10 @@ class DocumentIntelligenceConverter(DocumentConverter):
self.endpoint = endpoint
self.api_version = api_version
-
client_kwargs: dict[str, Any] = {
"endpoint": self.endpoint,
"credential": credential,
+ "user_agent": _USER_AGENT,
}
if self.api_version is not None:
client_kwargs["api_version"] = self.api_version
@@ -232,11 +436,11 @@ class DocumentIntelligenceConverter(DocumentConverter):
if mimetype.startswith(prefix):
return []
- return [
- DocumentAnalysisFeature.FORMULAS, # enable formula extraction
- DocumentAnalysisFeature.OCR_HIGH_RESOLUTION, # enable high resolution OCR
- DocumentAnalysisFeature.STYLE_FONT, # enable font style extraction
- ]
+ features = list(self._features) if self._features is not None else []
+
+ if self._query_fields and DocumentAnalysisFeature.QUERY_FIELDS not in features:
+ features.append(DocumentAnalysisFeature.QUERY_FIELDS)
+ return features
def convert(
self,
@@ -244,15 +448,32 @@ class DocumentIntelligenceConverter(DocumentConverter):
stream_info: StreamInfo,
**kwargs: Any, # Options to pass to the converter
) -> DocumentConverterResult:
+ # Build optional kwargs so that we only pass query_fields when the
+ # QUERY_FIELDS feature is actually enabled for this file type.
+ features = self._analysis_features(stream_info)
+ extra: dict = {}
+ if self._query_fields and DocumentAnalysisFeature.QUERY_FIELDS in features:
+ extra["query_fields"] = self._query_fields
+
# Extract the text using Azure Document Intelligence
poller = self.doc_intel_client.begin_analyze_document(
- model_id="prebuilt-layout",
+ model_id=self._model_id,
body=AnalyzeDocumentRequest(bytes_source=file_stream.read()),
- features=self._analysis_features(stream_info),
+ features=features,
output_content_format=CONTENT_FORMAT, # TODO: replace with "ContentFormat.MARKDOWN" when the bug is fixed
+ **extra,
)
result: AnalyzeResult = poller.result()
# remove comments from the markdown content generated by Doc Intelligence and append to markdown string
markdown_text = re.sub(r"<!--.*?-->", "", result.content, flags=re.DOTALL)
+
+ # Prepend YAML front matter when DI returned structured fields (e.g., from
+ # prebuilt-invoice/-receipt, custom models, or queryFields).
+ front_matter = _fields_to_front_matter(
+ getattr(result, "documents", None), model_id=self._model_id
+ )
+ if front_matter:
+ markdown_text = front_matter + markdown_text
+
return DocumentConverterResult(markdown=markdown_text)
diff --git a/packages/markitdown/tests/test_cli_misc.py b/packages/markitdown/tests/test_cli_misc.py
index 708de303..f45389ce 100644
--- a/packages/markitdown/tests/test_cli_misc.py
+++ b/packages/markitdown/tests/test_cli_misc.py
@@ -30,6 +30,24 @@ def test_version() -> None:
assert __version__ in result.stdout, f"Version not found in output: {result.stdout}"
+def test_docintel_help_lists_features_and_charges() -> None:
+ result = subprocess.run(
+ [sys.executable, "-m", "markitdown", "--help"],
+ capture_output=True,
+ text=True,
+ )
+
+ assert result.returncode == 0, f"CLI exited with error: {result.stderr}"
+ help_text = " ".join(result.stdout.split())
+ assert (
+ "barcodes, formulas, keyValuePairs, languages, ocrHighResolution, and styleFont"
+ in help_text
+ )
+ assert "incur additional charges" in help_text
+ assert "No add-ons are enabled by default" in help_text
+ assert "billable Document Intelligence queryFields add-on" in help_text
+
+
def test_invalid_flag() -> None:
result = subprocess.run(
[sys.executable, "-m", "markitdown", "--foobar"],
@@ -115,6 +133,54 @@ class TestCLIArgs:
"--use-cu/--use-content-understanding"
) in result.stderr, result.stderr
+ @pytest.mark.parametrize(
+ "raw_features, expected_features",
+ [
+ ("formulas,ocrHighResolution", ["formulas", "ocrHighResolution"]),
+ ("none", []),
+ ],
+ )
+ def test_use_docintel_wires_configuration(
+ self, monkeypatch, capsys, raw_features, expected_features
+ ):
+ import markitdown.__main__ as markitdown_cli
+
+ markitdown_instance = MagicMock()
+ markitdown_instance.convert.return_value.markdown = "converted"
+ markitdown_cls = MagicMock(return_value=markitdown_instance)
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "markitdown",
+ "--use-docintel",
+ "--endpoint",
+ "https://fake-di",
+ "--docintel-model-id",
+ "prebuilt-read",
+ "--docintel-features",
+ raw_features,
+ "--docintel-query-fields",
+ "Title,Author",
+ "fake.pdf",
+ ],
+ )
+ monkeypatch.setattr(markitdown_cli, "MarkItDown", markitdown_cls)
+
+ markitdown_cli.main()
+
+ markitdown_cls.assert_called_once_with(
+ enable_plugins=False,
+ docintel_endpoint="https://fake-di",
+ docintel_model_id="prebuilt-read",
+ docintel_features=expected_features,
+ docintel_query_fields=["Title", "Author"],
+ )
+ markitdown_instance.convert.assert_called_once_with(
+ "fake.pdf", stream_info=None, keep_data_uris=False
+ )
+ assert capsys.readouterr().out == "converted\n"
+
@pytest.mark.parametrize("raw", ["pdf,jpeg,wav", " , PDF, JPEG,, WAV, "])
@pytest.mark.parametrize(
"filename, content_type, analyzer",
diff --git a/packages/markitdown/tests/test_docintel_converter.py b/packages/markitdown/tests/test_docintel_converter.py
new file mode 100644
index 00000000..efbe8439
--- /dev/null
+++ b/packages/markitdown/tests/test_docintel_converter.py
@@ -0,0 +1,562 @@
+"""Unit tests for the DocumentIntelligenceConverter improvements.
+
+These tests exercise the converter without making any network calls. They use
+``__new__`` to bypass ``__init__`` (which would construct a real
+``DocumentIntelligenceClient``) and instead inject a mock client.
+"""
+
+import io
+from datetime import date
+from types import SimpleNamespace
+from unittest import mock
+
+import pytest
+import yaml
+from azure.ai.documentintelligence.models import AnalyzedDocument, DocumentField
+
+from markitdown._stream_info import StreamInfo
+from markitdown.converters import _doc_intel_converter as di_mod
+from markitdown.converters._doc_intel_converter import (
+ DocumentIntelligenceConverter,
+ DocumentIntelligenceFileType,
+ _USER_AGENT,
+ _field_value,
+ _fields_to_front_matter,
+ _yaml_dump,
+)
+
+# --------- helpers ---------------------------------------------------------
+
+
+def _bare_converter(
+ *,
+ file_types=None,
+ model_id="prebuilt-layout",
+ features=None,
+ query_fields=None,
+ client=None,
+):
+ """Build a converter without calling __init__ (no real DI client)."""
+ conv = DocumentIntelligenceConverter.__new__(DocumentIntelligenceConverter)
+ conv._file_types = file_types or [
+ DocumentIntelligenceFileType.PDF,
+ DocumentIntelligenceFileType.DOCX,
+ ]
+ conv._model_id = model_id
+ conv._features = list(features) if features is not None else None
+ conv._query_fields = list(query_fields) if query_fields else None
+ conv.endpoint = "https://example.cognitiveservices.azure.com/"
+ conv.api_version = "2024-11-30"
+ conv.doc_intel_client = client
+ return conv
+
+
+def _mock_field(**kwargs):
+ """A SimpleNamespace with all DocumentField value_* attrs defaulting to None."""
+ defaults = {
+ "value_string": None,
+ "value_boolean": None,
+ "value_integer": None,
+ "value_number": None,
+ "value_date": None,
+ "value_time": None,
+ "value_phone_number": None,
+ "value_country_region": None,
+ "value_selection_mark": None,
+ "value_signature": None,
+ "value_currency": None,
+ "value_address": None,
+ "value_array": None,
+ "value_object": None,
+ "content": None,
+ }
+ defaults.update(kwargs)
+ return SimpleNamespace(**defaults)
+
+
+# --------- Phase 1: API version + user agent -------------------------------
+
+
+def test_default_api_version_uses_sdk_default():
+ """The default api_version must defer to the Document Intelligence SDK."""
+ import inspect
+
+ sig = inspect.signature(DocumentIntelligenceConverter.__init__)
+ assert sig.parameters["api_version"].default is None
+
+
+def test_user_agent_string_format():
+ """User agent should start with 'markitdown-docintel/'."""
+ assert _USER_AGENT.startswith("markitdown-docintel/")
+ assert len(_USER_AGENT) > len("markitdown-docintel/")
+
+
+def test_client_constructed_with_user_agent_and_sdk_default_api_version():
+ """__init__ should pass user_agent without overriding the SDK API version."""
+ fake_client = mock.MagicMock()
+ with mock.patch.object(
+ di_mod, "DocumentIntelligenceClient", return_value=fake_client
+ ) as ctor:
+ DocumentIntelligenceConverter(
+ endpoint="https://example.cognitiveservices.azure.com/",
+ credential=mock.MagicMock(),
+ )
+ kwargs = ctor.call_args.kwargs
+ assert "api_version" not in kwargs
+ assert kwargs["user_agent"] == _USER_AGENT
+
+
+# --------- Phase 2: configurable model_id ----------------------------------
+
+
+def test_default_model_id():
+ """Default model_id preserves existing behavior."""
+ import inspect
+
+ sig = inspect.signature(DocumentIntelligenceConverter.__init__)
+ assert sig.parameters["model_id"].default == "prebuilt-layout"
+
+
+def test_convert_uses_default_model_id():
+ """Without overrides, convert() calls begin_analyze_document with prebuilt-layout."""
+ fake_poller = mock.MagicMock()
+ fake_poller.result.return_value = SimpleNamespace(content="# hi", documents=None)
+ client = mock.MagicMock()
+ client.begin_analyze_document.return_value = fake_poller
+
+ conv = _bare_converter(client=client)
+ conv.convert(
+ io.BytesIO(b"data"), StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ args, kwargs = client.begin_analyze_document.call_args
+ assert kwargs["model_id"] == "prebuilt-layout"
+
+
+def test_convert_uses_overridden_model_id():
+ fake_poller = mock.MagicMock()
+ fake_poller.result.return_value = SimpleNamespace(content="# hi", documents=None)
+ client = mock.MagicMock()
+ client.begin_analyze_document.return_value = fake_poller
+
+ conv = _bare_converter(model_id="prebuilt-invoice", client=client)
+ conv.convert(
+ io.BytesIO(b"data"), StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ assert (
+ client.begin_analyze_document.call_args.kwargs["model_id"] == "prebuilt-invoice"
+ )
+
+
+# --------- Phase 3: configurable features ---------------------------------
+
+
+def test_no_features_are_enabled_by_default():
+ conv = _bare_converter()
+ features = conv._analysis_features(
+ StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ assert features == []
+
+
+def test_empty_features_disable_all_addons():
+ conv = _bare_converter(features=[])
+
+ assert (
+ conv._analysis_features(
+ StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+ == []
+ )
+
+
+def test_explicit_features_are_copied():
+ configured = [di_mod.DocumentAnalysisFeature.OCR_HIGH_RESOLUTION]
+ conv = _bare_converter(features=configured)
+
+ resolved = conv._analysis_features(
+ StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+ resolved.append(di_mod.DocumentAnalysisFeature.STYLE_FONT)
+
+ assert configured == [di_mod.DocumentAnalysisFeature.OCR_HIGH_RESOLUTION]
+ assert conv._features == [di_mod.DocumentAnalysisFeature.OCR_HIGH_RESOLUTION]
+
+
+def test_explicit_features_are_not_sent_for_office_types():
+ conv = _bare_converter(features=[di_mod.DocumentAnalysisFeature.STYLE_FONT])
+
+ assert (
+ conv._analysis_features(
+ StreamInfo(
+ extension=".docx",
+ mimetype="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+ )
+ )
+ == []
+ )
+
+
+def test_query_fields_extend_explicit_features_without_mutating_them():
+ configured = [di_mod.DocumentAnalysisFeature.OCR_HIGH_RESOLUTION]
+ conv = _bare_converter(features=configured, query_fields=["VendorName"])
+
+ features = conv._analysis_features(
+ StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ assert features == [
+ di_mod.DocumentAnalysisFeature.OCR_HIGH_RESOLUTION,
+ di_mod.DocumentAnalysisFeature.QUERY_FIELDS,
+ ]
+ assert configured == [di_mod.DocumentAnalysisFeature.OCR_HIGH_RESOLUTION]
+
+
+# --------- Phase 4: YAML front matter --------------------------------------
+
+
+def test_field_value_typed_scalars():
+ assert _field_value(_mock_field(value_string="Contoso")) == "Contoso"
+ assert _field_value(_mock_field(value_integer=42)) == 42
+ assert _field_value(_mock_field(value_number=12.5)) == 12.5
+ assert _field_value(_mock_field(value_boolean=True)) is True
+ assert _field_value(_mock_field(value_date=date(2026, 3, 15))) == "2026-03-15"
+
+
+def test_field_value_currency():
+ cur = SimpleNamespace(amount=1250.0, currency_code="USD", currency_symbol="$")
+ assert _field_value(_mock_field(value_currency=cur)) == "1250.0 USD"
+
+
+def test_field_value_falls_back_to_content():
+ assert _field_value(_mock_field(content="raw text")) == "raw text"
+
+
+def test_field_value_array_of_scalars():
+ items = [_mock_field(value_string="A"), _mock_field(value_string="B")]
+ assert _field_value(_mock_field(value_array=items)) == ["A", "B"]
+
+
[email protected]("content", [None, ":selected: :unselected:"])
+def test_selection_group_preserves_selected_values_in_front_matter(content):
+ field = DocumentField(
+ type="selectionGroup", value_selection_group=["Email", "SMS"], content=content
+ )
+ doc = AnalyzedDocument(
+ doc_type="custom", fields={"ContactPreference": field}, spans=[], confidence=1.0
+ )
+
+ assert _field_value(field) == ["Email", "SMS"]
+ assert _fields_to_front_matter([doc]) == (
+ "---\nfields:\n ContactPreference:\n - Email\n - SMS\n---\n\n"
+ )
+
+
+def test_empty_selection_group_does_not_fall_back_to_content():
+ field = DocumentField(
+ type="selectionGroup", value_selection_group=[], content=":unselected:"
+ )
+ assert _field_value(field) == []
+
+
+def test_front_matter_preserves_separate_documents():
+ documents = [
+ AnalyzedDocument(
+ doc_type="invoice",
+ fields={
+ "VendorName": DocumentField(type="string", value_string="Vendor A"),
+ "InvoiceTotal": DocumentField(type="number", value_number=100.0),
+ },
+ spans=[],
+ confidence=1.0,
+ ),
+ AnalyzedDocument(
+ doc_type="invoice",
+ fields={
+ "VendorName": DocumentField(type="string", value_string="Vendor B"),
+ "InvoiceId": DocumentField(type="string", value_string="INV-B"),
+ },
+ spans=[],
+ confidence=1.0,
+ ),
+ ]
+
+ assert _fields_to_front_matter(documents, "prebuilt-invoice") == (
+ "---\n"
+ "modelId: prebuilt-invoice\n"
+ "documents:\n"
+ " -\n"
+ " fields:\n"
+ " VendorName: Vendor A\n"
+ " InvoiceTotal: 100.0\n"
+ " -\n"
+ " fields:\n"
+ " VendorName: Vendor B\n"
+ " InvoiceId: INV-B\n"
+ "---\n\n"
+ )
+
+
+def test_front_matter_preserves_empty_document_positions():
+ empty = SimpleNamespace(fields={})
+ populated = SimpleNamespace(fields={"Name": _mock_field(value_string="Contoso")})
+
+ assert _fields_to_front_matter([empty, populated, empty]) == (
+ "---\n"
+ "documents:\n"
+ " -\n"
+ " fields: {}\n"
+ " -\n"
+ " fields:\n"
+ " Name: Contoso\n"
+ " -\n"
+ " fields: {}\n"
+ "---\n\n"
+ )
+ assert _fields_to_front_matter([empty, empty]) == ""
+
+
+def test_fields_to_front_matter_empty_when_no_documents():
+ assert _fields_to_front_matter(None) == ""
+ assert _fields_to_front_matter([]) == ""
+
+
+def test_fields_to_front_matter_empty_when_no_fields():
+ doc = SimpleNamespace(fields={})
+ assert _fields_to_front_matter([doc]) == ""
+
+
+def test_fields_to_front_matter_basic():
+ doc = SimpleNamespace(
+ fields={
+ "VendorName": _mock_field(value_string="Contoso Ltd."),
+ "InvoiceTotal": _mock_field(value_number=1250.0),
+ }
+ )
+ fm = _fields_to_front_matter([doc], model_id="prebuilt-invoice")
+ assert fm.startswith("---\n")
+ assert fm.endswith("---\n\n")
+ assert "modelId: prebuilt-invoice" in fm
+ assert "fields:" in fm
+ assert " VendorName: Contoso Ltd." in fm
+ assert " InvoiceTotal: 1250.0" in fm
+
+
+def test_fields_to_front_matter_omits_model_id_when_not_provided():
+ doc = SimpleNamespace(fields={"X": _mock_field(value_string="y")})
+ fm = _fields_to_front_matter([doc])
+ assert "modelId:" not in fm
+ assert "fields:" in fm
+
+
+def test_fields_with_special_chars_are_quoted():
+ doc = SimpleNamespace(
+ fields={"Note": _mock_field(value_string="line1\nline2: with colon")}
+ )
+ fm = _fields_to_front_matter([doc])
+ # Value contains both \n and ':' so it must be quoted.
+ assert ' Note: "line1\\nline2: with colon"' in fm
+
+
+ "text",
+ [
+ "page one\fpage two",
+ "back\bspace",
+ "null\x00byte",
+ "vertical\vtab",
+ "control\x1fend",
+ "delete\x7fcharacter",
+ "next\x85line",
+ "control\x9fend",
+ "line\u2028separator",
+ "paragraph\u2029separator",
+ "noncharacter\ufffe",
+ "noncharacter\uffff",
+ "tab\tnewline\nreturn\r",
+ 'quote"backslash\\',
+ "café 😀",
+ "00123",
+ "[1, 2]",
+ "- pending",
+ "+12065550100",
+ "on",
+ "1e-7",
+ "1e20",
+ ],
+)
+def test_front_matter_string_keys_and_values_round_trip(text):
+ doc = AnalyzedDocument(
+ doc_type="custom",
+ fields={text: DocumentField(type="string", value_string=text)},
+ spans=[],
+ confidence=1.0,
+ )
+
+ front_matter = _fields_to_front_matter([doc])
+ assert front_matter.startswith("---\n")
+ assert front_matter.endswith("---\n\n")
+ assert yaml.safe_load(front_matter[4:-5]) == {"fields": {text: text}}
+
+
+ "value",
+ [
+ 1e-7,
+ -1e-7,
+ 1e20,
+ -1e20,
+ 1.25e-7,
+ 5e-324,
+ 1.7976931348623157e308,
+ 1.25,
+ 0.0,
+ -0.0,
+ ],
+)
+def test_front_matter_floats_round_trip(value):
+ doc = AnalyzedDocument(
+ doc_type="custom",
+ fields={"Measurement": DocumentField(type="number", value_number=value)},
+ spans=[],
+ confidence=1.0,
+ )
+
+ front_matter = _fields_to_front_matter([doc])
+ parsed = yaml.safe_load(front_matter[4:-5])["fields"]["Measurement"]
+ assert type(parsed) is float
+ assert repr(parsed) == repr(value)
+
+
+def test_yaml_dump_nested_dict():
+ out = _yaml_dump({"a": 1, "b": {"c": "x"}})
+ assert "a: 1" in out
+ assert "b:" in out
+ assert " c: x" in out
+
+
+def test_convert_prepends_front_matter_when_fields_present():
+ doc = SimpleNamespace(fields={"VendorName": _mock_field(value_string="Contoso")})
+ fake_poller = mock.MagicMock()
+ fake_poller.result.return_value = SimpleNamespace(
+ content="# Invoice\n\nbody", documents=[doc]
+ )
+ client = mock.MagicMock()
+ client.begin_analyze_document.return_value = fake_poller
+
+ conv = _bare_converter(model_id="prebuilt-invoice", client=client)
+ result = conv.convert(
+ io.BytesIO(b"data"), StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ assert result.markdown.startswith("---\n")
+ assert "modelId: prebuilt-invoice" in result.markdown
+ assert " VendorName: Contoso" in result.markdown
+ assert "# Invoice" in result.markdown
+
+
+def test_convert_no_front_matter_when_no_documents():
+ fake_poller = mock.MagicMock()
+ fake_poller.result.return_value = SimpleNamespace(
+ content="# Layout", documents=None
+ )
+ client = mock.MagicMock()
+ client.begin_analyze_document.return_value = fake_poller
+
+ conv = _bare_converter(client=client)
+ result = conv.convert(
+ io.BytesIO(b"data"), StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ assert not result.markdown.startswith("---")
+ assert result.markdown.startswith("# Layout")
+
+
+# --------- Phase 5: query fields -------------------------------------------
+
+
+def test_query_fields_adds_feature_for_ocr_types():
+ conv = _bare_converter(query_fields=["VendorName", "Total"])
+ features = conv._analysis_features(
+ StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+ from azure.ai.documentintelligence.models import DocumentAnalysisFeature
+
+ assert DocumentAnalysisFeature.QUERY_FIELDS in features
+
+
+def test_query_fields_skipped_for_office_types():
+ conv = _bare_converter(query_fields=["VendorName"])
+ features = conv._analysis_features(
+ StreamInfo(
+ extension=".docx",
+ mimetype="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+ )
+ )
+ # Office types skip OCR features entirely.
+ assert features == []
+
+
+def test_query_fields_passed_to_begin_analyze_document_for_pdf():
+ fake_poller = mock.MagicMock()
+ fake_poller.result.return_value = SimpleNamespace(content="x", documents=None)
+ client = mock.MagicMock()
+ client.begin_analyze_document.return_value = fake_poller
+
+ conv = _bare_converter(query_fields=["A", "B"], client=client)
+ conv.convert(
+ io.BytesIO(b"data"), StreamInfo(extension=".pdf", mimetype="application/pdf")
+ )
+
+ assert client.begin_analyze_document.call_args.kwargs.get("query_fields") == [
+ "A",
+ "B",
+ ]
+
+
+def test_query_fields_not_passed_for_office_types():
+ fake_poller = mock.MagicMock()
+ fake_poller.result.return_value = SimpleNamespace(content="x", documents=None)
+ client = mock.MagicMock()
+ client.begin_analyze_document.return_value = fake_poller
+
+ conv = _bare_converter(query_fields=["A"], client=client)
+ conv.convert(
+ io.BytesIO(b"data"),
+ StreamInfo(
+ extension=".docx",
+ mimetype="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+ ),
+ )
+
+ assert "query_fields" not in client.begin_analyze_document.call_args.kwargs
+
+
+# --------- _markitdown.py wiring -------------------------------------------
+
+
+def test_markitdown_forwards_docintel_kwargs(monkeypatch):
+ """MarkItDown(...) should forward Document Intelligence configuration."""
+ from markitdown import _markitdown as md_mod
+
+ captured = {}
+
+ class _Fake:
+ def __init__(self, **kwargs):
+ captured.update(kwargs)
+
+ monkeypatch.setattr(md_mod, "DocumentIntelligenceConverter", _Fake)
+
+ md_mod.MarkItDown(
+ docintel_endpoint="https://example.cognitiveservices.azure.com/",
+ docintel_model_id="prebuilt-invoice",
+ docintel_features=[],
+ docintel_query_fields=["A", "B"],
+ )
+
+ assert captured.get("endpoint") == "https://example.cognitiveservices.azure.com/"
+ assert captured.get("model_id") == "prebuilt-invoice"
+ assert captured.get("features") == []
+ assert captured.get("query_fields") == ["A", "B"]