Graph-Native Infrastructure for Context and Accountable AI Systems
Python agent-memoryaiai-governanceai-infrastructureartificial-intelligencecontext-engineeringcontext-graphsdata-engineeringdecision-intelligencedeveloper-toolsexplainable-aigenerative-aigraph-ragknowledge-graphllmontologyprovenancepythonreasoningsemantic-search
docs(parse): improve onboarding and align with implementation
Sameer63058cebb052c8d6
diff --git a/docs/reference/parse.md b/docs/reference/parse.mdindex 68e689ab..c6dda9d2 100644--- a/docs/reference/parse.md+++ b/docs/reference/parse.md@@ -6,14 +6,130 @@ icon: "file-lines"`semantica.parse` extracts structured text, layout, tables, and metadata from unstructured documents. `DocumentParser` handles clean machine-readable files; `DoclingParser` handles complex layouts, scanned PDFs, and multi-column documents.+## Getting Started++### Installation++The parse module works out of the box for standard formats:++```python+from semantica.parse import DocumentParser++parser = DocumentParser()+result = parser.parse("document.pdf")+print(result["full_text"]) # Extracted text content+```++For enhanced table extraction and complex layouts, install the Docling dependency:++```bash+pip install docling+```++```python+from semantica.parse import DoclingParser++parser = DoclingParser(export_format="markdown")+result = parser.parse("document.pdf", extract_tables=True)+print(result["tables"]) # Enhanced table extraction+```++### First Document Parsing++```python+from semantica.parse import DocumentParser++# Parse any supported format+parser = DocumentParser()+result = parser.parse("annual_report.pdf")++# Access extracted content+text = result["full_text"] # Complete document text+metadata = result["metadata"] # Document properties+pages = result.get("pages", []) # Page-level content++print(f"Extracted {len(text)} characters from {metadata.get('page_count', 0)} pages")+```++## Parser Selection Guide++### DocumentParser+- **Best for**: Clean PDFs, Word docs, HTML, plain text+- **Formats**: PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX+- **Strengths**: Fast processing, broad format support, no dependencies++### DoclingParser+- **Best for**: Complex layouts, merged-cell tables, scanned documents+- **Formats**: PDF, DOCX, PPTX, XLSX, HTML, images+- **Strengths**: Superior table extraction, OCR support, multi-column handling+- **Requirements**: `pip install docling`++**Simple rule**: Start with `DocumentParser`. Use `DoclingParser` when you need better table extraction or handle complex document layouts.++## Common Workflows++### Single Document Parsing++```python+from semantica.parse import DocumentParser++parser = DocumentParser()+result = parser.parse("contract.pdf")++# Check what was extracted+print(f"Text length: {len(result['full_text'])}")+print(f"Metadata: {result['metadata']}")+if "tables" in result:+ print(f"Tables found: {len(result['tables'])}")+```++### Batch Document Processing++```python+from semantica.parse import DocumentParser++parser = DocumentParser()+files = ["doc1.pdf", "doc2.docx", "doc3.html"]++# Process multiple files+results = parser.parse_batch(files, continue_on_error=True)++print(f"Successfully parsed: {results['success_count']}/{results['total']}")+for item in results["successful"]:+ file_path = item["file_path"]+ content = item["result"]["full_text"]+ print(f"{file_path}: {len(content)} characters")+```++### Enhanced Table Extraction++```python+from semantica.parse import DoclingParser++parser = DoclingParser(export_format="markdown")+result = parser.parse(+ "financial_report.pdf",+ extract_tables=True,+ extract_text=True+)++# Access structured table data+for i, table in enumerate(result["tables"]):+ print(f"Table {i+1}: {table['row_count']} rows, {table['col_count']} columns")+ print(f"Page: {table['page_number']}")++ # Table data is in rows format+ for row in table["rows"][:3]: # First 3 rows+ print(" | ".join(row))+```+## Exported Classes| Class | Role || --- | --- || `DocumentParser` | Auto-detects format — delegates to format-specific parser (PDF, DOCX, HTML, JSON, CSV, ...) |-| `DoclingParser` | Complex layouts, merged-cell tables, multi-column PDFs, and OCR (`pip install semantica[docling]`) |-| `ParsedDocument` | `{text, sections, tables, metadata, source_id}` — structured output from any parser |-| `DocumentMetadata` | `{title, author, created_date, page_count, language, word_count}` |+| `DoclingParser` | Complex layouts, merged-cell tables, multi-column PDFs, and OCR (`pip install docling`) |+| `DoclingMetadata` | Document metadata from Docling parsing || `PDFParser` | PDF text and metadata extraction || `WebParser` | URL fetch + HTML parsing || `EmailParser` | `.eml` / `.msg` email files with attachment extraction |@@ -27,11 +143,12 @@ Standard parser for clean, machine-readable documents:from semantica.parse import DocumentParserparser = DocumentParser()-parsed = parser.parse("data/report.pdf")+result = parser.parse("data/report.pdf")-print(parsed.text) # full clean text-print(parsed.metadata) # title, author, date, page_count, language, etc.-print(parsed.sections) # document structure as a list of Section objects+print(result["full_text"]) # Complete extracted text+print(result["metadata"]) # Document properties (title, author, page_count, etc.)+if "pages" in result: # Page-level content (when available)+ print(f"Pages: {len(result['pages'])}")```Supported formats: PDF, DOCX, HTML, TXT, JSON, CSV, PPTX, XLSX.@@ -48,16 +165,21 @@ pip install "semantica[docling]"from semantica.parse import DoclingParserparser = DoclingParser(- extract_tables=True, # structured table extraction with cell type detection- extract_images=True, # extract image regions for downstream OCR- output_format="markdown", # "markdown" | "html" | "json"+ export_format="markdown", # Export format: "markdown" | "html" | "json"+ enable_ocr=False # Enable OCR for scanned documents)-parsed = parser.parse("data/annual_report.pdf")+result = parser.parse(+ "data/annual_report.pdf",+ extract_tables=True, # Extract structured tables+ extract_images=False, # Extract image regions+ extract_text=True # Extract text content+)-print(parsed.text) # full clean text-print(parsed.tables) # structured TableData objects with headers and rows-print(parsed.sections) # document structure with heading hierarchy+print(result["full_text"]) # Complete extracted text+print(result["tables"]) # Structured table data+if "pages" in result: # Page-level content+ print(f"Pages: {len(result['pages'])}")```Use `DoclingParser` for:@@ -73,12 +195,12 @@ Use `DoclingParser` for:```pythonparser = DoclingParser(- ocr=True,- ocr_language=["en"], # ISO 639-1 codes; list for multi-language documents- extract_tables=True,+ enable_ocr=True, # Enable OCR via PdfPipelineOptions+ export_format="markdown")-parsed = parser.parse("data/scanned_contract.pdf")+result = parser.parse("data/scanned_contract.pdf")+print(result["full_text"]) # OCR-extracted text```## Supported Formats@@ -99,39 +221,41 @@ parsed = parser.parse("data/scanned_contract.pdf")| Archive | `.zip`, `.tar` | `FileIngestor` | Recursive extraction || Source code | `.py`, `.js`, `.java`, ... | `CodeParser` | AST-aware block detection |-## Parsed Document Object+## Parser Output Structure++Both parsers return dictionaries with the following structure:++```python+result = {+ "full_text": str, # Complete extracted text+ "metadata": dict, # Document properties and statistics+ "pages": List[dict], # Page-level content (when available)+ "tables": List[dict], # Structured table data (DoclingParser)+ "images": List[dict], # Image regions (DoclingParser)+ "total_pages": int, # Total page count+ "export_format": str # Format used for text extraction (DoclingParser)+}+```-Both parsers return a `ParsedDocument` with the same structure:+### Metadata Structure```python-@dataclass-class ParsedDocument:- text: str # full extracted text- sections: List[Section] # heading-based document structure- tables: List[TableData] # structured table data (DoclingParser only)- metadata: DocumentMetadata # title, author, dates, page count- source_id: str # links back to the original DataSource--@dataclass-class DocumentMetadata:- title: Optional[str]- author: Optional[str]- created_date: Optional[datetime]- page_count: int- language: Optional[str] # ISO 639-1 code- has_tables: bool- has_images: bool- word_count: int- format: str # "pdf" | "docx" | "pptx" | ...+metadata = {+ "file_path": str, # Source file path+ "page_count": int, # Number of pages+ "format": str, # File format ("pdf", "docx", etc.)+ # Additional fields vary by parser and document type+}```## DocumentParser Methods| Method | Returns | Description || ------ | ------- | ----------- |-| `parse(source)` | `ParsedDocument` | Auto-detect format and extract text, sections, metadata |-| `parse_batch(sources)` | `List[ParsedDocument]` | Process multiple sources in parallel |-| `is_supported(path)` | `bool` | Check if the file extension is supported |+| `parse(source)` | `dict` | Auto-detect format and extract text, metadata, tables |+| `parse_batch(sources)` | `dict` | Process multiple sources in parallel |+| `extract_text(path)` | `str` | Extract only text content from document |+| `extract_metadata(path)` | `dict` | Extract only metadata from document |## Integration with FileIngestor@@ -142,16 +266,19 @@ from semantica.ingest import FileIngestorfrom semantica.parse import DoclingParseringestor = FileIngestor()-parser = DoclingParser(extract_tables=True)+parser = DoclingParser(export_format="markdown")sources = ingestor.ingest("data/reports/")for source in sources:- parsed = parser.parse(source)- # → parsed.text, parsed.tables, parsed.sections+ result = parser.parse(source)+ # Access extracted content+ text = result["full_text"]+ tables = result["tables"]+ metadata = result["metadata"]```<Note>- Docling is an optional dependency. If `docling` is not installed, `DoclingParser` raises an `ImportError` with installation instructions. `DocumentParser` is always available and requires no extras.+ Docling is an optional dependency. If `docling` is not installed, `DoclingParser` raises an `ImportError` with installation instructions: `pip install docling`. `DocumentParser` is always available and requires no extras.</Note><CardGroup cols={2}>