Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Merge SearchSourceContent into DocumentContent (#768)
Louis Choquelcc408ed4b332
diff --git a/docs/building-methods/concepts/native-concepts.md b/docs/building-methods/concepts/native-concepts.mdindex 331adb396..3effe97a9 100644--- a/docs/building-methods/concepts/native-concepts.md+++ b/docs/building-methods/concepts/native-concepts.md@@ -27,7 +27,7 @@ Here are all the native concepts you can use out of the box:|-----------------|-------------|---------------------|| `Text` | A text | `TextContent` || `Image` | An image | `ImageContent` |-| `Document` | A document (PDF, DOCX, PPTX) | `DocumentContent` |+| `Document` | A document (PDF, DOCX, PPTX, web page) | `DocumentContent` || `TextAndImages` | Text with its associated images | `TextAndImagesContent` || `Number` | A number | `NumberContent` || `Page` | A document page with text, images, and optional page view | `PageContent` |@@ -80,15 +80,23 @@ Represents a document (PDF, DOCX, PPTX, etc.):```pythonclass DocumentContent(StuffContent):url: str+ public_url: str | None = Nonemime_type: str | None = None+ filename: str | None = None+ title: str | None = None+ snippet: str | None = None```**Fields:**-- `url`: Location of the document file (file path or URL)+- `url`: Location of the document file, storage URL, or web page URL+- `public_url`: Optional public-facing URL (when `url` is a private/internal reference)- `mime_type`: Optional MIME type of the document (e.g., "application/pdf", "application/vnd.openxmlformats-officedocument.wordprocessingml.document")+- `filename`: Optional filename of the document+- `title`: Optional title of the document or source+- `snippet`: Optional text snippet or excerpt from the document-**Use for:** Contracts, invoices, reports, presentations, any document file.+**Use for:** Contracts, invoices, reports, presentations, web pages, search source citations, any document file.### NumberContent@@ -188,16 +196,13 @@ Represents the result of a web search query. Produced by `PipeSearch`:```pythonclass SearchResultContent(StuffContent):answer: str- sources: list[SearchSourceContent] = []+ sources: list[DocumentContent] = []```**Fields:**- `answer`: The synthesized answer text from the search-- `sources`: A list of source citations, each containing:- - `name` (str): Source name/title- - `url` (str): Source URL- - `snippet` (str | None): Relevant excerpt from the source+- `sources`: A list of `DocumentContent` source citations (each with `url`, `title`, and `snippet`)**Use for:** Web search results, research findings, information retrieval with citations.diff --git a/docs/building-methods/pipes/pipe-operators/PipeExtract.md b/docs/building-methods/pipes/pipe-operators/PipeExtract.mdindex d56309d51..7994536a0 100644--- a/docs/building-methods/pipes/pipe-operators/PipeExtract.md+++ b/docs/building-methods/pipes/pipe-operators/PipeExtract.md@@ -1,14 +1,14 @@----description: "PipeExtract performs OCR on images and PDFs to extract text and embedded visuals. Turn documents into structured data for your AI methods."+description: "PipeExtract extracts structured content from documents (PDF, images, web pages). Turn documents into structured data for your AI methods."---# PipeExtract-The `PipeExtract` operator performs Optical Character Recognition (OCR) on images and PDF documents. It extracts text, embedded images, and provides full-page renderings.+The `PipeExtract` operator extracts structured content from documents. For PDFs and images, it performs OCR to extract text, embedded images, and full-page renderings. For web pages, it fetches and extracts page content.## How it works-`PipeExtract` takes a single input, which must be a `Document` or an `Image` (or a concept that refines one of them). It processes the input and produces a list of `PageContent` objects. Each `PageContent` object encapsulates all the information extracted from one page.+`PipeExtract` takes a single input, which must be a `Document` or an `Image` (or a concept that refines one of them). The document URL can be a file path, a storage URL, or a web page URL. It processes the input and produces a list of `PageContent` objects. Each `PageContent` object encapsulates all the information extracted from one page.The output is always a list. If the input is a single image, the output still contains one `Page` item.@@ -25,30 +25,37 @@ The `PageContent` object has the following structure:`PipeExtract` is configured in your pipeline's `.mthds` file.-### OCR Models and Backend System+### Extraction Models and Backend System-PipeExtract uses the unified inference backend system to manage OCR models. This means you can:+PipeExtract uses the unified inference backend system to manage extraction models. This means you can:-- Use different OCR providers (Mistral OCR, local PDF extraction via internal backend, etc.)-- Configure OCR models through the same backend system as LLMs and image generation models-- Use OCR presets for consistent configurations across your pipelines-- Route OCR requests to different backends based on your routing profile+- Use different extraction providers (Mistral OCR, Linkup Fetch, local PDF extraction via internal backend, etc.)+- Configure extraction models through the same backend system as LLMs and image generation models+- Use extraction presets for consistent configurations across your pipelines+- Route extraction requests to different backends based on your routing profile-Common OCR model handles:+Common extraction model handles:- `mistral-ocr`: Mistral's OCR model for high-quality text and image extraction- `pypdfium2-extract-pdf`: Local PDF text extraction (no API calls required)+- `linkup-fetch`: Web page content extraction from URLs-OCR presets are defined in your model deck configuration and can include parameters like `max_nb_images` and `image_min_size`.+Common model aliases:++- `@default-extract-web-page`: Default for web page extraction (Linkup Fetch)+- `@default-extract-document`: Default for document extraction (Azure Document Intelligence)+- `@default-text-from-pdf`: Fast local PDF text extraction (pypdfium2)++Extraction presets are defined in your model deck configuration and can include parameters like `max_nb_images` and `image_min_size`.### MTHDS Parameters| Parameter | Type | Description | Required || --------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------- || `type` | string | The type of the pipe: `PipeExtract` | Yes |-| `description` | string | A description of the OCR operation. | Yes |-| `inputs` | Fixed | The value must be of concept `Document` or `Image` (or a concept that refines one of them). | Yes |-| `output` | string | The output concept produced by the OCR operation. Use `Page[]`. | Yes |+| `description` | string | A description of the extraction operation. | Yes |+| `inputs` | Fixed | The value must be of concept `Document` or `Image` (or a concept that refines one of them). For web page extraction, the Document holds a web URL. | Yes |+| `output` | string | The output concept produced by the extraction operation. Use `Page[]`. | Yes || `max_page_images` | integer or null | Maximum number of images to extract from pages: `null` (or omit) for unlimited, `0` for no images, or a positive integer to limit. Defaults to the value in your Extract model preset. | No || `page_views` | boolean | If `true`, a high-fidelity image of each page will be included in the `page_view` field. Defaults to `false`. | No || `page_views_dpi` | integer | The resolution (in Dots Per Inch) for the generated page views when processing a PDF. Defaults to `150`. | No |@@ -80,6 +87,21 @@ The output of `PipeExtract` must be `Page[]`.To use this pipe, first load a PDF into the `ScannedDocument` concept. After the pipe runs, the output contains a list of `PageContent` objects, where each object has the extracted text and a 200 DPI image of the corresponding page.+### Example: Extracting a Web Page++This example defines a pipe that extracts content from a web page URL.++```toml+[pipe.extract_web_article]+type = "PipeExtract"+description = "Extract content from a web page"+inputs = { article_url = "Document" }+output = "Page[]"+model = "@default-extract-web-page"+```++Pass a web URL as the `document_uri` when running the pipe. PipeExtract fetches the page and extracts its content into `Page` objects, following the same pattern as document extraction.+## Related Documentation- [Generic Document Extraction Example](../../../cookbook/extract-generic.md) - Extract markdown from complex PDFs using visiondiff --git a/docs/building-methods/pipes/pipe-operators/PipeSearch.md b/docs/building-methods/pipes/pipe-operators/PipeSearch.mdindex f0a85758a..b4edb8e62 100644--- a/docs/building-methods/pipes/pipe-operators/PipeSearch.md+++ b/docs/building-methods/pipes/pipe-operators/PipeSearch.md@@ -13,7 +13,7 @@ The `PipeSearch` operator searches the web using a configurable search providerThe output is a `SearchResult` (or a concept that refines it), which contains:- `answer`: The synthesized answer text from the search-- `sources`: A list of sources, each with a `name`, `url`, and optional `snippet`+- `sources`: A list of sources, each with a `title`, `url`, and optional `snippet`## Configuration@@ -106,7 +106,7 @@ from_date = "2026-01-01"include_domains = ["reuters.com", "apnews.com", "bbc.com"]```-When the output concept is compatible with `SearchResult`, the output contains the synthesized `answer` and a list of `sources` with their names, URLs, and snippets. PipeSearch can also feed a structured output concept when your method needs a more specialized downstream shape.+When the output concept is compatible with `SearchResult`, the output contains the synthesized `answer` and a list of `sources` with their titles, URLs, and snippets. PipeSearch can also feed a structured output concept when your method needs a more specialized downstream shape.## Related Documentationdiff --git a/docs/features/document-extraction.md b/docs/features/document-extraction.mdindex dfc1d6285..e64f04aa6 100644--- a/docs/features/document-extraction.md+++ b/docs/features/document-extraction.md@@ -1,17 +1,17 @@---title: "Document Extraction"-description: "Multi-provider OCR and document processing with Mistral OCR, Azure Document Intelligence, docling, and Deepseek-OCR. PDF processing, layout analysis, and table recognition."+description: "Multi-provider OCR and document processing with Mistral OCR, Azure Document Intelligence, docling, Deepseek-OCR, and Linkup Fetch. PDF processing, web page extraction, layout analysis, and table recognition."---# Document Extraction-Multi-provider OCR and document processing with a unified interface.+Multi-provider OCR, document processing, and web content extraction with a unified interface.## OverviewFrom simple text extraction to advanced document understanding — Pipelex handles it all. Basic PDF text extraction works out of the box (via pypdfium2), but real documents demand more: OCR for scanned pages, layout analysis for complex structures, image extraction, and VLM-powered understanding.-Unlike LLM APIs (partly standardized around OpenAI's completions API), the OCR landscape is fragmented. Pipelex solves this with a unified interface: swap providers by changing your PipeExtract config, no code changes required.+Unlike LLM APIs (partly standardized around OpenAI's completions API), the OCR landscape is fragmented. Pipelex solves this with a unified interface: swap providers by changing your PipeExtract config, no code changes required. For web pages, Linkup Fetch extracts content directly from URLs using the same PipeExtract pattern.## Supported Providers@@ -22,6 +22,7 @@ Unlike LLM APIs (partly standardized around OpenAI's completions API), the OCR l| **docling** | Local SDK | IBM's open-source extraction library with local CPU processing and optional GPU acceleration || **Azure Document Intelligence** | Gateway | Enterprise-grade OCR with high accuracy for complex layouts, tables, and handwriting || **Deepseek-OCR** | Gateway | Open-source model optimized for markdown extraction from images |+| **Linkup Fetch** | Cloud API | Web page content extraction — fetches and extracts text from web URLs |## Key Capabilities@@ -31,10 +32,11 @@ Unlike LLM APIs (partly standardized around OpenAI's completions API), the OCR l- **Table recognition** — Automatic table detection and extraction- **Handwriting support** — Via providers that support handwriting recognition (e.g., Azure Document Intelligence)- **Multi-page processing** — Batch processing of document pages with per-page results+- **Web page extraction** — Fetch and extract content from web page URLs via Linkup Fetch## Documents in LLM Prompts-Include PDFs directly in your prompts using `@variable` syntax. PipeLLM automatically handles document rendering — single documents, multiple documents, and mixed content combining text, images, and PDFs are all supported.+Include PDFs directly in your prompts using `@variable` syntax. PipeLLM automatically handles document rendering — single documents, multiple documents, and mixed content combining text, images, and PDFs are all supported. Web page content extracted via PipeExtract follows the same pattern.## Related Documentationdiff --git a/pipelex/cli/agent_cli/commands/pipe_cmd.py b/pipelex/cli/agent_cli/commands/pipe_cmd.pyindex b4f233dcd..c25c110b2 100644--- a/pipelex/cli/agent_cli/commands/pipe_cmd.py+++ b/pipelex/cli/agent_cli/commands/pipe_cmd.py@@ -54,6 +54,7 @@ EXTRACT_TALENT_TO_MODEL: dict[str, str] = {"pdf-basic-text-extractor": "@default-text-from-pdf","image-text-extractor": "@default-extract-image","full-document-extractor": "@default-extract-document",+ "web-page-extractor": "@default-extract-web-page",}# Reverse mappings: preset name → talent name (for fuzzy resolution)diff --git a/pipelex/core/registry_models.py b/pipelex/core/registry_models.pyindex 6b6f88baa..50b7fe5e1 100644--- a/pipelex/core/registry_models.py+++ b/pipelex/core/registry_models.py@@ -10,7 +10,7 @@ from pipelex.core.stuffs.json_content import JSONContentfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.number_content import NumberContentfrom pipelex.core.stuffs.page_content import PageContent-from pipelex.core.stuffs.search_result_content import SearchResultContent, SearchSourceContent+from pipelex.core.stuffs.search_result_content import SearchResultContentfrom pipelex.core.stuffs.structured_content import StructuredContentfrom pipelex.core.stuffs.stuff import Stufffrom pipelex.core.stuffs.stuff_content import StuffContent@@ -88,7 +88,6 @@ class CoreRegistryModels(RegistryModels):PageContent,JSONContent,SearchResultContent,- SearchSourceContent,]EXPERIMENTAL: ClassVar[list[ModelType]] = [diff --git a/pipelex/core/stuffs/document_content.py b/pipelex/core/stuffs/document_content.pyindex 74543abf4..bd86f002f 100644--- a/pipelex/core/stuffs/document_content.py+++ b/pipelex/core/stuffs/document_content.py@@ -1,10 +1,12 @@from pydantic import Field, model_validator+from rich.text import Textfrom typing_extensions import overridefrom pipelex.cogt.templating.template_category import TemplateCategoryfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.tools.jinja2.jinja2_rendering import render_jinja2_syncfrom pipelex.tools.misc.http_utils import validate_url_resource_exists+from pipelex.tools.misc.pretty import PrettyPrintablefrom pipelex.tools.uri.uri_resolver import extract_filename_from_uri, resolve_urifrom pipelex.types import Self@@ -15,6 +17,8 @@ class DocumentContent(StuffContent):public_url: str | None = Field(default=None, description="The public HTTPS URL of the document")mime_type: str | None = Field(default=None, description="The MIME type of the document")filename: str | None = Field(default=None, description="The original filename of the document")+ title: str | None = Field(default=None, description="The title of the document or source")+ snippet: str | None = Field(default=None, description="A text snippet or excerpt from the document")@model_validator(mode="after")def _auto_populate_filename(self) -> Self:@@ -35,27 +39,57 @@ class DocumentContent(StuffContent):@property@overridedef short_desc(self) -> str:+ if self.title:+ return f"document: {self.title}"url_desc = resolve_uri(self.url).kind.descreturn f"{url_desc} of a document"@overridedef rendered_plain(self) -> str:- return self.url+ parts: list[str] = []+ if self.title:+ parts.append(f"- {self.title}: {self.url}")+ else:+ parts.append(self.url)+ if self.snippet:+ parts.append(f" {self.snippet}")+ return "\n".join(parts)@overridedef rendered_html(self) -> str:+ display_text = self.title or self.public_url or self.url# The |e filter escapes HTML special characters to prevent XSS attackstemplate_source = '<a href="{{ url|e }}" class="msg-document">{{ display_text|e }}</a>'+ if self.snippet:+ template_source += "<br/><small>{{ snippet|e }}</small>"+ context: dict[str, str] = {+ "url": self.public_url or self.url,+ "display_text": display_text,+ }+ if self.snippet:+ context["snippet"] = self.snippetreturn render_jinja2_sync(template_source=template_source,template_category=TemplateCategory.HTML,- templating_context={- "url": self.public_url or self.url,- "display_text": self.public_url or self.url,- },+ templating_context=context,)@overridedef rendered_markdown(self, level: int = 1, is_pretty: bool = False) -> str:- display_text = self.public_url or self.url- return f"[{display_text}]({self.url})"+ display_text = self.title or self.public_url or self.url+ result = f"[{display_text}]({self.url})"+ if self.snippet:+ result += f"\n {self.snippet}"+ return result++ @override+ def rendered_pretty(self, title: str | None = None, depth: int = 0) -> PrettyPrintable:+ source_text = Text()+ if self.title:+ source_text.append(self.title, style="bold")+ source_text.append(f" ({self.url})", style="dim cyan")+ else:+ source_text.append(self.url, style="dim cyan")+ if self.snippet:+ source_text.append(f"\n {self.snippet}", style="dim italic")+ return source_textdiff --git a/pipelex/core/stuffs/search_result_content.py b/pipelex/core/stuffs/search_result_content.pyindex 1c27f8f7c..52d28e2ec 100644--- a/pipelex/core/stuffs/search_result_content.py+++ b/pipelex/core/stuffs/search_result_content.py@@ -6,63 +6,17 @@ from rich.markdown import Markdownfrom rich.text import Textfrom typing_extensions import override+from pipelex.core.stuffs.document_content import DocumentContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.tools.misc.pretty import PrettyPrintablefrom pipelex.tools.typing.pydantic_utils import empty_list_factory_of-class SearchSourceContent(StuffContent):- """Represents a single search source with name, URL, and optional snippet."""-- name: str- url: str- snippet: str | None = None-- @property- @override- def short_desc(self) -> str:- return f"source: {self.name}"-- @override- def rendered_plain(self) -> str:- result = f"- {self.name}: {self.url}"- if self.snippet:- result += f"\n {self.snippet}"- return result-- @override- def rendered_html(self) -> str:- name_escaped = html_module.escape(self.name)- url_escaped = html_module.escape(self.url)- result = f'<li><a href="{url_escaped}">{name_escaped}</a>'- if self.snippet:- snippet_escaped = html_module.escape(self.snippet)- result += f"<br/><small>{snippet_escaped}</small>"- result += "</li>"- return result-- @override- def rendered_markdown(self, level: int = 1, is_pretty: bool = False) -> str:- result = f"- [{self.name}]({self.url})"- if self.snippet:- result += f"\n {self.snippet}"- return result-- @override- def rendered_pretty(self, title: str | None = None, depth: int = 0) -> PrettyPrintable:- source_text = Text()- source_text.append(self.name, style="bold")- source_text.append(f" ({self.url})", style="dim cyan")- if self.snippet:- source_text.append(f"\n {self.snippet}", style="dim italic")- return source_text--class SearchResultContent(StuffContent):"""Represents the result of a search query with an answer and list of sources."""answer: str- sources: list[SearchSourceContent] = Field(default_factory=empty_list_factory_of(SearchSourceContent))+ sources: list[DocumentContent] = Field(default_factory=empty_list_factory_of(DocumentContent))@property@override@@ -83,7 +37,7 @@ class SearchResultContent(StuffContent):answer_escaped = html_module.escape(self.answer)result = f"<div><p>{answer_escaped}</p>"if self.sources:- source_items = "".join(source.rendered_html() for source in self.sources)+ source_items = "".join(f"<li>{source.rendered_html()}</li>" for source in self.sources)result += f"<h4>Sources</h4><ul>{source_items}</ul>"result += "</div>"return result@@ -93,7 +47,7 @@ class SearchResultContent(StuffContent):result = self.answerif self.sources:result += "\n\n**Sources:**\n\n"- result += "\n".join(source.rendered_markdown(level=level, is_pretty=is_pretty) for source in self.sources)+ result += "\n".join(f"- {source.rendered_markdown(level=level, is_pretty=is_pretty)}" for source in self.sources)return result@overridediff --git a/pipelex/plugins/gateway/gateway_search_worker.py b/pipelex/plugins/gateway/gateway_search_worker.pyindex e2b5b38ef..4a3e2928e 100644--- a/pipelex/plugins/gateway/gateway_search_worker.py+++ b/pipelex/plugins/gateway/gateway_search_worker.py@@ -15,7 +15,8 @@ from pipelex.cogt.search.search_job import SearchJobfrom pipelex.cogt.search.search_worker_abstract import SearchWorkerAbstractfrom pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategoryfrom pipelex.config import get_config-from pipelex.core.stuffs.search_result_content import SearchResultContent, SearchSourceContent+from pipelex.core.stuffs.document_content import DocumentContent+from pipelex.core.stuffs.search_result_content import SearchResultContentfrom pipelex.plugins.gateway.gateway_deck import GatewayDeckfrom pipelex.plugins.gateway.gateway_exceptions import GatewaySearchResponseErrorfrom pipelex.plugins.gateway.gateway_factory import GatewayFactory@@ -87,10 +88,12 @@ class GatewaySearchWorker(SearchWorkerAbstract):result_dict: dict[str, Any] = json.loads(content_str)sources = [- SearchSourceContent(- name=source["name"],+ DocumentContent(+ title=source["name"],url=source["url"],+ public_url=source["url"],snippet=source["snippet"],+ mime_type="text/html",)for source in result_dict.get("sources", [])]diff --git a/pipelex/plugins/linkup/linkup_search_worker.py b/pipelex/plugins/linkup/linkup_search_worker.pyindex 438b8f575..e26ca680f 100644--- a/pipelex/plugins/linkup/linkup_search_worker.py+++ b/pipelex/plugins/linkup/linkup_search_worker.py@@ -9,7 +9,8 @@ from pipelex.cogt.search.search_depth import SearchDepthfrom pipelex.cogt.search.search_job import SearchJobfrom pipelex.cogt.search.search_worker_abstract import SearchWorkerAbstractfrom pipelex.cogt.usage.token_category import TokenCategory-from pipelex.core.stuffs.search_result_content import SearchResultContent, SearchSourceContent+from pipelex.core.stuffs.document_content import DocumentContent+from pipelex.core.stuffs.search_result_content import SearchResultContentfrom pipelex.hub import get_secrets_providerfrom pipelex.reporting.reporting_protocol import ReportingProtocolfrom pipelex.tools.typing.pydantic_utils import BaseModelTypeVar@@ -56,13 +57,15 @@ class LinkupSearchWorker(SearchWorkerAbstract):if search_tokens_usage := search_job.job_report.search_tokens_usage:search_tokens_usage.nb_tokens_by_category = {TokenCategory.INPUT: 1_000_000, TokenCategory.OUTPUT: 1_000_000}- sources: list[SearchSourceContent] = []+ sources: list[DocumentContent] = []for source in response.sources:sources.append(- SearchSourceContent(- name=source.name,+ DocumentContent(+ title=source.name,url=source.url,+ public_url=source.url,snippet=source.snippet,+ mime_type="text/html",))diff --git a/tests/integration/pipelex/cogt/test_search.py b/tests/integration/pipelex/cogt/test_search.pyindex c32240a1a..26a6ca154 100644--- a/tests/integration/pipelex/cogt/test_search.py+++ b/tests/integration/pipelex/cogt/test_search.py@@ -45,7 +45,7 @@ class TestSearch:assert result.answer, "Expected a non-empty answer"assert len(result.sources) > 0, "Expected at least one source"for source in result.sources:- assert source.name, "Expected source to have a name"+ assert source.title, "Expected source to have a title"assert source.url, "Expected source to have a URL"get_report_delegate().generate_report()diff --git a/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_data.py b/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_data.pyindex 5d9b45451..3dfd19c7b 100644--- a/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_data.py+++ b/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_data.py@@ -33,7 +33,6 @@ class PipeExtractTestCases:output="Page[]",extract_talent=ExtractTalent.PDF_BASIC_TEXT_EXTRACTOR,max_page_images=None,- page_image_captions=True,page_views=True,),PipeExtractBlueprint(@@ -43,7 +42,7 @@ class PipeExtractTestCases:output="Page[]",model="@default-text-from-pdf",max_page_images=None,- page_image_captions=True,+ page_image_captions=None,page_views=True,page_views_dpi=None,),diff --git a/tests/unit/pipelex/core/stuffs/document_content/test_data.py b/tests/unit/pipelex/core/stuffs/document_content/test_data.pyindex 1f1581f0d..67e1991a2 100644--- a/tests/unit/pipelex/core/stuffs/document_content/test_data.py+++ b/tests/unit/pipelex/core/stuffs/document_content/test_data.py@@ -15,6 +15,8 @@ class TestData:"mime_type": None,"public_url": None,"filename": None,+ "title": None,+ "snippet": None,}# Expected outputs for smart_dump (with optional fields)@@ -23,6 +25,8 @@ class TestData:"mime_type": "application/pdf","public_url": "Report.pdf","filename": None,+ "title": None,+ "snippet": None,}# Expected outputs for render methods