Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Refactor/uri resolver (#556)

Louis Choquel8f41f8e63c00
diff --git a/.pipelex/pipelex.toml b/.pipelex/pipelex.toml
index 2983ada3a..2032e08cc 100644
--- a/.pipelex/pipelex.toml
+++ b/.pipelex/pipelex.toml
@@ -112,3 +112,7 @@
# cost_report_base_name = "cost_report"
# cost_report_extension = "csv"
# cost_report_unit_scale = 1.0
+
+[pipelex.pipeline_execution_config]
+# Uncomment to disable conversion of data URLs to pipelex-storage:// URIs
+# is_normalize_data_urls_to_storage = false
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4e2b237e5..9ed023b0e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -34,6 +34,8 @@
See the [Build Commands documentation](https://docs.pipelex.com/home/9-tools/cli/#build-commands) for usage examples.
+- **Langfuse & OpenTelemetry Observability** — New OpenTelemetry-based observability system enables powerful tracing and Evals through Langfuse integration. Also supports OTLP-compatible backends (Datadog, Honeycomb, etc.). Configured via `.pipelex/telemetry.toml`.
+
### Added
- New backends & models
@@ -43,11 +45,13 @@
- **Mistral** Document AI (OCR) via Pipelex Gateway (disclaimer: it's flaky, needs many retries via tenacity)
- **Portkey AI** backend integration for unified access to multiple models through a single API key
- Mistral Document AI (OCR) via Pipelex Gateway (disclaimer: it's flaky, needs many retries via tenacity)
-- OpenTelemetry-based observability system with native support for Langfuse and OTLP-compatible backends (Datadog, Honeycomb, etc.), configured via `.pipelex/telemetry.toml`
+- **Unified URI Handling System**: New `pipelex.tools.uri` module providing type-safe parsing for HTTP/HTTPS URLs, local file paths, file URIs, `pipelex-storage://` URIs, and base64 data URLs.
+- **Automatic Data URL Normalization**: Pipeline pre-processing step that converts large `data:` URLs in `ImageContent` to `pipelex-storage://` URIs for improved performance. Configurable via `is_normalize_data_urls_to_storage` in `pipelex.toml`.
+- **`PreparedImage` Abstraction**: New models (`PreparedImageHttpUrl`, `PreparedImageBase64`) representing images ready for LLM provider APIs.
+- Comprehensive unit and integration tests for URI resolver, data URL normalization, and `pipelex-storage://` image handling.
- Pipelex Gateway service with terms of service management via `.pipelex/pipelex_service.toml` and interactive acceptance flow in `pipelex init`
- Configurable retry logic with exponential backoff for inference API calls, configurable in `pipelex.toml` under `[cogt.tenacity_config]`
- Context manager support for the `Pipelex` class (`with Pipelex.make(): ...`) for graceful shutdown
-- Validation for `PipeCompose` output concepts: must be strictly compatible with the Text concept
- Validation for Pipelex Bundle concept keys: cannot create a native concept
- Validation for `PipeSequence`: The multiplicity of the output of the sequence must be the same as the multiplicity of the output of the last step.
- Validation for `PipeFunc`: The multiplicity of the output of the pipe must be the same as the multiplicity of the output of the function: If the multiplicity is true, the return type of the function must be a subclass of `ListContent`. If the multiplicity is false, the return type of the function must not be a subclass of `ListContent`.
@@ -77,6 +81,10 @@
- `is_generate_cost_report_file_enabled` default changed from `true` to `false`
- `pipelex_override.toml` (final override) moved from repo root to `.pipelex/` directory
- `telemetry_override.toml` (personal telemetry settings) moved from repo root to `.pipelex/` directory
+- **Image Prompt Representation**: Redesigned `PromptImage` models—consolidated `PromptImagePath` and `PromptImageUrl` into `PromptImageUri`; now uses Pydantic discriminated union for URI, base64, and raw bytes sources.
+- **Centralized Image Preparation**: Moved image fetching and base64 conversion logic to `pipelex.cogt.image.prompt_image_utils`, simplifying LLM provider plugins (Anthropic, Google, Mistral, OpenAI).
+- **Unified Resource Loading**: Updated all file/URL reading components (PDF renderers, document extractors) to use the new URI handling system, replacing the legacy `pipelex.tools.misc.path_utils` module.
+- **Async HTTP Fetching**: Renamed `fetch_file_from_url_httpx_async` to `fetch_file_from_url_httpx`; removed redundant synchronous version.
- Documentation: clarified **Setup (first run)** vs **Configuration (TOML reference)**, added a Setup overview page, and added contributor docs for configuration defaults/overrides.
- `pipelex init` now creates a documented `telemetry.toml` template instead of prompting for preferences
- Model catalog updated with latest models (gpt-5.1, claude-4.5-opus, gemini-3.0-pro, etc.) and updated waterfalls in `base_deck.toml`
@@ -88,6 +96,10 @@
- Cost report now displays a note clarifying that it only includes LLM costs
- `description` field is not Optional anymore in the `PipeAbstract`, `PipeBlueprint` and `PipeSpec` classes.
+### Removed
+
+- **`openai_utils` Module**: Removed `pipelex.plugins.openai.openai_utils`; logic now in centralized image preparation utilities.
+
### Deprecated
- `pipelex_inference` backend in favor of `pipelex_gateway` (marked as "🛑 Legacy" in configuration template)
diff --git a/pipelex/cogt/content_generation/generated_content_factory.py b/pipelex/cogt/content_generation/generated_content_factory.py
index 9122427aa..9b4dc6c61 100644
--- a/pipelex/cogt/content_generation/generated_content_factory.py
+++ b/pipelex/cogt/content_generation/generated_content_factory.py
@@ -10,7 +10,7 @@ from pipelex.core.stuffs.page_content import PageContent
from pipelex.core.stuffs.text_and_images_content import TextAndImagesContent
from pipelex.core.stuffs.text_content import TextContent
from pipelex.tools.misc.base64_utils import extract_base64_str_from_base64_url_if_possible
-from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx_async
+from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx
from pipelex.tools.misc.image_utils import ImageFormat
from pipelex.tools.storage.storage_provider_abstract import StorageProviderAbstract
@@ -59,7 +59,7 @@ class GeneratedContentFactory:
return uri_format.format(primary_id=primary_id, secondary_id=secondary_id, hash=hash_digest, extension=extension)
async def _fetch_remote_content(self, url: str) -> bytes:
- return await fetch_file_from_url_httpx_async(url=url)
+ return await fetch_file_from_url_httpx(url=url)
async def make_image_content(
self,
diff --git a/pipelex/cogt/image/prepared_image.py b/pipelex/cogt/image/prepared_image.py
new file mode 100644
index 000000000..e77de1d26
--- /dev/null
+++ b/pipelex/cogt/image/prepared_image.py
@@ -0,0 +1,43 @@
+"""Prepared images ready to be sent to LLM APIs.
+
+This module defines the output of image preparation - images in a format
+that can be directly consumed by LLM provider APIs (either as HTTP URLs
+or as base64-encoded data with mime type).
+"""
+
+from typing import Annotated, Literal, Union
+
+from pydantic import BaseModel, Field
+
+from pipelex.tools.misc.base64_utils import make_base64_url
+from pipelex.tools.misc.filetype_utils import FileType
+
+
+class PreparedImageHttpUrl(BaseModel):
+ """An HTTP URL that can be passed directly to some LLMs."""
+
+ kind: Literal["http_url"] = "http_url"
+ url: str
+
+
+class PreparedImageBase64(BaseModel):
+ """Base64-encoded image data with mime type."""
+
+ kind: Literal["base64"] = "base64"
+ base64_data: str
+ file_type: FileType
+
+ @property
+ def mime_type(self) -> str:
+ """Return the MIME type of the image."""
+ return self.file_type.mime
+
+ def as_data_url(self) -> str:
+ """Convert to a data: URL for APIs that accept it."""
+ return make_base64_url(base64_data=self.base64_data, file_type=self.file_type)
+
+
+PreparedImage = Annotated[
+ Union[PreparedImageHttpUrl, PreparedImageBase64],
+ Field(discriminator="kind"),
+]
diff --git a/pipelex/cogt/image/prompt_image.py b/pipelex/cogt/image/prompt_image.py
index 13bcf800a..758541473 100644
--- a/pipelex/cogt/image/prompt_image.py
+++ b/pipelex/cogt/image/prompt_image.py
@@ -1,20 +1,19 @@
import base64
-from abc import ABC, abstractmethod
-from typing import Literal, Union
+from functools import cached_property
+from typing import Annotated, Literal, Union
-from pydantic import BaseModel, field_validator
+from pydantic import BaseModel, Field, field_validator
from typing_extensions import override
from pipelex.tools.misc.attribute_utils import AttributePolisher
-from pipelex.tools.misc.file_utils import MAX_FILE_PATH_LENGTH
from pipelex.tools.misc.filetype_utils import (
FileType,
detect_file_type_from_base64,
detect_file_type_from_bytes,
- detect_file_type_from_path,
)
from pipelex.tools.misc.http_utils import URL_MAX_LENGTH
-from pipelex.tools.typing.pydantic_utils import CustomBaseModel
+from pipelex.tools.uri.resolved_uri import ResolvedUri
+from pipelex.tools.uri.uri_resolver import resolve_uri
from pipelex.types import StrEnum
@@ -28,88 +27,58 @@ class PromptImageDetail(StrEnum):
return self.value
-class PromptImageTypedBase64(CustomBaseModel):
- base_64: bytes
- file_type: FileType
+class PromptImageUri(BaseModel):
+ """A prompt image specified by URI (path, URL, storage URI, or data URL)."""
+ kind: Literal["uri"] = "uri"
+ uri: str
-PromptImageTypedUrlOrBase64 = Union[str, PromptImageTypedBase64]
-
-
-class PromptImage(BaseModel, ABC):
- @abstractmethod
- def short_description(self) -> str:
- pass
-
-
-class PromptImagePath(PromptImage):
- file_path: str
-
- @field_validator("file_path", mode="before")
+ @field_validator("uri", mode="before")
@classmethod
- def validate_file_path(cls, file_path: str) -> str:
- if len(file_path) > MAX_FILE_PATH_LENGTH:
- msg = f"File path is too long: {file_path}"
+ def validate_uri(cls, uri: str) -> str:
+ if len(uri) > URL_MAX_LENGTH:
+ msg = f"URI is too long: {uri[:100]}..."
raise ValueError(msg)
- return file_path
+ return uri
- def get_file_type(self) -> FileType:
- return detect_file_type_from_path(self.file_path)
-
- def get_mime_type(self) -> str:
- return self.get_file_type().mime
+ @cached_property
+ def resolved(self) -> ResolvedUri:
+ """Lazily resolve the URI to a typed ResolvedUri."""
+ return resolve_uri(self.uri)
@override
def __str__(self) -> str:
- return f"PromptImagePath(file_path='{self.file_path}')"
-
- @override
- def short_description(self) -> str:
- return f"file path: {self.file_path}"
-
-
-class PromptImageUrl(PromptImage):
- url: str
-
- @field_validator("url", mode="before")
- @classmethod
- def validate_url(cls, url: str) -> str:
- if len(url) > URL_MAX_LENGTH:
- msg = f"URL is too long: {url}"
- raise ValueError(msg)
- return url
-
- @override
- def __str__(self) -> str:
- truncated_url = AttributePolisher.get_truncated_value(value=self.url)
- return f"PromptImageUrl(url='{truncated_url!r}')"
+ truncated_uri = AttributePolisher.get_truncated_value(value=self.uri)
+ return f"PromptImageUri(uri={truncated_uri!r})"
@override
def __format__(self, format_spec: str) -> str:
return self.__str__()
- @override
def short_description(self) -> str:
- return self.url
+ """Return a short description of the image."""
+ return f"{self.resolved.kind.desc}: {self.uri[:100]}"
-class PromptImageBase64(PromptImage):
- base_64: bytes
+class PromptImageBase64(BaseModel):
+ """A prompt image as base64-encoded string."""
+
+ kind: Literal["base64"] = "base64"
+ base64_data: str
def get_file_type(self) -> FileType:
- return detect_file_type_from_base64(self.base_64)
+ return detect_file_type_from_base64(self.base64_data)
def get_mime_type(self) -> str:
return self.get_file_type().mime
def get_decoded_bytes(self) -> bytes:
- return base64.b64decode(self.base_64)
+ return base64.b64decode(self.base64_data)
@override
def __str__(self) -> str:
- base_64_str = str(self.base_64)
- truncated_base_64 = AttributePolisher.get_truncated_value(value=base_64_str)
- return f"PromptImageBase64(base_64={truncated_base_64!r})"
+ truncated_base64 = AttributePolisher.get_truncated_value(value=self.base64_data)
+ return f"PromptImageBase64(base64_data={truncated_base64!r})"
@override
def __repr__(self) -> str:
@@ -119,31 +88,37 @@ class PromptImageBase64(PromptImage):
def __format__(self, format_spec: str) -> str:
return self.__str__()
- def make_prompt_image_typed_base64(self) -> PromptImageTypedBase64:
- return PromptImageTypedBase64(base_64=self.base_64, file_type=self.get_file_type())
-
- @override
def short_description(self) -> str:
- return f"base64: {self.base_64[:100].decode('ascii', errors='replace')}..."
+ """Return a short description of the image."""
+ return f"base64: {self.base64_data[:100]}..."
+
+class PromptImageBinary(BaseModel):
+ """A prompt image as raw binary bytes."""
-class PromptImageBinary(PromptImage):
- binary: bytes
+ kind: Literal["binary"] = "binary"
+ raw_bytes: bytes
def get_file_type(self) -> FileType:
- return detect_file_type_from_bytes(self.binary)
+ return detect_file_type_from_bytes(self.raw_bytes)
def get_mime_type(self) -> str:
return self.get_file_type().mime
@override
def __str__(self) -> str:
- return "PromptImageBinary(binary=...)"
+ return "PromptImageBinary(raw_bytes=...)"
@override
def __repr__(self) -> str:
return self.__str__()
- @override
def short_description(self) -> str:
- return f"binary: {self.binary[:50].hex()}..."
+ """Return a short description of the image."""
+ return f"binary: {self.raw_bytes[:50].hex()}..."
+
+
+PromptImage = Annotated[
+ Union[PromptImageUri, PromptImageBase64, PromptImageBinary],
+ Field(discriminator="kind"),
+]
diff --git a/pipelex/cogt/image/prompt_image_factory.py b/pipelex/cogt/image/prompt_image_factory.py
index 4c0cdfb66..63b470727 100644
--- a/pipelex/cogt/image/prompt_image_factory.py
+++ b/pipelex/cogt/image/prompt_image_factory.py
@@ -1,60 +1,50 @@
-import base64
+"""Factory for creating PromptImage instances."""
from pipelex.cogt.exceptions import PromptImageFactoryError
from pipelex.cogt.image.prompt_image import (
PromptImage,
PromptImageBase64,
PromptImageBinary,
- PromptImagePath,
- PromptImageTypedBase64,
- PromptImageUrl,
+ PromptImageUri,
+)
+from pipelex.tools.misc.base64_utils import (
+ extract_base64_str_from_base64_url_if_possible,
+ strip_base64_str_if_needed,
)
-from pipelex.tools.misc.base64_utils import make_base64_url, strip_base64_str_if_needed
-from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx_async
class PromptImageFactory:
@classmethod
def make_prompt_image(
cls,
- file_path: str | None = None,
- url: str | None = None,
- base_64: bytes | None = None,
- base_64_str: str | None = None,
+ uri: str | None = None,
+ base64_data: str | None = None,
+ raw_bytes: bytes | None = None,
) -> PromptImage:
- if base_64:
- return PromptImageBase64(base_64=base_64)
- elif base_64_str:
- stripped_base_64_str = strip_base64_str_if_needed(base_64_str)
- return PromptImageBase64(base_64=stripped_base_64_str.encode())
- elif file_path:
- return PromptImagePath(file_path=file_path)
- elif url:
- return PromptImageUrl(url=url)
- else:
- msg = "PromptImageFactory requires one of file_path, url, or image_bytes"
- raise PromptImageFactoryError(msg)
+ """Create a PromptImage from the provided input.
- @classmethod
- async def make_promptimagebase64_from_url_async(
- cls,
- prompt_image_url: PromptImageUrl,
- ) -> PromptImageBase64:
- raw_image_bytes = await fetch_file_from_url_httpx_async(prompt_image_url.url)
- base64_bytes = base64.b64encode(raw_image_bytes)
- return PromptImageBase64(base_64=base64_bytes)
+ Args:
+ uri: A URI string (file path, HTTP URL, pipelex-storage://, or data: URL)
+ base64_data: Base64 string (with or without data: prefix)
+ raw_bytes: Raw binary image data
- @classmethod
- async def make_promptimagebinary_from_url_async(
- cls,
- prompt_image_url: PromptImageUrl,
- ) -> PromptImageBinary:
- raw_image_bytes = await fetch_file_from_url_httpx_async(prompt_image_url.url)
- return PromptImageBinary(binary=raw_image_bytes)
+ Returns:
+ A PromptImage instance (PromptImageUri, PromptImageBase64, or PromptImageBinary)
- @classmethod
- def make_base_64_url_from_prompt_image_typed_base64(
- cls,
- prompt_image: PromptImageTypedBase64,
- ) -> str:
- return make_base64_url(base64_bytes=prompt_image.base_64, file_type=prompt_image.file_type)
+ Raises:
+ PromptImageFactoryError: If no valid input is provided
+ """
+ if raw_bytes:
+ return PromptImageBinary(raw_bytes=raw_bytes)
+ if base64_data:
+ stripped_base64_data = strip_base64_str_if_needed(base64_data)
+ return PromptImageBase64(base64_data=stripped_base64_data)
+ if uri:
+ # Check if it's a data URL and extract base64 data to avoid URL_MAX_LENGTH validation
+ extracted = extract_base64_str_from_base64_url_if_possible(uri)
+ if extracted is not None:
+ extracted_base64_data, _mime_type = extracted
+ return PromptImageBase64(base64_data=extracted_base64_data)
+ return PromptImageUri(uri=uri)
+ msg = "PromptImageFactory requires one of: uri, base64_data, or raw_bytes"
+ raise PromptImageFactoryError(msg)
diff --git a/pipelex/cogt/image/prompt_image_utils.py b/pipelex/cogt/image/prompt_image_utils.py
index dd6bb1779..cc933019f 100644
--- a/pipelex/cogt/image/prompt_image_utils.py
+++ b/pipelex/cogt/image/prompt_image_utils.py
@@ -1,96 +1,167 @@
+"""Utilities for preparing prompt images for LLM APIs.
+
+This module provides functions to convert PromptImage instances into
+PreparedImage instances that can be consumed by LLM provider APIs.
+"""
+
import asyncio
import base64
-from pipelex.cogt.exceptions import PromptImageFactoryError
+from pipelex.cogt.image.prepared_image import PreparedImage, PreparedImageBase64, PreparedImageHttpUrl
from pipelex.cogt.image.prompt_image import (
PromptImage,
PromptImageBase64,
- PromptImagePath,
- PromptImageTypedBase64,
- PromptImageTypedUrlOrBase64,
- PromptImageUrl,
+ PromptImageBinary,
+ PromptImageUri,
)
-from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
from pipelex.hub import get_storage_provider
-from pipelex.tools.misc.base64_utils import load_binary_as_base64_async
+from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx
from pipelex.tools.misc.file_utils import load_binary_async
-from pipelex.tools.misc.filetype_utils import (
- detect_file_type_from_base64,
- detect_file_type_from_bytes,
+from pipelex.tools.misc.filetype_utils import detect_file_type_from_base64, detect_file_type_from_bytes
+from pipelex.tools.uri.resolved_uri import (
+ ResolvedBase64DataUrl,
+ ResolvedHttpUrl,
+ ResolvedLocalPath,
+ ResolvedPipelexStorage,
)
-from pipelex.tools.misc.path_utils import clarify_path_or_url, is_pipelex_storage_uri
-async def promptimage_to_b64_async(prompt_image: PromptImage) -> bytes:
+async def prepare_prompt_image(
+ prompt_image: PromptImage,
+ is_http_url_enabled: bool,
+) -> PreparedImage:
+ """Prepare a single prompt image for LLM API consumption.
+
+ Args:
+ prompt_image: The input prompt image (URI, base64, or binary)
+ is_http_url_enabled: Whether to pass HTTP URLs directly to the LLM
+
+ Returns:
+ A PreparedImage (HttpUrl or Base64) ready for the LLM API
+ """
+ prepared: PreparedImage
match prompt_image:
- case PromptImagePath():
- return await load_binary_as_base64_async(prompt_image.file_path)
case PromptImageBase64():
- return prompt_image.base_64
- case PromptImageUrl():
- image_bytes = await PromptImageFactory.make_promptimagebase64_from_url_async(prompt_image)
- return image_bytes.base_64
- case _:
- msg = f"Unknown PromptImage type: {prompt_image}"
- raise PromptImageFactoryError(msg)
+ prepared = PreparedImageBase64(
+ base64_data=prompt_image.base64_data,
+ file_type=prompt_image.get_file_type(),
+ )
+
+ case PromptImageBinary():
+ base64_data = base64.b64encode(prompt_image.raw_bytes).decode("ascii")
+ prepared = PreparedImageBase64(
+ base64_data=base64_data,
+ file_type=prompt_image.get_file_type(),
+ )
+ case PromptImageUri():
+ match prompt_image.resolved:
+ case ResolvedHttpUrl():
+ if is_http_url_enabled:
+ prepared = PreparedImageHttpUrl(url=prompt_image.resolved.url)
+ else:
+ raw_bytes = await fetch_file_from_url_httpx(url=prompt_image.resolved.url)
+ prepared = PreparedImageBase64(
+ base64_data=base64.b64encode(raw_bytes).decode("ascii"),
+ file_type=detect_file_type_from_bytes(raw_bytes),
+ )
-async def promptimage_to_bytes_async(prompt_image: PromptImage) -> bytes:
+ case ResolvedLocalPath():
+ raw_bytes = await load_binary_async(prompt_image.resolved.path)
+ prepared = PreparedImageBase64(
+ base64_data=base64.b64encode(raw_bytes).decode("ascii"),
+ file_type=detect_file_type_from_bytes(raw_bytes),
+ )
+
+ case ResolvedPipelexStorage():
+ storage = get_storage_provider()
+ raw_bytes = storage.load(uri=prompt_image.resolved.storage_uri)
+ prepared = PreparedImageBase64(
+ base64_data=base64.b64encode(raw_bytes).decode("ascii"),
+ file_type=detect_file_type_from_bytes(raw_bytes),
+ )
+
+ case ResolvedBase64DataUrl():
+ # base64_data from ResolvedBase64DataUrl is already a string
+ base64_data = prompt_image.resolved.base64_data
+ prepared = PreparedImageBase64(
+ base64_data=base64_data,
+ file_type=detect_file_type_from_base64(base64_data),
+ )
+
+ return prepared
+
+
+async def prep_prompt_images(
+ prompt_images: list[PromptImage],
+ is_http_url_enabled: bool,
+) -> list[PreparedImage]:
+ """Prepare multiple prompt images in parallel.
+
+ Args:
+ prompt_images: List of input prompt images
+ is_http_url_enabled: Whether to pass HTTP URLs directly to the LLM
+
+ Returns:
+ List of PreparedImage instances ready for the LLM API
+ """
+ tasks = [prepare_prompt_image(prompt_image=img, is_http_url_enabled=is_http_url_enabled) for img in prompt_images]
+ return list(await asyncio.gather(*tasks))
+
+
+async def prepare_prompt_image_as_base64(prompt_image: PromptImage) -> PreparedImageBase64:
+ """Prepare a prompt image, always returning base64-encoded data.
+
+ This variant always fetches HTTP URLs and converts everything to base64.
+ Use this when the LLM API doesn't support HTTP URLs directly.
+
+ Args:
+ prompt_image: The input prompt image (URI, base64, or binary)
+
+ Returns:
+ A PreparedImageBase64 ready for the LLM API
+ """
match prompt_image:
- case PromptImagePath():
- return await load_binary_async(prompt_image.file_path)
case PromptImageBase64():
- return prompt_image.get_decoded_bytes()
- case PromptImageUrl():
- image_bytes = await PromptImageFactory.make_promptimagebinary_from_url_async(prompt_image)
- return image_bytes.binary
- case _:
- msg = f"Unknown PromptImage type: {prompt_image}"
- raise PromptImageFactoryError(msg)
+ return PreparedImageBase64(
+ base64_data=prompt_image.base64_data,
+ file_type=prompt_image.get_file_type(),
+ )
+ case PromptImageBinary():
+ base64_data = base64.b64encode(prompt_image.raw_bytes).decode("ascii")
+ return PreparedImageBase64(
+ base64_data=base64_data,
+ file_type=prompt_image.get_file_type(),
+ )
-async def promptimage_to_typed_bytes_or_url(
- prompt_image: PromptImage,
- is_http_url_enabled: bool,
-) -> PromptImageTypedUrlOrBase64:
- typed_bytes_or_url: PromptImageTypedUrlOrBase64
- if isinstance(prompt_image, PromptImageBase64):
- typed_bytes_or_url = prompt_image.make_prompt_image_typed_base64()
- elif isinstance(prompt_image, PromptImageUrl):
- # Check for Pipelex storage URI first
- if is_pipelex_storage_uri(prompt_image.url):
- storage_provider = get_storage_provider()
- image_bytes = storage_provider.load(uri=prompt_image.url)
- file_type = detect_file_type_from_bytes(image_bytes)
- base64_bytes = base64.b64encode(image_bytes)
- typed_bytes_or_url = PromptImageTypedBase64(base_64=base64_bytes, file_type=file_type)
- else:
- file_path, url = clarify_path_or_url(path_or_uri=prompt_image.url)
- if url:
- if is_http_url_enabled:
- typed_bytes_or_url = prompt_image.url
- else:
- prompt_image_b64 = await PromptImageFactory.make_promptimagebase64_from_url_async(prompt_image_url=prompt_image)
- file_type = detect_file_type_from_base64(prompt_image_b64.base_64)
- typed_bytes_or_url = PromptImageTypedBase64(base_64=prompt_image_b64.base_64, file_type=file_type)
- elif file_path:
- prompt_image_path = PromptImagePath(file_path=file_path)
- return await promptimage_to_typed_bytes_or_url(
- prompt_image=prompt_image_path,
- is_http_url_enabled=is_http_url_enabled,
- )
- else:
- msg = "No file path or URL found"
- raise PromptImageFactoryError(msg)
- elif isinstance(prompt_image, PromptImagePath):
- b64 = await load_binary_as_base64_async(prompt_image.file_path)
- typed_bytes_or_url = PromptImageTypedBase64(base_64=b64, file_type=prompt_image.get_file_type())
- else:
- msg = f"Unsupported PromptImage type: '{type(prompt_image).__name__}'"
- raise PromptImageFactoryError(msg)
- return typed_bytes_or_url
-
-
-async def prep_prompt_images(prompt_images: list[PromptImage], is_http_url_enabled: bool) -> list[PromptImageTypedUrlOrBase64]:
- tasks_to_prep_images = [promptimage_to_typed_bytes_or_url(prompt_image, is_http_url_enabled) for prompt_image in prompt_images]
- return await asyncio.gather(*tasks_to_prep_images)
+ case PromptImageUri():
+ match prompt_image.resolved:
+ case ResolvedHttpUrl():
+ raw_bytes = await fetch_file_from_url_httpx(url=prompt_image.resolved.url)
+ return PreparedImageBase64(
+ base64_data=base64.b64encode(raw_bytes).decode("ascii"),
+ file_type=detect_file_type_from_bytes(raw_bytes),
+ )
+
+ case ResolvedLocalPath():
+ raw_bytes = await load_binary_async(prompt_image.resolved.path)
+ return PreparedImageBase64(
+ base64_data=base64.b64encode(raw_bytes).decode("ascii"),
+ file_type=detect_file_type_from_bytes(raw_bytes),
+ )
+
+ case ResolvedPipelexStorage():
+ storage = get_storage_provider()
+ raw_bytes = storage.load(uri=prompt_image.resolved.storage_uri)
+ return PreparedImageBase64(
+ base64_data=base64.b64encode(raw_bytes).decode("ascii"),
+ file_type=detect_file_type_from_bytes(raw_bytes),
+ )
+
+ case ResolvedBase64DataUrl():
+ base64_data = prompt_image.resolved.base64_data
+ return PreparedImageBase64(
+ base64_data=base64_data,
+ file_type=detect_file_type_from_base64(base64_data),
+ )
diff --git a/pipelex/core/stuffs/image_content.py b/pipelex/core/stuffs/image_content.py
index 9a7d9c303..0ef63038c 100644
--- a/pipelex/core/stuffs/image_content.py
+++ b/pipelex/core/stuffs/image_content.py
@@ -9,8 +9,8 @@ from pipelex.cogt.image.image_size import ImageSize
from pipelex.cogt.templating.template_category import TemplateCategory
from pipelex.core.stuffs.stuff_content import StuffContent
from pipelex.tools.jinja2.jinja2_rendering import render_jinja2_sync
-from pipelex.tools.misc.path_utils import interpret_path_or_url
from pipelex.tools.misc.pretty import PrettyPrintable
+from pipelex.tools.uri.uri_resolver import resolve_uri
class ImageContent(StuffContent):
@@ -25,8 +25,8 @@ class ImageContent(StuffContent):
@property
@override
def short_desc(self) -> str:
- url_desc = interpret_path_or_url(path_or_uri=self.url).desc
- return f"{url_desc} or an image"
+ url_desc = resolve_uri(self.url).kind.desc
+ return f"{url_desc} of an image"
@override
def rendered_plain(self) -> str:
diff --git a/pipelex/core/stuffs/pdf_content.py b/pipelex/core/stuffs/pdf_content.py
index 49981e58a..e85f7197a 100644
--- a/pipelex/core/stuffs/pdf_content.py
+++ b/pipelex/core/stuffs/pdf_content.py
@@ -3,7 +3,7 @@ from typing_extensions import override
from pipelex.cogt.templating.template_category import TemplateCategory
from pipelex.core.stuffs.stuff_content import StuffContent
from pipelex.tools.jinja2.jinja2_rendering import render_jinja2_sync
-from pipelex.tools.misc.path_utils import interpret_path_or_url
+from pipelex.tools.uri.uri_resolver import resolve_uri
class PDFContent(StuffContent):
@@ -12,7 +12,7 @@ class PDFContent(StuffContent):
@property
@override
def short_desc(self) -> str:
- url_desc = interpret_path_or_url(path_or_uri=self.url).desc
+ url_desc = resolve_uri(self.url).kind.desc
return f"{url_desc} of a PDF document"
@override
diff --git a/pipelex/kit/configs/pipelex.toml b/pipelex/kit/configs/pipelex.toml
index 2983ada3a..2032e08cc 100644
--- a/pipelex/kit/configs/pipelex.toml
+++ b/pipelex/kit/configs/pipelex.toml
@@ -112,3 +112,7 @@
# cost_report_base_name = "cost_report"
# cost_report_extension = "csv"
# cost_report_unit_scale = 1.0
+
+[pipelex.pipeline_execution_config]
+# Uncomment to disable conversion of data URLs to pipelex-storage:// URIs
+# is_normalize_data_urls_to_storage = false
diff --git a/pipelex/pipe_operators/llm/llm_prompt_blueprint.py b/pipelex/pipe_operators/llm/llm_prompt_blueprint.py
index 221eb5695..829405835 100644
--- a/pipelex/pipe_operators/llm/llm_prompt_blueprint.py
+++ b/pipelex/pipe_operators/llm/llm_prompt_blueprint.py
@@ -64,7 +64,7 @@ class LLMPromptBlueprint(BaseModel):
accept_list=True,
)
if isinstance(prompt_image_content, ImageContent):
- user_image = PromptImageFactory.make_prompt_image(url=prompt_image_content.url)
+ user_image = PromptImageFactory.make_prompt_image(uri=prompt_image_content.url)
prompt_user_images[user_image_name] = user_image
elif isinstance(prompt_image_content, list):
prompt_image_content = cast("list[ImageContent]", prompt_image_content)
@@ -72,7 +72,7 @@ class LLMPromptBlueprint(BaseModel):
if not isinstance(image_item, ImageContent): # pyright: ignore[reportUnnecessaryIsInstance]
msg = f"Item of '{user_image_name}' is of type '{type(image_item).__name__}', it should be ImageContent"
raise LLMPromptBlueprintValueError(msg)
- user_image = PromptImageFactory.make_prompt_image(url=image_item.url)
+ user_image = PromptImageFactory.make_prompt_image(uri=image_item.url)
user_image_item_name = f"{user_image_name}[{image_index}]"
prompt_user_images[user_image_item_name] = user_image
else:
@@ -89,7 +89,7 @@ class LLMPromptBlueprint(BaseModel):
if isinstance(image_collection, (list, tuple)):
image_collection = cast("list[ImageContent] | tuple[ImageContent]", image_collection)
for image_index, image_item in enumerate(image_collection, start=1):
- user_image = PromptImageFactory.make_prompt_image(url=image_item.url)
+ user_image = PromptImageFactory.make_prompt_image(uri=image_item.url)
user_image_item_name = f"{user_image_name}[{image_index}]"
prompt_user_images[user_image_item_name] = user_image
else:
diff --git a/pipelex/pipelex.toml b/pipelex/pipelex.toml
index db5ba5089..60e2c7f04 100644
--- a/pipelex/pipelex.toml
+++ b/pipelex/pipelex.toml
@@ -264,6 +264,13 @@ choice_edge_style = "-----"
[pipelex.pipe_run_config]
pipe_stack_limit = 20
+####################################################################################################
+# Pipeline execution config
+####################################################################################################
+
+[pipelex.pipeline_execution_config]
+is_normalize_data_urls_to_storage = true
+
####################################################################################################
# Dry run config
####################################################################################################
diff --git a/pipelex/pipeline/input_normalizer.py b/pipelex/pipeline/input_normalizer.py
new file mode 100644
index 000000000..57123d54a
--- /dev/null
+++ b/pipelex/pipeline/input_normalizer.py
@@ -0,0 +1,206 @@
+"""Normalize pipeline inputs by converting data URLs to pipelex-storage:// URIs.
+
+This module provides functions to scan WorkingMemory and convert any ImageContent
+with data URLs (data:...;base64,...) to pipelex-storage:// URIs for more efficient
+pipeline processing.
+"""
+
+import base64
+from typing import Any, cast
+
+import shortuuid
+
+from pipelex.core.memory.working_memory import WorkingMemory
+from pipelex.core.stuffs.image_content import ImageContent
+from pipelex.core.stuffs.list_content import ListContent
+from pipelex.core.stuffs.structured_content import StructuredContent
+from pipelex.hub import get_storage_provider
+from pipelex.tools.misc.filetype_utils import detect_file_type_from_bytes
+from pipelex.tools.storage.storage_provider_abstract import StorageProviderAbstract
+from pipelex.tools.uri.resolved_uri import ResolvedBase64DataUrl
+from pipelex.tools.uri.uri_resolver import resolve_uri
+
+
+def normalize_data_urls_to_storage(working_memory: WorkingMemory) -> WorkingMemory:
+ """Convert all data URLs in ImageContent to pipelex-storage:// URIs.
+
+ Scans all stuffs in working memory and for any ImageContent with a data:...;base64,...
+ URL, stores the data and replaces the URL with a pipelex-storage:// URI.
+
+ This handles:
+ - Direct ImageContent
+ - ListContent containing ImageContent items
+ - StructuredContent with nested ImageContent fields (recursive)
+
+ Args:
+ working_memory: The working memory to normalize.
+
+ Returns:
+ The same WorkingMemory instance with normalized ImageContent URLs.
+ """
+ storage = get_storage_provider()
+
+ for stuff in working_memory.root.values():
+ content = stuff.content
+ normalized_content, changed = _normalize_value(value=content, storage=storage)
+ if changed:
+ stuff.content = normalized_content
+
+ return working_memory
+
+
+def _normalize_value(
+ value: Any,
+ storage: StorageProviderAbstract,
+) -> tuple[Any, bool]:
+ """Recursively normalize a value, converting data URLs in ImageContent to storage URIs.
+
+ Args:
+ value: The value to normalize (can be ImageContent, StructuredContent, list, or any other type).
+ storage: The storage provider to use.
+
+ Returns:
+ A tuple of (normalized_value, has_changed).
+ """
+ # Handle ImageContent directly
+ if isinstance(value, ImageContent):
+ normalized = _normalize_image_content(image_content=value, storage=storage)
+ return normalized, normalized is not value
+
+ # Handle StructuredContent (recursively process all fields)
+ if isinstance(value, StructuredContent):
+ return _normalize_structured_content(structured_content=value, storage=storage)
+
+ # Handle ListContent
+ if isinstance(value, ListContent):
+ return _normalize_list_content(list_content=value, storage=storage) # pyright: ignore[reportUnknownArgumentType]
+
+ # Handle plain lists (might contain ImageContent or StructuredContent)
+ if isinstance(value, list):
+ return _normalize_list(items=value, storage=storage) # pyright: ignore[reportUnknownArgumentType]
+
+ # Other types don't need normalization
+ return value, False
+
+
+def _normalize_structured_content(
+ structured_content: StructuredContent,
+ storage: StorageProviderAbstract,
+) -> tuple[StructuredContent, bool]:
+ """Normalize a StructuredContent by recursively processing all its fields.
+
+ Args:
+ structured_content: The structured content to normalize.
+ storage: The storage provider to use.
+
+ Returns:
+ A tuple of (normalized_content, has_changed).
+ """
+ updates: dict[str, Any] = {}
+ has_changes = False
+
+ for field_name, field_value in structured_content:
+ normalized_value, changed = _normalize_value(value=field_value, storage=storage)
+ if changed:
+ updates[field_name] = normalized_value
+ has_changes = True
+
+ if not has_changes:
+ return structured_content, False
+
+ # Create a new instance with updated fields
+ # Use model_copy with update to preserve all other fields
+ return structured_content.model_copy(update=updates), True
+
+
+def _normalize_list_content(
+ list_content: ListContent[Any],
+ storage: StorageProviderAbstract,
+) -> tuple[ListContent[Any], bool]:
+ """Normalize a ListContent by processing all its items.
+
+ Args:
+ list_content: The list content to normalize.
+ storage: The storage provider to use.
+
+ Returns:
+ A tuple of (normalized_list_content, has_changed).
+ """
+ raw_items = list_content.items # pyright: ignore[reportUnknownVariableType, reportUnknownMemberType]
+ if not raw_items:
+ return list_content, False
+
+ normalized_items, has_changes = _normalize_list(items=raw_items, storage=storage) # pyright: ignore[reportUnknownArgumentType]
+
+ if not has_changes:
+ return list_content, False
+
+ # Check the type of the first item to determine the ListContent type
+ first_item = normalized_items[0]
+ if isinstance(first_item, ImageContent):
+ return ListContent[ImageContent](items=cast("list[ImageContent]", normalized_items)), True
+
+ # For other types (e.g., StructuredContent subclasses), use generic ListContent
+ return ListContent(items=normalized_items), True
+
+
+def _normalize_list(
+ items: list[Any],
+ storage: StorageProviderAbstract,
+) -> tuple[list[Any], bool]:
+ """Normalize a list by processing all its items.
+
+ Args:
+ items: The list items to normalize.
+ storage: The storage provider to use.
+
+ Returns:
+ A tuple of (normalized_items, has_changed).
+ """
+ normalized_items: list[Any] = []
+ has_changes = False
+
+ for item in items:
+ normalized_item, changed = _normalize_value(value=item, storage=storage)
+ normalized_items.append(normalized_item)
+ if changed:
+ has_changes = True
+
+ return normalized_items, has_changes
+
+
+def _normalize_image_content(
+ image_content: ImageContent,
+ storage: StorageProviderAbstract,
+) -> ImageContent:
+ """Normalize a single ImageContent, converting data URLs to storage URIs.
+
+ Args:
+ image_content: The image content to normalize.
+ storage: The storage provider to use.
+
+ Returns:
+ The original ImageContent if no normalization needed, or a new ImageContent
+ with the normalized URL.
+ """
+ resolved_uri = resolve_uri(image_content.url)
+
+ if not isinstance(resolved_uri, ResolvedBase64DataUrl):
+ # Not a data URL, we can keep the original ImageContent without any changes
+ return image_content
+
+ # Decode base64 data and store
+ raw_bytes = base64.b64decode(resolved_uri.base64_data)
+ file_type = detect_file_type_from_bytes(raw_bytes)
+ key = f"normalized/{shortuuid.uuid()}.{file_type.extension}"
+ storage_uri = storage.store(data=raw_bytes, key=key)
+
+ return ImageContent(
+ url=storage_uri,
+ display_link=image_content.display_link,
+ mime_type=resolved_uri.mime_type or image_content.mime_type,
+ source_prompt=image_content.source_prompt,
+ source_negative_prompt=image_content.source_negative_prompt,
+ caption=image_content.caption,
+ size=image_content.size,
+ )
diff --git a/pipelex/pipeline/pipeline_run_setup.py b/pipelex/pipeline/pipeline_run_setup.py
index 821f42008..0e56efb0d 100644
--- a/pipelex/pipeline/pipeline_run_setup.py
+++ b/pipelex/pipeline/pipeline_run_setup.py
@@ -2,6 +2,7 @@ from pathlib import Path
from typing import TYPE_CHECKING
from pipelex.client.protocol import PipelineInputs
+from pipelex.config import get_config
from pipelex.core.memory.working_memory import WorkingMemory
from pipelex.core.memory.working_memory_factory import WorkingMemoryFactory
from pipelex.hub import (
@@ -22,6 +23,7 @@ from pipelex.pipe_run.pipe_run_params import (
)
from pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactory
from pipelex.pipeline.exceptions import PipeExecutionError
+from pipelex.pipeline.input_normalizer import normalize_data_urls_to_storage
from pipelex.pipeline.job_metadata import JobMetadata, OtelContext
from pipelex.pipeline.validate_bundle import validate_bundle
from pipelex.system.environment import get_optional_env
@@ -159,6 +161,10 @@ async def pipeline_run_setup(
search_domain_codes=search_domain_codes,
)
+ # Normalize data URLs to pipelex-storage:// URIs if configured
+ if working_memory and get_config().pipelex.pipeline_execution_config.is_normalize_data_urls_to_storage:
+ working_memory = normalize_data_urls_to_storage(working_memory)
+
if pipe_run_mode is None:
if run_mode_from_env := get_optional_env(key=FORCE_DRY_RUN_MODE_ENV_KEY):
pipe_run_mode = PipeRunMode(run_mode_from_env)
diff --git a/pipelex/plugins/anthropic/anthropic_factory.py b/pipelex/plugins/anthropic/anthropic_factory.py
index 9011628bf..9643d6d1a 100644
--- a/pipelex/plugins/anthropic/anthropic_factory.py
+++ b/pipelex/plugins/anthropic/anthropic_factory.py
@@ -9,10 +9,7 @@ from openai.types.chat import (
)
from pipelex.cogt.exceptions import CogtError
-from pipelex.cogt.image.prompt_image import (
- PromptImageTypedBase64,
- PromptImageTypedUrlOrBase64,
-)
+from pipelex.cogt.image.prepared_image import PreparedImage, PreparedImageBase64, PreparedImageHttpUrl
from pipelex.cogt.image.prompt_image_utils import prep_prompt_images
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.model_backends.backend import InferenceBackend
@@ -62,6 +59,30 @@ class AnthropicFactory:
aws_region=aws_region,
)
+ @staticmethod
+ def _make_image_block_param(prepped_image: PreparedImage) -> "ImageBlockParam":
+ """Convert a PreparedImage to an Anthropic ImageBlockParam."""
+ image_block_param: ImageBlockParam
+ match prepped_image:
+ case PreparedImageBase64():
+ image_block_param = {
+ "type": "image",
+ "source": {
+ "type": "base64",
+ "media_type": prepped_image.mime_type, # type: ignore[typeddict-item]
+ "data": prepped_image.base64_data,
+ }, # pyright: ignore[reportAssignmentType]
+ }
+ case PreparedImageHttpUrl():
+ image_block_param = {
+ "type": "image",
+ "source": {
+ "type": "url",
+ "url": prepped_image.url,
+ },
+ }
+ return image_block_param
+
@classmethod
async def make_user_message(
cls,
@@ -80,30 +101,7 @@ class AnthropicFactory:
if llm_prompt.user_images:
prepped_user_images = await prep_prompt_images(prompt_images=llm_prompt.user_images, is_http_url_enabled=False)
for prepped_image in prepped_user_images:
- image_block_param: ImageBlockParam
- if isinstance(prepped_image, PromptImageTypedBase64):
- mime = prepped_image.file_type.mime
- image_block_param = {
- "type": "image",
- "source": {
- "type": "base64",
- "media_type": mime, # type: ignore[typeddict-item]
- "data": prepped_image.base_64.decode("utf-8"),
- }, # pyright: ignore[reportAssignmentType]
- }
- elif isinstance(prepped_image, str): # pyright: ignore[reportUnnecessaryIsInstance]
- url = prepped_image
- image_block_param = {
- "type": "image",
- "source": {
- "type": "url",
- "url": url,
- },
- }
- else:
- msg = f"Unsupported PromptImageTypedBytesOrUrl type: '{type(prepped_image).__name__}'"
- raise AnthropicFactoryError(msg)
- content.append(image_block_param)
+ content.append(cls._make_image_block_param(prepped_image))
message = {
"role": "user",
@@ -113,40 +111,18 @@ class AnthropicFactory:
return message
# This creates a MessageParam disguised as a ChatCompletionMessageParam to please instructor type checking
- @staticmethod
+ @classmethod
def openai_typed_user_message(
+ cls,
user_content_txt: str,
- prepped_user_images: list[PromptImageTypedUrlOrBase64] | None = None,
+ prepped_user_images: list[PreparedImage] | None = None,
) -> ChatCompletionMessageParam:
text_block_param: TextBlockParam = {"type": "text", "text": user_content_txt}
message: MessageParam
if prepped_user_images is not None:
images_block_params: list[ImageBlockParam] = []
for prepped_image in prepped_user_images:
- image_block_param_in_loop: ImageBlockParam
- if isinstance(prepped_image, PromptImageTypedBase64):
- mime = prepped_image.file_type.mime
- image_block_param_in_loop = {
- "type": "image",
- "source": {
- "type": "base64",
- "media_type": mime, # type: ignore[typeddict-item]
- "data": prepped_image.base_64.decode("utf-8"),
- }, # pyright: ignore[reportAssignmentType]
- }
- elif isinstance(prepped_image, str): # pyright: ignore[reportUnnecessaryIsInstance]
- url = prepped_image
- image_block_param_in_loop = {
- "type": "image",
- "source": {
- "type": "url",
- "url": url,
- },
- }
- else:
- msg = f"Unsupported PromptImageTypedBytesOrUrl type: '{type(prepped_image).__name__}'"
- raise AnthropicFactoryError(msg)
- images_block_params.append(image_block_param_in_loop)
+ images_block_params.append(cls._make_image_block_param(prepped_image))
content: list[TextBlockParam | ImageBlockParam] = [*images_block_params, text_block_param]
message = {
@@ -173,7 +149,7 @@ class AnthropicFactory:
if system_content := llm_prompt.system_text:
messages.append(ChatCompletionSystemMessageParam(role="system", content=system_content))
- prepped_user_images: list[PromptImageTypedUrlOrBase64] | None
+ prepped_user_images: list[PreparedImage] | None
if llm_prompt.user_images:
prepped_user_images = await prep_prompt_images(prompt_images=llm_prompt.user_images, is_http_url_enabled=False)
else:
@@ -181,7 +157,7 @@ class AnthropicFactory:
# Concatenation ####
messages.append(
- AnthropicFactory.openai_typed_user_message(
+ cls.openai_typed_user_message(
user_content_txt=llm_prompt.user_text or "",
prepped_user_images=prepped_user_images,
),
diff --git a/pipelex/plugins/docling/docling_extract_worker.py b/pipelex/plugins/docling/docling_extract_worker.py
index 6b26dd2e7..2c9f6a7e3 100644
--- a/pipelex/plugins/docling/docling_extract_worker.py
+++ b/pipelex/plugins/docling/docling_extract_worker.py
@@ -12,7 +12,13 @@ from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.plugins.docling.docling_factory import DoclingFactory
from pipelex.plugins.docling.docling_sdk import DoclingSdk
from pipelex.reporting.reporting_protocol import ReportingProtocol
-from pipelex.tools.misc.path_utils import resolve_path_or_url_for_reading
+from pipelex.tools.uri.resolved_uri import (
+ ResolvedBase64DataUrl,
+ ResolvedHttpUrl,
+ ResolvedLocalPath,
+ ResolvedPipelexStorage,
+)
+from pipelex.tools.uri.uri_resolver import resolve_uri
class DoclingExtractWorker(ExtractWorkerAbstract):
@@ -53,7 +59,16 @@ class DoclingExtractWorker(ExtractWorkerAbstract):
async def _extract_from_source(self, source_uri: str) -> ExtractOutput:
"""Extract text from a source URI (file path, file:// URI, or http(s) URL)."""
- resolved_source = resolve_path_or_url_for_reading(path_or_uri=source_uri)
+ resolved_uri = resolve_uri(source_uri)
+ resolved_source: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ resolved_source = resolved_uri.url
+ case ResolvedLocalPath():
+ resolved_source = resolved_uri.path
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ msg = f"Unsupported URI type for Docling extraction: {resolved_uri.kind}"
+ raise ExtractInputError(msg)
# Run synchronous Docling conversion in a thread pool to avoid blocking
conversion_result = await asyncio.to_thread(self.docling_sdk.document_converter.convert, resolved_source)
diff --git a/pipelex/plugins/gateway/gateway_extract_worker.py b/pipelex/plugins/gateway/gateway_extract_worker.py
index ef2ebc7fa..08d621ec6 100644
--- a/pipelex/plugins/gateway/gateway_extract_worker.py
+++ b/pipelex/plugins/gateway/gateway_extract_worker.py
@@ -19,7 +19,7 @@ from pipelex.plugins.gateway.gateway_completions_factory import GatewayCompletio
from pipelex.plugins.gateway.gateway_deck import GatewayDeck
from pipelex.plugins.gateway.gateway_factory import GatewayFactory
from pipelex.reporting.reporting_protocol import ReportingProtocol
-from pipelex.tools.misc.base64_utils import make_base64_url_from_location_async
+from pipelex.tools.uri.uri_resolver import make_base64_url_from_any_uri
from pipelex.types import StrEnum
@@ -84,7 +84,7 @@ class GatewayExtractWorker(ExtractWorkerAbstract):
if extract_job.job_params.should_caption_images:
msg = f"Captioning is not implemented by '{self.inference_model.tag}'."
raise NotImplementedError(msg)
- base64_url = await make_base64_url_from_location_async(location=image_uri)
+ base64_url = await make_base64_url_from_any_uri(uri=image_uri)
extract_output = await self.extract_base64_url(
extract_job=extract_job,
base64_url=base64_url,
@@ -97,7 +97,7 @@ class GatewayExtractWorker(ExtractWorkerAbstract):
# TODO: handle model capability and skip UT when it's not supported
msg = f"Captioning is not implemented by '{self.inference_model.tag}'."
raise ExtractCapabilityError(msg)
- base64_url = await make_base64_url_from_location_async(location=pdf_uri)
+ base64_url = await make_base64_url_from_any_uri(uri=pdf_uri)
extract_output = await self.extract_base64_url(
extract_job=extract_job,
base64_url=base64_url,
diff --git a/pipelex/plugins/google/google_factory.py b/pipelex/plugins/google/google_factory.py
index 5254b0588..7232995bc 100644
--- a/pipelex/plugins/google/google_factory.py
+++ b/pipelex/plugins/google/google_factory.py
@@ -1,53 +1,32 @@
import asyncio
+import base64
from google import genai
from google.genai import types as genai_types
-from pipelex.cogt.exceptions import CogtError
-from pipelex.cogt.image.prompt_image import (
- PromptImage,
- PromptImageBase64,
- PromptImagePath,
- PromptImageUrl,
-)
-from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
+from pipelex.cogt.image.prompt_image import PromptImage
+from pipelex.cogt.image.prompt_image_utils import prepare_prompt_image_as_base64
from pipelex.cogt.llm.llm_prompt import LLMPrompt
from pipelex.cogt.model_backends.backend import InferenceBackend
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
-from pipelex.tools.misc.base64_utils import load_binary_async
-
-
-class GoogleFactoryError(CogtError):
- pass
class GoogleFactory:
- @staticmethod
- def make_google_client(backend: InferenceBackend) -> genai.Client:
+ @classmethod
+ def make_google_client(cls, backend: InferenceBackend) -> genai.Client:
"""Create a Google Gemini API client."""
return genai.Client(api_key=backend.api_key)
@classmethod
async def prepare_image_part(cls, prompt_image: PromptImage) -> genai_types.Part:
- """Convert a PromptImage to Google genai Part format."""
- image_bytes: bytes
- mime_type: str
-
- if isinstance(prompt_image, PromptImageBase64):
- image_bytes = prompt_image.get_decoded_bytes()
- mime_type = prompt_image.get_mime_type()
- return genai_types.Part.from_bytes(data=image_bytes, mime_type=mime_type)
- if isinstance(prompt_image, PromptImagePath):
- image_bytes = await load_binary_async(prompt_image.file_path)
- mime_type = prompt_image.get_mime_type()
- return genai_types.Part.from_bytes(data=image_bytes, mime_type=mime_type)
- if isinstance(prompt_image, PromptImageUrl):
- prompt_image_binary = await PromptImageFactory.make_promptimagebinary_from_url_async(prompt_image)
- image_bytes = prompt_image_binary.binary
- mime_type = prompt_image_binary.get_mime_type()
- return genai_types.Part.from_bytes(data=image_bytes, mime_type=mime_type)
- msg = f"Unsupported PromptImage type: '{type(prompt_image).__name__}'"
- raise GoogleFactoryError(msg)
+ """Convert a PromptImage to Google genai Part format.
+
+ Uses the unified prepare_prompt_image_as_base64() which supports all URI types
+ including pipelex-storage://.
+ """
+ prepared = await prepare_prompt_image_as_base64(prompt_image)
+ image_bytes = base64.b64decode(prepared.base64_data)
+ return genai_types.Part.from_bytes(data=image_bytes, mime_type=prepared.mime_type)
@classmethod
async def prepare_user_contents(cls, llm_prompt: LLMPrompt) -> genai_types.ContentListUnion:
diff --git a/pipelex/plugins/mistral/mistral_extract_worker.py b/pipelex/plugins/mistral/mistral_extract_worker.py
index 6ffb158cd..7aa0087e6 100644
--- a/pipelex/plugins/mistral/mistral_extract_worker.py
+++ b/pipelex/plugins/mistral/mistral_extract_worker.py
@@ -11,9 +11,14 @@ from pipelex.cogt.extract.extract_worker_abstract import ExtractWorkerAbstract
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.plugins.mistral.mistral_factory import MistralFactory
from pipelex.reporting.reporting_protocol import ReportingProtocol
-from pipelex.tools.misc.base64_utils import load_binary_as_base64_async
-from pipelex.tools.misc.filetype_utils import detect_file_type_from_base64
-from pipelex.tools.misc.path_utils import clarify_path_or_url
+from pipelex.tools.misc.base64_utils import make_base64_url_from_path
+from pipelex.tools.uri.resolved_uri import (
+ ResolvedBase64DataUrl,
+ ResolvedHttpUrl,
+ ResolvedLocalPath,
+ ResolvedPipelexStorage,
+)
+from pipelex.tools.uri.uri_resolver import resolve_uri
class MistralExtractWorker(ExtractWorkerAbstract):
@@ -43,13 +48,13 @@ class MistralExtractWorker(ExtractWorkerAbstract):
) -> ExtractOutput:
# TODO: report usage
if image_uri := extract_job.extract_input.image_uri:
- extract_output = await self._make_extract_output_from_image(
+ extract_output = await self._extract_page_from_image(
image_uri=image_uri,
should_caption_image=extract_job.job_params.should_caption_images,
)
elif pdf_uri := extract_job.extract_input.pdf_uri:
- extract_output = await self._make_extract_output_from_pdf(
+ extract_output = await self._extract_pages_from_pdf(
pdf_uri=pdf_uri,
should_include_images=extract_job.job_params.should_include_images,
should_caption_images=extract_job.job_params.should_caption_images,
@@ -59,7 +64,7 @@ class MistralExtractWorker(ExtractWorkerAbstract):
raise ExtractInputError(msg)
return extract_output
- async def _make_extract_output_from_image(
+ async def _extract_page_from_image(
self,
image_uri: str,
should_caption_image: bool = False,
@@ -67,14 +72,19 @@ class MistralExtractWorker(ExtractWorkerAbstract):
if should_caption_image:
msg = "Captioning is not implemented for Mistral OCR."
raise NotImplementedError(msg)
- image_path, image_url = clarify_path_or_url(path_or_uri=image_uri)
- if image_url:
- return await self._extract_from_image_url(image_url=image_url)
- else:
- assert image_path is not None
- return await self._extract_from_image_file(image_path=image_path)
-
- async def _make_extract_output_from_pdf(
+ resolved_uri = resolve_uri(image_uri)
+ image_url: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ image_url = resolved_uri.url
+ case ResolvedLocalPath():
+ image_url = await make_base64_url_from_path(path=resolved_uri.path)
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ msg = f"Unsupported URI type for Mistral image extraction: {resolved_uri.kind}"
+ raise ExtractInputError(msg)
+ return await self._extract_from_image_url(image_url=image_url)
+
+ async def _extract_pages_from_pdf(
self,
pdf_uri: str,
should_include_images: bool,
@@ -83,20 +93,20 @@ class MistralExtractWorker(ExtractWorkerAbstract):
if should_caption_images:
msg = "Captioning is not implemented for Mistral OCR."
raise ExtractCapabilityError(msg)
- pdf_path, pdf_url = clarify_path_or_url(path_or_uri=pdf_uri)
- extract_output: ExtractOutput
- if pdf_url:
- extract_output = await self.extract_from_pdf_url(
- pdf_url=pdf_url,
- should_include_images=should_include_images,
- )
- else: # pdf_path must be provided based on validation
- assert pdf_path is not None
- extract_output = await self.extract_from_pdf_file(
- pdf_path=pdf_path,
- should_include_images=should_include_images,
- )
- return extract_output
+ resolved_uri = resolve_uri(pdf_uri)
+ pdf_url: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ pdf_url = resolved_uri.url
+ case ResolvedLocalPath():
+ pdf_url = await self._get_signed_url_from_pdf_file(pdf_path=resolved_uri.path)
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ msg = f"Unsupported URI type for Mistral PDF extraction: {resolved_uri.kind}"
+ raise ExtractInputError(msg)
+ return await self._extract_from_pdf_url(
+ pdf_url=pdf_url,
+ should_include_images=should_include_images,
+ )
async def _extract_from_image_url(
self,
@@ -113,24 +123,7 @@ class MistralExtractWorker(ExtractWorkerAbstract):
mistral_extract_response=extract_response,
)
- async def _extract_from_image_file(
- self,
- image_path: str,
- ) -> ExtractOutput:
- b64 = await load_binary_as_base64_async(path=image_path)
-
- file_type = detect_file_type_from_base64(base64_data=b64)
- mime_type = file_type.mime
-
- extract_response = await self.mistral_client.ocr.process_async(
- model=self.inference_model.model_id,
- document={"type": "image_url", "image_url": f"data:{mime_type};base64,{b64.decode('utf-8')}"},
- )
- return await MistralFactory.make_extract_output_from_mistral_response(
- mistral_extract_response=extract_response,
- )
-
- async def extract_from_pdf_url(
+ async def _extract_from_pdf_url(
self,
pdf_url: str,
should_include_images: bool = False,
@@ -149,22 +142,16 @@ class MistralExtractWorker(ExtractWorkerAbstract):
should_include_images=should_include_images,
)
- async def extract_from_pdf_file(
+ async def _get_signed_url_from_pdf_file(
self,
pdf_path: str,
- should_include_images: bool = False,
- ) -> ExtractOutput:
- # Upload the file
+ ) -> str:
+ """Upload a PDF file to Mistral and return a signed URL for it."""
uploaded_file_id = await MistralFactory.upload_file_to_mistral_for_ocr(
mistral_client=self.mistral_client,
file_path=pdf_path,
)
-
- # Get signed URL
signed_url = await self.mistral_client.files.get_signed_url_async(
file_id=uploaded_file_id,
)
- return await self.extract_from_pdf_url(
- pdf_url=signed_url.url,
- should_include_images=should_include_images,
- )
+ return signed_url.url
diff --git a/pipelex/plugins/mistral/mistral_factory.py b/pipelex/plugins/mistral/mistral_factory.py
index 7a10b5a8a..fb45fbfce 100644
--- a/pipelex/plugins/mistral/mistral_factory.py
+++ b/pipelex/plugins/mistral/mistral_factory.py
@@ -1,3 +1,4 @@
+import asyncio
import os
import aiofiles
@@ -20,19 +21,18 @@ from openai.types.chat import (
ChatCompletionSystemMessageParam,
ChatCompletionUserMessageParam,
)
+from openai.types.chat.chat_completion_content_part_image_param import ImageURL as OpenAIImageURL
-from pipelex.cogt.exceptions import PromptImageFormatError
from pipelex.cogt.extract.bounding_box import BoundingBox
from pipelex.cogt.extract.extract_output import ExtractedImageFromPage, ExtractOutput, Page
from pipelex.cogt.image.image_size import ImageSize
-from pipelex.cogt.image.prompt_image import PromptImage, PromptImageBase64, PromptImagePath, PromptImageUrl
+from pipelex.cogt.image.prepared_image import PreparedImageBase64, PreparedImageHttpUrl
+from pipelex.cogt.image.prompt_image import PromptImage, PromptImageDetail
+from pipelex.cogt.image.prompt_image_utils import prep_prompt_images, prepare_prompt_image
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.model_backends.backend import InferenceBackend
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
from pipelex.plugins.mistral.mistral_exceptions import MistralExtractResponseError
-from pipelex.plugins.openai.openai_utils import make_image_url_obj
-from pipelex.tools.misc.base64_utils import load_binary_as_base64
-from pipelex.tools.misc.filetype_utils import detect_file_type_from_base64, detect_file_type_from_path
class MistralFactory:
@@ -51,14 +51,15 @@ class MistralFactory:
# Message
#########################################################
- def make_simple_messages(self, llm_job: LLMJob) -> list[Messages]:
+ async def make_simple_messages(self, llm_job: LLMJob) -> list[Messages]:
"""Makes a list of messages with a system message (if provided) and followed by a user message."""
messages: list[Messages] = []
user_content: list[ContentChunk] = []
if user_text := llm_job.llm_prompt.user_text:
user_content.append(TextChunk(text=user_text))
if user_images := llm_job.llm_prompt.user_images:
- user_content.extend(self.make_mistral_image_url(user_image) for user_image in user_images)
+ image_chunks = await asyncio.gather(*(self.make_mistral_image_url(prompt_image=img) for img in user_images))
+ user_content.extend(image_chunks)
if user_content:
messages.append(UserMessage(content=user_content))
@@ -67,25 +68,33 @@ class MistralFactory:
return messages
- def make_mistral_image_url(self, prompt_image: PromptImage) -> ImageURLChunk:
- if isinstance(prompt_image, PromptImageUrl):
- return ImageURLChunk(image_url=prompt_image.url)
- if isinstance(prompt_image, PromptImagePath):
- image_bytes = load_binary_as_base64(prompt_image.file_path).decode("utf-8")
- file_type = detect_file_type_from_path(prompt_image.file_path)
- return ImageURLChunk(image_url=f"data:{file_type.mime};base64,{image_bytes}")
- if isinstance(prompt_image, PromptImageBase64):
- image_bytes = prompt_image.base_64.decode("utf-8")
- file_type = detect_file_type_from_base64(prompt_image.base_64)
- return ImageURLChunk(image_url=f"data:{file_type.mime};base64,{image_bytes}")
- msg = f"prompt_image of type {type(prompt_image)} is not supported"
- raise PromptImageFormatError(msg)
+ async def make_mistral_image_url(self, prompt_image: PromptImage) -> ImageURLChunk:
+ """Convert a PromptImage to a Mistral ImageURLChunk.
+
+ Uses the unified prepare_prompt_image() which supports all URI types
+ including pipelex-storage://.
+ """
+ # Mistral accepts HTTP URLs directly, so we enable them
+ prepared = await prepare_prompt_image(prompt_image=prompt_image, is_http_url_enabled=True)
+
+ image_url: str
+ match prepared:
+ case PreparedImageBase64():
+ image_url = prepared.as_data_url()
+ case PreparedImageHttpUrl():
+ image_url = prepared.url
+
+ return ImageURLChunk(image_url=image_url)
async def make_simple_messages_openai_typed(
self,
llm_job: LLMJob,
) -> list[ChatCompletionMessageParam]:
- """Makes a list of messages with a system message (if provided) and followed by a user message."""
+ """Makes a list of messages with a system message (if provided) and followed by a user message.
+
+ Uses the unified prep_prompt_images() which supports all URI types
+ including pipelex-storage://.
+ """
llm_prompt = llm_job.llm_prompt
messages: list[ChatCompletionMessageParam] = []
user_contents: list[ChatCompletionContentPartParam] = []
@@ -98,8 +107,18 @@ class MistralFactory:
user_contents.append(user_part_text)
if user_images := llm_prompt.user_images:
- for prompt_image in user_images:
- image_url_obj = await make_image_url_obj(prompt_image=prompt_image, detail=llm_job.job_params.image_detail)
+ detail = llm_job.job_params.image_detail or PromptImageDetail.AUTO
+ # Mistral accepts HTTP URLs directly
+ prepared_images = await prep_prompt_images(prompt_images=user_images, is_http_url_enabled=True)
+ for prepared in prepared_images:
+ url: str
+ match prepared:
+ case PreparedImageBase64():
+ url = prepared.as_data_url()
+ case PreparedImageHttpUrl():
+ url = prepared.url
+
+ image_url_obj = OpenAIImageURL(url=url, detail=detail.as_openai_detail)
image_param = ChatCompletionContentPartImageParam(image_url=image_url_obj, type="image_url")
user_contents.append(image_param)
diff --git a/pipelex/plugins/mistral/mistral_llm_worker.py b/pipelex/plugins/mistral/mistral_llm_worker.py
index cd519cb8b..82ea2dccb 100644
--- a/pipelex/plugins/mistral/mistral_llm_worker.py
+++ b/pipelex/plugins/mistral/mistral_llm_worker.py
@@ -54,7 +54,7 @@ class MistralLLMWorker(LLMWorkerInternalAbstract):
llm_job: LLMJob,
) -> str:
job_params = llm_job.applied_job_params or llm_job.job_params
- messages = self.mistral_factory.make_simple_messages(llm_job=llm_job)
+ messages = await self.mistral_factory.make_simple_messages(llm_job=llm_job)
response: ChatCompletionResponse | None = await self.mistral_client_for_text.chat.complete_async(
messages=messages,
model=self.inference_model.model_id,
diff --git a/pipelex/plugins/openai/openai_completions_factory.py b/pipelex/plugins/openai/openai_completions_factory.py
index 33b9e1872..6b5ee4dd2 100644
--- a/pipelex/plugins/openai/openai_completions_factory.py
+++ b/pipelex/plugins/openai/openai_completions_factory.py
@@ -8,12 +8,12 @@ from openai.types.chat import (
ChatCompletionSystemMessageParam,
ChatCompletionUserMessageParam,
)
-from openai.types.chat.chat_completion_content_part_image_param import ImageURL
+from openai.types.chat.chat_completion_content_part_image_param import ImageURL as OpenAIImageURL
from openai.types.completion_usage import CompletionUsage
from typing_extensions import override
-from pipelex.cogt.image.prompt_image import PromptImageDetail, PromptImageTypedBase64
-from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
+from pipelex.cogt.image.prepared_image import PreparedImageBase64, PreparedImageHttpUrl
+from pipelex.cogt.image.prompt_image import PromptImageDetail
from pipelex.cogt.image.prompt_image_utils import prep_prompt_images
from pipelex.cogt.inference.inference_job_abstract import InferenceJobAbstract
from pipelex.cogt.llm.llm_job import LLMJob
@@ -46,13 +46,14 @@ class OpenAICompletionsFactory(PluginFactoryAbstract):
detail = llm_job.job_params.image_detail or PromptImageDetail.AUTO
prepped_images = await prep_prompt_images(prompt_images=llm_prompt.user_images, is_http_url_enabled=self.is_http_url_enabled)
for prepped_image in prepped_images:
- if isinstance(prepped_image, str):
- url = prepped_image
- else:
- assert isinstance(prepped_image, PromptImageTypedBase64)
- url = PromptImageFactory.make_base_64_url_from_prompt_image_typed_base64(prompt_image=prepped_image)
+ url: str
+ match prepped_image:
+ case PreparedImageHttpUrl():
+ url = prepped_image.url
+ case PreparedImageBase64():
+ url = prepped_image.as_data_url()
- image_url_obj = ImageURL(url=url, detail=detail.as_openai_detail)
+ image_url_obj = OpenAIImageURL(url=url, detail=detail.as_openai_detail)
image_param = ChatCompletionContentPartImageParam(image_url=image_url_obj, type="image_url")
user_contents.append(image_param)
diff --git a/pipelex/plugins/openai/openai_responses_factory.py b/pipelex/plugins/openai/openai_responses_factory.py
index b50aebacb..914958e30 100644
--- a/pipelex/plugins/openai/openai_responses_factory.py
+++ b/pipelex/plugins/openai/openai_responses_factory.py
@@ -6,8 +6,8 @@ from openai.types.responses import ResponseInputImageParam
from typing_extensions import override
from pipelex.cogt.exceptions import LLMPromptParameterError
-from pipelex.cogt.image.prompt_image import PromptImageDetail, PromptImageTypedBase64
-from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
+from pipelex.cogt.image.prepared_image import PreparedImageBase64, PreparedImageHttpUrl
+from pipelex.cogt.image.prompt_image import PromptImageDetail
from pipelex.cogt.image.prompt_image_utils import prep_prompt_images
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
@@ -44,11 +44,12 @@ class OpenAIResponsesFactory(PluginFactoryAbstract):
detail = llm_job.job_params.image_detail or PromptImageDetail.AUTO
prepped_images = await prep_prompt_images(prompt_images=llm_prompt.user_images, is_http_url_enabled=self.is_http_url_enabled)
for prepped_image in prepped_images:
- if isinstance(prepped_image, str):
- url = prepped_image
- else:
- assert isinstance(prepped_image, PromptImageTypedBase64)
- url = PromptImageFactory.make_base_64_url_from_prompt_image_typed_base64(prompt_image=prepped_image)
+ url: str
+ match prepped_image:
+ case PreparedImageHttpUrl():
+ url = prepped_image.url
+ case PreparedImageBase64():
+ url = prepped_image.as_data_url()
image_param = ResponseInputImageParam(type="input_image", image_url=url, detail=detail.as_openai_detail)
user_contents.append(image_param)
diff --git a/pipelex/plugins/openai/openai_utils.py b/pipelex/plugins/openai/openai_utils.py
deleted file mode 100644
index 689fa763c..000000000
--- a/pipelex/plugins/openai/openai_utils.py
+++ /dev/null
@@ -1,24 +0,0 @@
-from openai.types.chat.chat_completion_content_part_image_param import ImageURL
-
-from pipelex.cogt.exceptions import LLMPromptParameterError
-from pipelex.cogt.image.prompt_image import PromptImage, PromptImageBase64, PromptImageDetail, PromptImagePath, PromptImageUrl
-from pipelex.tools.misc.base64_utils import load_binary_as_base64
-
-
-async def make_image_url_obj(prompt_image: PromptImage, detail: PromptImageDetail | None) -> ImageURL:
- if detail is None:
- detail = PromptImageDetail.AUTO
- if isinstance(prompt_image, PromptImageUrl):
- url = prompt_image.url
- image_url_obj = ImageURL(url=url, detail=detail.as_openai_detail)
- elif isinstance(prompt_image, PromptImageBase64):
- # TODO: manage image type
- url_with_bytes: str = f"data:image/jpeg;base64,{prompt_image.base_64.decode('utf-8')}"
- image_url_obj = ImageURL(url=url_with_bytes, detail=detail.as_openai_detail)
- elif isinstance(prompt_image, PromptImagePath):
- image_bytes = load_binary_as_base64(path=prompt_image.file_path)
- return await make_image_url_obj(prompt_image=PromptImageBase64(base_64=image_bytes), detail=detail)
- else:
- msg = f"prompt_image of type {type(prompt_image)} is not supported"
- raise LLMPromptParameterError(msg)
- return image_url_obj
diff --git a/pipelex/system/configuration/configs.py b/pipelex/system/configuration/configs.py
index 47316b911..f30df0d5a 100644
--- a/pipelex/system/configuration/configs.py
+++ b/pipelex/system/configuration/configs.py
@@ -159,6 +159,10 @@ class BuilderConfig(ConfigModel):
default_directory_base_name: str
+class PipelineExecutionConfig(ConfigModel):
+ is_normalize_data_urls_to_storage: bool
+
+
class Pipelex(ConfigModel):
storage_config: StorageConfig
feature_config: FeatureConfig
@@ -173,6 +177,7 @@ class Pipelex(ConfigModel):
dry_run_config: DryRunConfig
pipe_run_config: PipeRunConfig
+ pipeline_execution_config: PipelineExecutionConfig
reporting_config: ReportingConfig
observer_config: ObserverConfig
scan_config: ScanConfig
diff --git a/pipelex/system/telemetry/telemetry_manager.py b/pipelex/system/telemetry/telemetry_manager.py
index 68033671d..3934d1c90 100644
--- a/pipelex/system/telemetry/telemetry_manager.py
+++ b/pipelex/system/telemetry/telemetry_manager.py
@@ -304,7 +304,7 @@ class TelemetryManager(TelemetryManagerAbstract):
and self.telemetry_config.pipelex_gateway.portkey
and self.telemetry_config.pipelex_gateway.portkey.force_debug_enabled
):
- log.info(
+ log.dev(
"Force-enabling Portkey logging (debug mode) because pipelex_gateway.portkey.force_debug_enabled is set in telemetry configuration"
)
is_debug = True
@@ -324,7 +324,7 @@ class TelemetryManager(TelemetryManagerAbstract):
and self.telemetry_config.pipelex_gateway.portkey.force_tracing_enabled
and not is_env_var_truthy(OTelConstants.DO_NOT_TRACK_ENV_VAR_KEY)
):
- log.info(
+ log.dev(
"Force-enabling Pipelex Gateway Portkey tracing "
"because pipelex_gateway.portkey.force_tracing_enabled is set in telemetry configuration"
)
diff --git a/pipelex/tools/misc/base64_utils.py b/pipelex/tools/misc/base64_utils.py
index c0a72fa70..7516162f4 100644
--- a/pipelex/tools/misc/base64_utils.py
+++ b/pipelex/tools/misc/base64_utils.py
@@ -2,59 +2,40 @@ import base64
import aiofiles
-from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx_async
-from pipelex.tools.misc.file_utils import load_binary, load_binary_async
+from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx
+from pipelex.tools.misc.file_utils import load_binary_async
from pipelex.tools.misc.filetype_utils import FileType, detect_file_type_from_bytes
-from pipelex.tools.misc.path_utils import clarify_path_or_url
-def load_binary_as_base64(path: str) -> bytes:
- with open(path, "rb") as file_pointer:
- return base64.b64encode(file_pointer.read())
-
-
-async def load_binary_as_base64_async(path: str) -> bytes:
+async def load_binary_as_base64(path: str) -> str:
+ """Load a file and return its contents as a base64-encoded string."""
async with aiofiles.open(path, "rb") as fp: # pyright: ignore[reportUnknownMemberType]
data_bytes = await fp.read()
- return base64.b64encode(data_bytes)
+ return base64.b64encode(data_bytes).decode("ascii")
-def make_base64_url_from_path(path: str) -> str:
- raw_bytes = load_binary(path=path)
- base64_bytes = base64.b64encode(raw_bytes)
- file_type = detect_file_type_from_bytes(raw_bytes=raw_bytes)
- return make_base64_url(base64_bytes=base64_bytes, file_type=file_type)
-
-
-async def make_base64_url_from_path_async(path: str) -> str:
+async def make_base64_url_from_path(path: str) -> str:
+ """Create a data: URL from a local file path."""
raw_bytes = await load_binary_async(path=path)
- base64_bytes = base64.b64encode(raw_bytes)
+ base64_data = base64.b64encode(raw_bytes).decode("ascii")
file_type = detect_file_type_from_bytes(raw_bytes=raw_bytes)
- return make_base64_url(base64_bytes=base64_bytes, file_type=file_type)
+ return make_base64_url(base64_data=base64_data, file_type=file_type)
-async def make_base64_url_from_http_url_async(url: str) -> str:
- raw_bytes = await fetch_file_from_url_httpx_async(url=url)
- base64_bytes = base64.b64encode(raw_bytes)
+async def make_base64_url_from_http_url(url: str) -> str:
+ """Fetch a URL and create a data: URL from its contents."""
+ raw_bytes = await fetch_file_from_url_httpx(url=url)
+ base64_data = base64.b64encode(raw_bytes).decode("ascii")
file_type = detect_file_type_from_bytes(raw_bytes=raw_bytes)
- return make_base64_url(base64_bytes=base64_bytes, file_type=file_type)
-
-
-async def make_base64_url_from_location_async(location: str) -> str:
- pdf_path, pdf_url = clarify_path_or_url(path_or_uri=location)
- if pdf_url:
- base64_url = await make_base64_url_from_http_url_async(url=pdf_url)
- else: # pdf_path must be provided based on validation
- assert pdf_path is not None
- base64_url = await make_base64_url_from_path_async(path=pdf_path)
- return base64_url
+ return make_base64_url(base64_data=base64_data, file_type=file_type)
def make_base64_url(
- base64_bytes: bytes,
+ base64_data: str,
file_type: FileType,
) -> str:
- return f"data:{file_type.mime};base64,{base64_bytes.decode('utf-8')}"
+ """Create a data: URL from base64 string and file type."""
+ return f"data:{file_type.mime};base64,{base64_data}"
def is_prefixed_base64_url(possibly_base64_url: str) -> bool:
diff --git a/pipelex/tools/misc/file_fetch_utils.py b/pipelex/tools/misc/file_fetch_utils.py
index 8612ef0a9..d867ba67f 100644
--- a/pipelex/tools/misc/file_fetch_utils.py
+++ b/pipelex/tools/misc/file_fetch_utils.py
@@ -4,7 +4,7 @@ from httpx import Response
from pipelex.tools.misc.http_utils import get_user_agent
-async def fetch_file_from_url_httpx_async(
+async def fetch_file_from_url_httpx(
url: str,
request_timeout: int | None = None,
) -> bytes:
@@ -18,20 +18,3 @@ async def fetch_file_from_url_httpx_async(
response.raise_for_status() # Raise exception for 4XX/5XX status codes
return response.content
-
-
-def fetch_file_from_url_httpx(
- url: str,
- request_timeout: int | None = None,
-) -> bytes:
- user_agent = get_user_agent()
- with httpx.Client(headers={"User-Agent": user_agent}) as client:
- response: Response = client.get(
- url,
- timeout=request_timeout,
- follow_redirects=True,
- )
- response.raise_for_status() # Raise exception for 4XX/5XX status codes
-
- bytes_content: bytes = response.content
- return bytes_content
diff --git a/pipelex/tools/misc/path_utils.py b/pipelex/tools/misc/path_utils.py
deleted file mode 100644
index 41eeffa15..000000000
--- a/pipelex/tools/misc/path_utils.py
+++ /dev/null
@@ -1,159 +0,0 @@
-import os
-import urllib.parse
-
-from pipelex.tools.storage.storage_provider_abstract import PIPELEX_STORAGE_SCHEME
-from pipelex.types import StrEnum
-
-
-class InterpretedPathOrUrl(StrEnum):
- HTTP_URL = "http_url"
- LOCAL_FILE_PATH_URL = "local_file_path_url"
- FILE_NAME = "file_name"
- FILE_PATH = "file_path"
- BASE_64 = "base_64"
- PIPELEX_STORAGE = "pipelex_storage"
-
- @property
- def desc(self) -> str:
- match self:
- case InterpretedPathOrUrl.HTTP_URL:
- return "HTTP URL"
- case InterpretedPathOrUrl.LOCAL_FILE_PATH_URL:
- return "Local file path URL"
- case InterpretedPathOrUrl.FILE_NAME:
- return "File Name"
- case InterpretedPathOrUrl.FILE_PATH:
- return "File path"
- case InterpretedPathOrUrl.BASE_64:
- return "Base 64"
- case InterpretedPathOrUrl.PIPELEX_STORAGE:
- return "Pipelex Storage"
-
-
-def interpret_path_or_url(path_or_uri: str) -> InterpretedPathOrUrl:
- """Determines whether a string represents a file URI, URL, file path, or Pipelex storage URI.
-
- This function analyzes the input string to categorize it as one of several types:
-
- - Pipelex storage URI (starts with "pipelex-storage://")
- - File URI (starts with "file://")
- - URL (starts with "http")
- - File path (contains OS path separator)
- - File name (anything else)
-
- Args:
- path_or_uri: The string to interpret, which could be a Pipelex storage URI,
- file URI, URL, or file path.
-
- Returns:
- InterpretedPathOrUrl: An enum value indicating the type of the input string:
-
- - PIPELEX_STORAGE for pipelex-storage:// URIs
- - FILE_URI for file:// URIs
- - URL for http(s) URLs
- - FILE_PATH for paths with OS separators
- - FILE_NAME for simple file names
- - BASE_64 for base64-encoded images
-
- Example:
- >>> interpret_path_or_url("pipelex-storage://images/photo.png")
- InterpretedPathOrUrl.PIPELEX_STORAGE
- >>> interpret_path_or_url("file:///home/user/file.txt")
- InterpretedPathOrUrl.FILE_URI
- >>> interpret_path_or_url("https://example.com")
- InterpretedPathOrUrl.URL
- >>> interpret_path_or_url("/home/user/file.txt")
- InterpretedPathOrUrl.FILE_PATH
- """
- if path_or_uri.startswith(PIPELEX_STORAGE_SCHEME):
- return InterpretedPathOrUrl.PIPELEX_STORAGE
- elif path_or_uri.startswith("file://"):
- return InterpretedPathOrUrl.LOCAL_FILE_PATH_URL
- elif path_or_uri.startswith("http"):
- return InterpretedPathOrUrl.HTTP_URL
- elif os.sep in path_or_uri:
- return InterpretedPathOrUrl.FILE_PATH
- else:
- return InterpretedPathOrUrl.FILE_NAME
-
-
-def clarify_path_or_url(path_or_uri: str) -> tuple[str | None, str | None]:
- """Separates a path_or_uri string into either a file path or online URL component.
-
- This function processes the input string to determine its type and returns
- the appropriate components. For file URIs, it converts them to regular file paths.
- Only one of the returned values will be non-None.
-
- Note:
- For Pipelex storage URIs (pipelex-storage://), use `is_pipelex_storage_uri()`
- to check and `get_storage_provider().load()` to read the content.
-
- Args:
- path_or_uri: The string to process, which could be a file URI, URL, or file path.
-
- Returns:
- A tuple containing:
-
- - file_path: The file path if the input is a file path or URI, None otherwise
- - url: The URL if the input is a URL, None otherwise
-
- Raises:
- NotImplementedError: If the input is a Base64 string or Pipelex storage URI.
-
- Example:
- >>> clarify_path_or_url("file:///home/user/file.txt")
- ('/home/user/file.txt', None)
- >>> clarify_path_or_url("https://example.com")
- (None, 'https://example.com')
- >>> clarify_path_or_url("/home/user/file.txt")
- ('/home/user/file.txt', None)
- """
- file_path: str | None
- url: str | None
- match interpret_path_or_url(path_or_uri):
- case InterpretedPathOrUrl.LOCAL_FILE_PATH_URL:
- parsed_uri = urllib.parse.urlparse(path_or_uri)
- file_path = urllib.parse.unquote(parsed_uri.path)
- url = None
- case InterpretedPathOrUrl.HTTP_URL:
- file_path = None
- url = path_or_uri
- case InterpretedPathOrUrl.FILE_PATH:
- # it's a file path
- file_path = path_or_uri
- url = None
- case InterpretedPathOrUrl.FILE_NAME:
- file_path = path_or_uri
- url = None
- case InterpretedPathOrUrl.BASE_64:
- msg = "Base 64 is not supported by clarify_path_or_url"
- raise NotImplementedError(msg)
- case InterpretedPathOrUrl.PIPELEX_STORAGE:
- msg = "Pipelex storage URIs are not supported by clarify_path_or_url; use is_pipelex_storage_uri() and storage_provider.load()"
- raise NotImplementedError(msg)
- return file_path, url
-
-
-def is_pipelex_storage_uri(path_or_uri: str) -> bool:
- """Check if a string is a Pipelex storage URI.
-
- Args:
- path_or_uri: The string to check.
-
- Returns:
- True if the string starts with the Pipelex storage scheme.
- """
- return path_or_uri.startswith(PIPELEX_STORAGE_SCHEME)
-
-
-def resolve_path_or_url_for_reading(path_or_uri: str) -> str:
- """Resolve a path-or-URI to a single string suitable for file-reading/conversion calls.
-
- This is a convenience wrapper around `clarify_path_or_url()` that returns a single value:
- - the http(s) URL if `path_or_uri` is a URL
- - otherwise a local file path (including normalized `file://...` URIs)
- """
- file_path, url = clarify_path_or_url(path_or_uri=path_or_uri)
- resolved = url or file_path
- assert resolved is not None
- return resolved
diff --git a/pipelex/tools/pdf/pypdfium2_renderer.py b/pipelex/tools/pdf/pypdfium2_renderer.py
index f120d58e9..4435a9607 100644
--- a/pipelex/tools/pdf/pypdfium2_renderer.py
+++ b/pipelex/tools/pdf/pypdfium2_renderer.py
@@ -12,9 +12,15 @@ from pipelex.cogt.extract.bounding_box import BoundingBox
from pipelex.cogt.extract.extract_output import ExtractedImageFromPage
from pipelex.cogt.image.image_size import ImageSize
from pipelex.system.exceptions import ToolError
-from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx_async
+from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx
from pipelex.tools.misc.image_utils import ImageFormat, pil_image_to_bytes
-from pipelex.tools.misc.path_utils import clarify_path_or_url
+from pipelex.tools.uri.resolved_uri import (
+ ResolvedBase64DataUrl,
+ ResolvedHttpUrl,
+ ResolvedLocalPath,
+ ResolvedPipelexStorage,
+)
+from pipelex.tools.uri.uri_resolver import resolve_uri
if TYPE_CHECKING:
from PIL import Image
@@ -243,25 +249,29 @@ class PyPdfium2Renderer:
return await asyncio.to_thread(self._extract_text_from_pdf_pages_sync, pdf_input)
async def render_pdf_pages_from_uri(self, pdf_uri: str, dpi: int) -> list[Image.Image]:
- pdf_path, pdf_url = clarify_path_or_url(path_or_uri=pdf_uri)
- if pdf_url:
- pdf_bytes = await fetch_file_from_url_httpx_async(url=pdf_url)
- return await self.render_pdf_pages(pdf_input=pdf_bytes, dpi=dpi)
- if pdf_path:
- return await self.render_pdf_pages(pdf_input=pdf_path, dpi=dpi)
- msg = f"Invalid PDF URI: {pdf_uri}"
- raise PyPdfium2RendererError(msg)
+ resolved_uri = resolve_uri(pdf_uri)
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ pdf_bytes = await fetch_file_from_url_httpx(url=resolved_uri.url)
+ return await self.render_pdf_pages(pdf_input=pdf_bytes, dpi=dpi)
+ case ResolvedLocalPath():
+ return await self.render_pdf_pages(pdf_input=resolved_uri.path, dpi=dpi)
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ msg = f"Unsupported URI type for PDF rendering: {resolved_uri.kind}"
+ raise PyPdfium2RendererError(msg)
async def extract_text_from_pdf_pages_from_uri(self, pdf_uri: str) -> list[str]:
"""Extract text from all pages of a PDF from URI."""
pdf_input: PdfInput
- pdf_path, pdf_url = clarify_path_or_url(path_or_uri=pdf_uri)
- if pdf_url:
- pdf_bytes = await fetch_file_from_url_httpx_async(url=pdf_url)
- pdf_input = pdf_bytes
- else:
- assert pdf_path is not None
- pdf_input = pdf_path
+ resolved_uri = resolve_uri(pdf_uri)
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ pdf_input = await fetch_file_from_url_httpx(url=resolved_uri.url)
+ case ResolvedLocalPath():
+ pdf_input = resolved_uri.path
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ msg = f"Unsupported URI type for PDF text extraction: {resolved_uri.kind}"
+ raise PyPdfium2RendererError(msg)
return await self.extract_text_from_pdf_pages(pdf_input=pdf_input)
async def extract_embedded_images_from_page(
@@ -335,13 +345,15 @@ class PyPdfium2Renderer:
Dictionary mapping page index (1-based) to list of ExtractedImageFromPage
"""
pdf_input: PdfInput
- pdf_path, pdf_url = clarify_path_or_url(path_or_uri=pdf_uri)
- if pdf_url:
- pdf_bytes = await fetch_file_from_url_httpx_async(url=pdf_url)
- pdf_input = pdf_bytes
- else:
- assert pdf_path is not None
- pdf_input = pdf_path
+ resolved_uri = resolve_uri(pdf_uri)
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ pdf_input = await fetch_file_from_url_httpx(url=resolved_uri.url)
+ case ResolvedLocalPath():
+ pdf_input = resolved_uri.path
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ msg = f"Unsupported URI type for PDF image extraction: {resolved_uri.kind}"
+ raise PyPdfium2RendererError(msg)
return await self._extract_embedded_images_from_pdf(
pdf_input=pdf_input,
max_depth=max_depth,
diff --git a/pipelex/tools/uri/__init__.py b/pipelex/tools/uri/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/pipelex/tools/uri/resolved_uri.py b/pipelex/tools/uri/resolved_uri.py
new file mode 100644
index 000000000..3014f0242
--- /dev/null
+++ b/pipelex/tools/uri/resolved_uri.py
@@ -0,0 +1,93 @@
+from pydantic import BaseModel
+
+from pipelex.types import StrEnum
+
+
+class UriKind(StrEnum):
+ """Enumeration of supported URI types."""
+
+ HTTP_URL = "http_url"
+ LOCAL_PATH = "local_path"
+ PIPELEX_STORAGE = "pipelex_storage"
+ BASE64_DATA_URL = "base64_data_url"
+
+ @property
+ def desc(self) -> str:
+ """Return a human-readable description of this URI kind."""
+ match self:
+ case UriKind.HTTP_URL:
+ return "HTTP URL"
+ case UriKind.LOCAL_PATH:
+ return "Local path"
+ case UriKind.PIPELEX_STORAGE:
+ return "Pipelex Storage"
+ case UriKind.BASE64_DATA_URL:
+ return "Base64 data URL"
+
+
+class ResolvedUriBase(BaseModel):
+ """Base class for all resolved URI types.
+
+ Attributes:
+ kind: The type of URI (discriminator field).
+ original: The original input string that was resolved.
+ """
+
+ kind: UriKind
+ original: str
+
+
+class ResolvedHttpUrl(ResolvedUriBase):
+ """Resolved HTTP or HTTPS URL.
+
+ Attributes:
+ url: The HTTP/HTTPS URL.
+ """
+
+ kind: UriKind = UriKind.HTTP_URL
+ url: str
+
+
+class ResolvedLocalPath(ResolvedUriBase):
+ """Resolved local file path.
+
+ This includes:
+ - Absolute paths (/home/user/file.txt)
+ - Relative paths (dir/file.txt)
+ - Simple file names (file.txt)
+ - Converted file:// URIs
+
+ Attributes:
+ path: The local file path.
+ """
+
+ kind: UriKind = UriKind.LOCAL_PATH
+ path: str
+
+
+class ResolvedPipelexStorage(ResolvedUriBase):
+ """Resolved pipelex-storage:// URI.
+
+ Attributes:
+ storage_uri: The full pipelex-storage:// URI (including scheme).
+ """
+
+ kind: UriKind = UriKind.PIPELEX_STORAGE
+ storage_uri: str
+
+
+class ResolvedBase64DataUrl(ResolvedUriBase):
+ """Resolved base64 data URL (data:{mime};base64,{data}).
+
+ Attributes:
+ mime_type: The MIME type from the data URL (e.g., "image/png").
+ base64_data: The base64-encoded data (without the data URL prefix).
+ """
+
+ kind: UriKind = UriKind.BASE64_DATA_URL
+ mime_type: str
+ base64_data: str
+
+
+# Type alias for the union of all resolved URI types
+ResolvedUri = ResolvedHttpUrl | ResolvedLocalPath | ResolvedPipelexStorage | ResolvedBase64DataUrl
diff --git a/pipelex/tools/uri/uri_resolver.py b/pipelex/tools/uri/uri_resolver.py
new file mode 100644
index 000000000..6bbb0baab
--- /dev/null
+++ b/pipelex/tools/uri/uri_resolver.py
@@ -0,0 +1,151 @@
+import urllib.parse
+
+from pipelex.tools.misc.base64_utils import (
+ make_base64_url_from_http_url,
+ make_base64_url_from_path,
+)
+from pipelex.tools.storage.storage_provider_abstract import PIPELEX_STORAGE_SCHEME
+from pipelex.tools.uri.resolved_uri import (
+ ResolvedBase64DataUrl,
+ ResolvedHttpUrl,
+ ResolvedLocalPath,
+ ResolvedPipelexStorage,
+ ResolvedUri,
+)
+
+# Base64 data URL markers
+BASE64_DATA_URL_PREFIX = "data:"
+BASE64_DATA_URL_MARKER = ";base64,"
+
+
+def resolve_uri(uri: str) -> ResolvedUri:
+ """Resolve a URI string to its typed representation.
+
+ This function parses the input string and returns the appropriate
+ ResolvedUri variant based on the URI type:
+
+ - HTTP/HTTPS URLs -> ResolvedHttpUrl
+ - file:// URIs -> ResolvedLocalPath (converted to path)
+ - Local paths -> ResolvedLocalPath
+ - pipelex-storage:// URIs -> ResolvedPipelexStorage
+ - data:...;base64,... URLs -> ResolvedBase64DataUrl
+
+ Args:
+ uri: The URI string to resolve.
+
+ Returns:
+ A ResolvedUri variant with the parsed URI information.
+
+ Example:
+ >>> resolved_uri = resolve_uri("https://example.com/image.png")
+ >>> resolved_uri.kind
+ UriKind.HTTP_URL
+ >>> resolved_uri.url
+ 'https://example.com/image.png'
+
+ >>> resolved_uri = resolve_uri("pipelex-storage://images/photo.png")
+ >>> resolved_uri.kind
+ UriKind.PIPELEX_STORAGE
+ >>> resolved_uri.storage_uri
+ 'pipelex-storage://images/photo.png'
+ """
+ # Check for base64 data URLs first (data:{mime};base64,{data})
+ if uri.startswith(BASE64_DATA_URL_PREFIX) and BASE64_DATA_URL_MARKER in uri:
+ return _resolve_base64_data_url(uri)
+
+ # Check for pipelex storage URIs
+ if uri.startswith(PIPELEX_STORAGE_SCHEME):
+ return ResolvedPipelexStorage(
+ original=uri,
+ storage_uri=uri,
+ )
+
+ # Check for file:// URIs
+ if uri.startswith("file://"):
+ return _resolve_file_uri(uri)
+
+ # Check for HTTP/HTTPS URLs
+ if uri.startswith(("http://", "https://")):
+ return ResolvedHttpUrl(
+ original=uri,
+ url=uri,
+ )
+
+ # Everything else is a local path (absolute, relative, or filename)
+ return ResolvedLocalPath(
+ original=uri,
+ path=uri,
+ )
+
+
+def _resolve_file_uri(uri: str) -> ResolvedLocalPath:
+ """Convert a file:// URI to a local path.
+
+ Args:
+ uri: A file:// URI string.
+
+ Returns:
+ ResolvedLocalPath with the converted path.
+ """
+ parsed = urllib.parse.urlparse(uri)
+ path = urllib.parse.unquote(parsed.path)
+ return ResolvedLocalPath(
+ original=uri,
+ path=path,
+ )
+
+
+def _resolve_base64_data_url(uri: str) -> ResolvedBase64DataUrl:
+ """Parse a base64 data URL into its components.
+
+ Args:
+ uri: A data URL in the format data:{mime};base64,{data}
+
+ Returns:
+ ResolvedBase64DataUrl with mime_type and base64_data.
+ """
+ # Format: data:{mime_type};base64,{base64_data}
+ # Remove "data:" prefix
+ content = uri[len(BASE64_DATA_URL_PREFIX) :]
+
+ # Split on ;base64,
+ parts = content.split(BASE64_DATA_URL_MARKER, 1)
+ mime_type = parts[0]
+ base64_data = parts[1] if len(parts) > 1 else ""
+
+ return ResolvedBase64DataUrl(
+ original=uri,
+ mime_type=mime_type,
+ base64_data=base64_data,
+ )
+
+
+async def make_base64_url_from_any_uri(uri: str) -> str:
+ """Convert a URI to a base64 data URL.
+
+ Resolves the URI and fetches/converts content to base64 format.
+ If the URI is already a data URL, returns it as-is.
+
+ Args:
+ uri: A URI string (http://, local path, or data: URL)
+
+ Returns:
+ A base64 data URL string containing the base64-encoded data, whichever way we got it.
+
+ Raises:
+ ValueError: If the URI type is not supported (e.g., pipelex-storage://)
+ """
+ base64_url: str
+ resolved_uri = resolve_uri(uri)
+ match resolved_uri:
+ case ResolvedBase64DataUrl():
+ # Already a data URL, return as-is
+ base64_url = resolved_uri.original
+ case ResolvedHttpUrl():
+ base64_url = await make_base64_url_from_http_url(url=resolved_uri.url)
+ case ResolvedLocalPath():
+ base64_url = await make_base64_url_from_path(path=resolved_uri.path)
+ case ResolvedPipelexStorage():
+ msg = f"Unsupported URI type for base64 URL creation: {resolved_uri.kind} (requires storage provider)"
+ raise ValueError(msg)
+ return base64_url
diff --git a/tests/cases/images.py b/tests/cases/images.py
index 1988a5291..f20768fd0 100644
--- a/tests/cases/images.py
+++ b/tests/cases/images.py
@@ -15,6 +15,7 @@ class ImageTestCases:
IMAGE_FILE_PATH_JPG_2 = f"{TEST_IMAGE_DIRECTORY}/solar_system.jpg"
IMAGE_FILE_PATH_PNG_2 = f"{TEST_IMAGE_DIRECTORY}/solar_system.png"
IMAGE_FILE_PATH_JPG_3 = f"{TEST_IMAGE_DIRECTORY}/eiffel_tower.jpg"
+ IMAGE_FILE_PATH_LOGO_TINY = f"{TEST_IMAGE_DIRECTORY}/logo-tiny.png"
# Remote URLs
IMAGE_URL_PNG = "https://pipelex-web.s3.amazonaws.com/tests/solar_system.png"
@@ -34,3 +35,35 @@ class ImageTestCases:
"https://www.w3.org/People/mimasa/test/imgformat/img/w3c_home.png",
"https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png",
]
+
+ # Base64-encoded test images for data URL testing
+ # A minimal valid PNG (1x1 red pixel) - useful for unit tests
+ MINIMAL_PNG_BASE64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFBQIAX8jx0gAAAABJRU5ErkJggg=="
+ MINIMAL_PNG_DATA_URL = f"data:image/png;base64,{MINIMAL_PNG_BASE64}"
+
+ # A minimal valid JPEG (1x1 pixel) - useful for unit tests
+ MINIMAL_JPEG_BASE64 = (
+ "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRof"
+ "Hh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/wAALCAABAAEBAREA/8QAFAAB"
+ "AAAAAAAAAAAAAAAAAAAACf/EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAD8AVN//2Q=="
+ )
+ MINIMAL_JPEG_DATA_URL = f"data:image/jpeg;base64,{MINIMAL_JPEG_BASE64}"
+
+ # Pipelex logo tiny (18x18 PNG) - useful for LLM vision tests with real image
+ LOGO_TINY_PNG_BASE64 = (
+ "iVBORw0KGgoAAAANSUhEUgAAABIAAAASCAIAAADZrBkAAAAAXklEQVR4AWOgPdg9FYSQwX8c"
+ "AE0Pus7/uAFhbcj2t7e3o+hkZMTpSDS/YHEqVhW4xRFswto2bdpEkjaEILHaTp06hRaSQDsJ"
+ "a/v48SNmBBDW1tTUhKmBNL8NZW30BwA1YiBFn3yfOQAAAABJRU5ErkJggg=="
+ )
+ LOGO_TINY_PNG_DATA_URL = f"data:image/png;base64,{LOGO_TINY_PNG_BASE64}"
+
+ # Data URL test cases for parametrize: (topic, data_url)
+ DATA_URL_TEST_CASES: ClassVar[list[tuple[str, str]]] = [
+ ("minimal_png", MINIMAL_PNG_DATA_URL),
+ ("minimal_jpeg", MINIMAL_JPEG_DATA_URL),
+ ]
+
+ # Data URL with actual image for LLM vision test: (topic, data_url)
+ DATA_URL_VISION_TEST_CASES: ClassVar[list[tuple[str, str]]] = [
+ ("logo_tiny", LOGO_TINY_PNG_DATA_URL),
+ ]
diff --git a/tests/conftest.py b/tests/conftest.py
index 2b5efc99b..f14c9880e 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -6,7 +6,7 @@ import shortuuid
from pytest_mock import MockerFixture
from pipelex import log
-from pipelex.hub import get_library_manager, set_current_library
+from pipelex.hub import get_library_manager, get_report_delegate, set_current_library
from pipelex.pipelex import Pipelex
from pipelex.pipeline.job_metadata import JobMetadata
from pipelex.system.pipelex_service.pipelex_service_config import (
@@ -140,15 +140,21 @@ def load_empty_library() -> Generator[Callable[[], str], None, None]:
@pytest.fixture
-def job_metadata(request: pytest.FixtureRequest) -> JobMetadata:
+def job_metadata(request: pytest.FixtureRequest) -> Generator[JobMetadata, None, None]:
"""Provide a JobMetadata instance with test-specific values.
Uses the test node ID as pipeline_run_id for better traceability in logs.
+ Opens a registry for the pipeline run ID before the test and closes it after.
"""
test_id: str = request.node.nodeid # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
random_code: str = shortuuid.uuid()[:5]
pipeline_run_id: str = f"{test_id}-{random_code}"
- return JobMetadata(
+
+ get_report_delegate().open_registry(pipeline_run_id=pipeline_run_id)
+
+ yield JobMetadata(
user_id="pytest",
pipeline_run_id=pipeline_run_id,
)
+
+ get_report_delegate().close_registry(pipeline_run_id=pipeline_run_id)
diff --git a/tests/data/images/logo-tiny.png b/tests/data/images/logo-tiny.png
new file mode 100644
index 000000000..3e4bd86ae
Binary files /dev/null and b/tests/data/images/logo-tiny.png differ
diff --git a/tests/integration/pipelex/cogt/image/__init__.py b/tests/integration/pipelex/cogt/image/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/integration/pipelex/cogt/image/test_data_url_image_flow.py b/tests/integration/pipelex/cogt/image/test_data_url_image_flow.py
new file mode 100644
index 000000000..36edb4e29
--- /dev/null
+++ b/tests/integration/pipelex/cogt/image/test_data_url_image_flow.py
@@ -0,0 +1,324 @@
+"""Integration tests for data URL handling in the image flow."""
+
+import base64
+from typing import cast
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.image.prepared_image import PreparedImageBase64
+from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
+from pipelex.cogt.image.prompt_image_utils import prepare_prompt_image
+from pipelex.core.concepts.concept_factory import ConceptFactory
+from pipelex.core.concepts.native.concept_native import NativeConceptCode
+from pipelex.core.memory.working_memory_factory import WorkingMemoryFactory
+from pipelex.core.stuffs.image_content import ImageContent
+from pipelex.core.stuffs.list_content import ListContent
+from pipelex.core.stuffs.structured_content import StructuredContent
+from pipelex.core.stuffs.stuff_factory import StuffFactory
+from pipelex.pipeline.input_normalizer import normalize_data_urls_to_storage
+from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider
+from tests.cases import ImageTestCases
+
+
+class ArticleWithImage(StructuredContent):
+ """Test StructuredContent with an embedded ImageContent."""
+
+ title: str
+ image: ImageContent
+ description: str | None = None
+
+
+class NestedArticle(StructuredContent):
+ """Test StructuredContent with nested StructuredContent containing ImageContent."""
+
+ main_article: ArticleWithImage
+ related_image: ImageContent | None = None
+
+
[email protected](loop_scope="class")
+class TestDataUrlImageFlow:
+ """Integration tests for data URL handling in image flows."""
+
+ @pytest.mark.parametrize(("_topic", "data_url"), ImageTestCases.DATA_URL_TEST_CASES)
+ async def test_image_content_with_data_url_to_prompt_image(self, _topic: str, data_url: str) -> None:
+ """Test that ImageContent with data URL converts correctly to PromptImage -> PreparedImage."""
+ # Create ImageContent with data URL
+ image_content = ImageContent(url=data_url)
+
+ # Convert to PromptImage (as done in llm_prompt_blueprint.py)
+ prompt_image = PromptImageFactory.make_prompt_image(uri=image_content.url)
+
+ # Prepare for LLM API consumption
+ prepared = await prepare_prompt_image(
+ prompt_image=prompt_image,
+ is_http_url_enabled=False,
+ )
+
+ # Verify result
+ assert isinstance(prepared, PreparedImageBase64), f"Expected PreparedImageBase64 for {_topic}"
+ # Verify base64 data can be decoded (valid format)
+ decoded_bytes = base64.b64decode(prepared.base64_data)
+ assert len(decoded_bytes) > 0
+
+ @pytest.mark.parametrize(("_topic", "data_url"), ImageTestCases.DATA_URL_TEST_CASES)
+ async def test_data_url_normalization_to_storage(
+ self,
+ mocker: MockerFixture,
+ _topic: str,
+ data_url: str,
+ ) -> None:
+ """Test that the normalizer converts data URLs to storage URIs."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+
+ # Create ImageContent with data URL
+ image_content = ImageContent(url=data_url)
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE),
+ content=image_content,
+ name="test_image",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+
+ # Verify the URL was converted to pipelex-storage://
+ normalized_stuff = normalized_memory.get_stuff("test_image")
+ assert isinstance(normalized_stuff.content, ImageContent)
+ assert normalized_stuff.content.url.startswith("pipelex-storage://")
+
+ # Verify the data is stored correctly and matches original
+ stored_bytes = provider.load(uri=normalized_stuff.content.url)
+ # Extract base64 data from the data URL for comparison
+ base64_data = data_url.split(",", 1)[1]
+ expected_bytes = base64.b64decode(base64_data)
+ assert stored_bytes == expected_bytes
+
+ @pytest.mark.parametrize(("_topic", "data_url"), ImageTestCases.DATA_URL_TEST_CASES)
+ async def test_data_url_round_trip(
+ self,
+ mocker: MockerFixture,
+ _topic: str,
+ data_url: str,
+ ) -> None:
+ """Test full flow: data URL -> storage -> prepared image (verifies bytes match)."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+ mocker.patch("pipelex.cogt.image.prompt_image_utils.get_storage_provider", return_value=provider)
+
+ # Create ImageContent with data URL
+ image_content = ImageContent(url=data_url)
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE),
+ content=image_content,
+ name="test_image",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize to storage
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+ normalized_stuff = normalized_memory.get_stuff("test_image")
+ assert isinstance(normalized_stuff.content, ImageContent)
+ storage_uri = normalized_stuff.content.url
+
+ # Convert back through PromptImage -> PreparedImage
+ prompt_image = PromptImageFactory.make_prompt_image(uri=storage_uri)
+ prepared = await prepare_prompt_image(
+ prompt_image=prompt_image,
+ is_http_url_enabled=False,
+ )
+
+ # Verify the round-trip preserves the data
+ assert isinstance(prepared, PreparedImageBase64)
+ # Extract original base64 for comparison
+ original_base64 = data_url.split(",", 1)[1]
+ assert prepared.base64_data == original_base64
+
+ async def test_normalization_with_list_of_images(
+ self,
+ mocker: MockerFixture,
+ ) -> None:
+ """Test that normalizer handles ListContent of ImageContent."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+
+ # Create ListContent with multiple ImageContent items using shared test data
+ image_contents = [
+ ImageContent(url=ImageTestCases.MINIMAL_PNG_DATA_URL),
+ ImageContent(url=ImageTestCases.MINIMAL_JPEG_DATA_URL),
+ ]
+ list_content = ListContent[ImageContent](items=image_contents)
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE),
+ content=list_content,
+ name="image_list",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+
+ # Verify all images were converted
+ normalized_stuff = normalized_memory.get_stuff("image_list")
+ normalized_content = normalized_stuff.content
+ assert isinstance(normalized_content, ListContent)
+ typed_list_content = cast("ListContent[ImageContent]", normalized_content)
+ for image_item in typed_list_content.items:
+ assert isinstance(image_item, ImageContent)
+ assert image_item.url.startswith("pipelex-storage://")
+
+ async def test_normalization_skips_non_data_urls(
+ self,
+ mocker: MockerFixture,
+ ) -> None:
+ """Test that normalizer skips URLs that are not data URLs."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+
+ # Create ImageContent with HTTP URL
+ http_url = "https://example.com/image.png"
+ image_content = ImageContent(url=http_url)
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE),
+ content=image_content,
+ name="test_image",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+
+ # Verify URL was NOT changed
+ normalized_stuff = normalized_memory.get_stuff("test_image")
+ assert isinstance(normalized_stuff.content, ImageContent)
+ assert normalized_stuff.content.url == http_url
+
+ async def test_normalization_with_structured_content(
+ self,
+ mocker: MockerFixture,
+ ) -> None:
+ """Test that normalizer handles StructuredContent with embedded ImageContent."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+
+ # Create StructuredContent with embedded ImageContent containing data URL
+ article = ArticleWithImage(
+ title="Test Article",
+ image=ImageContent(url=ImageTestCases.MINIMAL_PNG_DATA_URL),
+ description="A test article with an image",
+ )
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.DYNAMIC),
+ content=article,
+ name="test_article",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+
+ # Verify the embedded image URL was converted to pipelex-storage://
+ normalized_stuff = normalized_memory.get_stuff("test_article")
+ assert isinstance(normalized_stuff.content, ArticleWithImage)
+ normalized_article = normalized_stuff.content
+ assert normalized_article.title == "Test Article"
+ assert normalized_article.description == "A test article with an image"
+ assert normalized_article.image.url.startswith("pipelex-storage://")
+
+ # Verify the stored data is correct
+ stored_bytes = provider.load(uri=normalized_article.image.url)
+ expected_bytes = base64.b64decode(ImageTestCases.MINIMAL_PNG_BASE64)
+ assert stored_bytes == expected_bytes
+
+ async def test_normalization_with_nested_structured_content(
+ self,
+ mocker: MockerFixture,
+ ) -> None:
+ """Test that normalizer recursively handles nested StructuredContent."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+
+ # Create nested StructuredContent with ImageContent at multiple levels
+ nested = NestedArticle(
+ main_article=ArticleWithImage(
+ title="Main Article",
+ image=ImageContent(url=ImageTestCases.MINIMAL_PNG_DATA_URL),
+ ),
+ related_image=ImageContent(url=ImageTestCases.MINIMAL_JPEG_DATA_URL),
+ )
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.DYNAMIC),
+ content=nested,
+ name="nested_article",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+
+ # Verify all image URLs were converted
+ normalized_stuff = normalized_memory.get_stuff("nested_article")
+ assert isinstance(normalized_stuff.content, NestedArticle)
+ normalized_nested = normalized_stuff.content
+
+ # Check nested ImageContent
+ assert normalized_nested.main_article.image.url.startswith("pipelex-storage://")
+
+ # Check top-level ImageContent
+ assert normalized_nested.related_image is not None
+ assert normalized_nested.related_image.url.startswith("pipelex-storage://")
+
+ # Verify both stored data are correct
+ nested_stored = provider.load(uri=normalized_nested.main_article.image.url)
+ assert nested_stored == base64.b64decode(ImageTestCases.MINIMAL_PNG_BASE64)
+
+ related_stored = provider.load(uri=normalized_nested.related_image.url)
+ assert related_stored == base64.b64decode(ImageTestCases.MINIMAL_JPEG_BASE64)
+
+ async def test_normalization_with_list_in_structured_content(
+ self,
+ mocker: MockerFixture,
+ ) -> None:
+ """Test that normalizer handles lists of ImageContent within StructuredContent."""
+ # Setup in-memory storage
+ provider = InMemoryStorageProvider()
+ mocker.patch("pipelex.pipeline.input_normalizer.get_storage_provider", return_value=provider)
+
+ # Create a StructuredContent subclass with a list of images
+ class GalleryContent(StructuredContent):
+ title: str
+ images: list[ImageContent]
+
+ gallery = GalleryContent(
+ title="Test Gallery",
+ images=[
+ ImageContent(url=ImageTestCases.MINIMAL_PNG_DATA_URL),
+ ImageContent(url=ImageTestCases.MINIMAL_JPEG_DATA_URL),
+ ],
+ )
+ stuff = StuffFactory.make_stuff(
+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.DYNAMIC),
+ content=gallery,
+ name="gallery",
+ )
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(stuff=stuff)
+
+ # Normalize
+ normalized_memory = normalize_data_urls_to_storage(working_memory)
+
+ # Verify all images in the list were converted
+ normalized_stuff = normalized_memory.get_stuff("gallery")
+ assert isinstance(normalized_stuff.content, GalleryContent)
+ normalized_gallery = normalized_stuff.content
+ assert len(normalized_gallery.images) == 2
+ for image_item in normalized_gallery.images:
+ assert image_item.url.startswith("pipelex-storage://")
diff --git a/tests/integration/pipelex/cogt/image/test_pipelex_storage_image_flow.py b/tests/integration/pipelex/cogt/image/test_pipelex_storage_image_flow.py
new file mode 100644
index 000000000..1791306b9
--- /dev/null
+++ b/tests/integration/pipelex/cogt/image/test_pipelex_storage_image_flow.py
@@ -0,0 +1,143 @@
+"""Integration tests for the full pipelex-storage:// image flow.
+
+These tests verify the complete flow:
+1. Store an image via StorageProvider
+2. Create ImageContent with pipelex-storage:// URI
+3. Convert to PromptImage via factory
+4. Prepare via prep_prompt_images() for each provider type
+5. Verify all return valid PreparedImage
+"""
+
+import base64
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.image.prepared_image import PreparedImageBase64
+from pipelex.cogt.image.prompt_image import PromptImageUri
+from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
+from pipelex.cogt.image.prompt_image_utils import prep_prompt_images, prepare_prompt_image
+from pipelex.core.stuffs.image_content import ImageContent
+from pipelex.tools.misc.file_utils import load_binary
+from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider
+from pipelex.tools.uri.resolved_uri import ResolvedPipelexStorage
+from pipelex.tools.uri.uri_resolver import resolve_uri
+
+# Test image path from test data
+TEST_IMAGE_PATH = "tests/data/images/eiffel_tower.png"
+
+
+def storage_provider_with_test_image() -> tuple[InMemoryStorageProvider, str]:
+ """Create an in-memory storage with a real test image and return (provider, uri)."""
+ provider = InMemoryStorageProvider()
+ # Load a real test image
+ image_bytes = load_binary(path=TEST_IMAGE_PATH)
+ key = "pipeline_run_123/generated_image.png"
+ uri = provider.store(data=image_bytes, key=key)
+ return provider, uri
+
+
[email protected](loop_scope="class")
+class TestPipelexStorageImageFlow:
+ """Integration tests for the full pipelex-storage:// image flow."""
+
+ async def test_full_flow_storage_to_prepared_image(
+ self,
+ mocker: MockerFixture,
+ storage_provider_with_test_image: tuple[InMemoryStorageProvider, str],
+ ) -> None:
+ """Test the complete flow from storage to PreparedImage.
+
+ This simulates the real-world scenario where:
+ 1. An image generator stores an image → returns pipelex-storage:// URI
+ 2. That URI becomes an ImageContent.url
+ 3. User passes ImageContent to an LLM pipe as input
+ 4. The LLM pipe converts ImageContent → PromptImage → PreparedImage
+ """
+ provider, storage_uri = storage_provider_with_test_image
+
+ # Mock get_storage_provider to return our in-memory provider
+ mocker.patch("pipelex.cogt.image.prompt_image_utils.get_storage_provider", return_value=provider)
+
+ # Step 1: Create ImageContent as would be returned by GeneratedContentFactory
+ image_content = ImageContent(
+ url=storage_uri,
+ display_link=None,
+ mime_type="image/png",
+ )
+
+ # Step 2: Convert to PromptImage via factory (as done in llm_prompt_blueprint.py)
+ prompt_image = PromptImageFactory.make_prompt_image(uri=image_content.url)
+ assert isinstance(prompt_image, PromptImageUri)
+
+ # Step 3: Prepare for LLM API consumption
+ prepared = await prepare_prompt_image(
+ prompt_image=prompt_image,
+ is_http_url_enabled=False,
+ )
+
+ # Verify result
+ assert isinstance(prepared, PreparedImageBase64)
+ assert prepared.file_type.mime == "image/png"
+ # Verify base64 data is valid (can be decoded)
+ decoded = base64.b64decode(prepared.base64_data)
+ assert len(decoded) > 0
+
+ async def test_prep_prompt_images_batch_with_storage_uri(
+ self,
+ mocker: MockerFixture,
+ storage_provider_with_test_image: tuple[InMemoryStorageProvider, str],
+ ) -> None:
+ """Test batch preparation of multiple images including pipelex-storage:// URIs."""
+ provider, storage_uri = storage_provider_with_test_image
+
+ # Mock get_storage_provider
+ mocker.patch("pipelex.cogt.image.prompt_image_utils.get_storage_provider", return_value=provider)
+
+ # Create multiple PromptImages: one from storage, one from local path
+ prompt_images = [
+ PromptImageFactory.make_prompt_image(uri=storage_uri),
+ PromptImageFactory.make_prompt_image(uri=TEST_IMAGE_PATH),
+ ]
+
+ # Prepare all images in parallel
+ prepared_images = await prep_prompt_images(
+ prompt_images=prompt_images,
+ is_http_url_enabled=False,
+ )
+
+ # Verify all images were prepared
+ assert len(prepared_images) == 2
+ for prepared in prepared_images:
+ assert isinstance(prepared, PreparedImageBase64)
+ assert prepared.file_type.mime == "image/png"
+
+ async def test_storage_uri_resolves_correctly(
+ self,
+ storage_provider_with_test_image: tuple[InMemoryStorageProvider, str],
+ ) -> None:
+ """Test that pipelex-storage:// URIs are correctly resolved."""
+ _, storage_uri = storage_provider_with_test_image
+
+ resolved = resolve_uri(storage_uri)
+
+ assert isinstance(resolved, ResolvedPipelexStorage)
+ assert resolved.storage_uri == storage_uri
+
+ async def test_prompt_image_uri_caches_resolved(
+ self,
+ storage_provider_with_test_image: tuple[InMemoryStorageProvider, str],
+ ) -> None:
+ """Test that PromptImageUri.resolved property caches the ResolvedUri."""
+ _, storage_uri = storage_provider_with_test_image
+
+ prompt_image = PromptImageUri(uri=storage_uri)
+
+ # Access .resolved twice and verify it's the same instance (cached)
+ resolved1 = prompt_image.resolved
+ resolved2 = prompt_image.resolved
+
+ assert isinstance(resolved1, ResolvedPipelexStorage)
+ assert resolved1 is resolved2 # Same instance due to caching
diff --git a/tests/integration/pipelex/cogt/test_data.py b/tests/integration/pipelex/cogt/test_data.py
index 0267a9520..1b2ee7156 100644
--- a/tests/integration/pipelex/cogt/test_data.py
+++ b/tests/integration/pipelex/cogt/test_data.py
@@ -2,11 +2,12 @@ from typing import Any, ClassVar
from pydantic import BaseModel, Field, field_validator
-from pipelex.cogt.image.prompt_image import PromptImagePath
+from pipelex.cogt.image.prompt_image import PromptImageUri
from pipelex.cogt.llm.llm_prompt import LLMPrompt
from pipelex.cogt.llm.llm_prompt_template import LLMPromptTemplate
from pipelex.cogt.llm.llm_prompt_template_inputs import LLMPromptTemplateInputs
from pipelex.types import StrEnum
+from tests.cases import ImageTestCases
from tests.integration.pipelex.test_data import PipeTestCases
@@ -66,12 +67,12 @@ class LLMVisionTestCases:
PATH_IMG_GANTT_1 = f"{TEST_IMAGE_DIRECTORY}/diagram.png"
IMAGE_PATHS: ClassVar[list[tuple[str, str]]] = [ # topic, image_path
- ("Gantt Chart", PATH_IMG_GANTT_1),
+ # ("Gantt Chart", PATH_IMG_GANTT_1),
("AI Lympics PNG", PATH_IMG_PNG_1),
- ("Animal Lympics PNG", PATH_IMG_PNG_2),
+ # ("Animal Lympics PNG", PATH_IMG_PNG_2),
("AI Lympics JPEG", PATH_IMG_JPEG_1),
- ("Eiffel Tower", PATH_IMG_JPEG_3),
- ("Eiffel Tower", PATH_IMG_PNG_3),
+ # ("Eiffel Tower", PATH_IMG_JPEG_3),
+ # ("Eiffel Tower", PATH_IMG_PNG_3),
]
IMAGE_PATH_PAIRS: ClassVar[list[tuple[str, tuple[str, str]]]] = [ # topic, image_pair
("AI Lympics PNG", (PATH_IMG_PNG_1, PATH_IMG_PNG_2)),
@@ -88,6 +89,11 @@ class LLMVisionTestCases:
),
]
+ # Data URLs for vision tests (topic, data_url)
+ IMAGE_DATA_URLS: ClassVar[list[tuple[str, str]]] = [
+ ("Pipelex Logo Tiny", ImageTestCases.LOGO_TINY_PNG_DATA_URL),
+ ]
+
class LLMTestConstants:
USER_TEXT_SHORT = "In one short sentence, who is Bill Gates?"
@@ -183,12 +189,12 @@ class SerDeTestLLMCases:
"source_user_template_name": "markdown_reordering_vision_claude3_5_sonnet",
}
- # Prompt containing an image path --------------------------------------
- PROMPT_WITH_IMAGE_PATH: ClassVar[LLMPrompt] = LLMPrompt(
+ # Prompt containing an image URI --------------------------------------
+ PROMPT_WITH_IMAGE_URI: ClassVar[LLMPrompt] = LLMPrompt(
system_text="Some system text",
user_text="Some user text",
user_images=[
- PromptImagePath(file_path="some_file_path"),
+ PromptImageUri(uri="some_file_path"),
],
)
@@ -198,7 +204,7 @@ class SerDeTestLLMCases:
MY_PROMPT_TEMPLATE_MODEL_2,
]
PYDANTIC_EXAMPLES_USING_SUBCLASS: ClassVar[list[BaseModel]] = [
- PROMPT_WITH_IMAGE_PATH,
+ PROMPT_WITH_IMAGE_URI,
]
PYDANTIC_EXAMPLES_DICT: ClassVar[list[dict[str, Any]]] = [
DICT_1,
diff --git a/tests/integration/pipelex/cogt/test_llm_inference.py b/tests/integration/pipelex/cogt/test_llm_inference.py
index 1d235e668..1f9ae5e06 100644
--- a/tests/integration/pipelex/cogt/test_llm_inference.py
+++ b/tests/integration/pipelex/cogt/test_llm_inference.py
@@ -2,7 +2,7 @@ import pytest
from pipelex import log, pretty_print
from pipelex.cogt.exceptions import PromptImageFormatError
-from pipelex.cogt.image.prompt_image import PromptImagePath
+from pipelex.cogt.image.prompt_image import PromptImageUri
from pipelex.cogt.llm.llm_job_components import LLMJobParams
from pipelex.cogt.llm.llm_job_factory import LLMJobFactory
from pipelex.cogt.llm.llm_prompt import LLMPrompt
@@ -57,7 +57,7 @@ class TestLLMInference:
@pytest.mark.parametrize("image_path", [LLMVisionTestCases.PATH_IMG_PNG_1])
async def test_gen_text_from_image(self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, image_path: str):
log.info(f"test_gen_text_from_image: Testing llm_handle '{llm_handle}'")
- prompt_image = PromptImagePath(file_path=image_path)
+ prompt_image = PromptImageUri(uri=image_path)
llm_worker = get_inference_manager().get_llm_worker(llm_handle=llm_handle)
if not llm_worker.is_vision_supported:
msg = f"Vision is not supported for this LLM worker: '{llm_worker.desc}'"
@@ -83,7 +83,7 @@ class TestLLMInference:
@pytest.mark.parametrize("image_path", [LLMVisionTestCases.PATH_IMG_PNG_1])
async def test_gen_object_from_image(self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, image_path: str):
log.info(f"test_gen_object_from_image: Testing llm_handle '{llm_handle}'")
- prompt_image = PromptImagePath(file_path=image_path)
+ prompt_image = PromptImageUri(uri=image_path)
llm_worker = get_inference_manager().get_llm_worker(llm_handle=llm_handle)
if not llm_worker.is_vision_supported:
diff --git a/tests/integration/pipelex/cogt/test_llm_vision.py b/tests/integration/pipelex/cogt/test_llm_vision.py
index b013909f6..dd9fc28f8 100644
--- a/tests/integration/pipelex/cogt/test_llm_vision.py
+++ b/tests/integration/pipelex/cogt/test_llm_vision.py
@@ -2,7 +2,7 @@ import pytest
from pipelex import log, pretty_print
from pipelex.cogt.exceptions import LLMCapabilityError, PromptImageFormatError
-from pipelex.cogt.image.prompt_image import PromptImageBase64, PromptImagePath
+from pipelex.cogt.image.prompt_image import PromptImageBase64, PromptImageUri
from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
from pipelex.cogt.llm.llm_job_components import LLMJobParams
from pipelex.cogt.llm.llm_job_factory import LLMJobFactory
@@ -22,7 +22,7 @@ class TestLLMVision:
async def test_gen_text_from_vision_by_url(
self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, topic: str, image_uri: str
):
- prompt_image = PromptImageFactory.make_prompt_image(url=image_uri)
+ prompt_image = PromptImageFactory.make_prompt_image(uri=image_uri)
llm_worker = get_llm_worker(llm_handle=llm_handle)
log.info(f"Using llm_worker: {llm_worker.desc}")
llm_job = LLMJobFactory.make_llm_job(
@@ -47,8 +47,8 @@ class TestLLMVision:
async def test_gen_text_from_vision_by_bytes(
self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, topic: str, image_path: str
):
- image_bytes = load_binary_as_base64(path=image_path)
- prompt_image = PromptImageBase64(base_64=image_bytes)
+ base64_data = await load_binary_as_base64(path=image_path)
+ prompt_image = PromptImageBase64(base64_data=base64_data)
llm_worker = get_llm_worker(llm_handle=llm_handle)
llm_job = LLMJobFactory.make_llm_job(
llm_prompt=LLMPrompt(
@@ -71,7 +71,7 @@ class TestLLMVision:
async def test_gen_text_from_vision_by_path(
self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, topic: str, image_path: str
):
- prompt_image = PromptImagePath(file_path=image_path)
+ prompt_image = PromptImageUri(uri=image_path)
llm_worker = get_llm_worker(llm_handle=llm_handle)
llm_job = LLMJobFactory.make_llm_job(
llm_prompt=LLMPrompt(
@@ -94,8 +94,8 @@ class TestLLMVision:
async def test_gen_text_from_vision_2_images(
self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, topic: str, image_pair: tuple[str, str]
):
- prompt_image1 = PromptImagePath(file_path=image_pair[0])
- prompt_image2 = PromptImagePath(file_path=image_pair[1])
+ prompt_image1 = PromptImageUri(uri=image_pair[0])
+ prompt_image2 = PromptImageUri(uri=image_pair[1])
llm_worker = get_llm_worker(llm_handle=llm_handle)
llm_job = LLMJobFactory.make_llm_job(
llm_prompt=LLMPrompt(
@@ -113,3 +113,33 @@ class TestLLMVision:
pytest.skip(f"Vision capability not supported for this LLM: {llm_handle} because {exc}")
except PromptImageFormatError as exc:
pytest.skip(f"Prompt Image format not supported for this LLM: {llm_handle} because {exc}")
+
+ @pytest.mark.parametrize(("topic", "data_url"), LLMVisionTestCases.IMAGE_DATA_URLS)
+ async def test_gen_text_from_vision_by_data_url(
+ self, job_metadata: JobMetadata, llm_job_params: LLMJobParams, llm_handle: str, topic: str, data_url: str
+ ):
+ """Test LLM vision using a data URL (embedded base64 image).
+
+ This verifies that data URLs like 'data:image/png;base64,...' are correctly
+ handled by the LLM vision pipeline.
+ """
+ prompt_image = PromptImageFactory.make_prompt_image(uri=data_url)
+ llm_worker = get_llm_worker(llm_handle=llm_handle)
+ log.info(f"Using llm_worker: {llm_worker.desc}")
+ llm_job = LLMJobFactory.make_llm_job(
+ llm_prompt=LLMPrompt(
+ user_text=LLMVisionTestCases.VISION_USER_TEXT,
+ user_images=[prompt_image],
+ ),
+ job_metadata=job_metadata,
+ llm_job_params=llm_job_params,
+ )
+
+ try:
+ generated_text = await llm_worker.gen_text(llm_job=llm_job)
+ assert generated_text
+ pretty_print(generated_text, title=f"Vision of {topic} (data URL)")
+ except LLMCapabilityError as exc:
+ pytest.skip(f"Vision capability not supported for this LLM: {llm_handle} because {exc}")
+ except PromptImageFormatError as exc:
+ pytest.skip(f"Prompt Image format not supported for this LLM: {llm_handle} because {exc}")
diff --git a/tests/integration/pipelex/pipes/test_image_inputs.py b/tests/integration/pipelex/pipes/test_image_inputs.py
index 8908b7988..28b0b1e2d 100644
--- a/tests/integration/pipelex/pipes/test_image_inputs.py
+++ b/tests/integration/pipelex/pipes/test_image_inputs.py
@@ -237,3 +237,43 @@ class TestImageInputs:
assert pipe_output.main_stuff.concept.code == "Analysis"
# Verify the content is some kind of text output (analysis result)
assert pipe_output.main_stuff.content is not None
+
+ @pytest.mark.parametrize(("_topic", "data_url"), ImageTestCases.DATA_URL_TEST_CASES)
+ async def test_image_input_with_data_url(
+ self,
+ job_metadata: JobMetadata,
+ pipe_run_mode: PipeRunMode,
+ load_test_library: Callable[[list[Path]], None],
+ _topic: str,
+ data_url: str,
+ ) -> None:
+ """Test that ImageContent with data URL works as pipeline input.
+
+ This test verifies that a data URL (data:image/png;base64,...) can be used
+ as the URL in ImageContent and is correctly processed through the pipeline.
+ """
+ load_test_library([Path("tests/integration/pipelex/pipes/pipelines")])
+
+ # Create ImageContent with data URL
+ working_memory = WorkingMemoryFactory.make_from_single_stuff(
+ stuff=StuffFactory.make_stuff(
+ concept=get_native_concept(NativeConceptCode.IMAGE),
+ content=ImageContent(url=data_url),
+ name="image",
+ ),
+ )
+
+ # Run the extract_article_from_image pipe (which expects an Image input)
+ pipe_output = await get_pipe_router().run(
+ pipe_job=PipeJobFactory.make_pipe_job(
+ pipe=get_required_pipe(pipe_code="extract_article_from_image"),
+ pipe_run_params=PipeRunParamsFactory.make_run_params(pipe_run_mode=pipe_run_mode),
+ working_memory=working_memory,
+ job_metadata=job_metadata,
+ ),
+ )
+
+ # Verify the pipeline executed successfully
+ assert pipe_output is not None
+ assert pipe_output.working_memory is not None
+ assert pipe_output.main_stuff is not None
diff --git a/tests/integration/pipelex/test_data.py b/tests/integration/pipelex/test_data.py
index c392b9527..80e7de35e 100644
--- a/tests/integration/pipelex/test_data.py
+++ b/tests/integration/pipelex/test_data.py
@@ -164,9 +164,9 @@ class PipeExtractTestCases:
class ImageGenTestCases:
IMAGE_GEN_PROMPT_CONTENTS: ClassVar[list[tuple[str, str, str | None]]] = [ # topic, positive_text, negative_text
- ("dog wearing sunglasses", "a dog wearing sunglasses", "blue sky"),
+ # ("dog wearing sunglasses", "a dog wearing sunglasses", "blue sky"),
# ("otter playing poker", "an otter playing poker", None),
- # ("coding woman with dragon tatoo", "a woman with a dragon tatoo, wearing a tank top, coding in python", None),
+ ("coding woman with dragon tatoo", "a woman with a dragon tatoo, wearing a tank top, coding in python", "nude"),
]
diff --git a/tests/integration/pipelex/tools/httpx/test_httpx_fetching.py b/tests/integration/pipelex/tools/httpx/test_httpx_fetching.py
index 18b9b41f9..b1a15074e 100644
--- a/tests/integration/pipelex/tools/httpx/test_httpx_fetching.py
+++ b/tests/integration/pipelex/tools/httpx/test_httpx_fetching.py
@@ -1,6 +1,6 @@
import pytest
-from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx, fetch_file_from_url_httpx_async
+from pipelex.tools.misc.file_fetch_utils import fetch_file_from_url_httpx
from tests.cases import TestURLs
@@ -13,20 +13,7 @@ class TestHttpxFetching:
url: str,
) -> None:
assert (
- await fetch_file_from_url_httpx_async(
- url=url,
- request_timeout=60,
- )
- is not None
- )
-
- @pytest.mark.parametrize("url", TestURLs.PUBLIC_URLS)
- def test_fetch_file_from_url_httpx(
- self,
- url: str,
- ) -> None:
- assert (
- fetch_file_from_url_httpx(
+ await fetch_file_from_url_httpx(
url=url,
request_timeout=60,
)
diff --git a/tests/integration/pipelex/tools/uri/__init__.py b/tests/integration/pipelex/tools/uri/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/integration/pipelex/tools/uri/test_uri_resolver_integration.py b/tests/integration/pipelex/tools/uri/test_uri_resolver_integration.py
new file mode 100644
index 000000000..af3887078
--- /dev/null
+++ b/tests/integration/pipelex/tools/uri/test_uri_resolver_integration.py
@@ -0,0 +1,102 @@
+import base64
+
+import pytest
+
+from pipelex.tools.misc.base64_utils import make_base64_url_from_http_url
+from pipelex.tools.uri.resolved_uri import ResolvedHttpUrl, ResolvedLocalPath, UriKind
+from pipelex.tools.uri.uri_resolver import make_base64_url_from_any_uri, resolve_uri
+from tests.cases import ImageTestCases, TestURLs
+
+
[email protected](loop_scope="class")
+class TestUriResolverIntegration:
+ """Integration tests for URI resolution with real network calls."""
+
+ @pytest.mark.parametrize("url", TestURLs.PUBLIC_URLS)
+ async def test_make_base64_url_from_http_url_async_real_network(self, url: str) -> None:
+ """Test fetching and converting real HTTP URLs to base64 data URLs."""
+ result = await make_base64_url_from_http_url(url=url)
+
+ # Should start with data URL prefix
+ assert result.startswith("data:")
+ assert ";base64," in result
+
+ # Verify the base64 part is valid
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ # Should have some content
+ assert len(decoded) > 0
+
+ @pytest.mark.parametrize("url", TestURLs.PUBLIC_URLS)
+ async def test_make_base64_url_from_uri_async_with_http_url(self, url: str) -> None:
+ """Test full URI resolution workflow with HTTP URLs."""
+ # First verify the URL resolves correctly
+ resolved = resolve_uri(url)
+ assert resolved.kind == UriKind.HTTP_URL
+ assert isinstance(resolved, ResolvedHttpUrl)
+ assert resolved.url == url
+
+ # Then test the full conversion workflow
+ result = await make_base64_url_from_any_uri(url)
+
+ assert result.startswith("data:")
+ assert ";base64," in result
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert len(decoded) > 0
+
+ async def test_resolve_and_convert_local_file(self) -> None:
+ """Test URI resolution workflow with local files."""
+ file_path = ImageTestCases.IMAGE_FILE_PATH_PNG_1
+
+ # Verify resolution
+ resolved = resolve_uri(file_path)
+ assert resolved.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved, ResolvedLocalPath)
+ assert resolved.path == file_path
+
+ # Verify conversion
+ result = await make_base64_url_from_any_uri(file_path)
+
+ assert result.startswith("data:image/png;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded[:8] == b"\x89PNG\r\n\x1a\n"
+
+ async def test_data_url_passthrough_workflow(self) -> None:
+ """Test that data URLs pass through the entire workflow unchanged."""
+ original_data_url = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
+
+ # Verify resolution
+ resolved = resolve_uri(original_data_url)
+ assert resolved.kind == UriKind.BASE64_DATA_URL
+ assert resolved.original == original_data_url
+
+ # Verify passthrough
+ result = await make_base64_url_from_any_uri(original_data_url)
+ assert result == original_data_url
+
+ async def test_gcp_public_url_returns_png(self) -> None:
+ """Test that GCP public URL returns a valid PNG image."""
+ url = TestURLs.GCP_PUBLIC
+
+ result = await make_base64_url_from_any_uri(url)
+
+ assert result.startswith("data:image/png;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ # PNG magic bytes
+ assert decoded[:8] == b"\x89PNG\r\n\x1a\n"
+
+ async def test_aws_cloudfront_url_returns_jpeg(self) -> None:
+ """Test that AWS CloudFront URL returns a valid JPEG image."""
+ url = TestURLs.AWS_CLOUDFRONT
+
+ result = await make_base64_url_from_any_uri(url)
+
+ assert result.startswith("data:image/jpeg;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ # JPEG magic bytes
+ assert decoded[:2] == b"\xff\xd8"
diff --git a/tests/unit/pipelex/cogt/image/test_prompt_image_factory.py b/tests/unit/pipelex/cogt/image/test_prompt_image_factory.py
new file mode 100644
index 000000000..8ff7f942b
--- /dev/null
+++ b/tests/unit/pipelex/cogt/image/test_prompt_image_factory.py
@@ -0,0 +1,84 @@
+import pytest
+from pydantic import ValidationError
+
+from pipelex.cogt.image.prompt_image import PromptImageBase64, PromptImageUri
+from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
+from pipelex.tools.misc.base64_utils import make_base64_url_from_path
+from pipelex.tools.misc.http_utils import URL_MAX_LENGTH
+from tests.cases.images import ImageTestCases
+
+
+class TestPromptImageFactoryDataUrl:
+ """Test that PromptImageFactory correctly handles data URLs.
+
+ Data URLs (data:image/...;base64,...) should be converted to PromptImageBase64,
+ bypassing the URL_MAX_LENGTH (2048) validation that would otherwise fail
+ for realistic-sized images.
+ """
+
+ @pytest.mark.asyncio
+ @pytest.mark.parametrize(
+ ("_topic", "image_path"),
+ [
+ ("png_solar_system", ImageTestCases.IMAGE_FILE_PATH_PNG_2),
+ ("jpg_animal_lympics", ImageTestCases.IMAGE_FILE_PATH_JPG_1),
+ ("png_ai_lympics", ImageTestCases.IMAGE_FILE_PATH_PNG_1),
+ ("jpg_eiffel_tower", ImageTestCases.IMAGE_FILE_PATH_JPG_3),
+ ],
+ )
+ async def test_make_prompt_image_with_realistic_data_url(
+ self,
+ _topic: str,
+ image_path: str,
+ ) -> None:
+ """Test that realistic-sized data URLs work via the factory.
+
+ These tests load actual image files (~100KB-600KB) and create data URLs.
+ The data URLs will be tens to hundreds of thousands of characters,
+ far exceeding URL_MAX_LENGTH (2048).
+ Before the fix, this would fail with 'URI is too long'.
+ """
+ data_url = await make_base64_url_from_path(image_path)
+
+ result = PromptImageFactory.make_prompt_image(uri=data_url)
+
+ assert isinstance(result, PromptImageBase64)
+ assert len(result.base64_data) > 100_000 # Realistic image size
+
+ @pytest.mark.parametrize(
+ ("_topic", "uri"),
+ [
+ ("http_url", "https://example.com/image.png"),
+ ("https_url", "https://secure.example.com/photo.jpg"),
+ ("local_absolute_path", "/path/to/image.png"),
+ ("local_relative_path", "images/photo.jpg"),
+ ("storage_uri", "pipelex-storage://images/test.png"),
+ ],
+ )
+ def test_make_prompt_image_with_non_data_url_unchanged(
+ self,
+ _topic: str,
+ uri: str,
+ ) -> None:
+ """Test that non-data URIs still create PromptImageUri.
+
+ HTTP URLs, local paths, and storage URIs should continue to work
+ and should NOT be converted to PromptImageBase64.
+ """
+ result = PromptImageFactory.make_prompt_image(uri=uri)
+
+ assert isinstance(result, PromptImageUri)
+ assert result.uri == uri
+
+ def test_http_url_exceeding_max_length_fails(self) -> None:
+ """Test that HTTP URLs exceeding URL_MAX_LENGTH still fail validation.
+
+ Data URLs are exempt from URL_MAX_LENGTH, but HTTP URLs must still
+ be validated to prevent excessively long URLs.
+ """
+ # Create an HTTP URL that exceeds the limit
+ long_path = "a" * (URL_MAX_LENGTH + 100)
+ long_http_url = f"https://example.com/{long_path}.png"
+
+ with pytest.raises(ValidationError, match="URI is too long"):
+ PromptImageFactory.make_prompt_image(uri=long_http_url)
diff --git a/tests/unit/pipelex/plugins/__init__.py b/tests/unit/pipelex/plugins/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/unit/pipelex/plugins/test_plugin_pipelex_storage_images.py b/tests/unit/pipelex/plugins/test_plugin_pipelex_storage_images.py
new file mode 100644
index 000000000..7c0ee7849
--- /dev/null
+++ b/tests/unit/pipelex/plugins/test_plugin_pipelex_storage_images.py
@@ -0,0 +1,184 @@
+"""Unit tests for pipelex-storage:// URI support in LLM provider plugins.
+
+These tests verify that all plugins can handle images stored via pipelex-storage:// URIs.
+"""
+
+import pytest
+from google.genai import types as genai_types
+from mistralai.models import ImageURLChunk
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.image.prepared_image import PreparedImageBase64
+from pipelex.cogt.image.prompt_image import PromptImageUri
+from pipelex.cogt.image.prompt_image_utils import prepare_prompt_image_as_base64
+from pipelex.plugins.google.google_factory import GoogleFactory
+from pipelex.plugins.mistral.mistral_factory import MistralFactory
+from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider
+
+# Minimal PNG image (1x1 transparent pixel)
+MINIMAL_PNG_BYTES = bytes(
+ [
+ 0x89,
+ 0x50,
+ 0x4E,
+ 0x47,
+ 0x0D,
+ 0x0A,
+ 0x1A,
+ 0x0A, # PNG signature
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x0D,
+ 0x49,
+ 0x48,
+ 0x44,
+ 0x52, # IHDR chunk
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x01,
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x01, # 1x1
+ 0x08,
+ 0x06,
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x1F,
+ 0x15,
+ 0xC4,
+ 0x89, # 8-bit RGBA
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x0A,
+ 0x49,
+ 0x44,
+ 0x41,
+ 0x54, # IDAT chunk
+ 0x78,
+ 0x9C,
+ 0x63,
+ 0x00,
+ 0x01,
+ 0x00,
+ 0x00,
+ 0x05,
+ 0x00,
+ 0x01,
+ 0x0D,
+ 0x0A,
+ 0x2D,
+ 0xB4, # compressed data
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x00,
+ 0x49,
+ 0x45,
+ 0x4E,
+ 0x44, # IEND chunk
+ 0xAE,
+ 0x42,
+ 0x60,
+ 0x82, # CRC
+ ]
+)
+
+
+def storage_with_image() -> tuple[InMemoryStorageProvider, str]:
+ """Create an in-memory storage with a test image and return (provider, uri)."""
+ provider = InMemoryStorageProvider()
+ key = "test_images/sample.png"
+ uri = provider.store(data=MINIMAL_PNG_BYTES, key=key)
+ return provider, uri
+
+
+def prompt_image_with_storage_uri(storage_with_image: tuple[InMemoryStorageProvider, str]) -> PromptImageUri:
+ """Create a PromptImageUri pointing to a pipelex-storage:// URI."""
+ _, uri = storage_with_image
+ return PromptImageUri(uri=uri)
+
+
+class TestGoogleFactoryPipelexStorageSupport:
+ """Tests for GoogleFactory handling of pipelex-storage:// URIs."""
+
+ @pytest.mark.asyncio
+ async def test_prepare_image_part_with_pipelex_storage_uri(
+ self,
+ mocker: MockerFixture,
+ storage_with_image: tuple[InMemoryStorageProvider, str],
+ prompt_image_with_storage_uri: PromptImageUri,
+ ) -> None:
+ """Test that GoogleFactory.prepare_image_part() handles pipelex-storage:// URIs.
+
+ Currently EXPECTED TO FAIL - raises GoogleFactoryError.
+ After refactoring, this should return a valid genai Part.
+ """
+ provider, _ = storage_with_image
+
+ # Mock get_storage_provider to return our in-memory provider
+ mocker.patch("pipelex.cogt.image.prompt_image_utils.get_storage_provider", return_value=provider)
+
+ # This should NOT raise an error after refactoring
+ result = await GoogleFactory.prepare_image_part(prompt_image_with_storage_uri)
+
+ assert isinstance(result, genai_types.Part)
+
+
+class TestMistralFactoryPipelexStorageSupport:
+ """Tests for MistralFactory handling of pipelex-storage:// URIs."""
+
+ @pytest.mark.asyncio
+ async def test_make_mistral_image_url_with_pipelex_storage_uri(
+ self,
+ mocker: MockerFixture,
+ storage_with_image: tuple[InMemoryStorageProvider, str],
+ prompt_image_with_storage_uri: PromptImageUri,
+ ) -> None:
+ """Test that MistralFactory.make_mistral_image_url() handles pipelex-storage:// URIs.
+
+ Currently EXPECTED TO FAIL - raises PromptImageFormatError.
+ After refactoring, this should return a valid ImageURLChunk.
+ """
+ provider, _ = storage_with_image
+
+ # Mock get_storage_provider to return our in-memory provider
+ mocker.patch("pipelex.cogt.image.prompt_image_utils.get_storage_provider", return_value=provider)
+
+ factory = MistralFactory()
+ # This should NOT raise an error after refactoring
+ result = await factory.make_mistral_image_url(prompt_image_with_storage_uri)
+
+ assert isinstance(result, ImageURLChunk)
+ # Should be a data URL since we're converting from storage
+ image_url = result.image_url
+ assert isinstance(image_url, str)
+ assert image_url.startswith("data:image/")
+
+
+class TestPromptImageUtilsPipelexStorageSupport:
+ """Tests verifying that prompt_image_utils supports pipelex-storage:// correctly."""
+
+ @pytest.mark.asyncio
+ async def test_prepare_prompt_image_as_base64_with_pipelex_storage_uri(
+ self,
+ mocker: MockerFixture,
+ storage_with_image: tuple[InMemoryStorageProvider, str],
+ prompt_image_with_storage_uri: PromptImageUri,
+ ) -> None:
+ """Test that prepare_prompt_image_as_base64() correctly handles pipelex-storage:// URIs."""
+ provider, _ = storage_with_image
+
+ # Mock get_storage_provider to return our in-memory provider
+ mocker.patch("pipelex.cogt.image.prompt_image_utils.get_storage_provider", return_value=provider)
+
+ result = await prepare_prompt_image_as_base64(prompt_image_with_storage_uri)
+
+ assert isinstance(result, PreparedImageBase64)
+ assert result.file_type.mime == "image/png"
diff --git a/tests/unit/pipelex/tools/misc/base64/test_base64_utils.py b/tests/unit/pipelex/tools/misc/base64/test_base64_utils.py
new file mode 100644
index 000000000..c1b88629b
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/base64/test_base64_utils.py
@@ -0,0 +1,214 @@
+import base64
+
+import pytest
+
+from pipelex.tools.misc.base64_utils import (
+ extract_base64_str_from_base64_url_if_possible,
+ is_prefixed_base64_url,
+ load_binary_as_base64,
+ make_base64_url,
+ make_base64_url_from_path,
+ strip_base64_str_if_needed,
+)
+from pipelex.tools.misc.filetype_utils import FileType
+from tests.cases import FileHelperTestCases, ImageTestCases
+
+
+class TestBase64Utils:
+ """Tests for base64 utils."""
+
+ @pytest.mark.asyncio
+ async def test_load_binary_as_base64_async(self) -> None:
+ """Test asynchronous loading of binary file as base64 string."""
+ file_path = FileHelperTestCases.TEST_IMAGE
+ with open(file_path, "rb") as file_handle:
+ expected = base64.b64encode(file_handle.read()).decode("ascii")
+
+ result = await load_binary_as_base64(path=file_path)
+
+ assert result == expected
+ assert isinstance(result, str)
+
+ def test_make_base64_url_png(self) -> None:
+ """Test creating a base64 data URL for PNG."""
+ base64_data = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
+ file_type = FileType(extension="png", mime="image/png")
+
+ result = make_base64_url(base64_data=base64_data, file_type=file_type)
+
+ assert result == "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
+
+ def test_make_base64_url_jpeg(self) -> None:
+ """Test creating a base64 data URL for JPEG."""
+ base64_data = "/9j/4AAQSkZJRgABAQAAAQABAAD"
+ file_type = FileType(extension="jpg", mime="image/jpeg")
+
+ result = make_base64_url(base64_data=base64_data, file_type=file_type)
+
+ assert result == "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD"
+
+ def test_make_base64_url_pdf(self) -> None:
+ """Test creating a base64 data URL for PDF."""
+ base64_data = "JVBERi0xLjQK"
+ file_type = FileType(extension="pdf", mime="application/pdf")
+
+ result = make_base64_url(base64_data=base64_data, file_type=file_type)
+
+ assert result == "data:application/pdf;base64,JVBERi0xLjQK"
+
+ def test_make_base64_url_webp(self) -> None:
+ """Test creating a base64 data URL for WebP."""
+ base64_data = "UklGRhYAAABXRUJQ"
+ file_type = FileType(extension="webp", mime="image/webp")
+
+ result = make_base64_url(base64_data=base64_data, file_type=file_type)
+
+ assert result == "data:image/webp;base64,UklGRhYAAABXRUJQ"
+
+ @pytest.mark.asyncio
+ async def test_make_base64_url_from_path_async_png(self) -> None:
+ """Test async creating base64 URL from PNG file path."""
+ file_path = ImageTestCases.IMAGE_FILE_PATH_PNG_1
+
+ result = await make_base64_url_from_path(path=file_path)
+
+ assert result.startswith("data:image/png;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded[:8] == b"\x89PNG\r\n\x1a\n"
+
+ @pytest.mark.asyncio
+ async def test_make_base64_url_from_path_async_jpeg(self) -> None:
+ """Test async creating base64 URL from JPEG file path."""
+ file_path = ImageTestCases.IMAGE_FILE_PATH_JPG_1
+
+ result = await make_base64_url_from_path(path=file_path)
+
+ assert result.startswith("data:image/jpeg;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded[:2] == b"\xff\xd8"
+
+ @pytest.mark.parametrize(
+ "url",
+ [
+ "data:image/png;base64,iVBORw0KGgo=",
+ "data:image/jpeg;base64,/9j/4AAQ",
+ "data:application/pdf;base64,JVBERi0=",
+ "data:image/webp;base64,UklGRhYA",
+ "data:text/plain;base64,SGVsbG8=",
+ ],
+ )
+ def test_is_prefixed_base64_url_returns_true(self, url: str) -> None:
+ """Test that valid base64 data URLs return True."""
+ assert is_prefixed_base64_url(url) is True
+
+ @pytest.mark.parametrize(
+ "url",
+ [
+ "https://example.com/image.png",
+ "http://example.com/file.jpg",
+ "/path/to/file.png",
+ "relative/path.jpg",
+ "data:text/plain,Hello", # No ;base64,
+ "data:image/png,raw_data", # No ;base64,
+ "",
+ "base64,something", # Missing data: prefix
+ ],
+ )
+ def test_is_prefixed_base64_url_returns_false(self, url: str) -> None:
+ """Test that non-base64 URLs return False."""
+ assert is_prefixed_base64_url(url) is False
+
+ def test_strip_full_data_url(self) -> None:
+ """Test stripping full data URL prefix."""
+ base64_str = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg=="
+
+ result = strip_base64_str_if_needed(base64_str)
+
+ assert result == "iVBORw0KGgoAAAANSUhEUg=="
+
+ def test_strip_simple_comma_prefix(self) -> None:
+ """Test stripping when there's just a comma prefix."""
+ base64_str = "image/png;base64,iVBORw0KGgoAAAANSUhEUg=="
+
+ result = strip_base64_str_if_needed(base64_str)
+
+ assert result == "iVBORw0KGgoAAAANSUhEUg=="
+
+ def test_no_strip_needed(self) -> None:
+ """Test that plain base64 strings are returned as-is."""
+ base64_str = "iVBORw0KGgoAAAANSUhEUg=="
+
+ result = strip_base64_str_if_needed(base64_str)
+
+ assert result == "iVBORw0KGgoAAAANSUhEUg=="
+
+ def test_strip_with_different_mime_types(self) -> None:
+ """Test stripping with various MIME types."""
+ base64_str = "data:application/pdf;base64,JVBERi0xLjQK"
+
+ result = strip_base64_str_if_needed(base64_str)
+
+ assert result == "JVBERi0xLjQK"
+
+ def test_extract_png_base64_url(self) -> None:
+ """Test extracting from PNG data URL."""
+ url = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg=="
+
+ result = extract_base64_str_from_base64_url_if_possible(url)
+
+ assert result is not None
+ base64_str, mime_type = result
+ assert base64_str == "iVBORw0KGgoAAAANSUhEUg=="
+ assert mime_type == "image/png"
+
+ def test_extract_jpeg_base64_url(self) -> None:
+ """Test extracting from JPEG data URL."""
+ url = "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD"
+
+ result = extract_base64_str_from_base64_url_if_possible(url)
+
+ assert result is not None
+ base64_str, mime_type = result
+ assert base64_str == "/9j/4AAQSkZJRgABAQAAAQABAAD"
+ assert mime_type == "image/jpeg"
+
+ def test_extract_pdf_base64_url(self) -> None:
+ """Test extracting from PDF data URL."""
+ url = "data:application/pdf;base64,JVBERi0xLjQK"
+
+ result = extract_base64_str_from_base64_url_if_possible(url)
+
+ assert result is not None
+ base64_str, mime_type = result
+ assert base64_str == "JVBERi0xLjQK"
+ assert mime_type == "application/pdf"
+
+ def test_returns_none_for_non_data_url(self) -> None:
+ """Test that non-data URLs return None."""
+ url = "https://example.com/image.png"
+
+ result = extract_base64_str_from_base64_url_if_possible(url)
+
+ assert result is None
+
+ def test_returns_none_for_non_base64_data_url(self) -> None:
+ """Test that data URLs without ;base64, return None."""
+ url = "data:text/plain,Hello%20World"
+
+ result = extract_base64_str_from_base64_url_if_possible(url)
+
+ assert result is None
+
+ def test_returns_none_for_empty_string(self) -> None:
+ """Test that empty string returns None."""
+ result = extract_base64_str_from_base64_url_if_possible("")
+
+ assert result is None
+
+ def test_returns_none_for_local_path(self) -> None:
+ """Test that local paths return None."""
+ result = extract_base64_str_from_base64_url_if_possible("/path/to/file.png")
+
+ assert result is None
diff --git a/tests/unit/pipelex/tools/misc/base64/test_make_base64_url_from_http_url_async.py b/tests/unit/pipelex/tools/misc/base64/test_make_base64_url_from_http_url_async.py
new file mode 100644
index 000000000..364391087
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/base64/test_make_base64_url_from_http_url_async.py
@@ -0,0 +1,134 @@
+import base64
+
+import httpx
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.tools.misc.base64_utils import make_base64_url_from_http_url
+from tests.cases import ImageTestCases
+
+
[email protected](loop_scope="class")
+class TestMakeBase64UrlFromHttpUrlAsync:
+ """Tests for make_base64_url_from_http_url function with mocked HTTP calls."""
+
+ async def test_converts_png_url_to_base64_data_url(self, mocker: MockerFixture) -> None:
+ """Test that PNG HTTP URLs are converted to base64 data URLs."""
+ # Load real PNG bytes from test file
+ with open(ImageTestCases.IMAGE_FILE_PATH_PNG_1, "rb") as file_handle:
+ png_bytes = file_handle.read()
+
+ # Mock the HTTP fetch to return PNG bytes
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ return_value=png_bytes,
+ )
+
+ result = await make_base64_url_from_http_url(url="https://example.com/image.png")
+
+ assert result.startswith("data:image/png;base64,")
+ # Verify the base64 part decodes to original PNG bytes
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded == png_bytes
+
+ async def test_converts_jpeg_url_to_base64_data_url(self, mocker: MockerFixture) -> None:
+ """Test that JPEG HTTP URLs are converted to base64 data URLs."""
+ # Load real JPEG bytes from test file
+ with open(ImageTestCases.IMAGE_FILE_PATH_JPG_1, "rb") as file_handle:
+ jpeg_bytes = file_handle.read()
+
+ # Mock the HTTP fetch to return JPEG bytes
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ return_value=jpeg_bytes,
+ )
+
+ result = await make_base64_url_from_http_url(url="https://example.com/image.jpg")
+
+ assert result.startswith("data:image/jpeg;base64,")
+ # Verify the base64 part decodes to original JPEG bytes
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded == jpeg_bytes
+
+ async def test_propagates_http_error_on_404(self, mocker: MockerFixture) -> None:
+ """Test that HTTP 404 errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message="404 Not Found",
+ request=httpx.Request("GET", "https://example.com/not-found.png"),
+ response=httpx.Response(404),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await make_base64_url_from_http_url(url="https://example.com/not-found.png")
+
+ assert exc_info.value.response.status_code == 404
+
+ async def test_propagates_http_error_on_500(self, mocker: MockerFixture) -> None:
+ """Test that HTTP 500 errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message="500 Internal Server Error",
+ request=httpx.Request("GET", "https://example.com/error.png"),
+ response=httpx.Response(500),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await make_base64_url_from_http_url(url="https://example.com/error.png")
+
+ assert exc_info.value.response.status_code == 500
+
+ async def test_propagates_connection_error(self, mocker: MockerFixture) -> None:
+ """Test that connection errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.ConnectError("Connection refused"),
+ )
+
+ with pytest.raises(httpx.ConnectError):
+ await make_base64_url_from_http_url(url="https://unreachable.example.com/image.png")
+
+ async def test_propagates_timeout_error(self, mocker: MockerFixture) -> None:
+ """Test that timeout errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.TimeoutException("Request timed out"),
+ )
+
+ with pytest.raises(httpx.TimeoutException):
+ await make_base64_url_from_http_url(url="https://slow.example.com/image.png")
+
+ @pytest.mark.parametrize(
+ ("topic", "file_path", "expected_mime_prefix"),
+ [
+ ("png", ImageTestCases.IMAGE_FILE_PATH_PNG_1, "data:image/png;base64,"),
+ ("jpeg", ImageTestCases.IMAGE_FILE_PATH_JPG_1, "data:image/jpeg;base64,"),
+ ],
+ )
+ async def test_detects_file_type_from_content(
+ self,
+ mocker: MockerFixture,
+ topic: str,
+ file_path: str,
+ expected_mime_prefix: str,
+ ) -> None:
+ """Test that file type is correctly detected from content bytes, not URL."""
+ with open(file_path, "rb") as file_handle:
+ file_bytes = file_handle.read()
+
+ # Mock with a misleading URL extension
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ return_value=file_bytes,
+ )
+
+ # URL says .txt but content is image - should detect from content
+ result = await make_base64_url_from_http_url(url=f"https://example.com/file.txt?topic={topic}")
+
+ assert result.startswith(expected_mime_prefix)
diff --git a/tests/unit/pipelex/tools/misc/test_base64_utils.py b/tests/unit/pipelex/tools/misc/test_base64_utils.py
deleted file mode 100644
index b3fe09752..000000000
--- a/tests/unit/pipelex/tools/misc/test_base64_utils.py
+++ /dev/null
@@ -1,27 +0,0 @@
-import base64
-
-import pytest
-
-from pipelex.tools.misc.base64_utils import load_binary_as_base64, load_binary_as_base64_async
-from tests.cases import FileHelperTestCases
-
-
-class TestBase64Utils:
- def test_load_binary_as_base64(self) -> None:
- file_path = FileHelperTestCases.TEST_IMAGE
- with open(file_path, "rb") as f:
- expected = base64.b64encode(f.read())
-
- result = load_binary_as_base64(path=file_path)
-
- assert result == expected
-
- @pytest.mark.asyncio
- async def test_load_binary_as_base64_async(self) -> None:
- file_path = FileHelperTestCases.TEST_IMAGE
- with open(file_path, "rb") as f:
- expected = base64.b64encode(f.read())
-
- result = await load_binary_as_base64_async(path=file_path)
-
- assert result == expected
diff --git a/tests/unit/pipelex/tools/misc/test_path_utils.py b/tests/unit/pipelex/tools/misc/test_path_utils.py
deleted file mode 100644
index 539bcdecd..000000000
--- a/tests/unit/pipelex/tools/misc/test_path_utils.py
+++ /dev/null
@@ -1,208 +0,0 @@
-import os
-
-import pytest
-from pytest_mock import MockerFixture
-
-from pipelex.tools.misc.path_utils import (
- InterpretedPathOrUrl,
- clarify_path_or_url,
- interpret_path_or_url,
- is_pipelex_storage_uri,
-)
-from pipelex.tools.storage.storage_provider_abstract import PIPELEX_STORAGE_SCHEME
-
-
-class TestInterpretPathOrUrl:
- def test_interpret_file_uri(self):
- result = interpret_path_or_url("file:///home/user/file.txt")
- assert result == InterpretedPathOrUrl.LOCAL_FILE_PATH_URL
-
- def test_interpret_file_uri_windows(self):
- result = interpret_path_or_url("file:///C:/Users/user/file.txt")
- assert result == InterpretedPathOrUrl.LOCAL_FILE_PATH_URL
-
- def test_interpret_http_url(self):
- result = interpret_path_or_url("http://example.com")
- assert result == InterpretedPathOrUrl.HTTP_URL
-
- def test_interpret_https_url(self):
- result = interpret_path_or_url("https://example.com/path")
- assert result == InterpretedPathOrUrl.HTTP_URL
-
- def test_interpret_absolute_file_path_unix(self):
- result = interpret_path_or_url("/home/user/file.txt")
- assert result == InterpretedPathOrUrl.FILE_PATH
-
- def test_interpret_relative_file_path_unix(self):
- result = interpret_path_or_url("user/file.txt")
- assert result == InterpretedPathOrUrl.FILE_PATH
-
- def test_interpret_absolute_file_path_windows_style(self):
- # Test with actual Windows-style paths (backslashes)
- # On Unix systems, backslashes aren't recognized as path separators
- result = interpret_path_or_url("C:\\Users\\user\\file.txt")
- assert result == InterpretedPathOrUrl.FILE_NAME
-
- def test_interpret_relative_file_path_windows_style(self):
- # Test with actual Windows-style paths (backslashes)
- # On Unix systems, backslashes aren't recognized as path separators
- result = interpret_path_or_url("user\\file.txt")
- assert result == InterpretedPathOrUrl.FILE_NAME
-
- def test_interpret_path_with_current_os_separator(self, mocker: MockerFixture): # noqa: ARG002
- # Test with the current OS path separator
- path_with_sep = f"user{os.sep}file.txt"
- result = interpret_path_or_url(path_with_sep)
- assert result == InterpretedPathOrUrl.FILE_PATH
-
- def test_interpret_file_name_only(self):
- result = interpret_path_or_url("file.txt")
- assert result == InterpretedPathOrUrl.FILE_NAME
-
- def test_interpret_file_name_no_extension(self):
- result = interpret_path_or_url("filename")
- assert result == InterpretedPathOrUrl.FILE_NAME
-
- def test_interpret_empty_string(self):
- result = interpret_path_or_url("")
- assert result == InterpretedPathOrUrl.FILE_NAME
-
- def test_interpret_edge_case_http_in_path(self):
- result = interpret_path_or_url("/path/to/http_file.txt")
- assert result == InterpretedPathOrUrl.FILE_PATH
-
- def test_interpret_edge_case_file_in_url(self):
- result = interpret_path_or_url("http://example.com/file.txt")
- assert result == InterpretedPathOrUrl.HTTP_URL
-
- def test_interpret_pipelex_storage_uri(self):
- result = interpret_path_or_url(f"{PIPELEX_STORAGE_SCHEME}images/photo.png")
- assert result == InterpretedPathOrUrl.PIPELEX_STORAGE
-
- def test_interpret_pipelex_storage_uri_with_nested_path(self):
- result = interpret_path_or_url(f"{PIPELEX_STORAGE_SCHEME}user123/run456/image.jpg")
- assert result == InterpretedPathOrUrl.PIPELEX_STORAGE
-
- def test_interpret_pipelex_storage_takes_precedence(self):
- """Test that pipelex-storage:// is detected before checking for path separators."""
- # This URI contains a path separator but should be recognized as pipelex storage
- result = interpret_path_or_url(f"{PIPELEX_STORAGE_SCHEME}path/to/file.bin")
- assert result == InterpretedPathOrUrl.PIPELEX_STORAGE
-
-
-class TestClarifyPathOrUrl:
- def test_clarify_file_uri_unix(self):
- file_path, url = clarify_path_or_url("file:///home/user/file.txt")
- assert file_path == "/home/user/file.txt"
- assert url is None
-
- def test_clarify_file_uri_windows(self):
- file_path, url = clarify_path_or_url("file:///C:/Users/user/file.txt")
- assert file_path == "/C:/Users/user/file.txt"
- assert url is None
-
- def test_clarify_file_uri_with_spaces(self) -> None:
- file_path, url = clarify_path_or_url("file:///home/user/my%20file.txt")
- assert file_path == "/home/user/my file.txt"
- assert url is None
-
- def test_clarify_file_uri_with_special_chars(self):
- file_path, url = clarify_path_or_url("file:///home/user/file%20%26%20more.txt")
- assert file_path == "/home/user/file & more.txt"
- assert url is None
-
- def test_clarify_http_url(self):
- file_path, url = clarify_path_or_url("http://example.com")
- assert file_path is None
- assert url == "http://example.com"
-
- def test_clarify_https_url(self):
- file_path, url = clarify_path_or_url("https://example.com/path/to/file")
- assert file_path is None
- assert url == "https://example.com/path/to/file"
-
- def test_clarify_absolute_file_path(self):
- file_path, url = clarify_path_or_url("/home/user/file.txt")
- assert file_path == "/home/user/file.txt"
- assert url is None
-
- def test_clarify_relative_file_path(self):
- file_path, url = clarify_path_or_url("user/file.txt")
- assert file_path == "user/file.txt"
- assert url is None
-
- def test_clarify_file_name_only(self):
- file_path, url = clarify_path_or_url("file.txt")
- assert file_path == "file.txt"
- assert url is None
-
- def test_clarify_empty_string(self):
- file_path, url = clarify_path_or_url("")
- assert file_path == ""
- assert url is None
-
- def test_clarify_base_64_raises_not_implemented(self, mocker: MockerFixture):
- # Mock interpret_path_or_url to return BASE_64
- mocker.patch(
- "pipelex.tools.misc.path_utils.interpret_path_or_url",
- return_value=InterpretedPathOrUrl.BASE_64,
- )
- with pytest.raises(NotImplementedError, match="Base 64 is not supported by clarify_path_or_url"):
- clarify_path_or_url("some_base64_data")
-
- def test_clarify_pipelex_storage_raises_not_implemented(self):
- pipelex_uri = f"{PIPELEX_STORAGE_SCHEME}images/photo.png"
- with pytest.raises(NotImplementedError, match="Pipelex storage URIs are not supported by clarify_path_or_url"):
- clarify_path_or_url(pipelex_uri)
-
- def test_clarify_uses_interpret_path_or_url(self, mocker: MockerFixture):
- mock_interpret = mocker.patch(
- "pipelex.tools.misc.path_utils.interpret_path_or_url",
- return_value=InterpretedPathOrUrl.FILE_PATH,
- )
-
- clarify_path_or_url("/test/path")
-
- mock_interpret.assert_called_once_with("/test/path")
-
- def test_clarify_urlparse_called_for_file_uri(self, mocker: MockerFixture):
- mock_urlparse = mocker.patch("urllib.parse.urlparse")
- mock_urlparse.return_value.path = "/home/user/file.txt"
- mock_unquote = mocker.patch("urllib.parse.unquote", return_value="/home/user/file.txt")
-
- mocker.patch(
- "pipelex.tools.misc.path_utils.interpret_path_or_url",
- return_value=InterpretedPathOrUrl.LOCAL_FILE_PATH_URL,
- )
-
- file_path, url = clarify_path_or_url("file:///home/user/file.txt")
-
- mock_urlparse.assert_called_once_with("file:///home/user/file.txt")
- mock_unquote.assert_called_once_with("/home/user/file.txt")
- assert file_path == "/home/user/file.txt"
- assert url is None
-
-
-class TestIsPipelexStorageUri:
- def test_returns_true_for_pipelex_storage_uri(self):
- assert is_pipelex_storage_uri(f"{PIPELEX_STORAGE_SCHEME}images/photo.png") is True
-
- def test_returns_true_for_pipelex_storage_uri_with_nested_path(self):
- assert is_pipelex_storage_uri(f"{PIPELEX_STORAGE_SCHEME}user/run/image.jpg") is True
-
- def test_returns_false_for_http_url(self):
- assert is_pipelex_storage_uri("https://example.com/image.png") is False
-
- def test_returns_false_for_file_path(self):
- assert is_pipelex_storage_uri("/home/user/image.png") is False
-
- def test_returns_false_for_file_uri(self):
- assert is_pipelex_storage_uri("file:///home/user/image.png") is False
-
- def test_returns_false_for_empty_string(self):
- assert is_pipelex_storage_uri("") is False
-
- def test_returns_false_for_similar_prefix(self):
- # Ensure partial matches don't return true
- assert is_pipelex_storage_uri("pipelex-storag://images/photo.png") is False
- assert is_pipelex_storage_uri("pipelex-storage:/images/photo.png") is False
diff --git a/tests/unit/pipelex/tools/uri/__init__.py b/tests/unit/pipelex/tools/uri/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/unit/pipelex/tools/uri/test_make_base64_url_from_uri_async.py b/tests/unit/pipelex/tools/uri/test_make_base64_url_from_uri_async.py
new file mode 100644
index 000000000..9fbec6342
--- /dev/null
+++ b/tests/unit/pipelex/tools/uri/test_make_base64_url_from_uri_async.py
@@ -0,0 +1,133 @@
+import base64
+from pathlib import Path
+
+import httpx
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.tools.storage.storage_provider_abstract import PIPELEX_STORAGE_SCHEME
+from pipelex.tools.uri.uri_resolver import make_base64_url_from_any_uri
+from tests.cases import ImageTestCases
+
+
[email protected](loop_scope="class")
+class TestMakeBase64UrlFromUriAsync:
+ """Tests for make_base64_url_from_any_uri function."""
+
+ async def test_local_path_returns_data_url(self) -> None:
+ """Test that local file paths are converted to base64 data URLs."""
+ file_path = ImageTestCases.IMAGE_FILE_PATH_PNG_1
+
+ result = await make_base64_url_from_any_uri(file_path)
+
+ # Should start with data URL prefix for PNG
+ assert result.startswith("data:image/png;base64,")
+ # Verify the base64 part is valid
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ # PNG magic bytes
+ assert decoded[:8] == b"\x89PNG\r\n\x1a\n"
+
+ async def test_local_path_jpeg_returns_data_url(self) -> None:
+ """Test that JPEG files are converted correctly."""
+ file_path = ImageTestCases.IMAGE_FILE_PATH_JPG_1
+
+ result = await make_base64_url_from_any_uri(file_path)
+
+ # Should start with data URL prefix for JPEG
+ assert result.startswith("data:image/jpeg;base64,")
+ # Verify the base64 part is valid
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ # JPEG magic bytes
+ assert decoded[:2] == b"\xff\xd8"
+
+ async def test_data_url_passthrough(self) -> None:
+ """Test that existing data URLs are returned as-is."""
+ original_data_url = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg=="
+
+ result = await make_base64_url_from_any_uri(original_data_url)
+
+ assert result == original_data_url
+
+ async def test_data_url_passthrough_with_complex_content(self) -> None:
+ """Test passthrough with a more complex data URL."""
+ # A realistic base64 content string
+ base64_content = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
+ original_data_url = f"data:image/png;base64,{base64_content}"
+
+ result = await make_base64_url_from_any_uri(original_data_url)
+
+ assert result == original_data_url
+
+ async def test_pipelex_storage_raises_value_error(self) -> None:
+ """Test that pipelex-storage:// URIs raise ValueError."""
+ storage_uri = f"{PIPELEX_STORAGE_SCHEME}images/photo.png"
+
+ with pytest.raises(ValueError, match="Unsupported URI type"):
+ await make_base64_url_from_any_uri(storage_uri)
+
+ async def test_http_url_converts_to_base64_data_url(self, mocker: MockerFixture) -> None:
+ """Test that HTTP URLs are fetched and converted to base64 data URLs."""
+ # Load real PNG bytes from test file
+ with open(ImageTestCases.IMAGE_FILE_PATH_PNG_1, "rb") as file_handle:
+ png_bytes = file_handle.read()
+
+ # Mock the HTTP fetch to return PNG bytes
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ return_value=png_bytes,
+ )
+
+ result = await make_base64_url_from_any_uri("https://example.com/image.png")
+
+ assert result.startswith("data:image/png;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded == png_bytes
+
+ async def test_https_url_converts_to_base64_data_url(self, mocker: MockerFixture) -> None:
+ """Test that HTTPS URLs are fetched and converted to base64 data URLs."""
+ # Load real JPEG bytes from test file
+ with open(ImageTestCases.IMAGE_FILE_PATH_JPG_1, "rb") as file_handle:
+ jpeg_bytes = file_handle.read()
+
+ # Mock the HTTP fetch to return JPEG bytes
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ return_value=jpeg_bytes,
+ )
+
+ result = await make_base64_url_from_any_uri("https://secure.example.com/photo.jpg")
+
+ assert result.startswith("data:image/jpeg;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded == jpeg_bytes
+
+ async def test_http_url_propagates_network_errors(self, mocker: MockerFixture) -> None:
+ """Test that HTTP network errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message="404 Not Found",
+ request=httpx.Request("GET", "https://example.com/not-found.png"),
+ response=httpx.Response(404),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError):
+ await make_base64_url_from_any_uri("https://example.com/not-found.png")
+
+ async def test_file_uri_converts_to_base64_data_url(self) -> None:
+ """Test that file:// URIs are converted to base64 data URLs."""
+ # file:// URIs need absolute paths
+ absolute_path = Path(ImageTestCases.IMAGE_FILE_PATH_PNG_1).resolve()
+ file_uri = f"file://{absolute_path}"
+
+ result = await make_base64_url_from_any_uri(file_uri)
+
+ assert result.startswith("data:image/png;base64,")
+ base64_part = result.split(",", 1)[1]
+ decoded = base64.b64decode(base64_part)
+ assert decoded[:8] == b"\x89PNG\r\n\x1a\n"
diff --git a/tests/unit/pipelex/tools/uri/test_uri_resolver.py b/tests/unit/pipelex/tools/uri/test_uri_resolver.py
new file mode 100644
index 000000000..276567698
--- /dev/null
+++ b/tests/unit/pipelex/tools/uri/test_uri_resolver.py
@@ -0,0 +1,520 @@
+import os
+
+import pytest
+
+from pipelex.tools.storage.storage_provider_abstract import PIPELEX_STORAGE_SCHEME
+from pipelex.tools.uri.resolved_uri import (
+ ResolvedBase64DataUrl,
+ ResolvedHttpUrl,
+ ResolvedLocalPath,
+ ResolvedPipelexStorage,
+ ResolvedUri,
+ UriKind,
+)
+from pipelex.tools.uri.uri_resolver import resolve_uri
+
+
+class TestResolveUriHttpUrls:
+ """Tests for resolving HTTP/HTTPS URLs."""
+
+ def test_resolve_https_url(self) -> None:
+ """Test that HTTPS URLs are resolved_uri to ResolvedHttpUrl."""
+ uri = "https://example.com/image.png"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ assert isinstance(resolved_uri, ResolvedHttpUrl)
+ assert resolved_uri.url == uri
+ assert resolved_uri.original == uri
+
+ def test_resolve_http_url(self) -> None:
+ """Test that HTTP URLs are resolved_uri to ResolvedHttpUrl."""
+ uri = "http://example.com/path/to/file.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ assert isinstance(resolved_uri, ResolvedHttpUrl)
+ assert resolved_uri.url == uri
+ assert resolved_uri.original == uri
+
+ def test_resolve_http_url_with_query_params(self) -> None:
+ """Test that HTTP URLs with query parameters are preserved."""
+ uri = "https://example.com/api?key=value&other=123"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ assert isinstance(resolved_uri, ResolvedHttpUrl)
+ assert resolved_uri.url == uri
+
+ def test_resolve_http_url_with_fragment(self) -> None:
+ """Test that HTTP URLs with fragments are preserved."""
+ uri = "https://example.com/page#section"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ assert isinstance(resolved_uri, ResolvedHttpUrl)
+ assert resolved_uri.url == uri
+
+
+class TestResolveUriLocalPaths:
+ """Tests for resolving local file paths."""
+
+ def test_resolve_absolute_unix_path(self) -> None:
+ """Test that absolute Unix paths are resolved_uri to ResolvedLocalPath."""
+ uri = "/home/user/documents/file.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == uri
+ assert resolved_uri.original == uri
+
+ def test_resolve_relative_path_with_separator(self) -> None:
+ """Test that relative paths with OS separator are resolved_uri to ResolvedLocalPath."""
+ uri = f"user{os.sep}documents{os.sep}file.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == uri
+
+ def test_resolve_file_name_only(self) -> None:
+ """Test that simple file names are resolved_uri to ResolvedLocalPath."""
+ uri = "document.pdf"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == uri
+
+ def test_resolve_file_name_without_extension(self) -> None:
+ """Test that file names without extensions are resolved_uri to ResolvedLocalPath."""
+ uri = "readme"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == uri
+
+
+class TestResolveUriFileUri:
+ """Tests for resolving file:// URIs."""
+
+ def test_resolve_file_uri_unix(self) -> None:
+ """Test that file:// URIs are converted to local paths."""
+ uri = "file:///home/user/file.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == "/home/user/file.txt"
+ assert resolved_uri.original == uri
+
+ def test_resolve_file_uri_with_spaces(self) -> None:
+ """Test that file:// URIs with URL-encoded spaces are decoded."""
+ uri = "file:///home/user/my%20document.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == "/home/user/my document.txt"
+
+ def test_resolve_file_uri_with_special_chars(self) -> None:
+ """Test that file:// URIs with special characters are decoded."""
+ uri = "file:///home/user/file%20%26%20more.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == "/home/user/file & more.txt"
+
+ def test_resolve_file_uri_windows_style(self) -> None:
+ """Test that Windows file:// URIs are converted."""
+ uri = "file:///C:/Users/user/file.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ # The path includes the leading slash from the URI
+ assert resolved_uri.path == "/C:/Users/user/file.txt"
+
+
+class TestResolveUriPipelexStorage:
+ """Tests for resolving pipelex-storage:// URIs."""
+
+ def test_resolve_pipelex_storage_uri(self) -> None:
+ """Test that pipelex-storage:// URIs are resolved_uri to ResolvedPipelexStorage."""
+ uri = f"{PIPELEX_STORAGE_SCHEME}images/photo.png"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.PIPELEX_STORAGE
+ assert isinstance(resolved_uri, ResolvedPipelexStorage)
+ assert resolved_uri.storage_uri == uri
+ assert resolved_uri.original == uri
+
+ def test_resolve_pipelex_storage_uri_nested_path(self) -> None:
+ """Test pipelex-storage:// URIs with nested paths."""
+ uri = f"{PIPELEX_STORAGE_SCHEME}user123/run456/outputs/image.jpg"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.PIPELEX_STORAGE
+ assert isinstance(resolved_uri, ResolvedPipelexStorage)
+ assert resolved_uri.storage_uri == uri
+
+ def test_resolve_pipelex_storage_takes_precedence(self) -> None:
+ """Test that pipelex-storage:// is detected before path separator check."""
+ uri = f"{PIPELEX_STORAGE_SCHEME}path/with/separators/file.bin"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.PIPELEX_STORAGE
+ assert isinstance(resolved_uri, ResolvedPipelexStorage)
+
+
+class TestResolveUriBase64DataUrl:
+ """Tests for resolving base64 data URLs."""
+
+ def test_resolve_base64_data_url_png(self) -> None:
+ """Test that data: URLs with base64 encoding are resolved_uri to ResolvedBase64DataUrl."""
+ base64_content = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
+ uri = f"data:image/png;base64,{base64_content}"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.BASE64_DATA_URL
+ assert isinstance(resolved_uri, ResolvedBase64DataUrl)
+ assert resolved_uri.mime_type == "image/png"
+ assert resolved_uri.base64_data == base64_content
+ assert resolved_uri.original == uri
+
+ def test_resolve_base64_data_url_jpeg(self) -> None:
+ """Test base64 data URL with JPEG mime type."""
+ base64_content = "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRof"
+ uri = f"data:image/jpeg;base64,{base64_content}"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.BASE64_DATA_URL
+ assert isinstance(resolved_uri, ResolvedBase64DataUrl)
+ assert resolved_uri.mime_type == "image/jpeg"
+ assert resolved_uri.base64_data == base64_content
+
+ def test_resolve_base64_data_url_pdf(self) -> None:
+ """Test base64 data URL with PDF mime type."""
+ base64_content = "JVBERi0xLjQKJeLjz9MKMyAwIG9iago8PC9UeXBlL0NhdGFsb2cvUGFnZXMgMiAwIFI+PgplbmRvYmo="
+ uri = f"data:application/pdf;base64,{base64_content}"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.BASE64_DATA_URL
+ assert isinstance(resolved_uri, ResolvedBase64DataUrl)
+ assert resolved_uri.mime_type == "application/pdf"
+ assert resolved_uri.base64_data == base64_content
+
+ def test_resolve_base64_data_url_webp(self) -> None:
+ """Test base64 data URL with WebP mime type."""
+ base64_content = "UklGRhYAAABXRUJQVlA4TAoAAAAvAAAAABPpAA=="
+ uri = f"data:image/webp;base64,{base64_content}"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.BASE64_DATA_URL
+ assert isinstance(resolved_uri, ResolvedBase64DataUrl)
+ assert resolved_uri.mime_type == "image/webp"
+
+
+class TestResolveUriEdgeCases:
+ """Tests for edge cases and validation."""
+
+ def test_resolve_empty_string(self) -> None:
+ """Test that empty strings are resolved_uri to ResolvedLocalPath."""
+ uri = ""
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == ""
+
+ def test_resolve_http_in_filename(self) -> None:
+ """Test that 'http' in a file path doesn't trigger HTTP URL detection."""
+ uri = "/path/to/http_config.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert isinstance(resolved_uri, ResolvedLocalPath)
+ assert resolved_uri.path == uri
+
+ def test_resolve_file_in_http_url(self) -> None:
+ """Test that 'file' in an HTTP URL doesn't trigger file path detection."""
+ uri = "https://example.com/download/file.txt"
+
+ resolved_uri = resolve_uri(uri)
+
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ assert isinstance(resolved_uri, ResolvedHttpUrl)
+
+ def test_data_url_without_base64_is_not_base64_data_url(self) -> None:
+ """Test that data: URLs without ;base64, are treated as local paths."""
+ uri = "data:text/plain,Hello%20World"
+
+ resolved_uri = resolve_uri(uri)
+
+ # Without ;base64, this is not a base64 data URL
+ # It should fall through to local path (has no path separator)
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+
+
+class TestUriKindEnum:
+ """Tests for the UriKind StrEnum."""
+
+ def test_uri_kind_values(self) -> None:
+ """Test that UriKind has the expected values."""
+ assert UriKind.HTTP_URL == "http_url"
+ assert UriKind.LOCAL_PATH == "local_path"
+ assert UriKind.PIPELEX_STORAGE == "pipelex_storage"
+ assert UriKind.BASE64_DATA_URL == "base64_data_url"
+
+ def test_uri_kind_is_string(self) -> None:
+ """Test that UriKind values work as strings."""
+ assert str(UriKind.HTTP_URL) == "http_url"
+ assert f"{UriKind.LOCAL_PATH}" == "local_path"
+
+ def test_uri_kind_desc_property(self) -> None:
+ """Test that UriKind.desc returns human-readable descriptions."""
+ assert UriKind.HTTP_URL.desc == "HTTP URL"
+ assert UriKind.LOCAL_PATH.desc == "Local path"
+ assert UriKind.PIPELEX_STORAGE.desc == "Pipelex Storage"
+ assert UriKind.BASE64_DATA_URL.desc == "Base64 data URL"
+
+
+class TestResolvedUriModels:
+ """Tests for ResolvedUri model properties."""
+
+ def test_resolved_http_url_model(self) -> None:
+ """Test ResolvedHttpUrl model structure."""
+ resolved_uri = ResolvedHttpUrl(
+ original="https://example.com",
+ url="https://example.com",
+ )
+
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ assert resolved_uri.original == "https://example.com"
+ assert resolved_uri.url == "https://example.com"
+
+ def test_resolved_local_path_model(self) -> None:
+ """Test ResolvedLocalPath model structure."""
+ resolved_uri = ResolvedLocalPath(
+ original="/path/to/file",
+ path="/path/to/file",
+ )
+
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ assert resolved_uri.original == "/path/to/file"
+ assert resolved_uri.path == "/path/to/file"
+
+ def test_resolved_pipelex_storage_model(self) -> None:
+ """Test ResolvedPipelexStorage model structure."""
+ storage_uri = f"{PIPELEX_STORAGE_SCHEME}key/file.bin"
+ resolved_uri = ResolvedPipelexStorage(
+ original=storage_uri,
+ storage_uri=storage_uri,
+ )
+
+ assert resolved_uri.kind == UriKind.PIPELEX_STORAGE
+ assert resolved_uri.original == storage_uri
+ assert resolved_uri.storage_uri == storage_uri
+
+ def test_resolved_base64_data_url_model(self) -> None:
+ """Test ResolvedBase64DataUrl model structure."""
+ resolved_uri = ResolvedBase64DataUrl(
+ original="data:image/png;base64,abc123",
+ mime_type="image/png",
+ base64_data="abc123",
+ )
+
+ assert resolved_uri.kind == UriKind.BASE64_DATA_URL
+ assert resolved_uri.original == "data:image/png;base64,abc123"
+ assert resolved_uri.mime_type == "image/png"
+ assert resolved_uri.base64_data == "abc123"
+
+
+class TestResolvedUriTypeUnion:
+ """Tests for the ResolvedUri union type."""
+
+ @pytest.mark.parametrize(
+ ("uri", "expected_kind"),
+ [
+ ("https://example.com", UriKind.HTTP_URL),
+ ("http://example.com", UriKind.HTTP_URL),
+ ("/absolute/path.txt", UriKind.LOCAL_PATH),
+ ("relative/path.txt", UriKind.LOCAL_PATH),
+ ("filename.txt", UriKind.LOCAL_PATH),
+ ("file:///home/user/file.txt", UriKind.LOCAL_PATH),
+ (f"{PIPELEX_STORAGE_SCHEME}key/file.bin", UriKind.PIPELEX_STORAGE),
+ ("data:image/png;base64,abc", UriKind.BASE64_DATA_URL),
+ ],
+ )
+ def test_resolve_uri_returns_correct_kind(self, uri: str, expected_kind: UriKind) -> None:
+ """Test that resolve_uri returns the correct kind for various URIs."""
+ resolved_uri = resolve_uri(uri)
+ assert resolved_uri.kind == expected_kind
+
+ def test_resolved_uri_type_annotation(self) -> None:
+ """Test that ResolvedUri type annotation works for all variants."""
+ results: list[ResolvedUri] = [
+ resolve_uri("https://example.com"),
+ resolve_uri("/path/to/file"),
+ resolve_uri(f"{PIPELEX_STORAGE_SCHEME}key"),
+ resolve_uri("data:image/png;base64,abc"),
+ ]
+
+ kinds = [result.kind for result in results]
+ assert UriKind.HTTP_URL in kinds
+ assert UriKind.LOCAL_PATH in kinds
+ assert UriKind.PIPELEX_STORAGE in kinds
+ assert UriKind.BASE64_DATA_URL in kinds
+
+
+class TestResolvedUriMatchCase:
+ """Tests demonstrating match/case pattern matching with ResolvedUri types.
+
+ These tests verify that:
+ 1. Class pattern matching works correctly with ResolvedUri discriminated union
+ 2. Type narrowing allows direct attribute access without isinstance assertions
+ 3. All union variants are properly handled (exhaustiveness)
+ """
+
+ def test_match_case_http_url(self) -> None:
+ """Test match/case correctly narrows type to ResolvedHttpUrl."""
+ resolved_uri = resolve_uri("https://api.example.com/data")
+
+ result: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ # Type is narrowed: can access .url directly
+ result = f"HTTP: {resolved_uri.url}"
+ case ResolvedLocalPath():
+ result = f"Path: {resolved_uri.path}"
+ case ResolvedPipelexStorage():
+ result = f"Storage: {resolved_uri.storage_uri}"
+ case ResolvedBase64DataUrl():
+ result = f"Data: {resolved_uri.mime_type}"
+
+ assert result == "HTTP: https://api.example.com/data"
+
+ def test_match_case_local_path(self) -> None:
+ """Test match/case correctly narrows type to ResolvedLocalPath."""
+ resolved_uri = resolve_uri("/home/user/document.pdf")
+
+ result: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ result = f"HTTP: {resolved_uri.url}"
+ case ResolvedLocalPath():
+ # Type is narrowed: can access .path directly
+ result = f"Path: {resolved_uri.path}"
+ case ResolvedPipelexStorage():
+ result = f"Storage: {resolved_uri.storage_uri}"
+ case ResolvedBase64DataUrl():
+ result = f"Data: {resolved_uri.mime_type}"
+
+ assert result == "Path: /home/user/document.pdf"
+
+ def test_match_case_pipelex_storage(self) -> None:
+ """Test match/case correctly narrows type to ResolvedPipelexStorage."""
+ uri = f"{PIPELEX_STORAGE_SCHEME}run123/output.png"
+ resolved_uri = resolve_uri(uri)
+
+ result: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ result = f"HTTP: {resolved_uri.url}"
+ case ResolvedLocalPath():
+ result = f"Path: {resolved_uri.path}"
+ case ResolvedPipelexStorage():
+ # Type is narrowed: can access .storage_uri directly
+ result = f"Storage: {resolved_uri.storage_uri}"
+ case ResolvedBase64DataUrl():
+ result = f"Data: {resolved_uri.mime_type}"
+
+ assert result == f"Storage: {uri}"
+
+ def test_match_case_base64_data_url(self) -> None:
+ """Test match/case correctly narrows type to ResolvedBase64DataUrl."""
+ resolved_uri = resolve_uri("data:image/webp;base64,UklGRhYA")
+
+ result: str
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ result = f"HTTP: {resolved_uri.url}"
+ case ResolvedLocalPath():
+ result = f"Path: {resolved_uri.path}"
+ case ResolvedPipelexStorage():
+ result = f"Storage: {resolved_uri.storage_uri}"
+ case ResolvedBase64DataUrl():
+ # Type is narrowed: can access .mime_type and .base64_data directly
+ result = f"Data: {resolved_uri.mime_type}, {len(resolved_uri.base64_data)} chars"
+
+ assert result == "Data: image/webp, 8 chars"
+
+ def test_match_case_combined_cases(self) -> None:
+ """Test match/case with combined case patterns using | operator."""
+ test_uris = [
+ "https://example.com/file.txt",
+ "/local/file.txt",
+ f"{PIPELEX_STORAGE_SCHEME}key",
+ "data:text/plain;base64,SGVsbG8=",
+ ]
+
+ results: list[str] = []
+ for uri in test_uris:
+ resolved_uri = resolve_uri(uri)
+ match resolved_uri:
+ case ResolvedHttpUrl() | ResolvedLocalPath():
+ # Both provide path-like access
+ results.append("fetchable")
+ case ResolvedPipelexStorage() | ResolvedBase64DataUrl():
+ # Both require special handling
+ results.append("special")
+
+ assert results == ["fetchable", "fetchable", "special", "special"]
+
+ def test_match_case_access_common_attributes(self) -> None:
+ """Test that common attributes (kind, original) are accessible in all cases."""
+ uris = [
+ "https://example.com",
+ "/path/to/file",
+ f"{PIPELEX_STORAGE_SCHEME}key",
+ "data:image/png;base64,abc",
+ ]
+
+ for uri in uris:
+ resolved_uri = resolve_uri(uri)
+ # .original is always available on base class
+ assert resolved_uri.original == uri
+ # .kind is always available and matches the case
+ match resolved_uri:
+ case ResolvedHttpUrl():
+ assert resolved_uri.kind == UriKind.HTTP_URL
+ case ResolvedLocalPath():
+ assert resolved_uri.kind == UriKind.LOCAL_PATH
+ case ResolvedPipelexStorage():
+ assert resolved_uri.kind == UriKind.PIPELEX_STORAGE
+ case ResolvedBase64DataUrl():
+ assert resolved_uri.kind == UriKind.BASE64_DATA_URL
diff --git a/tests/unit/pipelex/tools/uri/test_uri_resolver_error_handling.py b/tests/unit/pipelex/tools/uri/test_uri_resolver_error_handling.py
new file mode 100644
index 000000000..1a3b07a23
--- /dev/null
+++ b/tests/unit/pipelex/tools/uri/test_uri_resolver_error_handling.py
@@ -0,0 +1,159 @@
+import httpx
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.tools.storage.storage_provider_abstract import PIPELEX_STORAGE_SCHEME
+from pipelex.tools.uri.uri_resolver import make_base64_url_from_any_uri
+
+
[email protected](loop_scope="class")
+class TestUriResolverErrorHandling:
+ """Tests for error handling in URI resolver functions."""
+
+ async def test_http_404_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that HTTP 404 errors are propagated through make_base64_url_from_any_uri."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message="404 Not Found",
+ request=httpx.Request("GET", "https://example.com/not-found.png"),
+ response=httpx.Response(404),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await make_base64_url_from_any_uri("https://example.com/not-found.png")
+
+ assert exc_info.value.response.status_code == 404
+
+ async def test_http_500_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that HTTP 500 errors are propagated through make_base64_url_from_any_uri."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message="500 Internal Server Error",
+ request=httpx.Request("GET", "https://example.com/error"),
+ response=httpx.Response(500),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await make_base64_url_from_any_uri("https://example.com/error")
+
+ assert exc_info.value.response.status_code == 500
+
+ async def test_http_403_forbidden_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that HTTP 403 Forbidden errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message="403 Forbidden",
+ request=httpx.Request("GET", "https://example.com/forbidden.png"),
+ response=httpx.Response(403),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await make_base64_url_from_any_uri("https://example.com/forbidden.png")
+
+ assert exc_info.value.response.status_code == 403
+
+ async def test_connection_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that connection errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.ConnectError("Connection refused"),
+ )
+
+ with pytest.raises(httpx.ConnectError):
+ await make_base64_url_from_any_uri("https://unreachable.example.com/image.png")
+
+ async def test_timeout_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that timeout errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.TimeoutException("Request timed out"),
+ )
+
+ with pytest.raises(httpx.TimeoutException):
+ await make_base64_url_from_any_uri("https://slow.example.com/image.png")
+
+ async def test_dns_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that DNS resolution errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.ConnectError("DNS resolution failed"),
+ )
+
+ with pytest.raises(httpx.ConnectError):
+ await make_base64_url_from_any_uri("https://nonexistent-domain-12345.com/image.png")
+
+ async def test_pipelex_storage_raises_value_error_with_descriptive_message(self) -> None:
+ """Test that pipelex-storage:// URIs raise ValueError with helpful message."""
+ storage_uri = f"{PIPELEX_STORAGE_SCHEME}bucket/path/file.png"
+
+ with pytest.raises(ValueError, match="Unsupported URI type"):
+ await make_base64_url_from_any_uri(storage_uri)
+
+ async def test_local_file_not_found_raises_file_not_found_error(self) -> None:
+ """Test that non-existent local files raise FileNotFoundError."""
+ non_existent_path = "/path/to/non/existent/file.png"
+
+ with pytest.raises(FileNotFoundError):
+ await make_base64_url_from_any_uri(non_existent_path)
+
+ async def test_local_file_permission_denied_raises_permission_error(self, mocker: MockerFixture) -> None:
+ """Test that permission errors for local files are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.load_binary_async",
+ side_effect=PermissionError("Permission denied"),
+ )
+
+ with pytest.raises(PermissionError):
+ await make_base64_url_from_any_uri("/restricted/file.png")
+
+ async def test_ssl_certificate_error_propagates(self, mocker: MockerFixture) -> None:
+ """Test that SSL certificate errors are propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.ConnectError("SSL certificate verification failed"),
+ )
+
+ with pytest.raises(httpx.ConnectError):
+ await make_base64_url_from_any_uri("https://expired-cert.example.com/image.png")
+
+ @pytest.mark.parametrize(
+ ("topic", "status_code"),
+ [
+ ("bad_request", 400),
+ ("unauthorized", 401),
+ ("payment_required", 402),
+ ("not_found", 404),
+ ("method_not_allowed", 405),
+ ("gone", 410),
+ ("internal_server_error", 500),
+ ("bad_gateway", 502),
+ ("service_unavailable", 503),
+ ("gateway_timeout", 504),
+ ],
+ )
+ async def test_various_http_status_codes_propagate(
+ self,
+ mocker: MockerFixture,
+ topic: str,
+ status_code: int,
+ ) -> None:
+ """Test that various HTTP status codes are properly propagated."""
+ mocker.patch(
+ "pipelex.tools.misc.base64_utils.fetch_file_from_url_httpx",
+ side_effect=httpx.HTTPStatusError(
+ message=f"{status_code} {topic}",
+ request=httpx.Request("GET", f"https://example.com/{topic}"),
+ response=httpx.Response(status_code),
+ ),
+ )
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await make_base64_url_from_any_uri(f"https://example.com/{topic}")
+
+ assert exc_info.value.response.status_code == status_code