Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix/anthorpic default image urls to base64 (#263)

Thomas Hebrard43fbddcb5515
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cd4d9703e..153cec558 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,22 +3,23 @@
## Unreleased
### Added
- - Added PIPELEX_INFERENCE LLM family enum value
- - Added support for PIPELEX_INFERENCE in OpenAI LLM worker
- - Added Azure OpenAI platform support for Grok models (grok-3 and grok-3-mini)
- - Added debug logging for PipeParallel output contents
- - Added TOML file filtering in LLM model library loading
+ - Added `PIPELEX_INFERENCE` LLM family enum value
+ - Added support for `PIPELEX_INFERENCE` in OpenAI LLM worker
+ - Added Azure OpenAI platform support for Grok models (`grok-3` and `grok-3-mini`)
+ - Added debug logging for `PipeParallel` output contents
+ - Added `TOML` file filtering in LLM model library loading
- Added error handling for Unicode decode errors in LLM model library
- - Added new test model configurations for pipelex and vertex_ai platforms
+ - Added new test model configurations for `pipelex` and `vertex_ai` platforms
### Changed
- - Improved error messages in StuffFactory to include concept code and stuff name
- - Disabled gen_object_supported for all Grok models (grok-3, grok-3-mini, grok-3-fast)
+ - Improved error messages in `StuffFactory` to include concept code and stuff name
+ - Disabled `is_gen_object_supported` for all Grok models (`grok-3`, `grok-3-mini`, `grok-3-fast`)
- Updated test configurations to use different LLM models and platforms
- - Modified Jinja2 filter to use default TagStyle.TICKS instead of raising error
+ - Modified `Jinja2` filter to use default `TagStyle.TICKS` instead of raising error
- Added proper error handling for Unicode decode errors when loading model libraries
- - Improved error handling in Anthropic plugin tests with specific AuthenticationError handling
- - Updated mypy to version 1.18.1
+ - Improved error handling in Anthropic plugin tests with specific `AuthenticationError` handling
+ - Updated `mypy` to version `1.18.1`
+ - Image handling in `AnthropicFactory` now converts image URLs to `base64` data URLs with proper MIME type prefix
## [v0.9.4] - 2025-09-06
diff --git a/pipelex/plugins/anthropic/anthropic_factory.py b/pipelex/plugins/anthropic/anthropic_factory.py
index 4cd56931a..bcabb7c7b 100644
--- a/pipelex/plugins/anthropic/anthropic_factory.py
+++ b/pipelex/plugins/anthropic/anthropic_factory.py
@@ -21,14 +21,14 @@ from pipelex.cogt.image.prompt_image import (
PromptImageTypedBytesOrUrl,
PromptImageUrl,
)
+from pipelex.cogt.image.prompt_image_factory import PromptImageFactory
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_models.llm_platform import LLMPlatform
from pipelex.cogt.llm.token_category import NbTokensByCategoryDict, TokenCategory
from pipelex.config import get_config
from pipelex.hub import get_plugin_manager, get_secrets_provider
-from pipelex.tools.misc.base_64_utils import (
- load_binary_as_base64_async,
-)
+from pipelex.tools.misc.base_64_utils import load_binary_as_base64_async
+from pipelex.tools.misc.filetype_utils import detect_file_type_from_base64
class AnthropicFactoryError(CogtError):
@@ -166,7 +166,9 @@ class AnthropicFactory:
if isinstance(prompt_image, PromptImageBytes):
typed_bytes_or_url = prompt_image.make_prompt_image_typed_bytes()
elif isinstance(prompt_image, PromptImageUrl):
- typed_bytes_or_url = prompt_image.url
+ image_bytes = await PromptImageFactory().make_promptimagebytes_from_url_async(prompt_image)
+ file_type = detect_file_type_from_base64(image_bytes.base_64)
+ typed_bytes_or_url = PromptImageTypedBytes(base_64=image_bytes.base_64, file_type=file_type)
elif isinstance(prompt_image, PromptImagePath):
b64 = await load_binary_as_base64_async(prompt_image.file_path)
typed_bytes_or_url = PromptImageTypedBytes(base_64=b64, file_type=prompt_image.get_file_type())