Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix/Gemini-image-config-by-spec · L-261004-3bf897 (#1326)

Louis Choquel513f1bac8aa9
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 513f4f3c3..e1c3b47cb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,6 +20,7 @@
### Fixed
+- **An image worker refusing a model's spec reports a configuration fault**: a worker refused a model with no `rules`, no `model_choice` or `endpoint_path` where it needs one, or on the Google Gemini paths an `aspect_ratio` taxonomy that is missing, unknown to this release or of another family, with `ImgGenParameterError` in the `content` category, so a server answered 422 and told the caller to change their input; and a rule value this release does not know escaped the OpenAI, Azure, fal, Hugging Face and gateway image workers as a bare `ValueError`. Each now raises `ImgGenParameterError` in the `configuration` category, naming the model, and answers 500, while a ratio or size the model's grid refuses is still the caller's 422.
- **A bare value is refused where it reads as a concept the input does not accept (Breaking)**: an input declared as a container native (`Html`, `Page`, `TextAndImages`, `SearchResult`, `Composite`), as `Choice` or `Rating`, or as a concept refining one, built a bare string as a `native.Text` and handed that text on, so the step reading the input met a text where it declared another concept. The value is now refused with `StructureValidationError` naming the concept it read as; send it in its `{concept, content}` envelope. The concept's own content sent without that envelope, such as `{"choice": "billing"}`, is refused with the same advice, where it was told to redeclare the input as `JSON`, and so is a bare number or boolean at a `Rating` or `Choice` input, such as a level `2` sent alone, where it was told to redeclare the input as `Anything`. A `Dynamic` input still keeps whatever its value reads as.
- **A name a template sets is read where Jinja binds it (Breaking)**: the input check of `PipeLLM`, `PipeCompose`, `PipeSearch`, `PipeImgGen` and `PipeDocGen`, `PipeDocGen`'s field check and the optional-input guard now treat a `{% set %}` name as the template's own inside the loop, macro or block that sets it and after an `{% if %}` every branch of which sets it, so a template setting a name in a loop body or in each branch of an `if` and reading it there is no longer refused as reading an undeclared input. A name only some branches set is still read from the input of that name. Newly refused: an input declared only so that such a set would load, which is now never read, a macro called before its `{% macro %}`, an optional input read unguarded in a macro argument's default, and a name a loop or a macro binds read inside a `{% block %}` without `scoped`, which Jinja reads from the input of that name.
- **File inputs are checked against the models that read them (Breaking)**: run setup now identifies every image and document input from its first bytes, refuses an `Image` input holding something other than an image with `PipelineInputNotAnImageError`, and refuses a document certain to reach a `PipeExtract` or `PipeLLM` step whose model does not read its format with `PipelineInputFormatUnsupportedError`, both 422 input errors naming the input, the step, the model and the formats it reads; a step reached only through a condition refuses the file itself with `ExtractInputFormatError` or `PromptDocumentFormatError`. A `PipeLLM` document its model does not read now raises `PromptDocumentFormatError` where it raised `LLMCapabilityError`, so a consumer branching on `error_type` must follow. `docling-extract-text` now declares the Word, PowerPoint, Excel, HTML, Markdown, CSV, plain-text, WebVTT and email files it already extracted, which `pipelex update` installs.
diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.md
index dceab9a92..ff36bf5cb 100644
--- a/docs/under-the-hood/error-model.md
+++ b/docs/under-the-hood/error-model.md
@@ -222,6 +222,8 @@ own_domain = self.error_category.error_domain if self.error_category is not None
The consequence worth knowing at the HTTP boundary: a **content-classified inference failure answers 422, not 500** — a content-policy refusal, a malformed prompt image, a bad prompt parameter are all properties of material the caller submitted. Everything else keeps the status it already had; only the report became truthful about why.
+The category a class declares is its default, and a raise site that knows better passes its own through the `error_category` argument. `ImgGenParameterError` is `CONTENT`, because an aspect ratio or a size the model's grid refuses is a property of the caller's request. A model spec its worker cannot use is not: no `rules`, a rule value this release does not know, a missing `model_choice` or `endpoint_path`, or, where a worker reads the Gemini geometry, a missing `aspect_ratio` taxonomy or one of another family. Those sites raise it with `error_category=InferenceErrorCategory.CONFIGURATION`, so the failure answers 500 and lands in server-error alerting like any other configuration fault, rather than a 422 telling the caller to change what they sent.
+
#### Classified where it is raised
Most classes state whose fault they are in their body: an `error_domain`, and `_authors_caller_facing_message = True` when their message is copy written for the caller, which STRICT disclosure then keeps instead of replacing it with `An internal error occurred.`. Every instance of such a class is the same kind of fault. A few classes are raised both for faults in the caller's own method or inputs and for faults that are not the caller's, so they cannot state it. For those, the site that raises the error and knows calls `as_caller_fault()` on it: the report then carries the `input` domain, so an HTTP surface answers 422, a caller-facing message, and the next step the raise site passes, if any.
diff --git a/pipelex/cogt/img_gen/img_gen_args_factory.py b/pipelex/cogt/img_gen/img_gen_args_factory.py
index 3190d7afc..fcbd7bb40 100644
--- a/pipelex/cogt/img_gen/img_gen_args_factory.py
+++ b/pipelex/cogt/img_gen/img_gen_args_factory.py
@@ -8,10 +8,11 @@ how each parameter should be formatted for the specific provider's API.
"""
import base64
-from typing import Any, TypeAlias
+from enum import StrEnum
+from typing import Any, TypeAlias, TypeVar
from pipelex import log
-from pipelex.cogt.exceptions import ImgGenParameterError
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory
from pipelex.cogt.image.image_size import ImageSize
from pipelex.cogt.image.prompt_image import PromptImage
from pipelex.cogt.image.prompt_image_utils import prep_prompt_images
@@ -42,6 +43,8 @@ from pipelex.tools.uri.prepared_file import PreparedFileBase64, PreparedFileHttp
ImageFileTuple: TypeAlias = tuple[str, bytes, str]
"""httpx-style multipart file part: (filename, content_bytes, mime_type)."""
+TaxonomyType = TypeVar("TaxonomyType", bound=StrEnum)
+
class ImgGenArgsFactory:
"""Factory that builds provider-specific API arguments from model rules and job parameters.
@@ -50,6 +53,27 @@ class ImgGenArgsFactory:
the appropriate taxonomy handler to generate the correct API arguments for each topic.
"""
+ @classmethod
+ def rule_taxonomy(
+ cls,
+ *,
+ taxonomy_class: type[TaxonomyType],
+ topic: ImgGenArgTopic,
+ taxonomy_value: str,
+ model_name: str,
+ ) -> TaxonomyType:
+ """The rule's value read as its topic's taxonomy.
+
+ Raises:
+ ImgGenParameterError: If this release does not know the value, in the `CONFIGURATION`
+ category since the rules are the model configuration's, not the caller's.
+ """
+ try:
+ return taxonomy_class(taxonomy_value)
+ except ValueError as exc:
+ msg = f"Image model '{model_name}' has an unknown {topic} taxonomy '{taxonomy_value}'"
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION) from exc
+
@classmethod
async def make_args_for_model(
cls,
@@ -82,7 +106,9 @@ class ImgGenArgsFactory:
for topic, taxonomy_value in model_rules.items():
match topic:
case ImgGenArgTopic.PROMPT:
- prompt_taxonomy = PromptTaxonomy(taxonomy_value)
+ prompt_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=PromptTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_prompt(
prompt_taxonomy=prompt_taxonomy,
@@ -91,7 +117,9 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.NUM_IMAGES:
- num_images_taxonomy = NumImagesTaxonomy(taxonomy_value)
+ num_images_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=NumImagesTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_num_images(
num_images_taxonomy=num_images_taxonomy,
@@ -99,7 +127,9 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.ASPECT_RATIO:
- aspect_ratio_taxonomy = AspectRatioTaxonomy(taxonomy_value)
+ aspect_ratio_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=AspectRatioTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_aspect_ratio(
aspect_ratio_taxonomy=aspect_ratio_taxonomy,
@@ -109,7 +139,9 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.BACKGROUND:
- background_taxonomy = BackgroundTaxonomy(taxonomy_value)
+ background_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=BackgroundTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_background(
background_taxonomy=background_taxonomy,
@@ -118,7 +150,9 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.INFERENCE:
- inference_taxonomy = InferenceTaxonomy(taxonomy_value)
+ inference_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=InferenceTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_inference(
inference_taxonomy=inference_taxonomy,
@@ -129,7 +163,9 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.SAFETY_CHECKER:
- safety_checker_taxonomy = SafetyCheckerTaxonomy(taxonomy_value)
+ safety_checker_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=SafetyCheckerTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_safety_checker(
safety_checker_taxonomy=safety_checker_taxonomy,
@@ -138,7 +174,9 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.OUTPUT_FORMAT:
- output_format_taxonomy = OutputFormatTaxonomy(taxonomy_value)
+ output_format_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=OutputFormatTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_output_format(
output_format_taxonomy=output_format_taxonomy,
@@ -146,21 +184,27 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.OUTPUT_COMPRESSION:
- output_compression_taxonomy = OutputCompressionTaxonomy(taxonomy_value)
+ output_compression_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=OutputCompressionTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_output_compression(
output_compression_taxonomy=output_compression_taxonomy,
)
)
case ImgGenArgTopic.SPECIFIC:
- specific_taxonomy = SpecificTaxonomy(taxonomy_value)
+ specific_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=SpecificTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_specific(
specific_taxonomy=specific_taxonomy,
)
)
case ImgGenArgTopic.MODEL_CHOICE:
- model_name_taxonomy = ModelChoiceTaxonomy(taxonomy_value)
+ model_name_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=ModelChoiceTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_model_name(
model_name_taxonomy=model_name_taxonomy,
@@ -169,14 +213,18 @@ class ImgGenArgsFactory:
)
)
case ImgGenArgTopic.INPUT_IMAGES:
- input_images_taxonomy = InputImagesTaxonomy(taxonomy_value)
+ input_images_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=InputImagesTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
input_images_args = await cls.make_args_from_input_images(
input_images_taxonomy=input_images_taxonomy,
input_images=img_gen_job.img_gen_prompt.input_images,
)
args_dict.update(input_images_args)
case ImgGenArgTopic.INPUT_FIDELITY:
- input_fidelity_taxonomy = InputFidelityTaxonomy(taxonomy_value)
+ input_fidelity_taxonomy = cls.rule_taxonomy(
+ taxonomy_class=InputFidelityTaxonomy, topic=topic, taxonomy_value=taxonomy_value, model_name=model_name
+ )
args_dict.update(
cls.make_args_from_input_fidelity(
input_fidelity_taxonomy=input_fidelity_taxonomy,
diff --git a/pipelex/cogt/img_gen/img_gen_gemini_mapping.py b/pipelex/cogt/img_gen/img_gen_gemini_mapping.py
index 28edf0c7f..fdd2678c8 100644
--- a/pipelex/cogt/img_gen/img_gen_gemini_mapping.py
+++ b/pipelex/cogt/img_gen/img_gen_gemini_mapping.py
@@ -14,7 +14,7 @@ import operator
from collections.abc import Mapping
from typing import ClassVar, Literal, NamedTuple
-from pipelex.cogt.exceptions import ImgGenParameterError
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory
from pipelex.cogt.image.image_size import ImageSize
from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, SizeTier
from pipelex.cogt.img_gen.img_gen_model_rules import AspectRatioTaxonomy, ImgGenArgTopic
@@ -132,7 +132,8 @@ class ImgGenGeminiMapping:
Raises:
ImgGenParameterError: If the model has no `aspect_ratio` rules configured
- or the configured taxonomy value is unknown.
+ or the configured taxonomy value is unknown, in the `CONFIGURATION`
+ category since the rules are the model configuration's, not the caller's.
"""
rules = inference_model.rules or {}
taxonomy_value = rules.get(ImgGenArgTopic.ASPECT_RATIO)
@@ -141,12 +142,12 @@ class ImgGenGeminiMapping:
f"Google image model '{inference_model.name}' has no 'aspect_ratio' rules configured; "
f"set rules.aspect_ratio to a Gemini taxonomy (e.g. 'gemini_3_flash')"
)
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
try:
return AspectRatioTaxonomy(taxonomy_value)
except ValueError as exc:
msg = f"Google image model '{inference_model.name}' has an unknown aspect_ratio taxonomy '{taxonomy_value}'"
- raise ImgGenParameterError(msg) from exc
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION) from exc
@classmethod
def optional_img_gen_taxonomy(cls, inference_model: InferenceModelSpec) -> AspectRatioTaxonomy | None:
@@ -165,6 +166,35 @@ class ImgGenGeminiMapping:
except ValueError:
return None
+ @classmethod
+ def optional_gemini_taxonomy(cls, inference_model: InferenceModelSpec) -> AspectRatioTaxonomy | None:
+ """The model's geometry taxonomy when it is a Gemini Image one, else None.
+
+ This is the spec-level test for "is this a Gemini image model": the rules name a Gemini
+ taxonomy for exactly those models, whatever handle or model id the catalog gives them.
+ A declared non-Gemini taxonomy answers None. A spec that declares no `aspect_ratio`, or a
+ value this release does not know, raises: the second may name a newer Gemini taxonomy,
+ the first says nothing either way, and answering None would drop the requested ratio and
+ size without a word. Both are the model configuration's fault, so they raise in the
+ `CONFIGURATION` category.
+ """
+ rules = inference_model.rules or {}
+ taxonomy_value = rules.get(ImgGenArgTopic.ASPECT_RATIO)
+ if taxonomy_value is None:
+ msg = f"Image model '{inference_model.name}' declares no aspect_ratio rule, so whether it takes Google's image_config cannot be decided"
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
+ try:
+ taxonomy = AspectRatioTaxonomy(taxonomy_value)
+ except ValueError as exc:
+ msg = (
+ f"Image model '{inference_model.name}' has an unknown aspect_ratio taxonomy '{taxonomy_value}', "
+ "so whether it takes Google's image_config cannot be decided"
+ )
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION) from exc
+ if not taxonomy.is_gemini:
+ return None
+ return taxonomy
+
@classmethod
def resolve_image_config(
cls,
@@ -270,7 +300,7 @@ class ImgGenGeminiMapping:
| AspectRatioTaxonomy.QWEN_IMAGE
):
msg = f"Taxonomy '{taxonomy}' configured for model '{model_name}' is not a Google Gemini image generation taxonomy"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
@classmethod
def dimensions_for_aspect_ratio_and_size(
diff --git a/pipelex/cogt/img_gen/img_gen_model_rules.py b/pipelex/cogt/img_gen/img_gen_model_rules.py
index 7029832a7..7f119d783 100644
--- a/pipelex/cogt/img_gen/img_gen_model_rules.py
+++ b/pipelex/cogt/img_gen/img_gen_model_rules.py
@@ -104,6 +104,30 @@ class AspectRatioTaxonomy(StrEnum):
GEMINI_3_FLASH = "gemini_3_flash"
GEMINI_3_FLASH_LITE = "gemini_3_flash_lite"
+ @property
+ def is_gemini(self) -> bool:
+ """Whether this is a Google Gemini Image taxonomy, whose geometry travels as Google's `image_config`.
+
+ It identifies a Gemini image model from its spec rather than from its name, since a catalog
+ may serve one under a handle that does not start with `gemini`.
+ """
+ match self:
+ case (
+ AspectRatioTaxonomy.GEMINI_2_5
+ | AspectRatioTaxonomy.GEMINI_3_PRO
+ | AspectRatioTaxonomy.GEMINI_3_FLASH
+ | AspectRatioTaxonomy.GEMINI_3_FLASH_LITE
+ ):
+ return True
+ case (
+ AspectRatioTaxonomy.FLUX
+ | AspectRatioTaxonomy.FLUX_11_ULTRA
+ | AspectRatioTaxonomy.GPT_IMAGE_LEGACY
+ | AspectRatioTaxonomy.GPT_IMAGE_2
+ | AspectRatioTaxonomy.QWEN_IMAGE
+ ):
+ return False
+
class InferenceTaxonomy(StrEnum):
"""Taxonomy for mapping inference-related parameters (steps, quality, guidance).
diff --git a/pipelex/providers/azure_rest/azure_img_gen_worker.py b/pipelex/providers/azure_rest/azure_img_gen_worker.py
index 01d4bb6c4..128219773 100644
--- a/pipelex/providers/azure_rest/azure_img_gen_worker.py
+++ b/pipelex/providers/azure_rest/azure_img_gen_worker.py
@@ -109,7 +109,7 @@ class AzureImgGenWorker(ImgGenWorkerAbstract):
) -> list[GeneratedImageRawDetails]:
if self.inference_model.rules is None:
msg = f"Model '{self.inference_model.name}' does not have rules configured"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
args_dict = await ImgGenArgsFactory.make_args_for_model(
model_rules=self.inference_model.rules,
diff --git a/pipelex/providers/fal/fal_img_gen_worker.py b/pipelex/providers/fal/fal_img_gen_worker.py
index 5103a0e87..9789bb3d9 100644
--- a/pipelex/providers/fal/fal_img_gen_worker.py
+++ b/pipelex/providers/fal/fal_img_gen_worker.py
@@ -6,7 +6,7 @@ from fal_client.client import FalClientError, FalClientHTTPError, FalClientTimeo
from typing_extensions import override
from pipelex import log
-from pipelex.cogt.exceptions import ImgGenParameterError, SdkTypeError
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory, SdkTypeError
from pipelex.cogt.image.generated_image import GeneratedImageRawDetails
from pipelex.cogt.img_gen.img_gen_args_factory import ImgGenArgsFactory
from pipelex.cogt.img_gen.img_gen_job import ImgGenJob
@@ -42,7 +42,7 @@ class FalImgGenWorker(ImgGenWorkerAbstract):
) -> Any:
if self.inference_model.rules is None:
msg = f"Model '{self.inference_model.name}' does not have rules configured"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
args_dict = await ImgGenArgsFactory.make_args_for_model(
model_rules=self.inference_model.rules,
img_gen_job=img_gen_job,
@@ -53,7 +53,7 @@ class FalImgGenWorker(ImgGenWorkerAbstract):
fal_application = args_dict.pop("model", None)
if fal_application is None:
msg = f"Model '{self.inference_model.name}' rules must include a 'model_choice' entry"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
log.verbose(args_dict, title=f"Fal arguments, application={fal_application}")
try:
handler = await self.fal_async_client.submit(
diff --git a/pipelex/providers/gateway/gateway_img_gen_worker.py b/pipelex/providers/gateway/gateway_img_gen_worker.py
index cee70c2f6..188d0f93b 100644
--- a/pipelex/providers/gateway/gateway_img_gen_worker.py
+++ b/pipelex/providers/gateway/gateway_img_gen_worker.py
@@ -74,7 +74,7 @@ class GatewayImgGenWorker(ImgGenWorkerAbstract):
) -> list[GeneratedImageRawDetails]:
if self.inference_model.rules is None:
msg = f"Model '{self.inference_model.name}' does not have rules configured"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
args_dict = await ImgGenArgsFactory.make_args_for_model(
model_rules=self.inference_model.rules,
img_gen_job=img_gen_job,
@@ -86,7 +86,7 @@ class GatewayImgGenWorker(ImgGenWorkerAbstract):
endpoint_path = self.inference_model.endpoint_path
if not endpoint_path:
msg = f"Model '{self.inference_model.name}' does not have an endpoint_path configured but it's required for this model/sdk."
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
config_id = GatewayDeck.get_config_id(headers=self.inference_model.extra_headers or {})
image_files: list[ImageFileTuple] | None = args_dict.pop("image", None)
# Given that different backends and different models can be used with this worker, we must interpret the response,
@@ -102,7 +102,7 @@ class GatewayImgGenWorker(ImgGenWorkerAbstract):
edits_endpoint_path = endpoint_path.replace("/images/generations", "/images/edits", 1)
if edits_endpoint_path == endpoint_path:
msg = f"Could not derive an /images/edits route from endpoint path '{endpoint_path}'"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
# OpenAI's multipart convention for /images/edits (matching the openai SDK's
# extract_files serialization): a single input image is the bare 'image' field,
# but multiple images must each go under 'image[]' — repeated bare 'image' parts
diff --git a/pipelex/providers/huggingface/huggingface_img_gen_worker.py b/pipelex/providers/huggingface/huggingface_img_gen_worker.py
index 9fb7c1bfc..cde91474e 100644
--- a/pipelex/providers/huggingface/huggingface_img_gen_worker.py
+++ b/pipelex/providers/huggingface/huggingface_img_gen_worker.py
@@ -6,7 +6,7 @@ from PIL import Image
from typing_extensions import override
from pipelex import log
-from pipelex.cogt.exceptions import ImgGenParameterError, SdkTypeError
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory, SdkTypeError
from pipelex.cogt.image.generated_image import GeneratedImageRawDetails
from pipelex.cogt.img_gen.img_gen_args_factory import ImgGenArgsFactory
from pipelex.cogt.img_gen.img_gen_job import ImgGenJob
@@ -40,7 +40,7 @@ class HuggingFaceImgGenWorker(ImgGenWorkerAbstract):
) -> Image.Image:
if self.inference_model.rules is None:
msg = f"Model '{self.inference_model.name}' does not have rules configured"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
args_dict = await ImgGenArgsFactory.make_args_for_model(
model_rules=self.inference_model.rules,
img_gen_job=img_gen_job,
@@ -52,7 +52,7 @@ class HuggingFaceImgGenWorker(ImgGenWorkerAbstract):
model_id = args_dict.pop("model", None)
if model_id is None:
msg = f"Model '{self.inference_model.name}' rules must include a 'model_choice' entry"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
try:
return await self.hf_async_client.text_to_image(
prompt=prompt,
diff --git a/pipelex/providers/manifold/manifold_factory.py b/pipelex/providers/manifold/manifold_factory.py
index 2bcf4dc39..dd47bf905 100644
--- a/pipelex/providers/manifold/manifold_factory.py
+++ b/pipelex/providers/manifold/manifold_factory.py
@@ -30,6 +30,7 @@ if TYPE_CHECKING:
from portkey_ai import AsyncPortkey
from pipelex.cogt.img_gen.img_gen_job_components import ImgGenJobParams
+ from pipelex.cogt.img_gen.img_gen_model_rules import AspectRatioTaxonomy
from pipelex.cogt.inference.inference_job_abstract import InferenceJobAbstract
from pipelex.cogt.model_backends.backend import InferenceBackend
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
@@ -105,25 +106,18 @@ class ManifoldFactory:
return bool(backend.extra_config.get("debug", False))
@classmethod
- def _make_gemini_image_config(cls, inference_model: InferenceModelSpec, *, job_params: ImgGenJobParams) -> dict[str, str]:
+ def _make_gemini_image_config(cls, taxonomy: AspectRatioTaxonomy, *, job_params: ImgGenJobParams, model_name: str) -> dict[str, str]:
"""The gemini `image_config` block, honouring the portable size.
- Same resolution as the native Google worker: a requested size (tier or exact) goes through
- the model's taxonomy rules, an unset size omits `image_size` so the provider applies its own
- default, and a model with no usable taxonomy falls back to the ratio alone rather than
- guessing.
+ Same resolution as the native Google worker: the requested ratio and size (tier or exact)
+ are checked against the model's Gemini taxonomy, and an unset size omits `image_size` so the
+ provider applies its own default.
"""
- if job_params.size is None:
- taxonomy = ImgGenGeminiMapping.optional_img_gen_taxonomy(inference_model)
- if taxonomy is None:
- return {"aspect_ratio": ImgGenGeminiMapping.aspect_ratio_literal(job_params.aspect_ratio)}
- else:
- taxonomy = ImgGenGeminiMapping.img_gen_taxonomy(inference_model)
resolved = ImgGenGeminiMapping.resolve_image_config(
taxonomy,
aspect_ratio=job_params.aspect_ratio,
size=job_params.size,
- model_name=inference_model.name,
+ model_name=model_name,
)
image_config: dict[str, str] = {"aspect_ratio": resolved.aspect_ratio}
if resolved.image_size is not None:
@@ -155,10 +149,22 @@ class ManifoldFactory:
# After the catalog's headers, so a catalog entry cannot restate whose call this is.
extra_headers.update(make_manifold_metadata_headers(job_metadata=inference_job.job_metadata))
- if isinstance(inference_job, LLMJob) and inference_model.model_id.lower().startswith("mistral-") and inference_job.job_params.seed is None:
- # Mistral models really want a non-null seed.
- extra_body["seed"] = random.randint(0, 1000000)
- elif isinstance(inference_job, ImgGenJob) and inference_model.model_id.startswith("gemini"):
- extra_body["image_config"] = cls._make_gemini_image_config(inference_model, job_params=inference_job.job_params)
+ if isinstance(inference_job, LLMJob):
+ # Mistral models really want a non-null seed. The spec carries no Mistral-specific field
+ # to key this on, so the model id's prefix decides, and it holds whether the catalog
+ # gives the provider's id or leaves the id to default to a `mistral-` handle.
+ if inference_model.model_id.lower().startswith("mistral-") and inference_job.job_params.seed is None:
+ extra_body["seed"] = random.randint(0, 1000000)
+ elif isinstance(inference_job, ImgGenJob):
+ # Decided by the spec, not the name: the catalog may serve a Gemini image model under a
+ # handle such as `nano-banana`, and the model id defaults to the handle when the catalog
+ # gives none. A Gemini taxonomy in the image rules is what makes it one, and a spec whose
+ # rules cannot say is refused rather than read as non-Gemini.
+ if (gemini_taxonomy := ImgGenGeminiMapping.optional_gemini_taxonomy(inference_model)) is not None:
+ extra_body["image_config"] = cls._make_gemini_image_config(
+ gemini_taxonomy,
+ job_params=inference_job.job_params,
+ model_name=inference_model.name,
+ )
return extra_headers, extra_body
diff --git a/pipelex/providers/manifold/manifold_img_gen_worker.py b/pipelex/providers/manifold/manifold_img_gen_worker.py
index dd7c484de..dcda5483c 100644
--- a/pipelex/providers/manifold/manifold_img_gen_worker.py
+++ b/pipelex/providers/manifold/manifold_img_gen_worker.py
@@ -87,7 +87,7 @@ class ManifoldImgGenWorker(ImgGenWorkerAbstract):
) -> list[GeneratedImageRawDetails]:
if self.inference_model.rules is None:
msg = f"Model '{self.inference_model.name}' does not have rules configured"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
args_dict = await ImgGenArgsFactory.make_args_for_model(
model_rules=self.inference_model.rules,
img_gen_job=img_gen_job,
diff --git a/pipelex/providers/openai/openai_img_gen_worker.py b/pipelex/providers/openai/openai_img_gen_worker.py
index 12e279317..a09e835d8 100644
--- a/pipelex/providers/openai/openai_img_gen_worker.py
+++ b/pipelex/providers/openai/openai_img_gen_worker.py
@@ -60,7 +60,7 @@ class OpenAIImgGenWorker(ImgGenWorkerAbstract):
) -> list[GeneratedImageRawDetails]:
if self.inference_model.rules is None:
msg = f"Model '{self.inference_model.name}' does not have rules configured"
- raise ImgGenParameterError(msg)
+ raise ImgGenParameterError(msg, error_category=InferenceErrorCategory.CONFIGURATION)
args_dict = await ImgGenArgsFactory.make_args_for_model(
model_rules=self.inference_model.rules,
diff --git a/subject_grants.toml b/subject_grants.toml
index 2f3bab7e8..5dcf8911d 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -1772,6 +1772,10 @@ rationale = "Derives the image size for the size tier; single operand."
param = "inference_model"
rationale = "Reads the aspect-ratio taxonomy off the inference model; single operand."
+["pipelex/cogt/img_gen/img_gen_gemini_mapping.py::ImgGenGeminiMapping.optional_gemini_taxonomy"]
+param = "inference_model"
+rationale = "Reads the Gemini aspect-ratio taxonomy off the inference model, None when it declares no Gemini one; single operand."
+
["pipelex/cogt/img_gen/img_gen_gemini_mapping.py::ImgGenGeminiMapping.optional_img_gen_taxonomy"]
param = "inference_model"
rationale = "Reads the aspect-ratio taxonomy off the inference model, None when it declares none; single operand."
@@ -5273,8 +5277,8 @@ param = "sdk"
rationale = "Predicate on the sdk; single operand."
["pipelex/providers/manifold/manifold_factory.py::ManifoldFactory._make_gemini_image_config"]
-param = "inference_model"
-rationale = "Factory from source: builds the result from the inference_model; single operand."
+param = "taxonomy"
+rationale = "Builds the image_config block from the Gemini aspect-ratio taxonomy; the job params and model name stay keyword-only."
["pipelex/providers/manifold/manifold_factory.py::ManifoldFactory.get_api_key"]
param = "backend"
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_gemini_mapping.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_gemini_mapping.py
index 2e7a7c56b..8b34bef35 100644
--- a/tests/unit/pipelex/cogt/img_gen/test_img_gen_gemini_mapping.py
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_gemini_mapping.py
@@ -1,6 +1,10 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
import pytest
-from pipelex.cogt.exceptions import ImgGenParameterError
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory
from pipelex.cogt.image.image_size import ImageSize
from pipelex.cogt.img_gen.img_gen_gemini_mapping import (
GeminiAspectRatioType,
@@ -9,6 +13,10 @@ from pipelex.cogt.img_gen.img_gen_gemini_mapping import (
)
from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, SizeTier
from pipelex.cogt.img_gen.img_gen_model_rules import AspectRatioTaxonomy
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
BANNER_ASPECT_RATIOS = [
AspectRatio.LANDSCAPE_4_1,
@@ -200,14 +208,65 @@ class TestImgGenGeminiMapping:
)
def test_non_google_taxonomy_rejected(self) -> None:
- """A non-Gemini taxonomy value has no Google resolution grids."""
- with pytest.raises(ImgGenParameterError, match="not a Google Gemini image generation taxonomy"):
+ """A non-Gemini taxonomy value has no Google resolution grids, which is the model configuration's fault."""
+ with pytest.raises(ImgGenParameterError, match="not a Google Gemini image generation taxonomy") as exc_info:
ImgGenGeminiMapping.dimensions_for_aspect_ratio_and_size(
AspectRatioTaxonomy.FLUX,
aspect_ratio=AspectRatio.SQUARE,
size="1K",
model_name="flux-dev",
)
+ assert exc_info.value.error_category == InferenceErrorCategory.CONFIGURATION
+
+ @pytest.mark.parametrize(
+ ("topic", "method_name", "rules", "error_match"),
+ [
+ ("required_without_rules", "img_gen_taxonomy", None, "no 'aspect_ratio' rules configured"),
+ ("required_unknown_value", "img_gen_taxonomy", {"aspect_ratio": "gemini_from_the_future"}, "unknown aspect_ratio taxonomy"),
+ ("gemini_without_rules", "optional_gemini_taxonomy", None, "declares no aspect_ratio rule"),
+ ("gemini_without_aspect_ratio", "optional_gemini_taxonomy", {"prompt": "positive_only"}, "declares no aspect_ratio rule"),
+ ("gemini_unknown_value", "optional_gemini_taxonomy", {"aspect_ratio": "gemini_from_the_future"}, "unknown aspect_ratio taxonomy"),
+ ],
+ )
+ def test_unusable_taxonomy_rules_are_a_configuration_fault(
+ self,
+ mocker: MockerFixture,
+ topic: str, # ruff: ignore[unused-method-argument]
+ method_name: str,
+ rules: dict[str, str] | None,
+ error_match: str,
+ ) -> None:
+ """Rules that cannot name the model's taxonomy are refused as the model configuration's fault, naming the model."""
+ inference_model = mocker.MagicMock(spec=InferenceModelSpec)
+ inference_model.name = "nano-banana-misconfigured"
+ inference_model.rules = rules
+
+ with pytest.raises(ImgGenParameterError, match=error_match) as exc_info:
+ getattr(ImgGenGeminiMapping, method_name)(inference_model)
+
+ assert exc_info.value.error_category == InferenceErrorCategory.CONFIGURATION
+ assert "nano-banana-misconfigured" in str(exc_info.value)
+
+ @pytest.mark.parametrize(
+ ("rules", "expected_taxonomy"),
+ [
+ ({"aspect_ratio": "gemini_3_pro"}, AspectRatioTaxonomy.GEMINI_3_PRO),
+ ({"aspect_ratio": "gpt_image_2"}, None),
+ ({"aspect_ratio": "flux"}, None),
+ ],
+ )
+ def test_optional_gemini_taxonomy_reads_the_declared_taxonomy(
+ self,
+ mocker: MockerFixture,
+ rules: dict[str, str],
+ expected_taxonomy: AspectRatioTaxonomy | None,
+ ) -> None:
+ """A declared Gemini taxonomy is returned and a declared non-Gemini one answers None."""
+ inference_model = mocker.MagicMock(spec=InferenceModelSpec)
+ inference_model.name = "some-image-model"
+ inference_model.rules = rules
+
+ assert ImgGenGeminiMapping.optional_gemini_taxonomy(inference_model) == expected_taxonomy
@pytest.mark.parametrize(
("taxonomy", "exact_size", "expected_ratio", "expected_image_size"),
@@ -316,14 +375,15 @@ class TestImgGenGeminiMapping:
assert resolved == ("16:9", "2K", 2752, 1536)
def test_resolve_image_config_rejects_tier_beyond_taxonomy(self) -> None:
- """A tier outside the taxonomy's grids is a validation error, not a silent downgrade."""
- with pytest.raises(ImgGenParameterError, match="does not support image size"):
+ """A tier outside the taxonomy's grids is a validation error of the caller's request, not a silent downgrade."""
+ with pytest.raises(ImgGenParameterError, match="does not support image size") as exc_info:
ImgGenGeminiMapping.resolve_image_config(
AspectRatioTaxonomy.GEMINI_2_5,
aspect_ratio=AspectRatio.SQUARE,
size=SizeTier.TWO_K,
model_name="nano-banana",
)
+ assert exc_info.value.error_category == InferenceErrorCategory.CONTENT
def test_resolve_image_config_rejects_half_k(self) -> None:
with pytest.raises(ImgGenParameterError, match=r"0\.5k"):
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_worker_config_faults.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_worker_config_faults.py
new file mode 100644
index 000000000..082c1a467
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_worker_config_faults.py
@@ -0,0 +1,140 @@
+"""An image worker refuses a model spec it cannot use as a configuration fault, not as the caller's input.
+
+A spec with no `rules`, a rule value this release does not know, or one missing the `model_choice` or
+`endpoint_path` its worker needs, is the backend configuration's fault: the job fails before any provider call with an `ImgGenParameterError`
+in the `CONFIGURATION` category, which the error model renders as a 500 rather than the 422 that
+tells a caller to change what they sent.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from fal_client import AsyncClient as FalAsyncClient
+from huggingface_hub import AsyncInferenceClient
+from openai import AsyncOpenAI
+from portkey_ai import AsyncPortkey
+
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory
+from pipelex.cogt.img_gen.img_gen_model_rules import ImgGenArgTopic
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.plugins.model_handle import ModelHandle
+from pipelex.providers.azure_rest.azure_img_gen_worker import AzureImgGenWorker
+from pipelex.providers.fal.fal_img_gen_worker import FalImgGenWorker
+from pipelex.providers.gateway.gateway_img_gen_worker import GatewayImgGenWorker
+from pipelex.providers.huggingface.huggingface_img_gen_worker import HuggingFaceImgGenWorker
+from pipelex.providers.manifold.manifold_img_gen_worker import ManifoldImgGenWorker
+from pipelex.providers.openai.openai_img_gen_worker import OpenAIImgGenWorker
+from tests.unit.pipelex.cogt.img_gen.conftest import make_img_gen_job
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+ from pipelex.cogt.img_gen.img_gen_model_rules import ImgGenModelRules
+ from pipelex.cogt.img_gen.img_gen_worker_abstract import ImgGenWorkerAbstract
+
+# Rules that every worker's args factory accepts, and that name neither a model choice nor a geometry.
+PROMPT_ONLY_RULES: ImgGenModelRules = {ImgGenArgTopic.PROMPT: "positive_only"}
+# Rules naming an aspect_ratio taxonomy no release knows, as a catalog newer than this release could.
+UNKNOWN_ASPECT_RATIO_RULES: ImgGenModelRules = {ImgGenArgTopic.ASPECT_RATIO: "gemini_9_turbo"}
+# Rules naming a prompt taxonomy no release knows, so the refusal is not particular to geometry.
+UNKNOWN_PROMPT_RULES: ImgGenModelRules = {ImgGenArgTopic.PROMPT: "weird"}
+
+
+def _model(*, backend_name: str, sdk: str, rules: ImgGenModelRules | None) -> InferenceModelSpec:
+ return InferenceModelSpec(
+ backend_name=backend_name,
+ name="misconfigured-image-model",
+ sdk=sdk,
+ model_type=ModelType.IMG_GEN,
+ model_id="misconfigured-image-model",
+ inputs=["text"],
+ outputs=["image"],
+ costs={},
+ thinking_mode=ThinkingMode.NONE,
+ max_tokens=None,
+ max_prompt_images=None,
+ rules=rules,
+ )
+
+
+def _make_worker(mocker: MockerFixture, *, worker_kind: str, rules: ImgGenModelRules | None) -> ImgGenWorkerAbstract:
+ """The worker, built around an SDK stand-in it accepts, for a model with these rules and no endpoint path."""
+ match worker_kind:
+ case "openai":
+ return OpenAIImgGenWorker(
+ sdk_instance=mocker.MagicMock(spec=AsyncOpenAI), inference_model=_model(backend_name="openai", sdk="openai_img_gen", rules=rules)
+ )
+ case "gateway":
+ return GatewayImgGenWorker(
+ sdk_instance=mocker.MagicMock(spec=AsyncPortkey),
+ inference_model=_model(backend_name="pipelex_gateway", sdk="gateway_img_gen", rules=rules),
+ )
+ case "manifold":
+ return ManifoldImgGenWorker(
+ sdk_instance=mocker.MagicMock(spec=AsyncPortkey),
+ inference_model=_model(backend_name="pipelex_manifold", sdk="manifold_img_gen", rules=rules),
+ )
+ case "fal":
+ return FalImgGenWorker(
+ sdk_instance=mocker.MagicMock(spec=FalAsyncClient), inference_model=_model(backend_name="fal", sdk="fal", rules=rules)
+ )
+ case "huggingface":
+ return HuggingFaceImgGenWorker(
+ sdk_instance=mocker.MagicMock(spec=AsyncInferenceClient),
+ inference_model=_model(backend_name="huggingface", sdk="huggingface_img_gen", rules=rules),
+ )
+ case "azure":
+ backend: Any = mocker.MagicMock(endpoint="https://azure.example", extra_config={"api_version": "2025-04-01-preview"}, api_key="test-key")
+ models_manager = mocker.MagicMock()
+ models_manager.get_required_inference_backend.return_value = backend
+ mocker.patch("pipelex.providers.azure_rest.azure_img_gen_worker.get_models_manager", return_value=models_manager)
+ return AzureImgGenWorker(
+ model_handle=ModelHandle(sdk="azure_rest_img_gen", backend="azure_openai"),
+ inference_model=_model(backend_name="azure_openai", sdk="azure_rest_img_gen", rules=rules),
+ )
+ case _:
+ msg = f"Unknown worker kind '{worker_kind}'"
+ raise ValueError(msg)
+
+
+class TestImgGenWorkerConfigFaults:
+ @pytest.mark.asyncio
+ @pytest.mark.parametrize(
+ ("worker_kind", "rules", "error_match"),
+ [
+ ("openai", None, "does not have rules configured"),
+ ("gateway", None, "does not have rules configured"),
+ ("manifold", None, "does not have rules configured"),
+ ("fal", None, "does not have rules configured"),
+ ("huggingface", None, "does not have rules configured"),
+ ("azure", None, "does not have rules configured"),
+ ("fal", PROMPT_ONLY_RULES, "must include a 'model_choice' entry"),
+ ("huggingface", PROMPT_ONLY_RULES, "must include a 'model_choice' entry"),
+ ("gateway", PROMPT_ONLY_RULES, "does not have an endpoint_path configured"),
+ ("openai", UNKNOWN_ASPECT_RATIO_RULES, "unknown aspect_ratio taxonomy 'gemini_9_turbo'"),
+ ("manifold", UNKNOWN_ASPECT_RATIO_RULES, "unknown aspect_ratio taxonomy 'gemini_9_turbo'"),
+ ("azure", UNKNOWN_ASPECT_RATIO_RULES, "unknown aspect_ratio taxonomy 'gemini_9_turbo'"),
+ ("openai", UNKNOWN_PROMPT_RULES, "unknown prompt taxonomy 'weird'"),
+ ("fal", UNKNOWN_PROMPT_RULES, "unknown prompt taxonomy 'weird'"),
+ ("huggingface", UNKNOWN_PROMPT_RULES, "unknown prompt taxonomy 'weird'"),
+ ],
+ )
+ async def test_unusable_model_spec_is_a_configuration_fault(
+ self,
+ mocker: MockerFixture,
+ worker_kind: str,
+ rules: ImgGenModelRules | None,
+ error_match: str,
+ ) -> None:
+ """The job fails before any provider call, in the CONFIGURATION category, naming the model."""
+ worker = _make_worker(mocker, worker_kind=worker_kind, rules=rules)
+
+ with pytest.raises(ImgGenParameterError, match=error_match) as exc_info:
+ await worker.gen_image(img_gen_job=make_img_gen_job())
+
+ assert exc_info.value.error_category == InferenceErrorCategory.CONFIGURATION
+ assert "misconfigured-image-model" in str(exc_info.value)
diff --git a/tests/unit/pipelex/providers/manifold/test_data.py b/tests/unit/pipelex/providers/manifold/test_data.py
index 777e68871..ab65dc482 100644
--- a/tests/unit/pipelex/providers/manifold/test_data.py
+++ b/tests/unit/pipelex/providers/manifold/test_data.py
@@ -1,7 +1,9 @@
-"""Job metadata the manifold tests hand to a request, and the header value it must produce."""
+"""Job metadata the manifold tests hand to a request, the header value it must produce, and catalog entries with the extras they yield."""
-from typing import ClassVar
+from typing import Any, ClassVar
+from pipelex.cogt.image.image_size import ImageSize
+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, SizeTier
from pipelex.system.job_metadata import JobMetadata, RunMetadata
@@ -31,3 +33,102 @@ class ManifoldMetadataTestData:
'{"org_id":"org_acme","api_key_id":"key_123","user_id":"user_42","pipeline_run_id":"run_abc",'
'"pipe_run_id":"0123456789abcdef","pipe_code":"summarize_doc","content_generation_job_id":"cgj_1"}'
)
+
+
+class ManifoldExtrasBySpecTestData:
+ """Manifold catalog entries, written as the tables a catalog holds, and the body extras each must produce.
+
+ The Gemini image entries carry no `model_id`, as the hosted catalog serves them: the spec's
+ model id then defaults to the handle, which does not start with `gemini`, so only the spec's
+ image rules can say that `image_config` is due. Each case is (handle, catalog entry, aspect
+ ratio, size, expected `image_config`).
+ """
+
+ GEMINI_IMAGE_CASES: ClassVar[list[tuple[str, dict[str, Any], AspectRatio, SizeTier | ImageSize | None, dict[str, str]]]] = [
+ (
+ "nano-banana",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "rules": {"aspect_ratio": "gemini_2_5"}},
+ AspectRatio.LANDSCAPE_16_9,
+ None,
+ {"aspect_ratio": "16:9"},
+ ),
+ (
+ "nano-banana-pro",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "rules": {"aspect_ratio": "gemini_3_pro"}},
+ AspectRatio.PORTRAIT_9_16,
+ SizeTier.TWO_K,
+ {"aspect_ratio": "9:16", "image_size": "2K"},
+ ),
+ (
+ "nano-banana-2",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "rules": {"aspect_ratio": "gemini_3_flash"}},
+ AspectRatio.LANDSCAPE_4_1,
+ SizeTier.FOUR_K,
+ {"aspect_ratio": "4:1", "image_size": "4K"},
+ ),
+ (
+ "nano-banana-2-lite",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "rules": {"aspect_ratio": "gemini_3_flash_lite"}},
+ AspectRatio.SQUARE,
+ SizeTier.ONE_K,
+ {"aspect_ratio": "1:1", "image_size": "1K"},
+ ),
+ (
+ "nano-banana",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "model_id": "gemini-2.5-flash-image", "rules": {"aspect_ratio": "gemini_2_5"}},
+ AspectRatio.PORTRAIT_3_4,
+ None,
+ {"aspect_ratio": "3:4"},
+ ),
+ ]
+
+ # (topic, handle, catalog entry): image models whose rules name a non-Gemini taxonomy get no `image_config`.
+ NON_GEMINI_IMAGE_CASES: ClassVar[list[tuple[str, str, dict[str, Any]]]] = [
+ (
+ "gpt_image_legacy",
+ "gpt-image-1",
+ {"model_type": "img_gen", "sdk": "manifold_img_gen", "rules": {"model_choice": "model_name", "aspect_ratio": "gpt_image_legacy"}},
+ ),
+ (
+ "gpt_image_2",
+ "gpt-image-2",
+ {"model_type": "img_gen", "sdk": "manifold_img_gen", "rules": {"model_choice": "model_name", "aspect_ratio": "gpt_image_2"}},
+ ),
+ ]
+
+ # (topic, handle, catalog entry, error match): an image spec that cannot say whether it is a Gemini one is refused rather
+ # than read as non-Gemini, since that would drop the requested ratio and size without a word. An `aspect_ratio` value
+ # this release does not know may name a newer Gemini taxonomy, and a spec declaring no `aspect_ratio` says nothing.
+ UNDECIDABLE_IMAGE_CASES: ClassVar[list[tuple[str, str, dict[str, Any], str]]] = [
+ (
+ "unknown_taxonomy",
+ "nano-banana-next",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "rules": {"aspect_ratio": "gemini_from_the_future"}},
+ "unknown aspect_ratio taxonomy 'gemini_from_the_future'",
+ ),
+ (
+ "gemini_id_without_rules",
+ "gemini-lookalike",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "model_id": "gemini-2.5-flash-image"},
+ "declares no aspect_ratio rule",
+ ),
+ (
+ "rules_without_aspect_ratio",
+ "nano-banana-bare",
+ {"model_type": "img_gen", "sdk": "manifold_completions", "rules": {"prompt": "positive_only"}},
+ "declares no aspect_ratio rule",
+ ),
+ ]
+
+ # (topic, handle, catalog entry, whether the job gets a seed): the Mistral seed still keys on the model id's prefix,
+ # which the one manifold Mistral handle carries whether or not the entry names the provider's id.
+ MISTRAL_SEED_CASES: ClassVar[list[tuple[str, str, dict[str, Any], bool]]] = [
+ ("mistral_handle_only", "mistral-large", {"sdk": "manifold_completions", "structure_method": "instructor/mistral_tools"}, True),
+ (
+ "mistral_provider_id",
+ "mistral-large",
+ {"sdk": "manifold_completions", "model_id": "Mistral-Large-3", "structure_method": "instructor/mistral_tools"},
+ True,
+ ),
+ ("not_mistral", "gpt-4o", {"sdk": "manifold_responses"}, False),
+ ]
diff --git a/tests/unit/pipelex/providers/manifold/test_manifold_extras_by_spec.py b/tests/unit/pipelex/providers/manifold/test_manifold_extras_by_spec.py
new file mode 100644
index 000000000..5fbc3fd78
--- /dev/null
+++ b/tests/unit/pipelex/providers/manifold/test_manifold_extras_by_spec.py
@@ -0,0 +1,143 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.cogt.exceptions import ImgGenParameterError, InferenceErrorCategory
+from pipelex.cogt.img_gen.img_gen_job import ImgGenJob
+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenJobConfig, ImgGenJobParams, ImgGenJobReport, SizeTier
+from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt
+from pipelex.cogt.llm.llm_job import LLMJob
+from pipelex.cogt.llm.llm_job_components import LLMJobConfig, LLMJobParams, LLMJobReport
+from pipelex.cogt.llm.llm_prompt import LLMPrompt
+from pipelex.cogt.model_backends.backend import PipelexBackend
+from pipelex.cogt.model_backends.model_spec_factory import InferenceModelSpecBlueprint, InferenceModelSpecFactory
+from pipelex.providers.manifold.manifold_factory import ManifoldFactory
+from tests.unit.pipelex.providers.manifold.test_data import ManifoldExtrasBySpecTestData, ManifoldMetadataTestData
+
+if TYPE_CHECKING:
+ from pipelex.cogt.image.image_size import ImageSize
+ from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+
+
+def _spec(*, handle: str, entry: dict[str, Any]) -> InferenceModelSpec:
+ """The spec the manifold backend would build from this catalog table, through the real factory."""
+ return InferenceModelSpecFactory.make_inference_model_spec(
+ PipelexBackend.MANIFOLD,
+ name=handle,
+ blueprint=InferenceModelSpecBlueprint.model_validate(entry),
+ backend_listed_constraints=[],
+ backend_valued_constraints={},
+ )
+
+
+def _img_gen_job(*, aspect_ratio: AspectRatio, size: SizeTier | ImageSize | None) -> ImgGenJob:
+ return ImgGenJob(
+ job_metadata=ManifoldMetadataTestData.JOB_METADATA,
+ img_gen_prompt=ImgGenPrompt(positive_text="a lighthouse at dusk"),
+ job_params=ImgGenJobParams(aspect_ratio=aspect_ratio, size=size, background=Background.AUTO),
+ job_config=ImgGenJobConfig(is_sync_mode=True),
+ job_report=ImgGenJobReport(),
+ )
+
+
+def _llm_job() -> LLMJob:
+ return LLMJob(
+ job_metadata=ManifoldMetadataTestData.JOB_METADATA,
+ llm_prompt=LLMPrompt(user_text="ping"),
+ job_params=LLMJobParams(temperature=0.5),
+ job_config=LLMJobConfig(schema_reask_max_attempts=1),
+ job_report=LLMJobReport(),
+ )
+
+
+class TestManifoldExtrasBySpec:
+ @pytest.mark.parametrize(
+ ("handle", "entry", "aspect_ratio", "size", "expected_image_config"),
+ ManifoldExtrasBySpecTestData.GEMINI_IMAGE_CASES,
+ )
+ def test_gemini_image_spec_gets_image_config(
+ self,
+ handle: str,
+ entry: dict[str, Any],
+ aspect_ratio: AspectRatio,
+ size: SizeTier | ImageSize | None,
+ expected_image_config: dict[str, str],
+ ) -> None:
+ """A Gemini taxonomy in the rules sends `image_config`, whether the model id is the provider's or defaults to the handle."""
+ spec = _spec(handle=handle, entry=entry)
+ assert spec.model_id == entry.get("model_id", handle)
+
+ _, extra_body = ManifoldFactory.make_extras(spec, inference_job=_img_gen_job(aspect_ratio=aspect_ratio, size=size), output_desc="Image")
+
+ assert extra_body == {"image_config": expected_image_config}
+
+ @pytest.mark.parametrize(
+ ("topic", "handle", "entry"),
+ ManifoldExtrasBySpecTestData.NON_GEMINI_IMAGE_CASES,
+ )
+ def test_non_gemini_image_spec_gets_no_image_config(
+ self,
+ topic: str, # ruff: ignore[unused-method-argument]
+ handle: str,
+ entry: dict[str, Any],
+ ) -> None:
+ """A spec whose rules name a non-Gemini taxonomy gets no `image_config`."""
+ spec = _spec(handle=handle, entry=entry)
+
+ _, extra_body = ManifoldFactory.make_extras(
+ spec, inference_job=_img_gen_job(aspect_ratio=AspectRatio.SQUARE, size=SizeTier.ONE_K), output_desc="Image"
+ )
+
+ assert extra_body == {}
+
+ @pytest.mark.parametrize("size", [None, SizeTier.ONE_K])
+ @pytest.mark.parametrize(
+ ("topic", "handle", "entry", "error_match"),
+ ManifoldExtrasBySpecTestData.UNDECIDABLE_IMAGE_CASES,
+ )
+ def test_undecidable_image_spec_is_refused_as_a_configuration_fault(
+ self,
+ topic: str, # ruff: ignore[unused-method-argument]
+ handle: str,
+ entry: dict[str, Any],
+ error_match: str,
+ size: SizeTier | None,
+ ) -> None:
+ """A spec that cannot say whether it is a Gemini one fails the job instead of dropping the ratio and size silently, as the catalog's fault."""
+ spec = _spec(handle=handle, entry=entry)
+
+ with pytest.raises(ImgGenParameterError, match=error_match) as exc_info:
+ ManifoldFactory.make_extras(spec, inference_job=_img_gen_job(aspect_ratio=AspectRatio.SQUARE, size=size), output_desc="Image")
+
+ assert exc_info.value.error_category == InferenceErrorCategory.CONFIGURATION
+
+ def test_size_beyond_the_gemini_taxonomy_is_refused(self) -> None:
+ """A handle-only Gemini spec still checks the request against its taxonomy's grids."""
+ spec = _spec(handle="nano-banana", entry=ManifoldExtrasBySpecTestData.GEMINI_IMAGE_CASES[0][1])
+
+ with pytest.raises(ImgGenParameterError, match="does not support image size '2K'"):
+ ManifoldFactory.make_extras(spec, inference_job=_img_gen_job(aspect_ratio=AspectRatio.SQUARE, size=SizeTier.TWO_K), output_desc="Image")
+
+ @pytest.mark.parametrize(
+ ("topic", "handle", "entry", "expects_seed"),
+ ManifoldExtrasBySpecTestData.MISTRAL_SEED_CASES,
+ )
+ def test_mistral_seed_follows_the_model_id_prefix(
+ self,
+ topic: str, # ruff: ignore[unused-method-argument]
+ handle: str,
+ entry: dict[str, Any],
+ expects_seed: bool,
+ ) -> None:
+ """The Mistral seed is sent for the manifold Mistral handle with or without a provider model id, and only for it."""
+ spec = _spec(handle=handle, entry=entry)
+
+ _, extra_body = ManifoldFactory.make_extras(spec, inference_job=_llm_job(), output_desc="text")
+
+ if expects_seed:
+ assert set(extra_body) == {"seed"}
+ assert 0 <= extra_body["seed"] <= 1000000
+ else:
+ assert extra_body == {}