Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Model-fault-detectors · L-261015-20208d (#1446)
Louis Choquel8df2ff9930dc
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6b8e1df14..0f1d884d1 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -21,7 +21,7 @@- **`PipeIOContractError` is the method's (Breaking)**: a structure that cannot be projected into its IO contract answers a 422 carrying `method.concept_invalid`, where it answered a 500.- **Storage and remote-file failures say whose they are**: an S3 or GCS refusal of the deployment's credentials (`AccessDenied`, an invalid or expired key, `Forbidden`) carries `pipelex.config.credential_refused`, a bucket that does not exist `pipelex.config.invalid` and missing credentials `pipelex.config.credential_missing`, apart from an outage, `pipelex.storage.unavailable`; a remote file that times out, refuses the connection or answers a 5xx, 408, 425 or 429 is `input.url_unreachable`, which a retry may cure, and any other failed fetch `input.file_unreadable`.- **Builtins raised where a reader sees them are coded classes**: refusals that raised a bare `ValueError`, `RuntimeError`, `ImportError` or `NotImplementedError` raise a coded `PipelexError`, such as `RunParameterError` for a run's metadata, extras or storage scope (still a `ValueError`), `ExportedPipeNotFoundError` and `ExportedPipeAmbiguousError` for a CLI pipe selector, `BundleClosureEmptyError`, `StorageNotConfiguredError`, `UrlSyntaxError`, `LocalFileMissingError` and `UnsupportedImageMimeTypeError`, and the classes that were rooted outside `PipelexError` (`PipeValidationError`, `ConceptValueError`, `QualifiedRefError`, `SemVerError`, `MethodNotFoundError` and the like) join it.-- **Inference failures carry the code their classification picks (Breaking)**: the classification of a provider's failure picks a code with its category, a `ServiceErrorCode` a plugin contributes names its `error_code`, and a category that repeats the call within the run must pick a code that invites a retry; an error built with a category alone reports the category's code (`provider.unavailable` for `TRANSIENT`, `runtime.misconfigured` for `CONFIGURATION`, `provider.quota_exhausted` for `CAPACITY`, `provider.outcome_unknown` for `AMBIGUOUS`). A provider 400 that carries no content-policy signal is `runtime.unexpected`, answered 500, where it was the caller's 422, and a local extractor's `OSError` is `runtime.unexpected`, no longer repeated within the run.+- **Inference failures carry the code their classification picks (Breaking)**: the classification of a provider's failure picks a code with its category, a `ServiceErrorCode` a plugin contributes names its `error_code`, and a category that repeats the call within the run must pick a code that invites a retry; an error built with a category alone reports the category's code (`provider.unavailable` for `TRANSIENT`, `runtime.misconfigured` for `CONFIGURATION`, `provider.quota_exhausted` for `CAPACITY`, `provider.outcome_unknown` for `AMBIGUOUS`). A provider 400 that carries no context-window or content-policy signal is `runtime.unexpected`, answered 500, where it was the caller's 422, and a local extractor's `OSError` is `runtime.unexpected`, no longer repeated within the run.- **A truncated completion says which limit cut it**: `LLMCompletionTruncatedError` carries `model.output_truncated` when the method's `max_tokens` cut the answer and `model.context_exceeded` when the context window did, both the method's to change.- **`ModelDeckPresetValidationError` is spelt right (Breaking)**: the error a model deck preset that fails validation raises was `ModelDeckPresetValidatonError`, and its reports carried that `error_type`; a consumer that branched on the old name branches on the new one, or on its code, `pipelex.config.invalid`.- **`pipelex-agent` reads `retryable` from the code (Breaking)**: the agent CLI's error envelope takes `retryable` from the report's code, and its table of retryable error types is gone, as are the domain-table entries of classes that now carry a code. The envelope names that code as `error_code`, the member a hosted refusal already carried, so an agent branches on the code whichever runner failed.@@ -35,6 +35,9 @@- **A template error found only at render is a located refusal**: a template naming a filter or a test that does not exist, which Jinja2 only finds when the branch renders, fails the run at the pipe with `Jinja2TemplateInvalidError`, naming the unknown filter or test and none of the template, coded `method.template_invalid` (a 422 STRICT disclosure keeps) when the template is the method's, in a `PipeCondition` expression, a `PipeLLM` prompt or system prompt and a `PipeDocGen` template, and `runtime.unexpected` when it is one of Pipelex's own, such as the deployment's structure prompt rendered in a `PipeLLM` step, where it failed as an unclassified 500. A filter refusing its argument, which may be a value of the run's data, is `pipe.template_render_failed` rather than the template's fault.- **A provider's 408 is retried**: a provider answering HTTP 408, that it gave up waiting, is a transient failure carrying `provider.timeout`, so the call is repeated within the run, a billed image call included, where it was read as the caller's input and the run failed on its first attempt.- **A `PipeFunc` returning a list of anything but stuff contents fails its output**: a function returning a list that holds a value other than a stuff content, such as a dict or a number, fails the step with `FuncOutputTypeError` (`pipe.output_invalid`), where a dict became an empty content and any other value failed as though the function had raised.+- **A prompt over the model's context window reads `model.context_exceeded`**: a provider 400, or another 4xx such as the 422 or 413 a gateway may answer, saying the inputs are over what the model can read, by OpenAI's `context_length_exceeded` code or by the wording OpenAI, Anthropic, Mistral, Bedrock and Gemini give it, in the error's message or its body, is the caller's `model.context_exceeded`, answered 422, where it read `runtime.unexpected`, or `input.too_large` on a 413. A language model's window holds the prompt and the output requested with `max_tokens` together, and a 400 the `max_tokens` sent caused reads the same, so its next step is to shorten the input or lower `max_tokens`, or choose a model with a larger context window; an image, extraction, search or judgment model's next step names what it reads instead. A 400 about a malformed request still reads `runtime.unexpected`.+- **A structured output cut at `max_tokens` reads `model.output_truncated`**: the Anthropic, OpenAI Chat Completions and Mistral workers turn instructor's `IncompleteOutputException`, raised for a structured answer cut at its output limit, into `LLMCompletionTruncatedError`, naming a structured output, the stop value, the output tokens and the `max_tokens` sent, where it escaped unclassified as `runtime.unexpected`. `LLMCompletionTruncatedError` takes an `output_kind`, a text or a structured output. When the Anthropic structured-output timeout lowered the limit that cut it, the error says so and advises a lower reasoning effort or a smaller output, never a higher `max_tokens`.+- **A Gemini image a safety filter blocked reads `model.content_refused`**: the Google image worker reads the answer's finish reason, or the prompt's block reason when no candidate came back, and raises `ImgGenGenerationError` with `model.content_refused`, answered 422 with a next step to revise the prompt or its images, where an `IMAGE_SAFETY` or `PROHIBITED_CONTENT` answer read as an empty one, `provider.response_invalid`. The message names the reason Gemini gave and what it says, so an image stopped for recitation or for an unsupported language reads the same code without being called a safety block. An answer the model's output token limit stopped before it held an image (`MAX_TOKENS`) raises the new `ImgGenOutputTruncatedError`, `model.output_truncated`, answered 422 with a next step to simplify the prompt or choose another image model, where it read `provider.response_invalid`.- **A file-type refusal no longer quotes the file**: `FileTypeError`'s message gives the number of bytes it read, where it quoted up to 300 bytes of the file's content.- **Pre-release documentation is published**: a push to a `pre-release/` branch deploys its documentation under the `pre-release` alias of docs.pipelex.com, where the deploy workflow called a make target with no recipe and published nothing.diff --git a/docs/building-methods/pipes/pipe-operators/PipeImgGen.md b/docs/building-methods/pipes/pipe-operators/PipeImgGen.mdindex c6f423177..766a703e3 100644--- a/docs/building-methods/pipes/pipe-operators/PipeImgGen.md+++ b/docs/building-methods/pipes/pipe-operators/PipeImgGen.md@@ -67,6 +67,8 @@ Common image generation model handles:Image generation presets are defined in your model deck configuration and can include parameters like `quality`, `guidance_scale`, and `safety_tolerance`.+A prompt or an input image a model's safety filter blocks fails the pipe with the code `model.content_refused`, naming the model and the reason the provider gave, so revising the prompt or its images is the fix. On a Gemini model such as `nano-banana-2`, that reason is the finish reason of the answer (`IMAGE_SAFETY`, `IMAGE_PROHIBITED_CONTENT` and their kin) or the block reason of the prompt, since Gemini answers a refusal with no image rather than with an error. A Gemini answer stopped because the image may reproduce an existing source (`RECITATION`, `IMAGE_RECITATION`) or because the prompt's language is not supported (`LANGUAGE`) reads the same code, and its message says which reason Gemini gave rather than blaming a safety filter. A Gemini answer whose output token limit stopped it before it held an image (`MAX_TOKENS`) fails the pipe with the code `model.output_truncated` instead, and its next step is a simpler prompt or another image model, since the pipe sets no output limit for an image.+### MTHDS Parameters| Parameter | Type | Description | Required |diff --git a/docs/building-methods/pipes/pipe-operators/PipeLLM.md b/docs/building-methods/pipes/pipe-operators/PipeLLM.mdindex 48030f214..a091708bf 100644--- a/docs/building-methods/pipes/pipe-operators/PipeLLM.md+++ b/docs/building-methods/pipes/pipe-operators/PipeLLM.md@@ -257,7 +257,7 @@ When `model` is specified as a table (inline LLM setting), it accepts the followA setting the model it resolves to cannot take is refused when the method loads, before a run spends anything: validation runs the check the model's worker runs before every call, so a reasoning setting on a model that does not reason, a reasoning budget on a model taking only an effort, or a budget its `max_tokens` cannot hold is an `llm_setting_refused_by_model` error on the `model` or `model_to_structure` field holding the setting. The setting checked is the one the output is generated with: `model` for a single text, the structuring setting for anything else.-A text the model cannot finish within `max_tokens`, or within its context window, fails the pipe with `LLMCompletionTruncatedError` rather than passing the partial text on as the pipe's output; a text the model refuses or a safety filter stops fails it with `LLMCompletionRefusedError`. The first names the output tokens used and the limit sent, so raising `max_tokens` or shortening the input is the fix; a model that reasons spends its reasoning from the same limit.+A text the model cannot finish within `max_tokens`, or within its context window, fails the pipe with `LLMCompletionTruncatedError` rather than passing the partial text on as the pipe's output, and so does a structured output cut at `max_tokens`, which the error names as a structured output; a text the model refuses or a safety filter stops fails it with `LLMCompletionRefusedError`. The first names the output tokens used and the limit sent, so raising `max_tokens` or shortening the input is the fix; a model that reasons spends its reasoning from the same limit. On an Anthropic model, a structured output's `max_tokens` is held to what the structured-output timeout allows, so when that lowered limit cut it the error says so and its fix is a lower reasoning effort or a smaller output, never a higher `max_tokens`. A request the provider refuses outright because the prompt plus the output it requests with `max_tokens` exceed the model's context window fails with the code `model.context_exceeded`: shortening the input or lowering `max_tokens`, which may be the model's default when the pipe sets none, or choosing a model with a larger context window, fixes it.A `prompt` or `system_prompt` that names a filter or a test Jinja2 does not know inside a conditional block loads, because Jinja2 resolves such a name only when the block renders. The run then fails at the pipe, before any model is called, with the code `method.template_invalid`: the message names the field and the unknown name, quotes none of the prompt, and comes with a next step, and a hosted run answers it with HTTP 422.diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.mdindex 14004eac0..ff1ff9b17 100644--- a/docs/under-the-hood/error-model.md+++ b/docs/under-the-hood/error-model.md@@ -273,7 +273,7 @@ The three steps live in three modules. Only the per-provider Extract functions s| Module | Step | What it owns ||--------|------|--------------|-| `pipelex/cogt/inference/error_classification.py` | Extract | `ProviderErrorMetadata`, `SDKErrorEnvelope`, `UserAction`, `UserActionKind`, the `extract_*_metadata` functions and the parsers they share (`parse_retry_after_seconds`, `error_code_from_body_code_first`), plus pure discriminators (`is_quota_exhaustion`, `is_content_policy_violation`, `is_network_error`, `is_model_not_allowed`) exposed as `@property` on the metadata |+| `pipelex/cogt/inference/error_classification.py` | Extract | `ProviderErrorMetadata`, `SDKErrorEnvelope`, `UserAction`, `UserActionKind`, the `extract_*_metadata` functions and the parsers they share (`parse_retry_after_seconds`, `error_code_from_body_code_first`), plus pure discriminators (`is_quota_exhaustion`, `is_content_policy_violation`, `is_context_window_exceeded`, `is_network_error`, `is_model_not_allowed`) exposed as `@property` on the metadata || `pipelex/cogt/inference/error_classify.py` | Classify | `classify_inference_error()` — provider-blind mapping from `ProviderErrorMetadata` → `ClassificationResult(category, user_action_kind, is_model_not_found, service_error_code, is_model_not_allowed)`, consulting the booted plugins' service error vocabulary ahead of the status ladder || `pipelex/cogt/inference/error_render.py` | Render | `render_inference_error()` — picks the `CogtError` subclass from `InferenceErrorFamily` plus `is_model_not_found` (e.g. `LLMModelNotFoundError` vs `LLMCompletionError`) |@@ -321,6 +321,8 @@ One refusal outside any service's own namespace is classified by the runtime itsOn structured-generation paths, `instructor` raises an `InstructorRetryException` from the exception that ended its retry loop. The loop re-asks only a response that fails the schema, so that exception is either the raw SDK exception of a call that failed in transport, or, once the re-ask budget is spent, the last parse failure. `extract_underlying_sdk_exception()` recovers it from the wrapper's cause, not from `failed_attempts`, which lists only parse failures and would name an earlier one when a re-ask then fails in transport. The recovered exception routes through the same per-provider categorization as the plain-text path. Once the re-ask budget is spent, the recovered exception is the last `pydantic.ValidationError`, which `_STATUSLESS_BY_TYPE_NAME` classifies as `CONTENT` / `CHANGE_INPUT` with `model.output_schema_mismatch`, so it answers 422. A recovered exception the classifier does not recognize lands in `UNKNOWN`.+One exception escapes the loop unwrapped. `instructor` reads the stop value of every structured answer it parses, and raises its `IncompleteOutputException` for one cut at the request's output limit: an OpenAI-shaped completion, Mistral's included, whose first choice finished on `length`, or an Anthropic message that stopped on `max_tokens`. It never re-asks such an answer, and it raises the exception as it is, carrying the cut answer as its `last_completion`, never inside an `InstructorRetryException`. The workers whose structuring mode can raise it, the Anthropic, OpenAI Chat Completions and Mistral workers, catch it and raise `LLMCompletionTruncatedError` from it through `LLMWorkerAbstract._incomplete_output_error`, which reads the answer's stop value and output tokens with `read_incomplete_output` (in `pipelex/cogt/llm/completion_stop.py`). A structured output cut at its limit therefore reads `model.output_truncated`, as a text cut at the same limit does, and names the same stop value, output tokens and `max_tokens`, the error's `output_kind` saying a structured output was cut rather than a text. When the Anthropic worker's structured-output timeout lowered the limit, the worker passes a `MaxTokensCeiling` naming the limit it was given and the timeout, and the error says the limit was lowered and to what, then gives as its next step, in its message and its `user_action`, a lower reasoning effort or a smaller output, since a higher `max_tokens` would be lowered the same way. The Gemini and Responses API structuring modes never raise it.+### Model and Provider AttributionInference-failure leaf errors (`LLMCompletionError`, `ImgGenGenerationError`, …) are raised deep inside a plugin and do not know which model handle invoked them. Each worker family fills that in at its public-method chokepoint:@@ -578,12 +580,16 @@ InferenceErrorCategory.TRANSIENT.is_retryable # True — only TRANSIENT| The deck's own default or override names a model it does not define | `ModelDeckChoiceUnresolvedError`, naming the deck setting: `pipelex.config.invalid`, the deployment's → HTTP 500, redacted, never the caller's `model.unknown`; a bad override is refused when the deck is built || Content-policy violation | `CONTENT` → not repeated; `UserAction(CHANGE_INPUT)`; `model.content_refused` → **HTTP 422** || Malformed prompt image / bad prompt parameter | `CONTENT` class-level (`PromptImageFormatError`, `LLMPromptParameterError`, …), each with its own code (`input.file_type_unsupported`, `pipe.prompt_empty`, …) |-| Any other provider **HTTP 400** | `CONTENT` / `CHANGE_INPUT`, but `runtime.unexpected` → HTTP 500: a 400 nothing recognises is most likely a request of ours the provider could not take, and only a content-policy signal makes it the caller's `model.content_refused` |+| A provider **HTTP 400**, or another 4xx such as the 422 or 413 a gateway may answer, saying the inputs are over what the model can read (OpenAI's `context_length_exceeded` code, which Azure OpenAI and OpenAI-compatible gateways relay, or a provider's own wording, read in the rendered message and in the body, where a raw `httpx` error such as Azure REST's keeps it: OpenAI's "maximum context length" and "exceeds the context window", Anthropic's "prompt is too long" and "exceed context limit", Bedrock's "Input is too long for requested model", Gemini's "exceeds the maximum number of tokens allowed") | `ProviderErrorMetadata.is_context_window_exceeded` → `CONTENT` / `CHANGE_INPUT`, not repeated; `model.context_exceeded`, the caller's → **HTTP 422**. The 400 arm and the other-4xx arm read it, ahead of the content-policy signal, for every family, and a 413 that carries no such signal keeps its payload-size reading, `input.too_large`. A language model's window holds the prompt and the output the request reserves with `max_tokens` together, and Anthropic's "exceed context limit" and OpenAI's "maximum context length" also refuse a request whose `max_tokens`, which may be the deck's default, pushes it over, so the language-model next step names both: shorten the input or lower `max_tokens`, or choose a model with a larger context window. The other families' next steps fit what their model reads: the prompt and its input images for an image model, the document for an extraction model, the inputs for a search or judgment model. Each phrase names the window or the inputs' length, so a 400 about a malformed request matches none of them |+| Any other provider **HTTP 400** | `CONTENT` / `CHANGE_INPUT`, but `runtime.unexpected` → HTTP 500: a 400 nothing recognises is most likely a request of ours the provider could not take, and only a context-window or a content-policy signal makes it the caller's, `model.context_exceeded` or `model.content_refused` || Local file extractor raises a builtin (docling, pypdfium2) | `FileNotFoundError` → `input.reference_not_found`, `ValueError` / `RuntimeError` → `input.file_unreadable`, both `CONTENT` → **HTTP 422**; `OSError` → `UNKNOWN`, `runtime.unexpected` (see `_LOCAL_EXTRACT_BY_TYPE_NAME`) || A text completion stops at its output limit or context window (`length`, `max_tokens`, `MAX_TOKENS`, an `incomplete` Responses answer at `max_output_tokens`, `model_context_window_exceeded`, `model_length`), with partial or empty text | Every LLM worker reads the stop before returning text and raises `LLMCompletionTruncatedError` → `CONTENT` / `CHANGE_INPUT`, not repeated: `model.output_truncated` at the output limit, which the method's `max_tokens` sets, and `model.context_exceeded` at the context window, both the method's → **HTTP 422**, caller-facing under STRICT. The message names the model, the pipe, the stop value, the output tokens used and the `max_tokens` sent, never the partial text. The error's `truncation_limit` tells the output limit from the context window (`model_context_window_exceeded`, `model_length`), and so does its next step: a higher `max_tokens` or a shorter input for the first, a shorter input, a lower reasoning effort or a larger model for the second, never a higher `max_tokens`. A gateway in non-strict mode passing each provider's own value through the completions shape is read the same way, since the classifier reads the union of the provider vocabularies || A text completion refused or stopped by a safety filter (`content_filter`, `refusal`, `content_filtered`, `guardrail_intervened`, Gemini's `SAFETY` and its kin), or a Gemini prompt blocked before any candidate (its `prompt_feedback.block_reason`, whatever its value) | `LLMCompletionRefusedError` → `CONTENT` / `CHANGE_INPUT`, not repeated; `model.content_refused` → **HTTP 422**, caller-facing under STRICT |+| A structured output cut at its output limit (`instructor`'s `IncompleteOutputException`, for an OpenAI or Mistral first choice finished on `length` or an Anthropic message stopped on `max_tokens`) | The Anthropic, OpenAI Chat Completions and Mistral workers raise `LLMCompletionTruncatedError` from it, at the output limit: `model.output_truncated` → **HTTP 422**, caller-facing under STRICT, with the same stop value, output tokens and `max_tokens` a cut text names (see [The `instructor` Unwrap](#the-instructor-unwrap)) |+| A Gemini image generation refused or stopped by a safety filter: a candidate whose `finish_reason` names a filter (`IMAGE_SAFETY`, `IMAGE_PROHIBITED_CONTENT`, `SAFETY` and their kin), or no candidate and a `prompt_feedback.block_reason` | The Google image worker reads the reason before it looks for an image, through `GoogleFactory.read_stop_signal`, which the Google text worker shares: the finish reason from Gemini's vocabulary with `classify_image_finish_reason`, and any block reason as a refusal. It raises `ImgGenGenerationError` → `CONTENT` / `CHANGE_INPUT`, not repeated; `model.content_refused` → **HTTP 422**, caller-facing under STRICT, naming the model's handle, the reason and what it says, so a recitation (`RECITATION`, `IMAGE_RECITATION`) or an unsupported language (`LANGUAGE`) is never called a safety filter's block. A candidate that holds no image for another reason (`NO_IMAGE`, `IMAGE_OTHER`) is still an unusable answer, `provider.response_invalid`, and a finish reason Gemini adds later is logged at warning level as an image generation's and taken as normal |+| A Gemini image generation its output token limit cut before the image: a candidate whose `finish_reason` is `MAX_TOKENS` and that holds no image | The Google image worker reads it with the refusals, before it looks for an image, and raises `ImgGenOutputTruncatedError`, a subclass of `ImgGenGenerationError` → `CONTENT` / `CHANGE_INPUT`, not repeated; `model.output_truncated` → **HTTP 422**, caller-facing under STRICT, naming the model's handle and the finish reason. The worker sends no output limit of its own, so the next step is a simpler prompt or another image model. An image the model finished before the limit is whole, and returned || A stop value no provider vocabulary lists | Logged at warning level with the model and the value, and taken as normal: the text is returned. An `incomplete` Responses answer whose reason is missing or unknown is the exception: logged at warning level and taken as a truncation, since the status alone says the text is unfinished |-| An Anthropic structured generation whose `max_tokens` its structured-output timeout lowers | Sent with the lowered limit; logged at warning level when the pipe set the limit, at debug level when it is the model's default; an error the lowered limit can explain, a structured output that failed validation or came back incomplete once the re-asks are spent, or a timeout, keeps its class and category and ends with a sentence saying the limit was lowered and to what; a rate limit, an authentication failure, an overload, a server error or a lost connection carries no such sentence |+| An Anthropic structured generation whose `max_tokens` its structured-output timeout lowers | Sent with the lowered limit; logged at warning level when the pipe set the limit, at debug level when it is the model's default; an error the lowered limit can explain, a structured output that failed validation once the re-asks are spent, or a timeout, keeps its class and category and ends with a sentence saying the limit was lowered and to what; a structured output cut at the lowered limit is a truncation that says it was lowered and to what, and whose next step, in its message and its `user_action`, is a lower reasoning effort or a smaller output, never a higher `max_tokens`; a rate limit, an authentication failure, an overload, a server error or a lost connection carries no such sentence || LLM returns schema-mismatched JSON | `instructor` re-asks; if exhausted → the last `pydantic.ValidationError` → `CONTENT` / `CHANGE_INPUT`, `model.output_schema_mismatch` → **HTTP 422** || The gateway does not allow the model for this account (`model_not_allowed_error`, HTTP 412) | `CONFIGURATION` / `CHANGE_MODEL`, never repeated; the advice names the model handle; `model.unavailable` → HTTP 500 || Connection dropped mid-request | `AMBIGUOUS` → not repeated (outcome unknown); `provider.outcome_unknown`, whose next step says to check the account, since the lost call may have been billed |diff --git a/pipelex/cogt/exceptions.py b/pipelex/cogt/exceptions.pyindex 582cce9be..8edfe6402 100644--- a/pipelex/cogt/exceptions.py+++ b/pipelex/cogt/exceptions.py@@ -1,7 +1,7 @@from __future__ import annotationsfrom enum import StrEnum-from typing import TYPE_CHECKING, ClassVar+from typing import TYPE_CHECKING, ClassVar, NamedTuplefrom mthds.protocol.problem.registry import CoreCodefrom typing_extensions import override@@ -536,7 +536,8 @@ class LLMCompletionError(CogtError):The Render step constructs it with the code its classification picked, and a worker raising it directly choosesone: ``provider.response_invalid`` for an answer with no usable text, ``runtime.unexpected`` for a structured- generation that failed with no cause to read. A worker wrapping a coded error in it takes that error's code.+ generation that failed with no cause to read. A worker wrapping a coded error in it takes that error's code. A+ text or a structured output cut short or refused is one of its subclasses, which carries its own code."""error_code: ClassVar[ErrorCodeDeclaration] = _inference_failure_declaration(fallback=CoreCode.RUNTIME_UNEXPECTED)@@ -551,20 +552,48 @@ class CompletionTruncationLimit(StrEnum):CONTEXT_WINDOW = "context_window"+class CompletionOutputKind(StrEnum):+ """What a completion was writing when it stopped, which its error names: a text, or a structured output."""++ TEXT = "text"+ STRUCTURED = "structured"++ @property+ def desc(self) -> str:+ """The words an error names this output by."""+ match self:+ case CompletionOutputKind.TEXT:+ return "text"+ case CompletionOutputKind.STRUCTURED:+ return "structured output"+++class MaxTokensCeiling(NamedTuple):+ """A max_tokens a worker sent below the one it was given, to fit its call's timeout, so a higher one is lowered too."""++ # The max_tokens the call was given: the pipe's setting, or the model's default+ requested_max_tokens: int+ # The call's timeout, which the max_tokens sent fits+ timeout_seconds: int++class LLMCompletionTruncatedError(LLMCompletionError):- """A text completion stopped before the model finished it: it hit its output limit, or its context window.+ """A completion stopped before the model finished it: it hit its output limit, or its context window.A worker raises it when its provider's stop signal says the text was cut (see- `pipelex.cogt.llm.completion_stop`), instead of handing back the partial or empty text as a result. It- is not retryable: the same request stops at the same limit. Its code follows the limit hit:- ``model.output_truncated`` at the output limit, the method's token cap, and ``model.context_exceeded`` at the- context window, which the inputs filled. Which remedy applies depends on the limit hit, carried as- `truncation_limit`: at the output limit, a higher `max_tokens` on the pipe or a shorter input, or, when the- request sent no `max_tokens` and the provider's default cut the text, a `max_tokens` set on the pipe up to- the model's limit, a lower reasoning effort or a shorter input; at the context window, which the input and- the output filled together, a shorter input, less thinking or a model with a larger window, never a higher- `max_tokens`. The message names only the pipe, the model's deck handle, the provider's stop value and the- token counts, never the partial text or the provider's body, so it is kept verbatim for the caller.+ `pipelex.cogt.llm.completion_stop`), instead of handing back the partial or empty text as a result, and when+ instructor reports a structured output cut at its output limit with its `IncompleteOutputException`; its+ `output_kind` says which of the two was cut. It is not retryable: the same request stops at the same limit.+ Its code follows the limit hit: ``model.output_truncated`` at the output limit, the method's token cap, and+ ``model.context_exceeded`` at the context window, which the inputs filled. Which remedy applies depends on the+ limit hit, carried as `truncation_limit`: at the output limit, a higher `max_tokens` on the pipe or a shorter+ input, or, when the request sent no `max_tokens` and the provider's default cut the text, a `max_tokens` set on+ the pipe up to the model's limit, a lower reasoning effort or a shorter input, or, when the worker sent a+ `max_tokens` lowered to fit its call's timeout (`max_tokens_ceiling`), a lower reasoning effort or a smaller+ output, since a higher `max_tokens` is lowered the same way; at the context window, which the input and the+ output filled together, a shorter input, less thinking or a model with a larger window, never a higher+ `max_tokens`. The message names only the pipe, the model's deck handle, the provider's stop value and the token+ counts, never the partial text or the provider's body, so it is kept verbatim for the caller."""error_code = ErrorCodeSet(default=CoreCode.MODEL_OUTPUT_TRUNCATED, others=(CoreCode.MODEL_CONTEXT_EXCEEDED,))@@ -577,11 +606,28 @@ class LLMCompletionTruncatedError(LLMCompletionError):model_handle: str,stop_reason: str,truncation_limit: CompletionTruncationLimit,+ output_kind: CompletionOutputKind,pipe_code: str | None = None,max_tokens: int | None = None,output_tokens: int | None = None,+ max_tokens_ceiling: MaxTokensCeiling | None = None,):- subject = f"the text of pipe '{pipe_code}'" if pipe_code else "its text"+ """Build the error, its message and its next step from the facts of the cut.++ Args:+ model_handle: The model's deck handle.+ stop_reason: The provider's stop value.+ truncation_limit: The limit the completion hit.+ output_kind: Whether a text or a structured output was cut.+ pipe_code: The pipe whose output it was, None when the job names none.+ max_tokens: The output limit the request sent, None when it sent none.+ output_tokens: The output tokens the provider counted, None when it gave no count.+ max_tokens_ceiling: Set when the worker sent a lower max_tokens than it was given, to fit its call's+ timeout: a higher max_tokens is then lowered the same way, so the next step names what else lifts the+ limit. Read at the output limit only.+ """+ output_desc = output_kind.desc+ subject = f"the {output_desc} of pipe '{pipe_code}'" if pipe_code else f"its {output_desc}"details = [f"stop reason '{stop_reason}'"]if output_tokens is not None:details.append(f"{output_tokens} output tokens used")@@ -589,10 +635,23 @@ class LLMCompletionTruncatedError(LLMCompletionError):details.append(f"max_tokens set to {max_tokens}")what_happened: strnext_step: str+ # A fact the next step rests on, said between the cut and the next step+ lowered_fact: str | None = Nonematch truncation_limit:case CompletionTruncationLimit.MAX_TOKENS:what_happened = "was cut off"- if max_tokens is not None:+ if max_tokens is not None and max_tokens_ceiling is not None:+ # The worker held the limit below the one it was given, so a higher one would be held the same way+ lowered_fact = (+ f"Its max_tokens was lowered from {max_tokens_ceiling.requested_max_tokens} to {max_tokens} "+ f"to fit the call's {max_tokens_ceiling.timeout_seconds}-second timeout."+ )+ next_step = (+ "Lower the pipe's reasoning effort, or ask it for a smaller output, since a higher max_tokens is lowered the same way."+ if pipe_code+ else "Lower the reasoning effort, or ask for a smaller output, since a higher max_tokens is lowered the same way."+ )+ elif max_tokens is not None:next_step = "Raise the pipe's max_tokens, or shorten its input." if pipe_code else "Raise max_tokens, or shorten the input."else:# The request left the limit to the provider's default, so there is no max_tokens of the pipe's to raise@@ -609,10 +668,13 @@ class LLMCompletionTruncatedError(LLMCompletionError):if pipe_codeelse "Shorten the input, lower the reasoning effort, or choose a model with a larger context window.")- message = (- f"The model '{model_handle}' {what_happened} before it finished {subject} ({', '.join(details)}), so the text is incomplete. {next_step}"- )- super().__init__(message, user_action=UserAction(kind=UserActionKind.CHANGE_INPUT, detail=next_step))+ sentences = [+ f"The model '{model_handle}' {what_happened} before it finished {subject} ({', '.join(details)}), so the {output_desc} is incomplete."+ ]+ if lowered_fact is not None:+ sentences.append(lowered_fact)+ sentences.append(next_step)+ super().__init__(" ".join(sentences), user_action=UserAction(kind=UserActionKind.CHANGE_INPUT, detail=next_step))match truncation_limit:case CompletionTruncationLimit.MAX_TOKENS:self.with_error_code(CoreCode.MODEL_OUTPUT_TRUNCATED)@@ -621,9 +683,11 @@ class LLMCompletionTruncatedError(LLMCompletionError):self.model_handle = model_handleself.stop_reason = stop_reasonself.truncation_limit = truncation_limit+ self.output_kind = output_kindself.pipe_code = pipe_codeself.max_tokens = max_tokensself.output_tokens = output_tokens+ self.max_tokens_ceiling = max_tokens_ceilingclass LLMCompletionRefusedError(LLMCompletionError):@@ -737,12 +801,38 @@ class ImgGenGenerationError(CogtError):"""An image generation that failed: a provider's failure the classification read, or an answer that cannot be used.The Render step constructs it with the code its classification picked, and a worker raising it directly chooses- one: ``provider.response_invalid`` for an answer with no usable image, ``provider.outcome_unknown`` for a call- that failed after it was sent, which may have generated, and billed, the image. With no code chosen and no coded- cause, it is ``provider.outcome_unknown``, since an image generation is a call no one may blindly repeat.+ one: ``provider.response_invalid`` for an answer with no usable image, ``model.content_refused`` for an answer+ whose finish or block reason says the provider refused the request on policy grounds, ``provider.outcome_unknown``+ for a call that failed after it was sent, which may have generated, and billed, the image. With no code chosen and+ no coded cause, it is ``provider.outcome_unknown``, since an image generation is a call no one may blindly repeat.+ """++ error_code: ClassVar[ErrorCodeDeclaration] = _inference_failure_declaration(fallback=CoreCode.PROVIDER_OUTCOME_UNKNOWN)+++class ImgGenOutputTruncatedError(ImgGenGenerationError):+ """An image generation the model's output token limit cut before it produced the image.++ A worker raises it when its provider's answer, a success all the same, says its output stopped at that limit and+ holds no image (a Gemini candidate whose finish reason is `MAX_TOKENS`), instead of reading it as an empty answer.+ It is ``model.output_truncated``, and not retryable: the same request stops at the same limit. The worker sends no+ output limit of its own, so the limit is the model's, which the image and the model's thinking share, and the+ next step is a simpler prompt or another model, never a setting of the pipe's. The message names only the model's+ deck handle and the provider's stop value, so it is kept verbatim for the caller."""- error_code = _inference_failure_declaration(fallback=CoreCode.PROVIDER_OUTCOME_UNKNOWN)+ error_code = CoreCode.MODEL_OUTPUT_TRUNCATED+ error_category = InferenceErrorCategory.CONTENT+ _authors_caller_facing_message = True++ def __init__(self, *, model_handle: str, stop_reason: str):+ next_step = "Simplify the image prompt, or choose another image model."+ message = (+ f"The model '{model_handle}' reached its output token limit before it generated the image (finish reason '{stop_reason}'). {next_step}"+ )+ super().__init__(message, user_action=UserAction(kind=UserActionKind.CHANGE_INPUT, detail=next_step))+ self.model_handle = model_handle+ self.stop_reason = stop_reasonclass ExtractCapabilityError(CogtError):diff --git a/pipelex/cogt/inference/error_classification.py b/pipelex/cogt/inference/error_classification.pyindex b6e3aedd2..e1dba9ed1 100644--- a/pipelex/cogt/inference/error_classification.py+++ b/pipelex/cogt/inference/error_classification.py@@ -161,6 +161,29 @@ class ProviderErrorMetadata(BaseModel):return Truereturn self.body is not None and _is_content_policy_violation(_stringify_for_scan(self.body))+ @property+ def is_context_window_exceeded(self) -> bool:+ """Whether the provider refused the request because its inputs are over what the model can read.++ For a language model, that is its context window, which holds the prompt and the output the request reserves+ with ``max_tokens`` together, so a refusal the ``max_tokens`` sent caused reads the same. Reads the structured+ error code first, ``provider_error_code`` and the code an error body carries+ (OpenAI puts ``context_length_exceeded`` in ``code`` beside a generic ``type``, which the Extract step+ prefers), then the wording each provider gives such a refusal, in its rendered message or in the in-process+ ``body``, as the content-policy check reads them: a raw ``httpx`` error, such as Azure REST's, renders only+ its status and URL, and keeps the provider's words in the body alone. The classifier reads it on a 400 and on+ any other 4xx, which a gateway may answer such a refusal with, and the Render step words its next step from+ the code the classifier picks, so a phrase in a 5xx, or quoted in a failure that reached no status, says+ nothing.+ """+ if self.provider_error_code in _CONTEXT_WINDOW_ERROR_CODES:+ return True+ if error_code_from_body_code_first(self.body) in _CONTEXT_WINDOW_ERROR_CODES:+ return True+ if _names_the_context_window(text=self.message):+ return True+ return self.body is not None and _names_the_context_window(text=_stringify_for_scan(self.body))+@propertydef is_network_error(self) -> bool:"""Whether this is a network/transport failure that never reached an HTTP status."""@@ -227,6 +250,35 @@ _CONTENT_POLICY_PATTERNS: tuple[str, ...] = ("blocked by safety",)+# The error code a provider gives a refusal of a prompt over the model's context window: OpenAI's, which Azure OpenAI+# and the OpenAI-compatible gateways relay.+_CONTEXT_WINDOW_ERROR_CODES: frozenset[str] = frozenset({"context_length_exceeded"})++# How each provider words that refusal in the message its SDK renders, every one a 400. Each phrase names the+# context window or the prompt's length, so a 400 about a malformed request matches none of them. A language model's+# window also holds the output the request reserves with `max_tokens`, so two of them also match a 400 the+# `max_tokens` sent caused, which the deck's default may be: Anthropic's "exceed context limit" and OpenAI's "maximum+# context length", which counts the completion beside the messages. The Render step's advice for the language-model+# family therefore names both the input and `max_tokens`.+_CONTEXT_WINDOW_PATTERNS: tuple[str, ...] = (+ # OpenAI's code, which its SDK renders into the message with the error body+ "context_length_exceeded",+ # OpenAI's earlier wording, also Mistral's and that of OpenAI-compatible servers such as vLLM, for the messages+ # alone or for the messages and the completion together+ "maximum context length",+ # OpenAI's current wording+ "exceeds the context window",+ # Anthropic's, directly or through Bedrock+ "prompt is too long",+ # Anthropic's when the prompt and the max_tokens requested together pass the window+ "exceed context limit",+ # Bedrock's ValidationException, in full: a 400 about one field's length, such as a tool description's, may say+ # "input is too long" of that field alone+ "input is too long for requested model",+ # Gemini's INVALID_ARGUMENT+ "exceeds the maximum number of tokens allowed",+)+_GOOGLE_QUOTA_PATTERNS: tuple[str, ...] = ("quota exceeded","resource has been exhausted",@@ -325,6 +377,12 @@ def _is_content_policy_violation(error_message: str) -> bool:return any(pattern in lower_message for pattern in _CONTENT_POLICY_PATTERNS)+def _names_the_context_window(*, text: str) -> bool:+ """Check if an error's text words a refusal of inputs over the model's context window, as some provider words it."""+ lower_text = text.lower()+ return any(pattern in lower_text for pattern in _CONTEXT_WINDOW_PATTERNS)++def _stringify_for_scan(body: Any) -> str:"""Render a metadata body into a lowercase string for content-policy / quota probing.diff --git a/pipelex/cogt/inference/error_classify.py b/pipelex/cogt/inference/error_classify.pyindex 7bf8f80de..e92144bc2 100644--- a/pipelex/cogt/inference/error_classify.py+++ b/pipelex/cogt/inference/error_classify.py@@ -247,6 +247,22 @@ def _classify_server_error(*, status_code: int, family: InferenceErrorFamily) ->)+def _bad_request_error_code(*, metadata: SDKErrorEnvelope) -> ErrorCode:+ """The code of a provider 400, which says what the provider refused when it says it, and is ours otherwise.++ Inputs over what the model can read are the caller's to shorten, and a content-policy signal is the caller's+ material refused. A 400 nothing recognises is most likely a request of ours the provider could not take. Every+ family reads the context-window signal: ``model.context_exceeded`` means the inputs exceed what the model can read,+ whether the model writes text, generates images or reads documents, as an image model whose prompt images exceed+ its limit already reports, so the code stays the same and the Render step words the advice for the family.+ """+ if metadata.is_context_window_exceeded:+ return CoreCode.MODEL_CONTEXT_EXCEEDED+ if metadata.is_content_policy_violation:+ return CoreCode.MODEL_CONTENT_REFUSED+ return CoreCode.RUNTIME_UNEXPECTED++def classify_inference_error(metadata: SDKErrorEnvelope, *, family: InferenceErrorFamily) -> ClassificationResult:"""Classify an inference SDK error from its structured metadata.@@ -346,23 +362,30 @@ def classify_inference_error(metadata: SDKErrorEnvelope, *, family: InferenceErr)if status_code == 400:- # A 400 nothing recognises is most likely a request of ours the provider could not take; only a- # content-policy signal says the caller's material was refused.return ClassificationResult(category=InferenceErrorCategory.CONTENT,user_action_kind=UserActionKind.CHANGE_INPUT,- error_code=CoreCode.MODEL_CONTENT_REFUSED if metadata.is_content_policy_violation else CoreCode.RUNTIME_UNEXPECTED,+ error_code=_bad_request_error_code(metadata=metadata),)if status_code >= 500:return _classify_server_error(status_code=status_code, family=family)if status_code >= 400:- # Unrecognized 4xx (e.g. 405, 409, 422) — 5xx is handled above. A safety/content- # policy rejection routed as 422 (FAL surfaces these here, signalled by- # ``provider_error_code = "ContentPolicyViolation"``) is rejected content, not a- # configuration issue — keep it CONTENT so downstream reporting and retry policy- # treat it as bad input.+ # Unrecognized 4xx (e.g. 405, 409, 413, 422) — 5xx is handled above. A refusal of the+ # caller's material may arrive on another status than 400, and is read as the 400 arm+ # reads it: inputs over what the model can read, which a gateway may answer 422 or 413,+ # and a safety/content policy rejection routed as 422 (FAL surfaces these here, signalled+ # by ``provider_error_code = "ContentPolicyViolation"``). Each is rejected content, not a+ # configuration issue — keep it CONTENT so downstream reporting and retry policy treat it+ # as bad input. The context-window signal comes first, as on a 400, and a 413 that+ # carries none keeps its payload-size reading below.+ if metadata.is_context_window_exceeded:+ return ClassificationResult(+ category=InferenceErrorCategory.CONTENT,+ user_action_kind=UserActionKind.CHANGE_INPUT,+ error_code=CoreCode.MODEL_CONTEXT_EXCEEDED,+ )if metadata.is_content_policy_violation:return ClassificationResult(category=InferenceErrorCategory.CONTENT,diff --git a/pipelex/cogt/inference/error_render.py b/pipelex/cogt/inference/error_render.pyindex 12cd8b560..fd0842559 100644--- a/pipelex/cogt/inference/error_render.py+++ b/pipelex/cogt/inference/error_render.py@@ -9,6 +9,8 @@ worker's error family, with a human-readable message and a structuredimport math+from mthds.protocol.problem.registry import CoreCode+from pipelex.cogt.exceptions import (CogtError,ExtractJobFailureError,@@ -25,6 +27,7 @@ from pipelex.cogt.exceptions import ()from pipelex.cogt.inference.error_classification import SDKErrorEnvelope, UserAction, UserActionKindfrom pipelex.cogt.inference.error_classify import ClassificationResult, InferenceErrorFamily+from pipelex.errors.registry.pipelex_codes import PipelexErrorCode# Generic failure error class per family — used when the error is not a missing model._FAILURE_CLASSES: dict[InferenceErrorFamily, type[CogtError]] = {@@ -69,7 +72,34 @@ def _render_model_not_allowed_detail(*, model_handle: str) -> str:)-def _render_detail(metadata: SDKErrorEnvelope, *, classification: ClassificationResult, model_handle: str) -> str:+def _render_context_exceeded_detail(*, family: InferenceErrorFamily) -> str:+ """Produce the advice for inputs over what the model can read, worded for what the family's model reads.++ The code, ``model.context_exceeded``, says the inputs exceed what the model can read, whatever the model. A language+ model's window holds the output a request reserves with ``max_tokens`` as well as the prompt, and its providers+ refuse both overflows alike, Anthropic's "input length and `max_tokens` exceed context limit" and OpenAI's+ "maximum context length" counting the completion beside the messages, so its advice names both. The+ ``max_tokens`` may be the deck's default rather than one the pipe set, and a pipe's own setting overrides it. An+ image model reads a prompt and its input images, and an extraction model a document, neither of which has a+ ``max_tokens`` to lower.+ """+ match family:+ case InferenceErrorFamily.LLM:+ return (+ "The prompt plus the output requested with max_tokens exceed the model's context window — shorten the input "+ "or lower max_tokens, or choose a model with a larger context window."+ )+ case InferenceErrorFamily.IMG_GEN:+ return "The prompt or its input images exceed what the image model can read — shorten the prompt or send fewer images."+ case InferenceErrorFamily.EXTRACT:+ return "The document exceeds what the extraction model can read — send a shorter document, or split it."+ case InferenceErrorFamily.SEARCH:+ return "The query built from the inputs exceeds what the search model can read — shorten the inputs."+ case InferenceErrorFamily.JUDGMENT:+ return "The material to judge exceeds what the judgment model can read — shorten it."+++def _render_detail(metadata: SDKErrorEnvelope, *, classification: ClassificationResult, family: InferenceErrorFamily, model_handle: str) -> str:"""Produce the free-form user-facing advice text for the classified error."""if classification.service_error_code is not None:# Branches ahead of the action kind rather than inside it: the service's own code names the@@ -99,6 +129,13 @@ def _render_detail(metadata: SDKErrorEnvelope, *, classification: Classificationcase UserActionKind.CHECK_CREDENTIALS:return "The provider rejected the credentials — check that the API key is valid and correctly configured."case UserActionKind.CHANGE_INPUT:+ # Worded from the code the classification picked, not from the metadata's wording, so a 5xx, or a parse+ # failure quoting the model's own text, never reads as the context window+ match classification.error_code:+ case CoreCode.MODEL_CONTEXT_EXCEEDED:+ return _render_context_exceeded_detail(family=family)+ case CoreCode() | PipelexErrorCode():+ passif metadata.is_content_policy_violation:return "Content was rejected by the provider's safety filters — revise the prompt."return "The provider rejected the request — review the prompt, parameters, and inputs."@@ -133,7 +170,7 @@ def render_inference_error(message = _format_message(metadata, model_desc=model_desc)user_action = UserAction(kind=classification.user_action_kind,- detail=_render_detail(metadata, classification=classification, model_handle=model_handle),+ detail=_render_detail(metadata, classification=classification, family=family, model_handle=model_handle),)if classification.is_model_not_found:not_found_class = _NOT_FOUND_CLASSES[family]diff --git a/pipelex/cogt/llm/completion_stop.py b/pipelex/cogt/llm/completion_stop.pyindex 4c6353a12..43f6b7190 100644--- a/pipelex/cogt/llm/completion_stop.py+++ b/pipelex/cogt/llm/completion_stop.py@@ -16,16 +16,25 @@ OpenAI-compatible servers send none. Two signals say on their own that the textwith them: a Responses API answer whose status is `incomplete` is never normal, an unknown or missing reason beingtaken as a truncation, and a Gemini answer with no candidate whose prompt feedback names a block reason is a refusal.-The classifier only turns a stop that was silently a success into an error. Structured generation is not read-here: its stops are the structuring library's business, and its configured re-asks are left as they are.+The Google image worker reads its answer's finish reason with `classify_image_finish_reason`, from Gemini's+vocabulary alone, the image values included, so a safety filter that refused the image, or an output token limit+that cut the answer before its image, is told from an empty answer, and the prompt's block reason as the text workers+read it.++The classifier only turns a stop that was silently a success into an error. Structured generation's stops are read+by the structuring library, instructor, whose configured re-asks are left as they are: it raises its+`IncompleteOutputException` for a structured answer cut at its output limit, carrying that answer, and+`read_incomplete_output` reads the answer's stop value and output tokens, so that the worker raises the same+truncation a text cut at its limit raises."""from collections.abc import Mappingfrom enum import StrEnumfrom types import MappingProxyType+from typing import Any, NamedTuplefrom pipelex import log-from pipelex.cogt.exceptions import CompletionTruncationLimit, LLMCompletionRefusedError, LLMCompletionTruncatedError+from pipelex.cogt.exceptions import CompletionOutputKind, CompletionTruncationLimit, LLMCompletionRefusedError, LLMCompletionTruncatedErrorclass CompletionStopOutcome(StrEnum):@@ -38,6 +47,24 @@ class CompletionStopOutcome(StrEnum):CONTEXT_WINDOW_EXCEEDED = "context_window_exceeded"REFUSED = "refused"+ @property+ def is_refused(self) -> bool:+ """Whether the model refused, or a content filter stopped the output."""+ match self:+ case CompletionStopOutcome.REFUSED:+ return True+ case CompletionStopOutcome.NORMAL | CompletionStopOutcome.TRUNCATED | CompletionStopOutcome.CONTEXT_WINDOW_EXCEEDED:+ return False++ @property+ def is_truncated(self) -> bool:+ """Whether an output limit cut the output: the one the request sent, or the model's own when it sent none."""+ match self:+ case CompletionStopOutcome.TRUNCATED:+ return True+ case CompletionStopOutcome.NORMAL | CompletionStopOutcome.CONTEXT_WINDOW_EXCEEDED | CompletionStopOutcome.REFUSED:+ return False+_NORMAL = CompletionStopOutcome.NORMAL_TRUNCATED = CompletionStopOutcome.TRUNCATED@@ -236,6 +263,30 @@ def classify_responses_stop(*, status: str | None, incomplete_reason: str | Nonereturn outcome+def classify_image_finish_reason(*, finish_reason: str, model_handle: str) -> CompletionStopOutcome:+ """Read the finish reason of a Gemini image generation's candidate, which says whether a safety filter refused it.++ The Google image worker receives Gemini's values alone, so it reads Gemini's vocabulary, the image values+ included, rather than the union the text workers read. A value the classifier does not know is logged at warning+ level as an image generation's, and taken as normal, so the worker looks for the image as before.++ Args:+ finish_reason: The candidate's `finish_reason`, as its value.+ model_handle: The model's deck handle, named by the warning about an unknown value.++ Returns:+ The outcome the value stands for: normal for an unknown value.+ """+ outcome = GEMINI_FINISH_REASONS.get(finish_reason.upper())+ if outcome is None:+ log.warning(+ "An image finish reason was not recognized, so it was taken as normal",+ fields={"model_handle": model_handle, "finish_reason": finish_reason},+ )+ return _NORMAL+ return outcome++def classify_prompt_block_reason(*, block_reason: str, model_handle: str) -> CompletionStopOutcome:"""Read the block reason of a Gemini answer that holds no candidate, which is always a refusal.@@ -259,6 +310,52 @@ def classify_prompt_block_reason(*, block_reason: str, model_handle: str) -> Comreturn outcome+class IncompleteOutput(NamedTuple):+ """What a structured answer instructor found cut at its output limit says about the cut."""++ # The provider's stop value, None when the answer carries none where instructor reads it+ stop_reason: str | None+ # The output tokens the provider counted, None when the answer carries no count+ output_tokens: int | None+++def read_incomplete_output(*, last_completion: Any) -> IncompleteOutput:+ """Read the stop value and the output tokens of the answer instructor's `IncompleteOutputException` carries.++ instructor raises that exception, as its `last_completion`, for an OpenAI-shaped completion whose first choice+ finished on `length`, Mistral's included, for such a choice by itself, and for an Anthropic message that stopped+ on `max_tokens`. The answer's type depends on the SDK and on the structuring mode, so it is read where instructor+ reads it, attribute by attribute: an Anthropic message's `stop_reason`, a choice's `finish_reason`, else the first+ choice's, and the `usage` its type carries, `output_tokens` for Anthropic and `completion_tokens` for the others.++ Args:+ last_completion: The answer instructor found cut, as it carries it.++ Returns:+ The answer's stop value and output tokens, each None when the answer does not carry it.+ """+ stop_reason = _read_stop_value(source=last_completion, attribute="stop_reason") or _read_stop_value(+ source=last_completion, attribute="finish_reason"+ )+ if stop_reason is None:+ choices: Any = getattr(last_completion, "choices", None)+ if isinstance(choices, list) and choices:+ stop_reason = _read_stop_value(source=choices[0], attribute="finish_reason")+ usage: Any = getattr(last_completion, "usage", None)+ output_tokens: int | None = None+ for usage_attribute in ("output_tokens", "completion_tokens"):+ count: Any = getattr(usage, usage_attribute, None)+ if isinstance(count, int):+ output_tokens = count+ break+ return IncompleteOutput(stop_reason=stop_reason, output_tokens=output_tokens)+++def _read_stop_value(*, source: Any, attribute: str) -> str | None:+ value: Any = getattr(source, attribute, None)+ return value if isinstance(value, str) else None++def raise_for_completion_stop(*,outcome: CompletionStopOutcome,@@ -297,6 +394,7 @@ def raise_for_completion_stop(model_handle=model_handle,stop_reason=stop_reason,truncation_limit=truncation_limit,+ output_kind=CompletionOutputKind.TEXT,pipe_code=pipe_code,max_tokens=max_tokens,output_tokens=output_tokens,diff --git a/pipelex/cogt/llm/llm_worker_abstract.py b/pipelex/cogt/llm/llm_worker_abstract.pyindex 1365205a1..390ba5462 100644--- a/pipelex/cogt/llm/llm_worker_abstract.py+++ b/pipelex/cogt/llm/llm_worker_abstract.py@@ -10,12 +10,20 @@ from opentelemetry.trace import NonRecordingSpan, Span, SpanContext, SpanKind, Sfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.exceptions import CogtError, LLMCapabilityError, LLMCompletionRefusedError, LLMCompletionTruncatedError+from pipelex.cogt.exceptions import (+ CogtError,+ CompletionOutputKind,+ CompletionTruncationLimit,+ LLMCapabilityError,+ LLMCompletionRefusedError,+ LLMCompletionTruncatedError,+ MaxTokensCeiling,+)from pipelex.cogt.inference.inference_call_summary import InferenceCallSummary, InferenceOperationfrom pipelex.cogt.inference.inference_constants import InferenceOutputTypefrom pipelex.cogt.inference.inference_worker_abstract import InferenceWorkerAbstractfrom pipelex.cogt.inference.prompt_file_checks import check_prompt_documents_are_read, check_prompt_images_are_images-from pipelex.cogt.llm.completion_stop import CompletionStopOutcome, classify_stop_reason, raise_for_completion_stop+from pipelex.cogt.llm.completion_stop import CompletionStopOutcome, classify_stop_reason, raise_for_completion_stop, read_incomplete_outputfrom pipelex.cogt.llm.thinking_mode import ThinkingModefrom pipelex.cogt.model_backends.constraints import ListedConstraint, ValuedConstraintfrom pipelex.cogt.usage.token_category import TokenCategory@@ -511,6 +519,44 @@ class LLMWorkerAbstract(InferenceWorkerAbstract, ABC):output_tokens=output_tokens,)+ def _incomplete_output_error(+ self,+ *,+ llm_job: LLMJob,+ last_completion: Any,+ max_tokens: int | None,+ max_tokens_ceiling: MaxTokensCeiling | None = None,+ ) -> LLMCompletionTruncatedError:+ """The error for a structured output instructor found cut at its output limit, for the worker to raise.++ instructor reads the stop value of every structured answer it parses, and raises its `IncompleteOutputException`+ for one cut at the request's output limit, `length` or `max_tokens`, which its re-asks never retry. A worker+ whose structuring mode can raise it catches it and raises this instead, from it, so the structured output reads+ `model.output_truncated`, as a text cut at the same limit does, rather than an exception no one classified, and+ the error says a structured output was cut.++ Args:+ llm_job: The job whose output it was: its pipe is named by the error.+ last_completion: The cut answer instructor's exception carries, whose stop value and output tokens are named.+ max_tokens: The output limit the request sent, None when it sent none.+ max_tokens_ceiling: Set by a worker that sent a lower max_tokens than it was given, to fit its call's+ timeout, so the error's next step does not ask for a higher one.++ Returns:+ The truncation at the output limit, naming the provider's stop value as the answer carries it.+ """+ incomplete_output = read_incomplete_output(last_completion=last_completion)+ return LLMCompletionTruncatedError(+ model_handle=self.inference_model.name,+ stop_reason=incomplete_output.stop_reason or "unknown",+ truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.STRUCTURED,+ pipe_code=llm_job.job_metadata.pipe_code,+ max_tokens=max_tokens,+ output_tokens=incomplete_output.output_tokens,+ max_tokens_ceiling=max_tokens_ceiling,+ )+def _check_can_perform_job(self, llm_job: LLMJob):# This can be overridden by subclasses for specific checksself._check_vision_support(llm_job=llm_job)diff --git a/pipelex/providers/anthropic/anthropic_llm_worker.py b/pipelex/providers/anthropic/anthropic_llm_worker.pyindex bab5db069..2133da8ad 100644--- a/pipelex/providers/anthropic/anthropic_llm_worker.py+++ b/pipelex/providers/anthropic/anthropic_llm_worker.py@@ -14,7 +14,7 @@ from pydantic.dataclasses import dataclass as pydantic_dataclassfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.exceptions import CogtError, InferenceErrorCategory, LLMCapabilityError, LLMCompletionError, SdkTypeError+from pipelex.cogt.exceptions import CogtError, InferenceErrorCategory, LLMCapabilityError, LLMCompletionError, MaxTokensCeiling, SdkTypeErrorfrom pipelex.cogt.inference.error_classification import (UserAction,UserActionKind,@@ -158,8 +158,9 @@ class AnthropicLLMWorker(LLMWorkerAbstract):A max_tokens the pipe's setting asked for and the call lowers is said at warning level, since the call thensends less than the method wrote. The model's own default lowered is said at debug level: a model whosedefault exceeds what the timeout allows has it lowered on every structured call, by design. Either way, an- error the call raises for a failure the lowered limit can explain, an output that failed validation or came- back incomplete, or a timeout, says the limit was lowered and to what (see `_with_note`).+ error the call raises for a failure the lowered limit can explain, an output that failed validation or a+ timeout, says the limit was lowered and to what (see `_with_note`). An output cut at the lowered limit says it+ itself, in a truncation whose next step does not ask for a higher max_tokens (see `MaxTokensCeiling`).Args:is_caller_setting: Whether the requested max_tokens is the pipe's setting rather than the model's default.@@ -192,9 +193,10 @@ class AnthropicLLMWorker(LLMWorkerAbstract):"""Whether a lowered max_tokens can explain the failure that ended a structured call.A lower limit can cut the structured output short, which instructor reports, once its re-asks are spent, as- an output that failed validation or came back incomplete; and the limit is lowered to fit the call's- timeout, so a timeout concerns it too. A failure the API or the network answered for its own reason, a rate- limit, an authentication failure, an overload, a server error or a lost connection, has nothing to do with it.+ an output that failed validation; and the limit is lowered to fit the call's timeout, so a timeout concerns+ it too. An output instructor reports cut at the limit is a truncation that names the lowered limit itself. A+ failure the API or the network answered for its own reason, a rate limit, an authentication failure, an+ overload, a server error or a lost connection, has nothing to do with it.Args:failure: The exception that ended the call, None when instructor's wrapper carries none.@@ -491,7 +493,7 @@ class AnthropicLLMWorker(LLMWorkerAbstract):sends_temperature = self.inference_model.accepts_temperature and not thinking_params.suppress_temperature# Deferred import: avoid pulling heavy SDK at module-load time- from instructor.core import InstructorRetryException # ruff: ignore[import-outside-top-level]+ from instructor.core import IncompleteOutputException, InstructorRetryException # ruff: ignore[import-outside-top-level]try:result_object, completion = await self.instructor_for_objects.chat.completions.create_with_completion(@@ -539,6 +541,21 @@ class AnthropicLLMWorker(LLMWorkerAbstract):),).with_error_code(CoreCode.RUNTIME_UNEXPECTED)raise self._with_note(error=fallback_error, note=lowered_max_tokens_note, failure=None) from instructor_exc+ except IncompleteOutputException as incomplete_exc:+ # instructor found the answer stopped at `max_tokens`. When the timeout lowered that limit, a higher one is+ # lowered the same way, so the error says so and words its next step for it, in its message and its action,+ # rather than taking the note after a next step that asks for a higher max_tokens+ max_tokens_ceiling = (+ MaxTokensCeiling(requested_max_tokens=requested_max_tokens, timeout_seconds=timeout_seconds)+ if effective_max_tokens < requested_max_tokens+ else None+ )+ raise self._incomplete_output_error(+ llm_job=llm_job,+ last_completion=incomplete_exc.last_completion,+ max_tokens=effective_max_tokens,+ max_tokens_ceiling=max_tokens_ceiling,+ ) from incomplete_excexcept (APIStatusError, APIConnectionError) as sdk_exc:metadata = extract_anthropic_metadata(sdk_exc)classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)diff --git a/pipelex/providers/google/google_factory.py b/pipelex/providers/google/google_factory.pyindex cc1601afc..36ffd3dcf 100644--- a/pipelex/providers/google/google_factory.py+++ b/pipelex/providers/google/google_factory.py@@ -1,7 +1,7 @@import asyncioimport base64from collections.abc import Coroutine-from typing import Any, ClassVar+from typing import Any, ClassVar, NamedTuplefrom google.genai import types as genai_typesfrom google.genai.client import Client as GoogleGenAiClient@@ -21,6 +21,15 @@ from pipelex.config import get_configfrom pipelex.tools.log.error_fields import error_fields+class GeminiStopSignal(NamedTuple):+ """How a Gemini answer says it stopped: at most one of the two reasons is set, and neither when it gives none."""++ # The first candidate's finish reason, as its value, when the answer holds a candidate+ finish_reason: str | None+ # The prompt feedback's block reason, as its value, when the answer holds no candidate: the prompt was blocked+ block_reason: str | None++class GoogleFactory:# The background tasks closing a Google async client, each held until it is done (see schedule_client_close)_pending_client_closes: ClassVar[set["asyncio.Task[None]"]] = set()@@ -108,6 +117,27 @@ class GoogleFactory:return genai_types.Content(parts=parts, role="user")+ @classmethod+ def read_stop_signal(cls, *, response: genai_types.GenerateContentResponse) -> GeminiStopSignal:+ """Read how a Gemini answer stopped, for the text and image workers alike.++ An answer holding a candidate says it in the first candidate's finish reason. An answer holding none was+ blocked before the model wrote anything, and says why in its prompt feedback's block reason, which is read+ only then: the two vocabularies share some words with different meanings.++ Args:+ response: The Gemini answer.++ Returns:+ The finish reason or the block reason the answer gives, each as its value, neither when it gives none.+ """+ if response.candidates:+ finish_reason = response.candidates[0].finish_reason+ return GeminiStopSignal(finish_reason=finish_reason.value if finish_reason else None, block_reason=None)+ if response.prompt_feedback and (block_reason := response.prompt_feedback.block_reason):+ return GeminiStopSignal(finish_reason=None, block_reason=block_reason.value)+ return GeminiStopSignal(finish_reason=None, block_reason=None)+@classmethoddef extract_text_from_response(cls, response: genai_types.GenerateContentResponse, *, model_desc: str) -> str:"""Extract text from a Google Gemini response, skipping thinking parts.diff --git a/pipelex/providers/google/google_img_gen_worker.py b/pipelex/providers/google/google_img_gen_worker.pyindex 5a26ae194..7255d13b3 100644--- a/pipelex/providers/google/google_img_gen_worker.py+++ b/pipelex/providers/google/google_img_gen_worker.py@@ -1,4 +1,6 @@import asyncio+from collections.abc import Mapping+from types import MappingProxyTypefrom typing import Anyimport httpx@@ -9,7 +11,7 @@ from mthds.protocol.problem.registry import CoreCodefrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.exceptions import ImgGenGenerationError, InferenceErrorCategory, SdkTypeError+from pipelex.cogt.exceptions import ImgGenGenerationError, ImgGenOutputTruncatedError, InferenceErrorCategory, SdkTypeErrorfrom pipelex.cogt.image.generated_image import GeneratedImageRawDetailsfrom pipelex.cogt.image.image_size import ImageSizefrom pipelex.cogt.img_gen.img_gen_gemini_mapping import ImgGenGeminiMapping@@ -18,11 +20,42 @@ from pipelex.cogt.img_gen.img_gen_worker_abstract import ImgGenWorkerAbstractfrom pipelex.cogt.inference.error_classification import UserAction, UserActionKind, extract_google_metadatafrom pipelex.cogt.inference.error_classify import InferenceErrorFamily, classify_inference_errorfrom pipelex.cogt.inference.error_render import render_inference_error+from pipelex.cogt.llm.completion_stop import classify_image_finish_reasonfrom pipelex.cogt.model_backends.model_spec import InferenceModelSpecfrom pipelex.providers.google.google_factory import GoogleFactoryfrom pipelex.reporting.reporting_protocol import ReportingProtocolfrom pipelex.tools.log.error_fields import error_fields+# What a reason Gemini gives for refusing an image says, in the words its error gives: the finish reasons that+# `classify_image_finish_reason` reads as a refusal, and the prompt feedback's block reasons, which share their words+# and their meanings. A recitation or an unsupported language is no safety filter's doing, so only the safety reasons+# name one. A reason not listed, such as `OTHER`, is named by its value alone.+_IMAGE_REFUSAL_CAUSES: Mapping[str, str] = MappingProxyType(+ {+ "SAFETY": "a safety filter flagged the content",+ "IMAGE_SAFETY": "a safety filter flagged the content",+ "PROHIBITED_CONTENT": "Gemini judged the content prohibited",+ "IMAGE_PROHIBITED_CONTENT": "Gemini judged the content prohibited",+ "BLOCKLIST": "the content holds a term from Gemini's blocklist",+ "SPII": "the content may hold sensitive personal information",+ "RECITATION": "the output may reproduce an existing source too closely",+ "IMAGE_RECITATION": "the output may reproduce an existing source too closely",+ "LANGUAGE": "the prompt's language is not one the model supports",+ "JAILBREAK": "Gemini took the prompt for a jailbreak attempt",+ "MODEL_ARMOR": "Model Armor blocked the prompt",+ }+)+++def _holds_image(*, response: genai_types.GenerateContentResponse) -> bool:+ """Whether the answer's first candidate holds image data, the part the worker returns as the image."""+ if not response.candidates:+ return False+ content = response.candidates[0].content+ if not content or not content.parts:+ return False+ return any(part.inline_data is not None and part.inline_data.data is not None for part in content.parts)+class GoogleImgGenWorker(ImgGenWorkerAbstract):"""Worker for generating images using Google Gemini Image models (Nano Banana)."""@@ -142,6 +175,10 @@ class GoogleImgGenWorker(ImgGenWorkerAbstract):img_gen_tokens_usage.nb_tokens_by_category.get(token_category, 0) + nb_tokens)+ # Read before the image, so a prompt or an image a filter blocked, or an answer cut before its image, is+ # reported for what stopped it, not as an empty answer+ self._check_stop(response=response)+# Extract image from responseif not response.candidates:msg = f"No candidates returned from model: {self.inference_model.desc}"@@ -203,6 +240,54 @@ class GoogleImgGenWorker(ImgGenWorkerAbstract):provider_metadata=None,).with_error_code(CoreCode.PROVIDER_RESPONSE_INVALID)+ def _check_stop(self, *, response: genai_types.GenerateContentResponse) -> None:+ """Raise when Gemini says it refused the image, or cut its answer before the image, before the worker looks for one.++ Gemini answers a blocked image request with a success all the same: a candidate whose finish reason names the+ filter that stopped it (`IMAGE_SAFETY`, `IMAGE_PROHIBITED_CONTENT`, `SAFETY` and their kin), and no image, or,+ for a prompt blocked before the model wrote anything, no candidate at all and a block reason in the prompt+ feedback, which is a refusal whatever its value, as the text workers read it. The finish reason reads from+ Gemini's vocabulary (see `pipelex.cogt.llm.completion_stop`), so an image refused on policy grounds is+ `model.content_refused`, the caller's input to revise, rather than an empty answer. The message names the+ reason Gemini gave and what it says, so a recitation or an unsupported language is never called a safety+ filter's block. A finish reason saying the model's output token limit cut the answer (`MAX_TOKENS`) is+ `model.output_truncated` when the candidate holds no image, the cause the answer states rather than an empty+ answer, while an image the model finished before the limit is whole, and returned. Any other finish reason+ leaves the answer to the checks that read what it holds.++ Raises:+ ImgGenGenerationError: With `model.content_refused`, when a finish reason or a block reason says the request+ was refused.+ ImgGenOutputTruncatedError: When the finish reason says the output token limit cut the answer, which holds+ no image.+ """+ model_handle = self.inference_model.name+ stop_signal = GoogleFactory.read_stop_signal(response=response)+ refusal_reason: str+ what_happened: str+ if stop_signal.finish_reason is not None:+ stop_outcome = classify_image_finish_reason(finish_reason=stop_signal.finish_reason, model_handle=model_handle)+ if stop_outcome.is_truncated and not _holds_image(response=response):+ raise ImgGenOutputTruncatedError(model_handle=model_handle, stop_reason=stop_signal.finish_reason)+ if not stop_outcome.is_refused:+ return+ refusal_reason = stop_signal.finish_reason+ what_happened = f"The model '{model_handle}' stopped before it generated the image (finish reason '{refusal_reason}')"+ elif stop_signal.block_reason is not None:+ # A prompt blocked before the model answered is a refusal whatever its reason, which the error names+ refusal_reason = stop_signal.block_reason+ what_happened = f"Gemini blocked the prompt before the model '{model_handle}' generated the image (block reason '{refusal_reason}')"+ else:+ return+ cause = _IMAGE_REFUSAL_CAUSES.get(refusal_reason.upper())+ next_step = "Revise the image prompt or its input images."+ msg = f"{what_happened}: {cause}. {next_step}" if cause else f"{what_happened}. {next_step}"+ raise (+ ImgGenGenerationError(msg, error_category=InferenceErrorCategory.CONTENT, provider_metadata=None)+ .with_error_code(CoreCode.MODEL_CONTENT_REFUSED)+ .as_caller_fault(user_action=UserAction(kind=UserActionKind.CHANGE_INPUT, detail=next_step))+ )+@overrideasync def _gen_image_list(self,diff --git a/pipelex/providers/google/google_llm_worker.py b/pipelex/providers/google/google_llm_worker.pyindex 7de05ad8b..342420f37 100644--- a/pipelex/providers/google/google_llm_worker.py+++ b/pipelex/providers/google/google_llm_worker.py@@ -267,15 +267,15 @@ class GoogleLLMWorker(LLMWorkerAbstract):llm_job.job_report.llm_tokens_usage.nb_tokens_by_category = GoogleFactory.extract_token_usage(response.usage_metadata)# Read before the text, so a candidate cut or filtered with no text left is reported for what stopped it- if response.candidates:- if finish_reason := response.candidates[0].finish_reason:- self._check_completion_stop(llm_job=llm_job, stop_reason=finish_reason.value, max_tokens=job_params.max_tokens)- elif response.prompt_feedback and (block_reason := response.prompt_feedback.block_reason):+ stop_signal = GoogleFactory.read_stop_signal(response=response)+ if stop_signal.finish_reason is not None:+ self._check_completion_stop(llm_job=llm_job, stop_reason=stop_signal.finish_reason, max_tokens=job_params.max_tokens)+ elif stop_signal.block_reason is not None:# A blocked prompt gets no candidate at all, its block reason in the prompt feedback: a refusal, named for its reasonself._raise_for_completion_stop(llm_job=llm_job,- outcome=classify_prompt_block_reason(block_reason=block_reason.value, model_handle=self.inference_model.name),- stop_reason=block_reason.value,+ outcome=classify_prompt_block_reason(block_reason=stop_signal.block_reason, model_handle=self.inference_model.name),+ stop_reason=stop_signal.block_reason,max_tokens=job_params.max_tokens,)diff --git a/pipelex/providers/mistral/mistral_llm_worker.py b/pipelex/providers/mistral/mistral_llm_worker.pyindex 8ff95faa7..cdaa790c9 100644--- a/pipelex/providers/mistral/mistral_llm_worker.py+++ b/pipelex/providers/mistral/mistral_llm_worker.py@@ -260,9 +260,10 @@ class MistralLLMWorker(LLMWorkerAbstract):reasoning_effort = self._resolve_reasoning_effort(inference_model=self.inference_model, job_params=job_params)self._log_reasoning_sent(api_name="Mistral", settings={"reasoning_effort": None if reasoning_effort is UNSET else reasoning_effort})# Deferred import: avoid pulling heavy SDK at module-load time- from instructor.core import InstructorRetryException # ruff: ignore[import-outside-top-level]+ from instructor.core import IncompleteOutputException, InstructorRetryException # ruff: ignore[import-outside-top-level]messages = await self.mistral_factory.make_simple_messages_openai_typed(llm_job=llm_job)+ max_tokens = job_params.max_tokens or self.default_max_tokenstry:result_object, completion = await self.instructor_for_objects.chat.completions.create_with_completion(@@ -270,7 +271,7 @@ class MistralLLMWorker(LLMWorkerAbstract):messages=messages,model=self.inference_model.model_id,temperature=job_params.temperature if self.inference_model.accepts_temperature else UNSET,- max_tokens=job_params.max_tokens or self.default_max_tokens,+ max_tokens=max_tokens,reasoning_effort=reasoning_effort,# instructor's retry is confined to schema re-ask: this validation-only AsyncRetrying# re-asks on a malformed/invalid output but never retries a transport error, which ends the@@ -302,6 +303,11 @@ class MistralLLMWorker(LLMWorkerAbstract):detail="Structured generation failed for an unrecognized reason — retry, and report this if it persists",),).with_error_code(CoreCode.RUNTIME_UNEXPECTED) from instructor_exc+ except IncompleteOutputException as incomplete_exc:+ # instructor found the answer's first choice finished on `length`+ raise self._incomplete_output_error(+ llm_job=llm_job, last_completion=incomplete_exc.last_completion, max_tokens=max_tokens+ ) from incomplete_excexcept (MistralError, httpx.TransportError) as sdk_exc:metadata = extract_mistral_metadata(sdk_exc)classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)diff --git a/pipelex/providers/openai/openai_completions_llm_worker.py b/pipelex/providers/openai/openai_completions_llm_worker.pyindex 787fa80b2..e5670a4bc 100644--- a/pipelex/providers/openai/openai_completions_llm_worker.py+++ b/pipelex/providers/openai/openai_completions_llm_worker.py@@ -212,7 +212,7 @@ class OpenAICompletionsLLMWorker(LLMWorkerAbstract):self._log_reasoning_sent(api_name="OpenAI Chat Completions", settings={"reasoning_effort": openai_reasoning_effort})messages = await self.openai_completions_factory.make_simple_messages(llm_job=llm_job)# Deferred import: avoid pulling heavy SDK at module-load time- from instructor.core import InstructorRetryException # ruff: ignore[import-outside-top-level]+ from instructor.core import IncompleteOutputException, InstructorRetryException # ruff: ignore[import-outside-top-level]extra_headers, extra_body = self.openai_completions_factory.make_extras(inference_model=self.inference_model, inference_job=llm_job, output_desc=schema.__name__@@ -255,6 +255,11 @@ class OpenAICompletionsLLMWorker(LLMWorkerAbstract):raise LLMCompletionError(msg, error_category=InferenceErrorCategory.UNKNOWN).with_error_code(CoreCode.RUNTIME_UNEXPECTED) from instructor_exc+ except IncompleteOutputException as incomplete_exc:+ # instructor found the answer's first choice finished on `length`+ raise self._incomplete_output_error(+ llm_job=llm_job, last_completion=incomplete_exc.last_completion, max_tokens=job_params.max_tokens+ ) from incomplete_excexcept (openai.APIStatusError, openai.APIConnectionError) as sdk_exc:metadata = extract_openai_metadata(sdk_exc)classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)diff --git a/tests/data/errors/error_identity.txt b/tests/data/errors/error_identity.txtindex 5b9a17356..da981ea16 100644--- a/tests/data/errors/error_identity.txt+++ b/tests/data/errors/error_identity.txt@@ -137,6 +137,7 @@ HuggingFaceProviderUnknownError | pipelex.config.invalidImgGenGenerationError | inherits cause, else provider.outcome_unknown; or chosen: runtime.unexpected, runtime.misconfigured, provider.rate_limited, provider.unavailable, provider.timeout, provider.failed, provider.response_invalid, provider.credential_refused, provider.quota_exhausted, model.unavailable, model.capability_unsupported, model.context_exceeded, model.content_refused, model.output_schema_mismatch, model.no_result, input.too_large, input.file_type_unsupported, input.file_unreadable, input.reference_not_found, input.url_unreachable, security.input_url_refused, security.method_url_refused, pipelex.config.invalid, pipelex.config.credential_missing, pipelex.billing.credit_exhaustedImgGenHandleNotFoundError | pipelex.config.invalidImgGenModelNotFoundError | model.unavailable+ImgGenOutputTruncatedError | model.output_truncatedImgGenParameterError | one of model.capability_unsupported, pipelex.config.invalid, runtime.unexpectedImgGenPromptError | pipe.prompt_emptyImgGenSettingValueError | method.model_setting_invaliddiff --git a/tests/unit/pipelex/cogt/inference/test_classify_inference_error.py b/tests/unit/pipelex/cogt/inference/test_classify_inference_error.pyindex 3e098183f..18c97ecb1 100644--- a/tests/unit/pipelex/cogt/inference/test_classify_inference_error.py+++ b/tests/unit/pipelex/cogt/inference/test_classify_inference_error.py@@ -141,6 +141,56 @@ class _TestCases:CoreCode.RUNTIME_UNEXPECTED,False,),+ (+ "bad_request_400_saying_the_prompt_is_over_the_context_window",+ ProviderName.ANTHROPIC,+ "BadRequestError",+ "prompt is too long: 208310 tokens > 200000 maximum",+ 400,+ _LLM,+ InferenceErrorCategory.CONTENT,+ UserActionKind.CHANGE_INPUT,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ False,+ ),+ (+ # A gateway may answer the window's refusal with another 4xx, which reads it as a 400 does+ "unprocessable_422_saying_the_prompt_is_too_long",+ ProviderName.OPENAI,+ "UnprocessableEntityError",+ "prompt is too long",+ 422,+ _LLM,+ InferenceErrorCategory.CONTENT,+ UserActionKind.CHANGE_INPUT,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ False,+ ),+ (+ "request_too_large_413_saying_the_context_window",+ ProviderName.GATEWAY,+ "APIStatusError",+ "Your input exceeds the context window of this model",+ 413,+ _LLM,+ InferenceErrorCategory.CONTENT,+ UserActionKind.CHANGE_INPUT,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ False,+ ),+ (+ # A 5xx is the provider's, whatever words it carries+ "internal_server_error_500_saying_the_prompt_is_too_long",+ ProviderName.OPENAI,+ "InternalServerError",+ "prompt is too long",+ 500,+ _LLM,+ InferenceErrorCategory.TRANSIENT,+ UserActionKind.WAIT_AND_RETRY,+ CoreCode.PROVIDER_FAILED,+ False,+ ),("bad_request_400_with_a_content_policy_signal",ProviderName.OPENAI,diff --git a/tests/unit/pipelex/cogt/inference/test_provider_error_metadata.py b/tests/unit/pipelex/cogt/inference/test_provider_error_metadata.pyindex 6f076d1f8..72f90c939 100644--- a/tests/unit/pipelex/cogt/inference/test_provider_error_metadata.py+++ b/tests/unit/pipelex/cogt/inference/test_provider_error_metadata.py@@ -7,10 +7,15 @@ from the exception chain.from __future__ import annotations+from typing import Any++import httpximport pytest+from mthds.protocol.problem.registry import CoreCodefrom pydantic import ValidationError-from pipelex.cogt.inference.error_classification import ProviderErrorMetadata+from pipelex.cogt.inference.error_classification import ProviderErrorMetadata, extract_azure_metadata+from pipelex.cogt.inference.error_classify import InferenceErrorFamily, classify_inference_errorfrom pipelex.cogt.inference.provider_name import ProviderName@@ -162,6 +167,59 @@ class TestProviderErrorMetadata:assert metadata.is_content_policy_violation is expected+ @pytest.mark.parametrize(+ ("_topic", "message", "provider_error_code", "body", "expected"),+ [+ ("openai_code_in_the_body", "Error code: 400", "invalid_request_error", {"code": "context_length_exceeded"}, True),+ ("gateway_code_first", "Request failed", "context_length_exceeded", None, True),+ ("openai_earlier_wording", "This model's maximum context length is 8192 tokens", None, None, True),+ ("openai_current_wording", "Your input exceeds the context window of this model", None, None, True),+ ("anthropic", "prompt is too long: 208310 tokens > 200000 maximum", None, None, True),+ ("anthropic_with_max_tokens", "input length and `max_tokens` exceed context limit: 197202 + 8192 > 200000", None, None, True),+ ("bedrock", "Input is too long for requested model.", None, None, True),+ ("gemini", "The input token count (1234567) exceeds the maximum number of tokens allowed (1048576).", None, None, True),+ (+ "a_parameter_refused",+ "Invalid value for 'temperature': must be between 0 and 2",+ "invalid_request_error",+ {"code": "invalid_value"},+ False,+ ),+ ("a_missing_field", "max_tokens: Field required", None, None, False),+ ("a_field_too_long", "tools.0.description: tool description input is too long", None, None, False),+ ("a_size_limit", "Request exceeds the maximum size", "request_too_large", None, False),+ ],+ )+ def test_is_context_window_exceeded(self, _topic: str, message: str, provider_error_code: str | None, body: Any, expected: bool) -> None:+ metadata = ProviderErrorMetadata(+ provider=ProviderName.OPENAI,+ sdk_exception_type="BadRequestError",+ message=message,+ status_code=400,+ provider_error_code=provider_error_code,+ body=body,+ )++ assert metadata.is_context_window_exceeded is expected++ def test_is_context_window_exceeded_reads_the_wording_in_the_body(self) -> None:+ """A raw httpx error renders only its status and URL, so the provider's wording is in the body alone."""+ body = {+ "error": {+ "code": "invalid_request",+ "message": "This model's maximum context length is 4096 tokens. However, your prompt resulted in 5210 tokens.",+ }+ }+ request = httpx.Request("POST", "https://test.openai.azure.com/openai/deployments/flux/images/generations")+ with pytest.raises(httpx.HTTPStatusError) as exc_info:+ httpx.Response(status_code=400, json=body, request=request).raise_for_status()+ metadata = extract_azure_metadata(exc_info.value)+ assert "context" not in metadata.message.lower()++ assert metadata.is_context_window_exceeded is True+ classification = classify_inference_error(metadata, family=InferenceErrorFamily.IMG_GEN)+ assert classification.error_code == CoreCode.MODEL_CONTEXT_EXCEEDED+def test_is_content_policy_violation_scans_body_when_message_is_empty(self) -> None:"""Azure REST puts safety phrasing only in the response body; the message stays the``HTTPStatusError`` envelope. The probe must still detect the violation by scanningdiff --git a/tests/unit/pipelex/cogt/llm/test_completion_stop.py b/tests/unit/pipelex/cogt/llm/test_completion_stop.pyindex 56e02ba6c..4109095b1 100644--- a/tests/unit/pipelex/cogt/llm/test_completion_stop.py+++ b/tests/unit/pipelex/cogt/llm/test_completion_stop.py@@ -11,7 +11,7 @@ from openai.types.chat.chat_completion import Choicefrom openai.types.responses.response import IncompleteDetailsfrom types_aiobotocore_bedrock_runtime.literals import StopReasonType-from pipelex.cogt.exceptions import CompletionTruncationLimit, LLMCompletionRefusedError, LLMCompletionTruncatedError+from pipelex.cogt.exceptions import CompletionOutputKind, CompletionTruncationLimit, LLMCompletionRefusedError, LLMCompletionTruncatedErrorfrom pipelex.cogt.llm import completion_stopfrom pipelex.cogt.llm.completion_stop import (ANTHROPIC_STOP_REASONS,@@ -23,6 +23,7 @@ from pipelex.cogt.llm.completion_stop import (OPENAI_RESPONSES_INCOMPLETE_REASONS,STOP_REASON_VOCABULARIES,CompletionStopOutcome,+ classify_image_finish_reason,classify_prompt_block_reason,classify_responses_stop,classify_stop_reason,@@ -112,6 +113,35 @@ class TestCompletionStop:fields={"model_handle": "gemini-test", "block_reason": "BRAND_NEW_BLOCK"},)+ @pytest.mark.parametrize(+ ("finish_reason", "expected_outcome"),+ [+ ("IMAGE_SAFETY", CompletionStopOutcome.REFUSED),+ ("image_prohibited_content", CompletionStopOutcome.REFUSED),+ ("NO_IMAGE", CompletionStopOutcome.NORMAL),+ ("STOP", CompletionStopOutcome.NORMAL),+ ],+ )+ def test_an_image_finish_reason_reads_from_gemini_vocabulary(+ self, mocker: MockerFixture, finish_reason: str, expected_outcome: CompletionStopOutcome+ ) -> None:+ warning = mocker.patch.object(completion_stop.log, "warning")++ assert classify_image_finish_reason(finish_reason=finish_reason, model_handle="nano-banana") == expected_outcome+ warning.assert_not_called()++ def test_an_unknown_image_finish_reason_is_logged_as_an_image_generations(self, mocker: MockerFixture) -> None:+ """An image job's unknown value warns as an image generation's, never as an LLM's stop reason."""+ warning = mocker.patch.object(completion_stop.log, "warning")++ outcome = classify_image_finish_reason(finish_reason="IMAGE_BRAND_NEW", model_handle="nano-banana")++ assert outcome == CompletionStopOutcome.NORMAL+ warning.assert_called_once_with(+ "An image finish reason was not recognized, so it was taken as normal",+ fields={"model_handle": "nano-banana", "finish_reason": "IMAGE_BRAND_NEW"},+ )+def test_the_vocabularies_agree_on_every_shared_value(self) -> None:"""Read as one table, no two vocabularies may read the same case-folded value differently."""readings: dict[str, set[CompletionStopOutcome]] = {}@@ -162,6 +192,7 @@ class TestCompletionStop:)assert exc_info.value.stop_reason == stop_reasonassert exc_info.value.truncation_limit == expected_limit+ assert exc_info.value.output_kind == CompletionOutputKind.TEXTassert exc_info.value.max_tokens == 4096assert exc_info.value.output_tokens == 4096diff --git a/tests/unit/pipelex/cogt/test_llm_completion_stop_errors.py b/tests/unit/pipelex/cogt/test_llm_completion_stop_errors.pyindex f8c1c3843..083c3b8ae 100644--- a/tests/unit/pipelex/cogt/test_llm_completion_stop_errors.py+++ b/tests/unit/pipelex/cogt/test_llm_completion_stop_errors.py@@ -4,11 +4,13 @@ import pytestfrom pipelex.base_exceptions import DisclosureMode, ErrorDomainfrom pipelex.cogt.exceptions import (+ CompletionOutputKind,CompletionTruncationLimit,InferenceErrorCategory,LLMCompletionError,LLMCompletionRefusedError,LLMCompletionTruncatedError,+ MaxTokensCeiling,)from pipelex.cogt.inference.error_classification import UserActionKindfrom pipelex.pipe_run.exceptions import PipeRouterError@@ -38,6 +40,7 @@ def _truncated_error() -> LLMCompletionTruncatedError:model_handle="claude-5-sonnet",stop_reason="max_tokens",truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.TEXT,pipe_code="ui_designer",max_tokens=4096,output_tokens=4096,@@ -49,6 +52,7 @@ def _context_window_error() -> LLMCompletionTruncatedError:model_handle="claude-5-sonnet",stop_reason="model_context_window_exceeded",truncation_limit=CompletionTruncationLimit.CONTEXT_WINDOW,+ output_kind=CompletionOutputKind.TEXT,pipe_code="ui_designer",max_tokens=4096,output_tokens=812,@@ -74,6 +78,7 @@ class TestLLMCompletionStopErrors:model_handle="gpt-test",stop_reason="length",truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.TEXT,pipe_code="ui_designer",output_tokens=16384,)@@ -98,6 +103,61 @@ class TestLLMCompletionStopErrors:assert error.user_action.detail == CONTEXT_WINDOW_NEXT_STEPassert "Raise" not in error.message+ def test_a_cut_structured_output_is_named_a_structured_output_not_a_text(self) -> None:+ error = LLMCompletionTruncatedError(+ model_handle="claude-5-sonnet",+ stop_reason="max_tokens",+ truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.STRUCTURED,+ pipe_code="ui_designer",+ max_tokens=4096,+ output_tokens=4096,+ )+ without_pipe = LLMCompletionTruncatedError(+ model_handle="gpt-test",+ stop_reason="length",+ truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.STRUCTURED,+ max_tokens=2048,+ )++ assert error.message == (+ "The model 'claude-5-sonnet' was cut off before it finished the structured output of pipe 'ui_designer' "+ "(stop reason 'max_tokens', 4096 output tokens used, max_tokens set to 4096), so the structured output is incomplete. "+ "Raise the pipe's max_tokens, or shorten its input."+ )+ assert without_pipe.message == (+ "The model 'gpt-test' was cut off before it finished its structured output (stop reason 'length', max_tokens set to 2048), "+ "so the structured output is incomplete. Raise max_tokens, or shorten the input."+ )+ assert "text" not in error.message+ assert error.output_kind == CompletionOutputKind.STRUCTURED++ def test_a_limit_lowered_to_fit_the_timeout_is_never_advised_raised(self) -> None:+ """A higher max_tokens is lowered to the same ceiling, so neither the message nor the action asks for one."""+ next_step = "Lower the pipe's reasoning effort, or ask it for a smaller output, since a higher max_tokens is lowered the same way."+ error = LLMCompletionTruncatedError(+ model_handle="claude-5-sonnet",+ stop_reason="max_tokens",+ truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.STRUCTURED,+ pipe_code="ui_designer",+ max_tokens=42666,+ output_tokens=42666,+ max_tokens_ceiling=MaxTokensCeiling(requested_max_tokens=64000, timeout_seconds=1200),+ )++ assert error.message == (+ "The model 'claude-5-sonnet' was cut off before it finished the structured output of pipe 'ui_designer' "+ "(stop reason 'max_tokens', 42666 output tokens used, max_tokens set to 42666), so the structured output is incomplete. "+ f"Its max_tokens was lowered from 64000 to 42666 to fit the call's 1200-second timeout. {next_step}"+ )+ assert error.user_action is not None+ assert error.user_action.kind == UserActionKind.CHANGE_INPUT+ assert error.user_action.detail == next_step+ assert "Raise" not in error.message+ assert error.max_tokens_ceiling == MaxTokensCeiling(requested_max_tokens=64000, timeout_seconds=1200)+def test_the_refusal_names_the_pipe_model_stop_and_next_step(self) -> None:error = _refused_error()@@ -108,13 +168,23 @@ class TestLLMCompletionStopErrors:def test_without_a_pipe_or_counts_the_messages_still_read_as_sentences(self) -> None:truncated = LLMCompletionTruncatedError(- model_handle="gpt-test", stop_reason="length", truncation_limit=CompletionTruncationLimit.MAX_TOKENS, max_tokens=2048+ model_handle="gpt-test",+ stop_reason="length",+ truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.TEXT,+ max_tokens=2048,)truncated_by_default = LLMCompletionTruncatedError(- model_handle="gpt-test", stop_reason="length", truncation_limit=CompletionTruncationLimit.MAX_TOKENS+ model_handle="gpt-test",+ stop_reason="length",+ truncation_limit=CompletionTruncationLimit.MAX_TOKENS,+ output_kind=CompletionOutputKind.TEXT,)context_window = LLMCompletionTruncatedError(- model_handle="mistral-test", stop_reason="model_length", truncation_limit=CompletionTruncationLimit.CONTEXT_WINDOW+ model_handle="mistral-test",+ stop_reason="model_length",+ truncation_limit=CompletionTruncationLimit.CONTEXT_WINDOW,+ output_kind=CompletionOutputKind.TEXT,)refused = LLMCompletionRefusedError(model_handle="gpt-test", stop_reason="refusal")diff --git a/tests/unit/pipelex/providers/google/test_google_factory_stop_signal.py b/tests/unit/pipelex/providers/google/test_google_factory_stop_signal.pynew file mode 100644index 000000000..dbda90908--- /dev/null+++ b/tests/unit/pipelex/providers/google/test_google_factory_stop_signal.py@@ -0,0 +1,42 @@+from __future__ import annotations++from typing import Any++import pytest+from google.genai import types as genai_types++from pipelex.providers.google.google_factory import GeminiStopSignal, GoogleFactory++# Gemini answers as its API sends them, read by the genai SDK as the workers receive them.+_STOP_SIGNAL_CASES: list[Any] = [+ pytest.param(+ {"candidates": [{"finishReason": "IMAGE_SAFETY", "index": 0}]},+ GeminiStopSignal(finish_reason="IMAGE_SAFETY", block_reason=None),+ id="candidate-with-finish-reason",+ ),+ pytest.param(+ {"candidates": [{"content": {"role": "model", "parts": [{"text": "Hello."}]}, "index": 0}]},+ GeminiStopSignal(finish_reason=None, block_reason=None),+ id="candidate-without-finish-reason",+ ),+ pytest.param(+ {"promptFeedback": {"blockReason": "PROHIBITED_CONTENT"}},+ GeminiStopSignal(finish_reason=None, block_reason="PROHIBITED_CONTENT"),+ id="blocked-prompt",+ ),+ # The block reason is read only when no candidate came back, the two vocabularies sharing words with other meanings+ pytest.param(+ {"candidates": [{"finishReason": "STOP", "index": 0}], "promptFeedback": {"blockReason": "SAFETY"}},+ GeminiStopSignal(finish_reason="STOP", block_reason=None),+ id="candidate-and-prompt-feedback",+ ),+ pytest.param({}, GeminiStopSignal(finish_reason=None, block_reason=None), id="no-candidate-and-no-feedback"),+]+++class TestGoogleFactoryStopSignal:+ @pytest.mark.parametrize(("answer", "expected_signal"), _STOP_SIGNAL_CASES)+ def test_the_answer_says_how_it_stopped(self, answer: dict[str, Any], expected_signal: GeminiStopSignal) -> None:+ response = genai_types.GenerateContentResponse.model_validate(answer)++ assert GoogleFactory.read_stop_signal(response=response) == expected_signaldiff --git a/tests/unit/pipelex/providers/google/test_google_img_gen_worker_refusal.py b/tests/unit/pipelex/providers/google/test_google_img_gen_worker_refusal.pynew file mode 100644index 000000000..480eef4fa--- /dev/null+++ b/tests/unit/pipelex/providers/google/test_google_img_gen_worker_refusal.py@@ -0,0 +1,245 @@+from __future__ import annotations++import base64+from typing import TYPE_CHECKING, Any++import httpx+import pytest+from google import genai+from google.genai import types as genai_types+from mthds.protocol.problem.registry import CoreCode++from pipelex.base_exceptions import ErrorDomain+from pipelex.cogt.exceptions import ImgGenGenerationError, ImgGenOutputTruncatedError, InferenceErrorCategory+from pipelex.cogt.img_gen.img_gen_gemini_mapping import ResolvedGeminiImageConfig+from pipelex.cogt.img_gen.img_gen_model_rules import AspectRatioTaxonomy, ImgGenArgTopic+from pipelex.cogt.inference.error_classification import UserActionKind+from pipelex.cogt.llm import completion_stop+from pipelex.providers.google.google_img_gen_worker import GoogleImgGenWorker++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++_MODEL_HANDLE = "nano-banana"+_MODEL_ID = "gemini-2.5-flash-image"+_IMAGE_BYTES = b"\x89PNG\r\n\x1a\nnot-really-a-png"+_USAGE: dict[str, Any] = {"promptTokenCount": 9, "totalTokenCount": 9}++# Gemini's answers on the wire, as its API sends them: a 200 whatever happened to the image.+++def _finished_without_image(finish_reason: str) -> dict[str, Any]:+ """A candidate the model stopped for the reason given, holding no content."""+ return {"candidates": [{"finishReason": finish_reason, "index": 0}], "usageMetadata": _USAGE, "modelVersion": _MODEL_ID}+++def _blocked_prompt(block_reason: str) -> dict[str, Any]:+ """No candidate at all: the prompt was blocked before the model wrote anything, its reason in the prompt feedback."""+ return {"promptFeedback": {"blockReason": block_reason}, "usageMetadata": _USAGE, "modelVersion": _MODEL_ID}+++_NO_CANDIDATE_AND_NO_FEEDBACK: dict[str, Any] = {"usageMetadata": _USAGE, "modelVersion": _MODEL_ID}++_GENERATED_IMAGE: dict[str, Any] = {+ "candidates": [+ {+ "content": {"role": "model", "parts": [{"inlineData": {"mimeType": "image/png", "data": base64.b64encode(_IMAGE_BYTES).decode()}}]},+ "finishReason": "STOP",+ "index": 0,+ }+ ],+ "usageMetadata": _USAGE,+ "modelVersion": _MODEL_ID,+}++# The model finished its image, then reached its output token limit: the image is whole+_GENERATED_IMAGE_AT_THE_LIMIT: dict[str, Any] = {+ **_GENERATED_IMAGE,+ "candidates": [{**_GENERATED_IMAGE["candidates"][0], "finishReason": "MAX_TOKENS"}],+}+++def _worker(mocker: MockerFixture, *, http_client: httpx.AsyncClient) -> GoogleImgGenWorker:+ """A worker over a real genai client, whose every request Gemini answers through the HTTP client given."""+ worker = object.__new__(GoogleImgGenWorker)+ inference_model = mocker.MagicMock()+ inference_model.desc = f"{_MODEL_HANDLE} → Model[{_MODEL_ID}]"+ inference_model.model_id = _MODEL_ID+ inference_model.name = _MODEL_HANDLE+ inference_model.rules = {ImgGenArgTopic.ASPECT_RATIO: AspectRatioTaxonomy.GEMINI_3_FLASH}+ worker.inference_model = inference_model+ client = genai.Client(api_key="unused-in-this-test", http_options=genai_types.HttpOptions(httpx_async_client=http_client))+ worker.genai_async_client = client.aio+ mocker.patch(+ "pipelex.providers.google.google_img_gen_worker.ImgGenGeminiMapping.resolve_image_config",+ return_value=ResolvedGeminiImageConfig(aspect_ratio="1:1", image_size=None, width=1024, height=1024),+ )+ return worker+++def _answering(body: dict[str, Any]) -> httpx.AsyncClient:+ def handle(request: httpx.Request) -> httpx.Response:+ return httpx.Response(status_code=200, json=body, request=request)++ return httpx.AsyncClient(transport=httpx.MockTransport(handle))+++def _img_gen_job(mocker: MockerFixture) -> Any:+ job = mocker.MagicMock()+ job.job_params.aspect_ratio = None+ job.job_params.size = None+ job.img_gen_prompt.positive_text = "a cute cat"+ job.img_gen_prompt.input_images = None+ job.job_report.img_gen_tokens_usage = None+ return job++[email protected](loop_scope="class")+class TestGoogleImgGenWorkerRefusal:+ @pytest.mark.parametrize(+ ("answer", "reason"),+ [+ (_finished_without_image("IMAGE_SAFETY"), "IMAGE_SAFETY"),+ (_finished_without_image("IMAGE_PROHIBITED_CONTENT"), "IMAGE_PROHIBITED_CONTENT"),+ (_finished_without_image("SAFETY"), "SAFETY"),+ (_blocked_prompt("PROHIBITED_CONTENT"), "PROHIBITED_CONTENT"),+ (_blocked_prompt("IMAGE_SAFETY"), "IMAGE_SAFETY"),+ ],+ )+ async def test_a_safety_block_reads_content_refused(self, mocker: MockerFixture, answer: dict[str, Any], reason: str) -> None:+ """A finish or block reason naming a filter is the caller's input refused, not an empty answer."""+ async with _answering(answer) as http_client:+ worker = _worker(mocker, http_client=http_client)+ with pytest.raises(ImgGenGenerationError) as exc_info:+ await worker._gen_image(img_gen_job=_img_gen_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert error.resolve_error_code() == CoreCode.MODEL_CONTENT_REFUSED+ assert error.error_category is InferenceErrorCategory.CONTENT+ assert error.user_action is not None+ assert error.user_action.kind is UserActionKind.CHANGE_INPUT+ assert f"'{reason}'" in error.message+ assert f"'{_MODEL_HANDLE}'" in error.message+ assert _MODEL_ID not in error.message+ report = error.to_error_report()+ assert report.code == CoreCode.MODEL_CONTENT_REFUSED+ assert report.error_domain == ErrorDomain.INPUT+ assert report.http_status == 422+ assert report.caller_facing_message is True++ @pytest.mark.parametrize(+ ("answer", "expected_message"),+ [+ pytest.param(+ _finished_without_image("RECITATION"),+ "The model 'nano-banana' stopped before it generated the image (finish reason 'RECITATION'): "+ "the output may reproduce an existing source too closely. Revise the image prompt or its input images.",+ id="recitation",+ ),+ pytest.param(+ _finished_without_image("IMAGE_RECITATION"),+ "The model 'nano-banana' stopped before it generated the image (finish reason 'IMAGE_RECITATION'): "+ "the output may reproduce an existing source too closely. Revise the image prompt or its input images.",+ id="image-recitation",+ ),+ pytest.param(+ _finished_without_image("LANGUAGE"),+ "The model 'nano-banana' stopped before it generated the image (finish reason 'LANGUAGE'): "+ "the prompt's language is not one the model supports. Revise the image prompt or its input images.",+ id="language",+ ),+ pytest.param(+ _finished_without_image("SPII"),+ "The model 'nano-banana' stopped before it generated the image (finish reason 'SPII'): "+ "the content may hold sensitive personal information. Revise the image prompt or its input images.",+ id="spii",+ ),+ pytest.param(+ _finished_without_image("IMAGE_SAFETY"),+ "The model 'nano-banana' stopped before it generated the image (finish reason 'IMAGE_SAFETY'): "+ "a safety filter flagged the content. Revise the image prompt or its input images.",+ id="image-safety",+ ),+ pytest.param(+ _blocked_prompt("OTHER"),+ "Gemini blocked the prompt before the model 'nano-banana' generated the image (block reason 'OTHER'). "+ "Revise the image prompt or its input images.",+ id="blocked-for-another-reason",+ ),+ ],+ )+ async def test_a_refusal_says_what_its_reason_says(self, mocker: MockerFixture, answer: dict[str, Any], expected_message: str) -> None:+ """A recitation or an unsupported language is named as such, never as a safety filter's block."""+ async with _answering(answer) as http_client:+ worker = _worker(mocker, http_client=http_client)+ with pytest.raises(ImgGenGenerationError) as exc_info:+ await worker._gen_image(img_gen_job=_img_gen_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ assert exc_info.value.resolve_error_code() == CoreCode.MODEL_CONTENT_REFUSED+ assert exc_info.value.message == expected_message++ @pytest.mark.parametrize(+ "answer",+ [+ # The model was asked for an image and made none, for no reason a filter gave+ _finished_without_image("NO_IMAGE"),+ _finished_without_image("IMAGE_OTHER"),+ _NO_CANDIDATE_AND_NO_FEEDBACK,+ ],+ )+ async def test_an_empty_answer_with_no_refusal_stays_unusable(self, mocker: MockerFixture, answer: dict[str, Any]) -> None:+ async with _answering(answer) as http_client:+ worker = _worker(mocker, http_client=http_client)+ with pytest.raises(ImgGenGenerationError) as exc_info:+ await worker._gen_image(img_gen_job=_img_gen_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ assert exc_info.value.resolve_error_code() == CoreCode.PROVIDER_RESPONSE_INVALID++ # genai warns that it does not know the value, and keeps it+ @pytest.mark.filterwarnings("ignore:IMAGE_BRAND_NEW is not a valid FinishReason")+ async def test_an_unknown_finish_reason_is_logged_as_an_image_generations(self, mocker: MockerFixture) -> None:+ """A value Gemini adds later is taken as normal and logged as an image generation's, never as an LLM's stop reason."""+ warning = mocker.patch.object(completion_stop.log, "warning")+ async with _answering(_finished_without_image("IMAGE_BRAND_NEW")) as http_client:+ worker = _worker(mocker, http_client=http_client)+ with pytest.raises(ImgGenGenerationError) as exc_info:+ await worker._gen_image(img_gen_job=_img_gen_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ assert exc_info.value.resolve_error_code() == CoreCode.PROVIDER_RESPONSE_INVALID+ warning.assert_called_once_with(+ "An image finish reason was not recognized, so it was taken as normal",+ fields={"model_handle": _MODEL_HANDLE, "finish_reason": "IMAGE_BRAND_NEW"},+ )++ async def test_an_answer_cut_before_its_image_reads_output_truncated(self, mocker: MockerFixture) -> None:+ """A finish reason of MAX_TOKENS with no image is the cut it states, not an empty answer to rephrase."""+ async with _answering(_finished_without_image("MAX_TOKENS")) as http_client:+ worker = _worker(mocker, http_client=http_client)+ with pytest.raises(ImgGenOutputTruncatedError) as exc_info:+ await worker._gen_image(img_gen_job=_img_gen_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert error.resolve_error_code() == CoreCode.MODEL_OUTPUT_TRUNCATED+ assert error.error_category is InferenceErrorCategory.CONTENT+ assert error.message == (+ "The model 'nano-banana' reached its output token limit before it generated the image (finish reason 'MAX_TOKENS'). "+ "Simplify the image prompt, or choose another image model."+ )+ assert error.user_action is not None+ assert error.user_action.kind is UserActionKind.CHANGE_INPUT+ assert error.user_action.detail == "Simplify the image prompt, or choose another image model."+ report = error.to_error_report()+ assert report.code == CoreCode.MODEL_OUTPUT_TRUNCATED+ assert report.http_status == 422+ assert report.retryable is False+ assert report.caller_facing_message is True++ @pytest.mark.parametrize("answer", [_GENERATED_IMAGE, _GENERATED_IMAGE_AT_THE_LIMIT], ids=["stopped", "stopped-at-the-limit"])+ async def test_a_generated_image_is_returned(self, mocker: MockerFixture, answer: dict[str, Any]) -> None:+ """An image the model finished is returned, even when the output token limit stopped the answer after it."""+ async with _answering(answer) as http_client:+ worker = _worker(mocker, http_client=http_client)+ generated = await worker._gen_image(img_gen_job=_img_gen_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ assert generated.actual_bytes == _IMAGE_BYTES+ assert generated.mime_type == "image/png"diff --git a/tests/unit/pipelex/providers/test_context_window_refusals.py b/tests/unit/pipelex/providers/test_context_window_refusals.pynew file mode 100644index 000000000..7bdc8a5aa--- /dev/null+++ b/tests/unit/pipelex/providers/test_context_window_refusals.py@@ -0,0 +1,481 @@+from __future__ import annotations++from typing import TYPE_CHECKING, Any++import anthropic+import botocore.session+import httpx+import openai+import pytest+from botocore.exceptions import ClientError+from botocore.stub import Stubber+from google.genai import errors as genai_errors+from mistralai.client import Mistral+from mistralai.client.errors import MistralError+from mistralai.client.models import UserMessage+from mthds.protocol.problem.registry import CoreCode++from pipelex.base_exceptions import ErrorDomain+from pipelex.cogt.exceptions import (+ CogtError,+ ExtractJobFailureError,+ ImgGenGenerationError,+ InferenceErrorCategory,+ JudgmentJobFailureError,+ LLMCompletionError,+ SearchJobFailureError,+)+from pipelex.cogt.inference.error_classification import (+ ProviderErrorMetadata,+ UserActionKind,+ extract_anthropic_metadata,+ extract_bedrock_metadata,+ extract_google_metadata,+ extract_mistral_metadata,+ extract_openai_metadata,+)+from pipelex.cogt.inference.error_classify import InferenceErrorFamily, classify_inference_error+from pipelex.cogt.inference.error_render import render_inference_error+from pipelex.cogt.inference.provider_name import ProviderName+from pipelex.providers.openai.openai_completions_llm_worker import OpenAICompletionsLLMWorker+from tests.helpers.completion_stop import make_text_llm_job++if TYPE_CHECKING:+ from collections.abc import Callable++ from pytest_mock import MockerFixture++ from pipelex.errors.error_codes import ErrorCode++_ORIGIN = "https://provider.example.com"++# The body of a provider's 400, as its API sends it.+_Body = dict[str, Any]++# Each provider's 400 for a prompt over the model's context window, worded as the provider words it.+_OPENAI_CONTEXT_LENGTH: _Body = {+ "error": {+ "message": (+ "This model's maximum context length is 128000 tokens. However, your messages resulted in 130412 tokens. "+ "Please reduce the length of the messages."+ ),+ "type": "invalid_request_error",+ "param": "messages",+ "code": "context_length_exceeded",+ }+}+_OPENAI_CONTEXT_WINDOW: _Body = {+ "error": {+ "message": "Your input exceeds the context window of this model. Please adjust your input and try again.",+ "type": "invalid_request_error",+ "param": "input",+ "code": "context_length_exceeded",+ }+}+# OpenAI's refusal when the messages and the completion the request reserves with max_tokens together pass the window+_OPENAI_MESSAGES_AND_COMPLETION: _Body = {+ "error": {+ "message": (+ "This model's maximum context length is 128000 tokens. However, you requested 130000 tokens "+ "(10000 in the messages, 120000 in the completion). Please reduce the length of the messages or completion."+ ),+ "type": "invalid_request_error",+ "param": "messages",+ "code": "context_length_exceeded",+ }+}+_ANTHROPIC_PROMPT_TOO_LONG: _Body = {+ "type": "error",+ "error": {"type": "invalid_request_error", "message": "prompt is too long: 208310 tokens > 200000 maximum"},+}+_ANTHROPIC_INPUT_AND_MAX_TOKENS: _Body = {+ "type": "error",+ "error": {+ "type": "invalid_request_error",+ "message": "input length and `max_tokens` exceed context limit: 197202 + 8192 > 200000, decrease input length or `max_tokens` and try again",+ },+}+_MISTRAL_PROMPT_TOO_LARGE: _Body = {+ "object": "error",+ "message": "Prompt contains 140000 tokens and 0 draft tokens, too large for model with 131072 maximum context length",+ "type": "invalid_request_message_error",+ "param": None,+ "code": "3051",+}+_GEMINI_INPUT_TOKEN_COUNT: _Body = {+ "error": {+ "code": 400,+ "message": "The input token count (1234567) exceeds the maximum number of tokens allowed (1048576).",+ "status": "INVALID_ARGUMENT",+ }+}+_BEDROCK_INPUT_TOO_LONG = "Input is too long for requested model."++# The same providers' 400s for a request of ours they could not take, which say nothing about the context window.+_OPENAI_INVALID_TYPE: _Body = {+ "error": {+ "message": "Invalid type for 'messages[0].content': expected one of a string or array of objects, but got an integer instead.",+ "type": "invalid_request_error",+ "param": "messages[0].content",+ "code": "invalid_type",+ }+}+_ANTHROPIC_FIELD_REQUIRED: _Body = {"type": "error", "error": {"type": "invalid_request_error", "message": "max_tokens: Field required"}}+_MISTRAL_INVALID_MESSAGE: _Body = {+ "object": "error",+ "message": "Assistant message must have either content or tool_calls, but not none.",+ "type": "invalid_request_message_error",+ "param": None,+ "code": "3230",+}+_GEMINI_INVALID_PAYLOAD: _Body = {+ "error": {"code": 400, "message": 'Invalid JSON payload received. Unknown name "foo": Cannot find field.', "status": "INVALID_ARGUMENT"}+}+_BEDROCK_INVALID_MODEL = "The provided model identifier is invalid."+# Anthropic's 413 for a request over its payload size, which says nothing about the context window+_ANTHROPIC_REQUEST_TOO_LARGE: _Body = {+ "type": "error",+ "error": {"type": "request_too_large", "message": "Request exceeds the maximum allowed number of bytes."},+}+++def _answering(*, body: _Body, status_code: int = 400) -> httpx.MockTransport:+ """A transport whose every request the provider answers with the status, a 400 unless told, carrying the body."""++ def handle(request: httpx.Request) -> httpx.Response:+ return httpx.Response(status_code=status_code, json=body, request=request)++ return httpx.MockTransport(handle)+++def _openai_refusal(body: _Body) -> BaseException:+ with httpx.Client(transport=_answering(body=body)) as http_client:+ client = openai.OpenAI(api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1", max_retries=0, http_client=http_client)+ with pytest.raises(openai.BadRequestError) as exc_info:+ client.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": "Summarize the contract."}])+ return exc_info.value+++def _anthropic_refusal(body: _Body) -> BaseException:+ with httpx.Client(transport=_answering(body=body)) as http_client:+ client = anthropic.Anthropic(api_key="unused-in-this-test", base_url=_ORIGIN, max_retries=0, http_client=http_client)+ with pytest.raises(anthropic.BadRequestError) as exc_info:+ client.messages.create(model="claude-sonnet-4-5", max_tokens=8192, messages=[{"role": "user", "content": "Summarize the contract."}])+ return exc_info.value+++def _openai_compatible_refusal(*, body: _Body, status_code: int) -> BaseException:+ """An OpenAI-compatible server, a gateway, answering with another 4xx than 400, raised by the OpenAI SDK."""+ with httpx.Client(transport=_answering(body=body, status_code=status_code)) as http_client:+ client = openai.OpenAI(api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1", max_retries=0, http_client=http_client)+ with pytest.raises(openai.APIStatusError) as exc_info:+ client.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": "Summarize the contract."}])+ return exc_info.value+++def _anthropic_status_refusal(*, body: _Body, status_code: int) -> BaseException:+ with httpx.Client(transport=_answering(body=body, status_code=status_code)) as http_client:+ client = anthropic.Anthropic(api_key="unused-in-this-test", base_url=_ORIGIN, max_retries=0, http_client=http_client)+ with pytest.raises(anthropic.APIStatusError) as exc_info:+ client.messages.create(model="claude-sonnet-4-5", max_tokens=8192, messages=[{"role": "user", "content": "Summarize the contract."}])+ return exc_info.value+++def _mistral_refusal(body: _Body) -> BaseException:+ with httpx.Client(transport=_answering(body=body)) as http_client:+ client = Mistral(api_key="unused-in-this-test", server_url=_ORIGIN, client=http_client)+ with pytest.raises(MistralError) as exc_info:+ client.chat.complete(model="mistral-large-latest", messages=[UserMessage(content="Summarize the contract.")])+ return exc_info.value+++def _gemini_refusal(body: _Body) -> BaseException:+ request = httpx.Request("POST", "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro:generateContent")+ with pytest.raises(genai_errors.ClientError) as exc_info:+ genai_errors.APIError.raise_for_response(httpx.Response(status_code=400, json=body, request=request)) # pyright: ignore[reportUnknownMemberType]+ return exc_info.value+++def _bedrock_refusal(message: str) -> BaseException:+ """Bedrock's Converse refusal with the message, raised by a real bedrock-runtime client from a stubbed answer."""+ # The bedrock-runtime client botocore builds from its service model, whose operations it types as nothing+ client: Any = botocore.session.get_session().create_client(+ "bedrock-runtime", region_name="us-east-1", aws_access_key_id="unused-in-this-test", aws_secret_access_key="unused-in-this-test"+ )+ stubber = Stubber(client)+ stubber.add_client_error("converse", service_error_code="ValidationException", service_message=message, http_status_code=400)+ with stubber, pytest.raises(ClientError) as exc_info:+ client.converse(modelId="anthropic.claude-sonnet-4-5", messages=[{"role": "user", "content": [{"text": "Summarize the contract."}]}])+ return exc_info.value+++# Each case: the provider's SDK raising its 400 from the provider's answer, the Extract step reading it, and the code+# the 400 must read.+_REFUSALS = [+ pytest.param(+ _openai_refusal, _OPENAI_CONTEXT_LENGTH, extract_openai_metadata, CoreCode.MODEL_CONTEXT_EXCEEDED, id="openai-maximum-context-length"+ ),+ pytest.param(_openai_refusal, _OPENAI_CONTEXT_WINDOW, extract_openai_metadata, CoreCode.MODEL_CONTEXT_EXCEEDED, id="openai-context-window"),+ pytest.param(+ _openai_refusal,+ _OPENAI_MESSAGES_AND_COMPLETION,+ extract_openai_metadata,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ id="openai-messages-and-completion",+ ),+ pytest.param(+ _anthropic_refusal, _ANTHROPIC_PROMPT_TOO_LONG, extract_anthropic_metadata, CoreCode.MODEL_CONTEXT_EXCEEDED, id="anthropic-prompt-too-long"+ ),+ pytest.param(+ _anthropic_refusal,+ _ANTHROPIC_INPUT_AND_MAX_TOKENS,+ extract_anthropic_metadata,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ id="anthropic-input-and-max-tokens",+ ),+ pytest.param(+ _mistral_refusal, _MISTRAL_PROMPT_TOO_LARGE, extract_mistral_metadata, CoreCode.MODEL_CONTEXT_EXCEEDED, id="mistral-prompt-too-large"+ ),+ pytest.param(_gemini_refusal, _GEMINI_INPUT_TOKEN_COUNT, extract_google_metadata, CoreCode.MODEL_CONTEXT_EXCEEDED, id="gemini-input-token-count"),+ pytest.param(_bedrock_refusal, _BEDROCK_INPUT_TOO_LONG, extract_bedrock_metadata, CoreCode.MODEL_CONTEXT_EXCEEDED, id="bedrock-input-too-long"),+ pytest.param(_openai_refusal, _OPENAI_INVALID_TYPE, extract_openai_metadata, CoreCode.RUNTIME_UNEXPECTED, id="openai-invalid-type"),+ pytest.param(+ _anthropic_refusal, _ANTHROPIC_FIELD_REQUIRED, extract_anthropic_metadata, CoreCode.RUNTIME_UNEXPECTED, id="anthropic-field-required"+ ),+ pytest.param(_mistral_refusal, _MISTRAL_INVALID_MESSAGE, extract_mistral_metadata, CoreCode.RUNTIME_UNEXPECTED, id="mistral-invalid-message"),+ pytest.param(_gemini_refusal, _GEMINI_INVALID_PAYLOAD, extract_google_metadata, CoreCode.RUNTIME_UNEXPECTED, id="gemini-invalid-payload"),+ pytest.param(_bedrock_refusal, _BEDROCK_INVALID_MODEL, extract_bedrock_metadata, CoreCode.RUNTIME_UNEXPECTED, id="bedrock-invalid-model"),+]+++class TestContextWindowRefusals:+ @pytest.mark.parametrize(("raise_refusal", "answer", "extract_metadata", "expected_error_code"), _REFUSALS)+ def test_a_provider_400_reads_context_exceeded_only_when_it_says_so(+ self,+ raise_refusal: Callable[[Any], BaseException],+ answer: Any,+ extract_metadata: Callable[[BaseException], ProviderErrorMetadata],+ expected_error_code: ErrorCode,+ ) -> None:+ """The provider's own 400, extracted, classified and rendered as a worker does, carries the code its wording earns."""+ sdk_exc = raise_refusal(answer)+ metadata = extract_metadata(sdk_exc)+ assert metadata.status_code == 400++ classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)+ rendered = render_inference_error(+ metadata=metadata,+ classification=classification,+ family=InferenceErrorFamily.LLM,+ model_desc="test-model-desc",+ model_handle="test-model",+ )++ assert classification.category is InferenceErrorCategory.CONTENT+ assert classification.user_action_kind is UserActionKind.CHANGE_INPUT+ assert classification.error_code == expected_error_code+ assert isinstance(rendered, LLMCompletionError)+ assert rendered.resolve_error_code() == expected_error_code+ assert rendered.user_action is not None+ assert ("context window" in rendered.user_action.detail) is (expected_error_code == CoreCode.MODEL_CONTEXT_EXCEEDED)++ @pytest.mark.parametrize(+ ("raise_refusal", "answer", "extract_metadata"),+ [+ pytest.param(_anthropic_refusal, _ANTHROPIC_INPUT_AND_MAX_TOKENS, extract_anthropic_metadata, id="anthropic-input-and-max-tokens"),+ pytest.param(_openai_refusal, _OPENAI_MESSAGES_AND_COMPLETION, extract_openai_metadata, id="openai-messages-and-completion"),+ pytest.param(_anthropic_refusal, _ANTHROPIC_PROMPT_TOO_LONG, extract_anthropic_metadata, id="anthropic-prompt-too-long"),+ ],+ )+ def test_a_language_models_window_advice_names_the_input_and_max_tokens(+ self,+ raise_refusal: Callable[[Any], BaseException],+ answer: Any,+ extract_metadata: Callable[[BaseException], ProviderErrorMetadata],+ ) -> None:+ """The window holds the prompt and the output max_tokens reserves, which may be the deck's default: either can be reduced."""+ metadata = extract_metadata(raise_refusal(answer))+ classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)+ rendered = render_inference_error(+ metadata=metadata,+ classification=classification,+ family=InferenceErrorFamily.LLM,+ model_desc="test-model-desc",+ model_handle="test-model",+ )++ assert rendered.resolve_error_code() == CoreCode.MODEL_CONTEXT_EXCEEDED+ assert rendered.user_action is not None+ assert rendered.user_action.detail == (+ "The prompt plus the output requested with max_tokens exceed the model's context window — shorten the input "+ "or lower max_tokens, or choose a model with a larger context window."+ )++ @pytest.mark.parametrize(+ ("family", "expected_class", "expected_detail"),+ [+ (+ InferenceErrorFamily.IMG_GEN,+ ImgGenGenerationError,+ "The prompt or its input images exceed what the image model can read — shorten the prompt or send fewer images.",+ ),+ (+ InferenceErrorFamily.EXTRACT,+ ExtractJobFailureError,+ "The document exceeds what the extraction model can read — send a shorter document, or split it.",+ ),+ (+ InferenceErrorFamily.SEARCH,+ SearchJobFailureError,+ "The query built from the inputs exceeds what the search model can read — shorten the inputs.",+ ),+ (+ InferenceErrorFamily.JUDGMENT,+ JudgmentJobFailureError,+ "The material to judge exceeds what the judgment model can read — shorten it.",+ ),+ ],+ )+ @pytest.mark.parametrize(+ ("raise_refusal", "answer", "extract_metadata"),+ [+ pytest.param(_gemini_refusal, _GEMINI_INPUT_TOKEN_COUNT, extract_google_metadata, id="gemini-input-token-count"),+ pytest.param(_anthropic_refusal, _ANTHROPIC_PROMPT_TOO_LONG, extract_anthropic_metadata, id="anthropic-prompt-too-long"),+ pytest.param(_bedrock_refusal, _BEDROCK_INPUT_TOO_LONG, extract_bedrock_metadata, id="bedrock-input-too-long"),+ ],+ )+ def test_another_familys_advice_fits_what_its_model_reads(+ self,+ raise_refusal: Callable[[Any], BaseException],+ answer: Any,+ extract_metadata: Callable[[BaseException], ProviderErrorMetadata],+ family: InferenceErrorFamily,+ expected_class: type[CogtError],+ expected_detail: str,+ ) -> None:+ """The inputs exceed what the model can read, whatever it does, but only a language model has max_tokens or a window to choose."""+ metadata = extract_metadata(raise_refusal(answer))+ classification = classify_inference_error(metadata, family=family)+ rendered = render_inference_error(+ metadata=metadata,+ classification=classification,+ family=family,+ model_desc="test-model-desc",+ model_handle="test-model",+ )++ assert isinstance(rendered, expected_class)+ assert rendered.resolve_error_code() == CoreCode.MODEL_CONTEXT_EXCEEDED+ assert rendered.user_action is not None+ assert rendered.user_action.kind is UserActionKind.CHANGE_INPUT+ assert rendered.user_action.detail == expected_detail+ assert "context window" not in rendered.user_action.detail+ assert "max_tokens" not in rendered.user_action.detail++ @pytest.mark.parametrize(+ ("raise_refusal", "extract_metadata", "expected_error_code"),+ [+ pytest.param(+ lambda: _openai_compatible_refusal(body=_OPENAI_CONTEXT_LENGTH, status_code=422),+ extract_openai_metadata,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ id="gateway-422-context-length",+ ),+ pytest.param(+ lambda: _openai_compatible_refusal(body=_OPENAI_CONTEXT_LENGTH, status_code=413),+ extract_openai_metadata,+ CoreCode.MODEL_CONTEXT_EXCEEDED,+ id="gateway-413-context-length",+ ),+ pytest.param(+ lambda: _anthropic_status_refusal(body=_ANTHROPIC_REQUEST_TOO_LARGE, status_code=413),+ extract_anthropic_metadata,+ CoreCode.INPUT_TOO_LARGE,+ id="anthropic-413-payload-size",+ ),+ ],+ )+ def test_another_4xx_reads_the_window_as_a_400_does(+ self,+ raise_refusal: Callable[[], BaseException],+ extract_metadata: Callable[[BaseException], ProviderErrorMetadata],+ expected_error_code: ErrorCode,+ ) -> None:+ """A gateway may answer the window's refusal with a 422 or a 413; a 413 over the payload size alone keeps its reading."""+ metadata = extract_metadata(raise_refusal())+ assert metadata.status_code != 400++ classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)+ rendered = render_inference_error(+ metadata=metadata,+ classification=classification,+ family=InferenceErrorFamily.LLM,+ model_desc="test-model-desc",+ model_handle="test-model",+ )++ assert classification.error_code == expected_error_code+ assert classification.user_action_kind is UserActionKind.CHANGE_INPUT+ assert rendered.resolve_error_code() == expected_error_code+ assert rendered.user_action is not None+ assert ("context window" in rendered.user_action.detail) is (expected_error_code == CoreCode.MODEL_CONTEXT_EXCEEDED)++ @pytest.mark.parametrize(+ ("status_code", "sdk_exception_type"),+ [+ (500, "InternalServerError"),+ (None, "ValidationError"),+ ],+ )+ def test_the_same_words_on_another_answer_never_read_as_the_window(self, status_code: int | None, sdk_exception_type: str) -> None:+ """Only a 4xx is read for the window: the words on a 5xx, or quoted in a parse failure, keep their own reading."""+ metadata = ProviderErrorMetadata(+ provider=ProviderName.OPENAI,+ sdk_exception_type=sdk_exception_type,+ message="prompt is too long: 208310 tokens > 200000 maximum",+ status_code=status_code,+ )++ classification = classify_inference_error(metadata, family=InferenceErrorFamily.LLM)+ rendered = render_inference_error(+ metadata=metadata,+ classification=classification,+ family=InferenceErrorFamily.LLM,+ model_desc="test-model-desc",+ model_handle="test-model",+ )++ assert classification.error_code != CoreCode.MODEL_CONTEXT_EXCEEDED+ assert rendered.user_action is not None+ assert "context window" not in rendered.user_action.detail++ @pytest.mark.asyncio+ async def test_the_worker_reports_the_prompt_over_the_window_as_the_callers_input(self, mocker: MockerFixture) -> None:+ """End to end through a worker: the run report reads `model.context_exceeded`, the caller's to fix, answered 422."""+ worker = object.__new__(OpenAICompletionsLLMWorker)+ inference_model = mocker.MagicMock()+ inference_model.name = "gpt-4o"+ inference_model.model_id = "gpt-4o-2024-08-06"+ inference_model.desc = "gpt-4o → Model[gpt-4o-2024-08-06]"+ inference_model.thinking_mode = None+ inference_model.accepts_temperature = True+ inference_model.listed_constraints = []+ worker.inference_model = inference_model+ completions_factory = mocker.MagicMock()+ completions_factory.make_simple_messages = mocker.AsyncMock(return_value=[{"role": "user", "content": "Summarize the contract."}])+ completions_factory.make_extras = mocker.MagicMock(return_value=({}, {}))+ worker.openai_completions_factory = completions_factory++ async with httpx.AsyncClient(transport=_answering(body=_OPENAI_CONTEXT_LENGTH)) as http_client:+ worker.openai_client_for_text = openai.AsyncOpenAI(+ api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1", max_retries=0, http_client=http_client+ )+ with pytest.raises(LLMCompletionError) as exc_info:+ await worker._gen_text(llm_job=make_text_llm_job()) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ report = exc_info.value.to_error_report()+ assert report.code == CoreCode.MODEL_CONTEXT_EXCEEDED+ assert report.error_domain == ErrorDomain.INPUT+ assert report.retryable is False+ assert report.http_status == 422+ assert isinstance(exc_info.value.__cause__, openai.BadRequestError)diff --git a/tests/unit/pipelex/providers/test_instructor_incomplete_output.py b/tests/unit/pipelex/providers/test_instructor_incomplete_output.pynew file mode 100644index 000000000..4a5403921--- /dev/null+++ b/tests/unit/pipelex/providers/test_instructor_incomplete_output.py@@ -0,0 +1,334 @@+# A structured output cut at its output limit reads `model.output_truncated`, driven through instructor's real loop.+#+# instructor reads the stop value of every structured answer it parses and raises its `IncompleteOutputException` for+# one cut at the request's output limit: an OpenAI or Mistral completion whose first choice finished on `length`, an+# Anthropic message that stopped on `max_tokens`. Its re-asks never retry it and it is never wrapped in an+# `InstructorRetryException`, so it escaped every worker as an exception nothing classified, `runtime.unexpected`.+#+# This module drives each worker whose structuring mode can raise it, the Anthropic, OpenAI completions and Mistral+# workers, over a real SDK client whose transport is an `httpx.MockTransport` answering with the provider's own cut+# answer, and through instructor's real loop with the schema-only retrying the workers pass. Gemini's and the Responses+# API's structuring modes never raise it, so their workers are not driven here.++from __future__ import annotations++import json+from typing import TYPE_CHECKING, Any++import anthropic+import httpx+import openai+import pytest+from instructor import Mode, from_anthropic, from_mistral, from_openai+from instructor.core import IncompleteOutputException+from mistralai.client import Mistral+from mthds.protocol.problem.registry import CoreCode++from pipelex.cogt.exceptions import CompletionOutputKind, CompletionTruncationLimit, LLMCompletionTruncatedError, MaxTokensCeiling+from pipelex.cogt.llm.instructor_retry import make_instructor_schema_retrying+from pipelex.providers.anthropic.anthropic_llm_worker import AnthropicLLMWorker+from pipelex.providers.mistral.mistral_llm_worker import MistralLLMWorker+from pipelex.providers.openai.openai_completions_llm_worker import OpenAICompletionsLLMWorker+from tests.helpers.completion_stop import STOP_TEST_PARTIAL_TEXT, STOP_TEST_PIPE_CODE, make_text_llm_job+from tests.helpers.instructor_test_utils import DummySchema++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++ from pipelex.cogt.llm.llm_job import LLMJob++_ORIGIN = "https://provider.example.com"+_MAX_TOKENS = 64+_OUTPUT_TOKENS = 64+_MAX_ATTEMPTS = 2++# A scripted HTTP answer: the status and the JSON body.+_ScriptedResponse = tuple[int, dict[str, Any]]++# The tool call's arguments as the model left them when the limit cut it: JSON that does not close.+_CUT_ARGUMENTS = '{"text": "' + STOP_TEST_PARTIAL_TEXT++_OPENAI_CUT_ANSWER: _ScriptedResponse = (+ 200,+ {+ "id": "chatcmpl-cut",+ "object": "chat.completion",+ "created": 0,+ "model": "some-model",+ "choices": [+ {+ "index": 0,+ "finish_reason": "length",+ "message": {+ "role": "assistant",+ "content": None,+ "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "DummySchema", "arguments": _CUT_ARGUMENTS}}],+ },+ }+ ],+ "usage": {"prompt_tokens": 12, "completion_tokens": _OUTPUT_TOKENS, "total_tokens": 12 + _OUTPUT_TOKENS},+ },+)++_ANTHROPIC_CUT_ANSWER: _ScriptedResponse = (+ 200,+ {+ "id": "msg_cut",+ "type": "message",+ "role": "assistant",+ "model": "some-model",+ "content": [{"type": "tool_use", "id": "toolu_1", "name": "DummySchema", "input": {}}],+ "stop_reason": "max_tokens",+ "stop_sequence": None,+ "usage": {"input_tokens": 12, "output_tokens": _OUTPUT_TOKENS},+ },+)++_MISTRAL_CUT_ANSWER: _ScriptedResponse = (+ 200,+ {+ "id": "cmpl-cut",+ "object": "chat.completion",+ "created": 0,+ "model": "mistral-large-latest",+ "choices": [+ {+ "index": 0,+ "finish_reason": "length",+ "message": {+ "role": "assistant",+ "content": "",+ "tool_calls": [{"id": "call1", "type": "function", "function": {"name": "DummySchema", "arguments": _CUT_ARGUMENTS}}],+ },+ }+ ],+ "usage": {"prompt_tokens": 12, "completion_tokens": _OUTPUT_TOKENS, "total_tokens": 12 + _OUTPUT_TOKENS},+ },+)++# An answer whose tool call fails `DummySchema`, which requires `text`: instructor re-asks it.+_OPENAI_INVALID_ANSWER: _ScriptedResponse = (+ 200,+ {+ "id": "chatcmpl-invalid",+ "object": "chat.completion",+ "created": 0,+ "model": "some-model",+ "choices": [+ {+ "index": 0,+ "finish_reason": "tool_calls",+ "message": {+ "role": "assistant",+ "content": None,+ "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "DummySchema", "arguments": json.dumps({"wrong": 1})}}],+ },+ }+ ],+ "usage": {"prompt_tokens": 12, "completion_tokens": 4, "total_tokens": 16},+ },+)+++class _ScriptedTransport:+ """Answers each request with the next scripted response, and counts the requests."""++ def __init__(self, responses: list[_ScriptedResponse]) -> None:+ self.responses = list(responses)+ self.nb_requests = 0++ def handle(self, request: httpx.Request) -> httpx.Response:+ status_code, body = self.responses[self.nb_requests]+ self.nb_requests += 1+ return httpx.Response(status_code=status_code, json=body, request=request)++ def make_http_client(self) -> httpx.AsyncClient:+ return httpx.AsyncClient(transport=httpx.MockTransport(self.handle))+++def _assert_names_a_structured_output(error: LLMCompletionTruncatedError) -> None:+ """The error says a structured output was cut, never a text."""+ assert error.output_kind == CompletionOutputKind.STRUCTURED+ assert f"finished the structured output of pipe '{STOP_TEST_PIPE_CODE}'" in error.message+ assert "so the structured output is incomplete" in error.message+ assert "text" not in error.message+++def _structured_job(*, max_tokens: int | None) -> LLMJob:+ """A real job for the pipe the stop tests name, re-asking once, as a structured output's job does."""+ llm_job = make_text_llm_job(max_tokens=max_tokens)+ llm_job.job_config.schema_reask_max_attempts = _MAX_ATTEMPTS+ return llm_job+++def _inference_model(mocker: MockerFixture, *, name: str, model_id: str) -> Any:+ inference_model = mocker.MagicMock()+ inference_model.name = name+ inference_model.model_id = model_id+ inference_model.desc = f"{name} → Model[{model_id}]"+ inference_model.thinking_mode = None+ inference_model.accepts_temperature = True+ inference_model.max_tokens = 4096+ inference_model.listed_constraints = []+ return inference_model+++def _openai_worker(mocker: MockerFixture, *, http_client: httpx.AsyncClient) -> OpenAICompletionsLLMWorker:+ worker = object.__new__(OpenAICompletionsLLMWorker)+ worker.inference_model = _inference_model(mocker, name="gpt-4o", model_id="gpt-4o-2024-08-06")+ completions_factory = mocker.MagicMock()+ completions_factory.make_simple_messages = mocker.AsyncMock(return_value=[{"role": "user", "content": "Summarize the contract."}])+ completions_factory.make_extras = mocker.MagicMock(return_value=({}, {}))+ worker.openai_completions_factory = completions_factory+ client = openai.AsyncOpenAI(api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1", max_retries=0, http_client=http_client)+ worker.instructor_for_objects = from_openai(client, mode=Mode.TOOLS)+ return worker+++def _anthropic_worker(mocker: MockerFixture, *, http_client: httpx.AsyncClient) -> AnthropicLLMWorker:+ worker = object.__new__(AnthropicLLMWorker)+ worker.extras_factory = None+ worker.inference_model = _inference_model(mocker, name="claude-4.5-sonnet", model_id="claude-sonnet-4-5-20250929")+ worker.default_max_tokens = 4096+ client = anthropic.AsyncAnthropic(api_key="unused-in-this-test", base_url=_ORIGIN, max_retries=0, http_client=http_client)+ worker.instructor_for_objects = from_anthropic(client)+ config = mocker.MagicMock()+ config.inference.llm.anthropic.structured_output_timeout_seconds = 1200+ mocker.patch("pipelex.providers.anthropic.anthropic_llm_worker.get_config", return_value=config)+ mocker.patch(+ "pipelex.providers.anthropic.anthropic_llm_worker.AnthropicFactory.make_simple_messages",+ new=mocker.AsyncMock(return_value=[{"role": "user", "content": "Summarize the contract."}]),+ )+ return worker+++def _mistral_worker(mocker: MockerFixture, *, http_client: httpx.AsyncClient) -> MistralLLMWorker:+ worker = object.__new__(MistralLLMWorker)+ worker.inference_model = _inference_model(mocker, name="mistral-large", model_id="mistral-large-latest")+ worker.default_max_tokens = 4096+ mistral_factory = mocker.MagicMock()+ mistral_factory.make_simple_messages_openai_typed = mocker.AsyncMock(return_value=[{"role": "user", "content": "Summarize the contract."}])+ worker.mistral_factory = mistral_factory+ client = Mistral(api_key="unused-in-this-test", server_url=_ORIGIN, async_client=http_client)+ worker.instructor_for_objects = from_mistral(client=client, mode=Mode.TOOLS, use_async=True)+ return worker++[email protected](loop_scope="class")+class TestInstructorIncompleteOutput:+ async def test_instructor_raises_the_cut_answer_unwrapped(self) -> None:+ """The shape the workers catch: instructor's own exception, carrying the cut answer, never its retry wrapper."""+ transport = _ScriptedTransport([_OPENAI_CUT_ANSWER])+ async with transport.make_http_client() as http_client:+ client = openai.AsyncOpenAI(api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1", max_retries=0, http_client=http_client)+ structured = from_openai(client, mode=Mode.TOOLS)+ with pytest.raises(IncompleteOutputException) as exc_info:+ await structured.chat.completions.create_with_completion(+ model="some-model",+ messages=[{"role": "user", "content": "Summarize the contract."}],+ response_model=DummySchema,+ max_retries=make_instructor_schema_retrying(max_attempts=_MAX_ATTEMPTS),+ )++ assert transport.nb_requests == 1+ assert type(exc_info.value) is IncompleteOutputException+ last_completion = exc_info.value.last_completion+ assert last_completion is not None+ assert last_completion.choices[0].finish_reason == "length"++ async def test_the_openai_completions_worker_reads_the_cut_as_output_truncated(self, mocker: MockerFixture) -> None:+ transport = _ScriptedTransport([_OPENAI_CUT_ANSWER])+ async with transport.make_http_client() as http_client:+ worker = _openai_worker(mocker, http_client=http_client)+ with pytest.raises(LLMCompletionTruncatedError) as exc_info:+ await worker._gen_object(llm_job=_structured_job(max_tokens=_MAX_TOKENS), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert transport.nb_requests == 1+ assert error.resolve_error_code() == CoreCode.MODEL_OUTPUT_TRUNCATED+ assert error.truncation_limit == CompletionTruncationLimit.MAX_TOKENS+ assert error.stop_reason == "length"+ assert error.pipe_code == STOP_TEST_PIPE_CODE+ assert error.max_tokens == _MAX_TOKENS+ assert error.output_tokens == _OUTPUT_TOKENS+ assert isinstance(error.__cause__, IncompleteOutputException)+ assert STOP_TEST_PARTIAL_TEXT not in error.message+ _assert_names_a_structured_output(error)++ async def test_a_cut_after_a_re_ask_is_still_the_truncation(self, mocker: MockerFixture) -> None:+ """A re-asked answer that comes back cut ends the loop with the same exception, which no wrapper hides."""+ transport = _ScriptedTransport([_OPENAI_INVALID_ANSWER, _OPENAI_CUT_ANSWER])+ async with transport.make_http_client() as http_client:+ worker = _openai_worker(mocker, http_client=http_client)+ with pytest.raises(LLMCompletionTruncatedError) as exc_info:+ await worker._gen_object(llm_job=_structured_job(max_tokens=None), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert transport.nb_requests == 2+ assert error.resolve_error_code() == CoreCode.MODEL_OUTPUT_TRUNCATED+ # The request sent no max_tokens, so the next step sets one rather than raising it+ assert error.max_tokens is None+ assert "no max_tokens sent" in error.message++ async def test_the_anthropic_worker_reads_the_cut_as_output_truncated(self, mocker: MockerFixture) -> None:+ transport = _ScriptedTransport([_ANTHROPIC_CUT_ANSWER])+ async with transport.make_http_client() as http_client:+ worker = _anthropic_worker(mocker, http_client=http_client)+ with pytest.raises(LLMCompletionTruncatedError) as exc_info:+ await worker._gen_object(llm_job=_structured_job(max_tokens=_MAX_TOKENS), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert transport.nb_requests == 1+ assert error.resolve_error_code() == CoreCode.MODEL_OUTPUT_TRUNCATED+ assert error.stop_reason == "max_tokens"+ assert error.pipe_code == STOP_TEST_PIPE_CODE+ assert error.max_tokens == _MAX_TOKENS+ assert error.output_tokens == _OUTPUT_TOKENS+ assert isinstance(error.__cause__, IncompleteOutputException)+ _assert_names_a_structured_output(error)+ # The limit sent is the one asked for, so raising it is the next step+ assert error.max_tokens_ceiling is None+ assert error.user_action is not None+ assert error.user_action.detail == "Raise the pipe's max_tokens, or shorten its input."++ async def test_the_anthropic_worker_says_its_timeout_lowered_the_limit_that_cut(self, mocker: MockerFixture) -> None:+ """A limit the structured call lowered to fit its timeout can explain the cut, so the error says so."""+ transport = _ScriptedTransport([_ANTHROPIC_CUT_ANSWER])+ async with transport.make_http_client() as http_client:+ worker = _anthropic_worker(mocker, http_client=http_client)+ mocker.patch(+ "pipelex.providers.anthropic.anthropic_llm_worker.AnthropicFactory.calculate_safe_max_tokens_for_timeout",+ return_value=_MAX_TOKENS,+ )+ with pytest.raises(LLMCompletionTruncatedError) as exc_info:+ await worker._gen_object(llm_job=_structured_job(max_tokens=8192), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert error.resolve_error_code() == CoreCode.MODEL_OUTPUT_TRUNCATED+ assert error.max_tokens == _MAX_TOKENS+ assert error.max_tokens_ceiling == MaxTokensCeiling(requested_max_tokens=8192, timeout_seconds=1200)+ assert f"lowered from 8192 to {_MAX_TOKENS}" in error.message+ assert error.message.count("lowered from") == 1+ # A higher max_tokens is lowered the same way, so neither the message nor the action asks for one+ next_step = "Lower the pipe's reasoning effort, or ask it for a smaller output, since a higher max_tokens is lowered the same way."+ assert error.message.endswith(next_step)+ assert "Raise" not in error.message+ assert error.user_action is not None+ assert error.user_action.detail == next_step++ async def test_the_mistral_worker_reads_the_cut_as_output_truncated(self, mocker: MockerFixture) -> None:+ transport = _ScriptedTransport([_MISTRAL_CUT_ANSWER])+ async with transport.make_http_client() as http_client:+ worker = _mistral_worker(mocker, http_client=http_client)+ with pytest.raises(LLMCompletionTruncatedError) as exc_info:+ await worker._gen_object(llm_job=_structured_job(max_tokens=_MAX_TOKENS), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]++ error = exc_info.value+ assert transport.nb_requests == 1+ assert error.resolve_error_code() == CoreCode.MODEL_OUTPUT_TRUNCATED+ assert error.stop_reason == "length"+ assert error.pipe_code == STOP_TEST_PIPE_CODE+ assert error.max_tokens == _MAX_TOKENS+ assert error.output_tokens == _OUTPUT_TOKENS+ assert isinstance(error.__cause__, IncompleteOutputException)+ _assert_names_a_structured_output(error)