Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Advice-promises-retry-after · L-260926-681ffe (#1378)
Louis Choquel18438f73d45d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2b2ffb50f..4f38aee9c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,12 @@# Changelog+## [Unreleased]++### Fixed++- **The `wait_and_retry` advice says to run again**: the `user_action.detail` of a transient inference failure said "the system will retry automatically", which no longer held by the time anyone read the failed run's report, and now says to wait, at least the provider's `Retry-After` delay when it gave one, then run it again; the advice on an empty or malformed Mistral response says the same.+- **A malformed `Retry-After` is no delay**: a provider's `Retry-After` that is negative or not a finite number (`inf`, `nan`, `1e999`) is ignored rather than taken as a delay, so the advice says to wait a moment instead of naming an impossible wait.+## [v0.79.0] - 2026-10-09### Highlightsdiff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.mdindex e3bf3df8b..544bb37e7 100644--- a/docs/under-the-hood/error-model.md+++ b/docs/under-the-hood/error-model.md@@ -298,6 +298,8 @@ class ProviderErrorMetadata(BaseModel):`UserAction` pairs a discrete `UserActionKind` (`WAIT_AND_RETRY`, `CHECK_BILLING`, `CHECK_CREDENTIALS`, `CHANGE_INPUT`, `CHANGE_MODEL`, `CONTACT_SUPPORT`, `UNKNOWN`) with a free-form `detail` string — so the CLI can render consistent guidance while keeping provider-specific text.+The `detail` is read on a failed run's report, once every automatic retry is spent, so it says what the reader can do next and never promises another attempt by the system: the `WAIT_AND_RETRY` advice says to wait, at least the provider's `Retry-After` delay when it gave one, then run it again.+### Service Error CodesNot every failure on an inference call comes from a provider. A gateway or a hosted inference service may refuse a request itself, before a model ever sees it, under error codes of its own, and those refusals arrive on statuses the ladder reads as a provider rejecting the prompt. The plugin that speaks to such a service contributes the codes it emits through `PluginRegistrar.add_service_error_codes`, each with its category, its action and its advice, and `classify_inference_error` consults that vocabulary ahead of the status ladder. The seam is described in [Service Error Codes](service-error-codes.md).diff --git a/pipelex/cogt/inference/error_classification.py b/pipelex/cogt/inference/error_classification.pyindex bc51e0828..b6e3aedd2 100644--- a/pipelex/cogt/inference/error_classification.py+++ b/pipelex/cogt/inference/error_classification.py@@ -7,6 +7,7 @@ wraps when ``instructor`` exhausts its retry loop."""import json+import mathfrom datetime import UTC, datetimefrom email.utils import parsedate_to_datetimefrom enum import StrEnum@@ -385,13 +386,20 @@ def parse_retry_after_seconds(value: Any) -> float | None:The HTTP spec allows two forms: a non-negative number of seconds, or anHTTP-date. Numeric values are returned directly; HTTP-date values areconverted to a delay relative to now, clamped to ``0.0`` when already past.+ A number the spec does not allow — negative, or one ``float`` accepts but no+ delay can be, such as ``inf``, ``nan`` or ``1e999`` — reads as no delay, so+ nothing downstream rounds or waits on it."""if value is None:return Nonetry:- return float(value)+ seconds = float(value)except (TypeError, ValueError):pass+ else:+ if math.isfinite(seconds) and seconds >= 0:+ return seconds+ return Noneif not isinstance(value, str):return Nonetry:diff --git a/pipelex/cogt/inference/error_render.py b/pipelex/cogt/inference/error_render.pyindex e21c1820f..3545e9bd9 100644--- a/pipelex/cogt/inference/error_render.py+++ b/pipelex/cogt/inference/error_render.py@@ -7,6 +7,7 @@ worker's error family, with a human-readable message and a structured``UserAction``."""+import mathfrom enum import StrEnumfrom pipelex.cogt.exceptions import (@@ -94,9 +95,14 @@ def _render_detail(metadata: SDKErrorEnvelope, *, classification: Classificationreturn _render_model_not_allowed_detail(model_handle=model_handle)match classification.user_action_kind:case UserActionKind.WAIT_AND_RETRY:+ # The advice is read on a failed run's report, once every automatic retry is spent, so+ # it says what the reader can do next and never promises another attempt by the system.+ # "At least" is a lower bound, so the provider's delay is rounded up, never down.if metadata.retry_after_seconds is not None:- return f"Transient provider error — the system will retry automatically after {metadata.retry_after_seconds:.0f}s."- return "Transient provider error — the system will retry automatically."+ wait_seconds = math.ceil(metadata.retry_after_seconds)+ if wait_seconds > 0:+ return f"Transient provider error — wait at least {wait_seconds}s, then run it again."+ return "Transient provider error — wait a moment, then run it again."case UserActionKind.CHECK_BILLING:return "Your account quota or credits are exhausted — check your billing dashboard."case UserActionKind.CHECK_CREDENTIALS:diff --git a/pipelex/providers/mistral/mistral_llm_worker.py b/pipelex/providers/mistral/mistral_llm_worker.pyindex 0e6dc04d7..c7a9d675c 100644--- a/pipelex/providers/mistral/mistral_llm_worker.py+++ b/pipelex/providers/mistral/mistral_llm_worker.py@@ -177,7 +177,7 @@ class MistralLLMWorker(LLMWorkerAbstract):provider_metadata=None,user_action=UserAction(kind=UserActionKind.WAIT_AND_RETRY,- detail="Mistral returned an empty response — the system will retry automatically",+ detail="Mistral returned an empty response — wait a moment, then run it again",),)if not response.choices:@@ -188,7 +188,7 @@ class MistralLLMWorker(LLMWorkerAbstract):provider_metadata=None,user_action=UserAction(kind=UserActionKind.WAIT_AND_RETRY,- detail="Mistral returned a response with no choices — the system will retry automatically",+ detail="Mistral returned a response with no choices — wait a moment, then run it again",),)message = response.choices[0].message@@ -200,7 +200,7 @@ class MistralLLMWorker(LLMWorkerAbstract):provider_metadata=None,user_action=UserAction(kind=UserActionKind.WAIT_AND_RETRY,- detail="Mistral returned a choice with no message — the system will retry automatically",+ detail="Mistral returned a choice with no message — wait a moment, then run it again",),)mistral_response_content = message.contentdiff --git a/tests/unit/pipelex/cogt/inference/test_parse_retry_after_seconds.py b/tests/unit/pipelex/cogt/inference/test_parse_retry_after_seconds.pynew file mode 100644index 000000000..7319b6b1b--- /dev/null+++ b/tests/unit/pipelex/cogt/inference/test_parse_retry_after_seconds.py@@ -0,0 +1,30 @@+from __future__ import annotations++from typing import Any++import pytest++from pipelex.cogt.inference.error_classification import parse_retry_after_seconds+++class TestParseRetryAfterSeconds:+ @pytest.mark.parametrize(+ ("header_value", "expected_seconds"),+ [+ pytest.param(None, None, id="absent"),+ pytest.param("12", 12.0, id="integer_seconds"),+ pytest.param("12.5", 12.5, id="fractional_seconds"),+ pytest.param("0", 0.0, id="zero"),+ pytest.param(7, 7.0, id="number_not_string"),+ pytest.param("inf", None, id="infinity"),+ pytest.param("-inf", None, id="negative_infinity"),+ pytest.param("nan", None, id="not_a_number"),+ pytest.param("1e999", None, id="overflows_to_infinity"),+ pytest.param("-5", None, id="negative"),+ pytest.param("soon", None, id="neither_number_nor_date"),+ pytest.param("Wed, 21 Oct 2015 07:28:00 GMT", 0.0, id="date_in_the_past"),+ ],+ )+ def test_reads_only_a_delay_the_spec_allows(self, header_value: Any, expected_seconds: float | None) -> None:+ """A Retry-After value is a non-negative finite number of seconds or an HTTP-date; anything else reads as no delay."""+ assert parse_retry_after_seconds(header_value) == expected_secondsdiff --git a/tests/unit/pipelex/cogt/inference/test_render_inference_error.py b/tests/unit/pipelex/cogt/inference/test_render_inference_error.pyindex 73d55f121..e9500df81 100644--- a/tests/unit/pipelex/cogt/inference/test_render_inference_error.py+++ b/tests/unit/pipelex/cogt/inference/test_render_inference_error.py@@ -26,6 +26,9 @@ from pipelex.cogt.inference.error_classification import ProviderErrorMetadata, Ufrom pipelex.cogt.inference.error_classify import ClassificationResultfrom pipelex.cogt.inference.error_render import InferenceErrorFamily, render_inference_errorfrom pipelex.cogt.inference.provider_name import ProviderName+from pipelex.pipe_run.exceptions import PipeRouterError+from pipelex.pipeline.exceptions import PipelineExecutionError+from pipelex.system.pipe_run_mode import PipeRunModeclass _TestCases:@@ -134,26 +137,57 @@ class TestRenderInferenceError:assert rendered.user_action is not Noneassert "safety filters" in rendered.user_action.detail- def test_retry_after_seconds_surfaced_in_detail(self) -> None:+ @pytest.mark.parametrize(+ ("retry_after_seconds", "expected_detail"),+ [+ pytest.param(None, "Transient provider error — wait a moment, then run it again.", id="no_retry_after"),+ pytest.param(12.0, "Transient provider error — wait at least 12s, then run it again.", id="retry_after"),+ pytest.param(12.4, "Transient provider error — wait at least 13s, then run it again.", id="retry_after_rounded_up"),+ pytest.param(0.0, "Transient provider error — wait a moment, then run it again.", id="retry_after_zero"),+ ],+ )+ def test_failed_run_report_says_to_wait_then_run_again(self, retry_after_seconds: float | None, expected_detail: str) -> None:+ """The advice a failed run's report carries for a transient inference error.++ The report is read once the run has failed and every automatic retry is spent, so the+ advice says what the reader can do, and never that the system will retry.+ """metadata = ProviderErrorMetadata(provider=ProviderName.OPENAI,sdk_exception_type="RateLimitError",message="Rate limited",status_code=429,- retry_after_seconds=12.0,+ retry_after_seconds=retry_after_seconds,)classification = ClassificationResult(category=InferenceErrorCategory.TRANSIENT,user_action_kind=UserActionKind.WAIT_AND_RETRY,)-- rendered = render_inference_error(+ root_fault = render_inference_error(metadata=metadata,classification=classification,family=InferenceErrorFamily.LLM,model_desc="gpt-fake",model_handle="fake-handle",)+ # Wrapped as the runner wraps it: located at the failing pipe, then reported as the run's failure.+ located = PipeRouterError.make_located(+ failure=root_fault,+ run_mode=PipeRunMode.LIVE,+ pipe_code="summarize",+ output_name=None,+ pipe_stack=["flow", "summarize"],+ )+ located.__cause__ = root_fault+ failed_run = PipelineExecutionError.make_for_run_failure(+ failure=located,+ run_mode=PipeRunMode.LIVE,+ entry_pipe_code="flow",+ output_name=None,+ )+ failed_run.__cause__ = located- assert rendered.user_action is not None- assert "12s" in rendered.user_action.detail+ report_payload = failed_run.to_error_report().to_dict()++ assert report_payload["user_action"] == {"kind": "wait_and_retry", "detail": expected_detail}+ assert "automatically" not in report_payload["user_action"]["detail"]diff --git a/tests/unit/pipelex/providers/anthropic/test_anthropic_worker_object_error_handling.py b/tests/unit/pipelex/providers/anthropic/test_anthropic_worker_object_error_handling.pyindex e23471b06..215d25d58 100644--- a/tests/unit/pipelex/providers/anthropic/test_anthropic_worker_object_error_handling.py+++ b/tests/unit/pipelex/providers/anthropic/test_anthropic_worker_object_error_handling.py@@ -118,7 +118,7 @@ class TestAnthropicWorkerObjectErrorHandling:assert exc_info.value.error_category is InferenceErrorCategory.TRANSIENTassert exc_info.value.user_action is not Noneassert exc_info.value.user_action.kind is UserActionKind.WAIT_AND_RETRY- assert "retry" in exc_info.value.user_action.detail.lower()+ assert "run it again" in exc_info.value.user_action.detailassert exc_info.value.__cause__ is wrappedmetadata = exc_info.value.provider_metadataassert metadata is not Nonediff --git a/tests/unit/pipelex/providers/anthropic/test_data.py b/tests/unit/pipelex/providers/anthropic/test_data.pyindex 95da1be19..da18ada53 100644--- a/tests/unit/pipelex/providers/anthropic/test_data.py+++ b/tests/unit/pipelex/providers/anthropic/test_data.py@@ -16,7 +16,7 @@ class AnthropicErrorHandlingTestData:"generic_rate_limit","rate_limit_error: Number of request tokens has exceeded your per-minute limit",InferenceErrorCategory.TRANSIENT,- "retry",+ "run it again",),("quota_exhaustion",diff --git a/tests/unit/pipelex/providers/bedrock/test_data.py b/tests/unit/pipelex/providers/bedrock/test_data.pyindex e12bde14d..e6e1b1991 100644--- a/tests/unit/pipelex/providers/bedrock/test_data.py+++ b/tests/unit/pipelex/providers/bedrock/test_data.py@@ -17,7 +17,7 @@ class BedrockLLMErrorHandlingTestData:"ThrottlingException","Rate exceeded for model",InferenceErrorCategory.TRANSIENT,- "retry",+ "run it again",),("throttling_quota",diff --git a/tests/unit/pipelex/providers/google/test_data.py b/tests/unit/pipelex/providers/google/test_data.pyindex 28caaf27a..4dcddeb0b 100644--- a/tests/unit/pipelex/providers/google/test_data.py+++ b/tests/unit/pipelex/providers/google/test_data.py@@ -48,7 +48,7 @@ class GoogleErrorHandlingTestData:429,"Too many requests, please slow down",InferenceErrorCategory.TRANSIENT,- "retry",+ "run it again",),("bad_request_400_content_policy",diff --git a/tests/unit/pipelex/providers/google/test_google_llm_worker_object_error_handling.py b/tests/unit/pipelex/providers/google/test_google_llm_worker_object_error_handling.pyindex 06f2e68c3..c4755e40b 100644--- a/tests/unit/pipelex/providers/google/test_google_llm_worker_object_error_handling.py+++ b/tests/unit/pipelex/providers/google/test_google_llm_worker_object_error_handling.py@@ -114,7 +114,7 @@ class TestGoogleLLMWorkerObjectErrorHandling:assert exc_info.value.error_category is InferenceErrorCategory.TRANSIENTassert exc_info.value.user_action is not Noneassert exc_info.value.user_action.kind is UserActionKind.WAIT_AND_RETRY- assert "retry" in exc_info.value.user_action.detail.lower()+ assert "run it again" in exc_info.value.user_action.detailmetadata = exc_info.value.provider_metadataassert metadata is not Noneassert metadata.status_code == 429diff --git a/tests/unit/pipelex/providers/mistral/test_data.py b/tests/unit/pipelex/providers/mistral/test_data.pyindex 4fceb8e92..7ff96258d 100644--- a/tests/unit/pipelex/providers/mistral/test_data.py+++ b/tests/unit/pipelex/providers/mistral/test_data.py@@ -31,7 +31,7 @@ class MistralLLMErrorHandlingTestData:429,"Rate limit exceeded. Please retry after 20s",InferenceErrorCategory.TRANSIENT,- "retry",+ "run it again",),("auth_error_401",@@ -99,7 +99,7 @@ class MistralExtractErrorHandlingTestData:429,"Rate limit exceeded. Please retry after 20s",InferenceErrorCategory.TRANSIENT,- "retry",+ "run it again",),("auth_error_401",diff --git a/tests/unit/pipelex/providers/mistral/test_mistral_llm_worker_object_error_handling.py b/tests/unit/pipelex/providers/mistral/test_mistral_llm_worker_object_error_handling.pyindex 17b38f95c..23ce93963 100644--- a/tests/unit/pipelex/providers/mistral/test_mistral_llm_worker_object_error_handling.py+++ b/tests/unit/pipelex/providers/mistral/test_mistral_llm_worker_object_error_handling.py@@ -82,7 +82,7 @@ class TestMistralLLMWorkerObjectErrorHandling:assert exc_info.value.error_category is InferenceErrorCategory.TRANSIENTassert exc_info.value.user_action is not Noneassert exc_info.value.user_action.kind is UserActionKind.WAIT_AND_RETRY- assert "retry" in exc_info.value.user_action.detail.lower()+ assert "run it again" in exc_info.value.user_action.detailassert exc_info.value.__cause__ is wrappedmetadata = exc_info.value.provider_metadataassert metadata is not Nonediff --git a/tests/unit/pipelex/providers/mistral/test_mistral_worker_error_handling.py b/tests/unit/pipelex/providers/mistral/test_mistral_worker_error_handling.pyindex c178d9878..92f9f3ff9 100644--- a/tests/unit/pipelex/providers/mistral/test_mistral_worker_error_handling.py+++ b/tests/unit/pipelex/providers/mistral/test_mistral_worker_error_handling.py@@ -138,11 +138,27 @@ class TestMistralWorkerErrorHandling:assert exc_info.value.user_action.kind is UserActionKind.CHANGE_INPUTassert exc_info.value.__cause__ is sdk_exc- async def test_llm_worker_choice_without_message_is_transient(self, mocker: MockerFixture) -> None:- """A choice whose message is missing is retried as a transient failure, not read as empty text."""+ @pytest.mark.parametrize(+ ("response_shape", "expected_detail"),+ [+ pytest.param("no_response", "Mistral returned an empty response — wait a moment, then run it again", id="no_response"),+ pytest.param("no_choices", "Mistral returned a response with no choices — wait a moment, then run it again", id="no_choices"),+ pytest.param(+ "choice_without_message",+ "Mistral returned a choice with no message — wait a moment, then run it again",+ id="choice_without_message",+ ),+ ],+ )+ async def test_llm_worker_malformed_response_is_transient(self, mocker: MockerFixture, response_shape: str, expected_detail: str) -> None:+ """A malformed response is a transient failure, not read as empty text, and its advice says to wait then run again."""worker = _make_mistral_llm_worker(mocker)- response = mocker.MagicMock()- response.choices = [mocker.MagicMock(message=None)]+ response: Any+ if response_shape == "no_response":+ response = None+ else:+ response = mocker.MagicMock()+ response.choices = [] if response_shape == "no_choices" else [mocker.MagicMock(message=None)]worker.mistral_client_for_text.chat.complete_async.return_value = response # type: ignore[attr-defined] # pyright: ignore[reportAttributeAccessIssue]with pytest.raises(LLMCompletionError) as exc_info:@@ -151,6 +167,7 @@ class TestMistralWorkerErrorHandling:assert exc_info.value.error_category is InferenceErrorCategory.TRANSIENTassert exc_info.value.user_action is not Noneassert exc_info.value.user_action.kind is UserActionKind.WAIT_AND_RETRY+ assert exc_info.value.user_action.detail == expected_detailasync def test_llm_worker_not_found_raises_llm_model_not_found_error(self, mocker: MockerFixture) -> None:"""A 404 MistralError specializes to LLMModelNotFoundError (CONFIGURATION) on the LLM path."""diff --git a/tests/unit/pipelex/providers/openai/test_data.py b/tests/unit/pipelex/providers/openai/test_data.pyindex 0b938082a..2579f9768 100644--- a/tests/unit/pipelex/providers/openai/test_data.py+++ b/tests/unit/pipelex/providers/openai/test_data.py@@ -16,7 +16,7 @@ class OpenAIErrorHandlingTestData:"generic_rate_limit","Rate limit exceeded. Please retry after 20s",InferenceErrorCategory.TRANSIENT,- "retry",+ "run it again",),("quota_insufficient",diff --git a/tests/unit/pipelex/providers/openai/test_openai_completions_worker_object_error_handling.py b/tests/unit/pipelex/providers/openai/test_openai_completions_worker_object_error_handling.pyindex e707f7634..8e1bcd47c 100644--- a/tests/unit/pipelex/providers/openai/test_openai_completions_worker_object_error_handling.py+++ b/tests/unit/pipelex/providers/openai/test_openai_completions_worker_object_error_handling.py@@ -108,7 +108,7 @@ class TestOpenAICompletionsWorkerObjectErrorHandling:assert exc_info.value.error_category is InferenceErrorCategory.TRANSIENTassert exc_info.value.user_action is not Noneassert exc_info.value.user_action.kind is UserActionKind.WAIT_AND_RETRY- assert "retry" in exc_info.value.user_action.detail.lower()+ assert "run it again" in exc_info.value.user_action.detailassert exc_info.value.__cause__ is wrappedmetadata = exc_info.value.provider_metadataassert metadata is not Nonediff --git a/tests/unit/pipelex/providers/openai/test_openai_responses_worker_object_error_handling.py b/tests/unit/pipelex/providers/openai/test_openai_responses_worker_object_error_handling.pyindex 37a539a48..40559a2ed 100644--- a/tests/unit/pipelex/providers/openai/test_openai_responses_worker_object_error_handling.py+++ b/tests/unit/pipelex/providers/openai/test_openai_responses_worker_object_error_handling.py@@ -109,7 +109,7 @@ class TestOpenAIResponsesWorkerObjectErrorHandling:assert exc_info.value.error_category is InferenceErrorCategory.TRANSIENTassert exc_info.value.user_action is not Noneassert exc_info.value.user_action.kind is UserActionKind.WAIT_AND_RETRY- assert "retry" in exc_info.value.user_action.detail.lower()+ assert "run it again" in exc_info.value.user_action.detailassert exc_info.value.__cause__ is wrappedmetadata = exc_info.value.provider_metadataassert metadata is not None