Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix/Gateway-429-read-as-quota · L-261027-4b60ef (#1494)

Louis Choquel651b28a3e9d0
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9f037b55c..f1fdc40d8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,11 @@
# Changelog
+## [Unreleased]
+
+### Fixed
+
+- **A gateway's rate limit is no longer read as a spent quota**: a 429 relayed by a Portkey-dialect gateway is now `provider.rate_limited`, retried with the provider's `Retry-After`, unless its error code is one the provider documents for a spent quota: OpenAI's `insufficient_quota`, `credit_balance_exhausted`, `organization_spend_limit_exceeded`, `project_spend_limit_exceeded` or `organization_usage_limit_exceeded`. A 429 whose message mentioned a quota, as Azure OpenAI's throttles do, used to fail at once as `provider.quota_exhausted` with advice to add credit. A 402 is still a spent quota.
+
## [v0.92.2] - 2026-10-26
### Added
diff --git a/docs/reliability/failure-classification.md b/docs/reliability/failure-classification.md
index 47ec27695..57e53a5c8 100644
--- a/docs/reliability/failure-classification.md
+++ b/docs/reliability/failure-classification.md
@@ -18,4 +18,12 @@ Whichever path runs, every inference failure is classified the moment it happens
Only `transient` failures are worth retrying — re-running a call that failed because your API key is wrong just wastes time and money. On the [durable path](durable-execution.md), the activity retry policy uses exactly this signal: a `transient` failure retries, every other category fails fast.
+## A spent quota or a rate limit
+
+A spent quota and a rate limit both refuse the call, often with the same HTTP status, and they differ only in what cures them: time cures a rate limit, while only a person, adding credit or raising a quota, cures a spent quota. For a failure a gateway relays, Pipelex tells them apart from what the provider says structurally, its status and its error code, and never from the words of its message: a gateway relays every provider's prose, and a throttle's message may well mention a quota, as Azure OpenAI's does by linking to its quota-increase page.
+
+Through a gateway, a `402` is a spent quota, whatever its code says. A `429` is a spent quota only when it carries an error code its provider documents as a spent quota: OpenAI's broad `insufficient_quota`, or one of the billing codes OpenAI's error-code guide names in `error.code`, `credit_balance_exhausted`, `organization_spend_limit_exceeded`, `project_spend_limit_exceeded` and `organization_usage_limit_exceeded`, all of which Azure OpenAI and the OpenAI-compatible providers relay. Every other `429`, including one with no code at all, is a rate limit: it is `transient`, so it is retried, and if it still fails it reaches you as `provider.rate_limited`, with the delay the provider asked for when it gave one.
+
+When the structure says nothing either way, Pipelex reads a `429` as a rate limit on purpose, because the two mistakes do not cost the same. A spent quota read as a rate limit costs a few refused calls, which are not billed, and some waiting before the run fails with the provider's code in its diagnostics. A rate limit read as a spent quota fails the run at once and sends you to add credit that cannot help. Each provider Pipelex calls directly, with your own keys, has a rule of its own, and some of those still read the provider's wording.
+
For the full mechanics of how errors are classified, carried, and reported, see [Error Model](../under-the-hood/error-model.md).
diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.md
index 42d369f9f..3105e339e 100644
--- a/docs/under-the-hood/error-model.md
+++ b/docs/under-the-hood/error-model.md
@@ -689,7 +689,7 @@ InferenceErrorCategory.TRANSIENT.is_retryable # True — only TRANSIENT
| Scenario | Behavior |
|----------|----------|
| Rate limit hit | `TRANSIENT` → repeated within the run; `provider.rate_limited`, whose verdict invites a retry and which answers 429, with a `Retry-After` header when the provider gave a delay; transport retry honors `Retry-After` |
-| Quota / billing exhausted | `CAPACITY` → not repeated; `UserAction(CHECK_BILLING)`; `provider.quota_exhausted`, the operator's → HTTP 502 |
+| Quota / billing exhausted | `CAPACITY` → not repeated; `UserAction(CHECK_BILLING)`; `provider.quota_exhausted`, the operator's → HTTP 502. Through a gateway, it is read from the status and the provider's error code alone, never the message: a 402 is a spent quota whatever its code, a 429 is one only when its provider code is in `_GATEWAY_QUOTA_ERROR_CODES` (OpenAI's broad `insufficient_quota` and the billing codes its error-code guide names in `error.code`, `credit_balance_exhausted`, `organization_spend_limit_exceeded`, `project_spend_limit_exceeded` and `organization_usage_limit_exceeded`, which Azure OpenAI and the OpenAI-compatible providers relay; the Extract step reads a body's `code` before its `type`, so a billing code beside a broad `type` is read as the billing code), and a gateway's 429 with any other code, or none, is a rate limit, whatever its message says |
| Bad API key | `CONFIGURATION` → not repeated; `provider.credential_refused`, the operator's → HTTP 502 |
| Model or deployment not found (provider HTTP 404) | Raises a dedicated `*ModelNotFoundError` sibling (`LLMModelNotFoundError`, `ImgGenModelNotFoundError`, `ExtractModelNotFoundError`, `SearchModelNotFoundError`) carrying `model.unavailable`, since a method cannot name a provider's raw model id; operator re-raises `PipeOperatorModelAvailabilityError` |
| Model reference unknown to the deck (a method naming `gpt-5.1`, a mistyped `@alias` or `$preset`) | `ModelChoiceNotFoundError` → `CONFIGURATION` category, but the code `model.unknown`, the method's → **HTTP 422**, whose sentence names the reference and whose `suggestions` parameter carries the close matches of every catalog the deck knows, an empty list when none is near, which are all a caller needs to fix the method. A handle the deck serves only as another type of model is `model.capability_unsupported` |
diff --git a/pipelex/cogt/inference/error_classification.py b/pipelex/cogt/inference/error_classification.py
index e842d041c..4ccdb9043 100644
--- a/pipelex/cogt/inference/error_classification.py
+++ b/pipelex/cogt/inference/error_classification.py
@@ -1,7 +1,7 @@
"""Helpers for classifying SDK errors into InferenceErrorCategory values.
-Pure functions that inspect error messages to discriminate between
-quota exhaustion vs rate limiting, detect content policy violations, and
+Pure functions that inspect error statuses, codes and messages to discriminate
+between quota exhaustion vs rate limiting, detect content policy violations, and
recover the underlying SDK exception that ``InstructorRetryException``
wraps when ``instructor`` exhausts its retry loop.
"""
@@ -112,8 +112,10 @@ class ProviderErrorMetadata(BaseModel):
def is_quota_exhaustion(self) -> bool:
"""Whether this error is a quota/credits exhaustion rather than rate limiting.
- Dispatches on ``provider`` because each provider phrases quota
- exhaustion differently; Mistral and Gateway also use HTTP 402.
+ Dispatches on ``provider`` because each provider signals quota
+ exhaustion differently; Mistral and Gateway also use HTTP 402. A
+ gateway relays every provider's prose, so its rule reads the status and
+ the provider's error code alone, never the message.
"""
match self.provider:
case ProviderName.OPENAI:
@@ -127,7 +129,7 @@ class ProviderErrorMetadata(BaseModel):
case ProviderName.BEDROCK:
return _is_quota_exhaustion_aws(self.message, provider_error_code=self.provider_error_code)
case ProviderName.GATEWAY:
- return _is_quota_exhaustion_gateway(self.message, status_code=self.status_code or 0)
+ return _is_quota_exhaustion_gateway(status_code=self.status_code or 0, provider_error_code=self.provider_error_code)
case (
ProviderName.AZURE
| ProviderName.FAL
@@ -307,15 +309,27 @@ _AWS_QUOTA_PATTERNS: tuple[str, ...] = (
"service quota",
)
-_GATEWAY_QUOTA_PATTERNS: tuple[str, ...] = (
- "quota",
- "billing limit",
- "billing quota",
- "insufficient_quota",
- "insufficient credit",
- "insufficient funds",
- "insufficient balance",
- "credits exhausted",
+# The provider error codes that make a gateway's 429 a spent quota rather than a rate limit. A gateway relays every
+# provider's prose, so the message decides nothing: Azure's throttle links to its quota-increase page, and Google's
+# per-minute limit says "check quota". A code joins this set only when the provider that sends it documents it as a
+# spent quota on a 429, with a comment naming that provider and a test case. A 402 needs no entry, since its status
+# decides it, and a gateway's own codes are read from the service error vocabulary before this rule runs.
+_GATEWAY_QUOTA_ERROR_CODES: frozenset[str] = frozenset(
+ {
+ # OpenAI's codes for a spent quota on a 429, which Azure OpenAI and the OpenAI-compatible providers relay in the
+ # same fields (https://developers.openai.com/api/docs/guides/error-codes). OpenAI names each billing cause in
+ # `error.code` and says `error.type` can still be `insufficient_quota`; the gateway's Extract step reads the
+ # body's code before its type, so the specific codes are listed beside the broad one.
+ "insufficient_quota",
+ # no prepaid credits remaining
+ "credit_balance_exhausted",
+ # the organization's enforced spend limit
+ "organization_spend_limit_exceeded",
+ # the project's enforced spend limit
+ "project_spend_limit_exceeded",
+ # the organization's OpenAI-assigned usage limit
+ "organization_usage_limit_exceeded",
+ }
)
@@ -364,16 +378,19 @@ def _is_quota_exhaustion_aws(error_message: str, *, provider_error_code: str | N
return any(pattern in lower_message for pattern in _AWS_QUOTA_PATTERNS)
-def _is_quota_exhaustion_gateway(error_message: str, *, status_code: int) -> bool:
- """Check if a Portkey/Gateway error indicates quota/credits exhaustion.
+def _is_quota_exhaustion_gateway(*, status_code: int, provider_error_code: str | None) -> bool:
+ """Check if a Portkey/Gateway error indicates quota/credits exhaustion, from its status and its provider code alone.
- HTTP 402 (Payment Required) is a definitive quota signal.
- HTTP 429 requires message inspection to distinguish quota from rate limiting.
+ HTTP 402 (Payment Required) is a spent quota whatever its code. HTTP 429 is a spent quota only when the provider's
+ error code is one of ``_GATEWAY_QUOTA_ERROR_CODES``; any other 429, one with no code included, is a rate limit,
+ which the classifier's 429 arm reads as ``provider.rate_limited``. The message is never read: a gateway relays
+ every provider's prose, and a word list keeps meeting a new provider's wording. When the structure says nothing
+ either way, a quota read as a rate limit costs some refused calls and some waiting, while a rate limit read as a
+ quota fails the run and sends a person to add credit that cannot help.
"""
if status_code == 402:
return True
- lower_message = error_message.lower()
- return status_code == 429 and any(pattern in lower_message for pattern in _GATEWAY_QUOTA_PATTERNS)
+ return status_code == 429 and provider_error_code in _GATEWAY_QUOTA_ERROR_CODES
def _is_content_policy_violation(error_message: str) -> bool:
diff --git a/pipelex/cogt/inference/error_classify.py b/pipelex/cogt/inference/error_classify.py
index e92144bc2..e4877d5e1 100644
--- a/pipelex/cogt/inference/error_classify.py
+++ b/pipelex/cogt/inference/error_classify.py
@@ -312,7 +312,9 @@ def classify_inference_error(metadata: SDKErrorEnvelope, *, family: InferenceErr
# Quota exhaustion is decided by the provider-aware ``is_quota_exhaustion``
# property and takes precedence over the HTTP status: providers signal it on
- # different statuses (OpenAI/Anthropic 429, Mistral/Gateway 402, AWS 400).
+ # different statuses (OpenAI/Anthropic 429, Mistral/Gateway 402, AWS 400). A
+ # gateway's 429 is a quota only on the provider's quota code; any other 429
+ # falls through to the arm below and is a rate limit.
if metadata.is_quota_exhaustion:
return ClassificationResult(
category=InferenceErrorCategory.CAPACITY,
diff --git a/subject_grants.toml b/subject_grants.toml
index d1ae791b5..8e5c897dc 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -1624,10 +1624,6 @@ rationale = "Predicate on the error_message; single operand."
param = "error_message"
rationale = "Predicate on the error_message; single operand."
-["pipelex/cogt/inference/error_classification.py::_is_quota_exhaustion_gateway"]
-param = "error_message"
-rationale = "Predicate on the error_message; single operand."
-
["pipelex/cogt/inference/error_classification.py::_is_quota_exhaustion_google"]
param = "error_message"
rationale = "Predicate on the error_message; single operand."
diff --git a/tests/unit/pipelex/cogt/inference/test_gateway_throttle_classification.py b/tests/unit/pipelex/cogt/inference/test_gateway_throttle_classification.py
new file mode 100644
index 000000000..554d596d4
--- /dev/null
+++ b/tests/unit/pipelex/cogt/inference/test_gateway_throttle_classification.py
@@ -0,0 +1,132 @@
+from __future__ import annotations
+
+import copy
+import json
+from typing import TYPE_CHECKING, Any
+
+import httpx
+import pytest
+from mthds.protocol.problem.registry import CoreCode
+from portkey_ai import Portkey
+from portkey_ai._vendor import openai as portkey_vendored_openai
+
+from pipelex.cogt.exceptions import InferenceErrorCategory
+from pipelex.cogt.inference.error_classification import UserActionKind, extract_gateway_metadata
+from pipelex.cogt.inference.error_classify import InferenceErrorFamily, classify_inference_error
+from pipelex.cogt.inference.error_render import render_inference_error
+from pipelex.cogt.inference.provider_name import ProviderName
+from pipelex.errors.problem import ProblemView
+from tests.unit.pipelex.cogt.test_data import GatewayThrottleTestData
+
+if TYPE_CHECKING:
+ from collections.abc import Callable
+
+_ORIGIN = "https://gateway.example.com"
+
+
+def _gateway_response(*, body: dict[str, Any], retry_after: str) -> tuple[httpx.Request, httpx.Response]:
+ request = httpx.Request("POST", f"{_ORIGIN}/v1/images/generations")
+ response = httpx.Response(
+ status_code=429,
+ request=request,
+ content=json.dumps(body).encode(),
+ headers={"content-type": "application/json", "retry-after": retry_after},
+ )
+ return request, response
+
+
+def _as_portkey_raises_it(*, body: dict[str, Any], retry_after: str) -> BaseException:
+ """Build the exception through Portkey's own factory, which keeps only the message string on ``body``."""
+ request, response = _gateway_response(body=body, retry_after=retry_after)
+ client = Portkey(api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1")
+ return client._make_status_error_from_response(request=request, response=response) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+
+def _as_portkey_s_vendored_openai_raises_it(*, body: dict[str, Any], retry_after: str) -> BaseException:
+ """Build the exception through the OpenAI client Portkey vendors, the hop a gateway's image call takes.
+
+ Its factory renders the whole body into the message and keeps the ``error`` section on ``body``, which is the
+ shape of the refusal's diagnostics on the hosted API.
+ """
+ _, response = _gateway_response(body=body, retry_after=retry_after)
+ client = portkey_vendored_openai.OpenAI(api_key="unused-in-this-test", base_url=f"{_ORIGIN}/v1")
+ return client._make_status_error_from_response(response) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+
+_HOPS = [
+ pytest.param(_as_portkey_raises_it, id="portkey-factory"),
+ pytest.param(_as_portkey_s_vendored_openai_raises_it, id="portkey-vendored-openai"),
+]
+
+
+class TestGatewayThrottleClassification:
+ @pytest.mark.parametrize("raise_it", _HOPS)
+ def test_azure_s_throttle_is_a_rate_limit_waited_for_the_delay_it_asks(self, raise_it: Callable[..., BaseException]) -> None:
+ """Azure's refusal, verbatim, through each hop that raises a gateway's failure, is a retryable rate limit.
+
+ The gateway's quota rule reads the status and the provider's code alone, so the message's link to Azure's
+ quota-increase page decides nothing: the refusal is ``provider.rate_limited``, repeated within the run, and
+ carries Azure's 57 seconds, where it used to fail at once as ``provider.quota_exhausted``.
+ """
+ exc = raise_it(body=GatewayThrottleTestData.AZURE_RATE_LIMIT_BODY, retry_after=GatewayThrottleTestData.AZURE_RETRY_AFTER)
+
+ metadata = extract_gateway_metadata(exc)
+ classification = classify_inference_error(metadata, family=InferenceErrorFamily.IMG_GEN)
+
+ assert metadata.provider == ProviderName.GATEWAY
+ assert metadata.status_code == 429
+ assert metadata.provider_error_code == GatewayThrottleTestData.AZURE_RATE_LIMIT_CODE
+ assert metadata.retry_after_seconds == 57
+ assert "quota" in metadata.message
+ assert metadata.is_quota_exhaustion is False
+ assert classification.error_code == CoreCode.PROVIDER_RATE_LIMITED
+ assert classification.category == InferenceErrorCategory.TRANSIENT
+ assert classification.category.is_retryable is True
+ assert classification.user_action_kind == UserActionKind.WAIT_AND_RETRY
+
+ rendered = render_inference_error(
+ metadata=metadata,
+ classification=classification,
+ family=InferenceErrorFamily.IMG_GEN,
+ model_desc="gpt-image-2 → Model[gpt-image-2]",
+ model_handle="gpt-image-2",
+ )
+ # As the worker's error enrichment does, which names the model and the backend the problem's sentence reads
+ rendered.fill_model_and_provider("gpt-image-2", backend_name="pipelex_manifold")
+ problem = rendered.to_problem(view=ProblemView.OPERATOR)
+
+ assert problem.code == CoreCode.PROVIDER_RATE_LIMITED
+ assert problem.is_retryable is True
+ assert problem.retry_after == 57
+
+ @pytest.mark.parametrize(
+ ("error_type", "error_code"),
+ [
+ pytest.param(None, GatewayThrottleTestData.OPENAI_QUOTA_CODE, id="broad-code"),
+ # OpenAI's billing shape: the cause in `code`, which the Extract step reads before the broad `type`
+ pytest.param(
+ GatewayThrottleTestData.OPENAI_QUOTA_CODE,
+ GatewayThrottleTestData.OPENAI_CREDIT_BALANCE_EXHAUSTED_CODE,
+ id="billing-code-beside-the-broad-type",
+ ),
+ ],
+ )
+ @pytest.mark.parametrize("raise_it", _HOPS)
+ def test_the_same_refusal_on_openai_s_quota_code_is_a_spent_quota(
+ self, raise_it: Callable[..., BaseException], error_type: str | None, error_code: str
+ ) -> None:
+ """The contrast: only the provider's code changes, and the code alone makes the 429 a spent quota."""
+ body = copy.deepcopy(GatewayThrottleTestData.AZURE_RATE_LIMIT_BODY)
+ body["error"]["type"] = error_type
+ body["error"]["code"] = error_code
+ exc = raise_it(body=body, retry_after=GatewayThrottleTestData.AZURE_RETRY_AFTER)
+
+ metadata = extract_gateway_metadata(exc)
+ classification = classify_inference_error(metadata, family=InferenceErrorFamily.IMG_GEN)
+
+ assert metadata.provider_error_code == error_code
+ assert metadata.is_quota_exhaustion is True
+ assert classification.error_code == CoreCode.PROVIDER_QUOTA_EXHAUSTED
+ assert classification.category == InferenceErrorCategory.CAPACITY
+ assert classification.category.is_retryable is False
+ assert classification.user_action_kind == UserActionKind.CHECK_BILLING
diff --git a/tests/unit/pipelex/cogt/test_data.py b/tests/unit/pipelex/cogt/test_data.py
index 121fe01e8..f1aad2775 100644
--- a/tests/unit/pipelex/cogt/test_data.py
+++ b/tests/unit/pipelex/cogt/test_data.py
@@ -1,4 +1,4 @@
-from typing import ClassVar
+from typing import Any, ClassVar
from mthds.protocol.problem.registry import CoreCode, ProblemDomain
@@ -28,3 +28,36 @@ class ExceptionTestData:
("ambiguous_is_an_unknown_outcome", InferenceErrorCategory.AMBIGUOUS, CoreCode.PROVIDER_OUTCOME_UNKNOWN, ProblemDomain.RUNTIME, True),
("unknown_keeps_the_fallback", InferenceErrorCategory.UNKNOWN, CoreCode.RUNTIME_UNEXPECTED, ProblemDomain.RUNTIME, False),
]
+
+
+class GatewayThrottleTestData:
+ # Azure OpenAI's refusal of a `gpt-image-2` call over its deployment's call rate, verbatim as a run on the hosted API
+ # received it through the gateway, which prefixes the provider's message with the provider's name. Its code is
+ # Azure's rate-limit code, and its last sentence links to Azure's quota-increase page, whose address holds the word
+ # "quota": a rule reading the message for that word took this throttle for a spent quota.
+ AZURE_RATE_LIMIT_MESSAGE: ClassVar[str] = (
+ "azure-openai error: Your requests to gpt-image-2 for gpt-image-2 in East US 2 have exceeded the call rate limit "
+ "for your current AIServices S0 pricing tier. This request was for ImageGenerations_Create under Azure OpenAI API "
+ "version 2025-04-01-preview. Please retry after 57 seconds. To increase your default rate limit, visit: "
+ "https://aka.ms/oai/quotaincrease."
+ )
+ AZURE_RATE_LIMIT_CODE: ClassVar[str] = "RateLimitReached"
+ # The delay the gateway forwards from Azure in the `retry-after` header
+ AZURE_RETRY_AFTER: ClassVar[str] = "57"
+ # The whole body the gateway answered with, at HTTP 429
+ AZURE_RATE_LIMIT_BODY: ClassVar[dict[str, Any]] = {
+ "error": {
+ "message": AZURE_RATE_LIMIT_MESSAGE,
+ "type": None,
+ "param": None,
+ "code": AZURE_RATE_LIMIT_CODE,
+ },
+ "provider": "azure-openai",
+ }
+
+ # OpenAI's broad code for a spent quota on a 429, which Azure OpenAI and the OpenAI-compatible providers relay
+ OPENAI_QUOTA_CODE: ClassVar[str] = "insufficient_quota"
+ OPENAI_QUOTA_MESSAGE: ClassVar[str] = "You exceeded your current quota, please check your plan and billing details."
+ # One of the billing codes OpenAI's error-code guide documents for a 429 in `error.code`, whose `error.type` can
+ # still be `insufficient_quota`
+ OPENAI_CREDIT_BALANCE_EXHAUSTED_CODE: ClassVar[str] = "credit_balance_exhausted"
diff --git a/tests/unit/pipelex/cogt/test_error_classification.py b/tests/unit/pipelex/cogt/test_error_classification.py
index b4ae421c9..afa5c5944 100644
--- a/tests/unit/pipelex/cogt/test_error_classification.py
+++ b/tests/unit/pipelex/cogt/test_error_classification.py
@@ -3,7 +3,8 @@
The quota and content-policy probes are exposed via the
``ProviderErrorMetadata.is_quota_exhaustion`` / ``is_content_policy_violation``
properties; the helpers backing them are module-private. These tests pin the
-per-provider phrase matching so changes to the patterns surface as test diffs.
+per-provider phrase matching so changes to the patterns surface as test diffs,
+and the gateway's rule, which reads the status and the provider's code alone.
"""
from typing import ClassVar
@@ -12,14 +13,18 @@ import pytest
from pipelex.cogt.inference.error_classification import ProviderErrorMetadata
from pipelex.cogt.inference.provider_name import ProviderName
+from tests.unit.pipelex.cogt.test_data import GatewayThrottleTestData
-def _metadata(provider: ProviderName, message: str, status_code: int | None = None) -> ProviderErrorMetadata:
+def _metadata(
+ provider: ProviderName, message: str, status_code: int | None = None, *, provider_error_code: str | None = None
+) -> ProviderErrorMetadata:
return ProviderErrorMetadata(
provider=provider,
sdk_exception_type="RateLimitError",
message=message,
status_code=status_code,
+ provider_error_code=provider_error_code,
)
@@ -86,22 +91,46 @@ class _TestCases:
("unrelated_error", "Access denied", False),
]
- # (topic, message, status_code, expected_is_quota)
- GATEWAY_QUOTA_CASES: ClassVar[list[tuple[str, str, int, bool]]] = [
- ("payment_required_402", "Payment required", 402, True),
- ("quota_429", "Your quota has been exhausted", 429, True),
- ("billing_429", "Billing limit exceeded", 429, True),
- ("insufficient_credits_429", "Insufficient credits", 429, True),
- ("insufficient_quota_429", "insufficient_quota for this account", 429, True),
- ("insufficient_funds_429", "Insufficient funds in your account", 429, True),
- ("insufficient_balance_429", "Insufficient balance to complete request", 429, True),
- ("credits_exhausted_429", "Credits exhausted on this key", 429, True),
- ("insufficient_permissions_429_not_quota", "Insufficient permissions to use this model", 429, False),
- ("insufficient_role_429_not_quota", "Insufficient role for the requested action", 429, False),
- ("insufficient_access_429_not_quota", "Insufficient access privileges", 429, False),
- ("generic_rate_limit_429", "Rate limit exceeded. Please retry", 429, False),
- ("server_error_500", "Internal server error", 500, False),
- ("payment_required_402_any_message", "random text", 402, True),
+ # (topic, message, status_code, provider_error_code, expected_is_quota)
+ # A gateway's rule reads the status and the provider's code, never the message: a 402 is a quota whatever it says,
+ # a 429 is a quota only on a documented quota code, and every other 429 is a rate limit, whatever its words.
+ GATEWAY_QUOTA_CASES: ClassVar[list[tuple[str, str, int, str | None, bool]]] = [
+ ("payment_required_402", "Payment required", 402, None, True),
+ ("payment_required_402_any_message", "random text", 402, None, True),
+ ("payment_required_402_any_code", "Payment required", 402, GatewayThrottleTestData.AZURE_RATE_LIMIT_CODE, True),
+ ("openai_insufficient_quota_code_429", GatewayThrottleTestData.OPENAI_QUOTA_MESSAGE, 429, GatewayThrottleTestData.OPENAI_QUOTA_CODE, True),
+ ("insufficient_quota_code_429_any_message", "random text", 429, GatewayThrottleTestData.OPENAI_QUOTA_CODE, True),
+ # OpenAI's billing codes, each naming the cause of a spent quota in `error.code`
+ (
+ "openai_credit_balance_exhausted_code_429",
+ "Credit balance exhausted",
+ 429,
+ GatewayThrottleTestData.OPENAI_CREDIT_BALANCE_EXHAUSTED_CODE,
+ True,
+ ),
+ ("openai_organization_spend_limit_code_429", "Organization spend limit reached", 429, "organization_spend_limit_exceeded", True),
+ ("openai_project_spend_limit_code_429", "Project spend limit reached", 429, "project_spend_limit_exceeded", True),
+ ("openai_organization_usage_limit_code_429", "Organization usage limit reached", 429, "organization_usage_limit_exceeded", True),
+ (
+ "azure_rate_limit_reached_429_linking_its_quota_increase_page",
+ GatewayThrottleTestData.AZURE_RATE_LIMIT_MESSAGE,
+ 429,
+ GatewayThrottleTestData.AZURE_RATE_LIMIT_CODE,
+ False,
+ ),
+ ("azure_message_429_no_code", GatewayThrottleTestData.AZURE_RATE_LIMIT_MESSAGE, 429, None, False),
+ ("generic_rate_limit_429_no_code", "Rate limit exceeded. Please retry", 429, None, False),
+ # Each message the former word list read as a quota, now a rate limit since no code says otherwise
+ ("quota_message_429_no_code", "Your quota has been exhausted", 429, None, False),
+ ("billing_message_429_no_code", "Billing limit exceeded", 429, None, False),
+ ("insufficient_credits_message_429_no_code", "Insufficient credits", 429, None, False),
+ ("insufficient_quota_message_429_no_code", "insufficient_quota for this account", 429, None, False),
+ ("insufficient_funds_message_429_no_code", "Insufficient funds in your account", 429, None, False),
+ ("insufficient_balance_message_429_no_code", "Insufficient balance to complete request", 429, None, False),
+ ("credits_exhausted_message_429_no_code", "Credits exhausted on this key", 429, None, False),
+ # The quota code decides a 429 only; on any other status the status ladder reads it
+ ("insufficient_quota_code_500_not_quota", "Internal server error", 500, GatewayThrottleTestData.OPENAI_QUOTA_CODE, False),
+ ("server_error_500", "Internal server error", 500, None, False),
]
CONTENT_POLICY_CASES: ClassVar[list[tuple[str, str, bool]]] = [
@@ -169,9 +198,12 @@ class TestErrorClassification:
assert _metadata(ProviderName.BEDROCK, error_message).is_quota_exhaustion == expected
@pytest.mark.parametrize(
- ("_topic", "error_message", "status_code", "expected"),
+ ("_topic", "error_message", "status_code", "provider_error_code", "expected"),
_TestCases.GATEWAY_QUOTA_CASES,
)
- def test_is_quota_exhaustion_gateway(self, _topic: str, error_message: str, status_code: int, expected: bool) -> None:
- """Discriminate Gateway quota exhaustion from generic rate limiting."""
- assert _metadata(ProviderName.GATEWAY, error_message, status_code).is_quota_exhaustion == expected
+ def test_is_quota_exhaustion_gateway(
+ self, _topic: str, error_message: str, status_code: int, provider_error_code: str | None, expected: bool
+ ) -> None:
+ """Discriminate Gateway quota exhaustion from rate limiting by the status and the provider's code alone."""
+ metadata = _metadata(ProviderName.GATEWAY, error_message, status_code, provider_error_code=provider_error_code)
+ assert metadata.is_quota_exhaustion == expected