Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Normalize DSPy LM errors (#9826)

Maxime Riveste608bc024c8e
diff --git a/docs/docs/api/utils/Errors.md b/docs/docs/api/utils/Errors.md
new file mode 100644
index 000000000..67af084a2
--- /dev/null
+++ b/docs/docs/api/utils/Errors.md
@@ -0,0 +1,59 @@
+# dspy errors
+
+DSPy exposes structured exception classes for LM failures and adapter parsing failures. LM errors inherit from `dspy.LMError`, which inherits from `dspy.DSPyError` and carries metadata such as `model`, `provider`, `status`, `request_id`, and `retry_after` when available.
+
+Use `dspy.LMError` to catch any LM call failure, or catch a concrete subclass when you need specific handling.
+
+```python
+try:
+ result = program(question="...")
+except dspy.ContextWindowExceededError as e:
+ print(f"Prompt was too long for {e.model}")
+except dspy.LMRateLimitError as e:
+ print(f"Rate limited; retry after {e.retry_after} seconds")
+except dspy.LMError as e:
+ print(f"LM failed with code={e.code}, provider={e.provider}, request_id={e.request_id}")
+```
+
+Use `dspy.is_retryable_lm_error(error)` to classify LM failures that are generally safe to retry: rate limits, timeouts, server errors, and transport errors. DSPy's built-in `dspy.LM` delegates provider retries to LiteLLM, but callers can use this helper after retries are exhausted. Retryability is advisory: respect provider policy and `retry_after` when present.
+
+```python
+try:
+ result = program(question="...")
+except dspy.LMError as e:
+ if dspy.is_retryable_lm_error(e):
+ # Schedule another attempt later.
+ raise
+ raise
+```
+
+## API Reference
+
+::: dspy.utils.exceptions
+ handler: python
+ options:
+ members:
+ - DSPyError
+ - LMError
+ - LMTransportError
+ - LMConfigurationError
+ - LMNotConfiguredError
+ - LMUnsupportedFeatureError
+ - LMProviderError
+ - LMUnexpectedError
+ - LMAuthError
+ - LMBillingError
+ - LMRateLimitError
+ - LMInvalidRequestError
+ - ContextWindowExceededError
+ - LMUnsupportedModelError
+ - LMTimeoutError
+ - LMServerError
+ - is_retryable_lm_error
+ - AdapterParseError
+ show_source: true
+ show_root_heading: false
+ heading_level: 3
+ docstring_style: google
+ show_root_full_path: true
+ show_object_full_path: false
diff --git a/docs/docs/faqs.md b/docs/docs/faqs.md
index 65615faba..84052a649 100644
--- a/docs/docs/faqs.md
+++ b/docs/docs/faqs.md
@@ -115,9 +115,9 @@ Modules can be frozen by setting their `._compiled` attribute to be True, indica
- **How do I deal with "context too long" errors?**
-If you're dealing with "context too long" errors in DSPy, you're likely using DSPy optimizers to include demonstrations within your prompt, and this is exceeding your current context window. Try reducing these parameters (e.g. `max_bootstrapped_demos` and `max_labeled_demos`). Additionally, you can also reduce the number of retrieved passages/docs/embeddings to ensure your prompt is fitting within your model context length.
+If you're dealing with "context too long" errors in DSPy, you're likely using DSPy optimizers to include demonstrations within your prompt, and this is exceeding your current context window. DSPy raises this as `dspy.ContextWindowExceededError`, a subclass of `dspy.LMInvalidRequestError`. Try reducing these parameters (e.g. `max_bootstrapped_demos` and `max_labeled_demos`). Additionally, you can also reduce the number of retrieved passages/docs/embeddings to ensure your prompt is fitting within your model context length.
-A more general fix is simply increasing the number of `max_tokens` specified to the LM request (e.g. `lm = dspy.OpenAI(model = ..., max_tokens = ...`).
+A more general fix is simply increasing the number of `max_tokens` specified to the LM request (e.g. `lm = dspy.OpenAI(model = ..., max_tokens = ...`). For other provider failures, catch `dspy.LMError` or a concrete subclass such as `dspy.LMRateLimitError`, `dspy.LMAuthError`, or `dspy.LMServerError`. See the [Errors API reference](api/utils/Errors.md) for details.
## Set Verbose Level
DSPy utilizes the [logging library](https://docs.python.org/3/library/logging.html) to print logs. If you want to debug your DSPy code, set the logging level to DEBUG with the example code below.
diff --git a/docs/docs/learn/programming/language_models.md b/docs/docs/learn/programming/language_models.md
index 9282c0887..2d9715cdb 100644
--- a/docs/docs/learn/programming/language_models.md
+++ b/docs/docs/learn/programming/language_models.md
@@ -250,6 +250,24 @@ lm.history[-1].keys() # access the last call to the LM, with all metadata
dict_keys(['prompt', 'messages', 'kwargs', 'response', 'outputs', 'usage', 'cost', 'timestamp', 'uuid', 'model', 'response_model', 'model_type])
```
+## Handling LM errors
+
+DSPy's built-in `dspy.LM` wraps provider and LiteLLM failures in structured DSPy exceptions. Catch `dspy.LMError` to handle any LM failure, or catch a more specific subclass for targeted behavior.
+
+```python linenums="1"
+try:
+ answer = qa(question="...")
+except dspy.ContextWindowExceededError:
+ # Reduce prompt size, retrieved passages, or demos before retrying.
+ raise
+except dspy.LMRateLimitError as e:
+ print(f"Rate limited by {e.provider}; retry after {e.retry_after} seconds")
+except dspy.LMError as e:
+ print(f"LM failed: code={e.code}, model={e.model}, request_id={e.request_id}")
+```
+
+All DSPy LM errors expose a stable `code` and may include `model`, `provider`, provider `status`, `request_id`, and `retry_after`. If an unknown exception is raised at the LM backend boundary, DSPy raises `dspy.LMUnexpectedError` instead of treating it as an adapter parse failure. See the [Errors API reference](../../api/utils/Errors.md) for the full hierarchy.
+
## Using the Responses API
By default, DSPy calls language models (LMs) using LiteLLM's [Chat Completions API](https://docs.litellm.ai/docs/completion), which is suitable for most standard models and tasks. However, some advanced models, such as OpenAI's reasoning models (e.g., `gpt-5` or other future models), may offer improved quality or additional features when accessed via the [Responses API](https://docs.litellm.ai/docs/response_api), which is supported in DSPy.
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index 0d5ab3c8b..c83c6b54e 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -158,6 +158,7 @@ nav:
- Embeddings: api/tools/Embeddings.md
- PythonInterpreter: api/tools/PythonInterpreter.md
- Utils:
+ - Errors: api/utils/Errors.md
- configure: api/utils/configure.md
- context: api/utils/context.md
- StatusMessage: api/utils/StatusMessage.md
diff --git a/dspy/__init__.py b/dspy/__init__.py
index 76afdb805..76f7c14b3 100644
--- a/dspy/__init__.py
+++ b/dspy/__init__.py
@@ -8,7 +8,26 @@ from dspy.evaluate import Evaluate # isort: skip
from dspy.clients import * # isort: skip
from dspy.adapters import Adapter, ChatAdapter, JSONAdapter, XMLAdapter, TwoStepAdapter, Image, Audio, File, History, Type, Tool, ToolCalls, Code, Reasoning # isort: skip
from dspy.primitives.sandbox_serializable import SandboxSerializable # isort: skip
-from dspy.utils.exceptions import ContextWindowExceededError
+from dspy.utils.exceptions import (
+ AdapterParseError,
+ ContextWindowExceededError,
+ DSPyError,
+ LMAuthError,
+ LMBillingError,
+ LMConfigurationError,
+ LMError,
+ LMInvalidRequestError,
+ LMNotConfiguredError,
+ LMProviderError,
+ LMRateLimitError,
+ LMServerError,
+ LMTimeoutError,
+ LMTransportError,
+ LMUnexpectedError,
+ LMUnsupportedFeatureError,
+ LMUnsupportedModelError,
+ is_retryable_lm_error,
+)
from dspy.utils.logging_utils import configure_dspy_loggers, disable_logging, enable_logging
from dspy.utils.asyncify import asyncify
from dspy.utils.syncify import syncify
diff --git a/dspy/adapters/chat_adapter.py b/dspy/adapters/chat_adapter.py
index 016c885d6..550a4723e 100644
--- a/dspy/adapters/chat_adapter.py
+++ b/dspy/adapters/chat_adapter.py
@@ -15,7 +15,7 @@ from dspy.adapters.utils import (
from dspy.clients.base_lm import BaseLM
from dspy.signatures.signature import Signature
from dspy.utils.callback import BaseCallback
-from dspy.utils.exceptions import AdapterParseError, ContextWindowExceededError
+from dspy.utils.exceptions import AdapterParseError, LMError
field_header_pattern = re.compile(r"\[\[ ## (\w+) ## \]\]")
@@ -74,14 +74,10 @@ class ChatAdapter(Adapter):
# fallback to JSONAdapter
from dspy.adapters.json_adapter import JSONAdapter
- if (
- isinstance(e, ContextWindowExceededError)
- or isinstance(self, JSONAdapter)
- or not self.use_json_adapter_fallback
- ):
- # On context window exceeded error, already using JSONAdapter, or use_json_adapter_fallback is False
- # we don't want to retry with a different adapter. Raise the original error instead of the fallback error.
- raise e
+ if isinstance(e, LMError) or isinstance(self, JSONAdapter) or not self.use_json_adapter_fallback:
+ # On LM errors, already using JSONAdapter, or use_json_adapter_fallback is False, we don't want to
+ # retry with a different adapter. Raise the original error instead of the fallback error.
+ raise
return JSONAdapter()(lm, lm_kwargs, signature, demos, inputs)
async def acall(
@@ -98,14 +94,10 @@ class ChatAdapter(Adapter):
# fallback to JSONAdapter
from dspy.adapters.json_adapter import JSONAdapter
- if (
- isinstance(e, ContextWindowExceededError)
- or isinstance(self, JSONAdapter)
- or not self.use_json_adapter_fallback
- ):
- # On context window exceeded error, already using JSONAdapter, or use_json_adapter_fallback is False
- # we don't want to retry with a different adapter. Raise the original error instead of the fallback error.
- raise e
+ if isinstance(e, LMError) or isinstance(self, JSONAdapter) or not self.use_json_adapter_fallback:
+ # On LM errors, already using JSONAdapter, or use_json_adapter_fallback is False, we don't want to
+ # retry with a different adapter. Raise the original error instead of the fallback error.
+ raise
return await JSONAdapter().acall(lm, lm_kwargs, signature, demos, inputs)
def format_field_description(self, signature: type[Signature]) -> str:
diff --git a/dspy/adapters/json_adapter.py b/dspy/adapters/json_adapter.py
index 59a3b6f56..d6ccec459 100644
--- a/dspy/adapters/json_adapter.py
+++ b/dspy/adapters/json_adapter.py
@@ -19,7 +19,7 @@ from dspy.adapters.utils import (
from dspy.clients.base_lm import BaseLM
from dspy.signatures.signature import Signature, SignatureMeta
from dspy.utils.callback import BaseCallback
-from dspy.utils.exceptions import AdapterParseError
+from dspy.utils.exceptions import AdapterParseError, LMError
logger = logging.getLogger(__name__)
@@ -73,6 +73,10 @@ class JSONAdapter(ChatAdapter):
)
lm_kwargs["response_format"] = structured_output_model
return super().__call__(lm, lm_kwargs, signature, demos, inputs)
+ except LMError:
+ # Provider/backend failures should propagate; the fallback below is only for local structured-output
+ # setup/schema failures where retrying in JSON mode is appropriate.
+ raise
except Exception:
logger.warning("Failed to use structured output format, falling back to JSON mode.")
lm_kwargs["response_format"] = {"type": "json_object"}
@@ -96,6 +100,10 @@ class JSONAdapter(ChatAdapter):
)
lm_kwargs["response_format"] = structured_output_model
return await super().acall(lm, lm_kwargs, signature, demos, inputs)
+ except LMError:
+ # Provider/backend failures should propagate; the fallback below is only for local structured-output
+ # setup/schema failures where retrying in JSON mode is appropriate.
+ raise
except Exception:
logger.warning("Failed to use structured output format, falling back to JSON mode.")
lm_kwargs["response_format"] = {"type": "json_object"}
diff --git a/dspy/adapters/two_step_adapter.py b/dspy/adapters/two_step_adapter.py
index 0e427dea2..012fa5d9b 100644
--- a/dspy/adapters/two_step_adapter.py
+++ b/dspy/adapters/two_step_adapter.py
@@ -9,6 +9,7 @@ from dspy.adapters.utils import get_field_description_string
from dspy.clients.base_lm import BaseLM
from dspy.signatures.field import InputField
from dspy.signatures.signature import Signature, make_signature
+from dspy.utils.exceptions import AdapterParseError, LMError
"""
NOTE/TODO/FIXME:
@@ -100,8 +101,15 @@ class TwoStepAdapter(Adapter):
)
return parsed_result[0]
+ except LMError:
+ raise
except Exception as e:
- raise ValueError(f"Failed to parse response from the original completion: {completion}") from e
+ raise AdapterParseError(
+ adapter_name="TwoStepAdapter",
+ signature=signature,
+ lm_response=completion,
+ message=f"Failed to parse response from the original completion: {e}",
+ ) from e
async def acall(
self,
@@ -141,8 +149,15 @@ class TwoStepAdapter(Adapter):
)
value = value[0]
+ except LMError:
+ raise
except Exception as e:
- raise ValueError(f"Failed to parse response from the original completion: {output}") from e
+ raise AdapterParseError(
+ adapter_name="TwoStepAdapter",
+ signature=signature,
+ lm_response=str(output),
+ message=f"Failed to parse response from the original completion: {e}",
+ ) from e
if tool_calls and tool_call_output_field_name:
tool_calls = [
diff --git a/dspy/clients/base_lm.py b/dspy/clients/base_lm.py
index 80e3a5f75..b08e02c0d 100644
--- a/dspy/clients/base_lm.py
+++ b/dspy/clients/base_lm.py
@@ -226,12 +226,13 @@ class BaseLM:
- [OpenAI text completion format](https://platform.openai.com/docs/api-reference/completions/object)
Raises:
- dspy.ContextWindowExceededError: When the request fails because the
- input exceeds the model's context window. DSPy adapters and
- modules rely on this error to trigger fallback behavior (e.g.
- truncating the prompt and retrying). Each subclass is
- responsible for catching its provider's native error and
- re-raising it as `dspy.ContextWindowExceededError`.
+ dspy.LMError: Base class for LM configuration, transport, provider,
+ and unsupported-feature failures. Notable subclasses include
+ `dspy.ContextWindowExceededError` for context-window failures,
+ which adapters use to avoid inappropriate fallback retries when
+ the prompt is too long. Each subclass should catch its
+ provider's native context-window error and re-raise it as
+ `dspy.ContextWindowExceededError`.
"""
raise NotImplementedError("Subclasses must implement this method.")
@@ -250,12 +251,13 @@ class BaseLM:
- [OpenAI text completion format](https://platform.openai.com/docs/api-reference/completions/object)
Raises:
- dspy.ContextWindowExceededError: When the request fails because the
- input exceeds the model's context window. DSPy adapters and
- modules rely on this error to trigger fallback behavior (e.g.
- truncating the prompt and retrying). Each subclass is
- responsible for catching its provider's native error and
- re-raising it as `dspy.ContextWindowExceededError`.
+ dspy.LMError: Base class for LM configuration, transport, provider,
+ and unsupported-feature failures. Notable subclasses include
+ `dspy.ContextWindowExceededError` for context-window failures,
+ which adapters use to avoid inappropriate fallback retries when
+ the prompt is too long. Each subclass should catch its
+ provider's native context-window error and re-raise it as
+ `dspy.ContextWindowExceededError`.
"""
raise NotImplementedError("Subclasses must implement this method.")
diff --git a/dspy/clients/lm.py b/dspy/clients/lm.py
index ca67acfad..ccece1579 100644
--- a/dspy/clients/lm.py
+++ b/dspy/clients/lm.py
@@ -17,7 +17,23 @@ from dspy.clients.openai import OpenAIProvider
from dspy.clients.provider import Provider, ReinforceJob, TrainingJob
from dspy.clients.utils_finetune import TrainDataFormat
from dspy.utils.callback import BaseCallback
-from dspy.utils.exceptions import ContextWindowExceededError
+from dspy.utils.exceptions import (
+ ContextWindowExceededError,
+ LMAuthError,
+ LMBillingError,
+ LMConfigurationError,
+ LMError,
+ LMInvalidRequestError,
+ LMNotConfiguredError,
+ LMProviderError,
+ LMRateLimitError,
+ LMServerError,
+ LMTimeoutError,
+ LMTransportError,
+ LMUnexpectedError,
+ LMUnsupportedFeatureError,
+ LMUnsupportedModelError,
+)
from .base_lm import BaseLM
@@ -108,9 +124,11 @@ class LM(BaseLM):
if model_pattern:
if (temperature and temperature != 1.0) or (max_tokens and max_tokens < 16000):
- raise ValueError(
+ raise LMConfigurationError(
"OpenAI's reasoning models require passing temperature=1.0 or None and max_tokens >= 16000 or None to "
- "`dspy.LM(...)`, e.g., dspy.LM('openai/gpt-5', temperature=1.0, max_tokens=16000)"
+ "`dspy.LM(...)`, e.g., dspy.LM('openai/gpt-5', temperature=1.0, max_tokens=16000)",
+ model=self.model,
+ provider=self._provider_name,
)
initial_kwargs = dict(temperature=temperature, max_completion_tokens=max_tokens, **kwargs)
else:
@@ -164,12 +182,53 @@ class LM(BaseLM):
return completion_fn, litellm_cache_args
+ def _wrap_litellm_exception(self, exc: Exception) -> LMError:
+ """Convert exceptions raised at the LiteLLM boundary into DSPy LM exceptions."""
+ if isinstance(exc, LMError):
+ return exc
+
+ status = _exception_status(exc)
+ provider = getattr(exc, "llm_provider", None) or self._provider_name
+ model = getattr(exc, "model", None) or self.model
+ message = _exception_message(exc)
+ metadata = {
+ "model": model,
+ "provider": provider,
+ "provider_code": _exception_provider_code(exc),
+ "status": status,
+ "request_id": _exception_request_id(exc),
+ "retry_after": _exception_retry_after(exc),
+ }
+
+ if is_litellm_context_window_error(exc):
+ return ContextWindowExceededError(message=message or "Context window exceeded", **metadata)
+
+ exc_cls = _lm_error_class_from_litellm_exception(exc) or _lm_error_class_from_status(status)
+ return exc_cls(message, **metadata)
+
def forward(
self,
prompt: str | None = None,
messages: list[dict[str, Any]] | None = None,
**kwargs
):
+ """Call the configured LM synchronously.
+
+ LiteLLM/provider exceptions are wrapped in DSPy's structured LM error
+ hierarchy before they are re-raised.
+
+ Args:
+ prompt: Optional prompt text. Ignored when `messages` is provided.
+ messages: Optional chat messages to send to the LM.
+ **kwargs: Per-call LM parameters that override defaults from `LM(...)`.
+
+ Raises:
+ dspy.LMError: Base class for wrapped LM configuration, transport,
+ provider, and unsupported-feature failures. Notable subclasses
+ include `dspy.ContextWindowExceededError` for context-window
+ failures, which adapters use to avoid inappropriate fallback
+ retries when the prompt is too long.
+ """
# Build the request.
kwargs = dict(kwargs)
cache = kwargs.pop("cache", self.cache)
@@ -197,9 +256,9 @@ class LM(BaseLM):
cache=litellm_cache_args,
)
except Exception as e:
- if is_litellm_context_window_error(e):
- raise ContextWindowExceededError(model=self.model) from e
- raise
+ if isinstance(e, LMError):
+ raise
+ raise self._wrap_litellm_exception(e) from e
self._check_truncation(results)
@@ -211,6 +270,23 @@ class LM(BaseLM):
messages: list[dict[str, Any]] | None = None,
**kwargs,
):
+ """Call the configured LM asynchronously.
+
+ LiteLLM/provider exceptions are wrapped in DSPy's structured LM error
+ hierarchy before they are re-raised.
+
+ Args:
+ prompt: Optional prompt text. Ignored when `messages` is provided.
+ messages: Optional chat messages to send to the LM.
+ **kwargs: Per-call LM parameters that override defaults from `LM(...)`.
+
+ Raises:
+ dspy.LMError: Base class for wrapped LM configuration, transport,
+ provider, and unsupported-feature failures. Notable subclasses
+ include `dspy.ContextWindowExceededError` for context-window
+ failures, which adapters use to avoid inappropriate fallback
+ retries when the prompt is too long.
+ """
# Build the request.
kwargs = dict(kwargs)
cache = kwargs.pop("cache", self.cache)
@@ -238,9 +314,9 @@ class LM(BaseLM):
cache=litellm_cache_args,
)
except Exception as e:
- if is_litellm_context_window_error(e):
- raise ContextWindowExceededError(model=self.model) from e
- raise
+ if isinstance(e, LMError):
+ raise
+ raise self._wrap_litellm_exception(e) from e
self._check_truncation(results)
@@ -261,10 +337,13 @@ class LM(BaseLM):
from dspy import settings as settings
if not self.provider.finetunable:
- raise ValueError(
+ raise LMUnsupportedFeatureError(
f"Provider {self.provider} does not support fine-tuning, please specify your provider by explicitly "
"setting `provider` when creating the `dspy.LM` instance. For example, "
- "`dspy.LM('openai/gpt-4.1-mini-2025-04-14', provider=dspy.OpenAIProvider())`."
+ "`dspy.LM('openai/gpt-4.1-mini-2025-04-14', provider=dspy.OpenAIProvider())`.",
+ model=self.model,
+ provider=self._provider_name,
+ features=["finetuning"],
)
def thread_function_wrapper():
@@ -288,8 +367,13 @@ class LM(BaseLM):
# TODO(GRPO Team): Should we return an initialized job here?
from dspy import settings as settings
- err = f"Provider {self.provider} does not implement the reinforcement learning interface."
- assert self.provider.reinforceable, err
+ if not self.provider.reinforceable:
+ raise LMUnsupportedFeatureError(
+ f"Provider {self.provider} does not implement the reinforcement learning interface.",
+ model=self.model,
+ provider=self._provider_name,
+ features=["reinforce"],
+ )
job = self.provider.ReinforceJob(lm=self, train_kwargs=train_kwargs)
job.initialize()
@@ -606,3 +690,130 @@ def _add_dspy_identifier_to_headers(headers: dict[str, Any] | None = None):
"User-Agent": f"DSPy/{dspy.__version__}",
**headers,
}
+
+#--------
+# Errors
+#--------
+
+def _safe_litellm_exception_class(name: str) -> type[Exception] | None:
+ cls = getattr(_get_litellm(), name, None)
+ return cls if isinstance(cls, type) and issubclass(cls, Exception) else None
+
+
+def _lm_error_class_from_litellm_exception(exc: Exception) -> type[LMError] | None:
+ message = _exception_message(exc).lower()
+ class_name = type(exc).__name__.lower()
+ if _exception_status(exc) is None and any(
+ phrase in message for phrase in ("api key", "apikey", "credentials", "environment variable")
+ ):
+ return LMNotConfiguredError
+ if "timeout" in class_name or "timed out" in message or "timeout" in message:
+ return LMTimeoutError
+ if "connection" in class_name or "network" in message or "connection" in message:
+ return LMTransportError
+
+ mappings = [
+ ("AuthenticationError", LMAuthError),
+ ("RateLimitError", LMRateLimitError),
+ ("NotFoundError", LMUnsupportedModelError),
+ ("UnsupportedParamsError", LMUnsupportedFeatureError),
+ ("UnprocessableEntityError", LMInvalidRequestError),
+ ("ContentPolicyViolationError", LMInvalidRequestError),
+ ("BadRequestError", LMInvalidRequestError),
+ ("InvalidRequestError", LMInvalidRequestError),
+ ("InternalServerError", LMServerError),
+ ("ServiceUnavailableError", LMServerError),
+ ("APIConnectionError", LMTransportError),
+ ("APIResponseValidationError", LMProviderError),
+ ("BudgetExceededError", LMBillingError),
+ ("RouterRateLimitError", LMRateLimitError),
+ ]
+ for litellm_name, dspy_cls in mappings:
+ litellm_cls = _safe_litellm_exception_class(litellm_name)
+ if litellm_cls is not None and isinstance(exc, litellm_cls):
+ return dspy_cls
+ return None
+
+
+def _lm_error_class_from_status(status: int | None) -> type[LMError]:
+ if status in (401, 403):
+ return LMAuthError
+ if status == 402:
+ return LMBillingError
+ if status == 404:
+ return LMUnsupportedModelError
+ if status == 408:
+ return LMTimeoutError
+ if status == 429:
+ return LMRateLimitError
+ if status is not None and 400 <= status < 500:
+ return LMInvalidRequestError
+ if status is not None and status >= 500:
+ return LMServerError
+ return LMUnexpectedError if status is None else LMProviderError
+
+
+# Best-effort LiteLLM/provider exception metadata extraction.
+#
+# LiteLLM exception metadata is not exposed as a single stable typed shape across providers, exception classes, and
+# LiteLLM versions. Keep the defensive getattr-based extraction localized here so the rest of DSPy sees structured
+# DSPyError metadata.
+def _exception_status(exc: Exception) -> int | None:
+ status = getattr(exc, "status_code", None)
+ if status is None:
+ response = getattr(exc, "response", None)
+ status = getattr(response, "status_code", None)
+ try:
+ return int(status) if status is not None else None
+ except (TypeError, ValueError):
+ return None
+
+
+def _exception_message(exc: Exception) -> str:
+ message = getattr(exc, "message", None)
+ if message is None:
+ message = str(exc)
+ return str(message)
+
+
+def _exception_headers(exc: Exception):
+ response = getattr(exc, "response", None)
+ return getattr(response, "headers", None) or getattr(exc, "headers", None) or {}
+
+
+def _exception_header(exc: Exception, name: str) -> str | None:
+ headers = _exception_headers(exc)
+ if not headers:
+ return None
+ try:
+ return headers.get(name) or headers.get(name.lower())
+ except AttributeError:
+ return None
+
+
+def _exception_request_id(exc: Exception) -> str | None:
+ return (
+ _exception_header(exc, "x-request-id")
+ or _exception_header(exc, "request-id")
+ or _exception_header(exc, "x-amzn-requestid")
+ or _exception_header(exc, "x-ms-request-id")
+ )
+
+
+def _exception_retry_after(exc: Exception) -> float | None:
+ retry_after = _exception_header(exc, "retry-after")
+ try:
+ return float(retry_after) if retry_after is not None else None
+ except (TypeError, ValueError):
+ return None
+
+
+def _exception_provider_code(exc: Exception) -> str | None:
+ body = getattr(exc, "body", None)
+ if isinstance(body, dict):
+ error = body.get("error")
+ if isinstance(error, dict) and error.get("code") is not None:
+ return str(error["code"])
+ if body.get("code") is not None:
+ return str(body["code"])
+ return None
diff --git a/dspy/utils/exceptions.py b/dspy/utils/exceptions.py
index 928bbe351..f088dbca7 100644
--- a/dspy/utils/exceptions.py
+++ b/dspy/utils/exceptions.py
@@ -1,29 +1,238 @@
+from __future__ import annotations
+
+from typing import Any
from dspy.signatures.signature import Signature
-class ContextWindowExceededError(Exception):
+class DSPyError(Exception):
+ """Base class for DSPy errors with structured metadata.
+
+ Args:
+ message: Human-readable error message.
+ code: Stable DSPy error code. Defaults to the class code.
+ model: Model identifier involved in the failure.
+ provider: Provider or backend that returned the error.
+ provider_code: Provider-specific error code, when available.
+ status: HTTP status code, when the error came from an HTTP response.
+ request_id: Provider request ID, when available.
+ retry_after: Suggested retry delay in seconds, when available.
+ """
+
+ default_code: str | None = None
+
+ def __init__(
+ self,
+ message: str = "",
+ *,
+ code: str | None = None,
+ model: str | None = None,
+ provider: str | None = None,
+ provider_code: str | None = None,
+ status: int | None = None,
+ request_id: str | None = None,
+ retry_after: float | None = None,
+ ):
+ self.message = message
+ self.code = code or self.default_code
+ self.model = model
+ self.provider = provider
+ self.provider_code = provider_code
+ self.status = status
+ self.request_id = request_id
+ self.retry_after = retry_after
+
+ prefix = f"[{model}] " if model else ""
+ super().__init__(f"{prefix}{message}" if message else prefix.rstrip())
+
+
+class LMError(DSPyError):
+ """Base class for language model errors.
+
+ Catch this class to handle any failure raised while configuring or calling
+ an LM. Concrete subclasses identify whether the failure came from local
+ configuration, transport, provider authentication, rate limits, invalid
+ requests, unsupported features, or provider server errors.
+ """
+
+ default_code = "lm_error"
+
+
+class LMTransportError(LMError):
+ """The LM request failed before the provider returned a response.
+
+ This commonly represents network, DNS, TLS, connection-reset, or similar
+ client-side transport failures.
+ """
+
+ default_code = "transport"
+
+
+class LMConfigurationError(LMError):
+ """The LM or provider client is not configured correctly."""
+
+ default_code = "configuration"
+
+
+class LMNotConfiguredError(LMConfigurationError):
+ """The LM is missing required provider configuration or credentials."""
+
+ default_code = "not_configured"
+
+
+class LMUnsupportedFeatureError(LMError):
+ """The LM, provider, or DSPy provider wrapper does not support a requested feature.
+
+ Args:
+ message: Human-readable error message.
+ features: Feature names that were requested but unavailable, such as
+ `"finetuning"`, `"reinforce"`, or `"structured_outputs"`.
+ issues: Optional detailed reasons the requested feature could not be
+ used.
+ **kwargs: Structured error metadata accepted by `DSPyError`.
+ """
+
+ default_code = "unsupported_feature"
+
+ def __init__(
+ self,
+ message: str = "",
+ *,
+ features: list[str] | None = None,
+ issues: list[str] | None = None,
+ **kwargs: Any,
+ ):
+ self.features = list(features or [])
+ self.issues = list(issues or [])
+ super().__init__(message, **kwargs)
+
+
+class LMProviderError(LMError):
+ """The provider returned an error response.
+
+ Provider errors include structured metadata when available, such as HTTP
+ `status`, provider `request_id`, provider-specific `provider_code`, and
+ `retry_after` for rate limits.
+ """
+
+ default_code = "provider"
+
+
+class LMUnexpectedError(LMError):
+ """An unexpected failure occurred at the LM provider boundary.
+
+ DSPy raises this when an exception is raised while calling the LM backend,
+ but the exception does not match a known provider error class and does not
+ include an HTTP status code. This keeps adapter fallback behavior from
+ treating unknown LM-boundary failures as parse failures while avoiding
+ over-classifying them as provider response errors.
+ """
+
+ default_code = "unexpected"
+
+
+class LMAuthError(LMProviderError):
+ """The provider rejected the request because authentication failed."""
+
+ default_code = "auth"
+
+
+class LMBillingError(LMProviderError):
+ """The provider rejected the request because billing or quota failed."""
+
+ default_code = "billing"
+
+
+class LMRateLimitError(LMProviderError):
+ """The provider rate-limited the request.
+
+ Check the `retry_after` attribute for a provider-suggested retry delay when
+ one is available.
+ """
+
+ default_code = "rate_limit"
+
+
+class LMInvalidRequestError(LMProviderError):
+ """The provider rejected the request shape or resource."""
+
+ default_code = "invalid_request"
+
+
+class ContextWindowExceededError(LMInvalidRequestError):
"""Raised when the prompt exceeds the model's context window.
- Any `BaseLM` subclass should raise this error (or a subclass of it) when the
- request fails because the input is too long for the model. Adapters and some
- modules rely on catching this specific type to decide whether a fallback
- retry is appropriate.
+ Any LM subclass should raise this error, or a subclass of it, when the
+ request fails because the input is too long for the model. Adapters and
+ some modules rely on catching this specific type to decide whether a
+ fallback retry is appropriate.
Args:
model: The model identifier that rejected the request.
message: Description of the error. Defaults to `"Context window exceeded"`.
+ **kwargs: Structured error metadata such as `provider`, `status`, or
+ `request_id`.
"""
- def __init__(self, *, model: str | None = None, message: str = "Context window exceeded"):
- self.model = model
- msg = message
- prefix = f"[{model}] " if model else ""
- super().__init__(f"{prefix}{msg}")
+ default_code = "context_window_exceeded"
+ def __init__(
+ self,
+ *,
+ model: str | None = None,
+ message: str = "Context window exceeded",
+ **kwargs: Any,
+ ):
+ super().__init__(message, model=model, **kwargs)
+
+
+class LMUnsupportedModelError(LMInvalidRequestError):
+ """The requested model is unavailable or unsupported by the provider."""
+
+ default_code = "unsupported_model"
+
+
+class LMTimeoutError(LMProviderError):
+ """The provider request timed out."""
+
+ default_code = "timeout"
+
+
+class LMServerError(LMProviderError):
+ """The provider failed while handling the request."""
+
+ default_code = "server"
+
+
+_RETRYABLE_LM_ERRORS = (LMRateLimitError, LMTimeoutError, LMServerError, LMTransportError)
+
+
+def is_retryable_lm_error(error: Exception) -> bool:
+ """Return whether an LM error is generally safe to retry.
+
+ DSPy's built-in `LM` delegates provider retries to LiteLLM, but callers and
+ higher-level orchestration code can use this helper to classify wrapped LM
+ failures after provider retries are exhausted. Retryability is advisory:
+ callers should still respect provider policy and `retry_after` when present.
+
+ Args:
+ error: The exception to classify.
+ """
+ return isinstance(error, _RETRYABLE_LM_ERRORS)
+
+
+class AdapterParseError(DSPyError):
+ """Raised when an adapter cannot parse an LM response into signature outputs.
+
+ Args:
+ adapter_name: Name of the adapter that failed to parse the response.
+ signature: DSPy signature whose output fields were expected.
+ lm_response: Raw LM response text or representation being parsed.
+ message: Optional additional context about the parse failure.
+ parsed_result: Partial parsed result, if any.
+ """
-class AdapterParseError(Exception):
- """Exception raised when adapter cannot parse the LM response."""
+ default_code = "adapter_parse_error"
def __init__(
self,
diff --git a/tests/adapters/test_json_adapter.py b/tests/adapters/test_json_adapter.py
index c6dd1f679..36abe4a7e 100644
--- a/tests/adapters/test_json_adapter.py
+++ b/tests/adapters/test_json_adapter.py
@@ -736,22 +736,6 @@ def test_json_adapter_not_using_structured_outputs_when_not_supported_by_model()
assert "response_format" not in call_kwargs
-def test_json_adapter_falls_back_when_structured_outputs_fails():
- class TestSignature(dspy.Signature):
- input1: str = dspy.InputField()
- output1: str = dspy.OutputField(desc="String output field")
-
- dspy.configure(lm=dspy.LM(model="openai/gpt-4o", cache=False), adapter=dspy.JSONAdapter())
- program = dspy.Predict(TestSignature)
- with mock.patch("litellm.completion") as mock_completion:
- mock_completion.side_effect = [Exception("Bad structured outputs!"), mock_completion.return_value]
- program(input1="Test input")
- assert mock_completion.call_count == 2
- _, first_call_kwargs = mock_completion.call_args_list[0]
- assert issubclass(first_call_kwargs.get("response_format"), pydantic.BaseModel)
- _, second_call_kwargs = mock_completion.call_args_list[1]
- assert second_call_kwargs.get("response_format") == {"type": "json_object"}
-
def test_json_adapter_with_structured_outputs_does_not_mutate_original_signature():
class OutputField3(pydantic.BaseModel):
@@ -1290,7 +1274,7 @@ async def test_json_adapter_on_pydantic_model_async():
assert result.answer.result == "Paris"
-def test_json_adapter_fallback_to_json_mode_on_structured_output_failure():
+def test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error():
class TestSignature(dspy.Signature):
question: str = dspy.InputField()
answer: str = dspy.OutputField(desc="String output field")
@@ -1299,25 +1283,15 @@ def test_json_adapter_fallback_to_json_mode_on_structured_output_failure():
program = dspy.Predict(TestSignature)
with mock.patch("litellm.completion") as mock_completion:
- # First call raises error to simulate structured output failure, second call returns a valid response
- mock_completion.side_effect = [
- RuntimeError("Structured output failed!"),
- ModelResponse(choices=[Choices(message=Message(content="{'answer': 'Test output'}"))]),
- ]
+ mock_completion.side_effect = RuntimeError("Structured output failed!")
- result = program(question="Dummy question!")
- # The parse should succeed on the second call
- assert mock_completion.call_count == 2
- assert result.answer == "Test output"
+ with pytest.raises(dspy.LMUnexpectedError, match="Structured output failed"):
+ program(question="Dummy question!")
- # The first call should have tried structured output
+ assert mock_completion.call_count == 1
_, first_call_kwargs = mock_completion.call_args_list[0]
assert issubclass(first_call_kwargs.get("response_format"), pydantic.BaseModel)
- # The second call should have used JSON mode
- _, second_call_kwargs = mock_completion.call_args_list[1]
- assert second_call_kwargs.get("response_format") == {"type": "json_object"}
-
def test_json_adapter_json_mode_no_structured_outputs():
class TestSignature(dspy.Signature):
@@ -1379,7 +1353,7 @@ async def test_json_adapter_json_mode_no_structured_outputs_async():
@pytest.mark.asyncio
-async def test_json_adapter_fallback_to_json_mode_on_structured_output_failure_async():
+async def test_json_adapter_does_not_fallback_to_json_mode_on_structured_output_lm_error_async():
class TestSignature(dspy.Signature):
question: str = dspy.InputField()
answer: str = dspy.OutputField(desc="String output field")
@@ -1387,26 +1361,16 @@ async def test_json_adapter_fallback_to_json_mode_on_structured_output_failure_a
program = dspy.Predict(TestSignature)
with mock.patch("litellm.acompletion") as mock_acompletion:
- # First call raises error to simulate structured output failure, second call returns a valid response
- mock_acompletion.side_effect = [
- RuntimeError("Structured output failed!"),
- ModelResponse(choices=[Choices(message=Message(content="{'answer': 'Test output'}"))]),
- ]
+ mock_acompletion.side_effect = RuntimeError("Structured output failed!")
with dspy.context(lm=dspy.LM(model="openai/gpt-4o-mini", cache=False), adapter=dspy.JSONAdapter()):
- result = await program.acall(question="Dummy question!")
- # The parse should succeed on the second call
- assert mock_acompletion.call_count == 2
- assert result.answer == "Test output"
+ with pytest.raises(dspy.LMUnexpectedError, match="Structured output failed"):
+ await program.acall(question="Dummy question!")
- # The first call should have tried structured output
+ assert mock_acompletion.call_count == 1
_, first_call_kwargs = mock_acompletion.call_args_list[0]
assert issubclass(first_call_kwargs.get("response_format"), pydantic.BaseModel)
- # The second call should have used JSON mode
- _, second_call_kwargs = mock_acompletion.call_args_list[1]
- assert second_call_kwargs.get("response_format") == {"type": "json_object"}
-
def test_error_message_on_json_adapter_failure():
class TestSignature(dspy.Signature):
@@ -1420,13 +1384,13 @@ def test_error_message_on_json_adapter_failure():
with mock.patch("litellm.completion") as mock_completion:
mock_completion.side_effect = RuntimeError("RuntimeError!")
- with pytest.raises(RuntimeError) as error:
+ with pytest.raises(dspy.LMUnexpectedError) as error:
program(question="Dummy question!")
assert "RuntimeError!" in str(error.value)
mock_completion.side_effect = ValueError("ValueError!")
- with pytest.raises(ValueError) as error:
+ with pytest.raises(dspy.LMUnexpectedError) as error:
program(question="Dummy question!")
assert "ValueError!" in str(error.value)
@@ -1443,13 +1407,13 @@ async def test_error_message_on_json_adapter_failure_async():
with mock.patch("litellm.acompletion") as mock_acompletion:
with dspy.context(lm=dspy.LM(model="openai/gpt-4o-mini", cache=False), adapter=dspy.JSONAdapter()):
mock_acompletion.side_effect = RuntimeError("RuntimeError!")
- with pytest.raises(RuntimeError) as error:
+ with pytest.raises(dspy.LMUnexpectedError) as error:
await program.acall(question="Dummy question!")
assert "RuntimeError!" in str(error.value)
mock_acompletion.side_effect = ValueError("ValueError!")
- with pytest.raises(ValueError) as error:
+ with pytest.raises(dspy.LMUnexpectedError) as error:
await program.acall(question="Dummy question!")
assert "ValueError!" in str(error.value)
diff --git a/tests/adapters/test_two_step_adapter.py b/tests/adapters/test_two_step_adapter.py
index 6eaee233a..3ae99e222 100644
--- a/tests/adapters/test_two_step_adapter.py
+++ b/tests/adapters/test_two_step_adapter.py
@@ -231,5 +231,5 @@ def test_two_step_adapter_parse_errors():
adapter = dspy.TwoStepAdapter(mock_extraction_lm)
- with pytest.raises(ValueError, match="Failed to parse response"):
+ with pytest.raises(dspy.AdapterParseError, match="Failed to parse response"):
adapter.parse(TestSignature, first_response)
diff --git a/tests/clients/test_lm.py b/tests/clients/test_lm.py
index b53430727..d5bd2f3e9 100644
--- a/tests/clients/test_lm.py
+++ b/tests/clients/test_lm.py
@@ -276,6 +276,68 @@ def test_lm_calls_support_pydantic_models(litellm_test_server):
lm("Query")
+def test_lm_wraps_litellm_errors_with_metadata():
+ lm = dspy.LM("openai/gpt-4o-mini")
+ response = mock.Mock()
+ response.status_code = 429
+ response.headers = {"x-request-id": "req-123", "retry-after": "2.5"}
+
+ error = litellm.RateLimitError(message="too many requests", llm_provider="openai", model="gpt-4o", response=response)
+ wrapped = lm._wrap_litellm_exception(error)
+
+ assert isinstance(wrapped, dspy.LMRateLimitError)
+ assert wrapped.model == "gpt-4o"
+ assert wrapped.provider == "openai"
+ assert wrapped.status == 429
+ assert wrapped.request_id == "req-123"
+ assert wrapped.retry_after == 2.5
+
+
+def test_lm_wraps_litellm_context_window_error():
+ lm = dspy.LM("openai/gpt-4o-mini")
+ error = litellm.ContextWindowExceededError(message="too long", llm_provider="openai", model="gpt-4o")
+ wrapped = lm._wrap_litellm_exception(error)
+
+ assert isinstance(wrapped, dspy.ContextWindowExceededError)
+ assert isinstance(wrapped, dspy.LMError)
+ assert wrapped.model == "gpt-4o"
+ assert wrapped.provider == "openai"
+
+
+def test_lm_wraps_unknown_boundary_error_as_unexpected_error():
+ lm = dspy.LM("openai/gpt-4o-mini")
+ wrapped = lm._wrap_litellm_exception(RuntimeError("local boundary failure"))
+
+ assert isinstance(wrapped, dspy.LMUnexpectedError)
+ assert wrapped.code == "unexpected"
+ assert wrapped.model == "openai/gpt-4o-mini"
+
+
+def test_lm_preserves_existing_lm_error_without_self_cause():
+ error = dspy.LMRateLimitError("rate limited", model="openai/gpt-4o-mini")
+ lm = dspy.LM("openai/gpt-4o-mini", cache=False)
+
+ with mock.patch("dspy.clients.lm.litellm_completion", side_effect=error):
+ with pytest.raises(dspy.LMRateLimitError) as exc_info:
+ lm("question")
+
+ assert exc_info.value is error
+ assert exc_info.value.__cause__ is None
+
+
+async def test_lm_preserves_existing_lm_error_without_self_cause_async():
+ error = dspy.LMRateLimitError("rate limited", model="openai/gpt-4o-mini")
+ lm = dspy.LM("openai/gpt-4o-mini", cache=False)
+
+ with mock.patch("dspy.clients.lm.alitellm_completion", side_effect=error):
+ with pytest.raises(dspy.LMRateLimitError) as exc_info:
+ await lm.acall("question")
+
+ assert exc_info.value is error
+ assert exc_info.value.__cause__ is None
+
+
def test_retry_number_set_correctly():
lm = dspy.LM("openai/gpt-4o-mini", num_retries=3)
with mock.patch("litellm.completion") as mock_completion:
@@ -297,7 +359,7 @@ def test_retry_made_on_system_errors():
lm = dspy.LM(model="openai/gpt-4o-mini", max_tokens=250, num_retries=3)
with mock.patch.object(litellm.OpenAIChatCompletion, "completion", side_effect=mock_create):
- with pytest.raises(RateLimitError):
+ with pytest.raises(dspy.LMRateLimitError):
lm("question")
assert retry_tracking[0] == 4
@@ -338,7 +400,7 @@ def test_reasoning_model_token_parameter():
def test_reasoning_model_requirements(model_name):
# Should raise assertion error if temperature or max_tokens requirements not met
with pytest.raises(
- ValueError,
+ dspy.LMConfigurationError,
match=r"reasoning models require passing temperature=1\.0 or None and max_tokens >= 16000 or None",
):
dspy.LM(
@@ -506,7 +568,7 @@ def test_exponential_backoff_retry():
lm = dspy.LM(model="openai/gpt-3.5-turbo", max_tokens=250, num_retries=3)
with mock.patch.object(litellm.OpenAIChatCompletion, "completion", side_effect=mock_create):
- with pytest.raises(RateLimitError):
+ with pytest.raises(dspy.LMRateLimitError):
lm("question")
# The first retry happens immediately regardless of the configuration
diff --git a/tests/teleprompt/test_bootstrap_trace.py b/tests/teleprompt/test_bootstrap_trace.py
index 020f4cba3..062bd3cc8 100644
--- a/tests/teleprompt/test_bootstrap_trace.py
+++ b/tests/teleprompt/test_bootstrap_trace.py
@@ -1,4 +1,4 @@
-from typing import Any
+from typing import Any, ClassVar
from unittest import mock
from litellm import Choices, Message, ModelResponse
@@ -63,14 +63,13 @@ def test_bootstrap_trace_data():
call_count = completion_side_effect.call_count
completion_side_effect.call_count += 1
- if call_count == 5: # Third call (0-indexed)
- # Return malformed response that will cause AdapterParseError
+ if call_count in (2, 3):
+ # Return malformed responses for both structured-output mode and JSON-mode fallback.
return ModelResponse(
choices=[Choices(message=Message(content="This is an invalid JSON!"))],
model="openai/gpt-4o-mini",
)
- else:
- return successful_responses[call_count]
+ return successful_responses[call_count if call_count < 2 else call_count - 2]
completion_side_effect.call_count = 0
@@ -80,6 +79,7 @@ def test_bootstrap_trace_data():
program=program,
dataset=dataset,
metric=exact_match_metric,
+ num_threads=1,
raise_on_error=False,
capture_failed_parses=True,
)
@@ -138,7 +138,7 @@ def test_bootstrap_trace_data_passes_callback_metadata(monkeypatch):
captured_metadata["value"] = callback_metadata
class _Result:
- results: list[Any] = []
+ results: ClassVar[list[Any]] = []
return _Result()
diff --git a/tests/utils/test_exceptions.py b/tests/utils/test_exceptions.py
index 21483b0bd..707ddea3e 100644
--- a/tests/utils/test_exceptions.py
+++ b/tests/utils/test_exceptions.py
@@ -1,9 +1,56 @@
import dspy
-from dspy.utils.exceptions import AdapterParseError, ContextWindowExceededError
+from dspy.utils.exceptions import (
+ AdapterParseError,
+ ContextWindowExceededError,
+ DSPyError,
+ LMError,
+ LMInvalidRequestError,
+)
+
+
+def test_lm_errors_are_exported_from_dspy():
+ assert dspy.DSPyError is not None
+ assert dspy.LMError is LMError
+ assert dspy.LMUnexpectedError is not None
+ assert dspy.AdapterParseError is AdapterParseError
+ assert dspy.is_retryable_lm_error is not None
+
+
+def test_retryable_lm_errors_classification():
+ assert dspy.is_retryable_lm_error(dspy.LMRateLimitError())
+ assert dspy.is_retryable_lm_error(dspy.LMTimeoutError())
+ assert dspy.is_retryable_lm_error(dspy.LMServerError())
+ assert dspy.is_retryable_lm_error(dspy.LMTransportError())
+ assert not dspy.is_retryable_lm_error(dspy.LMAuthError())
+ assert not dspy.is_retryable_lm_error(dspy.LMInvalidRequestError())
+ assert not dspy.is_retryable_lm_error(dspy.LMUnexpectedError())
+ assert not dspy.is_retryable_lm_error(ValueError("not an LM error"))
+
+
+def test_lm_error_metadata():
+ error = dspy.LMRateLimitError(
+ "rate limited",
+ model="openai/gpt-4o",
+ provider="openai",
+ status=429,
+ request_id="req-123",
+ retry_after=2.5,
+ )
+
+ assert error.code == "rate_limit"
+ assert error.model == "openai/gpt-4o"
+ assert error.provider == "openai"
+ assert error.status == 429
+ assert error.request_id == "req-123"
+ assert error.retry_after == 2.5
+ assert str(error) == "[openai/gpt-4o] rate limited"
def test_context_window_exceeded_error_defaults():
error = ContextWindowExceededError()
+ assert isinstance(error, LMInvalidRequestError)
+ assert isinstance(error, LMError)
+ assert error.code == "context_window_exceeded"
assert error.model is None
assert str(error) == "Context window exceeded"
@@ -33,6 +80,8 @@ def test_adapter_parse_error_basic():
error = AdapterParseError(adapter_name=adapter_name, signature=signature, lm_response=lm_response)
+ assert isinstance(error, DSPyError)
+ assert error.code == "adapter_parse_error"
assert error.adapter_name == adapter_name
assert error.signature == signature
assert error.lm_response == lm_response