DSPy: The framework for programming—not prompting—language models
Python
Makes BaseLM.__call__ support typed LMRequest / LMResponse while pre… (#9877)
Maxime Riveste031493268fb
diff --git a/docs/docs/community/normalized-lm-api-migration.md b/docs/docs/community/normalized-lm-api-migration.mdindex bf2b6a1d0..af6008580 100644--- a/docs/docs/community/normalized-lm-api-migration.md+++ b/docs/docs/community/normalized-lm-api-migration.md@@ -104,15 +104,17 @@ response.provider_dataThe typed path also makes direct `lm(...)` calls more expressive. Strings, typed messages, media parts, previous responses, and explicit `LMRequest` objects all flow through one call API.!!! warning "Experimental 3.3 API"- The typed LM symbols are importable without `experimental=True`. During the 3.3 release-candidate period, the direct typed `lm(...)` call path will land incrementally behind `dspy.context(experimental=True)` as the API settles. These helpers are expected to be available in DSPy 3.3. Key helpers such as `dspy.LMRequest`, `dspy.LMResponse`, `dspy.System`, `dspy.User`, `dspy.Assistant`, `dspy.ToolCall`, and `dspy.ToolResult` are available at the top level. The complete typed LM vocabulary is available under `dspy.core.types`, e.g. `dspy.core.types.LMTextPart`.+ The typed LM symbols are importable without `experimental=True`. In DSPy 3.3, direct typed `lm(...)` calls are available behind `dspy.context(experimental=True)` while the API settles. Key helpers such as `dspy.LMRequest`, `dspy.LMResponse`, `dspy.System`, `dspy.User`, `dspy.Assistant`, `dspy.ToolCall`, and `dspy.ToolResult` are available at the top level. The complete typed LM vocabulary is available under `dspy.core.types`, e.g. `dspy.core.types.LMTextPart` and `dspy.core.types.LMImagePart`.Multimodal request with instructions:```python+from dspy.core.types import LMImagePart+with dspy.context(experimental=True):response = lm(dspy.System("Be concise."),- dspy.User("Describe this image.", dspy.Image("https://example.com/dog.png")),+ dspy.User("Describe this image.", dspy.Image("https://example.com/dog.png")), #Coming soon!temperature=0.2,)```diff --git a/dspy/clients/base_lm.py b/dspy/clients/base_lm.pyindex e3b842eef..f11715a1d 100644--- a/dspy/clients/base_lm.py+++ b/dspy/clients/base_lm.py@@ -6,7 +6,15 @@ import uuidimport warningsfrom typing import Any, Literal, TextIO-from dspy.core.types import LMResponse+from dspy.clients.openai_format import (+ completion_to_lm_response,+ cost_from_response,+ lm_response_from_legacy_outputs,+ responses_to_lm_response,+ to_openai_chat_request,+ usage_from_response,+)+from dspy.core.types import LMHistoryEntry, LMRequest, LMResponsefrom dspy.dsp.utils import settingsfrom dspy.utils.callback import BaseCallback, with_callbacksfrom dspy.utils.inspect_history import pretty_print_history@@ -47,56 +55,109 @@ def _import_lm_class(class_path: str) -> type:class BaseLM:- """Base class for handling LLM calls.+ """Base class for DSPy language models.- Most users can directly use the `dspy.LM` class, which is a subclass of `BaseLM`. Users can also implement their- own subclasses of `BaseLM` to support custom LLM providers and inject custom logic. To do so, simply override the- `forward` method and make sure the return format is identical to the- [OpenAI response format](https://platform.openai.com/docs/api-reference/responses/object).+ Most users should use `dspy.LM`, which is a `BaseLM` subclass.- Subclasses whose state is captured by `BaseLM.__init__` can use the default `dump_state` and `load_state`- methods. Subclasses with extra persistent state should override both methods.+ For advanced use cases, such as custom language model backends, users can+ subclass `BaseLM` and implement `forward()`.- Examples:+ DSPy is migrating `forward()` from the legacy OpenAI/LiteLLM-shaped+ contract to a typed DSPy contract. During this migration, subclasses should+ declare which contract they implement with `forward_contract`:++ - `forward_contract = "typed_lm"`: implement+ `forward(request: dspy.LMRequest) -> dspy.LMResponse`. This is the+ preferred contract for new custom LMs.+ - `forward_contract = "legacy"`: implement+ `forward(prompt=None, messages=None, **kwargs)` and return an OpenAI-like+ provider response. This remains the default during the migration.++ `BaseLM.__call__()` is the compatibility boundary. In DSPy 3.3 and 3.4,+ ordinary calls preserve the legacy public return value, `list[str | dict]`:```python- from openai import OpenAI+ outputs = lm("What is DSPy?")+ outputs = lm(messages=[{"role": "user", "content": "What is DSPy?"}])+ ```- import dspy+ Calls can flow internally through the typed `LMRequest` / `LMResponse` path+ without changing the public return shape. The typed path is used when the+ caller passes an explicit `dspy.LMRequest`, when+ `dspy.context(experimental=True)` is active, or when the subclass declares+ `forward_contract = "typed_lm"`. It accepts richer direct-call inputs,+ including `dspy.System`, `dspy.User`, `dspy.Assistant`, `dspy.ToolResult`,+ content parts, and prior `dspy.LMResponse` objects. The public return value+ remains legacy outputs unless the caller explicitly opts into typed output+ with an `LMRequest` or `experimental=True`.+ Example typed direct call:- class MyLM(dspy.BaseLM):- @property- def supports_function_calling(self) -> bool:- return self.model.startswith("openai/gpt-4o")+ ```python+ with dspy.context(experimental=True):+ response = lm(+ dspy.System("You are concise."),+ dspy.User("What is DSPy?"),+ )+ print(response.text)+ ```- @property- def supports_reasoning(self) -> bool:- return self.model.startswith("anthropic/claude-3-7")+ `LMResponse` is designed to feel familiar to users of the legacy output+ list while carrying substantially more structure, including typed outputs,+ usage, cache status, provider metadata, tool calls, reasoning, citations,+ and multimodal content.- @property- def supports_response_schema(self) -> bool:- return self.model.startswith("openai/gpt-4o")+ LMs must be serializable as part of saved DSPy programs. The default+ `dump_state()` and `load_state()` implementations support subclasses whose+ persistent state is fully captured by `BaseLM.__init__()` arguments. If a+ subclass stores additional persistent state, override both methods.- @property- def supported_params(self) -> set[str]:- if self.model.startswith("openai/gpt-4o"):- return {"response_format"} # accepts response_format=...- return set()+ Examples:+ Preferred typed custom LM:- def forward(self, prompt, messages=None, **kwargs):- client = OpenAI()- return client.chat.completions.create(- model=self.model,- messages=messages or [{"role": "user", "content": prompt}],- **self.kwargs,- )+ ```python+ import dspy- lm = MyLM(model="gpt-4o-mini")- dspy.configure(lm=lm)- print(dspy.Predict("q->a")(q="Why did the chicken cross the kitchen?"))- ```+ class EchoLM(dspy.BaseLM):+ forward_contract = "typed_lm"++ def forward(self, request: dspy.LMRequest) -> dspy.LMResponse:+ return dspy.LMResponse.from_text("hello", model=request.model)+++ lm = EchoLM(model="test/echo")++ with dspy.context(experimental=True):+ response = lm(dspy.User("Say hello."))+ print(response.text)+ ```++ Legacy custom LM for an OpenAI-like provider:++ ```python+ from openai import OpenAI++ import dspy+++ class MyLegacyLM(dspy.BaseLM):+ forward_contract = "legacy"++ def forward(self, prompt=None, messages=None, **kwargs):+ client = OpenAI()+ return client.chat.completions.create(+ model=self.model,+ messages=messages or [{"role": "user", "content": prompt}],+ **self.kwargs,+ **kwargs,+ )+++ lm = MyLegacyLM(model="gpt-4o-mini")+ dspy.configure(lm=lm)+ print(dspy.Predict("q -> a")(q="Why did the chicken cross the kitchen?"))+ ```"""forward_contract: ForwardContract = "legacy"@@ -260,25 +321,287 @@ class BaseLM:@with_callbacksdef __call__(self,+ *items: Any,prompt: str | None = None,messages: list[dict[str, Any]] | None = None,- **kwargs- ) -> list[dict[str, Any] | str]:- response = self.forward(prompt=prompt, messages=messages, **kwargs)- outputs = self._process_lm_response(response, prompt, messages, **kwargs)+ request: LMRequest | None = None,+ **kwargs,+ ) -> LMResponse | list[dict[str, Any] | str]:+ """Call the language model synchronously.- return outputs+ The default call path preserves DSPy's legacy behavior and returns `list[str | dict]`. Calls flow internally+ through `LMRequest` / `LMResponse` when either:++ - `request=` is provided or the first positional argument is an `LMRequest`, or+ - `dspy.context(experimental=True)` is active, or+ - the subclass declares `forward_contract = "typed_lm"`.++ In the typed request path, positional `items` are normalized with `LMRequest.from_call()`. This supports+ a single prompt string, direct message objects such as `dspy.System(...)`, `dspy.User(...)`, and+ `dspy.Assistant(...)`, `dspy.ToolResult(...)` tool messages, and prior `LMResponse` values as assistant turns.++ Args:+ *items: Optional direct-call inputs. In the legacy path this may contain at most one prompt string. In the+ typed path it may contain normalized messages, message sequences, prior `LMResponse` values, or content+ parts accepted by `LMRequest.from_call()`.+ prompt: Optional prompt string. Do not combine with positional prompt input.+ messages: Optional OpenAI-chat-shaped messages. Do not combine with `items` or `prompt` in the typed path.+ request: Optional explicit normalized request. Call kwargs override request config when provided.+ **kwargs: Per-call generation parameters.++ Returns:+ `LMResponse` for explicit `LMRequest` calls or `experimental=True`; otherwise DSPy's legacy list of output+ strings or dictionaries, even when a typed LM subclass uses the typed path internally.+ """+ return_typed_response, forward_contract, normalized_request = self._prepare_lm_call(+ items=items,+ prompt=prompt,+ messages=messages,+ request=request,+ kwargs=kwargs,+ )+ if normalized_request is None:+ return self._legacy_call_direct(*items, prompt=prompt, messages=messages, **kwargs)++ if forward_contract == "typed_lm":+ response = self.forward(normalized_request)+ response = self._finalize_lm_response(normalized_request, self._validate_typed_lm_response(response))+ else:+ response = self._legacy_forward_as_lm_response(normalized_request)+ if return_typed_response:+ return response+ return response.to_legacy_outputs()@with_callbacksasync def acall(self,+ *items: Any,prompt: str | None = None,messages: list[dict[str, Any]] | None = None,- **kwargs+ request: LMRequest | None = None,+ **kwargs,+ ) -> LMResponse | list[dict[str, Any] | str]:+ """Asynchronously call the language model.++ This is the async equivalent of `__call__()`. It preserves legacy outputs by default and returns+ `dspy.LMResponse` for explicit `LMRequest` calls or experimental direct calls.+ """+ return_typed_response, forward_contract, normalized_request = self._prepare_lm_call(+ items=items,+ prompt=prompt,+ messages=messages,+ request=request,+ kwargs=kwargs,+ )+ if normalized_request is None:+ return await self._legacy_acall_direct(*items, prompt=prompt, messages=messages, **kwargs)++ if forward_contract == "typed_lm":+ response = await self.aforward(normalized_request)+ response = self._finalize_lm_response(normalized_request, self._validate_typed_lm_response(response))+ else:+ response = await self._legacy_aforward_as_lm_response(normalized_request)+ if return_typed_response:+ return response+ return response.to_legacy_outputs()++ def _prepare_lm_call(+ self,+ *,+ items: tuple[Any, ...],+ prompt: str | None,+ messages: list[dict[str, Any]] | None,+ request: LMRequest | None,+ kwargs: dict[str, Any],+ ) -> tuple[bool, ForwardContract, LMRequest | None]:+ explicit_request = request is not None or bool(items and isinstance(items[0], LMRequest))+ return_typed_response = explicit_request or bool(settings.get("experimental", False))+ forward_contract = self._get_forward_contract()+ if not return_typed_response and forward_contract != "typed_lm":+ return return_typed_response, forward_contract, None++ normalized_request = self._normalize_lm_call(+ *items,+ prompt=prompt,+ messages=messages,+ request=request,+ **kwargs,+ )+ return return_typed_response, forward_contract, normalized_request++ def _legacy_call_direct(+ self,+ *items: Any,+ prompt: str | None = None,+ messages: list[dict[str, Any]] | None = None,+ **kwargs: Any,+ ) -> list[dict[str, Any] | str]:+ """Execute the pre-typed synchronous call path and return legacy outputs."""+ prompt = self._legacy_prompt_from_items(items, prompt=prompt)+ response = self.forward(prompt=prompt, messages=messages, **kwargs)+ if isinstance(response, LMResponse):+ raise TypeError(+ f"{type(self).__name__}.forward() returned dspy.LMResponse on the legacy direct path. "+ "Set forward_contract='typed_lm' or pass an LMRequest/use dspy.context(experimental=True)."+ )+ return self._process_lm_response(response, prompt, messages, **kwargs)++ async def _legacy_acall_direct(+ self,+ *items: Any,+ prompt: str | None = None,+ messages: list[dict[str, Any]] | None = None,+ **kwargs: Any,) -> list[dict[str, Any] | str]:+ """Execute the pre-typed asynchronous call path and return legacy outputs."""+ prompt = self._legacy_prompt_from_items(items, prompt=prompt)response = await self.aforward(prompt=prompt, messages=messages, **kwargs)- outputs = self._process_lm_response(response, prompt, messages, **kwargs)- return outputs+ if isinstance(response, LMResponse):+ raise TypeError(+ f"{type(self).__name__}.aforward() returned dspy.LMResponse on the legacy direct path. "+ "Set forward_contract='typed_lm' or pass an LMRequest/use dspy.context(experimental=True)."+ )+ return self._process_lm_response(response, prompt, messages, **kwargs)++ def _legacy_prompt_from_items(self, items: tuple[Any, ...], *, prompt: str | None) -> str | None:+ """Validate and extract the one positional prompt accepted by legacy calls."""+ if len(items) > 1:+ raise TypeError(+ "Legacy BaseLM calls accept at most one positional prompt. "+ "Use dspy.context(experimental=True) or pass an LMRequest for typed multi-item LM calls."+ )+ if items and prompt is not None:+ raise TypeError("Pass a prompt either positionally or by keyword, not both.")+ if items and isinstance(items[0], LMRequest):+ raise TypeError("LMRequest calls require the typed LM path; this should be unreachable.")+ return items[0] if items else prompt++ def _normalize_lm_call(+ self,+ *items: Any,+ prompt: str | None = None,+ messages: list[dict[str, Any]] | None = None,+ request: LMRequest | None = None,+ **kwargs: Any,+ ) -> LMRequest:+ """Normalize direct call inputs or an explicit request into `LMRequest`."""+ if request is None and items and isinstance(items[0], LMRequest):+ request = items[0]+ items = items[1:]++ if request is not None:+ if items or prompt is not None or messages is not None:+ raise ValueError(+ "Pass either an LMRequest or direct-call inputs, not both. "+ "Use call kwargs to override request config."+ )+ return request.with_config_overrides(**kwargs) if kwargs else request++ merged_kwargs = {**self.kwargs, **kwargs}+ merged_kwargs.setdefault("cache", self.cache)+ return LMRequest.from_call(+ model=self.model,+ items=items,+ prompt=prompt,+ messages=messages,+ **merged_kwargs,+ )++ def _legacy_forward_as_lm_response(self, request: LMRequest) -> LMResponse:+ """Call a legacy `forward()` implementation and normalize its provider response."""+ data = self._legacy_forward_kwargs(request)+ messages = data.pop("messages", None)+ prompt = self._prompt_from_lm_request(request)+ if prompt is not None:+ messages = None+ response = self.forward(prompt=prompt, messages=messages, **data)+ typed_response = self._validate_legacy_lm_response(response, stacklevel=4)+ if typed_response is not None:+ return self._finalize_lm_response(request, typed_response)+ with settings.context(disable_history=True, usage_tracker=None):+ outputs = self._process_lm_response(response, prompt, messages, **data)+ lm_response = self._legacy_outputs_to_lm_response(outputs, request=request, provider_response=response)+ return self._finalize_lm_response(request, lm_response)++ async def _legacy_aforward_as_lm_response(self, request: LMRequest) -> LMResponse:+ """Async variant of `_legacy_forward_as_lm_response()`."""+ data = self._legacy_forward_kwargs(request)+ messages = data.pop("messages", None)+ prompt = self._prompt_from_lm_request(request)+ if prompt is not None:+ messages = None+ response = await self.aforward(prompt=prompt, messages=messages, **data)+ typed_response = self._validate_legacy_lm_response(response, stacklevel=4)+ if typed_response is not None:+ return self._finalize_lm_response(request, typed_response)+ with settings.context(disable_history=True, usage_tracker=None):+ outputs = self._process_lm_response(response, prompt, messages, **data)+ lm_response = self._legacy_outputs_to_lm_response(outputs, request=request, provider_response=response)+ return self._finalize_lm_response(request, lm_response)++ def _legacy_outputs_to_lm_response(+ self,+ outputs: list[dict[str, Any] | str | None],+ *,+ request: LMRequest,+ provider_response: Any,+ ) -> LMResponse:+ """Convert legacy post-processed outputs and provider metadata into `LMResponse`."""+ if self.model_type == "responses":+ response = responses_to_lm_response(provider_response, request)+ elif self.model_type in {"chat", "text"}:+ response = completion_to_lm_response(provider_response, request)+ else:+ response = lm_response_from_legacy_outputs(outputs, request)++ return response.model_copy(+ update={+ "model": getattr(provider_response, "model", None) or response.model,+ "usage": usage_from_response(provider_response),+ "cost": cost_from_response(provider_response),+ "cache_hit": bool(getattr(provider_response, "cache_hit", False)),+ "provider_response": provider_response,+ }+ )++ def _legacy_forward_kwargs(self, request: LMRequest) -> dict[str, Any]:+ """Convert a normalized request into kwargs for legacy `forward()` implementations."""+ data = to_openai_chat_request(request)+ data.pop("model", None)+ if request.config.cache is not None:+ if request.config.cache.enabled is not None:+ data["cache"] = request.config.cache.enabled+ if request.config.cache.rollout_id is not None:+ data["rollout_id"] = request.config.cache.rollout_id+ return data++ def _prompt_from_lm_request(self, request: LMRequest) -> str | None:+ """Return the legacy prompt when a normalized request is exactly one text user message."""+ if len(request.messages) != 1:+ return None+ message = request.messages[0]+ if message.role != "user" or len(message.parts) != 1:+ return None+ part = message.parts[0]+ return part.text if getattr(part, "type", None) == "text" else None++ def _finalize_lm_response(self, request: LMRequest, response: LMResponse) -> LMResponse:+ """Record usage and typed history for a normalized LM response."""+ if not getattr(response, "cache_hit", False) and settings.usage_tracker:+ usage = response.usage_as_dict()+ if usage:+ settings.usage_tracker.add_usage(self.model, usage)++ if not settings.disable_history:+ entry = LMHistoryEntry(+ request=request,+ response=response,+ timestamp=datetime.datetime.now().isoformat(),+ uuid=str(uuid.uuid4()),+ model_type=getattr(self, "model_type", None),+ )+ self.update_history(entry)+ return responsedef forward(self,@@ -288,12 +611,17 @@ class BaseLM:):"""Forward pass for the language model.- Subclasses must implement this method, and the response should be identical to either of the following formats:+ Subclasses must implement this method according to `forward_contract`.++ For `forward_contract = "legacy"`, implement+ `forward(prompt=None, messages=None, **kwargs)` and return one of these OpenAI-like provider responses:- [OpenAI response format](https://platform.openai.com/docs/api-reference/responses/object)- [OpenAI chat completion format](https://platform.openai.com/docs/api-reference/chat/object)- [OpenAI text completion format](https://platform.openai.com/docs/api-reference/completions/object)+ For `forward_contract = "typed_lm"`, implement `forward(request: dspy.LMRequest) -> dspy.LMResponse`.+Raises:dspy.LMError: Base class for LM configuration, transport, provider,and unsupported-feature failures. Notable subclasses include@@ -313,12 +641,17 @@ class BaseLM:):"""Async forward pass for the language model.- Subclasses must implement this method, and the response should be identical to either of the following formats:+ Subclasses that support async calls must implement this method according to `forward_contract`.++ For `forward_contract = "legacy"`, implement+ `aforward(prompt=None, messages=None, **kwargs)` and return one of these OpenAI-like provider responses:- [OpenAI response format](https://platform.openai.com/docs/api-reference/responses/object)- [OpenAI chat completion format](https://platform.openai.com/docs/api-reference/chat/object)- [OpenAI text completion format](https://platform.openai.com/docs/api-reference/completions/object)+ For `forward_contract = "typed_lm"`, implement `aforward(request: dspy.LMRequest) -> dspy.LMResponse`.+Raises:dspy.LMError: Base class for LM configuration, transport, provider,and unsupported-feature failures. Notable subclasses includediff --git a/dspy/core/types.py b/dspy/core/types.pyindex 5c12a3c4e..e0884286c 100644--- a/dspy/core/types.py+++ b/dspy/core/types.py@@ -1286,18 +1286,16 @@ def System(*parts: Any, name: str | None = None, metadata: dict[str, Any] | None"""Create a system message for a direct LM call.A system message gives model-level instructions, such as tone, scope, or- formatting rules. Pass text, media parts, or normalized `LMPart` objects;- DSPy stores them as one `LMMessage` with role `"system"`.+ formatting rules. Pass text or normalized `LMPart` objects; DSPy stores+ them as one `LMMessage` with role `"system"`.Args:- *parts: Text, DSPy media objects, or normalized LM parts to include in- the message.+ *parts: Text or normalized LM parts to include in the message.name: Optional sender name for providers that support named messages.metadata: Extra information to keep with the message.Returns:- An `LMMessage` that can be passed to `dspy.LanguageModel`, `dspy.LM`, or- `dspy.LMRequest`.+ An `LMMessage` that can be passed to `dspy.LM` or `dspy.LMRequest`.Examples:System instruction with a user turn:@@ -1305,10 +1303,12 @@ def System(*parts: Any, name: str | None = None, metadata: dict[str, Any] | None```pythonimport dspy- lm = dspy.LanguageModel(model="test/model")- request = lm.normalize_request(- dspy.System("You are concise."),- dspy.User("What is DSPy?"),+ request = dspy.LMRequest.from_call(+ model="openai/gpt-4o-mini",+ items=(+ dspy.System("You are concise."),+ dspy.User("What is DSPy?"),+ ),)```@@ -1328,8 +1328,7 @@ def Developer(*parts: Any, name: str | None = None, metadata: dict[str, Any] | Nyou want to keep implementation guidance separate from the user's request.Args:- *parts: Text, DSPy media objects, or normalized LM parts to include in- the message.+ *parts: Text or normalized LM parts to include in the message.name: Optional sender name for providers that support named messages.metadata: Extra information to keep with the message.@@ -1363,12 +1362,11 @@ def User(*parts: Any, name: str | None = None, metadata: dict[str, Any] | None ="""Create a user message for a direct LM call.A user message contains the request or data you want the model to answer.- Pass plain text for simple prompts, or mix text with images, audio, documents,- binary attachments, and normalized LM parts for multimodal calls.+ Pass plain text for simple prompts, or mix text with normalized image, audio,+ document, binary, and other `LMPart` objects for multimodal calls.Args:- *parts: Text, DSPy media objects, or normalized LM parts to include in- the message.+ *parts: Text or normalized LM parts to include in the message.name: Optional sender name for providers that support named messages.metadata: Extra information to keep with the message.@@ -1382,38 +1380,43 @@ def User(*parts: Any, name: str | None = None, metadata: dict[str, Any] | None =import dspylm = dspy.LM("openai/gpt-4o-mini")- response = lm(- dspy.User("What is DSPy?"),- dspy.Assistant("DSPy is a framework for programming LM pipelines."),- dspy.User("Say that in five words."),- )+ with dspy.context(experimental=True):+ response = lm(+ dspy.User("What is DSPy?"),+ dspy.Assistant("DSPy is a framework for programming LM pipelines."),+ dspy.User("Say that in five words."),+ )```Multi-turn call with media:```pythonimport dspy+ from dspy.core.types import LMImagePartlm = dspy.LM("openai/gpt-4o-mini")- response = lm(- dspy.System("Answer in one sentence."),- dspy.User(- "Describe this image.",- dspy.Image("https://example.com/dog.png"),- ),- )+ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Answer in one sentence."),+ dspy.User(+ "Describe this image.",+ LMImagePart(url="https://example.com/dog.png"),+ ),+ )```For a single user turn, pass the parts directly to `lm(...)` instead:```python- response = lm("Describe this image.", dspy.Image("https://example.com/dog.png"))+ with dspy.context(experimental=True):+ response = lm("Describe this image.", LMImagePart(url="https://example.com/dog.png"))```Explicit `LMRequest` for custom LM authors and advanced users:```pythonimport dspy+ from dspy.core.types import LMImagePartlm = dspy.LM("openai/gpt-4o-mini")request = dspy.LMRequest(@@ -1422,7 +1425,7 @@ def User(*parts: Any, name: str | None = None, metadata: dict[str, Any] | None =dspy.System("You are concise."),dspy.User("Describe this image.",- dspy.Image("https://example.com/dog.png"),+ LMImagePart(url="https://example.com/dog.png"),),],config=dspy.LMConfig(temperature=0.2, max_tokens=200),diff --git a/tests/clients/test_lm.py b/tests/clients/test_lm.pyindex ebd437602..c3d7ee2b1 100644--- a/tests/clients/test_lm.py+++ b/tests/clients/test_lm.py@@ -515,6 +515,320 @@ def test_base_lm_errors_when_explicit_legacy_forward_returns_lm_response():lm._validate_legacy_lm_response(response)+def test_base_lm_inherited_legacy_forward_returning_lm_response_errors_on_direct_call():+ class CustomLM(dspy.BaseLM):+ def forward(self, prompt=None, messages=None, **kwargs):+ return dspy.LMResponse.from_text(+ "ok",+ model="custom-model",+ usage={"prompt_tokens": 1, "completion_tokens": 2, "total_tokens": 3},+ )++ lm = CustomLM("custom-model")++ with pytest.raises(TypeError, match="legacy direct path"):+ lm("Query")++ assert len(lm.history) == 0+++# BaseLM direct-call compatibility tests.+#+# These cover the staged typed LM migration: legacy calls still return lists by default, explicit LMRequest calls and+# experimental direct calls return LMResponse, and typed-LM subclasses receive normalized LMRequest objects.+++def test_base_lm_default_call_keeps_legacy_outputs():+ class CustomLM(dspy.BaseLM):+ def forward(self, prompt=None, messages=None, **kwargs):+ assert prompt == "Query"+ assert messages is None+ return ModelResponse(+ choices=[Choices(message=Message(role="assistant", content="Hi!"))],+ usage={"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},+ model="custom-model",+ )++ assert CustomLM("custom-model")("Query") == ["Hi!"]+++def test_base_lm_experimental_call_returns_lm_response_through_legacy_bridge():+ class CustomLM(dspy.BaseLM):+ def __init__(self, *args, **kwargs):+ super().__init__(*args, **kwargs)+ self.seen = None++ def forward(self, prompt=None, messages=None, **kwargs):+ self.seen = {"prompt": prompt, "messages": messages, "kwargs": kwargs}+ return ModelResponse(+ choices=[Choices(message=Message(role="assistant", content="Hi!"), finish_reason="stop")],+ usage={"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},+ model="custom-model",+ )++ lm = CustomLM("custom-model", temperature=0.2)+ with dspy.context(experimental=True):+ response = lm("Query", rollout_id=7)++ assert isinstance(response, dspy.LMResponse)+ assert response.text == "Hi!"+ assert response.output.finish_reason == "stop"+ assert lm.seen["prompt"] == "Query"+ assert lm.seen["messages"] is None+ assert lm.seen["kwargs"]["temperature"] == 0.2+ assert lm.seen["kwargs"]["cache"] is True+ assert lm.seen["kwargs"]["rollout_id"] == 7+++def test_base_lm_explicit_lm_request_returns_lm_response_without_experimental():+ class CustomLM(dspy.BaseLM):+ def forward(self, prompt=None, messages=None, **kwargs):+ return ModelResponse(+ choices=[Choices(message=Message(role="assistant", content="Hi!"))],+ usage={},+ model="custom-model",+ )++ request = dspy.LMRequest.from_call(model="custom-model", prompt="Query")+ response = CustomLM("custom-model")(request)++ assert isinstance(response, dspy.LMResponse)+ assert response.text == "Hi!"+++def test_base_lm_legacy_bridge_records_typed_history_and_usage_once():+ class CustomLM(dspy.BaseLM):+ def forward(self, prompt=None, messages=None, **kwargs):+ return ModelResponse(+ choices=[Choices(message=Message(role="assistant", content="Hi!"))],+ usage={"prompt_tokens": 1, "completion_tokens": 2, "total_tokens": 3},+ model="custom-model",+ )++ lm = CustomLM("custom-model")+ request = dspy.LMRequest.from_call(model="custom-model", prompt="Query")++ with track_usage() as usage_tracker:+ response = lm(request)++ assert isinstance(response, dspy.LMResponse)+ assert response.text == "Hi!"+ assert len(lm.history) == 1+ assert lm.history[0].request == request+ assert lm.history[0].response == response+ total_usage = usage_tracker.get_total_tokens()["custom-model"]+ assert total_usage["prompt_tokens"] == 1+ assert total_usage["completion_tokens"] == 2+ assert total_usage["total_tokens"] == 3+++def test_base_lm_typed_forward_contract_uses_lm_request():+ class CustomLM(dspy.BaseLM):+ forward_contract = "typed_lm"++ def forward(self, request):+ assert isinstance(request, dspy.LMRequest)+ return dspy.LMResponse.from_text(f"model={request.model}; text={request.messages[0].text}")++ lm = CustomLM("custom-model")++ assert lm("Query") == ["model=custom-model; text=Query"]+ with dspy.context(experimental=True):+ response = lm("Query")+ assert isinstance(response, dspy.LMResponse)+ assert response.text == "model=custom-model; text=Query"+++def test_base_lm_typed_forward_contract_rejects_non_lm_response_at_call_time():+ class CustomLM(dspy.BaseLM):+ forward_contract = "typed_lm"++ def forward(self, request):+ return ["not typed"]++ with pytest.raises(TypeError, match="forward_contract='typed_lm'"):+ CustomLM("custom-model")("Query")+++def test_base_lm_request_call_rejects_mixed_inputs():+ class CustomLM(dspy.BaseLM):+ def forward(self, prompt=None, messages=None, **kwargs):+ raise AssertionError("forward should not be called")++ request = dspy.LMRequest.from_call(model="custom-model", prompt="Query")+ with pytest.raises(ValueError, match="Pass either an LMRequest or direct-call inputs"):+ CustomLM("custom-model")(request, "extra")+++def _model_response(text: str) -> ModelResponse:+ return ModelResponse(+ choices=[Choices(message=Message(role="assistant", content=text))],+ usage={},+ model="custom-model",+ )+++class _TypedContractLM(dspy.BaseLM):+ """Test double that records normalized requests received through the typed LM contract."""++ forward_contract = "typed_lm"++ def __init__(self, *args, outputs: list[str], **kwargs):+ super().__init__(*args, **kwargs)+ self.outputs = outputs+ self.requests = []++ def forward(self, request):+ assert isinstance(request, dspy.LMRequest)+ self.requests.append(request)+ return dspy.LMResponse.from_text(self.outputs[len(self.requests) - 1], model=request.model)+++def _direct_lm_case(lm_kind: str, outputs: list[str]):+ """Return a direct-call test double and helpers for inspecting normalized messages."""+ if lm_kind == "current_lm":+ patcher = mock.patch(+ "dspy.clients.lm.litellm_completion",+ side_effect=[_model_response(output) for output in outputs],+ )+ completion = patcher.start()+ lm = dspy.LM("custom-model", cache=False)++ def get_messages(index: int) -> list[dict[str, object]]:+ return completion.call_args_list[index].kwargs["request"]["messages"]++ def get_request(index: int):+ return None++ return lm, get_messages, get_request, patcher++ if lm_kind == "typed_lm":+ lm = _TypedContractLM("custom-model", outputs=outputs)++ def get_messages(index: int) -> list[dict[str, object]]:+ from dspy.clients.openai_format import to_openai_chat_request++ return to_openai_chat_request(lm.requests[index])["messages"]++ def get_request(index: int):+ return lm.requests[index]++ return lm, get_messages, get_request, None++ raise ValueError(f"Unknown lm_kind: {lm_kind}")++[email protected]("lm_kind", ["current_lm", "typed_lm"])+def test_base_lm_experimental_direct_messages_support_system_user_and_assistant_turns(lm_kind):+ lm, get_messages, get_request, patcher = _direct_lm_case(lm_kind, ["Five-word answer."])+ try:+ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Be concise."),+ dspy.User("What is DSPy?"),+ dspy.Assistant("DSPy is a framework for programming LM pipelines."),+ dspy.User("Say that in five words."),+ temperature=0.2,+ )+ finally:+ if patcher is not None:+ patcher.stop()++ assert isinstance(response, dspy.LMResponse)+ assert response.text == "Five-word answer."+ assert get_messages(0) == [+ {"role": "system", "content": "Be concise."},+ {"role": "user", "content": "What is DSPy?"},+ {"role": "assistant", "content": "DSPy is a framework for programming LM pipelines."},+ {"role": "user", "content": "Say that in five words."},+ ]+ if lm_kind == "typed_lm":+ assert get_request(0).config.temperature == 0.2++[email protected]("lm_kind", ["current_lm", "typed_lm"])+def test_base_lm_experimental_direct_messages_support_tool_call_transcripts(lm_kind):+ lm, get_messages, get_request, patcher = _direct_lm_case(lm_kind, ["It is 22 C in Paris."])+ try:+ with dspy.context(experimental=True):+ response = lm(+ dspy.User("What is the weather in Paris?"),+ dspy.Assistant(dspy.ToolCall(id="call_1", name="get_weather", args={"city": "Paris"})),+ dspy.ToolResult('{"temperature": "22 C"}', call_id="call_1", name="get_weather"),+ dspy.User("Summarize the result."),+ )+ finally:+ if patcher is not None:+ patcher.stop()++ assert response.text == "It is 22 C in Paris."+ assert get_messages(0) == [+ {"role": "user", "content": "What is the weather in Paris?"},+ {+ "role": "assistant",+ "content": None,+ "tool_calls": [+ {+ "type": "function",+ "function": {"name": "get_weather", "arguments": json.dumps({"city": "Paris"})},+ "id": "call_1",+ }+ ],+ },+ {"role": "tool", "content": '{"temperature": "22 C"}', "tool_call_id": "call_1", "name": "get_weather"},+ {"role": "user", "content": "Summarize the result."},+ ]+ if lm_kind == "typed_lm":+ assert isinstance(get_request(0), dspy.LMRequest)++[email protected]("lm_kind", ["current_lm", "typed_lm"])+def test_base_lm_experimental_direct_messages_can_reuse_lm_response_as_assistant_turn(lm_kind):+ lm, get_messages, get_request, patcher = _direct_lm_case(+ lm_kind,+ ["DSPy programs LM pipelines.", "DSPy programs pipelines."],+ )+ try:+ with dspy.context(experimental=True):+ first = lm("Explain DSPy in one sentence.")+ follow_up = lm(+ dspy.User("Explain DSPy in one sentence."),+ first,+ dspy.User("Now make it even shorter."),+ )+ finally:+ if patcher is not None:+ patcher.stop()++ assert first.text == "DSPy programs LM pipelines."+ assert follow_up.text == "DSPy programs pipelines."+ assert get_messages(0) == [{"role": "user", "content": "Explain DSPy in one sentence."}]+ assert get_messages(1) == [+ {"role": "user", "content": "Explain DSPy in one sentence."},+ {"role": "assistant", "content": "DSPy programs LM pipelines."},+ {"role": "user", "content": "Now make it even shorter."},+ ]+ if lm_kind == "typed_lm":+ assert isinstance(get_request(1), dspy.LMRequest)+++async def test_base_lm_async_explicit_lm_request_returns_lm_response():+ class CustomLM(dspy.BaseLM):+ async def aforward(self, prompt=None, messages=None, **kwargs):+ return ModelResponse(+ choices=[Choices(message=Message(role="assistant", content="Hi async!"))],+ usage={},+ model="custom-model",+ )++ request = dspy.LMRequest.from_call(model="custom-model", prompt="Query")+ response = await CustomLM("custom-model").acall(request)++ assert isinstance(response, dspy.LMResponse)+ assert response.text == "Hi async!"++def test_base_lm_tracks_usage_for_custom_subclasses():class CustomLM(dspy.BaseLM):def forward(self, prompt=None, messages=None, **kwargs):diff --git a/tests/clients/test_lm_direct_live.py b/tests/clients/test_lm_direct_live.pynew file mode 100644index 000000000..b0ccd3f6d--- /dev/null+++ b/tests/clients/test_lm_direct_live.py@@ -0,0 +1,311 @@+"""Live coverage for the experimental direct LM call interface.++These tests exercise provider behavior that cannot be verified with the mocked unit tests in `test_lm.py`: typed+message turns, tool-call transcripts, and reusing an `LMResponse` as an assistant turn across chat and Responses API+providers.++They are intentionally flat rather than parametrized so each test can be run individually from an editor or notebook-like+workflow. Each test skips unless the required provider credential is available.+"""++import os++import pytest++import dspy+++def _require_env(*keys: str) -> None:+ missing = [key for key in keys if not os.getenv(key)]+ if missing:+ pytest.skip(f"Missing live LM credentials: {', '.join(missing)}")+++def _text(response: dspy.LMResponse) -> str:+ assert isinstance(response, dspy.LMResponse)+ assert response.text is not None+ return response.text.strip()++[email protected]_call+def test_live_openai_chat_direct_system_user_assistant_multiturn():+ _require_env("OPENAI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_OPENAI_CHAT", "openai/gpt-5.5"),+ model_type="chat",+ cache=False,+ max_completion_tokens=64,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Follow the user's requested exact final token. No punctuation."),+ dspy.User("Reply with exactly: alpha"),+ dspy.Assistant("alpha"),+ dspy.User("Now reply with exactly: beta"),+ )++ assert "beta" in _text(response).lower()+ assert response.output.finish_reason is not None++[email protected]_call+def test_live_openai_chat_direct_tool_call_transcript():+ _require_env("OPENAI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_OPENAI_CHAT", "openai/gpt-4o-mini"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Use the supplied tool result. Keep the answer short."),+ dspy.User("What is the weather in Paris?"),+ dspy.Assistant(dspy.ToolCall(id="call_1", name="get_weather", args={"city": "Paris"})),+ dspy.ToolResult('{"temperature": "22 C"}', call_id="call_1", name="get_weather"),+ dspy.User("Answer with the temperature string from the tool result."),+ )++ text = _text(response).lower()+ assert "22" in text+ assert "c" in text++[email protected]_call+def test_live_openai_chat_direct_reuse_lm_response_as_assistant_turn():+ _require_env("OPENAI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_OPENAI_CHAT", "openai/gpt-4o-mini"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ first = lm(dspy.User("Reply with exactly: DSPy"))+ follow_up = lm(+ dspy.User("Reply with exactly: DSPy"),+ first,+ dspy.User("Repeat the previous assistant answer exactly."),+ )++ assert "dspy" in _text(first).lower()+ assert "dspy" in _text(follow_up).lower()++[email protected]_call+def test_live_openai_responses_direct_system_user_assistant_multiturn():+ _require_env("OPENAI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_OPENAI_RESPONSES", "openai/gpt-4.1-mini"),+ model_type="responses",+ cache=False,+ temperature=1.0,+ max_tokens=16000,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Follow the user's requested exact final token. No punctuation."),+ dspy.User("Reply with exactly: alpha"),+ dspy.Assistant("alpha"),+ dspy.User("Now reply with exactly: beta"),+ )++ assert "beta" in _text(response).lower()++[email protected]_call+def test_live_openai_responses_direct_tool_call_transcript():+ _require_env("OPENAI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_OPENAI_RESPONSES", "openai/gpt-4.1-mini"),+ model_type="responses",+ cache=False,+ temperature=1.0,+ max_tokens=16000,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Use the supplied tool result. Keep the answer short."),+ dspy.User("What is the weather in Paris?"),+ dspy.Assistant(dspy.ToolCall(id="call_1", name="get_weather", args={"city": "Paris"})),+ dspy.ToolResult('{"temperature": "22 C"}', call_id="call_1", name="get_weather"),+ dspy.User("Answer with the temperature string from the tool result."),+ )++ text = _text(response).lower()+ assert "22" in text+ assert "c" in text++[email protected]_call+def test_live_openai_responses_direct_reuse_lm_response_as_assistant_turn():+ _require_env("OPENAI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_OPENAI_RESPONSES", "openai/gpt-4.1-mini"),+ model_type="responses",+ cache=False,+ temperature=1.0,+ max_tokens=16000,+ )++ with dspy.context(experimental=True):+ first = lm(dspy.User("Reply with exactly: DSPy"))+ follow_up = lm(+ dspy.User("Reply with exactly: DSPy"),+ first,+ dspy.User("Repeat the previous assistant answer exactly."),+ )++ assert "dspy" in _text(first).lower()+ assert "dspy" in _text(follow_up).lower()++[email protected]_call+def test_live_anthropic_chat_direct_system_user_assistant_multiturn():+ _require_env("ANTHROPIC_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_ANTHROPIC", "anthropic/claude-3-5-haiku-latest"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Follow the user's requested exact final token. No punctuation."),+ dspy.User("Reply with exactly: alpha"),+ dspy.Assistant("alpha"),+ dspy.User("Now reply with exactly: beta"),+ )++ assert "beta" in _text(response).lower()++[email protected]_call+def test_live_anthropic_chat_direct_tool_call_transcript():+ _require_env("ANTHROPIC_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_ANTHROPIC", "anthropic/claude-3-5-haiku-latest"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Use the supplied tool result. Keep the answer short."),+ dspy.User("What is the weather in Paris?"),+ dspy.Assistant(dspy.ToolCall(id="call_1", name="get_weather", args={"city": "Paris"})),+ dspy.ToolResult('{"temperature": "22 C"}', call_id="call_1", name="get_weather"),+ dspy.User("Answer with the temperature string from the tool result."),+ )++ text = _text(response).lower()+ assert "22" in text+ assert "c" in text++[email protected]_call+def test_live_anthropic_chat_direct_reuse_lm_response_as_assistant_turn():+ _require_env("ANTHROPIC_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_ANTHROPIC", "anthropic/claude-3-5-haiku-latest"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ first = lm(dspy.User("Reply with exactly: DSPy"))+ follow_up = lm(+ dspy.User("Reply with exactly: DSPy"),+ first,+ dspy.User("Repeat the previous assistant answer exactly."),+ )++ assert "dspy" in _text(first).lower()+ assert "dspy" in _text(follow_up).lower()++[email protected]_call+def test_live_gemini_chat_direct_system_user_assistant_multiturn():+ _require_env("GEMINI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_GEMINI", "gemini/gemini-2.0-flash"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Follow the user's requested exact final token. No punctuation."),+ dspy.User("Reply with exactly: alpha"),+ dspy.Assistant("alpha"),+ dspy.User("Now reply with exactly: beta"),+ )++ assert "beta" in _text(response).lower()++[email protected]_call+def test_live_gemini_chat_direct_tool_call_transcript():+ _require_env("GEMINI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_GEMINI", "gemini/gemini-2.0-flash"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ response = lm(+ dspy.System("Use the supplied tool result. Keep the answer short."),+ dspy.User("What is the weather in Paris?"),+ dspy.Assistant(dspy.ToolCall(id="call_1", name="get_weather", args={"city": "Paris"})),+ dspy.ToolResult('{"temperature": "22 C"}', call_id="call_1", name="get_weather"),+ dspy.User("Answer with the temperature string from the tool result."),+ )++ text = _text(response).lower()+ assert "22" in text+ assert "c" in text++[email protected]_call+def test_live_gemini_chat_direct_reuse_lm_response_as_assistant_turn():+ _require_env("GEMINI_API_KEY")++ lm = dspy.LM(+ os.getenv("LM_FOR_TEST_DIRECT_GEMINI", "gemini/gemini-2.0-flash"),+ model_type="chat",+ cache=False,+ max_tokens=64,+ )++ with dspy.context(experimental=True):+ first = lm(dspy.User("Reply with exactly: DSPy"))+ follow_up = lm(+ dspy.User("Reply with exactly: DSPy"),+ first,+ dspy.User("Repeat the previous assistant answer exactly."),+ )++ assert "dspy" in _text(first).lower()+ assert "dspy" in _text(follow_up).lower()