Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Update bundled lm15 to rc3: native timeouts, reliable stops and error diagnostics (#10409)

Maxime Rivest7fd3612140a9
diff --git a/docs/docs/community/normalized-lm-api-migration.md b/docs/docs/community/normalized-lm-api-migration.md
index 4646dcba4..9689fa78a 100644
--- a/docs/docs/community/normalized-lm-api-migration.md
+++ b/docs/docs/community/normalized-lm-api-migration.md
@@ -59,9 +59,50 @@ Engine selection:
- `engine="litellm"`: explicitly use the compatibility backend.
Authentication failures, timeouts and provider errors never cause a switch to
-another backend. Native capability errors also raise rather than dropping the
-requested feature. Text completions and client settings not implemented by the
-native integration remain on LiteLLM.
+another backend. Text completions and client settings not implemented by the
+native integration remain on LiteLLM. The `timeout` setting, a number of seconds
+or an `httpx.Timeout`, stays native: it bounds the wait for the next byte of the
+reply (headers first, then each streamed chunk), the send, and a free
+connection; the native default is 600 seconds, the same as LiteLLM's. An
+`httpx.Timeout` component set to `None` means "wait forever", which the native
+engine cannot honour: under `engine="auto"` that call uses LiteLLM, and under
+`engine="lm15"` it raises `LMUnsupportedFeatureError` (feature `timeout`)
+rather than silently substituting a default.
+
+Text that no provider can receive (a lone surrogate character) raises a plain
+`ValueError` before any engine is chosen, on every engine setting, so it is
+never reported as an unexpected engine failure or retried.
+
+Typed Requests through the LiteLLM engine forward `top_k` as a LiteLLM
+argument on the Chat Completions path (LiteLLM translates it for providers
+that take it); on the Responses path, which has no such field, it is dropped
+and recorded under `"adaptations"`.
+
+A setting the native route cannot carry exactly as written is **adapted and
+recorded**, rather than silently ignored or unnecessarily refused (lm15 MAP-13): `seed` on Anthropic is
+left out, `temperature=1.5` on Anthropic becomes `1.0`, a thinking-summary
+level the wire lacks becomes `auto`. Each history entry carries the record
+under `"adaptations"` (`field`, `action`, `asked`, `applied`, `reason`); the
+lm15 `Response` in the entry carries the same. Nothing is printed. What lm15
+still refuses — an image inside a tool result on the Chat Completions wire, a
+stored cache object that does not exist on the provider, `n > 1` — is known
+before any request is sent, and under `engine="auto"` such a call goes to
+LiteLLM without a failed native attempt first; under `engine="lm15"` it raises,
+naming the field in `error.feature`.
+
+For stop sequences on the Responses API, the engine matters:
+
+- **Native lm15:** streams internally and closes at the stop. Final usage is
+ unavailable after an early cut; closing does not guarantee billing stops.
+- **Typed Requests through LiteLLM:** trims the completed reply using lm15's
+ stop helper. Generation is not stopped early; full provider usage is retained.
+ This is recorded in `response.adaptations` and history. Typed Responses
+ streaming through LiteLLM remains unsupported; use native lm15 for it.
+
+Both preserve scores for whole retained tokens. If a stop cuts through a token,
+`response.logprobs_complete` is false: the retained partial token has no invented
+score. The flag survives saved-response caching. Chat Completions still sends
+stop sequences to the provider directly.
### Caching, retries, usage and streaming
@@ -176,6 +217,13 @@ DSPy errors, including those from legacy plugins, are preserved by identity.
- `LMStreamAssemblyError` identifies an incomplete or invalid stream. It is not
automatically retried; `partial` may hold salvageable content, not a successful
result. Unknown usage stays unknown.
+- `LMCollectionLimitError` identifies a local collection budget being reached.
+ It is not retryable. `partial_events`, `rejected_event`, `limit`, `maximum`,
+ `retained_bytes`, and `retained_events` survive the engine boundary. Reading
+ `partial` assembles the incomplete turn on demand; wrapping the error does
+ not allocate combined text/audio. DSPy does not add a live-session API here.
+- `LMUnsupportedFeatureError.feature` preserves the unsupported setting path
+ reported by lm15. It also fills `features` when no explicit list was provided.
- Unknown engine failures become `LMUnexpectedError`, with the original exception
as `__cause__`. The canonical cause retains its exact lm15 code and SDK cause;
the public error retains useful request IDs, provider codes, retry hints, routing
diff --git a/dspy/__init__.py b/dspy/__init__.py
index e9dab23ed..1813f9c2b 100644
--- a/dspy/__init__.py
+++ b/dspy/__init__.py
@@ -18,6 +18,7 @@ from dspy.utils.exceptions import (
DSPyError,
LMAuthError,
LMBillingError,
+ LMCollectionLimitError,
LMConfigurationError,
LMError,
LMInvalidRequestError,
diff --git a/dspy/_vendor/lm15-provenance.txt b/dspy/_vendor/lm15-provenance.txt
index eb4bfe548..27598c413 100644
--- a/dspy/_vendor/lm15-provenance.txt
+++ b/dspy/_vendor/lm15-provenance.txt
@@ -1,5 +1,5 @@
source=https://github.com/cmpnd-ai/lm15-python.git
-version=1.0.0a1
-commit=7896f82aa1b9d73401aa40f52ece350c3b2efdae
-contract=e24b60e24b39a3e2b7bff8335bc84330b53bff39
-split=608cf26614f8581f7278efd242c5e8b2e23f6f2b
+version=1.0.0rc3
+commit=cd5d4d73a3cc3bbea9fc7901b7b3c26007e7bf37
+contract=7677980e36c8bd0b14ae9bfc79e49446f403ad22
+split=caf92f0781a0598b01cd7c25c81d250afd28babf
diff --git a/dspy/_vendor/lm15/__init__.py b/dspy/_vendor/lm15/__init__.py
index be42d5195..cf26a5b51 100644
--- a/dspy/_vendor/lm15/__init__.py
+++ b/dspy/_vendor/lm15/__init__.py
@@ -148,6 +148,8 @@ from .features import AccessPolicy, EndpointSupport
from .errors import (
LM15Error,
StreamAssemblyError,
+ CollectionLimitError,
+ StreamCleanupWarning,
TransportError,
LockTimeoutError,
ConfigurationError,
@@ -211,6 +213,12 @@ from .router import (
UnknownModelError,
)
+# ── Connection budget (lm15.transports) ───────────────────────────────
+from .transports import Timeouts
+
+# ── MAP-13 adaptations (lm15.adaptation) ──────────────────────────────
+from .adaptation import Adaptation, AdaptationPolicy
+
# ── Tool derivation (lm15.tools) ─────────────────────────────────────
from .tools import (
DerivedParam,
@@ -251,7 +259,7 @@ __all__ = [
"Role", "PartType", "FinishReason", "ReasoningEffort", "ErrorCode",
"StreamEventType", "ROLE_VALUES", "FINISH_REASONS", "ERROR_CODES",
# errors (the catchable taxonomy + the retry predicate)
- "LM15Error", "TransportError", "LockTimeoutError", "StreamAssemblyError", "ConfigurationError", "CapabilityError",
+ "LM15Error", "TransportError", "LockTimeoutError", "StreamAssemblyError", "CollectionLimitError", "StreamCleanupWarning", "ConfigurationError", "CapabilityError",
"ProviderError", "AuthError", "BillingError", "RateLimitError",
"InvalidRequestError", "ContextLengthError", "TimeoutError",
"ServerError", "UnsupportedModelError", "UnsupportedFeatureError",
@@ -272,6 +280,8 @@ __all__ = [
"ModelInfo", "ModelRegistry",
# router (lm15.router; the rule/preset tables live there too)
"LMRouter", "AsyncLMRouter", "RouterConfig", "Resolution",
+ "Timeouts",
+ "Adaptation", "AdaptationPolicy",
"UnknownModelError",
"AmbiguousModelError", "MissingCredentialError",
# tool derivation (lm15.tools)
diff --git a/dspy/_vendor/lm15/_authlock.py b/dspy/_vendor/lm15/_authlock.py
index 055f70ec1..63b49a53c 100644
--- a/dspy/_vendor/lm15/_authlock.py
+++ b/dspy/_vendor/lm15/_authlock.py
@@ -36,7 +36,7 @@ from contextlib import contextmanager
from pathlib import Path
from typing import Any, Iterator
-from .errors import LockTimeoutError
+from .errors import LockTimeoutError, NotConfiguredError
_DEFAULT_LOCK_TIMEOUT_S = 60.0
_LOCK_POLL_INTERVAL_S = 0.05
@@ -57,7 +57,17 @@ def _lock_dir() -> Path:
if override:
return Path(override).expanduser()
cache_home = os.environ.get("XDG_CACHE_HOME")
- base = Path(cache_home).expanduser() if cache_home else Path("~/.cache").expanduser()
+ if cache_home:
+ return Path(cache_home).expanduser() / "lm15" / "locks"
+ try:
+ base = Path("~/.cache").expanduser()
+ except RuntimeError as exc: # no home directory: a WASI guest, some containers
+ raise NotConfiguredError(
+ "No home directory to keep the credential lock in, so lm15 cannot "
+ "serialize credential refreshes against other processes. Reading "
+ "credentials still works; refreshing them from here does not.",
+ credential_hint="Set LM15_LOCK_DIR (or XDG_CACHE_HOME), or pass an explicit credential instead of a stored login",
+ ) from exc
return base / "lm15" / "locks"
@@ -73,8 +83,20 @@ def lock_path_for(path: Path) -> Path:
return _lock_dir() / f"{digest}.lock"
-if os.name == "posix":
+# Chosen by which primitive the platform actually has, not by `os.name`: WASI reports
+# "posix" and ships no fcntl, so a name test picks an implementation that cannot import.
+try:
import fcntl
+except ImportError: # pragma: no cover - Windows, and POSIX-ish builds without fcntl
+ fcntl = None
+
+try:
+ import msvcrt
+except ImportError: # pragma: no cover - every non-Windows platform
+ msvcrt = None
+
+
+if fcntl is not None:
def _try_lock(fd: int) -> bool:
try:
@@ -86,8 +108,7 @@ if os.name == "posix":
def _unlock(fd: int) -> None:
fcntl.flock(fd, fcntl.LOCK_UN)
-else: # pragma: no cover - exercised only on Windows
- import msvcrt
+elif msvcrt is not None: # pragma: no cover - exercised only on Windows
def _try_lock(fd: int) -> bool:
try:
@@ -102,6 +123,21 @@ else: # pragma: no cover - exercised only on Windows
except OSError:
pass
+else: # pragma: no cover - platforms with neither primitive, such as WASI
+
+ # NotConfiguredError, as in lm15-ts (`stores.ts`): a fact about this host's setup,
+ # not about a provider (CapabilityError) and not a timeout (LockTimeoutError).
+ def _try_lock(fd: int) -> bool:
+ raise NotConfiguredError(
+ "This platform provides no advisory file locking (neither fcntl nor msvcrt), "
+ "so lm15 cannot serialize credential refreshes against other processes. "
+ "Reading credentials still works; refreshing them from here does not.",
+ credential_hint="Pass an explicit credential instead of a stored login",
+ )
+
+ def _unlock(fd: int) -> None:
+ return None
+
@contextmanager
def hold_file_lock(
diff --git a/dspy/_vendor/lm15/_version.py b/dspy/_vendor/lm15/_version.py
index 06699f1bf..d958a62d9 100644
--- a/dspy/_vendor/lm15/_version.py
+++ b/dspy/_vendor/lm15/_version.py
@@ -1,3 +1,3 @@
"""The version of this source package, also used when building distributions."""
-__version__ = "1.0.0a1"
+__version__ = "1.0.0rc3"
diff --git a/dspy/_vendor/lm15/adaptation.py b/dspy/_vendor/lm15/adaptation.py
new file mode 100644
index 000000000..855cc693b
--- /dev/null
+++ b/dspy/_vendor/lm15/adaptation.py
@@ -0,0 +1,218 @@
+"""
+MAP-13 — adapt freely, never invisibly; refuse only when a guess could hurt.
+
+lm15 makes two promises, ranked: first, change the model or provider
+string and the program keeps working; second, never change what the
+caller asked for. The second is kept by VISIBILITY, not refusal. When a
+wire cannot take a setting as asked, the adapter does the obvious thing
+and records it here; the record rides ``Response.adaptations`` and
+``StreamStartEvent.adaptations``, and ``lm.plan(request)`` previews it
+with no network.
+
+Translations — the adapter's ordinary job (``stop`` → ``stop_sequences``,
+effort → budget by the MAP-7 table) — are never recorded. A record exists
+only where the wire got something other than what was asked.
+
+The policy (``AdaptationPolicy``) is set on the LM (``adaptations=``) and
+on ``RouterConfig``:
+
+- ``"note"`` (default): adapt and record.
+- ``"silent"``: adapt exactly as ``"note"`` does; the response carries no
+ record. The policy never changes what goes to the wire or what lm15
+ does after it.
+- ``"refuse"``: every DEVIATION — ``dropped``, ``clamped``, ``substituted``,
+ ``client_side`` — is an ``UnsupportedFeatureError`` before the wire (the
+ pre-2026-09-14 behaviour), carrying ``feature`` = the config path so a
+ policy layer can act without parsing prose. ``satisfied`` and
+ ``defaulted`` change nothing the caller asked for (the provider already
+ does it; the wire needed a value the caller left open) and are recorded
+ under every policy but ``"silent"``.
+
+Nothing here prints. The record is data on the response; a nagging
+warning channel gets ignored, then switched off, and the drop is silent
+again.
+
+How the record reaches the response without threading a parameter
+through every builder: ``collecting(policy)`` opens a scope (a context
+variable — thread-safe, task-safe, and copied into ``asyncio.to_thread``)
+around one ``build_request`` call; ``adapt(...)`` inside a builder appends
+to the open scope. ``BaseProviderLM._build`` is the only opener. A
+builder called with no scope open (a test, the vet harness's request
+direction) adapts under ``"note"`` and the record is simply not kept.
+"""
+from __future__ import annotations
+
+from contextlib import contextmanager
+from contextvars import ContextVar
+from dataclasses import dataclass, field
+from typing import Any, Iterator, Literal, get_args
+
+from .errors import UnsupportedFeatureError
+
+AdaptationAction = Literal["dropped", "clamped", "substituted", "client_side", "satisfied", "defaulted"]
+ADAPTATION_ACTIONS: tuple[str, ...] = get_args(AdaptationAction)
+
+# The actions that change what the caller asked for; "refuse" refuses these.
+DEVIATIONS: frozenset[str] = frozenset({"dropped", "clamped", "substituted", "client_side"})
+
+AdaptationPolicy = Literal["note", "silent", "refuse"]
+ADAPTATION_POLICIES: tuple[str, ...] = get_args(AdaptationPolicy)
+
+
+@dataclass(frozen=True, slots=True)
+class Adaptation:
+ """One thing the wire got that differs from what was asked.
+
+ - ``field``: the config path (``config.seed``, ``config.temperature``,
+ ``config.reasoning.summary``, ``tools``).
+ - ``action``: ``dropped`` (a hint with no home), ``clamped`` (a dial
+ to its nearest level), ``substituted`` (the closest spelling),
+ ``client_side`` (lm15 does it after the wire), ``satisfied`` (the
+ provider's default already is what was asked), ``defaulted`` (the
+ wire requires a value the caller did not set).
+ - ``asked`` / ``applied``: what the caller set / what went to the wire.
+ ``asked`` is absent for ``defaulted``; ``applied`` is absent for
+ ``dropped`` and ``satisfied``.
+ - ``reason``: one sentence naming the provider fact.
+ """
+
+ field: str
+ action: AdaptationAction
+ reason: str
+ asked: Any = None
+ applied: Any = None
+
+ def __post_init__(self) -> None:
+ if not isinstance(self.field, str) or not self.field:
+ raise ValueError("Adaptation.field must be a non-empty string")
+ if self.action not in ADAPTATION_ACTIONS:
+ raise ValueError(f"unsupported adaptation action: {self.action!r}")
+ if not isinstance(self.reason, str) or not self.reason:
+ raise ValueError("Adaptation.reason must be a non-empty string")
+
+
+def check_policy(value: object) -> AdaptationPolicy:
+ if value not in ADAPTATION_POLICIES:
+ raise ValueError(
+ f"adaptations must be one of {', '.join(map(repr, ADAPTATION_POLICIES))}, got {value!r}"
+ )
+ return value # type: ignore[return-value]
+
+
+@dataclass(slots=True)
+class AdaptationScope:
+ policy: AdaptationPolicy
+ provider: str | None
+ records: list[Adaptation] = field(default_factory=list)
+ # plan(): the wire request is built and discarded, so no credential is
+ # invoked and none is needed — like resolve(), plan() is offline.
+ planning: bool = False
+
+
+_scope: ContextVar[AdaptationScope | None] = ContextVar("lm15_adaptation_scope", default=None)
+
+
+@contextmanager
+def collecting(policy: AdaptationPolicy, *, provider: str | None = None, planning: bool = False) -> Iterator[AdaptationScope]:
+ """Open a scope for one request build. Nested scopes are independent."""
+ scope = AdaptationScope(policy=check_policy(policy), provider=provider, planning=planning)
+ token = _scope.set(scope)
+ try:
+ yield scope
+ finally:
+ _scope.reset(token)
+
+
+def adapt(
+ field: str,
+ action: AdaptationAction,
+ reason: str,
+ *,
+ asked: Any = None,
+ applied: Any = None,
+ provider: str | None = None,
+) -> None:
+ """Record one adaptation in the open scope, or raise under ``"refuse"``.
+
+ Builders call this at the point where they would have raised before
+ MAP-13, then do the adapted thing. The message under ``"refuse"`` is
+ the same sentence the note carries, so the two policies never say
+ different things about the same fact.
+ """
+ scope = _scope.get()
+ policy: AdaptationPolicy = scope.policy if scope is not None else "note"
+ who = provider or (scope.provider if scope is not None else None)
+ if policy == "refuse" and action in DEVIATIONS:
+ head = f"{who}: " if who else ""
+ raise UnsupportedFeatureError(
+ f"{head}{field} {_ACTION_VERB[action]}: {reason} (adaptations='refuse')",
+ provider=who,
+ feature=field,
+ )
+ if scope is None:
+ return
+ # Every policy records into the scope: the adapter's own behaviour
+ # (a client-side stop, a narrowed tool list) is read from these
+ # records, so "silent" must not empty them — it hides them on the
+ # response instead (BaseProviderLM._build). A policy that changed
+ # behaviour would be a second, invisible setting.
+ scope.records.append(Adaptation(field=field, action=action, reason=reason, asked=asked, applied=applied))
+
+
+_ACTION_VERB = {
+ "dropped": "would be dropped",
+ "clamped": "would be clamped",
+ "substituted": "would be substituted",
+ "client_side": "would be applied client-side",
+ "satisfied": "is already satisfied here",
+ "defaulted": "would be defaulted",
+}
+
+
+def current_policy() -> AdaptationPolicy:
+ scope = _scope.get()
+ return scope.policy if scope is not None else "note"
+
+
+def is_planning() -> bool:
+ """True inside ``plan()``: the build's bytes are discarded, so signing
+ and credential providers are skipped."""
+ scope = _scope.get()
+ return scope is not None and scope.planning
+
+
+def adaptation_to_dict(a: Adaptation) -> dict[str, Any]:
+ out: dict[str, Any] = {"field": a.field, "action": a.action, "reason": a.reason}
+ if a.asked is not None:
+ out["asked"] = a.asked
+ if a.applied is not None:
+ out["applied"] = a.applied
+ return out
+
+
+def adaptation_from_dict(d: dict[str, Any]) -> Adaptation:
+ return Adaptation(
+ field=d["field"],
+ action=d["action"],
+ reason=d["reason"],
+ asked=d.get("asked"),
+ applied=d.get("applied"),
+ )
+
+
+# ─── Shared clamps ───────────────────────────────────────────────────
+
+EFFORT_LADDER: tuple[str, ...] = ("minimal", "low", "medium", "high", "xhigh", "max")
+
+
+def nearest_effort(asked: str, available: "tuple[str, ...] | list[str]") -> str:
+ """The closest level to ``asked`` among ``available`` on the ordinal
+ effort ladder. A tie goes to the lower level: the cheaper guess is
+ the one a caller who set a dial would rather see recorded."""
+ levels = [lvl for lvl in available if lvl in EFFORT_LADDER]
+ if not levels:
+ raise ValueError(f"no comparable effort levels in {available!r}")
+ if asked in levels:
+ return asked
+ want = EFFORT_LADDER.index(asked) if asked in EFFORT_LADDER else 0
+ return min(levels, key=lambda lvl: (abs(EFFORT_LADDER.index(lvl) - want), EFFORT_LADDER.index(lvl)))
diff --git a/dspy/_vendor/lm15/auth.py b/dspy/_vendor/lm15/auth.py
index 981e6305f..fe6565dc0 100644
--- a/dspy/_vendor/lm15/auth.py
+++ b/dspy/_vendor/lm15/auth.py
@@ -84,12 +84,27 @@ __all__ = [
"write_xai_credential",
]
-CLAUDE_CODE_CREDENTIALS_PATH = Path("~/.claude/.credentials.json").expanduser()
+def _user_path(*parts: str) -> Path:
+ """Resolve a path under the user's home without raising where there is none.
+
+ Evaluated at import to define the credential-path constants below, so it must not
+ raise. `Path.expanduser()` raises RuntimeError wherever no home directory can be
+ determined -- a WASI guest, some container setups -- and these constants are
+ defaults a caller can always override, so an unexpanded path is a better answer
+ than a failed import. Anything that actually opens it still fails, and says why.
+ """
+ try:
+ return Path("~", *parts).expanduser()
+ except RuntimeError:
+ return Path("~", *parts)
+
+
+CLAUDE_CODE_CREDENTIALS_PATH = _user_path(".claude", ".credentials.json")
CLAUDE_CODE_CLIENT_ID = "9d1c250a-e61b-44d5-88ed-5944d1962f5e"
CLAUDE_CODE_TOKEN_URL = "https://platform.claude.com/v1/oauth/token"
CLAUDE_CODE_LOGIN_HINT = "Log in again: run `claude` and use /login (Claude subscription auth)"
-CODEX_CLI_AUTH_PATH = Path("~/.codex/auth.json").expanduser()
+CODEX_CLI_AUTH_PATH = _user_path(".codex", "auth.json")
OPENAI_CODEX_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"
OPENAI_CODEX_TOKEN_URL = "https://auth.openai.com/oauth/token"
OPENAI_CODEX_JWT_CLAIM_PATH = "https://api.openai.com/auth"
@@ -506,7 +521,7 @@ XAI_DEVICE_CODE_URL = "https://auth.x.ai/oauth2/device/code"
XAI_TOKEN_URL = "https://auth.x.ai/oauth2/token"
XAI_OAUTH_SCOPE = "openid profile email offline_access grok-cli:access api:access"
XAI_LOGIN_HINT = "Log in again: run lm15.auth.login_xai() (SuperGrok / X Premium subscription auth)"
-PI_AGENT_AUTH_PATH = Path("~/.pi/agent/auth.json").expanduser()
+PI_AGENT_AUTH_PATH = _user_path(".pi", "agent", "auth.json")
_XAI_PROVIDER_KEY = "xai"
_XAI_DEFAULT_TOKEN_LIFETIME_S = 3600
diff --git a/dspy/_vendor/lm15/compat.py b/dspy/_vendor/lm15/compat.py
index eb72102c7..bef3eecb7 100644
--- a/dspy/_vendor/lm15/compat.py
+++ b/dspy/_vendor/lm15/compat.py
@@ -191,6 +191,7 @@ OPENAI_RESPONSES_PRESETS: dict[str, OpenAIResponsesCompat] = {
cache_control="none",
tool_result_media="reject", # MAP-10: no receipt on this door — reject until one exists
),
+ "lmstudio": None, # type: ignore[dict-item] # filled below: ollama's policy object
"vllm": OpenAIResponsesCompat(
developer_role="system",
max_output_tokens_field="max_tokens",
@@ -278,8 +279,19 @@ OPENAI_RESPONSES_PRESETS: dict[str, OpenAIResponsesCompat] = {
# Default base URLs for the Responses presets that name a server (the
# registry's access policies point here so each URL has one copy).
+OPENAI_RESPONSES_PRESETS["lmstudio"] = OPENAI_RESPONSES_PRESETS["ollama"]
+
+# A server's OpenAI root is one address whichever OpenAI-shaped path is
+# used; the local engines' roots are the chat table's. A preset that names
+# a server absent here (qwen, deepseek, zai: no documented Responses root)
+# is REFUSED at construction without an explicit base_url — never sent to
+# the OpenAI cloud (preset_base_url below, 2026-09-11).
OPENAI_RESPONSES_PRESET_BASE_URLS: dict[str, str] = {
"openai": "https://api.openai.com/v1",
+ "ollama": "http://localhost:11434/v1",
+ "lmstudio": "http://localhost:1234/v1",
+ "vllm": "http://localhost:8000/v1",
+ "sglang": "http://localhost:30000/v1",
"openrouter": "https://openrouter.ai/api/v1",
# dev.meta.ai overview.md: one base URL for every Meta Model API surface.
"meta": "https://api.meta.ai/v1",
@@ -496,6 +508,33 @@ def _preset_key(name: str) -> str:
return _OPENAI_CHAT_PRESET_ALIASES.get(key, key)
+def preset_base_url(table: dict[str, str], name: str, *, dialect: str, default_preset: str) -> str:
+ """The address a preset name supplies for one dialect.
+
+ A name that names a server supplies that server's root. The dialect's
+ own default (``default_preset``: ``openai`` / ``anthropic``) is the
+ only name that resolves to the cloud default. Any other name with no
+ entry in ``table`` is refused with :class:`NotConfiguredError`: a
+ request meant for a named server must never be sent to the OpenAI
+ cloud with whatever key is around because a table lacked a row
+ (2026-09-11; before, ``compat="lmstudio"`` did exactly that).
+ """
+ key = _preset_key(name)
+ try:
+ return table[key]
+ except KeyError:
+ pass
+ if key == default_preset:
+ return table[default_preset]
+ from .errors import NotConfiguredError
+
+ raise NotConfiguredError(
+ f"compat {name!r} names a server whose {dialect} address lm15 does not "
+ f"know; pass base_url= (the server's OpenAI-compatible root, e.g. "
+ f"'http://localhost:PORT/v1')"
+ )
+
+
# Spelling aliases → canonical preset key. Every alias is permanent. One
# map serves all three dialect tables: a name means the same server in each.
_OPENAI_CHAT_PRESET_ALIASES: dict[str, str] = {
@@ -504,8 +543,7 @@ _OPENAI_CHAT_PRESET_ALIASES: dict[str, str] = {
"chat_completions": "openai",
"responses": "openai",
"openai_responses": "openai",
- "lmstudio": "ollama",
- "lm_studio": "ollama",
+ "lm_studio": "lmstudio",
"dashscope_qwen": "qwen",
"z_ai": "zai",
}
@@ -528,17 +566,40 @@ OPENAI_CHAT_PRESETS: dict[str, OpenAIChatCompat] = {
cache_control="openai",
tool_result_media="reject", # MAP-10: text-only tool row; gpt-5.4 received the USER image and not the tool image
),
- # ollama / LM Studio: max_tokens, no reasoning dial on the wire.
+ # ollama: max_tokens; reasoning_effort (and `reasoning: {effort}`) on
+ # the wire, mapped to Ollama's `think` by openai/openai.go
+ # `thinkFromReasoningEffort` (lm15-contract/research/tool-result-content/
+ # sources/ollama.txt:536-560): none → think:false, minimal → low,
+ # low|medium|high|max verbatim, xhigh → max; an unknown word is a 400.
+ # Until 2026-09-14 this said thinking_format="none" with "no receipt";
+ # THEORY.md §3.17. Source receipt; live receipt still owed.
"ollama": OpenAIChatCompat(
instruction_role="system",
max_tokens_field="max_tokens",
stream_usage="include",
- thinking_format="none",
+ thinking_format="reasoning_effort",
+ reasoning_efforts=("minimal", "low", "medium", "high", "xhigh", "max"),
tool_result_name="omit",
strict_tools="omit",
cache_control="none",
tool_result_media="reject", # MAP-10: openai.go builds image rows without ToolCallID; no receipt — reject until one exists
),
+ # LM Studio: its own policy at http://localhost:1234/v1
+ # (OPENAI_CHAT_PRESET_BASE_URLS). HYPOTHESIS, no receipt (THEORY.md
+ # §3.17): lmstudio.ai lists max_tokens and no reasoning dial, so
+ # thinking_format="none" — under MAP-13 a set dial is dropped and
+ # recorded, never refused on this unverified line. Until 2026-09-11
+ # the name was an alias of "ollama".
+ "lmstudio": OpenAIChatCompat(
+ instruction_role="system",
+ max_tokens_field="max_tokens",
+ stream_usage="include",
+ thinking_format="none",
+ tool_result_name="omit",
+ strict_tools="omit",
+ cache_control="none",
+ tool_result_media="reject",
+ ),
# Groq: server-executed builtin tools (browser_search / code_interpreter,
# live 2026-09-01); reasoning_effort dial; no cache_control field.
"groq": OpenAIChatCompat(
@@ -756,9 +817,12 @@ OPENAI_CHAT_PRESETS: dict[str, OpenAIChatCompat] = {
# Used by OpenAIChatLM when a compat preset is given by name and no
# explicit base_url overrides it; the provider registry's access policies
# point here so there is one copy of each URL.
+
OPENAI_CHAT_PRESET_BASE_URLS: dict[str, str] = {
"openai": "https://api.openai.com/v1",
"ollama": "http://localhost:11434/v1",
+ # lmstudio.ai docs (Local Server): "http://localhost:1234/v1".
+ "lmstudio": "http://localhost:1234/v1",
"groq": "https://api.groq.com/openai/v1",
"openrouter": "https://openrouter.ai/api/v1",
"xai": "https://api.x.ai/v1",
diff --git a/dspy/_vendor/lm15/errors.py b/dspy/_vendor/lm15/errors.py
index b26c3b338..f7c24c267 100644
--- a/dspy/_vendor/lm15/errors.py
+++ b/dspy/_vendor/lm15/errors.py
@@ -109,19 +109,69 @@ class LockTimeoutError(LM15Error):
super().__init__(message, **kwargs)
+class CollectionLimitError(LM15Error):
+ """A local turn collector reached its budget, not a provider failure.
+
+ Non-retryable. ``partial_events`` preserves the accepted events without
+ copying their payloads. ``partial`` materializes them on demand as an
+ incomplete Turn. A byte-limit failure also exposes ``rejected_event``:
+ it was received but not yielded or added to the collection. Process it
+ before resuming raw session reads if that content is needed.
+ """
+
+ default_code = "collection_limit"
+
+ def __init__(
+ self, message: str = "", *, limit: str | None = None,
+ maximum: int | None = None, retained_bytes: int = 0,
+ partial_events: tuple = (), rejected_event=None, **kwargs,
+ ) -> None:
+ self.limit = limit
+ self.maximum = maximum
+ self.retained_bytes = retained_bytes
+ self.partial_events = tuple(partial_events)
+ self.retained_events = len(self.partial_events)
+ self.rejected_event = rejected_event
+ super().__init__(message, **kwargs)
+
+ @property
+ def partial(self):
+ # Lazy: allocating combined text/audio while handling a collection
+ # limit would amplify memory pressure. Raw events are always available.
+ from dataclasses import replace
+ from .live import _materialize_turn
+
+ return replace(_materialize_turn(self.partial_events), ended_by="incomplete")
+
+
class StreamAssemblyError(LM15Error):
"""A stream could not be assembled into a Response without inventing a fact.
- Raised by the accumulator (MAP-9) when a tool call's fragments never
- carried a name: an unnamed call is not actionable (MAP-1), and guessing
- a name from the request dispatches the wrong function silently. This is
- an adapter defect, not model behaviour — every shipped dialect names a
- call on its first fragment — so the message points at the adapter.
+ Three defects raise it (MAP-9 and MAP-3, contract change
+ 2026-09-11-stream-completion):
+
+ - a tool call's fragments never carried a name (MAP-9): an unnamed call
+ is not actionable (MAP-1), and guessing a name from the request
+ dispatches the wrong function silently;
+ - the stream ended without an end event: the finish reason and usage
+ never arrived, and reporting the text as a finished turn would invent
+ both;
+ - an event arrived after the end event (MAP-3): it has no place in the
+ Response, and dropping it would be silent loss.
+
+ All three are adapter or source defects, not model behaviour — every
+ shipped dialect names a call on its first fragment and ends exactly
+ once — so the message points at the adapter.
``partial`` is everything that did assemble (text, thinking, other
- parts, usage, finish reason) with the unnamed call(s) left out, so a
+ parts, usage, finish reason) with the offending material left out, so a
caller that wants to salvage the turn can; ``part_index`` is the first
- offending part.
+ offending part (MAP-9 only).
+
+ What does NOT raise it: a failure after the end event that is not an
+ event — the source raising while it drains, or its ``close()`` raising.
+ The Response is complete; it is returned, and the failure is reported
+ as a :class:`StreamCleanupWarning`.
"""
default_code = "stream_assembly"
@@ -139,6 +189,19 @@ class StreamAssemblyError(LM15Error):
self.part_index = part_index
+class StreamCleanupWarning(RuntimeWarning):
+ """A stream's source failed after the Response was already complete.
+
+ Emitted (``warnings.warn``) when, after the end event has been yielded,
+ the source raises while draining or its ``close()``/``aclose()`` raises.
+ The provider finished the turn and billed it; the Response is returned
+ unchanged. The failure is about the connection's afterlife, not the
+ answer, so it is never raised from ``response`` — a caller who wants it
+ programmatically reads ``ResponseStream.cleanup_errors``, and a caller
+ who wants it fatal runs with ``-W error::lm15.errors.StreamCleanupWarning``.
+ """
+
+
class ConfigurationError(LM15Error):
"""Local SDK or provider-adapter configuration failure."""
@@ -146,10 +209,21 @@ class ConfigurationError(LM15Error):
class CapabilityError(LM15Error):
- """Requested capability is not supported by this provider adapter."""
+ """Requested capability is not supported by this provider adapter.
+
+ ``feature`` (MAP-13, 2026-09-14) is the config path of what was
+ refused — ``config.top_k``, ``config.reasoning.thinking_budget``,
+ ``messages[0].parts[1]``, ``tools[2]`` — so the caller's own policy
+ layer can drop it and retry without parsing the message. Absent
+ when the refusal is not about one addressable field.
+ """
default_code = "unsupported_feature"
+ def __init__(self, message: str = "", *, feature: str | None = None, **kwargs) -> None:
+ self.feature = feature
+ super().__init__(message, **kwargs)
+
class ProviderError(LM15Error):
"""The provider returned an error response."""
@@ -209,9 +283,9 @@ class AuthError(ProviderError):
)
if self.env_keys:
keys = " or ".join(f"{key}=..." for key in self.env_keys)
- guidance += f" - Set the provider API key in your environment: {keys}\n"
+ guidance += f" - Pass the key explicitly (api_key, or RouterConfig api_keys), or on a host with an environment set {keys}\n"
else:
- guidance += " - Set the provider API key in your environment\n"
+ guidance += " - Pass the key explicitly (api_key, or RouterConfig api_keys)\n"
if provider_name:
guidance += f" - Verify your {provider_name} account/project has access\n"
@@ -323,7 +397,7 @@ class NotConfiguredError(ConfigurationError):
guidance = "\n\n To fix:\n"
if self.env_keys:
keys = " or ".join(f"{key}=..." for key in self.env_keys)
- guidance += f" - Set the provider API key in your environment: {keys}\n"
+ guidance += f" - Pass the key explicitly (api_key, or RouterConfig api_keys), or on a host with an environment set {keys}\n"
if provider_name:
guidance += f" - Configure credentials for {provider_name}\n"
@@ -469,6 +543,7 @@ _CLASS_TO_CODE: dict[type[LM15Error], str] = {
TransportError: "transport",
LockTimeoutError: "lock_timeout",
StreamAssemblyError: "stream_assembly",
+ CollectionLimitError: "collection_limit",
ProviderError: "provider",
}
diff --git a/dspy/_vendor/lm15/live.py b/dspy/_vendor/lm15/live.py
index 41af1104d..8ac14a259 100644
--- a/dspy/_vendor/lm15/live.py
+++ b/dspy/_vendor/lm15/live.py
@@ -16,6 +16,7 @@ from collections.abc import Sequence
from dataclasses import dataclass
from typing import Any, Callable, Deque
+from .errors import CollectionLimitError, TransportError
from .types import (
ErrorDetail,
LiveClientAudioEvent,
@@ -73,11 +74,30 @@ def require_websocket_async_connect():
# Plain session iteration is the primary surface for that. `turn()` and
# `Turn` serve the half-duplex idiom (send, then listen until the turn
# ends) — the shape every scripted recipe and turn-based voice app has.
-# They are per-language ergonomics like BatchJob, not canonical wire
-# types: ports choose their own idiom; the transcript harness pins the
-# event stream, not this sugar.
+# Their shared behavior is specified in the contract's
+# 2026-09-11-shared-handles-and-profile-migration decision. They remain
+# provisional collectors, not new canonical wire types.
_TURN_TERMINAL = frozenset({"turn_end", "interrupted", "error"})
+DEFAULT_TURN_MAX_BYTES = 16 * 1024 * 1024
+DEFAULT_TURN_MAX_EVENTS = 10_000
+
+
+def _event_size(event: LiveServerEvent, remaining: int) -> int:
+ """Compact ASCII JSON byte charge, not a claim about resident memory.
+
+ Count incrementally and stop as soon as the event would exceed the
+ remaining budget. No serialized copy of the whole collection is built.
+ """
+ from .serde import live_server_event_to_dict
+
+ encoder = json.JSONEncoder(ensure_ascii=True, separators=(",", ":"), allow_nan=False)
+ size = 0
+ for chunk in encoder.iterencode(live_server_event_to_dict(event)):
+ size += len(chunk) # ASCII characters are exactly one byte each.
+ if size > remaining:
+ break
+ return size
@dataclass(frozen=True)
@@ -91,8 +111,9 @@ class Turn:
cancelled response's tokens precede its ``interrupted``. A ``tool_call`` ending mirrors the non-live
``finish_reason="tool_call"`` contract: the model is waiting for
YOUR result — answer with ``send_tool_result()`` and materialize the
- next turn. Materializing buffers text and audio in memory until the
- turn ends; for latency-sensitive playback iterate events instead.
+ next turn. turn() collects within a configurable byte/event budget.
+ Materializing adds combined text/audio copies; use raw session events
+ when retaining a whole turn is not wanted.
"""
ended_by: str
@@ -135,7 +156,9 @@ def _materialize_turn(events: tuple[LiveServerEvent, ...]) -> Turn:
if event.type == "text":
text_parts.append(event.text)
elif event.type == "audio":
- audio.extend(base64.b64decode(event.data))
+ if audio_media_type is not None and event.media_type is not None and audio_media_type != event.media_type:
+ raise ValueError("a Turn cannot concatenate different audio media types; consume raw events")
+ audio.extend(base64.b64decode(event.data, validate=True))
if audio_media_type is None and event.media_type is not None:
audio_media_type = event.media_type
elif event.type == "tool_call":
@@ -149,7 +172,7 @@ def _materialize_turn(events: tuple[LiveServerEvent, ...]) -> Turn:
usage = _sum_usage(usage, event.usage)
elif event.type == "error":
error = event.error
- ended_by = events[-1].type if events and events[-1].type in (_TURN_TERMINAL | {"tool_call"}) else "error"
+ ended_by = events[-1].type if events and events[-1].type in (_TURN_TERMINAL | {"tool_call"}) else "incomplete"
return Turn(
ended_by=ended_by,
text="".join(text_parts),
@@ -162,66 +185,157 @@ def _materialize_turn(events: tuple[LiveServerEvent, ...]) -> Turn:
)
-class TurnView:
- """Iterator over one turn's server events.
-
- Ends itself after yielding the terminal event (``turn_end`` /
- ``interrupted`` / ``error``) — the same self-ending idiom as
- ``stream()``. Tool calls are yielded mid-iteration (you hold the
- session, so you can answer and keep iterating); ``result()`` cannot
- answer for you, so it returns at a ``tool_call`` instead of
- deadlocking against a model that is waiting for your result.
- """
-
- def __init__(self, session: Any) -> None:
+class _TurnState:
+ def __init__(
+ self, session: Any, *, max_bytes: int = DEFAULT_TURN_MAX_BYTES,
+ max_events: int = DEFAULT_TURN_MAX_EVENTS,
+ ) -> None:
+ for name, value in (("max_bytes", max_bytes), ("max_events", max_events)):
+ if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
+ raise ValueError(f"{name} must be a positive integer; use raw session events for unbuffered reading")
+ self._max_bytes = max_bytes
+ self._max_events = max_events
+ self._retained_bytes = 0
self._session = session
self._done = False
+ self._events: list[LiveServerEvent] = []
+ self._failure: Exception | None = None
+ self._result: Turn | None = None
+ self._reading = False
+
+ @property
+ def retained_bytes(self) -> int:
+ """Accepted events' compact-JSON byte charge, not process memory."""
+ return self._retained_bytes
+
+ @property
+ def retained_events(self) -> int:
+ return len(self._events)
+
+ def snapshot(self) -> Turn:
+ """Collected data so far; incomplete is not a successful turn."""
+ if isinstance(self._failure, CollectionLimitError):
+ return self._failure.partial
+ return _materialize_turn(tuple(self._events))
+
+ def _limit_error(self, limit: str, maximum: int, event=None) -> CollectionLimitError:
+ self._done = True
+ return CollectionLimitError(
+ f"Live turn collection exceeded {limit}={maximum}; the turn is incomplete. "
+ "Raise this limit on session.turn(...), or process raw session events without collecting. "
+ "Inspect partial_events and rejected_event before continuing; the session remains open.",
+ limit=limit, maximum=maximum, retained_bytes=self._retained_bytes,
+ partial_events=tuple(self._events), rejected_event=event,
+ )
+
+ def _check_event_limit(self) -> None:
+ # Do not consume the next event when the count alone proves it
+ # cannot fit. This includes a terminal event, which also counts.
+ if len(self._events) >= self._max_events:
+ raise self._limit_error("max_events", self._max_events)
+
+ def close(self) -> None:
+ """Stop this view, not the underlying live session."""
+ if self._reading:
+ raise RuntimeError("stop the active turn reader before closing its view")
+ self._done = True
+
+ def _accept(self, event: LiveServerEvent | None) -> LiveServerEvent:
+ if event is None:
+ raise TransportError("live session closed before the turn reached a boundary")
+ size = _event_size(event, self._max_bytes - self._retained_bytes)
+ if size > self._max_bytes - self._retained_bytes:
+ raise self._limit_error("max_bytes", self._max_bytes, event)
+ self._events.append(event)
+ self._retained_bytes += size
+ if event.type in _TURN_TERMINAL:
+ self._done = True
+ return event
+
+ def _seal(self) -> Turn:
+ if self._failure is not None:
+ raise self._failure
+ result = self.snapshot()
+ if result.ended_by == "incomplete":
+ raise TransportError("turn view closed before the turn reached a boundary; inspect snapshot()")
+ self._done = True
+ self._result = result
+ return result
+
+
+class TurnView(_TurnState):
+ """One buffered half-duplex view. Iteration stops on terminal events;
+ result also stops at a tool call. Already-yielded events stay in the result.
+ Defaults: 16 MiB of compact ASCII JSON event data, at most 10,000 events.
+ Limits are per view and configurable via max_bytes/max_events. On overflow
+ CollectionLimitError preserves partial events and any consumed rejected
+ event. The session stays open; this is not a cancellation or billing promise.
+ Use raw session iteration when buffering is not wanted.
+ """
def __iter__(self):
return self
def __next__(self) -> LiveServerEvent:
+ if self._reading:
+ raise RuntimeError("turn view already has an active reader")
+ if self._failure is not None:
+ raise self._failure
if self._done:
raise StopIteration
- event = self._session.recv()
- if event.type in _TURN_TERMINAL:
- self._done = True
- return event
+ self._reading = True
+ try:
+ self._check_event_limit()
+ return self._accept(self._session.recv())
+ except Exception as exc:
+ self._failure = exc
+ raise
+ finally:
+ self._reading = False
def result(self) -> Turn:
- events: list[LiveServerEvent] = []
- for event in self:
- events.append(event)
- if event.type == "tool_call":
- break
- return _materialize_turn(tuple(events))
-
+ if self._result is not None:
+ return self._result
+ # A tool call just yielded by manual iteration must not be read past
+ # by result(): the application may still owe the model an answer.
+ if not self._events or self._events[-1].type != "tool_call":
+ for event in self:
+ if event.type == "tool_call":
+ break
+ return self._seal()
-class AsyncTurnView:
- """Async twin of :class:`TurnView`."""
- def __init__(self, session: Any) -> None:
- self._session = session
- self._done = False
+class AsyncTurnView(_TurnState):
+ """Native async twin; task cancellation propagates without inventing a turn."""
def __aiter__(self):
return self
async def __anext__(self) -> LiveServerEvent:
+ if self._reading:
+ raise RuntimeError("turn view already has an active reader")
+ if self._failure is not None:
+ raise self._failure
if self._done:
raise StopAsyncIteration
- event = await self._session.recv()
- if event.type in _TURN_TERMINAL:
- self._done = True
- return event
+ self._reading = True
+ try:
+ self._check_event_limit()
+ return self._accept(await self._session.recv())
+ except Exception as exc:
+ self._failure = exc
+ raise
+ finally:
+ self._reading = False
async def result(self) -> Turn:
- events: list[LiveServerEvent] = []
- async for event in self:
- events.append(event)
- if event.type == "tool_call":
- break
- return _materialize_turn(tuple(events))
+ if self._result is not None:
+ return self._result
+ if not self._events or self._events[-1].type != "tool_call":
+ async for event in self:
+ if event.type == "tool_call":
+ break
+ return self._seal()
class WebSocketLiveSession:
@@ -319,9 +433,12 @@ class WebSocketLiveSession:
for event in decoded:
self._pending.append(event)
- def turn(self) -> TurnView:
- """Iterate one turn; see :class:`TurnView` and :class:`Turn`."""
- return TurnView(self)
+ def turn(
+ self, *, max_bytes: int = DEFAULT_TURN_MAX_BYTES,
+ max_events: int = DEFAULT_TURN_MAX_EVENTS,
+ ) -> TurnView:
+ """Collect one bounded turn. For unbuffered processing, iterate this session."""
+ return TurnView(self, max_bytes=max_bytes, max_events=max_events)
def close(self) -> None:
if self._closed:
@@ -488,9 +605,12 @@ class AsyncWebSocketLiveSession:
for event in decoded:
self._pending.append(event)
- def turn(self) -> AsyncTurnView:
- """Iterate one turn; see :class:`AsyncTurnView` and :class:`Turn`."""
- return AsyncTurnView(self)
+ def turn(
+ self, *, max_bytes: int = DEFAULT_TURN_MAX_BYTES,
+ max_events: int = DEFAULT_TURN_MAX_EVENTS,
+ ) -> AsyncTurnView:
+ """Async collection with the same limits and recovery as TurnView."""
+ return AsyncTurnView(self, max_bytes=max_bytes, max_events=max_events)
async def close(self) -> None:
if self._closed:
diff --git a/dspy/_vendor/lm15/profiles.py b/dspy/_vendor/lm15/profiles.py
index d84175922..e77422522 100644
--- a/dspy/_vendor/lm15/profiles.py
+++ b/dspy/_vendor/lm15/profiles.py
@@ -1,13 +1,28 @@
"""
-lm15.profiles — Provider endpoint profiles and compatibility resolution.
-
-ProviderProfile connects endpoint metadata, model metadata, and typed
-compatibility policy. It is optional: provider adapters can still be used with
-just api_key/base_url and Request(model="...").
+lm15.profiles — provider endpoint profiles (DEPRECATED) and the Responses
+compat resolution that survives them.
+
+``ProviderProfile`` / ``EndpointProfile`` / ``OpenAILM.from_profile`` /
+``OpenAILM(profile=...)`` are deprecated as of 1.0.0rc2 and removed in
+1.0.0 (contract ``changes/2026-09-11-job-handles-live-turns-profiles.md``
+§ 3). Everything they expressed has one home now:
+
+- an endpoint's address and wire policy: ``OpenAILM(compat="ollama")``
+ (a preset name supplies its own address) or ``compat=`` + ``base_url=``;
+- a per-model policy: the chat dialect's ``OpenAIChatCompat.model_overrides``,
+ or per request ``Config.extensions["openai_responses_compat"]``;
+- a compat guessed from the base URL: nothing — say ``compat="openrouter"``.
+
+What stays, in ``lm15.compat`` where it belongs: the partial-compat
+semantics (``None`` inherits, ``"auto"`` is the adapter default),
+``merge_openai_responses_compat``, and the request-level hatch
+(:func:`openai_responses_compat_from_extensions`). ``ModelInfo`` and
+``ModelRegistry`` were never profiles; they live in ``lm15.models``.
"""
from __future__ import annotations
+import warnings
from dataclasses import dataclass, field
from typing import Any
@@ -65,8 +80,23 @@ def _check_json_object_or_none(value: object, field_name: str) -> None:
# ─── Provider profiles ───────────────────────────────────────────────
+_DEPRECATION = (
+ "lm15.profiles.{name} is deprecated and will be removed in lm15 1.0.0: "
+ "pass compat= (a preset name supplies its address) and base_url= to the "
+ "adapter, model_overrides on OpenAIChatCompat, or "
+ "Config.extensions['openai_responses_compat'] per request "
+ "(lm15-contract changes/2026-09-11-job-handles-live-turns-profiles.md § 3)"
+)
+
+
+def _deprecated(name: str) -> None:
+ warnings.warn(_DEPRECATION.format(name=name), DeprecationWarning, stacklevel=3)
+
+
@dataclass(frozen=True, slots=True)
class EndpointProfile:
+ """DEPRECATED (1.0.0rc2; removed in 1.0.0). See the module docstring."""
+
name: str
api_family: str
base_url: str | None = None
@@ -74,6 +104,7 @@ class EndpointProfile:
extensions: JsonObject | None = None
def __post_init__(self) -> None:
+ _deprecated("EndpointProfile")
_check_nonempty_text(self.name, "EndpointProfile.name")
_check_nonempty_text(self.api_family, "EndpointProfile.api_family")
if self.base_url is not None:
@@ -83,12 +114,15 @@ class EndpointProfile:
@dataclass(frozen=True, slots=True)
class ProviderProfile:
+ """DEPRECATED (1.0.0rc2; removed in 1.0.0). See the module docstring."""
+
provider: str
endpoints: dict[str, EndpointProfile] = field(default_factory=dict)
models: tuple[ModelInfo, ...] = ()
extensions: JsonObject | None = None
def __post_init__(self) -> None:
+ _deprecated("ProviderProfile")
_check_nonempty_text(self.provider, "ProviderProfile.provider")
object.__setattr__(self, "endpoints", dict(self.endpoints))
object.__setattr__(self, "models", tuple(self.models))
@@ -148,7 +182,9 @@ def openai_responses_compat_from_extensions(extensions: JsonObject | None) -> Op
{"compat": {"openai_responses": {...}}}
{"compat": {"openai": {...}}} # generic OpenAI alias
- This is an escape hatch. Normal configuration should use ProviderProfile.
+ The per-request hatch: the one place a Responses-dialect policy can
+ differ per model after profiles are gone. Normal configuration is
+ ``OpenAILM(compat=...)``.
"""
if not extensions:
return None
@@ -195,9 +231,14 @@ def resolve_openai_responses_compat(
Layering:
bound compat (``OpenAILM(compat=...)``), else the base URL default
- < endpoint compat
- < model compat
+ < endpoint compat (deprecated with profiles)
+ < model compat (deprecated with profiles)
< request extension override
+
+ The base-URL default is a guess about the server and is deprecated
+ with the profiles: when it would pick anything but OpenAI's own
+ policy it warns, naming the ``compat=`` spelling that says the same
+ thing explicitly.
"""
partial = base if base is not None else _default_openai_responses_compat_for_base_url(base_url)
@@ -216,14 +257,38 @@ def resolve_openai_responses_compat(
return resolve_openai_responses_compat_partial(partial)
+def _redacted_url(url: str) -> str:
+ """The URL with its userinfo and query dropped: a gateway address can
+ carry a token in either, and a warning is not the place to print it
+ (greptile on dspy#10409)."""
+ from urllib.parse import urlsplit, urlunsplit
+
+ try:
+ parts = urlsplit(url)
+ # netloc is "user:pass@host:port"; keep only what follows the "@".
+ # (Reading parts.port instead raised on an out-of-range port and
+ # turned a warning into a failure — greptile on dspy#10409.)
+ host = parts.netloc.rsplit("@", 1)[-1]
+ return urlunsplit((parts.scheme, host, parts.path, "", ""))
+ except ValueError:
+ return "<url>"
+
+
def _default_openai_responses_compat_for_base_url(base_url: str) -> OpenAIResponsesCompat:
lower = base_url.lower()
+ guessed: str | None = None
if "openrouter.ai" in lower:
- return OpenAIResponsesCompat.preset("openrouter")
- if "api.openai.com" in lower:
- return OpenAIResponsesCompat.preset("openai")
- if "api.meta.ai" in lower:
- return OpenAIResponsesCompat.preset("meta")
- # Preserve the current OpenAILM Responses serializer behavior for unknown
- # endpoints unless a profile/preset explicitly overrides it.
+ guessed = "openrouter"
+ elif "api.meta.ai" in lower:
+ guessed = "meta"
+ if guessed is not None:
+ warnings.warn(
+ f"OpenAILM guessed compat={guessed!r} from base_url {_redacted_url(base_url)!r}; that guess is deprecated "
+ f"and will be removed in lm15 1.0.0 — pass compat={guessed!r} explicitly (it also supplies "
+ "the address)",
+ DeprecationWarning,
+ stacklevel=4,
+ )
+ return OpenAIResponsesCompat.preset(guessed)
+ # OpenAI's own policy for OpenAI and for unknown endpoints, as before.
return OpenAIResponsesCompat.preset("openai")
diff --git a/dspy/_vendor/lm15/providers/anthropic.py b/dspy/_vendor/lm15/providers/anthropic.py
index 8668453ba..c6f34fbad 100644
--- a/dspy/_vendor/lm15/providers/anthropic.py
+++ b/dspy/_vendor/lm15/providers/anthropic.py
@@ -4,9 +4,10 @@ from datetime import datetime
import json
import os
-from dataclasses import dataclass, field
+from dataclasses import dataclass, field, replace
from typing import Any, Callable, ClassVar, Iterator, Mapping
+from ..adaptation import AdaptationPolicy, adapt, check_policy, nearest_effort
from ..errors import (
AuthError,
BillingError,
@@ -22,7 +23,7 @@ from ..errors import (
map_http_error,
)
from ..access import ANTHROPIC_API
-from ..compat import ANTHROPIC_PRESET_BASE_URLS, AnthropicCompat, ResolvedAnthropicCompat, resolve_anthropic_compat
+from ..compat import ANTHROPIC_PRESET_BASE_URLS, AnthropicCompat, ResolvedAnthropicCompat, preset_base_url, resolve_anthropic_compat
from ..features import ProviderManifest
from ..sse import SSEEvent
from ..transports import TransportRequest
@@ -83,7 +84,6 @@ ANTHROPIC_PROVIDER_EXECUTED_BLOCKS = {
"code_execution_tool_result",
}
-_DEFAULT_ANTHROPIC_VISIBLE_TOKENS = 1024
_DEFAULT_ANTHROPIC_THINKING_BUDGET = 1024
@@ -115,10 +115,12 @@ def _response_format_to_anthropic_output_config(format_config: dict[str, Any]) -
constrained); `name` is a label with no slot (dropped, stated in MAP-8).
"""
if format_config["type"] == "json_object":
+ # MAP-13 rule 4(c) until a live receipt shows an open schema is
+ # accepted (changes/2026-09-14-adapt-visibly.md §4 item 4).
raise UnsupportedFeatureError(
"anthropic: response_format json_object is not supported — the Messages API has no "
"any-JSON mode; give a json_schema (objects need additionalProperties: false)",
- provider="anthropic",
+ provider="anthropic", feature="config.response_format",
)
return {"format": {"type": "json_schema", "schema": format_config["schema"]}}
@@ -158,15 +160,44 @@ def _reasoning_thinking_budget(request: Request) -> int | None:
return EFFORT_THINKING_BUDGETS[reasoning.effort]
-def _max_tokens_for_anthropic(request: Request, thinking_budget: int | None) -> int:
+# Output ceilings by model class, for the `max_tokens` the Messages API
+# requires and the caller did not set (MAP-13 `defaulted`, decision
+# 2026-09-14 §4.8). Until then the default was 1024, which cut ordinary
+# answers off with nothing said. The 3.x classes have documented lower
+# ceilings and a value above them is a 400; everything else (4.x and later,
+# and any name this table does not know) gets 16384 — loud and actionable
+# if a model's ceiling is lower ("max_tokens: 16384 > N"), never a silent
+# truncation. A table that rots; `Config.max_tokens` overrides.
+_DEFAULT_MAX_TOKENS_BY_CLASS: tuple[tuple[str, int], ...] = (
+ ("claude-3-haiku", 4096), ("claude-3-opus", 4096), ("claude-3-sonnet", 4096),
+ ("claude-3-5-", 8192), ("claude-3.5-", 8192),
+)
+_DEFAULT_MAX_TOKENS = 16384
+
+
+def _default_max_tokens(model: str) -> int:
+ lowered = model.lower()
+ for marker, ceiling in _DEFAULT_MAX_TOKENS_BY_CLASS:
+ if marker in lowered:
+ return ceiling
+ return _DEFAULT_MAX_TOKENS
+
+
+def _max_tokens_for_anthropic(request: Request, thinking_budget: int | None, *, provider: str = "anthropic") -> int:
"""Manual class: max_tokens includes thinking, so the wire ceiling is the
budget plus the visible cap. Adaptive class (thinking_budget None):
Config.max_tokens is the total ceiling — provider semantics, stated in
- spec/types.md."""
+ spec/types.md. The Messages API requires the field: when the caller
+ set none, the class default is used and recorded (MAP-13)."""
+ visible = request.config.max_tokens
+ if visible is None:
+ visible = _default_max_tokens(request.model)
+ adapt("config.max_tokens", "defaulted",
+ "the Messages API requires max_tokens and none was set; the class default was used",
+ applied=visible, provider=provider)
if thinking_budget is None:
- return request.config.max_tokens or _DEFAULT_ANTHROPIC_VISIBLE_TOKENS
- visible_budget = request.config.max_tokens or _DEFAULT_ANTHROPIC_VISIBLE_TOKENS
- return thinking_budget + visible_budget
+ return visible
+ return thinking_budget + visible
def _finish_reason(stop_reason: str | None, *, has_tool_call: bool = False) -> str:
@@ -249,12 +280,15 @@ class AnthropicLM(BaseProviderLM):
settings: "Mapping[str, str] | None" = None
clock: "Callable[[], datetime] | None" = field(default=None, repr=False)
+ # MAP-13 policy: "note" (adapt and record), "silent", or "refuse".
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="anthropic", init=False)
account_id: str | None = field(default=None, init=False, repr=False)
manifest: ClassVar[ProviderManifest] = ANTHROPIC_API
_resolved_compat: ResolvedAnthropicCompat = field(init=False, repr=False, default=ResolvedAnthropicCompat())
def __post_init__(self) -> None:
+ check_policy(self.adaptations)
self._bind_access(self.access, credentials_path=self.credentials_path, default_base_url=_DEFAULT_BASE_URL, settings=self.settings)
compat = self.compat if self.compat is not None else self._registry_compat()
if isinstance(compat, str):
@@ -263,7 +297,9 @@ class AnthropicLM(BaseProviderLM):
# as OpenAIChatLM).
resolved = resolve_anthropic_compat(AnthropicCompat.preset(compat))
if self.base_url == _DEFAULT_BASE_URL:
- self.base_url = ANTHROPIC_PRESET_BASE_URLS.get(compat.lower(), _DEFAULT_BASE_URL)
+ self.base_url = preset_base_url(
+ ANTHROPIC_PRESET_BASE_URLS, compat, dialect="Messages", default_preset="anthropic"
+ )
elif isinstance(compat, AnthropicCompat):
resolved = resolve_anthropic_compat(compat)
else:
@@ -459,7 +495,7 @@ class AnthropicLM(BaseProviderLM):
# ToolResultBlockParam takes text, image and document blocks only.
raise UnsupportedFeatureError(
f"{self.provider}: a {part.type} part cannot reach a tool_result block (text, image and document only; MAP-10)",
- provider=self.provider,
+ provider=self.provider, feature=f"messages[*].tool_result.content[{part.type}]",
)
return {"type": "text", "text": parts_to_text((part,), provider=self.provider)}
@@ -471,6 +507,21 @@ class AnthropicLM(BaseProviderLM):
parts = [{"type": "text", "text": f"[developer]\n{text}"}]
return {"role": role, "content": parts}
+ def _allowed_subset(self, request: Request) -> tuple[str, ...] | None:
+ """The names of a proper-subset allowlist, else None. MAP-13: the
+ Messages API cannot restrict to a subset, so the adapter sends
+ ONLY those tools — that is what "may only call these" means — and
+ records it as client_side."""
+ tc = request.config.tool_choice
+ if tc is None or tc.mode == "none" or not tc.allowed:
+ return None
+ if len(tc.allowed) == 1 and tc.mode == "required":
+ return None
+ declared = {t.name for t in request.tools}
+ if set(tc.allowed) == declared:
+ return None
+ return tuple(tc.allowed)
+
def _tool_choice_payload(self, request: Request) -> dict[str, Any] | None:
tc = request.config.tool_choice
if tc is None:
@@ -488,20 +539,11 @@ class AnthropicLM(BaseProviderLM):
if len(tc.allowed) == 1 and tc.mode == "required":
payload["type"] = "tool"
payload["name"] = tc.allowed[0]
- elif set(tc.allowed) == {t.name for t in request.tools}:
- # Allowing every declared tool is no restriction at all.
- payload["type"] = "any" if tc.mode == "required" else "auto"
else:
- # A proper-subset allowlist has no Anthropic wire form.
- # Degrading to any/auto would let the model call excluded
- # tools — raise, never silently widen the caller's policy.
- raise UnsupportedFeatureError(
- "anthropic: tool_choice.allowed subsets are not supported — "
- "the Messages API can force one named tool or allow all "
- "declared tools, but cannot restrict to a subset. Send only "
- "the allowed tools in Request.tools instead",
- provider=self.provider,
- )
+ # Every declared tool, or (MAP-13) a proper subset that
+ # _payload has already narrowed the tools list to: either
+ # way the wire form is any/auto over the tools sent.
+ payload["type"] = "any" if tc.mode == "required" else "auto"
elif tc.mode == "required":
payload["type"] = "any"
else:
@@ -541,17 +583,18 @@ class AnthropicLM(BaseProviderLM):
# key / resource name mechanisms the Messages API does not have.
if use_cache and cache_cfg is not None:
if cache_cfg.key is not None:
- raise UnsupportedFeatureError(
- "anthropic: cache.key is not supported — the Messages API has no "
- "cache affinity key (OpenAI's prompt_cache_key); marks on blocks "
- "are the mechanism (prefix / prefix_until_index)",
- provider=self.provider,
- )
+ # MAP-13: a best-effort routing hint by definition; no home here.
+ adapt("config.cache.key", "dropped",
+ "the Messages API has no cache affinity key (OpenAI's prompt_cache_key); "
+ "marks on blocks are its mechanism and were placed",
+ asked=cache_cfg.key, provider=self.provider)
if cache_cfg.resource is not None:
+ # MAP-13 rule 4(b): the program references a stored object
+ # that does not exist on this provider.
raise UnsupportedFeatureError(
"anthropic: cache.resource is not supported — the Messages API has no "
"stored-cache tier; it caches by marks on blocks",
- provider=self.provider,
+ provider=self.provider, feature="config.cache.resource",
)
idx = None
if cache_cfg.prefix_until_index is not None:
@@ -580,47 +623,53 @@ class AnthropicLM(BaseProviderLM):
)
if reasoning is not None and not reasoning.is_off:
if compat.reasoning_efforts is not None and reasoning.effort not in compat.reasoning_efforts:
- # MAP-7 rule 2: a word with no native level raises here when
- # the server would not refuse it (Moonshot's Anthropic wire
- # answered 200 to `medium` and to `bogus`, live 2026-09-03).
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.effort={reasoning.effort!r} has no level on this server "
- f"(it accepts {', '.join(compat.reasoning_efforts)}) and would be accepted silently",
- provider=self.provider,
- )
+ # MAP-13: an effort word with no level here is clamped to the
+ # nearest declared level (the dial is ordinal); the server
+ # would have accepted the word silently (Moonshot answered
+ # 200 to `medium` and to `bogus`, live 2026-09-03).
+ nearest = nearest_effort(reasoning.effort, compat.reasoning_efforts)
+ adapt("config.reasoning.effort", "clamped",
+ f"this server has no {reasoning.effort!r} level (it accepts "
+ f"{', '.join(compat.reasoning_efforts)}) and would have accepted the word silently",
+ asked=reasoning.effort, applied=nearest, provider=self.provider)
+ reasoning = replace(reasoning, effort=nearest)
if reasoning.summary in ("concise", "detailed"):
- raise UnsupportedFeatureError(
- f"anthropic: reasoning.summary={reasoning.summary!r} is an OpenAI detail level; "
- "the Messages API returns thinking blocks whenever thinking runs (use 'auto' or None)",
- provider=self.provider,
- )
+ # MAP-13: a visibility level the wire lacks; thinking blocks
+ # are returned whenever thinking runs, which is "auto".
+ adapt("config.reasoning.summary", "substituted",
+ "the Messages API has no summary detail levels; it returns thinking blocks "
+ "whenever thinking runs, which is 'auto'",
+ asked=reasoning.summary, applied="auto", provider=self.provider)
+ reasoning = replace(reasoning, summary="auto")
if adaptive:
if reasoning.thinking_budget is not None:
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.thinking_budget is not supported on {request.model} — "
- + ("this server ignores budget_tokens (a silent no-op); effort is the dial"
+ # MAP-13: effort carries the intent (MAP-7 rule 5); the
+ # budget has no honoured field on this class.
+ adapt("config.reasoning.thinking_budget", "dropped",
+ ("this server ignores budget_tokens; effort is the dial"
if deepseek_thinking else
- "this server accepts budget_tokens without translating it (a silent no-op); "
- "effort is the dial (protocols--messages.md)"
+ "this server accepts budget_tokens without translating it; effort is the dial "
+ "(protocols--messages.md)"
if always_adaptive else
- "this model class takes thinking.type 'adaptive' with output_config.effort; "
+ f"{request.model} takes thinking.type 'adaptive' with output_config.effort; "
"budget_tokens is rejected by the API (live 2026-09-02)"),
- provider=self.provider,
- )
+ asked=reasoning.thinking_budget, provider=self.provider)
+ reasoning = replace(reasoning, thinking_budget=None)
# MAP-7: the word goes verbatim on the always-adaptive server;
# it answers an unsupported level with a 400 of its own.
if reasoning.effort == "minimal" and not (deepseek_thinking or always_adaptive or effort_only):
- raise UnsupportedFeatureError(
- "anthropic: reasoning.effort='minimal' has no level on this model class "
- "(output_config.effort is low|medium|high|xhigh|max); 'low' is the floor",
- provider=self.provider,
- )
+ adapt("config.reasoning.effort", "clamped",
+ "this model class has no 'minimal' level (output_config.effort is "
+ "low|medium|high|xhigh|max); 'low' is the floor",
+ asked="minimal", applied="low", provider=self.provider)
+ reasoning = replace(reasoning, effort="low")
+ request = replace(request, config=replace(request.config, reasoning=reasoning))
thinking_budget = None if adaptive else _reasoning_thinking_budget(request)
payload: dict[str, Any] = {
"model": request.model,
"messages": messages,
"stream": stream,
- "max_tokens": _max_tokens_for_anthropic(request, thinking_budget),
+ "max_tokens": _max_tokens_for_anthropic(request, thinking_budget, provider=self.provider),
}
if request.system:
@@ -632,44 +681,67 @@ class AnthropicLM(BaseProviderLM):
payload["system"] = [{"type": "text", "text": system_text, "cache_control": cache_marker}]
else:
payload["system"] = system_text
- if compat.sampling_params == "reject":
+ sampling_fixed = compat.sampling_params == "reject"
+ if sampling_fixed:
for name in ("temperature", "top_p", "top_k"):
if getattr(request.config, name) is not None:
# The server documents none of these and swallows them
- # silently (Moonshot, live 2026-09-03: temperature 0.5 is
- # HTTP 200 here, "only 1 is allowed" on its chat wire).
- raise UnsupportedFeatureError(
- f"{self.provider}: config.{name} is silently ignored by this server "
- "(the model's sampling is fixed); omit it",
- provider=self.provider,
- )
- if request.config.temperature is not None:
- payload["temperature"] = request.config.temperature
- if request.config.top_p is not None:
+ # silently (Moonshot, live 2026-09-03). MAP-13: omit and
+ # record — the note supplies the visibility.
+ adapt(f"config.{name}", "dropped",
+ "this server ignores sampling parameters (the model's sampling is fixed)",
+ asked=getattr(request.config, name), provider=self.provider)
+ for name in ("seed", "frequency_penalty", "presence_penalty"):
+ if getattr(request.config, name) is not None:
+ # MAP-13: a sampling hint with no field on the Messages API.
+ adapt(f"config.{name}", "dropped",
+ f"the Messages API has no {name} field",
+ asked=getattr(request.config, name), provider=self.provider)
+ if request.config.temperature is not None and not sampling_fixed:
+ temperature = request.config.temperature
+ if temperature > 1.0:
+ # MAP-13: the canonical range is 0–2; this wire's ceiling is
+ # 1.0 and both scales default to 1.0, so "hotter than allowed"
+ # becomes the hottest. Never rescaled.
+ adapt("config.temperature", "clamped",
+ "the Messages API accepts temperature in [0, 1]; the canonical range is [0, 2]",
+ asked=temperature, applied=1.0, provider=self.provider)
+ temperature = 1.0
+ payload["temperature"] = temperature
+ if request.config.top_p is not None and not sampling_fixed:
payload["top_p"] = request.config.top_p
- if request.config.top_k is not None:
+ if request.config.top_k is not None and not sampling_fixed:
payload["top_k"] = request.config.top_k
if request.config.stop:
payload["stop_sequences"] = list(request.config.stop)
if request.tools:
tools_wire: list[dict[str, Any]] = []
+ allowed_subset = self._allowed_subset(request)
for tool in request.tools:
+ if allowed_subset is not None and tool.name not in allowed_subset:
+ continue
if isinstance(tool, FunctionTool):
tools_wire.append({"name": tool.name, "description": tool.description, "input_schema": tool.parameters})
elif isinstance(tool, BuiltinTool):
tools_wire.append(_builtin_to_anthropic(tool))
+ if allowed_subset is not None:
+ adapt("config.tool_choice.allowed", "client_side",
+ "the Messages API cannot restrict to a subset of the declared tools; "
+ "only the allowed tools were sent, which is what the allowlist means",
+ asked=list(request.config.tool_choice.allowed), applied=[t["name"] for t in tools_wire],
+ provider=self.provider)
payload["tools"] = tools_wire
tool_choice = self._tool_choice_payload(request)
if tool_choice is not None:
tc = request.config.tool_choice
if compat.parallel_tool_calls == "reject" and tc is not None and tc.parallel is not None:
# disable_parallel_tool_use is documented as ignored
- # (guide--anthropic-api.md); a silent no-op is refused (MAP-8 §2).
- raise UnsupportedFeatureError(
- f"{self.provider}: tool_choice.parallel is silently ignored by this server "
- "(disable_parallel_tool_use is not applied); omit it",
- provider=self.provider,
- )
+ # (guide--anthropic-api.md). MAP-13: omit it and say so.
+ adapt("config.tool_choice.parallel", "dropped",
+ "this server accepts disable_parallel_tool_use and does not apply it "
+ "(guide--anthropic-api.md); the model may return several calls",
+ asked=tc.parallel, provider=self.provider)
+ tool_choice.pop("disable_parallel_tool_use", None)
payload["tool_choice"] = tool_choice
if deepseek_thinking:
# DeepSeek over the Anthropic wire (guide--anthropic-api.md; live
@@ -711,13 +783,14 @@ class AnthropicLM(BaseProviderLM):
# The server accepts output_config.format and ignores the
# schema (DeepSeek, live 2026-09-03: 200 with keys the schema
# never named). Silent, so refuse before the wire.
- raise UnsupportedFeatureError(
- f"{self.provider}: response_format is silently ignored by this server "
- "(output_config.format is accepted and not applied); describe the shape in the prompt",
- provider=self.provider,
- )
- output_config = _response_format_to_anthropic_output_config(request.config.response_format)
- payload["output_config"] = {**payload.get("output_config", {}), **output_config}
+ # MAP-13: omit and record; the caller can describe the shape in the prompt.
+ adapt("config.response_format", "dropped",
+ "this server accepts output_config.format and does not apply it; "
+ "describe the shape in the prompt",
+ asked=request.config.response_format, provider=self.provider)
+ else:
+ output_config = _response_format_to_anthropic_output_config(request.config.response_format)
+ payload["output_config"] = {**payload.get("output_config", {}), **output_config}
# Promoted cross-provider knobs (changes/2026-09-01-extensions-burn-down):
# user_id rides Anthropic's metadata.user_id; store has no Anthropic
# wire field — raise, never silently drop.
@@ -725,19 +798,25 @@ class AnthropicLM(BaseProviderLM):
payload["service_tier"] = request.config.service_tier
if request.config.user_id is not None:
payload["metadata"] = {"user_id": request.config.user_id}
- if request.config.store is not None:
- raise UnsupportedFeatureError(
- "anthropic: config.store is not supported — the Messages API has no "
- "response-storage opt-out field (OpenAI and Gemini carry it)",
- provider=self.provider,
- )
+ if request.config.store is False:
+ # MAP-13 "satisfied": the Messages API keeps no retrievable
+ # stored-response object, so an opt-out holds by construction.
+ adapt("config.store", "satisfied",
+ "the Messages API has no stored-response object to opt out of; "
+ "nothing retrievable is kept",
+ asked=False, provider=self.provider)
+ elif request.config.store is True:
+ adapt("config.store", "dropped",
+ "the Messages API has no stored-response object to opt into "
+ "(OpenAI and Gemini carry `store`)",
+ asked=True, provider=self.provider)
if request.config.logprobs is not None:
- raise UnsupportedFeatureError(
- "anthropic: config.logprobs is not supported — the Messages API "
- "does not expose token log probabilities (OpenAI and Gemini "
- "carry them)",
- provider=self.provider,
- )
+ # MAP-13 (decision 2026-09-14 §4.1): Response.logprobs is optional;
+ # the program sees absence, not a later crash.
+ adapt("config.logprobs", "dropped",
+ "the Messages API does not expose token log probabilities "
+ "(OpenAI and Gemini carry them); Response.logprobs will be absent",
+ asked=request.config.logprobs, provider=self.provider)
if request.config.extensions:
passthrough = {k: v for k, v in request.config.extensions.items() if k != "prompt_caching"}
payload.update(passthrough)
@@ -764,7 +843,6 @@ class AnthropicLM(BaseProviderLM):
endpoint="messages",
stream=stream,
model=request.model,
- read_timeout=120.0 if stream else 60.0,
)
# ─── Response parsing ───────────────────────────────────────────
@@ -1075,12 +1153,10 @@ class AnthropicLM(BaseProviderLM):
def _batch_submit_request(self, request: BatchRequest, upload_body: dict[str, Any] | None) -> TransportRequest:
if request.label is not None:
- raise UnsupportedFeatureError(
- "anthropic: batch labels are not supported — the Message Batches "
- "create body has no metadata field (verified live 2026-08-31); "
- "submit without a label and correlate by id",
- provider=self.provider,
- )
+ adapt("label", "dropped",
+ "the Message Batches create body has no metadata field (verified live "
+ "2026-08-31); correlate by id",
+ asked=request.label, provider=self.provider)
payload = {
"requests": [
{"custom_id": str(i), "params": self._payload(nested, stream=False)}
diff --git a/dspy/_vendor/lm15/providers/async_base.py b/dspy/_vendor/lm15/providers/async_base.py
index af6038761..0c3fd381c 100644
--- a/dspy/_vendor/lm15/providers/async_base.py
+++ b/dspy/_vendor/lm15/providers/async_base.py
@@ -21,6 +21,7 @@ import os
from dataclasses import dataclass, field
from typing import Any, AsyncIterator, Callable, ClassVar, Mapping, Protocol, TypeVar
+from ..adaptation import AdaptationPolicy
from ..errors import (
ProviderError,
TransportError as LM15TransportError,
@@ -46,7 +47,7 @@ from ..types import (
StreamEvent,
)
from .anthropic import AnthropicLM
-from .base import BaseProviderLM, Credential, HttpResponse, _attach_error_metadata
+from .base import BaseProviderLM, Credential, HttpResponse, _attach_error_metadata, _client_side_stop
from .claude_code import DEFAULT_CLAUDE_CODE_VERSION, ClaudeCodeLM
from .gemini import GeminiLM
from .openai import OpenAILM
@@ -144,23 +145,50 @@ class AsyncBaseProviderLM:
return await asyncio.to_thread(build, *args, **kwargs)
return build(*args, **kwargs)
+ # MAP-13 policy; every async dataclass declares the field (constructor
+ # parity with its sync sibling) and this is the fallback.
+ adaptations: AdaptationPolicy = "note"
+
+ def plan(self, request: Request):
+ """What a call WOULD adapt (MAP-13), no network, no credential; pure, so sync."""
+ return self._inner.plan(request, policy=self.adaptations)
+
async def complete(self, request: Request) -> Response:
- req = await self._build(self._inner.build_request, request, stream=False)
+ req, adaptations = await self._build(self._inner._build, request, stream=False, policy=self.adaptations)
+ if _client_side_stop(adaptations):
+ # Mirror of BaseProviderLM.complete: a client-side stop streams
+ # and closes at the cut (MAP-13); see the sync comment.
+ from ..result import amaterialize_response
+
+ return await amaterialize_response(self.stream(request), request)
resp = await self._send(req)
if resp.status >= 400:
raise self._inner._http_error(resp)
- return self._inner.parse_response(request, resp)
+ return self._inner._finish_response(request, self._inner.parse_response(request, resp), adaptations, policy=self.adaptations)
def stream(self, request: Request) -> AsyncIterator[StreamEvent]:
# MAP-3 (docs/mapping-rules.md): adapters may emit one end event per
# provider terminal frame; the coalescer merges them so the public
# stream yields exactly one final StreamEndEvent.
- from ..result import acoalesce_stream
-
- return acoalesce_stream(self._stream_raw(request), model=request.model)
-
- async def _stream_raw(self, request: Request) -> AsyncIterator[StreamEvent]:
- req = await self._build(self._inner.build_request, request, stream=True)
+ return self._astream(request)
+
+ async def _astream(self, request: Request) -> AsyncIterator[StreamEvent]:
+ from ..result import acoalesce_stream, atruncate_stream_at_stop
+ req, adaptations = await self._build(self._inner._build, request, stream=True, policy=self.adaptations)
+ events = acoalesce_stream(self._stream_raw(request, req), model=request.model, adaptations=self._inner._visible(adaptations, policy=self.adaptations))
+ if _client_side_stop(adaptations):
+ events = atruncate_stream_at_stop(events, request.config.stop)
+ try:
+ async for event in events:
+ yield event
+ finally:
+ aclose = getattr(events, "aclose", None)
+ if aclose is not None:
+ await aclose()
+
+ async def _stream_raw(self, request: Request, req: "TransportRequest | None" = None) -> AsyncIterator[StreamEvent]:
+ if req is None:
+ req = await self._build(self._inner.build_request, request, stream=True)
try:
async with self.transport.stream(req) as resp:
if resp.status >= 400:
@@ -187,6 +215,7 @@ class AsyncBaseProviderLM:
headers=resp.headers,
body=body,
http_version=resp.http_version,
+ provider=self.provider,
)
except NetworkTransportError as exc:
raise LM15TransportError(str(exc)) from exc
@@ -492,6 +521,7 @@ class AsyncOpenAILM(AsyncBaseProviderLM):
settings: "Mapping[str, str] | None" = field(default=None, kw_only=True)
clock: "Callable[[], datetime] | None" = field(default=None, repr=False, kw_only=True)
account_id: str | None = None
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
async def live(self, config: LiveConfig):
self._inner._require("live")
@@ -521,6 +551,7 @@ class AsyncOpenAILM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = OpenAILM(
+ adaptations=self.adaptations,
api_key=self.api_key,
transport=_ForbiddenTransport(),
base_url=self.base_url,
@@ -549,6 +580,7 @@ class AsyncAnthropicLM(AsyncBaseProviderLM):
credentials_path: "str | os.PathLike[str] | None" = field(default=None, repr=False)
settings: "Mapping[str, str] | None" = None
clock: "Callable[[], datetime] | None" = field(default=None, repr=False)
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="anthropic", init=False)
manifest: ClassVar[ProviderManifest] = AnthropicLM.manifest
@@ -557,6 +589,7 @@ class AsyncAnthropicLM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = AnthropicLM(
+ adaptations=self.adaptations,
api_key=self.api_key,
transport=_ForbiddenTransport(),
base_url=self.base_url,
@@ -584,6 +617,7 @@ class AsyncGeminiLM(AsyncBaseProviderLM):
credentials_path: "str | os.PathLike[str] | None" = field(default=None, repr=False)
settings: "Mapping[str, str] | None" = None
clock: "Callable[[], datetime] | None" = field(default=None, repr=False)
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="gemini", init=False)
manifest: ClassVar[ProviderManifest] = GeminiLM.manifest
@@ -592,6 +626,7 @@ class AsyncGeminiLM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = GeminiLM(
+ adaptations=self.adaptations,
api_key=self.api_key,
transport=_ForbiddenTransport(),
base_url=self.base_url,
@@ -635,6 +670,7 @@ class AsyncOpenAIChatLM(AsyncBaseProviderLM):
credentials_path: "str | os.PathLike[str] | None" = field(default=None, repr=False)
settings: "Mapping[str, str] | None" = None
clock: "Callable[[], datetime] | None" = field(default=None, repr=False)
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="openai-chat", init=False)
manifest: ClassVar[ProviderManifest] = OpenAIChatLM.manifest
@@ -643,6 +679,7 @@ class AsyncOpenAIChatLM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = OpenAIChatLM(
+ adaptations=self.adaptations,
api_key=self.api_key,
transport=_ForbiddenTransport(),
base_url=self.base_url,
@@ -685,6 +722,7 @@ class AsyncClaudeCodeLM(AsyncBaseProviderLM):
base_url: str = "https://api.anthropic.com/v1"
api_version: str = "2023-06-01"
claude_code_version: str = DEFAULT_CLAUDE_CODE_VERSION
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
# Not constructor params on the sync sibling either (it is not a dataclass).
provider: str = field(default="claude-code", init=False)
@@ -694,6 +732,7 @@ class AsyncClaudeCodeLM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = ClaudeCodeLM(
+ adaptations=self.adaptations,
api_key=self.api_key,
credentials_path=self.credentials_path,
transport=_ForbiddenTransport(),
@@ -750,6 +789,7 @@ class AsyncOpenAICodexLM(AsyncBaseProviderLM):
base_url: str = DEFAULT_CODEX_BASE_URL
originator: str = DEFAULT_CODEX_ORIGINATOR
client_version: str = DEFAULT_CODEX_CLIENT_VERSION
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
# Not constructor params on the sync sibling either (it is not a dataclass).
provider: str = field(default="openai-codex", init=False)
@@ -759,6 +799,7 @@ class AsyncOpenAICodexLM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = OpenAICodexLM(
+ adaptations=self.adaptations,
api_key=self.api_key,
account_id=self.account_id,
auth_path=self.auth_path,
@@ -824,6 +865,7 @@ class AsyncOpenAICodexLM(AsyncBaseProviderLM):
@dataclass(slots=True)
class AsyncXaiLM(AsyncBaseProviderLM):
"""Async mirror of :class:`XaiLM` (subscription OAuth or bearer key)."""
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
api_key: Credential | None = field(default=None, repr=False)
credentials_path: "str | os.PathLike[str] | None" = None
@@ -840,6 +882,7 @@ class AsyncXaiLM(AsyncBaseProviderLM):
def __post_init__(self) -> None:
self._inner = XaiLM(
+ adaptations=self.adaptations,
api_key=self.api_key,
credentials_path=self.credentials_path,
transport=_ForbiddenTransport(),
diff --git a/dspy/_vendor/lm15/providers/base.py b/dspy/_vendor/lm15/providers/base.py
index 469b1299a..6ecbd2744 100644
--- a/dspy/_vendor/lm15/providers/base.py
+++ b/dspy/_vendor/lm15/providers/base.py
@@ -20,6 +20,7 @@ from ..errors import (
map_http_error,
with_credential_hint,
)
+from ..adaptation import Adaptation, AdaptationPolicy, check_policy, collecting, is_planning
from ..credentials import AwsCredentials, CredentialLike, CredentialValue, coerce_credential
from ..features import EndpointSupport, ProviderManifest
from ..models import ModelInfo
@@ -162,6 +163,7 @@ class HttpResponse:
headers: list[tuple[str, str]]
body: bytes
http_version: str = "HTTP/1.1"
+ provider: str | None = None # who answered; names the ProviderError json() raises
def header(self, name: str) -> str | None:
lname = name.lower()
@@ -178,7 +180,34 @@ class HttpResponse:
return self.body.decode("utf-8", errors="replace")
def json(self):
- return json.loads(self.body)
+ """The body as JSON. A body that is not JSON — a gateway's HTML
+ error page behind a 200, a truncated reply, a captive portal — is
+ a provider reply that cannot become a Response without inventing
+ one, so it raises :class:`ProviderError` (ErrorCode ``provider``)
+ carrying the status and the first bytes of what arrived. A raw
+ ``JSONDecodeError`` would escape every ``except LM15Error`` and
+ say nothing about who sent what."""
+ try:
+ return json.loads(self.body)
+ except (ValueError, UnicodeDecodeError) as exc:
+ raise ProviderError(
+ _non_json_message(self, exc),
+ provider=self.provider,
+ status=self.status,
+ request_id=self.header("x-request-id") or self.header("request-id"),
+ ) from exc
+
+
+def _non_json_message(response: "HttpResponse", exc: Exception) -> str:
+ content_type = response.header("content-type") or "no content-type"
+ excerpt = response.body[:200].decode("utf-8", errors="replace")
+ if len(response.body) > 200:
+ excerpt += "…"
+ return (
+ f"the reply (HTTP {response.status}, {content_type}, {len(response.body)} bytes) is not JSON: "
+ f"{exc}. Body starts: {excerpt!r}. A gateway or proxy in front of the provider is the "
+ "usual cause; the request may or may not have been served."
+ )
class ProviderDialect(Protocol):
@@ -207,6 +236,10 @@ _SURFACE_WORD = {
}
+def _client_side_stop(adaptations: "tuple[Adaptation, ...]") -> bool:
+ return any(a.field == "config.stop" and a.action == "client_side" for a in adaptations)
+
+
class BaseProviderLM:
"""Shared synchronous provider LM implementation.
@@ -224,6 +257,9 @@ class BaseProviderLM:
manifest: ClassVar[ProviderManifest] = ProviderManifest(provider="unknown")
access: ProviderManifest
account_id: str | None = None
+ # MAP-13 policy: "note" (adapt and record), "silent", "refuse". Each
+ # dialect declares it as a constructor field; this is the fallback.
+ adaptations: AdaptationPolicy = "note"
_credential_source: str = "explicit"
# Cloud-host state (AUTH-10): the resolved host settings and the clock
# every time-dependent byte (SigV4 date, JWT iat/exp) is read from. The
@@ -335,7 +371,10 @@ class BaseProviderLM:
from ..access import auth_header
- credential = resolve_credential_value(self.api_key) if self.api_key is not None else None
+ # plan() builds and discards: no credential provider is invoked, no
+ # header is signed — the record of adaptations does not depend on it.
+ planning = is_planning()
+ credential = resolve_credential_value(self.api_key) if self.api_key is not None and not planning else None
hdrs = dict(headers.items()) if isinstance(headers, dict) else dict(headers or [])
if credential is not None and not isinstance(credential, AwsCredentials):
pair = auth_header(self.access, credential, api_key_header=self._api_key_header)
@@ -391,23 +430,80 @@ class BaseProviderLM:
provider=self.provider,
)
+ # ─── MAP-13: build with adaptations, plan, client-side steps ───
+
+ def _build(self, request: Request, stream: bool, *, policy: "AdaptationPolicy | None" = None, planning: bool = False) -> "tuple[TransportRequest, tuple[Adaptation, ...]]":
+ """``build_request`` inside an adaptation scope: the wire request and
+ the record of what differs from what was asked. The one place a
+ scope is opened; builders record through ``lm15.adaptation.adapt``.
+ ``policy`` overrides the adapter's own (the async mirror passes its);
+ ``planning`` skips credentials and signing (the bytes are discarded)."""
+ with collecting(check_policy(policy if policy is not None else self.adaptations), provider=self.provider, planning=planning) as scope:
+ req = self.build_request(request, stream=stream)
+ return req, tuple(scope.records)
+
+ def plan(self, request: Request, *, policy: "AdaptationPolicy | None" = None) -> "tuple[Adaptation, ...]":
+ """What a call with this request WOULD adapt, with no network and no
+ credential invoked (like ``resolve()``, offline). Raises what the
+ call would raise (a refusal under any policy, or every deviation
+ under ``adaptations="refuse"``). Returns the full record under
+ every policy, "silent" included: a preview that hid what it saw
+ would be no preview."""
+ return self._build(request, stream=False, policy=policy, planning=True)[1]
+
+ def _visible(self, adaptations: "tuple[Adaptation, ...]", *, policy: "AdaptationPolicy | None" = None) -> "tuple[Adaptation, ...]":
+ """What the response carries: everything under "note" (and "refuse",
+ which only ever holds satisfied/defaulted), nothing under "silent".
+ Behaviour is decided from the full record, never from this."""
+ return () if (policy if policy is not None else self.adaptations) == "silent" else adaptations
+
+ def _finish_response(self, request: Request, response: Response, adaptations: "tuple[Adaptation, ...]", *, policy: "AdaptationPolicy | None" = None) -> Response:
+ """Stamp the visible record on the response and apply client-side steps."""
+ from ..result import apply_client_side_stop
+
+ if _client_side_stop(adaptations):
+ response = apply_client_side_stop(response, request.config.stop)
+ visible = self._visible(adaptations, policy=policy)
+ if visible and not response.adaptations:
+ from dataclasses import replace
+
+ response = replace(response, adaptations=visible)
+ return response
+
def complete(self, request: Request) -> Response:
- req = self.build_request(request, stream=False)
+ req, adaptations = self._build(request, stream=False)
+ if _client_side_stop(adaptations):
+ # MAP-13 (decision 2026-09-14): a stop sequence the wire cannot
+ # take is honoured by streaming under the hood and closing the
+ # connection at the cut. Whether the provider then stops
+ # generating (and billing) on a closed connection is its own
+ # behaviour, not a promise made here. The price is the usage
+ # report, which only the final frame carries; it is "not
+ # reported", never estimated. A stream that never hits the
+ # sequence completes normally, usage included.
+ from ..result import materialize_response
+
+ return materialize_response(self.stream(request), request)
resp = self._send(req)
if resp.status >= 400:
raise self._http_error(resp)
- return self.parse_response(request, resp)
+ return self._finish_response(request, self.parse_response(request, resp), adaptations)
def stream(self, request: Request) -> Iterator[StreamEvent]:
# MAP-3 (docs/mapping-rules.md): adapters may emit one end event per
# provider terminal frame; the coalescer merges them so the public
# stream yields exactly one final StreamEndEvent.
- from ..result import coalesce_stream
+ from ..result import coalesce_stream, truncate_stream_at_stop
- return coalesce_stream(self._stream_raw(request), model=request.model)
+ req, adaptations = self._build(request, stream=True)
+ events = coalesce_stream(self._stream_raw(request, req), model=request.model, adaptations=self._visible(adaptations))
+ if _client_side_stop(adaptations):
+ events = truncate_stream_at_stop(events, request.config.stop)
+ return events
- def _stream_raw(self, request: Request) -> Iterator[StreamEvent]:
- req = self.build_request(request, stream=True)
+ def _stream_raw(self, request: Request, req: "TransportRequest | None" = None) -> Iterator[StreamEvent]:
+ if req is None:
+ req = self.build_request(request, stream=True)
self._ensure_transport_open()
try:
with self.transport.stream(req) as resp:
@@ -437,6 +533,7 @@ class BaseProviderLM:
headers=resp.headers,
body=body,
http_version=resp.http_version,
+ provider=self.provider,
)
except NetworkTransportError as exc:
raise LM15TransportError(str(exc)) from exc
@@ -502,7 +599,7 @@ class BaseProviderLM:
replaced before the next request.
"""
if isinstance(self.transport, StdlibTransport) and getattr(self.transport, "_closed", False):
- self.transport = default_transport()
+ self.transport = self.transport.copy()
def __enter__(self):
return self
@@ -737,13 +834,19 @@ class BaseProviderLM:
"""Submit to the provider's batch queue; returns the ticket snapshot."""
self._require("batches")
upload_body = None
- upload_req = self._batch_upload_request(request)
+ # MAP-13: the batch builders run under the adapter's policy so
+ # "refuse" refuses here too; a batch ticket has no adaptations field
+ # (provisional surface), so under "note" the record is not kept.
+ with collecting(check_policy(self.adaptations), provider=self.provider):
+ upload_req = self._batch_upload_request(request)
if upload_req is not None:
resp = self._send(upload_req)
if resp.status >= 400:
raise self._http_error(resp)
upload_body = resp.json()
- resp = self._send(self._batch_submit_request(request, upload_body))
+ with collecting(check_policy(self.adaptations), provider=self.provider):
+ submit_req = self._batch_submit_request(request, upload_body)
+ resp = self._send(submit_req)
if resp.status >= 400:
raise self._http_error(resp)
return self._batch_job_from_body(resp.text())
diff --git a/dspy/_vendor/lm15/providers/claude_code.py b/dspy/_vendor/lm15/providers/claude_code.py
index 500e02da6..23b55b3cf 100644
--- a/dspy/_vendor/lm15/providers/claude_code.py
+++ b/dspy/_vendor/lm15/providers/claude_code.py
@@ -18,6 +18,7 @@ from typing import ClassVar
from ..access import CLAUDE_CODE, DEFAULT_CLAUDE_CODE_SYSTEM_PROMPT, DEFAULT_CLAUDE_CODE_VERSION # noqa: F401
from ..features import ProviderManifest
from .anthropic import AnthropicLM
+from ..adaptation import AdaptationPolicy
from .base import Credential, SyncTransport, default_transport
@@ -35,6 +36,7 @@ class ClaudeCodeLM(AnthropicLM):
base_url: str = "https://api.anthropic.com/v1",
api_version: str = "2023-06-01",
claude_code_version: str = DEFAULT_CLAUDE_CODE_VERSION,
+ adaptations: "AdaptationPolicy" = "note",
) -> None:
self.claude_code_version = claude_code_version
policy = CLAUDE_CODE
@@ -47,6 +49,7 @@ class ClaudeCodeLM(AnthropicLM):
api_version=api_version,
access=policy,
credentials_path=credentials_path,
+ adaptations=adaptations,
)
@classmethod
diff --git a/dspy/_vendor/lm15/providers/common.py b/dspy/_vendor/lm15/providers/common.py
index c066e69da..0e1763b8e 100644
--- a/dspy/_vendor/lm15/providers/common.py
+++ b/dspy/_vendor/lm15/providers/common.py
@@ -74,7 +74,7 @@ def parts_to_text(parts: tuple[Part, ...], *, provider: str | None = None,
raise UnsupportedFeatureError(
f"{head}a {part.type} part cannot reach {where}, which takes text only; "
"no text rendering of a media part is made (MAP-10)",
- provider=provider,
+ provider=provider, feature=f"messages[*].parts[{part.type}]",
)
if isinstance(part, TextPart):
out.append(part.text)
@@ -137,7 +137,7 @@ def check_tool_result_media(provider: str, part: ToolResultPart, policy: str, *,
f"(compat tool_result_media={policy!r}, measured: lm15-contract/research/tool-result-content/). "
f"Carried natively by {_MEDIA_DOORS.get(p.type, 'no lm15 door yet')}; "
"or render the part to text yourself before building the tool result (MAP-10)",
- provider=provider,
+ provider=provider, feature=f"messages[*].tool_result[{part.id}].content[{p.type}]",
)
@@ -155,7 +155,22 @@ def extension_config(value: dict[str, Any] | None) -> dict[str, Any]:
def json_dumps(value: Any) -> bytes:
- return json.dumps(value, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
+ """Canonical request bytes. Text that is not valid Unicode (a lone
+ surrogate, U+D800..U+DFFF unpaired) has no UTF-8 form and cannot be
+ sent to ANY provider; it is refused here, before the wire, as the
+ input error it is — the alternative is an encode failure deep in the
+ transport that reads as a network fault."""
+ text = json.dumps(value, separators=(",", ":"), ensure_ascii=False)
+ try:
+ return text.encode("utf-8")
+ except UnicodeEncodeError as exc:
+ bad = text[exc.start:exc.end]
+ codepoints = " ".join(f"U+{ord(ch):04X}" for ch in bad)
+ raise ValueError(
+ f"request contains text that is not valid Unicode (lone surrogate {codepoints}), "
+ "which no provider can receive; repair the text first, e.g. "
+ "text.encode('utf-8', 'replace').decode('utf-8')"
+ ) from exc
def path_id(value: str, *, resource_name: bool = False) -> str:
@@ -294,6 +309,7 @@ def part_to_openai_input(part: Part, *, provider: str | None = None) -> dict[str
head = f"{provider}: " if provider else ""
raise UnsupportedFeatureError(
f"{head}a {part.type} part has no input block on the Responses wire (MAP-10)", provider=provider,
+ feature=f"messages[*].parts[{part.type}]",
)
diff --git a/dspy/_vendor/lm15/providers/gemini.py b/dspy/_vendor/lm15/providers/gemini.py
index cd249a50b..2a48b2531 100644
--- a/dspy/_vendor/lm15/providers/gemini.py
+++ b/dspy/_vendor/lm15/providers/gemini.py
@@ -7,9 +7,10 @@ import json
import os
import struct
import urllib.parse
-from dataclasses import dataclass, field
+from dataclasses import dataclass, field, replace
from typing import Any, Callable, ClassVar, Iterator, Mapping
+from ..adaptation import AdaptationPolicy, adapt, check_policy
from ..errors import (
AuthError,
BillingError,
@@ -403,12 +404,15 @@ class GeminiLM(BaseProviderLM):
settings: "Mapping[str, str] | None" = None
clock: "Callable[[], datetime] | None" = field(default=None, repr=False)
+ # MAP-13 policy: "note" (adapt and record), "silent", or "refuse".
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="gemini", init=False)
account_id: str | None = field(default=None, init=False, repr=False)
manifest: ClassVar[ProviderManifest] = GEMINI_API
_api_key_header: ClassVar[str] = "x-goog-api-key"
def __post_init__(self) -> None:
+ check_policy(self.adaptations)
self._bind_access(self.access, credentials_path=self.credentials_path, default_base_url=_DEFAULT_BASE_URL, settings=self.settings)
_error_status_map: ClassVar[dict[str, type[ProviderError]]] = {
@@ -574,14 +578,14 @@ class GeminiLM(BaseProviderLM):
raise UnsupportedFeatureError(
f"{self.provider}: a {p.type} part in tool_result {part.id!r} cannot reach a functionResponse — "
"multimodal function responses take images (png/jpeg/webp) and documents (pdf, text/plain) only (MAP-10)",
- provider=self.provider,
+ provider=self.provider, feature=f"messages[*].tool_result[{part.id}].content[{p.type}]",
)
name = part.name or (names or {}).get(part.id)
if not name:
raise UnsupportedFeatureError(
f"{self.provider}: tool_result {part.id!r} needs a function name on the Gemini wire and no preceding "
"assistant tool_call with that id is in the transcript; set ToolResultPart.name (MAP-10 rule 6)",
- provider=self.provider,
+ provider=self.provider, feature=f"messages[*].tool_result[{part.id}].name",
)
text = parts_to_text(text_parts, provider=self.provider, where="functionResponse.response")
if part.is_error:
@@ -654,28 +658,27 @@ class GeminiLM(BaseProviderLM):
if tc is None:
return None
if tc.parallel is False:
- # MAP-8 rule 2 (live 2026-09-02): no wire knob; two calls came back
- # on 2.5 and 3.7 with the preference set. The outcome is not
- # observable from usage, so the MAP-6 fallback exception does not
- # apply — raise.
- raise UnsupportedFeatureError(
- "gemini: tool_choice.parallel=False is not supported — GenerateContent has no "
- "parallel-tool-calls knob and returns several calls regardless (OpenAI and "
- "Anthropic carry it)",
- provider=self.provider,
- )
+ # MAP-13: no wire knob (live 2026-09-02: two calls came back on
+ # 2.5 and 3.7 with the preference set). A preference; agent
+ # loops iterate tool-call parts as a list anyway. Dropped and
+ # recorded — the record is the visibility MAP-8 rule 2 wanted.
+ adapt("config.tool_choice.parallel", "dropped",
+ "GenerateContent has no parallel-tool-calls knob and may return several calls "
+ "(OpenAI and Anthropic carry it)",
+ asked=False, provider=self.provider)
mode = {"none": "NONE", "required": "ANY", "auto": "AUTO"}[tc.mode]
cfg: dict[str, Any] = {"mode": mode}
if tc.allowed:
by_name = {t.name: t for t in request.tools}
builtins = [name for name in tc.allowed if isinstance(by_name.get(name), BuiltinTool)]
if builtins:
+ # MAP-13 rule 4(b): the program depends on the forced tool running.
raise UnsupportedFeatureError(
f"gemini: cannot force builtin tools {builtins} — "
"functionCallingConfig addresses function declarations only; "
"googleSearch/codeExecution have no tool_choice form "
"(OpenAI Responses and Anthropic carry builtin forcing)",
- provider=self.provider,
+ provider=self.provider, feature="config.tool_choice.allowed",
)
cfg["allowedFunctionNames"] = list(tc.allowed)
if tc.mode == "auto":
@@ -698,17 +701,15 @@ class GeminiLM(BaseProviderLM):
suffix_from = 0
if cache_cfg is not None and cache_cfg.mode != "off":
if cache_cfg.key is not None:
- raise UnsupportedFeatureError(
- "gemini: cache.key is not supported — GenerateContent has no cache "
- "affinity key; use cache.resource with a stored cache (lm.cache(prefix))",
- provider=self.provider,
- )
+ adapt("config.cache.key", "dropped",
+ "GenerateContent has no cache affinity key; implicit caching applies, and a "
+ "stored cache (lm.cache(prefix), cache.resource) is the explicit tier",
+ asked=cache_cfg.key, provider=self.provider)
if cache_cfg.retention is not None and cache_cfg.retention != "short":
- raise UnsupportedFeatureError(
- "gemini: cache.retention is not supported in-request — lifetime belongs to "
- "the stored cache (cache_create(..., ttl_seconds=...) / cache_update)",
- provider=self.provider,
- )
+ adapt("config.cache.retention", "dropped",
+ "GenerateContent takes no lifetime in-request; it belongs to the stored cache "
+ "(cache_create(..., ttl_seconds=...) / cache_update)",
+ asked=cache_cfg.retention, provider=self.provider)
if cache_cfg.resource is not None:
resource = cache_cfg.resource
if cache_cfg.prefix_until_index is not None:
@@ -734,6 +735,12 @@ class GeminiLM(BaseProviderLM):
generation_config["topP"] = _gemini_number(request.config.top_p)
if request.config.top_k is not None:
generation_config["topK"] = request.config.top_k
+ if request.config.seed is not None:
+ generation_config["seed"] = request.config.seed
+ if request.config.frequency_penalty is not None:
+ generation_config["frequencyPenalty"] = _gemini_number(request.config.frequency_penalty)
+ if request.config.presence_penalty is not None:
+ generation_config["presencePenalty"] = _gemini_number(request.config.presence_penalty)
if request.config.stop:
generation_config["stopSequences"] = list(request.config.stop)
if request.config.logprobs is not None:
@@ -749,38 +756,38 @@ class GeminiLM(BaseProviderLM):
if request.config.reasoning is not None:
reasoning = request.config.reasoning
level_class = gemini_level_class(request.model)
+ if reasoning.is_off and level_class:
+ # MAP-13 (decision 2026-09-14 §4.2): the Gemini 3 class has no
+ # honoured off switch (thinkingBudget 0 accepted, 58 tokens
+ # still spent on 3.7 Flash, live 2026-09-02); the closest to
+ # "none" is the lowest level, and the spend shows in
+ # usage.reasoning_tokens.
+ adapt("config.reasoning.effort", "substituted",
+ f"{request.model} cannot disable thinking (the Gemini 3 class honours no off switch); "
+ "the lowest level was sent and the thinking spend is visible in usage",
+ asked="off", applied="minimal", provider=self.provider)
+ reasoning = replace(reasoning, effort="minimal")
if reasoning.is_off:
- if level_class:
- # MAP-7 rule 4 / MAP-5: Gemini 3.x cannot fully disable
- # thinking; 3.7 Flash accepts thinkingBudget 0 and still
- # spent 58 tokens (live 2026-09-02) — a silent paid no-op.
- raise UnsupportedFeatureError(
- f"gemini: reasoning cannot be disabled on {request.model} — the Gemini 3 "
- "class has no full off switch (thinkingBudget 0 is accepted but not honoured); "
- "use effort='low' or a 2.5 model",
- provider=self.provider,
- )
generation_config["thinkingConfig"] = {"thinkingBudget": 0}
else:
if reasoning.summary in ("concise", "detailed"):
- raise UnsupportedFeatureError(
- f"gemini: reasoning.summary={reasoning.summary!r} is an OpenAI detail level; "
- "GenerateContent has includeThoughts only (use 'auto')",
- provider=self.provider,
- )
+ adapt("config.reasoning.summary", "substituted",
+ "GenerateContent has includeThoughts only, no detail levels; 'auto' shows the thoughts",
+ asked=reasoning.summary, applied="auto", provider=self.provider)
+ reasoning = replace(reasoning, summary="auto")
thinking: dict[str, Any] = {}
if reasoning.summary is not None:
thinking["includeThoughts"] = True # MAP-7 rule 7: only when asked
if reasoning.thinking_budget is not None:
thinking["thinkingBudget"] = reasoning.thinking_budget # 3.x: accepted, docs warn
elif level_class:
- if reasoning.effort in ("xhigh", "max"):
- raise UnsupportedFeatureError(
- f"gemini: reasoning.effort={reasoning.effort!r} has no thinkingLevel on the "
- "Gemini 3 class (minimal|low|medium|high); 'high' is the ceiling",
- provider=self.provider,
- )
- thinking["thinkingLevel"] = reasoning.effort
+ effort = reasoning.effort
+ if effort in ("xhigh", "max"):
+ adapt("config.reasoning.effort", "clamped",
+ "the Gemini 3 class has thinkingLevel minimal|low|medium|high; 'high' is the ceiling",
+ asked=effort, applied="high", provider=self.provider)
+ effort = "high"
+ thinking["thinkingLevel"] = effort
else:
thinking["thinkingBudget"] = EFFORT_THINKING_BUDGETS[reasoning.effort]
generation_config["thinkingConfig"] = thinking
@@ -825,11 +832,12 @@ class GeminiLM(BaseProviderLM):
if request.config.service_tier is not None:
payload["serviceTier"] = request.config.service_tier
if request.config.user_id is not None:
- raise UnsupportedFeatureError(
- "gemini: config.user_id is not supported — GenerateContent has no "
- "end-user attribution field (OpenAI and Anthropic carry it)",
- provider=self.provider,
- )
+ # MAP-13 (decision 2026-09-14 §4.5): attribution has no field
+ # here and nothing in the program depends on it at run time; a
+ # compliance policy sets adaptations="refuse".
+ adapt("config.user_id", "dropped",
+ "GenerateContent has no end-user attribution field (OpenAI and Anthropic carry it)",
+ asked=request.config.user_id, provider=self.provider)
if extensions:
passthrough = {k: v for k, v in extensions.items() if k not in {"prompt_caching", "output"}}
@@ -848,7 +856,6 @@ class GeminiLM(BaseProviderLM):
headers=self._auth_headers({"Content-Type": "application/json"}),
params=params,
payload=self._payload(request),
- read_timeout=120.0 if stream else 60.0,
)
# ─── Response parsing ───────────────────────────────────────────
diff --git a/dspy/_vendor/lm15/providers/openai.py b/dspy/_vendor/lm15/providers/openai.py
index 80ba427be..954493591 100644
--- a/dspy/_vendor/lm15/providers/openai.py
+++ b/dspy/_vendor/lm15/providers/openai.py
@@ -10,6 +10,7 @@ import urllib.parse
from dataclasses import dataclass, field
from typing import Any, Callable, ClassVar, Iterator, Mapping
+from ..adaptation import AdaptationPolicy, adapt, check_policy
from ..errors import (
AuthError,
BillingError,
@@ -27,7 +28,7 @@ from ..errors import (
)
from ..access import OPENAI_API, auth_header
from ..auth import extract_chatgpt_account_id
-from ..compat import OPENAI_RESPONSES_PRESET_BASE_URLS, OpenAIResponsesCompat, _preset_key
+from ..compat import OPENAI_RESPONSES_PRESET_BASE_URLS, OpenAIResponsesCompat, preset_base_url
from ..features import ProviderManifest
from ..result import materialize_response
from ..live import WebSocketLiveSession, require_websocket_sync_connect
@@ -197,15 +198,39 @@ def _cache_breakpoint_index(request: Request, cache_control: str) -> int | None:
return None
if cache_control != "openai":
return None
- return min(cache_cfg.prefix_until_index, len(request.messages) - 1)
+ asked = min(cache_cfg.prefix_until_index, len(request.messages) - 1)
+ # MAP-13: the wire carries the mark on a text block of a user/developer
+ # message only. A mark asked for elsewhere walks back to the nearest
+ # eligible message ("cache up to here" — the nearest boundary before
+ # "here" is the obvious answer); with none, the mark is dropped and
+ # implicit caching still applies.
+ for index in range(asked, -1, -1):
+ msg = request.messages[index]
+ if msg.role in ("assistant", "tool") or not msg.parts or not isinstance(msg.parts[-1], TextPart):
+ continue
+ if index != asked:
+ adapt("config.cache.prefix_until_index", "substituted",
+ f"message {asked} is a {request.messages[asked].role} message or does not end with "
+ "text; the Responses wire marks text blocks of user/developer messages only, so the "
+ "mark moved to the nearest eligible message before it",
+ asked=asked, applied=index)
+ return index
+ adapt("config.cache.prefix_until_index", "dropped",
+ f"no user/developer message ending with text at or before message {asked}; the Responses "
+ "wire marks text blocks only (implicit caching still applies)",
+ asked=asked)
+ return None
-def _has_explicit_breakpoint(request: Request, cache_control: str) -> bool:
+def _has_explicit_breakpoint(request: Request, cache_control: str, *, breakpoint_index: int | None = None) -> bool:
"""True when this request places a prompt_cache_breakpoint (prefix="stable"
- or prefix_until_index) — the cases where explicit mode belongs with it."""
+ or prefix_until_index) — the cases where explicit mode belongs with it.
+ ``breakpoint_index`` is the build's one computation of the index (it
+ records an adaptation when the mark moves; computing it twice recorded
+ it twice — greptile on dspy#10409)."""
# prefix="stable" places its mark on the system prompt; with no system
# there is no mark, and explicit mode with no mark would cache nothing.
- return _cache_breakpoint_index(request, cache_control) is not None or (
+ return breakpoint_index is not None or (
_cache_stable_prefix(request, cache_control) and bool(request.system)
)
@@ -219,7 +244,7 @@ def _cache_stable_prefix(request: Request, cache_control: str) -> bool:
)
-def _cache_common_payload(request: Request, payload: dict, cache_control: str, provider: str) -> None:
+def _cache_common_payload(request: Request, payload: dict, cache_control: str, provider: str, *, breakpoint_index: int | None = None) -> None:
"""Shared MAP-6 fields for both OpenAI dialects: off switch, key, retention.
``cache_control="openai_implicit"`` forwards the key and the retention
@@ -241,7 +266,7 @@ def _cache_common_payload(request: Request, payload: dict, cache_control: str, p
raise UnsupportedFeatureError(
f"{provider}: cache.resource is not supported — this provider has no stored-cache "
"tier; it caches every prompt prefix automatically",
- provider=provider,
+ provider=provider, feature="config.cache.resource",
)
return
if cache_cfg.mode == "off":
@@ -260,7 +285,7 @@ def _cache_common_payload(request: Request, payload: dict, cache_control: str, p
# prompt_cache_retention: "24h"; every pinned 5.6 body already
# echoes 24h as its default. Sending it is honest and harmless.
payload["prompt_cache_retention"] = "24h"
- if openai_model_has_cache_options(request.model) and _has_explicit_breakpoint(request, cache_control):
+ if openai_model_has_cache_options(request.model) and _has_explicit_breakpoint(request, cache_control, breakpoint_index=breakpoint_index):
# A placed breakpoint means "cache up to here". Without explicit
# mode the 5.6 class also writes the volatile suffix at 1.25x on
# every warm call (pinned: openai.prompt_cache_breakpoint wrote 18
@@ -271,7 +296,7 @@ def _cache_common_payload(request: Request, payload: dict, cache_control: str, p
raise UnsupportedFeatureError(
f"{provider}: cache.resource is not supported — this provider has no stored-cache "
"tier; it caches by marks on blocks (prefix / prefix_until_index) and automatically",
- provider=provider,
+ provider=provider, feature="config.cache.resource",
)
@@ -452,17 +477,31 @@ class OpenAILM(BaseProviderLM):
clock: "Callable[[], datetime] | None" = field(default=None, repr=False, kw_only=True)
account_id: str | None = None
+ # MAP-13 policy: "note" (adapt and record), "silent", or "refuse".
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="openai", init=False)
manifest: ClassVar[ProviderManifest] = OPENAI_API
_compat_base: OpenAIResponsesCompat | None = field(init=False, repr=False, default=None)
def __post_init__(self) -> None:
+ check_policy(self.adaptations)
+ if self.profile is not None:
+ import warnings
+
+ warnings.warn(
+ "OpenAILM(profile=...) is deprecated and will be removed in lm15 1.0.0: pass compat= "
+ "(and base_url=) instead; per-request policy goes in Config.extensions['openai_responses_compat']",
+ DeprecationWarning,
+ stacklevel=3,
+ )
self._bind_access(self.access, credentials_path=self.credentials_path, default_base_url=_DEFAULT_BASE_URL, settings=self.settings)
compat = self.compat if self.compat is not None else self._registry_compat()
if isinstance(compat, str):
self._compat_base = OpenAIResponsesCompat.preset(compat)
if self.base_url == _DEFAULT_BASE_URL:
- self.base_url = OPENAI_RESPONSES_PRESET_BASE_URLS.get(_preset_key(compat), _DEFAULT_BASE_URL)
+ self.base_url = preset_base_url(
+ OPENAI_RESPONSES_PRESET_BASE_URLS, compat, dialect="Responses", default_preset="openai"
+ )
elif isinstance(compat, OpenAIResponsesCompat):
self._compat_base = compat
elif compat is not None:
@@ -533,6 +572,15 @@ class OpenAILM(BaseProviderLM):
profile: ProviderProfile,
transport: SyncTransport | None = None,
) -> "OpenAILM":
+ """DEPRECATED (1.0.0rc2; removed in 1.0.0): ``OpenAILM(api_key=..., compat=..., base_url=...)``."""
+ import warnings
+
+ warnings.warn(
+ "OpenAILM.from_profile is deprecated and will be removed in lm15 1.0.0: "
+ "use OpenAILM(api_key=..., compat=<preset or OpenAIResponsesCompat>, base_url=...)",
+ DeprecationWarning,
+ stacklevel=2,
+ )
endpoint = profile.endpoint("inference")
base_url = endpoint.base_url if endpoint and endpoint.base_url else "https://api.openai.com/v1"
return cls(
@@ -813,12 +861,13 @@ class OpenAILM(BaseProviderLM):
def _payload(self, request: Request, stream: bool) -> dict[str, Any]:
compat = self._compat(request)
+ breakpoint_index = _cache_breakpoint_index(request, compat.cache_control) # once: it may record
payload: dict[str, Any] = {
"model": request.model,
"input": self._build_input(
request.messages,
compat,
- breakpoint_index=_cache_breakpoint_index(request, compat.cache_control),
+ breakpoint_index=breakpoint_index,
),
"stream": stream,
}
@@ -843,16 +892,26 @@ class OpenAILM(BaseProviderLM):
if request.config.top_p is not None:
payload["top_p"] = request.config.top_p
if request.config.stop:
- raise UnsupportedFeatureError(
- f"{self.provider}: config.stop has no field on the Responses wire (the Chat Completions "
- "dialect carries `stop`); a silent omission would run the model past the sequence",
- provider=self.provider,
- )
+ # MAP-13 client_side: the Responses wire has no stop field; the
+ # text is cut at the first sequence after the wire (complete)
+ # or as it streams (the source is closed at the cut).
+ adapt("config.stop", "client_side",
+ "the Responses wire has no stop field; the reply is streamed and the connection "
+ "closed at the first stop sequence (whether the provider then stops generating, "
+ "and billing, is its own behaviour); the usage report rides only the final frame, "
+ "so it is not reported when the cut happens (never estimated)",
+ asked=list(request.config.stop), applied=list(request.config.stop), provider=self.provider)
if request.config.top_k is not None:
- raise UnsupportedFeatureError(
- f"{self.provider}: config.top_k has no field on the Responses wire (Anthropic and Gemini carry it)",
- provider=self.provider,
- )
+ adapt("config.top_k", "dropped",
+ "the Responses wire has no top_k (Anthropic and Gemini carry it)",
+ asked=request.config.top_k, provider=self.provider)
+ for name in ("seed", "frequency_penalty", "presence_penalty"):
+ if getattr(request.config, name) is not None:
+ # The Responses API dropped these from the Chat Completions
+ # wire (no field in the reference); the chat dialect carries them.
+ adapt(f"config.{name}", "dropped",
+ f"the Responses wire has no {name} field (the Chat Completions dialect carries it)",
+ asked=getattr(request.config, name), provider=self.provider)
if request.config.logprobs is not None:
# Verified live 2026-09-01: include triggers per-token logprobs;
# top_logprobs (0–20) controls the alternatives count.
@@ -889,19 +948,17 @@ class OpenAILM(BaseProviderLM):
# 2026-09-02: gpt-5.6-sol rejects minimal, gpt-5.4-mini
# rejects max). No budget exists on this wire.
if reasoning.thinking_budget is not None:
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.thinking_budget is not supported — this wire "
- "has no thinking token budget; use effort (Anthropic's manual class and "
- "Gemini take a budget)",
- provider=self.provider,
- )
+ # MAP-13: effort carries the intent (MAP-7 rule 5); no
+ # budget field exists on this wire.
+ adapt("config.reasoning.thinking_budget", "dropped",
+ "this wire has no thinking token budget; effort carries the intent "
+ "(Anthropic's manual class and Gemini take a budget)",
+ asked=reasoning.thinking_budget, provider=self.provider)
effort = reasoning.effort
if reasoning.summary in ("concise", "detailed") and compat.reasoning_format != "responses_reasoning":
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.summary={reasoning.summary!r} is an OpenAI Responses "
- "detail level; this wire has no summary levels (use 'auto')",
- provider=self.provider,
- )
+ adapt("config.reasoning.summary", "substituted",
+ "this wire has no summary detail levels; 'auto' is what it shows",
+ asked=reasoning.summary, applied="auto", provider=self.provider)
if compat.reasoning_format == "responses_reasoning":
reasoning_payload: dict[str, Any] = {"effort": effort}
if reasoning.summary is not None:
@@ -945,7 +1002,7 @@ class OpenAILM(BaseProviderLM):
payload["chat_template_kwargs"] = {"enable_thinking": False}
# Prompt caching (MAP-6): off switch, key, retention, resource.
- _cache_common_payload(request, payload, compat.cache_control, self.provider)
+ _cache_common_payload(request, payload, compat.cache_control, self.provider, breakpoint_index=breakpoint_index)
if compat.routing is not None:
payload["provider"] = compat.routing
@@ -994,7 +1051,6 @@ class OpenAILM(BaseProviderLM):
model=request.model,
headers=self._headers(),
payload=self._payload(request, stream=stream),
- read_timeout=120.0 if stream else 60.0,
)
# ─── Response parsing ───────────────────────────────────────────
diff --git a/dspy/_vendor/lm15/providers/openai_chat.py b/dspy/_vendor/lm15/providers/openai_chat.py
index 10f2b5db0..23e68c242 100644
--- a/dspy/_vendor/lm15/providers/openai_chat.py
+++ b/dspy/_vendor/lm15/providers/openai_chat.py
@@ -15,15 +15,17 @@ from datetime import datetime
import json
import mimetypes
import os
-from dataclasses import dataclass, field
+from dataclasses import dataclass, field, replace
from typing import Any, Callable, ClassVar, Iterator, Mapping
from ..compat import (
OPENAI_CHAT_PRESET_BASE_URLS,
+ preset_base_url,
OpenAIChatCompat,
ResolvedOpenAIChatCompat,
resolve_openai_chat_compat,
)
+from ..adaptation import AdaptationPolicy, adapt, check_policy, nearest_effort
from ..errors import ProviderError, ServerError, UnsupportedFeatureError
from ..access import OPENAI_CHAT_API
from ..features import ProviderManifest
@@ -139,7 +141,7 @@ def _chat_content_parts(msg: Message, *, force_array: bool = False, provider: st
raise UnsupportedFeatureError(
f"{provider}: a {part.type} part in a {msg.role} message has no slot on the Chat Completions wire "
"(text and image_url only); the OpenAI Responses, Anthropic and Gemini dialects carry it (MAP-10)",
- provider=provider,
+ provider=provider, feature=f"messages[*].parts[{part.type}]",
)
else:
out.append({"type": "text", "text": parts_to_text((part,), provider=provider)})
@@ -201,20 +203,18 @@ def _response_format_to_chat(format_config: dict[str, Any]) -> dict[str, Any]:
# OpenAI documents that no canonical field expresses and that the builder
# re-emits verbatim (payload.update(extensions)), so they round-trip.
_INGEST_EXTENSIONS_KEYS: frozenset[str] = frozenset({
- "seed", "logit_bias", "presence_penalty", "frequency_penalty", "metadata",
- "verbosity", "moderation", "provider",
+ "logit_bias", "metadata", "verbosity", "moderation", "provider",
+ # prediction: a latency hint (predicted outputs); harmless verbatim on
+ # OpenAI, dropped-with-note elsewhere (decision 2026-09-14 §4.9).
+ "prediction",
})
# Top-level keys refused with the reason a canonical Request cannot carry them.
_INGEST_REFUSED_KEYS: dict[str, str] = {
"n": "lm15 reads one choice per response; n>1 would silently lose choices — fan out in the caller",
- "functions": "the deprecated function-calling shape; declare tools with {type: function, function: {...}}",
- "function_call": "the deprecated function-calling shape; use tool_choice",
"audio": "audio output parameters have no canonical slot on the chat surface",
"modalities": "output modality selection has no canonical slot on the chat surface",
- "prediction": "predicted-output content has no canonical slot",
"web_search_options": "a server-executed search the chat dialect cannot map to parts (MAP-1); the Responses dialect carries web_search as a BuiltinTool",
- "top_k": "the Chat Completions wire has no top_k (the builder raises on Config.top_k for the same reason); servers that take it do so through extensions",
}
# Call-mode keys: they say HOW the request is sent, not WHAT is asked. A
@@ -227,7 +227,11 @@ _INGEST_CALL_MODE_KEYS: frozenset[str] = frozenset({"stream", "stream_options"})
# the tables above, and an unlisted key is refused).
_INGEST_CONFIG_KEYS: frozenset[str] = frozenset({
"model", "messages", "tools", "tool_choice", "parallel_tool_calls",
- "max_completion_tokens", "max_tokens", "temperature", "top_p", "stop",
+ # functions / function_call: the deprecated function-calling shape,
+ # translated to tools / tool_choice (MAP-13: a pure spelling change).
+ "functions", "function_call",
+ "max_completion_tokens", "max_tokens", "temperature", "top_p", "top_k", "stop",
+ "seed", "frequency_penalty", "presence_penalty",
"logprobs", "top_logprobs", "response_format", "service_tier", "store",
"user", "safety_identifier", "user_id",
"reasoning_effort", "reasoning", "thinking", "enable_thinking", "chat_template_kwargs", "reasoning_format",
@@ -591,6 +595,15 @@ def _ingest_tools(provider: str, raw: Any, compat: ResolvedOpenAIChatCompat) ->
return tools
+def _tool_choice_from_function_call(raw: Any) -> Any:
+ """The deprecated ``function_call`` spelling → the ``tool_choice`` shape."""
+ if raw in ("none", "auto"):
+ return raw
+ if isinstance(raw, Mapping) and "name" in raw:
+ return {"type": "function", "function": {"name": raw["name"]}}
+ raise ValueError(f"function_call must be 'none', 'auto', or {{name}}; got {raw!r}")
+
+
def _ingest_tool_choice(provider: str, raw: Any, parallel: Any) -> ToolChoice | None:
"""Inverse of OpenAIChatLM._tool_choice_payload plus parallel_tool_calls."""
mode: str | None = None
@@ -815,7 +828,7 @@ def _ingest_config(
if len(set(map(repr, values.values()))) > 1:
raise ValueError(f"max_tokens and max_completion_tokens disagree: {values}")
kwargs["max_tokens"] = next(iter(values.values()))
- for key in ("temperature", "top_p", "service_tier", "store"):
+ for key in ("temperature", "top_p", "top_k", "seed", "frequency_penalty", "presence_penalty", "service_tier", "store"):
if key in body:
kwargs[key] = body[key]
if "stop" in body:
@@ -829,7 +842,12 @@ def _ingest_config(
raise ValueError("top_logprobs requires logprobs: true")
if "response_format" in body:
kwargs["response_format"] = _ingest_response_format(provider, body["response_format"])
- kwargs["tool_choice"] = _ingest_tool_choice(provider, body.get("tool_choice"), body.get("parallel_tool_calls"))
+ if "function_call" in body and "tool_choice" in body:
+ raise ValueError("function_call and tool_choice cannot both be given")
+ kwargs["tool_choice"] = _ingest_tool_choice(
+ provider, _tool_choice_from_function_call(body["function_call"]) if "function_call" in body else body.get("tool_choice"),
+ body.get("parallel_tool_calls"),
+ )
user_keys = [k for k in ("user", "safety_identifier", "user_id") if k in body]
if "user_id" in user_keys and compat.user_field != "user_id":
@@ -863,7 +881,14 @@ def _ingest_openai_chat(provider: str, body: Mapping[str, Any], compat: Resolved
if "messages" not in body:
raise ValueError("messages is required")
system, messages, system_breakpoint, breakpoint_index = _ingest_messages(provider, body["messages"], compat)
- tools = _ingest_tools(provider, body.get("tools"), compat)
+ if "functions" in body and "tools" in body:
+ raise ValueError("functions and tools cannot both be given")
+ raw_tools = body.get("tools")
+ if "functions" in body:
+ if not isinstance(body["functions"], list):
+ raise TypeError("functions must be an array")
+ raw_tools = [{"type": "function", "function": fn} for fn in body["functions"]]
+ tools = _ingest_tools(provider, raw_tools, compat)
config = _ingest_config(provider, body, compat, system_breakpoint=system_breakpoint, breakpoint_index=breakpoint_index)
return Request(model=model, messages=tuple(messages), system=system, tools=tuple(tools), config=config)
@@ -929,7 +954,7 @@ def _response_from_chat_body(
raise UnsupportedFeatureError(
f"{provider}: the body carries {len(choices)} choices; a canonical Response is one message — "
"name the choice to read (choice=i) and read each one, or send no n",
- provider=provider,
+ provider=provider, feature="n",
)
index = 0
else:
@@ -1075,6 +1100,8 @@ class OpenAIChatLM(BaseProviderLM):
settings: "Mapping[str, str] | None" = None
clock: "Callable[[], datetime] | None" = field(default=None, repr=False)
+ # MAP-13 policy: "note" (adapt and record), "silent", or "refuse".
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
provider: str = field(default="openai-chat", init=False)
account_id: str | None = field(default=None, init=False, repr=False)
manifest: ClassVar[ProviderManifest] = OPENAI_CHAT_API
@@ -1091,13 +1118,15 @@ class OpenAIChatLM(BaseProviderLM):
normalize_error = OpenAILM.normalize_error
def __post_init__(self) -> None:
+ check_policy(self.adaptations)
self._bind_access(self.access, credentials_path=self.credentials_path, default_base_url=_DEFAULT_BASE_URL, settings=self.settings)
compat = self.compat if self.compat is not None else self._registry_compat()
if isinstance(compat, str):
- preset_key = compat.lower().replace("-", "_").replace(" ", "_")
partial = OpenAIChatCompat.preset(compat)
if self.base_url == _DEFAULT_BASE_URL:
- self.base_url = OPENAI_CHAT_PRESET_BASE_URLS.get(preset_key, _DEFAULT_BASE_URL)
+ self.base_url = preset_base_url(
+ OPENAI_CHAT_PRESET_BASE_URLS, compat, dialect="Chat Completions", default_preset="openai"
+ )
elif isinstance(compat, OpenAIChatCompat):
partial = compat
else:
@@ -1137,7 +1166,7 @@ class OpenAIChatLM(BaseProviderLM):
# ─── Request serialization ──────────────────────────────────────
- def _build_messages(self, request: Request, compat: ResolvedOpenAIChatCompat) -> list[dict[str, Any]]:
+ def _build_messages(self, request: Request, compat: ResolvedOpenAIChatCompat, *, breakpoint_index: int | None = None) -> list[dict[str, Any]]:
messages: list[dict[str, Any]] = []
if request.system:
system_text = request.system if isinstance(request.system, str) else parts_to_text(request.system)
@@ -1150,7 +1179,6 @@ class OpenAIChatLM(BaseProviderLM):
else:
messages.append({"role": compat.instruction_role, "content": system_text})
- breakpoint_index = _cache_breakpoint_index(request, compat.cache_control)
for msg_index, msg in enumerate(request.messages):
if msg_index == breakpoint_index and msg.role in ("assistant", "tool"):
raise _breakpoint_unsupported(self.provider, msg_index, msg.role)
@@ -1236,7 +1264,7 @@ class OpenAIChatLM(BaseProviderLM):
raise UnsupportedFeatureError(
f"{self.provider}: builtin tool {tool.name!r} has no Groq wire "
f"mapping — supported: {sorted(_GROQ_BUILTIN_MAP)}",
- provider=self.provider,
+ provider=self.provider, feature=f"tools[{tool.name}]",
)
entry: dict[str, Any] = {"type": wire_type}
if tool.config:
@@ -1248,7 +1276,7 @@ class OpenAIChatLM(BaseProviderLM):
"unproven servers may silently ignore unknown tool types. Use "
"compat='groq' for Groq's server-executed tools, or the OpenAI "
"Responses / Anthropic / Gemini providers",
- provider=self.provider,
+ provider=self.provider, feature=f"tools[{tool.name}]",
)
def _tool_choice_payload(self, request: Request) -> Any:
@@ -1266,7 +1294,7 @@ class OpenAIChatLM(BaseProviderLM):
f"{self.provider}: cannot force builtin tools {builtins} — the "
"Chat Completions wire has no hosted-tool tool_choice form "
"(OpenAI Responses and Anthropic carry it)",
- provider=self.provider,
+ provider=self.provider, feature="config.tool_choice.allowed",
)
if len(entries) == 1 and tc.mode == "required":
return {"type": "function", "function": {"name": entries[0].name}}
@@ -1295,9 +1323,10 @@ class OpenAIChatLM(BaseProviderLM):
def _payload(self, request: Request, stream: bool) -> dict[str, Any]:
compat = self._compat_for(request.model)
+ breakpoint_index = _cache_breakpoint_index(request, compat.cache_control) # once: it may record
payload: dict[str, Any] = {
"model": request.model,
- "messages": self._build_messages(request, compat),
+ "messages": self._build_messages(request, compat, breakpoint_index=breakpoint_index),
}
if stream:
payload["stream"] = True
@@ -1310,12 +1339,15 @@ class OpenAIChatLM(BaseProviderLM):
if request.config.top_p is not None:
payload["top_p"] = request.config.top_p
if request.config.top_k is not None:
- # No wire slot on Chat Completions (port.md rule 4: a raise or an
- # extensions door, never omission).
- raise UnsupportedFeatureError(
- f"{self.provider}: config.top_k has no field on the Chat Completions wire; servers that accept "
- "top_k take it through extensions", provider=self.provider,
- )
+ # MAP-13: a sampling hint with no field on this wire; servers
+ # that take one do so through extensions.
+ adapt("config.top_k", "dropped",
+ "the Chat Completions wire has no top_k (Anthropic and Gemini carry it; servers "
+ "that accept it take it through extensions)",
+ asked=request.config.top_k, provider=self.provider)
+ for name in ("seed", "frequency_penalty", "presence_penalty"):
+ if getattr(request.config, name) is not None:
+ payload[name] = getattr(request.config, name)
if request.config.stop:
payload["stop"] = list(request.config.stop)
if request.config.logprobs is not None:
@@ -1344,17 +1376,31 @@ class OpenAIChatLM(BaseProviderLM):
if tool_choice is not None:
tc = request.config.tool_choice
if compat.forced_tool_choice == "reject" and (tc.mode != "auto" or tc.allowed):
- # MAP-8: the server documents tool_choice=auto only and ignores
- # every other form without an error (Z.AI, live 2026-09-03:
- # required → text answer, none → a tool call). A silent widen
- # is worse than an error; omit tool_choice or use "auto".
- raise UnsupportedFeatureError(
- f"{self.provider}: tool_choice mode={tc.mode!r}"
- + (f" allowed={list(tc.allowed)}" if tc.allowed else "")
- + " is silently ignored by this server (only 'auto' is honoured); "
- "omit tool_choice, or send only the tools you want callable",
- provider=self.provider,
- )
+ # The server documents tool_choice=auto only and ignores every
+ # other form without an error (Z.AI, live 2026-09-03: required
+ # → text answer, none → a tool call). MAP-13: "none" and an
+ # allowlist have a client-side form — send no tools / only
+ # those tools — and are recorded; "required" cannot be forced
+ # and the program depends on the call (rule 4b): refused.
+ if tc.mode == "required":
+ raise UnsupportedFeatureError(
+ f"{self.provider}: tool_choice mode='required' is silently ignored by this server "
+ "(only 'auto' is honoured) and a forced call cannot be reproduced client-side",
+ provider=self.provider, feature="config.tool_choice.mode",
+ )
+ if tc.mode == "none":
+ adapt("config.tool_choice.mode", "client_side",
+ "this server ignores tool_choice='none'; no tools were sent, which is the same outcome",
+ asked="none", applied="no tools sent", provider=self.provider)
+ payload.pop("tools", None)
+ else:
+ kept = [t for t in payload.get("tools", []) if t.get("function", {}).get("name") in tc.allowed]
+ adapt("config.tool_choice.allowed", "client_side",
+ "this server ignores tool_choice allowlists; only the allowed tools were sent, "
+ "which is what the allowlist means",
+ asked=list(tc.allowed), applied=[t["function"]["name"] for t in kept], provider=self.provider)
+ payload["tools"] = kept
+ tool_choice = "auto"
payload["tool_choice"] = tool_choice
if request.config.tool_choice and request.config.tool_choice.parallel is not None:
payload["parallel_tool_calls"] = request.config.tool_choice.parallel
@@ -1363,41 +1409,50 @@ class OpenAIChatLM(BaseProviderLM):
# The server accepts response_format.type=json_schema and
# ignores it (Z.AI, live 2026-09-03: HTTP 200, fenced JSON with
# keys the schema never named). json_object is honoured.
- raise UnsupportedFeatureError(
- f"{self.provider}: response_format type "
- f"{request.config.response_format['type']!r} is silently ignored by this "
- "server; use {'type': 'json_object'} and describe the shape in the prompt",
- provider=self.provider,
- )
- payload["response_format"] = _response_format_to_chat(request.config.response_format)
+ adapt("config.response_format", "dropped",
+ f"this server accepts response_format type {request.config.response_format['type']!r} "
+ "and does not apply it; use {'type': 'json_object'} and describe the shape in the prompt",
+ asked=request.config.response_format, provider=self.provider)
+ else:
+ payload["response_format"] = _response_format_to_chat(request.config.response_format)
if request.config.reasoning:
reasoning = request.config.reasoning
- if not reasoning.is_off:
+ if compat.thinking_format == "none":
+ # No reasoning dial on this server. MAP-13: the dial is
+ # dropped and recorded — the model may reason at its own
+ # default and the tokens show in usage. (The 2026-09-11
+ # refusal here rested on "ollama has no dial", which was a
+ # preset written without a receipt; Ollama maps
+ # reasoning_effort to `think` — THEORY.md §3.17.)
+ adapt("config.reasoning", "dropped",
+ "this server has no reasoning dial on its wire (compat thinking_format='none'); "
+ "the model reasons at its own default; pass the server's own knob through extensions",
+ asked={"effort": reasoning.effort}, provider=self.provider)
+ reasoning = None
+ if reasoning is not None and not reasoning.is_off:
# MAP-7: verbatim effort; no budget on this wire; summary
# levels are Responses-only; "auto" maps to the dialect's
# visibility knob where one exists (Groq include_reasoning).
if reasoning.thinking_budget is not None:
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.thinking_budget is not supported — the Chat "
- "Completions wire has no thinking token budget; use effort",
- provider=self.provider,
- )
+ adapt("config.reasoning.thinking_budget", "dropped",
+ "the Chat Completions wire has no thinking token budget; effort carries the intent",
+ asked=reasoning.thinking_budget, provider=self.provider)
if reasoning.summary in ("concise", "detailed"):
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.summary={reasoning.summary!r} is an OpenAI Responses "
- "detail level; the Chat Completions wire has none (use 'auto')",
- provider=self.provider,
- )
+ adapt("config.reasoning.summary", "substituted",
+ "the Chat Completions wire has no summary detail levels; 'auto' is what it shows",
+ asked=reasoning.summary, applied="auto", provider=self.provider)
+ reasoning = replace(reasoning, summary="auto")
effort = reasoning.effort
if compat.reasoning_efforts is not None and effort not in compat.reasoning_efforts:
- # MAP-7 rule 2: a word with no native level raises here
- # when the server would not refuse it (Moonshot kimi-k3
+ # MAP-13: clamp to the nearest declared level; the server
+ # would have accepted the word silently (Moonshot kimi-k3
# answered 200 to `medium` and to `bogus`, live 2026-09-03).
- raise UnsupportedFeatureError(
- f"{self.provider}: reasoning.effort={effort!r} has no level on this server "
- f"(it accepts {', '.join(compat.reasoning_efforts)}) and would be accepted silently",
- provider=self.provider,
- )
+ nearest = nearest_effort(effort, compat.reasoning_efforts)
+ adapt("config.reasoning.effort", "clamped",
+ f"this server has no {effort!r} level (it accepts "
+ f"{', '.join(compat.reasoning_efforts)}) and would have accepted the word silently",
+ asked=effort, applied=nearest, provider=self.provider)
+ effort = nearest
if compat.builtin_tools == "groq" and reasoning.summary == "auto":
# Groq's visibility knob (MAP-7 rule 7): "parsed" returns
# the trace as message.reasoning. Live 2026-09-02: Qwen
@@ -1431,7 +1486,7 @@ class OpenAIChatLM(BaseProviderLM):
"enable_thinking": True,
"preserve_thinking": True,
}
- else:
+ elif reasoning is not None:
# Explicit off must reach the wire; omission lets
# reasoning-by-default models spend hidden reasoning tokens
# (verified live 2026-09-01 on Groq: gpt-oss-20b spent 45
@@ -1454,7 +1509,7 @@ class OpenAIChatLM(BaseProviderLM):
payload["chat_template_kwargs"] = {"enable_thinking": False}
# Prompt caching (MAP-6): off switch, key, retention, resource.
- _cache_common_payload(request, payload, compat.cache_control, self.provider)
+ _cache_common_payload(request, payload, compat.cache_control, self.provider, breakpoint_index=breakpoint_index)
if compat.routing is not None:
payload["provider"] = compat.routing
@@ -1494,7 +1549,6 @@ class OpenAIChatLM(BaseProviderLM):
model=request.model,
headers=self._headers(),
payload=self._payload(request, stream=stream),
- read_timeout=120.0 if stream else 60.0,
)
# ─── Response parsing ───────────────────────────────────────────
diff --git a/dspy/_vendor/lm15/providers/openai_codex.py b/dspy/_vendor/lm15/providers/openai_codex.py
index 57afb4352..eac387222 100644
--- a/dspy/_vendor/lm15/providers/openai_codex.py
+++ b/dspy/_vendor/lm15/providers/openai_codex.py
@@ -24,6 +24,7 @@ from ..access import ( # noqa: F401
OPENAI_CODEX,
)
from ..features import ProviderManifest
+from ..adaptation import AdaptationPolicy
from .base import Credential, SyncTransport, default_transport
from .openai import MODEL_LIST_HINT, OpenAILM # noqa: F401
@@ -43,6 +44,7 @@ class OpenAICodexLM(OpenAILM):
base_url: str = DEFAULT_CODEX_BASE_URL,
originator: str = DEFAULT_CODEX_ORIGINATOR,
client_version: str = DEFAULT_CODEX_CLIENT_VERSION,
+ adaptations: "AdaptationPolicy" = "note",
) -> None:
self.originator = originator
self.client_version = client_version
@@ -61,6 +63,7 @@ class OpenAICodexLM(OpenAILM):
access=policy,
credentials_path=auth_path,
account_id=account_id,
+ adaptations=adaptations,
)
@classmethod
diff --git a/dspy/_vendor/lm15/providers/xai.py b/dspy/_vendor/lm15/providers/xai.py
index cd9a26e13..1919531b1 100644
--- a/dspy/_vendor/lm15/providers/xai.py
+++ b/dspy/_vendor/lm15/providers/xai.py
@@ -31,12 +31,15 @@ explicitly (``api_key=`` / ``RouterConfig.api_keys``). Run
from __future__ import annotations
+from dataclasses import replace
+
import base64
import json
import os
from typing import Any, ClassVar
from ..access import DEFAULT_XAI_BASE_URL, XAI
+from ..adaptation import adapt
from ..errors import ProviderError, UnsupportedFeatureError
from ..features import ProviderManifest
from ..transports import TransportRequest
@@ -65,6 +68,7 @@ class XaiLM(OpenAIChatLM):
credentials_path: str | os.PathLike[str] | None = None,
transport: SyncTransport | None = None,
base_url: str = DEFAULT_XAI_BASE_URL,
+ adaptations: "AdaptationPolicy" = "note",
) -> None:
super().__init__(
api_key=api_key,
@@ -73,6 +77,7 @@ class XaiLM(OpenAIChatLM):
compat="xai",
access=XAI,
credentials_path=credentials_path,
+ adaptations=adaptations,
)
def _payload(self, request: Request, stream: bool) -> dict[str, Any]:
@@ -85,12 +90,13 @@ class XaiLM(OpenAIChatLM):
# (Config(reasoning=None)) — they never reason anyway.
reasoning = request.config.reasoning
if reasoning is not None and reasoning.is_off:
- raise UnsupportedFeatureError(
- "xai: reasoning cannot be disabled — Grok reasoning models have no "
- "off switch, and xAI silently ignores disable fields on the wire. "
- "Omit the reasoning config, or pick a non-reasoning Grok model.",
- provider=self.provider,
- )
+ # MAP-13 (decision 2026-09-14 §4.2): no off switch exists; the
+ # lowest level is the closest and the spend shows in usage.
+ adapt("config.reasoning.effort", "substituted",
+ "Grok reasoning models have no off switch and api.x.ai ignores disable fields "
+ "(158 reasoning tokens on an explicit off, live 2026-09-01); the lowest level was sent",
+ asked="off", applied="low", provider=self.provider)
+ request = replace(request, config=replace(request.config, reasoning=replace(reasoning, effort="low")))
# docs.x.ai (models page, 2026-09-01): "logprobs and top_logprobs are
# not supported by models grok-4.20 and newer. These fields will be
# silently ignored if set." Verified live 2026-09-01 on grok-4.6:
@@ -98,30 +104,33 @@ class XaiLM(OpenAIChatLM):
# model served today is 4.20 or newer, so sending the field is a
# guaranteed silent no-op — raise instead.
if request.config.logprobs is not None:
- raise UnsupportedFeatureError(
- "xai: config.logprobs is not supported — grok-4.20 and newer "
- "silently ignore logprobs/top_logprobs on the wire (docs.x.ai, "
- "verified live 2026-09-01). OpenAI and Gemini carry logprobs.",
- provider=self.provider,
- )
+ adapt("config.logprobs", "dropped",
+ "grok-4.20 and newer ignore logprobs/top_logprobs (docs.x.ai, live 2026-09-01); "
+ "Response.logprobs will be absent (OpenAI and Gemini carry them)",
+ asked=request.config.logprobs, provider=self.provider)
+ request = replace(request, config=replace(request.config, logprobs=None))
tc = request.config.tool_choice
if tc is not None and tc.allowed and not (len(tc.allowed) == 1 and tc.mode == "required"):
- # MAP-8 rule 1 (live 2026-09-02): api.x.ai accepts allowed_tools and
- # ignores it — with {lookup} allowed and weather asked, it called
- # weather. A silent widen; the forced single-function form held.
- raise UnsupportedFeatureError(
- "xai: tool_choice.allowed subsets are silently ignored by api.x.ai "
- "(verified live 2026-09-02); force a single tool with mode='required', "
- "or send only the allowed tools in Request.tools",
- provider=self.provider,
- )
+ # api.x.ai accepts allowed_tools and ignores it (live 2026-09-02:
+ # with {lookup} allowed and weather asked, it called weather).
+ # MAP-13 client_side: send only the allowed tools — what the
+ # allowlist means — and record it.
+ kept = tuple(t for t in request.tools if t.name in tc.allowed)
+ adapt("config.tool_choice.allowed", "client_side",
+ "api.x.ai ignores tool_choice allowlists (live 2026-09-02); only the allowed tools "
+ "were sent, which is what the allowlist means",
+ asked=list(tc.allowed), applied=[t.name for t in kept], provider=self.provider)
+ request = replace(request, tools=kept,
+ config=replace(request.config, tool_choice=replace(tc, allowed=())))
+ tc = request.config.tool_choice
if tc is not None and tc.mode == "required" and request.config.response_format is not None:
# MAP-8 rule 3 (live 2026-09-02): a forced tool next to a
# response_format returned JSON text and no call.
+ # MAP-13 rule 4(b): the program depends on the call.
raise UnsupportedFeatureError(
"xai: a forced tool (mode='required') cannot be combined with response_format — "
"api.x.ai returns JSON text and drops the call (verified live 2026-09-02)",
- provider=self.provider,
+ provider=self.provider, feature="config.tool_choice.mode",
)
return super()._payload(request, stream)
diff --git a/dspy/_vendor/lm15/result.py b/dspy/_vendor/lm15/result.py
index a9bb726e0..bc117decb 100644
--- a/dspy/_vendor/lm15/result.py
+++ b/dspy/_vendor/lm15/result.py
@@ -21,10 +21,11 @@ any execute-tools-until-done loop belongs to the layer above lm15.
from __future__ import annotations
import json
+from collections import deque
from dataclasses import dataclass, field, replace
from typing import Any, AsyncIterator, Iterator
-from .errors import LM15Error, StreamAssemblyError, error_class_for_code
+from .errors import LM15Error, StreamAssemblyError, StreamCleanupWarning, error_class_for_code
from .types import (
AudioDelta,
AudioPart,
@@ -100,13 +101,17 @@ class StreamAccumulator:
message_continuation: list[ContinuationState] = field(default_factory=list)
part_continuation: dict[int, list[ContinuationState]] = field(default_factory=dict)
logprob_seq: list[TokenLogprob] = field(default_factory=list)
+ logprobs_complete: bool = True
provider_data: dict[str, Any] | None = None
+ adaptations: tuple = ()
def push(self, event: StreamEvent) -> None:
"""Fold one canonical stream event into the accumulated state."""
if event.type == "start":
self.started_id = event.id or self.started_id
self.started_model = event.model or self.started_model
+ if event.adaptations:
+ self.adaptations = event.adaptations
return
if event.type == "end":
@@ -125,6 +130,7 @@ class StreamAccumulator:
self.text_parts.setdefault(delta.part_index, []).append(delta.text or "")
if delta.logprobs:
self.logprob_seq.extend(delta.logprobs)
+ self.logprobs_complete = self.logprobs_complete and delta.logprobs_complete
elif delta.type == "thinking":
self.thinking_parts.setdefault(delta.part_index, []).append(delta.text or "")
@@ -259,7 +265,9 @@ class StreamAccumulator:
finish_reason=finish,
usage=self.usage or Usage(),
logprobs=tuple(self.logprob_seq) if self.logprob_seq else None,
+ logprobs_complete=self.logprobs_complete,
provider_data=self.provider_data,
+ adaptations=self.adaptations,
)
@@ -287,12 +295,19 @@ class ResponseStream:
self._failure: BaseException | None = None
self._done = False
self._source_closed = False
+ #: Failures that followed the end event (a read error while draining,
+ #: a close() that raised). The Response is complete regardless; each
+ #: was also emitted as a StreamCleanupWarning.
+ self.cleanup_errors: tuple[BaseException, ...] = ()
self._event_iter = self._pump()
def _close_source(self, primary: BaseException | None = None) -> None:
if not self._source_closed:
self._source_closed = True
- _close_events(self._source, primary, self._response)
+ _close_events(self._source, primary, self._response, self._record_cleanup)
+
+ def _record_cleanup(self, exc: BaseException) -> None:
+ self.cleanup_errors = (*self.cleanup_errors, exc)
def close(self) -> None:
"""Stop reading and release the source, without draining it.
@@ -301,7 +316,7 @@ class ResponseStream:
promise that the provider stops generating or billing immediately.
"""
if not self._done and self._response is None and self._failure is None:
- self._failure = RuntimeError("stream closed before completion")
+ self._failure = _closed_early(self._accumulator)
try:
self._event_iter.close()
finally:
@@ -371,20 +386,23 @@ class ResponseStream:
def _pump(self) -> Iterator[StreamEvent]:
primary = None
try:
- for event in self._source:
- _check_terminal(event, self._response)
- self._accumulator.push(event)
- if event.type == "end":
- self._response = self._accumulator.response()
- yield event
+ try:
+ for event in self._source:
+ _check_terminal(event, self._response)
+ self._accumulator.push(event)
+ if event.type == "end":
+ self._response = self._accumulator.response()
+ yield event
+ except Exception as exc:
+ if not _after_completion(exc, self._response):
+ raise
+ _warn_cleanup(exc, self._record_cleanup)
if self._response is None:
raise _incomplete(self._accumulator)
except BaseException as exc:
- primary = _after_completion_error(exc, self._response)
+ primary = exc
if self._failure is None and not (isinstance(exc, GeneratorExit) and self._response is not None):
- self._failure = primary
- if primary is not exc:
- raise primary from exc
+ self._failure = exc
raise
finally:
self._done = True
@@ -414,17 +432,22 @@ class AsyncResponseStream:
self._failure: BaseException | None = None
self._done = False
self._source_closed = False
+ #: See :attr:`ResponseStream.cleanup_errors`.
+ self.cleanup_errors: tuple[BaseException, ...] = ()
self._event_gen = self._pump()
async def _close_source(self, primary: BaseException | None = None) -> None:
if not self._source_closed:
self._source_closed = True
- await _aclose_events(self._source, primary, self._response)
+ await _aclose_events(self._source, primary, self._response, self._record_cleanup)
+
+ def _record_cleanup(self, exc: BaseException) -> None:
+ self.cleanup_errors = (*self.cleanup_errors, exc)
async def aclose(self) -> None:
"""Stop reading and release the source, without draining it."""
if not self._done and self._response is None and self._failure is None:
- self._failure = RuntimeError("stream closed before completion")
+ self._failure = _closed_early(self._accumulator)
try:
await self._event_gen.aclose()
finally:
@@ -465,20 +488,23 @@ class AsyncResponseStream:
async def _pump(self) -> AsyncIterator[StreamEvent]:
primary = None
try:
- async for event in self._source:
- _check_terminal(event, self._response)
- self._accumulator.push(event)
- if event.type == "end":
- self._response = self._accumulator.response()
- yield event
+ try:
+ async for event in self._source:
+ _check_terminal(event, self._response)
+ self._accumulator.push(event)
+ if event.type == "end":
+ self._response = self._accumulator.response()
+ yield event
+ except Exception as exc:
+ if not _after_completion(exc, self._response):
+ raise
+ _warn_cleanup(exc, self._record_cleanup)
if self._response is None:
raise _incomplete(self._accumulator)
except BaseException as exc:
- primary = _after_completion_error(exc, self._response)
+ primary = exc
if self._failure is None and not (isinstance(exc, GeneratorExit) and self._response is not None):
- self._failure = primary
- if primary is not exc:
- raise primary from exc
+ self._failure = exc
raise
finally:
self._done = True
@@ -497,18 +523,21 @@ def materialize_response(events: Iterator[StreamEvent], request: Request) -> Res
response = None
primary = None
try:
- for event in events:
- _check_terminal(event, response)
- accumulator.push(event)
- if event.type == "end":
- response = accumulator.response()
+ try:
+ for event in events:
+ _check_terminal(event, response)
+ accumulator.push(event)
+ if event.type == "end":
+ response = accumulator.response()
+ except Exception as exc:
+ if not _after_completion(exc, response):
+ raise
+ _warn_cleanup(exc)
if response is None:
raise _incomplete(accumulator)
return response
except BaseException as exc:
- primary = _after_completion_error(exc, response)
- if primary is not exc:
- raise primary from exc
+ primary = exc
raise
finally:
_close_events(events, primary, response)
@@ -520,18 +549,21 @@ async def amaterialize_response(events: AsyncIterator[StreamEvent], request: Req
response = None
primary = None
try:
- async for event in events:
- _check_terminal(event, response)
- accumulator.push(event)
- if event.type == "end":
- response = accumulator.response()
+ try:
+ async for event in events:
+ _check_terminal(event, response)
+ accumulator.push(event)
+ if event.type == "end":
+ response = accumulator.response()
+ except Exception as exc:
+ if not _after_completion(exc, response):
+ raise
+ _warn_cleanup(exc)
if response is None:
raise _incomplete(accumulator)
return response
except BaseException as exc:
- primary = _after_completion_error(exc, response)
- if primary is not exc:
- raise primary from exc
+ primary = exc
raise
finally:
await _aclose_events(events, primary, response)
@@ -539,7 +571,11 @@ async def amaterialize_response(events: AsyncIterator[StreamEvent], request: Req
def _check_terminal(event, response):
if response is not None:
- raise StreamAssemblyError("Stream emitted an event after completion", partial=response)
+ raise StreamAssemblyError(
+ "Stream emitted an event after its end event (MAP-3: the end event is "
+ "final); the source that produced this stream is defective",
+ partial=response,
+ )
if event.type == "error":
raise _exception_from_error(event)
@@ -551,19 +587,61 @@ def _incomplete(accumulator):
partial = exc.partial
except Exception:
partial = None
- return StreamAssemblyError("Stream ended without a completion event", partial=partial)
+ return StreamAssemblyError(
+ "Stream ended without an end event: its finish reason and usage never "
+ "arrived, so the text is not a finished turn (MAP-3)",
+ partial=partial,
+ )
-def _after_completion_error(exc, response):
- if response is not None and isinstance(exc, Exception) and not isinstance(exc, (StreamAssemblyError, Warning)):
- return StreamAssemblyError("Stream failed after response completion", partial=response)
- return exc
+def _closed_early(accumulator):
+ """The caller closed the stream before its end event: the same fact as
+ a stream that ended without one (MAP-3) — no finish reason, no usage,
+ the text not a finished turn — so the same error, inside the family
+ (every failure lm15 raises is an LM15Error). ``partial`` is what had
+ arrived."""
+ try:
+ partial = accumulator.response()
+ except StreamAssemblyError as exc:
+ partial = exc.partial
+ except Exception:
+ partial = None
+ return StreamAssemblyError(
+ "Stream closed before its end event: the response was not completed "
+ "(close() was called while the stream was still open; MAP-3)",
+ partial=partial,
+ )
+
+def _after_completion(exc, response):
+ """Is ``exc`` a source failure that followed a complete Response?
+
+ True only when the end event was seen and the failure is not itself an
+ assembly defect: the Response is complete, the provider billed it, and
+ the failure concerns the connection's afterlife. Such a failure is
+ reported (StreamCleanupWarning), never raised in place of the Response.
+ """
+ return response is not None and not isinstance(exc, StreamAssemblyError)
-def _cleanup_failure(primary, cleanup, response):
+
+def _warn_cleanup(exc, record=None):
+ import warnings
+
+ if record is not None:
+ record(exc)
+ warnings.warn(
+ f"stream source failed after the response was complete "
+ f"({type(exc).__name__}: {exc}); the Response is returned unchanged",
+ StreamCleanupWarning,
+ stacklevel=3,
+ )
+
+
+def _cleanup_failure(primary, cleanup, response, record):
if primary is None or isinstance(primary, GeneratorExit):
if response is not None:
- raise StreamAssemblyError("Stream cleanup failed after response completion", partial=response) from cleanup
+ _warn_cleanup(cleanup, record)
+ return
raise cleanup
try:
primary.cleanup_errors = (*getattr(primary, "cleanup_errors", ()), cleanup)
@@ -573,22 +651,22 @@ def _cleanup_failure(primary, cleanup, response):
pass # Optional diagnostics must not replace the primary failure.
-def _close_events(source, primary=None, response=None):
+def _close_events(source, primary=None, response=None, record=None):
try:
close = getattr(source, "close", None)
if close is not None:
close()
except Exception as cleanup:
- _cleanup_failure(primary, cleanup, response)
+ _cleanup_failure(primary, cleanup, response, record)
-async def _aclose_events(source, primary=None, response=None):
+async def _aclose_events(source, primary=None, response=None, record=None):
try:
close = getattr(source, "aclose", None)
if close is not None:
await close()
except Exception as cleanup:
- _cleanup_failure(primary, cleanup, response)
+ _cleanup_failure(primary, cleanup, response, record)
# ─── Conversion utilities ────────────────────────────────────────────
@@ -600,15 +678,20 @@ def response_to_events(response: Response) -> Iterator[StreamEvent]:
response contains a valid Part that has no Delta representation, this
function raises instead of silently dropping content.
"""
- yield StreamStartEvent(id=response.id, model=response.model)
+ yield StreamStartEvent(id=response.id, model=response.model, adaptations=response.adaptations)
# Response.logprobs is message-level; the delta vocabulary carries
# logprobs on text deltas. Emitting the whole sequence on the first
# text delta makes Response -> events -> Response lossless.
pending_logprobs = response.logprobs or ()
+ pending_complete = response.logprobs_complete
+ if not pending_complete and not any(isinstance(p, TextPart) for p in response.message.parts):
+ raise TypeError("Cannot stream incomplete logprobs without a TextPart to carry their coverage")
for idx, part in enumerate(response.message.parts):
if isinstance(part, TextPart):
- yield StreamDeltaEvent(delta=TextDelta(text=part.text, part_index=idx, logprobs=pending_logprobs))
+ yield StreamDeltaEvent(delta=TextDelta(text=part.text, part_index=idx, logprobs=pending_logprobs,
+ logprobs_complete=pending_complete))
pending_logprobs = ()
+ pending_complete = True
elif isinstance(part, ThinkingPart):
yield StreamDeltaEvent(delta=ThinkingDelta(text=part.text, part_index=idx))
elif isinstance(part, ToolCallPart):
@@ -679,6 +762,14 @@ def response_to_events(response: Response) -> Iterator[StreamEvent]:
)
+def _stamp_start(event: StreamStartEvent, adaptations: tuple) -> StreamStartEvent:
+ if not adaptations or event.adaptations:
+ return event
+ from dataclasses import replace
+
+ return replace(event, adaptations=tuple(adaptations))
+
+
class _EndProviderData:
"""MAP-3 / D9: which adapter end event's ``provider_data`` the merged
end carries. Rank 2: a frame that supplied usage. Rank 1: a frame that
@@ -702,7 +793,7 @@ class _EndProviderData:
def coalesce_stream(
- events: Iterator[StreamEvent], *, model: str | None = None
+ events: Iterator[StreamEvent], *, model: str | None = None, adaptations: tuple = ()
) -> Iterator[StreamEvent]:
"""Enforce MAP-3 and MAP-4: one final StreamEndEvent, one leading StreamStartEvent.
@@ -725,19 +816,28 @@ def coalesce_stream(
start passes through; duplicates after the first are dropped. Error
events never force a start: a stream that fails to open has no start.
+ ``adaptations`` (MAP-13) are stamped on the start event, provider-sent
+ or synthesized: they are known before the first byte.
+
See docs/mapping-rules.md MAP-3 and MAP-4.
"""
primary = None
+ ended = None
try:
- yield from _coalesce_stream(events, model=model)
+ for event in _coalesce_stream(events, model=model, adaptations=adaptations):
+ if event.type == "end":
+ ended = event
+ yield event
except BaseException as exc:
primary = exc
raise
finally:
- _close_events(events, primary)
+ # After the merged end event the stream is complete: a close() that
+ # raises is reported, not raised (contract 2026-09-11-stream-completion).
+ _close_events(events, primary, ended)
-def _coalesce_stream(events: Iterator[StreamEvent], *, model: str | None) -> Iterator[StreamEvent]:
+def _coalesce_stream(events: Iterator[StreamEvent], *, model: str | None, adaptations: tuple = ()) -> Iterator[StreamEvent]:
started = False
saw_end = False
finish_reason = None
@@ -748,7 +848,7 @@ def _coalesce_stream(events: Iterator[StreamEvent], *, model: str | None) -> Ite
if started:
continue
started = True
- yield event
+ yield _stamp_start(event, adaptations)
continue
if event.type == "end":
saw_end = True
@@ -760,11 +860,11 @@ def _coalesce_stream(events: Iterator[StreamEvent], *, model: str | None) -> Ite
continue
if not started and event.type == "delta":
started = True
- yield StreamStartEvent(model=model)
+ yield StreamStartEvent(model=model, adaptations=adaptations)
yield event
if saw_end:
if not started:
- yield StreamStartEvent(model=model)
+ yield StreamStartEvent(model=model, adaptations=adaptations)
yield StreamEndEvent(
finish_reason=finish_reason,
usage=usage,
@@ -773,7 +873,7 @@ def _coalesce_stream(events: Iterator[StreamEvent], *, model: str | None) -> Ite
async def acoalesce_stream(
- events: "AsyncIterator[StreamEvent]", *, model: str | None = None
+ events: "AsyncIterator[StreamEvent]", *, model: str | None = None, adaptations: tuple = ()
) -> "AsyncIterator[StreamEvent]":
"""Async mirror of :func:`coalesce_stream` — same MAP-3/MAP-4 semantics.
@@ -785,17 +885,20 @@ async def acoalesce_stream(
are dropped; error events never force a start.
"""
primary = None
+ ended = None
try:
- async for event in _acoalesce_stream(events, model=model):
+ async for event in _acoalesce_stream(events, model=model, adaptations=adaptations):
+ if event.type == "end":
+ ended = event
yield event
except BaseException as exc:
primary = exc
raise
finally:
- await _aclose_events(events, primary)
+ await _aclose_events(events, primary, ended)
-async def _acoalesce_stream(events: AsyncIterator[StreamEvent], *, model: str | None) -> AsyncIterator[StreamEvent]:
+async def _acoalesce_stream(events: AsyncIterator[StreamEvent], *, model: str | None, adaptations: tuple = ()) -> AsyncIterator[StreamEvent]:
started = False
saw_end = False
finish_reason = None
@@ -806,7 +909,7 @@ async def _acoalesce_stream(events: AsyncIterator[StreamEvent], *, model: str |
if started:
continue
started = True
- yield event
+ yield _stamp_start(event, adaptations)
continue
if event.type == "end":
saw_end = True
@@ -818,11 +921,11 @@ async def _acoalesce_stream(events: AsyncIterator[StreamEvent], *, model: str |
continue
if not started and event.type == "delta":
started = True
- yield StreamStartEvent(model=model)
+ yield StreamStartEvent(model=model, adaptations=adaptations)
yield event
if saw_end:
if not started:
- yield StreamStartEvent(model=model)
+ yield StreamStartEvent(model=model, adaptations=adaptations)
yield StreamEndEvent(
finish_reason=finish_reason,
usage=usage,
@@ -830,6 +933,208 @@ async def _acoalesce_stream(events: AsyncIterator[StreamEvent], *, model: str |
)
+# ─── MAP-13 client-side stop ─────────────────────────────────────────
+#
+# A wire with no stop field (OpenAI Responses) gets the sequence applied
+# here: the visible text is cut at the first occurrence, the finish reason
+# becomes "stop", and the source is closed at the cut. Whether the
+# provider then stops generating (and billing) on a closed connection is
+# the provider's behaviour, not a promise lm15 can make; what lm15 does
+# promise is that the end event carries no usage after a cut (the final
+# frame was never read) — "not reported", never estimated.
+
+
+def _first_stop(text: str, stop: tuple[str, ...]) -> tuple[int, str] | None:
+ best: tuple[int, str] | None = None
+ for seq in stop:
+ if not seq:
+ continue
+ idx = text.find(seq)
+ if idx >= 0 and (best is None or idx < best[0]):
+ best = (idx, seq)
+ return best
+
+
+def apply_client_side_stop(response: Response, stop: tuple[str, ...]) -> Response:
+ """Cut the response's visible text at the first stop sequence.
+
+ The text parts are one stream in document order: a sequence that
+ starts at the end of one part and finishes at the start of the next
+ is a hit (a provider's own stop works on the token stream and knows
+ no block boundary). The part holding the start is cut there; every
+ later part is removed.
+ """
+ from dataclasses import replace
+
+ if not stop:
+ return response
+ text_parts = [(i, p) for i, p in enumerate(response.message.parts) if isinstance(p, TextPart)]
+ joined = "".join(p.text for _, p in text_parts)
+ hit = _first_stop(joined, stop)
+ if hit is None:
+ return response
+ # Locate the part that holds the hit's start.
+ offset = 0
+ cut_index, cut_at = text_parts[-1][0], 0
+ for index, part in text_parts:
+ if offset + len(part.text) > hit[0]:
+ cut_index, cut_at = index, hit[0] - offset
+ break
+ offset += len(part.text)
+ parts: list[Part] = []
+ for index, part in enumerate(response.message.parts):
+ if index > cut_index:
+ break
+ if index == cut_index:
+ parts.append(replace(part, text=part.text[:cut_at]))
+ else:
+ parts.append(part)
+ if not parts:
+ parts = [TextPart(text="")]
+ message = replace(response.message, parts=tuple(parts))
+ scores, incomplete = _scores_before_cut(response.logprobs or (), joined, hit[0])
+ return replace(response, message=message, finish_reason="stop", logprobs=scores or None,
+ logprobs_complete=response.logprobs_complete and not incomplete)
+
+
+def _scores_before_cut(
+ scores: tuple[TokenLogprob, ...], text: str, cut_at: int,
+) -> tuple[tuple[TokenLogprob, ...], bool]:
+ """Keep original scores for whole retained tokens; never score a token fragment.
+
+ Byte boundaries matter: a provider token can itself contain only part of
+ a Unicode character. Only use token spellings as a fallback when their
+ concatenated UTF-8 bytes exactly reproduce the original text.
+ """
+ if not scores or cut_at == 0:
+ return (), False
+ if cut_at == len(text):
+ return scores, False
+ try:
+ token_bytes = [bytes(s.bytes) if s.bytes is not None else s.token.encode("utf-8") for s in scores]
+ original = text.encode("utf-8")
+ boundary = len(text[:cut_at].encode("utf-8"))
+ except (ValueError, UnicodeError):
+ return (), True
+ if b"".join(token_bytes) != original:
+ # These scores cannot be placed reliably on the shortened text.
+ return (), True
+ end = 0
+ for index, data in enumerate(token_bytes):
+ if end == boundary:
+ return scores[:index], False
+ end += len(data)
+ if end > boundary:
+ return scores[:index], True
+ return scores, False
+
+
+class _StopCutter:
+ """Keep original events until their text is safe, then pass them unchanged.
+
+ Text parts form one text stream. A possible stop suffix holds its
+ entire event, plus intervening events to preserve order. Only the
+ event actually cut is reconstructed; unrelated fields survive via
+ dataclasses.replace. This can delay delivery by an event boundary,
+ but never splits token scores just to release text earlier.
+ """
+
+ def __init__(self, stop: tuple[str, ...]) -> None:
+ self.stop = tuple(s for s in stop if s)
+ self.hold = max((len(s) for s in self.stop), default=1) - 1
+ self.segments: deque[StreamEvent] = deque()
+ self.cut = False
+
+ def _take(self, count: int, *, cutting: bool = False) -> list[StreamEvent]:
+ out: list[StreamEvent] = []
+ while self.segments:
+ event = self.segments[0]
+ if event.type != "delta" or not isinstance(event.delta, TextDelta):
+ out.append(self.segments.popleft())
+ continue
+ delta = event.delta
+ if cutting and count == 0:
+ break
+ if len(delta.text) <= count:
+ out.append(self.segments.popleft())
+ count -= len(delta.text)
+ elif cutting:
+ scores, incomplete = _scores_before_cut(delta.logprobs, delta.text, count)
+ out.append(replace(event, delta=replace(delta, text=delta.text[:count], logprobs=scores,
+ logprobs_complete=delta.logprobs_complete and not incomplete)))
+ break
+ else:
+ break
+ return out
+
+ def feed(self, event: StreamEvent) -> list[StreamEvent]:
+ if event.type != "delta" or not isinstance(event.delta, TextDelta):
+ if not self.segments:
+ return [event]
+ self.segments.append(event)
+ return []
+ self.segments.append(event)
+ buf = "".join(e.delta.text for e in self.segments
+ if e.type == "delta" and isinstance(e.delta, TextDelta))
+ hit = _first_stop(buf, self.stop)
+ if hit is not None:
+ out = self._take(hit[0], cutting=True)
+ self.segments.clear()
+ self.cut = True
+ return out
+ return self._take(max(0, len(buf) - self.hold))
+
+ def flush(self) -> list[StreamEvent]:
+ out = list(self.segments)
+ self.segments.clear()
+ return out
+
+
+def _cut_events(cutter: _StopCutter, event: StreamEvent):
+ if event.type in ("end", "error"):
+ yield from cutter.flush()
+ yield event
+ else:
+ yield from cutter.feed(event)
+
+
+def truncate_stream_at_stop(events: Iterator[StreamEvent], stop: tuple[str, ...]) -> Iterator[StreamEvent]:
+ cutter = _StopCutter(stop)
+ if not cutter.stop:
+ yield from events
+ return
+ try:
+ for event in events:
+ for out in _cut_events(cutter, event):
+ yield out
+ if cutter.cut:
+ yield StreamEndEvent(finish_reason="stop")
+ return
+ finally:
+ close = getattr(events, "close", None)
+ if close is not None:
+ close()
+
+
+async def atruncate_stream_at_stop(events: AsyncIterator[StreamEvent], stop: tuple[str, ...]) -> AsyncIterator[StreamEvent]:
+ cutter = _StopCutter(stop)
+ if not cutter.stop:
+ async for event in events:
+ yield event
+ return
+ try:
+ async for event in events:
+ for out in _cut_events(cutter, event):
+ yield out
+ if cutter.cut:
+ yield StreamEndEvent(finish_reason="stop")
+ return
+ finally:
+ aclose = getattr(events, "aclose", None)
+ if aclose is not None:
+ await aclose()
+
+
# ─── Internal helpers ────────────────────────────────────────────────
def _raise_non_streamable_part(part: Part, *, reason: str | None = None) -> None:
diff --git a/dspy/_vendor/lm15/router.py b/dspy/_vendor/lm15/router.py
index 75efc9ca8..674001de1 100644
--- a/dspy/_vendor/lm15/router.py
+++ b/dspy/_vendor/lm15/router.py
@@ -57,11 +57,23 @@ recommended front door. Needing a custom ``base_url``/transport/compat
(azure, a self-hosted gateway) is the documented escape hatch: ``lm()``
returns the ordinary provider LM — keep it and configure it yourself
next time.
+
+Connections
+-----------
+A router owns ONE transport (connection pool) shared by every LM it
+builds, so ``RouterConfig(timeouts=..., max_connections=...)`` is the
+whole connection budget of that router: ``Timeouts(read=1800)`` for a
+slow local model, ``max_connections=200`` for a wide evaluation. Pass
+``transport=`` instead to bring your own (then the two knobs are refused:
+configure the transport you pass). ``close()`` / ``aclose()`` (or ``with``)
+releases every socket; a router that is simply dropped closes its idle
+sockets when collected.
"""
from __future__ import annotations
import os
+import threading
from dataclasses import dataclass, field, replace
from types import MappingProxyType
from typing import AsyncIterator, Iterator, Literal, Mapping, overload
@@ -72,6 +84,8 @@ from .errors import AmbiguousModelError, NotConfiguredError, UnknownModelError
from .models import ModelInfo, ModelRegistry
from .providers import Credential
from .registry import PROVIDERS, ProviderDefinition, canonical_provider as _canonical_provider
+from .adaptation import AdaptationPolicy, check_policy
+from .transports import Timeouts
from .types import Request, Response, StreamEvent
__all__ = [
@@ -362,6 +376,13 @@ class RouterConfig:
proxy in front of OpenAI, a vLLM server on another port. Not for
the cloud doors (azure, bedrock, vertex): their URL is derived from
``settings`` (resource, region), and an entry for one is refused.
+
+ ``timeouts`` (:class:`lm15.Timeouts`) and ``max_connections`` shape the
+ one transport the router builds and shares across its LMs; defaults
+ are the provider SDKs' (connect 10 s, read/write/pool 600 s, 100
+ connections). ``transport`` replaces that transport with one you
+ built; it cannot be combined with the two knobs, which would then
+ silently not apply.
"""
registry: ModelRegistry | None = None
@@ -377,6 +398,44 @@ class RouterConfig:
transport: object | None = None # SyncTransport for LMRouter, AsyncTransport
# for AsyncLMRouter; passed to every LM the
# router constructs (tests, custom pooling)
+ timeouts: Timeouts | None = field(default=None, kw_only=True)
+ max_connections: int | None = field(default=None, kw_only=True)
+ # MAP-13: "note" (adapt and record on the response), "silent" (adapt,
+ # record nothing), "refuse" (every adaptation is an error before the
+ # wire). Applied to every LM the router builds.
+ adaptations: AdaptationPolicy = field(default="note", kw_only=True)
+
+ def __post_init__(self) -> None:
+ check_policy(self.adaptations)
+ if self.timeouts is not None and not isinstance(self.timeouts, Timeouts):
+ raise TypeError(
+ f"RouterConfig(timeouts=...) takes lm15.Timeouts, got {type(self.timeouts).__name__}; "
+ "e.g. Timeouts(read=600)"
+ )
+ if self.max_connections is not None:
+ from .transports._limits import check_max_connections
+
+ check_max_connections(self.max_connections)
+ if self.transport is not None and (self.timeouts is not None or self.max_connections is not None):
+ raise NotConfiguredError(
+ "RouterConfig(transport=...) cannot be combined with timeouts= or max_connections=: "
+ "they configure the transport lm15 would build, and would silently not apply to "
+ "the one you passed. Configure that transport directly (StdlibTransport(read_timeout=...))."
+ )
+
+ def transport_kwargs(self) -> dict:
+ """Constructor keywords for the default transport this config asks for."""
+ kwargs: dict = {}
+ if self.timeouts is not None:
+ kwargs.update(
+ connect_timeout=self.timeouts.connect,
+ read_timeout=self.timeouts.read,
+ write_timeout=self.timeouts.write,
+ pool_timeout=self.timeouts.pool,
+ )
+ if self.max_connections is not None:
+ kwargs["max_connections"] = self.max_connections
+ return kwargs
# ------------------------------------------------------------- internals ----
@@ -645,12 +704,16 @@ def _env_key_for(provider: str, config: RouterConfig, adapters: Mapping[str, typ
return env_keys[0]
-def _build_lm(resolution: Resolution, config: RouterConfig, adapters: Mapping[str, type]):
+def _build_lm(resolution: Resolution, config: RouterConfig, adapters: Mapping[str, type], transport: object | None = None):
cls = _adapter_for(resolution.provider, adapters)
definition = _bound(resolution.provider, adapters)
extra: dict = {}
- if config.transport is not None:
+ if transport is not None:
+ extra["transport"] = transport
+ elif config.transport is not None:
extra["transport"] = config.transport
+ if config.adaptations != "note":
+ extra["adaptations"] = config.adaptations
base_url = _base_url_entry(config, resolution.provider)
if base_url is not None:
if definition is not None and definition.hosted:
@@ -733,6 +796,51 @@ def _build_lm(resolution: Resolution, config: RouterConfig, adapters: Mapping[st
return cls(api_key=api_key, **extra)
+PLANNING_KEY = "lm15-planning"
+
+
+def _build_planning_lm(resolution: Resolution, config: RouterConfig, adapters: Mapping[str, type], transport: object | None):
+ """A throwaway LM for ``plan()``: the build's bytes are discarded, so it
+ carries a placeholder credential under every policy — no stored login
+ is read or refreshed, no cloud chain is walked, no environment key is
+ needed — and placeholder host settings where a cloud door would
+ otherwise refuse to render its URL. Never cached. Everything else
+ (compat preset, access policy, base_url, adaptations policy) is the
+ real route's, so the plan is the call's."""
+ cls = _adapter_for(resolution.provider, adapters)
+ definition = _bound(resolution.provider, adapters)
+ extra: dict = {"api_key": PLANNING_KEY, "adaptations": config.adaptations}
+ if transport is not None:
+ extra["transport"] = transport
+ elif config.transport is not None:
+ extra["transport"] = config.transport
+ base_url = _base_url_entry(config, resolution.provider)
+ if base_url is not None and not (definition is not None and definition.hosted):
+ extra["base_url"] = base_url
+ account_field = getattr(cls, "__dataclass_fields__", {}).get("account_id")
+ if account_field is not None and account_field.init:
+ # The Codex backend derives an account id from its token at
+ # construction; a placeholder token has none, so name one.
+ extra["account_id"] = PLANNING_KEY
+ if definition is None:
+ return cls(**extra)
+ if definition.hosted:
+ from .cloud.hosts import resolve_settings
+
+ env = config.env if config.env is not None else os.environ
+ given = dict((config.settings or {}).get(resolution.provider) or {})
+ try:
+ settings = resolve_settings(definition.access.host, given, env, provider=resolution.provider)
+ except NotConfiguredError:
+ placeholders = {setting.name: given.get(setting.name) or setting.default or "planning"
+ for setting in definition.access.host.settings}
+ settings = resolve_settings(definition.access.host, placeholders, None, provider=resolution.provider)
+ if definition.compat is not None:
+ extra["compat"] = definition.compat
+ return cls(access=definition.access, settings=settings, **extra)
+ return cls(compat=definition.compat, access=definition.access, **extra)
+
+
def _routed_request(request: Request, resolution: Resolution) -> Request:
if request.model == resolution.model:
return request
@@ -768,7 +876,7 @@ _CLIENT_KEYWORDS: Mapping[str, str] = MappingProxyType({
"api_key": "LMRouter(RouterConfig(api_keys={provider: key})) or the environment",
"api_base": "LMRouter(RouterConfig(base_urls={provider: url}))",
"base_url": "LMRouter(RouterConfig(base_urls={provider: url}))",
- "timeout": "RouterConfig(transport=...)",
+ "timeout": "RouterConfig(timeouts=Timeouts(read=...))",
"num_retries": "your own retry loop over lm15.RETRYABLE_ERRORS (lm15 never retries)",
"max_retries": "your own retry loop over lm15.RETRYABLE_ERRORS (lm15 never retries)",
"headers": "RouterConfig(transport=...)",
@@ -778,7 +886,7 @@ _CLIENT_KEYWORDS: Mapping[str, str] = MappingProxyType({
"cache": "your own cache keyed on the Request (lm15 has no response cache)",
"caching": "your own cache keyed on the Request (lm15 has no response cache)",
"mock_response": "lm15.testing.FakeLM",
- "drop_params": "nothing: lm15 refuses what it cannot carry instead of dropping it",
+ "drop_params": "RouterConfig(adaptations='silent'): lm15 adapts what a wire cannot carry and records it on the response (MAP-13); 'silent' keeps no record, 'refuse' raises instead",
"custom_llm_provider": "the model string's prefix",
})
@@ -830,16 +938,49 @@ class LMRouter:
Four methods, no state you can't see: config is frozen; the only
mutation is an LM cache keyed by provider (one LM per provider,
- built lazily, reused).
+ built lazily, reused) and the one transport those LMs share.
"""
def __init__(self, config: RouterConfig = RouterConfig()) -> None:
_check_provider_keyed(config, self._adapters)
self.config = config
self._lms: dict[str, object] = {}
+ self._transport: object | None = config.transport
+ # One lock for the LM cache and the shared transport: routers are
+ # used from many threads (an evaluation's workers), and two first
+ # calls at once must not each build a pool that close() cannot
+ # find (found in review of dspy#10409).
+ self._lock = threading.RLock()
_adapters: Mapping[str, type] = ADAPTERS
+ def _shared_transport(self):
+ """The transport every LM of this router uses: the configured one,
+ else one StdlibTransport built from ``timeouts``/``max_connections``
+ on first use and shared, so the router's pool is one pool."""
+ with self._lock:
+ if self._transport is None:
+ from .transports import StdlibTransport
+
+ self._transport = StdlibTransport(**self.config.transport_kwargs())
+ return self._transport
+
+ def close(self) -> None:
+ """Close every connection this router holds. Idempotent; the router
+ may be used again afterwards (a fresh transport is built)."""
+ with self._lock:
+ transport, self._transport = self._transport, None
+ self._lms.clear()
+ close = getattr(transport, "close", None)
+ if callable(close):
+ close()
+
+ def __enter__(self) -> "LMRouter":
+ return self
+
+ def __exit__(self, exc_type, exc, tb) -> None:
+ self.close()
+
def resolve(self, model: str) -> Resolution:
"""Offline lookup; invokes no credential providers and returns no secrets.
@@ -856,12 +997,27 @@ class LMRouter:
hatch is built in: keep it, configure transports yourself.
"""
resolution = self.resolve(model)
- lm = self._lms.get(resolution.provider)
- if lm is None:
- lm = _build_lm(resolution, self.config, self._adapters)
- self._lms[resolution.provider] = lm
+ with self._lock:
+ lm = self._lms.get(resolution.provider)
+ if lm is None:
+ lm = _build_lm(resolution, self.config, self._adapters, self._shared_transport())
+ self._lms[resolution.provider] = lm
return lm
+ def plan(self, request: Request):
+ """MAP-13 pre-flight: what this request WOULD adapt on its route, no
+ network and no credential (like ``resolve()``); raises what the call
+ would raise. A route with no key gets a throwaway planning LM (not
+ cached): the build's bytes are discarded, so no key is needed."""
+ resolution = self.resolve(request.model)
+ with self._lock:
+ lm = self._lms.get(resolution.provider)
+ if lm is None:
+ # Not lm(): constructing the real LM reads or refreshes a stored
+ # login and walks cloud chains — I/O a pre-flight must not do.
+ lm = _build_planning_lm(resolution, self.config, self._adapters, self._shared_transport())
+ return lm.plan(_routed_request(request, resolution))
+
def complete(self, request: Request) -> Response:
resolution = self.resolve(request.model)
return self.lm(request.model).complete(_routed_request(request, resolution))
@@ -954,20 +1110,50 @@ class AsyncLMRouter:
_check_provider_keyed(config, self._adapters)
self.config = config
self._lms: dict[str, object] = {}
+ self._transport: object | None = config.transport
+ self._lock = threading.RLock() # construction is sync; see LMRouter
_adapters: Mapping[str, type] = ASYNC_ADAPTERS
+ def _shared_transport(self):
+ with self._lock:
+ if self._transport is None:
+ from .transports import StdlibAsyncTransport
+
+ self._transport = StdlibAsyncTransport(**self.config.transport_kwargs())
+ return self._transport
+
+ async def aclose(self) -> None:
+ """Close every connection this router holds. An async transport's
+ pool belongs to the loop that used it; call this on that loop
+ before it ends (one router per loop). Idempotent."""
+ with self._lock:
+ transport, self._transport = self._transport, None
+ self._lms.clear()
+ aclose = getattr(transport, "aclose", None)
+ if callable(aclose):
+ await aclose()
+
+ async def __aenter__(self) -> "AsyncLMRouter":
+ return self
+
+ async def __aexit__(self, exc_type, exc, tb) -> None:
+ await self.aclose()
+
def resolve(self, model: str) -> Resolution:
return _resolve(model, self.config, self._adapters)
def lm(self, model: str):
resolution = self.resolve(model)
- lm = self._lms.get(resolution.provider)
- if lm is None:
- lm = _build_lm(resolution, self.config, self._adapters)
- self._lms[resolution.provider] = lm
+ with self._lock:
+ lm = self._lms.get(resolution.provider)
+ if lm is None:
+ lm = _build_lm(resolution, self.config, self._adapters, self._shared_transport())
+ self._lms[resolution.provider] = lm
return lm
+ plan = LMRouter.plan
+
async def complete(self, request: Request) -> Response:
resolution = self.resolve(request.model)
return await self.lm(request.model).complete(_routed_request(request, resolution))
diff --git a/dspy/_vendor/lm15/serde.py b/dspy/_vendor/lm15/serde.py
index af4a30dd3..39b82aafb 100644
--- a/dspy/_vendor/lm15/serde.py
+++ b/dspy/_vendor/lm15/serde.py
@@ -10,6 +10,7 @@ from __future__ import annotations
from typing import Any
+from .adaptation import adaptation_from_dict, adaptation_to_dict
from .models import (
InferenceModelInfo,
InferencePricing,
@@ -490,6 +491,9 @@ def config_to_dict(c: Config) -> dict[str, Any]:
"tool_choice": tool_choice_to_dict(c.tool_choice) if c.tool_choice else None,
"reasoning": reasoning_to_dict(c.reasoning) if c.reasoning else None,
"cache": cache_config_to_dict(c.cache) if c.cache else None,
+ "seed": c.seed, # 0 is data (a seed), not emptiness — emitted
+ "frequency_penalty": c.frequency_penalty, # 0.0 is data (explicitly none) — emitted
+ "presence_penalty": c.presence_penalty,
"service_tier": c.service_tier,
"user_id": c.user_id,
"store": c.store, # False is data (opt-out), not emptiness — emitted
@@ -526,6 +530,9 @@ def config_from_dict(d: dict[str, Any]) -> Config:
tool_choice=tool_choice_from_dict(tool_choice) if tool_choice is not None else None,
reasoning=reasoning_from_dict(reasoning) if reasoning is not None else None,
cache=cache_config_from_dict(cache) if cache is not None else None,
+ seed=d.get("seed"),
+ frequency_penalty=d.get("frequency_penalty"),
+ presence_penalty=d.get("presence_penalty"),
service_tier=d.get("service_tier"),
user_id=d.get("user_id"),
store=d.get("store"),
@@ -614,6 +621,8 @@ def delta_to_dict(d: Delta) -> dict[str, Any]:
out["text"] = d.text
if d.logprobs:
out["logprobs"] = _logprobs_to_json(d.logprobs)
+ if not d.logprobs_complete:
+ out["logprobs_complete"] = False
elif isinstance(d, ThinkingDelta):
out["text"] = d.text
elif isinstance(d, AudioDelta):
@@ -654,6 +663,7 @@ def delta_from_dict(d: dict[str, Any]) -> Delta:
text=d.get("text", ""),
part_index=part_index,
logprobs=_logprobs_from_json(d.get("logprobs")) or (),
+ logprobs_complete=d.get("logprobs_complete", True),
)
if t == "thinking":
return ThinkingDelta(text=d.get("text", ""), part_index=part_index)
@@ -730,7 +740,10 @@ def usage_from_dict(d: dict[str, Any]) -> Usage:
def stream_event_to_dict(e: StreamEvent) -> dict[str, Any]:
if isinstance(e, StreamStartEvent):
- return _clean_mapping({"type": e.type, "id": e.id, "model": e.model})
+ return _clean_mapping({
+ "type": e.type, "id": e.id, "model": e.model,
+ "adaptations": [adaptation_to_dict(a) for a in e.adaptations] or None,
+ })
if isinstance(e, StreamDeltaEvent):
return {"type": e.type, "delta": delta_to_dict(e.delta)}
if isinstance(e, StreamEndEvent):
@@ -748,7 +761,10 @@ def stream_event_to_dict(e: StreamEvent) -> dict[str, Any]:
def stream_event_from_dict(d: dict[str, Any]) -> StreamEvent:
t = d["type"]
if t == "start":
- return StreamStartEvent(id=d.get("id"), model=d.get("model"))
+ return StreamStartEvent(
+ id=d.get("id"), model=d.get("model"),
+ adaptations=tuple(adaptation_from_dict(a) for a in d.get("adaptations") or ()),
+ )
if t == "delta":
return StreamDeltaEvent(delta=delta_from_dict(d["delta"]))
if t == "end":
@@ -803,6 +819,8 @@ def response_to_dict(r: Response, *, include_provider_data: bool = False) -> dic
"finish_reason": r.finish_reason,
"usage": usage_to_dict(r.usage),
"logprobs": _logprobs_to_json(r.logprobs),
+ "logprobs_complete": None if r.logprobs_complete else False,
+ "adaptations": [adaptation_to_dict(a) for a in r.adaptations] or None,
}
if include_provider_data and r.provider_data is not None:
out["provider_data"] = r.provider_data
@@ -817,7 +835,9 @@ def response_from_dict(d: dict[str, Any]) -> Response:
finish_reason=d["finish_reason"],
usage=usage_from_dict(d["usage"]) if isinstance(d.get("usage"), dict) else Usage(),
logprobs=_logprobs_from_json(d.get("logprobs")),
+ logprobs_complete=d.get("logprobs_complete", True),
provider_data=d.get("provider_data"),
+ adaptations=tuple(adaptation_from_dict(a) for a in d.get("adaptations") or ()),
)
diff --git a/dspy/_vendor/lm15/transports/__init__.py b/dspy/_vendor/lm15/transports/__init__.py
index 99991dd27..90f66406c 100644
--- a/dspy/_vendor/lm15/transports/__init__.py
+++ b/dspy/_vendor/lm15/transports/__init__.py
@@ -5,6 +5,8 @@ Public API:
TransportRequest, TransportResponse — transport-level request/response models
StdlibTransport — sync transport (blocking, socket-based)
StdlibAsyncTransport — async transport (asyncio-native)
+ FetchTransport — async transport over the host's fetch (Pyodide: a page, a worker)
+ Timeouts — per-operation timeouts (connect/read/write/pool)
TransportError + subclasses
"""
@@ -18,16 +20,21 @@ from ._exceptions import (
WriteError,
WriteTimeout,
)
+from ._limits import DEFAULT_MAX_CONNECTIONS, Timeouts
from ._types import TransportRequest, TransportResponse, AsyncTransportResponse
from ._sync import StdlibTransport
from ._async import StdlibAsyncTransport
+from ._fetch import FetchTransport
__all__ = [
+ "Timeouts",
+ "DEFAULT_MAX_CONNECTIONS",
"TransportRequest",
"TransportResponse",
"AsyncTransportResponse",
"StdlibTransport",
"StdlibAsyncTransport",
+ "FetchTransport",
"TransportError",
"ConnectError",
"ConnectTimeout",
diff --git a/dspy/_vendor/lm15/transports/_async.py b/dspy/_vendor/lm15/transports/_async.py
index b299a901d..2908c1435 100644
--- a/dspy/_vendor/lm15/transports/_async.py
+++ b/dspy/_vendor/lm15/transports/_async.py
@@ -11,14 +11,21 @@ Mirror of _sync.py but async end-to-end. Key differences:
CancelledError without awaiting anything that could itself be cancelled.
- Timeouts use a cancellation-safe wait helper, including on Python 3.10/3.11
where asyncio.wait_for can swallow simultaneous caller cancellation.
+- Lifetime: `aclose()` (or `async with`) closes every connection. A
+ transport collected without it closes its idle connections from a
+ finalizer; when their event loop is already gone the socket underneath
+ is closed directly, since the asyncio transport can no longer do it.
+- TLS lives in `_ssl.py`, which this module imports on the first https request
+ and not before. That module needs the stdlib `ssl`; this one does not, so a
+ CPython build without `ssl` still carries plain HTTP through here.
"""
from __future__ import annotations
import asyncio
import select
import socket
-import ssl
-from typing import AsyncIterator
+import weakref
+from typing import TYPE_CHECKING, AsyncIterator
from ._exceptions import (
ConnectError,
@@ -34,16 +41,25 @@ from ._http11 import (
ResponseHeadParser,
build_request_head,
)
+from ._limits import (
+ DEFAULT_CONNECT_TIMEOUT,
+ DEFAULT_MAX_CONNECTIONS,
+ DEFAULT_POOL_TIMEOUT,
+ DEFAULT_READ_TIMEOUT,
+ DEFAULT_WRITE_TIMEOUT,
+ check_max_connections,
+ pool_timeout_hint,
+ read_timeout_hint,
+)
from ._proxy import ProxyRoute, connect_payload, proxy_route_for, route_origin
-from ._ssl import make_ssl_context
from ._timeouts import wait_for
from ._types import AsyncTransportResponse, TransportRequest
from ._url import ParsedURL, parse_url
+if TYPE_CHECKING:
+ from ._ssl import TLS
+
-_DEFAULT_CONNECT_TIMEOUT = 10.0
-_DEFAULT_READ_TIMEOUT = 60.0
-_DEFAULT_WRITE_TIMEOUT = 60.0
_READ_CHUNK = 64 * 1024
@@ -90,10 +106,40 @@ class _AsyncConnection:
if self.closed:
return
self.closed = True
- try:
- self.writer.close()
- except Exception:
- pass
+ _close_writer(self.writer)
+
+
+def _close_writer(writer: asyncio.StreamWriter) -> None:
+ """Close a StreamWriter; when its loop is already closed (the owner was
+ collected after ``asyncio.run`` returned) asyncio cannot schedule the
+ close, so the socket underneath is closed directly.
+
+ The fallback reaches asyncio's private ``_sock`` (through the TLS
+ protocol's inner transport when there is one) and does what the
+ loop's own ``_call_connection_lost`` would have done: close it and
+ forget it, so the transport's finalizer neither warns nor double-
+ closes. Best effort by construction; a future asyncio that renames
+ the field degrades to the warning, never to an exception."""
+ try:
+ writer.close()
+ return
+ except Exception:
+ pass
+ transport = getattr(writer, "transport", None)
+ ssl_protocol = getattr(transport, "_ssl_protocol", None)
+ if ssl_protocol is not None:
+ transport = getattr(ssl_protocol, "_transport", None)
+ sock = getattr(transport, "_sock", None)
+ if sock is None:
+ return
+ try:
+ sock.close()
+ except Exception:
+ pass
+ try:
+ transport._sock = None
+ except Exception:
+ pass
# ─── Async pool ──────────────────────────────────────────────────────
@@ -108,6 +154,7 @@ class _AsyncConnectionPool:
self._slot = asyncio.Semaphore(max_connections)
self._total_opened = 0
self._closed = False
+ weakref.finalize(self, _close_idle, self._idle)
async def acquire_slot(self, timeout: float | None = None) -> None:
if self._closed:
@@ -118,7 +165,7 @@ class _AsyncConnectionPool:
try:
await wait_for(self._slot.acquire(), timeout=timeout, cancel_result=lambda _: self._slot.release())
except asyncio.TimeoutError:
- raise TransportError("timed out waiting for connection pool slot")
+ raise TransportError(pool_timeout_hint(timeout, self._max))
def release_slot(self) -> None:
self._slot.release()
@@ -172,17 +219,30 @@ class _AsyncConnectionPool:
self._in_use.clear()
+def _close_idle(idle: dict) -> None:
+ for q in list(idle.values()):
+ for conn in q:
+ conn.close()
+ idle.clear()
+
+
# ─── Transport ───────────────────────────────────────────────────────
class StdlibAsyncTransport:
+ """Async HTTP/1.1 transport on asyncio streams. Same knobs and per-
+ operation timeout semantics as :class:`StdlibTransport`. Its pool
+ belongs to the event loop that first used it; one transport per loop.
+ """
+
def __init__(
self,
*,
- connect_timeout: float = _DEFAULT_CONNECT_TIMEOUT,
- read_timeout: float = _DEFAULT_READ_TIMEOUT,
- write_timeout: float = _DEFAULT_WRITE_TIMEOUT,
- max_connections: int = 10,
+ connect_timeout: float = DEFAULT_CONNECT_TIMEOUT,
+ read_timeout: float = DEFAULT_READ_TIMEOUT,
+ write_timeout: float = DEFAULT_WRITE_TIMEOUT,
+ pool_timeout: float | None = DEFAULT_POOL_TIMEOUT,
+ max_connections: int = DEFAULT_MAX_CONNECTIONS,
verify: bool = True,
ca_bundle: str | None = None,
user_agent: str = "lm15/stdlib",
@@ -192,21 +252,45 @@ class StdlibAsyncTransport:
self._connect_timeout = connect_timeout
self._read_timeout = read_timeout
self._write_timeout = write_timeout
+ self._pool_timeout = pool_timeout
+ self._max_connections = check_max_connections(max_connections)
self._user_agent = user_agent
self._verify = verify
self._ca_bundle = ca_bundle
self._proxy = proxy
self._trust_env = trust_env
- self._ssl_ctx: ssl.SSLContext | None = None
- self._pool = _AsyncConnectionPool(max_connections)
+ self._tls: TLS | None = None
+ self._pool = _AsyncConnectionPool(self._max_connections)
self._closed = False
- def _get_ssl_ctx(self) -> ssl.SSLContext:
- if self._ssl_ctx is None:
- self._ssl_ctx = make_ssl_context(
- verify=self._verify, ca_bundle=self._ca_bundle
- )
- return self._ssl_ctx
+ @property
+ def max_connections(self) -> int:
+ return self._max_connections
+
+ def copy(self) -> "StdlibAsyncTransport":
+ """A fresh, open transport with this one's configuration and none of
+ its connections (an interactive runner closed the original)."""
+ return type(self)(
+ connect_timeout=self._connect_timeout,
+ read_timeout=self._read_timeout,
+ write_timeout=self._write_timeout,
+ pool_timeout=self._pool_timeout,
+ max_connections=self._max_connections,
+ verify=self._verify,
+ ca_bundle=self._ca_bundle,
+ user_agent=self._user_agent,
+ proxy=self._proxy,
+ trust_env=self._trust_env,
+ )
+
+ def _tls_half(self) -> "TLS":
+ """Build the TLS half on first https use. `._ssl` needs the stdlib `ssl` module,
+ which a reduced build may not have — and a plain-HTTP caller never asks for it."""
+ from ._ssl import TLS
+
+ if self._tls is None:
+ self._tls = TLS(verify=self._verify, ca_bundle=self._ca_bundle)
+ return self._tls
def pool_stats(self) -> dict:
return self._pool.stats()
@@ -240,7 +324,7 @@ class StdlibAsyncTransport:
read_timeout = request.read_timeout or self._read_timeout
write_timeout = request.write_timeout or self._write_timeout
- await self._pool.acquire_slot(timeout=connect_timeout * 5)
+ await self._pool.acquire_slot(timeout=self._pool_timeout)
slot_released = {"done": False}
def release_slot_once() -> None:
@@ -301,7 +385,9 @@ class StdlibAsyncTransport:
)
except asyncio.TimeoutError as exc:
aborted = True
- raise ReadTimeout("read timed out mid-body") from exc
+ raise ReadTimeout(
+ f"read timed out mid-body: {read_timeout_hint(read_timeout)}"
+ ) from exc
except asyncio.CancelledError:
aborted = True
raise
@@ -314,6 +400,9 @@ class StdlibAsyncTransport:
except ProtocolError:
aborted = True
raise
+ tail = decoder.drain()
+ if tail:
+ yield tail
break
try:
for out in decoder.feed(data):
@@ -356,55 +445,42 @@ class StdlibAsyncTransport:
origin: tuple[str, str, int],
connect_timeout: float,
) -> _AsyncConnection:
- ctx = self._get_ssl_ctx() if parsed.is_tls else None
+ tls = self._tls_half() if parsed.is_tls else None
- if proxy is not None and parsed.is_tls:
+ if proxy is not None and tls is not None:
# CONNECT over a raw socket, then hand it to open_connection
# for the end-to-end TLS handshake (works on 3.10; StreamWriter
# gained start_tls only in 3.11).
tunnel = await self._connect_tunnel(parsed, proxy, timeout=connect_timeout)
+ reader, writer = await tls.connect_over(
+ tunnel, server_hostname=parsed.host, timeout=connect_timeout
+ )
+ return _AsyncConnection(origin, reader, writer)
+
+ connect_host = proxy.host if proxy is not None else parsed.host
+ connect_port = proxy.port if proxy is not None else parsed.port
+ if tls is not None:
+ reader, writer = await tls.connect(
+ host=connect_host,
+ port=connect_port,
+ server_hostname=parsed.host,
+ timeout=connect_timeout,
+ )
+ else:
try:
reader, writer = await wait_for(
- asyncio.open_connection(
- sock=tunnel, ssl=ctx, server_hostname=parsed.host
- ),
+ asyncio.open_connection(host=connect_host, port=connect_port),
timeout=connect_timeout,
cancel_result=lambda pair: pair[1].close(),
)
except asyncio.TimeoutError as exc:
- tunnel.close()
- raise ConnectTimeout("TLS handshake through proxy timed out") from exc
- except ssl.SSLError as exc:
- tunnel.close()
- raise ConnectError(f"TLS handshake failed: {exc}") from exc
+ raise ConnectTimeout(
+ f"timed out connecting to {connect_host}:{connect_port}"
+ ) from exc
except OSError as exc:
- tunnel.close()
- raise ConnectError(f"TLS handshake through proxy failed: {exc}") from exc
- return _AsyncConnection(origin, reader, writer)
-
- connect_host = proxy.host if proxy is not None else parsed.host
- connect_port = proxy.port if proxy is not None else parsed.port
- try:
- reader, writer = await wait_for(
- asyncio.open_connection(
- host=connect_host,
- port=connect_port,
- ssl=ctx,
- server_hostname=parsed.host if ctx else None,
- ),
- timeout=connect_timeout,
- cancel_result=lambda pair: pair[1].close(),
- )
- except asyncio.TimeoutError as exc:
- raise ConnectTimeout(
- f"timed out connecting to {connect_host}:{connect_port}"
- ) from exc
- except ssl.SSLError as exc:
- raise ConnectError(f"TLS handshake failed: {exc}") from exc
- except OSError as exc:
- raise ConnectError(
- f"failed to connect to {connect_host}:{connect_port}: {exc}"
- ) from exc
+ raise ConnectError(
+ f"failed to connect to {connect_host}:{connect_port}: {exc}"
+ ) from exc
# TCP_NODELAY on the underlying socket
sock = writer.get_extra_info("socket")
@@ -526,7 +602,9 @@ class StdlibAsyncTransport:
conn.reader.read(_READ_CHUNK), timeout=read_timeout
)
except asyncio.TimeoutError as exc:
- raise ReadTimeout("read timed out waiting for headers") from exc
+ raise ReadTimeout(
+ f"read timed out waiting for headers: {read_timeout_hint(read_timeout)}"
+ ) from exc
except OSError as exc:
raise ReadError(f"read failed: {exc}") from exc
if not data:
diff --git a/dspy/_vendor/lm15/transports/_fetch.py b/dspy/_vendor/lm15/transports/_fetch.py
new file mode 100644
index 000000000..c0470d6b9
--- /dev/null
+++ b/dspy/_vendor/lm15/transports/_fetch.py
@@ -0,0 +1,155 @@
+"""The async transport over the host's ``fetch``: lm15 in a page.
+
+Under Pyodide — CPython compiled to WebAssembly, in a browser page, a
+worker, or Node — there is no socket, so :class:`StdlibAsyncTransport`
+cannot open one. The host has ``fetch``. :class:`FetchTransport` is the
+same :class:`AsyncTransport` protocol over it: the same
+``AsyncTransportResponse``, chunk by chunk as the body arrives, cancelled
+by dropping the response before its end. Every adapter's async mirror
+(``AsyncOpenAIChatLM``, ``AsyncAnthropicLM``, …) takes it as
+``transport=`` and is otherwise unchanged: the request bytes are the
+ones the stdlib transport would send.
+
+What a page cannot do is not hidden here: a cross-origin request goes
+only where the server's CORS headers allow it (the browser refuses
+before any bytes leave, and ``fetch`` reports a bare ``TypeError`` with
+no status — surfaced as :class:`TransportError` naming that
+possibility); a separate connect timeout does not exist (``fetch`` has
+one signal); there is no proxy configuration. The TypeScript port's
+``docs/browser.md`` states the same line.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from typing import Any, AsyncIterator
+
+from ._exceptions import TransportError
+from ._types import AsyncTransportResponse, TransportRequest
+
+try: # the host bridge exists only under Pyodide
+ import js as _js # type: ignore[import-not-found]
+ from pyodide.ffi import JsProxy, to_js # type: ignore[import-not-found]
+except ImportError: # pragma: no cover - exercised under CPython only by the import guard test
+ _js = None
+ JsProxy = None # type: ignore[assignment]
+ to_js = None # type: ignore[assignment]
+
+from ._limits import DEFAULT_READ_TIMEOUT, read_timeout_hint
+
+
+def _require_pyodide() -> None:
+ if _js is None:
+ raise TransportError(
+ "FetchTransport runs under Pyodide (a page, a worker, or Node hosting Pyodide); "
+ "on CPython use StdlibAsyncTransport"
+ )
+
+
+class FetchTransport:
+ """The :class:`AsyncTransport` over the host's ``fetch``.
+
+ ``fetch`` may be given (a test double, a wrapped ``fetch``); by default
+ it is the host's global. ``read_timeout`` bounds the wait for each body
+ chunk, as the stdlib transport's does; a request's own
+ ``read_timeout`` overrides it.
+ """
+
+ def __init__(self, *, fetch: Any = None, read_timeout: float = DEFAULT_READ_TIMEOUT) -> None:
+ _require_pyodide()
+ self._fetch = fetch if fetch is not None else _js.fetch
+ self._read_timeout = read_timeout
+ self._closed = False
+
+ def stream(self, request: TransportRequest) -> "_FetchStreamCM":
+ return _FetchStreamCM(self, request)
+
+ async def aclose(self) -> None:
+ self._closed = True
+
+ async def _do_stream(self, request: TransportRequest) -> AsyncTransportResponse:
+ if self._closed:
+ raise TransportError("transport is closed")
+ if request.connect_timeout is not None:
+ raise TransportError("fetch has no separate connect timeout; leave connect_timeout unset")
+ controller = _js.AbortController.new()
+ headers = _js.Headers.new()
+ for name, value in request.headers:
+ headers.append(name, value)
+ options: dict[str, Any] = {"method": request.method, "headers": headers, "signal": controller.signal}
+ if request.body:
+ options["body"] = to_js(request.body)
+ try:
+ response = await self._fetch(request.url, to_js(options, dict_converter=_js.Object.fromEntries))
+ except Exception as exc: # a JS TypeError: network, or the browser's CORS refusal — fetch does not say which
+ raise TransportError(
+ f"{request.method} {request.url.split('?', 1)[0]}: fetch failed ({_message(exc)}); "
+ "a browser reports a CORS refusal and a network failure the same way"
+ ) from exc
+
+ header_pairs: list[tuple[str, str]] = []
+ for pair in response.headers.entries():
+ header_pairs.append((str(pair[0]), str(pair[1])))
+ read_timeout = request.read_timeout or self._read_timeout
+ reader = response.body.getReader() if response.body is not None else None
+ state = {"consumed": False}
+
+ async def chunks() -> AsyncIterator[bytes]:
+ if reader is None:
+ state["consumed"] = True
+ return
+ while True:
+ try:
+ result = await asyncio.wait_for(reader.read(), read_timeout)
+ except asyncio.TimeoutError as exc:
+ controller.abort()
+ raise TransportError(f"response body read timed out: {read_timeout_hint(read_timeout)}") from exc
+ except Exception as exc:
+ raise TransportError(f"response body read failed ({_message(exc)})") from exc
+ if result.done:
+ state["consumed"] = True
+ return
+ yield bytes(result.value.to_py())
+
+ async def release(body_consumed: bool) -> None:
+ # Dropping the response before its end cancels the request: the
+ # socket closes, the server sees it (the browser smoke checks this).
+ if not (body_consumed or state["consumed"]):
+ controller.abort()
+ if reader is not None:
+ try:
+ await reader.cancel()
+ except Exception:
+ pass
+
+ return AsyncTransportResponse(
+ status=int(response.status),
+ reason=str(response.statusText or ""),
+ headers=header_pairs,
+ http_version="HTTP/1.1", # fetch does not expose the negotiated version
+ chunks=chunks(),
+ release=release,
+ )
+
+
+class _FetchStreamCM:
+ def __init__(self, transport: FetchTransport, request: TransportRequest) -> None:
+ self._transport = transport
+ self._request = request
+ self._response: AsyncTransportResponse | None = None
+
+ def __await__(self):
+ return self._transport._do_stream(self._request).__await__()
+
+ async def __aenter__(self) -> AsyncTransportResponse:
+ self._response = await self._transport._do_stream(self._request)
+ return self._response
+
+ async def __aexit__(self, exc_type, exc, tb) -> None:
+ if self._response is not None:
+ await self._response.aclose()
+
+
+def _message(exc: BaseException) -> str:
+ text = str(exc).strip()
+ return text.splitlines()[0] if text else type(exc).__name__
diff --git a/dspy/_vendor/lm15/transports/_http11.py b/dspy/_vendor/lm15/transports/_http11.py
index 43713b3f7..ada132ed2 100644
--- a/dspy/_vendor/lm15/transports/_http11.py
+++ b/dspy/_vendor/lm15/transports/_http11.py
@@ -8,9 +8,13 @@ decoded body chunks + a completion flag.
This is deliberately minimal:
- HTTP/1.1 only (no HTTP/2, no HTTP/1.0 write side)
- - No Content-Encoding handling (gzip/deflate/br/zstd) — LLM APIs don't
- compress responses in practice, and we'd rather not pull in zlib paths
- until we need them. Set Accept-Encoding: identity to be explicit.
+ - Requests advertise ``Accept-Encoding: identity`` (a compressed SSE
+ stream sits in a proxy's buffer until the window fills, which defeats
+ streaming). Responses that arrive compressed anyway — gateways and
+ CDNs ignore the header routinely — are decoded: ``gzip``/``x-gzip``
+ and ``deflate`` through the stdlib ``zlib``, incrementally, so a
+ compressed stream still streams. ``br`` and ``zstd`` have no stdlib
+ codec and raise a ProtocolError that names the coding.
- No trailers, no pipelining, no 100-continue.
- Chunk extensions (RFC 7230 §4.1.1) are parsed and discarded.
"""
@@ -180,7 +184,17 @@ class ResponseHeadParser:
# ─── Framing decision ───
def body_decoder(self, request_method: str) -> "_BodyDecoder":
- """Pick the right decoder based on headers (RFC 7230 §3.3.3)."""
+ """Pick the right decoder based on headers (RFC 7230 §3.3.3), wrapped
+ in a content decoder when the reply is compressed."""
+ framing = self._framing_decoder(request_method)
+ if framing.complete:
+ return framing
+ codings = content_codings(self.headers_all("content-encoding"))
+ if not codings:
+ return framing
+ return ContentDecoder(framing, codings)
+
+ def _framing_decoder(self, request_method: str) -> "_BodyDecoder":
# No body on 1xx, 204, 304, or HEAD
if (
self.status // 100 == 1
@@ -230,7 +244,8 @@ class ResponseHeadParser:
class _BodyDecoder:
- """Interface: feed(data) -> iter of decoded chunks, eof() to signal close."""
+ """Interface: feed(data) -> iter of decoded chunks, eof() to signal close,
+ drain() for any bytes an EOF-delimited body releases at close."""
complete: bool
leftover: bytes
@@ -240,6 +255,10 @@ class _BodyDecoder:
def eof(self) -> None:
raise NotImplementedError
+ def drain(self) -> bytes:
+ """Bytes released by eof() (a content decoder's final flush)."""
+ return b""
+
class NoBodyDecoder(_BodyDecoder):
def __init__(self) -> None:
@@ -414,3 +433,172 @@ class EOFDecoder(_BodyDecoder):
def eof(self) -> None:
self.complete = True
+
+
+# ─── Content decoding (Content-Encoding) ─────────────────────────────
+
+
+_SUPPORTED_CODINGS = frozenset({"gzip", "x-gzip", "deflate"})
+
+
+def content_codings(values: list[str]) -> list[str]:
+ """The codings applied to a body, in the order they must be UNDONE
+ (RFC 9110 §8.4: the header lists them in the order applied, so the
+ last one listed is the outermost). ``identity`` is a no-op and is
+ dropped. Anything lm15 cannot undo raises a ProtocolError naming it —
+ the alternative is handing compressed bytes to a JSON parser, which
+ surfaces as a baffling decode error far from the cause."""
+ codings: list[str] = []
+ for value in values:
+ for token in value.split(","):
+ coding = token.strip().lower()
+ if not coding or coding == "identity":
+ continue
+ if coding not in _SUPPORTED_CODINGS:
+ raise ProtocolError(
+ f"response is Content-Encoding: {coding!r}, which lm15 cannot decode "
+ "(it asked for identity; gzip and deflate are decoded, br and zstd are not)"
+ )
+ codings.append(coding)
+ codings.reverse()
+ return codings
+
+
+class _Inflater:
+ """One content coding, possibly containing several gzip members.
+
+ Deflate normally has a two-byte zlib header (RFC 1950). Wait for
+ both bytes before selecting wrapped or legacy raw deflate, regardless
+ of how the transport split them. Never reinterpret a checksum failure
+ as another format after output has already been delivered.
+ """
+
+ __slots__ = ("_coding", "_obj", "_started", "_prefix")
+
+ def __init__(self, coding: str) -> None:
+ import zlib
+
+ self._coding = coding
+ self._started = False
+ self._prefix = b""
+ self._obj = None if coding == "deflate" else zlib.decompressobj(16 + zlib.MAX_WBITS)
+
+ def decompress(self, data: bytes) -> bytes:
+ import zlib
+
+ if not data:
+ return b""
+ self._started = True
+ if self._obj is None:
+ needed = 2 - len(self._prefix)
+ self._prefix += data[:needed]
+ data = data[needed:]
+ if len(self._prefix) < 2:
+ return b""
+ cmf, flg = self._prefix
+ wrapped = (cmf & 0x0F) == 8 and (cmf >> 4) <= 7 and ((cmf << 8) | flg) % 31 == 0
+ self._obj = zlib.decompressobj(zlib.MAX_WBITS if wrapped else -zlib.MAX_WBITS)
+ data = self._prefix + data
+ self._prefix = b""
+
+ output: list[bytes] = []
+ while data:
+ if self._obj.eof:
+ if self._coding == "deflate":
+ raise ProtocolError("malformed deflate body: data after the compressed stream")
+ # Match stdlib gzip's allowance for zero padding between
+ # members or after the last member. Other trailing bytes
+ # must form a valid member; they are never silently lost.
+ data = data.lstrip(b"\x00")
+ if not data:
+ break
+ self._obj = zlib.decompressobj(16 + zlib.MAX_WBITS)
+ try:
+ output.append(self._obj.decompress(data))
+ except zlib.error as exc:
+ raise ProtocolError(f"malformed {self._coding} body: {exc}") from exc
+ # zlib stops at ONE gzip member. Carry the remaining bytes
+ # into a fresh member decoder, including a split next header.
+ data = self._obj.unused_data
+ return b"".join(output)
+
+ def finish(self) -> bytes:
+ import zlib
+
+ if not self._started:
+ return b""
+ if self._obj is None or not self._obj.eof:
+ raise ProtocolError(
+ f"{self._coding} body ended before the compressed stream did (truncated)"
+ )
+ try:
+ return self._obj.flush()
+ except zlib.error as exc:
+ raise ProtocolError(f"malformed {self._coding} body: {exc}") from exc
+
+
+class ContentDecoder(_BodyDecoder):
+ """Undo Content-Encoding on top of a framing decoder.
+
+ The framing decoder (chunked / content-length / to-EOF) decides where
+ the body ends; this decoder turns each framed chunk back into the
+ bytes the origin produced. Incremental, so a compressed SSE stream
+ still yields events as they arrive. ``complete`` and ``leftover``
+ mirror the framing decoder's — the framing decides connection reuse,
+ not the compression.
+ """
+
+ def __init__(self, framing: _BodyDecoder, codings: list[str]) -> None:
+ self._framing = framing
+ self._inflaters = [_Inflater(c) for c in codings]
+ self._finished = False
+ self._drain = b""
+
+ @property
+ def complete(self) -> bool: # type: ignore[override]
+ return self._framing.complete
+
+ @property
+ def leftover(self) -> bytes: # type: ignore[override]
+ return self._framing.leftover
+
+ def _inflate(self, data: bytes) -> bytes:
+ for inflater in self._inflaters:
+ data = inflater.decompress(data)
+ if not data:
+ return b""
+ return data
+
+ def _finish(self) -> bytes:
+ if self._finished:
+ return b""
+ self._finished = True
+ # Flush each layer and push the tail through the layers above it.
+ out = b""
+ for index, inflater in enumerate(self._inflaters):
+ tail = inflater.finish()
+ for outer in self._inflaters[index + 1:]:
+ tail = outer.decompress(tail)
+ out += tail
+ return out
+
+ def feed(self, data: bytes) -> Iterator[bytes]:
+ for framed in self._framing.feed(data):
+ out = self._inflate(framed)
+ if out:
+ yield out
+ if self._framing.complete:
+ tail = self._finish()
+ if tail:
+ yield tail
+
+ def eof(self) -> None:
+ self._framing.eof()
+ # An EOF-delimited body finishes here (a framed one already did in
+ # feed). The flush may release bytes; the transport collects them
+ # through drain() since there is no further feed call to carry them.
+ self._drain += self._finish()
+
+ def drain(self) -> bytes:
+ out, self._drain = self._drain, b""
+ return out
diff --git a/dspy/_vendor/lm15/transports/_limits.py b/dspy/_vendor/lm15/transports/_limits.py
new file mode 100644
index 000000000..f6ff85b36
--- /dev/null
+++ b/dspy/_vendor/lm15/transports/_limits.py
@@ -0,0 +1,83 @@
+"""
+Timeouts and connection limits — the two knobs every transport takes.
+
+Defaults follow the provider SDKs, not general-purpose HTTP clients: a
+model call that thinks for three minutes before its first byte is normal,
+and a client that gives up at 60 seconds turns that into a "network
+failure" and, under a retry loop, restarts the generation each time.
+OpenAI, Anthropic and litellm all wait 600 seconds by default; lm15 does
+the same. Connect stays short (10 s): a host that does not answer a SYN
+in ten seconds is down or unreachable, and a long wait there only delays
+the error.
+
+Every timeout is per operation, not per request: ``read`` bounds the wait
+for the next byte, so a slow stream that keeps trickling never times out,
+and a stalled one fails at ``read`` seconds after its last byte.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+DEFAULT_CONNECT_TIMEOUT = 10.0
+DEFAULT_READ_TIMEOUT = 600.0
+DEFAULT_WRITE_TIMEOUT = 600.0
+DEFAULT_POOL_TIMEOUT = 600.0
+DEFAULT_MAX_CONNECTIONS = 100
+
+
+@dataclass(frozen=True, slots=True)
+class Timeouts:
+ """How long to wait, in seconds, at each step of a request.
+
+ - ``connect``: TCP + TLS (and a proxy CONNECT) to the host.
+ - ``read``: the next byte of the reply — headers first, then each body
+ chunk. A model that is still thinking sends nothing, so this is the
+ one to raise for slow local models and long non-streaming answers.
+ - ``write``: sending the request bytes.
+ - ``pool``: a free connection when ``max_connections`` are all busy.
+ ``None`` waits as long as it takes.
+
+ Every value must be positive (or ``None`` for ``pool``); a zero or
+ negative timeout is a configuration mistake, not "no timeout".
+ """
+
+ connect: float = DEFAULT_CONNECT_TIMEOUT
+ read: float = DEFAULT_READ_TIMEOUT
+ write: float = DEFAULT_WRITE_TIMEOUT
+ pool: float | None = DEFAULT_POOL_TIMEOUT
+
+ def __post_init__(self) -> None:
+ for name in ("connect", "read", "write"):
+ _check_positive(name, getattr(self, name))
+ if self.pool is not None:
+ _check_positive("pool", self.pool)
+
+
+def _check_positive(name: str, value: object) -> None:
+ if isinstance(value, bool) or not isinstance(value, (int, float)) or not value > 0:
+ raise ValueError(
+ f"Timeouts.{name} must be a positive number of seconds, got {value!r}"
+ )
+
+
+def check_max_connections(value: object) -> int:
+ if isinstance(value, bool) or not isinstance(value, int) or value < 1:
+ raise ValueError(f"max_connections must be a positive integer, got {value!r}")
+ return value
+
+
+def read_timeout_hint(seconds: float) -> str:
+ """The tail of a read-timeout message: says whose limit fired and how
+ to move it. A user who cannot tell a client timeout from a dead
+ server retries the wrong thing."""
+ return (
+ f"no bytes for {seconds:g}s; this is lm15's read timeout, not a server "
+ f"failure — raise it with Timeouts(read=...) if the model needs longer"
+ )
+
+
+def pool_timeout_hint(seconds: float, max_connections: int) -> str:
+ return (
+ f"all {max_connections} connections were busy for {seconds:g}s; raise "
+ f"max_connections, or Timeouts(pool=...) to wait longer"
+ )
diff --git a/dspy/_vendor/lm15/transports/_ssl.py b/dspy/_vendor/lm15/transports/_ssl.py
index 4b213015b..b9ed10625 100644
--- a/dspy/_vendor/lm15/transports/_ssl.py
+++ b/dspy/_vendor/lm15/transports/_ssl.py
@@ -1,22 +1,110 @@
-"""SSL context factory.
+"""TLS for the stdlib transports: one context, and the ways to put it on a socket.
-We rely on the stdlib `ssl` module's `create_default_context`, which on
-Python 3.10+ loads the system trust store correctly on Linux/macOS/Windows.
-No certifi bundle is shipped — set SSL_CERT_FILE if your system store is
-broken, or pass an explicit ca_bundle= to the transport.
+The socket transports load this module only when a request names an HTTPS URL.
+This keeps plain HTTP usable on Python builds without ``ssl``. If such a build
+does request HTTPS, :class:`TLS` raises a transport error that names the host's
+fetch transport as the Pyodide alternative.
+
+We rely on the stdlib ``create_default_context``, which on Python 3.10+ loads
+the system trust store correctly on Linux, macOS, and Windows. No certifi bundle
+is shipped: set ``SSL_CERT_FILE`` if the system store is broken, or pass an
+explicit ``ca_bundle=`` to the transport.
"""
from __future__ import annotations
-import ssl
+import asyncio
+import socket
+
+try:
+ import ssl
+except ImportError: # pragma: no cover - Pyodide and reduced Python builds
+ ssl = None # type: ignore[assignment]
+
+from ._exceptions import ConnectError, ConnectTimeout
+from ._timeouts import wait_for
+
+
+def _close(sock: socket.socket) -> None:
+ try:
+ sock.close()
+ except Exception:
+ pass
+
+
+def _is_ssl_error(exc: OSError) -> bool:
+ return ssl is not None and isinstance(exc, ssl.SSLError)
+
+
+class TLS:
+ """The TLS half of a transport, holding the context its connections share."""
+
+ __slots__ = ("_ctx",)
+
+ def __init__(self, *, verify: bool = True, ca_bundle: str | None = None) -> None:
+ if ssl is None: # pragma: no cover - Pyodide
+ raise ConnectError(
+ "this Python has no ssl module (Pyodide?), so the socket transports "
+ "cannot open TLS; use lm15.transports.FetchTransport, the host's fetch"
+ )
+ if not verify:
+ self._ctx = ssl._create_unverified_context()
+ return
+ self._ctx = ssl.create_default_context()
+ if ca_bundle:
+ self._ctx.load_verify_locations(cafile=ca_bundle)
+
+ def wrap(
+ self, sock: socket.socket, *, server_hostname: str, timeout: float
+ ) -> socket.socket:
+ """Return ``sock`` with TLS, or close it and describe the handshake failure."""
+ try:
+ sock.settimeout(timeout)
+ return self._ctx.wrap_socket(sock, server_hostname=server_hostname)
+ except OSError as exc:
+ _close(sock)
+ raise ConnectError(f"TLS handshake failed: {exc}") from exc
+ async def connect(
+ self, *, host: str, port: int, server_hostname: str, timeout: float
+ ) -> tuple[asyncio.StreamReader, asyncio.StreamWriter]:
+ """Open a TCP connection and run its TLS handshake in one asyncio call."""
+ try:
+ return await wait_for(
+ asyncio.open_connection(
+ host=host,
+ port=port,
+ ssl=self._ctx,
+ server_hostname=server_hostname,
+ ),
+ timeout=timeout,
+ cancel_result=lambda pair: pair[1].close(),
+ )
+ except asyncio.TimeoutError as exc:
+ raise ConnectTimeout(f"timed out connecting to {host}:{port}") from exc
+ except OSError as exc:
+ if _is_ssl_error(exc):
+ raise ConnectError(f"TLS handshake failed: {exc}") from exc
+ raise ConnectError(f"failed to connect to {host}:{port}: {exc}") from exc
-def make_ssl_context(
- *, verify: bool = True, ca_bundle: str | None = None
-) -> ssl.SSLContext:
- if not verify:
- ctx = ssl._create_unverified_context()
- return ctx
- ctx = ssl.create_default_context()
- if ca_bundle:
- ctx.load_verify_locations(cafile=ca_bundle)
- return ctx
+ async def connect_over(
+ self, sock: socket.socket, *, server_hostname: str, timeout: float
+ ) -> tuple[asyncio.StreamReader, asyncio.StreamWriter]:
+ """Run TLS over a socket that a proxy has already tunneled to the origin."""
+ try:
+ return await wait_for(
+ asyncio.open_connection(
+ sock=sock,
+ ssl=self._ctx,
+ server_hostname=server_hostname,
+ ),
+ timeout=timeout,
+ cancel_result=lambda pair: pair[1].close(),
+ )
+ except asyncio.TimeoutError as exc:
+ _close(sock)
+ raise ConnectTimeout("TLS handshake through proxy timed out") from exc
+ except OSError as exc:
+ _close(sock)
+ if _is_ssl_error(exc):
+ raise ConnectError(f"TLS handshake failed: {exc}") from exc
+ raise ConnectError(f"TLS handshake through proxy failed: {exc}") from exc
diff --git a/dspy/_vendor/lm15/transports/_sync.py b/dspy/_vendor/lm15/transports/_sync.py
index d943453f3..431c593bb 100644
--- a/dspy/_vendor/lm15/transports/_sync.py
+++ b/dspy/_vendor/lm15/transports/_sync.py
@@ -1,5 +1,5 @@
"""
-Sync transport built on the stdlib `socket` + `ssl` modules.
+Sync transport built on the stdlib `socket` module.
Design:
@@ -15,18 +15,25 @@ Design:
socket, readable means the server sent EOF — so we drop it and open fresh.
- If the server sent `Connection: close`, or the body is still outstanding
when the response closes, the connection is closed rather than reused.
+- TLS lives in `_ssl.py`, which this module imports on the first https request
+ and not before. That module needs the stdlib `ssl`; this one does not, so a
+ CPython build without `ssl` still carries plain HTTP through here.
- Proxies: `proxy=` pins one explicitly; otherwise `trust_env=True` (default)
honors HTTP_PROXY / HTTPS_PROXY / ALL_PROXY / NO_PROXY. Plain-HTTP targets
are forwarded absolute-URI; TLS targets are tunneled with CONNECT and the
handshake runs end-to-end to the origin (see `_proxy.py`).
+- Lifetime: `close()` (or `with`) closes every socket. A transport that is
+ garbage-collected without it closes its idle sockets from a finalizer so
+ a forgotten short-lived client does not leak file descriptors or warn
+ under `-W error`; sockets still in use belong to their open response.
"""
from __future__ import annotations
import select
import socket
-import ssl
import threading
-from typing import Iterator
+import weakref
+from typing import TYPE_CHECKING, Iterator
from ._exceptions import (
ConnectError,
@@ -45,15 +52,24 @@ from ._http11 import (
ResponseHeadParser,
build_request_head,
)
+from ._limits import (
+ DEFAULT_CONNECT_TIMEOUT,
+ DEFAULT_MAX_CONNECTIONS,
+ DEFAULT_POOL_TIMEOUT,
+ DEFAULT_READ_TIMEOUT,
+ DEFAULT_WRITE_TIMEOUT,
+ check_max_connections,
+ pool_timeout_hint,
+ read_timeout_hint,
+)
from ._proxy import ProxyRoute, connect_payload, proxy_route_for, route_origin
-from ._ssl import make_ssl_context
from ._types import TransportRequest, TransportResponse
from ._url import ParsedURL, parse_url
+if TYPE_CHECKING:
+ from ._ssl import TLS
+
-_DEFAULT_CONNECT_TIMEOUT = 10.0
-_DEFAULT_READ_TIMEOUT = 60.0
-_DEFAULT_WRITE_TIMEOUT = 60.0
_READ_CHUNK = 64 * 1024
@@ -115,13 +131,17 @@ class _ConnectionPool:
self._slot = threading.Semaphore(max_connections)
self._total_opened = 0
self._closed = False
+ # Close idle sockets when the pool itself is collected. A finalizer
+ # (not __del__) so nothing here runs during interpreter teardown
+ # with half-dead globals; it captures the idle dict, never the pool.
+ weakref.finalize(self, _close_idle, self._idle)
def acquire_slot(self, timeout: float | None = None) -> None:
if self._closed:
raise TransportError("transport is closed")
ok = self._slot.acquire(timeout=timeout)
if not ok:
- raise TransportError("timed out waiting for connection pool slot")
+ raise TransportError(pool_timeout_hint(timeout or 0.0, self._max))
def release_slot(self) -> None:
self._slot.release()
@@ -181,19 +201,33 @@ class _ConnectionPool:
self._in_use.clear()
+def _close_idle(idle: dict) -> None:
+ for q in list(idle.values()):
+ for conn in q:
+ conn.close()
+ idle.clear()
+
+
# ─── Transport ───────────────────────────────────────────────────────
class StdlibTransport:
- """Sync HTTP/1.1 transport using only the Python standard library."""
+ """Sync HTTP/1.1 transport using only the Python standard library.
+
+ Timeouts are per operation (``_limits.py``): ``read_timeout`` bounds the
+ wait for the NEXT byte, so a streaming reply that keeps arriving never
+ trips it. ``pool_timeout`` bounds the wait for a free connection when
+ ``max_connections`` are busy; ``None`` waits indefinitely.
+ """
def __init__(
self,
*,
- connect_timeout: float = _DEFAULT_CONNECT_TIMEOUT,
- read_timeout: float = _DEFAULT_READ_TIMEOUT,
- write_timeout: float = _DEFAULT_WRITE_TIMEOUT,
- max_connections: int = 10,
+ connect_timeout: float = DEFAULT_CONNECT_TIMEOUT,
+ read_timeout: float = DEFAULT_READ_TIMEOUT,
+ write_timeout: float = DEFAULT_WRITE_TIMEOUT,
+ pool_timeout: float | None = DEFAULT_POOL_TIMEOUT,
+ max_connections: int = DEFAULT_MAX_CONNECTIONS,
verify: bool = True,
ca_bundle: str | None = None,
user_agent: str = "lm15/stdlib",
@@ -203,23 +237,47 @@ class StdlibTransport:
self._connect_timeout = connect_timeout
self._read_timeout = read_timeout
self._write_timeout = write_timeout
+ self._pool_timeout = pool_timeout
+ self._max_connections = check_max_connections(max_connections)
self._user_agent = user_agent
self._verify = verify
self._ca_bundle = ca_bundle
self._proxy = proxy
self._trust_env = trust_env
- self._ssl_ctx: ssl.SSLContext | None = None
- self._ssl_lock = threading.Lock()
- self._pool = _ConnectionPool(max_connections)
+ self._tls: TLS | None = None
+ self._tls_lock = threading.Lock()
+ self._pool = _ConnectionPool(self._max_connections)
self._closed = False
- def _get_ssl_ctx(self) -> ssl.SSLContext:
- with self._ssl_lock:
- if self._ssl_ctx is None:
- self._ssl_ctx = make_ssl_context(
- verify=self._verify, ca_bundle=self._ca_bundle
- )
- return self._ssl_ctx
+ @property
+ def max_connections(self) -> int:
+ return self._max_connections
+
+ def copy(self) -> "StdlibTransport":
+ """A fresh, open transport with this one's configuration and none of
+ its connections (an interactive runner closed the original)."""
+ return type(self)(
+ connect_timeout=self._connect_timeout,
+ read_timeout=self._read_timeout,
+ write_timeout=self._write_timeout,
+ pool_timeout=self._pool_timeout,
+ max_connections=self._max_connections,
+ verify=self._verify,
+ ca_bundle=self._ca_bundle,
+ user_agent=self._user_agent,
+ proxy=self._proxy,
+ trust_env=self._trust_env,
+ )
+
+ def _tls_half(self) -> "TLS":
+ """Build the TLS half on first https use. `._ssl` needs the stdlib `ssl` module,
+ which a reduced build may not have — and a plain-HTTP caller never asks for it."""
+ from ._ssl import TLS
+
+ with self._tls_lock:
+ if self._tls is None:
+ self._tls = TLS(verify=self._verify, ca_bundle=self._ca_bundle)
+ return self._tls
def pool_stats(self) -> dict:
return self._pool.stats()
@@ -256,7 +314,7 @@ class StdlibTransport:
write_timeout = request.write_timeout or self._write_timeout
# Acquire a pool slot FIRST (may block if we're at max_connections)
- self._pool.acquire_slot(timeout=connect_timeout * 5)
+ self._pool.acquire_slot(timeout=self._pool_timeout)
slot_released = False
def release_slot_once() -> None:
@@ -337,6 +395,10 @@ class StdlibTransport:
origin: tuple[str, str, int],
connect_timeout: float,
) -> _SyncConnection:
+ # Built before the socket, as on the async side: a missing `ssl` or an unreadable
+ # ca_bundle then fails with nothing yet open.
+ tls = self._tls_half() if parsed.is_tls else None
+
connect_host = proxy.host if proxy is not None else parsed.host
connect_port = proxy.port if proxy is not None else parsed.port
try:
@@ -367,17 +429,8 @@ class StdlibTransport:
pass
raise
- if parsed.is_tls:
- try:
- ctx = self._get_ssl_ctx()
- sock.settimeout(connect_timeout)
- sock = ctx.wrap_socket(sock, server_hostname=parsed.host)
- except (ssl.SSLError, OSError) as exc:
- try:
- sock.close()
- except Exception:
- pass
- raise ConnectError(f"TLS handshake failed: {exc}") from exc
+ if tls is not None:
+ sock = tls.wrap(sock, server_hostname=parsed.host, timeout=connect_timeout)
return _SyncConnection(origin, sock)
@@ -463,7 +516,9 @@ class StdlibTransport:
try:
data = conn.sock.recv(_READ_CHUNK)
except socket.timeout as exc:
- raise ReadTimeout(f"read timed out waiting for headers: {exc}") from exc
+ raise ReadTimeout(
+ f"read timed out waiting for headers: {read_timeout_hint(read_timeout)}"
+ ) from exc
except OSError as exc:
raise ReadError(f"read failed: {exc}") from exc
if not data:
@@ -499,7 +554,9 @@ class StdlibTransport:
data = conn.sock.recv(_READ_CHUNK)
except socket.timeout as exc:
aborted = True
- raise ReadTimeout(f"read timed out mid-body: {exc}") from exc
+ raise ReadTimeout(
+ f"read timed out mid-body: {read_timeout_hint(read_timeout)}"
+ ) from exc
except OSError as exc:
aborted = True
raise ReadError(f"read failed: {exc}") from exc
@@ -510,6 +567,9 @@ class StdlibTransport:
except ProtocolError:
aborted = True
raise
+ tail = decoder.drain()
+ if tail:
+ yield tail
break
try:
for out in decoder.feed(data):
diff --git a/dspy/_vendor/lm15/types.py b/dspy/_vendor/lm15/types.py
index 42da49327..f941f5257 100644
--- a/dspy/_vendor/lm15/types.py
+++ b/dspy/_vendor/lm15/types.py
@@ -51,6 +51,8 @@ from os import PathLike
from pathlib import Path
from typing import Any, Literal, TypeAlias, TypeVar, get_args
+from .adaptation import Adaptation
+
# ─── Literal vocabularies ────────────────────────────────────────────
@@ -97,6 +99,7 @@ ErrorCode = Literal[
"transport",
"lock_timeout",
"stream_assembly",
+ "collection_limit",
"provider",
]
ERROR_CODES = frozenset(get_args(ErrorCode))
@@ -194,6 +197,15 @@ def _validate_json_field(obj: object, field_name: str, *, required: bool = False
_check_json_object(getattr(obj, field_name), field_name=field_name, required=required)
+def _validate_adaptations(obj: object, field_name: str) -> None:
+ value = getattr(obj, "adaptations")
+ if isinstance(value, list):
+ value = tuple(value)
+ object.__setattr__(obj, "adaptations", value)
+ if not isinstance(value, tuple) or not all(isinstance(a, Adaptation) for a in value):
+ raise TypeError(f"{field_name} must be a tuple of Adaptation")
+
+
def _validate_extensions_field(obj: object) -> None:
"""Validate extensions, normalizing an empty mapping to None."""
if getattr(obj, "extensions") == {}:
@@ -1249,13 +1261,19 @@ class TextDelta:
``logprobs`` carries the token logprobs for exactly the tokens in this
fragment, when the request asked for them (``Config.logprobs``) and the
provider streams them per chunk (verified live for OpenAI Responses,
- 2026-09-01). Materialization concatenates fragment logprobs in arrival
- order into ``Response.logprobs``.
+ 2026-09-01). Materialization concatenates fragment logprobs in arrival
+ order into ``Response.logprobs``. ``logprobs_complete=False`` means
+ local text editing left some retained text without its original score
+ (for example, a stop inside a token). Scores always describe original,
+ whole provider tokens. True does not promise the provider supplied any.
"""
text: str
part_index: int = 0
logprobs: tuple[TokenLogprob, ...] = ()
+ # False when local text editing removed scores that cannot describe
+ # the retained text. True does not promise the provider supplied scores.
+ logprobs_complete: bool = field(default=True, kw_only=True)
type: Literal["text"] = field(default="text", init=False)
def __post_init__(self) -> None:
@@ -1263,6 +1281,7 @@ class TextDelta:
_validate_part_index(self.part_index)
_validate_text(self.text, field_name="TextDelta.text")
_validate_logprobs_field(self, "logprobs", allow_none=False)
+ _validate_bool(self.logprobs_complete, field_name="TextDelta.logprobs_complete")
@dataclass(frozen=True, slots=True)
@@ -1487,11 +1506,15 @@ class StreamStartEvent:
id: str | None = None
model: str | None = None
+ # MAP-13: what the wire got that differs from what was asked, known
+ # before the first byte and so carried by the first event.
+ adaptations: tuple[Adaptation, ...] = ()
type: Literal["start"] = field(default="start", init=False)
def __post_init__(self) -> None:
_validate_optional_text(self.id, field_name="StreamStartEvent.id", allow_empty=False)
_validate_optional_text(self.model, field_name="StreamStartEvent.model", allow_empty=False)
+ _validate_adaptations(self, "StreamStartEvent.adaptations")
@dataclass(frozen=True, slots=True)
@@ -1758,6 +1781,9 @@ class Config:
"""
max_tokens: int | None = None
+ # temperature's canonical range is 0–2 (OpenAI's and Gemini's); a wire
+ # whose ceiling is 1 (Anthropic) clamps and records it (MAP-13). Never
+ # rescaled: both scales default to 1.0 and 0–1 is the same cool half.
temperature: float | None = None
top_p: float | None = None
top_k: int | None = None
@@ -1766,6 +1792,16 @@ class Config:
tool_choice: ToolChoice | None = None
reasoning: Reasoning | None = None
cache: CacheConfig | None = None
+ # Sampling knobs promoted from extensions 2026-09-14 (MAP-13 audit):
+ # OpenAI (both dialects), Gemini (generationConfig.seed /
+ # frequencyPenalty / presencePenalty) and every OpenAI-compatible
+ # server carry them; as extensions they reached Gemini at the wrong
+ # path. seed is best-effort determinism everywhere it exists; the
+ # penalties are in [-2, 2] on every wire that has them. A wire
+ # without them (Anthropic) drops and records.
+ seed: int | None = None
+ frequency_penalty: float | None = None
+ presence_penalty: float | None = None
# Cross-provider knobs promoted from extensions (2026-09-01 burn-down):
# service_tier is an OPEN string namespace — the tier concept is shared
# (OpenAI, Anthropic), the value vocabulary is provider-owned, like
@@ -1800,10 +1836,22 @@ class Config:
):
raise TypeError(f"{field_name} must be numeric")
_coerce_float_field(self, field_name)
- if self.temperature is not None and self.temperature < 0:
- raise ValueError("temperature must be >= 0")
+ if self.temperature is not None and not (0 <= self.temperature <= 2):
+ raise ValueError("temperature must be in [0, 2]")
if self.top_p is not None and not (0 <= self.top_p <= 1):
raise ValueError("top_p must be in [0, 1]")
+ _coerce_int_field(self, "seed")
+ if self.seed is not None and (isinstance(self.seed, bool) or not isinstance(self.seed, int)):
+ raise TypeError("seed must be an int")
+ for field_name in ("frequency_penalty", "presence_penalty"):
+ value = getattr(self, field_name)
+ if value is not None and (
+ isinstance(value, bool) or not isinstance(value, (int, float))
+ ):
+ raise TypeError(f"{field_name} must be numeric")
+ _coerce_float_field(self, field_name)
+ if value is not None and not (-2 <= value <= 2):
+ raise ValueError(f"{field_name} must be in [-2, 2]")
if any(not isinstance(s, str) or not s for s in self.stop):
raise ValueError("stop must contain non-empty strings")
if self.tool_choice is not None and not isinstance(self.tool_choice, ToolChoice):
@@ -2084,6 +2132,11 @@ class Response:
``Response`` keeps only minimal convenience properties. Use
``response.message.first(...)`` and ``response.message.parts_of(...)``
for variant-specific content access.
+
+ ``logprobs_complete=False`` means local text editing could not preserve
+ scores for all retained text. Remaining scores describe whole original
+ tokens only; never infer a probability for the unscored text. True is
+ the default and does not promise the provider supplied scores at all.
"""
id: str | None
@@ -2098,6 +2151,12 @@ class Response:
# (TextDelta.part_index) and in provider_data, not here.
logprobs: tuple[TokenLogprob, ...] | None = None
provider_data: ProviderData | None = None
+ # MAP-13: what the wire got that differs from what was asked (a
+ # dropped hint, a clamped dial, a client-side stop). Empty when the
+ # request went out exactly as written. Never printed; data.
+ adaptations: tuple[Adaptation, ...] = ()
+ # Same meaning as TextDelta.logprobs_complete; preserved in canonical JSON.
+ logprobs_complete: bool = field(default=True, kw_only=True)
def __post_init__(self) -> None:
_validate_optional_text(self.id, field_name="Response.id", allow_empty=False)
@@ -2112,6 +2171,8 @@ class Response:
raise TypeError("Response.usage must be a Usage")
_validate_logprobs_field(self, "logprobs", allow_none=True)
_validate_json_field(self, "provider_data")
+ _validate_adaptations(self, "Response.adaptations")
+ _validate_bool(self.logprobs_complete, field_name="Response.logprobs_complete")
def __repr__(self) -> str:
display_text = self.text
@@ -2127,6 +2188,10 @@ class Response:
fields.append(("citations", repr(citations)))
if self.logprobs is not None:
fields.append(("logprobs", f"<{len(self.logprobs)} tokens>"))
+ if not self.logprobs_complete:
+ fields.append(("logprobs_complete", "False"))
+ if self.adaptations:
+ fields.append(("adaptations", repr([f"{a.field}:{a.action}" for a in self.adaptations])))
if self.id is not None:
fields.append(("id", repr(self.id)))
if self.provider_data is not None:
diff --git a/dspy/_vendor/lm15/vet.py b/dspy/_vendor/lm15/vet.py
index 7dcb48424..74333320c 100644
--- a/dspy/_vendor/lm15/vet.py
+++ b/dspy/_vendor/lm15/vet.py
@@ -21,6 +21,7 @@ from typing import Any, Callable, Iterator, Literal, get_args, get_origin
from . import types as lm15_types
from . import serde
+from .adaptation import adaptation_to_dict
from .errors import AmbiguousModelError, LM15Error, StreamAssemblyError, UnknownModelError, canonical_error_code
from .providers import HttpResponse
from .providers.base import BaseProviderLM
@@ -254,10 +255,19 @@ def _base_url(msg: JsonObject) -> str | None:
def op_build_request(msg: JsonObject) -> JsonObject:
+ """The wire request, plus ``adaptations`` (MAP-13) when the build
+ recorded any: what the wire got that differs from what was asked.
+ ``reason`` is the adapter's own wording and is not part of the reply
+ (ports word their own, as with error messages)."""
lm = _adapter(msg)
request = serde.request_from_dict(msg["canonical_request"])
- transport_req = lm.build_request(request, stream=bool(msg.get("stream", False)))
- return normalize_transport_request(transport_req)
+ transport_req, adaptations = lm._build(request, stream=bool(msg.get("stream", False)))
+ out = normalize_transport_request(transport_req)
+ if adaptations:
+ out["adaptations"] = [
+ {k: v for k, v in adaptation_to_dict(a).items() if k != "reason"} for a in adaptations
+ ]
+ return out
def op_ingest_openai_chat(msg: JsonObject) -> JsonObject:
@@ -822,6 +832,11 @@ def _error_reply(req_id: Any, exc: BaseException) -> JsonObject:
error: JsonObject = {"type": type(exc).__name__, "message": str(exc)}
if isinstance(exc, LM15Error) and exc.code is not None:
error["code"] = exc.code
+ feature = getattr(exc, "feature", None)
+ if isinstance(feature, str) and feature:
+ # MAP-13: the config path a refusal is about, so a policy layer
+ # can act on it; pinned by cases as expect_lm15.raises.feature.
+ error["feature"] = feature
if isinstance(exc, StreamAssemblyError) and exc.partial is not None:
error["partial_response"] = _response_result(exc.partial)["canonical_response"]
if isinstance(exc, (UnknownModelError, AmbiguousModelError)):
diff --git a/dspy/_vendor/lm15/video_jobs.py b/dspy/_vendor/lm15/video_jobs.py
index 4e7dba909..95ca02fd6 100644
--- a/dspy/_vendor/lm15/video_jobs.py
+++ b/dspy/_vendor/lm15/video_jobs.py
@@ -1,49 +1,49 @@
-"""
-lm15.video_jobs — the VideoJob handle (ticket ergonomics over the pure ops).
-
-A video job is a ticket, exactly like a batch job: the provider cooks,
-``job.id`` is the ticket number, and the id is a plain string you can
-store anywhere. On xAI it is the ONLY copy — the wire has no list
-endpoint — so the primary pattern is store-the-id and re-attach.
+"""Explicit ticket handles over video_submit/status/result/list.
-The handle is per-language sugar over four pure operations
-(``video_submit`` / ``video_status`` / ``video_result`` /
-``video_list``); ports implement the ops, the handle idiom is theirs.
-
-``refresh()`` / ``wait()`` update the handle's snapshot in place and
-return the handle — the batch precedent (a mutating refresh avoids the
-forgot-to-reassign-stale-status bug).
+Snapshot properties never perform I/O. Waiting never submits or retries a failed
+request, and abandoning a wait does not cancel provider generation or billing.
+See the contract's 2026-09-11-shared-handles-and-profile-migration decision.
"""
-
from __future__ import annotations
import asyncio
+import math
import time
from typing import TYPE_CHECKING
from .types import VideoJobInfo, VideoPart, VideoStatus
-if TYPE_CHECKING: # pragma: no cover - annotations only
+if TYPE_CHECKING:
from .providers.async_base import AsyncBaseProviderLM
from .providers.base import BaseProviderLM
-__all__ = ["AsyncVideoJob", "VideoJob"]
+__all__ = ["AsyncVideoJob", "VideoJob", "VideoWaitTimeout"]
-_DEFAULT_POLL_SECONDS = 5.0 # video jobs finish in seconds-to-minutes, not hours
+class VideoWaitTimeout(TimeoutError):
+ """Local polling deadline, not an HTTP timeout; the job may still be running."""
-class VideoJob:
- """A live handle on one provider-side video job."""
+ def __init__(self, info: VideoJobInfo):
+ self.info = info
+ super().__init__(f"video {info.id} still {info.status!r} when the wait deadline expired")
- def __init__(self, lm: "BaseProviderLM", info: VideoJobInfo) -> None:
- self._lm = lm
- self._info = info
- # ─── Snapshot access ─────────────────────────────────────────────
+def _validate_wait(poll_every: float, timeout: float | None) -> None:
+ for name, value, positive in (("poll_every", poll_every, True), ("timeout", timeout, False)):
+ if value is None and name == "timeout":
+ continue
+ if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value):
+ raise ValueError(f"{name} must be a finite number")
+ if value < 0 or (positive and value == 0):
+ raise ValueError(f"{name} must be {'positive' if positive else 'non-negative'}")
+
+
+class _Snapshot:
+ _info: VideoJobInfo
@property
def info(self) -> VideoJobInfo:
- """The frozen snapshot from the last provider contact."""
+ """The frozen snapshot from the last successful provider contact."""
return self._info
@property
@@ -59,76 +59,77 @@ class VideoJob:
return self._info.progress
def __repr__(self) -> str:
- return f"VideoJob(id={self._info.id!r}, status={self._info.status!r}, progress={self._info.progress!r})"
+ return f"{type(self).__name__}(id={self.id!r}, status={self.status!r}, progress={self.progress!r})"
- # ─── Operations ──────────────────────────────────────────────────
- def refresh(self) -> "VideoJob":
- self._info = self._lm.video_status(self._info.id)
+class VideoJob(_Snapshot):
+ def __init__(self, lm: BaseProviderLM, info: VideoJobInfo) -> None:
+ self._lm, self._info = lm, info
+
+ def refresh(self) -> VideoJob:
+ self._info = self._lm.video_status(self.id)
return self
- def wait(self, poll_every: float = _DEFAULT_POLL_SECONDS, timeout: float | None = None) -> "VideoJob":
- """Poll until the job is terminal; returns the handle (check
- ``status`` — ``failed`` returns, it does not raise, mirroring
- batch's entry-level honesty)."""
+ def wait(self, poll_every: float = 5.0, timeout: float | None = 300.0) -> VideoJob:
+ """Wait for a terminal status, including failure; inspect status afterward.
+
+ timeout=None deliberately waits indefinitely. The sync deadline bounds
+ polling, not an in-flight status call: its transport timeout still applies.
+ """
+ _validate_wait(poll_every, timeout)
deadline = None if timeout is None else time.monotonic() + timeout
- while True:
- if self._info.done:
- return self
+ while not self.info.done:
+ remaining = None if deadline is None else deadline - time.monotonic()
+ if remaining is not None and remaining <= 0:
+ raise VideoWaitTimeout(self.info)
+ time.sleep(poll_every if remaining is None else min(poll_every, remaining))
if deadline is not None and time.monotonic() >= deadline:
- raise TimeoutError(
- f"video {self._info.id} still {self._info.status!r} after {timeout}s"
- )
- time.sleep(poll_every)
+ raise VideoWaitTimeout(self.info)
self.refresh()
+ return self
def result(self) -> VideoPart:
- """The finished video as a VideoPart (URL- or bytes-addressed,
- the provider's own delivery mode)."""
- return self._lm.video_result(self._info.id)
-
+ """Fetch through the raw result operation; never implicitly wait."""
+ return self._lm.video_result(self.id)
-class AsyncVideoJob:
- """Async mirror of :class:`VideoJob`."""
- def __init__(self, lm: "AsyncBaseProviderLM", info: VideoJobInfo) -> None:
- self._lm = lm
- self._info = info
-
- @property
- def info(self) -> VideoJobInfo:
- return self._info
+class AsyncVideoJob(_Snapshot):
+ def __init__(self, lm: AsyncBaseProviderLM, info: VideoJobInfo) -> None:
+ self._lm, self._info = lm, info
- @property
- def id(self) -> str:
- return self._info.id
-
- @property
- def status(self) -> VideoStatus:
- return self._info.status
-
- @property
- def progress(self) -> int | None:
- return self._info.progress
-
- def __repr__(self) -> str:
- return f"AsyncVideoJob(id={self._info.id!r}, status={self._info.status!r}, progress={self._info.progress!r})"
-
- async def refresh(self) -> "AsyncVideoJob":
- self._info = await self._lm.video_status(self._info.id)
+ async def refresh(self) -> AsyncVideoJob:
+ self._info = await self._lm.video_status(self.id)
return self
- async def wait(self, poll_every: float = _DEFAULT_POLL_SECONDS, timeout: float | None = None) -> "AsyncVideoJob":
+ async def wait(self, poll_every: float = 5.0, timeout: float | None = 300.0) -> AsyncVideoJob:
+ """Native cancellable wait. Neither cancellation nor timeout cancels the job."""
+ from .transports._timeouts import wait_for
+
+ _validate_wait(poll_every, timeout)
deadline = None if timeout is None else time.monotonic() + timeout
- while True:
- if self._info.done:
- return self
- if deadline is not None and time.monotonic() >= deadline:
- raise TimeoutError(
- f"video {self._info.id} still {self._info.status!r} after {timeout}s"
- )
- await asyncio.sleep(poll_every)
- await self.refresh()
+ while not self.info.done:
+ remaining = None if deadline is None else deadline - time.monotonic()
+ if remaining is not None and remaining <= 0:
+ raise VideoWaitTimeout(self.info)
+ await asyncio.sleep(poll_every if remaining is None else min(poll_every, remaining))
+ remaining = None if deadline is None else deadline - time.monotonic()
+ if remaining is not None and remaining <= 0:
+ raise VideoWaitTimeout(self.info)
+ # Return provider exceptions as data so a provider's own TimeoutError
+ # is never mistaken for the local wait deadline.
+ async def status():
+ try:
+ return await self._lm.video_status(self.id), None
+ except Exception as exc:
+ return None, exc
+ try:
+ info, error = await wait_for(status(), timeout=remaining)
+ except asyncio.TimeoutError as exc:
+ raise VideoWaitTimeout(self.info) from exc
+ if error is not None:
+ raise error
+ self._info = info
+ return self
async def result(self) -> VideoPart:
- return await self._lm.video_result(self._info.id)
+ return await self._lm.video_result(self.id)
diff --git a/dspy/clients/backend_selection.py b/dspy/clients/backend_selection.py
index b0beef6db..f7c0bbf21 100644
--- a/dspy/clients/backend_selection.py
+++ b/dspy/clients/backend_selection.py
@@ -8,12 +8,15 @@ before execution. No provider client is constructed and no credential is invoked
import os
from dataclasses import dataclass, field
-from dspy.clients.engines.lm15_engine import LM15Engine
+from dspy.clients.engines.lm15_engine import LM15Engine, timeouts_for
from dspy.lm15 import RouterConfig, UnknownModelError, UnsupportedFeatureError
CLIENT_KEYS = {"api_key", "api_base", "base_url", "headers", "extra_headers", "timeout", "api_version",
"azure_ad_token_provider", "organization", "project", "extra_query", "custom_llm_provider"}
-NATIVE_CLIENT_KEYS = {"api_key", "api_base", "base_url"}
+# timeout: since lm15 1.0.0rc2 the native engine takes it as RouterConfig
+# timeouts (a number of seconds, or an httpx.Timeout); before, its presence
+# sent the call to LiteLLM.
+NATIVE_CLIENT_KEYS = {"api_key", "api_base", "base_url", "timeout"}
@dataclass(frozen=True)
@@ -45,6 +48,11 @@ def select_backend(lm, options=None):
if (set(clients) - NATIVE_CLIENT_KEYS) or (resolution.provider.startswith("azure") and
any(key in clients for key in ("api_base", "base_url"))):
native = False
+ # A timeout the native engine cannot honor as written (a disabled
+ # httpx component) is a client setting LiteLLM carries; a malformed
+ # one stays a local TypeError/ValueError on every engine.
+ if native and "timeout" in clients:
+ timeouts_for(clients["timeout"])
except (UnknownModelError, UnsupportedFeatureError):
if spec == "lm15":
raise
diff --git a/dspy/clients/engines/litellm_engine.py b/dspy/clients/engines/litellm_engine.py
index b233ea256..7f594aca3 100644
--- a/dspy/clients/engines/litellm_engine.py
+++ b/dspy/clients/engines/litellm_engine.py
@@ -3,6 +3,7 @@
import json
from dataclasses import replace
+from dspy._vendor.lm15.result import apply_client_side_stop
from dspy._vendor.lm15.sse import SSEEvent
from dspy.clients._litellm import get_litellm
from dspy.clients.call_context import completed_legacy
@@ -12,6 +13,7 @@ from dspy.clients.engines.litellm_errors import litellm_errors
from dspy.clients.legacy_outputs import plain
from dspy.clients.lm15_boundary import request_kwargs, response_value
from dspy.lm15 import (
+ Adaptation,
ConfigurationError,
OpenAIChatLM,
ProviderError,
@@ -123,7 +125,19 @@ class _LiteLLMConfig:
"LiteLLMEngine streaming currently requires model_type='chat'. "
"Use the native Responses engine for Responses streaming.",
)
- data = request_kwargs(request, self.model_type)
+ # This engine receives a completed Responses reply, not lm15's
+ # provider stream. Own the stop operation explicitly: remove it
+ # only here, paired with _response's shared lm15 text cutter.
+ wire_request = request
+ if self.model_type == "responses" and request.config.stop:
+ wire_request = replace(request, config=replace(request.config, stop=()))
+ data = request_kwargs(wire_request, self.model_type)
+ if self.model_type == "chat" and request.config.top_k is not None:
+ # The conversion speaks OpenAI's chat dialect, which has no top_k
+ # field. LiteLLM translates top_k for providers that take it
+ # (Anthropic, Gemini, local servers), so the setting is forwarded
+ # as a LiteLLM argument rather than dropped by the conversion.
+ data["top_k"] = request.config.top_k
data.update(self.client_options)
data.update(model=request.model, num_retries=0, cache={"no-cache": True, "no-store": True})
if self.model_type == "chat":
@@ -133,6 +147,25 @@ class _LiteLLMConfig:
data["stream_options"] = {"include_usage": True}
return data
+ def _response(self, raw, request):
+ response = response_value(raw, self.model_type, request)
+ if self.model_type == "responses" and request.config.top_k is not None:
+ note = Adaptation(
+ field="config.top_k", action="dropped", asked=request.config.top_k,
+ reason="The Responses API has no top_k field; the setting was not sent.",
+ )
+ response = replace(response, adaptations=(*response.adaptations, note))
+ if self.model_type == "responses" and request.config.stop:
+ response = apply_client_side_stop(response, request.config.stop)
+ note = Adaptation(
+ field="config.stop", action="client_side",
+ asked=list(request.config.stop), applied=list(request.config.stop),
+ reason="The Responses API has no stop field; DSPy trims the completed LiteLLM reply. "
+ "Full provider usage is retained; generation is not stopped early.",
+ )
+ response = replace(response, adaptations=(*response.adaptations, note))
+ return response
+
def __repr__(self):
return f"{type(self).__name__}(model_type={self.model_type!r})"
@@ -159,7 +192,7 @@ class LiteLLMEngine(_LiteLLMConfig):
fn = litellm.responses if self.model_type == "responses" else litellm.completion
with litellm_errors(model=request.model):
raw = fn(**data)
- return response_value(raw, self.model_type, request)
+ return self._response(raw, request)
def stream(self, request: Request):
data = self._arguments(request, streaming=True)
@@ -203,7 +236,7 @@ class AsyncLiteLLMEngine(_LiteLLMConfig):
fn = litellm.aresponses if self.model_type == "responses" else litellm.acompletion
with litellm_errors(model=request.model):
raw = await fn(**data)
- return response_value(raw, self.model_type, request)
+ return self._response(raw, request)
async def stream(self, request: Request):
data = self._arguments(request, streaming=True)
diff --git a/dspy/clients/engines/lm15_engine.py b/dspy/clients/engines/lm15_engine.py
index fd15705a0..038945cbb 100644
--- a/dspy/clients/engines/lm15_engine.py
+++ b/dspy/clients/engines/lm15_engine.py
@@ -1,8 +1,8 @@
"""Native lm15 routing, with canonical errors and no retry policy."""
+import math
import threading
from collections.abc import AsyncIterator, Iterator
-from contextlib import AsyncExitStack, ExitStack
from dataclasses import replace
from dspy._vendor.lm15.router import LITELLM_PROVIDER_PREFIXES
@@ -17,10 +17,47 @@ from dspy.lm15 import (
Request,
Response,
RouterConfig,
+ Timeouts,
UnsupportedFeatureError,
)
+def timeouts_for(timeout) -> Timeouts | None:
+ """lm15 ``Timeouts`` from an LM ``timeout`` setting, or None for lm15's own defaults.
+
+ A number of seconds bounds every wait the way LiteLLM's ``timeout`` did:
+ the next byte (read), the send (write), and a free connection (pool).
+ Connecting keeps lm15's short default. An ``httpx.Timeout`` maps each of
+ its components. In httpx a component set to ``None`` means "wait
+ forever"; lm15 bounds every wait and has no "forever", so such a value
+ is refused as unsupported rather than silently replaced by a default.
+ """
+ if timeout is None:
+ return None
+ names = ("connect", "read", "write", "pool")
+ if any(hasattr(timeout, name) for name in names):
+ parts = {name: getattr(timeout, name, None) for name in names}
+ disabled = sorted(name for name, value in parts.items() if value is None)
+ if disabled:
+ raise UnsupportedFeatureError(
+ f"timeout disables {', '.join(disabled)} (None means wait forever); the native engine "
+ "bounds every wait, so pass a number of seconds for each component or use engine='litellm'",
+ feature="timeout",
+ )
+ else:
+ parts = {"connect": None, "read": timeout, "write": timeout, "pool": timeout}
+ kwargs = {}
+ for name, value in parts.items():
+ if value is None:
+ continue
+ if isinstance(value, bool) or not isinstance(value, (int, float)):
+ raise TypeError(f"timeout must be a number of seconds or an httpx.Timeout, not {type(timeout).__name__}")
+ if not math.isfinite(value) or value <= 0:
+ raise ValueError("timeout must be a positive finite number of seconds")
+ kwargs[name] = float(value)
+ return Timeouts(**kwargs) if kwargs else None
+
+
def _model_string(model: str, model_type: str) -> str:
"""Keep endpoint selection deliberate, including model ids containing ':' ."""
head, separator, rest = model.partition("/")
@@ -61,8 +98,8 @@ class _Routing:
with self._lock:
if self._closed:
raise ConfigurationError("Engine is closed")
- # Serialize lazy construction so parallel first calls do not leak
- # duplicate owned transports. Borrowed transports stay caller-owned.
+ # Serialize lazy construction so parallel first calls do not race
+ # the router's one shared transport. Borrowed transports stay caller-owned.
provider = self._providers.get(resolution.provider)
if provider is None:
provider = self.router.lm(f"{resolution.provider}:{resolution.model}")
@@ -74,6 +111,24 @@ class _Routing:
)
return provider, replace(request, model=resolution.model)
+ def plan(self, request: Request):
+ """What this request WOULD adapt on its route (lm15 MAP-13), with no network.
+
+ Offline like ``resolve``: no credential is read or invoked, so a
+ model with no key configured still plans. Raises the refusal the
+ call would raise, so engine selection can fall back before any I/O
+ instead of after a failed attempt.
+ """
+ validate_request(request)
+ resolution = self.resolve(request.model)
+ with self._lock:
+ if self._closed:
+ raise ConfigurationError("Engine is closed")
+ # Under the engine's lock like _target: the router is thread-safe
+ # itself (lm15 rc2), and this keeps "closed" and "planning"
+ # from interleaving.
+ return self.router.plan(replace(request, model=f"{resolution.provider}:{resolution.model}"))
+
class LM15Engine(_Routing):
"""One synchronous native attempt. Close after its active calls finish."""
@@ -95,13 +150,13 @@ class LM15Engine(_Routing):
def close(self):
with self._lock:
self._closed = True
- providers = list(self._providers.values())
self._providers.clear()
- self.router._lms.clear()
+ # The router owns the one transport its providers share (lm15 rc2);
+ # a caller-supplied transport stays the caller's to close.
if self.config.transport is None:
- with ExitStack() as cleanup:
- for provider in providers:
- cleanup.callback(provider.close)
+ self.router.close()
+ else:
+ self.router._lms.clear()
class AsyncLM15Engine(_Routing):
@@ -130,10 +185,8 @@ class AsyncLM15Engine(_Routing):
async def aclose(self):
with self._lock:
self._closed = True
- providers = list(self._providers.values())
self._providers.clear()
- self.router._lms.clear()
if self.config.transport is None:
- async with AsyncExitStack() as cleanup:
- for provider in providers:
- cleanup.push_async_callback(provider.aclose)
+ await self.router.aclose()
+ else:
+ self.router._lms.clear()
diff --git a/dspy/clients/errors.py b/dspy/clients/errors.py
index 9ccaa6af1..ffdc62390 100644
--- a/dspy/clients/errors.py
+++ b/dspy/clients/errors.py
@@ -4,6 +4,7 @@ Engines speak dspy.lm15 errors. Only DSPy-owned boundaries translate them into
public DSPy errors; SDK-specific interpretation belongs to the owning engine.
"""
+import re
from contextlib import contextmanager
from dspy import lm15
@@ -11,6 +12,7 @@ from dspy.utils.exceptions import (
ContextWindowExceededError,
LMAuthError,
LMBillingError,
+ LMCollectionLimitError,
LMConfigurationError,
LMError,
LMInvalidRequestError,
@@ -39,6 +41,7 @@ ERROR_MAPPING = (
(lm15.TimeoutError, LMTimeoutError),
(lm15.LockTimeoutError, LMLockTimeoutError),
(lm15.StreamAssemblyError, LMStreamAssemblyError),
+ (lm15.CollectionLimitError, LMCollectionLimitError),
(lm15.ServerError, LMServerError),
(lm15.CapabilityError, LMUnsupportedFeatureError),
(lm15.NotConfiguredError, LMNotConfiguredError),
@@ -49,6 +52,15 @@ ERROR_MAPPING = (
)
+_ROUTER_CONFIG_KEYS = re.compile(r"pass RouterConfig\(api_keys=\{'[^']+': \"\.\.\.\"\}\)")
+
+
+def _dspy_remedy(message: str) -> str:
+ """Say the remedy in DSPy's terms: a DSPy user cannot pass a RouterConfig."""
+ message = _ROUTER_CONFIG_KEYS.sub('pass api_key="..." to dspy.LM(...)', message)
+ return message.replace("(api_key, or RouterConfig api_keys)", "(api_key= on dspy.LM)")
+
+
def wrap_error(exc: Exception, *, model: str, provider: str | None = None) -> Exception:
"""Project a canonical failure without guessing from arbitrary attributes/text.
@@ -62,19 +74,30 @@ def wrap_error(exc: Exception, *, model: str, provider: str | None = None) -> Ex
if not isinstance(exc, lm15.LM15Error):
return LMUnexpectedError(str(exc), model=model, provider=provider)
target = next((target for source, target in ERROR_MAPPING if isinstance(exc, source)), LMUnexpectedError)
+ # The message is preserved verbatim (the contract pins it) except for the
+ # one local case whose remedy names a RouterConfig: a missing key.
+ message = _dspy_remedy(exc.message) if isinstance(exc, lm15.NotConfiguredError) else exc.message
+ details = {}
+ if isinstance(exc, lm15.CapabilityError):
+ details["feature"] = exc.feature
+ if isinstance(exc, lm15.CollectionLimitError):
+ details["source"] = exc
wrapped = target(
- message=exc.message,
+ message=message,
model=getattr(exc, "model", None) or model,
provider=exc.provider or provider,
provider_code=exc.provider_code,
status=exc.status,
request_id=exc.request_id,
retry_after=exc.retry_after,
+ **details,
)
for name in (
"partial", "part_index", "env_keys", "credential_hint", "path", "lock_path",
"providers", "candidates", "rules_tried", "catalog_searched", "cleanup_errors",
):
+ if name == "partial" and isinstance(exc, lm15.CollectionLimitError):
+ continue # Even hasattr() would evaluate its lazy partial property.
if hasattr(exc, name):
setattr(wrapped, name, getattr(exc, name))
return wrapped
diff --git a/dspy/clients/execution.py b/dspy/clients/execution.py
index 2b8759b8f..64e39c8a1 100644
--- a/dspy/clients/execution.py
+++ b/dspy/clients/execution.py
@@ -19,7 +19,7 @@ from dspy.clients.call_result import CACHE_FORMAT, CallResult, combine, usage_di
from dspy.clients.engines.legacy_engine import AsyncLegacyEngine, LegacyEngine
from dspy.clients.engines.lifecycle import aclosing_stream, closing_stream
from dspy.clients.engines.litellm_engine import AsyncLiteLLMEngine, LiteLLMEngine
-from dspy.clients.engines.lm15_engine import AsyncLM15Engine, LM15Engine
+from dspy.clients.engines.lm15_engine import AsyncLM15Engine, LM15Engine, timeouts_for
from dspy.clients.engines.stream_guard import achecked_stream, checked_stream
from dspy.clients.engines.streaming import ListenerBridge
from dspy.clients.errors import error_boundary
@@ -32,6 +32,7 @@ from dspy.lm15 import (
Response,
RouterConfig,
StreamAssemblyError,
+ UnsupportedFeatureError,
request_from_openai_chat,
)
from dspy.utils.exceptions import LMUnsupportedFeatureError, is_retryable_lm_error
@@ -68,6 +69,31 @@ class PreparedCall:
return key
+def _check_encodable(value, where):
+ """Refuse text no provider can receive before any engine is chosen.
+
+ lm15's builder already raises the same ValueError, but only inside the
+ engine attempt, where a forced native engine reports it as unexpected.
+ Nested containers cover message lists, tool inputs and typed Requests.
+ """
+ if isinstance(value, str):
+ try:
+ value.encode("utf-8")
+ except UnicodeEncodeError as exc:
+ bad = " ".join(f"U+{ord(ch):04X}" for ch in value[exc.start:exc.end])
+ raise ValueError(
+ f"{where} contains text that is not valid Unicode (lone surrogate {bad}), which no "
+ "provider can receive; repair the text first, e.g. text.encode('utf-8', 'replace').decode('utf-8')"
+ ) from None
+ elif isinstance(value, dict):
+ for key, item in value.items():
+ _check_encodable(key, where)
+ _check_encodable(item, where)
+ elif isinstance(value, (list, tuple, set, frozenset)):
+ for item in value:
+ _check_encodable(item, where)
+
+
def prepare(lm, prompt, messages, kwargs, *, asynchronous=False, direct=False):
kwargs = dict(kwargs)
request = kwargs.pop("request", None)
@@ -100,6 +126,7 @@ def prepare(lm, prompt, messages, kwargs, *, asynchronous=False, direct=False):
from dspy.clients.engines.base import validate_request
validate_request(request)
+ _check_encodable(request_to_dict(request), "request")
extra = set(kwargs) - {"cache", "rollout_id"}
if extra:
raise TypeError(f"Generation options belong in Request.config: {sorted(extra)}")
@@ -121,6 +148,7 @@ def prepare(lm, prompt, messages, kwargs, *, asynchronous=False, direct=False):
if hasattr(lm, "_warn_zero_temp_rollout"):
lm._warn_zero_temp_rollout(merged.get("temperature"), merged.get("rollout_id"))
rendered = messages or [{"role": "user", "content": prompt}]
+ _check_encodable(rendered, "messages" if messages else "prompt")
if getattr(lm, "use_developer_role", False) and lm.model_type == "responses":
rendered = [{**m, "role": "developer"} if m.get("role") == "system" else m for m in rendered]
n = merged.get("n", 1)
@@ -198,6 +226,8 @@ def _select_engine(lm, call, asynchronous):
canonical = _canonical(call, compat=resolution.compat)
except (LM15Error, TypeError, ValueError) as exc:
if spec == "lm15" or call.request is not None or call.legacy.get("prompt_cache") is not None:
+ if isinstance(exc, UnsupportedFeatureError):
+ raise # The outer boundary preserves its feature and other diagnostics.
raise LMUnsupportedFeatureError(str(exc), model=lm.model) from exc
# This is a representational refusal BEFORE execution. Preserve the
# original body on the compatibility backend, never retry elsewhere.
@@ -215,18 +245,30 @@ def _select_engine(lm, call, asynchronous):
# Long-lived sync pools and a separate async pool per event loop. Copies
# share the store; a copy with changed client settings gets a distinct key.
loop = asyncio.get_running_loop() if asynchronous else None
- key = (loop, lm.model, lm.model_type, tuple(sorted((k, v if isinstance(v, (str, int, float, type(None))) else id(v))
- for k, v in clients.items())))
+ timeouts = timeouts_for(clients.get("timeout"))
+ key = (loop, lm.model, lm.model_type, timeouts,
+ tuple(sorted((k, v if isinstance(v, (str, int, float, type(None))) else id(v))
+ for k, v in clients.items() if k != "timeout")))
with lm._engine_lock:
backend = lm._engine_store.get(key)
if backend is None:
provider = resolution.provider
api_keys = {provider: clients["api_key"]} if "api_key" in clients else None
url = clients.get("api_base") or clients.get("base_url")
- config = RouterConfig(api_keys=api_keys, base_urls={provider: url} if url else None)
+ config = RouterConfig(api_keys=api_keys, base_urls={provider: url} if url else None, timeouts=timeouts)
cls = AsyncLM15Engine if asynchronous else LM15Engine
backend = cls(config, model_type=lm.model_type)
lm._engine_store[key] = backend
+ if spec == "auto" and canonical is not None and call.legacy.get("prompt_cache") is None:
+ # The migration promise: an input the native route cannot carry
+ # selects LiteLLM BEFORE any I/O. lm15 MAP-13 adapts most settings
+ # and records it; what it still refuses is known from plan() with
+ # no network, so the fallback happens here, not after a failed call.
+ try:
+ backend.plan(canonical)
+ except UnsupportedFeatureError:
+ cls = AsyncLiteLLMEngine if asynchronous else LiteLLMEngine
+ return cls(model_type=lm.model_type, **clients), canonical if call.request else None, None
return backend, canonical, resolution.provider
@@ -561,5 +603,11 @@ def finalize(lm, call, result):
entry["cost_details"] = result.cost_details
if call.request:
entry["request"] = call.request
+ # lm15 MAP-13: what the wire got that differs from what was asked
+ # (a dropped seed, a clamped temperature). Data on the entry, never
+ # printed; absent when the request went out as written.
+ adaptations = tuple(a for response in result.responses for a in getattr(response, "adaptations", ()))
+ if adaptations:
+ entry["adaptations"] = adaptations
lm.update_history(entry)
return result.typed(call.request, lm.model_type) if call.request else result.outputs
diff --git a/dspy/clients/lm.py b/dspy/clients/lm.py
index a3b30b1d2..a4ccde3a8 100644
--- a/dspy/clients/lm.py
+++ b/dspy/clients/lm.py
@@ -36,8 +36,10 @@ def _get_litellm():
def _is_openai_reasoning_model(model: str) -> bool:
model_family = model.split("/")[-1].lower() if "/" in model else model.lower()
+ # Every gpt-5 generation, including dotted versions (gpt-5.6-luna), is a
+ # reasoning family. The "-chat" variants are the documented exception.
return re.match(
- r"^(?:o[1345](?:-(?:mini|nano|pro))?(?:-\d{4}-\d{2}-\d{2})?|gpt-5(?!-chat)(?:-.*)?)$",
+ r"^(?:o[1345](?:-(?:mini|nano|pro))?(?:-\d{4}-\d{2}-\d{2})?|gpt-5(?:\.\d+)*(?!-chat)(?:-.*)?)$",
model_family,
) is not None
diff --git a/dspy/utils/exceptions.py b/dspy/utils/exceptions.py
index 1d0248450..8a7d9c5e4 100644
--- a/dspy/utils/exceptions.py
+++ b/dspy/utils/exceptions.py
@@ -81,6 +81,31 @@ class LMLockTimeoutError(LMError):
super().__init__(message, **kwargs)
+class LMCollectionLimitError(LMError):
+ """A local collector reached its budget; never automatically retry.
+
+ Retains the canonical source error so its partial result remains lazy:
+ wrapping a memory-limit failure must not allocate combined text/audio.
+ ``partial_events`` and ``rejected_event`` are available without that work.
+ """
+
+ default_code = "collection_limit"
+
+ def __init__(self, message: str = "", *, source=None, **kwargs: Any):
+ self._source = source
+ self.limit = getattr(source, "limit", None)
+ self.maximum = getattr(source, "maximum", None)
+ self.retained_bytes = getattr(source, "retained_bytes", 0)
+ self.retained_events = getattr(source, "retained_events", 0)
+ self.partial_events = getattr(source, "partial_events", ())
+ self.rejected_event = getattr(source, "rejected_event", None)
+ super().__init__(message, **kwargs)
+
+ @property
+ def partial(self):
+ return self._source.partial if self._source is not None else None
+
+
class LMConfigurationError(LMError):
"""The LM or provider client is not configured correctly."""
@@ -98,6 +123,7 @@ class LMUnsupportedFeatureError(LMError):
Args:
message: Human-readable error message.
+ feature: One unsupported setting path reported by the engine.
features: Feature names that were requested but unavailable, such as
`"finetuning"`, `"reinforce"`, or `"structured_outputs"`.
issues: Optional detailed reasons the requested feature could not be
@@ -112,10 +138,12 @@ class LMUnsupportedFeatureError(LMError):
message: str = "",
*,
features: list[str] | None = None,
+ feature: str | None = None,
issues: list[str] | None = None,
**kwargs: Any,
):
- self.features = list(features or [])
+ self.feature = feature
+ self.features = list(features) if features is not None else ([feature] if feature is not None else [])
self.issues = list(issues or [])
super().__init__(message, **kwargs)
diff --git a/tests/clients/test_backend_capabilities.py b/tests/clients/test_backend_capabilities.py
index ac9a6a8a1..b67273cce 100644
--- a/tests/clients/test_backend_capabilities.py
+++ b/tests/clients/test_backend_capabilities.py
@@ -43,7 +43,6 @@ def conflicting_metadata(monkeypatch):
@pytest.mark.parametrize("model,options", [
("openai/gpt-4o", {"custom_llm_provider": "alternate"}),
("openai/gpt-4o", {"headers": {"x-test": "yes"}}),
- ("openai/gpt-4o", {"timeout": 10}),
("azure/gpt-4o", {"api_base": "https://example.invalid"}),
("unmapped/model", {}),
("openai/gpt-4o", {"model_type": "text"}),
diff --git a/tests/clients/test_engine_errors.py b/tests/clients/test_engine_errors.py
index b63a96d61..e395cfbfd 100644
--- a/tests/clients/test_engine_errors.py
+++ b/tests/clients/test_engine_errors.py
@@ -24,6 +24,7 @@ EXPECTED = {
lm15.TransportError: dspy.LMTransportError,
lm15.LockTimeoutError: dspy.LMLockTimeoutError,
lm15.StreamAssemblyError: dspy.LMStreamAssemblyError,
+ lm15.CollectionLimitError: dspy.LMCollectionLimitError,
lm15.ConfigurationError: dspy.LMConfigurationError,
lm15.NotConfiguredError: dspy.LMNotConfiguredError,
lm15.UnknownModelError: dspy.LMConfigurationError,
@@ -282,7 +283,7 @@ async def test_custom_request_refusal_is_projected_before_adapter_fallback(async
lm = dspy.LM("custom", engine=engine, async_engine=AsyncEngine(engine), cache=False)
adapter = dspy.ChatAdapter()
monkeypatch.setattr(adapter, "_make_json_adapter_fallback", lambda: pytest.fail("Setup failures are not parse failures"))
- args = (lm, {"prediction": {"type": "content", "content": "x"}}, dspy.Signature("question -> answer"), [], {"question": "hi"})
+ args = (lm, {"audio": {"voice": "alloy", "format": "wav"}}, dspy.Signature("question -> answer"), [], {"question": "hi"})
with pytest.raises(dspy.LMUnsupportedFeatureError) as caught:
if asynchronous:
await adapter.acall(*args)
@@ -292,7 +293,7 @@ async def test_custom_request_refusal_is_projected_before_adapter_fallback(async
assert engine.calls == 0
[email protected]("code", ["auth", "billing", "rate_limit", "invalid_request", "context_length", "timeout", "server", "unsupported_model", "unsupported_feature", "not_configured", "unknown_model", "ambiguous_model", "transport", "lock_timeout", "stream_assembly", "provider"])
[email protected]("code", sorted(lm15.ERROR_CODES))
def test_every_canonical_error_event_uses_the_same_projection(code):
from dspy._vendor.lm15.errors import error_class_for_code
from dspy.clients.engines.stream_guard import error_from_event
diff --git a/tests/clients/test_engine_failure_lifecycle.py b/tests/clients/test_engine_failure_lifecycle.py
index a119b6ecc..96c98581f 100644
--- a/tests/clients/test_engine_failure_lifecycle.py
+++ b/tests/clients/test_engine_failure_lifecycle.py
@@ -367,12 +367,15 @@ async def test_materializing_backend_keeps_completed_usage_on_cleanup_failure(as
engine = Backend()
lm = dspy.LM("custom", engine=engine, async_engine=AsyncBackend(engine), cache=True, num_retries=2)
- with track_usage() as tracker, pytest.raises(dspy.LMStreamAssemblyError) as caught:
- await invoke(lm, asynchronous, "hello")
- assert caught.value.partial.text == "ok"
+ # lm15 1.0.0rc1 (contract 2026-09-11-stream-completion): a failure after
+ # the end event never withholds the complete Response — it is returned,
+ # the failure is a StreamCleanupWarning. No retry, usage kept, cached.
+ with track_usage() as tracker, pytest.warns(lm15.StreamCleanupWarning):
+ outputs = await invoke(lm, asynchronous, "hello")
+ assert outputs == ["ok"]
assert tracker.get_total_tokens()[lm.model]["total_tokens"] == 3
assert engine.calls == 1
- assert not dspy.cache.memory_cache
+ assert dspy.cache.memory_cache
@pytest.mark.asyncio
diff --git a/tests/clients/test_gauntlet_followups.py b/tests/clients/test_gauntlet_followups.py
new file mode 100644
index 000000000..b1b3539ca
--- /dev/null
+++ b/tests/clients/test_gauntlet_followups.py
@@ -0,0 +1,162 @@
+"""Gauntlet follow-ups fixed directly in DSPy, checked without provider calls."""
+
+import re
+
+import pytest
+
+import dspy
+from dspy import lm15
+from dspy.clients.backend_selection import select_backend
+from dspy.clients.engines.lm15_engine import timeouts_for
+from dspy.clients.execution import _engine, prepare
+from dspy.clients.lm import _is_openai_reasoning_model
+
+# ─── gpt-5.6 style names are reasoning models ────────────────────────
+
[email protected]("model,expected", [
+ ("openai/gpt-5.6-luna", True),
+ ("openai/gpt-5.6", True),
+ ("openai/gpt-5.6-mini-2026-03-17", True),
+ ("azure/gpt-5.12-pro", True),
+ ("openai/gpt-5", True),
+ ("openai/gpt-5-mini", True),
+ ("openai/GPT-5.6-Luna", True),
+ ("openai/gpt-5.6-chat", False),
+ ("openai/gpt-5-chat-latest", False),
+ ("openai/gpt-5.6chat", False), # not a known naming form; do not guess a reasoning family
+ ("openai/gpt-50", False),
+ ("openai/gpt-5x", False),
+ ("openai/gpt-4.1", False),
+])
+def test_reasoning_model_detection_handles_dotted_versions(model, expected):
+ assert _is_openai_reasoning_model(model) is expected
+
+
+def test_dotted_reasoning_model_gets_validation_and_token_field():
+ lm = dspy.LM("openai/gpt-5.6-luna", temperature=1.0, max_tokens=16_000)
+ assert lm.kwargs["max_completion_tokens"] == 16_000 and "max_tokens" not in lm.kwargs
+ with pytest.raises(dspy.LMConfigurationError, match="reasoning models require"):
+ dspy.LM("openai/gpt-5.6-luna", temperature=0.7)
+ with pytest.raises(dspy.LMConfigurationError):
+ dspy.LM("openai/gpt-5.6-luna", max_tokens=1000)
+ # The existing gpt-5 rule treats temperature 0 as "unset"; gpt-5.6 now
+ # follows that same rule rather than a different one.
+ assert dspy.LM("openai/gpt-5.6-luna", temperature=0.0).kwargs["temperature"] == 0.0
+
+
+# ─── malformed text fails as an input error on every engine choice ──
+
[email protected]("asynchronous", [False, True])
[email protected]("engine", ["auto", "lm15", "litellm"])
[email protected]("where", ["prompt", "messages", "request", "system"])
+async def test_lone_surrogate_is_refused_before_execution(asynchronous, engine, where, monkeypatch):
+ import litellm
+
+ monkeypatch.setattr(litellm, "completion", lambda **kw: pytest.fail("must not reach the provider SDK"))
+ monkeypatch.setattr(litellm, "acompletion", lambda **kw: pytest.fail("must not reach the provider SDK"))
+ lm = dspy.LM("openai/gpt-4o", engine=engine, api_key="k", cache=False, num_retries=3)
+ bad = "hello \ud800 world"
+ if where == "prompt":
+ args, kwargs = (bad,), {}
+ elif where == "messages":
+ args, kwargs = (), {"messages": [{"role": "user", "content": bad}]}
+ elif where == "system":
+ args, kwargs = (lm15.Request(model=lm.model, system=bad, messages=(lm15.Message.user("hi"),)),), {}
+ else:
+ args, kwargs = (lm15.Request(model=lm.model, messages=(lm15.Message.user(bad),)),), {}
+ with pytest.raises(ValueError, match="U\\+D800") as caught:
+ if asynchronous:
+ await lm.acall(*args, **kwargs)
+ else:
+ lm(*args, **kwargs)
+ assert not isinstance(caught.value, dspy.LMError)
+ assert not lm.history
+
+
+def test_valid_unicode_is_not_refused():
+ lm = dspy.LM("openai/gpt-4o", engine="lm15", api_key="k", cache=False)
+ call = prepare(lm, "héllo 🌍 \u200f", None, {})
+ assert call.legacy["messages"][0]["content"] == "héllo 🌍 \u200f"
+
+
+# ─── disabled timeouts are not silently replaced by 600 seconds ─────
+
+def test_httpx_timeout_with_a_disabled_component_is_not_native():
+ httpx = pytest.importorskip("httpx")
+ with pytest.raises(lm15.UnsupportedFeatureError, match="disables pool, read, write") as caught:
+ timeouts_for(httpx.Timeout(connect=5.0, read=None, write=None, pool=None))
+ assert caught.value.feature == "timeout"
+ with pytest.raises(lm15.UnsupportedFeatureError, match="disables connect, pool, read, write"):
+ timeouts_for(httpx.Timeout(None))
+ assert timeouts_for(httpx.Timeout(connect=5.0, read=900.0, write=30.0, pool=15.0)) == lm15.Timeouts(
+ connect=5.0, read=900.0, write=30.0, pool=15.0,
+ )
+
+
[email protected]("value", [None, "unset"])
+def test_disabled_timeout_selects_litellm_under_auto_and_refuses_under_lm15(value):
+ httpx = pytest.importorskip("httpx")
+ timeout = httpx.Timeout(None) if value is None else httpx.Timeout(connect=5.0, read=None, write=None, pool=None)
+ assert not select_backend(dspy.LM("openai/gpt-4o", api_key="k", cache=False, timeout=timeout)).native
+ with pytest.raises(lm15.UnsupportedFeatureError, match="disables"):
+ select_backend(dspy.LM("openai/gpt-4o", api_key="k", cache=False, engine="lm15", timeout=timeout))
+ # Through the public call the refusal is a DSPy error naming the setting.
+ with pytest.raises(dspy.LMUnsupportedFeatureError, match="disables") as caught:
+ dspy.LM("openai/gpt-4o", api_key="k", cache=False, engine="lm15", timeout=timeout)("hi")
+ assert caught.value.feature == "timeout"
+ assert select_backend(dspy.LM("openai/gpt-4o", api_key="k", cache=False, timeout=httpx.Timeout(60.0))).native
+
+
+# ─── top_k survives the typed LiteLLM route ─────────────────────────
+
[email protected]("asynchronous", [False, True])
+async def test_typed_litellm_chat_forwards_top_k_and_records_nothing(asynchronous, monkeypatch):
+ import litellm
+
+ calls = []
+
+ def reply(**kwargs):
+ calls.append(kwargs)
+ return {"model": "claude-haiku-4-5", "choices": [{"index": 0,
+ "message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}]}
+
+ async def areply(**kwargs):
+ return reply(**kwargs)
+
+ monkeypatch.setattr(litellm, "completion", reply)
+ monkeypatch.setattr(litellm, "acompletion", areply)
+ lm = dspy.LM("anthropic/claude-haiku-4-5", engine="litellm", cache=False, num_retries=0)
+ request = lm15.Request(model=lm.model, messages=(lm15.Message.user("hi"),), config=lm15.Config(top_k=7))
+ response = await (lm.acall(request) if asynchronous else __import__("anyio").to_thread.run_sync(lambda: lm(request)))
+ assert calls[0]["top_k"] == 7
+ assert response.text == "ok" and response.adaptations == ()
+
+
+def test_typed_litellm_responses_records_a_dropped_top_k(monkeypatch):
+ import litellm
+
+ calls = []
+
+ def reply(**kwargs):
+ calls.append(kwargs)
+ return {"id": "r", "model": "gpt-4o", "status": "completed",
+ "output": [{"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "ok"}]}],
+ "usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2}}
+
+ monkeypatch.setattr(litellm, "responses", reply)
+ lm = dspy.LM("openai/gpt-4o", model_type="responses", engine="litellm", cache=False, num_retries=0)
+ request = lm15.Request(model=lm.model, messages=(lm15.Message.user("hi"),), config=lm15.Config(top_k=7))
+ response = lm(request)
+ assert "top_k" not in calls[0]
+ note, = response.adaptations
+ assert (note.field, note.action, note.asked) == ("config.top_k", "dropped", 7)
+ assert re.search(r"Responses", note.reason)
+
+
+def test_native_anthropic_top_k_stays_native():
+ lm = dspy.LM("anthropic/claude-haiku-4-5", api_key="k", cache=False)
+ backend, canonical, provider = _engine(lm, prepare(lm, "hi", None, {"top_k": 7}), False)
+ assert provider == "anthropic" and canonical.config.top_k == 7
+ assert type(backend).__name__ == "LM15Engine"
diff --git a/tests/clients/test_lm15_adaptations_and_timeouts.py b/tests/clients/test_lm15_adaptations_and_timeouts.py
new file mode 100644
index 000000000..3174922db
--- /dev/null
+++ b/tests/clients/test_lm15_adaptations_and_timeouts.py
@@ -0,0 +1,261 @@
+"""The native engine on lm15 1.0.0rc2: timeouts, plan()-before-I/O, adaptations, remedies.
+
+Found by the DSPy gauntlet (cmpnd-ai/breaka-your-lm, 2026-09-13): `timeout=`
+moved every call to LiteLLM; a slow local model died at lm15's fixed 60 s;
+a refusal surfaced only after a failed attempt; the missing-key message
+named a RouterConfig a DSPy user cannot pass.
+"""
+
+import asyncio
+import json
+
+import pytest
+
+import dspy
+from dspy._vendor.lm15.testing import FakeResponse, FakeTransport
+from dspy._vendor.lm15.transports import StdlibAsyncTransport, StdlibTransport
+from dspy.clients.backend_selection import select_backend
+from dspy.clients.engines import LiteLLMEngine, LM15Engine
+from dspy.clients.engines.lm15_engine import timeouts_for
+from dspy.clients.execution import _engine, prepare
+from dspy.lm15 import Message, Request, Timeouts
+
+
+def _wire(text="hello"):
+ body = {"id": "c", "model": "gpt-4o-mini", "choices": [{"index": 0, "message": {"role": "assistant", "content": text},
+ "finish_reason": "stop"}], "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}}
+ return FakeResponse(status=200, body=json.dumps(body).encode())
+
+
+# ─── timeout= stays native and reaches the transport ─────────────────
+
+def test_timeout_is_a_native_setting_now():
+ lm = dspy.LM("openai/gpt-4o", timeout=30, api_key="k", cache=False)
+ assert select_backend(lm).native
+
+
[email protected]("timeout,expected", [
+ (30, Timeouts(read=30.0, write=30.0, pool=30.0)),
+ (0.5, Timeouts(read=0.5, write=0.5, pool=0.5)),
+ (None, None),
+])
+def test_timeouts_for_a_number_bounds_every_wait_but_connect(timeout, expected):
+ assert timeouts_for(timeout) == expected
+
+
+def test_timeouts_for_httpx_timeout_maps_each_component():
+ httpx = pytest.importorskip("httpx")
+ assert timeouts_for(httpx.Timeout(120.0)) == Timeouts(connect=120.0, read=120.0, write=120.0, pool=120.0)
+ assert timeouts_for(httpx.Timeout(connect=5.0, read=900.0, write=30.0, pool=15.0)) == Timeouts(
+ connect=5.0, read=900.0, write=30.0, pool=15.0,
+ )
+ # httpx None means "wait forever", which lm15 cannot honour: refused,
+ # never silently replaced by the 600 s default (gauntlet follow-up).
+ with pytest.raises(dspy.lm15.UnsupportedFeatureError, match="disables pool, write"):
+ timeouts_for(httpx.Timeout(connect=5.0, read=900.0, write=None, pool=None))
+
+
[email protected]("bad", [0, -1, float("inf"), float("nan"), True, "10"])
+def test_timeouts_for_rejects_what_it_cannot_honour(bad):
+ with pytest.raises((TypeError, ValueError)):
+ timeouts_for(bad)
+
+
[email protected]("asynchronous", [False, True])
+def test_timeout_reaches_the_engines_shared_transport(asynchronous):
+ lm = dspy.LM("openai/gpt-4o", timeout=1800, api_key="k", cache=False)
+ call = prepare(lm, "hello", None, {})
+
+ def check(backend):
+ transport = backend.router._shared_transport()
+ assert isinstance(transport, StdlibAsyncTransport if asynchronous else StdlibTransport)
+ assert transport._read_timeout == 1800.0 and transport._pool_timeout == 1800.0
+ assert transport._connect_timeout == 10.0 # lm15's own connect default stays
+ return transport
+
+ async def run_async():
+ # Async pools belong to their loop: select and close on the same one.
+ backend, _, _ = _engine(lm, call, True)
+ transport = check(backend)
+ await lm.aclose()
+ return transport
+
+ if asynchronous:
+ transport = asyncio.run(run_async())
+ else:
+ backend, _, _ = _engine(lm, call, False)
+ transport = check(backend)
+ lm.close()
+ assert transport._closed
+
+
+def test_different_timeouts_get_different_engines_and_close_releases_them():
+ lm = dspy.LM("openai/gpt-4o", api_key="k", cache=False)
+ fast, _, _ = _engine(lm, prepare(lm, "hello", None, {"timeout": 5}), False)
+ slow, _, _ = _engine(lm, prepare(lm, "hello", None, {"timeout": 500}), False)
+ again, _, _ = _engine(lm, prepare(lm, "hello", None, {"timeout": 5}), False)
+ assert fast is again and fast is not slow
+ assert fast.router._shared_transport()._read_timeout == 5.0
+ assert slow.router._shared_transport()._read_timeout == 500.0
+ lm.close()
+ assert fast.router._transport is None and slow.router._transport is None
+
+
+# ─── plan() before I/O: the fallback promise, kept offline ───────────
+
+def test_auto_engine_falls_back_before_io_on_a_refusal(monkeypatch):
+ calls = []
+
+ def compatible(engine, lm, request, **context):
+ calls.append(request)
+ from dspy.clients.call_result import CallResult
+
+ return CallResult(outputs=["compatible"], response_model=lm.model)
+
+ monkeypatch.setattr(LiteLLMEngine, "complete_legacy", compatible)
+ lm = dspy.LM("openai/gpt-4o", api_key="k", cache=False)
+ # An image inside a tool result has no slot on the Chat Completions wire:
+ # lm15 refuses at build time (MAP-13 rule 4b). The refusal is known from
+ # plan() with no network, so the call goes to LiteLLM before any I/O.
+ png = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="
+ messages = [{"role": "user", "content": "look"},
+ {"role": "assistant", "content": None, "tool_calls": [{"id": "c1", "type": "function", "function": {"name": "f", "arguments": "{}"}}]},
+ {"role": "tool", "tool_call_id": "c1", "content": [{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{png}"}}]}]
+ backend, _, _ = _engine(lm, prepare(lm, None, messages, {}), False)
+ assert isinstance(backend, LiteLLMEngine)
+
+
+def test_plan_raises_the_refusal_with_its_field():
+ from dspy.lm15 import ImagePart, ToolResultPart
+
+ engine = LM15Engine(model_type="chat")
+ png = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="
+ request = Request(model="openai/gpt-4o", messages=(
+ Message.user("look"),
+ Message.tool((ToolResultPart(id="c1", content=(ImagePart(media_type="image/png", data=png),)),)),
+ ))
+ with pytest.raises(dspy.lm15.UnsupportedFeatureError) as err:
+ engine.plan(request)
+ assert err.value.feature == "messages[*].tool_result[c1].content[image]"
+
+
+def test_plan_needs_no_key():
+ engine = LM15Engine(model_type="chat")
+ request = Request(model="openai/gpt-4o", messages=(Message.user("hi"),), config=dspy.lm15.Config(top_k=3))
+ plan = engine.plan(request)
+ assert [(a.field, a.action) for a in plan] == [("config.top_k", "dropped")]
+
+
+# ─── adaptations ride the history entry ──────────────────────────────
+
+def test_history_entry_records_adaptations(monkeypatch):
+ import dspy.clients.execution as execution
+ from dspy.lm15 import RouterConfig
+
+ transport = FakeTransport([_wire()])
+ monkeypatch.setattr(execution, "RouterConfig", lambda **kwargs: RouterConfig(
+ **{**{k: v for k, v in kwargs.items() if k != "timeouts"}, "api_keys": {"openai-chat": "fake"}, "transport": transport},
+ ))
+ lm = dspy.LM("openai/gpt-4o-mini", cache=False)
+ assert lm("hello", top_k=3) == ["hello"]
+ entry = lm.history[-1]
+ assert [(a.field, a.action, a.asked) for a in entry["adaptations"]] == [("config.top_k", "dropped", 3)]
+ assert "top_k" not in json.loads(transport.requests[0].body)
+ assert entry["response"].adaptations == entry["adaptations"]
+
+
+# ─── the remedy is said in DSPy's words ──────────────────────────────
+
+def test_missing_key_remedy_names_dspy_lm(monkeypatch):
+ monkeypatch.delenv("OPENAI_API_KEY", raising=False)
+ lm = dspy.LM("openai/gpt-4o", cache=False)
+ with pytest.raises(dspy.LMNotConfiguredError) as err:
+ lm("hello")
+ text = str(err.value)
+ assert 'pass api_key="..." to dspy.LM(...)' in text and "RouterConfig" not in text
+
+
+# ─── review findings, through dspy.LM ────────────────────────────────
+
+def test_simultaneous_first_calls_share_one_pool_and_close_releases_it(monkeypatch):
+ import threading
+
+ from dspy.clients.engines import LM15Engine as Engine
+
+ calls = []
+ monkeypatch.setattr(Engine, "complete", lambda self, request: calls.append(request) or dspy.lm15.Response(
+ None, request.model, dspy.lm15.Message.assistant("ok"), "stop", dspy.lm15.Usage()))
+ lm = dspy.LM("openai/gpt-4o", api_key="k", cache=False, timeout=30)
+ barrier = threading.Barrier(8)
+ transports: set[int] = set()
+
+ def go():
+ barrier.wait()
+ assert lm("hello") == ["ok"]
+ backend, _, _ = _engine(lm, prepare(lm, "hello", None, {}), False)
+ transports.add(id(backend.router._shared_transport()))
+
+ threads = [threading.Thread(target=go) for _ in range(8)]
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()
+ assert len(transports) == 1
+ backend, _, _ = _engine(lm, prepare(lm, "hello", None, {}), False)
+ transport = backend.router._shared_transport()
+ lm.close()
+ assert transport._closed
+
+
+def test_stop_word_split_across_stream_chunks_is_honoured(monkeypatch):
+ import dspy.clients.execution as execution
+ from dspy.lm15 import RouterConfig
+
+ frames = [
+ 'event: response.created\ndata: {"type":"response.created","response":{"id":"r","model":"gpt-5"}}\n\n',
+ 'event: response.output_item.added\ndata: {"type":"response.output_item.added","output_index":0,"item":{"type":"message","id":"m","role":"assistant","content":[]}}\n\n',
+ ]
+ for piece in ("alpha ", "S", "T", "O", "P", " beta"):
+ frames.append("event: response.output_text.delta\ndata: " + json.dumps({"type": "response.output_text.delta", "output_index": 0, "content_index": 0, "delta": piece}) + "\n\n")
+ frames.append('event: response.completed\ndata: {"type":"response.completed","response":{"id":"r","model":"gpt-5","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":9}}}\n\n')
+ transport = FakeTransport([FakeResponse(status=200, body="".join(frames).encode())])
+ monkeypatch.setattr(execution, "RouterConfig", lambda **kwargs: RouterConfig(
+ **{**{k: v for k, v in kwargs.items() if k != "timeouts"}, "api_keys": {"openai": "fake"}, "transport": transport},
+ ))
+ lm = dspy.LM("openai/gpt-5", model_type="responses", cache=False)
+ outputs = lm("hello", stop=["STOP"])
+ assert outputs[0]["text"] == "alpha "
+ assert [a.action for a in lm.history[-1]["adaptations"]] == ["client_side"]
+
+
+def test_plan_reads_no_stored_login(monkeypatch):
+ from dspy._vendor.lm15 import access
+
+ class Forbidden(dict):
+ def get(self, key, default=None):
+ raise AssertionError("engine selection must not read a stored login")
+
+ monkeypatch.setattr(access, "_CREDENTIAL_LOADERS", Forbidden())
+ monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
+ lm = dspy.LM("anthropic/claude-sonnet-4-5", cache=False)
+ backend, _, _ = _engine(lm, prepare(lm, "hello", None, {"seed": 7}), False)
+ assert isinstance(backend, LM15Engine) # selected without a key, without a login
+
+
+def test_stop_word_spanning_two_text_parts_is_honoured(monkeypatch):
+ import dspy.clients.execution as execution
+ from dspy.lm15 import RouterConfig
+
+ frames = [
+ 'event: response.created\ndata: {"type":"response.created","response":{"id":"r","model":"gpt-5"}}\n\n',
+ 'event: response.output_item.added\ndata: {"type":"response.output_item.added","output_index":0,"item":{"type":"message","id":"m","role":"assistant","content":[]}}\n\n',
+ "event: response.output_text.delta\ndata: " + json.dumps({"type": "response.output_text.delta", "output_index": 0, "content_index": 0, "delta": "alpha S"}) + "\n\n",
+ "event: response.output_text.delta\ndata: " + json.dumps({"type": "response.output_text.delta", "output_index": 0, "content_index": 1, "delta": "TOP beta"}) + "\n\n",
+ 'event: response.completed\ndata: {"type":"response.completed","response":{"id":"r","model":"gpt-5","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":9}}}\n\n',
+ ]
+ transport = FakeTransport([FakeResponse(status=200, body="".join(frames).encode())])
+ monkeypatch.setattr(execution, "RouterConfig", lambda **kwargs: RouterConfig(
+ **{**{k: v for k, v in kwargs.items() if k != "timeouts"}, "api_keys": {"openai": "fake"}, "transport": transport},
+ ))
+ lm = dspy.LM("openai/gpt-5", model_type="responses", cache=False)
+ assert lm("hello", stop=["STOP"])[0]["text"] == "alpha "
diff --git a/tests/clients/test_lm15_rc3_integration.py b/tests/clients/test_lm15_rc3_integration.py
new file mode 100644
index 000000000..e8bf519b5
--- /dev/null
+++ b/tests/clients/test_lm15_rc3_integration.py
@@ -0,0 +1,173 @@
+"""The rc3 integration through DSPy's public LM boundary, without provider calls."""
+
+import copy
+
+import pytest
+
+import dspy
+from dspy import lm15
+from dspy.clients.call_result import CallResult
+from dspy.clients.errors import wrap_error
+from tests.clients.test_engine_errors import AsyncEngine, Engine, Sink, invoke
+
+
+def response_body(text="hi preSTOPtail"):
+ return {
+ "id": "r", "model": "gpt-4o", "status": "completed",
+ "output": [{"type": "message", "role": "assistant", "content": [
+ {"type": "output_text", "text": text, "logprobs": [
+ {"token": token, "logprob": -0.25, "bytes": list(token.encode()), "top_logprobs": []}
+ for token in ("hi ", "preSTOPtail")
+ ]},
+ ]}],
+ "usage": {"input_tokens": 2, "output_tokens": 3, "total_tokens": 5},
+ }
+
+
[email protected]("asynchronous", [False, True])
[email protected]("stop", [(), ("NEVER",), ("STOP",), ("hi",)])
+async def test_typed_litellm_responses_honors_stops_and_preserves_usage(asynchronous, stop, monkeypatch):
+ import litellm
+
+ calls = []
+ body = response_body()
+ original = copy.deepcopy(body)
+
+ def reply(**kwargs):
+ calls.append(kwargs)
+ return body
+
+ async def areply(**kwargs):
+ return reply(**kwargs)
+
+ monkeypatch.setattr(litellm, "responses", reply)
+ monkeypatch.setattr(litellm, "aresponses", areply)
+ lm = dspy.LM("openai/gpt-4o", model_type="responses", engine="litellm", cache=False, num_retries=0)
+ request = lm15.Request(model=lm.model, messages=(lm15.Message.user("hi"),),
+ config=lm15.Config(stop=stop, logprobs=0))
+ response = await invoke(lm, asynchronous, request)
+ expected = "hi pre" if stop == ("STOP",) else "" if stop == ("hi",) else "hi preSTOPtail"
+ assert response.text == expected
+ assert response.usage == lm15.Usage(input_tokens=2, output_tokens=3, total_tokens=5)
+ assert len(calls) == 1 and "stop" not in calls[0]
+ assert body == original # leave the provider evidence intact
+ if stop:
+ note, = response.adaptations
+ assert (note.field, note.action, note.asked, note.applied) == (
+ "config.stop", "client_side", list(stop), list(stop),
+ )
+ assert "completed" in note.reason and "not stopped early" in note.reason
+ assert lm.history[-1]["adaptations"] == response.adaptations
+ else:
+ assert response.adaptations == ()
+ if stop == ("STOP",):
+ assert [token.token for token in response.logprobs] == ["hi "]
+ assert not response.logprobs_complete
+ elif stop == ("hi",):
+ assert response.logprobs is None
+ else:
+ assert [token.token for token in response.logprobs] == ["hi ", "preSTOPtail"]
+ assert response.logprobs_complete
+ # DSPy's saved-response format keeps both the adjustment and score coverage.
+ restored = CallResult.load(CallResult.native(response).dump()).responses[0]
+ assert restored == response
+
+
[email protected]("asynchronous", [False, True])
+async def test_litellm_chat_still_sends_stop_to_provider(asynchronous, monkeypatch):
+ import litellm
+
+ calls = []
+
+ def reply(**kwargs):
+ calls.append(kwargs)
+ return {"model": "gpt-4o", "choices": [{"index": 0,
+ "message": {"role": "assistant", "content": "stopped upstream"}, "finish_reason": "stop"}]}
+
+ async def areply(**kwargs):
+ return reply(**kwargs)
+
+ monkeypatch.setattr(litellm, "completion", reply)
+ monkeypatch.setattr(litellm, "acompletion", areply)
+ lm = dspy.LM("openai/gpt-4o", engine="litellm", cache=False, num_retries=0)
+ request = lm15.Request(model=lm.model, messages=(lm15.Message.user("hi"),), config=lm15.Config(stop=("STOP",)))
+ response = await invoke(lm, asynchronous, request)
+ assert calls[0]["stop"] == ["STOP"]
+ assert response.text == "stopped upstream" and response.adaptations == ()
+
+
[email protected]("asynchronous", [False, True])
[email protected]("streaming", [False, True])
+async def test_unsupported_feature_path_reaches_public_error(asynchronous, streaming):
+ original = lm15.UnsupportedFeatureError("cannot carry image", feature="messages[1].parts[0]")
+ engine = Engine(original)
+ lm = dspy.LM("custom", engine=engine, async_engine=AsyncEngine(engine), cache=False, num_retries=2)
+ with dspy.context(send_stream=Sink() if streaming else None), pytest.raises(dspy.LMUnsupportedFeatureError) as caught:
+ await invoke(lm, asynchronous, "hello")
+ assert caught.value.feature == original.feature
+ assert caught.value.features == [original.feature]
+ assert caught.value.__cause__ is original and engine.calls == 1
+
+
[email protected]("asynchronous", [False, True])
[email protected]("streaming", [False, True])
+async def test_collection_limit_is_preserved_without_retry_or_eager_materialization(asynchronous, streaming, monkeypatch):
+ from dspy._vendor.lm15.types import LiveServerTextEvent
+
+ accepted = LiveServerTextEvent(text="partial")
+ rejected = LiveServerTextEvent(text="private rejected payload")
+ original = lm15.CollectionLimitError("collection limit reached", limit="max_bytes", maximum=100,
+ retained_bytes=50, partial_events=(accepted,), rejected_event=rejected)
+ materializations = []
+
+ def partial(error):
+ materializations.append(error)
+ return "lazy partial result"
+
+ monkeypatch.setattr(lm15.CollectionLimitError, "partial", property(partial))
+ engine = Engine(original)
+ lm = dspy.LM("custom", engine=engine, async_engine=AsyncEngine(engine), cache=False, num_retries=3)
+ with dspy.context(send_stream=Sink() if streaming else None), pytest.raises(dspy.LMError) as caught:
+ await invoke(lm, asynchronous, "hello")
+ error = caught.value
+ assert type(error) is dspy.LMCollectionLimitError
+ assert error.code == "collection_limit" and not dspy.is_retryable_lm_error(error)
+ assert error.__cause__ is original and engine.calls == 1
+ assert error.partial_events == (accepted,) and error.rejected_event is rejected
+ assert (error.limit, error.maximum, error.retained_bytes, error.retained_events) == ("max_bytes", 100, 50, 1)
+ assert not materializations # wrapping must not access the lazy property
+ assert error.partial == "lazy partial result" and materializations == [original]
+ assert "private rejected payload" not in str(error)
+ assert not lm.history
+
+
+def test_collection_limit_wrap_does_not_require_raising_to_access_partial():
+ original = lm15.CollectionLimitError("limit")
+ wrapped = wrap_error(original, model="custom")
+ assert wrapped.partial.ended_by == "incomplete"
+
+
[email protected]("asynchronous", [False, True])
+async def test_request_conversion_refusal_keeps_the_feature(asynchronous, monkeypatch):
+ import dspy.clients.execution as execution
+
+ original = lm15.UnsupportedFeatureError("cannot convert", feature="config.custom")
+
+ def refuse(*args, **kwargs):
+ raise original
+
+ monkeypatch.setattr(execution, "_canonical", refuse)
+ lm = dspy.LM("openai/gpt-4o", engine="lm15", api_key="unused", cache=False, num_retries=0)
+ with pytest.raises(dspy.LMUnsupportedFeatureError) as caught:
+ await invoke(lm, asynchronous, "hello")
+ assert caught.value.feature == "config.custom" and caught.value.__cause__ is original
+
+
+def test_explicit_feature_list_is_preserved():
+ error = dspy.LMUnsupportedFeatureError("unsupported", feature="config.x", features=["custom"])
+ assert error.feature == "config.x" and error.features == ["custom"]
diff --git a/tests/clients/test_lm_engine_execution.py b/tests/clients/test_lm_engine_execution.py
index 327c7501a..bded0383c 100644
--- a/tests/clients/test_lm_engine_execution.py
+++ b/tests/clients/test_lm_engine_execution.py
@@ -146,8 +146,8 @@ def test_unrepresentable_ordinary_options_choose_compatibility_before_io(monkeyp
return CallResult(outputs=["compatible"], response_model=lm.model)
monkeypatch.setattr(LiteLLMEngine, "complete_legacy", complete)
- assert dspy.LM("openai/gpt-4o-mini")("hello", prediction={"type": "content", "content": "hello"}) == ["compatible"]
- assert calls[0]["prediction"]["content"] == "hello"
+ assert dspy.LM("openai/gpt-4o-mini")("hello", audio={"voice": "alloy", "format": "wav"}) == ["compatible"]
+ assert calls[0]["audio"]["voice"] == "alloy"
assert not transport.requests
diff --git a/tests/clients/test_prompt_cache.py b/tests/clients/test_prompt_cache.py
index 0056d349b..85ff4928d 100644
--- a/tests/clients/test_prompt_cache.py
+++ b/tests/clients/test_prompt_cache.py
@@ -121,7 +121,7 @@ def test_unrepresentable_input_does_not_fallback_with_prompt_cache(monkeypatch):
monkeypatch.setattr(LiteLLMEngine, "complete_legacy", forbidden)
lm = dspy.LM("openai/gpt-4o", cache=False, prompt_cache=CacheConfig(prefix="stable"))
with pytest.raises(dspy.LMUnsupportedFeatureError):
- lm("hello", prediction={"type": "content", "content": "hello"})
+ lm("hello", audio={"voice": "alloy", "format": "wav"})
def test_provider_cache_options_cannot_conflict():