Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

feature/Responses-temperatureunsupported · L-261006-b8d7f9 (#1362)

Louis Choquel3fa5f82aef6f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3b020814c..5b8507cd5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,7 @@
- **A manual thinking budget leaves room for the answer and stays in the model's range (Breaking)**: an Anthropic or Gemini thinking budget, resolved from an effort or set explicitly, is cut to leave a quarter of `max_tokens` for the answer, where it could take all but one token of it, and held within the range the model declares with the new `min_thinking_budget` and `max_thinking_budget` valued constraints. The kit's Anthropic models declare the API's minimum of 1,024 tokens, so the 512 that `minimal` maps to is raised to it, and its Gemini 2.5 models declare their ranges, so `max` effort's 65,536 tokens, which every Gemini 2.5 model refused, are cut to the model's maximum even without a `max_tokens`. A `max_tokens` too small to hold the model's minimum beside the answer reserve is refused with an `LLMCapabilityError` naming the numbers, where the request was sent, and a reasoning effort of `none` on a Gemini model that always thinks, which the kit marks with the new `thinking_cannot_be_disabled` listed constraint, is refused rather than sent as a budget of 0 that Gemini answers with a 400; a project that copied the kit's `anthropic.toml`, `bedrock.toml` or `google.toml` gets the bounds by declaring them in its copy.
- **Mistral reasoning is sent as `reasoning_effort`**: a reasoning effort on a Mistral model is sent as `reasoning_effort = "high"`, the only reasoning value Mistral's reasoning models accept, where it was sent as `prompt_mode = "reasoning"`, which every current Mistral model refuses, so any reasoning effort on Mistral failed. The `reasoning` level of `[inference.llm.mistral.effort_to_level_map]` is unchanged.
- **A structured output on Google without a system prompt no longer fails**: a structured call on the `google` backend whose prompt has no system text failed with `TypeError` "generate_content() got an unexpected keyword argument 'system'", and is now sent with no system instruction.
+- **A model listing `temperature_unsupported` gets no temperature on any SDK**: a model whose backend file lists the `temperature_unsupported` constraint is now called without a temperature on the OpenAI Responses, Google, Mistral and Bedrock-native SDKs too, where those sent one anyway and only the Anthropic and OpenAI chat-completions SDKs honoured the constraint. The model spec answers it as the new `accepts_temperature` property, and a Bedrock client's `chat` now takes `temperature=None` to send none.
## [v0.76.0] - 2026-10-06
diff --git a/docs/configuration/config-technical/inference-backend-config.md b/docs/configuration/config-technical/inference-backend-config.md
index 5f364893b..6f1ba0f12 100644
--- a/docs/configuration/config-technical/inference-backend-config.md
+++ b/docs/configuration/config-technical/inference-backend-config.md
@@ -237,6 +237,23 @@ On the `mistral` SDK, a reasoning setting on a structured output needs `instruct
The `google` backend uses `instructor/genai_structured_outputs`, Gemini's native JSON output. `instructor/genai_tools` works on it too: Gemini returns the function-call arguments as plain values, so pipelex validates them in pydantic's lax mode, where a string reaches an enum field as its member.
+#### Temperature constraints
+
+A job's temperature runs from 0 to 1, and three constraints adapt it to what a model's provider takes:
+
+- `temperature_unsupported`, a listed constraint, is for a model whose provider refuses a temperature: no temperature is sent to it, on any SDK, and the model samples at its own default.
+- `fixed_temperature`, a valued constraint, is for a model that takes one value only: the job's temperature is replaced by it, with a warning when the two differ.
+- `temperature_must_be_multiplied_by_2`, a listed constraint, is for a provider whose scale runs from 0 to 2: the job's temperature is doubled.
+
+```toml
+# anthropic.toml
+["claude-4.7-opus"]
+model_id = "claude-opus-4-7"
+listed_constraints = ["temperature_unsupported"]
+```
+
+A reasoning setting drops the temperature too on some SDKs, whatever the constraints say: the OpenAI SDKs, chat completions and Responses, send no temperature beside a reasoning effort, and the Anthropic SDKs none while thinking is on.
+
#### Thinking budget bounds
A model that thinks on a token budget (`thinking_mode = "manual"` on the `anthropic`, `bedrock_anthropic` and `google` SDKs) may declare the range of budgets its provider accepts, as two valued constraints, both inclusive and both optional:
diff --git a/pipelex/cogt/model_backends/model_spec.py b/pipelex/cogt/model_backends/model_spec.py
index ed45be13c..04b6fa531 100644
--- a/pipelex/cogt/model_backends/model_spec.py
+++ b/pipelex/cogt/model_backends/model_spec.py
@@ -71,6 +71,11 @@ class InferenceModelSpec(ConfigModel):
"""The largest manual thinking budget the provider accepts for this model, when it declares one."""
return self.valued_constraints.get(ValuedConstraint.MAX_THINKING_BUDGET)
+ @property
+ def accepts_temperature(self) -> bool:
+ """Whether the provider takes a temperature for this model: every LLM worker omits it when the model lists `temperature_unsupported`."""
+ return ListedConstraint.TEMPERATURE_UNSUPPORTED not in self.listed_constraints
+
@property
def tag(self) -> str:
return rf"{self.name} → \[{self.sdk}@{self.backend_name}]({self.model_id})"
diff --git a/pipelex/providers/anthropic/anthropic_llm_worker.py b/pipelex/providers/anthropic/anthropic_llm_worker.py
index f3aaadb95..7d52cc2c6 100644
--- a/pipelex/providers/anthropic/anthropic_llm_worker.py
+++ b/pipelex/providers/anthropic/anthropic_llm_worker.py
@@ -33,7 +33,6 @@ from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.structured_output import StructureMethod
from pipelex.cogt.llm.thinking_budget import fit_thinking_budget
from pipelex.cogt.llm.thinking_mode import ThinkingMode
-from pipelex.cogt.model_backends.constraints import ListedConstraint
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.config import get_config
from pipelex.plugins.backend_extras_factory import BackendExtrasFactory
@@ -254,15 +253,15 @@ class AnthropicLLMWorker(LLMWorkerAbstract):
thinking_params = self._build_thinking_params(job_params=job_params, max_tokens=max_tokens)
log.verbose(thinking_params, title="Thinking params")
log.verbose(max_tokens, title="Max tokens")
+ sends_temperature = self.inference_model.accepts_temperature and not thinking_params.suppress_temperature
try:
# Use streaming internally to avoid SDK long-request protection
- temperature_unsupported = ListedConstraint.TEMPERATURE_UNSUPPORTED in self.inference_model.listed_constraints
async with self.anthropic_async_client.messages.stream(
messages=[message],
system=llm_job.llm_prompt.system_text or omit,
model=self.inference_model.model_id,
- temperature=omit if (thinking_params.suppress_temperature or temperature_unsupported) else job_params.temperature,
+ temperature=job_params.temperature if sends_temperature else omit,
max_tokens=max_tokens,
thinking=thinking_params.thinking or omit,
output_config=thinking_params.output_config or omit,
@@ -392,11 +391,11 @@ class AnthropicLLMWorker(LLMWorkerAbstract):
# The thinking budget is fitted against the max_tokens this call actually sends
thinking_params = self._build_thinking_params(job_params=job_params, max_tokens=effective_max_tokens)
log.verbose(thinking_params, title="Thinking params")
+ sends_temperature = self.inference_model.accepts_temperature and not thinking_params.suppress_temperature
# Deferred import: avoid pulling heavy SDK at module-load time
from instructor.core import InstructorRetryException # ruff: ignore[import-outside-top-level]
- temperature_unsupported = ListedConstraint.TEMPERATURE_UNSUPPORTED in self.inference_model.listed_constraints
try:
result_object, completion = await self.instructor_for_objects.chat.completions.create_with_completion(
messages=messages,
@@ -407,7 +406,7 @@ class AnthropicLLMWorker(LLMWorkerAbstract):
# SDK client floor (Tier 1) alone.
max_retries=make_instructor_schema_retrying(max_attempts=llm_job.job_config.schema_reask_max_attempts),
model=self.inference_model.model_id,
- temperature=omit if (thinking_params.suppress_temperature or temperature_unsupported) else job_params.temperature,
+ temperature=job_params.temperature if sends_temperature else omit,
max_tokens=effective_max_tokens,
thinking=thinking_params.thinking or omit,
output_config=thinking_params.output_config or omit,
diff --git a/pipelex/providers/bedrock/bedrock_client_aiobotocore.py b/pipelex/providers/bedrock/bedrock_client_aiobotocore.py
index 6aa15e3eb..ff75598f0 100644
--- a/pipelex/providers/bedrock/bedrock_client_aiobotocore.py
+++ b/pipelex/providers/bedrock/bedrock_client_aiobotocore.py
@@ -5,7 +5,7 @@ from typing_extensions import override
from pipelex import log
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
-from pipelex.providers.bedrock.bedrock_client_protocol import BedrockClientProtocol
+from pipelex.providers.bedrock.bedrock_client_protocol import BedrockClientProtocol, make_converse_params
from pipelex.providers.bedrock.bedrock_message import BedrockMessageDictList
if TYPE_CHECKING:
@@ -25,19 +25,10 @@ class BedrockClientAiobotocore(BedrockClientProtocol):
*,
system_text: str | None,
model: str,
- temperature: float,
+ temperature: float | None,
max_tokens: int | None = None,
) -> tuple[str, NbTokensByCategoryDict]:
- params: dict[str, Any] = {
- "modelId": model,
- "messages": messages,
- "inferenceConfig": {
- "temperature": temperature,
- "maxTokens": max_tokens,
- },
- }
- if system_text:
- params["system"] = [{"text": system_text}]
+ params = make_converse_params(messages=messages, system_text=system_text, model=model, temperature=temperature, max_tokens=max_tokens)
async with self.session.create_client("bedrock-runtime", region_name=self.aws_region) as bedrock_runtime_client: # pyright: ignore[reportUnknownMemberType]
conversation_response: ConverseResponseTypeDef = await bedrock_runtime_client.converse(**params)
diff --git a/pipelex/providers/bedrock/bedrock_client_boto3.py b/pipelex/providers/bedrock/bedrock_client_boto3.py
index 69d2f817b..b8d350547 100644
--- a/pipelex/providers/bedrock/bedrock_client_boto3.py
+++ b/pipelex/providers/bedrock/bedrock_client_boto3.py
@@ -6,7 +6,7 @@ from typing_extensions import override
from pipelex import log
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
-from pipelex.providers.bedrock.bedrock_client_protocol import BedrockClientProtocol
+from pipelex.providers.bedrock.bedrock_client_protocol import BedrockClientProtocol, make_converse_params
from pipelex.providers.bedrock.bedrock_message import BedrockMessageDictList
@@ -22,19 +22,10 @@ class BedrockClientBoto3(BedrockClientProtocol):
*,
system_text: str | None,
model: str,
- temperature: float,
+ temperature: float | None,
max_tokens: int | None = None,
) -> tuple[str, NbTokensByCategoryDict]:
- params: dict[str, Any] = {
- "modelId": model,
- "messages": messages,
- "inferenceConfig": {
- "temperature": temperature,
- "maxTokens": max_tokens,
- },
- }
- if system_text:
- params["system"] = [{"text": system_text}]
+ params = make_converse_params(messages=messages, system_text=system_text, model=model, temperature=temperature, max_tokens=max_tokens)
# ``to_thread`` rather than ``run_in_executor``: it carries the context over, so the SDK's own log
# lines in the thread name the LLM span under `pipelex.*` and the caller's current span in their
diff --git a/pipelex/providers/bedrock/bedrock_client_protocol.py b/pipelex/providers/bedrock/bedrock_client_protocol.py
index c45dee503..548eeb6c6 100644
--- a/pipelex/providers/bedrock/bedrock_client_protocol.py
+++ b/pipelex/providers/bedrock/bedrock_client_protocol.py
@@ -1,4 +1,4 @@
-from typing import Protocol, runtime_checkable
+from typing import Any, Protocol, runtime_checkable
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict
from pipelex.providers.bedrock.bedrock_message import BedrockMessageDictList
@@ -12,6 +12,31 @@ class BedrockClientProtocol(Protocol):
*,
system_text: str | None,
model: str,
- temperature: float,
+ temperature: float | None,
max_tokens: int | None = None,
) -> tuple[str, NbTokensByCategoryDict]: ...
+
+
+def make_converse_params(
+ *,
+ messages: BedrockMessageDictList,
+ system_text: str | None,
+ model: str,
+ temperature: float | None,
+ max_tokens: int | None,
+) -> dict[str, Any]:
+ """Build the parameters of a Bedrock Converse call, which every Bedrock client sends.
+
+ A temperature of None leaves the key out of the inference config, for a model whose provider refuses one.
+ """
+ inference_config: dict[str, Any] = {"maxTokens": max_tokens}
+ if temperature is not None:
+ inference_config["temperature"] = temperature
+ params: dict[str, Any] = {
+ "modelId": model,
+ "messages": messages,
+ "inferenceConfig": inference_config,
+ }
+ if system_text:
+ params["system"] = [{"text": system_text}]
+ return params
diff --git a/pipelex/providers/bedrock/bedrock_llm_worker.py b/pipelex/providers/bedrock/bedrock_llm_worker.py
index df02adb12..117129f50 100644
--- a/pipelex/providers/bedrock/bedrock_llm_worker.py
+++ b/pipelex/providers/bedrock/bedrock_llm_worker.py
@@ -68,7 +68,7 @@ class BedrockLLMWorker(LLMWorkerAbstract):
messages=message.to_dict_list(),
system_text=llm_job.llm_prompt.system_text,
model=self.inference_model.model_id,
- temperature=job_params.temperature,
+ temperature=job_params.temperature if self.inference_model.accepts_temperature else None,
max_tokens=job_params.max_tokens or self.default_max_tokens,
)
except ClientError as exc:
diff --git a/pipelex/providers/google/google_llm_worker.py b/pipelex/providers/google/google_llm_worker.py
index 8f1efa178..05a867fc0 100644
--- a/pipelex/providers/google/google_llm_worker.py
+++ b/pipelex/providers/google/google_llm_worker.py
@@ -224,7 +224,7 @@ class GoogleLLMWorker(LLMWorkerAbstract):
# Build generation config
generation_config = genai_types.GenerateContentConfig(
- temperature=job_params.temperature,
+ temperature=job_params.temperature if self.inference_model.accepts_temperature else None,
max_output_tokens=job_params.max_tokens,
candidate_count=1, # Generate one candidate
thinking_config=thinking_config,
@@ -310,7 +310,8 @@ class GoogleLLMWorker(LLMWorkerAbstract):
# top-level OpenAI-style kwargs: a `GenerateContentConfig` passed as `generation_config`
# or `config` is dropped.
**system_kwargs,
- temperature=job_params.temperature,
+ # None sets no temperature in the config the handlers build
+ temperature=job_params.temperature if self.inference_model.accepts_temperature else None,
max_tokens=job_params.max_tokens,
n=1,
strict=self._validates_structured_output_strictly(),
diff --git a/pipelex/providers/mistral/mistral_llm_worker.py b/pipelex/providers/mistral/mistral_llm_worker.py
index 9d476d003..e79009666 100644
--- a/pipelex/providers/mistral/mistral_llm_worker.py
+++ b/pipelex/providers/mistral/mistral_llm_worker.py
@@ -124,7 +124,7 @@ class MistralLLMWorker(LLMWorkerAbstract):
response: ChatCompletionResponse | None = await self.mistral_client_for_text.chat.complete_async(
messages=messages,
model=self.inference_model.model_id,
- temperature=job_params.temperature,
+ temperature=job_params.temperature if self.inference_model.accepts_temperature else UNSET,
max_tokens=job_params.max_tokens or self.default_max_tokens,
reasoning_effort=reasoning_effort,
)
@@ -246,7 +246,7 @@ class MistralLLMWorker(LLMWorkerAbstract):
response_model=schema,
messages=messages,
model=self.inference_model.model_id,
- temperature=job_params.temperature,
+ temperature=job_params.temperature if self.inference_model.accepts_temperature else UNSET,
max_tokens=job_params.max_tokens or self.default_max_tokens,
reasoning_effort=reasoning_effort,
# instructor's retry is confined to schema re-ask: this validation-only AsyncRetrying
diff --git a/pipelex/providers/openai/openai_completions_llm_worker.py b/pipelex/providers/openai/openai_completions_llm_worker.py
index 4cd601dac..cd34c923c 100644
--- a/pipelex/providers/openai/openai_completions_llm_worker.py
+++ b/pipelex/providers/openai/openai_completions_llm_worker.py
@@ -24,7 +24,6 @@ from pipelex.cogt.llm.llm_job_components import LLMJobParams
from pipelex.cogt.llm.llm_utils import dump_error, dump_kwargs, dump_response_from_structured_gen
from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.thinking_mode import ThinkingMode
-from pipelex.cogt.model_backends.constraints import ListedConstraint
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.config import get_config
from pipelex.providers.openai.openai_completions_factory import OpenAICompletionsFactory
@@ -138,10 +137,9 @@ class OpenAICompletionsLLMWorker(LLMWorkerAbstract):
extra_headers, extra_body = self.openai_completions_factory.make_extras(
inference_model=self.inference_model, inference_job=llm_job, output_desc=InferenceOutputType.TEXT
)
- temperature_unsupported = ListedConstraint.TEMPERATURE_UNSUPPORTED in self.inference_model.listed_constraints
response = await self.openai_client_for_text.chat.completions.create(
model=self.inference_model.model_id,
- temperature=omit if (openai_reasoning_effort is not None or temperature_unsupported) else job_params.temperature,
+ temperature=omit if (openai_reasoning_effort is not None or not self.inference_model.accepts_temperature) else job_params.temperature,
max_tokens=job_params.max_tokens or omit,
seed=job_params.seed,
messages=messages,
@@ -218,11 +216,10 @@ class OpenAICompletionsLLMWorker(LLMWorkerAbstract):
extra_headers, extra_body = self.openai_completions_factory.make_extras(
inference_model=self.inference_model, inference_job=llm_job, output_desc=schema.__name__
)
- temperature_unsupported = ListedConstraint.TEMPERATURE_UNSUPPORTED in self.inference_model.listed_constraints
try:
result_object, completion = await self.instructor_for_objects.chat.completions.create_with_completion(
model=self.inference_model.model_id,
- temperature=omit if (openai_reasoning_effort is not None or temperature_unsupported) else job_params.temperature,
+ temperature=omit if (openai_reasoning_effort is not None or not self.inference_model.accepts_temperature) else job_params.temperature,
max_tokens=job_params.max_tokens or NOT_GIVEN,
seed=job_params.seed,
messages=messages,
diff --git a/pipelex/providers/openai/openai_responses_llm_worker.py b/pipelex/providers/openai/openai_responses_llm_worker.py
index 9b70bed3c..7f4b9e843 100644
--- a/pipelex/providers/openai/openai_responses_llm_worker.py
+++ b/pipelex/providers/openai/openai_responses_llm_worker.py
@@ -147,7 +147,7 @@ class OpenAIResponsesLLMWorker(LLMWorkerAbstract):
response = await self.openai_client_for_responses.responses.create(
model=self.inference_model.model_id,
instructions=llm_job.llm_prompt.system_text,
- temperature=omit if openai_reasoning is not None else job_params.temperature,
+ temperature=omit if (openai_reasoning is not None or not self.inference_model.accepts_temperature) else job_params.temperature,
max_output_tokens=job_params.max_tokens or omit,
input=input_items,
reasoning=openai_reasoning if openai_reasoning is not None else omit,
@@ -225,7 +225,7 @@ class OpenAIResponsesLLMWorker(LLMWorkerAbstract):
max_retries=make_instructor_schema_retrying(max_attempts=llm_job.job_config.schema_reask_max_attempts),
model=self.inference_model.model_id,
instructions=llm_job.llm_prompt.system_text,
- temperature=omit if openai_reasoning is not None else job_params.temperature,
+ temperature=omit if (openai_reasoning is not None or not self.inference_model.accepts_temperature) else job_params.temperature,
max_output_tokens=job_params.max_tokens or NOT_GIVEN,
reasoning=openai_reasoning if openai_reasoning is not None else omit,
extra_headers=extra_headers,
diff --git a/tests/helpers/anthropic_structured_request.py b/tests/helpers/anthropic_structured_request.py
index 3d9c5521a..b72a844d8 100644
--- a/tests/helpers/anthropic_structured_request.py
+++ b/tests/helpers/anthropic_structured_request.py
@@ -69,6 +69,7 @@ def make_structured_worker(
thinking_mode: ThinkingMode = ThinkingMode.NONE,
default_max_tokens: int = 4096,
responses: list[Message] | None = None,
+ accepts_temperature: bool = True,
) -> tuple[AnthropicLLMWorker, AsyncMock]:
"""A worker whose instructor client is built as the worker builds it, over an SDK call that records the request."""
from instructor import from_anthropic # ruff: ignore[import-outside-top-level]
@@ -87,7 +88,7 @@ def make_structured_worker(
model.desc = "test-model-desc"
model.model_id = "claude-test"
model.name = "claude-test"
- model.listed_constraints = []
+ model.accepts_temperature = accepts_temperature
model.structure_method = structure_method
model.thinking_mode = thinking_mode
model.min_thinking_budget = ANTHROPIC_MIN_THINKING_BUDGET
diff --git a/tests/helpers/google_structured_request.py b/tests/helpers/google_structured_request.py
index 7840a85c2..d81d2cf49 100644
--- a/tests/helpers/google_structured_request.py
+++ b/tests/helpers/google_structured_request.py
@@ -64,8 +64,13 @@ def make_structured_worker(
thinking_mode: ThinkingMode | None = None,
min_thinking_budget: int | None = None,
max_thinking_budget: int | None = None,
+ accepts_temperature: bool = True,
) -> GoogleLLMWorker:
- """A worker over a real instructor client whose ``generate_content`` records what reaches it in ``captured``."""
+ """A worker over a real instructor client whose ``generate_content`` records what reaches it in ``captured``.
+
+ The worker's text client is the same SDK client, so its text call is captured too, and answered with the
+ response the structure method shapes: a text part under JSON output.
+ """
client = genai.Client(api_key="test-key")
response_payload = payload or {"text": "ok"}
@@ -86,7 +91,9 @@ def make_structured_worker(
mock_model.thinking_mode = thinking_mode
mock_model.min_thinking_budget = min_thinking_budget
mock_model.max_thinking_budget = max_thinking_budget
+ mock_model.accepts_temperature = accepts_temperature
worker.inference_model = mock_model
+ worker.genai_async_client = client.aio
worker.instructor_for_objects = from_genai(client=client, mode=structure_method.as_instructor_mode(), use_async=True)
config = mocker.MagicMock()
diff --git a/tests/helpers/mistral_request.py b/tests/helpers/mistral_request.py
index dd53b9b7f..bcfef33a5 100644
--- a/tests/helpers/mistral_request.py
+++ b/tests/helpers/mistral_request.py
@@ -72,6 +72,7 @@ def make_worker(
thinking_mode: ThinkingMode,
response: ChatCompletionResponse,
structure_method: StructureMethod = StructureMethod.INSTRUCTOR_MISTRAL_TOOLS,
+ accepts_temperature: bool = True,
) -> tuple[MistralLLMWorker, AsyncMock]:
"""A worker whose text client and instructor client share one SDK client, whose call records the request."""
from instructor import from_mistral # ruff: ignore[import-outside-top-level]
@@ -87,6 +88,7 @@ def make_worker(
model.name = "magistral-test"
model.thinking_mode = thinking_mode
model.structure_method = structure_method
+ model.accepts_temperature = accepts_temperature
worker.inference_model = model
worker.default_max_tokens = 4096
worker.mistral_factory = MistralFactory()
diff --git a/tests/helpers/openai_completions_request.py b/tests/helpers/openai_completions_request.py
new file mode 100644
index 000000000..e59466b1e
--- /dev/null
+++ b/tests/helpers/openai_completions_request.py
@@ -0,0 +1,100 @@
+"""Fakes for driving the OpenAI chat-completions worker's text and structured calls over a recording OpenAI SDK client.
+
+The worker's text client and its instructor client, built in tool mode as the worker builds it, share one SDK
+client whose ``chat.completions.create`` records the request and answers with the completion given, so a test
+asserts what is actually sent, through instructor on the structured path.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import TYPE_CHECKING, Any
+
+from openai import AsyncOpenAI, Omit
+from openai.types.chat import ChatCompletion, ChatCompletionMessage, ChatCompletionMessageToolCall
+from openai.types.chat.chat_completion import Choice
+from openai.types.chat.chat_completion_message_tool_call import Function
+
+from pipelex.cogt.llm.structured_output import StructureMethod
+from pipelex.providers.openai.openai_completions_factory import OpenAICompletionsFactory
+from pipelex.providers.openai.openai_completions_llm_worker import OpenAICompletionsLLMWorker
+from pipelex.providers.openai.openai_config import OpenAIConfig
+
+if TYPE_CHECKING:
+ from unittest.mock import AsyncMock
+
+ from pytest_mock import MockerFixture
+
+ from pipelex.cogt.llm.thinking_mode import ThinkingMode
+
+OPENAI_LEVEL_MAP: dict[str, str] = {
+ "none": "none",
+ "minimal": "minimal",
+ "low": "low",
+ "medium": "medium",
+ "high": "high",
+ "xhigh": "xhigh",
+ "max": "xhigh",
+}
+
+
+def _completion(choice: Choice) -> ChatCompletion:
+ return ChatCompletion(id="chatcmpl_test", object="chat.completion", created=0, model="test-model", choices=[choice])
+
+
+def tool_call_completion() -> ChatCompletion:
+ tool_call = ChatCompletionMessageToolCall(
+ id="call_test",
+ type="function",
+ function=Function(name="DummySchema", arguments=json.dumps({"text": "answer"})),
+ )
+ message = ChatCompletionMessage(role="assistant", content=None, tool_calls=[tool_call])
+ return _completion(Choice(index=0, finish_reason="tool_calls", message=message))
+
+
+def text_completion() -> ChatCompletion:
+ return _completion(Choice(index=0, finish_reason="stop", message=ChatCompletionMessage(role="assistant", content="answer")))
+
+
+def make_worker(
+ mocker: MockerFixture,
+ *,
+ thinking_mode: ThinkingMode,
+ response: ChatCompletion,
+ accepts_temperature: bool = True,
+) -> tuple[OpenAICompletionsLLMWorker, AsyncMock]:
+ """A worker whose text client and instructor client share one SDK client, whose call records the request."""
+ from instructor import from_openai # ruff: ignore[import-outside-top-level]
+
+ sdk_client = AsyncOpenAI(api_key="test-key")
+ create = mocker.AsyncMock(return_value=response)
+ mocker.patch.object(sdk_client.chat.completions, "create", new=create)
+
+ worker = object.__new__(OpenAICompletionsLLMWorker)
+ model = mocker.MagicMock()
+ model.desc = "test-model-desc"
+ model.model_id = "test-model"
+ model.name = "test-model"
+ model.thinking_mode = thinking_mode
+ model.accepts_temperature = accepts_temperature
+ model.extra_headers = None
+ worker.inference_model = model
+ worker.openai_completions_factory = OpenAICompletionsFactory(is_http_url_enabled=False)
+ worker.openai_client_for_text = sdk_client
+ worker.instructor_for_objects = from_openai(client=sdk_client, mode=StructureMethod.INSTRUCTOR_OPENAI_TOOLS.as_instructor_mode())
+
+ config = mocker.MagicMock()
+ config.inference.llm.openai = OpenAIConfig(effort_to_level_map=OPENAI_LEVEL_MAP)
+ mocker.patch("pipelex.providers.openai.openai_completions_llm_worker.get_config", return_value=config)
+ return worker, create
+
+
+def sent_request(create: AsyncMock) -> dict[str, Any]:
+ create.assert_awaited_once()
+ await_args = create.await_args
+ assert await_args is not None
+ return dict(await_args.kwargs)
+
+
+def is_omitted(request: dict[str, Any], key: str) -> bool:
+ return key not in request or isinstance(request[key], Omit)
diff --git a/tests/helpers/openai_responses_request.py b/tests/helpers/openai_responses_request.py
new file mode 100644
index 000000000..2ec513714
--- /dev/null
+++ b/tests/helpers/openai_responses_request.py
@@ -0,0 +1,114 @@
+"""Fakes for driving the OpenAI Responses worker's text and structured calls over a recording OpenAI SDK client.
+
+The worker's text client and its instructor client, built in ``RESPONSES_TOOLS`` mode as the worker builds it,
+share one SDK client whose ``responses.create`` records the request and answers with the response given, so a
+test asserts what is actually sent, through instructor on the structured path. The hosted plane's
+``manifold_responses`` SDK runs this worker, so this is the request that reaches the gateway.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import TYPE_CHECKING, Any
+
+from openai import AsyncOpenAI, Omit
+from openai.types.responses import Response, ResponseFunctionToolCall, ResponseOutputMessage, ResponseOutputText
+
+from pipelex.cogt.llm.structured_output import StructureMethod
+from pipelex.providers.openai.openai_config import OpenAIConfig
+from pipelex.providers.openai.openai_responses_factory import OpenAIResponsesFactory
+from pipelex.providers.openai.openai_responses_llm_worker import OpenAIResponsesLLMWorker
+
+if TYPE_CHECKING:
+ from unittest.mock import AsyncMock
+
+ from openai.types.responses import ResponseOutputItem
+ from pytest_mock import MockerFixture
+
+ from pipelex.cogt.llm.thinking_mode import ThinkingMode
+
+OPENAI_LEVEL_MAP: dict[str, str] = {
+ "none": "none",
+ "minimal": "minimal",
+ "low": "low",
+ "medium": "medium",
+ "high": "high",
+ "xhigh": "xhigh",
+ "max": "xhigh",
+}
+
+
+def _response(output: list[ResponseOutputItem]) -> Response:
+ return Response(
+ id="resp_test",
+ created_at=0,
+ model="gpt-test",
+ object="response",
+ output=output,
+ parallel_tool_calls=False,
+ tool_choice="auto",
+ tools=[],
+ )
+
+
+def function_call_response() -> Response:
+ return _response(
+ [ResponseFunctionToolCall(type="function_call", call_id="call_test", name="DummySchema", arguments=json.dumps({"text": "answer"}))]
+ )
+
+
+def text_response() -> Response:
+ message = ResponseOutputMessage(
+ id="msg_test",
+ type="message",
+ role="assistant",
+ status="completed",
+ content=[ResponseOutputText(type="output_text", text="answer", annotations=[])],
+ )
+ return _response([message])
+
+
+def make_worker(
+ mocker: MockerFixture,
+ *,
+ thinking_mode: ThinkingMode,
+ response: Response,
+ accepts_temperature: bool = True,
+) -> tuple[OpenAIResponsesLLMWorker, AsyncMock]:
+ """A worker whose text client and instructor client share one SDK client, whose call records the request."""
+ from instructor import from_openai # ruff: ignore[import-outside-top-level]
+
+ sdk_client = AsyncOpenAI(api_key="test-key")
+ create = mocker.AsyncMock(return_value=response)
+ mocker.patch.object(sdk_client.responses, "create", new=create)
+
+ worker = object.__new__(OpenAIResponsesLLMWorker)
+ model = mocker.MagicMock()
+ model.desc = "test-model-desc"
+ model.model_id = "gpt-test"
+ model.name = "gpt-test"
+ model.thinking_mode = thinking_mode
+ model.accepts_temperature = accepts_temperature
+ model.extra_headers = None
+ worker.inference_model = model
+ factory = OpenAIResponsesFactory(is_http_url_enabled=False)
+ mocker.patch.object(factory, "make_input_items", new_callable=mocker.AsyncMock, return_value=[{"role": "user", "content": "Generate."}])
+ worker.openai_responses_factory = factory
+ worker.openai_client_for_responses = sdk_client
+ worker.instructor_for_objects = from_openai(client=sdk_client, mode=StructureMethod.INSTRUCTOR_OPENAI_RESPONSES_TOOLS.as_instructor_mode())
+
+ config = mocker.MagicMock()
+ config.inference.llm.openai = OpenAIConfig(effort_to_level_map=OPENAI_LEVEL_MAP)
+ mocker.patch("pipelex.providers.openai.openai_responses_llm_worker.get_config", return_value=config)
+ return worker, create
+
+
+def sent_request(create: AsyncMock) -> dict[str, Any]:
+ create.assert_awaited_once()
+ await_args = create.await_args
+ assert await_args is not None
+ return dict(await_args.kwargs)
+
+
+def is_omitted(request: dict[str, Any], key: str) -> bool:
+ return key not in request or isinstance(request[key], Omit)
diff --git a/tests/unit/pipelex/cogt/model_backends/test_model_spec_temperature.py b/tests/unit/pipelex/cogt/model_backends/test_model_spec_temperature.py
new file mode 100644
index 000000000..cee3a6a69
--- /dev/null
+++ b/tests/unit/pipelex/cogt/model_backends/test_model_spec_temperature.py
@@ -0,0 +1,42 @@
+"""A model spec says whether its provider takes a temperature, from the `temperature_unsupported` listed constraint.
+
+Every LLM worker reads this one answer, so a model listing the constraint gets no temperature on any SDK.
+"""
+
+import pytest
+
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.cogt.model_backends.constraints import ListedConstraint
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.cogt.usage.cost_category import CostCategory
+
+
+def _make_spec(*, listed_constraints: list[ListedConstraint]) -> InferenceModelSpec:
+ return InferenceModelSpec(
+ backend_name="test",
+ name="test-model",
+ sdk="test_sdk",
+ model_type=ModelType.LLM,
+ model_id="test-model-id",
+ outputs=["text"],
+ costs={CostCategory.INPUT: 1.0, CostCategory.OUTPUT: 1.0},
+ thinking_mode=ThinkingMode.MANUAL,
+ max_tokens=None,
+ max_prompt_images=None,
+ listed_constraints=listed_constraints,
+ )
+
+
+class TestModelSpecTemperature:
+ @pytest.mark.parametrize(
+ ("listed_constraints", "expected"),
+ [
+ ([], True),
+ ([ListedConstraint.TEMPERATURE_MUST_BE_MULTIPLIED_BY_2], True),
+ ([ListedConstraint.TEMPERATURE_UNSUPPORTED], False),
+ ([ListedConstraint.THINKING_CANNOT_BE_DISABLED, ListedConstraint.TEMPERATURE_UNSUPPORTED], False),
+ ],
+ )
+ def test_accepts_temperature_reads_the_listed_constraint(self, listed_constraints: list[ListedConstraint], expected: bool) -> None:
+ assert _make_spec(listed_constraints=listed_constraints).accepts_temperature is expected
diff --git a/tests/unit/pipelex/providers/anthropic/test_anthropic_structure_method.py b/tests/unit/pipelex/providers/anthropic/test_anthropic_structure_method.py
index e3ccb0593..b61a14b9e 100644
--- a/tests/unit/pipelex/providers/anthropic/test_anthropic_structure_method.py
+++ b/tests/unit/pipelex/providers/anthropic/test_anthropic_structure_method.py
@@ -69,6 +69,13 @@ class TestAnthropicStructureMethod:
assert isinstance(request.get("thinking", Omit()), Omit)
assert request["temperature"] == 0.5
+ async def test_a_model_refusing_temperature_omits_it_without_thinking(self, mocker: MockerFixture) -> None:
+ worker, create = make_structured_worker(mocker, structure_method=StructureMethod.INSTRUCTOR_ANTHROPIC_TOOLS, accepts_temperature=False)
+
+ await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert isinstance(sent_request(create).get("temperature", Omit()), Omit)
+
async def test_json_mode_with_thinking_adds_no_tool_choice(self, mocker: MockerFixture) -> None:
"""A JSON mode defines no tool, so thinking adds no tool choice and no tool steering to its request."""
worker, create = make_structured_worker(
diff --git a/tests/unit/pipelex/providers/anthropic/test_anthropic_text_request.py b/tests/unit/pipelex/providers/anthropic/test_anthropic_text_request.py
new file mode 100644
index 000000000..63ba3ffb5
--- /dev/null
+++ b/tests/unit/pipelex/providers/anthropic/test_anthropic_text_request.py
@@ -0,0 +1,96 @@
+"""The temperature the Anthropic worker's text call sends.
+
+A temperature is sent unless thinking is on, or the model lists `temperature_unsupported`, whose provider
+refuses one. The text call streams, so the SDK client's ``messages.stream`` is replaced by one that records its
+arguments and hands back a final message.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from anthropic import Omit
+from anthropic.types import TextBlock
+
+from pipelex.cogt.llm.llm_job_components import ReasoningEffort
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.providers.anthropic.anthropic_config import AnthropicConfig
+from pipelex.providers.anthropic.anthropic_llm_worker import AnthropicLLMWorker
+from tests.helpers.anthropic_structured_request import ANTHROPIC_LEVEL_MAP, ANTHROPIC_MIN_THINKING_BUDGET, anthropic_budget, make_message
+from tests.helpers.instructor_test_utils import make_llm_job
+
+if TYPE_CHECKING:
+ from unittest.mock import MagicMock
+
+ from pytest_mock import MockerFixture
+
+
+def _make_text_worker(
+ mocker: MockerFixture,
+ *,
+ thinking_mode: ThinkingMode = ThinkingMode.NONE,
+ accepts_temperature: bool = True,
+) -> tuple[AnthropicLLMWorker, MagicMock]:
+ """A worker whose streaming call records its arguments and answers with one text block."""
+ stream = mocker.MagicMock()
+ stream.get_final_message = mocker.AsyncMock(return_value=make_message(TextBlock(type="text", text="answer")))
+ stream_context = mocker.MagicMock()
+ stream_context.__aenter__ = mocker.AsyncMock(return_value=stream)
+ stream_context.__aexit__ = mocker.AsyncMock(return_value=False)
+ sdk_client = mocker.MagicMock()
+ sdk_client.messages.stream = mocker.MagicMock(return_value=stream_context)
+
+ worker = object.__new__(AnthropicLLMWorker)
+ worker.extras_factory = None
+ model = mocker.MagicMock()
+ model.desc = "test-model-desc"
+ model.model_id = "claude-test"
+ model.name = "claude-test"
+ model.thinking_mode = thinking_mode
+ model.accepts_temperature = accepts_temperature
+ model.min_thinking_budget = ANTHROPIC_MIN_THINKING_BUDGET
+ model.max_thinking_budget = None
+ worker.inference_model = model
+ worker.default_max_tokens = 4096
+ worker.anthropic_async_client = sdk_client
+
+ config = mocker.MagicMock()
+ config.inference.llm.anthropic = AnthropicConfig(structured_output_timeout_seconds=1200, effort_to_level_map=ANTHROPIC_LEVEL_MAP)
+ config.inference.llm.get_reasoning_budget = mocker.MagicMock(side_effect=anthropic_budget)
+ mocker.patch("pipelex.providers.anthropic.anthropic_llm_worker.get_config", return_value=config)
+ return worker, sdk_client.messages.stream
+
+
+def _sent_request(stream: MagicMock) -> dict[str, Any]:
+ stream.assert_called_once()
+ return dict(stream.call_args.kwargs)
+
+
[email protected](loop_scope="class")
+class TestAnthropicTextRequest:
+ async def test_without_thinking_the_temperature_is_sent(self, mocker: MockerFixture) -> None:
+ worker, stream = _make_text_worker(mocker)
+
+ result = await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert result == "answer"
+ assert _sent_request(stream)["temperature"] == 0.5
+
+ async def test_thinking_drops_the_temperature(self, mocker: MockerFixture) -> None:
+ worker, stream = _make_text_worker(mocker, thinking_mode=ThinkingMode.ADAPTIVE)
+ llm_job = make_llm_job(mocker)
+ llm_job.job_params.reasoning_effort = ReasoningEffort.HIGH
+
+ await worker._gen_text(llm_job=llm_job) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ request = _sent_request(stream)
+ assert request["thinking"] == {"type": "adaptive"}
+ assert isinstance(request["temperature"], Omit)
+
+ async def test_a_model_refusing_temperature_omits_it_without_thinking(self, mocker: MockerFixture) -> None:
+ worker, stream = _make_text_worker(mocker, accepts_temperature=False)
+
+ await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert isinstance(_sent_request(stream)["temperature"], Omit)
diff --git a/tests/unit/pipelex/providers/bedrock/test_bedrock_converse_request.py b/tests/unit/pipelex/providers/bedrock/test_bedrock_converse_request.py
new file mode 100644
index 000000000..686b4ec23
--- /dev/null
+++ b/tests/unit/pipelex/providers/bedrock/test_bedrock_converse_request.py
@@ -0,0 +1,108 @@
+"""The Converse request the Bedrock clients send, and the temperature the Bedrock worker hands them.
+
+Both clients build their request with ``make_converse_params``. A temperature is sent unless the model lists
+`temperature_unsupported`, whose provider refuses one: the worker then hands the client none, and the request's
+inference config carries no temperature key at all.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.providers.bedrock.bedrock_client_boto3 import BedrockClientBoto3
+from pipelex.providers.bedrock.bedrock_client_protocol import make_converse_params
+from pipelex.providers.bedrock.bedrock_llm_worker import BedrockLLMWorker
+
+if TYPE_CHECKING:
+ from unittest.mock import AsyncMock
+
+ from pytest_mock import MockerFixture
+
+_MESSAGES: list[dict[str, Any]] = [{"role": "user", "content": [{"text": "Hello."}]}]
+
+
+class _RecordingConverse:
+ """Stands in for the boto3 runtime client: records the parameters ``converse`` receives."""
+
+ def __init__(self) -> None:
+ self.params: list[dict[str, Any]] = []
+
+ def converse(self, **params: Any) -> dict[str, Any]:
+ self.params.append(params)
+ return {"usage": {"inputTokens": 1, "outputTokens": 2}, "output": {"message": {"content": [{"text": "answer"}]}}}
+
+
+def _make_worker(mocker: MockerFixture, *, accepts_temperature: bool) -> tuple[BedrockLLMWorker, AsyncMock]:
+ worker = object.__new__(BedrockLLMWorker)
+ model = mocker.MagicMock()
+ model.desc = "test-model-desc"
+ model.model_id = "mistral.test"
+ model.name = "mistral.test"
+ model.accepts_temperature = accepts_temperature
+ worker.inference_model = model
+ worker.default_max_tokens = 4096
+ chat = mocker.AsyncMock(return_value=("answer", {}))
+ client = mocker.MagicMock()
+ client.chat = chat
+ worker.bedrock_client_for_text = client
+ return worker, chat
+
+
+def _make_llm_job(mocker: MockerFixture) -> Any:
+ job = mocker.MagicMock()
+ job.applied_job_params = None
+ job.job_params.temperature = 0.5
+ job.job_params.max_tokens = None
+ job.job_params.reasoning_effort = None
+ job.job_params.reasoning_budget = None
+ job.job_report.llm_tokens_usage = None
+ job.llm_prompt.system_text = "system"
+ job.llm_prompt.user_text = "Hello."
+ job.llm_prompt.user_images = []
+ return job
+
+
+class TestMakeConverseParams:
+ def test_a_temperature_goes_in_the_inference_config(self) -> None:
+ params = make_converse_params(messages=_MESSAGES, system_text="Be brief.", model="mistral.test", temperature=0.5, max_tokens=100)
+
+ assert params == {
+ "modelId": "mistral.test",
+ "messages": _MESSAGES,
+ "inferenceConfig": {"temperature": 0.5, "maxTokens": 100},
+ "system": [{"text": "Be brief."}],
+ }
+
+ def test_without_a_temperature_the_inference_config_has_no_temperature_key(self) -> None:
+ params = make_converse_params(messages=_MESSAGES, system_text=None, model="mistral.test", temperature=None, max_tokens=100)
+
+ assert params == {"modelId": "mistral.test", "messages": _MESSAGES, "inferenceConfig": {"maxTokens": 100}}
+
+
[email protected](loop_scope="class")
+class TestBedrockConverseRequest:
+ async def test_the_boto3_client_sends_the_built_request(self) -> None:
+ client = BedrockClientBoto3(aws_region="us-east-1")
+ recording = _RecordingConverse()
+ client.boto3_client = recording
+
+ text, _ = await client.chat(messages=_MESSAGES, system_text=None, model="mistral.test", temperature=None, max_tokens=100)
+
+ assert text == "answer"
+ assert recording.params == [
+ make_converse_params(messages=_MESSAGES, system_text=None, model="mistral.test", temperature=None, max_tokens=100)
+ ]
+
+ @pytest.mark.parametrize(("accepts_temperature", "expected_temperature"), [(True, 0.5), (False, None)])
+ async def test_the_worker_hands_the_client_a_temperature_only_when_the_model_takes_one(
+ self, mocker: MockerFixture, accepts_temperature: bool, expected_temperature: float | None
+ ) -> None:
+ worker, chat = _make_worker(mocker, accepts_temperature=accepts_temperature)
+
+ await worker._gen_text(llm_job=_make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ chat.assert_awaited_once()
+ assert chat.await_args is not None
+ assert chat.await_args.kwargs["temperature"] == expected_temperature
diff --git a/tests/unit/pipelex/providers/google/test_google_llm_worker_structured_request.py b/tests/unit/pipelex/providers/google/test_google_llm_worker_structured_request.py
index a931e0269..ae5549cad 100644
--- a/tests/unit/pipelex/providers/google/test_google_llm_worker_structured_request.py
+++ b/tests/unit/pipelex/providers/google/test_google_llm_worker_structured_request.py
@@ -74,6 +74,15 @@ class TestGoogleLLMWorkerStructuredRequest:
assert config.temperature == 0.5
assert config.max_output_tokens is None
+ @pytest.mark.parametrize("structure_method", GENAI_STRUCTURE_METHODS)
+ async def test_a_model_refusing_temperature_gets_none(self, mocker: MockerFixture, structure_method: StructureMethod) -> None:
+ captured: dict[str, Any] = {}
+ worker = make_structured_worker(mocker, structure_method=structure_method, captured=captured, accepts_temperature=False)
+
+ await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert captured["config"].temperature is None
+
@pytest.mark.parametrize("structure_method", GENAI_STRUCTURE_METHODS)
async def test_an_enum_field_is_parsed_from_its_string_value(self, mocker: MockerFixture, structure_method: StructureMethod) -> None:
"""Gemini hands an enum back as its string value, under tool calling and JSON output alike."""
diff --git a/tests/unit/pipelex/providers/google/test_google_llm_worker_text_request.py b/tests/unit/pipelex/providers/google/test_google_llm_worker_text_request.py
new file mode 100644
index 000000000..dbd18167f
--- /dev/null
+++ b/tests/unit/pipelex/providers/google/test_google_llm_worker_text_request.py
@@ -0,0 +1,43 @@
+"""The temperature the Google LLM worker's text call sends.
+
+Gemini keeps a temperature beside thinking, so one is sent unless the model lists `temperature_unsupported`,
+whose provider refuses one.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from google.genai import types as genai_types
+
+from pipelex.cogt.llm.structured_output import StructureMethod
+from tests.helpers.google_structured_request import make_structured_worker
+from tests.helpers.instructor_test_utils import make_llm_job
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
[email protected](loop_scope="class")
+class TestGoogleLLMWorkerTextRequest:
+ async def test_the_temperature_is_sent(self, mocker: MockerFixture) -> None:
+ captured: dict[str, Any] = {}
+ worker = make_structured_worker(mocker, structure_method=StructureMethod.INSTRUCTOR_GENAI_STRUCTURED_OUTPUTS, captured=captured)
+
+ result = await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert result == '{"text": "ok"}'
+ config = captured["config"]
+ assert isinstance(config, genai_types.GenerateContentConfig)
+ assert config.temperature == 0.5
+
+ async def test_a_model_refusing_temperature_gets_none(self, mocker: MockerFixture) -> None:
+ captured: dict[str, Any] = {}
+ worker = make_structured_worker(
+ mocker, structure_method=StructureMethod.INSTRUCTOR_GENAI_STRUCTURED_OUTPUTS, captured=captured, accepts_temperature=False
+ )
+
+ await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert captured["config"].temperature is None
diff --git a/tests/unit/pipelex/providers/mistral/test_mistral_structured_request.py b/tests/unit/pipelex/providers/mistral/test_mistral_structured_request.py
index b7169bdda..61031ead9 100644
--- a/tests/unit/pipelex/providers/mistral/test_mistral_structured_request.py
+++ b/tests/unit/pipelex/providers/mistral/test_mistral_structured_request.py
@@ -46,6 +46,13 @@ class TestMistralStructuredRequest:
assert is_unset(request, "reasoning_effort")
assert request["temperature"] == 0.5
+ async def test_a_model_refusing_temperature_omits_it(self, mocker: MockerFixture) -> None:
+ worker, complete_async = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=tool_call_response(), accepts_temperature=False)
+
+ await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert is_unset(sent_request(complete_async), "temperature")
+
async def test_a_model_without_reasoning_refuses_it_as_its_text_path_does(self, mocker: MockerFixture) -> None:
worker, complete_async = make_worker(mocker, thinking_mode=ThinkingMode.NONE, response=tool_call_response())
llm_job = make_llm_job(mocker)
diff --git a/tests/unit/pipelex/providers/mistral/test_mistral_text_request.py b/tests/unit/pipelex/providers/mistral/test_mistral_text_request.py
index 2d9ae403e..c2f2fb2c4 100644
--- a/tests/unit/pipelex/providers/mistral/test_mistral_text_request.py
+++ b/tests/unit/pipelex/providers/mistral/test_mistral_text_request.py
@@ -1,6 +1,7 @@
"""The request the Mistral worker's text call sends, with and without a reasoning setting.
-Mistral refuses `prompt_mode="reasoning"` on every current model and takes `reasoning_effort` instead.
+Mistral refuses `prompt_mode="reasoning"` on every current model and takes `reasoning_effort` instead. It takes a
+temperature beside it, so one is sent unless the model lists `temperature_unsupported`.
"""
from __future__ import annotations
@@ -37,4 +38,13 @@ class TestMistralTextRequest:
await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
- assert is_unset(sent_request(complete_async), "reasoning_effort")
+ request = sent_request(complete_async)
+ assert is_unset(request, "reasoning_effort")
+ assert request["temperature"] == 0.5
+
+ async def test_a_model_refusing_temperature_omits_it(self, mocker: MockerFixture) -> None:
+ worker, complete_async = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_response(), accepts_temperature=False)
+
+ await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert is_unset(sent_request(complete_async), "temperature")
diff --git a/tests/unit/pipelex/providers/openai/test_openai_completions_structured_request.py b/tests/unit/pipelex/providers/openai/test_openai_completions_structured_request.py
index fcbf79178..6b03e9f1b 100644
--- a/tests/unit/pipelex/providers/openai/test_openai_completions_structured_request.py
+++ b/tests/unit/pipelex/providers/openai/test_openai_completions_structured_request.py
@@ -9,130 +9,52 @@ Gemini through this worker take it.
from __future__ import annotations
-import json
-from typing import TYPE_CHECKING, Any
+from typing import TYPE_CHECKING
import pytest
-from openai import AsyncOpenAI, Omit
-from openai.types.chat import ChatCompletion, ChatCompletionMessage, ChatCompletionMessageToolCall
-from openai.types.chat.chat_completion import Choice
-from openai.types.chat.chat_completion_message_tool_call import Function
from pipelex.cogt.exceptions import LLMCapabilityError
from pipelex.cogt.llm.llm_job_components import ReasoningEffort
-from pipelex.cogt.llm.structured_output import StructureMethod
from pipelex.cogt.llm.thinking_mode import ThinkingMode
-from pipelex.cogt.model_backends.constraints import ListedConstraint
-from pipelex.providers.openai.openai_completions_factory import OpenAICompletionsFactory
-from pipelex.providers.openai.openai_completions_llm_worker import OpenAICompletionsLLMWorker
-from pipelex.providers.openai.openai_config import OpenAIConfig
from tests.helpers.instructor_test_utils import DummySchema, make_llm_job
+from tests.helpers.openai_completions_request import is_omitted, make_worker, sent_request, tool_call_completion
if TYPE_CHECKING:
- from unittest.mock import AsyncMock
-
from pytest_mock import MockerFixture
-_OPENAI_LEVEL_MAP: dict[str, str] = {
- "none": "none",
- "minimal": "minimal",
- "low": "low",
- "medium": "medium",
- "high": "high",
- "xhigh": "xhigh",
- "max": "xhigh",
-}
-
-
-def _tool_call_completion() -> ChatCompletion:
- tool_call = ChatCompletionMessageToolCall(
- id="call_test",
- type="function",
- function=Function(name="DummySchema", arguments=json.dumps({"text": "answer"})),
- )
- return ChatCompletion(
- id="chatcmpl_test",
- object="chat.completion",
- created=0,
- model="test-model",
- choices=[Choice(index=0, finish_reason="tool_calls", message=ChatCompletionMessage(role="assistant", content=None, tool_calls=[tool_call]))],
- )
-
-
-def _make_worker(
- mocker: MockerFixture,
- *,
- thinking_mode: ThinkingMode,
- listed_constraints: list[ListedConstraint] | None = None,
-) -> tuple[OpenAICompletionsLLMWorker, AsyncMock]:
- """A worker whose instructor client is built as the worker builds it, over an SDK call that records the request."""
- from instructor import from_openai # ruff: ignore[import-outside-top-level]
-
- sdk_client = AsyncOpenAI(api_key="test-key")
- create = mocker.AsyncMock(return_value=_tool_call_completion())
- mocker.patch.object(sdk_client.chat.completions, "create", new=create)
-
- worker = object.__new__(OpenAICompletionsLLMWorker)
- model = mocker.MagicMock()
- model.desc = "test-model-desc"
- model.model_id = "test-model"
- model.name = "test-model"
- model.thinking_mode = thinking_mode
- model.extra_headers = None
- model.listed_constraints = listed_constraints or []
- worker.inference_model = model
- worker.openai_completions_factory = OpenAICompletionsFactory(is_http_url_enabled=False)
- worker.instructor_for_objects = from_openai(client=sdk_client, mode=StructureMethod.INSTRUCTOR_OPENAI_TOOLS.as_instructor_mode())
-
- config = mocker.MagicMock()
- config.inference.llm.openai = OpenAIConfig(effort_to_level_map=_OPENAI_LEVEL_MAP)
- mocker.patch("pipelex.providers.openai.openai_completions_llm_worker.get_config", return_value=config)
- return worker, create
-
-
-def _sent_request(create: AsyncMock) -> dict[str, Any]:
- create.assert_awaited_once()
- await_args = create.await_args
- assert await_args is not None
- return dict(await_args.kwargs)
-
-
-def _is_omitted(request: dict[str, Any], key: str) -> bool:
- return key not in request or isinstance(request[key], Omit)
-
@pytest.mark.asyncio(loop_scope="class")
class TestOpenAICompletionsStructuredRequest:
async def test_a_reasoning_effort_reaches_the_api_and_drops_the_temperature(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.MANUAL)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=tool_call_completion())
llm_job = make_llm_job(mocker)
llm_job.job_params.reasoning_effort = ReasoningEffort.LOW
result = await worker._gen_object(llm_job=llm_job, schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
assert result.text == "answer"
- request = _sent_request(create)
+ request = sent_request(create)
assert request["reasoning_effort"] == "low"
- assert _is_omitted(request, "temperature")
+ assert is_omitted(request, "temperature")
async def test_without_a_reasoning_setting_the_request_is_unchanged(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.MANUAL)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=tool_call_completion())
await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
- request = _sent_request(create)
- assert _is_omitted(request, "reasoning_effort")
+ request = sent_request(create)
+ assert is_omitted(request, "reasoning_effort")
assert request["temperature"] == 0.5
async def test_a_model_refusing_temperature_still_omits_it_without_reasoning(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, listed_constraints=[ListedConstraint.TEMPERATURE_UNSUPPORTED])
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=tool_call_completion(), accepts_temperature=False)
await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
- assert _is_omitted(_sent_request(create), "temperature")
+ assert is_omitted(sent_request(create), "temperature")
async def test_a_model_without_reasoning_refuses_it_as_its_text_path_does(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.NONE)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.NONE, response=tool_call_completion())
llm_job = make_llm_job(mocker)
llm_job.job_params.reasoning_effort = ReasoningEffort.HIGH
diff --git a/tests/unit/pipelex/providers/openai/test_openai_completions_text_request.py b/tests/unit/pipelex/providers/openai/test_openai_completions_text_request.py
new file mode 100644
index 000000000..be8176d79
--- /dev/null
+++ b/tests/unit/pipelex/providers/openai/test_openai_completions_text_request.py
@@ -0,0 +1,50 @@
+"""The temperature the OpenAI chat-completions worker's text call sends.
+
+A temperature is sent unless a reasoning effort is, or the model lists `temperature_unsupported`, whose
+provider refuses one.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import pytest
+
+from pipelex.cogt.llm.llm_job_components import ReasoningEffort
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from tests.helpers.instructor_test_utils import make_llm_job
+from tests.helpers.openai_completions_request import is_omitted, make_worker, sent_request, text_completion
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
[email protected](loop_scope="class")
+class TestOpenAICompletionsTextRequest:
+ async def test_without_a_reasoning_setting_the_temperature_is_sent(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_completion())
+
+ result = await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert result == "answer"
+ request = sent_request(create)
+ assert request["temperature"] == 0.5
+ assert is_omitted(request, "reasoning_effort")
+
+ async def test_a_reasoning_effort_drops_the_temperature(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_completion())
+ llm_job = make_llm_job(mocker)
+ llm_job.job_params.reasoning_effort = ReasoningEffort.LOW
+
+ await worker._gen_text(llm_job=llm_job) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ request = sent_request(create)
+ assert request["reasoning_effort"] == "low"
+ assert is_omitted(request, "temperature")
+
+ async def test_a_model_refusing_temperature_omits_it_without_reasoning(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_completion(), accepts_temperature=False)
+
+ await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert is_omitted(sent_request(create), "temperature")
diff --git a/tests/unit/pipelex/providers/openai/test_openai_responses_structured_request.py b/tests/unit/pipelex/providers/openai/test_openai_responses_structured_request.py
index 2ef57bfd6..cb532482a 100644
--- a/tests/unit/pipelex/providers/openai/test_openai_responses_structured_request.py
+++ b/tests/unit/pipelex/providers/openai/test_openai_responses_structured_request.py
@@ -8,116 +8,54 @@ preset on a structured pipe reaches the gateway through exactly this request.
from __future__ import annotations
-import json
-from typing import TYPE_CHECKING, Any
+from typing import TYPE_CHECKING
import pytest
-from openai import AsyncOpenAI, Omit
-from openai.types.responses import Response, ResponseFunctionToolCall
from pipelex.cogt.exceptions import LLMCapabilityError
from pipelex.cogt.llm.llm_job_components import ReasoningEffort
-from pipelex.cogt.llm.structured_output import StructureMethod
from pipelex.cogt.llm.thinking_mode import ThinkingMode
-from pipelex.providers.openai.openai_config import OpenAIConfig
-from pipelex.providers.openai.openai_responses_factory import OpenAIResponsesFactory
-from pipelex.providers.openai.openai_responses_llm_worker import OpenAIResponsesLLMWorker
from tests.helpers.instructor_test_utils import DummySchema, make_llm_job
+from tests.helpers.openai_responses_request import function_call_response, is_omitted, make_worker, sent_request
if TYPE_CHECKING:
- from unittest.mock import AsyncMock
-
from pytest_mock import MockerFixture
-_OPENAI_LEVEL_MAP: dict[str, str] = {
- "none": "none",
- "minimal": "minimal",
- "low": "low",
- "medium": "medium",
- "high": "high",
- "xhigh": "xhigh",
- "max": "xhigh",
-}
-
-
-def _function_call_response() -> Response:
- return Response(
- id="resp_test",
- created_at=0,
- model="gpt-test",
- object="response",
- output=[ResponseFunctionToolCall(type="function_call", call_id="call_test", name="DummySchema", arguments=json.dumps({"text": "answer"}))],
- parallel_tool_calls=False,
- tool_choice="auto",
- tools=[],
- )
-
-
-def _make_worker(mocker: MockerFixture, *, thinking_mode: ThinkingMode) -> tuple[OpenAIResponsesLLMWorker, AsyncMock]:
- """A worker whose instructor client is built as the worker builds it, over an SDK call that records the request."""
- from instructor import from_openai # ruff: ignore[import-outside-top-level]
-
- sdk_client = AsyncOpenAI(api_key="test-key")
- create = mocker.AsyncMock(return_value=_function_call_response())
- mocker.patch.object(sdk_client.responses, "create", new=create)
-
- worker = object.__new__(OpenAIResponsesLLMWorker)
- model = mocker.MagicMock()
- model.desc = "test-model-desc"
- model.model_id = "gpt-test"
- model.name = "gpt-test"
- model.thinking_mode = thinking_mode
- model.extra_headers = None
- worker.inference_model = model
- factory = OpenAIResponsesFactory(is_http_url_enabled=False)
- mocker.patch.object(factory, "make_input_items", new_callable=mocker.AsyncMock, return_value=[{"role": "user", "content": "Generate."}])
- worker.openai_responses_factory = factory
- worker.instructor_for_objects = from_openai(client=sdk_client, mode=StructureMethod.INSTRUCTOR_OPENAI_RESPONSES_TOOLS.as_instructor_mode())
-
- config = mocker.MagicMock()
- config.inference.llm.openai = OpenAIConfig(effort_to_level_map=_OPENAI_LEVEL_MAP)
- mocker.patch("pipelex.providers.openai.openai_responses_llm_worker.get_config", return_value=config)
- return worker, create
-
-
-def _sent_request(create: AsyncMock) -> dict[str, Any]:
- create.assert_awaited_once()
- await_args = create.await_args
- assert await_args is not None
- return dict(await_args.kwargs)
-
-
-def _is_omitted(request: dict[str, Any], key: str) -> bool:
- return key not in request or isinstance(request[key], Omit)
-
@pytest.mark.asyncio(loop_scope="class")
class TestOpenAIResponsesStructuredRequest:
async def test_a_reasoning_effort_reaches_the_api_and_drops_the_temperature(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.MANUAL)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=function_call_response())
llm_job = make_llm_job(mocker)
llm_job.job_params.reasoning_effort = ReasoningEffort.HIGH
result = await worker._gen_object(llm_job=llm_job, schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
assert result.text == "answer"
- request = _sent_request(create)
+ request = sent_request(create)
assert request["reasoning"] == {"effort": "high"}
- assert _is_omitted(request, "temperature")
+ assert is_omitted(request, "temperature")
# The structuring path itself is unchanged: the response tool is still the forced one
assert request["tool_choice"] == {"type": "function", "name": "DummySchema"}
async def test_without_a_reasoning_setting_the_request_is_unchanged(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.MANUAL)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=function_call_response())
await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
- request = _sent_request(create)
- assert _is_omitted(request, "reasoning")
+ request = sent_request(create)
+ assert is_omitted(request, "reasoning")
assert request["temperature"] == 0.5
+ async def test_a_model_refusing_temperature_omits_it_without_reasoning(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=function_call_response(), accepts_temperature=False)
+
+ await worker._gen_object(llm_job=make_llm_job(mocker), schema=DummySchema) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert is_omitted(sent_request(create), "temperature")
+
async def test_a_model_without_reasoning_refuses_it_as_its_text_path_does(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.NONE)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.NONE, response=function_call_response())
llm_job = make_llm_job(mocker)
llm_job.job_params.reasoning_effort = ReasoningEffort.HIGH
@@ -126,7 +64,7 @@ class TestOpenAIResponsesStructuredRequest:
create.assert_not_awaited()
async def test_a_reasoning_budget_is_refused_as_its_text_path_does(self, mocker: MockerFixture) -> None:
- worker, create = _make_worker(mocker, thinking_mode=ThinkingMode.MANUAL)
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=function_call_response())
llm_job = make_llm_job(mocker)
llm_job.job_params.reasoning_budget = 4096
diff --git a/tests/unit/pipelex/providers/openai/test_openai_responses_text_request.py b/tests/unit/pipelex/providers/openai/test_openai_responses_text_request.py
new file mode 100644
index 000000000..79351cba4
--- /dev/null
+++ b/tests/unit/pipelex/providers/openai/test_openai_responses_text_request.py
@@ -0,0 +1,50 @@
+"""The temperature the OpenAI Responses worker's text call sends.
+
+A temperature is sent unless a reasoning setting is, or the model lists `temperature_unsupported`, whose
+provider refuses one.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import pytest
+
+from pipelex.cogt.llm.llm_job_components import ReasoningEffort
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from tests.helpers.instructor_test_utils import make_llm_job
+from tests.helpers.openai_responses_request import is_omitted, make_worker, sent_request, text_response
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
[email protected](loop_scope="class")
+class TestOpenAIResponsesTextRequest:
+ async def test_without_a_reasoning_setting_the_temperature_is_sent(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_response())
+
+ result = await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert result == "answer"
+ request = sent_request(create)
+ assert request["temperature"] == 0.5
+ assert is_omitted(request, "reasoning")
+
+ async def test_a_reasoning_effort_drops_the_temperature(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_response())
+ llm_job = make_llm_job(mocker)
+ llm_job.job_params.reasoning_effort = ReasoningEffort.LOW
+
+ await worker._gen_text(llm_job=llm_job) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ request = sent_request(create)
+ assert request["reasoning"] == {"effort": "low"}
+ assert is_omitted(request, "temperature")
+
+ async def test_a_model_refusing_temperature_omits_it_without_reasoning(self, mocker: MockerFixture) -> None:
+ worker, create = make_worker(mocker, thinking_mode=ThinkingMode.MANUAL, response=text_response(), accepts_temperature=False)
+
+ await worker._gen_text(llm_job=make_llm_job(mocker)) # ruff: ignore[private-member-access] # pyright: ignore[reportPrivateUsage]
+
+ assert is_omitted(sent_request(create), "temperature")