Skip to content

Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

Python

Honor reasoning tiers on numeric-budget providers (#1150)

Ali Khokhar65a342ede49b
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 8bd007f0..6047a722 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -611,14 +611,18 @@ provider-neutral `ReasoningPolicy`. It represents three distinct facts:
- `control`: provider default, explicitly off, or explicitly on;
- `effort`: the client's named effort when one was supplied;
-- `budget_tokens`: an exact positive client budget, never a derived value.
+- `budget_tokens`: an exact positive client budget when one was supplied.
+
+When a numeric-budget provider needs a budget, `ReasoningPolicy` expresses named
+effort through FCC's single product scale: `minimal`/`low=512`, `medium=1024`,
+`high=2048`, `xhigh=4096`, and `max=8192`. Exact client budgets take precedence.
The application layer resolves configuration and client input into this value;
the API layer may replace it for a product policy such as the safety classifier;
providers receive it unchanged. Provider adapters alone translate the subset
their documented wire API can represent. The shared OpenAI-chat implementation
uses small encoder objects for named effort, reasoning objects, thinking
-objects, chat-template booleans, exact llama.cpp budgets, and split reasoning
+objects, chat-template booleans, numeric llama.cpp budgets, and split reasoning
output. Specialized providers keep only translations that cannot be expressed
by those encoders.
@@ -631,11 +635,11 @@ for a valid continuation.
The boundary has four hard rules:
1. Never inspect an upstream model name or version to select reasoning behavior.
-2. Never convert a named effort into a fabricated token budget, or use the
- output-token limit as a reasoning budget.
-3. Forward an exact token budget only where the provider documents one; otherwise
- translate a supported named or boolean control and leave unsupported precision
- to the provider.
+2. Prefer a provider's named effort vocabulary; use FCC's documented numeric
+ scale only when the provider exposes a numeric budget rather than named effort.
+3. Never use the output-token limit as a reasoning budget. Forward exact or
+ FCC-mapped budgets only through documented numeric fields; otherwise translate
+ a supported named or boolean control and leave unsupported precision upstream.
4. Provider-default intent emits no compute-control field. Explicit off requests
an upstream disable where supported and always suppresses reasoning output at
the FCC protocol boundary.
diff --git a/README.md b/README.md
index 7c377a94..2497fdd3 100644
--- a/README.md
+++ b/README.md
@@ -205,7 +205,7 @@ For example, route Opus to `nvidia_nim/moonshotai/kimi-k2.6`, Sonnet to `open_ro
Open **Admin UI → Model Config → Reasoning** to choose how FCC handles client reasoning controls. The default **From client** option preserves reasoning effort sent by Claude Code, Codex, or Pi; when the client sends no control, the provider keeps its own default.
-You can instead select **Off**, **Low**, **Medium**, **High**, **X-High**, or **Max**. Fable, Opus, Sonnet, and Haiku each have the same choices plus **Inherit**, which uses the root policy. FCC translates each choice only into controls documented by that provider, so unsupported precision safely remains provider-defined.
+You can instead select **Off**, **Low**, **Medium**, **High**, **X-High**, or **Max**. Fable, Opus, Sonnet, and Haiku each have the same choices plus **Inherit**, which uses the root policy. Providers with named effort receive those names; numeric-budget providers map **Low=512**, **Medium=1,024**, **High=2,048**, **X-High=4,096**, and **Max=8,192** reasoning tokens; boolean providers receive on or off. Unsupported controls safely remain provider-defined.
<a id="connect-your-client"></a>
diff --git a/pyproject.toml b/pyproject.toml
index de3b4fa8..f60c1972 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "free-claude-code"
-version = "4.8.1"
+version = "4.8.2"
description = "Local proxy connecting coding agents to OpenAI-compatible AI providers"
readme = "README.md"
requires-python = ">=3.14.0"
diff --git a/src/free_claude_code/core/reasoning.py b/src/free_claude_code/core/reasoning.py
index 6e8dd6ae..252d8bc2 100644
--- a/src/free_claude_code/core/reasoning.py
+++ b/src/free_claude_code/core/reasoning.py
@@ -22,6 +22,22 @@ class ReasoningEffort(StrEnum):
XHIGH = "xhigh"
MAX = "max"
+ @property
+ def budget_tokens(self) -> int:
+ """Return FCC's numeric token budget for this effort."""
+
+ return _EFFORT_BUDGET_TOKENS[self]
+
+
+_EFFORT_BUDGET_TOKENS = {
+ ReasoningEffort.MINIMAL: 512,
+ ReasoningEffort.LOW: 512,
+ ReasoningEffort.MEDIUM: 1_024,
+ ReasoningEffort.HIGH: 2_048,
+ ReasoningEffort.XHIGH: 4_096,
+ ReasoningEffort.MAX: 8_192,
+}
+
@dataclass(frozen=True, slots=True)
class ReasoningPolicy:
@@ -89,5 +105,17 @@ class ReasoningPolicy:
or self.budget_tokens is not None
)
+ @property
+ def numeric_budget_tokens(self) -> int | None:
+ """Express this intent as an exact or FCC-mapped numeric budget."""
+
+ if self.control is ReasoningControl.OFF:
+ return None
+ if self.budget_tokens is not None:
+ return self.budget_tokens
+ if self.effort is None:
+ return None
+ return self.effort.budget_tokens
+
DEFAULT_REASONING_POLICY = ReasoningPolicy.provider_default()
diff --git a/src/free_claude_code/providers/nvidia_nim/request_options.py b/src/free_claude_code/providers/nvidia_nim/request_options.py
index f218916a..5c4cc9a0 100644
--- a/src/free_claude_code/providers/nvidia_nim/request_options.py
+++ b/src/free_claude_code/providers/nvidia_nim/request_options.py
@@ -79,6 +79,7 @@ def apply_nim_request_options(
extra_body.update(deepcopy(request_extra))
for key in (
"reasoning",
+ "reasoning_budget",
"reasoning_effort",
"reasoning_tokens",
"thinking",
@@ -98,8 +99,8 @@ def apply_nim_request_options(
enabled = reasoning.control is not ReasoningControl.OFF
chat_template_kwargs["thinking"] = enabled
chat_template_kwargs["enable_thinking"] = enabled
- if enabled and reasoning.budget_tokens is not None:
- chat_template_kwargs["reasoning_budget"] = reasoning.budget_tokens
+ if enabled and (budget := reasoning.numeric_budget_tokens) is not None:
+ chat_template_kwargs["reasoning_budget"] = budget
req_top_k = request_data.top_k
top_k = req_top_k if req_top_k is not None else nim.top_k
diff --git a/src/free_claude_code/providers/openai_chat/reasoning.py b/src/free_claude_code/providers/openai_chat/reasoning.py
index 37412798..0c36a82f 100644
--- a/src/free_claude_code/providers/openai_chat/reasoning.py
+++ b/src/free_claude_code/providers/openai_chat/reasoning.py
@@ -110,13 +110,13 @@ class ChatTemplateReasoning:
@dataclass(frozen=True, slots=True)
class LlamaCppReasoning:
- """Encode llama.cpp's exact per-request numeric thinking budget."""
+ """Encode llama.cpp's per-request numeric thinking budget."""
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
if policy.control is ReasoningControl.OFF:
_extra_body(body)["thinking_budget_tokens"] = 0
- elif policy.budget_tokens is not None:
- _extra_body(body)["thinking_budget_tokens"] = policy.budget_tokens
+ elif (budget := policy.numeric_budget_tokens) is not None:
+ _extra_body(body)["thinking_budget_tokens"] = budget
@dataclass(frozen=True, slots=True)
diff --git a/tests/application/test_reasoning.py b/tests/application/test_reasoning.py
index f3572ce6..fbbc527a 100644
--- a/tests/application/test_reasoning.py
+++ b/tests/application/test_reasoning.py
@@ -40,7 +40,7 @@ def test_client_reasoning_preserves_effort_and_exact_budget() -> None:
)
-def test_named_effort_does_not_invent_a_token_budget() -> None:
+def test_named_effort_preserves_intent_without_exact_client_budget() -> None:
policy = client_reasoning_policy(_request(output_config={"effort": "high"}))
assert policy == ReasoningPolicy(
@@ -127,3 +127,43 @@ def test_reasoning_budget_requires_a_positive_integer(budget: int) -> None:
def test_reasoning_budget_requires_explicit_on_control() -> None:
with pytest.raises(ValueError, match="control to be on"):
ReasoningPolicy(budget_tokens=100)
+
+
+ ("effort", "expected"),
+ (
+ (ReasoningEffort.MINIMAL, 512),
+ (ReasoningEffort.LOW, 512),
+ (ReasoningEffort.MEDIUM, 1_024),
+ (ReasoningEffort.HIGH, 2_048),
+ (ReasoningEffort.XHIGH, 4_096),
+ (ReasoningEffort.MAX, 8_192),
+ ),
+)
+def test_reasoning_effort_has_one_fcc_numeric_budget(
+ effort: ReasoningEffort, expected: int
+) -> None:
+ assert ReasoningPolicy.on(effort=effort).numeric_budget_tokens == expected
+
+
+def test_exact_reasoning_budget_takes_precedence_over_effort_mapping() -> None:
+ policy = ReasoningPolicy.on(
+ effort=ReasoningEffort.XHIGH,
+ budget_tokens=777,
+ )
+
+ assert policy.numeric_budget_tokens == 777
+
+
+ "policy",
+ (
+ ReasoningPolicy.provider_default(),
+ ReasoningPolicy.off(),
+ ReasoningPolicy.on(),
+ ),
+)
+def test_reasoning_without_numeric_intensity_has_no_budget(
+ policy: ReasoningPolicy,
+) -> None:
+ assert policy.numeric_budget_tokens is None
diff --git a/tests/providers/test_nvidia_nim.py b/tests/providers/test_nvidia_nim.py
index 9b6f12f1..a46c906f 100644
--- a/tests/providers/test_nvidia_nim.py
+++ b/tests/providers/test_nvidia_nim.py
@@ -8,7 +8,7 @@ from httpx import Request, Response
from free_claude_code.config.nim import NimSettings
from free_claude_code.config.provider_catalog import NVIDIA_NIM_DEFAULT_BASE
from free_claude_code.core.failures import ExecutionFailure
-from free_claude_code.core.reasoning import ReasoningPolicy
+from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from free_claude_code.providers.nvidia_nim.tool_schema import (
NIM_TOOL_ARGUMENT_ALIASES_KEY,
@@ -767,14 +767,15 @@ async def test_stream_response_retries_without_reasoning_budget(nim_provider):
events = [
e
async for e in nim_provider.stream_response(
- req, reasoning=ReasoningPolicy.on(budget_tokens=77)
+ req,
+ reasoning=ReasoningPolicy.on(effort=ReasoningEffort.XHIGH),
)
]
assert mock_create.await_count == 2
first_call = mock_create.await_args_list[0].kwargs
second_call = mock_create.await_args_list[1].kwargs
- assert first_call["extra_body"]["chat_template_kwargs"]["reasoning_budget"] == 77
+ assert first_call["extra_body"]["chat_template_kwargs"]["reasoning_budget"] == 4096
assert "reasoning_budget" not in second_call["extra_body"]
assert "reasoning_budget" not in second_call["extra_body"]["chat_template_kwargs"]
assert second_call["extra_body"]["chat_template_kwargs"]["enable_thinking"] is True
diff --git a/tests/providers/test_nvidia_nim_request.py b/tests/providers/test_nvidia_nim_request.py
index f9d95128..c45ec4b6 100644
--- a/tests/providers/test_nvidia_nim_request.py
+++ b/tests/providers/test_nvidia_nim_request.py
@@ -101,14 +101,59 @@ class TestSetExtra:
class TestBuildRequestBody:
- def test_named_effort_enables_boolean_chat_template_control(self, req):
- policy = ReasoningPolicy(effort=ReasoningEffort.HIGH)
+ @pytest.mark.parametrize(
+ ("effort", "expected_budget"),
+ (
+ (ReasoningEffort.MINIMAL, 512),
+ (ReasoningEffort.LOW, 512),
+ (ReasoningEffort.MEDIUM, 1_024),
+ (ReasoningEffort.HIGH, 2_048),
+ (ReasoningEffort.XHIGH, 4_096),
+ (ReasoningEffort.MAX, 8_192),
+ ),
+ )
+ def test_named_effort_enables_thinking_with_numeric_budget(
+ self,
+ req,
+ effort: ReasoningEffort,
+ expected_budget: int,
+ ):
+ policy = ReasoningPolicy(effort=effort)
body = build_request_body(req, NimSettings(), reasoning=policy)
assert body["extra_body"]["chat_template_kwargs"] == {
"thinking": True,
"enable_thinking": True,
+ "reasoning_budget": expected_budget,
+ }
+
+ def test_named_effort_replaces_client_reasoning_budgets(self):
+ req = make_messages_request(
+ model="test",
+ thinking=None,
+ extra_body={
+ "reasoning_budget": 99,
+ "chat_template_kwargs": {
+ "reasoning_budget": 100,
+ "custom": "value",
+ },
+ },
+ )
+
+ body = build_request_body(
+ req,
+ NimSettings(),
+ reasoning=ReasoningPolicy(effort=ReasoningEffort.HIGH),
+ )
+
+ extra_body = body["extra_body"]
+ assert "reasoning_budget" not in extra_body
+ assert extra_body["chat_template_kwargs"] == {
+ "custom": "value",
+ "thinking": True,
+ "enable_thinking": True,
+ "reasoning_budget": 2048,
}
def test_max_tokens_capped_by_nim(self, req):
diff --git a/tests/providers/test_openai_chat_reasoning.py b/tests/providers/test_openai_chat_reasoning.py
index abcc6fc5..10906aa8 100644
--- a/tests/providers/test_openai_chat_reasoning.py
+++ b/tests/providers/test_openai_chat_reasoning.py
@@ -88,17 +88,17 @@ def test_chat_template_encoder_maps_named_effort_to_boolean_capability() -> None
assert body == {"extra_body": {"chat_template_kwargs": {"thinking": True}}}
-def test_llamacpp_encoder_forwards_only_exact_budget_or_off() -> None:
- default_body: dict = {}
+def test_llamacpp_encoder_maps_effort_preserves_exact_budget_and_disables() -> None:
+ effort_body: dict = {}
budget_body: dict = {}
off_body: dict = {}
encoder = LlamaCppReasoning()
- encoder.encode(default_body, ReasoningPolicy.on(effort=ReasoningEffort.HIGH))
+ encoder.encode(effort_body, ReasoningPolicy.on(effort=ReasoningEffort.HIGH))
encoder.encode(budget_body, ReasoningPolicy.on(budget_tokens=256))
encoder.encode(off_body, ReasoningPolicy.off())
- assert default_body == {}
+ assert effort_body == {"extra_body": {"thinking_budget_tokens": 2048}}
assert budget_body == {"extra_body": {"thinking_budget_tokens": 256}}
assert off_body == {"extra_body": {"thinking_budget_tokens": 0}}
diff --git a/uv.lock b/uv.lock
index e316a2b0..d096616b 100644
--- a/uv.lock
+++ b/uv.lock
@@ -561,7 +561,7 @@ wheels = [
[[package]]
name = "free-claude-code"
-version = "4.8.1"
+version = "4.8.2"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },