Skip to content

Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

Python

Coordinate provider recovery under concurrent load (#1205)

Ali Khokharaf12e7b2bb95
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 86e6044c..381482d1 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -519,27 +519,43 @@ does not select a concrete adapter.
[providers/runtime/](src/free_claude_code/providers/runtime/) owns construction details for one
closable provider generation: construction policy, resolved provider
-configuration, lazy provider instances, provider-owned rate limiters, and
+configuration, lazy provider instances, provider-owned admission controllers, and
cleanup. [providers/runtime/factory.py](src/free_claude_code/providers/runtime/factory.py)
constructs ordinary provider IDs from `OPENAI_CHAT_PROFILES` and keeps a sparse
factory mapping only for adapters with real state or algorithms. The union of
those two construction owners must exactly equal the neutral provider catalog.
-`ProviderRuntime` directly guarantees one provider and limiter per provider ID
-within a generation; there is no pass-through cache object, process singleton,
-or second limiter registry. Provider admission combines a strict proactive window with
-one reactive backoff deadline. Positive backoffs can only extend that deadline,
-and admission loops until proactive capacity and the final reactive check are
-simultaneously available. The proactive timestamp is recorded only when that
-check succeeds, so a concurrent 429/5xx cannot be missed, shortened, consume
-unused quota, or release queued requests as an expiry burst. Retired generations
-retain their own synchronization state until request leases drain, while new
-generations and separate server instances never reuse it. Hot replacement
-therefore begins with fresh quota state; an old and new generation enforce
-independent budgets while old request leases drain. Application-level generation
-publication, request leases, model metadata, discovery orchestration, and
-configured-model validation belong to `ProviderRuntimeManager` in the runtime
-package. This separates a single generation's resources from process-lifetime
-state.
+`ProviderRuntime` directly guarantees one provider and admission controller per
+provider ID within a generation; there is no pass-through cache object, process
+singleton, or second admission registry.
+
+[providers/admission.py](src/free_claude_code/providers/admission.py) owns the
+complete shared upstream-admission lifecycle for that provider generation. A
+strict sliding window admits each real attempt before a concurrency bulkhead;
+the bulkhead is held only while an upstream operation or stream is active, never
+during retry backoff. The first retryable failure before upstream acceptance
+opens one recovery episode and elects that logical execution as leader. The
+leader alone waits and sends half-open probes. Concurrent failures coalesce into
+the same episode, later callers wait, and already active streams continue. A
+stale in-flight success or failure cannot close or extend the episode, while
+leader cancellation transfers ownership to a waiter.
+
+A successful probe closes the episode and releases waiters through ordinary
+rate and concurrency admission. A non-retryable probe response also closes it
+because the provider has responded; only that request receives the rejection.
+If the leader exhausts its attempt budget, that terminal outcome stays attached
+to every coalesced logical execution, even if a later recovery generation starts.
+New work fails fast during the provider-directed cooldown; once it expires,
+exactly one new caller becomes the next probe. There is no background retry
+worker, copied request queue, or second scheduling system.
+
+Retired generations retain their own synchronization state until request leases
+drain, while new generations and separate server instances never reuse it. Hot
+replacement therefore begins with fresh quota and recovery state; an old and new
+generation enforce independent budgets while old request leases drain.
+Application-level generation publication, request leases, model metadata,
+discovery orchestration, and configured-model validation belong to
+`ProviderRuntimeManager` in the runtime package. This separates a single
+generation's resources from process-lifetime state.
[application/model_metadata.py](src/free_claude_code/application/model_metadata.py) owns the immutable
`ProviderModelInfo` value consumed by the application catalog. Provider-specific
@@ -767,19 +783,34 @@ construction for those failures.
Concrete adapters may supply one narrow semantic override for an upstream quirk
that the shared SDK cannot express correctly. The concrete adapter owns the
exact upstream marker, while the shared failure policy owns its canonical
-meaning and wording. The limiter uses that meaning for retry qualification and
-its existing provider-wide reactive backoff while retaining the raw exception,
-so exhausted retries still receive the original HTTP status/body through the
-shared redaction and diagnostic path. For NVCF's function-scoped failure this
-deliberately keeps the simple one-limiter-per-provider policy; a degraded NIM
-function can therefore briefly delay other NIM models during backoff. No
-provider-specific marker enters `core/`, another provider, or an API adapter.
+meaning and wording. Admission uses that meaning for retry qualification while
+retaining the raw exception, so exhausted retries still receive the original
+HTTP status/body through the shared redaction and diagnostic path. For NVCF's
+function-scoped failure this deliberately keeps the simple
+one-controller-per-provider policy; a degraded NIM function can therefore
+briefly pause other NIM models during shared recovery. No provider-specific
+marker enters `core/`, another provider, or an API adapter.
+
[providers/stream_recovery.py](src/free_claude_code/providers/stream_recovery.py)
-owns the 0.75-second/65,536-byte holdback, four transparent early retries after
-the first attempt, and five midstream recovery attempts. Provider opening keeps
-its existing five-attempt exponential-backoff budget. `ExecutionFailure.retryable`
-records provider-policy eligibility; it never tells the client to retry after FCC
-has finalized the failure.
+owns only the 0.75-second/65,536-byte commit holdback and the choice between
+transparent replay, request-local continuation/tool salvage, and final failure.
+`ProviderRetrySession` owns one five-attempt budget for the whole logical
+execution: initial opening, deterministic request-shape corrections, early
+replay, continuation, and tool repair all consume that same budget. There are no
+nested retry counters. Deterministic corrections retry immediately; transient
+failures use exponential backoff with jitter and honor `Retry-After` as a
+minimum. When partial output exists, the last available attempt is reserved for
+continuation or repair instead of replaying the full request again. Completed
+tool calls can be salvaged without an upstream attempt.
+
+For streams, upstream acceptance is the first received chunk. Retryable failure
+before that point participates in provider-wide coordinated recovery. Failure
+after that point remains request-local so one interrupted connection does not
+freeze healthy parallel streams, but any continuation still consumes the same
+execution budget. The OpenAI SDK's internal retries remain disabled so FCC is
+the only retry owner. `ExecutionFailure.retryable` records provider-policy
+eligibility; it never tells the client to retry after FCC has finalized the
+failure.
The OpenAI-chat provider remains an upstream adapter: it converts OpenAI chat
chunks into ledger operations. After retry, continuation, and tool salvage are
diff --git a/pyproject.toml b/pyproject.toml
index c64489f8..154d9e5e 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "free-claude-code"
-version = "4.11.0"
+version = "4.11.1"
description = "Local proxy connecting coding agents to OpenAI-compatible AI providers"
readme = "README.md"
requires-python = ">=3.14.0"
diff --git a/smoke/capabilities.py b/smoke/capabilities.py
index 10c31e0c..7898f2d6 100644
--- a/smoke/capabilities.py
+++ b/smoke/capabilities.py
@@ -150,12 +150,12 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = (
"provider_proxy_timeout_config",
"free_claude_code.providers.runtime.ProviderRuntime",
"provider proxy, timeout, and rate-limit settings",
- "provider client and instance-owned limiter config",
+ "provider client and generation-owned admission config",
"provider construction failure",
(
"tests/api/test_dependencies.py",
"tests/providers/test_provider_runtime.py",
- "tests/providers/test_provider_rate_limit.py",
+ "tests/providers/test_provider_admission.py",
),
),
CapabilityContract(
@@ -279,12 +279,12 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = (
"provider_runtime",
"rate_limit_and_disconnect",
"smart_rate_limiting",
- "free_claude_code.providers.rate_limit.ProviderRateLimiter",
+ "free_claude_code.providers.admission.ProviderAdmissionController",
"concurrent provider requests and transient upstream/disconnect failures",
- "provider-specific classification, proactive throttle, retry, cleanup",
+ "provider-specific classification, proactive throttle, coordinated retry, cleanup",
"mapped provider error or smoke skip for upstream disconnect",
(
- "tests/providers/test_provider_rate_limit.py",
+ "tests/providers/test_provider_admission.py",
"tests/providers/test_nvidia_nim_degraded_retry.py",
),
("test_client_disconnect_mid_stream_does_not_crash_server",),
diff --git a/smoke/features.py b/smoke/features.py
index ce5805e1..37fa898a 100644
--- a/smoke/features.py
+++ b/smoke/features.py
@@ -198,7 +198,7 @@ FEATURE_INVENTORY: tuple[FeatureCoverage, ...] = (
"Transient retries and disconnect cleanup preserve follow-up requests",
"public_surface",
(
- "tests/providers/test_provider_rate_limit.py",
+ "tests/providers/test_provider_admission.py",
"tests/providers/test_nvidia_nim_degraded_retry.py",
),
("test_client_disconnect_mid_stream_does_not_crash_server",),
diff --git a/src/free_claude_code/core/anthropic/streaming/recovery.py b/src/free_claude_code/core/anthropic/streaming/recovery.py
index e488edc6..162fec06 100644
--- a/src/free_claude_code/core/anthropic/streaming/recovery.py
+++ b/src/free_claude_code/core/anthropic/streaming/recovery.py
@@ -129,7 +129,6 @@ def make_text_recovery_body(
recovery = deepcopy(body)
recovery.pop("tools", None)
recovery.pop("tool_choice", None)
- recovery["stream"] = True
messages = _copied_messages(recovery)
if partial_text:
messages.append({"role": "assistant", "content": partial_text})
@@ -152,7 +151,6 @@ def make_tool_repair_body(
recovery = deepcopy(body)
recovery.pop("tools", None)
recovery.pop("tool_choice", None)
- recovery["stream"] = True
messages = _copied_messages(recovery)
messages.append(
{
diff --git a/src/free_claude_code/providers/admission.py b/src/free_claude_code/providers/admission.py
new file mode 100644
index 00000000..d4a73375
--- /dev/null
+++ b/src/free_claude_code/providers/admission.py
@@ -0,0 +1,717 @@
+"""Provider-owned admission, concurrency, and coordinated retry lifecycle."""
+
+import asyncio
+import math
+import random
+import time
+from collections.abc import Awaitable, Callable
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from email.utils import parsedate_to_datetime
+from typing import TypeVar
+
+from loguru import logger
+
+from free_claude_code.core.rate_limit import StrictSlidingWindowLimiter
+from free_claude_code.core.trace import trace_event
+from free_claude_code.providers.failure_policy import (
+ ProviderFailureOverride,
+ ProviderRecoveryExhausted,
+ is_retryable_provider_error,
+ retryable_upstream_status,
+)
+
+T = TypeVar("T")
+
+UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS = 5
+DEFAULT_UPSTREAM_BASE_DELAY = 2.0
+DEFAULT_UPSTREAM_MAX_DELAY = 60.0
+DEFAULT_UPSTREAM_JITTER = 1.0
+
+
+class ProviderRetrySession:
+ """One non-multiplying upstream-attempt budget for a logical execution."""
+
+ def __init__(self, *, max_attempts: int, request_id: str | None = None) -> None:
+ if max_attempts <= 0:
+ raise ValueError("max_attempts must be > 0")
+ self._max_attempts = max_attempts
+ self._request_id = request_id
+ self._attempts_started = 0
+ self._terminal_error: Exception | None = None
+
+ @property
+ def max_attempts(self) -> int:
+ return self._max_attempts
+
+ @property
+ def request_id(self) -> str | None:
+ return self._request_id
+
+ @property
+ def attempts_started(self) -> int:
+ return self._attempts_started
+
+ @property
+ def can_attempt(self) -> bool:
+ return self._attempts_started < self._max_attempts
+
+ @property
+ def attempts_remaining(self) -> int:
+ return self._max_attempts - self._attempts_started
+
+ def _claim_attempt(self) -> int:
+ if not self.can_attempt:
+ raise RuntimeError("provider retry session is exhausted")
+ self._attempts_started += 1
+ return self._attempts_started
+
+ def _fail_recovery(self, error: Exception) -> None:
+ self._terminal_error = error
+
+ def _terminal_failure(self) -> Exception | None:
+ return self._terminal_error
+
+
+@dataclass(frozen=True, slots=True)
+class _GatePermit:
+ generation: int | None
+ probe: bool
+
+
+@dataclass(slots=True)
+class _RecoveryEpisode:
+ generation: int
+ leader: ProviderRetrySession | None
+ ready_at: float
+ last_error: Exception
+ waiters: set[ProviderRetrySession] = field(default_factory=set)
+ probe_active: bool = False
+ terminal_until: float | None = None
+
+
+class ProviderAttempt:
+ """One admitted upstream attempt and its held concurrency slot."""
+
+ def __init__(
+ self,
+ controller: ProviderAdmissionController,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> None:
+ self._controller = controller
+ self._session = session
+ self._permit = permit
+ self._resolved = False
+ self._accepted = False
+ self._failure_retryable: bool | None = None
+ self._closed = False
+
+ @property
+ def accepted(self) -> bool:
+ """Return whether upstream acceptance has resolved this attempt."""
+ return self._accepted
+
+ @property
+ def failure_retryable(self) -> bool | None:
+ """Return the provider classification recorded for a failed attempt."""
+ return self._failure_retryable
+
+ async def succeeded(self) -> None:
+ """Record a valid upstream response and close a recovery episode if probing."""
+ if self._resolved:
+ return
+ await self._controller._attempt_succeeded(self._session, self._permit)
+ self._resolved = True
+ self._accepted = True
+
+ async def retry_immediately(self) -> None:
+ """Keep probe ownership for a bounded request-shape correction."""
+ if self._resolved:
+ return
+ await self._controller._attempt_corrected(self._session, self._permit)
+ self._resolved = True
+
+ async def retry(
+ self,
+ error: Exception,
+ *,
+ provider_failure_override: ProviderFailureOverride | None = None,
+ ) -> bool:
+ """Record a retryable failure and return whether this execution may retry."""
+ if self._resolved:
+ return False
+ effective_error = (
+ provider_failure_override(error)
+ if provider_failure_override is not None
+ else None
+ )
+ if effective_error is None:
+ effective_error = error
+ status = retryable_upstream_status(effective_error)
+ retryable = is_retryable_provider_error(effective_error)
+ self._failure_retryable = retryable
+ if not retryable:
+ await self._controller._attempt_rejected(self._session, self._permit)
+ self._resolved = True
+ return False
+ should_retry = await self._controller._attempt_failed(
+ self._session,
+ self._permit,
+ error=error,
+ status=status,
+ )
+ self._resolved = True
+ return should_retry
+
+ async def aclose(self) -> None:
+ """Release attempt ownership and its concurrency slot exactly once."""
+ if self._closed:
+ return
+ self._closed = True
+ try:
+ if not self._resolved:
+ await asyncio.shield(
+ self._controller._attempt_abandoned(self._session, self._permit)
+ )
+ finally:
+ self._controller._release_concurrency()
+
+
+class ProviderAdmissionController:
+ """Coordinate one provider's rate, concurrency, and recovery state.
+
+ Normal attempts pass through a strict sliding window and concurrency bulkhead.
+ The first shared transient failure opens one recovery episode. Exactly one
+ logical execution owns its half-open probes; concurrent callers wait for that
+ episode instead of starting independent retry loops.
+ """
+
+ def __init__(
+ self,
+ *,
+ provider_name: str,
+ rate_limit: int = 40,
+ rate_window: float = 60.0,
+ max_concurrency: int = 5,
+ max_attempts: int = UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS,
+ base_delay: float = DEFAULT_UPSTREAM_BASE_DELAY,
+ max_delay: float = DEFAULT_UPSTREAM_MAX_DELAY,
+ jitter: float = DEFAULT_UPSTREAM_JITTER,
+ ) -> None:
+ if rate_limit <= 0:
+ raise ValueError("rate_limit must be > 0")
+ if rate_window <= 0:
+ raise ValueError("rate_window must be > 0")
+ if max_concurrency <= 0:
+ raise ValueError("max_concurrency must be > 0")
+ if max_attempts <= 0:
+ raise ValueError("max_attempts must be > 0")
+ if base_delay < 0:
+ raise ValueError("base_delay must be >= 0")
+ if max_delay < base_delay:
+ raise ValueError("max_delay must be >= base_delay")
+ if jitter < 0:
+ raise ValueError("jitter must be >= 0")
+
+ self._provider_name = provider_name
+ self._max_attempts = max_attempts
+ self._base_delay = base_delay
+ self._max_delay = max_delay
+ self._jitter = jitter
+ self._proactive_limiter = StrictSlidingWindowLimiter(
+ rate_limit, float(rate_window)
+ )
+ self._concurrency_sem = asyncio.Semaphore(max_concurrency)
+ self._condition = asyncio.Condition()
+ self._episode: _RecoveryEpisode | None = None
+ self._next_generation = 1
+ logger.info(
+ "Provider admission initialized for {} ({} req / {}s, "
+ "max_concurrency={}, max_attempts={})",
+ provider_name,
+ rate_limit,
+ rate_window,
+ max_concurrency,
+ max_attempts,
+ )
+
+ def new_retry_session(
+ self,
+ *,
+ request_id: str | None = None,
+ ) -> ProviderRetrySession:
+ """Return a fresh logical-execution retry budget."""
+ return ProviderRetrySession(
+ max_attempts=self._max_attempts,
+ request_id=request_id,
+ )
+
+ async def open_attempt(self, session: ProviderRetrySession) -> ProviderAttempt:
+ """Wait for provider admission and hold one active-operation slot."""
+ if not session.can_attempt:
+ raise RuntimeError("provider retry session is exhausted")
+
+ while True:
+ permit = await self._wait_for_gate(session)
+ slot_acquired = False
+ try:
+ admitted = await self._proactive_limiter.acquire_if(
+ lambda permit=permit: self._permit_is_current(session, permit)
+ )
+ if not admitted:
+ await self._abandon_probe_permit(session, permit)
+ continue
+ await self._concurrency_sem.acquire()
+ slot_acquired = True
+ if not self._permit_is_current(session, permit):
+ self._concurrency_sem.release()
+ slot_acquired = False
+ await self._abandon_probe_permit(session, permit)
+ continue
+ session._claim_attempt()
+ return ProviderAttempt(self, session, permit)
+ except BaseException:
+ if slot_acquired:
+ self._concurrency_sem.release()
+ await self._abandon_probe_permit(session, permit)
+ raise
+
+ async def run_with_retry(
+ self,
+ fn: Callable[[], Awaitable[T]],
+ *,
+ provider_failure_override: ProviderFailureOverride | None = None,
+ request_id: str | None = None,
+ ) -> T:
+ """Run one non-streaming provider operation through coordinated retries."""
+ session = self.new_retry_session(request_id=request_id)
+ while True:
+ attempt = await self.open_attempt(session)
+ try:
+ result = await fn()
+ except asyncio.CancelledError:
+ raise
+ except Exception as error:
+ should_retry = await attempt.retry(
+ error,
+ provider_failure_override=provider_failure_override,
+ )
+ if not should_retry:
+ raise
+ else:
+ await attempt.succeeded()
+ return result
+ finally:
+ await attempt.aclose()
+
+ async def _wait_for_gate(self, session: ProviderRetrySession) -> _GatePermit:
+ while True:
+ if (terminal_error := session._terminal_failure()) is not None:
+ raise ProviderRecoveryExhausted(terminal_error)
+ sleep_delay: float | None = None
+ claimed_generation: int | None = None
+ async with self._condition:
+ episode = self._episode
+ if episode is None:
+ return _GatePermit(generation=None, probe=False)
+
+ now = time.monotonic()
+ if episode.terminal_until is not None:
+ if now < episode.terminal_until:
+ raise ProviderRecoveryExhausted(episode.last_error)
+ episode = self._start_recovery_episode(
+ leader=session,
+ ready_at=now,
+ last_error=episode.last_error,
+ )
+
+ if episode.leader is None:
+ episode.leader = session
+ episode.waiters.discard(session)
+ if episode.leader is session:
+ if episode.probe_active:
+ await self._condition.wait()
+ continue
+ sleep_delay = max(0.0, episode.ready_at - now)
+ claimed_generation = episode.generation
+ if sleep_delay == 0:
+ episode.probe_active = True
+ return self._probe_permit(session, episode)
+ else:
+ episode.waiters.add(session)
+ try:
+ await self._condition.wait()
+ except asyncio.CancelledError:
+ current = self._episode
+ if (
+ current is not None
+ and current.generation == episode.generation
+ ):
+ current.waiters.discard(session)
+ raise
+ continue
+
+ if sleep_delay is None or claimed_generation is None:
+ continue
+ try:
+ if sleep_delay > 0:
+ logger.warning(
+ "Provider {} recovery active, waiting {:.1f}s for one probe",
+ self._provider_name,
+ sleep_delay,
+ )
+ await asyncio.sleep(sleep_delay)
+ async with self._condition:
+ episode = self._episode
+ if (
+ episode is not None
+ and episode.generation == claimed_generation
+ and episode.terminal_until is None
+ and episode.leader is session
+ and not episode.probe_active
+ ):
+ episode.probe_active = True
+ return self._probe_permit(session, episode)
+ except asyncio.CancelledError:
+ await self._abandon_waiting_leader(session, claimed_generation)
+ raise
+
+ def _probe_permit(
+ self,
+ session: ProviderRetrySession,
+ episode: _RecoveryEpisode,
+ ) -> _GatePermit:
+ trace_event(
+ stage="provider",
+ event="provider.recovery.probe",
+ source="provider",
+ provider=self._provider_name,
+ request_id=session.request_id,
+ generation=episode.generation,
+ attempt=session.attempts_started + 1,
+ max_attempts=session.max_attempts,
+ )
+ return _GatePermit(generation=episode.generation, probe=True)
+
+ def _permit_is_current(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> bool:
+ episode = self._episode
+ if not permit.probe:
+ return episode is None
+ return (
+ episode is not None
+ and episode.generation == permit.generation
+ and episode.terminal_until is None
+ and episode.leader is session
+ and episode.probe_active
+ )
+
+ async def _attempt_succeeded(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> None:
+ if not permit.probe:
+ return
+ async with self._condition:
+ episode = self._matching_probe(session, permit)
+ if episode is None:
+ return
+ self._episode = None
+ self._condition.notify_all()
+ trace_event(
+ stage="provider",
+ event="provider.recovery.closed",
+ source="provider",
+ provider=self._provider_name,
+ request_id=session.request_id,
+ generation=permit.generation,
+ attempt=session.attempts_started,
+ outcome="success",
+ )
+
+ async def _attempt_corrected(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> None:
+ if not permit.probe:
+ return
+ async with self._condition:
+ episode = self._matching_probe(session, permit)
+ if episode is None:
+ return
+ episode.probe_active = False
+ episode.ready_at = time.monotonic()
+ self._condition.notify_all()
+
+ async def _attempt_rejected(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> None:
+ """Close a probe episode when upstream responds with a final rejection."""
+ if not permit.probe:
+ return
+ async with self._condition:
+ episode = self._matching_probe(session, permit)
+ if episode is None:
+ return
+ self._episode = None
+ self._condition.notify_all()
+ trace_event(
+ stage="provider",
+ event="provider.recovery.closed",
+ source="provider",
+ provider=self._provider_name,
+ request_id=session.request_id,
+ generation=permit.generation,
+ attempt=session.attempts_started,
+ outcome="rejected",
+ )
+
+ async def _attempt_abandoned(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> None:
+ if not permit.probe:
+ return
+ async with self._condition:
+ episode = self._matching_probe(session, permit)
+ if episode is None:
+ return
+ episode.leader = None
+ episode.probe_active = False
+ episode.ready_at = time.monotonic()
+ self._condition.notify_all()
+
+ async def _attempt_failed(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ *,
+ error: Exception,
+ status: int | None,
+ ) -> bool:
+ can_retry = session.can_attempt
+ delay = self._retry_delay(error, session.attempts_started)
+ became_leader = False
+ exhausted_episode = False
+
+ async with self._condition:
+ episode = self._episode
+ matching_probe = self._matching_probe(session, permit)
+ if can_retry:
+ if episode is None:
+ episode = self._start_recovery_episode(
+ leader=session,
+ ready_at=time.monotonic() + delay,
+ last_error=error,
+ )
+ became_leader = True
+ elif matching_probe is not None:
+ episode.last_error = error
+ episode.probe_active = False
+ episode.ready_at = time.monotonic() + delay
+ became_leader = True
+ elif episode.terminal_until is not None:
+ session._fail_recovery(episode.last_error)
+ else:
+ episode.waiters.add(session)
+ self._condition.notify_all()
+ elif episode is None or matching_probe is not None:
+ terminal_delay = self._retry_delay(
+ error,
+ session.attempts_started,
+ )
+ if episode is None:
+ episode = self._start_recovery_episode(
+ leader=None,
+ ready_at=time.monotonic(),
+ last_error=error,
+ request_id=session.request_id,
+ )
+ episode.last_error = error
+ episode.leader = None
+ episode.probe_active = False
+ episode.terminal_until = time.monotonic() + terminal_delay
+ for waiter in episode.waiters:
+ waiter._fail_recovery(error)
+ episode.waiters.clear()
+ exhausted_episode = True
+ self._condition.notify_all()
+
+ label = self._failure_label(status, error)
+ if became_leader:
+ logger.warning(
+ "{}, attempt {}/{} failed; one provider recovery probe in {:.1f}s",
+ label,
+ session.attempts_started,
+ session.max_attempts,
+ delay,
+ )
+ trace_event(
+ stage="provider",
+ event="provider.retry.scheduled",
+ source="provider",
+ provider=self._provider_name,
+ request_id=session.request_id,
+ status_code=status,
+ exc_type=type(error).__name__,
+ attempt=session.attempts_started,
+ max_attempts=session.max_attempts,
+ delay_s=round(delay, 3),
+ coordinated=True,
+ )
+ elif can_retry:
+ trace_event(
+ stage="provider",
+ event="provider.retry.coalesced",
+ source="provider",
+ provider=self._provider_name,
+ request_id=session.request_id,
+ status_code=status,
+ exc_type=type(error).__name__,
+ attempt=session.attempts_started,
+ max_attempts=session.max_attempts,
+ )
+ else:
+ logger.warning(
+ "{} retry exhausted (attempts={})",
+ label,
+ session.attempts_started,
+ )
+ trace_event(
+ stage="provider",
+ event="provider.retry.exhausted",
+ source="provider",
+ provider=self._provider_name,
+ request_id=session.request_id,
+ status_code=status,
+ exc_type=type(error).__name__,
+ attempts=session.attempts_started,
+ episode_exhausted=exhausted_episode,
+ )
+ return can_retry
+
+ def _start_recovery_episode(
+ self,
+ *,
+ leader: ProviderRetrySession | None,
+ ready_at: float,
+ last_error: Exception,
+ request_id: str | None = None,
+ ) -> _RecoveryEpisode:
+ episode = _RecoveryEpisode(
+ generation=self._next_generation,
+ leader=leader,
+ ready_at=ready_at,
+ last_error=last_error,
+ )
+ self._next_generation += 1
+ self._episode = episode
+ trace_event(
+ stage="provider",
+ event="provider.recovery.opened",
+ source="provider",
+ provider=self._provider_name,
+ request_id=(leader.request_id if leader is not None else request_id),
+ generation=episode.generation,
+ )
+ return episode
+
+ def _matching_probe(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> _RecoveryEpisode | None:
+ episode = self._episode
+ if (
+ not permit.probe
+ or episode is None
+ or episode.generation != permit.generation
+ or episode.leader is not session
+ or not episode.probe_active
+ ):
+ return None
+ return episode
+
+ async def _abandon_probe_permit(
+ self,
+ session: ProviderRetrySession,
+ permit: _GatePermit,
+ ) -> None:
+ if not permit.probe:
+ return
+ async with self._condition:
+ episode = self._matching_probe(session, permit)
+ if episode is None:
+ return
+ episode.leader = None
+ episode.probe_active = False
+ episode.ready_at = time.monotonic()
+ self._condition.notify_all()
+
+ async def _abandon_waiting_leader(
+ self,
+ session: ProviderRetrySession,
+ generation: int,
+ ) -> None:
+ async with self._condition:
+ episode = self._episode
+ if (
+ episode is None
+ or episode.generation != generation
+ or episode.leader is not session
+ or episode.probe_active
+ ):
+ return
+ episode.leader = None
+ self._condition.notify_all()
+
+ def _release_concurrency(self) -> None:
+ self._concurrency_sem.release()
+
+ def _retry_delay(self, error: Exception, attempt: int) -> float:
+ exponent = max(0, attempt - 1)
+ backoff = min(self._base_delay * (2**exponent), self._max_delay)
+ backoff += random.uniform(0, self._jitter)
+ retry_after = _retry_after_seconds(error)
+ return max(backoff, retry_after or 0.0)
+
+ @staticmethod
+ def _failure_label(status: int | None, error: Exception) -> str:
+ if status == 429:
+ return "Rate limited (429)"
+ if status is not None:
+ return f"Upstream server error ({status})"
+ return f"Provider transient error ({type(error).__name__})"
+
+
+def _retry_after_seconds(error: Exception) -> float | None:
+ response = getattr(error, "response", None)
+ headers = getattr(response, "headers", None)
+ if headers is None:
+ return None
+ value = headers.get("retry-after")
+ if not isinstance(value, str) or not value.strip():
+ return None
+ stripped = value.strip()
+ try:
+ seconds = float(stripped)
+ except ValueError:
+ try:
+ retry_at = parsedate_to_datetime(stripped)
+ except TypeError, ValueError, OverflowError:
+ return None
+ if retry_at.tzinfo is None:
+ retry_at = retry_at.replace(tzinfo=UTC)
+ seconds = (retry_at - datetime.now(UTC)).total_seconds()
+ if not math.isfinite(seconds):
+ return None
+ return max(0.0, seconds)
diff --git a/src/free_claude_code/providers/cloudflare/client.py b/src/free_claude_code/providers/cloudflare/client.py
index 250f0cd2..7215b796 100644
--- a/src/free_claude_code/providers/cloudflare/client.py
+++ b/src/free_claude_code/providers/cloudflare/client.py
@@ -11,6 +11,7 @@ from free_claude_code.application.errors import ApplicationUnavailableError
from free_claude_code.application.model_metadata import ProviderModelInfo
from free_claude_code.config.provider_catalog import CLOUDFLARE_AI_REST_ROOT
from free_claude_code.core.anthropic import ReasoningReplayMode
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.http import maybe_await_aclose
from free_claude_code.providers.model_listing import (
@@ -25,7 +26,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatRequestPolicy,
validate_extra_body_does_not_override_canonical_fields,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
_REQUEST_POLICY = OpenAIChatRequestPolicy(
provider_name="CLOUDFLARE",
@@ -70,7 +70,7 @@ class CloudflareProvider(OpenAIChatProvider):
config: ProviderConfig,
*,
account_id: str,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
):
base_url = cloudflare_ai_base_url(config.base_url, account_id)
self._model_search_url = _cloudflare_model_search_url(
@@ -88,7 +88,7 @@ class CloudflareProvider(OpenAIChatProvider):
super().__init__(
replace(config, base_url=base_url),
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
async def cleanup(self) -> None:
@@ -102,13 +102,22 @@ class CloudflareProvider(OpenAIChatProvider):
async def list_model_infos(self) -> frozenset[ProviderModelInfo]:
"""Return Cloudflare Workers AI model ids from account model search."""
- response = await self._model_list_client.get(
- self._model_search_url,
- params={"format": "openrouter"},
- headers=self._model_list_headers(),
- )
+
+ async def request() -> httpx.Response:
+ response = await self._model_list_client.get(
+ self._model_search_url,
+ params={"format": "openrouter"},
+ headers=self._model_list_headers(),
+ )
+ try:
+ response.raise_for_status()
+ except Exception:
+ await maybe_await_aclose(response)
+ raise
+ return response
+
+ response = await self._admission.run_with_retry(request)
try:
- response.raise_for_status()
try:
payload = response.json()
except ValueError as exc:
diff --git a/src/free_claude_code/providers/deepseek/client.py b/src/free_claude_code/providers/deepseek/client.py
index 07bfb0b7..2dec3ca9 100644
--- a/src/free_claude_code/providers/deepseek/client.py
+++ b/src/free_claude_code/providers/deepseek/client.py
@@ -4,6 +4,7 @@ from typing import Any
from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import (
NO_REASONING,
@@ -11,7 +12,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProvider,
usage_int,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .compat import DEEPSEEK_REQUEST_POLICY, build_deepseek_request_body
@@ -24,11 +24,13 @@ _PROFILE = OpenAIChatProfile(
class DeepSeekProvider(OpenAIChatProvider):
"""DeepSeek using ``https://api.deepseek.com`` Chat Completions."""
- def __init__(self, config: ProviderConfig, *, rate_limiter: ProviderRateLimiter):
+ def __init__(
+ self, config: ProviderConfig, *, admission: ProviderAdmissionController
+ ):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
def _build_request_body(
diff --git a/src/free_claude_code/providers/failure_policy.py b/src/free_claude_code/providers/failure_policy.py
index f65fb5fd..412918d5 100644
--- a/src/free_claude_code/providers/failure_policy.py
+++ b/src/free_claude_code/providers/failure_policy.py
@@ -16,7 +16,6 @@ from free_claude_code.core.diagnostics import (
)
from free_claude_code.core.failures import ExecutionFailure, FailureKind
-MarkRateLimited = Callable[[float], None]
ProviderFailureOverride = Callable[[Exception], ExecutionFailure | None]
_RATE_LIMIT_MARKERS = frozenset({"rate_limit", "rate limit", "too many requests"})
@@ -36,16 +35,28 @@ _INVALID_REQUEST_MESSAGE = "Invalid request sent to provider."
_OVERLOADED_MESSAGE = "Provider is currently overloaded. Please retry."
+class ProviderRecoveryExhausted(RuntimeError):
+ """A shared provider recovery episode exhausted its single probe budget."""
+
+ def __init__(self, last_error: Exception) -> None:
+ super().__init__("Provider recovery retry budget was exhausted.")
+ self.last_error = last_error
+
+
+class RetryableProviderProtocolError(RuntimeError):
+ """A malformed upstream protocol result eligible for provider retry."""
+
+
def classify_provider_failure(
exc: Exception,
*,
provider_name: str,
read_timeout_s: float | None,
request_id: str | None,
- mark_rate_limited: MarkRateLimited,
provider_failure_override: ProviderFailureOverride | None = None,
) -> ExecutionFailure:
"""Return one detailed canonical failure after provider retries are exhausted."""
+ exc = underlying_provider_error(exc)
if isinstance(exc, ExecutionFailure):
failure = exc
message = failure.message
@@ -63,7 +74,6 @@ def classify_provider_failure(
failure = _classify_provider_failure(
exc,
read_timeout_s=read_timeout_s,
- mark_rate_limited=mark_rate_limited,
)
message = format_execution_failure_message(
failure,
@@ -81,6 +91,8 @@ def overloaded_provider_failure() -> ExecutionFailure:
def retryable_transient_status(exc: BaseException) -> int | None:
"""Infer a retryable HTTP-like status from one upstream exception."""
+ if isinstance(exc, ProviderRecoveryExhausted):
+ return None
if isinstance(exc, ExecutionFailure):
status = exc.status_code
return status if exc.retryable and _is_retryable_status(status) else None
@@ -110,6 +122,8 @@ def retryable_transient_status(exc: BaseException) -> int | None:
def is_transient_overload_error(exc: BaseException) -> bool:
"""Return whether an upstream exception reports overload or capacity pressure."""
+ if isinstance(exc, ProviderRecoveryExhausted):
+ return False
if isinstance(exc, ExecutionFailure):
return exc.kind == FailureKind.OVERLOADED
return _has_marker(transient_error_text(exc), _OVERLOAD_MARKERS)
@@ -130,6 +144,8 @@ def transient_error_text(exc: BaseException) -> str:
def is_retryable_provider_error(exc: BaseException) -> bool:
"""Return whether provider policy permits stream retry or recovery."""
+ if isinstance(exc, ProviderRecoveryExhausted):
+ return False
if isinstance(exc, ExecutionFailure):
return exc.retryable
if isinstance(exc, openai.AuthenticationError | openai.BadRequestError):
@@ -148,6 +164,7 @@ def is_retryable_provider_error(exc: BaseException) -> bool:
httpx.NetworkError,
openai.APITimeoutError,
openai.APIConnectionError,
+ RetryableProviderProtocolError,
),
)
@@ -158,34 +175,14 @@ def retryable_upstream_status(exc: BaseException) -> int | None:
return status if status is not None and _is_retryable_status(status) else None
-def retryable_upstream_transport_error(exc: BaseException) -> bool:
- """Return whether a pre-response transport failure can be retried."""
- if isinstance(exc, ExecutionFailure):
- return exc.retryable and retryable_transient_status(exc) is None
- if isinstance(exc, openai.AuthenticationError | openai.BadRequestError):
- return False
- return isinstance(
- exc,
- (
- TimeoutError,
- httpx.TimeoutException,
- httpx.ConnectError,
- httpx.ReadError,
- httpx.WriteError,
- httpx.RemoteProtocolError,
- httpx.NetworkError,
- openai.APITimeoutError,
- openai.APIConnectionError,
- ),
- )
-
-
def provider_error_message(
exc: BaseException,
*,
read_timeout_s: float | None = None,
) -> str:
"""Map raw provider exception types to stable customer-facing wording."""
+ if isinstance(exc, Exception):
+ exc = underlying_provider_error(exc)
if isinstance(exc, ExecutionFailure):
return exc.message
if isinstance(exc, httpx.ReadTimeout):
@@ -213,17 +210,13 @@ def _classify_provider_failure(
exc: Exception,
*,
read_timeout_s: float | None,
- mark_rate_limited: MarkRateLimited,
) -> ExecutionFailure:
if isinstance(exc, ExecutionFailure):
- if exc.kind == FailureKind.RATE_LIMIT:
- mark_rate_limited(60)
return exc
if isinstance(exc, openai.AuthenticationError):
return _failure(FailureKind.AUTHENTICATION, 401, _AUTHENTICATION_MESSAGE, False)
if isinstance(exc, openai.RateLimitError):
- mark_rate_limited(60)
return _failure(FailureKind.RATE_LIMIT, 429, _RATE_LIMIT_MESSAGE, True)
if isinstance(exc, openai.BadRequestError):
return _failure(
@@ -248,7 +241,6 @@ def _classify_provider_failure(
if isinstance(exc, openai.APIError):
status = retryable_transient_status(exc)
if status == 429:
- mark_rate_limited(60)
return _failure(FailureKind.RATE_LIMIT, 429, _RATE_LIMIT_MESSAGE, True)
if is_transient_overload_error(exc):
return overloaded_provider_failure()
@@ -269,7 +261,6 @@ def _classify_provider_failure(
FailureKind.AUTHENTICATION, 401, _AUTHENTICATION_MESSAGE, False
)
if status == 429:
- mark_rate_limited(60)
return _failure(FailureKind.RATE_LIMIT, 429, _RATE_LIMIT_MESSAGE, True)
if status == 400:
return _failure(
@@ -389,5 +380,12 @@ def _has_marker(text: str, markers: frozenset[str]) -> bool:
return any(marker in text for marker in markers)
+def underlying_provider_error(exc: Exception) -> Exception:
+ """Return the raw failure retained by an exhausted recovery wrapper."""
+ while isinstance(exc, ProviderRecoveryExhausted):
+ exc = exc.last_error
+ return exc
+
+
def _is_retryable_status(status: int | None) -> bool:
return isinstance(status, int) and (status == 429 or 500 <= status <= 599)
diff --git a/src/free_claude_code/providers/gemini/client.py b/src/free_claude_code/providers/gemini/client.py
index c20670b0..46a55889 100644
--- a/src/free_claude_code/providers/gemini/client.py
+++ b/src/free_claude_code/providers/gemini/client.py
@@ -2,6 +2,7 @@
from free_claude_code.core.anthropic import ReasoningReplayMode
from free_claude_code.core.reasoning import ReasoningEffort
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.google_openai import GoogleOpenAIProvider
from free_claude_code.providers.openai_chat import (
@@ -9,7 +10,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProfile,
OpenAIChatRequestPolicy,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
_REQUEST_POLICY = OpenAIChatRequestPolicy(
provider_name="GEMINI",
@@ -34,9 +34,11 @@ _PROFILE = OpenAIChatProfile(
class GeminiProvider(GoogleOpenAIProvider):
"""Gemini API using ``https://generativelanguage.googleapis.com/v1beta/openai/``."""
- def __init__(self, config: ProviderConfig, *, rate_limiter: ProviderRateLimiter):
+ def __init__(
+ self, config: ProviderConfig, *, admission: ProviderAdmissionController
+ ):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
diff --git a/src/free_claude_code/providers/github_models/client.py b/src/free_claude_code/providers/github_models/client.py
index 4cd88c3c..5e8c0011 100644
--- a/src/free_claude_code/providers/github_models/client.py
+++ b/src/free_claude_code/providers/github_models/client.py
@@ -7,6 +7,7 @@ import httpx
from free_claude_code.application.model_metadata import ProviderModelInfo
from free_claude_code.core.anthropic import ReasoningReplayMode
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.http import maybe_await_aclose
from free_claude_code.providers.model_listing import (
@@ -19,7 +20,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProvider,
OpenAIChatRequestPolicy,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
GITHUB_MODELS_CATALOG_URL = "https://models.github.ai/catalog/models"
GITHUB_MODELS_API_VERSION = "2026-03-10"
@@ -35,7 +35,9 @@ _REQUIRED_MODEL_CAPABILITIES = frozenset({"streaming", "tool-calling"})
class GitHubModelsProvider(OpenAIChatProvider):
"""GitHub Models OpenAI-compatible inference provider."""
- def __init__(self, config: ProviderConfig, *, rate_limiter: ProviderRateLimiter):
+ def __init__(
+ self, config: ProviderConfig, *, admission: ProviderAdmissionController
+ ):
self._catalog_url = GITHUB_MODELS_CATALOG_URL
self._model_list_client = httpx.AsyncClient(
proxy=config.proxy or None,
@@ -49,7 +51,7 @@ class GitHubModelsProvider(OpenAIChatProvider):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
default_headers=_github_models_default_headers(),
)
@@ -64,12 +66,21 @@ class GitHubModelsProvider(OpenAIChatProvider):
async def list_model_infos(self) -> frozenset[ProviderModelInfo]:
"""Return stream/tool-capable GitHub Models catalog ids."""
- response = await self._model_list_client.get(
- self._catalog_url,
- headers=self._model_list_headers(),
- )
+
+ async def request() -> httpx.Response:
+ response = await self._model_list_client.get(
+ self._catalog_url,
+ headers=self._model_list_headers(),
+ )
+ try:
+ response.raise_for_status()
+ except Exception:
+ await maybe_await_aclose(response)
+ raise
+ return response
+
+ response = await self._admission.run_with_retry(request)
try:
- response.raise_for_status()
try:
payload = response.json()
except ValueError as exc:
diff --git a/src/free_claude_code/providers/google_openai/provider.py b/src/free_claude_code/providers/google_openai/provider.py
index 4f7cbe7c..554a7ae6 100644
--- a/src/free_claude_code/providers/google_openai/provider.py
+++ b/src/free_claude_code/providers/google_openai/provider.py
@@ -11,6 +11,7 @@ from free_claude_code.core.reasoning import (
ReasoningControl,
ReasoningPolicy,
)
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import (
OpenAIAsyncCredentialProvider,
@@ -18,7 +19,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProvider,
build_openai_chat_request_body,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .quirks import apply_google_request_quirks, google_thinking_config
@@ -51,14 +51,14 @@ class GoogleOpenAIProvider(OpenAIChatProvider):
config: ProviderConfig,
*,
profile: OpenAIChatProfile,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
api_key_provider: OpenAIAsyncCredentialProvider | None = None,
default_headers: Mapping[str, str] | None = None,
) -> None:
super().__init__(
config,
profile=profile,
- rate_limiter=rate_limiter,
+ admission=admission,
api_key_provider=api_key_provider,
default_headers=default_headers,
)
diff --git a/src/free_claude_code/providers/lmstudio/client.py b/src/free_claude_code/providers/lmstudio/client.py
index fcd4d9a6..243957d8 100644
--- a/src/free_claude_code/providers/lmstudio/client.py
+++ b/src/free_claude_code/providers/lmstudio/client.py
@@ -24,6 +24,7 @@ from free_claude_code.core.reasoning import (
ReasoningEffort,
ReasoningPolicy,
)
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import (
NamedEffortReasoning,
@@ -31,7 +32,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProvider,
OpenAIChatRequestPolicy,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
_PROFILE = OpenAIChatProfile(
OpenAIChatRequestPolicy(
@@ -64,11 +64,13 @@ class LMStudioProvider(OpenAIChatProvider):
# dying mid-stream.
_CONTEXT_CACHE_TTL_S = 30.0
- def __init__(self, config: ProviderConfig, *, rate_limiter: ProviderRateLimiter):
+ def __init__(
+ self, config: ProviderConfig, *, admission: ProviderAdmissionController
+ ):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
self._loaded_context_cache: tuple[float, int | None] = (0.0, None)
diff --git a/src/free_claude_code/providers/mistral/client.py b/src/free_claude_code/providers/mistral/client.py
index fb872303..503ce1a6 100644
--- a/src/free_claude_code/providers/mistral/client.py
+++ b/src/free_claude_code/providers/mistral/client.py
@@ -7,6 +7,7 @@ from loguru import logger
from free_claude_code.core.anthropic import ReasoningReplayMode
from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import (
NO_REASONING,
@@ -15,7 +16,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatRequestPolicy,
build_openai_chat_request_body,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .reasoning import (
apply_mistral_reasoning_request_shape,
@@ -34,11 +34,13 @@ _PROFILE = OpenAIChatProfile(_REQUEST_POLICY, NO_REASONING)
class MistralProvider(OpenAIChatProvider):
"""Mistral API using ``https://api.mistral.ai/v1/chat/completions``."""
- def __init__(self, config: ProviderConfig, *, rate_limiter: ProviderRateLimiter):
+ def __init__(
+ self, config: ProviderConfig, *, admission: ProviderAdmissionController
+ ):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
def _build_request_body(
@@ -67,6 +69,5 @@ class MistralProvider(OpenAIChatProvider):
)
return retry_body
- async def _create_stream(self, body: dict) -> tuple[Any, dict]:
- stream, final_body = await super()._create_stream(body)
- return normalize_mistral_stream(stream), final_body
+ def _normalize_stream(self, stream: Any) -> Any:
+ return normalize_mistral_stream(stream)
diff --git a/src/free_claude_code/providers/nvidia_nim/client.py b/src/free_claude_code/providers/nvidia_nim/client.py
index 10ca669f..a75124bb 100644
--- a/src/free_claude_code/providers/nvidia_nim/client.py
+++ b/src/free_claude_code/providers/nvidia_nim/client.py
@@ -11,6 +11,7 @@ from free_claude_code.config.nim import NimSettings
from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.core.failures import ExecutionFailure
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.failure_policy import (
overloaded_provider_failure,
@@ -20,7 +21,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProfile,
OpenAIChatProvider,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .request_options import NIM_REQUEST_POLICY, build_nim_request_body
from .retry import (
@@ -48,12 +48,12 @@ class NvidiaNimProvider(OpenAIChatProvider):
config: ProviderConfig,
*,
nim_settings: NimSettings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
self._nim_settings = nim_settings
diff --git a/src/free_claude_code/providers/open_router/client.py b/src/free_claude_code/providers/open_router/client.py
index 311dd557..03ef869f 100644
--- a/src/free_claude_code/providers/open_router/client.py
+++ b/src/free_claude_code/providers/open_router/client.py
@@ -10,6 +10,7 @@ from free_claude_code.core.anthropic import ReasoningReplayMode
from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.core.anthropic.streaming import AnthropicStreamLedger
from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.model_listing import (
extract_openrouter_tool_model_ids,
@@ -22,7 +23,6 @@ from free_claude_code.providers.openai_chat import (
ReasoningObject,
validate_extra_body_does_not_override_canonical_fields,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
_REQUEST_POLICY = OpenAIChatRequestPolicy(
provider_name="OPENROUTER",
@@ -36,27 +36,35 @@ _REQUEST_POLICY = OpenAIChatRequestPolicy(
class OpenRouterProvider(OpenAIChatProvider):
"""OpenRouter provider using the OpenAI-compatible Chat Completions API."""
- def __init__(self, config: ProviderConfig, *, rate_limiter: ProviderRateLimiter):
+ def __init__(
+ self, config: ProviderConfig, *, admission: ProviderAdmissionController
+ ):
super().__init__(
config,
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
)
async def list_model_ids(self) -> frozenset[str]:
"""Only advertise OpenRouter models that can run Claude Code tools."""
- payload = await self._client.models.list()
+ payload = await self._list_models_payload()
return extract_openrouter_tool_model_ids(
payload, provider_name=self._provider_name
)
async def list_model_infos(self) -> frozenset[ProviderModelInfo]:
"""Advertise OpenRouter tool models with reasoning capability metadata."""
- payload = await self._client.models.list()
+ payload = await self._list_models_payload()
return extract_openrouter_tool_model_infos(
payload, provider_name=self._provider_name
)
+ async def _list_models_payload(self) -> Any:
+ return await self._admission.run_with_retry(
+ self._client.models.list,
+ provider_failure_override=self._provider_failure_override,
+ )
+
def _handle_extra_reasoning(
self, delta: Any, ledger: AnthropicStreamLedger, *, output_reasoning: bool
) -> Iterator[str]:
diff --git a/src/free_claude_code/providers/openai_chat/__init__.py b/src/free_claude_code/providers/openai_chat/__init__.py
index 160f1db4..48be7343 100644
--- a/src/free_claude_code/providers/openai_chat/__init__.py
+++ b/src/free_claude_code/providers/openai_chat/__init__.py
@@ -1,7 +1,7 @@
"""OpenAI-compatible provider family."""
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .base_url import openai_v1_base_url
from .extra_body import validate_extra_body_does_not_override_canonical_fields
@@ -20,7 +20,7 @@ from .usage import usage_int
def create_openai_chat_provider(
provider_id: str,
config: ProviderConfig,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> OpenAIChatProvider:
"""Construct one profile-driven provider."""
profile = OPENAI_CHAT_PROFILES.get(provider_id)
@@ -29,7 +29,7 @@ def create_openai_chat_provider(
return OpenAIChatProvider(
config,
profile=profile,
- rate_limiter=rate_limiter,
+ admission=admission,
default_headers=(
{"User-Agent": profile.user_agent} if profile.user_agent else None
),
diff --git a/src/free_claude_code/providers/openai_chat/provider.py b/src/free_claude_code/providers/openai_chat/provider.py
index 1f531d28..f61f31f6 100644
--- a/src/free_claude_code/providers/openai_chat/provider.py
+++ b/src/free_claude_code/providers/openai_chat/provider.py
@@ -29,16 +29,22 @@ from free_claude_code.core.anthropic.streaming import (
from free_claude_code.core.failures import ExecutionFailure
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
from free_claude_code.core.trace import provider_chat_body_snapshot, trace_event
+from free_claude_code.providers.admission import (
+ ProviderAdmissionController,
+ ProviderAttempt,
+ ProviderRetrySession,
+)
from free_claude_code.providers.base import BaseProvider, ProviderConfig
-from free_claude_code.providers.failure_policy import classify_provider_failure
+from free_claude_code.providers.failure_policy import (
+ classify_provider_failure,
+ underlying_provider_error,
+)
from free_claude_code.providers.http import (
close_provider_stream,
maybe_await_aclose,
)
from free_claude_code.providers.model_listing import extract_openai_model_ids
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from free_claude_code.providers.stream_recovery import (
- MIDSTREAM_RECOVERY_ATTEMPTS,
RecoveryController,
RecoveryFailureAction,
TruncatedProviderStreamError,
@@ -74,7 +80,7 @@ class OpenAIChatProvider(BaseProvider):
config: ProviderConfig,
*,
profile: OpenAIChatProfile,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
default_headers: Mapping[str, str] | None = None,
api_key_provider: OpenAIAsyncCredentialProvider | None = None,
):
@@ -86,7 +92,7 @@ class OpenAIChatProvider(BaseProvider):
# Learned per-model output-token caps from upstream 400 rejections, so
# later requests clamp proactively instead of paying the 400 each time.
self._model_output_caps: dict[str, int] = {}
- self._rate_limiter = rate_limiter
+ self._admission = admission
http_client = None
if config.proxy:
http_client = httpx.AsyncClient(
@@ -120,7 +126,10 @@ class OpenAIChatProvider(BaseProvider):
async def list_model_ids(self) -> frozenset[str]:
"""Return model ids from the provider's OpenAI-compatible models endpoint."""
- payload = await self._client.models.list()
+ payload = await self._admission.run_with_retry(
+ self._client.models.list,
+ provider_failure_override=self._provider_failure_override,
+ )
return extract_openai_model_ids(payload, provider_name=self._provider_name)
def _build_request_body(
@@ -177,26 +186,58 @@ class OpenAIChatProvider(BaseProvider):
"""Return provider-specific Anthropic usage fields for final SSE usage."""
return {}
- async def _create_stream(self, body: dict) -> tuple[Any, dict]:
+ async def _create_stream(
+ self,
+ body: dict,
+ retry_session: ProviderRetrySession,
+ ) -> tuple[Any, dict, ProviderAttempt]:
"""Create a streaming chat completion with bounded request fallbacks."""
body = self._apply_learned_output_cap(body)
used_retry_kinds: set[str] = set()
- while True:
+ while retry_session.can_attempt:
+ attempt = await self._admission.open_attempt(retry_session)
+ stream: Any | None = None
+ retain_attempt = False
try:
create_body = self._prepare_create_body(body)
- stream = await self._rate_limiter.execute_with_retry(
- self._client.chat.completions.create,
- provider_failure_override=self._provider_failure_override,
+ stream = await self._client.chat.completions.create(
**create_body,
stream=True,
)
- return stream, body
+ stream = self._normalize_stream(stream)
+ retain_attempt = True
+ return stream, body, attempt
+ except asyncio.CancelledError:
+ raise
except Exception as error:
retry_body = self._next_create_retry_body(error, body, used_retry_kinds)
- if retry_body is None:
+ if retry_body is not None and retry_session.can_attempt:
+ await attempt.retry_immediately()
+ body = retry_body
+ continue
+ should_retry = await attempt.retry(
+ error,
+ provider_failure_override=self._provider_failure_override,
+ )
+ if not should_retry:
raise
- body = retry_body
+ finally:
+ if not retain_attempt:
+ if stream is not None:
+ await close_provider_stream(
+ stream,
+ active_error=sys.exception(),
+ provider_name=self._provider_name,
+ request_id=retry_session.request_id,
+ )
+ await attempt.aclose()
+
+ raise RuntimeError("provider retry session exhausted without a final error")
+
+ def _normalize_stream(self, stream: Any) -> Any:
+ """Return the provider-specific stream view consumed by the base runner."""
+ return stream
def _next_create_retry_body(
self,
@@ -304,9 +345,9 @@ class _OpenAIChatStreamRunner:
tag = self._provider._provider_name
req_tag = f" request_id={self._request_id}" if self._request_id else ""
ledger = self._new_ledger()
- recovery = RecoveryController(
- provider_name=tag,
- request_id=self._request_id,
+ recovery = RecoveryController()
+ retry_session = self._provider._admission.new_retry_session(
+ request_id=self._request_id
)
def hold_event(event: str) -> Iterator[str]:
@@ -342,219 +383,246 @@ class _OpenAIChatStreamRunner:
tool_argument_aliases: dict[str, dict[str, str]] = {}
tool_argument_alias_buffers: dict[int, str] = {}
- async with self._provider._rate_limiter.concurrency_slot():
- while True:
- if not ledger.message_started:
- for event in hold_event(ledger.message_start()):
- yield event
- stream: Any | None = None
- stream_opened = False
- try:
- stream, body = await self._provider._create_stream(body)
- stream_opened = True
- tool_argument_aliases = self._provider._tool_argument_aliases(body)
- async for chunk in stream:
- chunk_usage = getattr(chunk, "usage", None)
- if chunk_usage is not None:
- usage_info = chunk_usage
-
- if not chunk.choices:
- continue
-
- choice = chunk.choices[0]
- delta = choice.delta
- if delta is None:
- continue
-
- if choice.finish_reason:
- finish_reason = choice.finish_reason
- logger.debug("{} finish_reason: {}", tag, finish_reason)
+ while True:
+ if not ledger.message_started:
+ for event in hold_event(ledger.message_start()):
+ yield event
+ stream: Any | None = None
+ attempt: ProviderAttempt | None = None
+ stream_opened = False
+ try:
+ stream, body, attempt = await self._provider._create_stream(
+ body,
+ retry_session,
+ )
+ stream_opened = True
+ tool_argument_aliases = self._provider._tool_argument_aliases(body)
+ async for chunk in stream:
+ if not attempt.accepted:
+ await attempt.succeeded()
+ chunk_usage = getattr(chunk, "usage", None)
+ if chunk_usage is not None:
+ usage_info = chunk_usage
- reasoning = self._provider._profile.reasoning_delta(delta)
- if output_reasoning and reasoning is not None:
- for event in hold_events(ledger.ensure_thinking_block()):
+ if not chunk.choices:
+ continue
+
+ choice = chunk.choices[0]
+ delta = choice.delta
+ if delta is None:
+ continue
+
+ if choice.finish_reason:
+ finish_reason = choice.finish_reason
+ logger.debug("{} finish_reason: {}", tag, finish_reason)
+
+ reasoning = self._provider._profile.reasoning_delta(delta)
+ if output_reasoning and reasoning is not None:
+ for event in hold_events(ledger.ensure_thinking_block()):
+ yield event
+ if reasoning:
+ for event in hold_event(
+ ledger.emit_thinking_delta(reasoning)
+ ):
yield event
- if reasoning:
+
+ for event in self._provider._handle_extra_reasoning(
+ delta,
+ ledger,
+ output_reasoning=output_reasoning,
+ ):
+ for out_event in hold_event(event):
+ yield out_event
+
+ if delta.content:
+ for part in think_parser.feed(delta.content):
+ if part.type == ContentType.THINKING:
+ if not output_reasoning:
+ continue
+ for event in hold_events(
+ ledger.ensure_thinking_block()
+ ):
+ yield event
for event in hold_event(
- ledger.emit_thinking_delta(reasoning)
+ ledger.emit_thinking_delta(part.content)
):
yield event
+ else:
+ (
+ filtered_text,
+ detected_tools,
+ ) = heuristic_parser.feed(part.content)
- for event in self._provider._handle_extra_reasoning(
- delta,
- ledger,
- output_reasoning=output_reasoning,
- ):
- for out_event in hold_event(event):
- yield out_event
-
- if delta.content:
- for part in think_parser.feed(delta.content):
- if part.type == ContentType.THINKING:
- if not output_reasoning:
- continue
+ if filtered_text:
for event in hold_events(
- ledger.ensure_thinking_block()
+ ledger.ensure_text_block()
):
yield event
for event in hold_event(
- ledger.emit_thinking_delta(part.content)
+ ledger.emit_text_delta(filtered_text)
):
yield event
- else:
- (
- filtered_text,
- detected_tools,
- ) = heuristic_parser.feed(part.content)
-
- if filtered_text:
- for event in hold_events(
- ledger.ensure_text_block()
- ):
- yield event
- for event in hold_event(
- ledger.emit_text_delta(filtered_text)
- ):
- yield event
-
- for tool_use in detected_tools:
- for event in iter_heuristic_tool_use_sse(
- ledger, tool_use
- ):
- for out_event in hold_event(event):
- yield out_event
-
- if delta.tool_calls:
- for event in hold_events(ledger.close_content_blocks()):
- yield event
- for tool_call in delta.tool_calls:
- extra_content = tool_call_extra_content(tool_call)
- tool_call_info = {
- "index": tool_call.index,
- "id": tool_call.id,
- "function": {
- "name": tool_call.function.name,
- "arguments": tool_call.function.arguments,
- },
- }
- if extra_content:
- tool_call_info["extra_content"] = extra_content
- for event in self._tool_calls.process_tool_call(
- tool_call_info,
- ledger,
- tool_argument_aliases=tool_argument_aliases,
- tool_argument_alias_buffers=tool_argument_alias_buffers,
- ):
- for out_event in hold_event(event):
- yield out_event
- if finish_reason is None:
- raise TruncatedProviderStreamError(
- "Provider stream ended without finish_reason."
- )
- break
-
- except asyncio.CancelledError, GeneratorExit:
- raise
- except Exception as error:
- generated_output = has_committed_sse_output(ledger)
- complete_tool_salvageable = (
- generated_output
- and ledger.has_emitted_tool_block()
- and all_emitted_tools_complete(ledger, self._request)
- )
- decision = recovery.advance_failure(
- error,
- stream_opened=stream_opened,
- generated_output=generated_output,
- complete_tool_salvageable=complete_tool_salvageable,
- )
- if decision.action == RecoveryFailureAction.EARLY_RETRY:
- ledger = self._new_ledger()
- think_parser = ThinkTagParser()
- heuristic_parser = HeuristicToolParser()
- finish_reason = None
- usage_info = None
- tool_argument_aliases = {}
- tool_argument_alias_buffers = {}
- continue
+ for tool_use in detected_tools:
+ for event in iter_heuristic_tool_use_sse(
+ ledger, tool_use
+ ):
+ for out_event in hold_event(event):
+ yield out_event
- if decision.action == RecoveryFailureAction.MIDSTREAM_RECOVERY:
- try:
- recovery_events = await self._recovery_events(
- body=body,
- ledger=ledger,
- error=error,
+ if delta.tool_calls:
+ for event in hold_events(ledger.close_content_blocks()):
+ yield event
+ for tool_call in delta.tool_calls:
+ extra_content = tool_call_extra_content(tool_call)
+ tool_call_info = {
+ "index": tool_call.index,
+ "id": tool_call.id,
+ "function": {
+ "name": tool_call.function.name,
+ "arguments": tool_call.function.arguments,
+ },
+ }
+ if extra_content:
+ tool_call_info["extra_content"] = extra_content
+ for event in self._tool_calls.process_tool_call(
+ tool_call_info,
+ ledger,
+ tool_argument_aliases=tool_argument_aliases,
tool_argument_alias_buffers=tool_argument_alias_buffers,
- output_reasoning=output_reasoning,
- )
- except Exception as recovery_error:
- trace_event(
- stage="provider",
- event="provider.recovery.failed",
- source="provider",
- provider=tag,
- request_id=self._request_id,
- exc_type=type(recovery_error).__name__,
- )
- recovery_events = None
- if recovery_events is not None:
- for event in recovery.flush_uncommitted(decision):
- yield event
- for event in recovery_events:
- yield event
- return
+ ):
+ for out_event in hold_event(event):
+ yield out_event
- self._provider._log_stream_transport_error(
- tag, req_tag, error, request_id=self._request_id
+ if finish_reason is None:
+ raise TruncatedProviderStreamError(
+ "Provider stream ended without finish_reason."
)
- failure = classify_provider_failure(
+ break
+
+ except asyncio.CancelledError, GeneratorExit:
+ raise
+ except Exception as error:
+ if attempt is not None and not attempt.accepted:
+ await attempt.retry(
error,
- provider_name=tag,
- read_timeout_s=self._provider._config.http_read_timeout,
- request_id=self._request_id,
- mark_rate_limited=(
- self._provider._rate_limiter.extend_reactive_block
- ),
provider_failure_override=(
self._provider._provider_failure_override
),
)
- error_trace: dict[str, Any] = {
- "stage": "provider",
- "event": "provider.response.error",
- "source": "provider",
- "provider": tag,
- "request_id": self._request_id,
- "exc_type": type(error).__name__,
- "failure_kind": failure.kind.value,
- "status_code": failure.status_code,
- "provider_retryable": failure.retryable,
- }
- if self._provider._config.log_api_error_tracebacks:
- error_trace["error_message"] = failure.message
- trace_event(**error_trace)
- if (
- not decision.committed
- and decision.has_buffered
- and complete_tool_salvageable
- ):
- for event in recovery.flush():
+ generated_output = has_committed_sse_output(ledger)
+ complete_tool_salvageable = (
+ generated_output
+ and ledger.has_emitted_tool_block()
+ and all_emitted_tools_complete(ledger, self._request)
+ )
+ decision = recovery.advance_failure(
+ error,
+ stream_opened=stream_opened,
+ generated_output=generated_output,
+ complete_tool_salvageable=complete_tool_salvageable,
+ attempts_remaining=retry_session.attempts_remaining,
+ retryable_override=(
+ attempt.failure_retryable if attempt is not None else None
+ ),
+ )
+ if decision.action == RecoveryFailureAction.EARLY_RETRY:
+ trace_event(
+ stage="provider",
+ event="provider.recovery.early_retry",
+ source="provider",
+ provider=tag,
+ request_id=self._request_id,
+ attempts_started=retry_session.attempts_started,
+ max_attempts=retry_session.max_attempts,
+ retryable=True,
+ )
+ ledger = self._new_ledger()
+ think_parser = ThinkTagParser()
+ heuristic_parser = HeuristicToolParser()
+ finish_reason = None
+ usage_info = None
+ tool_argument_aliases = {}
+ tool_argument_alias_buffers = {}
+ continue
+
+ if decision.action == RecoveryFailureAction.MIDSTREAM_RECOVERY:
+ try:
+ recovery_events = await self._recovery_events(
+ body=body,
+ ledger=ledger,
+ error=error,
+ tool_argument_alias_buffers=tool_argument_alias_buffers,
+ output_reasoning=output_reasoning,
+ retry_session=retry_session,
+ )
+ except Exception as recovery_error:
+ trace_event(
+ stage="provider",
+ event="provider.recovery.failed",
+ source="provider",
+ provider=tag,
+ request_id=self._request_id,
+ exc_type=type(recovery_error).__name__,
+ )
+ recovery_events = None
+ if recovery_events is not None:
+ for event in recovery.flush_uncommitted(decision):
+ yield event
+ for event in recovery_events:
yield event
- elif not decision.committed:
- recovery.discard()
- raise failure from error
- for event in ledger.close_unclosed_blocks():
+ return
+
+ reported_error = underlying_provider_error(error)
+ self._provider._log_stream_transport_error(
+ tag, req_tag, reported_error, request_id=self._request_id
+ )
+ failure = classify_provider_failure(
+ reported_error,
+ provider_name=tag,
+ read_timeout_s=self._provider._config.http_read_timeout,
+ request_id=self._request_id,
+ provider_failure_override=(
+ self._provider._provider_failure_override
+ ),
+ )
+ error_trace: dict[str, Any] = {
+ "stage": "provider",
+ "event": "provider.response.error",
+ "source": "provider",
+ "provider": tag,
+ "request_id": self._request_id,
+ "exc_type": type(reported_error).__name__,
+ "failure_kind": failure.kind.value,
+ "status_code": failure.status_code,
+ "provider_retryable": failure.retryable,
+ }
+ if self._provider._config.log_api_error_tracebacks:
+ error_trace["error_message"] = failure.message
+ trace_event(**error_trace)
+ if (
+ not decision.committed
+ and decision.has_buffered
+ and complete_tool_salvageable
+ ):
+ for event in recovery.flush():
yield event
+ elif not decision.committed:
+ recovery.discard()
raise failure from error
- finally:
- if stream is not None:
- await close_provider_stream(
- stream,
- active_error=sys.exception(),
- provider_name=tag,
- request_id=self._request_id,
- )
+ for event in ledger.close_unclosed_blocks():
+ yield event
+ raise failure from error
+ finally:
+ if stream is not None:
+ await close_provider_stream(
+ stream,
+ active_error=sys.exception(),
+ provider_name=tag,
+ request_id=self._request_id,
+ )
+ if attempt is not None:
+ await attempt.aclose()
remaining = think_parser.flush()
if remaining:
@@ -650,18 +718,28 @@ class _OpenAIChatStreamRunner:
yield event
async def _collect_recovery_text(
- self, body: dict[str, Any], *, include_reasoning: bool
+ self,
+ body: dict[str, Any],
+ *,
+ include_reasoning: bool,
+ retry_session: ProviderRetrySession,
) -> tuple[str, str]:
"""Collect a complete text/reasoning continuation stream."""
last_error: Exception | None = None
- for attempt in range(MIDSTREAM_RECOVERY_ATTEMPTS):
+ while retry_session.can_attempt:
stream: Any | None = None
+ attempt: ProviderAttempt | None = None
try:
- stream, _ = await self._provider._create_stream(body)
+ stream, _, attempt = await self._provider._create_stream(
+ body,
+ retry_session,
+ )
text_parts: list[str] = []
thinking_parts: list[str] = []
terminal_seen = False
async for chunk in stream:
+ if not attempt.accepted:
+ await attempt.succeeded()
if not getattr(chunk, "choices", None):
continue
choice = chunk.choices[0]
@@ -684,7 +762,17 @@ class _OpenAIChatStreamRunner:
return "".join(text_parts), "".join(thinking_parts)
except Exception as error:
last_error = error
- if not is_retryable_stream_error(error):
+ retryable = is_retryable_stream_error(error)
+ if attempt is not None and not attempt.accepted:
+ await attempt.retry(
+ error,
+ provider_failure_override=(
+ self._provider._provider_failure_override
+ ),
+ )
+ if attempt.failure_retryable is not None:
+ retryable = attempt.failure_retryable
+ if not retryable or not retry_session.can_attempt:
raise
trace_event(
stage="provider",
@@ -692,13 +780,15 @@ class _OpenAIChatStreamRunner:
source="provider",
provider=self._provider._provider_name,
recovery_kind="openai_text",
- attempt=attempt + 1,
- max_attempts=MIDSTREAM_RECOVERY_ATTEMPTS,
+ attempts_started=retry_session.attempts_started,
+ max_attempts=retry_session.max_attempts,
exc_type=type(error).__name__,
)
finally:
if stream is not None:
await maybe_await_aclose(stream)
+ if attempt is not None:
+ await attempt.aclose()
if last_error is not None:
raise last_error
return "", ""
@@ -711,6 +801,7 @@ class _OpenAIChatStreamRunner:
error: Exception,
tool_argument_alias_buffers: dict[int, str],
output_reasoning: bool,
+ retry_session: ProviderRetrySession,
) -> list[str] | None:
"""Build terminal recovery events when the interrupted stream permits it."""
if not is_retryable_stream_error(error):
@@ -722,6 +813,7 @@ class _OpenAIChatStreamRunner:
body=body,
ledger=ledger,
tool_argument_alias_buffers=tool_argument_alias_buffers,
+ retry_session=retry_session,
)
if repair_events is None:
return None
@@ -752,7 +844,9 @@ class _OpenAIChatStreamRunner:
recovery_body = make_text_recovery_body(body, partial_text, partial_thinking)
text, thinking = await self._collect_recovery_text(
- recovery_body, include_reasoning=output_reasoning
+ recovery_body,
+ include_reasoning=output_reasoning,
+ retry_session=retry_session,
)
text_suffix = continuation_suffix(partial_text, text)
thinking_suffix = continuation_suffix(partial_thinking, thinking)
@@ -787,6 +881,7 @@ class _OpenAIChatStreamRunner:
body: dict[str, Any],
ledger: AnthropicStreamLedger,
tool_argument_alias_buffers: dict[int, str],
+ retry_session: ProviderRetrySession,
) -> list[str] | None:
schemas = tool_schemas_by_name(self._request)
events: list[str] = []
@@ -814,9 +909,13 @@ class _OpenAIChatStreamRunner:
input_schema=schema.input_schema if schema is not None else None,
)
accepted_suffix: str | None = None
- for attempt in range(MIDSTREAM_RECOVERY_ATTEMPTS):
+ repair_attempt = 0
+ while retry_session.can_attempt:
+ repair_attempt += 1
text, _ = await self._collect_recovery_text(
- recovery_body, include_reasoning=False
+ recovery_body,
+ include_reasoning=False,
+ retry_session=retry_session,
)
repair = accept_tool_json_repair(
repair_prefix,
@@ -832,7 +931,7 @@ class _OpenAIChatStreamRunner:
source="provider",
provider=self._provider._provider_name,
tool_name=state.name,
- attempt=attempt + 1,
+ attempt=repair_attempt,
)
break
if accepted_suffix is None:
diff --git a/src/free_claude_code/providers/rate_limit.py b/src/free_claude_code/providers/rate_limit.py
deleted file mode 100644
index 5ec73377..00000000
--- a/src/free_claude_code/providers/rate_limit.py
+++ /dev/null
@@ -1,233 +0,0 @@
-"""Provider-owned upstream rate limiting and retry policy."""
-
-import asyncio
-import random
-import time
-from collections.abc import AsyncIterator, Callable
-from contextlib import asynccontextmanager
-from typing import Any, TypeVar
-
-from loguru import logger
-
-from free_claude_code.core.rate_limit import StrictSlidingWindowLimiter
-from free_claude_code.core.trace import trace_event
-from free_claude_code.providers.failure_policy import (
- ProviderFailureOverride,
- retryable_upstream_status,
- retryable_upstream_transport_error,
-)
-
-T = TypeVar("T")
-
-UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS = 5
-DEFAULT_UPSTREAM_MAX_RETRIES = UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS - 1
-
-
-class ProviderRateLimiter:
- """
- Rate limiter owned by one provider instance.
-
- Blocks that provider's requests when a rate-limit error is encountered
- (reactive) and throttles its requests with a strict rolling window
- (proactive).
-
- Optionally enforces a max_concurrency cap: at most N provider streams
- may be open simultaneously, independent of the sliding window.
-
- Proactive limits - throttles requests to stay within API limits.
- Reactive limits - pauses all requests when a 429 or 5xx retry backoff is active.
- Concurrency limit - caps simultaneously open streams.
- """
-
- def __init__(
- self,
- rate_limit: int = 40,
- rate_window: float = 60.0,
- max_concurrency: int = 5,
- ):
- if rate_limit <= 0:
- raise ValueError("rate_limit must be > 0")
- if rate_window <= 0:
- raise ValueError("rate_window must be > 0")
- if max_concurrency <= 0:
- raise ValueError("max_concurrency must be > 0")
-
- self._rate_limit = rate_limit
- self._rate_window = float(rate_window)
- self._max_concurrency = max_concurrency
- self._proactive_limiter = StrictSlidingWindowLimiter(
- self._rate_limit, self._rate_window
- )
- self._blocked_until: float = 0
- self._concurrency_sem = asyncio.Semaphore(max_concurrency)
- logger.info(
- "ProviderRateLimiter initialized "
- f"({rate_limit} req / {rate_window}s, max_concurrency={max_concurrency})"
- )
-
- async def wait_if_blocked(self) -> bool:
- """
- Wait if currently rate limited or throttle to meet quota.
-
- Returns:
- True if was reactively blocked and waited, False otherwise.
- """
- # A reactive deadline can be installed or extended while this task waits
- # for proactive capacity. Commit the proactive timestamp only if that
- # deadline is still clear, so retries neither burst nor consume unused quota.
- waited_reactively = False
- while True:
- waited_reactively = (
- await self._wait_for_reactive_block() or waited_reactively
- )
- if await self._proactive_limiter.acquire_if(lambda: not self.is_blocked()):
- return waited_reactively
-
- async def _wait_for_reactive_block(self) -> bool:
- waited = False
- while (wait_time := self.remaining_wait()) > 0:
- logger.warning(
- "Provider rate limit active (reactive), waiting {:.1f}s...",
- wait_time,
- )
- await asyncio.sleep(wait_time)
- waited = True
- return waited
-
- def extend_reactive_block(self, seconds: float) -> None:
- """
- Extend this provider's reactive block by at least ``seconds`` from now.
-
- Args:
- seconds: Positive minimum duration for the resulting block.
- """
- if seconds <= 0:
- raise ValueError("reactive block duration must be > 0")
- now = time.monotonic()
- self._blocked_until = max(self._blocked_until, now + seconds)
- logger.warning(
- "Provider rate limit set for {:.1f}s (reactive)",
- max(0.0, self._blocked_until - now),
- )
-
- def is_blocked(self) -> bool:
- """Check if currently reactively blocked."""
- return time.monotonic() < self._blocked_until
-
- def remaining_wait(self) -> float:
- """Get remaining reactive wait time in seconds."""
- return max(0.0, self._blocked_until - time.monotonic())
-
- @asynccontextmanager
- async def concurrency_slot(self) -> AsyncIterator[None]:
- """Async context manager that holds one concurrency slot for a stream.
-
- Blocks until a slot is available (controlled by max_concurrency).
- """
- await self._concurrency_sem.acquire()
- try:
- yield
- finally:
- self._concurrency_sem.release()
-
- async def execute_with_retry(
- self,
- fn: Callable[..., Any],
- *args: Any,
- provider_failure_override: ProviderFailureOverride | None = None,
- max_retries: int = DEFAULT_UPSTREAM_MAX_RETRIES,
- base_delay: float = 2.0,
- max_delay: float = 60.0,
- jitter: float = 1.0,
- **kwargs: Any,
- ) -> Any:
- """Execute an async callable with rate limiting and retry on transient limits.
-
- Waits for the proactive limiter before each attempt. On ``429`` (rate limit)
- or upstream ``5xx`` server errors, applies exponential backoff with jitter
- and sets the reactive block before retrying. Pre-response transport errors
- use the same attempt budget and backoff schedule without setting the
- reactive provider block.
-
- Args:
- fn: Async callable to execute.
- provider_failure_override: Optional provider-specific semantic
- classifier applied before shared retry qualification.
- max_retries: Maximum number of retry attempts after the first failure.
- base_delay: Base delay in seconds for exponential backoff.
- max_delay: Maximum delay cap in seconds.
- jitter: Maximum random jitter in seconds added to each delay.
-
- Returns:
- The result of the callable.
-
- Raises:
- The last exception if all retries are exhausted.
- """
- last_exc: Exception | None = None
- total_attempts = 1 + max_retries
-
- for attempt in range(total_attempts):
- await self.wait_if_blocked()
-
- try:
- return await fn(*args, **kwargs)
- except Exception as e:
- effective_error = (
- provider_failure_override(e)
- if provider_failure_override is not None
- else None
- )
- if effective_error is None:
- effective_error = e
- status = retryable_upstream_status(effective_error)
- transport_error = status is None and retryable_upstream_transport_error(
- effective_error
- )
- if status is None and not transport_error:
- raise
-
- if status is None:
- label = f"Provider transport error ({type(e).__name__})"
- else:
- label = (
- "Rate limited (429)"
- if status == 429
- else f"Upstream server error ({status})"
- )
- last_exc = e
- if attempt >= max_retries:
- logger.warning(
- "{} retry exhausted after {} retries (attempts={})",
- label,
- max_retries,
- total_attempts,
- )
- break
-
- delay = min(base_delay * (2**attempt), max_delay)
- delay += random.uniform(0, jitter)
- attempt_no = attempt + 1
- logger.warning(
- "{}, attempt {}/{}. Retrying in {:.1f}s...",
- label,
- attempt_no,
- total_attempts,
- delay,
- )
- trace_event(
- stage="provider",
- event="provider.retry.scheduled",
- source="provider",
- status_code=status,
- exc_type=type(e).__name__,
- attempt=attempt_no,
- max_attempts=total_attempts,
- delay_s=round(delay, 3),
- )
- if status is not None:
- self.extend_reactive_block(delay)
- await asyncio.sleep(delay)
-
- assert last_exc is not None
- raise last_exc
diff --git a/src/free_claude_code/providers/runtime/factory.py b/src/free_claude_code/providers/runtime/factory.py
index 00216fa8..e6e547bd 100644
--- a/src/free_claude_code/providers/runtime/factory.py
+++ b/src/free_claude_code/providers/runtime/factory.py
@@ -5,102 +5,102 @@ from collections.abc import Callable
from free_claude_code.application.errors import UnknownProviderError
from free_claude_code.config.provider_catalog import PROVIDER_CATALOG
from free_claude_code.config.settings import Settings
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import BaseProvider, ProviderConfig
from free_claude_code.providers.openai_chat import (
OPENAI_CHAT_PROFILES,
create_openai_chat_provider,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .config import build_provider_config
ProviderFactory = Callable[
- [ProviderConfig, Settings, ProviderRateLimiter], BaseProvider
+ [ProviderConfig, Settings, ProviderAdmissionController], BaseProvider
]
def _create_nvidia_nim(
config: ProviderConfig,
settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
return NvidiaNimProvider(
config,
nim_settings=settings.nim,
- rate_limiter=rate_limiter,
+ admission=admission,
)
def _create_open_router(
config: ProviderConfig,
_settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.open_router import OpenRouterProvider
- return OpenRouterProvider(config, rate_limiter=rate_limiter)
+ return OpenRouterProvider(config, admission=admission)
def _create_mistral(
config: ProviderConfig,
_settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.mistral import MistralProvider
- return MistralProvider(config, rate_limiter=rate_limiter)
+ return MistralProvider(config, admission=admission)
def _create_deepseek(
config: ProviderConfig,
_settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.deepseek import DeepSeekProvider
- return DeepSeekProvider(config, rate_limiter=rate_limiter)
+ return DeepSeekProvider(config, admission=admission)
def _create_lmstudio(
config: ProviderConfig,
_settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.lmstudio import LMStudioProvider
- return LMStudioProvider(config, rate_limiter=rate_limiter)
+ return LMStudioProvider(config, admission=admission)
def _create_cloudflare(
config: ProviderConfig,
settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.cloudflare import CloudflareProvider
return CloudflareProvider(
config,
account_id=settings.cloudflare_account_id,
- rate_limiter=rate_limiter,
+ admission=admission,
)
def _create_gemini(
config: ProviderConfig,
_settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.gemini import GeminiProvider
- return GeminiProvider(config, rate_limiter=rate_limiter)
+ return GeminiProvider(config, admission=admission)
def _create_vertex(
config: ProviderConfig,
settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.vertex import VertexProvider
@@ -108,18 +108,18 @@ def _create_vertex(
config,
project_id=settings.vertex_project_id,
location=settings.vertex_location,
- rate_limiter=rate_limiter,
+ admission=admission,
)
def _create_github_models(
config: ProviderConfig,
_settings: Settings,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
) -> BaseProvider:
from free_claude_code.providers.github_models import GitHubModelsProvider
- return GitHubModelsProvider(config, rate_limiter=rate_limiter)
+ return GitHubModelsProvider(config, admission=admission)
_SPECIAL_PROVIDER_FACTORIES: dict[str, ProviderFactory] = {
@@ -153,12 +153,13 @@ def create_provider(provider_id: str, settings: Settings) -> BaseProvider:
raise UnknownProviderError.for_provider(provider_id, PROVIDER_CATALOG)
config = build_provider_config(descriptor, settings)
- rate_limiter = ProviderRateLimiter(
+ admission = ProviderAdmissionController(
+ provider_name=provider_id,
rate_limit=config.rate_limit or 40,
rate_window=config.rate_window or 60.0,
max_concurrency=config.max_concurrency,
)
factory = _SPECIAL_PROVIDER_FACTORIES.get(provider_id)
if factory is not None:
- return factory(config, settings, rate_limiter)
- return create_openai_chat_provider(provider_id, config, rate_limiter)
+ return factory(config, settings, admission)
+ return create_openai_chat_provider(provider_id, config, admission)
diff --git a/src/free_claude_code/providers/stream_recovery.py b/src/free_claude_code/providers/stream_recovery.py
index 4ea18167..7d79ebe6 100644
--- a/src/free_claude_code/providers/stream_recovery.py
+++ b/src/free_claude_code/providers/stream_recovery.py
@@ -9,18 +9,14 @@ import httpx
import openai
from free_claude_code.core.failures import ExecutionFailure
-from free_claude_code.core.trace import trace_event
-from .failure_policy import retryable_transient_status
+from .failure_policy import RetryableProviderProtocolError, retryable_transient_status
-EARLY_TRANSPARENT_TOTAL_ATTEMPTS = 5
-EARLY_TRANSPARENT_MAX_RETRIES = EARLY_TRANSPARENT_TOTAL_ATTEMPTS - 1
-MIDSTREAM_RECOVERY_ATTEMPTS = 5
EARLY_HOLDBACK_SECONDS = 0.75
RECOVERY_BUFFER_MAX_BYTES = 65_536
-class TruncatedProviderStreamError(RuntimeError):
+class TruncatedProviderStreamError(RetryableProviderProtocolError):
"""An upstream stream ended without its required terminal marker."""
@@ -40,8 +36,6 @@ class RecoveryDecision:
retryable: bool
committed: bool
has_buffered: bool
- early_retry_attempt: int | None = None
- midstream_recovery_attempt: int | None = None
class RecoveryHoldbackBuffer:
@@ -97,14 +91,10 @@ class RecoveryHoldbackBuffer:
class RecoveryController:
- """Own holdback and retry counters for one provider stream lifecycle."""
+ """Own commit-boundary holdback for one provider stream lifecycle."""
- def __init__(self, *, provider_name: str, request_id: str | None) -> None:
- self._provider_name = provider_name
- self._request_id = request_id
+ def __init__(self) -> None:
self._holdback = RecoveryHoldbackBuffer()
- self._early_retry_count = 0
- self._midstream_recovery_count = 0
@property
def committed(self) -> bool:
@@ -114,14 +104,6 @@ class RecoveryController:
def has_buffered(self) -> bool:
return self._holdback.has_buffered
- @property
- def early_retries(self) -> int:
- return self._early_retry_count
-
- @property
- def midstream_recoveries(self) -> int:
- return self._midstream_recovery_count
-
def push(self, event: str) -> list[str]:
return self._holdback.push(event)
@@ -143,50 +125,46 @@ class RecoveryController:
stream_opened: bool,
generated_output: bool,
complete_tool_salvageable: bool,
+ attempts_remaining: int,
+ retryable_override: bool | None = None,
) -> RecoveryDecision:
- retryable = is_retryable_stream_error(error)
+ retryable = (
+ is_retryable_stream_error(error)
+ if retryable_override is None
+ else retryable_override
+ )
committed = self._holdback.committed
has_buffered = self._holdback.has_buffered
+ retry_available = attempts_remaining > 0
+ reserve_last_attempt_for_recovery = generated_output and attempts_remaining == 1
if (
retryable
+ and retry_available
and stream_opened
and not committed
and not complete_tool_salvageable
- and self._early_retry_count < EARLY_TRANSPARENT_MAX_RETRIES
+ and not reserve_last_attempt_for_recovery
):
- self._early_retry_count += 1
self._holdback.discard()
self._holdback = RecoveryHoldbackBuffer()
- trace_event(
- stage="provider",
- event="provider.recovery.early_retry",
- source="provider",
- provider=self._provider_name,
- request_id=self._request_id,
- retry_attempt=self._early_retry_count,
- retryable=True,
- )
return RecoveryDecision(
action=RecoveryFailureAction.EARLY_RETRY,
retryable=True,
committed=False,
has_buffered=has_buffered,
- early_retry_attempt=self._early_retry_count,
)
if (
retryable
and generated_output
- and self._midstream_recovery_count < MIDSTREAM_RECOVERY_ATTEMPTS
+ and (retry_available or complete_tool_salvageable)
):
- self._midstream_recovery_count += 1
return RecoveryDecision(
action=RecoveryFailureAction.MIDSTREAM_RECOVERY,
retryable=True,
committed=committed,
has_buffered=has_buffered,
- midstream_recovery_attempt=self._midstream_recovery_count,
)
return RecoveryDecision(
diff --git a/src/free_claude_code/providers/vertex/client.py b/src/free_claude_code/providers/vertex/client.py
index e90c2864..14d34b55 100644
--- a/src/free_claude_code/providers/vertex/client.py
+++ b/src/free_claude_code/providers/vertex/client.py
@@ -6,6 +6,7 @@ from typing import Any
import httpx
from free_claude_code.core.anthropic import ReasoningReplayMode
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.google_openai import (
GoogleOpenAIProvider,
@@ -17,7 +18,6 @@ from free_claude_code.providers.openai_chat import (
OpenAIChatProfile,
OpenAIChatRequestPolicy,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from .auth import GoogleAccessTokenProvider
from .endpoint import vertex_openai_base_url, vertex_publisher_models_url
@@ -39,7 +39,7 @@ class VertexProvider(GoogleOpenAIProvider):
*,
project_id: str,
location: str,
- rate_limiter: ProviderRateLimiter,
+ admission: ProviderAdmissionController,
access_token_provider: GoogleAccessTokenProvider | None = None,
) -> None:
self._project_id = project_id.strip()
@@ -61,7 +61,7 @@ class VertexProvider(GoogleOpenAIProvider):
super().__init__(
replace(config, base_url=base_url),
profile=_PROFILE,
- rate_limiter=rate_limiter,
+ admission=admission,
api_key_provider=self._access_token_provider,
default_headers={"x-goog-user-project": self._project_id},
)
@@ -114,7 +114,7 @@ class VertexProvider(GoogleOpenAIProvider):
raise
return response
- response = await self._rate_limiter.execute_with_retry(request)
+ response = await self._admission.run_with_retry(request)
try:
try:
return response.json()
diff --git a/tests/api/test_app_lifespan_and_errors.py b/tests/api/test_app_lifespan_and_errors.py
index dcf79364..e24897e8 100644
--- a/tests/api/test_app_lifespan_and_errors.py
+++ b/tests/api/test_app_lifespan_and_errors.py
@@ -356,7 +356,7 @@ async def test_bootstrap_constructs_isolated_runtime_resource_graphs() -> None:
assert isinstance(first_provider, NvidiaNimProvider)
assert isinstance(second_provider, NvidiaNimProvider)
- assert first_provider._rate_limiter is not second_provider._rate_limiter
+ assert first_provider._admission is not second_provider._admission
assert first.runtime._transcriber is not second.runtime._transcriber
finally:
await first_lease.release()
diff --git a/tests/conftest.py b/tests/conftest.py
index 813e9787..bfb155d4 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -8,7 +8,7 @@ from unittest.mock import AsyncMock, MagicMock
import pytest
from free_claude_code.config.settings import Settings
-from tests.providers.support import passthrough_rate_limiter
+from tests.providers.support import immediate_admission
# Set mock environment BEFORE any imports that use Settings
os.environ.setdefault("NVIDIA_NIM_API_KEY", "test_key")
@@ -49,7 +49,7 @@ def nim_provider(provider_config):
return NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -57,7 +57,7 @@ def nim_provider(provider_config):
def open_router_provider(provider_config):
from free_claude_code.providers.open_router import OpenRouterProvider
- return OpenRouterProvider(provider_config, rate_limiter=passthrough_rate_limiter())
+ return OpenRouterProvider(provider_config, admission=immediate_admission())
@pytest.fixture
@@ -71,7 +71,7 @@ def lmstudio_provider(provider_config):
rate_limit=provider_config.rate_limit,
rate_window=provider_config.rate_window,
)
- return LMStudioProvider(lmstudio_config, rate_limiter=passthrough_rate_limiter())
+ return LMStudioProvider(lmstudio_config, admission=immediate_admission())
@pytest.fixture
@@ -88,7 +88,7 @@ def llamacpp_provider(provider_config):
return create_openai_chat_provider(
"llamacpp",
llamacpp_config,
- passthrough_rate_limiter(),
+ immediate_admission(),
)
diff --git a/tests/core/anthropic/test_stream_repair.py b/tests/core/anthropic/test_stream_repair.py
index 1f8c7e30..eca8b102 100644
--- a/tests/core/anthropic/test_stream_repair.py
+++ b/tests/core/anthropic/test_stream_repair.py
@@ -4,6 +4,8 @@ from free_claude_code.core.anthropic.streaming import (
ToolSchema,
accept_tool_json_repair,
continuation_suffix,
+ make_text_recovery_body,
+ make_tool_repair_body,
)
@@ -45,3 +47,24 @@ def test_tool_json_repair_requires_append_only_schema_valid_json() -> None:
)
is None
)
+
+
+def test_recovery_bodies_do_not_own_transport_flags() -> None:
+ body = {
+ "messages": [{"role": "user", "content": "hello"}],
+ "tools": [{"type": "function"}],
+ "tool_choice": "auto",
+ }
+
+ text_body = make_text_recovery_body(body, "partial")
+ tool_body = make_tool_repair_body(
+ body,
+ tool_name="Echo",
+ prefix='{"message":',
+ input_schema={"type": "object"},
+ )
+
+ assert "stream" not in text_body
+ assert "stream" not in tool_body
+ assert "tools" not in text_body
+ assert "tools" not in tool_body
diff --git a/tests/providers/support.py b/tests/providers/support.py
index 16cfe541..62926006 100644
--- a/tests/providers/support.py
+++ b/tests/providers/support.py
@@ -1,85 +1,49 @@
-"""Provider test doubles with explicit limiter ownership."""
-
-from collections.abc import Callable
-from typing import Any
+"""Provider test helpers with explicit admission ownership."""
from free_claude_code.application.reasoning import client_reasoning_policy
from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.core.reasoning import ReasoningPolicy
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import (
OpenAIChatProvider,
create_openai_chat_provider,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
REASONING_DEFAULT = ReasoningPolicy.provider_default()
REASONING_ON = ReasoningPolicy.on()
REASONING_OFF = ReasoningPolicy.off()
-class PassthroughProviderRateLimiter(ProviderRateLimiter):
- """Skip retry timing while retaining the real concurrency context manager."""
-
- def __init__(self) -> None:
- super().__init__(
- rate_limit=1_000_000,
- rate_window=1.0,
- max_concurrency=1_000,
- )
-
- async def execute_with_retry(
- self,
- fn: Callable[..., Any],
- *args: Any,
- **kwargs: Any,
- ) -> Any:
- kwargs.pop("provider_failure_override", None)
- return await fn(*args, **kwargs)
-
-
-class ImmediateRetryProviderRateLimiter(ProviderRateLimiter):
- """Run the real retry policy without exercising wall-clock backoff."""
-
- async def execute_with_retry(
- self,
- fn: Callable[..., Any],
- *args: Any,
- **kwargs: Any,
- ) -> Any:
- kwargs.update(base_delay=0.0, max_delay=0.0, jitter=0.0)
- return await super().execute_with_retry(fn, *args, **kwargs)
-
- def extend_reactive_block(self, seconds: float) -> None:
- """Leave reactive timing to the limiter's dedicated unit tests."""
- del seconds
-
-
-def passthrough_rate_limiter() -> ProviderRateLimiter:
- """Return a fresh limiter test double for one provider instance."""
- return PassthroughProviderRateLimiter()
+def immediate_admission(
+ *,
+ provider_name: str = "TEST",
+ max_attempts: int = 5,
+) -> ProviderAdmissionController:
+ """Return a real controller with deterministic zero-delay recovery."""
+ return ProviderAdmissionController(
+ provider_name=provider_name,
+ rate_limit=1_000_000,
+ rate_window=1.0,
+ max_concurrency=1_000,
+ max_attempts=max_attempts,
+ base_delay=0.0,
+ max_delay=0.0,
+ jitter=0.0,
+ )
def profiled_provider(
provider_id: str,
config: ProviderConfig,
*,
- rate_limiter: ProviderRateLimiter | None = None,
+ admission: ProviderAdmissionController | None = None,
) -> OpenAIChatProvider:
"""Construct one declarative provider for a focused behavior test."""
return create_openai_chat_provider(
provider_id,
config,
- rate_limiter or passthrough_rate_limiter(),
- )
-
-
-def retrying_rate_limiter() -> ProviderRateLimiter:
- """Return a limiter that exercises retry policy without elapsed time."""
- return ImmediateRetryProviderRateLimiter(
- rate_limit=1_000_000,
- rate_window=1.0,
- max_concurrency=1_000,
+ admission or immediate_admission(provider_name=provider_id),
)
diff --git a/tests/providers/test_bedrock.py b/tests/providers/test_bedrock.py
index 32d7c821..f360ab4c 100644
--- a/tests/providers/test_bedrock.py
+++ b/tests/providers/test_bedrock.py
@@ -9,7 +9,7 @@ from free_claude_code.config.provider_catalog import BEDROCK_DEFAULT_BASE
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
BEDROCK_MODEL = "openai.gpt-oss-120b"
@@ -18,7 +18,7 @@ def _provider(base_url: str = BEDROCK_DEFAULT_BASE) -> OpenAIChatProvider:
return profiled_provider(
"bedrock",
ProviderConfig(api_key="bedrock-key", base_url=base_url),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_cerebras.py b/tests/providers/test_cerebras.py
index 395f0ce9..c4eb9ccb 100644
--- a/tests/providers/test_cerebras.py
+++ b/tests/providers/test_cerebras.py
@@ -9,7 +9,7 @@ from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -64,7 +64,7 @@ def cerebras_config():
@pytest.fixture
def cerebras_provider(cerebras_config):
return profiled_provider(
- "cerebras", cerebras_config, rate_limiter=passthrough_rate_limiter()
+ "cerebras", cerebras_config, admission=immediate_admission()
)
@@ -74,7 +74,7 @@ def test_init(cerebras_config):
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
provider = profiled_provider(
- "cerebras", cerebras_config, rate_limiter=passthrough_rate_limiter()
+ "cerebras", cerebras_config, admission=immediate_admission()
)
assert provider._api_key == "test_cerebras_key"
assert provider._base_url == CEREBRAS_DEFAULT_BASE
@@ -125,7 +125,7 @@ def test_replay_is_independent_of_current_turn_reasoning_control():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
body = provider._build_request_body(
make_reasoning_tool_history_request(), reasoning=REASONING_OFF
diff --git a/tests/providers/test_cloudflare.py b/tests/providers/test_cloudflare.py
index af1adfe6..45bd1892 100644
--- a/tests/providers/test_cloudflare.py
+++ b/tests/providers/test_cloudflare.py
@@ -16,7 +16,7 @@ from free_claude_code.providers.cloudflare import (
CloudflareProvider,
cloudflare_ai_base_url,
)
-from tests.providers.support import passthrough_rate_limiter, reasoning_for
+from tests.providers.support import immediate_admission, reasoning_for
_ACCOUNT_ID = "account-123"
_BASE_URL = f"{CLOUDFLARE_AI_REST_ROOT}/accounts/{_ACCOUNT_ID}/ai/v1"
@@ -38,7 +38,7 @@ def cloudflare_provider(cloudflare_config: ProviderConfig) -> CloudflareProvider
return CloudflareProvider(
cloudflare_config,
account_id=_ACCOUNT_ID,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -73,7 +73,7 @@ def test_missing_account_id_raises_authentication_error(
) -> None:
with pytest.raises(ApplicationUnavailableError, match="CLOUDFLARE_ACCOUNT_ID"):
CloudflareProvider(
- cloudflare_config, account_id=" ", rate_limiter=passthrough_rate_limiter()
+ cloudflare_config, account_id=" ", admission=immediate_admission()
)
@@ -89,7 +89,7 @@ def test_init_composes_account_scoped_openai_chat_base_url(
provider = CloudflareProvider(
cloudflare_config,
account_id=_ACCOUNT_ID,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
assert provider._api_key == "test-cloudflare-token"
diff --git a/tests/providers/test_codestral.py b/tests/providers/test_codestral.py
index 762413cc..def47b56 100644
--- a/tests/providers/test_codestral.py
+++ b/tests/providers/test_codestral.py
@@ -7,7 +7,7 @@ import pytest
from free_claude_code.config.provider_catalog import CODESTRAL_DEFAULT_BASE
from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
def make_request(**overrides):
@@ -27,7 +27,7 @@ def codestral_config():
@pytest.fixture
def codestral_provider(codestral_config):
return profiled_provider(
- "mistral_codestral", codestral_config, rate_limiter=passthrough_rate_limiter()
+ "mistral_codestral", codestral_config, admission=immediate_admission()
)
@@ -39,7 +39,7 @@ def test_init(codestral_config):
provider = profiled_provider(
"mistral_codestral",
codestral_config,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
assert provider._api_key == "test_codestral_key"
assert provider._base_url == CODESTRAL_DEFAULT_BASE
@@ -69,7 +69,7 @@ def test_build_request_body_global_disable_blocks_reasoning_mapping():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
body = provider._build_request_body(req)
diff --git a/tests/providers/test_cohere.py b/tests/providers/test_cohere.py
index d46ac6a3..1b15bed8 100644
--- a/tests/providers/test_cohere.py
+++ b/tests/providers/test_cohere.py
@@ -10,7 +10,7 @@ from free_claude_code.config.provider_catalog import COHERE_DEFAULT_BASE
from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -32,9 +32,7 @@ def cohere_config():
@pytest.fixture
def cohere_provider(cohere_config):
- return profiled_provider(
- "cohere", cohere_config, rate_limiter=passthrough_rate_limiter()
- )
+ return profiled_provider("cohere", cohere_config, admission=immediate_admission())
def test_default_base_url_constant():
@@ -46,7 +44,7 @@ def test_init_uses_default_base_url_and_api_key(cohere_config):
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
provider = profiled_provider(
- "cohere", cohere_config, rate_limiter=passthrough_rate_limiter()
+ "cohere", cohere_config, admission=immediate_admission()
)
assert provider._api_key == "test_cohere_key"
@@ -58,9 +56,7 @@ def test_init_strips_trailing_slash(cohere_config):
config = replace(cohere_config, base_url=f"{COHERE_DEFAULT_BASE}/")
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
- provider = profiled_provider(
- "cohere", config, rate_limiter=passthrough_rate_limiter()
- )
+ provider = profiled_provider("cohere", config, admission=immediate_admission())
assert provider._base_url == COHERE_DEFAULT_BASE
@@ -153,7 +149,7 @@ def test_build_request_body_maps_reasoning_off_to_none():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
request = make_request(thinking={"type": "disabled"})
diff --git a/tests/providers/test_deepseek.py b/tests/providers/test_deepseek.py
index e7b1d3ee..001b04d5 100644
--- a/tests/providers/test_deepseek.py
+++ b/tests/providers/test_deepseek.py
@@ -24,7 +24,7 @@ from free_claude_code.providers.deepseek import DeepSeekProvider
from tests.providers.support import (
REASONING_OFF,
REASONING_ON,
- passthrough_rate_limiter,
+ immediate_admission,
reasoning_for,
)
@@ -41,7 +41,7 @@ def deepseek_config():
@pytest.fixture
def deepseek_provider(deepseek_config):
- return DeepSeekProvider(deepseek_config, rate_limiter=passthrough_rate_limiter())
+ return DeepSeekProvider(deepseek_config, admission=immediate_admission())
async def _capture_openai_wire_body(body: dict) -> dict:
@@ -82,9 +82,7 @@ def test_init(deepseek_config):
with patch(
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_client:
- provider = DeepSeekProvider(
- deepseek_config, rate_limiter=passthrough_rate_limiter()
- )
+ provider = DeepSeekProvider(deepseek_config, admission=immediate_admission())
assert provider._api_key == "test_deepseek_key"
assert provider._base_url == "https://api.deepseek.com"
assert mock_client.called
@@ -211,7 +209,7 @@ def test_build_request_body_encodes_reasoning_off():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
request = MessagesRequest.model_validate(
{
@@ -523,7 +521,7 @@ def test_thinking_off_strips_thinking_history():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
request = MessagesRequest.model_validate(
{
@@ -552,7 +550,7 @@ def test_thinking_off_still_replays_required_tool_reasoning():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
request = MessagesRequest.model_validate(
{
@@ -653,7 +651,7 @@ def test_preflight_strips_user_image():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
# Should not raise; image is stripped.
provider.preflight_stream(request, reasoning=REASONING_ON)
@@ -676,7 +674,7 @@ def test_preflight_rejects_mcp_servers():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with pytest.raises(InvalidRequestError, match="mcp_servers"):
provider.preflight_stream(request)
@@ -695,7 +693,7 @@ def test_preflight_rejects_listed_server_tools_in_tools_list():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with pytest.raises(InvalidRequestError, match="web_search"):
provider.preflight_stream(request)
@@ -732,7 +730,7 @@ def test_preflight_rejects_server_tool_result_blocks():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with pytest.raises(InvalidRequestError, match=r"web_search_tool_result|server"):
provider.preflight_stream(request)
diff --git a/tests/providers/test_execution_failure_boundary.py b/tests/providers/test_execution_failure_boundary.py
index dcce33f5..4746efa7 100644
--- a/tests/providers/test_execution_failure_boundary.py
+++ b/tests/providers/test_execution_failure_boundary.py
@@ -14,7 +14,7 @@ from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.http import close_provider_stream
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter
+from tests.providers.support import immediate_admission
class _FailingStream:
@@ -60,7 +60,7 @@ def _provider() -> NvidiaNimProvider:
rate_window=60,
),
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_failure_policy.py b/tests/providers/test_failure_policy.py
index 10e8d788..e2cba6f9 100644
--- a/tests/providers/test_failure_policy.py
+++ b/tests/providers/test_failure_policy.py
@@ -2,7 +2,6 @@
from collections.abc import Callable
from dataclasses import dataclass
-from unittest.mock import Mock
import httpx
import openai
@@ -13,7 +12,12 @@ from free_claude_code.core.diagnostics import (
attach_upstream_error_body,
)
from free_claude_code.core.failures import ExecutionFailure, FailureKind
-from free_claude_code.providers.failure_policy import classify_provider_failure
+from free_claude_code.providers.failure_policy import (
+ ProviderRecoveryExhausted,
+ classify_provider_failure,
+ is_retryable_provider_error,
+ retryable_upstream_status,
+)
def _openai_status_error(
@@ -57,7 +61,6 @@ class _ClassificationCase:
kind: FailureKind
status_code: int
retryable: bool
- rate_limit_block_seconds: int | None = None
_CASES = (
@@ -82,7 +85,6 @@ _CASES = (
FailureKind.RATE_LIMIT,
429,
True,
- 60,
),
_ClassificationCase(
"openai_bad_request",
@@ -126,7 +128,6 @@ _CASES = (
FailureKind.RATE_LIMIT,
429,
True,
- 60,
),
_ClassificationCase(
"statusless_openai_overload_body",
@@ -209,14 +210,11 @@ _CASES = (
def test_raw_provider_failure_maps_to_canonical_failure(
case: _ClassificationCase,
) -> None:
- mark_rate_limited = Mock()
-
failure = classify_provider_failure(
case.error(),
provider_name="TEST_PROVIDER",
read_timeout_s=30.0,
request_id="req_classification",
- mark_rate_limited=mark_rate_limited,
)
assert isinstance(failure, ExecutionFailure)
@@ -226,10 +224,6 @@ def test_raw_provider_failure_maps_to_canonical_failure(
assert failure.message.strip()
assert "Request ID: req_classification" in failure.message
assert "SECRET" not in failure.message
- if case.rate_limit_block_seconds is None:
- mark_rate_limited.assert_not_called()
- else:
- mark_rate_limited.assert_called_once_with(case.rate_limit_block_seconds)
def test_classification_preserves_useful_body_while_redacting_credentials() -> None:
@@ -243,7 +237,6 @@ def test_classification_preserves_useful_body_while_redacting_credentials() -> N
provider_name="LOCAL",
read_timeout_s=60.0,
request_id="req_body",
- mark_rate_limited=Mock(),
)
assert failure.kind is FailureKind.INVALID_REQUEST
@@ -274,7 +267,6 @@ def test_auth_failure_preserves_model_error_body_instead_of_masking_it() -> None
provider_name="OPENCODE_GO",
read_timeout_s=60.0,
request_id="req_model",
- mark_rate_limited=Mock(),
)
assert failure.kind is FailureKind.AUTHENTICATION
@@ -299,7 +291,6 @@ def test_empty_http_error_body_is_reported_explicitly() -> None:
provider_name="EMPTY",
read_timeout_s=30.0,
request_id="req_empty",
- mark_rate_limited=Mock(),
)
assert failure.kind is FailureKind.UPSTREAM
@@ -314,7 +305,6 @@ def test_http_405_diagnostic_names_rejected_upstream_endpoint() -> None:
provider_name="LOCAL",
read_timeout_s=30.0,
request_id="req_405",
- mark_rate_limited=Mock(),
)
assert failure.kind is FailureKind.UPSTREAM
@@ -340,7 +330,6 @@ def test_connection_cause_chain_is_redacted_and_capped() -> None:
provider_name="NIM",
read_timeout_s=30.0,
request_id="req_cause",
- mark_rate_limited=Mock(),
)
assert "Caused by:" in failure.message
@@ -368,8 +357,26 @@ def test_attached_streamed_error_body_remains_bounded() -> None:
provider_name="LONG",
read_timeout_s=30.0,
request_id="req_long",
- mark_rate_limited=Mock(),
)
assert f"truncated after {ERROR_DETAIL_DISPLAY_CAP_BYTES} bytes" in failure.message
assert "x" * 100 in failure.message
+
+
+def test_shared_recovery_exhaustion_preserves_last_provider_failure() -> None:
+ raw_error = _http_status_error(503, "provider still unavailable")
+ exhausted = ProviderRecoveryExhausted(raw_error)
+
+ failure = classify_provider_failure(
+ exhausted,
+ provider_name="SHARED",
+ read_timeout_s=30.0,
+ request_id="req_exhausted",
+ )
+
+ assert failure.kind is FailureKind.OVERLOADED
+ assert failure.status_code == 529
+ assert "provider still unavailable" in failure.message
+ assert "Request ID: req_exhausted" in failure.message
+ assert retryable_upstream_status(exhausted) is None
+ assert not is_retryable_provider_error(exhausted)
diff --git a/tests/providers/test_fireworks.py b/tests/providers/test_fireworks.py
index f0df293b..71887fd0 100644
--- a/tests/providers/test_fireworks.py
+++ b/tests/providers/test_fireworks.py
@@ -12,7 +12,7 @@ from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -28,7 +28,7 @@ def fireworks_provider():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -84,7 +84,7 @@ def test_replay_is_independent_of_current_turn_reasoning_control():
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
request = MessagesRequest.model_validate(
{
diff --git a/tests/providers/test_gemini.py b/tests/providers/test_gemini.py
index 5874897b..37414601 100644
--- a/tests/providers/test_gemini.py
+++ b/tests/providers/test_gemini.py
@@ -11,7 +11,7 @@ from free_claude_code.providers.google_openai import (
GOOGLE_SKIP_THOUGHT_SIGNATURE_VALIDATOR,
)
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, reasoning_for
+from tests.providers.support import immediate_admission, reasoning_for
def make_request(**overrides):
@@ -38,7 +38,7 @@ def gemini_config():
@pytest.fixture
def gemini_provider(gemini_config):
- return GeminiProvider(gemini_config, rate_limiter=passthrough_rate_limiter())
+ return GeminiProvider(gemini_config, admission=immediate_admission())
def test_init(gemini_config):
@@ -46,9 +46,7 @@ def test_init(gemini_config):
with patch(
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
- provider = GeminiProvider(
- gemini_config, rate_limiter=passthrough_rate_limiter()
- )
+ provider = GeminiProvider(gemini_config, admission=immediate_admission())
assert provider._api_key == "test_gemini_key"
assert (
provider._base_url
@@ -110,7 +108,7 @@ def test_build_request_body_reasoning_off_sets_reasoning_none():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request(thinking={"type": "disabled"})
body = provider._build_request_body(req, reasoning=reasoning_for(req))
diff --git a/tests/providers/test_github_models.py b/tests/providers/test_github_models.py
index 422c7e00..f921de1d 100644
--- a/tests/providers/test_github_models.py
+++ b/tests/providers/test_github_models.py
@@ -15,7 +15,7 @@ from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.github_models import GitHubModelsProvider
from free_claude_code.providers.github_models.client import GITHUB_MODELS_CATALOG_URL
from free_claude_code.providers.model_listing import ModelListResponseError
-from tests.providers.support import REASONING_ON, passthrough_rate_limiter
+from tests.providers.support import REASONING_ON, immediate_admission
@pytest.fixture
@@ -32,9 +32,7 @@ def github_models_config() -> ProviderConfig:
def github_models_provider(
github_models_config: ProviderConfig,
) -> GitHubModelsProvider:
- return GitHubModelsProvider(
- github_models_config, rate_limiter=passthrough_rate_limiter()
- )
+ return GitHubModelsProvider(github_models_config, admission=immediate_admission())
def _request(model: str = "openai/gpt-4.1") -> MessagesRequest:
@@ -76,7 +74,7 @@ def test_init_uses_default_base_url_api_key_and_github_headers(
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
provider = GitHubModelsProvider(
- github_models_config, rate_limiter=passthrough_rate_limiter()
+ github_models_config, admission=immediate_admission()
)
assert provider._api_key == "test-github-models-token"
@@ -97,7 +95,7 @@ def test_init_strips_trailing_slash(github_models_config: ProviderConfig) -> Non
)
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
- provider = GitHubModelsProvider(config, rate_limiter=passthrough_rate_limiter())
+ provider = GitHubModelsProvider(config, admission=immediate_admission())
assert provider._base_url == GITHUB_MODELS_DEFAULT_BASE
diff --git a/tests/providers/test_groq.py b/tests/providers/test_groq.py
index 857a0390..8404c631 100644
--- a/tests/providers/test_groq.py
+++ b/tests/providers/test_groq.py
@@ -7,7 +7,7 @@ import pytest
from free_claude_code.config.provider_catalog import GROQ_DEFAULT_BASE
from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
def make_request(**overrides):
@@ -26,9 +26,7 @@ def groq_config():
@pytest.fixture
def groq_provider(groq_config):
- return profiled_provider(
- "groq", groq_config, rate_limiter=passthrough_rate_limiter()
- )
+ return profiled_provider("groq", groq_config, admission=immediate_admission())
def test_init(groq_config):
@@ -37,7 +35,7 @@ def test_init(groq_config):
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
provider = profiled_provider(
- "groq", groq_config, rate_limiter=passthrough_rate_limiter()
+ "groq", groq_config, admission=immediate_admission()
)
assert provider._api_key == "test_groq_key"
assert provider._base_url == GROQ_DEFAULT_BASE
@@ -67,7 +65,7 @@ def test_build_request_body_global_disable_blocks_reasoning_mapping():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
body = provider._build_request_body(req)
diff --git a/tests/providers/test_huggingface.py b/tests/providers/test_huggingface.py
index bb13f187..8987d51c 100644
--- a/tests/providers/test_huggingface.py
+++ b/tests/providers/test_huggingface.py
@@ -10,7 +10,7 @@ from free_claude_code.core.anthropic import ReasoningReplayMode
from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
def make_request(**overrides):
@@ -30,7 +30,7 @@ def huggingface_config():
@pytest.fixture
def huggingface_provider(huggingface_config):
return profiled_provider(
- "huggingface", huggingface_config, rate_limiter=passthrough_rate_limiter()
+ "huggingface", huggingface_config, admission=immediate_admission()
)
@@ -43,7 +43,7 @@ def test_init_uses_default_base_url_and_api_key(huggingface_config):
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
provider = profiled_provider(
- "huggingface", huggingface_config, rate_limiter=passthrough_rate_limiter()
+ "huggingface", huggingface_config, admission=immediate_admission()
)
assert provider._api_key == "test_hf_key"
@@ -56,7 +56,7 @@ def test_init_strips_trailing_slash(huggingface_config):
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
provider = profiled_provider(
- "huggingface", config, rate_limiter=passthrough_rate_limiter()
+ "huggingface", config, admission=immediate_admission()
)
assert provider._base_url == HUGGINGFACE_DEFAULT_BASE
diff --git a/tests/providers/test_kimi.py b/tests/providers/test_kimi.py
index 618a2199..687a1a67 100644
--- a/tests/providers/test_kimi.py
+++ b/tests/providers/test_kimi.py
@@ -12,7 +12,7 @@ from free_claude_code.core.anthropic.models import Message, MessagesRequest
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.support import (
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -28,7 +28,7 @@ def kimi_provider():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_kimi_code.py b/tests/providers/test_kimi_code.py
index f5964c53..9a1dd9ed 100644
--- a/tests/providers/test_kimi_code.py
+++ b/tests/providers/test_kimi_code.py
@@ -11,7 +11,7 @@ from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.support import (
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -36,7 +36,7 @@ def kimi_code_provider() -> OpenAIChatProvider:
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_llamacpp.py b/tests/providers/test_llamacpp.py
index 412512d4..977ce58c 100644
--- a/tests/providers/test_llamacpp.py
+++ b/tests/providers/test_llamacpp.py
@@ -11,7 +11,7 @@ from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -24,7 +24,7 @@ def provider() -> OpenAIChatProvider:
return profiled_provider(
"llamacpp",
ProviderConfig(api_key="llamacpp", base_url="http://localhost:8080/v1"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -44,7 +44,7 @@ def test_init_normalizes_openai_base_url(configured: str, expected: str) -> None
provider = profiled_provider(
"llamacpp",
ProviderConfig(api_key="llamacpp", base_url=configured),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
assert provider._base_url == expected
@@ -63,7 +63,7 @@ def test_init_uses_openai_chat_client() -> None:
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as openai_client:
provider = profiled_provider(
- "llamacpp", config, rate_limiter=passthrough_rate_limiter()
+ "llamacpp", config, admission=immediate_admission()
)
assert provider._provider_name == "LLAMACPP"
diff --git a/tests/providers/test_lmstudio.py b/tests/providers/test_lmstudio.py
index 6d0a70af..418918eb 100644
--- a/tests/providers/test_lmstudio.py
+++ b/tests/providers/test_lmstudio.py
@@ -14,7 +14,7 @@ from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
REASONING_ON,
- passthrough_rate_limiter,
+ immediate_admission,
)
@@ -34,7 +34,7 @@ def lmstudio_config():
@pytest.fixture
def lmstudio_provider(lmstudio_config):
- return LMStudioProvider(lmstudio_config, rate_limiter=passthrough_rate_limiter())
+ return LMStudioProvider(lmstudio_config, admission=immediate_admission())
def test_init(lmstudio_config):
@@ -42,9 +42,7 @@ def test_init(lmstudio_config):
with patch(
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
- provider = LMStudioProvider(
- lmstudio_config, rate_limiter=passthrough_rate_limiter()
- )
+ provider = LMStudioProvider(lmstudio_config, admission=immediate_admission())
assert provider._api_key == "lm-studio"
assert provider._base_url == LMSTUDIO_DEFAULT_BASE
assert provider._provider_name == "LMSTUDIO"
diff --git a/tests/providers/test_minimax.py b/tests/providers/test_minimax.py
index 2051565e..119bf464 100644
--- a/tests/providers/test_minimax.py
+++ b/tests/providers/test_minimax.py
@@ -17,7 +17,7 @@ from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -49,7 +49,7 @@ def minimax_provider():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_mistral.py b/tests/providers/test_mistral.py
index efb863e8..65fe0947 100644
--- a/tests/providers/test_mistral.py
+++ b/tests/providers/test_mistral.py
@@ -14,7 +14,7 @@ from free_claude_code.providers.mistral import MistralProvider
from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
reasoning_for,
)
@@ -35,7 +35,7 @@ def mistral_config():
@pytest.fixture
def mistral_provider(mistral_config):
- return MistralProvider(mistral_config, rate_limiter=passthrough_rate_limiter())
+ return MistralProvider(mistral_config, admission=immediate_admission())
def test_init(mistral_config):
@@ -43,9 +43,7 @@ def test_init(mistral_config):
with patch(
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
- provider = MistralProvider(
- mistral_config, rate_limiter=passthrough_rate_limiter()
- )
+ provider = MistralProvider(mistral_config, admission=immediate_admission())
assert provider._api_key == "test_mistral_key"
assert provider._base_url == MISTRAL_DEFAULT_BASE
mock_openai.assert_called_once()
@@ -146,7 +144,7 @@ def test_build_request_body_reasoning_off_uses_native_none():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
body = provider._build_request_body(req, reasoning=REASONING_OFF)
@@ -163,7 +161,7 @@ def test_reasoning_off_keeps_replay_separate_from_new_turn_compute():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request(
system=None,
diff --git a/tests/providers/test_model_validation.py b/tests/providers/test_model_validation.py
index bfa7d3c4..06512477 100644
--- a/tests/providers/test_model_validation.py
+++ b/tests/providers/test_model_validation.py
@@ -26,7 +26,7 @@ from free_claude_code.providers.openai_chat import OpenAIChatProvider
from free_claude_code.providers.runtime import ProviderRuntime
from free_claude_code.providers.runtime.model_cache import ProviderModelCache
from free_claude_code.runtime.provider_manager import ProviderRuntimeManager
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
def _settings(
@@ -77,7 +77,7 @@ async def test_nim_lists_openai_compatible_model_ids() -> None:
config = ProviderConfig(api_key="test-key", base_url=NVIDIA_NIM_DEFAULT_BASE)
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
provider = NvidiaNimProvider(
- config, nim_settings=NimSettings(), rate_limiter=passthrough_rate_limiter()
+ config, nim_settings=NimSettings(), admission=immediate_admission()
)
with patch.object(
@@ -96,12 +96,12 @@ async def test_nim_lists_openai_compatible_model_ids() -> None:
profiled_provider(
"llamacpp",
ProviderConfig(api_key="llamacpp", base_url="http://localhost:8080/v1"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
),
profiled_provider(
"ollama",
ProviderConfig(api_key="ollama", base_url="http://localhost:11434"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
),
],
)
@@ -123,7 +123,7 @@ async def test_local_openai_chat_providers_list_model_ids(
async def test_deepseek_lists_models_from_root_endpoint() -> None:
provider = DeepSeekProvider(
ProviderConfig(api_key="deepseek-key", base_url=DEEPSEEK_DEFAULT_BASE),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with patch.object(
provider._client.models,
@@ -141,7 +141,7 @@ async def test_wafer_lists_models_from_default_models_endpoint() -> None:
provider = profiled_provider(
"wafer",
ProviderConfig(api_key="wafer-key", base_url=WAFER_DEFAULT_BASE),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with patch.object(
provider._client.models,
@@ -158,7 +158,7 @@ async def test_wafer_lists_models_from_default_models_endpoint() -> None:
async def test_openrouter_lists_only_tool_capable_models() -> None:
provider = OpenRouterProvider(
ProviderConfig(api_key="open-router-key", base_url=OPENROUTER_DEFAULT_BASE),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with patch.object(
provider._client.models,
@@ -193,7 +193,7 @@ async def test_openrouter_lists_only_tool_capable_models() -> None:
async def test_openrouter_lists_tool_metadata_with_thinking_support() -> None:
provider = OpenRouterProvider(
ProviderConfig(api_key="open-router-key", base_url=OPENROUTER_DEFAULT_BASE),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with patch.object(
provider._client.models,
@@ -234,7 +234,7 @@ async def test_openrouter_lists_tool_metadata_with_thinking_support() -> None:
async def test_openrouter_lists_empty_set_when_no_tool_capable_models() -> None:
provider = OpenRouterProvider(
ProviderConfig(api_key="open-router-key", base_url=OPENROUTER_DEFAULT_BASE),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with patch.object(
provider._client.models,
@@ -254,7 +254,7 @@ async def test_openrouter_lists_empty_set_when_no_tool_capable_models() -> None:
async def test_openrouter_model_metadata_rejects_malformed_ids() -> None:
provider = OpenRouterProvider(
ProviderConfig(api_key="open-router-key", base_url=OPENROUTER_DEFAULT_BASE),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with (
patch.object(
@@ -275,7 +275,7 @@ async def test_model_listing_rejects_malformed_payload() -> None:
provider = profiled_provider(
"llamacpp",
ProviderConfig(api_key="llamacpp", base_url="http://localhost:8080/v1"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with (
patch.object(
@@ -294,7 +294,7 @@ async def test_model_listing_propagates_upstream_errors() -> None:
provider = profiled_provider(
"llamacpp",
ProviderConfig(api_key="llamacpp", base_url="http://localhost:8080/v1"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
with (
patch.object(
diff --git a/tests/providers/test_nvidia_nim.py b/tests/providers/test_nvidia_nim.py
index a46c906f..bb36a699 100644
--- a/tests/providers/test_nvidia_nim.py
+++ b/tests/providers/test_nvidia_nim.py
@@ -9,6 +9,7 @@ from free_claude_code.config.nim import NimSettings
from free_claude_code.config.provider_catalog import NVIDIA_NIM_DEFAULT_BASE
from free_claude_code.core.failures import ExecutionFailure
from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
+from free_claude_code.providers.admission import UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from free_claude_code.providers.nvidia_nim.tool_schema import (
NIM_TOOL_ARGUMENT_ALIASES_KEY,
@@ -17,7 +18,7 @@ from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
REASONING_ON,
- passthrough_rate_limiter,
+ immediate_admission,
reasoning_for,
)
@@ -113,7 +114,7 @@ async def test_init(provider_config):
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
assert provider._api_key == "test_key"
assert provider._base_url == "https://test.api.nvidia.com/v1"
@@ -136,7 +137,7 @@ async def test_init_uses_configurable_timeouts():
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
NvidiaNimProvider(
- config, nim_settings=NimSettings(), rate_limiter=passthrough_rate_limiter()
+ config, nim_settings=NimSettings(), admission=immediate_admission()
)
call_kwargs = mock_openai.call_args[1]
timeout = call_kwargs["timeout"]
@@ -151,7 +152,7 @@ async def test_build_request_body(provider_config):
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
body = provider._build_request_body(req, reasoning=reasoning_for(req))
@@ -177,7 +178,7 @@ async def test_build_request_body_encodes_explicit_reasoning_off(
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
body = provider._build_request_body(req, reasoning=REASONING_OFF)
@@ -197,7 +198,7 @@ async def test_build_request_body_omits_reasoning_when_request_disables_thinking
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
req.thinking.enabled = False
@@ -343,7 +344,7 @@ async def test_stream_response_suppresses_thinking_when_disabled(provider_config
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request()
@@ -388,7 +389,7 @@ async def test_stream_response_retries_without_chat_template(provider_config):
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(chat_template="custom_template"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request(model="mistralai/mixtral-8x7b-instruct-v0.1")
@@ -445,7 +446,7 @@ async def test_stream_response_retries_without_chat_template_kwargs_issue_993(
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request(model="mistralai/mistral-small-4-119b-2603")
@@ -498,7 +499,7 @@ async def test_stream_response_does_not_retry_unrelated_bad_request(provider_con
provider = NvidiaNimProvider(
provider_config,
nim_settings=NimSettings(chat_template="custom_template"),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_request(model="mistralai/mixtral-8x7b-instruct-v0.1")
@@ -925,7 +926,11 @@ async def test_stream_response_unrelated_internal_error_does_not_downgrade(
with pytest.raises(ExecutionFailure) as exc_info:
[e async for e in nim_provider.stream_response(req)]
- assert mock_create.await_count == 1
+ assert mock_create.await_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
+ assert all(
+ call.kwargs == mock_create.await_args_list[0].kwargs
+ for call in mock_create.await_args_list
+ )
assert "Provider API request failed" in exc_info.value.message
@@ -946,5 +951,9 @@ async def test_stream_response_internal_reasoning_content_error_does_not_downgra
with pytest.raises(ExecutionFailure) as exc_info:
[e async for e in nim_provider.stream_response(req)]
- assert mock_create.await_count == 1
+ assert mock_create.await_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
+ assert all(
+ call.kwargs == mock_create.await_args_list[0].kwargs
+ for call in mock_create.await_args_list
+ )
assert "Provider API request failed" in exc_info.value.message
diff --git a/tests/providers/test_nvidia_nim_degraded_retry.py b/tests/providers/test_nvidia_nim_degraded_retry.py
index 55d35375..4b50ea39 100644
--- a/tests/providers/test_nvidia_nim_degraded_retry.py
+++ b/tests/providers/test_nvidia_nim_degraded_retry.py
@@ -8,6 +8,10 @@ import pytest
from free_claude_code.config.nim import NimSettings
from free_claude_code.core.failures import ExecutionFailure, FailureKind
+from free_claude_code.providers.admission import (
+ UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS,
+ ProviderAdmissionController,
+)
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.failure_policy import (
overloaded_provider_failure,
@@ -15,11 +19,6 @@ from free_claude_code.providers.failure_policy import (
)
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from free_claude_code.providers.open_router import OpenRouterProvider
-from free_claude_code.providers.rate_limit import (
- DEFAULT_UPSTREAM_MAX_RETRIES,
- UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS,
- ProviderRateLimiter,
-)
from tests.providers.request_factory import make_messages_request
_FUNCTION_ID = "87ea0ddc-cff1-4bca-bf8b-3bd98a35ddd0"
@@ -39,11 +38,16 @@ def _config(base_url: str) -> ProviderConfig:
)
-def _limiter() -> ProviderRateLimiter:
- return ProviderRateLimiter(
+def _admission(*, max_attempts: int = UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS):
+ return ProviderAdmissionController(
+ provider_name="nvidia_nim",
rate_limit=1_000_000,
rate_window=1.0,
max_concurrency=1_000,
+ max_attempts=max_attempts,
+ base_delay=0.0,
+ max_delay=0.0,
+ jitter=0.0,
)
@@ -82,18 +86,18 @@ def _successful_stream(text: str = "Recovered"):
return stream()
-def _nim(limiter: ProviderRateLimiter) -> NvidiaNimProvider:
+def _nim(admission: ProviderAdmissionController) -> NvidiaNimProvider:
return NvidiaNimProvider(
_config("https://integrate.api.nvidia.com/v1"),
nim_settings=NimSettings(),
- rate_limiter=limiter,
+ admission=admission,
)
@pytest.mark.asyncio
async def test_degraded_function_retries_unchanged_request_then_succeeds() -> None:
- limiter = _limiter()
- provider = _nim(limiter)
+ admission = _admission()
+ provider = _nim(admission)
with (
patch.object(
@@ -102,11 +106,6 @@ async def test_degraded_function_retries_unchanged_request_then_succeeds() -> No
new_callable=AsyncMock,
side_effect=[_bad_request(), _successful_stream()],
) as create,
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch(
- "free_claude_code.providers.rate_limit.asyncio.sleep",
- new_callable=AsyncMock,
- ) as sleep,
):
events = [
event
@@ -117,8 +116,6 @@ async def test_degraded_function_retries_unchanged_request_then_succeeds() -> No
assert create.await_count == 2
assert create.call_args_list[0].kwargs == create.call_args_list[1].kwargs
- extend_block.assert_called_once()
- sleep.assert_awaited_once()
event_text = "".join(events)
assert "Recovered" in event_text
assert "event: message_stop" in event_text
@@ -127,8 +124,8 @@ async def test_degraded_function_retries_unchanged_request_then_succeeds() -> No
@pytest.mark.asyncio
async def test_degraded_function_exhaustion_is_detailed_redacted_overload() -> None:
- limiter = _limiter()
- provider = _nim(limiter)
+ admission = _admission()
+ provider = _nim(admission)
error = _bad_request(
body_extra={
"authorization": "Bearer NIM_AUTH_SECRET",
@@ -143,11 +140,6 @@ async def test_degraded_function_exhaustion_is_detailed_redacted_overload() -> N
new_callable=AsyncMock,
side_effect=error,
) as create,
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch(
- "free_claude_code.providers.rate_limit.asyncio.sleep",
- new_callable=AsyncMock,
- ) as sleep,
patch("free_claude_code.providers.openai_chat.provider.trace_event") as trace,
pytest.raises(ExecutionFailure) as exc_info,
):
@@ -159,8 +151,6 @@ async def test_degraded_function_exhaustion_is_detailed_redacted_overload() -> N
]
assert create.await_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
- assert extend_block.call_count == DEFAULT_UPSTREAM_MAX_RETRIES
- assert sleep.await_count == DEFAULT_UPSTREAM_MAX_RETRIES
failure = exc_info.value
assert failure.kind is FailureKind.OVERLOADED
@@ -195,8 +185,8 @@ async def test_degraded_function_exhaustion_is_detailed_redacted_overload() -> N
)
@pytest.mark.asyncio
async def test_unrelated_nim_bad_request_is_not_retried(detail: str) -> None:
- limiter = _limiter()
- provider = _nim(limiter)
+ admission = _admission()
+ provider = _nim(admission)
with (
patch.object(
@@ -205,18 +195,11 @@ async def test_unrelated_nim_bad_request_is_not_retried(detail: str) -> None:
new_callable=AsyncMock,
side_effect=_bad_request(detail),
) as create,
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch(
- "free_claude_code.providers.rate_limit.asyncio.sleep",
- new_callable=AsyncMock,
- ) as sleep,
pytest.raises(ExecutionFailure) as exc_info,
):
[event async for event in provider.stream_response(make_messages_request())]
assert create.await_count == 1
- extend_block.assert_not_called()
- sleep.assert_not_awaited()
assert exc_info.value.kind is FailureKind.INVALID_REQUEST
assert exc_info.value.status_code == 400
assert exc_info.value.retryable is False
@@ -224,9 +207,9 @@ async def test_unrelated_nim_bad_request_is_not_retried(detail: str) -> None:
@pytest.mark.asyncio
async def test_degraded_wording_remains_non_retryable_for_other_providers() -> None:
- limiter = _limiter()
+ admission = _admission()
provider = OpenRouterProvider(
- _config("https://openrouter.ai/api/v1"), rate_limiter=limiter
+ _config("https://openrouter.ai/api/v1"), admission=admission
)
error = _bad_request()
@@ -239,25 +222,18 @@ async def test_degraded_wording_remains_non_retryable_for_other_providers() -> N
new_callable=AsyncMock,
side_effect=error,
) as create,
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch(
- "free_claude_code.providers.rate_limit.asyncio.sleep",
- new_callable=AsyncMock,
- ) as sleep,
pytest.raises(ExecutionFailure) as exc_info,
):
[event async for event in provider.stream_response(make_messages_request())]
assert create.await_count == 1
- extend_block.assert_not_called()
- sleep.assert_not_awaited()
assert exc_info.value.kind is FailureKind.INVALID_REQUEST
assert exc_info.value.status_code == 400
@pytest.mark.asyncio
-async def test_limiter_override_preserves_raw_exception_after_exhaustion() -> None:
- limiter = _limiter()
+async def test_admission_override_preserves_raw_exception_after_exhaustion() -> None:
+ admission = _admission(max_attempts=2)
errors = (_bad_request(), _bad_request())
attempts = 0
@@ -269,17 +245,11 @@ async def test_limiter_override_preserves_raw_exception_after_exhaustion() -> No
override = Mock(return_value=overloaded_provider_failure())
with (
- patch.object(limiter, "extend_reactive_block"),
- patch(
- "free_claude_code.providers.rate_limit.asyncio.sleep",
- new_callable=AsyncMock,
- ),
pytest.raises(openai.BadRequestError) as exc_info,
):
- await limiter.execute_with_retry(
+ await admission.run_with_retry(
fail,
provider_failure_override=override,
- max_retries=1,
)
assert attempts == 2
diff --git a/tests/providers/test_ollama.py b/tests/providers/test_ollama.py
index 4aee854e..5268fe93 100644
--- a/tests/providers/test_ollama.py
+++ b/tests/providers/test_ollama.py
@@ -17,7 +17,7 @@ from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -30,7 +30,7 @@ def _provider(base_url: str = OLLAMA_DEFAULT_BASE) -> OpenAIChatProvider:
return profiled_provider(
"ollama",
ProviderConfig(api_key="ollama", base_url=base_url),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -41,7 +41,7 @@ def _cloud_provider() -> OpenAIChatProvider:
api_key="ollama-cloud-key",
base_url=OLLAMA_CLOUD_DEFAULT_BASE,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_open_router.py b/tests/providers/test_open_router.py
index 308b919c..b5b29535 100644
--- a/tests/providers/test_open_router.py
+++ b/tests/providers/test_open_router.py
@@ -18,7 +18,7 @@ from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.request_factory import make_messages_request
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
reasoning_for,
)
@@ -52,7 +52,7 @@ def open_router_provider():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_openai_chat_output_cap.py b/tests/providers/test_openai_chat_output_cap.py
index 125960dd..898e42b6 100644
--- a/tests/providers/test_openai_chat_output_cap.py
+++ b/tests/providers/test_openai_chat_output_cap.py
@@ -16,7 +16,7 @@ from free_claude_code.providers.openai_chat.output_cap import (
parse_output_token_cap,
)
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
class _BadRequest(Exception):
@@ -120,7 +120,7 @@ def groq_provider():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -140,7 +140,11 @@ async def test_create_stream_clamps_and_learns_on_cap_rejection(groq_provider):
create = AsyncMock(side_effect=[error, object()])
with patch.object(groq_provider._client.chat.completions, "create", create):
- _stream, used_body = await groq_provider._create_stream(body)
+ _stream, used_body, attempt = await groq_provider._create_stream(
+ body,
+ groq_provider._admission.new_retry_session(),
+ )
+ await attempt.aclose()
assert create.call_count == 2
assert create.call_args_list[1].kwargs["max_completion_tokens"] == 40960
@@ -162,7 +166,11 @@ async def test_learned_cap_clamps_next_request_without_a_400(groq_provider):
create = AsyncMock(return_value=object())
with patch.object(groq_provider._client.chat.completions, "create", create):
- _stream, used_body = await groq_provider._create_stream(body)
+ _stream, used_body, attempt = await groq_provider._create_stream(
+ body,
+ groq_provider._admission.new_retry_session(),
+ )
+ await attempt.aclose()
assert create.call_count == 1
assert create.call_args.kwargs["max_completion_tokens"] == 40960
@@ -184,7 +192,10 @@ async def test_unrelated_400_is_not_clamped_and_propagates(groq_provider):
patch.object(groq_provider._client.chat.completions, "create", create),
pytest.raises(Exception, match="wizard"),
):
- await groq_provider._create_stream(body)
+ await groq_provider._create_stream(
+ body,
+ groq_provider._admission.new_retry_session(),
+ )
assert create.call_count == 1
assert groq_provider._model_output_caps == {}
diff --git a/tests/providers/test_openai_chat_usage.py b/tests/providers/test_openai_chat_usage.py
index 109c2b70..2c23fa93 100644
--- a/tests/providers/test_openai_chat_usage.py
+++ b/tests/providers/test_openai_chat_usage.py
@@ -26,7 +26,7 @@ from free_claude_code.providers.openai_chat.usage import (
usage_int,
)
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter
+from tests.providers.support import immediate_admission
class _UsageTestProvider(OpenAIChatProvider):
@@ -45,7 +45,7 @@ class _UsageTestProvider(OpenAIChatProvider):
),
NO_REASONING,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
def _build_request_body(
@@ -216,7 +216,11 @@ async def test_openai_chat_stream_retries_without_usage_when_option_is_rejected(
)
with patch.object(provider._client.chat.completions, "create", create):
- _stream_obj, used_body = await provider._create_stream(body)
+ _stream_obj, used_body, attempt = await provider._create_stream(
+ body,
+ provider._admission.new_retry_session(),
+ )
+ await attempt.aclose()
assert create.await_count == 2
assert create.await_args_list[0].kwargs["stream_options"] == {"include_usage": True}
diff --git a/tests/providers/test_openai_compat_5xx_retry.py b/tests/providers/test_openai_compat_5xx_retry.py
index af8fff00..0edb621e 100644
--- a/tests/providers/test_openai_compat_5xx_retry.py
+++ b/tests/providers/test_openai_compat_5xx_retry.py
@@ -12,7 +12,7 @@ from free_claude_code.core.failures import ExecutionFailure
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import retrying_rate_limiter
+from tests.providers.support import immediate_admission
def _internal_5xx(code: int) -> openai.InternalServerError:
@@ -46,7 +46,7 @@ async def test_nim_stream_retries_on_openai_5xx_then_streams(status_code):
provider = NvidiaNimProvider(
config,
nim_settings=NimSettings(),
- rate_limiter=retrying_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_messages_request()
@@ -90,7 +90,7 @@ async def test_nim_stream_retries_on_pre_stream_connection_error_then_streams():
provider = NvidiaNimProvider(
config,
nim_settings=NimSettings(),
- rate_limiter=retrying_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_messages_request()
@@ -134,7 +134,7 @@ async def test_nim_stream_connection_error_exhausted_emits_cause_chain():
provider = NvidiaNimProvider(
config,
nim_settings=NimSettings(),
- rate_limiter=retrying_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_messages_request()
error = _connection_error("upstream disconnected")
@@ -189,7 +189,7 @@ async def test_nim_stream_openai_5xx_exhausted_emits_user_message(
provider = NvidiaNimProvider(
config,
nim_settings=NimSettings(),
- rate_limiter=retrying_rate_limiter(),
+ admission=immediate_admission(),
)
req = make_messages_request()
diff --git a/tests/providers/test_opencode.py b/tests/providers/test_opencode.py
index e457ea45..8d99d862 100644
--- a/tests/providers/test_opencode.py
+++ b/tests/providers/test_opencode.py
@@ -3,7 +3,7 @@
from free_claude_code.core.anthropic.models import MessagesRequest
from free_claude_code.providers.base import ProviderConfig
from tests.providers.support import (
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -18,7 +18,7 @@ def test_build_request_body_omits_empty_reasoning_content() -> None:
rate_limit=1,
rate_window=1,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
request = MessagesRequest.model_validate(
{
diff --git a/tests/providers/test_provider_admission.py b/tests/providers/test_provider_admission.py
new file mode 100644
index 00000000..7c24e63f
--- /dev/null
+++ b/tests/providers/test_provider_admission.py
@@ -0,0 +1,743 @@
+"""Provider-owned admission and coordinated recovery contracts."""
+
+import asyncio
+import time
+from collections.abc import Callable
+from datetime import UTC, datetime, timedelta
+from email.utils import format_datetime
+from unittest.mock import patch
+
+import httpx
+import pytest
+
+from free_claude_code.core.failures import ExecutionFailure, FailureKind
+from free_claude_code.providers.admission import (
+ UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS,
+ ProviderAdmissionController,
+ ProviderRetrySession,
+ _retry_after_seconds,
+)
+from free_claude_code.providers.failure_policy import ProviderRecoveryExhausted
+from free_claude_code.providers.stream_recovery import TruncatedProviderStreamError
+
+
+def _controller(
+ *,
+ provider_name: str = "TEST",
+ rate_limit: int = 1_000_000,
+ rate_window: float = 1.0,
+ max_concurrency: int = 1_000,
+ max_attempts: int = UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS,
+ base_delay: float = 0.0,
+ max_delay: float = 0.0,
+) -> ProviderAdmissionController:
+ return ProviderAdmissionController(
+ provider_name=provider_name,
+ rate_limit=rate_limit,
+ rate_window=rate_window,
+ max_concurrency=max_concurrency,
+ max_attempts=max_attempts,
+ base_delay=base_delay,
+ max_delay=max_delay,
+ jitter=0.0,
+ )
+
+
+def _status_error(
+ status: int,
+ *,
+ retry_after: str | None = None,
+) -> httpx.HTTPStatusError:
+ request = httpx.Request("POST", "https://provider.test/chat/completions")
+ headers = {"retry-after": retry_after} if retry_after is not None else None
+ response = httpx.Response(status, request=request, headers=headers)
+ return httpx.HTTPStatusError(
+ f"upstream returned {status}",
+ request=request,
+ response=response,
+ )
+
+
+ ("factory", "message"),
+ [
+ (
+ lambda: ProviderAdmissionController(provider_name="TEST", rate_limit=0),
+ "rate_limit",
+ ),
+ (
+ lambda: ProviderAdmissionController(provider_name="TEST", rate_window=0),
+ "rate_window",
+ ),
+ (
+ lambda: ProviderAdmissionController(
+ provider_name="TEST", max_concurrency=0
+ ),
+ "max_concurrency",
+ ),
+ (
+ lambda: ProviderAdmissionController(provider_name="TEST", max_attempts=0),
+ "max_attempts",
+ ),
+ (
+ lambda: ProviderAdmissionController(provider_name="TEST", base_delay=-1),
+ "base_delay",
+ ),
+ (
+ lambda: ProviderAdmissionController(
+ provider_name="TEST", base_delay=2, max_delay=1
+ ),
+ "max_delay",
+ ),
+ ],
+)
+def test_admission_rejects_invalid_configuration(
+ factory: Callable[[], ProviderAdmissionController], message: str
+) -> None:
+ with pytest.raises(ValueError, match=message):
+ factory()
+
+
+def test_retry_session_exposes_one_bounded_execution_budget() -> None:
+ session = ProviderRetrySession(max_attempts=2)
+
+ assert session.max_attempts == 2
+ assert session.attempts_started == 0
+ assert session.attempts_remaining == 2
+ assert session.can_attempt
+ assert session._claim_attempt() == 1
+ assert session._claim_attempt() == 2
+ assert not session.can_attempt
+ assert session.attempts_remaining == 0
+ with pytest.raises(RuntimeError, match="exhausted"):
+ session._claim_attempt()
+
+
+async def test_proactive_rate_limit_is_a_strict_rolling_window() -> None:
+ controller = _controller(rate_limit=1, rate_window=0.04)
+
+ first = await controller.open_attempt(controller.new_retry_session())
+ await first.succeeded()
+ await first.aclose()
+
+ started = time.monotonic()
+ second = await controller.open_attempt(controller.new_retry_session())
+ elapsed = time.monotonic() - started
+ await second.succeeded()
+ await second.aclose()
+
+ assert elapsed >= 0.03
+
+
+async def test_concurrency_bulkhead_limits_active_attempts() -> None:
+ controller = _controller(max_concurrency=2)
+ first = await controller.open_attempt(controller.new_retry_session())
+ second = await controller.open_attempt(controller.new_retry_session())
+
+ third_task = asyncio.create_task(
+ controller.open_attempt(controller.new_retry_session())
+ )
+ await asyncio.sleep(0)
+ assert not third_task.done()
+
+ await first.succeeded()
+ await first.aclose()
+ third = await asyncio.wait_for(third_task, timeout=1)
+
+ await second.succeeded()
+ await second.aclose()
+ await third.succeeded()
+ await third.aclose()
+
+
+async def test_attempt_cancellation_releases_concurrency() -> None:
+ controller = _controller(max_concurrency=1)
+ entered = asyncio.Event()
+
+ async def never_finishes() -> None:
+ entered.set()
+ await asyncio.Event().wait()
+
+ task = asyncio.create_task(controller.run_with_retry(never_finishes))
+ await entered.wait()
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+
+ assert (
+ await asyncio.wait_for(
+ controller.run_with_retry(lambda: asyncio.sleep(0, result="ok")),
+ timeout=1,
+ )
+ == "ok"
+ )
+
+
+async def test_run_with_retry_uses_one_five_attempt_budget() -> None:
+ controller = _controller()
+ attempts = 0
+ error = _status_error(503)
+
+ async def fail() -> None:
+ nonlocal attempts
+ attempts += 1
+ raise error
+
+ with pytest.raises(httpx.HTTPStatusError) as exc_info:
+ await controller.run_with_retry(fail)
+
+ assert attempts == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
+ assert exc_info.value is error
+
+
+async def test_run_with_retry_succeeds_without_multiplying_attempts() -> None:
+ controller = _controller(max_concurrency=1)
+ attempts = 0
+
+ async def recover() -> str:
+ nonlocal attempts
+ attempts += 1
+ if attempts < 3:
+ raise _status_error(429)
+ return "recovered"
+
+ assert (
+ await asyncio.wait_for(controller.run_with_retry(recover), timeout=1)
+ == "recovered"
+ )
+ assert attempts == 3
+
+
+async def test_recovery_traces_keep_the_logical_request_id() -> None:
+ controller = _controller(max_attempts=2)
+ attempts = 0
+
+ async def recover() -> str:
+ nonlocal attempts
+ attempts += 1
+ if attempts == 1:
+ raise _status_error(503)
+ return "recovered"
+
+ with patch("free_claude_code.providers.admission.trace_event") as trace:
+ result = await controller.run_with_retry(recover, request_id="req_trace")
+
+ assert result == "recovered"
+ recovery_rows = [
+ call.kwargs
+ for call in trace.call_args_list
+ if call.kwargs.get("event", "").startswith("provider.recovery")
+ or call.kwargs.get("event") == "provider.retry.scheduled"
+ ]
+ assert {row["event"] for row in recovery_rows} == {
+ "provider.recovery.opened",
+ "provider.recovery.probe",
+ "provider.recovery.closed",
+ "provider.retry.scheduled",
+ }
+ assert all(row["request_id"] == "req_trace" for row in recovery_rows)
+ assert recovery_rows[-1]["outcome"] == "success"
+
+
+async def test_direct_exhaustion_trace_keeps_the_logical_request_id() -> None:
+ controller = _controller(max_attempts=1)
+ error = _status_error(503)
+
+ async def fail() -> None:
+ raise error
+
+ with (
+ patch("free_claude_code.providers.admission.trace_event") as trace,
+ pytest.raises(httpx.HTTPStatusError),
+ ):
+ await controller.run_with_retry(fail, request_id="req_terminal")
+
+ rows = [call.kwargs for call in trace.call_args_list]
+ assert {row["event"] for row in rows} == {
+ "provider.recovery.opened",
+ "provider.retry.exhausted",
+ }
+ assert all(row["request_id"] == "req_terminal" for row in rows)
+
+
+async def test_non_retryable_error_is_attempted_once() -> None:
+ controller = _controller()
+ attempts = 0
+
+ async def reject() -> None:
+ nonlocal attempts
+ attempts += 1
+ raise _status_error(400)
+
+ with pytest.raises(httpx.HTTPStatusError):
+ await controller.run_with_retry(reject)
+
+ assert attempts == 1
+
+
+async def test_pre_response_protocol_failure_opens_coordinated_recovery() -> None:
+ controller = _controller()
+ session = controller.new_retry_session()
+ attempt = await controller.open_attempt(session)
+
+ assert await attempt.retry(
+ TruncatedProviderStreamError("stream ended before its first chunk")
+ )
+ assert attempt.failure_retryable is True
+ await attempt.aclose()
+
+ probe = await controller.open_attempt(session)
+ await probe.succeeded()
+ await probe.aclose()
+
+
+async def test_provider_override_can_classify_retryable_semantics() -> None:
+ controller = _controller()
+ attempts = 0
+
+ async def degraded() -> str:
+ nonlocal attempts
+ attempts += 1
+ if attempts == 1:
+ raise _status_error(400)
+ return "healthy"
+
+ def classify(error: Exception) -> ExecutionFailure | None:
+ del error
+ return ExecutionFailure(
+ kind=FailureKind.OVERLOADED,
+ status_code=529,
+ message="temporarily degraded",
+ retryable=True,
+ )
+
+ assert (
+ await controller.run_with_retry(
+ degraded,
+ provider_failure_override=classify,
+ )
+ == "healthy"
+ )
+ assert attempts == 2
+
+
+async def test_one_leader_backs_off_while_followers_coalesce() -> None:
+ controller = _controller(base_delay=2.0, max_delay=60.0)
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ follower = await controller.open_attempt(follower_session)
+ error = _status_error(429)
+
+ assert await leader.retry(error)
+ assert await follower.retry(error)
+ await leader.aclose()
+ await follower.aclose()
+
+ real_sleep = asyncio.sleep
+ sleep_started = asyncio.Event()
+ release_sleep = asyncio.Event()
+ delays: list[float] = []
+
+ async def controlled_sleep(delay: float) -> None:
+ delays.append(delay)
+ sleep_started.set()
+ await release_sleep.wait()
+
+ with patch(
+ "free_claude_code.providers.admission.asyncio.sleep",
+ side_effect=controlled_sleep,
+ ):
+ leader_probe_task = asyncio.create_task(controller.open_attempt(leader_session))
+ await sleep_started.wait()
+ follower_attempt_task = asyncio.create_task(
+ controller.open_attempt(follower_session)
+ )
+ await real_sleep(0)
+
+ assert len(delays) == 1
+ assert 1.9 <= delays[0] <= 2.0
+ assert not follower_attempt_task.done()
+
+ release_sleep.set()
+ leader_probe = await asyncio.wait_for(leader_probe_task, timeout=1)
+ await real_sleep(0)
+ assert not follower_attempt_task.done()
+
+ await leader_probe.succeeded()
+ await leader_probe.aclose()
+ resumed_follower = await asyncio.wait_for(follower_attempt_task, timeout=1)
+
+ await resumed_follower.succeeded()
+ await resumed_follower.aclose()
+
+
+async def test_cancelled_follower_leaves_recovery_episode() -> None:
+ controller = _controller(base_delay=1.0, max_delay=1.0)
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ follower = await controller.open_attempt(follower_session)
+
+ assert await leader.retry(_status_error(503))
+ assert await follower.retry(_status_error(503))
+ await leader.aclose()
+ await follower.aclose()
+
+ follower_wait = asyncio.create_task(controller.open_attempt(follower_session))
+ await asyncio.sleep(0)
+ episode = controller._episode
+ assert episode is not None
+ assert follower_session in episode.waiters
+
+ follower_wait.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await follower_wait
+
+ assert follower_session not in episode.waiters
+
+
+async def test_cancelled_backoff_leader_transfers_to_a_waiter() -> None:
+ controller = _controller(base_delay=2.0, max_delay=60.0)
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ follower = await controller.open_attempt(follower_session)
+
+ assert await leader.retry(_status_error(503))
+ assert await follower.retry(_status_error(503))
+ await leader.aclose()
+ await follower.aclose()
+
+ real_sleep = asyncio.sleep
+ first_sleep_started = asyncio.Event()
+ second_sleep_started = asyncio.Event()
+ release_second_sleep = asyncio.Event()
+ sleep_calls = 0
+
+ async def controlled_sleep(delay: float) -> None:
+ nonlocal sleep_calls
+ assert 1.9 <= delay <= 2.0
+ sleep_calls += 1
+ if sleep_calls == 1:
+ first_sleep_started.set()
+ await asyncio.Event().wait()
+ second_sleep_started.set()
+ await release_second_sleep.wait()
+
+ with patch(
+ "free_claude_code.providers.admission.asyncio.sleep",
+ side_effect=controlled_sleep,
+ ):
+ leader_task = asyncio.create_task(controller.open_attempt(leader_session))
+ await first_sleep_started.wait()
+ follower_task = asyncio.create_task(controller.open_attempt(follower_session))
+ await real_sleep(0)
+
+ leader_task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await leader_task
+
+ await second_sleep_started.wait()
+ release_second_sleep.set()
+ probe = await asyncio.wait_for(follower_task, timeout=1)
+
+ await probe.succeeded()
+ await probe.aclose()
+
+
+async def test_stale_in_flight_success_cannot_close_recovery_episode() -> None:
+ controller = _controller()
+ leader_session = controller.new_retry_session()
+ stale_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ stale = await controller.open_attempt(stale_session)
+
+ assert await leader.retry(_status_error(503))
+ await leader.aclose()
+ await stale.succeeded()
+ await stale.aclose()
+
+ waiting = asyncio.create_task(
+ controller.open_attempt(controller.new_retry_session())
+ )
+ await asyncio.sleep(0)
+ assert not waiting.done()
+
+ probe = await controller.open_attempt(leader_session)
+ await probe.succeeded()
+ await probe.aclose()
+ resumed = await asyncio.wait_for(waiting, timeout=1)
+ await resumed.succeeded()
+ await resumed.aclose()
+
+
+async def test_abandoned_probe_transfers_leadership() -> None:
+ controller = _controller()
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ follower = await controller.open_attempt(follower_session)
+
+ assert await leader.retry(_status_error(503))
+ assert await follower.retry(_status_error(503))
+ await leader.aclose()
+ await follower.aclose()
+
+ abandoned_probe = await controller.open_attempt(leader_session)
+ follower_probe_task = asyncio.create_task(controller.open_attempt(follower_session))
+ await asyncio.sleep(0)
+ assert not follower_probe_task.done()
+
+ await abandoned_probe.aclose()
+ follower_probe = await asyncio.wait_for(follower_probe_task, timeout=1)
+ await follower_probe.succeeded()
+ await follower_probe.aclose()
+
+
+async def test_cancelled_probe_resolution_remains_abandonable() -> None:
+ controller = _controller(max_concurrency=1)
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+
+ assert await leader.retry(_status_error(503))
+ await leader.aclose()
+
+ probe = await controller.open_attempt(leader_session)
+ await controller._condition.acquire()
+ resolution = asyncio.create_task(probe.succeeded())
+ await asyncio.sleep(0)
+ resolution.cancel()
+ controller._condition.release()
+ with pytest.raises(asyncio.CancelledError):
+ await resolution
+
+ follower = asyncio.create_task(controller.open_attempt(follower_session))
+ await probe.aclose()
+ replacement = await asyncio.wait_for(follower, timeout=1)
+ await replacement.succeeded()
+ await replacement.aclose()
+
+
+async def test_cancelled_probe_close_releases_slot_and_transfers_leadership() -> None:
+ controller = _controller(max_concurrency=1)
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+
+ assert await leader.retry(_status_error(503))
+ await leader.aclose()
+
+ probe = await controller.open_attempt(leader_session)
+ await controller._condition.acquire()
+ close = asyncio.create_task(probe.aclose())
+ await asyncio.sleep(0)
+ close.cancel()
+ controller._condition.release()
+ with pytest.raises(asyncio.CancelledError):
+ await close
+
+ replacement = await asyncio.wait_for(
+ controller.open_attempt(follower_session),
+ timeout=1,
+ )
+ await replacement.succeeded()
+ await replacement.aclose()
+
+
+async def test_non_retryable_probe_closes_episode_and_only_rejects_leader() -> None:
+ controller = _controller()
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ follower = await controller.open_attempt(follower_session)
+
+ assert await leader.retry(_status_error(503))
+ assert await follower.retry(_status_error(503))
+ await leader.aclose()
+ await follower.aclose()
+
+ probe = await controller.open_attempt(leader_session)
+ follower_task = asyncio.create_task(controller.open_attempt(follower_session))
+ await asyncio.sleep(0)
+ assert not follower_task.done()
+
+ assert not await probe.retry(_status_error(400))
+ assert probe.failure_retryable is False
+ await probe.aclose()
+ resumed = await asyncio.wait_for(follower_task, timeout=1)
+ await resumed.succeeded()
+ await resumed.aclose()
+
+
+async def test_probe_exhaustion_fails_waiters_and_opens_after_cooldown() -> None:
+ controller = _controller(
+ max_attempts=2,
+ base_delay=0.02,
+ max_delay=0.02,
+ )
+ leader_session = controller.new_retry_session()
+ follower_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ follower = await controller.open_attempt(follower_session)
+ error = _status_error(429)
+
+ assert await leader.retry(error)
+ assert await follower.retry(error)
+ await leader.aclose()
+ await follower.aclose()
+
+ follower_wait = asyncio.create_task(controller.open_attempt(follower_session))
+ probe = await controller.open_attempt(leader_session)
+ assert not await probe.retry(error)
+ await probe.aclose()
+
+ with pytest.raises(ProviderRecoveryExhausted) as waiter_error:
+ await follower_wait
+ assert waiter_error.value.last_error is error
+
+ with pytest.raises(ProviderRecoveryExhausted):
+ await controller.open_attempt(controller.new_retry_session())
+
+ await asyncio.sleep(0.03)
+ recovery = await controller.open_attempt(controller.new_retry_session())
+ await recovery.succeeded()
+ await recovery.aclose()
+
+
+async def test_exhausted_waiter_cannot_join_a_later_recovery_generation() -> None:
+ controller = _controller(
+ max_attempts=2,
+ base_delay=0.01,
+ max_delay=0.01,
+ )
+ leader_session = controller.new_retry_session()
+ delayed_waiter_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ delayed_waiter = await controller.open_attempt(delayed_waiter_session)
+ error = _status_error(503)
+
+ assert await leader.retry(error)
+ assert await delayed_waiter.retry(error)
+ await leader.aclose()
+ await delayed_waiter.aclose()
+
+ probe = await controller.open_attempt(leader_session)
+ assert not await probe.retry(error)
+ await probe.aclose()
+
+ await asyncio.sleep(0.02)
+ later = await controller.open_attempt(controller.new_retry_session())
+ await later.succeeded()
+ await later.aclose()
+
+ with pytest.raises(ProviderRecoveryExhausted) as exc_info:
+ await controller.open_attempt(delayed_waiter_session)
+ assert exc_info.value.last_error is error
+
+
+async def test_late_in_flight_failure_keeps_exhausted_generation_outcome() -> None:
+ controller = _controller(
+ max_attempts=2,
+ base_delay=0.01,
+ max_delay=0.01,
+ )
+ leader_session = controller.new_retry_session()
+ stale_session = controller.new_retry_session()
+ leader = await controller.open_attempt(leader_session)
+ stale = await controller.open_attempt(stale_session)
+ error = _status_error(503)
+
+ assert await leader.retry(error)
+ await leader.aclose()
+ probe = await controller.open_attempt(leader_session)
+ assert not await probe.retry(error)
+ await probe.aclose()
+
+ assert await stale.retry(_status_error(503))
+ await stale.aclose()
+ await asyncio.sleep(0.02)
+
+ with pytest.raises(ProviderRecoveryExhausted) as exc_info:
+ await controller.open_attempt(stale_session)
+ assert exc_info.value.last_error is error
+
+
+async def test_retry_after_is_a_minimum_backoff() -> None:
+ controller = _controller(max_attempts=2)
+ attempts = 0
+
+ async def recover() -> str:
+ nonlocal attempts
+ attempts += 1
+ if attempts == 1:
+ raise _status_error(429, retry_after="7")
+ return "ok"
+
+ with patch(
+ "free_claude_code.providers.admission.asyncio.sleep",
+ return_value=None,
+ ) as sleep:
+ assert await controller.run_with_retry(recover) == "ok"
+
+ sleep.assert_awaited_once()
+ await_args = sleep.await_args
+ assert await_args is not None
+ assert 6.9 <= await_args.args[0] <= 7.0
+
+
+def test_retry_after_accepts_http_date_and_rejects_invalid_values() -> None:
+ future = format_datetime(datetime.now(UTC) + timedelta(seconds=10), usegmt=True)
+
+ parsed = _retry_after_seconds(_status_error(429, retry_after=future))
+
+ assert parsed is not None
+ assert 8 <= parsed <= 10
+ assert _retry_after_seconds(_status_error(429, retry_after="invalid")) is None
+ assert _retry_after_seconds(_status_error(429, retry_after="nan")) is None
+ assert _retry_after_seconds(_status_error(429, retry_after="inf")) is None
+
+
+async def test_provider_controllers_do_not_share_recovery_state() -> None:
+ first = _controller(provider_name="FIRST")
+ second = _controller(provider_name="SECOND")
+ first_session = first.new_retry_session()
+ first_attempt = await first.open_attempt(first_session)
+
+ assert await first_attempt.retry(_status_error(503))
+ await first_attempt.aclose()
+
+ independent = await asyncio.wait_for(
+ second.open_attempt(second.new_retry_session()),
+ timeout=1,
+ )
+ await independent.succeeded()
+ await independent.aclose()
+
+ probe = await first.open_attempt(first_session)
+ await probe.succeeded()
+ await probe.aclose()
diff --git a/tests/providers/test_provider_rate_limit.py b/tests/providers/test_provider_rate_limit.py
deleted file mode 100644
index 61c6cc46..00000000
--- a/tests/providers/test_provider_rate_limit.py
+++ /dev/null
@@ -1,753 +0,0 @@
-import asyncio
-import time
-from unittest.mock import AsyncMock, patch
-
-import httpx
-import openai
-import pytest
-from httpx import Request
-
-import free_claude_code.providers.rate_limit as rate_limit_module
-from free_claude_code.providers.failure_policy import (
- retryable_upstream_status,
- retryable_upstream_transport_error,
-)
-from free_claude_code.providers.rate_limit import (
- DEFAULT_UPSTREAM_MAX_RETRIES,
- UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS,
- ProviderRateLimiter,
-)
-
-
-def test_upstream_transient_retry_total_attempts_is_five() -> None:
- assert UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS == 5
- assert DEFAULT_UPSTREAM_MAX_RETRIES == 4
-
-
-def _statusless_api_error(message: str, body: object | None) -> openai.APIError:
- return openai.APIError(message, request=Request("POST", "http://x"), body=body)
-
-
-def test_retryable_upstream_status_reads_statusless_api_error_body_status() -> None:
- exc = _statusless_api_error(
- "stream embedded error",
- {"error": {"message": "internal failure", "code": 500}},
- )
-
- assert retryable_upstream_status(exc) == 500
-
-
-def test_retryable_upstream_status_reads_statusless_resource_exhausted_text() -> None:
- exc = _statusless_api_error(
- "ResourceExhausted: limit reached while generating response",
- {"error": {"message": "ResourceExhausted: limit reached"}},
- )
-
- assert retryable_upstream_status(exc) == 503
-
-
-def test_retryable_upstream_transport_error_classifies_connection_failures() -> None:
- request = Request("POST", "http://x")
- assert retryable_upstream_transport_error(
- openai.APIConnectionError(request=request)
- )
- assert retryable_upstream_transport_error(httpx.ConnectError("connect failed"))
- assert retryable_upstream_transport_error(httpx.ReadError("read failed"))
- assert retryable_upstream_transport_error(httpx.WriteError("write failed"))
- assert retryable_upstream_transport_error(
- httpx.RemoteProtocolError("server disconnected")
- )
- assert retryable_upstream_transport_error(TimeoutError("timed out"))
-
-
-def test_retryable_upstream_transport_error_rejects_request_errors() -> None:
- from httpx import Response
-
- request = Request("POST", "http://x")
- assert not retryable_upstream_transport_error(
- openai.BadRequestError(
- "bad request",
- response=Response(400, request=request),
- body={},
- )
- )
- assert not retryable_upstream_transport_error(
- httpx.HTTPStatusError(
- "bad request",
- request=request,
- response=Response(400, request=request),
- )
- )
-
-
-class TestProviderRateLimiter:
- """Tests for providers.rate_limit.ProviderRateLimiter."""
-
- @pytest.mark.asyncio
- async def test_proactive_throttling(self):
- """
- Test proactive throttling.
- Logic ported from verify_provider_limiter.py
- """
- # Re-init with tight limits: 1 request per 0.25 second
- limiter = ProviderRateLimiter(rate_limit=1, rate_window=0.25)
-
- start_time = time.monotonic()
-
- async def call_limiter():
- await limiter.wait_if_blocked()
- return time.monotonic()
-
- # 5 requests.
- # R0 -> 0s
- # R1 -> 0.25s
- # R2 -> 0.50s
- # R3 -> 0.75s
- # R4 -> 1.00s
- results = [await call_limiter() for _ in range(5)]
-
- total_time = time.monotonic() - start_time
-
- assert len(results) == 5
- # Should take at least ~1.0s
- assert total_time >= 0.9, f"Throttling failed, took too fast: {total_time:.2f}s"
-
- @pytest.mark.asyncio
- async def test_reactive_blocking(self):
- """
- Test reactive blocking when a deadline is extended.
- Logic ported from verify_provider_limiter.py
- """
- limiter = ProviderRateLimiter()
-
- start_time = time.monotonic()
-
- # Manually block for 1.5s
- block_time = 1.5
- limiter.extend_reactive_block(block_time)
-
- assert limiter.is_blocked()
-
- async def call_limiter():
- return await limiter.wait_if_blocked()
-
- # Run 2 calls concurrently
- # They should both wait for the block time
- results = await asyncio.gather(call_limiter(), call_limiter())
-
- total_time = time.monotonic() - start_time
-
- # Both should report having waited reactively
- assert all(results) is True
- assert total_time >= block_time - 0.1, (
- f"Reactive block failed, took {total_time:.2f}s"
- )
-
- @pytest.mark.asyncio
- async def test_reactive_block_at_proactive_commit_retries_without_reserving(
- self,
- monkeypatch: pytest.MonkeyPatch,
- ) -> None:
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- proactive_attempts = 0
- proactive_reservations = 0
- reactive_checks = 0
-
- async def acquire_if_allowed(_allowed) -> bool:
- nonlocal proactive_attempts, proactive_reservations
- proactive_attempts += 1
- if proactive_attempts == 1:
- return False
- proactive_reservations += 1
- return True
-
- async def wait_reactively() -> bool:
- nonlocal reactive_checks
- reactive_checks += 1
- return reactive_checks == 2
-
- monkeypatch.setattr(
- limiter._proactive_limiter,
- "acquire_if",
- acquire_if_allowed,
- )
- monkeypatch.setattr(
- limiter,
- "_wait_for_reactive_block",
- wait_reactively,
- )
-
- assert await limiter.wait_if_blocked() is True
- assert proactive_attempts == 2
- assert proactive_reservations == 1
- assert reactive_checks == 2
-
- @pytest.mark.asyncio
- async def test_reactive_wait_rechecks_an_extended_deadline(
- self,
- monkeypatch: pytest.MonkeyPatch,
- ) -> None:
- now = 0.0
- sleep_delays: list[float] = []
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- async def advance_time(delay: float) -> None:
- nonlocal now
- sleep_delays.append(delay)
- if len(sleep_delays) == 1:
- now = 5.0
- limiter.extend_reactive_block(10.0)
- now = 10.0
- return
- now += delay
-
- monkeypatch.setattr(rate_limit_module.time, "monotonic", lambda: now)
- monkeypatch.setattr(rate_limit_module.asyncio, "sleep", advance_time)
- monkeypatch.setattr(
- limiter._proactive_limiter,
- "acquire_if",
- AsyncMock(return_value=True),
- )
- limiter.extend_reactive_block(10.0)
-
- assert await limiter.wait_if_blocked() is True
- assert sleep_delays == [10.0, 5.0]
- assert limiter.is_blocked() is False
-
- @pytest.mark.asyncio
- async def test_reactive_wait_propagates_cancellation(
- self,
- monkeypatch: pytest.MonkeyPatch,
- ) -> None:
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- sleep_started = asyncio.Event()
-
- async def wait_forever(_delay: float) -> None:
- sleep_started.set()
- await asyncio.Event().wait()
-
- monkeypatch.setattr(rate_limit_module.asyncio, "sleep", wait_forever)
- limiter.extend_reactive_block(60.0)
- waiter = asyncio.create_task(limiter.wait_if_blocked())
- await sleep_started.wait()
-
- waiter.cancel()
-
- with pytest.raises(asyncio.CancelledError):
- await waiter
- assert limiter.is_blocked() is True
-
- @pytest.mark.asyncio
- async def test_proactive_wait_propagates_cancellation(
- self,
- monkeypatch: pytest.MonkeyPatch,
- ) -> None:
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- acquire_started = asyncio.Event()
-
- async def wait_forever(_allowed) -> bool:
- acquire_started.set()
- await asyncio.Event().wait()
- return True
-
- monkeypatch.setattr(limiter._proactive_limiter, "acquire_if", wait_forever)
- waiter = asyncio.create_task(limiter.wait_if_blocked())
- await acquire_started.wait()
-
- waiter.cancel()
-
- with pytest.raises(asyncio.CancelledError):
- await waiter
-
- def test_shorter_reactive_backoff_does_not_shorten_existing_deadline(
- self,
- monkeypatch: pytest.MonkeyPatch,
- ) -> None:
- now = 100.0
- monkeypatch.setattr(rate_limit_module.time, "monotonic", lambda: now)
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- limiter.extend_reactive_block(10.0)
- now = 102.0
- limiter.extend_reactive_block(1.0)
-
- assert limiter.remaining_wait() == 8.0
-
- @pytest.mark.parametrize("seconds", [0.0, -1.0])
- def test_reactive_backoff_duration_must_be_positive(self, seconds: float) -> None:
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- with pytest.raises(ValueError, match="reactive block duration must be > 0"):
- limiter.extend_reactive_block(seconds)
-
- @pytest.mark.asyncio
- async def test_remaining_wait_when_not_blocked(self):
- """remaining_wait() should return 0 when not blocked."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- assert limiter.remaining_wait() == 0
-
- @pytest.mark.asyncio
- async def test_remaining_wait_decreases(self):
- """remaining_wait() should decrease over time."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- limiter.extend_reactive_block(2.0)
-
- wait1 = limiter.remaining_wait()
- assert wait1 > 1.5
-
- await asyncio.sleep(0.5)
- wait2 = limiter.remaining_wait()
- assert wait2 < wait1
-
- @pytest.mark.asyncio
- async def test_is_blocked_false_initially(self):
- """is_blocked() should be False for a fresh limiter."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- assert limiter.is_blocked() is False
-
- @pytest.mark.asyncio
- async def test_high_rate_limit_no_throttling(self):
- """Very high rate limit should not cause throttling."""
- limiter = ProviderRateLimiter(rate_limit=10000, rate_window=60)
-
- start = time.monotonic()
- for _ in range(20):
- await limiter.wait_if_blocked()
- duration = time.monotonic() - start
-
- # 20 requests with 10000 limit should be near-instant
- assert duration < 1.0, f"High rate limit caused throttling: {duration:.2f}s"
-
- @pytest.mark.asyncio
- async def test_instances_are_independent(self):
- """Each provider limiter owns independent reactive state."""
- limiter1 = ProviderRateLimiter(rate_limit=10, rate_window=1)
- limiter2 = ProviderRateLimiter(rate_limit=10, rate_window=1)
-
- limiter1.extend_reactive_block(1)
-
- assert limiter1 is not limiter2
- assert limiter1.is_blocked() is True
- assert limiter2.is_blocked() is False
-
- @pytest.mark.asyncio
- async def test_wait_if_blocked_returns_false_when_not_blocked(self):
- """wait_if_blocked should return False when not reactively blocked."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
- result = await limiter.wait_if_blocked()
- assert result is False
-
- @pytest.mark.asyncio
- async def test_proactive_strict_rolling_window(self):
- """
- Proactive limiter should enforce a strict rolling window:
- for any i, t[i+rate_limit] - t[i] >= rate_window (within tolerance).
- """
- rate_limit = 2
- rate_window = 0.5
- limiter = ProviderRateLimiter(rate_limit=rate_limit, rate_window=rate_window)
-
- acquired: list[float] = []
-
- async def acquire():
- await limiter.wait_if_blocked()
- acquired.append(time.monotonic())
-
- # Trigger concurrency; without strict rolling windows, this can burst.
- await asyncio.gather(*(acquire() for _ in range(5)))
-
- acquired.sort()
- assert len(acquired) == 5
-
- tolerance = 0.05
- for i in range(len(acquired) - rate_limit):
- assert acquired[i + rate_limit] - acquired[i] >= rate_window - tolerance, (
- f"Rolling window violated at i={i}: "
- f"dt={acquired[i + rate_limit] - acquired[i]:.3f}s"
- )
-
- @pytest.mark.asyncio
- async def test_init_rate_limit_zero_raises(self):
- """rate_limit <= 0 raises ValueError."""
- with pytest.raises(ValueError, match="rate_limit must be > 0"):
- ProviderRateLimiter(rate_limit=0, rate_window=60)
-
- @pytest.mark.asyncio
- async def test_init_rate_window_zero_raises(self):
- """rate_window <= 0 raises ValueError."""
- with pytest.raises(ValueError, match="rate_window must be > 0"):
- ProviderRateLimiter(rate_limit=10, rate_window=0)
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_exhaust_retries_raises(self):
- """When all 429 retries exhausted, last exception is raised."""
- import openai
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- def make_429():
- return openai.RateLimitError(
- "rate limited",
- response=Response(429, request=Request("POST", "http://x")),
- body={},
- )
-
- async def fail():
- raise make_429()
-
- with pytest.raises(openai.RateLimitError):
- await limiter.execute_with_retry(
- fail, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_retry(self):
- """429 then success returns result."""
- import openai
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- def make_429():
- return openai.RateLimitError(
- "rate limited",
- response=Response(429, request=Request("POST", "http://x")),
- body={},
- )
-
- call_count = 0
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- raise make_429()
- return "ok"
-
- result = await limiter.execute_with_retry(
- fail_then_ok, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
- assert result == "ok"
- assert call_count == 2
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_httpx_429(self):
- """HTTP 429 as httpx.HTTPStatusError then success returns result."""
- import httpx
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- r = Response(429, request=Request("POST", "http://x"), text="slow")
- raise httpx.HTTPStatusError(
- "Too Many Requests", request=r.request, response=r
- )
- return "ok"
-
- result = await limiter.execute_with_retry(
- fail_then_ok, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
- assert result == "ok"
- assert call_count == 2
-
- @pytest.mark.parametrize("status_code", [500, 502, 503, 504])
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_openai_internal_server_error_5xx(
- self, status_code
- ):
- """5xx as openai.InternalServerError then success."""
- import openai
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- def make_upstream_error():
- return openai.InternalServerError(
- "unavailable",
- response=Response(status_code, request=Request("POST", "http://x")),
- body={},
- )
-
- call_count = 0
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- raise make_upstream_error()
- return "ok"
-
- result = await limiter.execute_with_retry(
- fail_then_ok, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
- assert result == "ok"
- assert call_count == 2
-
- @pytest.mark.parametrize("status_code", [500, 502, 503, 504])
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_httpx_5xx(self, status_code):
- """HTTP 5xx as httpx.HTTPStatusError then success."""
- import httpx
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- r = Response(
- status_code, request=Request("POST", "http://x"), text="error"
- )
- raise httpx.HTTPStatusError(
- "Server Error", request=r.request, response=r
- )
- return "ok"
-
- result = await limiter.execute_with_retry(
- fail_then_ok, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
- assert result == "ok"
- assert call_count == 2
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_statusless_transient_api_error(self):
- """Status-less SDK APIError transient markers participate in backoff retry."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- raise _statusless_api_error(
- "ResourceExhausted: limit reached while generating response",
- {"error": {"message": "ResourceExhausted: limit reached"}},
- )
- return "ok"
-
- result = await limiter.execute_with_retry(
- fail_then_ok, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
-
- assert result == "ok"
- assert call_count == 2
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_openai_connection_retry(self):
- """Pre-response OpenAI SDK connection errors retry then succeed."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
- request = Request("POST", "http://x")
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- raise openai.APIConnectionError(request=request)
- return "ok"
-
- with (
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch("asyncio.sleep", new_callable=AsyncMock) as sleep,
- ):
- result = await limiter.execute_with_retry(
- fail_then_ok,
- max_retries=2,
- base_delay=0.01,
- max_delay=0.1,
- jitter=0,
- )
-
- assert result == "ok"
- assert call_count == 2
- extend_block.assert_not_called()
- sleep.assert_awaited_once()
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_succeeds_on_httpx_transport_retry(self):
- """Pre-response HTTPX transport errors retry then succeed."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
-
- async def fail_then_ok():
- nonlocal call_count
- call_count += 1
- if call_count == 1:
- raise httpx.RemoteProtocolError("server disconnected")
- return "ok"
-
- with (
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch("asyncio.sleep", new_callable=AsyncMock) as sleep,
- ):
- result = await limiter.execute_with_retry(
- fail_then_ok,
- max_retries=2,
- base_delay=0.01,
- max_delay=0.1,
- jitter=0,
- )
-
- assert result == "ok"
- assert call_count == 2
- extend_block.assert_not_called()
- sleep.assert_awaited_once()
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_exhaust_transport_error_attempts(self):
- """Transport retries exhaust after the shared 5 total attempts."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
-
- async def always_fail():
- nonlocal call_count
- call_count += 1
- raise httpx.ConnectError("connect failed")
-
- with (
- patch.object(limiter, "extend_reactive_block") as extend_block,
- patch("asyncio.sleep", new_callable=AsyncMock) as sleep,
- pytest.raises(httpx.ConnectError),
- ):
- await limiter.execute_with_retry(
- always_fail,
- base_delay=0.01,
- max_delay=0.1,
- jitter=0,
- )
-
- assert call_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
- extend_block.assert_not_called()
- assert sleep.await_count == DEFAULT_UPSTREAM_MAX_RETRIES
-
- @pytest.mark.parametrize("status_code", [500, 502, 503, 504])
- @pytest.mark.asyncio
- async def test_execute_with_retry_exhaust_openai_5xx_raises(self, status_code):
- """When all 5xx retries exhausted (OpenAI SDK), last InternalServerError is raised."""
- import openai
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- exc = openai.InternalServerError(
- "unavailable",
- response=Response(status_code, request=Request("POST", "http://x")),
- body={},
- )
-
- async def always_fail():
- raise exc
-
- with pytest.raises(openai.InternalServerError):
- await limiter.execute_with_retry(
- always_fail, max_retries=2, base_delay=0.01, max_delay=0.1, jitter=0
- )
-
- @pytest.mark.asyncio
- async def test_execute_with_retry_httpx_400_raises_immediately(self):
- """Non-retryable 4xx is not wrapped by execute_with_retry loop."""
- import httpx
- from httpx import Request, Response
-
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60)
-
- call_count = 0
-
- async def bad_request():
- nonlocal call_count
- call_count += 1
- r = Response(400, request=Request("POST", "http://x"), text="bad request")
- raise httpx.HTTPStatusError("Bad Request", request=r.request, response=r)
-
- with pytest.raises(httpx.HTTPStatusError):
- await limiter.execute_with_retry(
- bad_request,
- max_retries=2,
- base_delay=0.01,
- max_delay=0.1,
- jitter=0,
- )
-
- assert call_count == 1
-
- @pytest.mark.asyncio
- async def test_max_concurrency_zero_raises(self):
- """max_concurrency <= 0 raises ValueError."""
- with pytest.raises(ValueError, match="max_concurrency must be > 0"):
- ProviderRateLimiter(rate_limit=10, rate_window=60, max_concurrency=0)
-
- @pytest.mark.asyncio
- async def test_concurrency_slot_limits_simultaneous_streams(self):
- """At most max_concurrency streams can hold a slot simultaneously."""
- max_concurrency = 2
- limiter = ProviderRateLimiter(
- rate_limit=100, rate_window=60, max_concurrency=max_concurrency
- )
-
- peak_concurrent = 0
- current_concurrent = 0
- lock = asyncio.Lock()
-
- async def stream_task(hold_time: float) -> None:
- nonlocal peak_concurrent, current_concurrent
- async with limiter.concurrency_slot():
- async with lock:
- current_concurrent += 1
- if current_concurrent > peak_concurrent:
- peak_concurrent = current_concurrent
- await asyncio.sleep(hold_time)
- async with lock:
- current_concurrent -= 1
-
- # Launch 5 tasks that each hold the slot; only 2 can be active at once
- await asyncio.gather(*(stream_task(0.05) for _ in range(5)))
-
- assert peak_concurrent <= max_concurrency, (
- f"Concurrency exceeded: peak={peak_concurrent}, max={max_concurrency}"
- )
-
- @pytest.mark.asyncio
- async def test_concurrency_slot_releases_on_exception(self):
- """Slot is released even when the body raises an exception."""
- limiter = ProviderRateLimiter(rate_limit=100, rate_window=60, max_concurrency=1)
- assert limiter._concurrency_sem is not None
-
- with pytest.raises(RuntimeError):
- async with limiter.concurrency_slot():
- raise RuntimeError("boom")
-
- # Semaphore value should be restored (1 available again)
- assert limiter._concurrency_sem._value == 1
-
- @pytest.mark.asyncio
- async def test_constructor_sets_max_concurrency(self):
- """Constructor applies max_concurrency to an independent limiter."""
- limiter = ProviderRateLimiter(rate_limit=10, rate_window=60, max_concurrency=3)
- assert limiter._concurrency_sem is not None
- assert limiter._concurrency_sem._value == 3
-
- @pytest.mark.asyncio
- async def test_provider_owned_instances_are_isolated(self):
- """Independent provider limiters do not share reactive block state."""
- nim = ProviderRateLimiter(rate_limit=10, rate_window=60)
- openrouter = ProviderRateLimiter(rate_limit=20, rate_window=30)
-
- assert nim is not openrouter
- nim.extend_reactive_block(1.0)
-
- assert nim.is_blocked() is True
- assert openrouter.is_blocked() is False
diff --git a/tests/providers/test_provider_runtime.py b/tests/providers/test_provider_runtime.py
index 7f19be1e..b84833fa 100644
--- a/tests/providers/test_provider_runtime.py
+++ b/tests/providers/test_provider_runtime.py
@@ -20,6 +20,7 @@ from free_claude_code.config.provider_catalog import (
VERCEL_AI_GATEWAY_DEFAULT_BASE,
ZAI_DEFAULT_BASE,
)
+from free_claude_code.providers.admission import ProviderAdmissionController
from free_claude_code.providers.cloudflare import CloudflareProvider
from free_claude_code.providers.deepseek import DeepSeekProvider
from free_claude_code.providers.gemini import GeminiProvider
@@ -32,7 +33,6 @@ from free_claude_code.providers.openai_chat import (
OPENAI_CHAT_PROFILES,
OpenAIChatProvider,
)
-from free_claude_code.providers.rate_limit import ProviderRateLimiter
from free_claude_code.providers.runtime import (
ProviderRuntime,
build_provider_config,
@@ -441,27 +441,28 @@ def test_create_provider_instantiates_each_builtin():
"sambanova": OpenAIChatProvider,
"cerebras": OpenAIChatProvider,
}
- sentinel_limiter = MagicMock(spec=ProviderRateLimiter)
+ sentinel_admission = MagicMock(spec=ProviderAdmissionController)
with (
patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"),
patch("httpx.AsyncClient"),
patch(
- "free_claude_code.providers.runtime.factory.ProviderRateLimiter",
- return_value=sentinel_limiter,
- ) as limiter_factory,
+ "free_claude_code.providers.runtime.factory.ProviderAdmissionController",
+ return_value=sentinel_admission,
+ ) as admission_factory,
):
for provider_id, provider_cls in cases.items():
provider = create_provider(provider_id, settings)
assert isinstance(provider, provider_cls)
- assert provider._rate_limiter is sentinel_limiter
- limiter_factory.assert_called_once_with(
+ assert provider._admission is sentinel_admission
+ admission_factory.assert_called_once_with(
+ provider_name=provider_id,
rate_limit=7,
rate_window=11,
max_concurrency=3,
)
- limiter_factory.reset_mock()
+ admission_factory.reset_mock()
assert set(cases) == set(PROVIDER_CATALOG)
@@ -476,7 +477,7 @@ def test_provider_runtime_caches_by_provider_id():
assert first is second
-def test_provider_runtime_provider_owns_one_limiter() -> None:
+def test_provider_runtime_provider_owns_one_admission_controller() -> None:
runtime = ProviderRuntime(_make_settings())
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
@@ -485,10 +486,10 @@ def test_provider_runtime_provider_owns_one_limiter() -> None:
assert isinstance(first, NvidiaNimProvider)
assert isinstance(second, NvidiaNimProvider)
- assert first._rate_limiter is second._rate_limiter
+ assert first._admission is second._admission
-def test_separate_provider_runtimes_never_share_limiters() -> None:
+def test_separate_provider_runtimes_never_share_admission_controllers() -> None:
first_runtime = ProviderRuntime(_make_settings())
second_runtime = ProviderRuntime(_make_settings())
@@ -499,10 +500,10 @@ def test_separate_provider_runtimes_never_share_limiters() -> None:
assert isinstance(first, NvidiaNimProvider)
assert isinstance(second, NvidiaNimProvider)
assert first is not second
- assert first._rate_limiter is not second._rate_limiter
+ assert first._admission is not second._admission
-def test_different_providers_in_one_runtime_have_independent_limiters() -> None:
+def test_different_providers_have_independent_admission_controllers() -> None:
runtime = ProviderRuntime(_make_settings())
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
@@ -511,7 +512,7 @@ def test_different_providers_in_one_runtime_have_independent_limiters() -> None:
assert isinstance(nim, NvidiaNimProvider)
assert isinstance(open_router, OpenRouterProvider)
- assert nim._rate_limiter is not open_router._rate_limiter
+ assert nim._admission is not open_router._admission
def test_unknown_provider_raises_unknown_provider_type_error():
diff --git a/tests/providers/test_provider_transport_logging.py b/tests/providers/test_provider_transport_logging.py
index d8cf830e..3e2624f3 100644
--- a/tests/providers/test_provider_transport_logging.py
+++ b/tests/providers/test_provider_transport_logging.py
@@ -1,7 +1,6 @@
"""Tests for credential-safe provider transport logging."""
import logging
-from contextlib import asynccontextmanager
from unittest.mock import AsyncMock, patch
import httpx
@@ -13,7 +12,7 @@ from free_claude_code.core.failures import ExecutionFailure
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter
+from tests.providers.support import immediate_admission
def _provider(*, verbose: bool = False) -> NvidiaNimProvider:
@@ -24,15 +23,10 @@ def _provider(*, verbose: bool = False) -> NvidiaNimProvider:
log_api_error_tracebacks=verbose,
),
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
-@asynccontextmanager
-async def _noop_slot():
- yield
-
-
@pytest.mark.asyncio
async def test_stream_failure_default_logs_exclude_exception_text(caplog) -> None:
provider = _provider()
@@ -43,7 +37,6 @@ async def test_stream_failure_default_logs_exclude_exception_text(caplog) -> Non
new_callable=AsyncMock,
side_effect=RuntimeError("SECRET_OPENAI_COMPAT"),
),
- patch.object(provider._rate_limiter, "concurrency_slot", _noop_slot),
caplog.at_level(logging.ERROR),
pytest.raises(ExecutionFailure),
):
@@ -68,7 +61,6 @@ async def test_stream_failure_default_logs_cause_types_only(caplog) -> None:
new_callable=AsyncMock,
side_effect=error,
),
- patch.object(provider._rate_limiter, "concurrency_slot", _noop_slot),
caplog.at_level(logging.ERROR),
pytest.raises(ExecutionFailure),
):
@@ -92,7 +84,6 @@ async def test_stream_failure_verbose_traceback_redacts_credentials(caplog) -> N
"api_key=SECRET_OPENAI_COMPAT useful traceback detail"
),
),
- patch.object(provider._rate_limiter, "concurrency_slot", _noop_slot),
caplog.at_level(logging.ERROR),
pytest.raises(ExecutionFailure),
):
diff --git a/tests/providers/test_sambanova.py b/tests/providers/test_sambanova.py
index ddc859d6..fbdb7c9b 100644
--- a/tests/providers/test_sambanova.py
+++ b/tests/providers/test_sambanova.py
@@ -9,7 +9,7 @@ from free_claude_code.config.provider_catalog import SAMBANOVA_DEFAULT_BASE
from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
def make_request(**overrides):
@@ -29,7 +29,7 @@ def sambanova_config():
@pytest.fixture
def sambanova_provider(sambanova_config):
return profiled_provider(
- "sambanova", sambanova_config, rate_limiter=passthrough_rate_limiter()
+ "sambanova", sambanova_config, admission=immediate_admission()
)
@@ -42,7 +42,7 @@ def test_init_uses_default_base_url_and_api_key(sambanova_config):
"free_claude_code.providers.openai_chat.provider.AsyncOpenAI"
) as mock_openai:
provider = profiled_provider(
- "sambanova", sambanova_config, rate_limiter=passthrough_rate_limiter()
+ "sambanova", sambanova_config, admission=immediate_admission()
)
assert provider._api_key == "test_sambanova_key"
@@ -55,7 +55,7 @@ def test_init_strips_trailing_slash(sambanova_config):
with patch("free_claude_code.providers.openai_chat.provider.AsyncOpenAI"):
provider = profiled_provider(
- "sambanova", config, rate_limiter=passthrough_rate_limiter()
+ "sambanova", config, admission=immediate_admission()
)
assert provider._base_url == SAMBANOVA_DEFAULT_BASE
diff --git a/tests/providers/test_stream_recovery.py b/tests/providers/test_stream_recovery.py
index f0ad6170..293540c4 100644
--- a/tests/providers/test_stream_recovery.py
+++ b/tests/providers/test_stream_recovery.py
@@ -1,15 +1,13 @@
-"""Provider-owned stream retry and holdback policy."""
+"""Provider stream commit-boundary and recovery policy."""
import httpx
import openai
from free_claude_code.providers.stream_recovery import (
- EARLY_TRANSPARENT_MAX_RETRIES,
- EARLY_TRANSPARENT_TOTAL_ATTEMPTS,
- MIDSTREAM_RECOVERY_ATTEMPTS,
RecoveryController,
RecoveryFailureAction,
RecoveryHoldbackBuffer,
+ TruncatedProviderStreamError,
is_retryable_stream_error,
)
@@ -24,16 +22,10 @@ def _statusless_openai_api_error(
)
-def test_early_transparent_retry_total_attempts_is_five() -> None:
- assert EARLY_TRANSPARENT_TOTAL_ATTEMPTS == 5
- assert EARLY_TRANSPARENT_MAX_RETRIES == 4
-
-
-def test_midstream_recovery_attempts_total_is_five() -> None:
- assert MIDSTREAM_RECOVERY_ATTEMPTS == 5
-
-
-def test_retryable_stream_error_classifies_transport_and_http_status() -> None:
+def test_retryable_stream_error_classifies_protocol_transport_and_status() -> None:
+ assert is_retryable_stream_error(
+ TruncatedProviderStreamError("missing terminal marker")
+ )
assert is_retryable_stream_error(httpx.ReadError("cut off"))
request = httpx.Request("GET", "https://example.test")
@@ -98,8 +90,8 @@ def test_retryable_stream_error_does_not_retry_bad_request_status() -> None:
)
-def test_recovery_controller_advances_early_retry_and_discards_holdback() -> None:
- controller = RecoveryController(provider_name="TEST", request_id="REQ")
+def test_early_retry_discards_uncommitted_holdback() -> None:
+ controller = RecoveryController()
assert controller.push("hidden") == []
decision = controller.advance_failure(
@@ -107,60 +99,82 @@ def test_recovery_controller_advances_early_retry_and_discards_holdback() -> Non
stream_opened=True,
generated_output=True,
complete_tool_salvageable=False,
+ attempts_remaining=2,
)
assert decision.action == RecoveryFailureAction.EARLY_RETRY
- assert decision.early_retry_attempt == 1
- assert controller.early_retries == 1
+ assert decision.retryable
+ assert decision.has_buffered
assert not controller.committed
assert not controller.has_buffered
assert controller.flush() == []
-def test_recovery_controller_retries_statusless_transient_api_error() -> None:
- controller = RecoveryController(provider_name="TEST", request_id="REQ")
+def test_early_retry_requires_remaining_execution_budget() -> None:
+ controller = RecoveryController()
+ assert controller.push("hidden") == []
decision = controller.advance_failure(
- _statusless_openai_api_error(
- "ResourceExhausted: limit reached while generating response",
- {"error": {"message": "ResourceExhausted: limit reached"}},
- ),
+ httpx.ReadError("early cutoff"),
stream_opened=True,
- generated_output=False,
+ generated_output=True,
complete_tool_salvageable=False,
+ attempts_remaining=0,
)
- assert decision.action == RecoveryFailureAction.EARLY_RETRY
+ assert decision.action == RecoveryFailureAction.FINAL_ERROR
assert decision.retryable
- assert decision.early_retry_attempt == 1
+ assert controller.has_buffered
-def test_recovery_controller_respects_early_retry_limit() -> None:
- controller = RecoveryController(provider_name="TEST", request_id=None)
-
- for attempt in range(1, EARLY_TRANSPARENT_MAX_RETRIES + 1):
- decision = controller.advance_failure(
- httpx.ReadError("cutoff"),
- stream_opened=True,
- generated_output=False,
- complete_tool_salvageable=False,
- )
- assert decision.action == RecoveryFailureAction.EARLY_RETRY
- assert decision.early_retry_attempt == attempt
+def test_last_attempt_is_reserved_for_partial_output_recovery() -> None:
+ controller = RecoveryController()
+ assert controller.push("partial") == []
decision = controller.advance_failure(
- httpx.ReadError("cutoff"),
+ httpx.ReadError("early cutoff"),
stream_opened=True,
+ generated_output=True,
+ complete_tool_salvageable=False,
+ attempts_remaining=1,
+ )
+
+ assert decision.action == RecoveryFailureAction.MIDSTREAM_RECOVERY
+ assert decision.has_buffered
+ assert controller.has_buffered
+
+
+def test_create_failure_is_owned_by_admission_not_stream_recovery() -> None:
+ decision = RecoveryController().advance_failure(
+ httpx.ConnectError("connect failed"),
+ stream_opened=False,
generated_output=False,
complete_tool_salvageable=False,
+ attempts_remaining=1,
)
assert decision.action == RecoveryFailureAction.FINAL_ERROR
- assert controller.early_retries == EARLY_TRANSPARENT_MAX_RETRIES
+ assert decision.retryable
-def test_recovery_controller_classifies_midstream_recovery_after_commit() -> None:
- controller = RecoveryController(provider_name="TEST", request_id=None)
+def test_statusless_transient_api_error_allows_early_retry() -> None:
+ decision = RecoveryController().advance_failure(
+ _statusless_openai_api_error(
+ "ResourceExhausted: limit reached while generating response",
+ {"error": {"message": "ResourceExhausted: limit reached"}},
+ ),
+ stream_opened=True,
+ generated_output=False,
+ complete_tool_salvageable=False,
+ attempts_remaining=1,
+ )
+
+ assert decision.action == RecoveryFailureAction.EARLY_RETRY
+ assert decision.retryable
+
+
+def test_committed_output_allows_midstream_recovery() -> None:
+ controller = RecoveryController()
assert controller.push("event: content_block_delta\n\n") == []
assert controller.flush() == ["event: content_block_delta\n\n"]
@@ -169,6 +183,7 @@ def test_recovery_controller_classifies_midstream_recovery_after_commit() -> Non
stream_opened=True,
generated_output=True,
complete_tool_salvageable=False,
+ attempts_remaining=1,
)
assert decision.action == RecoveryFailureAction.MIDSTREAM_RECOVERY
@@ -177,8 +192,8 @@ def test_recovery_controller_classifies_midstream_recovery_after_commit() -> Non
assert controller.flush_uncommitted(decision) == []
-def test_recovery_controller_flushes_uncommitted_midstream_decision() -> None:
- controller = RecoveryController(provider_name="TEST", request_id=None)
+def test_uncommitted_complete_tool_can_be_salvaged() -> None:
+ controller = RecoveryController()
assert controller.push("event: content_block_delta\n\n") == []
decision = controller.advance_failure(
@@ -186,40 +201,35 @@ def test_recovery_controller_flushes_uncommitted_midstream_decision() -> None:
stream_opened=True,
generated_output=True,
complete_tool_salvageable=True,
+ attempts_remaining=0,
)
assert decision.action == RecoveryFailureAction.MIDSTREAM_RECOVERY
assert not decision.committed
assert decision.has_buffered
- assert not controller.committed
- assert controller.has_buffered
-
assert controller.flush_uncommitted(decision) == ["event: content_block_delta\n\n"]
- assert not decision.committed
- assert decision.has_buffered
assert controller.committed
assert not controller.has_buffered
-def test_recovery_controller_non_retryable_error_is_final() -> None:
+def test_non_retryable_error_is_final() -> None:
request = httpx.Request("POST", "https://example.test/messages")
error = httpx.HTTPStatusError(
"bad request",
request=request,
response=httpx.Response(400, request=request),
)
- controller = RecoveryController(provider_name="TEST", request_id=None)
- decision = controller.advance_failure(
+ decision = RecoveryController().advance_failure(
error,
stream_opened=True,
generated_output=True,
complete_tool_salvageable=False,
+ attempts_remaining=2,
)
assert decision.action == RecoveryFailureAction.FINAL_ERROR
assert not decision.retryable
- assert controller.early_retries == 0
def test_holdback_buffers_until_delay_then_commits() -> None:
@@ -232,8 +242,7 @@ def test_holdback_buffers_until_delay_then_commits() -> None:
assert not holdback.committed
now[0] += 0.01
- flushed = holdback.push("event: content_block_stop\n\n")
- assert flushed == [
+ assert holdback.push("event: content_block_stop\n\n") == [
"event: content_block_start\n\n",
"event: content_block_delta\n\n",
"event: content_block_stop\n\n",
diff --git a/tests/providers/test_streaming_errors.py b/tests/providers/test_streaming_errors.py
index 7d81932b..b84236e1 100644
--- a/tests/providers/test_streaming_errors.py
+++ b/tests/providers/test_streaming_errors.py
@@ -18,6 +18,7 @@ from free_claude_code.core.anthropic.streaming import (
)
from free_claude_code.core.failures import ExecutionFailure
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
+from free_claude_code.providers.admission import UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from free_claude_code.providers.openai_chat.provider import (
@@ -28,12 +29,9 @@ from free_claude_code.providers.openai_chat.tool_calls import (
has_committed_sse_output,
iter_heuristic_tool_use_sse,
)
-from free_claude_code.providers.stream_recovery import (
- MIDSTREAM_RECOVERY_ATTEMPTS,
- TruncatedProviderStreamError,
-)
+from free_claude_code.providers.stream_recovery import TruncatedProviderStreamError
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import REASONING_OFF, passthrough_rate_limiter
+from tests.providers.support import REASONING_OFF, immediate_admission
class AsyncStreamMock:
@@ -75,7 +73,7 @@ def _make_provider():
return NvidiaNimProvider(
config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -215,6 +213,30 @@ def _assert_error_not_in_text_deltas_after_tool(
class TestStreamingExceptionHandling:
+ @pytest.mark.asyncio
+ async def test_stream_normalization_failure_closes_raw_stream(self):
+ provider = _make_provider()
+ stream = ClosableAsyncStreamMock([])
+ retry_session = provider._admission.new_retry_session()
+
+ with (
+ patch.object(
+ provider._client.chat.completions,
+ "create",
+ new_callable=AsyncMock,
+ return_value=stream,
+ ),
+ patch.object(
+ provider,
+ "_normalize_stream",
+ side_effect=ValueError("invalid stream wrapper"),
+ ),
+ pytest.raises(ValueError, match="invalid stream wrapper"),
+ ):
+ await provider._create_stream({"messages": []}, retry_session)
+
+ assert stream.closed
+
"""Tests for error paths during stream_response."""
@pytest.mark.asyncio
@@ -230,12 +252,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
side_effect=RuntimeError("API failed"),
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
error = await _collect_stream_error(provider, request)
@@ -254,12 +270,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
side_effect=httpx.ReadTimeout(""),
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
patch("asyncio.sleep", new_callable=AsyncMock),
):
error = await _collect_stream_error(
@@ -287,12 +297,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
error = await _collect_stream_error(provider, request)
@@ -316,12 +320,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events, error = await _collect_stream_and_error(provider, request)
event_text = "".join(events)
@@ -352,12 +350,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -388,12 +380,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -424,12 +410,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
event_text = "".join(events)
@@ -454,12 +434,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -486,12 +460,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -517,12 +485,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -561,12 +523,6 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request, reasoning=REASONING_OFF)
@@ -761,8 +717,8 @@ class TestStreamingExceptionHandling:
)
@pytest.mark.asyncio
- async def test_precommit_openai_holdback_retries_without_leaking_partial(self):
- """A retryable early cutoff before holdback commit is retried invisibly."""
+ async def test_precommit_retry_emits_one_unduplicated_downstream_lifecycle(self):
+ """An abandoned attempt contributes no frame to the successful replay."""
provider = _make_provider()
request = _make_request()
first_stream = AsyncStreamMock(
@@ -787,12 +743,64 @@ class TestStreamingExceptionHandling:
event_text = "".join(events)
assert mock_create.await_count == 2
assert "hidden" not in event_text
- assert "visible" in event_text
parsed = parse_sse_text(event_text)
- assert parsed[0].event == "message_start"
+ text_deltas = [
+ event.data.get("delta", {}).get("text", "")
+ for event in parsed
+ if event.event == "content_block_delta"
+ ]
+ assert text_deltas == ["visible"]
assert sum(event.event == "message_start" for event in parsed) == 1
+ assert sum(event.event == "content_block_start" for event in parsed) == 1
+ assert sum(event.event == "content_block_stop" for event in parsed) == 1
+ assert sum(event.event == "message_delta" for event in parsed) == 1
+ assert sum(event.event == "message_stop" for event in parsed) == 1
+ assert parsed[0].event == "message_start"
assert parsed[-1].event == "message_stop"
+ @pytest.mark.asyncio
+ async def test_primary_replay_and_continuation_share_five_attempts(self):
+ """Four replays plus continuation emit one unduplicated response."""
+ provider = _make_provider()
+ request = _make_request()
+ primary_streams = [
+ AsyncStreamMock([_make_chunk(content="hello")]) for _ in range(4)
+ ]
+ continuation = AsyncStreamMock(
+ [
+ _make_chunk(content="hello world"),
+ _make_chunk(finish_reason="stop"),
+ ]
+ )
+
+ with patch.object(
+ provider._client.chat.completions,
+ "create",
+ new_callable=AsyncMock,
+ side_effect=[*primary_streams, continuation],
+ ) as create:
+ events = await _collect_stream(provider, request)
+
+ assert create.await_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
+ assert all(
+ call.kwargs["messages"] == create.await_args_list[0].kwargs["messages"]
+ for call in create.await_args_list[:4]
+ )
+ assert (
+ create.await_args_list[4].kwargs["messages"]
+ != (create.await_args_list[0].kwargs["messages"])
+ )
+ parsed = parse_sse_text("".join(events))
+ text = "".join(
+ event.data.get("delta", {}).get("text", "")
+ for event in parsed
+ if event.event == "content_block_delta"
+ )
+ assert text == "hello world"
+ assert sum(event.event == "message_start" for event in parsed) == 1
+ assert sum(event.event == "message_delta" for event in parsed) == 1
+ assert sum(event.event == "message_stop" for event in parsed) == 1
+
@pytest.mark.asyncio
async def test_clean_eof_after_text_continues_with_overlap_trim(self):
"""A truncated text stream is continued and duplicate overlap is trimmed."""
@@ -824,6 +832,11 @@ class TestStreamingExceptionHandling:
]
assert text_deltas == ["hello wor", "ld"]
assert "".join(text_deltas) == "hello world"
+ assert sum(event.event == "message_start" for event in parsed) == 1
+ assert sum(event.event == "content_block_start" for event in parsed) == 1
+ assert sum(event.event == "content_block_stop" for event in parsed) == 1
+ assert sum(event.event == "message_delta" for event in parsed) == 1
+ assert sum(event.event == "message_stop" for event in parsed) == 1
assert any(
event.event == "message_delta"
and event.data.get("delta", {}).get("stop_reason") == "end_turn"
@@ -870,21 +883,28 @@ class TestStreamingExceptionHandling:
"""Recovery collectors reject truncated OpenAI-chat continuation streams."""
streams = [
ClosableAsyncStreamMock([_make_chunk(content=f"world {index}")])
- for index in range(MIDSTREAM_RECOVERY_ATTEMPTS)
+ for index in range(UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS)
]
- create_stream = AsyncMock(side_effect=[(stream, {}) for stream in streams])
provider = _make_provider()
runner = _make_stream_runner(provider)
+ retry_session = provider._admission.new_retry_session()
with (
- patch.object(provider, "_create_stream", create_stream),
+ patch.object(
+ provider._client.chat.completions,
+ "create",
+ new_callable=AsyncMock,
+ side_effect=streams,
+ ) as create,
pytest.raises(TruncatedProviderStreamError),
):
await runner._collect_recovery_text(
- {"messages": []}, include_reasoning=True
+ {"messages": []},
+ include_reasoning=True,
+ retry_session=retry_session,
)
- assert create_stream.await_count == MIDSTREAM_RECOVERY_ATTEMPTS
+ assert create.await_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
assert all(stream.closed for stream in streams)
@pytest.mark.asyncio
@@ -895,21 +915,28 @@ class TestStreamingExceptionHandling:
[_make_chunk(content=f"partial {index}")],
error=TimeoutError("recovery cutoff"),
)
- for index in range(MIDSTREAM_RECOVERY_ATTEMPTS)
+ for index in range(UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS)
]
- create_stream = AsyncMock(side_effect=[(stream, {}) for stream in streams])
provider = _make_provider()
runner = _make_stream_runner(provider)
+ retry_session = provider._admission.new_retry_session()
with (
- patch.object(provider, "_create_stream", create_stream),
+ patch.object(
+ provider._client.chat.completions,
+ "create",
+ new_callable=AsyncMock,
+ side_effect=streams,
+ ) as create,
pytest.raises(TimeoutError),
):
await runner._collect_recovery_text(
- {"messages": []}, include_reasoning=True
+ {"messages": []},
+ include_reasoning=True,
+ retry_session=retry_session,
)
- assert create_stream.await_count == MIDSTREAM_RECOVERY_ATTEMPTS
+ assert create.await_count == UPSTREAM_TRANSIENT_TOTAL_ATTEMPTS
assert all(stream.closed for stream in streams)
@pytest.mark.asyncio
@@ -921,23 +948,69 @@ class TestStreamingExceptionHandling:
_make_chunk(finish_reason="stop"),
]
)
- create_stream = AsyncMock(
- return_value=(
- stream,
- {},
- )
- )
provider = _make_provider()
runner = _make_stream_runner(provider)
+ retry_session = provider._admission.new_retry_session()
- with patch.object(provider, "_create_stream", create_stream):
+ with patch.object(
+ provider._client.chat.completions,
+ "create",
+ new_callable=AsyncMock,
+ return_value=stream,
+ ):
result = await runner._collect_recovery_text(
- {"messages": []}, include_reasoning=True
+ {"messages": []},
+ include_reasoning=True,
+ retry_session=retry_session,
)
assert result == ("world", "")
assert stream.closed is True
+ @pytest.mark.asyncio
+ async def test_recovery_collect_text_honors_provider_retry_classification(self):
+ """Provider semantics apply before the first recovery chunk as well."""
+ provider = _make_provider()
+ runner = _make_stream_runner(provider)
+ retry_session = provider._admission.new_retry_session()
+ request = httpx.Request(
+ "POST", "https://test.api.nvidia.com/v1/chat/completions"
+ )
+ degraded = openai.BadRequestError(
+ "Bad Request",
+ response=httpx.Response(400, request=request),
+ body={
+ "status": 400,
+ "detail": (
+ "Function id 'test-function': DEGRADED function cannot be invoked"
+ ),
+ },
+ )
+ rejected = ClosableAsyncStreamMock([], error=degraded)
+ recovered = ClosableAsyncStreamMock(
+ [
+ _make_chunk(content="world"),
+ _make_chunk(finish_reason="stop"),
+ ]
+ )
+
+ with patch.object(
+ provider._client.chat.completions,
+ "create",
+ new_callable=AsyncMock,
+ side_effect=[rejected, recovered],
+ ) as create:
+ result = await runner._collect_recovery_text(
+ {"messages": []},
+ include_reasoning=True,
+ retry_session=retry_session,
+ )
+
+ assert result == ("world", "")
+ assert create.await_count == 2
+ assert rejected.closed
+ assert recovered.closed
+
def test_text_recovery_body_preserves_thinking_context(self):
"""Continuation prompts include emitted thinking without provider-specific fields."""
body = {
@@ -954,6 +1027,7 @@ class TestStreamingExceptionHandling:
assert "tools" not in recovery_body
assert "tool_choice" not in recovery_body
+ assert "stream" not in recovery_body
assert recovery_body["messages"][-2] == {
"role": "assistant",
"content": "visible answer",
@@ -981,12 +1055,14 @@ class TestStreamingExceptionHandling:
new_callable=AsyncMock,
return_value=("visible answer done", "hidden reasoning more"),
) as mock_collect:
+ retry_session = runner._provider._admission.new_retry_session()
events = await runner._recovery_events(
body={"messages": [{"role": "user", "content": "hello"}]},
ledger=ledger,
error=TimeoutError("cutoff"),
tool_argument_alias_buffers={},
output_reasoning=True,
+ retry_session=retry_session,
)
assert events is not None
@@ -1109,8 +1185,8 @@ class TestStreamingExceptionHandling:
assert not any(event.event == "error" for event in parsed)
@pytest.mark.asyncio
- async def test_stream_rate_limited_retries_via_execute_with_retry(self):
- """When rate limited, execute_with_retry handles retries transparently."""
+ async def test_stream_rate_limit_uses_the_execution_retry_session(self):
+ """A create-time 429 consumes one attempt before a successful retry."""
provider = _make_provider()
request = _make_request()
@@ -1118,25 +1194,28 @@ class TestStreamingExceptionHandling:
chunk2 = _make_chunk(finish_reason="stop")
stream_mock = AsyncStreamMock([chunk1, chunk2])
+ response = httpx.Response(
+ 429,
+ request=httpx.Request(
+ "POST", "https://test.api.nvidia.com/v1/chat/completions"
+ ),
+ )
+ error = httpx.HTTPStatusError(
+ "rate limited",
+ request=response.request,
+ response=response,
+ )
+
with patch.object(
provider._client.chat.completions,
"create",
new_callable=AsyncMock,
- return_value=stream_mock,
- ):
- # Mock execute_with_retry to pass through to the actual function
- async def _passthrough(fn, *args, **kwargs):
- return await fn(*args, **kwargs)
-
- with patch.object(
- provider._rate_limiter,
- "execute_with_retry",
- new_callable=AsyncMock,
- side_effect=_passthrough,
- ):
- events = await _collect_stream(provider, request)
+ side_effect=[error, stream_mock],
+ ) as create:
+ events = await _collect_stream(provider, request)
event_text = "".join(events)
+ assert create.await_count == 2
assert "Response" in event_text
@@ -1382,12 +1461,6 @@ class TestStreamChunkEdgeCases:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -1418,12 +1491,6 @@ class TestStreamChunkEdgeCases:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
events = await _collect_stream(provider, request)
@@ -1449,12 +1516,6 @@ class TestStreamChunkEdgeCases:
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
error = await _collect_stream_error(provider, request)
@@ -1506,12 +1567,6 @@ async def test_openai_compat_stream_ends_with_contract_when_tool_name_never_arri
new_callable=AsyncMock,
return_value=stream_mock,
),
- patch.object(
- provider._rate_limiter,
- "wait_if_blocked",
- new_callable=AsyncMock,
- return_value=False,
- ),
):
error = await _collect_stream_error(provider, request)
diff --git a/tests/providers/test_subagent_interception.py b/tests/providers/test_subagent_interception.py
index f8e1e990..fc51c20a 100644
--- a/tests/providers/test_subagent_interception.py
+++ b/tests/providers/test_subagent_interception.py
@@ -11,7 +11,7 @@ from free_claude_code.providers.nvidia_nim import NvidiaNimProvider
from free_claude_code.providers.openai_chat.tool_calls import (
OpenAIToolCallAssembler,
)
-from tests.providers.support import passthrough_rate_limiter
+from tests.providers.support import immediate_admission
@pytest.mark.asyncio
@@ -21,7 +21,7 @@ async def test_task_tool_interception():
provider = NvidiaNimProvider(
config,
nim_settings=NimSettings(),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
# Mock request and stream ledger with real StreamBlockLedger
diff --git a/tests/providers/test_vercel.py b/tests/providers/test_vercel.py
index 20f45546..89f06c03 100644
--- a/tests/providers/test_vercel.py
+++ b/tests/providers/test_vercel.py
@@ -8,7 +8,7 @@ import pytest
from free_claude_code.config.provider_catalog import VERCEL_AI_GATEWAY_DEFAULT_BASE
from free_claude_code.providers.base import ProviderConfig
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, profiled_provider
+from tests.providers.support import immediate_admission, profiled_provider
def make_request(**overrides):
@@ -30,7 +30,7 @@ def vercel_provider(vercel_config):
return profiled_provider(
"vercel",
vercel_config,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
@@ -45,7 +45,7 @@ def test_init_uses_default_base_url_and_api_key(vercel_config):
provider = profiled_provider(
"vercel",
vercel_config,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
assert provider._api_key == "test_vercel_key"
@@ -60,7 +60,7 @@ def test_init_strips_trailing_slash(vercel_config):
provider = profiled_provider(
"vercel",
config,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
assert provider._base_url == VERCEL_AI_GATEWAY_DEFAULT_BASE
diff --git a/tests/providers/test_vertex.py b/tests/providers/test_vertex.py
index 1aaedbc8..6905c54c 100644
--- a/tests/providers/test_vertex.py
+++ b/tests/providers/test_vertex.py
@@ -23,7 +23,7 @@ from free_claude_code.providers.vertex.endpoint import (
)
from free_claude_code.providers.vertex.models import extract_vertex_model_page
from tests.providers.request_factory import make_messages_request
-from tests.providers.support import passthrough_rate_limiter, reasoning_for
+from tests.providers.support import immediate_admission, reasoning_for
_PROJECT_ID = "my-project"
_GLOBAL_OPENAI_BASE = (
@@ -85,7 +85,7 @@ def _provider(
ProviderConfig(api_key="", base_url=VERTEX_AI_API_ROOT),
project_id=_PROJECT_ID,
location=location,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
access_token_provider=token_provider or _token_provider(),
)
diff --git a/tests/providers/test_wafer.py b/tests/providers/test_wafer.py
index 4877c736..9d45e82a 100644
--- a/tests/providers/test_wafer.py
+++ b/tests/providers/test_wafer.py
@@ -13,7 +13,7 @@ from free_claude_code.providers.openai_chat import (
)
from tests.providers.support import (
REASONING_OFF,
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -34,7 +34,7 @@ def wafer_provider(wafer_config):
return profiled_provider(
"wafer",
wafer_config,
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/providers/test_zai.py b/tests/providers/test_zai.py
index d0fa9d11..08c98d21 100644
--- a/tests/providers/test_zai.py
+++ b/tests/providers/test_zai.py
@@ -11,7 +11,7 @@ from free_claude_code.core.anthropic.models import Message, MessagesRequest
from free_claude_code.providers.base import ProviderConfig
from free_claude_code.providers.openai_chat import OpenAIChatProvider
from tests.providers.support import (
- passthrough_rate_limiter,
+ immediate_admission,
profiled_provider,
reasoning_for,
)
@@ -27,7 +27,7 @@ def zai_provider():
rate_limit=10,
rate_window=60,
),
- rate_limiter=passthrough_rate_limiter(),
+ admission=immediate_admission(),
)
diff --git a/tests/runtime/test_provider_manager.py b/tests/runtime/test_provider_manager.py
index d0ac3d29..dda19d9b 100644
--- a/tests/runtime/test_provider_manager.py
+++ b/tests/runtime/test_provider_manager.py
@@ -106,7 +106,7 @@ async def test_replacement_keeps_leased_generation_open_until_final_release() ->
@pytest.mark.asyncio
-async def test_real_hot_replacement_owns_a_limiter_per_provider_generation() -> None:
+async def test_hot_replacement_owns_admission_per_provider_generation() -> None:
first_settings = _settings("nvidia_nim/one")
second_settings = _settings("nvidia_nim/two")
clients: list[MagicMock] = []
@@ -135,7 +135,7 @@ async def test_real_hot_replacement_owns_a_limiter_per_provider_generation() ->
assert isinstance(old_provider, NvidiaNimProvider)
assert isinstance(new_provider, NvidiaNimProvider)
assert new_provider is not old_provider
- assert new_provider._rate_limiter is not old_provider._rate_limiter
+ assert new_provider._admission is not old_provider._admission
assert old_lease.resolve_provider("nvidia_nim") is old_provider
clients[0].close.assert_not_awaited()
diff --git a/uv.lock b/uv.lock
index 677d9481..e4a5ef17 100644
--- a/uv.lock
+++ b/uv.lock
@@ -620,7 +620,7 @@ wheels = [
[[package]]
name = "free-claude-code"
-version = "4.11.0"
+version = "4.11.1"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },