Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)
Python
Split API request handling by product surface (#923)
Ali Khokhardb60452c0c18
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.mdindex 9fe57418..4fbfb43b 100644--- a/ARCHITECTURE.md+++ b/ARCHITECTURE.md@@ -32,9 +32,10 @@ flowchart LRBots[Discord or Telegram Bots] --> Messaging[Messaging Bridge]Messaging --> ClientCLI[Managed Client CLI Sessions]ClientCLI --> ProxyAPI- ProxyAPI --> Pipeline[ApiRequestPipeline]- Pipeline --> Router[ModelRouter]- Pipeline --> Providers[ProviderRegistry]+ ProxyAPI --> Handlers[API Product Handlers]+ Handlers --> Router[ModelRouter]+ Handlers --> Executor[ProviderExecutionService]+ Executor --> Providers[ProviderRegistry]Providers --> OpenAIChat[OpenAI Chat Providers]Providers --> NativeAnthropic[Anthropic Messages Providers]```@@ -43,8 +44,9 @@ flowchart LRThe installable wheel packages are declared in [pyproject.toml](pyproject.toml):-- [api/](api/) owns the FastAPI app, route handlers, request pipeline, model- catalog, admin APIs, local optimizations, and server-tool handling.+- [api/](api/) owns the FastAPI app, route handlers, API product handlers, shared+ provider execution, model catalog, admin APIs, local optimizations, and+ server-tool handling.- [cli/](cli/) owns console entrypoints, client CLI launchers, process/sessionmanagement, and client adapter contracts.- [config/](config/) owns settings, provider metadata, filesystem paths,@@ -105,10 +107,11 @@ The current package boundaries are intentional, but several modules still carrylarge orchestration responsibilities. Treat these as refactor targets, not asnew places to add unrelated behavior:-- [api/request_pipeline.py](api/request_pipeline.py) coordinates routing,- message-only intercepts, provider execution, token counting, and Responses- adaptation. Keep route handlers thin and keep new pre-provider behavior as- explicit pipeline intercepts.+- [api/handlers/](api/handlers/) owns customer-facing API product flows:+ Claude Messages, OpenAI Responses, and token counting. Keep route handlers+ thin, keep Claude-only behavior in the Messages handler, and use+ [api/provider_execution.py](api/provider_execution.py) only for shared+ provider resolution, preflight, tracing, token counting, and streaming.- [providers/transports/](providers/transports/) owns provider transportfamilies. The OpenAI-chat and native Anthropic transport packages split thintransport bases from per-request stream runners, recovery event construction,@@ -221,36 +224,42 @@ proxy auth is disabled. Otherwise the token may be supplied through `x-api-key`,`Authorization: Bearer ...`, or `anthropic-auth-token`. Comparisons useconstant-time matching.-`ApiRequestPipeline` in-[api/request_pipeline.py](api/request_pipeline.py) coordinates request handling.-It validates non-empty messages, resolves models, runs explicit message-only-intercepts for local server tools and optimizations, resolves a provider,-preflights the upstream request, then streams Anthropic SSE back to the caller.+[api/handlers/](api/handlers/) owns the public API product flows.+`MessagesHandler` validates non-empty messages, resolves models, applies+Claude-only safety-classifier and local optimization policy, handles local web+server tools, then streams Anthropic SSE. `ResponsesHandler` owns streaming-only+OpenAI Responses validation and conversion for Codex clients. `TokenCountHandler`+owns Anthropic token counting. Shared provider execution lives in+[api/provider_execution.py](api/provider_execution.py), which resolves a+provider, preflights the upstream request, emits trace events, counts input+tokens, and returns an Anthropic SSE iterator.```mermaidsequenceDiagramparticipant Clientparticipant Route as FastAPIRoute- participant Pipeline as ApiRequestPipeline+ participant Handler as ProductHandlerparticipant Router as ModelRouter+ participant Exec as ProviderExecutionparticipant Registry as ProviderRegistryparticipant ProviderClient->>Route: POST /v1/messagesRoute->>Route: require_api_key- Route->>Pipeline: create_message- Pipeline->>Router: resolve model and thinking- Pipeline->>Pipeline: server tools or optimizations- Pipeline->>Registry: resolve provider+ Route->>Handler: create message+ Handler->>Router: resolve model and thinking+ Handler->>Handler: server tools or optimizations+ Handler->>Exec: stream routed request+ Exec->>Registry: resolve providerRegistry->>Provider: cached or new provider- Pipeline->>Provider: preflight_stream- Pipeline->>Provider: stream_response+ Exec->>Provider: preflight_stream+ Exec->>Provider: stream_responseProvider-->>Client: Anthropic SSE events```-OpenAI Responses uses the same provider execution path without message-only-intercepts. `create_response()` delegates Responses protocol work to the-`OpenAIResponsesAdapter` in+OpenAI Responses uses the same provider execution primitive without importing+Claude-only message intercepts. `ResponsesHandler` delegates protocol work to+the `OpenAIResponsesAdapter` in[core/openai_responses/adapter.py](core/openai_responses/adapter.py). The adapterconverts the Responses payload into an Anthropic Messages payload beforeprovider execution, then converts Anthropic SSE back to Responses SSE.@@ -423,25 +432,25 @@ common low-value client requests before they reach a provider:- suggestion mode;- filepath extraction.-The service runs these only after model routing and after local server-tool+The Messages handler runs these only after model routing and after local server-toolhandling. Each optimization is controlled by settings flags.Claude Code auto-mode safety-classifier requests are a message-only routing-policy, not a short-circuit response. After routing, the pipeline detects the+policy, not a short-circuit response. After routing, the Messages handler detects thenarrow classifier prompt shape and forces thinking off before provider executionso Claude Code receives a parser-readable `<block>yes</block>` or`<block>no</block>` verdict.Local `web_search` and `web_fetch` handling lives under[api/web_tools/](api/web_tools/). When `ENABLE_WEB_SERVER_TOOLS` is true, the-service can stream local Anthropic server-tool responses without sending the+Messages handler can stream local Anthropic server-tool responses without sending therequest upstream. [api/web_tools/egress.py](api/web_tools/egress.py) enforces URLscheme and private-network restrictions for `web_fetch`.OpenAI-chat upstream providers are identified by`ProviderDescriptor.transport_type == "openai_chat"` in[config/provider_catalog.py](config/provider_catalog.py). They cannot safely-represent Anthropic server-tool blocks, so the service rejects unsupported+represent Anthropic server-tool blocks, so the Messages handler rejects unsupportedserver-tool requests before provider execution instead of performing a lossyconversion. Forced `web_search` or `web_fetch` requests are handled locally when`ENABLE_WEB_SERVER_TOOLS` is true; otherwise OpenAI-chat upstreams reject themdiff --git a/api/handlers/__init__.py b/api/handlers/__init__.pynew file mode 100644index 00000000..e14c5d70--- /dev/null+++ b/api/handlers/__init__.py@@ -0,0 +1,7 @@+"""Product-flow handlers for public API routes."""++from .messages import MessagesHandler+from .responses import ResponsesHandler+from .token_count import TokenCountHandler++__all__ = ["MessagesHandler", "ResponsesHandler", "TokenCountHandler"]diff --git a/api/handlers/messages.py b/api/handlers/messages.pynew file mode 100644index 00000000..a20b47af--- /dev/null+++ b/api/handlers/messages.py@@ -0,0 +1,172 @@+"""Claude Messages API product flow."""++from __future__ import annotations++from collections.abc import Callable+from dataclasses import replace++from loguru import logger++from api.detection import is_safety_classifier_request+from api.model_router import ModelRouter, RoutedMessagesRequest+from api.models.anthropic import MessagesRequest+from api.optimization_handlers import try_optimizations+from api.provider_execution import ProviderExecutionService, TokenCounter+from api.request_errors import require_non_empty_messages, unexpected_http_exception+from api.response_streams import anthropic_sse_streaming_response+from api.web_tools.egress import WebFetchEgressPolicy+from api.web_tools.request import (+ is_web_server_tool_request,+ openai_chat_upstream_server_tool_error,+)+from api.web_tools.streaming import stream_web_server_tool_response+from config.provider_catalog import PROVIDER_CATALOG+from config.settings import Settings+from core.anthropic import get_token_count+from core.trace import trace_event+from providers.base import BaseProvider+from providers.exceptions import InvalidRequestError, ProviderError++ProviderGetter = Callable[[str], BaseProvider]+MessageIntercept = Callable[[RoutedMessagesRequest], object | None]++_OPENAI_CHAT_UPSTREAM_IDS = frozenset(+ provider_id+ for provider_id, descriptor in PROVIDER_CATALOG.items()+ if descriptor.transport_type == "openai_chat"+)+++class MessagesHandler:+ """Handle Anthropic-compatible Messages requests."""++ def __init__(+ self,+ settings: Settings,+ provider_getter: ProviderGetter,+ *,+ model_router: ModelRouter | None = None,+ token_counter: TokenCounter = get_token_count,+ provider_execution: ProviderExecutionService | None = None,+ ) -> None:+ self._settings = settings+ self._model_router = model_router or ModelRouter(settings)+ self._token_counter = token_counter+ self._provider_execution = provider_execution or ProviderExecutionService(+ settings,+ provider_getter,+ token_counter=token_counter,+ )+ self._message_intercepts: tuple[MessageIntercept, ...] = (+ self._intercept_web_server_tool,+ self._intercept_local_optimization,+ )++ def create(self, request_data: MessagesRequest) -> object:+ """Create an Anthropic-compatible message response."""+ try:+ require_non_empty_messages(request_data.messages)+ routed = self._model_router.resolve_messages_request(request_data)+ routed = self._apply_message_routing_policies(routed)+ self._reject_unsupported_server_tools(routed)++ intercepted = self._run_message_intercepts(routed)+ if intercepted is not None:+ return intercepted++ logger.debug("No optimization matched, routing to provider")+ return anthropic_sse_streaming_response(+ self._provider_execution.stream(+ routed,+ wire_api="messages",+ raw_log_label="FULL_PAYLOAD",+ raw_log_payload=routed.request.model_dump(),+ )+ )+ except ProviderError:+ raise+ except Exception as exc:+ raise unexpected_http_exception(+ self._settings, exc, context="CREATE_MESSAGE_ERROR"+ ) from exc++ def _reject_unsupported_server_tools(self, routed: RoutedMessagesRequest) -> None:+ if routed.resolved.provider_id not in _OPENAI_CHAT_UPSTREAM_IDS:+ return+ tool_err = openai_chat_upstream_server_tool_error(+ routed.request,+ web_tools_enabled=self._settings.enable_web_server_tools,+ )+ if tool_err is not None:+ raise InvalidRequestError(tool_err)++ def _apply_message_routing_policies(+ self, routed: RoutedMessagesRequest+ ) -> RoutedMessagesRequest:+ if not is_safety_classifier_request(routed.request):+ return routed+ changed = routed.resolved.thinking_enabled+ trace_event(+ stage="routing",+ event="api.optimization.safety_classifier_no_thinking",+ source="api",+ model=routed.request.model,+ changed=changed,+ )+ if not changed:+ return routed+ return RoutedMessagesRequest(+ request=routed.request,+ resolved=replace(routed.resolved, thinking_enabled=False),+ )++ def _run_message_intercepts(self, routed: RoutedMessagesRequest) -> object | None:+ for intercept in self._message_intercepts:+ result = intercept(routed)+ if result is not None:+ return result+ return None++ def _intercept_web_server_tool(+ self, routed: RoutedMessagesRequest+ ) -> object | None:+ if not self._settings.enable_web_server_tools:+ return None+ if not is_web_server_tool_request(routed.request):+ return None++ input_tokens = self._token_counter(+ routed.request.messages, routed.request.system, routed.request.tools+ )+ trace_event(+ stage="routing",+ event="api.optimization.web_server_tool",+ source="api",+ model=routed.request.model,+ )+ egress = WebFetchEgressPolicy(+ allow_private_network_targets=self._settings.web_fetch_allow_private_networks,+ allowed_schemes=self._settings.web_fetch_allowed_scheme_set(),+ )+ return anthropic_sse_streaming_response(+ stream_web_server_tool_response(+ routed.request,+ input_tokens=input_tokens,+ web_fetch_egress=egress,+ verbose_client_errors=self._settings.log_api_error_tracebacks,+ ),+ )++ def _intercept_local_optimization(+ self, routed: RoutedMessagesRequest+ ) -> object | None:+ optimized = try_optimizations(routed.request, self._settings)+ if optimized is None:+ return None+ trace_event(+ stage="routing",+ event="api.optimization.short_circuit",+ source="api",+ model=routed.request.model,+ )+ return optimizeddiff --git a/api/handlers/responses.py b/api/handlers/responses.pynew file mode 100644index 00000000..09ad0e60--- /dev/null+++ b/api/handlers/responses.py@@ -0,0 +1,113 @@+"""OpenAI Responses API product flow for Codex clients."""++from __future__ import annotations++from collections.abc import Callable++from fastapi.responses import JSONResponse++from api.model_router import ModelRouter+from api.models.anthropic import MessagesRequest+from api.models.openai_responses import OpenAIResponsesRequest+from api.provider_execution import ProviderExecutionService+from api.request_errors import (+ http_status_for_unexpected_api_exception,+ log_unexpected_api_exception,+ require_non_empty_messages,+)+from api.response_streams import openai_responses_sse_streaming_response+from config.settings import Settings+from core.anthropic import get_user_facing_error_message+from core.openai_responses import OpenAIResponsesAdapter+from providers.base import BaseProvider+from providers.exceptions import InvalidRequestError, ProviderError++ProviderGetter = Callable[[str], BaseProvider]+++class ResponsesHandler:+ """Handle streaming OpenAI Responses-compatible requests."""++ def __init__(+ self,+ settings: Settings,+ provider_getter: ProviderGetter,+ *,+ model_router: ModelRouter | None = None,+ responses_adapter: OpenAIResponsesAdapter | None = None,+ provider_execution: ProviderExecutionService | None = None,+ ) -> None:+ self._settings = settings+ self._model_router = model_router or ModelRouter(settings)+ self._responses_adapter = responses_adapter or OpenAIResponsesAdapter()+ self._provider_execution = provider_execution or ProviderExecutionService(+ settings,+ provider_getter,+ )++ async def create(self, request_data: OpenAIResponsesRequest) -> object:+ """Create a streaming OpenAI Responses-compatible response."""+ request_payload = request_data.model_dump(mode="json", exclude_none=True)+ if request_data.stream is False:+ invalid_request = InvalidRequestError(+ "FCC /v1/responses supports streaming only; omit stream or set stream=true."+ )+ return JSONResponse(+ status_code=invalid_request.status_code,+ content=self._responses_adapter.error_payload(+ message=invalid_request.message,+ error_type=invalid_request.error_type,+ ),+ )++ try:+ anthropic_payload = self._responses_adapter.to_anthropic_payload(+ request_payload+ )+ response_request = MessagesRequest(**anthropic_payload)+ require_non_empty_messages(response_request.messages)+ routed = self._model_router.resolve_messages_request(response_request)++ streamed = self._provider_execution.stream(+ routed,+ wire_api="responses",+ raw_log_label="FULL_RESPONSES_PAYLOAD",+ raw_log_payload=request_payload,+ )+ return openai_responses_sse_streaming_response(+ self._responses_adapter.iter_sse_from_anthropic(+ streamed,+ request_payload,+ ),+ headers=self._responses_adapter.sse_headers,+ )+ except OpenAIResponsesAdapter.ConversionError as exc:+ invalid_request = InvalidRequestError(str(exc))+ return JSONResponse(+ status_code=invalid_request.status_code,+ content=self._responses_adapter.error_payload(+ message=invalid_request.message,+ error_type=invalid_request.error_type,+ ),+ )+ except ProviderError as exc:+ return JSONResponse(+ status_code=exc.status_code,+ content=self._responses_adapter.error_payload(+ message=exc.message,+ error_type=exc.error_type,+ ),+ )+ except Exception as exc:+ log_unexpected_api_exception(+ self._settings,+ exc,+ context="CREATE_RESPONSE_ERROR",+ )+ return JSONResponse(+ status_code=http_status_for_unexpected_api_exception(exc),+ content=self._responses_adapter.error_payload(+ message=get_user_facing_error_message(exc),+ error_type="api_error",+ ),+ )diff --git a/api/handlers/token_count.py b/api/handlers/token_count.pynew file mode 100644index 00000000..241a7e5a--- /dev/null+++ b/api/handlers/token_count.py@@ -0,0 +1,80 @@+"""Anthropic token-count API product flow."""++from __future__ import annotations++import uuid++from fastapi import HTTPException+from loguru import logger++from api.model_router import ModelRouter+from api.models.anthropic import TokenCountRequest+from api.models.responses import TokenCountResponse+from api.provider_execution import TokenCounter+from api.request_errors import (+ http_status_for_unexpected_api_exception,+ log_unexpected_api_exception,+ require_non_empty_messages,+)+from config.settings import Settings+from core.anthropic import get_token_count, get_user_facing_error_message+from core.trace import api_messages_request_snapshot, trace_event+from providers.exceptions import ProviderError+++class TokenCountHandler:+ """Handle Anthropic-compatible token count requests."""++ def __init__(+ self,+ settings: Settings,+ *,+ model_router: ModelRouter | None = None,+ token_counter: TokenCounter = get_token_count,+ ) -> None:+ self._settings = settings+ self._model_router = model_router or ModelRouter(settings)+ self._token_counter = token_counter++ def count(self, request_data: TokenCountRequest) -> TokenCountResponse:+ """Count tokens for a request after applying configured model routing."""+ request_id = f"req_{uuid.uuid4().hex[:12]}"+ with logger.contextualize(request_id=request_id):+ try:+ require_non_empty_messages(request_data.messages)+ routed = self._model_router.resolve_token_count_request(request_data)+ tokens = self._token_counter(+ routed.request.messages, routed.request.system, routed.request.tools+ )+ trace_event(+ stage="routing",+ event="api.route.resolved",+ source="api",+ kind="count_tokens",+ provider_id=routed.resolved.provider_id,+ provider_model=routed.resolved.provider_model,+ provider_model_ref=routed.resolved.provider_model_ref,+ gateway_model=routed.request.model,+ )+ trace_event(+ stage="ingress",+ event="api.count_tokens.completed",+ source="api",+ message_count=len(routed.request.messages),+ input_tokens=tokens,+ snapshot=api_messages_request_snapshot(routed.request),+ )+ return TokenCountResponse(input_tokens=tokens)+ except ProviderError:+ raise+ except Exception as exc:+ log_unexpected_api_exception(+ self._settings,+ exc,+ context="COUNT_TOKENS_ERROR",+ request_id=request_id,+ )+ raise HTTPException(+ status_code=http_status_for_unexpected_api_exception(exc),+ detail=get_user_facing_error_message(exc),+ ) from excdiff --git a/api/provider_execution.py b/api/provider_execution.pynew file mode 100644index 00000000..4d019eb3--- /dev/null+++ b/api/provider_execution.py@@ -0,0 +1,111 @@+"""Shared provider execution primitive for API product handlers."""++from __future__ import annotations++import uuid+from collections.abc import AsyncIterator, Callable+from typing import Any++from loguru import logger++from config.settings import Settings+from core.anthropic import get_token_count+from core.trace import api_messages_request_snapshot, trace_event, traced_async_stream+from providers.base import BaseProvider++from .model_router import RoutedMessagesRequest++TokenCounter = Callable[[list[Any], str | list[Any] | None, list[Any] | None], int]+ProviderGetter = Callable[[str], BaseProvider]+++class ProviderExecutionService:+ """Resolve a provider and execute one routed Anthropic Messages stream."""++ def __init__(+ self,+ settings: Settings,+ provider_getter: ProviderGetter,+ *,+ token_counter: TokenCounter = get_token_count,+ ) -> None:+ self._settings = settings+ self._provider_getter = provider_getter+ self._token_counter = token_counter++ def stream(+ self,+ routed: RoutedMessagesRequest,+ *,+ wire_api: str,+ raw_log_label: str,+ raw_log_payload: Any,+ ) -> AsyncIterator[str]:+ provider = self._provider_getter(routed.resolved.provider_id)+ provider.preflight_stream(+ routed.request,+ thinking_enabled=routed.resolved.thinking_enabled,+ )++ route_trace: dict[str, Any] = {+ "stage": "routing",+ "event": "api.route.resolved",+ "source": "api",+ "provider_id": routed.resolved.provider_id,+ "provider_model": routed.resolved.provider_model,+ "provider_model_ref": routed.resolved.provider_model_ref,+ "gateway_model": routed.request.model,+ "thinking_enabled": routed.resolved.thinking_enabled,+ }+ if wire_api == "responses":+ route_trace["wire_api"] = "responses"+ trace_event(**route_trace)++ request_id = f"req_{uuid.uuid4().hex[:12]}"+ trace_event(+ stage="ingress",+ event=(+ "api.responses.request.received"+ if wire_api == "responses"+ else "api.request.received"+ ),+ source="api",+ message_count=len(routed.request.messages),+ snapshot=api_messages_request_snapshot(routed.request),+ request_id=request_id,+ )++ if self._settings.log_raw_api_payloads:+ logger.debug(f"{raw_log_label} [{{}}]: {{}}", request_id, raw_log_payload)++ input_tokens = self._token_counter(+ routed.request.messages,+ routed.request.system,+ routed.request.tools,+ )+ return traced_async_stream(+ provider.stream_response(+ routed.request,+ input_tokens=input_tokens,+ request_id=request_id,+ thinking_enabled=routed.resolved.thinking_enabled,+ ),+ stage="egress",+ source="api",+ complete_event=(+ "api.responses.stream_completed"+ if wire_api == "responses"+ else "api.response.stream_completed"+ ),+ interrupted_event=(+ "api.responses.stream_interrupted"+ if wire_api == "responses"+ else "api.response.stream_interrupted"+ ),+ chunk_event=None,+ extra={+ "request_id": request_id,+ "provider_id": routed.resolved.provider_id,+ "gateway_model": routed.request.model,+ },+ )diff --git a/api/request_errors.py b/api/request_errors.pynew file mode 100644index 00000000..cb193bab--- /dev/null+++ b/api/request_errors.py@@ -0,0 +1,58 @@+"""Shared API request validation and safe error logging."""++from __future__ import annotations++import traceback+from typing import Any++from fastapi import HTTPException+from loguru import logger++from config.settings import Settings+from core.anthropic import get_user_facing_error_message+from providers.exceptions import InvalidRequestError+++def require_non_empty_messages(messages: list[Any]) -> None:+ if not messages:+ raise InvalidRequestError("messages cannot be empty")+++def http_status_for_unexpected_api_exception(_exc: BaseException) -> int:+ return 500+++def log_unexpected_api_exception(+ settings: Settings,+ exc: BaseException,+ *,+ context: str,+ request_id: str | None = None,+) -> None:+ """Log API failures without echoing exception text unless opted in."""+ if settings.log_api_error_tracebacks:+ if request_id is not None:+ logger.error("{} request_id={}: {}", context, request_id, exc)+ else:+ logger.error("{}: {}", context, exc)+ logger.error(traceback.format_exc())+ return+ if request_id is not None:+ logger.error(+ "{} request_id={} exc_type={}",+ context,+ request_id,+ type(exc).__name__,+ )+ else:+ logger.error("{} exc_type={}", context, type(exc).__name__)+++def unexpected_http_exception(+ settings: Settings, exc: Exception, *, context: str+) -> HTTPException:+ log_unexpected_api_exception(settings, exc, context=context)+ return HTTPException(+ status_code=http_status_for_unexpected_api_exception(exc),+ detail=get_user_facing_error_message(exc),+ )diff --git a/api/request_pipeline.py b/api/request_pipeline.pydeleted file mode 100644index eb0cacd4..00000000--- a/api/request_pipeline.py+++ /dev/null@@ -1,424 +0,0 @@-"""API request pipeline for routing, intercepts, and provider execution."""--from __future__ import annotations--import traceback-import uuid-from collections.abc import AsyncIterator, Callable-from dataclasses import replace-from typing import Any--from fastapi import HTTPException-from fastapi.responses import JSONResponse, StreamingResponse-from loguru import logger--from config.provider_catalog import PROVIDER_CATALOG-from config.settings import Settings-from core.anthropic import get_token_count, get_user_facing_error_message-from core.anthropic.streaming import ANTHROPIC_SSE_RESPONSE_HEADERS-from core.openai_responses import OpenAIResponsesAdapter-from core.trace import api_messages_request_snapshot, trace_event, traced_async_stream-from providers.base import BaseProvider-from providers.exceptions import InvalidRequestError, ProviderError--from .detection import is_safety_classifier_request-from .model_router import ModelRouter, RoutedMessagesRequest-from .models.anthropic import MessagesRequest, TokenCountRequest-from .models.openai_responses import OpenAIResponsesRequest-from .models.responses import TokenCountResponse-from .optimization_handlers import try_optimizations-from .web_tools.egress import WebFetchEgressPolicy-from .web_tools.request import (- is_web_server_tool_request,- openai_chat_upstream_server_tool_error,-)-from .web_tools.streaming import stream_web_server_tool_response--TokenCounter = Callable[[list[Any], str | list[Any] | None, list[Any] | None], int]-ProviderGetter = Callable[[str], BaseProvider]-MessageIntercept = Callable[[RoutedMessagesRequest], object | None]--# Providers that use ``/chat/completions`` + Anthropic-to-OpenAI conversion.-_OPENAI_CHAT_UPSTREAM_IDS = frozenset(- provider_id- for provider_id, descriptor in PROVIDER_CATALOG.items()- if descriptor.transport_type == "openai_chat"-)---def anthropic_sse_streaming_response(body: AsyncIterator[str]) -> StreamingResponse:- """Return a streaming response for Anthropic-style SSE streams."""- return StreamingResponse(- body,- media_type="text/event-stream",- headers=ANTHROPIC_SSE_RESPONSE_HEADERS,- )---def openai_responses_sse_streaming_response(- body: AsyncIterator[str],-) -> StreamingResponse:- """Return a streaming response for OpenAI Responses-style SSE."""- return StreamingResponse(- body,- media_type="text/event-stream",- headers=OpenAIResponsesAdapter.sse_headers,- )---def _http_status_for_unexpected_pipeline_exception(_exc: BaseException) -> int:- """HTTP status for uncaught non-provider failures."""- return 500---def _log_unexpected_pipeline_exception(- settings: Settings,- exc: BaseException,- *,- context: str,- request_id: str | None = None,-) -> None:- """Log API failures without echoing exception text unless opted in."""- if settings.log_api_error_tracebacks:- if request_id is not None:- logger.error("{} request_id={}: {}", context, request_id, exc)- else:- logger.error("{}: {}", context, exc)- logger.error(traceback.format_exc())- return- if request_id is not None:- logger.error(- "{} request_id={} exc_type={}",- context,- request_id,- type(exc).__name__,- )- else:- logger.error("{} exc_type={}", context, type(exc).__name__)---def _require_non_empty_messages(messages: list[Any]) -> None:- if not messages:- raise InvalidRequestError("messages cannot be empty")---class ApiRequestPipeline:- """Coordinate API request intercepts, routing, and provider stream execution."""-- def __init__(- self,- settings: Settings,- provider_getter: ProviderGetter,- model_router: ModelRouter | None = None,- token_counter: TokenCounter = get_token_count,- responses_adapter: OpenAIResponsesAdapter | None = None,- ) -> None:- self._settings = settings- self._provider_getter = provider_getter- self._model_router = model_router or ModelRouter(settings)- self._token_counter = token_counter- self._responses_adapter = responses_adapter or OpenAIResponsesAdapter()- self._message_intercepts: tuple[MessageIntercept, ...] = (- self._intercept_web_server_tool,- self._intercept_local_optimization,- )-- def create_message(self, request_data: MessagesRequest) -> object:- """Create an Anthropic-compatible message response."""- try:- _require_non_empty_messages(request_data.messages)- routed = self._model_router.resolve_messages_request(request_data)- routed = self._apply_message_routing_policies(routed)- self._reject_unsupported_server_tools(routed)-- intercepted = self._run_message_intercepts(routed)- if intercepted is not None:- return intercepted-- logger.debug("No optimization matched, routing to provider")- return anthropic_sse_streaming_response(- self._provider_stream(- routed,- wire_api="messages",- raw_log_label="FULL_PAYLOAD",- raw_log_payload=routed.request.model_dump(),- )- )- except ProviderError:- raise- except Exception as e:- _log_unexpected_pipeline_exception(- self._settings, e, context="CREATE_MESSAGE_ERROR"- )- raise HTTPException(- status_code=_http_status_for_unexpected_pipeline_exception(e),- detail=get_user_facing_error_message(e),- ) from e-- async def create_response(self, request_data: OpenAIResponsesRequest) -> object:- """Create a streaming OpenAI Responses-compatible response."""- request_payload = request_data.model_dump(mode="json", exclude_none=True)- if request_data.stream is False:- invalid_request = InvalidRequestError(- "FCC /v1/responses supports streaming only; omit stream or set stream=true."- )- return JSONResponse(- status_code=invalid_request.status_code,- content=self._responses_adapter.error_payload(- message=invalid_request.message,- error_type=invalid_request.error_type,- ),- )-- try:- anthropic_payload = self._responses_adapter.to_anthropic_payload(- request_payload- )- response_request = MessagesRequest(**anthropic_payload)- _require_non_empty_messages(response_request.messages)- routed = self._model_router.resolve_messages_request(response_request)- self._reject_unsupported_server_tools(routed)-- streamed = self._provider_stream(- routed,- wire_api="responses",- raw_log_label="FULL_RESPONSES_PAYLOAD",- raw_log_payload=request_payload,- )- return openai_responses_sse_streaming_response(- self._responses_adapter.iter_sse_from_anthropic(- streamed,- request_payload,- )- )- except OpenAIResponsesAdapter.ConversionError as exc:- invalid_request = InvalidRequestError(str(exc))- return JSONResponse(- status_code=invalid_request.status_code,- content=self._responses_adapter.error_payload(- message=invalid_request.message,- error_type=invalid_request.error_type,- ),- )- except ProviderError as exc:- return JSONResponse(- status_code=exc.status_code,- content=self._responses_adapter.error_payload(- message=exc.message,- error_type=exc.error_type,- ),- )- except Exception as e:- _log_unexpected_pipeline_exception(- self._settings,- e,- context="CREATE_RESPONSE_ERROR",- )- return JSONResponse(- status_code=_http_status_for_unexpected_pipeline_exception(e),- content=self._responses_adapter.error_payload(- message=get_user_facing_error_message(e),- error_type="api_error",- ),- )-- def count_tokens(self, request_data: TokenCountRequest) -> TokenCountResponse:- """Count tokens for a request after applying configured model routing."""- request_id = f"req_{uuid.uuid4().hex[:12]}"- with logger.contextualize(request_id=request_id):- try:- _require_non_empty_messages(request_data.messages)- routed = self._model_router.resolve_token_count_request(request_data)- tokens = self._token_counter(- routed.request.messages, routed.request.system, routed.request.tools- )- trace_event(- stage="routing",- event="api.route.resolved",- source="api",- kind="count_tokens",- provider_id=routed.resolved.provider_id,- provider_model=routed.resolved.provider_model,- provider_model_ref=routed.resolved.provider_model_ref,- gateway_model=routed.request.model,- )- trace_event(- stage="ingress",- event="api.count_tokens.completed",- source="api",- message_count=len(routed.request.messages),- input_tokens=tokens,- snapshot=api_messages_request_snapshot(routed.request),- )- return TokenCountResponse(input_tokens=tokens)- except ProviderError:- raise- except Exception as e:- _log_unexpected_pipeline_exception(- self._settings,- e,- context="COUNT_TOKENS_ERROR",- request_id=request_id,- )- raise HTTPException(- status_code=_http_status_for_unexpected_pipeline_exception(e),- detail=get_user_facing_error_message(e),- ) from e-- def _reject_unsupported_server_tools(self, routed: RoutedMessagesRequest) -> None:- if routed.resolved.provider_id not in _OPENAI_CHAT_UPSTREAM_IDS:- return- tool_err = openai_chat_upstream_server_tool_error(- routed.request,- web_tools_enabled=self._settings.enable_web_server_tools,- )- if tool_err is not None:- raise InvalidRequestError(tool_err)-- def _apply_message_routing_policies(- self, routed: RoutedMessagesRequest- ) -> RoutedMessagesRequest:- if not is_safety_classifier_request(routed.request):- return routed- changed = routed.resolved.thinking_enabled- trace_event(- stage="routing",- event="api.optimization.safety_classifier_no_thinking",- source="api",- model=routed.request.model,- changed=changed,- )- if not changed:- return routed- return RoutedMessagesRequest(- request=routed.request,- resolved=replace(routed.resolved, thinking_enabled=False),- )-- def _run_message_intercepts(self, routed: RoutedMessagesRequest) -> object | None:- for intercept in self._message_intercepts:- result = intercept(routed)- if result is not None:- return result- return None-- def _intercept_web_server_tool(- self, routed: RoutedMessagesRequest- ) -> object | None:- if not self._settings.enable_web_server_tools:- return None- if not is_web_server_tool_request(routed.request):- return None-- input_tokens = self._token_counter(- routed.request.messages, routed.request.system, routed.request.tools- )- trace_event(- stage="routing",- event="api.optimization.web_server_tool",- source="api",- model=routed.request.model,- )- egress = WebFetchEgressPolicy(- allow_private_network_targets=self._settings.web_fetch_allow_private_networks,- allowed_schemes=self._settings.web_fetch_allowed_scheme_set(),- )- return anthropic_sse_streaming_response(- stream_web_server_tool_response(- routed.request,- input_tokens=input_tokens,- web_fetch_egress=egress,- verbose_client_errors=self._settings.log_api_error_tracebacks,- ),- )-- def _intercept_local_optimization(- self, routed: RoutedMessagesRequest- ) -> object | None:- optimized = try_optimizations(routed.request, self._settings)- if optimized is None:- return None- trace_event(- stage="routing",- event="api.optimization.short_circuit",- source="api",- model=routed.request.model,- )- return optimized-- def _provider_stream(- self,- routed: RoutedMessagesRequest,- *,- wire_api: str,- raw_log_label: str,- raw_log_payload: Any,- ) -> AsyncIterator[str]:- provider = self._provider_getter(routed.resolved.provider_id)- provider.preflight_stream(- routed.request,- thinking_enabled=routed.resolved.thinking_enabled,- )-- route_trace: dict[str, Any] = {- "stage": "routing",- "event": "api.route.resolved",- "source": "api",- "provider_id": routed.resolved.provider_id,- "provider_model": routed.resolved.provider_model,- "provider_model_ref": routed.resolved.provider_model_ref,- "gateway_model": routed.request.model,- "thinking_enabled": routed.resolved.thinking_enabled,- }- if wire_api == "responses":- route_trace["wire_api"] = "responses"- trace_event(**route_trace)-- request_id = f"req_{uuid.uuid4().hex[:12]}"- trace_event(- stage="ingress",- event=(- "api.responses.request.received"- if wire_api == "responses"- else "api.request.received"- ),- source="api",- message_count=len(routed.request.messages),- snapshot=api_messages_request_snapshot(routed.request),- request_id=request_id,- )-- if self._settings.log_raw_api_payloads:- logger.debug(f"{raw_log_label} [{{}}]: {{}}", request_id, raw_log_payload)-- input_tokens = self._token_counter(- routed.request.messages,- routed.request.system,- routed.request.tools,- )- return traced_async_stream(- provider.stream_response(- routed.request,- input_tokens=input_tokens,- request_id=request_id,- thinking_enabled=routed.resolved.thinking_enabled,- ),- stage="egress",- source="api",- complete_event=(- "api.responses.stream_completed"- if wire_api == "responses"- else "api.response.stream_completed"- ),- interrupted_event=(- "api.responses.stream_interrupted"- if wire_api == "responses"- else "api.response.stream_interrupted"- ),- chunk_event=None,- extra={- "request_id": request_id,- "provider_id": routed.resolved.provider_id,- "gateway_model": routed.request.model,- },- )diff --git a/api/response_streams.py b/api/response_streams.pynew file mode 100644index 00000000..5651880e--- /dev/null+++ b/api/response_streams.py@@ -0,0 +1,31 @@+"""FastAPI streaming response wrappers for public API wire formats."""++from __future__ import annotations++from collections.abc import AsyncIterator, Mapping++from fastapi.responses import StreamingResponse++from core.anthropic.streaming import ANTHROPIC_SSE_RESPONSE_HEADERS+++def anthropic_sse_streaming_response(body: AsyncIterator[str]) -> StreamingResponse:+ """Return a streaming response for Anthropic-style SSE streams."""+ return StreamingResponse(+ body,+ media_type="text/event-stream",+ headers=ANTHROPIC_SSE_RESPONSE_HEADERS,+ )+++def openai_responses_sse_streaming_response(+ body: AsyncIterator[str],+ *,+ headers: Mapping[str, str],+) -> StreamingResponse:+ """Return a streaming response for OpenAI Responses-style SSE."""+ return StreamingResponse(+ body,+ media_type="text/event-stream",+ headers=dict(headers),+ )diff --git a/api/routes.py b/api/routes.pyindex 08b7fef6..66b19d78 100644--- a/api/routes.py+++ b/api/routes.py@@ -10,29 +10,51 @@ from providers.registry import ProviderRegistryfrom . import dependenciesfrom .dependencies import get_settings, require_api_key+from .handlers import MessagesHandler, ResponsesHandler, TokenCountHandlerfrom .model_catalog import build_models_list_responsefrom .models.anthropic import MessagesRequest, TokenCountRequestfrom .models.openai_responses import OpenAIResponsesRequestfrom .models.responses import ModelsListResponse-from .request_pipeline import ApiRequestPipelinerouter = APIRouter()-def get_request_pipeline(+def _provider_getter(request: Request, settings: Settings):+ return lambda provider_type: dependencies.resolve_provider(+ provider_type, app=request.app, settings=settings+ )+++def get_messages_handler(request: Request,settings: Settings = Depends(get_settings),-) -> ApiRequestPipeline:- """Build the API request pipeline for route handlers."""- return ApiRequestPipeline(+) -> MessagesHandler:+ """Build the Claude Messages product handler for route handlers."""+ return MessagesHandler(settings,- provider_getter=lambda provider_type: dependencies.resolve_provider(- provider_type, app=request.app, settings=settings- ),+ provider_getter=_provider_getter(request, settings),token_counter=get_token_count,)+def get_responses_handler(+ request: Request,+ settings: Settings = Depends(get_settings),+) -> ResponsesHandler:+ """Build the OpenAI Responses product handler for route handlers."""+ return ResponsesHandler(+ settings,+ provider_getter=_provider_getter(request, settings),+ )+++def get_token_count_handler(+ settings: Settings = Depends(get_settings),+) -> TokenCountHandler:+ """Build the token-count product handler for route handlers."""+ return TokenCountHandler(settings, token_counter=get_token_count)++def _probe_response(allow: str) -> Response:"""Return an empty success response for compatibility probes."""return Response(status_code=204, headers={"Allow": allow})@@ -44,11 +66,11 @@ def _probe_response(allow: str) -> Response:@router.post("/v1/messages")async def create_message(request_data: MessagesRequest,- pipeline: ApiRequestPipeline = Depends(get_request_pipeline),+ handler: MessagesHandler = Depends(get_messages_handler),_auth=Depends(require_api_key),):"""Create a message (always streaming)."""- return pipeline.create_message(request_data)+ return handler.create(request_data)@router.api_route("/v1/messages", methods=["HEAD", "OPTIONS"])@@ -60,11 +82,11 @@ async def probe_messages(_auth=Depends(require_api_key)):@router.post("/v1/responses")async def create_response(request_data: OpenAIResponsesRequest,- pipeline: ApiRequestPipeline = Depends(get_request_pipeline),+ handler: ResponsesHandler = Depends(get_responses_handler),_auth=Depends(require_api_key),):"""Create an OpenAI Responses-compatible response through this proxy."""- return await pipeline.create_response(request_data)+ return await handler.create(request_data)@router.api_route("/v1/responses", methods=["HEAD", "OPTIONS"])@@ -76,11 +98,11 @@ async def probe_responses(_auth=Depends(require_api_key)):@router.post("/v1/messages/count_tokens")async def count_tokens(request_data: TokenCountRequest,- pipeline: ApiRequestPipeline = Depends(get_request_pipeline),+ handler: TokenCountHandler = Depends(get_token_count_handler),_auth=Depends(require_api_key),):"""Count tokens for a request."""- return pipeline.count_tokens(request_data)+ return handler.count(request_data)@router.api_route("/v1/messages/count_tokens", methods=["HEAD", "OPTIONS"])diff --git a/pyproject.toml b/pyproject.tomlindex cfc77f28..d0eb37dd 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -4,7 +4,7 @@ build-backend = "hatchling.build"[project]name = "free-claude-code"-version = "2.3.15"+version = "2.3.16"description = "Middleware between Claude Code CLI (Anthropic API) and NVIDIA NIM"readme = "README.md"requires-python = ">=3.14.0"diff --git a/smoke/capabilities.py b/smoke/capabilities.pyindex 2c5d080d..25d9805b 100644--- a/smoke/capabilities.py+++ b/smoke/capabilities.py@@ -29,7 +29,7 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = ("api_compatibility","routes_and_probes","anthropic_api_routes",- "api.request_pipeline.ApiRequestPipeline",+ "api.routes","Anthropic-compatible HTTP requests","JSON or Anthropic SSE responses","HTTPException or Anthropic error payload",@@ -51,7 +51,7 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = ("api_compatibility","client_request_shapes","drop_in_claude_code_replacement",- "api.request_pipeline.ApiRequestPipeline",+ "api.handlers.messages.MessagesHandler","Claude Code, VS Code, and JetBrains shaped requests","compatible streaming response","provider or validation error in Anthropic shape",@@ -62,7 +62,7 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = ("api_compatibility","responses_api","drop_in_codex_replacement",- "api.request_pipeline.ApiRequestPipeline.create_response",+ "api.handlers.responses.ResponsesHandler","OpenAI Responses requests from Codex","Responses SSE or JSON response","OpenAI-shaped error or conversion error",@@ -81,7 +81,7 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = ("api_compatibility","client_extensions","vscode_extension",- "api.request_pipeline.ApiRequestPipeline",+ "api.handlers.messages.MessagesHandler","VS Code beta query/header variants","accepted Anthropic-compatible response","HTTP error only for auth/provider failures",@@ -92,7 +92,7 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = ("api_compatibility","client_extensions","intellij_extension",- "api.request_pipeline.ApiRequestPipeline",+ "api.handlers.messages.MessagesHandler","JetBrains/ACP request variants","accepted Anthropic-compatible response","HTTP error only for auth/provider failures",diff --git a/tests/api/test_request_pipeline.py b/tests/api/test_api_handlers.pysimilarity index 73%rename from tests/api/test_request_pipeline.pyrename to tests/api/test_api_handlers.pyindex 95834786..beca4ee1 100644--- a/tests/api/test_request_pipeline.py+++ b/tests/api/test_api_handlers.py@@ -7,9 +7,9 @@ from unittest.mock import MagicMock, patchimport pytestfrom fastapi.responses import StreamingResponse-from api.models.anthropic import Message, MessagesRequest+from api.handlers import MessagesHandler, ResponsesHandler, TokenCountHandler+from api.models.anthropic import Message, MessagesRequest, TokenCountRequestfrom api.models.openai_responses import OpenAIResponsesRequest-from api.request_pipeline import ApiRequestPipelinefrom config.settings import Settingsfrom providers.base import BaseProvider, ProviderConfig@@ -79,16 +79,16 @@ def _trace_events(trace_mock: MagicMock, event: str) -> list[dict[str, Any]]:@pytest.mark.asyncio-async def test_pipeline_provider_execution_passes_routed_request_and_stream_metadata():+async def test_messages_handler_passes_routed_request_and_stream_metadata() -> None:provider = FakeProvider()- pipeline = ApiRequestPipeline(Settings(), provider_getter=lambda _: provider)+ handler = MessagesHandler(Settings(), provider_getter=lambda _: provider)request = MessagesRequest(model="nvidia_nim/test-model",max_tokens=100,messages=[Message(role="user", content="hi")],)- response = pipeline.create_message(request)+ response = handler.create(request)assert isinstance(response, StreamingResponse)body = await _streaming_body_text(response)@@ -101,9 +101,9 @@ async def test_pipeline_provider_execution_passes_routed_request_and_stream_meta@pytest.mark.asyncio-async def test_pipeline_forces_no_thinking_for_safety_classifier_messages():+async def test_messages_handler_forces_no_thinking_for_safety_classifier() -> None:provider = FakeProvider()- pipeline = ApiRequestPipeline(Settings(), provider_getter=lambda _: provider)+ handler = MessagesHandler(Settings(), provider_getter=lambda _: provider)request = MessagesRequest(model="nvidia_nim/test-model",max_tokens=100,@@ -111,8 +111,8 @@ async def test_pipeline_forces_no_thinking_for_safety_classifier_messages():messages=[Message(role="user", content=_CLASSIFIER_USER)],)- with patch("api.request_pipeline.trace_event") as trace_mock:- response = pipeline.create_message(request)+ with patch("api.handlers.messages.trace_event") as trace_mock:+ response = handler.create(request)assert isinstance(response, StreamingResponse)await _streaming_body_text(response)@@ -134,9 +134,9 @@ async def test_pipeline_forces_no_thinking_for_safety_classifier_messages():@pytest.mark.asyncio-async def test_pipeline_preserves_thinking_for_non_classifier_messages():+async def test_messages_handler_preserves_thinking_for_non_classifier() -> None:provider = FakeProvider()- pipeline = ApiRequestPipeline(Settings(), provider_getter=lambda _: provider)+ handler = MessagesHandler(Settings(), provider_getter=lambda _: provider)request = MessagesRequest(model="nvidia_nim/test-model",max_tokens=100,@@ -152,8 +152,8 @@ async def test_pipeline_preserves_thinking_for_non_classifier_messages():],)- with patch("api.request_pipeline.trace_event") as trace_mock:- response = pipeline.create_message(request)+ with patch("api.handlers.messages.trace_event") as trace_mock:+ response = handler.create(request)assert isinstance(response, StreamingResponse)await _streaming_body_text(response)@@ -166,9 +166,9 @@ async def test_pipeline_preserves_thinking_for_non_classifier_messages():@pytest.mark.asyncio-async def test_pipeline_keeps_existing_no_thinking_for_classifier_messages():+async def test_messages_handler_keeps_existing_no_thinking_for_classifier() -> None:provider = FakeProvider()- pipeline = ApiRequestPipeline(Settings(), provider_getter=lambda _: provider)+ handler = MessagesHandler(Settings(), provider_getter=lambda _: provider)request = MessagesRequest(model="claude-3-freecc-no-thinking/nvidia_nim/test-model",max_tokens=100,@@ -176,8 +176,8 @@ async def test_pipeline_keeps_existing_no_thinking_for_classifier_messages():messages=[Message(role="user", content=_CLASSIFIER_USER)],)- with patch("api.request_pipeline.trace_event") as trace_mock:- response = pipeline.create_message(request)+ with patch("api.handlers.messages.trace_event") as trace_mock:+ response = handler.create(request)assert isinstance(response, StreamingResponse)await _streaming_body_text(response)@@ -196,9 +196,9 @@ async def test_pipeline_keeps_existing_no_thinking_for_classifier_messages():]-def test_pipeline_message_optimization_intercepts_before_provider_execution():+def test_messages_handler_optimization_intercepts_before_provider_execution() -> None:provider_getter = MagicMock()- pipeline = ApiRequestPipeline(Settings(), provider_getter=provider_getter)+ handler = MessagesHandler(Settings(), provider_getter=provider_getter)request = MessagesRequest(model="nvidia_nim/test-model",max_tokens=100,@@ -206,23 +206,23 @@ def test_pipeline_message_optimization_intercepts_before_provider_execution():)optimized = object()- with patch("api.request_pipeline.try_optimizations", return_value=optimized):- assert pipeline.create_message(request) is optimized+ with patch("api.handlers.messages.try_optimizations", return_value=optimized):+ assert handler.create(request) is optimizedprovider_getter.assert_not_called()@pytest.mark.asyncio-async def test_pipeline_responses_bypass_message_only_optimizations():+async def test_responses_handler_bypasses_message_only_optimizations() -> None:provider = FakeProvider()- pipeline = ApiRequestPipeline(Settings(), provider_getter=lambda _: provider)+ handler = ResponsesHandler(Settings(), provider_getter=lambda _: provider)with patch(- "api.request_pipeline.try_optimizations",+ "api.handlers.messages.try_optimizations",side_effect=AssertionError("Responses must not use message optimizations"),):- response = await pipeline.create_response(- request_data=OpenAIResponsesRequest(+ response = await handler.create(+ OpenAIResponsesRequest(model="nvidia_nim/test-model",input="quota check",)@@ -235,13 +235,13 @@ async def test_pipeline_responses_bypass_message_only_optimizations():@pytest.mark.asyncio-async def test_pipeline_responses_do_not_apply_safety_classifier_policy():+async def test_responses_handler_does_not_apply_safety_classifier_policy() -> None:provider = FakeProvider()- pipeline = ApiRequestPipeline(Settings(), provider_getter=lambda _: provider)+ handler = ResponsesHandler(Settings(), provider_getter=lambda _: provider)- with patch("api.request_pipeline.trace_event") as trace_mock:- response = await pipeline.create_response(- request_data=OpenAIResponsesRequest(+ with patch("api.handlers.messages.trace_event") as trace_mock:+ response = await handler.create(+ OpenAIResponsesRequest(model="nvidia_nim/test-model",input=_CLASSIFIER_USER,instructions=_CLASSIFIER_SYSTEM,@@ -257,3 +257,19 @@ async def test_pipeline_responses_do_not_apply_safety_classifier_policy():_trace_events(trace_mock, "api.optimization.safety_classifier_no_thinking")== [])+++def test_token_count_handler_routes_and_counts_tokens() -> None:+ handler = TokenCountHandler(+ Settings(),+ token_counter=lambda messages, system, tools: len(messages) + 41,+ )++ response = handler.count(+ TokenCountRequest(+ model="nvidia_nim/test-model",+ messages=[Message(role="user", content="hi")],+ )+ )++ assert response.input_tokens == 42diff --git a/tests/api/test_openai_responses.py b/tests/api/test_openai_responses.pyindex c29c4aaf..909a11ca 100644--- a/tests/api/test_openai_responses.py+++ b/tests/api/test_openai_responses.py@@ -81,7 +81,7 @@ def test_create_response_stream_bypasses_local_message_optimizations() -> None:with (patch("api.dependencies.resolve_provider", return_value=provider),patch(- "api.request_pipeline.try_optimizations",+ "api.handlers.messages.try_optimizations",side_effect=AssertionError("Responses must not use message optimizations"),),TestClient(app) as client,diff --git a/tests/api/test_safe_logging.py b/tests/api/test_safe_logging.pyindex f45bbcb6..aa685f07 100644--- a/tests/api/test_safe_logging.py+++ b/tests/api/test_safe_logging.py@@ -7,9 +7,9 @@ from unittest.mock import MagicMock, patchimport pytestfrom fastapi import HTTPException-from api import request_pipeline as pipeline_mod+from api import provider_execution, request_errors+from api.handlers import MessagesHandler, TokenCountHandlerfrom api.models.anthropic import Message, MessagesRequest-from api.request_pipeline import ApiRequestPipelinefrom config.settings import Settingsfrom core.anthropic import AnthropicStreamLedger@@ -23,7 +23,7 @@ def test_create_message_skips_full_payload_debug_log_by_default():yield "event: ping\ndata: {}\n\n"mock_provider.stream_response = fake_stream- service = ApiRequestPipeline(settings, provider_getter=lambda _: mock_provider)+ service = MessagesHandler(settings, provider_getter=lambda _: mock_provider)request = MessagesRequest(model="claude-3-haiku-20240307",@@ -31,8 +31,8 @@ def test_create_message_skips_full_payload_debug_log_by_default():messages=[Message(role="user", content="secret-user-text")],)- with patch.object(pipeline_mod.logger, "debug") as mock_debug:- service.create_message(request)+ with patch.object(provider_execution.logger, "debug") as mock_debug:+ service.create(request)full_payload_calls = [c@@ -51,15 +51,15 @@ def test_create_message_logs_full_payload_when_opt_in():yield "event: ping\ndata: {}\n\n"mock_provider.stream_response = fake_stream- service = ApiRequestPipeline(settings, provider_getter=lambda _: mock_provider)+ service = MessagesHandler(settings, provider_getter=lambda _: mock_provider)request = MessagesRequest(model="claude-3-haiku-20240307",max_tokens=10,messages=[Message(role="user", content="visible")],)- with patch.object(pipeline_mod.logger, "debug") as mock_debug:- service.create_message(request)+ with patch.object(provider_execution.logger, "debug") as mock_debug:+ service.create(request)keys = [c.args[0] for c in mock_debug.call_args_list if c.args]assert any(k == "FULL_PAYLOAD [{}]: {}" for k in keys)@@ -103,7 +103,7 @@ def test_create_message_unexpected_error_default_logs_exclude_exception_text():raise RuntimeError(secret)mock_provider.stream_response = stream_boom- service = ApiRequestPipeline(settings, provider_getter=lambda _: mock_provider)+ service = MessagesHandler(settings, provider_getter=lambda _: mock_provider)request = MessagesRequest(model="claude-3-haiku-20240307",max_tokens=10,@@ -111,10 +111,10 @@ def test_create_message_unexpected_error_default_logs_exclude_exception_text():)with (- patch.object(pipeline_mod.logger, "error") as log_err,+ patch.object(request_errors.logger, "error") as log_err,pytest.raises(HTTPException),):- service.create_message(request)+ service.create(request)blob = _flatten_log_calls(log_err)assert secret not in blob@@ -134,7 +134,7 @@ def test_create_message_unexpected_error_always_returns_500():raise WeirdError("no")mock_provider.stream_response = stream_boom- service = ApiRequestPipeline(settings, provider_getter=lambda _: mock_provider)+ service = MessagesHandler(settings, provider_getter=lambda _: mock_provider)request = MessagesRequest(model="claude-3-haiku-20240307",max_tokens=10,@@ -142,7 +142,7 @@ def test_create_message_unexpected_error_always_returns_500():)with pytest.raises(HTTPException) as excinfo:- service.create_message(request)+ service.create(request)assert excinfo.value.status_code == 500@@ -181,9 +181,8 @@ def test_count_tokens_unexpected_error_default_logs_exclude_exception_text():def boom(*_a, **_kw):raise ValueError(secret)- service = ApiRequestPipeline(+ service = TokenCountHandler(settings,- provider_getter=lambda _: MagicMock(),token_counter=boom,)from api.models.anthropic import TokenCountRequest@@ -194,10 +193,10 @@ def test_count_tokens_unexpected_error_default_logs_exclude_exception_text():)with (- patch.object(pipeline_mod.logger, "error") as log_err,+ patch.object(request_errors.logger, "error") as log_err,pytest.raises(HTTPException),):- service.count_tokens(req)+ service.count(req)blob = _flatten_log_calls(log_err)assert secret not in blobdiff --git a/tests/api/test_web_server_tools.py b/tests/api/test_web_server_tools.pyindex 140b8040..6ca12301 100644--- a/tests/api/test_web_server_tools.py+++ b/tests/api/test_web_server_tools.py@@ -5,9 +5,9 @@ import httpximport pytestimport api.web_tools.constants as web_tool_constants+from api.handlers import MessagesHandlerfrom api.model_router import ModelRouter, ResolvedModel, RoutedMessagesRequestfrom api.models.anthropic import Message, MessagesRequest, Tool-from api.request_pipeline import ApiRequestPipelinefrom api.web_tools import egress as web_egressfrom api.web_tools.egress import (WebFetchEgressPolicy,@@ -112,7 +112,7 @@ def test_service_rejects_forced_server_tool_on_openai_when_disabled("""OpenAI Chat upstreams cannot run forced server tools without the local handler."""settings = Settings()assert settings.enable_web_server_tools is False- service = ApiRequestPipeline(+ service = MessagesHandler(settings,provider_getter=lambda _: MagicMock(),model_router=FixedProviderModelRouter(settings, provider_id),@@ -130,7 +130,7 @@ def test_service_rejects_forced_server_tool_on_openai_when_disabled(tool_choice={"type": "tool", "name": "web_search"},)with pytest.raises(InvalidRequestError, match="ENABLE_WEB_SERVER_TOOLS"):- service.create_message(request)+ service.create(request)@pytest.mark.parametrize(@@ -598,7 +598,7 @@ def test_service_rejects_listed_server_tools_on_openai_chat(provider_id: str,) -> None:settings = Settings()- service = ApiRequestPipeline(+ service = MessagesHandler(settings,provider_getter=lambda _: MagicMock(),model_router=FixedProviderModelRouter(settings, provider_id),@@ -610,7 +610,7 @@ def test_service_rejects_listed_server_tools_on_openai_chat(tools=[Tool(name="web_search", type="web_search_20250305")],)with pytest.raises(InvalidRequestError, match="OpenAI Chat upstreams"):- service.create_message(request)+ service.create(request)@pytest.mark.parametrize("provider_id", _ANTHROPIC_MESSAGES_PROVIDER_IDS)@@ -626,7 +626,7 @@ def test_listed_server_tools_routed_on_anthropic_messages_providers(mock_provider = MagicMock()mock_provider.stream_response = fake_stream- service = ApiRequestPipeline(+ service = MessagesHandler(settings,provider_getter=lambda _: mock_provider,model_router=FixedProviderModelRouter(settings, provider_id),@@ -637,7 +637,7 @@ def test_listed_server_tools_routed_on_anthropic_messages_providers(messages=[Message(role="user", content="q")],tools=[Tool(name="web_search", type="web_search_20250305")],)- service.create_message(request)+ service.create(request)mock_provider.preflight_stream.assert_called()@@ -654,7 +654,7 @@ def test_forced_server_tools_routed_on_anthropic_messages_providers_when_local_dmock_provider = MagicMock()mock_provider.stream_response = fake_stream- service = ApiRequestPipeline(+ service = MessagesHandler(settings,provider_getter=lambda _: mock_provider,model_router=FixedProviderModelRouter(settings, provider_id),@@ -666,5 +666,5 @@ def test_forced_server_tools_routed_on_anthropic_messages_providers_when_local_dtools=[Tool(name="web_search", type="web_search_20250305")],tool_choice={"type": "tool", "name": "web_search"},)- service.create_message(request)+ service.create(request)mock_provider.preflight_stream.assert_called()diff --git a/tests/contracts/test_capability_map.py b/tests/contracts/test_capability_map.pyindex 4b6e41a2..ccf3e8f2 100644--- a/tests/contracts/test_capability_map.py+++ b/tests/contracts/test_capability_map.py@@ -46,3 +46,14 @@ def test_capability_contracts_are_decision_complete() -> None:assert contract.failure.strip(), contractassert contract.pytest_tests or contract.smoke_tests, contract+++def test_capability_contract_owners_do_not_reference_removed_request_pipeline() -> None:+ stale = [+ contract+ for contract in CAPABILITY_CONTRACTS+ if "api.request_pipeline" in contract.owner+ or "ApiRequestPipeline" in contract.owner+ ]++ assert stale == []diff --git a/tests/contracts/test_import_boundaries.py b/tests/contracts/test_import_boundaries.pyindex 109a9555..2b224216 100644--- a/tests/contracts/test_import_boundaries.py+++ b/tests/contracts/test_import_boundaries.py@@ -203,8 +203,11 @@ def test_anthropic_stream_engine_owns_provider_stream_state() -> None:def test_openai_responses_uses_adapter_boundary() -> None:repo_root = Path(__file__).resolve().parents[2]responses_root = repo_root / "core" / "openai_responses"+ api_root = repo_root / "api"+ handlers_root = api_root / "handlers"assert not (repo_root / "api" / "services.py").exists()+ assert not (api_root / "request_pipeline.py").exists()assert not (responses_root / "conversion.py").exists()assert not (responses_root / "sse.py").exists()assert not (responses_root / "output.py").exists()@@ -223,12 +226,16 @@ def test_openai_responses_uses_adapter_boundary() -> None:}:assert (responses_root / filename).exists()- pipeline_text = (repo_root / "api" / "request_pipeline.py").read_text(- encoding="utf-8"+ responses_handler = handlers_root / "responses.py"+ responses_handler_text = responses_handler.read_text(encoding="utf-8")+ assert (+ "from core.openai_responses import OpenAIResponsesAdapter"+ in responses_handler_text)- assert "from core.openai_responses import OpenAIResponsesAdapter" in pipeline_textroutes_text = (repo_root / "api" / "routes.py").read_text(encoding="utf-8")- assert "ApiRequestPipeline" in routes_text+ assert "ApiRequestPipeline" not in routes_text+ assert "request_pipeline" not in routes_text+ assert "from .handlers import" in routes_textassert "api.services" not in routes_textfor old_helper in {"responses_request_to_anthropic_payload",@@ -237,7 +244,7 @@ def test_openai_responses_uses_adapter_boundary() -> None:"collect_openai_response_from_anthropic_sse","iter_message_response_as_openai_responses",}:- assert old_helper not in pipeline_text+ assert old_helper not in responses_handler_textoffenders: list[str] = []for path in (repo_root / "api").rglob("*.py"):@@ -247,6 +254,27 @@ def test_openai_responses_uses_adapter_boundary() -> None:offenders.append(f"{rel}: {imported}")assert sorted(offenders) == []+ adapter_importers: list[str] = []+ for path in (repo_root / "api").rglob("*.py"):+ imports = set(_imports_from(path, repo_root))+ if "core.openai_responses" in imports:+ adapter_importers.append(path.relative_to(repo_root).as_posix())+ assert sorted(adapter_importers) == ["api/handlers/responses.py"]++ response_handler_imports = set(_imports_from(responses_handler, repo_root))+ for forbidden in {+ "api.optimization_handlers",+ "api.detection",+ "api.web_tools",+ }:+ assert forbidden not in response_handler_imports++ provider_execution_text = (api_root / "provider_execution.py").read_text(+ encoding="utf-8"+ )+ assert "StreamingResponse" not in provider_execution_text+ assert "OpenAIResponsesAdapter" not in provider_execution_text+adapter_text = (responses_root / "adapter.py").read_text(encoding="utf-8")for deleted_api in {"from_anthropic_message",diff --git a/uv.lock b/uv.lockindex e73cf773..a0b33736 100644--- a/uv.lock+++ b/uv.lock@@ -561,7 +561,7 @@ wheels = [[[package]]name = "free-claude-code"-version = "2.3.15"+version = "2.3.16"source = { editable = "." }dependencies = [{ name = "aiohttp" },