Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)
Python
Updated README with per-model mapping, fixed test .env isolation
Alishahryar1a7d88d5cbd22
diff --git a/README.md b/README.mdindex 088eb306..ddba8dee 100644--- a/README.md+++ b/README.md@@ -32,7 +32,7 @@ A lightweight proxy that routes Claude Code's Anthropic API calls to **NVIDIA NI| **Zero Cost** | 40 req/min free on NVIDIA NIM. Free models on OpenRouter. Fully local with LM Studio || **Drop-in Replacement** | Set 2 env vars. No modifications to Claude Code CLI or VSCode extension needed || **3 Providers** | NVIDIA NIM, OpenRouter (hundreds of models), LM Studio (local & offline) |-| **Per-Tier Model Mapping** | Route Opus / Sonnet / Haiku requests to different models and providers. Mix providers freely per tier |+| **Per-Model Mapping** | Route Opus / Sonnet / Haiku requests to different models and providers. Mix providers freely per model || **Thinking Token Support** | Parses `<think>` tags and `reasoning_content` into native Claude thinking blocks || **Heuristic Tool Parser** | Models outputting tool calls as text are auto-parsed into structured tool use || **Request Optimization** | 5 categories of trivial API calls intercepted locally, saving quota and latency |@@ -68,7 +68,11 @@ Choose your provider and edit `.env`:```dotenvNVIDIA_NIM_API_KEY="nvapi-your-key-here"-MODEL="nvidia_nim/z-ai/glm4.7"++MODEL_OPUS="nvidia_nim/moonshotai/kimi-k2.5"+MODEL_SONNET="nvidia_nim/z-ai/glm5"+MODEL_HAIKU="nvidia_nim/stepfun-ai/step-3.5-flash"+MODEL="nvidia_nim/z-ai/glm4.7" # fallback```</details>@@ -78,7 +82,11 @@ MODEL="nvidia_nim/z-ai/glm4.7"```dotenvOPENROUTER_API_KEY="sk-or-your-key-here"-MODEL="open_router/stepfun/step-3.5-flash:free"++MODEL_OPUS="open_router/deepseek/deepseek-r1-0528:free"+MODEL_SONNET="open_router/openai/gpt-oss-120b:free"+MODEL_HAIKU="open_router/stepfun/step-3.5-flash:free"+MODEL="open_router/stepfun/step-3.5-flash:free" # fallback```</details>@@ -87,24 +95,27 @@ MODEL="open_router/stepfun/step-3.5-flash:free"<summary><b>LM Studio</b> (fully local, no API key)</summary>```dotenv-MODEL="lmstudio/lmstudio-community/qwen2.5-7b-instruct"+MODEL_OPUS="lmstudio/unsloth/MiniMax-M2.5-GGUF"+MODEL_SONNET="lmstudio/unsloth/Qwen3.5-35B-A3B-GGUF"+MODEL_HAIKU="lmstudio/unsloth/GLM-4.7-Flash-GGUF"+MODEL="lmstudio/unsloth/GLM-4.7-Flash-GGUF" # fallback```</details><details>-<summary><b>Per-Tier Model Mapping</b> (mix providers per Claude tier)</summary>+<summary><b>Mix providers</b> (use multiple providers together)</summary>-Claude Code sends requests tagged as Opus, Sonnet, or Haiku. You can route each tier to a different model — even across different providers. `MODEL` acts as the fallback when a tier-specific variable is not set.+Each `MODEL_*` variable can use a different provider. `MODEL` is the fallback for unrecognized Claude models.```dotenvNVIDIA_NIM_API_KEY="nvapi-your-key-here"OPENROUTER_API_KEY="sk-or-your-key-here"-MODEL_OPUS="nvidia_nim/z-ai/glm4.7"-MODEL_SONNET="open_router/arcee-ai/trinity-large-preview:free"-MODEL_HAIKU="open_router/stepfun/step-3.5-flash:free"-MODEL="nvidia_nim/z-ai/glm4.7" # fallback for unrecognized tiers+MODEL_OPUS="nvidia_nim/moonshotai/kimi-k2.5"+MODEL_SONNET="open_router/deepseek/deepseek-r1-0528:free"+MODEL_HAIKU="lmstudio/unsloth/GLM-4.7-Flash-GGUF"+MODEL="nvidia_nim/z-ai/glm4.7" # fallback```</details>@@ -198,7 +209,7 @@ alias claude-kimi='ANTHROPIC_BASE_URL="http://localhost:8082" ANTHROPIC_AUTH_TOK```- **Transparent proxy**: Claude Code sends standard Anthropic API requests to the proxy server-- **Per-tier model routing**: Opus / Sonnet / Haiku requests are resolved to their tier-specific model and provider, with the default `MODEL` as fallback+- **Per-model routing**: Opus / Sonnet / Haiku requests are resolved to their model-specific backend and provider, with the default `MODEL` as fallback- **Request optimization**: 5 categories of trivial requests (quota probes, title generation, prefix detection, suggestions, filepath extraction) are intercepted and responded to instantly without using API quota- **Format translation**: real requests are translated from Anthropic format to the provider's OpenAI-compatible format and streamed back- **Thinking tokens**: `<think>` tags and `reasoning_content` fields are converted into native Claude thinking blocks so Claude Code renders them correctly@@ -390,9 +401,9 @@ Browse: [model.lmstudio.ai](https://model.lmstudio.ai)| Variable | Description | Default || --------------------------------- | ---------------------------------------------------------------------------------- | -------------------------------------- || `MODEL` | Fallback model (prefix format: `provider/model/name`; invalid prefix causes error) | `nvidia_nim/stepfun-ai/step-3.5-flash` |-| `MODEL_OPUS` | Model for Claude Opus tier requests (optional, falls back to `MODEL`) | — |-| `MODEL_SONNET` | Model for Claude Sonnet tier requests (optional, falls back to `MODEL`) | — |-| `MODEL_HAIKU` | Model for Claude Haiku tier requests (optional, falls back to `MODEL`) | — |+| `MODEL_OPUS` | Model for Claude Opus requests (optional, falls back to `MODEL`) | — |+| `MODEL_SONNET` | Model for Claude Sonnet requests (optional, falls back to `MODEL`) | — |+| `MODEL_HAIKU` | Model for Claude Haiku requests (optional, falls back to `MODEL`) | — || `NVIDIA_NIM_API_KEY` | NVIDIA API key (NIM provider) | required || `OPENROUTER_API_KEY` | OpenRouter API key (OpenRouter provider) | required || `LM_STUDIO_BASE_URL` | LM Studio server URL | `http://localhost:1234/v1` |diff --git a/api/models/anthropic.py b/api/models/anthropic.pyindex 87f4698a..c03cd359 100644--- a/api/models/anthropic.py+++ b/api/models/anthropic.py@@ -102,7 +102,7 @@ class MessagesRequest(BaseModel):@model_validator(mode="after")def map_model(self) -> MessagesRequest:- """Map any Claude model name to the configured model (tier-aware)."""+ """Map any Claude model name to the configured model (model-aware)."""settings = get_settings()if self.original_model is None:self.original_model = self.model@@ -128,7 +128,7 @@ class TokenCountRequest(BaseModel):@field_validator("model")@classmethoddef validate_model_field(cls, v: str, info) -> str:- """Map any Claude model name to the configured model (tier-aware)."""+ """Map any Claude model name to the configured model (model-aware)."""settings = get_settings()resolved_full = settings.resolve_model(v)return Settings.parse_model_name(resolved_full)diff --git a/api/routes.py b/api/routes.pyindex 2c5f2df3..105bc210 100644--- a/api/routes.py+++ b/api/routes.py@@ -41,7 +41,7 @@ async def create_message(if optimized is not None:return optimized- # Resolve provider from the tier-aware model mapping+ # Resolve provider from the model-aware mappingprovider_type = Settings.parse_provider_type(request_data.resolved_provider_model or settings.model)diff --git a/config/settings.py b/config/settings.pyindex 7c30507e..cdff8f80 100644--- a/config/settings.py+++ b/config/settings.py@@ -2,14 +2,11 @@from functools import lru_cache-from dotenv import load_dotenvfrom pydantic import Field, field_validator, model_validatorfrom pydantic_settings import BaseSettings, SettingsConfigDictfrom .nim import NimSettings-load_dotenv()-class Settings(BaseSettings):"""Application settings loaded from environment variables."""@@ -37,7 +34,7 @@ class Settings(BaseSettings):# Format: provider_type/model/namemodel: str = "nvidia_nim/meta/llama3-70b-instruct"- # Per-tier model overrides (optional, falls back to MODEL)+ # Per-model overrides (optional, falls back to MODEL)# Each can use a different providermodel_opus: str | None = Field(default=None, validation_alias="MODEL_OPUS")model_sonnet: str | None = Field(default=None, validation_alias="MODEL_SONNET")@@ -176,8 +173,8 @@ class Settings(BaseSettings):def resolve_model(self, claude_model_name: str) -> str:"""Resolve a Claude model name to the configured provider/model string.- Classifies the incoming model into a tier (opus/sonnet/haiku) and- returns the tier-specific model if configured, otherwise the fallback MODEL.+ Classifies the incoming Claude model (opus/sonnet/haiku) and+ returns the model-specific override if configured, otherwise the fallback MODEL."""name_lower = claude_model_name.lower()if "opus" in name_lower and self.model_opus is not None:diff --git a/tests/api/test_models_validators.py b/tests/api/test_models_validators.pyindex bd567018..b6089dea 100644--- a/tests/api/test_models_validators.py+++ b/tests/api/test_models_validators.py@@ -10,6 +10,9 @@ from config.settings import Settingsdef mock_settings():settings = Settings()settings.model = "nvidia_nim/target-model-from-settings"+ settings.model_opus = None+ settings.model_sonnet = None+ settings.model_haiku = Nonereturn settings@@ -76,7 +79,7 @@ def test_messages_request_resolved_provider_model_default(mock_settings):)-def test_messages_request_tier_aware_opus_override():+def test_messages_request_model_aware_opus_override():"""Opus model routes to MODEL_OPUS when set."""settings = Settings()settings.model = "nvidia_nim/fallback-model"@@ -93,7 +96,7 @@ def test_messages_request_tier_aware_opus_override():assert request.original_model == "claude-opus-4-20250514"-def test_messages_request_tier_aware_haiku_override():+def test_messages_request_model_aware_haiku_override():"""Haiku model routes to MODEL_HAIKU when set."""settings = Settings()settings.model = "nvidia_nim/fallback-model"@@ -109,7 +112,7 @@ def test_messages_request_tier_aware_haiku_override():assert request.resolved_provider_model == "lmstudio/qwen2.5-7b"-def test_messages_request_tier_aware_sonnet_override():+def test_messages_request_model_aware_sonnet_override():"""Sonnet model routes to MODEL_SONNET when set."""settings = Settings()settings.model = "nvidia_nim/fallback-model"@@ -127,10 +130,13 @@ def test_messages_request_tier_aware_sonnet_override():)-def test_messages_request_tier_fallback_when_not_set():- """When tier override is None, falls back to MODEL."""+def test_messages_request_model_fallback_when_not_set():+ """When model override is None, falls back to MODEL."""settings = Settings()settings.model = "nvidia_nim/fallback-model"+ settings.model_opus = None+ settings.model_sonnet = None+ settings.model_haiku = None# model_opus is Nonewith patch("api.models.anthropic.get_settings", return_value=settings):@@ -143,8 +149,8 @@ def test_messages_request_tier_fallback_when_not_set():assert request.resolved_provider_model == "nvidia_nim/fallback-model"-def test_token_count_request_tier_aware():- """TokenCountRequest also uses tier-aware resolution."""+def test_token_count_request_model_aware():+ """TokenCountRequest also uses model-aware resolution."""settings = Settings()settings.model = "nvidia_nim/fallback-model"settings.model_haiku = "lmstudio/qwen2.5-7b"diff --git a/tests/config/test_config.py b/tests/config/test_config.pyindex f45df73d..10862eed 100644--- a/tests/config/test_config.py+++ b/tests/config/test_config.py@@ -310,11 +310,11 @@ class TestSettingsOptionalStr:assert s.whisper_device == device-class TestPerTierModelMapping:- """Test per-tier model fields and resolve_model()."""+class TestPerModelMapping:+ """Test per-model fields and resolve_model()."""- def test_tier_fields_default_none(self):- """Per-tier model fields default to None."""+ def test_model_fields_default_none(self):+ """Per-model fields default to None."""from config.settings import Settingss = Settings()@@ -411,20 +411,20 @@ class TestPerTierModelMapping:assert s.resolve_model("claude-3-5-haiku-20241022") == "lmstudio/qwen2.5-7b"assert s.resolve_model("claude-haiku-4-20250514") == "lmstudio/qwen2.5-7b"- def test_resolve_model_fallback_when_tier_not_set(self):- """resolve_model falls back to MODEL when tier override is None."""+ def test_resolve_model_fallback_when_override_not_set(self):+ """resolve_model falls back to MODEL when model override is None."""from config.settings import Settingss = Settings()s.model = "nvidia_nim/fallback-model"- # No tier overrides set+ # No model overrides setassert s.resolve_model("claude-opus-4-20250514") == "nvidia_nim/fallback-model"assert (s.resolve_model("claude-sonnet-4-20250514") == "nvidia_nim/fallback-model")assert s.resolve_model("claude-3-haiku-20240307") == "nvidia_nim/fallback-model"- def test_resolve_model_unknown_tier_falls_back(self):+ def test_resolve_model_unknown_model_falls_back(self):"""resolve_model falls back to MODEL for unrecognized model names."""from config.settings import Settings@@ -435,7 +435,7 @@ class TestPerTierModelMapping:assert s.resolve_model("some-unknown-model") == "nvidia_nim/fallback-model"def test_resolve_model_case_insensitive(self):- """Tier classification is case-insensitive."""+ """Model classification is case-insensitive."""from config.settings import Settingss = Settings()diff --git a/tests/conftest.py b/tests/conftest.pyindex c4f837c7..d64e6b23 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -29,6 +29,16 @@ from providers.base import ProviderConfigfrom providers.nvidia_nim import NvidiaNimProvider[email protected](autouse=True)+def _isolate_from_dotenv(monkeypatch):+ """Prevent Pydantic BaseSettings from reading the .env file during tests."""+ from config.settings import Settings++ monkeypatch.setattr(+ Settings, "model_config", {**Settings.model_config, "env_file": None}+ )++@pytest.fixturedef provider_config():return ProviderConfig(