Skip to content

Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

Python

Updated README with per-model mapping, fixed test .env isolation

Alishahryar1a7d88d5cbd22
diff --git a/README.md b/README.md
index 088eb306..ddba8dee 100644
--- a/README.md
+++ b/README.md
@@ -32,7 +32,7 @@ A lightweight proxy that routes Claude Code's Anthropic API calls to **NVIDIA NI
| **Zero Cost** | 40 req/min free on NVIDIA NIM. Free models on OpenRouter. Fully local with LM Studio |
| **Drop-in Replacement** | Set 2 env vars. No modifications to Claude Code CLI or VSCode extension needed |
| **3 Providers** | NVIDIA NIM, OpenRouter (hundreds of models), LM Studio (local & offline) |
-| **Per-Tier Model Mapping** | Route Opus / Sonnet / Haiku requests to different models and providers. Mix providers freely per tier |
+| **Per-Model Mapping** | Route Opus / Sonnet / Haiku requests to different models and providers. Mix providers freely per model |
| **Thinking Token Support** | Parses `<think>` tags and `reasoning_content` into native Claude thinking blocks |
| **Heuristic Tool Parser** | Models outputting tool calls as text are auto-parsed into structured tool use |
| **Request Optimization** | 5 categories of trivial API calls intercepted locally, saving quota and latency |
@@ -68,7 +68,11 @@ Choose your provider and edit `.env`:
```dotenv
NVIDIA_NIM_API_KEY="nvapi-your-key-here"
-MODEL="nvidia_nim/z-ai/glm4.7"
+
+MODEL_OPUS="nvidia_nim/moonshotai/kimi-k2.5"
+MODEL_SONNET="nvidia_nim/z-ai/glm5"
+MODEL_HAIKU="nvidia_nim/stepfun-ai/step-3.5-flash"
+MODEL="nvidia_nim/z-ai/glm4.7" # fallback
```
</details>
@@ -78,7 +82,11 @@ MODEL="nvidia_nim/z-ai/glm4.7"
```dotenv
OPENROUTER_API_KEY="sk-or-your-key-here"
-MODEL="open_router/stepfun/step-3.5-flash:free"
+
+MODEL_OPUS="open_router/deepseek/deepseek-r1-0528:free"
+MODEL_SONNET="open_router/openai/gpt-oss-120b:free"
+MODEL_HAIKU="open_router/stepfun/step-3.5-flash:free"
+MODEL="open_router/stepfun/step-3.5-flash:free" # fallback
```
</details>
@@ -87,24 +95,27 @@ MODEL="open_router/stepfun/step-3.5-flash:free"
<summary><b>LM Studio</b> (fully local, no API key)</summary>
```dotenv
-MODEL="lmstudio/lmstudio-community/qwen2.5-7b-instruct"
+MODEL_OPUS="lmstudio/unsloth/MiniMax-M2.5-GGUF"
+MODEL_SONNET="lmstudio/unsloth/Qwen3.5-35B-A3B-GGUF"
+MODEL_HAIKU="lmstudio/unsloth/GLM-4.7-Flash-GGUF"
+MODEL="lmstudio/unsloth/GLM-4.7-Flash-GGUF" # fallback
```
</details>
<details>
-<summary><b>Per-Tier Model Mapping</b> (mix providers per Claude tier)</summary>
+<summary><b>Mix providers</b> (use multiple providers together)</summary>
-Claude Code sends requests tagged as Opus, Sonnet, or Haiku. You can route each tier to a different model — even across different providers. `MODEL` acts as the fallback when a tier-specific variable is not set.
+Each `MODEL_*` variable can use a different provider. `MODEL` is the fallback for unrecognized Claude models.
```dotenv
NVIDIA_NIM_API_KEY="nvapi-your-key-here"
OPENROUTER_API_KEY="sk-or-your-key-here"
-MODEL_OPUS="nvidia_nim/z-ai/glm4.7"
-MODEL_SONNET="open_router/arcee-ai/trinity-large-preview:free"
-MODEL_HAIKU="open_router/stepfun/step-3.5-flash:free"
-MODEL="nvidia_nim/z-ai/glm4.7" # fallback for unrecognized tiers
+MODEL_OPUS="nvidia_nim/moonshotai/kimi-k2.5"
+MODEL_SONNET="open_router/deepseek/deepseek-r1-0528:free"
+MODEL_HAIKU="lmstudio/unsloth/GLM-4.7-Flash-GGUF"
+MODEL="nvidia_nim/z-ai/glm4.7" # fallback
```
</details>
@@ -198,7 +209,7 @@ alias claude-kimi='ANTHROPIC_BASE_URL="http://localhost:8082" ANTHROPIC_AUTH_TOK
```
- **Transparent proxy**: Claude Code sends standard Anthropic API requests to the proxy server
-- **Per-tier model routing**: Opus / Sonnet / Haiku requests are resolved to their tier-specific model and provider, with the default `MODEL` as fallback
+- **Per-model routing**: Opus / Sonnet / Haiku requests are resolved to their model-specific backend and provider, with the default `MODEL` as fallback
- **Request optimization**: 5 categories of trivial requests (quota probes, title generation, prefix detection, suggestions, filepath extraction) are intercepted and responded to instantly without using API quota
- **Format translation**: real requests are translated from Anthropic format to the provider's OpenAI-compatible format and streamed back
- **Thinking tokens**: `<think>` tags and `reasoning_content` fields are converted into native Claude thinking blocks so Claude Code renders them correctly
@@ -390,9 +401,9 @@ Browse: [model.lmstudio.ai](https://model.lmstudio.ai)
| Variable | Description | Default |
| --------------------------------- | ---------------------------------------------------------------------------------- | -------------------------------------- |
| `MODEL` | Fallback model (prefix format: `provider/model/name`; invalid prefix causes error) | `nvidia_nim/stepfun-ai/step-3.5-flash` |
-| `MODEL_OPUS` | Model for Claude Opus tier requests (optional, falls back to `MODEL`) | — |
-| `MODEL_SONNET` | Model for Claude Sonnet tier requests (optional, falls back to `MODEL`) | — |
-| `MODEL_HAIKU` | Model for Claude Haiku tier requests (optional, falls back to `MODEL`) | — |
+| `MODEL_OPUS` | Model for Claude Opus requests (optional, falls back to `MODEL`) | — |
+| `MODEL_SONNET` | Model for Claude Sonnet requests (optional, falls back to `MODEL`) | — |
+| `MODEL_HAIKU` | Model for Claude Haiku requests (optional, falls back to `MODEL`) | — |
| `NVIDIA_NIM_API_KEY` | NVIDIA API key (NIM provider) | required |
| `OPENROUTER_API_KEY` | OpenRouter API key (OpenRouter provider) | required |
| `LM_STUDIO_BASE_URL` | LM Studio server URL | `http://localhost:1234/v1` |
diff --git a/api/models/anthropic.py b/api/models/anthropic.py
index 87f4698a..c03cd359 100644
--- a/api/models/anthropic.py
+++ b/api/models/anthropic.py
@@ -102,7 +102,7 @@ class MessagesRequest(BaseModel):
@model_validator(mode="after")
def map_model(self) -> MessagesRequest:
- """Map any Claude model name to the configured model (tier-aware)."""
+ """Map any Claude model name to the configured model (model-aware)."""
settings = get_settings()
if self.original_model is None:
self.original_model = self.model
@@ -128,7 +128,7 @@ class TokenCountRequest(BaseModel):
@field_validator("model")
@classmethod
def validate_model_field(cls, v: str, info) -> str:
- """Map any Claude model name to the configured model (tier-aware)."""
+ """Map any Claude model name to the configured model (model-aware)."""
settings = get_settings()
resolved_full = settings.resolve_model(v)
return Settings.parse_model_name(resolved_full)
diff --git a/api/routes.py b/api/routes.py
index 2c5f2df3..105bc210 100644
--- a/api/routes.py
+++ b/api/routes.py
@@ -41,7 +41,7 @@ async def create_message(
if optimized is not None:
return optimized
- # Resolve provider from the tier-aware model mapping
+ # Resolve provider from the model-aware mapping
provider_type = Settings.parse_provider_type(
request_data.resolved_provider_model or settings.model
)
diff --git a/config/settings.py b/config/settings.py
index 7c30507e..cdff8f80 100644
--- a/config/settings.py
+++ b/config/settings.py
@@ -2,14 +2,11 @@
from functools import lru_cache
-from dotenv import load_dotenv
from pydantic import Field, field_validator, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
from .nim import NimSettings
-load_dotenv()
-
class Settings(BaseSettings):
"""Application settings loaded from environment variables."""
@@ -37,7 +34,7 @@ class Settings(BaseSettings):
# Format: provider_type/model/name
model: str = "nvidia_nim/meta/llama3-70b-instruct"
- # Per-tier model overrides (optional, falls back to MODEL)
+ # Per-model overrides (optional, falls back to MODEL)
# Each can use a different provider
model_opus: str | None = Field(default=None, validation_alias="MODEL_OPUS")
model_sonnet: str | None = Field(default=None, validation_alias="MODEL_SONNET")
@@ -176,8 +173,8 @@ class Settings(BaseSettings):
def resolve_model(self, claude_model_name: str) -> str:
"""Resolve a Claude model name to the configured provider/model string.
- Classifies the incoming model into a tier (opus/sonnet/haiku) and
- returns the tier-specific model if configured, otherwise the fallback MODEL.
+ Classifies the incoming Claude model (opus/sonnet/haiku) and
+ returns the model-specific override if configured, otherwise the fallback MODEL.
"""
name_lower = claude_model_name.lower()
if "opus" in name_lower and self.model_opus is not None:
diff --git a/tests/api/test_models_validators.py b/tests/api/test_models_validators.py
index bd567018..b6089dea 100644
--- a/tests/api/test_models_validators.py
+++ b/tests/api/test_models_validators.py
@@ -10,6 +10,9 @@ from config.settings import Settings
def mock_settings():
settings = Settings()
settings.model = "nvidia_nim/target-model-from-settings"
+ settings.model_opus = None
+ settings.model_sonnet = None
+ settings.model_haiku = None
return settings
@@ -76,7 +79,7 @@ def test_messages_request_resolved_provider_model_default(mock_settings):
)
-def test_messages_request_tier_aware_opus_override():
+def test_messages_request_model_aware_opus_override():
"""Opus model routes to MODEL_OPUS when set."""
settings = Settings()
settings.model = "nvidia_nim/fallback-model"
@@ -93,7 +96,7 @@ def test_messages_request_tier_aware_opus_override():
assert request.original_model == "claude-opus-4-20250514"
-def test_messages_request_tier_aware_haiku_override():
+def test_messages_request_model_aware_haiku_override():
"""Haiku model routes to MODEL_HAIKU when set."""
settings = Settings()
settings.model = "nvidia_nim/fallback-model"
@@ -109,7 +112,7 @@ def test_messages_request_tier_aware_haiku_override():
assert request.resolved_provider_model == "lmstudio/qwen2.5-7b"
-def test_messages_request_tier_aware_sonnet_override():
+def test_messages_request_model_aware_sonnet_override():
"""Sonnet model routes to MODEL_SONNET when set."""
settings = Settings()
settings.model = "nvidia_nim/fallback-model"
@@ -127,10 +130,13 @@ def test_messages_request_tier_aware_sonnet_override():
)
-def test_messages_request_tier_fallback_when_not_set():
- """When tier override is None, falls back to MODEL."""
+def test_messages_request_model_fallback_when_not_set():
+ """When model override is None, falls back to MODEL."""
settings = Settings()
settings.model = "nvidia_nim/fallback-model"
+ settings.model_opus = None
+ settings.model_sonnet = None
+ settings.model_haiku = None
# model_opus is None
with patch("api.models.anthropic.get_settings", return_value=settings):
@@ -143,8 +149,8 @@ def test_messages_request_tier_fallback_when_not_set():
assert request.resolved_provider_model == "nvidia_nim/fallback-model"
-def test_token_count_request_tier_aware():
- """TokenCountRequest also uses tier-aware resolution."""
+def test_token_count_request_model_aware():
+ """TokenCountRequest also uses model-aware resolution."""
settings = Settings()
settings.model = "nvidia_nim/fallback-model"
settings.model_haiku = "lmstudio/qwen2.5-7b"
diff --git a/tests/config/test_config.py b/tests/config/test_config.py
index f45df73d..10862eed 100644
--- a/tests/config/test_config.py
+++ b/tests/config/test_config.py
@@ -310,11 +310,11 @@ class TestSettingsOptionalStr:
assert s.whisper_device == device
-class TestPerTierModelMapping:
- """Test per-tier model fields and resolve_model()."""
+class TestPerModelMapping:
+ """Test per-model fields and resolve_model()."""
- def test_tier_fields_default_none(self):
- """Per-tier model fields default to None."""
+ def test_model_fields_default_none(self):
+ """Per-model fields default to None."""
from config.settings import Settings
s = Settings()
@@ -411,20 +411,20 @@ class TestPerTierModelMapping:
assert s.resolve_model("claude-3-5-haiku-20241022") == "lmstudio/qwen2.5-7b"
assert s.resolve_model("claude-haiku-4-20250514") == "lmstudio/qwen2.5-7b"
- def test_resolve_model_fallback_when_tier_not_set(self):
- """resolve_model falls back to MODEL when tier override is None."""
+ def test_resolve_model_fallback_when_override_not_set(self):
+ """resolve_model falls back to MODEL when model override is None."""
from config.settings import Settings
s = Settings()
s.model = "nvidia_nim/fallback-model"
- # No tier overrides set
+ # No model overrides set
assert s.resolve_model("claude-opus-4-20250514") == "nvidia_nim/fallback-model"
assert (
s.resolve_model("claude-sonnet-4-20250514") == "nvidia_nim/fallback-model"
)
assert s.resolve_model("claude-3-haiku-20240307") == "nvidia_nim/fallback-model"
- def test_resolve_model_unknown_tier_falls_back(self):
+ def test_resolve_model_unknown_model_falls_back(self):
"""resolve_model falls back to MODEL for unrecognized model names."""
from config.settings import Settings
@@ -435,7 +435,7 @@ class TestPerTierModelMapping:
assert s.resolve_model("some-unknown-model") == "nvidia_nim/fallback-model"
def test_resolve_model_case_insensitive(self):
- """Tier classification is case-insensitive."""
+ """Model classification is case-insensitive."""
from config.settings import Settings
s = Settings()
diff --git a/tests/conftest.py b/tests/conftest.py
index c4f837c7..d64e6b23 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -29,6 +29,16 @@ from providers.base import ProviderConfig
from providers.nvidia_nim import NvidiaNimProvider
[email protected](autouse=True)
+def _isolate_from_dotenv(monkeypatch):
+ """Prevent Pydantic BaseSettings from reading the .env file during tests."""
+ from config.settings import Settings
+
+ monkeypatch.setattr(
+ Settings, "model_config", {**Settings.model_config, "env_file": None}
+ )
+
+
@pytest.fixture
def provider_config():
return ProviderConfig(