Skip to content

Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

Python

Add client-independent model fallback (#1463)

Ali Khokhara186fd0f4136
diff --git a/.env.example b/.env.example
index 96531f34..fd46e625 100644
--- a/.env.example
+++ b/.env.example
@@ -185,7 +185,7 @@ LLAMACPP_BASE_URL="http://localhost:8080/v1"
OLLAMA_BASE_URL="http://localhost:11434"
-# All Claude model requests are mapped to these models, plain model is fallback
+# Default model and optional Claude tier overrides
# Format: provider_type/model/name
# Valid providers: "nvidia_nim" | "openai" | "azure_openai" | "open_router" | "gemini" | "vertex" | "deepseek" | "mistral" | "mistral_codestral" | "opencode_zen" | "opencode_go" | "vercel" | "bedrock" | "huggingface" | "cohere" | "github_models" | "wafer" | "kimi" | "kimi_code" | "minimax" | "cerebras" | "groq" | "cline_pass" | "xai" | "qwencloud" | "qwencloud_coding" | "together" | "deepinfra" | "siliconflow" | "nebius" | "chutes" | "featherless" | "agnes" | "zenmux" | "wandb" | "sambanova" | "kilo" | "fireworks" | "novita" | "cloudflare" | "zai" | "ollama_cloud" | "lmstudio" | "llamacpp" | "ollama" | "tokenrouter" | "nararoute"
# MODEL_FABLE=<provider/model-id>
@@ -193,6 +193,9 @@ OLLAMA_BASE_URL="http://localhost:11434"
# MODEL_SONNET=<provider/model-id>
# MODEL_HAIKU=<provider/model-id>
MODEL="nvidia_nim/nvidia/nemotron-3-super-120b-a12b"
+# Optional ordered cross-client fallbacks after retryable provider failures exhaust.
+# A request may reach and consume usage from more than one provider.
+# MODEL_FALLBACKS="groq/llama-3.3-70b-versatile,open_router/openrouter/free"
# Optional live smoke model overrides. Provider smoke runs once per configured
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index e8f5c80d..c8404134 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -453,16 +453,19 @@ server tools, then streams Anthropic SSE. `ResponsesHandler` owns streaming-only
OpenAI Responses validation and conversion for Codex clients. `TokenCountHandler`
owns Anthropic token counting. Shared provider execution lives in
[application/execution.py](src/free_claude_code/application/execution.py). `ProviderExecutor` resolves the narrow
-consumer-owned `ProviderPort`, synchronously preflights the upstream request,
-emits trace events, counts input tokens, and returns an Anthropic SSE iterator.
+consumer-owned `ProviderPort`, synchronously preflights the primary upstream
+request, emits trace events, counts input tokens, and returns an Anthropic SSE
+iterator. It also owns application-level model fallback after provider-owned
+retries are exhausted; providers do not select alternate models.
It receives only a provider resolver and the few scalar collaborators it needs;
it does not depend on FastAPI, provider implementations, or the full settings
object. The executor also owns FCC's provider-progress deadline: every wait for
the next non-empty provider chunk is limited by the Settings-owned
`PROVIDER_PROGRESS_TIMEOUT`, which defaults to 600 seconds and is projected into
-the executor as a validated scalar. Admission, retries, backoff, and recovery
-before a chunk all consume that same wait; a non-empty emitted chunk renews the
-next window, while empty keepalives do not. The timeout context ends before the
+the executor as a validated scalar. Admission, retries, backoff, recovery, and
+any pre-output fallback transitions all consume that same wait; switching
+candidates never resets it. A non-empty emitted chunk renews the next window,
+while empty keepalives do not. The timeout context ends before the
executor yields, so downstream response backpressure is not mistaken for
stalled provider work and cannot receive cancellation from the generator's
timer. Expiry becomes a protocol-neutral, non-retryable 504 `ExecutionFailure`;
@@ -558,8 +561,8 @@ It supports two forms:
- Gateway model IDs decoded by [core/gateway_model_ids.py](src/free_claude_code/core/gateway_model_ids.py).
If the incoming model is not direct, `ModelRouter` maps it by Claude tier. Names
-containing `fable`, `opus`, `sonnet`, or `haiku` use the matching tier override when set,
-otherwise they fall back to `MODEL`.
+containing `fable`, `opus`, `sonnet`, or `haiku` use the matching tier override
+when set, otherwise they use the Default Model in `MODEL`.
The router also selects the applicable reasoning preference. Direct provider
refs use the root policy; Claude tier routes use a non-inherited tier override
@@ -567,17 +570,42 @@ or the root fallback; the no-thinking gateway variant forces `off`.
[application/reasoning.py](src/free_claude_code/application/reasoning.py) then
combines that preference with the concrete client request exactly once. The
resulting `ReasoningPolicy` preserves independent control, named effort, and an
-exact client token budget without guessing provider behavior. `ResolvedModel`
-owns the selected route and preference; `RoutedMessagesRequest` owns the final
-request-scoped policy passed to execution.
+exact client token budget without guessing provider behavior.
+`ResolvedModelRoute` owns the public model, one canonical primary
+`ProviderModelTarget`, the ordered canonical targets from `MODEL_FALLBACKS`, and
+the reasoning preference. It omits only a fallback exactly equal to the resolved
+primary; another model on the same provider remains valid.
+`RoutedMessagesRequest` owns the final request-scoped policy passed to execution.
Routing keeps model identity split at the application boundary. The routed
request carries the provider model sent upstream, while
-`ResolvedModel.original_model` remains the stable gateway model exposed in
+`ResolvedModelRoute.original_model` remains the stable gateway model exposed in
Anthropic responses and traces. `ProviderExecutor` passes both identities
explicitly; providers, local optimizations, and local server tools must never
publish the private upstream model as the response model.
+Fallback execution is a bounded first-frame state machine in
+[application/execution.py](src/free_claude_code/application/execution.py). The
+executor opens the primary first and resolves each later provider lazily. It
+advances only when the current candidate raises a retryable `ExecutionFailure`
+before emitting any non-empty protocol chunk and another configured target
+exists. It closes the abandoned iterator before resolving the next provider and
+deep-copies the original routed request for every candidate, changing only the
+upstream model. Authentication, permission, billing, invalid request, context
+overflow, deterministic preflight, unexpected exceptions, empty completion,
+timeouts, cancellation, and disconnect do not advance. Once any protocol frame
+is emitted, the candidate is committed and existing terminal-error behavior is
+authoritative; this prevents duplicate lifecycles and output. Final exhaustion
+re-raises the last exact failure.
+
+Every candidate keeps the original request ID, public response model, input
+token count, reasoning policy, messages, system prompt, tools, and generation
+metadata. One request-generation lease therefore snapshots both the primary and
+fallback list: Admin hot apply affects only requests that acquire the next
+generation. Safe `model_fallback.started` and `model_fallback.selected` trace
+events expose canonical route refs, candidate positions, failure kind, status,
+wire API, and generation without prompt or raw upstream error content.
+
`GET /v1/models` advertises:
- configured provider model refs;
@@ -589,8 +617,8 @@ Provider model discovery and optional thinking metadata live in the
application-level catalog owned by `ProviderRuntimeManager`.
[providers/runtime/discovery.py](src/free_claude_code/providers/runtime/discovery.py)
is the sole owner of provider model-list queries and cache population. Startup
-synchronously warms the providers referenced by model routing before clients can
-perform their one-time model fetch, then a background pass fills the remaining
+synchronously warms the providers referenced by primary and fallback routing
+before clients can perform their one-time model fetch, then a background pass fills the remaining
configured provider catalogs without querying successful warm-ups again.
Discovery is an adapter operation, not an assumption that every upstream has an
OpenAI `/models` route. For example, Vertex translates that operation to
diff --git a/README.md b/README.md
index 1559eee1..9cdba54e 100644
--- a/README.md
+++ b/README.md
@@ -23,6 +23,7 @@
- **48 ToS-friendly providers. 1.3B+ free tokens every month.** Use free, paid, subscription, and local models from one searchable UI without putting your account at risk. FCC follows provider terms and removes integrations if they stop being allowed.
- **5 coding agents. One model catalog.** Run Claude Code, Codex, Pi, OpenCode, or Cline with your FCC models.
+- **Ordered model fallback for every client.** After a provider exhausts retryable failures, FCC can try your next configured model without restarting the turn.
- **Up to 90% fewer terminal-output tokens.** Optional [RTK](https://github.com/rtk-ai/rtk) filters common command output, while five FCC optimizations handle quota probes, command-prefix detection, titles, suggestions, and filepaths without calling a provider.
- **Terminal, desktop, IDE, or phone.** Work through native launchers, VS Code, Codex App, JetBrains, Discord, or Telegram.
- **Voice notes in. Code out.** Talk to your agent using local Whisper or NVIDIA NIM transcription.
@@ -151,6 +152,10 @@ extensions unchanged.
models, enter `<provider-id>/<exact-provider-model-id>` manually.
4. Click **Validate**, then **Apply**.
+Optional: add an ordered **Fallback Models** list under **Model Config**. It
+applies to every connected client. A failed request may reach and consume usage
+from more than one provider before succeeding.
+
<details>
<summary><strong>Provider catalog</strong></summary>
diff --git a/e2e/conftest.py b/e2e/conftest.py
index b51c6e05..58cf5836 100644
--- a/e2e/conftest.py
+++ b/e2e/conftest.py
@@ -104,6 +104,7 @@ def _close_manager(manager: ProviderRuntimeManager) -> None:
@pytest.fixture
def admin_base_url(
+ request: pytest.FixtureRequest,
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> Iterator[str]:
@@ -124,6 +125,8 @@ def admin_base_url(
monkeypatch.setenv("FCC_OPEN_BROWSER", "false")
monkeypatch.setenv("PROXY_AUTH_ENABLED", "false")
monkeypatch.setenv("ANTHROPIC_AUTH_TOKEN", "e2e-proxy-token")
+ for key, value in getattr(request, "param", {}).items():
+ monkeypatch.setenv(key, value)
monkeypatch.setattr(paths, "config_dir_path", lambda: config_dir)
monkeypatch.setattr(env_migrations, "legacy_env_paths", lambda: ())
monkeypatch.setattr(env_migrations, "verified_checkout_env_path", lambda: None)
diff --git a/e2e/test_admin_models.py b/e2e/test_admin_models.py
new file mode 100644
index 00000000..9feb0f43
--- /dev/null
+++ b/e2e/test_admin_models.py
@@ -0,0 +1,132 @@
+"""Rendered model-fallback controls for the local Admin UI."""
+
+import pytest
+from playwright.sync_api import Page, expect
+
+
+def _open_models(page: Page, admin_base_url: str) -> None:
+ page.emulate_media(reduced_motion="reduce")
+ page.goto(f"{admin_base_url}/admin")
+ expect(page.locator("#messageArea")).to_have_text("")
+ page.get_by_role("button", name="Model Config", exact=True).click()
+
+
+def _refresh_openrouter_models(page: Page) -> None:
+ page.get_by_role("button", name="Providers", exact=True).click()
+ card = page.locator('[data-provider="open_router"]')
+ card.get_by_role("button", name="Refresh models", exact=True).click()
+ expect(card.locator(".provider-check-result")).to_have_text("2 models available")
+ page.get_by_role("button", name="Model Config", exact=True).click()
+
+
+def test_fallback_editor_adds_filters_reorders_and_removes_models(
+ page: Page,
+ admin_base_url: str,
+) -> None:
+ page.set_viewport_size({"width": 1280, "height": 720})
+ _open_models(page, admin_base_url)
+ _refresh_openrouter_models(page)
+
+ editor = page.locator('.field[data-key="MODEL_FALLBACKS"]')
+ add_input = editor.locator("#field-MODEL_FALLBACKS-add")
+ stored_value = editor.locator("#field-MODEL_FALLBACKS")
+ expect(editor.locator(".model-list-empty")).to_have_text(
+ "No fallback models configured."
+ )
+
+ add_input.fill("model-b")
+ expect(
+ editor.get_by_role("option", name="open_router/vendor/model-b", exact=True)
+ ).to_be_visible()
+ add_input.press("ArrowDown")
+ add_input.press("Enter")
+ expect(add_input).to_have_value("open_router/vendor/model-b")
+ editor.get_by_role("button", name="Add", exact=True).click()
+
+ add_input.fill("groq/custom-model")
+ editor.get_by_role("button", name="Add", exact=True).click()
+ expect(stored_value).to_have_value("open_router/vendor/model-b,groq/custom-model")
+ expect(page.locator("#dirtyState")).to_have_text("1 unsaved change")
+ expect(page.locator("#applyButton")).to_be_enabled()
+
+ editor.get_by_role("button", name="Move groq/custom-model up", exact=True).click()
+ expect(stored_value).to_have_value("groq/custom-model,open_router/vendor/model-b")
+ expect(
+ editor.get_by_role("button", name="Move groq/custom-model up", exact=True)
+ ).to_be_disabled()
+ expect(
+ editor.get_by_role(
+ "button", name="Move open_router/vendor/model-b down", exact=True
+ )
+ ).to_be_disabled()
+
+ editor.get_by_role("button", name="Remove groq/custom-model", exact=True).click()
+ expect(stored_value).to_have_value("open_router/vendor/model-b")
+ editor.get_by_role(
+ "button", name="Remove open_router/vendor/model-b", exact=True
+ ).click()
+ expect(stored_value).to_have_value("")
+ expect(editor.locator(".model-list-empty")).to_be_visible()
+ expect(page.locator("#dirtyState")).to_have_text("No changes")
+ expect(page.locator("#applyButton")).to_be_disabled()
+
+
+def test_fallback_editor_rejects_blank_and_duplicate_rows(
+ page: Page,
+ admin_base_url: str,
+) -> None:
+ page.set_viewport_size({"width": 1280, "height": 720})
+ _open_models(page, admin_base_url)
+
+ editor = page.locator('.field[data-key="MODEL_FALLBACKS"]')
+ add_input = editor.locator("#field-MODEL_FALLBACKS-add")
+ add_button = editor.get_by_role("button", name="Add", exact=True)
+
+ add_button.click()
+ expect(page.locator("#messageArea")).to_have_text(
+ "Enter a full provider/model fallback."
+ )
+ add_input.fill("groq/custom-model")
+ add_button.click()
+ add_input.fill("groq/custom-model")
+ add_button.click()
+ expect(page.locator("#messageArea")).to_have_text(
+ "That fallback model is already in the list."
+ )
+ expect(editor.locator(".model-list-row")).to_have_count(1)
+
+
+ "admin_base_url",
+ [{"MODEL_FALLBACKS": "groq/model-a,open_router/model-b"}],
+ indirect=True,
+)
+def test_process_owned_fallback_list_locks_every_mutation(
+ page: Page,
+ admin_base_url: str,
+) -> None:
+ page.set_viewport_size({"width": 1280, "height": 720})
+ _open_models(page, admin_base_url)
+
+ editor = page.locator('.field[data-key="MODEL_FALLBACKS"]')
+ expect(editor.locator("#field-MODEL_FALLBACKS-add")).to_be_disabled()
+ expect(editor.get_by_role("button", name="Add", exact=True)).to_be_disabled()
+ expect(editor.locator(".model-list-action")).to_have_count(6)
+ for button in editor.locator(".model-list-action").all():
+ expect(button).to_be_disabled()
+
+
+def test_fallback_editor_remains_usable_at_narrow_viewport(
+ page: Page,
+ admin_base_url: str,
+) -> None:
+ page.set_viewport_size({"width": 390, "height": 844})
+ _open_models(page, admin_base_url)
+
+ editor = page.locator('.field[data-key="MODEL_FALLBACKS"]')
+ editor.scroll_into_view_if_needed()
+ expect(editor.locator("#field-MODEL_FALLBACKS-add")).to_be_visible()
+ expect(editor.get_by_role("button", name="Add", exact=True)).to_be_visible()
+ assert page.evaluate(
+ "document.documentElement.scrollWidth <= document.documentElement.clientWidth"
+ )
diff --git a/pyproject.toml b/pyproject.toml
index 8f8a182d..234e9e0c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "free-claude-code"
-version = "5.7.0"
+version = "5.8.0"
description = "Local proxy connecting coding agents to OpenAI-compatible AI providers"
readme = "README.md"
requires-python = ">=3.14.0"
diff --git a/smoke/capabilities.py b/smoke/capabilities.py
index 4f8515a9..531ac2e3 100644
--- a/smoke/capabilities.py
+++ b/smoke/capabilities.py
@@ -144,6 +144,21 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = (
("tests/application/test_routing.py", "tests/config/test_config.py"),
("test_mixed_provider_model_mapping_when_configured",),
),
+ CapabilityContract(
+ "provider_routing",
+ "retry_exhaustion_fallback",
+ "model_fallback",
+ "free_claude_code.application.execution.ProviderExecutor",
+ "resolved primary plus ordered MODEL_FALLBACKS route plan",
+ "one protocol lifecycle from the first candidate that emits a frame",
+ "last exact failure, or no transition after commit/cancellation/nonretryable error",
+ (
+ "tests/application/test_execution.py",
+ "tests/api/test_model_fallback.py",
+ "tests/config/test_config.py",
+ ),
+ ("test_model_fallback_e2e",),
+ ),
CapabilityContract(
"provider_routing",
"provider_runtime_config",
diff --git a/smoke/features.py b/smoke/features.py
index 2eb1fd09..0a61ea73 100644
--- a/smoke/features.py
+++ b/smoke/features.py
@@ -162,6 +162,20 @@ FEATURE_INVENTORY: tuple[FeatureCoverage, ...] = (
("MODEL", "MODEL_FABLE", "MODEL_OPUS", "MODEL_SONNET", "MODEL_HAIKU"),
"configured mixed-provider mappings must resolve consistently",
),
+ FeatureCoverage(
+ "model_fallback",
+ "Retryable pre-response failures can move through an ordered model list",
+ (
+ "tests/application/test_execution.py",
+ "tests/api/test_model_fallback.py",
+ "tests/config/test_config.py",
+ ),
+ (),
+ ("test_model_fallback_e2e",),
+ ("api",),
+ (),
+ "always runnable with isolated controlled providers",
+ ),
FeatureCoverage(
"thinking_token_support",
"Thinking history, adaptive thinking, and redacted blocks are accepted",
diff --git a/smoke/product/test_model_fallback_product_live.py b/smoke/product/test_model_fallback_product_live.py
new file mode 100644
index 00000000..e232ee89
--- /dev/null
+++ b/smoke/product/test_model_fallback_product_live.py
@@ -0,0 +1,55 @@
+"""Credential-free product smoke for application-owned model fallback."""
+
+import pytest
+
+from free_claude_code.core.anthropic.stream_contracts import parse_sse_text
+from tests.api.model_fallback_support import (
+ ControlledFallbackProvider,
+ execution_failure,
+ fallback_client,
+ messages_payload,
+ responses_payload,
+)
+
+pytestmark = [pytest.mark.live, pytest.mark.smoke_target("api")]
+
+
+ ("path", "payload", "start_event", "complete_event"),
+ (
+ (
+ "/v1/messages",
+ messages_payload(stream=True),
+ "message_start",
+ "message_stop",
+ ),
+ (
+ "/v1/responses",
+ responses_payload(),
+ "response.created",
+ "response.completed",
+ ),
+ ),
+)
+def test_model_fallback_e2e(
+ path: str,
+ payload: dict[str, object],
+ start_event: str,
+ complete_event: str,
+) -> None:
+ primary = ControlledFallbackProvider(
+ failure=execution_failure("controlled primary overload")
+ )
+ fallback = ControlledFallbackProvider(text="MODEL_FALLBACK_SMOKE_OK")
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post(path, json=payload)
+
+ assert response.status_code == 200, response.text
+ events = parse_sse_text(response.text)
+ names = [event.event for event in events]
+ assert names.count(start_event) == 1
+ assert names.count(complete_event) == 1
+ assert "MODEL_FALLBACK_SMOKE_OK" in response.text
+ assert "controlled primary overload" not in response.text
+ assert primary.close_calls == fallback.close_calls == 1
diff --git a/src/free_claude_code/api/admin_static/admin.css b/src/free_claude_code/api/admin_static/admin.css
index a29fb4be..a19c851a 100644
--- a/src/free_claude_code/api/admin_static/admin.css
+++ b/src/free_claude_code/api/admin_static/admin.css
@@ -611,6 +611,59 @@ textarea:focus-visible,
line-height: 1.4;
}
+.field[data-key="MODEL_FALLBACKS"] {
+ grid-column: 1 / -1;
+}
+
+.model-list-editor,
+.model-list-rows {
+ display: grid;
+ gap: 8px;
+}
+
+.model-list-add {
+ display: grid;
+ grid-template-columns: minmax(0, 1fr) auto;
+ gap: 8px;
+ align-items: start;
+}
+
+.model-list-row {
+ display: grid;
+ grid-template-columns: minmax(0, 1fr) auto auto auto;
+ gap: 4px;
+ align-items: center;
+ min-height: 44px;
+ padding: 6px 8px 6px 12px;
+ border: 1px solid var(--line);
+ border-radius: var(--radius-md);
+ background: var(--panel-strong);
+}
+
+.model-list-value {
+ min-width: 0;
+ overflow-wrap: anywhere;
+ color: var(--text-strong);
+ font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
+ font-size: 13px;
+}
+
+.model-list-action {
+ min-height: 32px;
+ padding: 4px 8px;
+}
+
+.model-list-editor button:disabled {
+ cursor: not-allowed;
+ opacity: 0.45;
+}
+
+.model-list-empty {
+ padding: 8px 2px;
+ color: var(--muted);
+ font-size: 13px;
+}
+
.field input:focus,
.field select:focus,
.field textarea:focus {
@@ -831,6 +884,14 @@ textarea:focus-visible,
grid-template-columns: 1fr;
}
+ .model-list-add {
+ grid-template-columns: 1fr;
+ }
+
+ .model-list-row {
+ grid-template-columns: 1fr repeat(3, auto);
+ }
+
.action-meta {
flex-direction: column;
align-items: stretch;
diff --git a/src/free_claude_code/api/admin_static/admin.js b/src/free_claude_code/api/admin_static/admin.js
index 153bbe48..5846096a 100644
--- a/src/free_claude_code/api/admin_static/admin.js
+++ b/src/free_claude_code/api/admin_static/admin.js
@@ -584,10 +584,14 @@ function renderField(field) {
});
}
- const control =
- field.type === "model" || field.type === "optional_model"
- ? new ModelCombobox(input, field).element
- : input;
+ let control = input;
+ if (field.type === "model" || field.type === "optional_model") {
+ control = new ModelCombobox(input, field).element;
+ } else if (field.type === "model_list") {
+ const editor = new ModelListEditor(input, field);
+ label.htmlFor = editor.inputId;
+ control = editor.element;
+ }
wrapper.append(label, control);
if (field.secret && field.nullable && field.configured && !field.locked) {
const removeButton = document.createElement("button");
@@ -644,6 +648,13 @@ function inputForField(field) {
return input;
}
+ if (field.type === "model_list") {
+ const input = document.createElement("input");
+ input.type = "hidden";
+ input.value = field.value || "";
+ return input;
+ }
+
const input = document.createElement("input");
input.type = field.type === "number" ? "number" : "text";
if (field.type === "secret") {
@@ -839,6 +850,132 @@ class ModelCombobox {
}
}
+class ModelListEditor {
+ constructor(input, field) {
+ this.input = input;
+ this.field = field;
+ this.values = input.value
+ ? input.value.split(",").map((value) => value.trim()).filter(Boolean)
+ : [];
+ this.inputId = `field-${field.key}-add`;
+
+ this.element = document.createElement("div");
+ this.element.className = "model-list-editor";
+
+ const addRow = document.createElement("div");
+ addRow.className = "model-list-add";
+ this.addInput = document.createElement("input");
+ this.addInput.id = this.inputId;
+ this.addInput.type = "text";
+ this.addInput.autocomplete = "off";
+ this.addInput.placeholder = "provider/model";
+ this.addInput.disabled = field.locked;
+ const addCombobox = new ModelCombobox(this.addInput, {
+ ...field,
+ key: `${field.key}-add`,
+ label: "fallback model",
+ type: "model",
+ });
+
+ this.addButton = document.createElement("button");
+ this.addButton.type = "button";
+ this.addButton.className = "secondary-button";
+ this.addButton.textContent = "Add";
+ this.addButton.disabled = field.locked;
+ this.addButton.addEventListener("click", () => this.add());
+ addRow.append(addCombobox.element, this.addButton);
+
+ this.rows = document.createElement("div");
+ this.rows.className = "model-list-rows";
+ this.element.append(input, addRow, this.rows);
+ this.renderRows();
+ }
+
+ add() {
+ const value = this.addInput.value.trim();
+ if (!value) {
+ showMessage("Enter a full provider/model fallback.", "error");
+ return;
+ }
+ if (this.values.includes(value)) {
+ showMessage("That fallback model is already in the list.", "error");
+ return;
+ }
+ this.values.push(value);
+ this.addInput.value = "";
+ showMessage("");
+ this.sync();
+ }
+
+ move(index, offset) {
+ const destination = index + offset;
+ if (destination < 0 || destination >= this.values.length) return;
+ [this.values[index], this.values[destination]] = [
+ this.values[destination],
+ this.values[index],
+ ];
+ this.sync();
+ }
+
+ remove(index) {
+ this.values.splice(index, 1);
+ this.sync();
+ }
+
+ sync() {
+ this.input.value = this.values.join(",");
+ this.input.dataset.remove = "false";
+ this.input.dispatchEvent(new Event("input", { bubbles: true }));
+ this.renderRows();
+ }
+
+ renderRows() {
+ this.rows.innerHTML = "";
+ if (this.values.length === 0) {
+ const empty = document.createElement("div");
+ empty.className = "model-list-empty";
+ empty.textContent = "No fallback models configured.";
+ this.rows.appendChild(empty);
+ return;
+ }
+
+ this.values.forEach((value, index) => {
+ const row = document.createElement("div");
+ row.className = "model-list-row";
+
+ const model = document.createElement("span");
+ model.className = "model-list-value";
+ model.textContent = value;
+
+ const up = this.actionButton("Move up", `Move ${value} up`, () =>
+ this.move(index, -1),
+ );
+ up.disabled = this.field.locked || index === 0;
+ const down = this.actionButton("Move down", `Move ${value} down`, () =>
+ this.move(index, 1),
+ );
+ down.disabled = this.field.locked || index === this.values.length - 1;
+ const remove = this.actionButton("Remove", `Remove ${value}`, () =>
+ this.remove(index),
+ );
+ remove.disabled = this.field.locked;
+
+ row.append(model, up, down, remove);
+ this.rows.appendChild(row);
+ });
+ }
+
+ actionButton(text, label, action) {
+ const button = document.createElement("button");
+ button.type = "button";
+ button.className = "ghost-button model-list-action";
+ button.textContent = text;
+ button.setAttribute("aria-label", label);
+ button.addEventListener("click", action);
+ return button;
+ }
+}
+
function option(value, label) {
const optionEl = document.createElement("option");
optionEl.value = value;
diff --git a/src/free_claude_code/api/handlers/messages.py b/src/free_claude_code/api/handlers/messages.py
index e5fbb4ce..15873026 100644
--- a/src/free_claude_code/api/handlers/messages.py
+++ b/src/free_claude_code/api/handlers/messages.py
@@ -12,6 +12,7 @@ from free_claude_code.api.optimization_handlers import try_optimizations
from free_claude_code.api.request_errors import (
http_status_for_unexpected_api_exception,
log_unexpected_api_exception,
+ ordinary_application_error_response,
require_non_empty_messages,
unexpected_http_exception,
)
@@ -152,6 +153,8 @@ class MessagesHandler:
raise
except asyncio.CancelledError:
raise
+ except ApplicationError:
+ raise
except ExecutionFailure as exc:
return self._execution_failure_response(exc, request_id=request_id)
except BaseExceptionGroup as exc:
@@ -200,6 +203,12 @@ class MessagesHandler:
def _pre_start_error_response(
self, exc: BaseException, *, request_id: str
) -> Response:
+ if isinstance(exc, ApplicationError):
+ return ordinary_application_error_response(
+ exc,
+ wire_api="messages",
+ request_id=request_id,
+ )
failure = find_execution_failure(exc)
if failure is not None:
return self._execution_failure_response(failure, request_id=request_id)
diff --git a/src/free_claude_code/api/handlers/responses.py b/src/free_claude_code/api/handlers/responses.py
index 0ab83315..84f0a988 100644
--- a/src/free_claude_code/api/handlers/responses.py
+++ b/src/free_claude_code/api/handlers/responses.py
@@ -5,6 +5,7 @@ from fastapi.responses import JSONResponse
from free_claude_code.api.request_errors import (
http_status_for_unexpected_api_exception,
log_unexpected_api_exception,
+ ordinary_application_error_response,
require_non_empty_messages,
)
from free_claude_code.api.request_ids import new_request_id
@@ -120,6 +121,12 @@ class ResponsesHandler:
def _pre_start_error_response(
self, exc: BaseException, *, request_id: str
) -> JSONResponse:
+ if isinstance(exc, ApplicationError):
+ return ordinary_application_error_response(
+ exc,
+ wire_api="responses",
+ request_id=request_id,
+ )
failure = find_execution_failure(exc)
if failure is not None:
return self._execution_failure_response(failure, request_id=request_id)
diff --git a/src/free_claude_code/api/handlers/token_count.py b/src/free_claude_code/api/handlers/token_count.py
index 3e5e05df..f2bea621 100644
--- a/src/free_claude_code/api/handlers/token_count.py
+++ b/src/free_claude_code/api/handlers/token_count.py
@@ -55,9 +55,9 @@ class TokenCountHandler:
source="api",
request_id=request_id,
kind="count_tokens",
- provider_id=routed.resolved.provider_id,
- provider_model=routed.resolved.provider_model,
- provider_model_ref=routed.resolved.provider_model_ref,
+ provider_id=routed.resolved.primary.provider_id,
+ provider_model=routed.resolved.primary.provider_model,
+ provider_model_ref=routed.resolved.primary.provider_model_ref,
gateway_model=routed.resolved.original_model,
)
request_snapshot = anthropic_request_snapshot(routed.request)
diff --git a/src/free_claude_code/application/execution.py b/src/free_claude_code/application/execution.py
index 62793a8a..ca4b1fca 100644
--- a/src/free_claude_code/application/execution.py
+++ b/src/free_claude_code/application/execution.py
@@ -23,7 +23,7 @@ from free_claude_code.core.trace import (
)
from .ports import ProviderResolver
-from .routing import RoutedMessagesRequest
+from .routing import ProviderModelTarget, RoutedMessagesRequest
TokenCounter = Callable[
[list[Message], str | list[SystemContent] | None, list[Tool] | None],
@@ -77,6 +77,69 @@ class ProviderExecutor:
retryable=False,
)
+ def _trace_fallback_started(
+ self,
+ *,
+ request_id: str,
+ wire_api: WireApi,
+ failed: ProviderModelTarget,
+ selected: ProviderModelTarget,
+ failure: ExecutionFailure,
+ candidate_index: int,
+ candidate_count: int,
+ ) -> None:
+ fields: dict[str, object] = {
+ "stage": "execution",
+ "event": "free_claude_code.model_fallback.started",
+ "source": "application",
+ "request_id": request_id,
+ "wire_api": wire_api,
+ "from_provider_model_ref": failed.provider_model_ref,
+ "to_provider_model_ref": selected.provider_model_ref,
+ "candidate_index": candidate_index,
+ "candidate_count": candidate_count,
+ "failure_kind": failure.kind.value,
+ "status_code": failure.status_code,
+ "provider_retryable": True,
+ }
+ if self._generation_id is not None:
+ fields["generation_id"] = self._generation_id
+ trace_event(**fields)
+ logger.info(
+ "Model fallback: request_id={} from={} to={} candidate={}/{} "
+ "failure_kind={} status_code={}",
+ request_id,
+ failed.provider_model_ref,
+ selected.provider_model_ref,
+ candidate_index,
+ candidate_count,
+ failure.kind.value,
+ failure.status_code,
+ )
+
+ def _trace_fallback_selected(
+ self,
+ *,
+ request_id: str,
+ wire_api: WireApi,
+ selected: ProviderModelTarget,
+ candidate_index: int,
+ candidate_count: int,
+ ) -> None:
+ fields: dict[str, object] = {
+ "stage": "execution",
+ "event": "free_claude_code.model_fallback.selected",
+ "source": "application",
+ "request_id": request_id,
+ "wire_api": wire_api,
+ "selected_provider_model_ref": selected.provider_model_ref,
+ "candidate_index": candidate_index,
+ "candidate_count": candidate_count,
+ }
+ if self._generation_id is not None:
+ fields["generation_id"] = self._generation_id
+ trace_event(**fields)
+
def stream(
self,
routed: RoutedMessagesRequest,
@@ -87,11 +150,14 @@ class ProviderExecutor:
request_id: str,
) -> AsyncIterator[str]:
"""Preflight synchronously, then return the traced provider stream."""
- provider = self._provider_resolver(routed.resolved.provider_id)
- provider.preflight_stream(
- routed.request,
+ primary = routed.resolved.primary
+ primary_provider = self._provider_resolver(primary.provider_id)
+ primary_request = routed.request.model_copy(deep=True)
+ primary_provider.preflight_stream(
+ primary_request,
reasoning=routed.reasoning,
)
+ candidates = (primary, *routed.resolved.fallbacks)
gateway_model = routed.resolved.original_model
route_trace: dict[str, object] = {
@@ -99,9 +165,10 @@ class ProviderExecutor:
"event": "free_claude_code.api.route.resolved",
"source": "api",
"request_id": request_id,
- "provider_id": routed.resolved.provider_id,
- "provider_model": routed.resolved.provider_model,
- "provider_model_ref": routed.resolved.provider_model_ref,
+ "provider_id": primary.provider_id,
+ "provider_model": primary.provider_model,
+ "provider_model_ref": primary.provider_model_ref,
+ "fallback_count": len(routed.resolved.fallbacks),
"gateway_model": gateway_model,
"reasoning_control": routed.reasoning.control.value,
"reasoning_effort": (
@@ -142,53 +209,106 @@ class ProviderExecutor:
)
async def provider_body() -> AsyncIterator[str]:
- provider_stream: AsyncIterator[str] | None = None
- try:
- provider_stream = provider.stream_response(
- routed.request,
- input_tokens=input_tokens,
- request_id=request_id,
- response_model=gateway_model,
- reasoning=routed.reasoning,
+ loop = asyncio.get_running_loop()
+ progress_deadline = loop.time() + self._progress_timeout_seconds
+ for index, target in enumerate(candidates):
+ provider = (
+ primary_provider
+ if index == 0
+ else self._provider_resolver(target.provider_id)
)
- loop = asyncio.get_running_loop()
- progress_deadline = loop.time() + self._progress_timeout_seconds
- while True:
- if loop.time() >= progress_deadline:
- raise self._progress_timeout_failure(
- request_id=request_id,
- provider_id=routed.resolved.provider_id,
- )
- progress_timeout = asyncio.timeout_at(progress_deadline)
- try:
- async with progress_timeout:
- chunk = await anext(provider_stream)
- except StopAsyncIteration:
- break
- except TimeoutError as exc:
- if not progress_timeout.expired():
- raise
- raise self._progress_timeout_failure(
- request_id=request_id,
- provider_id=routed.resolved.provider_id,
- ) from exc
- if not chunk:
- await asyncio.sleep(0)
- continue
- yield chunk
- progress_deadline = loop.time() + self._progress_timeout_seconds
- finally:
- if provider_stream is not None:
- await close_stream_input(
- provider_stream,
- owner="provider_executor",
- source="api",
- preserved_error=sys.exception(),
+ candidate_request = (
+ primary_request
+ if index == 0
+ else routed.request.model_copy(
+ update={"model": target.provider_model},
+ deep=True,
+ )
+ )
+ if index > 0:
+ provider.preflight_stream(
+ candidate_request,
+ reasoning=routed.reasoning,
+ )
+
+ provider_stream: AsyncIterator[str] | None = None
+ candidate_committed = False
+ advance_failure: ExecutionFailure | None = None
+ try:
+ provider_stream = provider.stream_response(
+ candidate_request,
+ input_tokens=input_tokens,
+ request_id=request_id,
+ response_model=gateway_model,
+ reasoning=routed.reasoning,
)
+ while True:
+ if loop.time() >= progress_deadline:
+ raise self._progress_timeout_failure(
+ request_id=request_id,
+ provider_id=target.provider_id,
+ )
+ progress_timeout = asyncio.timeout_at(progress_deadline)
+ try:
+ async with progress_timeout:
+ chunk = await anext(provider_stream)
+ except StopAsyncIteration:
+ break
+ except TimeoutError as exc:
+ if not progress_timeout.expired():
+ raise
+ raise self._progress_timeout_failure(
+ request_id=request_id,
+ provider_id=target.provider_id,
+ ) from exc
+ if not chunk:
+ await asyncio.sleep(0)
+ continue
+ if not candidate_committed:
+ candidate_committed = True
+ if index > 0:
+ self._trace_fallback_selected(
+ request_id=request_id,
+ wire_api=wire_api,
+ selected=target,
+ candidate_index=index + 1,
+ candidate_count=len(candidates),
+ )
+ yield chunk
+ progress_deadline = loop.time() + self._progress_timeout_seconds
+ except ExecutionFailure as failure:
+ if (
+ candidate_committed
+ or not failure.retryable
+ or index + 1 >= len(candidates)
+ ):
+ raise
+ advance_failure = failure
+ finally:
+ if provider_stream is not None:
+ await close_stream_input(
+ provider_stream,
+ owner="provider_executor",
+ source="api",
+ preserved_error=sys.exception(),
+ )
+
+ if advance_failure is None:
+ return
+ next_target = candidates[index + 1]
+ self._trace_fallback_started(
+ request_id=request_id,
+ wire_api=wire_api,
+ failed=target,
+ selected=next_target,
+ failure=advance_failure,
+ candidate_index=index + 2,
+ candidate_count=len(candidates),
+ )
stream_trace: dict[str, object] = {
"request_id": request_id,
- "provider_id": routed.resolved.provider_id,
+ "initial_provider_id": primary.provider_id,
"gateway_model": gateway_model,
}
if self._generation_id is not None:
diff --git a/src/free_claude_code/application/routing.py b/src/free_claude_code/application/routing.py
index 898656ac..37b6a85e 100644
--- a/src/free_claude_code/application/routing.py
+++ b/src/free_claude_code/application/routing.py
@@ -27,25 +27,35 @@ _ROUTE_SETTINGS = (
@dataclass(frozen=True, slots=True)
-class ResolvedModel:
- original_model: str
+class ProviderModelTarget:
+ """One canonical provider/model execution target."""
+
provider_id: str
provider_model: str
provider_model_ref: str
+
+
+@dataclass(frozen=True, slots=True)
+class ResolvedModelRoute:
+ """One public model resolved to a primary and ordered fallbacks."""
+
+ original_model: str
+ primary: ProviderModelTarget
+ fallbacks: tuple[ProviderModelTarget, ...]
reasoning_preference: ReasoningPreference
@dataclass(frozen=True, slots=True)
class RoutedMessagesRequest:
request: MessagesRequest
- resolved: ResolvedModel
+ resolved: ResolvedModelRoute
reasoning: ReasoningPolicy
@dataclass(frozen=True, slots=True)
class RoutedTokenCountRequest:
request: TokenCountRequest
- resolved: ResolvedModel
+ resolved: ResolvedModelRoute
class ModelRouter:
@@ -54,7 +64,7 @@ class ModelRouter:
def __init__(self, settings: Settings):
self._settings = settings
- def resolve(self, claude_model_name: str) -> ResolvedModel:
+ def resolve(self, claude_model_name: str) -> ResolvedModelRoute:
(
direct_provider_id,
direct_provider_model,
@@ -73,30 +83,52 @@ class ModelRouter:
direct_provider_model,
reasoning_preference.value,
)
- return ResolvedModel(
+ primary = self._target(direct_provider_id, direct_provider_model)
+ return ResolvedModelRoute(
original_model=claude_model_name,
- provider_id=direct_provider_id,
- provider_model=direct_provider_model,
- provider_model_ref=claude_model_name,
+ primary=primary,
+ fallbacks=self._fallback_targets(primary),
reasoning_preference=reasoning_preference,
)
provider_model_ref = self._resolve_model_ref(claude_model_name)
reasoning_preference = self._resolve_reasoning_preference(claude_model_name)
- provider_id = parse_provider_type(provider_model_ref)
- self._validate_provider_id(provider_id)
- provider_model = parse_model_name(provider_model_ref)
- if provider_model != claude_model_name:
+ primary = self._target_from_ref(provider_model_ref)
+ if primary.provider_model != claude_model_name:
logger.debug(
- "MODEL MAPPING: '{}' -> '{}'", claude_model_name, provider_model
+ "MODEL MAPPING: '{}' -> '{}'",
+ claude_model_name,
+ primary.provider_model,
)
- return ResolvedModel(
+ return ResolvedModelRoute(
original_model=claude_model_name,
+ primary=primary,
+ fallbacks=self._fallback_targets(primary),
+ reasoning_preference=reasoning_preference,
+ )
+
+ def _target_from_ref(self, provider_model_ref: str) -> ProviderModelTarget:
+ return self._target(
+ parse_provider_type(provider_model_ref),
+ parse_model_name(provider_model_ref),
+ )
+
+ def _target(self, provider_id: str, provider_model: str) -> ProviderModelTarget:
+ self._validate_provider_id(provider_id)
+ return ProviderModelTarget(
provider_id=provider_id,
provider_model=provider_model,
- provider_model_ref=provider_model_ref,
- reasoning_preference=reasoning_preference,
+ provider_model_ref=f"{provider_id}/{provider_model}",
+ )
+
+ def _fallback_targets(
+ self, primary: ProviderModelTarget
+ ) -> tuple[ProviderModelTarget, ...]:
+ configured = tuple(
+ self._target_from_ref(model_ref)
+ for model_ref in self._settings.model_fallbacks or ()
)
+ return tuple(target for target in configured if target != primary)
@staticmethod
def _validate_provider_id(provider_id: str) -> None:
@@ -161,7 +193,7 @@ class ModelRouter:
"""Return an internal routed request context."""
resolved = self.resolve(request.model)
routed = request.model_copy(deep=True)
- routed.model = resolved.provider_model
+ routed.model = resolved.primary.provider_model
return RoutedMessagesRequest(
request=routed,
resolved=resolved,
@@ -177,6 +209,6 @@ class ModelRouter:
"""Return an internal token-count request context."""
resolved = self.resolve(request.model)
routed = request.model_copy(
- update={"model": resolved.provider_model}, deep=True
+ update={"model": resolved.primary.provider_model}, deep=True
)
return RoutedTokenCountRequest(request=routed, resolved=resolved)
diff --git a/src/free_claude_code/config/admin/manifest.py b/src/free_claude_code/config/admin/manifest.py
index 485abadb..f7695838 100644
--- a/src/free_claude_code/config/admin/manifest.py
+++ b/src/free_claude_code/config/admin/manifest.py
@@ -91,7 +91,7 @@ _NON_PROVIDER_FIELDS: tuple[ConfigFieldSpec, ...] = (
"models",
"model",
settings_attr="model",
- description="Fallback provider/model route for all Claude model names.",
+ description="Provider/model used when no tier-specific override applies.",
),
ConfigFieldSpec(
"MODEL_FABLE",
@@ -125,6 +125,18 @@ _NON_PROVIDER_FIELDS: tuple[ConfigFieldSpec, ...] = (
settings_attr="model_haiku",
description="Select None to use the Default Model for Haiku requests.",
),
+ ConfigFieldSpec(
+ "MODEL_FALLBACKS",
+ "Fallback Models",
+ "models",
+ "model_list",
+ settings_attr="model_fallbacks",
+ description=(
+ "Tried in order after the selected model exhausts retryable provider "
+ "failures. Applies to every client. One request may reach multiple "
+ "providers and consume usage at each."
+ ),
+ ),
ConfigFieldSpec(
"REASONING_POLICY",
"Reasoning Policy",
diff --git a/src/free_claude_code/config/admin/specs.py b/src/free_claude_code/config/admin/specs.py
index 48b30340..e3a2969e 100644
--- a/src/free_claude_code/config/admin/specs.py
+++ b/src/free_claude_code/config/admin/specs.py
@@ -13,6 +13,7 @@ type FieldType = Literal[
"boolean",
"model",
"optional_model",
+ "model_list",
"select",
"textarea",
]
diff --git a/src/free_claude_code/config/admin/values.py b/src/free_claude_code/config/admin/values.py
index 365dd962..d60d7306 100644
--- a/src/free_claude_code/config/admin/values.py
+++ b/src/free_claude_code/config/admin/values.py
@@ -25,6 +25,8 @@ def normalize_for_env(value: object) -> str | None:
return "true" if value else "false"
if isinstance(value, Enum):
return str(value.value)
+ if isinstance(value, tuple) and all(isinstance(item, str) for item in value):
+ return ",".join(value)
return str(value).strip()
diff --git a/src/free_claude_code/config/model_refs.py b/src/free_claude_code/config/model_refs.py
index 19dc88bf..fb4487b0 100644
--- a/src/free_claude_code/config/model_refs.py
+++ b/src/free_claude_code/config/model_refs.py
@@ -19,18 +19,28 @@ class ChatModelConfig(Protocol):
model_opus: str | None
model_sonnet: str | None
model_haiku: str | None
+ model_fallbacks: tuple[str, ...] | None
+
+
+def split_provider_model_ref(model_ref: str) -> tuple[str, str]:
+ """Split one complete ``provider/model`` reference."""
+
+ provider_id, separator, model_id = model_ref.partition("/")
+ if not separator or not provider_id or not model_id:
+ raise ValueError("Model reference must contain provider and model names.")
+ return provider_id, model_id
def parse_provider_type(model_ref: str) -> str:
"""Extract provider type from any 'provider/model' string."""
- return model_ref.split("/", 1)[0]
+ return split_provider_model_ref(model_ref)[0]
def parse_model_name(model_ref: str) -> str:
"""Extract model name from any 'provider/model' string."""
- return model_ref.split("/", 1)[1]
+ return split_provider_model_ref(model_ref)[1]
def configured_chat_model_refs(
@@ -46,6 +56,7 @@ def configured_chat_model_refs(
settings.model_opus,
settings.model_sonnet,
settings.model_haiku,
+ *(settings.model_fallbacks or ()),
)
if model_ref is not None
)
diff --git a/src/free_claude_code/config/settings.py b/src/free_claude_code/config/settings.py
index 1fb91906..7a8d7ad3 100644
--- a/src/free_claude_code/config/settings.py
+++ b/src/free_claude_code/config/settings.py
@@ -29,6 +29,18 @@ def _empty_to_none(value: object) -> object:
return value
+def _parse_model_fallbacks(value: object) -> object:
+ if value is None:
+ return None
+ if isinstance(value, str):
+ if not value.strip():
+ return None
+ return tuple(part.strip() for part in value.split(","))
+ if isinstance(value, list | tuple) and not value:
+ return None
+ return value
+
+
NonEmptyString = Annotated[
str,
StringConstraints(strip_whitespace=True, min_length=1),
@@ -37,6 +49,26 @@ OptionalNonEmptyString = Annotated[
NonEmptyString | None,
BeforeValidator(_empty_to_none),
]
+OptionalModelFallbacks = Annotated[
+ tuple[NonEmptyString, ...] | None,
+ BeforeValidator(_parse_model_fallbacks),
+]
+
+
+def _validate_model_ref(value: str) -> str:
+ provider, separator, model = value.partition("/")
+ if not separator or not provider:
+ raise ValueError(
+ "Model must be prefixed with provider type. "
+ f"Valid providers: {', '.join(SUPPORTED_PROVIDER_IDS)}. "
+ "Format: provider_type/model/name"
+ )
+ if provider not in SUPPORTED_PROVIDER_IDS:
+ supported = ", ".join(f"'{item}'" for item in SUPPORTED_PROVIDER_IDS)
+ raise ValueError(f"Invalid provider: '{provider}'. Supported: {supported}")
+ if not model:
+ raise ValueError("Model reference must include a non-empty model suffix.")
+ return value
class Settings(BaseModel):
@@ -319,7 +351,7 @@ class Settings(BaseModel):
validation_alias="MODEL",
)
- # Per-model overrides (optional, falls back to MODEL)
+ # Per-model overrides (optional, use MODEL when unset)
# Each can use a different provider
model_fable: OptionalNonEmptyString = Field(
default=None, validation_alias="MODEL_FABLE"
@@ -333,6 +365,10 @@ class Settings(BaseModel):
model_haiku: OptionalNonEmptyString = Field(
default=None, validation_alias="MODEL_HAIKU"
)
+ model_fallbacks: OptionalModelFallbacks = Field(
+ default=None,
+ validation_alias="MODEL_FALLBACKS",
+ )
# ==================== Per-Provider Proxy ====================
openai_proxy: OptionalNonEmptyString = Field(
@@ -727,17 +763,19 @@ class Settings(BaseModel):
def validate_model_format(cls, v: str | None) -> str | None:
if v is None:
return None
- if "/" not in v:
- raise ValueError(
- f"Model must be prefixed with provider type. "
- f"Valid providers: {', '.join(SUPPORTED_PROVIDER_IDS)}. "
- f"Format: provider_type/model/name"
- )
- provider = v.split("/", 1)[0]
- if provider not in SUPPORTED_PROVIDER_IDS:
- supported = ", ".join(f"'{p}'" for p in SUPPORTED_PROVIDER_IDS)
- raise ValueError(f"Invalid provider: '{provider}'. Supported: {supported}")
- return v
+ return _validate_model_ref(v)
+
+ @field_validator("model_fallbacks")
+ @classmethod
+ def validate_model_fallbacks(
+ cls, value: tuple[str, ...] | None
+ ) -> tuple[str, ...] | None:
+ if value is None:
+ return None
+ validated = tuple(_validate_model_ref(model_ref) for model_ref in value)
+ if len(validated) != len(set(validated)):
+ raise ValueError("MODEL_FALLBACKS must not contain duplicate model refs.")
+ return validated
@model_validator(mode="after")
def check_nvidia_nim_api_key(self) -> Settings:
diff --git a/tests/api/model_fallback_support.py b/tests/api/model_fallback_support.py
new file mode 100644
index 00000000..ce7a4336
--- /dev/null
+++ b/tests/api/model_fallback_support.py
@@ -0,0 +1,176 @@
+"""Controlled provider boundary for model-fallback API and product tests."""
+
+from collections.abc import AsyncIterator, Iterator
+from contextlib import contextmanager
+from unittest.mock import patch
+
+from fastapi.testclient import TestClient
+
+from free_claude_code.application.errors import InvalidRequestError
+from free_claude_code.config.settings import Settings
+from free_claude_code.core.anthropic import MessagesRequest
+from free_claude_code.core.anthropic.streaming import format_sse_event
+from free_claude_code.core.failures import ExecutionFailure, FailureKind
+from free_claude_code.core.reasoning import ReasoningPolicy
+from tests.api.support import create_test_app
+
+
+def execution_failure(message: str, *, retryable: bool = True) -> ExecutionFailure:
+ return ExecutionFailure(
+ kind=FailureKind.OVERLOADED,
+ status_code=529,
+ message=message,
+ retryable=retryable,
+ )
+
+
+def text_stream(text: str, *, model: str) -> list[str]:
+ return [
+ format_sse_event(
+ "message_start",
+ {
+ "type": "message_start",
+ "message": {
+ "id": "msg_fallback",
+ "type": "message",
+ "role": "assistant",
+ "model": model,
+ "content": [],
+ "stop_reason": None,
+ "stop_sequence": None,
+ "usage": {"input_tokens": 3, "output_tokens": 0},
+ },
+ },
+ ),
+ format_sse_event(
+ "content_block_start",
+ {
+ "type": "content_block_start",
+ "index": 0,
+ "content_block": {"type": "text", "text": ""},
+ },
+ ),
+ format_sse_event(
+ "content_block_delta",
+ {
+ "type": "content_block_delta",
+ "index": 0,
+ "delta": {"type": "text_delta", "text": text},
+ },
+ ),
+ format_sse_event(
+ "content_block_stop",
+ {"type": "content_block_stop", "index": 0},
+ ),
+ format_sse_event(
+ "message_delta",
+ {
+ "type": "message_delta",
+ "delta": {"stop_reason": "end_turn", "stop_sequence": None},
+ "usage": {"input_tokens": 3, "output_tokens": 4},
+ },
+ ),
+ format_sse_event("message_stop", {"type": "message_stop"}),
+ ]
+
+
+class ControlledFallbackProvider:
+ def __init__(
+ self,
+ *,
+ failure: ExecutionFailure | None = None,
+ chunks_before_failure: tuple[str, ...] = (),
+ text: str | None = None,
+ preflight_error: InvalidRequestError | None = None,
+ ) -> None:
+ self._failure = failure
+ self._chunks_before_failure = chunks_before_failure
+ self._text = text
+ self._preflight_error = preflight_error
+ self.preflight_models: list[str] = []
+ self.stream_models: list[str] = []
+ self.response_models: list[str] = []
+ self.close_calls = 0
+
+ def preflight_stream(
+ self,
+ request: MessagesRequest,
+ *,
+ reasoning: ReasoningPolicy,
+ ) -> None:
+ del reasoning
+ self.preflight_models.append(request.model)
+ if self._preflight_error is not None:
+ raise self._preflight_error
+
+ async def stream_response(
+ self,
+ request: MessagesRequest,
+ input_tokens: int = 0,
+ *,
+ request_id: str | None = None,
+ response_model: str | None = None,
+ reasoning: ReasoningPolicy,
+ ) -> AsyncIterator[str]:
+ del input_tokens, request_id, reasoning
+ self.stream_models.append(request.model)
+ public_model = response_model or request.model
+ self.response_models.append(public_model)
+ try:
+ for chunk in self._chunks_before_failure:
+ yield chunk
+ if self._failure is not None:
+ raise self._failure
+ assert self._text is not None
+ for chunk in text_stream(self._text, model=public_model):
+ yield chunk
+ finally:
+ self.close_calls += 1
+
+
+@contextmanager
+def fallback_client(
+ primary: ControlledFallbackProvider,
+ fallback: ControlledFallbackProvider,
+) -> Iterator[TestClient]:
+ app = create_test_app(
+ Settings(
+ model="nvidia_nim/primary-model",
+ model_fallbacks=("groq/fallback-model",),
+ )
+ )
+
+ def resolve(
+ provider_id: str,
+ *,
+ lease: object,
+ ) -> ControlledFallbackProvider:
+ del lease
+ return {"nvidia_nim": primary, "groq": fallback}[provider_id]
+
+ with (
+ patch(
+ "free_claude_code.api.routes.resolve_provider",
+ side_effect=resolve,
+ ),
+ TestClient(app) as client,
+ ):
+ yield client
+
+
+def messages_payload(*, stream: bool) -> dict[str, object]:
+ return {
+ "model": "nvidia_nim/primary-model",
+ "messages": [{"role": "user", "content": "Hello"}],
+ "max_tokens": 32,
+ "stream": stream,
+ }
+
+
+def responses_payload() -> dict[str, object]:
+ return {
+ "model": "nvidia_nim/primary-model",
+ "input": "Hello",
+ "max_output_tokens": 32,
+ "stream": True,
+ }
diff --git a/tests/api/test_admin.py b/tests/api/test_admin.py
index 2bc556e9..778a37a3 100644
--- a/tests/api/test_admin.py
+++ b/tests/api/test_admin.py
@@ -34,6 +34,7 @@ def _set_home(monkeypatch, tmp_path: Path) -> None:
def _clear_process_config(monkeypatch) -> None:
for key in (
"MODEL",
+ "MODEL_FALLBACKS",
"NVIDIA_NIM_API_KEY",
"HUGGINGFACE_API_KEY",
"OPENROUTER_API_KEY",
@@ -442,7 +443,14 @@ def test_admin_config_masks_secrets_and_exposes_manifest(monkeypatch, tmp_path):
field["key"]: field["type"]
for field in body["fields"]
if field["key"]
- in {"MODEL", "MODEL_FABLE", "MODEL_OPUS", "MODEL_SONNET", "MODEL_HAIKU"}
+ in {
+ "MODEL",
+ "MODEL_FABLE",
+ "MODEL_OPUS",
+ "MODEL_SONNET",
+ "MODEL_HAIKU",
+ "MODEL_FALLBACKS",
+ }
}
assert model_field_types == {
"MODEL": "model",
@@ -450,7 +458,17 @@ def test_admin_config_masks_secrets_and_exposes_manifest(monkeypatch, tmp_path):
"MODEL_OPUS": "optional_model",
"MODEL_SONNET": "optional_model",
"MODEL_HAIKU": "optional_model",
+ "MODEL_FALLBACKS": "model_list",
}
+ fallback_field = next(
+ field for field in body["fields"] if field["key"] == "MODEL_FALLBACKS"
+ )
+ assert fallback_field["label"] == "Fallback Models"
+ assert fallback_field["value"] is None
+ assert fallback_field["nullable"] is True
+ assert fallback_field["restart_required"] is False
+ assert "every client" in fallback_field["description"]
+ assert "multiple providers" in fallback_field["description"]
reasoning_policy = next(
field for field in body["fields"] if field["key"] == "REASONING_POLICY"
)
@@ -626,6 +644,56 @@ def test_admin_apply_hot_publishes_provider_progress_timeout(monkeypatch, tmp_pa
assert "PROVIDER_PROGRESS_TIMEOUT=900" in managed_env.read_text(encoding="utf-8")
+def test_admin_apply_hot_publishes_ordered_model_fallbacks(monkeypatch, tmp_path):
+ _set_home(monkeypatch, tmp_path)
+ _clear_process_config(monkeypatch)
+ app = create_test_app()
+ value = "open_router/vendor/model-a,groq/vendor/model-b"
+
+ response = _local_client(app).post(
+ "/admin/api/config/apply",
+ json={"values": {"MODEL_FALLBACKS": value}},
+ )
+
+ assert response.status_code == 200
+ body = response.json()
+ assert body["applied"] is True
+ assert body["restart"]["required"] is False
+ assert provider_manager_for_app(app).current_settings().model_fallbacks == (
+ "open_router/vendor/model-a",
+ "groq/vendor/model-b",
+ )
+ managed_env = tmp_path / ".fcc" / ".env"
+ assert f"MODEL_FALLBACKS={value}" in managed_env.read_text(encoding="utf-8")
+
+ loaded = _local_client(app).get("/admin/api/config").json()
+ field = next(item for item in loaded["fields"] if item["key"] == "MODEL_FALLBACKS")
+ assert field["value"] == value
+ assert field["source"] == "managed_env"
+
+
+def test_admin_apply_null_removes_model_fallbacks(monkeypatch, tmp_path):
+ _set_home(monkeypatch, tmp_path)
+ _clear_process_config(monkeypatch)
+ env_file = tmp_path / ".fcc" / ".env"
+ env_file.parent.mkdir(parents=True)
+ env_file.write_text(
+ "FCC_CONFIG_SCHEMA=1\nMODEL_FALLBACKS=open_router/vendor/model-a\n",
+ encoding="utf-8",
+ )
+ app = create_test_app()
+
+ response = _local_client(app).post(
+ "/admin/api/config/apply",
+ json={"values": {"MODEL_FALLBACKS": None}},
+ )
+
+ assert response.status_code == 200
+ assert response.json()["applied"] is True
+ assert provider_manager_for_app(app).current_settings().model_fallbacks is None
+ assert "MODEL_FALLBACKS=" not in env_file.read_text(encoding="utf-8")
+
+
def test_admin_rejects_invalid_provider_progress_timeout_without_writing(
monkeypatch,
tmp_path,
@@ -732,6 +800,23 @@ def test_admin_validate_rejects_bad_model_shape(monkeypatch, tmp_path):
assert any("provider type" in error for error in body["errors"])
+def test_admin_validate_rejects_duplicate_model_fallbacks(monkeypatch, tmp_path):
+ _set_home(monkeypatch, tmp_path)
+ _clear_process_config(monkeypatch)
+ app = create_test_app()
+ duplicate = "groq/vendor/model,groq/vendor/model"
+
+ response = _local_client(app).post(
+ "/admin/api/config/validate",
+ json={"values": {"MODEL_FALLBACKS": duplicate}},
+ )
+
+ assert response.status_code == 200
+ body = response.json()
+ assert body["valid"] is False
+ assert any("duplicate" in error.lower() for error in body["errors"])
+
+
def test_admin_apply_writes_complete_managed_env_and_masks_preview(
monkeypatch, tmp_path
):
@@ -1286,6 +1371,29 @@ def test_admin_process_env_values_are_locked_and_not_written(monkeypatch, tmp_pa
assert "deepseek/managed-model" not in env_file.read_text("utf-8")
+def test_admin_process_model_fallbacks_are_locked_and_not_written(
+ monkeypatch, tmp_path
+):
+ _set_home(monkeypatch, tmp_path)
+ _clear_process_config(monkeypatch)
+ monkeypatch.setenv("MODEL_FALLBACKS", "open_router/process-model")
+ app = create_test_app()
+
+ config = _local_client(app).get("/admin/api/config").json()
+ field = next(item for item in config["fields"] if item["key"] == "MODEL_FALLBACKS")
+ assert field["locked"] is True
+ assert field["source"] == "process"
+
+ response = _local_client(app).post(
+ "/admin/api/config/apply",
+ json={"values": {"MODEL_FALLBACKS": "groq/managed-model"}},
+ )
+
+ assert response.status_code == 200
+ env_file = tmp_path / ".fcc" / ".env"
+ assert "MODEL_FALLBACKS=" not in env_file.read_text("utf-8")
+
+
def test_admin_never_reads_arbitrary_current_directory_env(monkeypatch, tmp_path):
_set_home(monkeypatch, tmp_path)
_clear_process_config(monkeypatch)
diff --git a/tests/api/test_model_fallback.py b/tests/api/test_model_fallback.py
new file mode 100644
index 00000000..9001fe7e
--- /dev/null
+++ b/tests/api/test_model_fallback.py
@@ -0,0 +1,198 @@
+"""Public model-fallback contracts across both wire APIs."""
+
+import pytest
+
+from free_claude_code.application.errors import InvalidRequestError
+from free_claude_code.core.anthropic.stream_contracts import parse_sse_text
+from free_claude_code.core.anthropic.streaming import format_sse_event
+from free_claude_code.core.failures import ExecutionFailure, FailureKind
+from tests.api.model_fallback_support import (
+ ControlledFallbackProvider,
+ execution_failure,
+ fallback_client,
+ messages_payload,
+ responses_payload,
+ text_stream,
+)
+
+
+def test_messages_fallback_emits_one_lifecycle_with_original_model() -> None:
+ primary = ControlledFallbackProvider(
+ failure=execution_failure("primary overloaded")
+ )
+ fallback = ControlledFallbackProvider(text="fallback worked")
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post("/v1/messages", json=messages_payload(stream=True))
+
+ assert response.status_code == 200
+ events = parse_sse_text(response.text)
+ event_names = [event.event for event in events]
+ assert event_names.count("message_start") == 1
+ assert event_names.count("message_stop") == 1
+ assert "fallback worked" in response.text
+ assert "primary overloaded" not in response.text
+ assert fallback.stream_models == ["fallback-model"]
+ assert fallback.response_models == ["nvidia_nim/primary-model"]
+ assert primary.close_calls == fallback.close_calls == 1
+
+
+def test_responses_fallback_emits_one_stable_response_lifecycle() -> None:
+ primary = ControlledFallbackProvider(
+ failure=execution_failure("primary overloaded")
+ )
+ fallback = ControlledFallbackProvider(text="fallback worked")
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post("/v1/responses", json=responses_payload())
+
+ assert response.status_code == 200
+ events = parse_sse_text(response.text)
+ names = [event.event for event in events]
+ assert names.count("response.created") == 1
+ assert names.count("response.completed") == 1
+ assert "response.failed" not in names
+ response_ids = {
+ event.data["response"]["id"]
+ for event in events
+ if isinstance(event.data.get("response"), dict)
+ }
+ assert len(response_ids) == 1
+ assert "fallback worked" in response.text
+
+
+ ("path", "payload"),
+ (
+ ("/v1/messages", messages_payload(stream=True)),
+ ("/v1/responses", responses_payload()),
+ ),
+)
+def test_final_failure_uses_last_candidate_typed_error(
+ path: str,
+ payload: dict[str, object],
+) -> None:
+ primary = ControlledFallbackProvider(
+ failure=execution_failure("primary overloaded")
+ )
+ final_failure = ExecutionFailure(
+ kind=FailureKind.RATE_LIMIT,
+ status_code=429,
+ message="fallback rate limited",
+ retryable=True,
+ )
+ fallback = ControlledFallbackProvider(failure=final_failure)
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post(path, json=payload)
+
+ assert response.status_code == 429
+ assert response.headers["x-should-retry"] == "false"
+ payload = response.json()
+ assert payload["error"]["type"] == "rate_limit_error"
+ assert payload["error"]["message"] == "fallback rate limited"
+ assert "primary overloaded" not in response.text
+
+
+ ("path", "payload"),
+ (
+ ("/v1/messages", messages_payload(stream=True)),
+ ("/v1/responses", responses_payload()),
+ ),
+)
+def test_lazy_fallback_preflight_error_remains_ordinary(
+ path: str,
+ payload: dict[str, object],
+) -> None:
+ primary = ControlledFallbackProvider(
+ failure=execution_failure("primary overloaded")
+ )
+ fallback = ControlledFallbackProvider(
+ preflight_error=InvalidRequestError("fallback configuration is invalid")
+ )
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post(path, json=payload)
+
+ assert response.status_code == 400
+ assert "x-should-retry" not in response.headers
+ payload = response.json()
+ assert payload["error"]["type"] == "invalid_request_error"
+ assert payload["error"]["message"] == "fallback configuration is invalid"
+ assert fallback.stream_models == []
+
+
+def test_postframe_failure_never_opens_fallback_for_streaming_messages() -> None:
+ first = format_sse_event(
+ "message_start",
+ {"type": "message_start", "message": {}},
+ )
+ primary = ControlledFallbackProvider(
+ chunks_before_failure=(first,),
+ failure=execution_failure("primary failed after start"),
+ )
+ fallback = ControlledFallbackProvider(text="must not run")
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post("/v1/messages", json=messages_payload(stream=True))
+
+ assert response.status_code == 200
+ events = parse_sse_text(response.text)
+ assert [event.event for event in events] == ["message_start", "error"]
+ assert fallback.preflight_models == []
+ assert fallback.stream_models == []
+
+
+def test_postframe_failure_never_opens_fallback_for_responses() -> None:
+ first = format_sse_event(
+ "message_start",
+ {
+ "type": "message_start",
+ "message": {
+ "id": "msg_primary",
+ "type": "message",
+ "role": "assistant",
+ "model": "nvidia_nim/primary-model",
+ "content": [],
+ "stop_reason": None,
+ "stop_sequence": None,
+ "usage": {"input_tokens": 3, "output_tokens": 0},
+ },
+ },
+ )
+ primary = ControlledFallbackProvider(
+ chunks_before_failure=(first,),
+ failure=execution_failure("primary failed after start"),
+ )
+ fallback = ControlledFallbackProvider(text="must not run")
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post("/v1/responses", json=responses_payload())
+
+ assert response.status_code == 200
+ events = parse_sse_text(response.text)
+ assert [event.event for event in events] == [
+ "response.created",
+ "response.failed",
+ ]
+ assert fallback.preflight_models == []
+ assert fallback.stream_models == []
+
+
+def test_nonstreaming_partial_failure_discards_content_without_fallback() -> None:
+ partial_text = "PARTIAL_ASSISTANT_OUTPUT"
+ partial = text_stream(partial_text, model="nvidia_nim/primary-model")[:3]
+ primary = ControlledFallbackProvider(
+ chunks_before_failure=tuple(partial),
+ failure=execution_failure("primary stream failed"),
+ )
+ fallback = ControlledFallbackProvider(text="must not run")
+
+ with fallback_client(primary, fallback) as client:
+ response = client.post("/v1/messages", json=messages_payload(stream=False))
+
+ assert response.status_code == 529
+ assert response.headers["x-should-retry"] == "false"
+ assert partial_text not in response.text
+ assert fallback.stream_models == []
diff --git a/tests/api/test_model_listing.py b/tests/api/test_model_listing.py
index f1073ffa..c250d912 100644
--- a/tests/api/test_model_listing.py
+++ b/tests/api/test_model_listing.py
@@ -11,6 +11,7 @@ def _settings(
model_fable: str | None = None,
model_opus: str | None = "open_router/anthropic/claude-opus",
model_haiku: str | None = "deepseek/deepseek-chat",
+ model_fallbacks: tuple[str, ...] | None = None,
) -> Settings:
return Settings.model_construct(
model=model,
@@ -18,6 +19,7 @@ def _settings(
model_opus=model_opus,
model_sonnet=None,
model_haiku=model_haiku,
+ model_fallbacks=model_fallbacks,
proxy_auth_enabled=False,
proxy_auth_token="freecc",
deepseek_api_key="deepseek-key",
@@ -135,6 +137,23 @@ def test_models_list_includes_cached_wafer_models():
assert "claude-3-freecc-no-thinking/wafer/MiniMax-M2.7" in ids
+def test_models_list_includes_fallback_refs_without_cached_discovery():
+ app = create_test_app(
+ _settings(
+ model_opus=None,
+ model_haiku=None,
+ model_fallbacks=("wafer/DeepSeek-V4-Pro",),
+ )
+ )
+
+ response = TestClient(app).get("/v1/models")
+
+ assert response.status_code == 200
+ ids = [item["id"] for item in response.json()["data"]]
+ assert "anthropic/wafer/DeepSeek-V4-Pro" in ids
+ assert "claude-3-freecc-no-thinking/wafer/DeepSeek-V4-Pro" in ids
+
+
def test_models_list_works_with_empty_discovery_catalog():
app = create_test_app(_settings())
diff --git a/tests/api/test_web_server_tools.py b/tests/api/test_web_server_tools.py
index a1bd907d..8c0db6c6 100644
--- a/tests/api/test_web_server_tools.py
+++ b/tests/api/test_web_server_tools.py
@@ -24,7 +24,8 @@ from free_claude_code.api.web_tools.streaming import stream_web_server_tool_resp
from free_claude_code.application.errors import InvalidRequestError
from free_claude_code.application.routing import (
ModelRouter,
- ResolvedModel,
+ ProviderModelTarget,
+ ResolvedModelRoute,
RoutedMessagesRequest,
)
from free_claude_code.config.provider_catalog import PROVIDER_CATALOG
@@ -71,15 +72,19 @@ class FixedProviderModelRouter(ModelRouter):
self, request: MessagesRequest
) -> RoutedMessagesRequest:
provider_model = self._fixed_provider_model or request.model
- resolved = ResolvedModel(
- original_model=request.model,
+ target = ProviderModelTarget(
provider_id=self._fixed_provider_id,
provider_model=provider_model,
provider_model_ref=f"{self._fixed_provider_id}/{provider_model}",
+ )
+ resolved = ResolvedModelRoute(
+ original_model=request.model,
+ primary=target,
+ fallbacks=(),
reasoning_preference=ReasoningPreference.OFF,
)
routed = request.model_copy(deep=True)
- routed.model = resolved.provider_model
+ routed.model = resolved.primary.provider_model
return RoutedMessagesRequest(
request=routed,
resolved=resolved,
diff --git a/tests/application/test_execution.py b/tests/application/test_execution.py
index 058c4e0f..a9581a11 100644
--- a/tests/application/test_execution.py
+++ b/tests/application/test_execution.py
@@ -6,8 +6,13 @@ from unittest.mock import MagicMock, patch
import pytest
+from free_claude_code.application.errors import InvalidRequestError
from free_claude_code.application.execution import ProviderExecutor
-from free_claude_code.application.routing import ResolvedModel, RoutedMessagesRequest
+from free_claude_code.application.routing import (
+ ProviderModelTarget,
+ ResolvedModelRoute,
+ RoutedMessagesRequest,
+)
from free_claude_code.config.reasoning import ReasoningPreference
from free_claude_code.core.anthropic.models import Message, MessagesRequest
from free_claude_code.core.async_iterators import AsyncCloseable
@@ -81,7 +86,14 @@ class EmptyForever:
EMPTY_FOREVER = EmptyForever()
-type StreamStep = str | WaitStep | EmptyForever | BaseException
+
+
+class DelayStep:
+ def __init__(self, seconds: float) -> None:
+ self.seconds = seconds
+
+
+type StreamStep = str | WaitStep | DelayStep | EmptyForever | BaseException
class ControlledProvider(FakeProvider):
@@ -112,6 +124,8 @@ class ControlledProvider(FakeProvider):
elif isinstance(step, WaitStep):
step.started.set()
await step.release.wait()
+ elif isinstance(step, DelayStep):
+ await asyncio.sleep(step.seconds)
elif isinstance(step, EmptyForever):
while True:
yield ""
@@ -131,6 +145,20 @@ class FailingPreflightProvider(FakeProvider):
raise ValueError("invalid provider request")
+class ApplicationErrorPreflightProvider(FakeProvider):
+ def __init__(self, error: InvalidRequestError) -> None:
+ super().__init__()
+ self._error = error
+
+ def preflight_stream(
+ self,
+ request: MessagesRequest,
+ *,
+ reasoning: ReasoningPolicy,
+ ) -> None:
+ raise self._error
+
+
class FailingStreamConstructionProvider(FakeProvider):
def stream_response(
self,
@@ -144,18 +172,27 @@ class FailingStreamConstructionProvider(FakeProvider):
raise RuntimeError("stream construction failed")
-def _routed_request() -> RoutedMessagesRequest:
+def _target(provider_id: str, provider_model: str) -> ProviderModelTarget:
+ return ProviderModelTarget(
+ provider_id=provider_id,
+ provider_model=provider_model,
+ provider_model_ref=f"{provider_id}/{provider_model}",
+ )
+
+
+def _routed_request(
+ *fallbacks: ProviderModelTarget,
+) -> RoutedMessagesRequest:
request = MessagesRequest(
model="provider-model",
messages=[Message(role="user", content="hello")],
)
return RoutedMessagesRequest(
request=request,
- resolved=ResolvedModel(
+ resolved=ResolvedModelRoute(
original_model="gateway-model",
- provider_id="provider",
- provider_model="provider-model",
- provider_model_ref="provider/provider-model",
+ primary=_target("provider", "provider-model"),
+ fallbacks=fallbacks,
reasoning_preference=ReasoningPreference.CLIENT,
),
reasoning=ReasoningPolicy.on(),
@@ -215,6 +252,355 @@ async def test_executor_uses_structural_provider_port_and_preflights_eagerly() -
assert provider.stream_close_calls == 1
+def _execution_failure(
+ message: str,
+ *,
+ retryable: bool = True,
+) -> ExecutionFailure:
+ return ExecutionFailure(
+ kind=FailureKind.OVERLOADED,
+ status_code=529,
+ message=message,
+ retryable=retryable,
+ )
+
+
+async def test_primary_success_never_resolves_fallback() -> None:
+ primary = FakeProvider()
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": primary, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(resolve, progress_timeout_seconds=60.0)
+ stream = executor.stream(
+ _routed_request(_target("fallback", "fallback-model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_primary_success",
+ )
+
+ assert [chunk async for chunk in stream] == ["event: message_stop\ndata: {}\n\n"]
+ assert resolved_ids == ["provider"]
+ assert fallback.preflight_calls == []
+ assert fallback.stream_calls == []
+
+
+async def test_retryable_preframe_failure_selects_fallback_after_closing_primary() -> (
+ None
+):
+ order: list[str] = []
+ primary = ControlledProvider([_execution_failure("primary overloaded")])
+ fallback = ControlledProvider(["fallback-frame"])
+
+ def resolve(provider_id: str) -> FakeProvider:
+ if provider_id == "fallback":
+ assert primary.stream_close_calls == 1
+ order.append("fallback-resolved")
+ return {"provider": primary, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(
+ resolve,
+ token_counter=lambda _messages, _system, _tools: 17,
+ progress_timeout_seconds=60.0,
+ generation_id=7,
+ )
+ routed = _routed_request(_target("fallback", "fallback-model"))
+
+ with patch("free_claude_code.application.execution.trace_event") as trace_mock:
+ chunks = [
+ chunk
+ async for chunk in executor.stream(
+ routed,
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_fallback",
+ )
+ ]
+
+ assert chunks == ["fallback-frame"]
+ assert order == ["fallback-resolved"]
+ assert primary.stream_close_calls == 1
+ assert fallback.stream_close_calls == 1
+ assert fallback.preflight_calls[0][0].model == "fallback-model"
+ assert fallback.stream_calls[0] == {
+ "request": fallback.preflight_calls[0][0],
+ "input_tokens": 17,
+ "request_id": "req_fallback",
+ "response_model": "gateway-model",
+ "reasoning": ReasoningPolicy.on(),
+ }
+ events = [call.kwargs for call in trace_mock.call_args_list]
+ started = next(
+ event
+ for event in events
+ if event.get("event") == "free_claude_code.model_fallback.started"
+ )
+ assert started == {
+ "stage": "execution",
+ "event": "free_claude_code.model_fallback.started",
+ "source": "application",
+ "request_id": "req_fallback",
+ "wire_api": "messages",
+ "from_provider_model_ref": "provider/provider-model",
+ "to_provider_model_ref": "fallback/fallback-model",
+ "candidate_index": 2,
+ "candidate_count": 2,
+ "failure_kind": "overloaded",
+ "status_code": 529,
+ "provider_retryable": True,
+ "generation_id": 7,
+ }
+ selected = next(
+ event
+ for event in events
+ if event.get("event") == "free_claude_code.model_fallback.selected"
+ )
+ assert selected["selected_provider_model_ref"] == "fallback/fallback-model"
+ assert selected["candidate_index"] == 2
+
+
+async def test_fallback_chain_preserves_exact_last_failure() -> None:
+ first = _execution_failure("first")
+ second = _execution_failure("second")
+ providers = {
+ "provider": ControlledProvider([first]),
+ "second": ControlledProvider([second]),
+ }
+ executor = ProviderExecutor(
+ providers.__getitem__,
+ progress_timeout_seconds=60.0,
+ )
+ stream = executor.stream(
+ _routed_request(_target("second", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_exhausted",
+ )
+
+ with pytest.raises(ExecutionFailure) as exc_info:
+ await anext(stream)
+
+ assert exc_info.value is second
+ assert all(provider.stream_close_calls == 1 for provider in providers.values())
+
+
+async def test_multiple_retryable_failures_walk_fallbacks_in_order() -> None:
+ providers = {
+ "provider": ControlledProvider([_execution_failure("primary")]),
+ "second": ControlledProvider([_execution_failure("second")]),
+ "third": ControlledProvider(["third-frame"]),
+ }
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return providers[provider_id]
+
+ executor = ProviderExecutor(resolve, progress_timeout_seconds=60.0)
+ stream = executor.stream(
+ _routed_request(
+ _target("second", "second-model"),
+ _target("third", "third-model"),
+ ),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_ordered_fallbacks",
+ )
+
+ assert [chunk async for chunk in stream] == ["third-frame"]
+ assert resolved_ids == ["provider", "second", "third"]
+ assert all(provider.stream_close_calls == 1 for provider in providers.values())
+
+
+async def test_empty_primary_completion_does_not_select_fallback() -> None:
+ primary = ControlledProvider([])
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": primary, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(resolve, progress_timeout_seconds=60.0)
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_empty_primary",
+ )
+
+ assert [chunk async for chunk in stream] == []
+ assert resolved_ids == ["provider"]
+ assert primary.stream_close_calls == 1
+
+
+async def test_unexpected_primary_failure_does_not_select_fallback() -> None:
+ primary = ControlledProvider([RuntimeError("unexpected")])
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": primary, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(resolve, progress_timeout_seconds=60.0)
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_unexpected_primary",
+ )
+
+ with pytest.raises(RuntimeError, match="unexpected"):
+ await anext(stream)
+
+ assert resolved_ids == ["provider"]
+ assert primary.stream_close_calls == 1
+
+
+async def test_nonretryable_failure_never_resolves_fallback() -> None:
+ primary_failure = _execution_failure("rejected", retryable=False)
+ primary = ControlledProvider([primary_failure])
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": primary, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(resolve, progress_timeout_seconds=60.0)
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_rejected",
+ )
+
+ with pytest.raises(ExecutionFailure) as exc_info:
+ await anext(stream)
+
+ assert exc_info.value is primary_failure
+ assert resolved_ids == ["provider"]
+
+
+async def test_failure_after_first_frame_never_selects_fallback() -> None:
+ primary_failure = _execution_failure("after frame")
+ primary = ControlledProvider(["first-frame", primary_failure])
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": primary, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(resolve, progress_timeout_seconds=60.0)
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_committed",
+ )
+
+ assert await anext(stream) == "first-frame"
+ with pytest.raises(ExecutionFailure) as exc_info:
+ await anext(stream)
+
+ assert exc_info.value is primary_failure
+ assert resolved_ids == ["provider"]
+
+
+async def test_lazy_fallback_preflight_application_error_stops_unchanged() -> None:
+ primary = ControlledProvider([_execution_failure("primary")])
+ preflight_error = InvalidRequestError("fallback is misconfigured")
+ fallback = ApplicationErrorPreflightProvider(preflight_error)
+ providers = {"provider": primary, "fallback": fallback}
+ executor = ProviderExecutor(
+ providers.__getitem__,
+ progress_timeout_seconds=60.0,
+ )
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_preflight",
+ )
+
+ with pytest.raises(InvalidRequestError) as exc_info:
+ await anext(stream)
+
+ assert exc_info.value is preflight_error
+ assert primary.stream_close_calls == 1
+ assert fallback.stream_calls == []
+
+
+async def test_candidate_requests_are_isolated_from_provider_mutation() -> None:
+ class MutatingProvider(ControlledProvider):
+ async def stream_response(
+ self,
+ request: MessagesRequest,
+ input_tokens: int = 0,
+ *,
+ request_id: str | None = None,
+ response_model: str | None = None,
+ reasoning: ReasoningPolicy,
+ ) -> AsyncIterator[str]:
+ request.messages[0].content = "mutated"
+ async for chunk in super().stream_response(
+ request,
+ input_tokens,
+ request_id=request_id,
+ response_model=response_model,
+ reasoning=reasoning,
+ ):
+ yield chunk
+
+ primary = MutatingProvider([_execution_failure("primary")])
+ fallback = FakeProvider()
+ providers = {"provider": primary, "fallback": fallback}
+ routed = _routed_request(_target("fallback", "model"))
+ executor = ProviderExecutor(
+ providers.__getitem__,
+ progress_timeout_seconds=60.0,
+ )
+
+ assert [
+ chunk
+ async for chunk in executor.stream(
+ routed,
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_isolated",
+ )
+ ]
+ fallback_request = fallback.stream_calls[0]["request"]
+ assert isinstance(fallback_request, MessagesRequest)
+ assert fallback_request.messages[0].content == "hello"
+ assert routed.request.messages[0].content == "hello"
+
+
@pytest.mark.asyncio
async def test_closing_executor_stream_closes_provider_stream_once() -> None:
provider = FakeProvider()
@@ -242,14 +628,21 @@ async def test_closing_executor_stream_closes_provider_stream_once() -> None:
@pytest.mark.asyncio
async def test_stream_construction_failure_remains_deferred_to_iteration() -> None:
provider = FailingStreamConstructionProvider()
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": provider, "fallback": fallback}[provider_id]
+
executor = ProviderExecutor(
- lambda _provider_id: provider,
+ resolve,
progress_timeout_seconds=60.0,
token_counter=lambda _messages, _system, _tools: 17,
)
stream = executor.stream(
- _routed_request(),
+ _routed_request(_target("fallback", "model")),
wire_api="messages",
raw_log_label="FULL_PAYLOAD",
raw_log_payload={},
@@ -259,6 +652,8 @@ async def test_stream_construction_failure_remains_deferred_to_iteration() -> No
with pytest.raises(RuntimeError, match="stream construction failed"):
await anext(stream)
+ assert resolved_ids == ["provider"]
+
def test_executor_preflight_failure_stays_before_token_count_and_stream() -> None:
provider = FailingPreflightProvider()
@@ -376,6 +771,48 @@ async def test_empty_chunks_do_not_renew_provider_progress() -> None:
assert provider.stream_close_calls == 1
+async def test_fallback_transition_does_not_reset_shared_progress_deadline() -> None:
+ timeout_seconds = 0.2
+ primary = ControlledProvider(
+ [DelayStep(0.06), _execution_failure("primary overloaded")]
+ )
+ fallback_wait = WaitStep()
+ fallback = ControlledProvider([fallback_wait])
+ providers = {"provider": primary, "fallback": fallback}
+ executor = ProviderExecutor(
+ providers.__getitem__,
+ progress_timeout_seconds=timeout_seconds,
+ )
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
+ request_id="req_shared_deadline",
+ )
+ loop = asyncio.get_running_loop()
+ started = loop.time()
+
+ with (
+ patch("free_claude_code.application.execution.trace_event") as trace_mock,
+ pytest.raises(ExecutionFailure) as exc_info,
+ ):
+ await anext(stream)
+
+ elapsed = loop.time() - started
+ assert fallback_wait.started.is_set()
+ assert exc_info.value.kind == FailureKind.TIMEOUT
+ assert elapsed < 0.24
+ assert primary.stream_close_calls == fallback.stream_close_calls == 1
+ timeout_trace = next(
+ call.kwargs
+ for call in trace_mock.call_args_list
+ if call.kwargs.get("event") == "free_claude_code.provider.progress_timeout"
+ )
+ assert timeout_trace["provider_id"] == "fallback"
+
+
@pytest.mark.asyncio
async def test_provider_owned_timeout_is_not_reclassified() -> None:
provider = ControlledProvider([TimeoutError("provider-owned timeout")])
@@ -395,9 +832,22 @@ async def test_provider_owned_timeout_is_not_reclassified() -> None:
async def test_cancelling_progress_wait_remains_cancellation() -> None:
wait = WaitStep()
provider = ControlledProvider([wait])
- stream = _executor_stream(
- provider,
- timeout_seconds=1.0,
+ fallback = FakeProvider()
+ resolved_ids: list[str] = []
+
+ def resolve(provider_id: str) -> FakeProvider:
+ resolved_ids.append(provider_id)
+ return {"provider": provider, "fallback": fallback}[provider_id]
+
+ executor = ProviderExecutor(
+ resolve,
+ progress_timeout_seconds=1.0,
+ )
+ stream = executor.stream(
+ _routed_request(_target("fallback", "model")),
+ wire_api="messages",
+ raw_log_label="FULL_PAYLOAD",
+ raw_log_payload={},
request_id="req_cancelled_progress",
)
task = asyncio.ensure_future(anext(stream))
@@ -408,3 +858,4 @@ async def test_cancelling_progress_wait_remains_cancellation() -> None:
await task
assert provider.stream_close_calls == 1
+ assert resolved_ids == ["provider"]
diff --git a/tests/application/test_routing.py b/tests/application/test_routing.py
index 207211ed..009b7c90 100644
--- a/tests/application/test_routing.py
+++ b/tests/application/test_routing.py
@@ -35,9 +35,10 @@ def test_model_router_resolves_default_model(settings):
resolved = ModelRouter(settings).resolve("claude-3-opus")
assert resolved.original_model == "claude-3-opus"
- assert resolved.provider_id == "nvidia_nim"
- assert resolved.provider_model == "fallback-model"
- assert resolved.provider_model_ref == "nvidia_nim/fallback-model"
+ assert resolved.primary.provider_id == "nvidia_nim"
+ assert resolved.primary.provider_model == "fallback-model"
+ assert resolved.primary.provider_model_ref == "nvidia_nim/fallback-model"
+ assert resolved.fallbacks == ()
assert resolved.reasoning_preference is ReasoningPreference.CLIENT
@@ -52,7 +53,9 @@ def test_model_router_applies_opus_override(settings):
routed = ModelRouter(settings).resolve_messages_request(request)
assert routed.request.model == "deepseek/deepseek-r1"
- assert routed.resolved.provider_model_ref == "open_router/deepseek/deepseek-r1"
+ assert (
+ routed.resolved.primary.provider_model_ref == "open_router/deepseek/deepseek-r1"
+ )
assert routed.resolved.original_model == "claude-opus-4-20250514"
assert routed.reasoning.control is ReasoningControl.DEFAULT
assert request.model == "claude-opus-4-20250514"
@@ -70,7 +73,10 @@ def test_model_router_applies_fable_override(settings):
)
assert routed.request.model == "anthropic/claude-fable-5"
- assert routed.resolved.provider_model_ref == "open_router/anthropic/claude-fable-5"
+ assert (
+ routed.resolved.primary.provider_model_ref
+ == "open_router/anthropic/claude-fable-5"
+ )
assert routed.resolved.original_model == "claude-fable-5"
@@ -113,7 +119,7 @@ def test_model_router_applies_haiku_override(settings):
)
assert routed.request.model == "qwen2.5-7b"
- assert routed.resolved.provider_model_ref == "lmstudio/qwen2.5-7b"
+ assert routed.resolved.primary.provider_model_ref == "lmstudio/qwen2.5-7b"
def test_model_router_applies_sonnet_override(settings):
@@ -129,7 +135,8 @@ def test_model_router_applies_sonnet_override(settings):
assert routed.request.model == "meta/llama-3.3-70b-instruct"
assert (
- routed.resolved.provider_model_ref == "nvidia_nim/meta/llama-3.3-70b-instruct"
+ routed.resolved.primary.provider_model_ref
+ == "nvidia_nim/meta/llama-3.3-70b-instruct"
)
@@ -144,9 +151,9 @@ def test_model_router_routes_prefixed_provider_model_directly(settings):
assert routed.request.model == "deepseek-chat"
assert routed.resolved.original_model == "deepseek/deepseek-chat"
- assert routed.resolved.provider_id == "deepseek"
- assert routed.resolved.provider_model == "deepseek-chat"
- assert routed.resolved.provider_model_ref == "deepseek/deepseek-chat"
+ assert routed.resolved.primary.provider_id == "deepseek"
+ assert routed.resolved.primary.provider_model == "deepseek-chat"
+ assert routed.resolved.primary.provider_model_ref == "deepseek/deepseek-chat"
def test_model_router_routes_explicit_opencode_zen_prefix(settings):
@@ -159,8 +166,8 @@ def test_model_router_routes_explicit_opencode_zen_prefix(settings):
)
assert routed.request.model == "kimi-k2.6"
- assert routed.resolved.provider_id == "opencode_zen"
- assert routed.resolved.provider_model_ref == "opencode_zen/kimi-k2.6"
+ assert routed.resolved.primary.provider_id == "opencode_zen"
+ assert routed.resolved.primary.provider_model_ref == "opencode_zen/kimi-k2.6"
def test_model_router_routes_wafer_provider_model_directly(settings):
@@ -173,9 +180,9 @@ def test_model_router_routes_wafer_provider_model_directly(settings):
)
assert routed.request.model == "DeepSeek-V4-Pro"
- assert routed.resolved.provider_id == "wafer"
- assert routed.resolved.provider_model == "DeepSeek-V4-Pro"
- assert routed.resolved.provider_model_ref == "wafer/DeepSeek-V4-Pro"
+ assert routed.resolved.primary.provider_id == "wafer"
+ assert routed.resolved.primary.provider_model == "DeepSeek-V4-Pro"
+ assert routed.resolved.primary.provider_model_ref == "wafer/DeepSeek-V4-Pro"
def test_model_router_routes_minimax_provider_model_directly(settings):
@@ -188,9 +195,9 @@ def test_model_router_routes_minimax_provider_model_directly(settings):
)
assert routed.request.model == "MiniMax-M3"
- assert routed.resolved.provider_id == "minimax"
- assert routed.resolved.provider_model == "MiniMax-M3"
- assert routed.resolved.provider_model_ref == "minimax/MiniMax-M3"
+ assert routed.resolved.primary.provider_id == "minimax"
+ assert routed.resolved.primary.provider_model == "MiniMax-M3"
+ assert routed.resolved.primary.provider_model_ref == "minimax/MiniMax-M3"
def test_model_router_routes_gateway_encoded_provider_model_directly(settings):
@@ -207,11 +214,11 @@ def test_model_router_routes_gateway_encoded_provider_model_directly(settings):
routed.resolved.original_model
== "anthropic/nvidia_nim/deepseek-ai/deepseek-v4-pro"
)
- assert routed.resolved.provider_id == "nvidia_nim"
- assert routed.resolved.provider_model == "deepseek-ai/deepseek-v4-pro"
+ assert routed.resolved.primary.provider_id == "nvidia_nim"
+ assert routed.resolved.primary.provider_model == "deepseek-ai/deepseek-v4-pro"
assert (
- routed.resolved.provider_model_ref
- == "anthropic/nvidia_nim/deepseek-ai/deepseek-v4-pro"
+ routed.resolved.primary.provider_model_ref
+ == "nvidia_nim/deepseek-ai/deepseek-v4-pro"
)
@@ -229,8 +236,8 @@ def test_model_router_routes_no_thinking_gateway_model_directly(settings):
routed.resolved.original_model
== "claude-3-freecc-no-thinking/nvidia_nim/deepseek-ai/deepseek-v4-pro"
)
- assert routed.resolved.provider_id == "nvidia_nim"
- assert routed.resolved.provider_model == "deepseek-ai/deepseek-v4-pro"
+ assert routed.resolved.primary.provider_id == "nvidia_nim"
+ assert routed.resolved.primary.provider_model == "deepseek-ai/deepseek-v4-pro"
assert routed.reasoning.control is ReasoningControl.OFF
@@ -245,8 +252,8 @@ def test_direct_provider_model_uses_root_policy_without_model_name_guessing(sett
)
)
- assert routed.resolved.provider_id == "open_router"
- assert routed.resolved.provider_model == "anthropic/claude-opus-4"
+ assert routed.resolved.primary.provider_id == "open_router"
+ assert routed.resolved.primary.provider_model == "anthropic/claude-opus-4"
assert routed.reasoning.effort is ReasoningEffort.LOW
@@ -263,6 +270,41 @@ def test_model_router_routes_token_count_request(settings):
assert request.model == "claude-3-haiku-20240307"
+def test_model_router_appends_global_fallbacks_and_skips_only_primary(settings):
+ settings.model_fallbacks = (
+ "nvidia_nim/fallback-model",
+ "nvidia_nim/alternate-model",
+ "open_router/vendor/model-b",
+ )
+
+ resolved = ModelRouter(settings).resolve("claude-2.1")
+
+ assert [target.provider_model_ref for target in resolved.fallbacks] == [
+ "nvidia_nim/alternate-model",
+ "open_router/vendor/model-b",
+ ]
+
+
+def test_direct_gateway_route_preserves_original_and_deduplicates_canonical_target(
+ settings,
+):
+ settings.model_fallbacks = (
+ "nvidia_nim/deepseek-ai/deepseek-v4-pro",
+ "groq/vendor/model-b",
+ )
+ model = "anthropic/nvidia_nim/deepseek-ai/deepseek-v4-pro"
+
+ resolved = ModelRouter(settings).resolve(model)
+
+ assert resolved.original_model == model
+ assert (
+ resolved.primary.provider_model_ref == "nvidia_nim/deepseek-ai/deepseek-v4-pro"
+ )
+ assert [target.provider_model_ref for target in resolved.fallbacks] == [
+ "groq/vendor/model-b"
+ ]
+
+
def test_model_router_logs_mapping(settings):
with patch("free_claude_code.application.routing.logger.debug") as mock_log:
ModelRouter(settings).resolve("claude-2.1")
diff --git a/tests/config/test_config.py b/tests/config/test_config.py
index f1090b2b..5bf6f24a 100644
--- a/tests/config/test_config.py
+++ b/tests/config/test_config.py
@@ -197,6 +197,42 @@ def test_optional_strings_share_one_normalization_rule() -> None:
assert settings.allowed_dir is None
[email protected]("value", [None, "", " ", (), []])
+def test_model_fallbacks_empty_values_disable_fallback(value: object) -> None:
+ settings = Settings.model_validate({"MODEL_FALLBACKS": value})
+
+ assert settings.model_fallbacks is None
+
+
+ "value",
+ [
+ "open_router/vendor/model-a, groq/vendor/model-b ",
+ ("open_router/vendor/model-a", " groq/vendor/model-b "),
+ ["open_router/vendor/model-a", "groq/vendor/model-b"],
+ ],
+)
+def test_model_fallbacks_preserve_order_and_trim_members(value: object) -> None:
+ settings = Settings.model_validate({"MODEL_FALLBACKS": value})
+
+ assert settings.model_fallbacks == (
+ "open_router/vendor/model-a",
+ "groq/vendor/model-b",
+ )
+
+
+ "value",
+ [
+ "open_router/vendor/model-a,,groq/vendor/model-b",
+ "open_router/vendor/model-a,open_router/vendor/model-a",
+ ],
+)
+def test_model_fallbacks_reject_blank_and_duplicate_members(value: str) -> None:
+ with pytest.raises(ValidationError):
+ Settings.model_validate({"MODEL_FALLBACKS": value})
+
+
@pytest.mark.parametrize(
"field",
["MODEL", "HOST", "WHISPER_MODEL", "LOG_LEVEL", "ANTHROPIC_AUTH_TOKEN"],
@@ -213,19 +249,33 @@ def test_model_validation_and_routing() -> None:
)
router = ModelRouter(settings)
- assert router.resolve("claude-opus-4").provider_model_ref == (
+ assert router.resolve("claude-opus-4").primary.provider_model_ref == (
"open_router/anthropic/claude-opus"
)
- assert router.resolve("unknown").provider_model_ref == "deepseek/fallback"
+ assert router.resolve("unknown").primary.provider_model_ref == "deepseek/fallback"
with pytest.raises(ValidationError, match="Invalid provider"):
Settings(model="unknown/model")
+ "field",
+ ["MODEL", "MODEL_FABLE", "MODEL_OPUS", "MODEL_SONNET", "MODEL_HAIKU"],
+)
+def test_model_settings_reject_empty_model_suffix(field: str) -> None:
+ with pytest.raises(ValidationError, match="model suffix"):
+ Settings.model_validate({field: "open_router/"})
+
+
def test_configured_chat_model_refs_are_unique() -> None:
settings = Settings(
model="deepseek/fallback",
model_fable="open_router/anthropic/claude-fable",
model_sonnet="deepseek/fallback",
+ model_fallbacks=(
+ "groq/vendor/model-a",
+ "open_router/anthropic/claude-fable",
+ "lmstudio/vendor/model-b",
+ ),
)
refs = configured_chat_model_refs(settings)
@@ -233,6 +283,8 @@ def test_configured_chat_model_refs_are_unique() -> None:
assert [ref.model_ref for ref in refs] == [
"deepseek/fallback",
"open_router/anthropic/claude-fable",
+ "groq/vendor/model-a",
+ "lmstudio/vendor/model-b",
]
diff --git a/tests/providers/test_model_discovery.py b/tests/providers/test_model_discovery.py
index 7c87cb58..c5e3444d 100644
--- a/tests/providers/test_model_discovery.py
+++ b/tests/providers/test_model_discovery.py
@@ -39,6 +39,7 @@ def _settings(
model_opus: str | None = None,
model_sonnet: str | None = None,
model_haiku: str | None = None,
+ model_fallbacks: tuple[str, ...] | None = None,
nvidia_nim_api_key: str = "",
open_router_api_key: str = "",
deepseek_api_key: str = "",
@@ -53,6 +54,7 @@ def _settings(
model_opus=model_opus,
model_sonnet=model_sonnet,
model_haiku=model_haiku,
+ model_fallbacks=model_fallbacks,
nvidia_nim_api_key=nvidia_nim_api_key,
open_router_api_key=open_router_api_key,
deepseek_api_key=deepseek_api_key,
@@ -392,7 +394,7 @@ class FakeProvider(BaseProvider):
@pytest.mark.asyncio
async def test_runtime_warm_caches_all_referenced_provider_models() -> None:
settings = _settings(
- model_opus="open_router/anthropic/claude-opus",
+ model_fallbacks=("open_router/anthropic/claude-opus",),
nvidia_nim_api_key="nim-key",
open_router_api_key="open-router-key",
)
diff --git a/uv.lock b/uv.lock
index 88c0652a..1976ac7a 100644
--- a/uv.lock
+++ b/uv.lock
@@ -607,7 +607,7 @@ wheels = [
[[package]]
name = "free-claude-code"
-version = "5.7.0"
+version = "5.8.0"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },