Skip to content

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

feat(backend): single-source LLM model catalog (cutover + Kimi K3) (#13627)

Nicholas Tindleb901d6942a9e
diff --git a/autogpt_platform/AGENTS.md b/autogpt_platform/AGENTS.md
index 582086b253..3c72d39071 100644
--- a/autogpt_platform/AGENTS.md
+++ b/autogpt_platform/AGENTS.md
@@ -49,6 +49,7 @@ AutoGPT Platform is a monorepo containing:
- **`dev`** is the main development branch. All PRs should target `dev`.
- **`master`** is the production branch. Only used for production releases.
+- Exception: **LLM-catalog-only diffs** (`backend/data/llm_registry/catalog.py`) may use a `hotfix/*` branch targeting `master` for incident-speed changes (model kills, routing swaps) — CD deploys on merge. See `docs/platform/contributing/managing-llm-models.md`.
### Creating Pull Requests
diff --git a/autogpt_platform/backend/.env.default b/autogpt_platform/backend/.env.default
index 3669aeeee8..b294318f9b 100644
--- a/autogpt_platform/backend/.env.default
+++ b/autogpt_platform/backend/.env.default
@@ -106,6 +106,13 @@ NVIDIA_API_KEY=
# OpenRouter-style provider/model slugs do not resolve. Extended-thinking
# mode is automatically downgraded to fast under this transport because
# the Claude Agent SDK CLI requires Anthropic's wire protocol.
+# Note: on the managed cloud platform (BEHAVE_AS=cloud) CHAT_*_MODEL is
+# the BOTTOM layer of AutoPilot model resolution — the LLM catalog's
+# copilot routing cells (backend/data/llm_registry/catalog.py) and the
+# per-user LaunchDarkly "copilot-model-routing" flag take precedence
+# when set. On self-hosted installs the catalog cells are skipped and
+# these env vars stay authoritative. See
+# docs/platform/contributing/managing-llm-models.md.
CHAT_API_KEY=
CHAT_BASE_URL=
CHAT_USE_LOCAL=false
diff --git a/autogpt_platform/backend/AGENTS.md b/autogpt_platform/backend/AGENTS.md
index 2b4db5f200..396a3400ec 100644
--- a/autogpt_platform/backend/AGENTS.md
+++ b/autogpt_platform/backend/AGENTS.md
@@ -126,6 +126,10 @@ Key models (defined in `schema.prisma`):
## Common Development Tasks
+### Adding / editing / retiring an LLM model
+
+Model definitions, costs, and AutoPilot routing are catalog-as-code in `backend/data/llm_registry/catalog.py` — edit the file, open a PR (catalog-only diffs may ride `hotfix/*`→`master` for incident-speed changes). The catalog is the single source: metadata and billing dicts are derived from it at import. A block-selectable model additionally needs one `LLMModel` name line in `backend/data/llm_registry/llm_models.py` (an import-time check enforces the pairing); copilot-only models need just the catalog entry. Retire a model with a catalog PR (`is_enabled: False`) plus `python -m backend.data.llm_registry.retire <slug> --replacement <slug> --yes` to migrate existing graph nodes (dry-run by default, revertable). Full reference: [Managing LLM Models](@../../docs/platform/contributing/managing-llm-models.md).
+
### Adding a new block
Follow the comprehensive [Block SDK Guide](@../../docs/platform/block-sdk-guide.md) which covers:
diff --git a/autogpt_platform/backend/backend/api/features/admin/execution_analytics_routes.py b/autogpt_platform/backend/backend/api/features/admin/execution_analytics_routes.py
index 45912dae67..c101e1e53d 100644
--- a/autogpt_platform/backend/backend/api/features/admin/execution_analytics_routes.py
+++ b/autogpt_platform/backend/backend/api/features/admin/execution_analytics_routes.py
@@ -7,7 +7,7 @@ from autogpt_libs.auth import get_user_id, requires_admin_user
from fastapi import APIRouter, HTTPException, Security
from pydantic import BaseModel, Field
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import DEFAULT_LLM_MODEL, LLMModel
from backend.data.analytics import (
AccuracyTrendsResponse,
get_accuracy_trends_and_alerts,
@@ -125,11 +125,11 @@ async def get_execution_analytics_config(
"""
logger.info(f"Admin user {admin_user_id} requesting execution analytics config")
- # Generate model list from LlmModel enum with provider information
+ # Generate model list from LLMModel enum with provider information
available_models = []
# Function to generate friendly display names from model values
- def generate_model_label(model: LlmModel) -> str:
+ def generate_model_label(model: LLMModel) -> str:
"""Generate a user-friendly label from the model enum value."""
value = model.value
@@ -176,9 +176,10 @@ async def get_execution_analytics_config(
# Return with provider prefix for clarity
return f"{provider_name}: {model_name}"
- # Include all LlmModel values (no more filtering by hardcoded list)
- recommended_model = LlmModel.GPT4O_MINI.value
- for model in LlmModel:
+ # Label the platform default so the admin dropdown's "(Recommended)"
+ # tracks the catalog's is_recommended model instead of a hardcoded slug.
+ recommended_model = DEFAULT_LLM_MODEL.value
+ for model in LLMModel:
label = generate_model_label(model)
# Add "(Recommended)" suffix to the recommended model
if model.value == recommended_model:
diff --git a/autogpt_platform/backend/backend/api/features/builder/db.py b/autogpt_platform/backend/backend/api/features/builder/db.py
index 0a99132bc9..921007bf11 100644
--- a/autogpt_platform/backend/backend/api/features/builder/db.py
+++ b/autogpt_platform/backend/backend/api/features/builder/db.py
@@ -19,7 +19,7 @@ from backend.blocks._base import (
BlockSchema,
BlockType,
)
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import LLMModel
from backend.integrations.providers import ProviderName
from backend.util.cache import cached
from backend.util.models import Pagination
@@ -37,7 +37,7 @@ from .model import (
)
logger = logging.getLogger(__name__)
-llm_models = [name.name.lower().replace("_", " ") for name in LlmModel]
+llm_models = [name.name.lower().replace("_", " ") for name in LLMModel]
MAX_LIBRARY_AGENT_RESULTS = 100
MAX_MARKETPLACE_AGENT_RESULTS = 100
@@ -46,7 +46,7 @@ MIN_SCORE_FOR_FILTERED_RESULTS = 10.0
# Boost blocks over marketplace agents in search results
BLOCK_SCORE_BOOST = 50.0
-# Bonus when a block exposes an LlmModel field and the query names an LLM model
+# Bonus when a block exposes an LLMModel field and the query names an LLM model
LLM_MODEL_MATCH_BONUS = 20.0
# Block IDs to exclude from search results
@@ -450,7 +450,7 @@ def _text_search_blocks(
Scoring:
- Base: text relevance via _score_primary_fields, plus BLOCK_SCORE_BOOST
to prioritize blocks over marketplace agents in combined results
- - +20 if the block has an LlmModel field and the query matches an LLM model name
+ - +20 if the block has an LLMModel field and the query matches an LLM model name
"""
if not include_blocks and not include_integrations:
return [], 0, 0
@@ -701,7 +701,7 @@ def _contains_type(annotation: Any, target: type) -> bool:
def _schema_has_llm_model(schema_cls: type[BlockSchema]) -> bool:
return any(
- _contains_type(field.annotation, LlmModel)
+ _contains_type(field.annotation, LLMModel)
for field in schema_cls.model_fields.values()
)
diff --git a/autogpt_platform/backend/backend/api/features/chat/routes.py b/autogpt_platform/backend/backend/api/features/chat/routes.py
index b383db1a7b..81a38dbe1e 100644
--- a/autogpt_platform/backend/backend/api/features/chat/routes.py
+++ b/autogpt_platform/backend/backend/api/features/chat/routes.py
@@ -22,7 +22,7 @@ from backend.copilot.active_turns import (
inflight_turn_limit_message,
)
from backend.copilot.builder_context import resolve_session_permissions
-from backend.copilot.config import ChatConfig, CopilotLlmModel, CopilotMode
+from backend.copilot.config import ChatConfig, CopilotLLMModel, CopilotMode
from backend.copilot.db import get_chat_messages_paginated
from backend.copilot.executor.utils import enqueue_cancel_task, schedule_chat_turn
from backend.copilot.model import (
@@ -216,7 +216,7 @@ class StreamChatRequest(BaseModel):
description="Autopilot mode: 'fast' for baseline LLM, 'extended_thinking' for Claude Agent SDK. "
"If None, uses the server default (extended_thinking).",
)
- model: CopilotLlmModel | None = Field(
+ model: CopilotLLMModel | None = Field(
default=None,
description="Model tier: 'standard' for the default model, 'advanced' for the highest-capability model. "
"If None, the server applies per-user LD targeting then falls back to config.",
diff --git a/autogpt_platform/backend/backend/api/features/search/content_handlers.py b/autogpt_platform/backend/backend/api/features/search/content_handlers.py
index 7dad60ebbc..0e3d0acdf1 100644
--- a/autogpt_platform/backend/backend/api/features/search/content_handlers.py
+++ b/autogpt_platform/backend/backend/api/features/search/content_handlers.py
@@ -18,7 +18,7 @@ from typing import TYPE_CHECKING, Any, get_args, get_origin
from prisma.enums import ContentType
from backend.blocks import get_blocks
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import LLMModel
from backend.data.db import query_raw_with_schema
from backend.util.docs import get_docs_root
from backend.util.text import split_camelcase
@@ -363,9 +363,9 @@ def _build_block_content_item(block_id: str, block: AnyBlockSchema) -> ContentIt
for provider in info.provider
]
- # Check if block has LlmModel field in input schema
+ # Check if block has LLMModel field in input schema
has_llm_model_field = any(
- _contains_type(field.annotation, LlmModel)
+ _contains_type(field.annotation, LLMModel)
for field in block.input_schema.model_fields.values()
)
diff --git a/autogpt_platform/backend/backend/api/rest_api.py b/autogpt_platform/backend/backend/api/rest_api.py
index 76136f1179..edd9bbb007 100644
--- a/autogpt_platform/backend/backend/api/rest_api.py
+++ b/autogpt_platform/backend/backend/api/rest_api.py
@@ -54,6 +54,7 @@ import backend.api.features.workspace.routes as team_routes
import backend.data.block
import backend.data.db
import backend.data.graph
+import backend.data.llm_registry
import backend.data.org_migration
import backend.data.redis_client
import backend.data.user
@@ -160,6 +161,9 @@ async def lifespan_context(app: fastapi.FastAPI):
await backend.integrations.webhooks.utils.migrate_legacy_triggered_graphs()
await backend.data.org_migration.run_migration()
+ # Fail-hard: the catalog is load-bearing — a broken load stops the boot.
+ backend.data.llm_registry.load_catalog()
+
with launch_darkly_context():
yield
diff --git a/autogpt_platform/backend/backend/blocks/ai_condition.py b/autogpt_platform/backend/backend/blocks/ai_condition.py
index db8c023b99..153f630857 100644
--- a/autogpt_platform/backend/backend/blocks/ai_condition.py
+++ b/autogpt_platform/backend/backend/blocks/ai_condition.py
@@ -14,7 +14,7 @@ from backend.blocks.llm import (
AIBlockBase,
AICredentials,
AICredentialsField,
- LlmModel,
+ LLMModel,
LLMResponse,
llm_call,
)
@@ -76,7 +76,7 @@ class AIConditionBlock(AIBlockBase):
placeholder="Leave empty to use input_value, or enter a specific value",
default=None,
)
- model: LlmModel = SchemaField(
+ model: LLMModel = SchemaField(
title="LLM Model",
default=DEFAULT_LLM_MODEL,
description="The language model to use for evaluating the condition.",
@@ -134,7 +134,7 @@ class AIConditionBlock(AIBlockBase):
async def llm_call(
self,
credentials: APIKeyCredentials,
- llm_model: LlmModel,
+ llm_model: LLMModel,
prompt: list,
max_tokens: int,
) -> LLMResponse:
diff --git a/autogpt_platform/backend/backend/blocks/llm.py b/autogpt_platform/backend/backend/blocks/llm.py
index 05b0261464..a18567b406 100644
--- a/autogpt_platform/backend/backend/blocks/llm.py
+++ b/autogpt_platform/backend/backend/blocks/llm.py
@@ -6,10 +6,9 @@ import math
import re
import secrets
from abc import ABC
-from collections.abc import Mapping
-from enum import Enum, EnumMeta
+from enum import Enum
from json import JSONDecodeError
-from typing import Any, Iterable, List, Literal, NamedTuple, Optional, cast
+from typing import Any, Iterable, List, Literal, Optional, cast
import anthropic
import openai
@@ -24,6 +23,14 @@ from backend.blocks._base import (
BlockSchemaInput,
BlockSchemaOutput,
)
+from backend.data.llm_registry.llm_models import DEFAULT_LLM_MODEL as DEFAULT_LLM_MODEL
+from backend.data.llm_registry.llm_models import (
+ LEGACY_MODEL_MAPPINGS as LEGACY_MODEL_MAPPINGS,
+)
+from backend.data.llm_registry.llm_models import MODEL_METADATA as MODEL_METADATA
+from backend.data.llm_registry.llm_models import LLMModel as LLMModel
+from backend.data.llm_registry.llm_models import LLMModelMeta as LLMModelMeta
+from backend.data.llm_registry.llm_models import ModelMetadata as ModelMetadata
from backend.data.model import (
APIKeyCredentials,
CredentialsField,
@@ -92,741 +99,11 @@ def AICredentialsField() -> AICredentials:
description="API key for the LLM provider.",
discriminator="model",
discriminator_mapping={
- model.value: model.metadata.provider for model in LlmModel
+ model.value: model.metadata.provider for model in LLMModel
},
)
-class ModelMetadata(NamedTuple):
- provider: str
- context_window: int
- max_output_tokens: int | None
- display_name: str
- provider_name: str
- creator_name: str
- price_tier: Literal[1, 2, 3]
-
-
-class LlmModelMeta(EnumMeta):
- pass
-
-
-class LlmModel(str, Enum, metaclass=LlmModelMeta):
- @classmethod
- def _missing_(cls, value: object) -> "LlmModel | None":
- """Resolve provider-prefixed model names.
-
- Handles two shapes:
-
- 1. OpenRouter aliases for Anthropic models whose direct-API
- identifier carries a snapshot suffix that the OpenRouter slug
- drops — e.g. ``anthropic/claude-haiku-4-5`` ↔ enum value
- ``claude-haiku-4-5-20251001``. Looked up via
- ``_OPENROUTER_ALIASES`` (defined below the class so it can hold
- ``LlmModel`` members directly).
- 2. Generic provider prefix strip — e.g.
- ``anthropic/claude-sonnet-4-6`` → ``claude-sonnet-4-6``.
- """
- if not isinstance(value, str):
- return None
- alias = _OPENROUTER_ALIASES.get(value)
- if alias is not None:
- return alias
- if "/" in value:
- stripped = value.split("/", 1)[1]
- try:
- return cls(stripped)
- except ValueError:
- return None
- return None
-
- # OpenAI models — O-series reasoning
- O4_MINI = "o4-mini"
- O3 = "o3-2025-04-16"
- O3_MINI = "o3-mini"
- O3_PRO = "o3-pro"
- O1 = "o1"
- O1_MINI = "o1-mini"
- # GPT-5.6 models (current flagship)
- GPT5_6_SOL = "gpt-5.6-sol"
- GPT5_6_TERRA = "gpt-5.6-terra"
- GPT5_6_LUNA = "gpt-5.6-luna"
- # GPT-5.5 models
- GPT5_5 = "gpt-5.5-2026-04-23"
- GPT5_5_PRO = "gpt-5.5-pro"
- # GPT-5.4 models
- GPT5_4 = "gpt-5.4-2026-03-05"
- GPT5_4_MINI = "gpt-5.4-mini-2026-03-17"
- GPT5_4_NANO = "gpt-5.4-nano-2026-03-17"
- GPT5_4_PRO = "gpt-5.4-pro"
- # GPT-5.3 models
- GPT5_3 = "gpt-5.3-chat-latest"
- GPT5_3_CODEX = "gpt-5.3-codex"
- # GPT-5.2 models
- GPT5_2 = "gpt-5.2-2025-12-11"
- GPT5_2_PRO = "gpt-5.2-pro"
- # GPT-5.1 models
- GPT5_1 = "gpt-5.1-2025-11-13"
- GPT5_1_CODEX = "gpt-5.1-codex"
- # GPT-5 models
- GPT5 = "gpt-5-2025-08-07"
- GPT5_MINI = "gpt-5-mini-2025-08-07"
- GPT5_NANO = "gpt-5-nano-2025-08-07"
- GPT5_PRO = "gpt-5-pro"
- GPT5_CHAT = "gpt-5-chat-latest"
- # GPT-4.x models
- GPT41 = "gpt-4.1-2025-04-14"
- GPT41_MINI = "gpt-4.1-mini-2025-04-14"
- GPT41_NANO = "gpt-4.1-nano"
- GPT4O_MINI = "gpt-4o-mini"
- GPT4O = "gpt-4o"
- # Anthropic models
- CLAUDE_4_5_OPUS = "claude-opus-4-5-20251101"
- CLAUDE_4_5_SONNET = "claude-sonnet-4-5-20250929"
- CLAUDE_4_5_HAIKU = "claude-haiku-4-5-20251001"
- CLAUDE_4_6_OPUS = "claude-opus-4-6"
- CLAUDE_4_7_OPUS = "claude-opus-4-7"
- CLAUDE_4_6_SONNET = "claude-sonnet-4-6"
- # AI/ML API models
- AIML_API_LLAMA3_3_70B = "meta-llama/Llama-3.3-70B-Instruct-Turbo"
- # Groq models
- LLAMA3_3_70B = "llama-3.3-70b-versatile"
- LLAMA3_1_8B = "llama-3.1-8b-instant"
- # Ollama models
- OLLAMA_LLAMA3_3 = "llama3.3"
- OLLAMA_LLAMA3_2 = "llama3.2"
- OLLAMA_LLAMA3_8B = "llama3"
- OLLAMA_LLAMA3_405B = "llama3.1:405b"
- OLLAMA_DOLPHIN = "dolphin-mistral:latest"
- # OpenRouter models
- OPENAI_GPT_OSS_120B = "openai/gpt-oss-120b"
- OPENAI_GPT_OSS_20B = "openai/gpt-oss-20b"
- GEMINI_2_5_PRO = "google/gemini-2.5-pro"
- GEMINI_3_1_PRO_PREVIEW = "google/gemini-3.1-pro-preview"
- GEMINI_3_FLASH_PREVIEW = "google/gemini-3-flash-preview"
- GEMINI_2_5_FLASH = "google/gemini-2.5-flash"
- GEMINI_2_0_FLASH = "google/gemini-2.0-flash-001"
- GEMINI_3_1_FLASH_LITE_PREVIEW = "google/gemini-3.1-flash-lite-preview"
- GEMINI_2_5_FLASH_LITE = "google/gemini-2.5-flash-lite"
- GEMINI_2_0_FLASH_LITE = "google/gemini-2.0-flash-lite-001"
- MISTRAL_LARGE_3 = "mistralai/mistral-large-2512"
- MISTRAL_MEDIUM_3_1 = "mistralai/mistral-medium-3.1"
- MISTRAL_SMALL_3_2 = "mistralai/mistral-small-3.2-24b-instruct"
- CODESTRAL = "mistralai/codestral-2508"
- COHERE_COMMAND_A_03_2025 = "cohere/command-a-03-2025"
- COHERE_COMMAND_A_TRANSLATE_08_2025 = "cohere/command-a-translate-08-2025"
- COHERE_COMMAND_A_REASONING_08_2025 = "cohere/command-a-reasoning-08-2025"
- COHERE_COMMAND_A_VISION_07_2025 = "cohere/command-a-vision-07-2025"
- DEEPSEEK_CHAT = "deepseek/deepseek-chat" # Actually: DeepSeek V3
- DEEPSEEK_R1_0528 = "deepseek/deepseek-r1-0528"
- PERPLEXITY_SONAR = "perplexity/sonar"
- PERPLEXITY_SONAR_PRO = "perplexity/sonar-pro"
- PERPLEXITY_SONAR_REASONING_PRO = "perplexity/sonar-reasoning-pro"
- PERPLEXITY_SONAR_DEEP_RESEARCH = "perplexity/sonar-deep-research"
- NOUSRESEARCH_HERMES_3_LLAMA_3_1_405B = "nousresearch/hermes-3-llama-3.1-405b"
- NOUSRESEARCH_HERMES_3_LLAMA_3_1_70B = "nousresearch/hermes-3-llama-3.1-70b"
- AMAZON_NOVA_LITE_V1 = "amazon/nova-lite-v1"
- AMAZON_NOVA_MICRO_V1 = "amazon/nova-micro-v1"
- AMAZON_NOVA_PRO_V1 = "amazon/nova-pro-v1"
- MICROSOFT_PHI_4 = "microsoft/phi-4"
- GRYPHE_MYTHOMAX_L2_13B = "gryphe/mythomax-l2-13b"
- META_LLAMA_4_SCOUT = "meta-llama/llama-4-scout"
- META_LLAMA_4_MAVERICK = "meta-llama/llama-4-maverick"
- GROK_3 = "x-ai/grok-3"
- GROK_4 = "x-ai/grok-4"
- GROK_4_FAST = "x-ai/grok-4-fast"
- GROK_4_1_FAST = "x-ai/grok-4.1-fast"
- GROK_4_20 = "x-ai/grok-4.20"
- GROK_4_20_MULTI_AGENT = "x-ai/grok-4.20-multi-agent"
- GROK_CODE_FAST_1 = "x-ai/grok-code-fast-1"
- KIMI_K2_5 = "moonshotai/kimi-k2.5"
- KIMI_K2_6 = "moonshotai/kimi-k2.6"
- KIMI_K2_THINKING = "moonshotai/kimi-k2-thinking"
- QWEN3_235B_A22B_THINKING = "qwen/qwen3-235b-a22b-thinking-2507"
- QWEN3_CODER = "qwen/qwen3-coder"
- # Z.ai (Zhipu) models
- ZAI_GLM_4_6 = "z-ai/glm-4.6"
- ZAI_GLM_4_6V = "z-ai/glm-4.6v"
- ZAI_GLM_4_7 = "z-ai/glm-4.7"
- ZAI_GLM_4_7_FLASH = "z-ai/glm-4.7-flash"
- ZAI_GLM_5 = "z-ai/glm-5"
- ZAI_GLM_5_TURBO = "z-ai/glm-5-turbo"
- ZAI_GLM_5V_TURBO = "z-ai/glm-5v-turbo"
- # Llama API models
- LLAMA_API_LLAMA_4_SCOUT = "Llama-4-Scout-17B-16E-Instruct-FP8"
- LLAMA_API_LLAMA4_MAVERICK = "Llama-4-Maverick-17B-128E-Instruct-FP8"
- LLAMA_API_LLAMA3_3_8B = "Llama-3.3-8B-Instruct"
- LLAMA_API_LLAMA3_3_70B = "Llama-3.3-70B-Instruct"
- # v0 by Vercel models
- V0_1_5_MD = "v0-1.5-md"
- V0_1_5_LG = "v0-1.5-lg"
- V0_1_0_MD = "v0-1.0-md"
-
- @classmethod
- def __get_pydantic_json_schema__(cls, schema, handler):
- json_schema = handler(schema)
- llm_model_metadata = {}
- for model in cls:
- model_name = model.value
- metadata = model.metadata
- llm_model_metadata[model_name] = {
- "creator": metadata.creator_name,
- "creator_name": metadata.creator_name,
- "title": metadata.display_name,
- "provider": metadata.provider,
- "provider_name": metadata.provider_name,
- "name": model_name,
- "price_tier": metadata.price_tier,
- }
- json_schema["llm_model"] = True
- json_schema["llm_model_metadata"] = llm_model_metadata
- return json_schema
-
- @property
- def metadata(self) -> ModelMetadata:
- return MODEL_METADATA[self]
-
- @property
- def provider(self) -> str:
- return self.metadata.provider
-
- @property
- def context_window(self) -> int:
- return self.metadata.context_window
-
- @property
- def max_output_tokens(self) -> int | None:
- return self.metadata.max_output_tokens
-
-
-# OpenRouter exposes Anthropic models under canonical ``anthropic/<model>``
-# slugs that drop the snapshot-date suffix Anthropic's own API uses
-# (``claude-haiku-4-5-20251001`` → ``anthropic/claude-haiku-4-5``). The
-# generic provider-prefix strip in ``_missing_`` can't reverse the date
-# truncation, so map the OpenRouter slugs to ``LlmModel`` members here.
-# Only models whose canonical enum value carries a ``-YYYYMMDD`` snapshot
-# suffix need entries; values without a snapshot (4.6/4.7+) are already
-# covered by the prefix-strip path alone. Stored as ``LlmModel`` instances
-# (not strings) so a rename or snapshot rotation on the enum follows the
-# alias automatically — a stale entry becomes a load-time ``AttributeError``
-# rather than a silent ``_missing_`` miss at runtime.
-_OPENROUTER_ALIASES: Mapping[str, LlmModel] = {
- "anthropic/claude-haiku-4-5": LlmModel.CLAUDE_4_5_HAIKU,
- "anthropic/claude-opus-4-5": LlmModel.CLAUDE_4_5_OPUS,
- "anthropic/claude-sonnet-4-5": LlmModel.CLAUDE_4_5_SONNET,
- "openai/gpt-5.4": LlmModel.GPT5_4,
- "openai/gpt-5.4-mini": LlmModel.GPT5_4_MINI,
- "openai/gpt-5.4-nano": LlmModel.GPT5_4_NANO,
- "openai/gpt-5.5": LlmModel.GPT5_5,
-}
-
-
-MODEL_METADATA = {
- # https://platform.openai.com/docs/models
- # O-series reasoning models
- LlmModel.O4_MINI: ModelMetadata(
- "openai", 200000, 100000, "O4 Mini", "OpenAI", "OpenAI", 2
- ),
- LlmModel.O3: ModelMetadata("openai", 200000, 100000, "O3", "OpenAI", "OpenAI", 2),
- LlmModel.O3_MINI: ModelMetadata(
- "openai", 200000, 100000, "O3 Mini", "OpenAI", "OpenAI", 1
- ), # o3-mini-2025-01-31
- LlmModel.O3_PRO: ModelMetadata(
- "openai", 200000, 100000, "O3 Pro", "OpenAI", "OpenAI", 3
- ),
- LlmModel.O1: ModelMetadata(
- "openai", 200000, 100000, "O1", "OpenAI", "OpenAI", 3
- ), # o1-2024-12-17
- LlmModel.O1_MINI: ModelMetadata(
- "openai", 128000, 65536, "O1 Mini", "OpenAI", "OpenAI", 2
- ), # o1-mini-2024-09-12
- # GPT-5.6 models (current flagship, July 2026)
- LlmModel.GPT5_6_SOL: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.6 Sol", "OpenAI", "OpenAI", 2
- ),
- LlmModel.GPT5_6_TERRA: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.6 Terra", "OpenAI", "OpenAI", 2
- ),
- LlmModel.GPT5_6_LUNA: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.6 Luna", "OpenAI", "OpenAI", 1
- ),
- # GPT-5.5 models
- LlmModel.GPT5_5: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.5", "OpenAI", "OpenAI", 2
- ),
- LlmModel.GPT5_5_PRO: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.5 Pro", "OpenAI", "OpenAI", 3
- ),
- # GPT-5.4 models (March 2026)
- LlmModel.GPT5_4: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.4", "OpenAI", "OpenAI", 2
- ),
- LlmModel.GPT5_4_MINI: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.4 Mini", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT5_4_NANO: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.4 Nano", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT5_4_PRO: ModelMetadata(
- "openai", 1050000, 128000, "GPT-5.4 Pro", "OpenAI", "OpenAI", 3
- ),
- # GPT-5.3 models
- LlmModel.GPT5_3: ModelMetadata(
- "openai", 128000, 16384, "GPT-5.3", "OpenAI", "OpenAI", 2
- ),
- LlmModel.GPT5_3_CODEX: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.3 Codex", "OpenAI", "OpenAI", 2
- ),
- # GPT-5.2 models
- LlmModel.GPT5_2: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.2", "OpenAI", "OpenAI", 3
- ),
- LlmModel.GPT5_2_PRO: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.2 Pro", "OpenAI", "OpenAI", 3
- ),
- # GPT-5.1 models
- LlmModel.GPT5_1: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.1", "OpenAI", "OpenAI", 2
- ),
- LlmModel.GPT5_1_CODEX: ModelMetadata(
- "openai", 400000, 128000, "GPT-5.1 Codex", "OpenAI", "OpenAI", 2
- ),
- # GPT-5 models
- LlmModel.GPT5: ModelMetadata(
- "openai", 400000, 128000, "GPT-5", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT5_MINI: ModelMetadata(
- "openai", 400000, 128000, "GPT-5 Mini", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT5_NANO: ModelMetadata(
- "openai", 400000, 128000, "GPT-5 Nano", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT5_PRO: ModelMetadata(
- "openai", 400000, 128000, "GPT-5 Pro", "OpenAI", "OpenAI", 3
- ),
- LlmModel.GPT5_CHAT: ModelMetadata(
- "openai", 400000, 16384, "GPT-5 Chat Latest", "OpenAI", "OpenAI", 2
- ),
- # GPT-4.x models
- LlmModel.GPT41: ModelMetadata(
- "openai", 1047576, 32768, "GPT-4.1", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT41_MINI: ModelMetadata(
- "openai", 1047576, 32768, "GPT-4.1 Mini", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT41_NANO: ModelMetadata(
- "openai", 1047576, 32768, "GPT-4.1 Nano", "OpenAI", "OpenAI", 1
- ),
- LlmModel.GPT4O_MINI: ModelMetadata(
- "openai", 128000, 16384, "GPT-4o Mini", "OpenAI", "OpenAI", 1
- ), # gpt-4o-mini-2024-07-18
- LlmModel.GPT4O: ModelMetadata(
- "openai", 128000, 16384, "GPT-4o", "OpenAI", "OpenAI", 2
- ), # gpt-4o-2024-08-06
- # https://docs.anthropic.com/en/docs/about-claude/models
- LlmModel.CLAUDE_4_6_OPUS: ModelMetadata(
- "anthropic", 200000, 128000, "Claude Opus 4.6", "Anthropic", "Anthropic", 3
- ), # claude-opus-4-6
- LlmModel.CLAUDE_4_7_OPUS: ModelMetadata(
- "anthropic", 200000, 128000, "Claude Opus 4.7", "Anthropic", "Anthropic", 3
- ), # claude-opus-4-7
- LlmModel.CLAUDE_4_6_SONNET: ModelMetadata(
- "anthropic", 200000, 64000, "Claude Sonnet 4.6", "Anthropic", "Anthropic", 3
- ), # claude-sonnet-4-6
- LlmModel.CLAUDE_4_5_OPUS: ModelMetadata(
- "anthropic", 200000, 64000, "Claude Opus 4.5", "Anthropic", "Anthropic", 3
- ), # claude-opus-4-5-20251101
- LlmModel.CLAUDE_4_5_SONNET: ModelMetadata(
- "anthropic", 200000, 64000, "Claude Sonnet 4.5", "Anthropic", "Anthropic", 3
- ), # claude-sonnet-4-5-20250929
- LlmModel.CLAUDE_4_5_HAIKU: ModelMetadata(
- "anthropic", 200000, 64000, "Claude Haiku 4.5", "Anthropic", "Anthropic", 2
- ), # claude-haiku-4-5-20251001
- # https://docs.aimlapi.com/api-overview/model-database/text-models
- LlmModel.AIML_API_LLAMA3_3_70B: ModelMetadata(
- "aiml_api", 128000, None, "Llama 3.3 70B Instruct Turbo", "AI/ML", "Meta", 1
- ),
- # https://console.groq.com/docs/models
- LlmModel.LLAMA3_3_70B: ModelMetadata(
- "groq", 128000, 32768, "Llama 3.3 70B Versatile", "Groq", "Meta", 1
- ),
- LlmModel.LLAMA3_1_8B: ModelMetadata(
- "groq", 128000, 8192, "Llama 3.1 8B Instant", "Groq", "Meta", 1
- ),
- # https://ollama.com/library
- LlmModel.OLLAMA_LLAMA3_3: ModelMetadata(
- "ollama", 8192, None, "Llama 3.3", "Ollama", "Meta", 1
- ),
- LlmModel.OLLAMA_LLAMA3_2: ModelMetadata(
- "ollama", 8192, None, "Llama 3.2", "Ollama", "Meta", 1
- ),
- LlmModel.OLLAMA_LLAMA3_8B: ModelMetadata(
- "ollama", 8192, None, "Llama 3", "Ollama", "Meta", 1
- ),
- LlmModel.OLLAMA_LLAMA3_405B: ModelMetadata(
- "ollama", 8192, None, "Llama 3.1 405B", "Ollama", "Meta", 1
- ),
- LlmModel.OLLAMA_DOLPHIN: ModelMetadata(
- "ollama", 32768, None, "Dolphin Mistral Latest", "Ollama", "Mistral AI", 1
- ),
- # https://openrouter.ai/models
- LlmModel.GEMINI_2_5_PRO: ModelMetadata(
- "open_router",
- 1048576,
- 65536,
- "Gemini 2.5 Pro",
- "OpenRouter",
- "Google",
- 2,
- ),
- LlmModel.GEMINI_3_1_PRO_PREVIEW: ModelMetadata(
- "open_router",
- 1048576,
- 65536,
- "Gemini 3.1 Pro Preview",
- "OpenRouter",
- "Google",
- 2,
- ),
- LlmModel.GEMINI_3_FLASH_PREVIEW: ModelMetadata(
- "open_router",
- 1048576,
- 65536,
- "Gemini 3 Flash Preview",
- "OpenRouter",
- "Google",
- 1,
- ),
- LlmModel.GEMINI_2_5_FLASH: ModelMetadata(
- "open_router", 1048576, 65535, "Gemini 2.5 Flash", "OpenRouter", "Google", 1
- ),
- LlmModel.GEMINI_2_0_FLASH: ModelMetadata(
- "open_router", 1048576, 8192, "Gemini 2.0 Flash 001", "OpenRouter", "Google", 1
- ),
- LlmModel.GEMINI_3_1_FLASH_LITE_PREVIEW: ModelMetadata(
- "open_router",
- 1048576,
- 65536,
- "Gemini 3.1 Flash Lite Preview",
- "OpenRouter",
- "Google",
- 1,
- ),
- LlmModel.GEMINI_2_5_FLASH_LITE: ModelMetadata(
- "open_router",
- 1048576,
- 65535,
- "Gemini 2.5 Flash Lite",
- "OpenRouter",
- "Google",
- 1,
- ),
- LlmModel.GEMINI_2_0_FLASH_LITE: ModelMetadata(
- "open_router",
- 1048576,
- 8192,
- "Gemini 2.0 Flash Lite 001",
- "OpenRouter",
- "Google",
- 1,
- ),
- LlmModel.MISTRAL_LARGE_3: ModelMetadata(
- "open_router",
- 262144,
- None,
- "Mistral Large 3 2512",
- "OpenRouter",
- "Mistral AI",
- 2,
- ),
- LlmModel.MISTRAL_MEDIUM_3_1: ModelMetadata(
- "open_router",
- 131072,
- None,
- "Mistral Medium 3.1",
- "OpenRouter",
- "Mistral AI",
- 2,
- ),
- LlmModel.MISTRAL_SMALL_3_2: ModelMetadata(
- "open_router",
- 131072,
- 131072,
- "Mistral Small 3.2 24B",
- "OpenRouter",
- "Mistral AI",
- 1,
- ),
- LlmModel.CODESTRAL: ModelMetadata(
- "open_router",
- 256000,
- None,
- "Codestral 2508",
- "OpenRouter",
- "Mistral AI",
- 1,
- ),
- LlmModel.COHERE_COMMAND_A_03_2025: ModelMetadata(
- "open_router", 256000, 8192, "Command A 03.2025", "OpenRouter", "Cohere", 2
- ),
- LlmModel.COHERE_COMMAND_A_TRANSLATE_08_2025: ModelMetadata(
- "open_router",
- 128000,
- 8192,
- "Command A Translate 08.2025",
- "OpenRouter",
- "Cohere",
- 2,
- ),
- LlmModel.COHERE_COMMAND_A_REASONING_08_2025: ModelMetadata(
- "open_router",
- 256000,
- 32768,
- "Command A Reasoning 08.2025",
- "OpenRouter",
- "Cohere",
- 3,
- ),
- LlmModel.COHERE_COMMAND_A_VISION_07_2025: ModelMetadata(
- "open_router",
- 128000,
- 8192,
- "Command A Vision 07.2025",
- "OpenRouter",
- "Cohere",
- 2,
- ),
- LlmModel.DEEPSEEK_CHAT: ModelMetadata(
- "open_router", 64000, 2048, "DeepSeek Chat", "OpenRouter", "DeepSeek", 1
- ),
- LlmModel.DEEPSEEK_R1_0528: ModelMetadata(
- "open_router", 163840, 163840, "DeepSeek R1 0528", "OpenRouter", "DeepSeek", 1
- ),
- LlmModel.PERPLEXITY_SONAR: ModelMetadata(
- "open_router", 127000, 8000, "Sonar", "OpenRouter", "Perplexity", 1
- ),
- LlmModel.PERPLEXITY_SONAR_PRO: ModelMetadata(
- "open_router", 200000, 8000, "Sonar Pro", "OpenRouter", "Perplexity", 2
- ),
- LlmModel.PERPLEXITY_SONAR_REASONING_PRO: ModelMetadata(
- "open_router",
- 128000,
- 8000,
- "Sonar Reasoning Pro",
- "OpenRouter",
- "Perplexity",
- 2,
- ),
- LlmModel.PERPLEXITY_SONAR_DEEP_RESEARCH: ModelMetadata(
- "open_router",
- 128000,
- 16000,
- "Sonar Deep Research",
- "OpenRouter",
- "Perplexity",
- 3,
- ),
- LlmModel.NOUSRESEARCH_HERMES_3_LLAMA_3_1_405B: ModelMetadata(
- "open_router",
- 131000,
- 4096,
- "Hermes 3 Llama 3.1 405B",
- "OpenRouter",
- "Nous Research",
- 1,
- ),
- LlmModel.NOUSRESEARCH_HERMES_3_LLAMA_3_1_70B: ModelMetadata(
- "open_router",
- 12288,
- 12288,
- "Hermes 3 Llama 3.1 70B",
- "OpenRouter",
- "Nous Research",
- 1,
- ),
- LlmModel.OPENAI_GPT_OSS_120B: ModelMetadata(
- "open_router", 131072, 131072, "GPT-OSS 120B", "OpenRouter", "OpenAI", 1
- ),
- LlmModel.OPENAI_GPT_OSS_20B: ModelMetadata(
- "open_router", 131072, 32768, "GPT-OSS 20B", "OpenRouter", "OpenAI", 1
- ),
- LlmModel.AMAZON_NOVA_LITE_V1: ModelMetadata(
- "open_router", 300000, 5120, "Nova Lite V1", "OpenRouter", "Amazon", 1
- ),
- LlmModel.AMAZON_NOVA_MICRO_V1: ModelMetadata(
- "open_router", 128000, 5120, "Nova Micro V1", "OpenRouter", "Amazon", 1
- ),
- LlmModel.AMAZON_NOVA_PRO_V1: ModelMetadata(
- "open_router", 300000, 5120, "Nova Pro V1", "OpenRouter", "Amazon", 1
- ),
- LlmModel.MICROSOFT_PHI_4: ModelMetadata(
- "open_router", 16384, 16384, "Phi-4", "OpenRouter", "Microsoft", 1
- ),
- LlmModel.GRYPHE_MYTHOMAX_L2_13B: ModelMetadata(
- "open_router", 4096, 4096, "MythoMax L2 13B", "OpenRouter", "Gryphe", 1
- ),
- LlmModel.META_LLAMA_4_SCOUT: ModelMetadata(
- "open_router", 131072, 131072, "Llama 4 Scout", "OpenRouter", "Meta", 1
- ),
- LlmModel.META_LLAMA_4_MAVERICK: ModelMetadata(
- "open_router", 1048576, 1000000, "Llama 4 Maverick", "OpenRouter", "Meta", 1
- ),
- LlmModel.GROK_3: ModelMetadata(
- "open_router",
- 131072,
- 131072,
- "Grok 3",
- "OpenRouter",
- "xAI",
- 2,
- ),
- LlmModel.GROK_4: ModelMetadata(
- "open_router", 256000, 256000, "Grok 4", "OpenRouter", "xAI", 3
- ),
- LlmModel.GROK_4_FAST: ModelMetadata(
- "open_router", 2000000, 30000, "Grok 4 Fast", "OpenRouter", "xAI", 1
- ),
- LlmModel.GROK_4_1_FAST: ModelMetadata(
- "open_router", 2000000, 30000, "Grok 4.1 Fast", "OpenRouter", "xAI", 1
- ),
- LlmModel.GROK_4_20: ModelMetadata(
- "open_router", 2000000, 100000, "Grok 4.20", "OpenRouter", "xAI", 3
- ),
- LlmModel.GROK_4_20_MULTI_AGENT: ModelMetadata(
- "open_router",
- 2000000,
- 100000,
- "Grok 4.20 Multi-Agent",
- "OpenRouter",
- "xAI",
- 3,
- ),
- LlmModel.GROK_CODE_FAST_1: ModelMetadata(
- "open_router", 256000, 10000, "Grok Code Fast 1", "OpenRouter", "xAI", 1
- ),
- LlmModel.KIMI_K2_5: ModelMetadata(
- "open_router", 262144, 262144, "Kimi K2.5", "OpenRouter", "Moonshot AI", 1
- ),
- LlmModel.KIMI_K2_6: ModelMetadata(
- "open_router", 262144, 262144, "Kimi K2.6", "OpenRouter", "Moonshot AI", 2
- ),
- LlmModel.KIMI_K2_THINKING: ModelMetadata(
- "open_router",
- 262144,
- 262144,
- "Kimi K2 Thinking",
- "OpenRouter",
- "Moonshot AI",
- 2,
- ),
- LlmModel.QWEN3_235B_A22B_THINKING: ModelMetadata(
- "open_router",
- 262144,
- 262144,
- "Qwen 3 235B A22B Thinking 2507",
- "OpenRouter",
- "Qwen",
- 1,
- ),
- LlmModel.QWEN3_CODER: ModelMetadata(
- "open_router", 262144, 262144, "Qwen 3 Coder", "OpenRouter", "Qwen", 3
- ),
- # https://openrouter.ai/models?q=z-ai
- LlmModel.ZAI_GLM_4_6: ModelMetadata(
- "open_router", 204800, 204800, "GLM 4.6", "OpenRouter", "Z.ai", 1
- ),
- LlmModel.ZAI_GLM_4_6V: ModelMetadata(
- "open_router", 131072, 131072, "GLM 4.6V", "OpenRouter", "Z.ai", 1
- ),
- LlmModel.ZAI_GLM_4_7: ModelMetadata(
- "open_router", 202752, 65535, "GLM 4.7", "OpenRouter", "Z.ai", 1
- ),
- LlmModel.ZAI_GLM_4_7_FLASH: ModelMetadata(
- "open_router", 202752, 202752, "GLM 4.7 Flash", "OpenRouter", "Z.ai", 1
- ),
- LlmModel.ZAI_GLM_5: ModelMetadata(
- "open_router", 80000, 80000, "GLM 5", "OpenRouter", "Z.ai", 2
- ),
- LlmModel.ZAI_GLM_5_TURBO: ModelMetadata(
- "open_router", 202752, 131072, "GLM 5 Turbo", "OpenRouter", "Z.ai", 3
- ),
- LlmModel.ZAI_GLM_5V_TURBO: ModelMetadata(
- "open_router", 202752, 131072, "GLM 5V Turbo", "OpenRouter", "Z.ai", 3
- ),
- # Llama API models
- LlmModel.LLAMA_API_LLAMA_4_SCOUT: ModelMetadata(
- "llama_api",
- 128000,
- 4028,
- "Llama 4 Scout 17B 16E Instruct FP8",
- "Llama API",
- "Meta",
- 1,
- ),
- LlmModel.LLAMA_API_LLAMA4_MAVERICK: ModelMetadata(
- "llama_api",
- 128000,
- 4028,
- "Llama 4 Maverick 17B 128E Instruct FP8",
- "Llama API",
- "Meta",
- 1,
- ),
- LlmModel.LLAMA_API_LLAMA3_3_8B: ModelMetadata(
- "llama_api", 128000, 4028, "Llama 3.3 8B Instruct", "Llama API", "Meta", 1
- ),
- LlmModel.LLAMA_API_LLAMA3_3_70B: ModelMetadata(
- "llama_api", 128000, 4028, "Llama 3.3 70B Instruct", "Llama API", "Meta", 1
- ),
- # v0 by Vercel models
- LlmModel.V0_1_5_MD: ModelMetadata("v0", 128000, 64000, "v0 1.5 MD", "V0", "V0", 1),
- LlmModel.V0_1_5_LG: ModelMetadata("v0", 512000, 64000, "v0 1.5 LG", "V0", "V0", 1),
- LlmModel.V0_1_0_MD: ModelMetadata("v0", 128000, 64000, "v0 1.0 MD", "V0", "V0", 1),
-}
-
-DEFAULT_LLM_MODEL = LlmModel.GPT5_6_TERRA
-
-# Family-aware mapping for legacy model values that have been retired from the
-# `LlmModel` enum. Used by both the Prisma migration that rewrites stored graph
-# definitions and by the boot-time safety net (`migrate_llm_models` in
-# backend/data/graph.py) so a Claude Opus user lands on a newer Opus instead of
-# the global GPT default. Keep this in sync with
-# migrations/20260512120000_retire_deprecated_llm_models/migration.sql.
-LEGACY_MODEL_MAPPINGS: dict[str, LlmModel] = {
- "claude-3-haiku-20240307": LlmModel.CLAUDE_4_5_HAIKU,
- "claude-opus-4-20250514": LlmModel.CLAUDE_4_7_OPUS,
- "claude-sonnet-4-20250514": LlmModel.CLAUDE_4_6_SONNET,
- "claude-opus-4-1-20250805": LlmModel.CLAUDE_4_7_OPUS,
- "gpt-4-turbo": LlmModel.GPT41,
- "o1": LlmModel.O3,
- "o1-mini": LlmModel.O3_MINI,
- "google/gemini-2.5-pro-preview-03-25": LlmModel.GEMINI_2_5_PRO,
- "google/gemini-2.5-flash-lite-preview-06-17": LlmModel.GEMINI_2_5_FLASH,
- "cohere/command-r-08-2024": LlmModel.COHERE_COMMAND_A_03_2025,
- "cohere/command-r-plus-08-2024": LlmModel.COHERE_COMMAND_A_03_2025,
- "mistralai/mistral-nemo": LlmModel.MISTRAL_SMALL_3_2,
- "microsoft/wizardlm-2-8x22b": LlmModel.MICROSOFT_PHI_4,
- "moonshotai/kimi-k2": LlmModel.KIMI_K2_6,
- "moonshotai/kimi-k2-0905": LlmModel.KIMI_K2_6,
- "z-ai/glm-4-32b": LlmModel.ZAI_GLM_4_6,
- "z-ai/glm-4.5": LlmModel.ZAI_GLM_4_6,
- "z-ai/glm-4.5-air": LlmModel.ZAI_GLM_4_7_FLASH,
- "z-ai/glm-4.5-air:free": LlmModel.ZAI_GLM_4_7_FLASH,
- "z-ai/glm-4.5v": LlmModel.ZAI_GLM_4_6V,
- # AI/ML API stragglers — no direct same-family successor on AI/ML's current
- # catalogue, so they all map to the closest open-weight Meta/Llama option
- # that AI/ML still serves.
- "Qwen/Qwen2.5-72B-Instruct-Turbo": LlmModel.AIML_API_LLAMA3_3_70B,
- "nvidia/llama-3.1-nemotron-70b-instruct": LlmModel.AIML_API_LLAMA3_3_70B,
- "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo": LlmModel.AIML_API_LLAMA3_3_70B,
- "meta-llama/Llama-3.2-3B-Instruct-Turbo": LlmModel.AIML_API_LLAMA3_3_70B,
-}
-
-for model in LlmModel:
- if model not in MODEL_METADATA:
- raise ValueError(f"Missing MODEL_METADATA metadata for model: {model}")
-
-
class LLMResponse(BaseModel):
raw_response: Any
prompt: List[Any]
@@ -938,7 +215,7 @@ def extract_openai_tool_calls(response) -> list[ToolContentBlock] | None:
def get_parallel_tool_calls_param(
- llm_model: LlmModel, parallel_tool_calls: bool | None
+ llm_model: LLMModel, parallel_tool_calls: bool | None
) -> bool | openai.Omit:
"""Get the appropriate parallel_tool_calls parameter for OpenAI-compatible APIs."""
if llm_model.startswith("o") or parallel_tool_calls is None:
@@ -948,7 +225,7 @@ def get_parallel_tool_calls_param(
async def llm_call(
credentials: APIKeyCredentials,
- llm_model: LlmModel,
+ llm_model: LLMModel,
prompt: list[dict],
max_tokens: int | None,
force_json_output: bool = False,
@@ -983,7 +260,7 @@ async def llm_call(
async def _llm_call(
credentials: APIKeyCredentials,
- llm_model: LlmModel,
+ llm_model: LLMModel,
prompt: list[dict],
max_tokens: int | None,
force_json_output: bool = False,
@@ -1055,7 +332,7 @@ async def _llm_call(
# API, etc.) live in ``backend/util/llm/providers.py`` so the dream
# pass, copilot chat, and any future server-side caller route
# through one implementation. This wrapper keeps the block-layer
- # framing on top: ``LlmModel``-aware token budget, retry-on-bad-
+ # framing on top: ``LLMModel``-aware token budget, retry-on-bad-
# shape (caller), ``NodeExecutionStats`` (caller).
from backend.util.llm.providers import ProviderResponse, call_provider
@@ -1123,7 +400,7 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
default=False,
description="Whether the response should be a list of objects in the expected format.",
)
- model: LlmModel = SchemaField(
+ model: LLMModel = SchemaField(
title="LLM Model",
default=DEFAULT_LLM_MODEL,
description="The language model to use for answering the prompt.",
@@ -1225,7 +502,7 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
async def llm_call(
self,
credentials: APIKeyCredentials,
- llm_model: LlmModel,
+ llm_model: LLMModel,
prompt: list[dict],
max_tokens: int | None,
force_json_output: bool = False,
@@ -1599,7 +876,7 @@ class AITextGeneratorBlock(AIBlockBase):
description="The prompt to send to the language model. You can use any of the {keys} from Prompt Values to fill in the prompt with values from the prompt values dictionary by putting them in curly braces.",
placeholder="Enter your prompt here...",
)
- model: LlmModel = SchemaField(
+ model: LLMModel = SchemaField(
title="LLM Model",
default=DEFAULT_LLM_MODEL,
description="The language model to use for answering the prompt.",
@@ -1695,7 +972,7 @@ class AITextSummarizerBlock(AIBlockBase):
description="The text to summarize.",
placeholder="Enter the text to summarize here...",
)
- model: LlmModel = SchemaField(
+ model: LLMModel = SchemaField(
title="LLM Model",
default=DEFAULT_LLM_MODEL,
description="The language model to use for summarizing the text.",
@@ -1912,7 +1189,7 @@ class AIConversationBlock(AIBlockBase):
messages: List[Any] = SchemaField(
description="List of messages in the conversation.",
)
- model: LlmModel = SchemaField(
+ model: LLMModel = SchemaField(
title="LLM Model",
default=DEFAULT_LLM_MODEL,
description="The language model to use for the conversation.",
@@ -2026,7 +1303,7 @@ class AIListGeneratorBlock(AIBlockBase):
default=None,
advanced=False,
)
- model: LlmModel = SchemaField(
+ model: LLMModel = SchemaField(
title="LLM Model",
default=DEFAULT_LLM_MODEL,
description="The language model to use for generating the list.",
diff --git a/autogpt_platform/backend/backend/blocks/orchestrator.py b/autogpt_platform/backend/backend/blocks/orchestrator.py
index d64a85fca3..74377d84de 100644
--- a/autogpt_platform/backend/backend/blocks/orchestrator.py
+++ b/autogpt_platform/backend/backend/blocks/orchestrator.py
@@ -433,7 +433,7 @@ class OrchestratorBlock(Block):
description="The prompt to send to the language model.",
placeholder="Enter your prompt here...",
)
- model: llm.LlmModel = SchemaField(
+ model: llm.LLMModel = SchemaField(
title="LLM Model",
default=llm.DEFAULT_LLM_MODEL,
description="The language model to use for answering the prompt.",
diff --git a/autogpt_platform/backend/backend/blocks/stagehand/blocks.py b/autogpt_platform/backend/backend/blocks/stagehand/blocks.py
index afd302c4f8..63b0627e47 100644
--- a/autogpt_platform/backend/backend/blocks/stagehand/blocks.py
+++ b/autogpt_platform/backend/backend/blocks/stagehand/blocks.py
@@ -8,7 +8,7 @@ from backend.blocks.llm import (
MODEL_METADATA,
AICredentials,
AICredentialsField,
- LlmModel,
+ LLMModel,
ModelMetadata,
)
from backend.blocks.stagehand._config import stagehand as stagehand_provider
@@ -26,11 +26,10 @@ from backend.sdk import (
logger = logging.getLogger(__name__)
-class StagehandRecommendedLlmModel(str, Enum):
- """
- This is subset of LLModel from autogpt_platform/backend/backend/blocks/llm.py
-
- It contains only the models recommended by Stagehand
+class StagehandRecommendedLLMModel(str, Enum):
+ """A subset of ``LLMModel`` (canonical home:
+ ``backend/data/llm_registry/llm_models.py``, re-exported from
+ ``blocks/llm.py``) containing only the models recommended by Stagehand.
"""
# OpenAI
@@ -47,7 +46,7 @@ class StagehandRecommendedLlmModel(str, Enum):
Returns the provider name for the model in the required format for Stagehand:
provider/model_name
"""
- model_metadata = MODEL_METADATA[LlmModel(self.value)]
+ model_metadata = MODEL_METADATA[LLMModel(self.value)]
model_name = self.value
if len(model_name.split("/")) == 1 and not self.value.startswith(
@@ -63,19 +62,19 @@ class StagehandRecommendedLlmModel(str, Enum):
@property
def provider(self) -> str:
- return MODEL_METADATA[LlmModel(self.value)].provider
+ return MODEL_METADATA[LLMModel(self.value)].provider
@property
def metadata(self) -> ModelMetadata:
- return MODEL_METADATA[LlmModel(self.value)]
+ return MODEL_METADATA[LLMModel(self.value)]
@property
def context_window(self) -> int:
- return MODEL_METADATA[LlmModel(self.value)].context_window
+ return MODEL_METADATA[LLMModel(self.value)].context_window
@property
def max_output_tokens(self) -> int | None:
- return MODEL_METADATA[LlmModel(self.value)].max_output_tokens
+ return MODEL_METADATA[LLMModel(self.value)].max_output_tokens
class StagehandObserveBlock(Block):
@@ -90,10 +89,10 @@ class StagehandObserveBlock(Block):
description="Browserbase project ID (required if using Browserbase)",
)
# Model selection and credentials (provider-discriminated like llm.py)
- model: StagehandRecommendedLlmModel = SchemaField(
+ model: StagehandRecommendedLLMModel = SchemaField(
title="LLM Model",
description="LLM to use for Stagehand (provider is inferred)",
- default=StagehandRecommendedLlmModel.CLAUDE_4_6_SONNET,
+ default=StagehandRecommendedLLMModel.CLAUDE_4_6_SONNET,
advanced=False,
)
model_credentials: AICredentials = AICredentialsField()
@@ -173,10 +172,10 @@ class StagehandActBlock(Block):
description="Browserbase project ID (required if using Browserbase)",
)
# Model selection and credentials (provider-discriminated like llm.py)
- model: StagehandRecommendedLlmModel = SchemaField(
+ model: StagehandRecommendedLLMModel = SchemaField(
title="LLM Model",
description="LLM to use for Stagehand (provider is inferred)",
- default=StagehandRecommendedLlmModel.CLAUDE_4_6_SONNET,
+ default=StagehandRecommendedLLMModel.CLAUDE_4_6_SONNET,
advanced=False,
)
model_credentials: AICredentials = AICredentialsField()
@@ -269,10 +268,10 @@ class StagehandExtractBlock(Block):
description="Browserbase project ID (required if using Browserbase)",
)
# Model selection and credentials (provider-discriminated like llm.py)
- model: StagehandRecommendedLlmModel = SchemaField(
+ model: StagehandRecommendedLLMModel = SchemaField(
title="LLM Model",
description="LLM to use for Stagehand (provider is inferred)",
- default=StagehandRecommendedLlmModel.CLAUDE_4_6_SONNET,
+ default=StagehandRecommendedLLMModel.CLAUDE_4_6_SONNET,
advanced=False,
)
model_credentials: AICredentials = AICredentialsField()
diff --git a/autogpt_platform/backend/backend/blocks/test/test_llm.py b/autogpt_platform/backend/backend/blocks/test/test_llm.py
index 5c1ca2bff3..10156b90e7 100644
--- a/autogpt_platform/backend/backend/blocks/test/test_llm.py
+++ b/autogpt_platform/backend/backend/blocks/test/test_llm.py
@@ -85,7 +85,7 @@ class TestLLMStatsTracking:
response = await llm.llm_call(
credentials=anthropic_creds,
- llm_model=llm.LlmModel.CLAUDE_4_5_HAIKU,
+ llm_model=llm.LLMModel.CLAUDE_4_5_HAIKU,
prompt=[{"role": "user", "content": "Hello"}],
max_tokens=100,
)
@@ -1143,33 +1143,33 @@ class TestUserErrorStatusCodeHandling:
mock_exception.assert_not_called()
-class TestLlmModelMissing:
- """Test that LlmModel handles provider-prefixed model names."""
+class TestLLMModelMissing:
+ """Test that LLMModel handles provider-prefixed model names."""
def test_provider_prefixed_model_resolves(self):
"""Provider-prefixed model string should resolve to the correct enum member."""
assert (
- llm.LlmModel("anthropic/claude-sonnet-4-6")
- == llm.LlmModel.CLAUDE_4_6_SONNET
+ llm.LLMModel("anthropic/claude-sonnet-4-6")
+ == llm.LLMModel.CLAUDE_4_6_SONNET
)
def test_bare_model_still_works(self):
"""Bare (non-prefixed) model string should still resolve correctly."""
- assert llm.LlmModel("claude-sonnet-4-6") == llm.LlmModel.CLAUDE_4_6_SONNET
+ assert llm.LLMModel("claude-sonnet-4-6") == llm.LLMModel.CLAUDE_4_6_SONNET
def test_invalid_prefixed_model_raises(self):
"""Unknown provider-prefixed model string should raise ValueError."""
with pytest.raises(ValueError):
- llm.LlmModel("invalid/nonexistent-model")
+ llm.LLMModel("invalid/nonexistent-model")
def test_slash_containing_value_direct_lookup(self):
"""Enum values with '/' (e.g., OpenRouter models) should resolve via direct lookup, not _missing_."""
- assert llm.LlmModel("google/gemini-2.5-pro") == llm.LlmModel.GEMINI_2_5_PRO
+ assert llm.LLMModel("google/gemini-2.5-pro") == llm.LLMModel.GEMINI_2_5_PRO
def test_double_prefixed_slash_model(self):
"""Double-prefixed value should still resolve by stripping first prefix."""
assert (
- llm.LlmModel("extra/google/gemini-2.5-pro") == llm.LlmModel.GEMINI_2_5_PRO
+ llm.LLMModel("extra/google/gemini-2.5-pro") == llm.LLMModel.GEMINI_2_5_PRO
)
@@ -1278,7 +1278,7 @@ class TestAnthropicCacheControl:
with patch("anthropic.AsyncAnthropic", return_value=mock_client):
await llm.llm_call(
credentials=credentials,
- llm_model=llm.LlmModel.CLAUDE_4_6_SONNET,
+ llm_model=llm.LLMModel.CLAUDE_4_6_SONNET,
prompt=[
{"role": "system", "content": "You are an assistant."},
{"role": "user", "content": "Hello"},
@@ -1333,7 +1333,7 @@ class TestAnthropicCacheControl:
with patch("anthropic.AsyncAnthropic", return_value=mock_client):
await llm.llm_call(
credentials=credentials,
- llm_model=llm.LlmModel.CLAUDE_4_6_SONNET,
+ llm_model=llm.LLMModel.CLAUDE_4_6_SONNET,
prompt=[
{"role": "system", "content": "System."},
{"role": "user", "content": "Do something"},
@@ -1371,7 +1371,7 @@ class TestAnthropicCacheControl:
with patch("anthropic.AsyncAnthropic", return_value=mock_client):
await llm.llm_call(
credentials=credentials,
- llm_model=llm.LlmModel.CLAUDE_4_6_SONNET,
+ llm_model=llm.LLMModel.CLAUDE_4_6_SONNET,
prompt=[
{"role": "system", "content": "System."},
{"role": "user", "content": "Hello"},
@@ -1412,7 +1412,7 @@ class TestAnthropicCacheControl:
with patch("anthropic.AsyncAnthropic", return_value=mock_client):
await llm.llm_call(
credentials=credentials,
- llm_model=llm.LlmModel.CLAUDE_4_6_SONNET,
+ llm_model=llm.LLMModel.CLAUDE_4_6_SONNET,
prompt=[{"role": "user", "content": "Hi"}],
max_tokens=50,
)
@@ -1447,7 +1447,7 @@ class TestAnthropicCacheControl:
with patch("anthropic.AsyncAnthropic", return_value=mock_client):
await llm.llm_call(
credentials=credentials,
- llm_model=llm.LlmModel.CLAUDE_4_6_SONNET,
+ llm_model=llm.LLMModel.CLAUDE_4_6_SONNET,
prompt=[
{"role": "system", "content": " \n\t "},
{"role": "user", "content": "Hi"},
@@ -1551,37 +1551,37 @@ class TestLLMRequestTimeout:
), f"Expected exactly 1 call (no retry on timeout), got {call_count['n']}"
-class TestLlmModelMissingHandler:
- """``LlmModel._missing_`` resolves provider-prefixed slugs back to enum
+class TestLLMModelMissingHandler:
+ """``LLMModel._missing_`` resolves provider-prefixed slugs back to enum
members. Used by the dry-run simulator to send an OpenRouter slug
(``anthropic/claude-haiku-4-5``) that OpenRouter's OpenAI-compat endpoint
accepts, while still validating through ``OrchestratorBlock.Input.model``
- which is typed as ``LlmModel``."""
+ which is typed as ``LLMModel``."""
def test_openrouter_alias_maps_to_canonical(self) -> None:
# The OpenRouter slug for Haiku 4.5 drops the snapshot suffix that
# Anthropic's direct API uses — pure prefix strip wouldn't find it,
# so the alias map is required.
assert (
- llm.LlmModel("anthropic/claude-haiku-4-5") is llm.LlmModel.CLAUDE_4_5_HAIKU
+ llm.LLMModel("anthropic/claude-haiku-4-5") is llm.LLMModel.CLAUDE_4_5_HAIKU
)
- assert llm.LlmModel("anthropic/claude-opus-4-5") is llm.LlmModel.CLAUDE_4_5_OPUS
+ assert llm.LLMModel("anthropic/claude-opus-4-5") is llm.LLMModel.CLAUDE_4_5_OPUS
assert (
- llm.LlmModel("anthropic/claude-sonnet-4-5")
- is llm.LlmModel.CLAUDE_4_5_SONNET
+ llm.LLMModel("anthropic/claude-sonnet-4-5")
+ is llm.LLMModel.CLAUDE_4_5_SONNET
)
def test_prefix_strip_still_works(self) -> None:
# 4.6/4.7 enum values already match OpenRouter's slug after prefix
# strip — no alias-map entry needed.
- assert llm.LlmModel("anthropic/claude-opus-4-7") is llm.LlmModel.CLAUDE_4_7_OPUS
- assert llm.LlmModel("anthropic/claude-opus-4-6") is llm.LlmModel.CLAUDE_4_6_OPUS
+ assert llm.LLMModel("anthropic/claude-opus-4-7") is llm.LLMModel.CLAUDE_4_7_OPUS
+ assert llm.LLMModel("anthropic/claude-opus-4-6") is llm.LLMModel.CLAUDE_4_6_OPUS
def test_unknown_slug_raises(self) -> None:
# Bare unknown strings still fail loudly — the alias map is an
# additive shortcut, not a silent catch-all.
with pytest.raises(ValueError):
- llm.LlmModel("anthropic/claude-haiku-999")
+ llm.LLMModel("anthropic/claude-haiku-999")
def test_all_claude_snapshots_reachable_via_openrouter_slug(self) -> None:
"""Every Claude enum value whose canonical form carries a trailing
@@ -1596,12 +1596,12 @@ class TestLlmModelMissingHandler:
# constructed correctly: ``claude-haiku-4-5-20251001`` →
# ``anthropic/claude-haiku-4-5`` (NOT ``anthropic/haiku-4-5``).
snapshot_re = re.compile(r"^(claude-.+)-\d{8}$")
- for member in llm.LlmModel:
+ for member in llm.LLMModel:
match = snapshot_re.match(member.value)
if match is None:
continue
slug = f"anthropic/{match.group(1)}"
- assert llm.LlmModel(slug) is member, (
+ assert llm.LLMModel(slug) is member, (
f"OpenRouter alias missing for {member.value} — expected slug "
f"{slug!r} to resolve back to {member.name}"
)
diff --git a/autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py b/autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
index d1ed9ad0a1..42d20cab28 100644
--- a/autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
+++ b/autogpt_platform/backend/backend/blocks/test/test_orchestrator_execution_mode.py
@@ -10,7 +10,7 @@ from unittest.mock import AsyncMock, MagicMock, patch
import pytest
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import LLMModel
from backend.blocks.orchestrator import (
ExecutionMode,
OrchestratorBlock,
@@ -57,7 +57,7 @@ class TestExecutionModeEnum:
def _make_model_stub(provider: str, value: str):
- """Create a lightweight stub that behaves like LlmModel for validation."""
+ """Create a lightweight stub that behaves like LLMModel for validation."""
metadata = MagicMock()
metadata.provider = provider
stub = MagicMock()
@@ -103,14 +103,14 @@ class TestExtendedThinkingProviderValidation:
assert not model_name.startswith("claude")
def test_real_gpt4o_model_rejected(self):
- """Verify a real LlmModel enum member (GPT4O) fails the provider check."""
- model = LlmModel.GPT4O
+ """Verify a real LLMModel enum member (GPT4O) fails the provider check."""
+ model = LLMModel.GPT4O
provider = model.metadata.provider
assert provider not in ("anthropic", "open_router")
def test_real_claude_model_passes(self):
- """Verify a real LlmModel enum member (CLAUDE_4_6_SONNET) passes."""
- model = LlmModel.CLAUDE_4_6_SONNET
+ """Verify a real LLMModel enum member (CLAUDE_4_6_SONNET) passes."""
+ model = LLMModel.CLAUDE_4_6_SONNET
provider = model.metadata.provider
model_name = model.value
assert provider in ("anthropic", "open_router")
@@ -150,7 +150,7 @@ class TestExtendedThinkingValidationRaisesInBlock:
async def test_non_anthropic_provider_raises_valueerror(self):
"""EXTENDED_THINKING + openai provider raises ValueError."""
block = OrchestratorBlock()
- input_data = _make_input_data(model=LlmModel.GPT4O)
+ input_data = _make_input_data(model=LLMModel.GPT4O)
with (
patch.object(
diff --git a/autogpt_platform/backend/backend/copilot/anthropic_rate_card.py b/autogpt_platform/backend/backend/copilot/anthropic_rate_card.py
index a281ade626..364390c385 100644
--- a/autogpt_platform/backend/backend/copilot/anthropic_rate_card.py
+++ b/autogpt_platform/backend/backend/copilot/anthropic_rate_card.py
@@ -19,6 +19,11 @@ refresh: ``poetry run python scripts/refresh_anthropic_rates.py``.
Anthropic prompt-caching docs (rates we apply when the source omits
the 1h cache-write field on a model):
https://docs.claude.com/en/docs/build-with-claude/prompt-caching
+
+Scope note: this rate card exists only for direct-Anthropic cost
+recovery on the OpenAI-compat transport. Per-model catalog costs (the
+ones the platform manages) live in ``backend/data/llm_registry/
+catalog.py`` — see docs/platform/contributing/managing-llm-models.md.
"""
from __future__ import annotations
diff --git a/autogpt_platform/backend/backend/copilot/baseline/reasoning.py b/autogpt_platform/backend/backend/copilot/baseline/reasoning.py
index 0cc5939e84..7424a7a1a6 100644
--- a/autogpt_platform/backend/backend/copilot/baseline/reasoning.py
+++ b/autogpt_platform/backend/backend/copilot/baseline/reasoning.py
@@ -153,8 +153,8 @@ def _is_reasoning_route(model: str) -> bool:
OpenRouter exposes reasoning tokens via a unified ``reasoning`` request
param that works on any provider that supports extended thinking —
- currently Anthropic (Claude Opus / Sonnet) and Moonshot (Kimi K2.6 +
- kimi-k2-thinking) advertise it in their ``supported_parameters``.
+ currently Anthropic (Claude Opus / Sonnet) and Moonshot (Kimi K2.6,
+ kimi-k2-thinking, K3) advertise it in their ``supported_parameters``.
Other providers silently drop the field, but we skip it anyway to keep
the payload tight and avoid confusing cache diagnostics.
diff --git a/autogpt_platform/backend/backend/copilot/baseline/service.py b/autogpt_platform/backend/backend/copilot/baseline/service.py
index bf60a5bb24..e0aef56b2f 100644
--- a/autogpt_platform/backend/backend/copilot/baseline/service.py
+++ b/autogpt_platform/backend/backend/copilot/baseline/service.py
@@ -42,7 +42,7 @@ from backend.copilot.builder_context import (
build_builder_context_turn_prefix,
build_builder_system_prompt_suffix,
)
-from backend.copilot.config import CopilotLlmModel, CopilotMode
+from backend.copilot.config import CopilotLLMModel, CopilotMode
from backend.copilot.context import get_workspace_manager, set_execution_context
from backend.copilot.expert_context import build_expert_identity_suffix
from backend.copilot.graphiti.config import is_enabled_for_user
@@ -53,12 +53,13 @@ from backend.copilot.local_context_probe import (
from backend.copilot.model import (
ChatMessage,
ChatSession,
+ RoutingSource,
get_chat_session,
maybe_append_user_message,
upsert_chat_session,
)
from backend.copilot.model_normalize import normalize_model_for_transport
-from backend.copilot.model_router import resolve_model
+from backend.copilot.model_router import ResolvedModel, resolve_model_route
from backend.copilot.moonshot import is_moonshot_model
from backend.copilot.pending_message_helpers import (
combine_pending_with_current,
@@ -404,16 +405,17 @@ def _filter_tools_by_permissions(
async def _resolve_baseline_model(
- tier: CopilotLlmModel | None, user_id: str | None
-) -> str:
+ tier: CopilotLLMModel | None, user_id: str | None
+) -> ResolvedModel:
"""Pick the model for the baseline path based on the per-request tier.
- Delegates to :func:`copilot.model_router.resolve_model` so the
- ``(fast, tier)`` cell is LD-overridable per user. ``None`` tier
- maps to ``"standard"``.
+ Delegates to :func:`copilot.model_router.resolve_model_route` so the
+ ``(fast, tier)`` cell resolves LD → registry cell → env. ``None`` tier
+ maps to ``"standard"``. The routing source rides along so persisted
+ assistant messages can be stamped for product-intelligence segmentation.
"""
tier_name = "advanced" if tier == "advanced" else "standard"
- return await resolve_model("fast", tier_name, user_id, config=config)
+ return await resolve_model_route("fast", tier_name, user_id, config=config)
@dataclass
@@ -425,6 +427,11 @@ class _BaselineStreamState:
"""
model: str = ""
+ # Which routing layer picked ``model`` — stamped onto persisted assistant
+ # messages for product-intelligence segmentation. The baseline path only
+ # ever produces "ld" | "catalog" | "env" ("fallback" is SDK-only, marking
+ # a CLI 529-overload swap); typed as the shared RoutingSource superset.
+ routing_source: RoutingSource = "env"
# Live delivery channel drained concurrently by ``stream_chat_completion_baseline``
# so reasoning / text / tool events reach the SSE wire **during** the upstream
# LLM stream, not after ``_baseline_llm_caller`` returns. Before this was a
@@ -1236,6 +1243,8 @@ def _baseline_conversation_updater(
if state is not None and tool_results:
assistant_msg = ChatMessage(
role="assistant",
+ model=state.model,
+ routing_source=state.routing_source,
content=response.response_text or "",
tool_calls=[
{
@@ -1560,7 +1569,7 @@ async def stream_chat_completion_baseline(
permissions: "CopilotPermissions | None" = None,
context: dict[str, str] | None = None,
mode: CopilotMode | None = None,
- model: CopilotLlmModel | None = None,
+ model: CopilotLLMModel | None = None,
request_arrival_at: float = 0.0,
organization_id: str | None = None,
team_id: str | None = None,
@@ -1643,7 +1652,9 @@ async def stream_chat_completion_baseline(
# would be rejected by the direct client. Pass the baseline-side
# ``config`` so monkeypatch fixtures targeting this module's
# ``config`` symbol drive the decision.
- resolved_model = await _resolve_baseline_model(model, user_id)
+ resolved_route = await _resolve_baseline_model(model, user_id)
+ resolved_model = resolved_route.model
+ routing_source = resolved_route.source
try:
active_model = normalize_model_for_transport(resolved_model, config)
except ValueError as exc:
@@ -1664,6 +1675,7 @@ async def stream_chat_completion_baseline(
active_model = normalize_model_for_transport(tier_default, config)
except ValueError:
raise exc
+ routing_source = "env"
logger.warning(
"[Baseline] [%s] LD model %r rejected for tier=%s (%s); falling "
"back to tier default %s",
@@ -2068,7 +2080,7 @@ async def stream_chat_completion_baseline(
logger.warning("[Baseline] Langfuse trace context setup failed")
_stream_error = False # Track whether an error occurred during streaming
- state = _BaselineStreamState(model=active_model)
+ state = _BaselineStreamState(model=active_model, routing_source=routing_source)
# Bind extracted module-level callbacks to this request's state/session
# using functools.partial so they satisfy the Protocol signatures.
@@ -2220,7 +2232,12 @@ async def stream_chat_completion_baseline(
current_session = _session_holder[0]
if text_only_text.strip():
current_session.messages.append(
- ChatMessage(role="assistant", content=text_only_text)
+ ChatMessage(
+ role="assistant",
+ content=text_only_text,
+ model=state.model,
+ routing_source=state.routing_source,
+ )
)
for _buffered in state.session_messages:
current_session.messages.append(_buffered)
@@ -2490,7 +2507,14 @@ async def stream_chat_completion_baseline(
if final_text.startswith(recorded):
final_text = final_text[len(recorded) :]
if final_text.strip():
- session.messages.append(ChatMessage(role="assistant", content=final_text))
+ session.messages.append(
+ ChatMessage(
+ role="assistant",
+ content=final_text,
+ model=state.model,
+ routing_source=state.routing_source,
+ )
+ )
try:
await upsert_chat_session(session)
except Exception as persist_err:
diff --git a/autogpt_platform/backend/backend/copilot/baseline/service_unit_test.py b/autogpt_platform/backend/backend/copilot/baseline/service_unit_test.py
index a3a873fa20..8f9d34e4ab 100644
--- a/autogpt_platform/backend/backend/copilot/baseline/service_unit_test.py
+++ b/autogpt_platform/backend/backend/copilot/baseline/service_unit_test.py
@@ -149,6 +149,43 @@ class TestBaselineConversationUpdater:
assert messages[1]["tool_call_id"] == "tc_1"
assert messages[1]["content"] == "Found result"
+ def test_stamps_model_and_routing_source_on_persisted_assistant(self):
+ """The persisted assistant ChatMessage (state.session_messages) must
+ carry the turn's model and routing_source so product-intelligence can
+ segment quality by model/layer — stamped from the stream state. Tool
+ rows stay unstamped (model is None)."""
+ messages: list = []
+ builder = self._make_transcript_builder()
+ state = _BaselineStreamState(
+ model="anthropic/claude-sonnet-4-6", routing_source="ld"
+ )
+ response = LLMLoopResponse(
+ response_text="searching",
+ tool_calls=[LLMToolCall(id="tc_1", name="search", arguments="{}")],
+ raw_response=None,
+ prompt_tokens=0,
+ completion_tokens=0,
+ )
+ tool_results = [
+ ToolCallResult(tool_call_id="tc_1", tool_name="search", content="ok"),
+ ]
+
+ _baseline_conversation_updater(
+ messages,
+ response,
+ tool_results=tool_results,
+ transcript_builder=builder,
+ model="anthropic/claude-sonnet-4-6",
+ state=state,
+ )
+
+ persisted = [m for m in state.session_messages if m.role == "assistant"]
+ assert len(persisted) == 1
+ assert persisted[0].model == "anthropic/claude-sonnet-4-6"
+ assert persisted[0].routing_source == "ld"
+ tool_rows = [m for m in state.session_messages if m.role == "tool"]
+ assert tool_rows and tool_rows[0].model is None
+
# Transcript: user + assistant(tool_use) + user(tool_result)
assert builder.entry_count == 3
diff --git a/autogpt_platform/backend/backend/copilot/baseline/transcript_integration_test.py b/autogpt_platform/backend/backend/copilot/baseline/transcript_integration_test.py
index 69783dc373..ce174f7a51 100644
--- a/autogpt_platform/backend/backend/copilot/baseline/transcript_integration_test.py
+++ b/autogpt_platform/backend/backend/copilot/baseline/transcript_integration_test.py
@@ -73,28 +73,31 @@ class TestResolveBaselineModel:
Baseline reads the ``fast_*_model`` cells of the (path, tier) matrix
and never falls through to the SDK-side ``thinking_*_model`` cells.
Without a user_id (so no LD context) the resolver returns the
- ``ChatConfig`` static default; per-user overrides are exercised in
- ``copilot/model_router_test.py``.
+ ``ChatConfig`` static default stamped ``source="env"``; per-user
+ overrides are exercised in ``copilot/model_router_test.py``.
"""
@pytest.mark.asyncio
async def test_advanced_tier_selects_fast_advanced_model(self):
- assert (
- await _resolve_baseline_model("advanced", None)
- == config.fast_advanced_model
+ assert await _resolve_baseline_model("advanced", None) == (
+ config.fast_advanced_model,
+ "env",
)
@pytest.mark.asyncio
async def test_standard_tier_selects_fast_standard_model(self):
- assert (
- await _resolve_baseline_model("standard", None)
- == config.fast_standard_model
+ assert await _resolve_baseline_model("standard", None) == (
+ config.fast_standard_model,
+ "env",
)
@pytest.mark.asyncio
async def test_none_tier_selects_fast_standard_model(self):
"""Baseline users without a tier get the fast-standard default."""
- assert await _resolve_baseline_model(None, None) == config.fast_standard_model
+ assert await _resolve_baseline_model(None, None) == (
+ config.fast_standard_model,
+ "env",
+ )
def test_fast_standard_default_is_sonnet(self):
"""Shipped default: Sonnet on the baseline standard cell — the
diff --git a/autogpt_platform/backend/backend/copilot/config.py b/autogpt_platform/backend/backend/copilot/config.py
index fcc812f3e7..d7e56392c5 100644
--- a/autogpt_platform/backend/backend/copilot/config.py
+++ b/autogpt_platform/backend/backend/copilot/config.py
@@ -184,7 +184,7 @@ CopilotMode = Literal["fast", "extended_thinking"]
# default to Opus today).
# None means no preference — falls through to LD per-user targeting, then config.
# Using tier names instead of model names keeps the contract model-agnostic.
-CopilotLlmModel = Literal["standard", "advanced"]
+CopilotLLMModel = Literal["standard", "advanced"]
class ChatConfig(BaseSettings):
@@ -193,7 +193,7 @@ class ChatConfig(BaseSettings):
# Chat model tiers — a 2×2 of (path, tier). ``path`` = ``CopilotMode``
# (``"fast"`` → baseline OpenAI-compat / any OpenRouter model;
# ``"extended_thinking"`` → Claude Agent SDK, Anthropic-only CLI).
- # ``tier`` = ``CopilotLlmModel`` (``"standard"`` / ``"advanced"``).
+ # ``tier`` = ``CopilotLLMModel`` (``"standard"`` / ``"advanced"``).
# Each cell has its own config so the two paths can evolve
# independently (cheap provider on baseline, Anthropic on SDK) at each
# tier without conflating one path's needs with the other's constraint.
diff --git a/autogpt_platform/backend/backend/copilot/db.py b/autogpt_platform/backend/backend/copilot/db.py
index 6e8c3c7e99..2f725403ef 100644
--- a/autogpt_platform/backend/backend/copilot/db.py
+++ b/autogpt_platform/backend/backend/copilot/db.py
@@ -527,6 +527,11 @@ async def add_chat_messages_batch(
if msg.get("duration_ms") is not None:
data["durationMs"] = msg["duration_ms"]
+ if msg.get("model") is not None:
+ data["model"] = msg["model"]
+ if msg.get("routing_source") is not None:
+ data["routingSource"] = msg["routing_source"]
+
messages_data.append(data)
# Run create_many and session update in parallel within transaction
@@ -797,6 +802,32 @@ async def update_message_content_by_sequence(
return False
+async def update_chat_message_stamps(
+ session_id: str,
+ sequence: int,
+ model: str | None,
+ routing_source: str | None,
+) -> bool:
+ """Back-fill model/routingSource on an already-persisted message row.
+
+ Mid-turn flushes persist assistant rows (assigning sequences) BEFORE
+ the end-of-turn stamping runs; this repairs those rows so the
+ analytics columns survive in the DB. Same mechanism and authorization
+ reasoning as ``update_chat_message_tool_calls``.
+ """
+ result = await PrismaChatMessage.prisma().update(
+ where={"sessionId_sequence": {"sessionId": session_id, "sequence": sequence}},
+ data={"model": model, "routingSource": routing_source},
+ )
+ if not result:
+ logger.warning(
+ f"No message found to update stamps for session "
+ f"{session_id}, sequence {sequence}"
+ )
+ return False
+ return True
+
+
async def update_chat_message_tool_calls(
session_id: str,
sequence: int,
diff --git a/autogpt_platform/backend/backend/copilot/db_test.py b/autogpt_platform/backend/backend/copilot/db_test.py
index 20ee057e56..35843939ed 100644
--- a/autogpt_platform/backend/backend/copilot/db_test.py
+++ b/autogpt_platform/backend/backend/copilot/db_test.py
@@ -900,3 +900,127 @@ async def test_update_chat_message_tool_calls_not_found():
assert result is False
mock_logger.warning.assert_called_once()
+
+
+# ---------- Stamping columns: the Prisma persistence boundary ----------
+
+
+async def test_batch_persist_maps_stamp_columns_to_prisma_names():
+ """The snake_case message fields must land on the camelCase Prisma
+ columns — a mapping typo here silently stops analytics populating."""
+ from backend.copilot.db import add_chat_messages_batch
+
+ captured: dict = {}
+
+ with (
+ patch.object(PrismaChatMessage, "prisma") as mock_msg,
+ patch.object(PrismaChatSession, "prisma") as mock_sess,
+ patch("backend.copilot.db.db.transaction") as mock_tx,
+ ):
+ mock_tx.return_value.__aenter__ = AsyncMock(return_value=None)
+ mock_tx.return_value.__aexit__ = AsyncMock(return_value=None)
+
+ async def _capture(data):
+ captured["rows"] = data
+
+ mock_msg.return_value.create_many = AsyncMock(side_effect=_capture)
+ mock_msg.return_value.find_first = AsyncMock(return_value=None)
+ mock_sess.return_value.update = AsyncMock()
+
+ await add_chat_messages_batch(
+ session_id=SESSION_ID,
+ start_sequence=0,
+ messages=[
+ {
+ "role": "assistant",
+ "content": "hi",
+ "model": "claude-sonnet-4-6",
+ "routing_source": "catalog",
+ }
+ ],
+ )
+
+ (row,) = captured["rows"]
+ assert row["model"] == "claude-sonnet-4-6"
+ assert row["routingSource"] == "catalog"
+ assert "routing_source" not in row
+
+
+def test_from_db_restores_stamp_columns():
+ """Prisma camelCase columns come back as the snake_case message fields."""
+ from backend.copilot.model import ChatMessage
+
+ prisma_msg = PrismaChatMessage.model_construct(
+ id="m-1",
+ sessionId=SESSION_ID,
+ sequence=1,
+ role="assistant",
+ content="hi",
+ model="claude-sonnet-4-6",
+ routingSource="catalog",
+ createdAt=datetime.now(UTC),
+ )
+ restored = ChatMessage.from_db(prisma_msg)
+ assert restored.model == "claude-sonnet-4-6"
+ assert restored.routing_source == "catalog"
+
+
+def test_from_db_null_stamps_stay_null():
+ """Pre-feature rows (NULL columns) must not grow phantom values."""
+ from backend.copilot.model import ChatMessage
+
+ prisma_msg = PrismaChatMessage.model_construct(
+ id="m-2",
+ sessionId=SESSION_ID,
+ sequence=2,
+ role="assistant",
+ content="old",
+ model=None,
+ routingSource=None,
+ createdAt=datetime.now(UTC),
+ )
+ restored = ChatMessage.from_db(prisma_msg)
+ assert restored.model is None
+ assert restored.routing_source is None
+
+
+async def test_update_chat_message_stamps_maps_prisma_columns():
+ """The stamp back-fill writes the camelCase Prisma columns."""
+ from backend.copilot.db import update_chat_message_stamps
+
+ with patch.object(PrismaChatMessage, "prisma") as mock_msg:
+ update = AsyncMock(return_value=object())
+ mock_msg.return_value.update = update
+
+ ok = await update_chat_message_stamps(
+ session_id=SESSION_ID,
+ sequence=7,
+ model="claude-sonnet-4-6",
+ routing_source="fallback",
+ )
+
+ assert ok is True
+ kwargs = update.call_args.kwargs
+ assert kwargs["data"] == {
+ "model": "claude-sonnet-4-6",
+ "routingSource": "fallback",
+ }
+ assert kwargs["where"]["sessionId_sequence"]["sequence"] == 7
+
+
+async def test_update_chat_message_stamps_not_found_returns_false():
+ """No matching row → False (the back-fill loop logs and keeps going)."""
+ from backend.copilot.db import update_chat_message_stamps
+
+ with patch.object(PrismaChatMessage, "prisma") as mock_msg:
+ mock_msg.return_value.update = AsyncMock(return_value=None)
+ ok = await update_chat_message_stamps(
+ session_id=SESSION_ID,
+ sequence=99,
+ model="claude-sonnet-4-6",
+ routing_source="env",
+ )
+ assert ok is False
diff --git a/autogpt_platform/backend/backend/copilot/executor/manager.py b/autogpt_platform/backend/backend/copilot/executor/manager.py
index 5b790b7d93..d3c6e87dfb 100644
--- a/autogpt_platform/backend/backend/copilot/executor/manager.py
+++ b/autogpt_platform/backend/backend/copilot/executor/manager.py
@@ -17,6 +17,7 @@ from pika.exceptions import AMQPChannelError, AMQPConnectionError
from pika.spec import Basic, BasicProperties
from prometheus_client import Gauge, start_http_server
+import backend.data.llm_registry
from backend.copilot import engine_switch
from backend.copilot.executor.utils import schedule_turn
from backend.data import redis_client as redis
@@ -42,6 +43,20 @@ from .utils import (
logger = TruncatedLogger(logging.getLogger(__name__), prefix="[CoPilotExecutor]")
settings = Settings()
+
+def _load_catalog() -> None:
+ """Load the LLM catalog into THIS process's registry cache.
+
+ Copilot turns run in this executor, not the rest API process — without
+ this, routing cells and serve-time gating silently no-op (an empty
+ registry gates nothing by design). Fail-hard: the catalog is
+ load-bearing (blocks and billing already built from it at import), so
+ a failure here is a bug that should stop the process, not silently
+ disable gating.
+ """
+ backend.data.llm_registry.load_catalog()
+
+
# Prometheus metrics
active_tasks_gauge = Gauge(
"copilot_executor_active_tasks",
@@ -96,6 +111,8 @@ class CoPilotExecutor(AppProcess):
logger.info(f"Pod assigned executor_id: {self.executor_id}")
logger.info(f"Spawn max-{self.pool_size} workers...")
+ _load_catalog()
+
# Materialise the active-tasks lock NOW, before any worker threads
# exist, so subsequent multi-threaded reads of the lazy property
# never race on its check-then-init pattern. ``__init__`` can't
diff --git a/autogpt_platform/backend/backend/copilot/executor/utils.py b/autogpt_platform/backend/backend/copilot/executor/utils.py
index 0c0159a414..e6e48f9b78 100644
--- a/autogpt_platform/backend/backend/copilot/executor/utils.py
+++ b/autogpt_platform/backend/backend/copilot/executor/utils.py
@@ -16,7 +16,7 @@ from backend.copilot.active_turns import (
get_inflight_turn_limit,
inflight_turn_limit_message,
)
-from backend.copilot.config import CopilotLlmModel, CopilotMode
+from backend.copilot.config import CopilotLLMModel, CopilotMode
from backend.copilot.permissions import CopilotPermissions
from backend.data.rabbitmq import Exchange, ExchangeType, Queue, RabbitMQConfig
from backend.util.logging import TruncatedLogger, is_structured_logging_enabled
@@ -208,7 +208,7 @@ class CoPilotExecutionEntry(BaseModel):
mode: CopilotMode | None = None
"""Autopilot mode override: 'fast' or 'extended_thinking'. None = server default."""
- model: CopilotLlmModel | None = None
+ model: CopilotLLMModel | None = None
"""Per-request model tier: 'standard' or 'advanced'. None = server default."""
permissions: CopilotPermissions | None = None
@@ -247,7 +247,7 @@ async def enqueue_copilot_turn(
organization_id: str | None = None,
team_id: str | None = None,
mode: CopilotMode | None = None,
- model: CopilotLlmModel | None = None,
+ model: CopilotLLMModel | None = None,
permissions: CopilotPermissions | None = None,
request_arrival_at: float = 0.0,
) -> None:
@@ -306,7 +306,7 @@ async def schedule_turn(
organization_id: str | None = None,
team_id: str | None = None,
mode: CopilotMode | None = None,
- model: CopilotLlmModel | None = None,
+ model: CopilotLLMModel | None = None,
permissions: CopilotPermissions | None = None,
request_arrival_at: float = 0.0,
) -> None:
@@ -391,7 +391,7 @@ async def dispatch_turn(
organization_id: str | None = None,
team_id: str | None = None,
mode: CopilotMode | None = None,
- model: CopilotLlmModel | None = None,
+ model: CopilotLLMModel | None = None,
permissions: CopilotPermissions | None = None,
request_arrival_at: float = 0.0,
) -> None:
@@ -473,7 +473,7 @@ async def schedule_chat_turn(
organization_id: str | None = None,
team_id: str | None = None,
mode: CopilotMode | None = None,
- model: CopilotLlmModel | None = None,
+ model: CopilotLLMModel | None = None,
permissions: CopilotPermissions | None = None,
request_arrival_at: float = 0.0,
) -> str | None:
diff --git a/autogpt_platform/backend/backend/copilot/model.py b/autogpt_platform/backend/backend/copilot/model.py
index a9455170b6..e7b900e7f2 100644
--- a/autogpt_platform/backend/backend/copilot/model.py
+++ b/autogpt_platform/backend/backend/copilot/model.py
@@ -3,7 +3,7 @@ import logging
import uuid
from contextlib import asynccontextmanager
from datetime import UTC, datetime
-from typing import Any, AsyncIterator, Self, cast
+from typing import Any, AsyncIterator, Literal, Self, cast
from openai.types.chat import (
ChatCompletionAssistantMessageParam,
@@ -36,6 +36,14 @@ logger = logging.getLogger(__name__)
config = ChatConfig()
+# The routing layer that picked a turn's model. Defined here — on the
+# persistence model, the lowest layer — so ChatMessage can type its
+# stamped column and model_router / the SDK service import the ONE
+# definition instead of re-declaring the Literal. ("fallback" is stamped,
+# never routed: it marks a CLI 529-overload fallback to a different model.)
+RoutingSource = Literal["ld", "catalog", "env", "fallback"]
+
+
# Redis cache key prefix for chat sessions
CHAT_SESSION_CACHE_PREFIX = "chat:session:"
@@ -105,6 +113,26 @@ class ChatMessage(BaseModel):
duration_ms: int | None = None
created_at: datetime | None = None
+ # Which LLM served this assistant turn ("model" — visible to clients,
+ # the model-badge UX) and which routing layer picked it
+ # ("ld" | "catalog" | "env" | "fallback" — the last when the CLI's
+ # overload fallback served a different model than the routed one).
+ # Product-intelligence mirrors these to segment quality judgments by
+ # model. None on user/tool rows.
+ #
+ # routing_source is excluded from serialized payloads: "ld" vs "env"
+ # would let a client infer LaunchDarkly cohort membership. It still
+ # persists — the DB save path maps fields explicitly (not via
+ # model_dump) and the back-fill reads attributes.
+ model: str | None = None
+ routing_source: RoutingSource | None = Field(default=None, exclude=True)
+
+ stamps_pending_save: bool = Field(default=False, exclude=True)
+ """True when model/routing_source were stamped after this row was already
+ persisted (mid-turn flush assigned its sequence before end-of-turn
+ stamping ran). The save path back-fills flagged rows — same mechanism
+ as ``tool_calls_pending_save`` below."""
+
tool_calls_pending_save: bool = Field(default=False, exclude=True)
"""True when ``tool_calls`` mutated after this row was already persisted.
@@ -158,6 +186,10 @@ class ChatMessage(BaseModel):
created_at=prisma_message.createdAt,
session_id=prisma_message.sessionId,
metadata=_parse_json_field(prisma_message.metadata),
+ model=prisma_message.model,
+ # DB column is a plain string; the values are always ones we wrote
+ # (this PR owns the column) and Pydantic re-validates on construct.
+ routing_source=cast("RoutingSource | None", prisma_message.routingSource),
)
@@ -927,6 +959,8 @@ async def _save_session_to_db(
"refusal": msg.refusal,
"tool_calls": msg.tool_calls,
"function_call": msg.function_call,
+ "model": msg.model,
+ "routing_source": msg.routing_source,
}
)
logger.info(
@@ -948,6 +982,7 @@ async def _save_session_to_db(
for i, msg in enumerate(new_messages):
msg.sequence = actual_start + i
msg.tool_calls_pending_save = False
+ msg.stamps_pending_save = False
# Back-fill tool_calls onto rows that were flushed before their
# tool_use blocks arrived (see ChatMessage.tool_calls_pending_save).
@@ -962,6 +997,30 @@ async def _save_session_to_db(
if m.tool_calls_pending_save and m.sequence is not None
]
+ # Same repair for stamps: rows flushed mid-turn got their sequence
+ # before end-of-turn stamping ran (see ChatMessage.stamps_pending_save).
+ pending_stamps = [
+ m for m in session.messages if m.stamps_pending_save and m.sequence is not None
+ ]
+
+ async def _backfill_stamps(msg: ChatMessage) -> None:
+ assert msg.sequence is not None
+ try:
+ updated = await db.update_chat_message_stamps(
+ session_id=session.session_id,
+ sequence=msg.sequence,
+ model=msg.model,
+ routing_source=msg.routing_source,
+ )
+ except Exception as e:
+ logger.error(
+ f"Failed to back-fill stamps for session "
+ f"{session.session_id} seq {msg.sequence}: {e}"
+ )
+ return
+ if updated:
+ msg.stamps_pending_save = False
+
async def _backfill(msg: ChatMessage) -> None:
assert msg.sequence is not None # narrowed by the filter above
try:
@@ -985,8 +1044,14 @@ async def _save_session_to_db(
f"{session.session_id} seq {msg.sequence} (row not found)"
)
- if pending:
- await asyncio.gather(*(_backfill(m) for m in pending))
+ # One concurrent pass for both repairs — a mid-turn-flushed assistant
+ # row with tools commonly needs BOTH back-fills; success/failure flags
+ # stay independent per mechanism.
+ if pending or pending_stamps:
+ await asyncio.gather(
+ *(_backfill(m) for m in pending),
+ *(_backfill_stamps(m) for m in pending_stamps),
+ )
async def append_and_save_message(
diff --git a/autogpt_platform/backend/backend/copilot/model_router.py b/autogpt_platform/backend/backend/copilot/model_router.py
index 181e736b6d..f7238a9f2a 100644
--- a/autogpt_platform/backend/backend/copilot/model_router.py
+++ b/autogpt_platform/backend/backend/copilot/model_router.py
@@ -1,11 +1,24 @@
-"""LaunchDarkly-aware model selection for the copilot.
+"""Model selection for the copilot: LaunchDarkly → registry cell → env.
-Each cell of the ``(mode, tier)`` matrix has a static default baked into
-``ChatConfig`` (see ``copilot/config.py``) and a single JSON-valued
-LaunchDarkly feature flag — ``copilot-model-routing`` — that can override
-it per-user. This module centralises the lookup so both the baseline
-and SDK paths agree on the selection rule and so A/B experiments can
-target a single cell without shipping a config change.
+Each cell of the ``(mode, tier)`` matrix resolves through three layers:
+
+1. The JSON-valued LaunchDarkly flag ``copilot-model-routing`` (per-user —
+ cohort experiments and rollouts live here, and the flag returns model
+ slugs directly).
+2. The catalog's routing cell (``catalog.py`` ``routing`` section, surface
+ ``"copilot"``) — our cloud's deployment config, shipped with the code.
+3. The static ``ChatConfig`` default (env vars) — the bootstrap floor.
+
+On our cloud, the registry is the serve-time gate for layers 1 and 2: a slug
+the catalog doesn't know, or one with ``is_enabled=False`` (the kill switch),
+is refused — loudly (log + Sentry) — and resolution falls through to the next
+layer. ``visibility=HIDDEN`` models DO serve when routed: that's the
+pre-launch testing state (registered + routable, not shown in pickers).
+Self-hosted installs and local transports skip the gate entirely (their LD
+and env slugs are their own business — the shipped catalog must not veto an
+operator's custom model). An EMPTY registry never gates anything either;
+production processes always load it (fail-hard boot), so that branch is
+defense-in-depth for exotic embedders only.
Matrix:
@@ -24,23 +37,120 @@ LD payload shape::
}
Missing mode, missing tier-within-mode, non-string cell value, non-dict
-payload, or LD failure all fall back to the corresponding ``ChatConfig``
-default.
+payload, or LD failure all fall through to the next layer.
"""
from __future__ import annotations
import logging
-from typing import Literal
+from typing import Literal, NamedTuple
+
+import sentry_sdk
+import backend.data.llm_registry as llm_registry
from backend.copilot.config import ChatConfig
+from backend.copilot.model import RoutingSource
+from backend.data.llm_registry.llm_models import LLMModel, transport_slug_candidates
from backend.util.feature_flag import Flag, get_feature_flag_value
+from backend.util.settings import BehaveAs, Settings
logger = logging.getLogger(__name__)
+settings = Settings()
ModelMode = Literal["fast", "thinking"]
ModelTier = Literal["standard", "advanced"]
+ROUTE_SURFACE_COPILOT = "copilot"
+
+
+class ResolvedModel(NamedTuple):
+ model: str
+ source: RoutingSource
+
+
+def _catalog_lookup(slug: str) -> "llm_registry.RegistryModel | None":
+ """Look up *slug* in the catalog, tolerating transport spellings.
+
+ The catalog registers Claude models under bare canonical enum slugs
+ (``claude-opus-4-7``) while LaunchDarkly payloads and env defaults use
+ OpenRouter forms (``anthropic/claude-opus-4.6``). Gate on the model,
+ not the spelling: try the exact slug, then the ``anthropic/``-stripped
+ tail, then its dots→dashes form.
+ """
+ candidates = transport_slug_candidates(slug)
+ for candidate in candidates:
+ model = llm_registry.get_model(candidate)
+ if model is not None:
+ return model
+ # Anthropic's API slugs carry a -YYYYMMDD snapshot suffix that the
+ # OpenRouter canonical form drops (anthropic/claude-haiku-4-5 ↔ catalog
+ # claude-haiku-4-5-20251001) — resolve via the date-stripped index
+ # built at catalog load (O(1), no per-turn scan).
+ for candidate in candidates:
+ model = llm_registry.get_model_by_date_stripped_slug(candidate)
+ if model is not None:
+ return model
+ # Transport spellings are fuzzy (prefix-strip, dot/dash, snapshot-date)
+ # and deliberately don't know the enum's exact alias map, so an
+ # OpenRouter OpenAI slug whose catalog entry carries a -YYYY-MM-DD
+ # snapshot (openai/gpt-5.4 → gpt-5.4-2026-03-05) slips past them. Honor
+ # the enum's alias resolution as a final gate so an LD/env cell set to
+ # such a slug resolves instead of being refused as unknown.
+ try:
+ member = LLMModel(slug)
+ except ValueError:
+ return None
+ return llm_registry.get_model(member.value)
+
+
+_sentry_reported: set[tuple[str, str]] = set()
+_unloaded_reported = False
+
+
+async def _registry_refuses(slug: str, layer: RoutingSource) -> str | None:
+ """Return a refusal reason if the registry gates *slug*, else None.
+
+ An empty registry gates nothing (dormant-registry installs must keep
+ exact pre-registry behavior). Unknown slug or kill-switched model is
+ refused; HIDDEN visibility serves fine when explicitly routed.
+ """
+ if not llm_registry.has_models():
+ global _unloaded_reported
+ if not llm_registry.is_loaded() and not _unloaded_reported:
+ # Empty-because-dormant is legitimate; empty-because-nobody-
+ # called-load_catalog() in this process is a wiring bug that
+ # would silently disable gating and cells — say so, once.
+ _unloaded_reported = True
+ logger.error(
+ "[model_router] registry gating skipped: load_catalog() was "
+ "never called in this process — routing cells and serve-time "
+ "gating are inactive"
+ )
+ return None
+ model = _catalog_lookup(slug)
+ if model is None:
+ reason = "unknown to the model registry"
+ elif not model.is_enabled:
+ reason = "disabled in the model registry (kill switch)"
+ else:
+ return None
+ logger.warning(
+ "[model_router] %s-layer slug %r refused: %s — falling through",
+ layer,
+ slug,
+ reason,
+ )
+ # Log every refusal (greppable), but Sentry only once per (layer, slug)
+ # per process — a bad LD slug refuses on EVERY turn until fixed, and one
+ # event per turn during an incident is noise, not signal.
+ if (layer, slug) not in _sentry_reported:
+ _sentry_reported.add((layer, slug))
+ sentry_sdk.capture_message(
+ f"copilot routing refused {layer} slug {slug!r}: {reason}",
+ level="warning",
+ )
+ return reason
+
def _config_default(config: ChatConfig, mode: ModelMode, tier: ModelTier) -> str:
if mode == "fast":
@@ -56,24 +166,8 @@ def _config_default(config: ChatConfig, mode: ModelMode, tier: ModelTier) -> str
)
-async def resolve_model(
- mode: ModelMode,
- tier: ModelTier,
- user_id: str | None,
- *,
- config: ChatConfig,
-) -> str:
- """Return the model identifier for a ``(mode, tier)`` cell.
-
- Consults ``copilot-model-routing`` (JSON) for *user_id* — LD targeting is
- still per-user so cohorts can receive different routing — and falls back to
- the ``ChatConfig`` default on missing user, missing / invalid flag payload,
- or non-string cell value.
- """
- fallback = _config_default(config, mode, tier).strip()
- if not user_id:
- return fallback
-
+async def _ld_cell_value(mode: ModelMode, tier: ModelTier, user_id: str) -> str | None:
+ """Extract the (mode, tier) slug from the LD JSON flag, or None."""
try:
payload: object = await get_feature_flag_value(
Flag.COPILOT_MODEL_ROUTING.value, user_id, default=None
@@ -81,27 +175,25 @@ async def resolve_model(
except Exception:
logger.warning(
"[model_router] LD lookup failed for copilot-model-routing — "
- "using config default %s for (%s, %s)",
- fallback,
+ "falling through for (%s, %s)",
mode,
tier,
exc_info=True,
)
- return fallback
+ return None
if payload is None:
- return fallback
+ return None
if not isinstance(payload, dict):
logger.warning(
"[model_router] copilot-model-routing expected a JSON object, got %r — "
- "using config default %s for (%s, %s)",
+ "falling through for (%s, %s)",
payload,
- fallback,
mode,
tier,
)
- return fallback
+ return None
mode_cell = payload.get(mode)
if mode in payload and not isinstance(mode_cell, dict):
@@ -109,14 +201,13 @@ async def resolve_model(
# a {tier: model} dict — surface the typo in logs.
logger.warning(
"[model_router] copilot-model-routing[%s] expected a JSON object, "
- "got %r — using config default %s for tier %s",
+ "got %r — falling through for tier %s",
mode,
mode_cell,
- fallback,
tier,
)
if not isinstance(mode_cell, dict):
- return fallback
+ return None
value = mode_cell.get(tier)
if isinstance(value, str) and value.strip():
@@ -129,10 +220,74 @@ async def resolve_model(
)
logger.warning(
"[model_router] copilot-model-routing[%s][%s] returned %s — "
- "using config default %s",
+ "falling through",
mode,
tier,
reason,
- fallback,
)
- return fallback
+ return None
+
+
+async def _env_floor(
+ config: ChatConfig, mode: ModelMode, tier: ModelTier
+) -> ResolvedModel:
+ """Serve the env default — the LAST layer, served even when the catalog
+ refuses it (refusing would leave nothing). A kill switch pointing here
+ is an incident the operator must hear about: log + Sentry, then serve.
+ """
+ env_slug = _config_default(config, mode, tier).strip()
+ if await _registry_refuses(env_slug, "env") is not None:
+ logger.error(
+ "[model_router] env default %r is refused by the catalog "
+ "but served anyway (last-resort floor) — change the "
+ "CHAT_*_MODEL default or the routing cell",
+ env_slug,
+ )
+ return ResolvedModel(env_slug, "env")
+
+
+async def resolve_model_route(
+ mode: ModelMode,
+ tier: ModelTier,
+ user_id: str | None,
+ *,
+ config: ChatConfig,
+) -> ResolvedModel:
+ """Resolve a ``(mode, tier)`` cell through LD → registry cell → env.
+
+ Every layer's slug is validated against the registry (see module
+ docstring); a refused slug falls through to the next layer. The returned
+ ``source`` is stamped onto persisted chat messages so product
+ intelligence can segment quality metrics by model and routing layer.
+ """
+ # The catalog gates (and cells apply) on OUR CLOUD's hosted transports
+ # only — they are not rules for everyone:
+ # - self-hosted installs (behave_as != CLOUD) keep LD/env authority over
+ # their own slugs; the shipped catalog must not veto an operator's
+ # custom model (they can't edit our catalog to register it)
+ # - local transports (Ollama/vLLM) pass slugs through verbatim; catalog
+ # gating would refuse every local model and a cloud-slug cell would
+ # 404 at request time
+ # Both resolve LD → env, exactly as before the catalog existed.
+ gated = (
+ settings.config.behave_as == BehaveAs.CLOUD
+ and config.baseline_provider != "local"
+ )
+
+ if user_id:
+ ld_slug = await _ld_cell_value(mode, tier, user_id)
+ if ld_slug and (not gated or await _registry_refuses(ld_slug, "ld") is None):
+ return ResolvedModel(ld_slug, "ld")
+
+ if not gated:
+ return ResolvedModel(_config_default(config, mode, tier).strip(), "env")
+
+ cell_slug = llm_registry.get_route(ROUTE_SURFACE_COPILOT, mode, tier)
+ if cell_slug and await _registry_refuses(cell_slug, "catalog") is None:
+ # Cells carry TRANSPORT-READY spellings (e.g. the vendor-prefixed
+ # dot form ``anthropic/claude-sonnet-4.6`` OpenRouter serves) and are
+ # returned verbatim; the catalog guard tests enforce the convention,
+ # and the slug-tolerant gate above maps them to catalog identity.
+ return ResolvedModel(cell_slug, "catalog")
+
+ return await _env_floor(config, mode, tier)
diff --git a/autogpt_platform/backend/backend/copilot/model_router_test.py b/autogpt_platform/backend/backend/copilot/model_router_test.py
index d89f4daf4f..559a0c8665 100644
--- a/autogpt_platform/backend/backend/copilot/model_router_test.py
+++ b/autogpt_platform/backend/backend/copilot/model_router_test.py
@@ -1,12 +1,51 @@
"""Tests for the LD-aware model resolver."""
import logging
+import textwrap
from unittest.mock import AsyncMock, patch
import pytest
+import backend.data.llm_registry.registry as reg
from backend.copilot.config import ChatConfig
-from backend.copilot.model_router import _config_default, resolve_model
+from backend.copilot.model_router import _config_default, resolve_model_route
+
+
+def _function_calls(module_name: str, obj_name: str, callee: str) -> bool:
+ """AST-verified: *obj_name* in *module_name* contains a real CALL of
+ *callee* — a commented-out call cannot pass (unlike source grep)."""
+ import ast
+ import importlib
+ import inspect
+
+ module = importlib.import_module(module_name)
+ src = textwrap.dedent(inspect.getsource(getattr(module, obj_name)))
+ for node in ast.walk(ast.parse(src)):
+ if isinstance(node, ast.Call):
+ f = node.func
+ name = f.attr if isinstance(f, ast.Attribute) else getattr(f, "id", None)
+ if name == callee:
+ return True
+ return False
+
+
+async def resolve_model(mode, tier, user_id, *, config):
+ """Test-local stand-in for the deleted back-compat wrapper: these tests
+ assert on resolution VALUES; the routing source is covered elsewhere."""
+ return (await resolve_model_route(mode, tier, user_id, config=config)).model
+
+
[email protected](autouse=True)
+def _empty_catalog_by_default():
+ """These tests assert the resolver's ungated (pre-catalog) behavior, which
+ requires an EMPTY registry — but other test modules (registry_test) load
+ the real catalog into the module globals and legitimately leave it there.
+ Snapshot, clear, restore, so suite order can never change outcomes.
+ TestRegistryGating's own fixture layers its populated state on top."""
+ old = (reg._dynamic_models, reg._date_stripped_models, reg._routes)
+ reg._dynamic_models, reg._date_stripped_models, reg._routes = {}, {}, {}
+ yield
+ reg._dynamic_models, reg._date_stripped_models, reg._routes = old
def _make_config() -> ChatConfig:
@@ -314,3 +353,537 @@ class TestResolveModel:
await resolve_model("thinking", "advanced", "u", config=cfg)
assert calls == ["copilot-model-routing"] * 4
+
+
+class TestRegistryGating:
+ """LD → registry cell → env resolution with serve-time slug validation."""
+
+ @pytest.fixture(autouse=True)
+ def registry_state(self, mocker):
+ import backend.data.llm_registry.registry as reg
+ from backend.data.llm_registry.registry import (
+ RegistryModel,
+ RegistryModelMetadata,
+ )
+
+ def make(slug, *, enabled=True, visibility="GA"):
+ return RegistryModel(
+ slug=slug,
+ display_name=slug,
+ metadata=RegistryModelMetadata(
+ provider="open_router",
+ context_window=100000,
+ max_output_tokens=8192,
+ display_name=slug,
+ provider_name="OpenRouter",
+ creator_name="Test",
+ price_tier=2,
+ ),
+ provider_display_name="OpenRouter",
+ is_enabled=enabled,
+ visibility=visibility,
+ )
+
+ self.reg = reg
+ self.make = make
+ old_models, old_routes = reg._dynamic_models, reg._routes
+ reg._dynamic_models = {
+ "known/model": make("known/model"),
+ "disabled/model": make("disabled/model", enabled=False),
+ "hidden/model": make("hidden/model", visibility="HIDDEN"),
+ "cell/model": make("cell/model"),
+ # The env-floor defaults resolve through the gate too (served
+ # regardless, but logged) — register them so refusal
+ # assertions only count the layer under test.
+ "claude-sonnet-4-6": make("claude-sonnet-4-6"),
+ "claude-opus-4-7": make("claude-opus-4-7"),
+ }
+ reg._routes = {}
+ # Mirror load_catalog's derived index for the seeded models.
+ from backend.data.llm_registry.llm_models import MODEL_DATE_SUFFIX_RE
+
+ reg._date_stripped_models = {
+ stripped: m
+ for slug, m in reg._dynamic_models.items()
+ if (stripped := MODEL_DATE_SUFFIX_RE.sub("", slug)) != slug
+ }
+ # Cell tests exercise the cloud path — the test env's default
+ # behave_as is LOCAL, which (correctly) skips cells entirely.
+ import backend.copilot.model_router as router_mod
+ from backend.util.settings import BehaveAs
+
+ mocker.patch.object(router_mod.settings.config, "behave_as", BehaveAs.CLOUD)
+ router_mod._sentry_reported.clear()
+ self.sentry = mocker.patch(
+ "backend.copilot.model_router.sentry_sdk.capture_message"
+ )
+ yield
+ reg._dynamic_models, reg._routes = old_models, old_routes
+
+ def _ld(self, mocker, slug):
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value={"fast": {"standard": slug}}),
+ )
+
+ @pytest.mark.asyncio
+ async def test_known_ld_slug_serves_with_ld_source(self, mocker):
+ from backend.copilot.model_router import resolve_model_route
+
+ self._ld(mocker, "known/model")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("known/model", "ld")
+
+ @pytest.mark.asyncio
+ async def test_unknown_ld_slug_refused_and_falls_through(self, mocker):
+ from backend.copilot.model_router import resolve_model_route
+
+ self._ld(mocker, "typo/model")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved.source == "env"
+ self.sentry.assert_called_once()
+ assert "'typo/model'" in self.sentry.call_args.args[0]
+
+ @pytest.mark.asyncio
+ async def test_sentry_fires_once_per_refused_slug(self, mocker):
+ """A bad LD slug refuses on every turn; Sentry hears about it once
+ per process, the log hears about it every time."""
+ from backend.copilot.model_router import resolve_model_route
+
+ self._ld(mocker, "typo/model")
+ for _ in range(3):
+ await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ self.sentry.assert_called_once()
+
+ @pytest.mark.asyncio
+ async def test_ld_date_suffixed_anthropic_slug_serves(self, mocker):
+ """OpenRouter drops the -YYYYMMDD snapshot suffix Anthropic's API
+ slugs carry; an LD experiment routing to the 4.5 family must match
+ the suffixed catalog slug, not be refused as unknown."""
+ from backend.copilot.model_router import resolve_model_route
+
+ self.reg._dynamic_models["claude-haiku-4-5-20251001"] = self.make(
+ "claude-haiku-4-5-20251001"
+ )
+ self.reg._date_stripped_models["claude-haiku-4-5"] = self.reg._dynamic_models[
+ "claude-haiku-4-5-20251001"
+ ]
+ self._ld(mocker, "anthropic/claude-haiku-4-5")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("anthropic/claude-haiku-4-5", "ld")
+ self.sentry.assert_not_called()
+
+ @pytest.mark.asyncio
+ async def test_ld_vendor_prefixed_openai_slug_serves(self, mocker):
+ """LD may route with ANY vendor prefix (openai/gpt-5.4), while the
+ catalog stores the bare (possibly date-suffixed) slug — the gate
+ must match the model, not the vendor spelling.
+
+ The OpenAI snapshot suffix is -YYYY-MM-DD (dashed), which the
+ transport-spelling fallbacks and the -YYYYMMDD date-stripped index
+ both miss; resolution here rides the enum's exact alias map
+ (openai/gpt-5.4 -> gpt-5.4-2026-03-05). The index below is built
+ the same way load_catalog derives it, so this slug is deliberately
+ absent from it — proving the enum-alias gate, not a hand-seeded
+ index entry, is what resolves the slug.
+ """
+ self.reg._dynamic_models["gpt-5.4-2026-03-05"] = self.make("gpt-5.4-2026-03-05")
+ from backend.data.llm_registry.llm_models import MODEL_DATE_SUFFIX_RE
+
+ self.reg._date_stripped_models = {
+ stripped: m
+ for slug, m in self.reg._dynamic_models.items()
+ if (stripped := MODEL_DATE_SUFFIX_RE.sub("", slug)) != slug
+ }
+ assert "gpt-5.4" not in self.reg._date_stripped_models
+ self._ld(mocker, "openai/gpt-5.4")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("openai/gpt-5.4", "ld")
+ self.sentry.assert_not_called()
+
+ @pytest.mark.asyncio
+ async def test_disabled_ld_slug_refused_kill_switch(self, mocker):
+ from backend.copilot.model_router import resolve_model_route
+
+ self._ld(mocker, "disabled/model")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved.source == "env"
+ self.sentry.assert_called_once()
+
+ @pytest.mark.asyncio
+ async def test_hidden_ld_slug_serves(self, mocker):
+ """HIDDEN = registered + routable, just not shown in pickers."""
+ from backend.copilot.model_router import resolve_model_route
+
+ self._ld(mocker, "hidden/model")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("hidden/model", "ld")
+ self.sentry.assert_not_called()
+
+ @pytest.mark.asyncio
+ async def test_db_cell_used_when_no_ld(self, mocker):
+ from backend.copilot.model_router import resolve_model_route
+
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value=None),
+ )
+ self.reg._routes = {("copilot", "fast", "standard"): "cell/model"}
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("cell/model", "catalog")
+
+ @pytest.mark.asyncio
+ async def test_ld_beats_db_cell(self, mocker):
+ from backend.copilot.model_router import resolve_model_route
+
+ self._ld(mocker, "known/model")
+ self.reg._routes = {("copilot", "fast", "standard"): "cell/model"}
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("known/model", "ld")
+
+ @pytest.mark.asyncio
+ async def test_stale_db_cell_refused_falls_to_env(self, mocker):
+ from backend.copilot.model_router import resolve_model_route
+
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value=None),
+ )
+ self.reg._routes = {("copilot", "fast", "standard"): "disabled/model"}
+ cfg = _make_config()
+ resolved = await resolve_model_route("fast", "standard", "user-1", config=cfg)
+ assert resolved == (cfg.fast_standard_model, "env")
+ assert "refused catalog slug" in self.sentry.call_args.args[0]
+
+ @pytest.mark.asyncio
+ async def test_no_user_skips_ld_but_uses_db_cell(self):
+ from backend.copilot.model_router import resolve_model_route
+
+ self.reg._routes = {("copilot", "fast", "standard"): "cell/model"}
+ resolved = await resolve_model_route(
+ "fast", "standard", None, config=_make_config()
+ )
+ assert resolved == ("cell/model", "catalog")
+
+ @pytest.mark.asyncio
+ async def test_empty_registry_gates_nothing(self, mocker):
+ """Dormant-registry installs keep exact pre-registry behavior."""
+ from backend.copilot.model_router import resolve_model_route
+
+ self.reg._dynamic_models = {}
+ self._ld(mocker, "totally/unregistered")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("totally/unregistered", "ld")
+ self.sentry.assert_not_called()
+
+ @pytest.mark.asyncio
+ async def test_ld_openrouter_spelling_matches_bare_catalog_slug(self, mocker):
+ """LD sends anthropic/claude-opus-4.6-style slugs; the catalog holds
+ bare dashed enum slugs. The gate must match the model, not the
+ spelling."""
+ from backend.copilot.model_router import resolve_model_route
+
+ self.reg._dynamic_models["claude-opus-4-6"] = self.make("claude-opus-4-6")
+ self._ld(mocker, "anthropic/claude-opus-4.6")
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("anthropic/claude-opus-4.6", "ld")
+ self.sentry.assert_not_called()
+
+ @pytest.mark.asyncio
+ async def test_cell_value_is_returned_verbatim(self, mocker):
+ """Cells carry transport-ready spellings and are returned untouched;
+ the slug-tolerant gate maps them to catalog identity."""
+ from backend.copilot.model_router import resolve_model_route
+
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value=None),
+ )
+ self.reg._dynamic_models["claude-sonnet-4-6"] = self.make("claude-sonnet-4-6")
+ self.reg._routes = {
+ ("copilot", "fast", "standard"): "anthropic/claude-sonnet-4.6"
+ }
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ assert resolved == ("anthropic/claude-sonnet-4.6", "catalog")
+
+ def test_catalog_cell_values_normalize_on_every_transport(self, mocker):
+ """The REAL catalog routing cells must be transport-ready: OpenRouter
+ sends them verbatim (so they must be genuine OR slugs — the dot
+ forms), and the direct-Anthropic family strips the vendor prefix and
+ dedots without raising."""
+ from unittest.mock import PropertyMock
+
+ from backend.copilot.config import ChatConfig
+ from backend.copilot.model_normalize import normalize_model_for_transport
+ from backend.data.llm_registry import get_catalog
+
+ # Cells ship empty (claiming one is an explicit catalog PR), so this
+ # governs any real cells that exist PLUS the canonical spellings the
+ # runbook tells operators to use — the transport property must hold
+ # before anyone claims a cell.
+ cells = [
+ slug
+ for modes in get_catalog().routing.values()
+ for tiers in modes.values()
+ for slug in tiers.values()
+ ] + [
+ "anthropic/claude-sonnet-4.6",
+ "anthropic/claude-opus-4.7",
+ ]
+
+ def cfg_for(transport: str) -> ChatConfig:
+ cfg = ChatConfig()
+ mocker.patch.object(
+ type(cfg),
+ "effective_transport",
+ new_callable=PropertyMock,
+ return_value=transport,
+ )
+ return cfg
+
+ for slug in cells:
+ # OpenRouter passes cells through verbatim — the cell IS the wire slug.
+ assert normalize_model_for_transport(slug, cfg_for("openrouter")) == slug
+ # Direct-Anthropic strips the vendor prefix and dedots.
+ direct = normalize_model_for_transport(slug, cfg_for("subscription"))
+ assert "/" not in direct and "." not in direct
+ assert direct.startswith("claude-")
+
+
+class TestLocalTransportSkipsCells:
+ """Catalog cells hold cloud slugs; local transports must ignore them."""
+
+ @pytest.mark.asyncio
+ async def test_local_transport_resolves_env_not_cell(self, mocker):
+ from unittest.mock import PropertyMock
+
+ import backend.data.llm_registry.registry as reg
+ from backend.copilot.model_router import resolve_model_route
+
+ old_routes = reg._routes
+ reg._routes = {("copilot", "fast", "standard"): "claude-sonnet-4-6"}
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value=None),
+ )
+ cfg = _make_config()
+ mocker.patch.object(
+ type(cfg),
+ "baseline_provider",
+ new_callable=PropertyMock,
+ return_value="local",
+ )
+ try:
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=cfg
+ )
+ finally:
+ reg._routes = old_routes
+
+ assert resolved == (cfg.fast_standard_model, "env")
+
+
+class TestExecutorCatalogLoad:
+ """The copilot executor process must load the catalog (turns run there,
+ not in the rest API process — an unloaded registry no-ops all routing)."""
+
+ def test_loader_calls_load_catalog(self, mocker):
+ from backend.copilot.executor import manager
+
+ load = mocker.patch.object(manager.backend.data.llm_registry, "load_catalog")
+ manager._load_catalog()
+ load.assert_called_once()
+
+ def test_loader_fails_hard_on_broken_catalog(self, mocker):
+ """Post-cutover the catalog is load-bearing — a load failure must
+ stop the process, not silently disable routing cells and gating."""
+ from backend.copilot.executor import manager
+
+ mocker.patch.object(
+ manager.backend.data.llm_registry,
+ "load_catalog",
+ side_effect=RuntimeError("bad catalog"),
+ )
+ with pytest.raises(RuntimeError, match="bad catalog"):
+ manager._load_catalog()
+
+ def test_executor_run_invokes_the_loader(self):
+ assert _function_calls(
+ "backend.copilot.executor.manager", "CoPilotExecutor", "_load_catalog"
+ ), (
+ "CoPilotExecutor.run must load the LLM catalog — turns execute in "
+ "this process, and without the load every routing cell and "
+ "serve-time gate silently no-ops"
+ )
+
+
+class TestSelfHostedSkipsCells:
+ """Cells are cloud deployment config: behave_as != CLOUD ignores them."""
+
+ @pytest.mark.asyncio
+ async def test_self_hosted_ld_slug_not_catalog_gated(self, mocker):
+ """A self-hosted operator's LD flag may route to their own model
+ (e.g. a custom Ollama tag) that the shipped catalog has never heard
+ of — the catalog must not veto it. Gating is cloud-only."""
+ import backend.data.llm_registry.registry as reg
+ from backend.copilot.model_router import resolve_model_route
+
+ old_models, old_routes = reg._dynamic_models, reg._routes
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value={"fast": {"standard": "qwen3:8b-custom"}}),
+ )
+ try:
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=_make_config()
+ )
+ finally:
+ reg._dynamic_models, reg._routes = old_models, old_routes
+
+ assert resolved == ("qwen3:8b-custom", "ld")
+
+ @pytest.mark.asyncio
+ async def test_self_hosted_cloud_transport_resolves_env_not_cell(self, mocker):
+ import backend.data.llm_registry.registry as reg
+ from backend.copilot.model_router import resolve_model_route
+
+ old_models, old_routes = reg._dynamic_models, reg._routes
+ # behave_as defaults to LOCAL in the test env — that IS the case
+ # under test; no patch needed. Transport stays cloud (openrouter).
+ reg._routes = {("copilot", "fast", "standard"): "anthropic/claude-sonnet-4.6"}
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value=None),
+ )
+ cfg = _make_config()
+ try:
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=cfg
+ )
+ finally:
+ reg._dynamic_models, reg._routes = old_models, old_routes
+
+ assert resolved == (cfg.fast_standard_model, "env")
+
+
+class TestEnvFloorIncidentPath:
+ """The env default is served even when the catalog refuses it (the last
+ layer cannot fall through), but the refusal must be LOUD."""
+
+ @pytest.mark.asyncio
+ async def test_killed_env_default_serves_but_screams(self, mocker):
+ import backend.copilot.model_router as router_mod
+ import backend.data.llm_registry.registry as reg
+ from backend.copilot.model_router import resolve_model_route
+ from backend.util.settings import BehaveAs
+
+ mocker.patch.object(router_mod.settings.config, "behave_as", BehaveAs.CLOUD)
+ router_mod._sentry_reported.clear()
+ sentry = mocker.patch("backend.copilot.model_router.sentry_sdk.capture_message")
+ mocker.patch(
+ "backend.copilot.model_router.get_feature_flag_value",
+ new=AsyncMock(return_value=None),
+ )
+
+ cfg = _make_config()
+ # Register the env default as KILL-SWITCHED
+ from backend.data.llm_registry.registry import (
+ RegistryModel,
+ RegistryModelMetadata,
+ )
+
+ def _entry(slug, enabled):
+ return RegistryModel(
+ slug=slug,
+ display_name=slug,
+ metadata=RegistryModelMetadata(
+ provider="anthropic",
+ context_window=1000,
+ max_output_tokens=None,
+ display_name=slug,
+ provider_name="Anthropic",
+ creator_name="Anthropic",
+ price_tier=1,
+ ),
+ provider_display_name="Anthropic",
+ is_enabled=enabled,
+ )
+
+ old = (reg._dynamic_models, reg._routes)
+ reg._dynamic_models = {"claude-sonnet-4-6": _entry("claude-sonnet-4-6", False)}
+ reg._routes = {}
+ try:
+ resolved = await resolve_model_route(
+ "fast", "standard", "user-1", config=cfg
+ )
+ finally:
+ reg._dynamic_models, reg._routes = old
+
+ # Serves anyway (last resort) but Sentry heard about it.
+ assert resolved == (cfg.fast_standard_model, "env")
+ sentry.assert_called_once()
+
+
+class TestUnloadedRegistryWiring:
+ """Empty-because-never-loaded is a wiring bug and must log loudly once,
+ distinct from a legitimately dormant empty registry."""
+
+ @pytest.mark.asyncio
+ async def test_never_loaded_process_logs_wiring_error_once(self, mocker, caplog):
+ import logging
+
+ import backend.copilot.model_router as router_mod
+ import backend.data.llm_registry.registry as reg
+ from backend.copilot.model_router import _registry_refuses
+
+ old_models, old_loaded = reg._dynamic_models, reg._loaded
+ reg._dynamic_models, reg._loaded = {}, False
+ router_mod._unloaded_reported = False
+ try:
+ with caplog.at_level(logging.ERROR):
+ assert await _registry_refuses("x/y", "ld") is None
+ assert await _registry_refuses("x/y", "ld") is None
+ finally:
+ reg._dynamic_models, reg._loaded = old_models, old_loaded
+ router_mod._unloaded_reported = False
+
+ wiring_errors = [r for r in caplog.records if "never called" in r.message]
+ assert len(wiring_errors) == 1
+
+
+class TestRestApiCatalogLoad:
+ """The REST API lifespan must load the catalog (same wiring guarantee
+ the executor gets)."""
+
+ def test_lifespan_invokes_the_loader(self):
+ assert _function_calls(
+ "backend.api.rest_api", "lifespan_context", "load_catalog"
+ ), (
+ "rest_api's lifespan must load the LLM catalog — without it "
+ "routing cells and serve-time gating silently no-op in the "
+ "API process"
+ )
diff --git a/autogpt_platform/backend/backend/copilot/model_test.py b/autogpt_platform/backend/backend/copilot/model_test.py
index b14f4121da..d74e5d6579 100644
--- a/autogpt_platform/backend/backend/copilot/model_test.py
+++ b/autogpt_platform/backend/backend/copilot/model_test.py
@@ -1573,3 +1573,69 @@ async def test_backfill_partial_failure_clears_only_successes(
assert m2.tool_calls_pending_save is True
assert m3.tool_calls_pending_save is True
assert mock_db.update_chat_message_tool_calls.await_count == 3
+
+
[email protected](loop_scope="session")
+async def test_save_session_to_db_backfills_stamps_on_flushed_rows(
+ mocker: MockerFixture,
+) -> None:
+ """An assistant row flushed before end-of-turn stamping gets its
+ model/routingSource back-filled on the next save, flag cleared."""
+ flushed = ChatMessage(
+ role="assistant",
+ content="answer",
+ sequence=7,
+ model="claude-sonnet-4-6",
+ routing_source="env",
+ stamps_pending_save=True,
+ )
+ session = _make_session_with_messages(flushed)
+
+ mock_db = mocker.MagicMock()
+ mock_db.update_chat_session = mocker.AsyncMock()
+ mock_db.add_chat_messages_batch = mocker.AsyncMock(return_value=8)
+ mock_db.update_chat_message_stamps = mocker.AsyncMock(return_value=True)
+ mocker.patch("backend.copilot.model.chat_db", return_value=mock_db)
+
+ await _save_session_to_db(
+ session, existing_message_count=8, skip_existence_check=True
+ )
+
+ mock_db.update_chat_message_stamps.assert_awaited_once_with(
+ session_id=session.session_id,
+ sequence=7,
+ model="claude-sonnet-4-6",
+ routing_source="env",
+ )
+ assert flushed.stamps_pending_save is False
+
+
[email protected](loop_scope="session")
+async def test_save_session_to_db_stamp_backfill_failure_keeps_flag(
+ mocker: MockerFixture,
+) -> None:
+ """A failed stamp back-fill keeps the flag set so any later save
+ retries — mirror of the tool_calls failure semantics."""
+ flushed = ChatMessage(
+ role="assistant",
+ content="answer",
+ sequence=7,
+ model="claude-sonnet-4-6",
+ routing_source="catalog",
+ stamps_pending_save=True,
+ )
+ session = _make_session_with_messages(flushed)
+
+ mock_db = mocker.MagicMock()
+ mock_db.update_chat_session = mocker.AsyncMock()
+ mock_db.add_chat_messages_batch = mocker.AsyncMock(return_value=8)
+ mock_db.update_chat_message_stamps = mocker.AsyncMock(
+ side_effect=RuntimeError("db down")
+ )
+ mocker.patch("backend.copilot.model.chat_db", return_value=mock_db)
+
+ await _save_session_to_db(
+ session, existing_message_count=8, skip_existence_check=True
+ )
+
+ assert flushed.stamps_pending_save is True
diff --git a/autogpt_platform/backend/backend/copilot/moonshot.py b/autogpt_platform/backend/backend/copilot/moonshot.py
index c117e76120..3cb138f8a1 100644
--- a/autogpt_platform/backend/backend/copilot/moonshot.py
+++ b/autogpt_platform/backend/backend/copilot/moonshot.py
@@ -31,40 +31,59 @@ cache-control gating don't handle on their own:
site; see ``baseline/service.py::_supports_prompt_cache_markers``.)
Detection is prefix-based (``moonshotai/``). Moonshot routes every Kimi
-SKU through the same Anthropic-compat surface and currently prices them
-identically, so a new ``moonshotai/kimi-k3.0`` slug transparently
+SKU through the same Anthropic-compat surface, so a new slug transparently
inherits both the rate card and the cache-control gate without editing
-this file. Per-slug overrides are in :data:`_RATE_OVERRIDES_USD_PER_MTOK`
-for when Moonshot eventually splits prices.
+this file. SKUs priced off the shared K2.x default (Kimi K3 broke the
+uniform pricing at 5x) carry per-slug entries in
+:data:`_RATE_OVERRIDES_USD_PER_MTOK`.
"""
from __future__ import annotations
-# All Moonshot slugs share these rates as of April 2026 — Moonshot prices
-# every Kimi K2.x SKU at $0.60/$2.80 per million (input/output) via
-# OpenRouter. Cache-read / cache-write discounts are NOT applied here:
+from backend.data.llm_registry.catalog import get_catalog
+
+# Every Kimi K2.x SKU shares these rates — $0.60/$2.80 per million
+# (input/output) via OpenRouter as of April 2026.
+# Cache-read / cache-write discounts are NOT applied here:
# OpenRouter currently exposes only a single input price per Moonshot
# endpoint; the real billed amount (with cache savings) lands via the
# reconcile path. Keep in sync with https://platform.moonshot.ai/docs/pricing.
_DEFAULT_MOONSHOT_RATE_USD_PER_MTOK: tuple[float, float] = (0.60, 2.80)
-# Per-slug overrides for when Moonshot splits pricing across SKUs. Empty
-# today — every slug matching ``moonshotai/`` falls back to
-# :data:`_DEFAULT_MOONSHOT_RATE_USD_PER_MTOK`.
-_RATE_OVERRIDES_USD_PER_MTOK: dict[str, tuple[float, float]] = {}
# Vendor prefix — matches any OpenRouter slug Moonshot ships. Keep as a
# module constant so the prefix check stays in exactly one place.
_MOONSHOT_PREFIX = "moonshotai/"
+def _overrides_from_catalog() -> dict[str, tuple[float, float]]:
+ """Per-slug USD overrides for SKUs priced off the shared K2.x default.
+
+ Authored in the catalog (``provider_*_usd_per_1m`` on the model's cost
+ entry — e.g. Kimi K3's $3/$15); slugs without them fall back to
+ :data:`_DEFAULT_MOONSHOT_RATE_USD_PER_MTOK`.
+ """
+ return {
+ m.slug: (m.cost.provider_input_usd_per_1m, m.cost.provider_output_usd_per_1m)
+ for m in get_catalog().models
+ if m.slug.startswith(_MOONSHOT_PREFIX)
+ and m.cost is not None
+ and m.cost.provider_input_usd_per_1m is not None
+ and m.cost.provider_output_usd_per_1m is not None
+ }
+
+
+# Import-time snapshot is safe: the catalog is immutable within a process
+# (the file only changes at deploy), same lifecycle as every projection.
+_RATE_OVERRIDES_USD_PER_MTOK: dict[str, tuple[float, float]] = _overrides_from_catalog()
+
+
def is_moonshot_model(model: str | None) -> bool:
"""True when *model* is a Moonshot OpenRouter slug.
- Prefix match against ``moonshotai/`` covers every Kimi SKU Moonshot
- ships today (``kimi-k2``, ``kimi-k2.5``, ``kimi-k2.6``,
- ``kimi-k2-thinking``) plus any future SKU Moonshot publishes under
- the same namespace. Used by both pricing and cache-control gating.
+ Prefix match against ``moonshotai/`` covers every Kimi SKU under that
+ namespace — present and future — with no per-SKU roster to maintain.
+ Used by both pricing and cache-control gating.
"""
return isinstance(model, str) and model.startswith(_MOONSHOT_PREFIX)
diff --git a/autogpt_platform/backend/backend/copilot/moonshot_test.py b/autogpt_platform/backend/backend/copilot/moonshot_test.py
index 7fcea124bf..0ece3417eb 100644
--- a/autogpt_platform/backend/backend/copilot/moonshot_test.py
+++ b/autogpt_platform/backend/backend/copilot/moonshot_test.py
@@ -22,7 +22,8 @@ class TestIsMoonshotModel:
"moonshotai/kimi-k2-thinking",
"moonshotai/kimi-k2.5",
"moonshotai/kimi-k2",
- "moonshotai/kimi-k3.0", # Future SKU must match transparently.
+ "moonshotai/kimi-k3",
+ "moonshotai/kimi-k4", # Future SKU must match transparently.
],
)
def test_moonshot_slugs_match(self, model: str) -> None:
@@ -50,15 +51,20 @@ class TestIsMoonshotModel:
class TestRateCardUsd:
- """Rate card defaults to the shared Moonshot price for every SKU."""
+ """Shared K2.x default plus per-slug overrides for split-priced SKUs."""
def test_moonshot_default_rate(self) -> None:
assert rate_card_usd("moonshotai/kimi-k2.6") == (0.60, 2.80)
+ def test_kimi_k3_premium_override(self) -> None:
+ # K3 is split-priced ($3/$15 per Mtok) — the override must win
+ # over the shared K2.x default.
+ assert rate_card_usd("moonshotai/kimi-k3") == (3.00, 15.00)
+
def test_future_moonshot_sku_inherits_default(self) -> None:
# Verifies the prefix-based fallback — new SKUs don't need a code
# edit to get a reasonable rate card.
- assert rate_card_usd("moonshotai/kimi-k3.0") == (0.60, 2.80)
+ assert rate_card_usd("moonshotai/kimi-k4") == (0.60, 2.80)
def test_non_moonshot_returns_none(self) -> None:
assert rate_card_usd("anthropic/claude-sonnet-4.6") is None
diff --git a/autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.py b/autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.py
index 93082fc7e2..87f5e3ee30 100644
--- a/autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.py
+++ b/autogpt_platform/backend/backend/copilot/sdk/retry_scenarios_test.py
@@ -1039,6 +1039,11 @@ def _make_sdk_patches(
claude_agent_max_thinking_tokens=0,
claude_agent_thinking_effort=None,
claude_agent_fallback_model=None,
+ # Real strings: the stamp path canonicalizes the model for
+ # fallback detection; auto-MagicMock attributes would leak
+ # into regex-based comparison.
+ claude_agent_model="claude-sonnet-4-6",
+ thinking_standard_model="anthropic/claude-sonnet-4-6",
),
),
(f"{_SVC}.get_user_tier", dict(new_callable=AsyncMock, return_value=None)),
diff --git a/autogpt_platform/backend/backend/copilot/sdk/service.py b/autogpt_platform/backend/backend/copilot/sdk/service.py
index 44c1ce6944..ff2ca9fcad 100644
--- a/autogpt_platform/backend/backend/copilot/sdk/service.py
+++ b/autogpt_platform/backend/backend/copilot/sdk/service.py
@@ -47,7 +47,7 @@ from backend.executor.cluster_lock import AsyncClusterLock
from backend.util.exceptions import NotFoundError
from backend.util.settings import Settings
-from ..config import ChatConfig, CopilotLlmModel, CopilotMode
+from ..config import ChatConfig, CopilotLLMModel, CopilotMode
from ..constants import (
COPILOT_ERROR_PREFIX,
COPILOT_RETRYABLE_ERROR_PREFIX,
@@ -61,7 +61,9 @@ from ..session_cleanup import prune_orphan_tool_calls
from ..context import encode_cwd_for_cli, get_workspace_manager
from ..graphiti.config import is_enabled_for_user
from ..model_normalize import normalize_model_for_transport
-from ..model_router import resolve_model
+from backend.data.llm_registry.llm_models import MODEL_DATE_SUFFIX_RE
+
+from ..model_router import ResolvedModel, RoutingSource, resolve_model_route
from ..moonshot import (
is_moonshot_model as _is_moonshot_model,
override_cost_usd as _override_cost_for_moonshot,
@@ -1947,15 +1949,17 @@ def _resolve_sdk_model() -> str | None:
async def _resolve_thinking_model_for_user(
- tier: "CopilotLlmModel",
+ tier: "CopilotLLMModel",
user_id: str | None,
-) -> str:
+) -> ResolvedModel:
"""LD-aware thinking-tier model pick for a specific user.
Consults ``copilot-model-routing[thinking][{tier}]`` and falls back
- to the ``ChatConfig`` default on missing user / missing flag.
+ to the ``ChatConfig`` default on missing user / missing flag. Returns
+ the model together with which routing layer picked it, so persisted
+ assistant messages can be stamped for product-intelligence.
"""
- return await resolve_model("thinking", tier, user_id, config=config)
+ return await resolve_model_route("thinking", tier, user_id, config=config)
def _resolve_fallback_model() -> str | None:
@@ -1981,11 +1985,11 @@ def _resolve_env_model(sdk_model: str | None, fallback_model: str | None) -> str
async def _resolve_sdk_model_for_request(
- model: "CopilotLlmModel | None",
+ model: "CopilotLLMModel | None",
session_id: str,
user_id: str | None = None,
-) -> str | None:
- """Resolve the SDK model string for a turn.
+) -> tuple[str | None, RoutingSource]:
+ """Resolve the SDK model string for a turn, plus which layer picked it.
Priority (highest first):
1. ``config.claude_agent_model`` — unconditional override, bypasses LD.
@@ -2001,9 +2005,9 @@ async def _resolve_sdk_model_for_request(
4. ``ChatConfig`` static default for the tier.
"""
if config.claude_agent_model:
- return config.claude_agent_model
+ return config.claude_agent_model, "env"
- tier_name: "CopilotLlmModel" = "advanced" if model == "advanced" else "standard"
+ tier_name: "CopilotLLMModel" = "advanced" if model == "advanced" else "standard"
# Strip at read time so a stray trailing space in ``CHAT_*_MODEL`` (a
# common ``.env`` pitfall) doesn't make the ``resolved == tier_default``
# comparison below spuriously diverge — ``resolve_model`` already strips
@@ -2025,7 +2029,7 @@ async def _resolve_sdk_model_for_request(
# must be honoured. Without this, a subscription-mode deployment
# silently ignores the ``copilot-model-routing[thinking][standard]``
# flag entirely, which defeats the point of cohort-based routing.
- ld_overrides_default = resolved != tier_default
+ ld_overrides_default = resolved.model != tier_default
if (
not ld_overrides_default
and tier_name == "standard"
@@ -2035,9 +2039,9 @@ async def _resolve_sdk_model_for_request(
"[SDK] [%s] Subscription default (tier=standard, LD unset)",
session_id[:12] if session_id else "?",
)
- return None
+ return None, "env"
try:
- sdk_model = _normalize_model_name(resolved)
+ sdk_model = _normalize_model_name(resolved.model)
except ValueError as exc:
# The per-user LD value didn't pass ``_normalize_model_name``'s
# vendor check (most commonly: a ``moonshotai/kimi-*`` slug on a
@@ -2059,19 +2063,19 @@ async def _resolve_sdk_model_for_request(
"[SDK] [%s] LD model %r rejected for tier=%s (%s); falling "
"back to tier default %s",
session_id[:12] if session_id else "?",
- resolved,
+ resolved.model,
tier_name,
exc,
sdk_model,
)
- return sdk_model
+ return sdk_model, "env"
logger.info(
"[SDK] [%s] Resolved model for tier=%s: %s",
session_id[:12] if session_id else "?",
tier_name,
sdk_model,
)
- return sdk_model
+ return sdk_model, resolved.source
_MAX_TRANSIENT_BACKOFF_SECONDS = 30
@@ -3918,7 +3922,7 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues
file_ids: list[str] | None = None,
permissions: "CopilotPermissions | None" = None,
mode: CopilotMode | None = None,
- model: CopilotLlmModel | None = None,
+ model: CopilotLLMModel | None = None,
request_arrival_at: float = 0.0,
organization_id: str | None = None,
team_id: str | None = None,
@@ -3954,6 +3958,11 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues
# Type narrowing: session is guaranteed ChatSession after the check above
session = cast(ChatSession, session)
+ # Stamping state for THIS turn: messages beyond this index were created
+ # by the current turn and get model/routing_source at persist time.
+ # Pre-feature history rows (NULL model) must never be back-stamped.
+ routing_source: RoutingSource = "env"
+
# The session row is the tenancy anchor; the turn entry's org/team only
# backfills sessions created before org tagging (pre-migration rows).
if session.organization_id is None and organization_id:
@@ -3982,6 +3991,13 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues
# Stop mid-tool-call so the next turn's --resume transcript is well-formed.
prune_orphan_tool_calls(session.messages, log_prefix=f"[SDK] [{session_id[:12]}]")
+ # Capture the turn-start boundary AFTER the error-marker and orphan-tool
+ # cleanup above: those pops shrink session.messages, so a count taken
+ # before them would overshoot and leave this turn's first assistant
+ # row(s) below _stamp_turn_messages' start_index — unstamped — exactly
+ # on the error-recovery turns whose routing we most want recorded.
+ pre_turn_message_count = len(session.messages)
+
# Strip any user-injected <user_context> tags on every turn.
# Only the server-injected prefix on the first message is trusted.
if message:
@@ -4275,7 +4291,9 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues
# Resolve model (request tier → LD per-user override → config default).
# Done BEFORE build_sdk_env so model-aware env vars (e.g. the
# Moonshot autocompact gate) can branch on the resolved slug.
- sdk_model = await _resolve_sdk_model_for_request(model, session_id, user_id)
+ sdk_model, routing_source = await _resolve_sdk_model_for_request(
+ model, session_id, user_id
+ )
fallback_model = _resolve_fallback_model()
# sdk_cwd routes the CLI's temp dir into the per-session workspace
@@ -5322,6 +5340,16 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues
# Without this, messages disappear after refresh because they were never
# saved to the database.
if session is not None:
+ # sdk_model (not effective_model): in subscription mode no
+ # model was requested — the CLI's default must not read as a
+ # fallback divergence against a fabricated "request".
+ _stamp_turn_messages(
+ session.messages,
+ start_index=pre_turn_message_count,
+ requested_model=sdk_model,
+ actual_model=state.observed_model if state is not None else None,
+ routing_source=routing_source,
+ )
try:
await asyncio.shield(upsert_chat_session(session))
logger.info(
@@ -5584,3 +5612,65 @@ async def _fetch_graphiti_context(
ctx = await fetch_warm_context(user_id, message or "") or ""
return True, ctx
+
+
+def _canonical_model(model: str) -> str:
+ """Spelling-insensitive model identity for fallback detection:
+ vendor prefix stripped, dots→dashes, trailing -YYYYMMDD dropped."""
+ tail = model.lower().split("/")[-1]
+ if tail.startswith("anthropic."):
+ tail = tail[len("anthropic.") :]
+ return MODEL_DATE_SUFFIX_RE.sub("", tail.replace(".", "-"))
+
+
+def _same_model(a: str, b: str | None) -> bool:
+ return b is not None and _canonical_model(a) == _canonical_model(b)
+
+
+def _stamp_turn_messages(
+ messages: list[ChatMessage],
+ *,
+ start_index: int,
+ requested_model: str | None,
+ actual_model: str | None,
+ routing_source: RoutingSource,
+) -> None:
+ """Stamp THIS turn's assistant messages with the serving model and
+ routing layer.
+
+ The SDK adapters build messages far from the resolution context, so
+ the stamp lands at persist time — bounded to ``start_index`` (the
+ session length when the turn began) so pre-feature history rows with
+ NULL model are never back-stamped with today's values, and never
+ overwriting an already-stamped row.
+
+ ``actual_model`` is the model observed on ``AssistantMessage.model``
+ (survives retries). When it names a DIFFERENT MODEL than the requested
+ resolution the CLI's overload fallback served the turn — the stamp
+ records the actual model with ``routing_source="fallback"`` so
+ analytics never attribute a fallback-served turn to the layer that
+ routed a different model. The comparison is canonicalized (vendor
+ prefix, dot/dash, date-suffix spellings) so a spelling difference
+ between the request and the CLI's report never fakes a fallback.
+
+ ``requested_model=None`` means no specific model was requested
+ (subscription mode lets the CLI pick its default) — there is nothing
+ to diverge from, so the observed model stamps with the original
+ routing source, never "fallback".
+ """
+ model = actual_model or requested_model
+ if (
+ actual_model is not None
+ and requested_model is not None
+ and not _same_model(actual_model, requested_model)
+ ):
+ routing_source = "fallback"
+ for msg in messages[start_index:]:
+ if msg.role == "assistant" and msg.model is None:
+ msg.model = model
+ msg.routing_source = routing_source
+ if msg.sequence is not None:
+ # Row already flushed to the DB mid-turn — flag it so the
+ # save path back-fills the columns (insert only covers
+ # unsequenced rows).
+ msg.stamps_pending_save = True
diff --git a/autogpt_platform/backend/backend/copilot/sdk/service_helpers_test.py b/autogpt_platform/backend/backend/copilot/sdk/service_helpers_test.py
index 57c2825b6d..d87d32315c 100644
--- a/autogpt_platform/backend/backend/copilot/sdk/service_helpers_test.py
+++ b/autogpt_platform/backend/backend/copilot/sdk/service_helpers_test.py
@@ -16,6 +16,7 @@ from claude_agent_sdk import AssistantMessage, TextBlock, ToolUseBlock
from backend.copilot import config as cfg_mod
from backend.copilot.config import ChatConfig
+from ..model_router import ResolvedModel
from .conftest import build_test_transcript as _build_transcript
from .service import (
_RETRY_TARGET_TOKENS,
@@ -589,13 +590,16 @@ class TestResolveSdkModelForRequestTransportAware:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="anthropic/claude-opus-4.7"),
+ new=AsyncMock(
+ return_value=ResolvedModel("anthropic/claude-opus-4.7", "ld")
+ ),
):
resolved = await _resolve_sdk_model_for_request(
model="advanced", session_id="sess-adv", user_id="user-1"
)
- # NOT the OpenRouter slug, NOT None — the CLI-friendly hyphenated form.
- assert resolved == "claude-opus-4-7"
+ # NOT the OpenRouter slug, NOT None — the CLI-friendly hyphenated
+ # form; and the LD routing source rides along for message stamping.
+ assert resolved == ("claude-opus-4-7", "ld")
@pytest.mark.asyncio
async def test_subscription_standard_no_override_returns_none(
@@ -616,12 +620,43 @@ class TestResolveSdkModelForRequestTransportAware:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="anthropic/claude-sonnet-4-6"),
+ new=AsyncMock(
+ return_value=ResolvedModel("anthropic/claude-sonnet-4-6", "env")
+ ),
):
resolved = await _resolve_sdk_model_for_request(
model="standard", session_id="sess-std", user_id="user-1"
)
- assert resolved is None
+ assert resolved == (None, "env")
+
+ @pytest.mark.asyncio
+ async def test_env_default_stamps_env_source(
+ self, monkeypatch: pytest.MonkeyPatch, _clean_config_env: None
+ ):
+ """No LD opinion, no subscription: the config default serves and the
+ routing source says so — this is the value persisted onto SDK-path
+ assistant messages."""
+ cfg = cfg_mod.ChatConfig(
+ thinking_standard_model="anthropic/claude-sonnet-4-6",
+ claude_agent_model=None,
+ use_openrouter=True,
+ api_key="or-key",
+ base_url="https://openrouter.ai/api/v1",
+ use_claude_code_subscription=False,
+ )
+ monkeypatch.setattr("backend.copilot.sdk.service.config", cfg)
+
+ with patch(
+ "backend.copilot.sdk.service._resolve_thinking_model_for_user",
+ new=AsyncMock(
+ return_value=ResolvedModel("anthropic/claude-sonnet-4-6", "env")
+ ),
+ ):
+ model, source = await _resolve_sdk_model_for_request(
+ model="standard", session_id="sess-env", user_id="user-1"
+ )
+ assert source == "env"
+ assert model is not None
# ---------------------------------------------------------------------------
@@ -1450,3 +1485,200 @@ class TestEmptyToolCallNoArgException:
# not the saturation failure).
assert result.count == 0
assert result.tripped is False
+
+
+# ---------------------------------------------------------------------------
+# _stamp_turn_messages — turn-bounded model/routing_source stamping
+# ---------------------------------------------------------------------------
+
+
+class TestStampTurnMessages:
+ """The SDK path stamps at persist time; the stamp must be bounded to
+ the turn (never back-stamping pre-feature NULL history) and must not
+ overwrite already-stamped rows."""
+
+ def _messages(self):
+ from backend.copilot.model import ChatMessage
+
+ return [
+ ChatMessage(role="assistant", content="old", sequence=0),
+ ChatMessage(role="user", content="q", sequence=1),
+ ChatMessage(role="assistant", content="new", sequence=2),
+ ]
+
+ def test_stamps_only_this_turns_assistant_rows(self):
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = self._messages()
+ _stamp_turn_messages(
+ msgs,
+ start_index=1,
+ requested_model="claude-sonnet-4-6",
+ actual_model=None,
+ routing_source="env",
+ )
+ assert msgs[0].model is None # pre-turn history untouched
+ assert msgs[1].model is None # user rows untouched
+ assert msgs[2].model == "claude-sonnet-4-6"
+ assert msgs[2].routing_source == "env"
+
+ def test_never_overwrites_an_existing_stamp(self):
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = self._messages()
+ msgs[2].model = "already/stamped"
+ msgs[2].routing_source = "ld"
+ _stamp_turn_messages(
+ msgs,
+ start_index=0,
+ requested_model="claude-sonnet-4-6",
+ actual_model=None,
+ routing_source="env",
+ )
+ assert msgs[2].model == "already/stamped"
+ assert msgs[2].routing_source == "ld"
+
+ def test_fallback_divergence_restamps_source(self):
+ """When the CLI's overload fallback served a different model than
+ the routed one, the stamp records the ACTUAL model and marks the
+ source as "fallback" — never attributing the turn to a layer that
+ routed a different model."""
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = self._messages()
+ _stamp_turn_messages(
+ msgs,
+ start_index=0,
+ requested_model="moonshotai/kimi-k2.6",
+ actual_model="claude-sonnet-4-6",
+ routing_source="ld",
+ )
+ assert msgs[0].model == "claude-sonnet-4-6"
+ assert msgs[0].routing_source == "fallback"
+
+ def test_matching_observed_model_keeps_source(self):
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = self._messages()
+ _stamp_turn_messages(
+ msgs,
+ start_index=0,
+ requested_model="claude-sonnet-4-6",
+ actual_model="claude-sonnet-4-6",
+ routing_source="ld",
+ )
+ assert msgs[0].routing_source == "ld"
+
+ def test_subscription_default_never_marks_fallback(self):
+ """Subscription mode requests NO model (requested_model=None) — the
+ CLI's default choice is the resolution itself, not a divergence."""
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = self._messages()
+ _stamp_turn_messages(
+ msgs,
+ start_index=0,
+ requested_model=None,
+ actual_model="claude-opus-4-6",
+ routing_source="env",
+ )
+ assert msgs[0].model == "claude-opus-4-6"
+ assert msgs[0].routing_source == "env" # NOT "fallback"
+
+ def test_flushed_rows_flagged_for_stamp_backfill(self):
+ """A row the mid-turn flush already persisted (has a sequence) gets
+ flagged so the save path back-fills its columns — the insert path
+ only covers unsequenced rows."""
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = self._messages()
+ msgs[2].sequence = 7 # flushed mid-turn
+ _stamp_turn_messages(
+ msgs,
+ start_index=1,
+ requested_model="claude-sonnet-4-6",
+ actual_model=None,
+ routing_source="env",
+ )
+ assert msgs[2].model == "claude-sonnet-4-6"
+ assert msgs[2].stamps_pending_save is True
+ assert msgs[1].stamps_pending_save is False # user row untouched
+
+ def test_turn_boundary_captured_after_message_cleanup(self):
+ """The stamp's ``start_index`` (``pre_turn_message_count``) MUST be
+ taken after the turn-start cleanup that pops trailing error markers
+ and prunes orphan tool rows. Captured before, it overshoots by the
+ number of popped rows and leaves this turn's first assistant row(s)
+ below ``start_index`` — unstamped — precisely on error-recovery
+ turns. Guard the ordering at the source level: the enormous turn
+ handler can't be driven in a unit test, so pin the invariant that
+ the count assignment follows the last cleanup call."""
+ import inspect
+
+ from backend.copilot.sdk.service import stream_chat_completion_sdk
+
+ src = inspect.getsource(stream_chat_completion_sdk)
+ assign_at = src.index("pre_turn_message_count = len(session.messages)")
+ prune_at = src.index("prune_orphan_tool_calls(session.messages")
+ marker_pop_at = src.index("Removing stale error marker")
+ assert (
+ assign_at > prune_at
+ ), "pre_turn_message_count must be captured AFTER prune_orphan_tool_calls"
+ assert (
+ assign_at > marker_pop_at
+ ), "pre_turn_message_count must be captured AFTER error-marker cleanup"
+
+
+class TestChatMessageStampRoundTrip:
+ """model survives client-facing serialization; routing_source is
+ deliberately EXCLUDED from payloads (clients could infer LD cohort
+ membership from "ld" vs "env") while persisting via the explicit
+ field-mapped DB path."""
+
+ def test_model_serializes_routing_source_excluded(self):
+ from backend.copilot.model import ChatMessage
+
+ msg = ChatMessage(
+ role="assistant",
+ content="x",
+ sequence=3,
+ model="claude-sonnet-4-6",
+ routing_source="catalog",
+ )
+ dumped = msg.model_dump()
+ assert dumped["model"] == "claude-sonnet-4-6"
+ assert "routing_source" not in dumped
+ # The DB path reads the attribute, not the dump.
+ assert msg.routing_source == "catalog"
+
+
+class TestSameModelCanonicalization:
+ """Spelling differences between the requested resolution and the CLI's
+ report must never fake a fallback; real model changes must."""
+
+ def test_spelling_variants_are_same_model(self):
+ from backend.copilot.sdk.service import _same_model
+
+ assert _same_model("anthropic/claude-opus-4.7", "claude-opus-4-7")
+ assert _same_model("claude-haiku-4-5-20251001", "anthropic/claude-haiku-4-5")
+ assert _same_model("anthropic.claude-sonnet-4-6", "claude-sonnet-4.6")
+
+ def test_different_models_diverge(self):
+ from backend.copilot.sdk.service import _same_model
+
+ assert not _same_model("claude-sonnet-4-6", "moonshotai/kimi-k2.6")
+ assert not _same_model("claude-opus-4-7", "claude-sonnet-4-6")
+
+ def test_fallback_not_faked_by_spelling(self):
+ from backend.copilot.model import ChatMessage
+ from backend.copilot.sdk.service import _stamp_turn_messages
+
+ msgs = [ChatMessage(role="assistant", content="x", sequence=None)]
+ _stamp_turn_messages(
+ msgs,
+ start_index=0,
+ requested_model="claude-opus-4-7",
+ actual_model="anthropic/claude-opus-4.7",
+ routing_source="ld",
+ )
+ assert msgs[0].routing_source == "ld" # same model, not "fallback"
diff --git a/autogpt_platform/backend/backend/copilot/sdk/service_test.py b/autogpt_platform/backend/backend/copilot/sdk/service_test.py
index 1fe118b4a0..43bb6a48c2 100644
--- a/autogpt_platform/backend/backend/copilot/sdk/service_test.py
+++ b/autogpt_platform/backend/backend/copilot/sdk/service_test.py
@@ -11,6 +11,7 @@ import pytest
from backend.copilot import config as cfg_mod
from backend.copilot.builder_context import BUILDER_BLOCKED_TOOLS
+from backend.copilot.model_router import ResolvedModel
from backend.copilot.permissions import CopilotPermissions, all_known_tool_names
from backend.data.sharing.workspace_refs import extract_workspace_file_ids
@@ -572,15 +573,16 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="moonshotai/kimi-k2.6"),
+ new=AsyncMock(return_value=ResolvedModel("moonshotai/kimi-k2.6", "ld")),
):
resolved = await _resolve_sdk_model_for_request(
model="standard", session_id="sess-abc", user_id="user-1"
)
# Fallback == tier-specific config default (thinking_standard_model
- # normalised to hyphen-form for direct-Anthropic mode).
- assert resolved == "claude-sonnet-4-6"
+ # normalised to hyphen-form for direct-Anthropic mode) — an env
+ # fallback, so the routing source must NOT claim "ld".
+ assert resolved == ("claude-sonnet-4-6", "env")
@pytest.mark.asyncio
async def test_openrouter_mode_accepts_ld_kimi_value(
@@ -600,12 +602,12 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="moonshotai/kimi-k2.6"),
+ new=AsyncMock(return_value=ResolvedModel("moonshotai/kimi-k2.6", "ld")),
):
resolved = await _resolve_sdk_model_for_request(
model="standard", session_id="sess-abc", user_id="user-1"
)
- assert resolved == "moonshotai/kimi-k2.6"
+ assert resolved == ("moonshotai/kimi-k2.6", "ld")
@pytest.mark.asyncio
async def test_advanced_tier_fallback_uses_advanced_default_not_standard(
@@ -630,14 +632,14 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="moonshotai/kimi-k2.6"),
+ new=AsyncMock(return_value=ResolvedModel("moonshotai/kimi-k2.6", "ld")),
):
resolved = await _resolve_sdk_model_for_request(
model="advanced", session_id="sess-adv", user_id="user-1"
)
# Direct-Anthropic normalises anthropic/claude-opus-4.7 → claude-opus-4-7
- assert resolved == "claude-opus-4-7"
+ assert resolved == ("claude-opus-4-7", "env")
@pytest.mark.asyncio
async def test_standard_ld_override_wins_over_subscription(
@@ -666,14 +668,14 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="moonshotai/kimi-k2.6"),
+ new=AsyncMock(return_value=ResolvedModel("moonshotai/kimi-k2.6", "ld")),
):
resolved = await _resolve_sdk_model_for_request(
model="standard", session_id="sess-std-sub", user_id="user-1"
)
# Kimi can't be served by the subscription CLI; fail-soft to
- # the tier default normalised for the active transport.
- assert resolved == "claude-sonnet-4-6"
+ # the tier default normalised for the active transport (env source).
+ assert resolved == ("claude-sonnet-4-6", "env")
@pytest.mark.asyncio
async def test_standard_subscription_survives_trailing_whitespace_in_env(
@@ -698,12 +700,14 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="anthropic/claude-sonnet-4-6"),
+ new=AsyncMock(
+ return_value=ResolvedModel("anthropic/claude-sonnet-4-6", "ld")
+ ),
):
resolved = await _resolve_sdk_model_for_request(
model="standard", session_id="sess-ws", user_id="user-1"
)
- assert resolved is None, (
+ assert resolved == (None, "env"), (
"LD value semantically matches the whitespace-padded config "
"default — subscription mode must still win and return None"
)
@@ -730,12 +734,14 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="anthropic/claude-sonnet-4-6"),
+ new=AsyncMock(
+ return_value=ResolvedModel("anthropic/claude-sonnet-4-6", "ld")
+ ),
):
resolved = await _resolve_sdk_model_for_request(
model="standard", session_id="sess-std-nop", user_id="user-1"
)
- assert resolved is None
+ assert resolved == (None, "env")
@pytest.mark.asyncio
async def test_advanced_tier_consults_ld_under_subscription(
@@ -761,12 +767,14 @@ class TestResolveSdkModelForRequestLdFallback:
with patch(
"backend.copilot.sdk.service._resolve_thinking_model_for_user",
- new=AsyncMock(return_value="anthropic/claude-opus-4.7"),
+ new=AsyncMock(
+ return_value=ResolvedModel("anthropic/claude-opus-4.7", "ld")
+ ),
):
resolved = await _resolve_sdk_model_for_request(
model="advanced", session_id="sess-adv-sub", user_id="user-1"
)
- assert resolved == "claude-opus-4-7"
+ assert resolved == ("claude-opus-4-7", "ld")
# ---------------------------------------------------------------------------
diff --git a/autogpt_platform/backend/backend/copilot/service.py b/autogpt_platform/backend/backend/copilot/service.py
index ee5f6cee82..2e0d485e1d 100644
--- a/autogpt_platform/backend/backend/copilot/service.py
+++ b/autogpt_platform/backend/backend/copilot/service.py
@@ -29,7 +29,7 @@ from backend.util.llm.providers import call_provider_openai_compat_sync
from backend.util.settings import AppEnvironment, Settings
from .anthropic_rate_card import compute_anthropic_cost_usd
-from .config import ChatConfig, CopilotLlmModel
+from .config import ChatConfig, CopilotLLMModel
from .expert_context import build_expert_context
from .model import (
ChatMessage,
@@ -51,7 +51,7 @@ _TITLE_ELLIPSIS = "..."
_TITLE_TRUNCATED_MAX_CHARS = _TITLE_MAX_CHARS - len(_TITLE_ELLIPSIS)
-def resolve_chat_model(tier: CopilotLlmModel | None) -> str:
+def resolve_chat_model(tier: CopilotLLMModel | None) -> str:
"""Return the configured SDK model for the given tier.
The SDK (extended-thinking) path is Anthropic-only — the Claude Agent
diff --git a/autogpt_platform/backend/backend/copilot/tools/test_dry_run.py b/autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
index 61327a5ecd..25c5007b9a 100644
--- a/autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
+++ b/autogpt_platform/backend/backend/copilot/tools/test_dry_run.py
@@ -5,7 +5,7 @@ from unittest.mock import AsyncMock, MagicMock, patch
import pytest
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import LLMModel
from backend.copilot.tools.helpers import execute_block
from backend.copilot.tools.models import BlockOutputResponse, ErrorResponse
from backend.copilot.tools.run_block import RunBlockTool
@@ -586,9 +586,9 @@ def test_prepare_dry_run_orchestrator_block():
assert result is not None
# Model is overridden to the simulation model (not the user's model).
assert result["model"] != "gpt-4o"
- # Simulation model must parse as a real LlmModel so OrchestratorBlock's
+ # Simulation model must parse as a real LLMModel so OrchestratorBlock's
# Pydantic input validation accepts it.
- assert LlmModel(result["model"]) is not None
+ assert LLMModel(result["model"]) is not None
# Capped to min(original, 10); user's 10 passes through unchanged.
assert result["agent_mode_max_iterations"] == 10
assert result["_dry_run_api_key"] == "sk-or-test-key"
diff --git a/autogpt_platform/backend/backend/data/block_cost_config.py b/autogpt_platform/backend/backend/data/block_cost_config.py
index 122ff1fd6e..d7411682cd 100644
--- a/autogpt_platform/backend/backend/data/block_cost_config.py
+++ b/autogpt_platform/backend/backend/data/block_cost_config.py
@@ -5,10 +5,12 @@ from pydantic import BaseModel
from backend.blocks._base import Block, BlockCost, BlockCostType, TokenRateDisplay
from backend.data.block import BlockInput
+from backend.data.llm_registry.catalog import get_catalog
from backend.data.model import APIKeyCredentials
if TYPE_CHECKING:
from backend.data.model import NodeExecutionStats
+
from backend.blocks.ai_condition import AIConditionBlock
from backend.blocks.ai_image_customizer import AIImageCustomizerBlock, GeminiImageModel
from backend.blocks.ai_image_generator_block import AIImageGeneratorBlock, ImageGenModel
@@ -43,7 +45,7 @@ from backend.blocks.llm import (
AIStructuredResponseGeneratorBlock,
AITextGeneratorBlock,
AITextSummarizerBlock,
- LlmModel,
+ LLMModel,
)
from backend.blocks.mem0 import (
AddMemoryBlock,
@@ -101,119 +103,32 @@ from backend.integrations.credentials_store import (
# =============== Configure the cost for each LLM Model call =============== #
-MODEL_COST: dict[LlmModel, int] = {
- LlmModel.O4_MINI: 2,
- LlmModel.O3: 4,
- LlmModel.O3_MINI: 2,
- LlmModel.O3_PRO: 30,
- LlmModel.O1: 16,
- LlmModel.O1_MINI: 2,
- # GPT-5.6 models
- LlmModel.GPT5_6_SOL: 10,
- LlmModel.GPT5_6_TERRA: 6,
- LlmModel.GPT5_6_LUNA: 2,
- # GPT-5.5 models
- LlmModel.GPT5_5: 10,
- LlmModel.GPT5_5_PRO: 60,
- # GPT-5 models
- LlmModel.GPT5_4: 6,
- LlmModel.GPT5_4_MINI: 2,
- LlmModel.GPT5_4_NANO: 1,
- LlmModel.GPT5_4_PRO: 60,
- LlmModel.GPT5_3: 5,
- LlmModel.GPT5_3_CODEX: 5,
- LlmModel.GPT5_2: 6,
- LlmModel.GPT5_2_PRO: 60,
- LlmModel.GPT5_1: 5,
- LlmModel.GPT5_1_CODEX: 5,
- LlmModel.GPT5: 2,
- LlmModel.GPT5_MINI: 1,
- LlmModel.GPT5_NANO: 1,
- LlmModel.GPT5_PRO: 60,
- LlmModel.GPT5_CHAT: 5,
- LlmModel.GPT41: 2,
- LlmModel.GPT41_MINI: 1,
- LlmModel.GPT41_NANO: 1,
- LlmModel.GPT4O_MINI: 1,
- LlmModel.GPT4O: 3,
- LlmModel.CLAUDE_4_6_OPUS: 14,
- LlmModel.CLAUDE_4_7_OPUS: 14,
- LlmModel.CLAUDE_4_6_SONNET: 9,
- LlmModel.CLAUDE_4_5_HAIKU: 4,
- LlmModel.CLAUDE_4_5_OPUS: 14,
- LlmModel.CLAUDE_4_5_SONNET: 9,
- LlmModel.AIML_API_LLAMA3_3_70B: 1,
- LlmModel.LLAMA3_3_70B: 1,
- LlmModel.LLAMA3_1_8B: 1,
- LlmModel.OLLAMA_LLAMA3_3: 1,
- LlmModel.OLLAMA_LLAMA3_2: 1,
- LlmModel.OLLAMA_LLAMA3_8B: 1,
- LlmModel.OLLAMA_LLAMA3_405B: 1,
- LlmModel.OLLAMA_DOLPHIN: 1,
- LlmModel.OPENAI_GPT_OSS_120B: 1,
- LlmModel.OPENAI_GPT_OSS_20B: 1,
- LlmModel.GEMINI_2_5_PRO: 4,
- LlmModel.GEMINI_3_1_PRO_PREVIEW: 5,
- LlmModel.GEMINI_3_FLASH_PREVIEW: 2,
- LlmModel.GEMINI_2_5_FLASH: 1,
- LlmModel.GEMINI_2_0_FLASH: 1,
- LlmModel.GEMINI_3_1_FLASH_LITE_PREVIEW: 1,
- LlmModel.GEMINI_2_5_FLASH_LITE: 1,
- LlmModel.GEMINI_2_0_FLASH_LITE: 1,
- LlmModel.MISTRAL_LARGE_3: 2,
- LlmModel.MISTRAL_MEDIUM_3_1: 2,
- LlmModel.MISTRAL_SMALL_3_2: 1,
- LlmModel.CODESTRAL: 1,
- LlmModel.COHERE_COMMAND_A_03_2025: 3,
- LlmModel.COHERE_COMMAND_A_TRANSLATE_08_2025: 3,
- LlmModel.COHERE_COMMAND_A_REASONING_08_2025: 6,
- LlmModel.COHERE_COMMAND_A_VISION_07_2025: 3,
- LlmModel.DEEPSEEK_CHAT: 2,
- LlmModel.DEEPSEEK_R1_0528: 1,
- LlmModel.PERPLEXITY_SONAR: 1,
- LlmModel.PERPLEXITY_SONAR_PRO: 5,
- LlmModel.PERPLEXITY_SONAR_REASONING_PRO: 5,
- LlmModel.PERPLEXITY_SONAR_DEEP_RESEARCH: 10,
- LlmModel.NOUSRESEARCH_HERMES_3_LLAMA_3_1_405B: 1,
- LlmModel.NOUSRESEARCH_HERMES_3_LLAMA_3_1_70B: 1,
- LlmModel.AMAZON_NOVA_LITE_V1: 1,
- LlmModel.AMAZON_NOVA_MICRO_V1: 1,
- LlmModel.AMAZON_NOVA_PRO_V1: 1,
- LlmModel.MICROSOFT_PHI_4: 1,
- LlmModel.GRYPHE_MYTHOMAX_L2_13B: 1,
- LlmModel.META_LLAMA_4_SCOUT: 1,
- LlmModel.META_LLAMA_4_MAVERICK: 1,
- LlmModel.LLAMA_API_LLAMA_4_SCOUT: 1,
- LlmModel.LLAMA_API_LLAMA4_MAVERICK: 1,
- LlmModel.LLAMA_API_LLAMA3_3_8B: 1,
- LlmModel.LLAMA_API_LLAMA3_3_70B: 1,
- LlmModel.GROK_3: 3,
- LlmModel.GROK_4: 9,
- LlmModel.GROK_4_FAST: 1,
- LlmModel.GROK_4_1_FAST: 1,
- LlmModel.GROK_4_20: 5,
- LlmModel.GROK_4_20_MULTI_AGENT: 5,
- LlmModel.GROK_CODE_FAST_1: 1,
- LlmModel.KIMI_K2_5: 1,
- LlmModel.KIMI_K2_6: 2,
- LlmModel.KIMI_K2_THINKING: 2,
- LlmModel.QWEN3_235B_A22B_THINKING: 1,
- LlmModel.QWEN3_CODER: 9,
- # Z.ai (Zhipu) models
- LlmModel.ZAI_GLM_4_6: 1,
- LlmModel.ZAI_GLM_4_6V: 1,
- LlmModel.ZAI_GLM_4_7: 1,
- LlmModel.ZAI_GLM_4_7_FLASH: 1,
- LlmModel.ZAI_GLM_5: 2,
- LlmModel.ZAI_GLM_5_TURBO: 4,
- LlmModel.ZAI_GLM_5V_TURBO: 4,
- # v0 by Vercel models
- LlmModel.V0_1_5_MD: 1,
- LlmModel.V0_1_5_LG: 2,
- LlmModel.V0_1_0_MD: 1,
-}
-for model in LlmModel:
+def _enum_members_by_value() -> dict[str, LLMModel]:
+ return {m.value: m for m in LLMModel}
+
+
+def _model_cost_from_catalog() -> dict[LLMModel, int]:
+ """Flat credits-per-run tiers, read from the catalog — the single
+ source of truth for model pricing.
+
+ Deliberately does NOT filter on ``is_enabled``: the kill switch stops
+ a model from being SERVED to new work, but stored graphs that still
+ reference it keep executing — and an execution that happens must be
+ billed. Filtering here would make killed-model usage free."""
+ members = _enum_members_by_value()
+ costs: dict[LLMModel, int] = {}
+ for model in get_catalog().models:
+ member = members.get(model.slug)
+ if member is None or model.cost is None or model.cost.run_credits is None:
+ continue
+ costs[member] = model.cost.run_credits
+ return costs
+
+
+MODEL_COST: dict[LLMModel, int] = _model_cost_from_catalog()
+
+for model in LLMModel:
if model not in MODEL_COST:
raise ValueError(f"Missing MODEL_COST for model: {model}")
@@ -234,120 +149,31 @@ class TokenRate(BaseModel):
# TOKEN_COST populates gradually as we migrate LLM blocks to the TOKENS
-# cost type. Entries not yet listed fall back to the flat MODEL_COST tier
-# via the RUN-based LLM_COST list. Rates below are credits/1M tokens at the
-# current credit-to-USD conversion (1 credit ≈ $0.01), with a uniform 1.5x
-# margin over the published provider price (nearest-integer rounded).
-TOKEN_COST: dict[LlmModel, TokenRate] = {
- # Anthropic Opus current ($5/$25/$0.50/$6.25 per 1M).
- LlmModel.CLAUDE_4_6_OPUS: TokenRate(
- input=750, output=3750, cache_read=75, cache_creation=938
- ),
- LlmModel.CLAUDE_4_7_OPUS: TokenRate(
- input=750, output=3750, cache_read=75, cache_creation=938
- ),
- LlmModel.CLAUDE_4_5_OPUS: TokenRate(
- input=750, output=3750, cache_read=75, cache_creation=938
- ),
- # Anthropic Sonnet ($3/$15/$0.30/$3.75).
- LlmModel.CLAUDE_4_6_SONNET: TokenRate(
- input=450, output=2250, cache_read=45, cache_creation=563
- ),
- LlmModel.CLAUDE_4_5_SONNET: TokenRate(
- input=450, output=2250, cache_read=45, cache_creation=563
- ),
- # Anthropic Haiku 4.5 ($1/$5/$0.10/$1.25).
- LlmModel.CLAUDE_4_5_HAIKU: TokenRate(
- input=150, output=750, cache_read=15, cache_creation=188
- ),
- # OpenAI
- LlmModel.GPT5_6_SOL: TokenRate(input=750, output=4500),
- LlmModel.GPT5_6_TERRA: TokenRate(input=375, output=2250),
- LlmModel.GPT5_6_LUNA: TokenRate(input=150, output=900),
- LlmModel.GPT5_5: TokenRate(input=750, output=4500),
- LlmModel.GPT5_5_PRO: TokenRate(input=4500, output=27000),
- LlmModel.GPT5_4: TokenRate(input=375, output=2250),
- LlmModel.GPT5_4_MINI: TokenRate(input=113, output=675),
- LlmModel.GPT5_4_NANO: TokenRate(input=30, output=188),
- LlmModel.GPT5_4_PRO: TokenRate(input=4500, output=27000),
- LlmModel.GPT5_3: TokenRate(input=263, output=2100),
- LlmModel.GPT5_3_CODEX: TokenRate(input=263, output=2100),
- LlmModel.GPT5_2: TokenRate(input=263, output=2100),
- LlmModel.GPT5_2_PRO: TokenRate(input=3150, output=25200),
- LlmModel.GPT5_1: TokenRate(input=188, output=1500),
- LlmModel.GPT5_1_CODEX: TokenRate(input=188, output=1500),
- LlmModel.GPT5: TokenRate(input=188, output=1500),
- LlmModel.GPT5_MINI: TokenRate(input=38, output=300),
- LlmModel.GPT5_NANO: TokenRate(input=8, output=60),
- LlmModel.GPT5_PRO: TokenRate(input=2250, output=18000),
- LlmModel.GPT5_CHAT: TokenRate(input=188, output=1500),
- LlmModel.GPT4O: TokenRate(input=375, output=1500),
- LlmModel.GPT4O_MINI: TokenRate(input=23, output=90),
- LlmModel.GPT41: TokenRate(input=300, output=1200),
- LlmModel.GPT41_MINI: TokenRate(input=60, output=240),
- LlmModel.GPT41_NANO: TokenRate(input=15, output=60),
- LlmModel.O4_MINI: TokenRate(input=165, output=660),
- LlmModel.O3: TokenRate(input=300, output=1200),
- LlmModel.O3_MINI: TokenRate(input=165, output=660),
- LlmModel.O3_PRO: TokenRate(input=3000, output=12000),
- LlmModel.O1: TokenRate(input=2250, output=9000),
- LlmModel.O1_MINI: TokenRate(input=165, output=660),
- # Google Gemini (uses <=200k context tier pricing).
- LlmModel.GEMINI_2_5_PRO: TokenRate(input=188, output=1500),
- LlmModel.GEMINI_2_5_FLASH: TokenRate(input=45, output=375),
- LlmModel.GEMINI_2_0_FLASH: TokenRate(input=15, output=60),
- LlmModel.GEMINI_2_0_FLASH_LITE: TokenRate(input=11, output=45),
- LlmModel.GEMINI_2_5_FLASH_LITE: TokenRate(input=15, output=60),
- LlmModel.GEMINI_3_1_PRO_PREVIEW: TokenRate(input=300, output=1800),
- LlmModel.GEMINI_3_FLASH_PREVIEW: TokenRate(input=75, output=450),
- LlmModel.GEMINI_3_1_FLASH_LITE_PREVIEW: TokenRate(input=38, output=225),
- # xAI Grok. docs.x.ai currently lists only Grok 4.20 and grok-4-1-fast;
- # the rest (grok-3, grok-4-0709, grok-4-fast, grok-code-fast-1) were
- # removed from the public pricing page but remain callable via the
- # API. Rates below match their launch pricing (verified historically):
- # grok-3 / grok-4 $3/$15, grok-4-fast / grok-4-1-fast $0.20/$0.50,
- # grok-code-fast-1 $0.20/$1.50.
- LlmModel.GROK_3: TokenRate(input=450, output=2250),
- LlmModel.GROK_4: TokenRate(input=450, output=2250),
- LlmModel.GROK_4_FAST: TokenRate(input=30, output=75),
- LlmModel.GROK_4_1_FAST: TokenRate(input=30, output=75),
- LlmModel.GROK_4_20: TokenRate(input=300, output=900),
- LlmModel.GROK_CODE_FAST_1: TokenRate(input=30, output=225),
- # DeepSeek: both `deepseek-chat` and `deepseek-reasoner` now alias to
- # `deepseek-v4-flash` (non-thinking + thinking modes) at unified
- # $0.14/$0.28 per 1M (Sept 2025 price unification).
- LlmModel.DEEPSEEK_CHAT: TokenRate(input=21, output=42),
- LlmModel.DEEPSEEK_R1_0528: TokenRate(input=21, output=42),
- # Mistral — models route through OpenRouter (ModelMetadata provider =
- # "open_router"). TOKEN_COST here is the safety floor when OpenRouter
- # fails to return x-total-cost; rates below match OpenRouter's current
- # pass-through pricing (NOT Mistral's direct /v1/chat prices, which
- # we never call).
- LlmModel.MISTRAL_LARGE_3: TokenRate(input=300, output=900),
- LlmModel.MISTRAL_MEDIUM_3_1: TokenRate(input=60, output=300),
- LlmModel.MISTRAL_SMALL_3_2: TokenRate(input=15, output=45),
- LlmModel.CODESTRAL: TokenRate(input=45, output=135),
- # Cohere
- LlmModel.COHERE_COMMAND_A_03_2025: TokenRate(input=375, output=1500),
- # Moonshot Kimi
- # K2.5 / K2.6 aren't on Moonshot's direct pricing page today; OpenRouter
- # passes through $0.44/$2.00 for K2.5 and $0.7448/$4.655 for K2.6.
- LlmModel.KIMI_K2_5: TokenRate(input=66, output=300),
- LlmModel.KIMI_K2_6: TokenRate(input=112, output=698),
- LlmModel.KIMI_K2_THINKING: TokenRate(input=90, output=375),
- # Perplexity Sonar
- LlmModel.PERPLEXITY_SONAR: TokenRate(input=150, output=150),
- LlmModel.PERPLEXITY_SONAR_PRO: TokenRate(input=450, output=2250),
- LlmModel.PERPLEXITY_SONAR_REASONING_PRO: TokenRate(input=300, output=1200),
- LlmModel.PERPLEXITY_SONAR_DEEP_RESEARCH: TokenRate(input=300, output=1200),
- # Groq (LLama + OpenAI OSS). Maverick not listed on Groq; using Meta rate.
- LlmModel.LLAMA3_3_70B: TokenRate(input=89, output=119),
- LlmModel.LLAMA3_1_8B: TokenRate(input=8, output=12),
- LlmModel.META_LLAMA_4_SCOUT: TokenRate(input=17, output=51),
- LlmModel.META_LLAMA_4_MAVERICK: TokenRate(input=75, output=116),
- LlmModel.OPENAI_GPT_OSS_120B: TokenRate(input=23, output=90),
- LlmModel.OPENAI_GPT_OSS_20B: TokenRate(input=11, output=45),
-}
+# cost type: only catalog models carrying per-1M rates get an entry; the
+# rest fall back to the flat MODEL_COST tier via the RUN-based LLM_COST
+# list. Rates live in the catalog (credits/1M tokens at the current
+# credit-to-USD conversion, 1 credit ≈ $0.01, uniform 1.5x margin over
+# the published provider price).
+def _token_cost_from_catalog() -> dict[LLMModel, TokenRate]:
+ members = _enum_members_by_value()
+ rates: dict[LLMModel, TokenRate] = {}
+ for model in get_catalog().models:
+ member = members.get(model.slug)
+ cost = model.cost
+ if member is None or cost is None:
+ continue
+ if cost.input_credits_per_1m is None or cost.output_credits_per_1m is None:
+ continue
+ rates[member] = TokenRate(
+ input=cost.input_credits_per_1m,
+ output=cost.output_credits_per_1m,
+ cache_read=cost.cache_read_credits_per_1m or 0.0,
+ cache_creation=cost.cache_creation_credits_per_1m or 0.0,
+ )
+ return rates
+
+
+TOKEN_COST: dict[LLMModel, TokenRate] = _token_cost_from_catalog()
def compute_token_credits(
@@ -381,13 +207,13 @@ def compute_token_credits(
return max(0, math.ceil(total / 1_000_000))
-def _lookup_llm_model(raw: "str | LlmModel | None") -> "LlmModel | None":
+def _lookup_llm_model(raw: "str | LLMModel | None") -> "LLMModel | None":
if raw is None:
return None
- if isinstance(raw, LlmModel):
+ if isinstance(raw, LLMModel):
return raw
try:
- return LlmModel(raw)
+ return LLMModel(raw)
except ValueError:
return None
@@ -399,7 +225,7 @@ def _lookup_llm_model(raw: "str | LlmModel | None") -> "LlmModel | None":
_USD_PER_1M_DIVISOR = 150
-def _token_rate_display(model: LlmModel) -> TokenRateDisplay | None:
+def _token_rate_display(model: LLMModel) -> TokenRateDisplay | None:
"""Return the published per-1M-token USD `TokenRateDisplay` for `model`,
or None if the model has no TOKEN_COST entry. cache_* fields are set
only when the provider publishes a distinct cached-token rate (most
@@ -420,7 +246,7 @@ def _token_rate_display(model: LlmModel) -> TokenRateDisplay | None:
)
-def _tokens_llm_cost(model: LlmModel, credentials: APIKeyCredentials) -> BlockCost:
+def _tokens_llm_cost(model: LLMModel, credentials: APIKeyCredentials) -> BlockCost:
"""Build a TOKENS BlockCost for `model` + `credentials`, attaching the
public per-1M-token USD rates when the model has a TOKEN_COST entry.
"""
@@ -439,7 +265,7 @@ def _tokens_llm_cost(model: LlmModel, credentials: APIKeyCredentials) -> BlockCo
)
-def _groq_llm_cost(model: LlmModel) -> BlockCost:
+def _groq_llm_cost(model: LLMModel) -> BlockCost:
"""Groq variant of `_tokens_llm_cost` — keeps the legacy id-only
cost_filter shape so older graphs that stored just the credential id
continue to match.
@@ -455,7 +281,7 @@ def _groq_llm_cost(model: LlmModel) -> BlockCost:
)
-def _open_router_llm_cost(model: LlmModel) -> BlockCost:
+def _open_router_llm_cost(model: LLMModel) -> BlockCost:
"""OpenRouter variant — bills via COST_USD against x-total-cost, but
still exposes the same per-1M-token USD rates so the builder UI shows
the "$X in / $Y out per 1M tokens" pair instead of "Pay-as-you-go".
@@ -1340,7 +1166,7 @@ BLOCK_COSTS: dict[Type[Block], list[BlockCost]] = {
def _validate_codex_costs() -> None:
"""Keep Codex pricing exhaustive.
- ``LlmModel`` has module-load completeness guards; ``CodexModel`` does not, so
+ ``LLMModel`` has module-load completeness guards; ``CodexModel`` does not, so
a Codex model added without a matching ``BlockCost`` would silently bill as
free.
"""
diff --git a/autogpt_platform/backend/backend/data/db_manager.py b/autogpt_platform/backend/backend/data/db_manager.py
index 6d2ccfd820..39b9c4d5a4 100644
--- a/autogpt_platform/backend/backend/data/db_manager.py
+++ b/autogpt_platform/backend/backend/data/db_manager.py
@@ -499,6 +499,7 @@ class DatabaseManager(AppService):
get_next_sequence = _(chat_db.get_next_sequence)
update_tool_message_content = _(chat_db.update_tool_message_content)
update_message_content_by_sequence = _(chat_db.update_message_content_by_sequence)
+ update_chat_message_stamps = _(chat_db.update_chat_message_stamps)
update_chat_message_tool_calls = _(chat_db.update_chat_message_tool_calls)
update_chat_session_title = _(chat_db.update_chat_session_title)
update_chat_session_pinned = _(chat_db.update_chat_session_pinned)
@@ -778,6 +779,7 @@ class DatabaseManagerAsyncClient(AppServiceClient):
get_next_sequence = d.get_next_sequence
update_tool_message_content = d.update_tool_message_content
update_message_content_by_sequence = d.update_message_content_by_sequence
+ update_chat_message_stamps = d.update_chat_message_stamps
update_chat_message_tool_calls = d.update_chat_message_tool_calls
update_chat_session_title = d.update_chat_session_title
update_chat_session_pinned = d.update_chat_session_pinned
diff --git a/autogpt_platform/backend/backend/data/graph.py b/autogpt_platform/backend/backend/data/graph.py
index 4f20ffc28b..c13e520625 100644
--- a/autogpt_platform/backend/backend/data/graph.py
+++ b/autogpt_platform/backend/backend/data/graph.py
@@ -27,7 +27,7 @@ from backend.blocks import get_block, get_blocks
from backend.blocks._base import Block, BlockType, EmptySchema
from backend.blocks.agent import AgentExecutorBlock
from backend.blocks.io import AgentInputBlock, AgentOutputBlock
-from backend.blocks.llm import LEGACY_MODEL_MAPPINGS, LlmModel
+from backend.blocks.llm import LEGACY_MODEL_MAPPINGS, LLMModel
from backend.data.tenancy import get_user_team_ids, visibility_filter
from backend.integrations.providers import ProviderName
from backend.util import type as type_utils
@@ -1976,10 +1976,10 @@ async def fix_llm_provider_credentials():
)
-def _legacy_value_aliases(legacy_value: str, replacement: LlmModel) -> set[str]:
+def _legacy_value_aliases(legacy_value: str, replacement: LLMModel) -> set[str]:
"""Stored-value forms that should map to ``replacement`` for one legacy slug.
- ``LlmModel._missing_`` accepts provider-prefixed inputs at write time, so
+ ``LLMModel._missing_`` accepts provider-prefixed inputs at write time, so
historical rows may carry either the bare slug or ``<provider>/<slug>``
even when the canonical enum value is unprefixed. Vendor-prefixed legacy
values (e.g. ``google/...``) need no alias.
@@ -1989,11 +1989,11 @@ def _legacy_value_aliases(legacy_value: str, replacement: LlmModel) -> set[str]:
return {legacy_value, f"{replacement.metadata.provider}/{legacy_value}"}
-async def migrate_llm_models(fallback: LlmModel):
+async def migrate_llm_models(fallback: LLMModel):
"""
Rewrite legacy LLM model values to in-enum equivalents.
- Runs in two passes per LlmModel field:
+ Runs in two passes per LLMModel field:
1. Family-aware: for each (legacy_value, replacement) in
LEGACY_MODEL_MAPPINGS, rewrite that exact legacy value to its mapped
replacement so e.g. Claude Opus lands on a newer Opus, not the global
@@ -2002,19 +2002,19 @@ async def migrate_llm_models(fallback: LlmModel):
Both passes run against two tables:
* ``AgentNode.constantInput`` — saved graph definitions (scoped by
- ``agentBlockId`` because we know the LlmModel field name per block).
+ ``agentBlockId`` because we know the LLMModel field name per block).
* ``AgentNodeExecutionInputOutput.data`` where ``agentPresetId`` is set —
preset input overrides; scoped only by the field-value match since
preset rows don't carry the block id.
- Note: Only updates top level LlmModel SchemaFields of blocks (won't update nested fields).
+ Note: Only updates top level LLMModel SchemaFields of blocks (won't update nested fields).
"""
logger.info("Migrating LLM models")
llm_model_fields = _find_llm_model_fields()
if not llm_model_fields:
return
- enum_values = [v.value for v in LlmModel]
+ enum_values = [v.value for v in LLMModel]
escaped_enum_values = repr(tuple(enum_values)) # hack but works
node_targeted_query = """
@@ -2088,12 +2088,12 @@ async def migrate_llm_models(fallback: LlmModel):
def _find_llm_model_fields() -> dict[str, str]:
- """Return ``{block_id: field_name}`` for every top-level LlmModel field."""
+ """Return ``{block_id: field_name}`` for every top-level LLMModel field."""
llm_model_fields: dict[str, str] = {}
for block_type in get_blocks().values():
block = block_type()
for field_name, field in block.input_schema.model_fields.items():
- if field.annotation == LlmModel:
+ if field.annotation == LLMModel:
llm_model_fields[block.id] = field_name
return llm_model_fields
diff --git a/autogpt_platform/backend/backend/data/graph_test.py b/autogpt_platform/backend/backend/data/graph_test.py
index 4197cbfe59..9c303564fc 100644
--- a/autogpt_platform/backend/backend/data/graph_test.py
+++ b/autogpt_platform/backend/backend/data/graph_test.py
@@ -14,7 +14,7 @@ from backend.blocks._base import BlockSchema, BlockSchemaInput
from backend.blocks.basic import StoreValueBlock
from backend.blocks.code_executor import ExecuteCodeBlock
from backend.blocks.io import AgentInputBlock, AgentOutputBlock
-from backend.blocks.llm import LEGACY_MODEL_MAPPINGS, LlmModel
+from backend.blocks.llm import LEGACY_MODEL_MAPPINGS, LLMModel
from backend.data.graph import (
Graph,
GraphModel,
@@ -2184,7 +2184,7 @@ async def test_migrate_llm_models_uses_schema_prefix_placeholder():
"backend.data.graph.execute_raw_with_schema",
new_callable=AsyncMock,
) as mock_execute:
- await migrate_llm_models(next(iter(LlmModel)))
+ await migrate_llm_models(next(iter(LLMModel)))
for call in mock_execute.await_args_list:
query_template = call.args[0]
@@ -2202,7 +2202,7 @@ async def test_migrate_llm_models_matches_provider_prefixed_legacy_values():
values (e.g. ``anthropic/claude-sonnet-4-20250514``) to the family-aware
replacement, not let them fall through to the global ``fallback`` model.
- ``LlmModel._missing_`` accepts prefixed inputs at write time, so historical
+ ``LLMModel._missing_`` accepts prefixed inputs at write time, so historical
rows may carry either the bare or prefixed form."""
bare_legacy_value = "claude-sonnet-4-20250514"
expected_replacement = LEGACY_MODEL_MAPPINGS[bare_legacy_value].value
@@ -2212,7 +2212,7 @@ async def test_migrate_llm_models_matches_provider_prefixed_legacy_values():
"backend.data.graph.execute_raw_with_schema",
new_callable=AsyncMock,
) as mock_execute:
- await migrate_llm_models(next(iter(LlmModel)))
+ await migrate_llm_models(next(iter(LLMModel)))
# Targeted query has 5 SQL params ($1..$5), so its call has 6 positional
# args: (query, [path], replacement, block_id, path, stored_value).
@@ -2244,7 +2244,7 @@ async def test_migrate_llm_models_covers_preset_overrides():
"backend.data.graph.execute_raw_with_schema",
new_callable=AsyncMock,
) as mock_execute:
- await migrate_llm_models(next(iter(LlmModel)))
+ await migrate_llm_models(next(iter(LLMModel)))
queries = [call.args[0] for call in mock_execute.await_args_list]
assert any(
diff --git a/autogpt_platform/backend/backend/data/llm_registry/__init__.py b/autogpt_platform/backend/backend/data/llm_registry/__init__.py
new file mode 100644
index 0000000000..fcd7ee74d3
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/__init__.py
@@ -0,0 +1,48 @@
+"""LLM catalog — catalog-as-code model management.
+
+The canonical catalog file (``catalog.py``) holds model facts, per-model
+credit costs, and routing cells; ``load_catalog()`` builds the in-process
+view every consumer reads through.
+"""
+
+from .catalog import get_catalog
+from .catalog_model import (
+ CATALOG_SCHEMA_VERSION,
+ CatalogCreator,
+ CatalogModel,
+ CatalogModelCost,
+ CatalogPayload,
+ CatalogProvider,
+)
+from .registry import (
+ RegistryModel,
+ RegistryModelMetadata,
+ get_all_models,
+ get_model,
+ get_model_by_date_stripped_slug,
+ get_route,
+ has_models,
+ is_loaded,
+ load_catalog,
+)
+
+__all__ = [
+ # Catalog
+ "get_catalog",
+ "CATALOG_SCHEMA_VERSION",
+ "CatalogCreator",
+ "CatalogModel",
+ "CatalogModelCost",
+ "CatalogPayload",
+ "CatalogProvider",
+ # Registry view
+ "RegistryModel",
+ "RegistryModelMetadata",
+ "get_all_models",
+ "get_model",
+ "get_model_by_date_stripped_slug",
+ "get_route",
+ "has_models",
+ "is_loaded",
+ "load_catalog",
+]
diff --git a/autogpt_platform/backend/backend/data/llm_registry/catalog.py b/autogpt_platform/backend/backend/data/llm_registry/catalog.py
new file mode 100644
index 0000000000..7ea4e0a607
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/catalog.py
@@ -0,0 +1,1422 @@
+"""The canonical LLM catalog — catalog-as-code.
+
+This file IS the model database. Edit it by PR; catalog-only diffs are
+fast-tracked by the /review bot and may ride hotfix/* branches straight
+to master for incident-speed changes (kills, routing swaps). CD deploys
+it; every environment loads it at startup into the in-process registry
+cache (``registry.load_catalog``).
+
+Field defaults keep entries short: visibility=GA, is_enabled=True,
+capabilities empty. See ``catalog_model.py`` for every field and
+``catalog_test.py`` for the integrity + cost-drift guards that review
+this file so humans don't have to.
+
+
+Deliberate exception to the ~300-line file guideline: this file IS
+the model database (data-as-code); splitting it would scatter the one
+place model facts live.
+"""
+
+from datetime import datetime, timezone
+
+from backend.data.llm_registry.catalog_model import (
+ CATALOG_SCHEMA_VERSION,
+ CatalogCreator,
+ CatalogModel,
+ CatalogModelCost,
+ CatalogPayload,
+ CatalogProvider,
+)
+
+_catalog_cache: CatalogPayload | None = None
+
+
+def get_catalog() -> CatalogPayload:
+ """Build-once accessor for the catalog payload.
+
+ Raises on an invalid catalog literal. The catalog is LOAD-BEARING:
+ ``llm_models``/``block_cost_config`` call this at module import, so a
+ bad catalog stops every process at boot (the intended fail-hard —
+ CI's catalog tests catch it long before a deploy does)."""
+ global _catalog_cache
+ if _catalog_cache is None:
+ _catalog_cache = _build_catalog()
+ return _catalog_cache
+
+
+def _build_catalog() -> CatalogPayload:
+ return CatalogPayload(
+ schema_version=CATALOG_SCHEMA_VERSION,
+ generated_at=datetime(2026, 7, 18, tzinfo=timezone.utc),
+ providers=[
+ CatalogProvider(name="aiml_api", display_name="AI/ML API"),
+ CatalogProvider(name="anthropic", display_name="Anthropic"),
+ CatalogProvider(name="groq", display_name="Groq"),
+ CatalogProvider(name="llama_api", display_name="Llama API"),
+ CatalogProvider(name="ollama", display_name="Ollama"),
+ CatalogProvider(name="open_router", display_name="OpenRouter"),
+ CatalogProvider(name="openai", display_name="OpenAI"),
+ CatalogProvider(name="v0", display_name="v0 by Vercel"),
+ ],
+ creators=[
+ CatalogCreator(name="amazon", display_name="Amazon"),
+ CatalogCreator(name="anthropic", display_name="Anthropic"),
+ CatalogCreator(name="cohere", display_name="Cohere"),
+ CatalogCreator(name="deepseek", display_name="DeepSeek"),
+ CatalogCreator(name="google", display_name="Google"),
+ CatalogCreator(name="gryphe", display_name="Gryphe"),
+ CatalogCreator(name="meta", display_name="Meta"),
+ CatalogCreator(name="microsoft", display_name="Microsoft"),
+ CatalogCreator(name="mistral-ai", display_name="Mistral AI"),
+ CatalogCreator(name="moonshot-ai", display_name="Moonshot AI"),
+ CatalogCreator(name="nous-research", display_name="Nous Research"),
+ CatalogCreator(name="openai", display_name="OpenAI"),
+ CatalogCreator(name="perplexity", display_name="Perplexity"),
+ CatalogCreator(name="qwen", display_name="Qwen"),
+ CatalogCreator(name="v0", display_name="v0 by Vercel"),
+ CatalogCreator(name="xai", display_name="xAI"),
+ CatalogCreator(name="z.ai", display_name="Z.ai"),
+ ],
+ models=[
+ # ----- AI/ML API -----
+ CatalogModel(
+ slug="meta-llama/Llama-3.3-70B-Instruct-Turbo",
+ display_name="Llama 3.3 70B Instruct Turbo",
+ provider="aiml_api",
+ creator="meta",
+ context_window=128000,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ # ----- Anthropic -----
+ CatalogModel(
+ slug="claude-haiku-4-5-20251001",
+ display_name="Claude Haiku 4.5",
+ provider="anthropic",
+ creator="anthropic",
+ context_window=200000,
+ max_output_tokens=64000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=4,
+ input_credits_per_1m=150.0,
+ output_credits_per_1m=750.0,
+ cache_read_credits_per_1m=15.0,
+ cache_creation_credits_per_1m=188.0,
+ ),
+ ),
+ CatalogModel(
+ slug="claude-opus-4-5-20251101",
+ display_name="Claude Opus 4.5",
+ provider="anthropic",
+ creator="anthropic",
+ context_window=200000,
+ max_output_tokens=64000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=14,
+ input_credits_per_1m=750.0,
+ output_credits_per_1m=3750.0,
+ cache_read_credits_per_1m=75.0,
+ cache_creation_credits_per_1m=938.0,
+ ),
+ ),
+ CatalogModel(
+ slug="claude-opus-4-6",
+ display_name="Claude Opus 4.6",
+ provider="anthropic",
+ creator="anthropic",
+ context_window=200000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=14,
+ input_credits_per_1m=750.0,
+ output_credits_per_1m=3750.0,
+ cache_read_credits_per_1m=75.0,
+ cache_creation_credits_per_1m=938.0,
+ ),
+ ),
+ CatalogModel(
+ slug="claude-opus-4-7",
+ display_name="Claude Opus 4.7",
+ provider="anthropic",
+ creator="anthropic",
+ context_window=200000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=14,
+ input_credits_per_1m=750.0,
+ output_credits_per_1m=3750.0,
+ cache_read_credits_per_1m=75.0,
+ cache_creation_credits_per_1m=938.0,
+ ),
+ ),
+ CatalogModel(
+ slug="claude-sonnet-4-5-20250929",
+ display_name="Claude Sonnet 4.5",
+ provider="anthropic",
+ creator="anthropic",
+ context_window=200000,
+ max_output_tokens=64000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=9,
+ input_credits_per_1m=450.0,
+ output_credits_per_1m=2250.0,
+ cache_read_credits_per_1m=45.0,
+ cache_creation_credits_per_1m=563.0,
+ ),
+ ),
+ CatalogModel(
+ slug="claude-sonnet-4-6",
+ display_name="Claude Sonnet 4.6",
+ provider="anthropic",
+ creator="anthropic",
+ context_window=200000,
+ max_output_tokens=64000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=9,
+ input_credits_per_1m=450.0,
+ output_credits_per_1m=2250.0,
+ cache_read_credits_per_1m=45.0,
+ cache_creation_credits_per_1m=563.0,
+ ),
+ ),
+ # ----- Groq -----
+ CatalogModel(
+ slug="llama-3.1-8b-instant",
+ display_name="Llama 3.1 8B Instant",
+ provider="groq",
+ creator="meta",
+ context_window=128000,
+ max_output_tokens=8192,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=8.0, output_credits_per_1m=12.0
+ ),
+ ),
+ CatalogModel(
+ slug="llama-3.3-70b-versatile",
+ display_name="Llama 3.3 70B Versatile",
+ provider="groq",
+ creator="meta",
+ context_window=128000,
+ max_output_tokens=32768,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=89.0,
+ output_credits_per_1m=119.0,
+ ),
+ ),
+ # ----- Llama API -----
+ CatalogModel(
+ slug="Llama-3.3-70B-Instruct",
+ display_name="Llama 3.3 70B Instruct",
+ provider="llama_api",
+ creator="meta",
+ context_window=128000,
+ max_output_tokens=4028,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="Llama-3.3-8B-Instruct",
+ display_name="Llama 3.3 8B Instruct",
+ provider="llama_api",
+ creator="meta",
+ context_window=128000,
+ max_output_tokens=4028,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="Llama-4-Maverick-17B-128E-Instruct-FP8",
+ display_name="Llama 4 Maverick 17B 128E Instruct FP8",
+ provider="llama_api",
+ creator="meta",
+ context_window=128000,
+ max_output_tokens=4028,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="Llama-4-Scout-17B-16E-Instruct-FP8",
+ display_name="Llama 4 Scout 17B 16E Instruct FP8",
+ provider="llama_api",
+ creator="meta",
+ context_window=128000,
+ max_output_tokens=4028,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ # ----- Ollama -----
+ CatalogModel(
+ slug="dolphin-mistral:latest",
+ display_name="Dolphin Mistral Latest",
+ provider="ollama",
+ creator="mistral-ai",
+ context_window=32768,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="llama3",
+ display_name="Llama 3",
+ provider="ollama",
+ creator="meta",
+ context_window=8192,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="llama3.1:405b",
+ display_name="Llama 3.1 405B",
+ provider="ollama",
+ creator="meta",
+ context_window=8192,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="llama3.2",
+ display_name="Llama 3.2",
+ provider="ollama",
+ creator="meta",
+ context_window=8192,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="llama3.3",
+ display_name="Llama 3.3",
+ provider="ollama",
+ creator="meta",
+ context_window=8192,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ # ----- OpenRouter -----
+ CatalogModel(
+ slug="amazon/nova-lite-v1",
+ display_name="Nova Lite V1",
+ provider="open_router",
+ creator="amazon",
+ context_window=300000,
+ max_output_tokens=5120,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="amazon/nova-micro-v1",
+ display_name="Nova Micro V1",
+ provider="open_router",
+ creator="amazon",
+ context_window=128000,
+ max_output_tokens=5120,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="amazon/nova-pro-v1",
+ display_name="Nova Pro V1",
+ provider="open_router",
+ creator="amazon",
+ context_window=300000,
+ max_output_tokens=5120,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="cohere/command-a-03-2025",
+ display_name="Command A 03.2025",
+ provider="open_router",
+ creator="cohere",
+ context_window=256000,
+ max_output_tokens=8192,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=3,
+ input_credits_per_1m=375.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="cohere/command-a-reasoning-08-2025",
+ display_name="Command A Reasoning 08.2025",
+ provider="open_router",
+ creator="cohere",
+ context_window=256000,
+ max_output_tokens=32768,
+ price_tier=3,
+ cost=CatalogModelCost(run_credits=6),
+ ),
+ CatalogModel(
+ slug="cohere/command-a-translate-08-2025",
+ display_name="Command A Translate 08.2025",
+ provider="open_router",
+ creator="cohere",
+ context_window=128000,
+ max_output_tokens=8192,
+ price_tier=2,
+ cost=CatalogModelCost(run_credits=3),
+ ),
+ CatalogModel(
+ slug="cohere/command-a-vision-07-2025",
+ display_name="Command A Vision 07.2025",
+ provider="open_router",
+ creator="cohere",
+ context_window=128000,
+ max_output_tokens=8192,
+ price_tier=2,
+ cost=CatalogModelCost(run_credits=3),
+ ),
+ CatalogModel(
+ slug="deepseek/deepseek-chat",
+ display_name="DeepSeek Chat",
+ provider="open_router",
+ creator="deepseek",
+ context_window=64000,
+ max_output_tokens=2048,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2, input_credits_per_1m=21.0, output_credits_per_1m=42.0
+ ),
+ ),
+ CatalogModel(
+ slug="deepseek/deepseek-r1-0528",
+ display_name="DeepSeek R1 0528",
+ provider="open_router",
+ creator="deepseek",
+ context_window=163840,
+ max_output_tokens=163840,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=21.0, output_credits_per_1m=42.0
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-2.0-flash-001",
+ display_name="Gemini 2.0 Flash 001",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=8192,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=15.0, output_credits_per_1m=60.0
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-2.0-flash-lite-001",
+ display_name="Gemini 2.0 Flash Lite 001",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=8192,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=11.0, output_credits_per_1m=45.0
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-2.5-flash",
+ display_name="Gemini 2.5 Flash",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=65535,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=45.0,
+ output_credits_per_1m=375.0,
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-2.5-flash-lite",
+ display_name="Gemini 2.5 Flash Lite",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=65535,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=15.0, output_credits_per_1m=60.0
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-2.5-pro",
+ display_name="Gemini 2.5 Pro",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=65536,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=4,
+ input_credits_per_1m=188.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-3-flash-preview",
+ display_name="Gemini 3 Flash Preview",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=65536,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=75.0,
+ output_credits_per_1m=450.0,
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-3.1-flash-lite-preview",
+ display_name="Gemini 3.1 Flash Lite Preview",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=65536,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=38.0,
+ output_credits_per_1m=225.0,
+ ),
+ ),
+ CatalogModel(
+ slug="google/gemini-3.1-pro-preview",
+ display_name="Gemini 3.1 Pro Preview",
+ provider="open_router",
+ creator="google",
+ context_window=1048576,
+ max_output_tokens=65536,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=1800.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gryphe/mythomax-l2-13b",
+ display_name="MythoMax L2 13B",
+ provider="open_router",
+ creator="gryphe",
+ context_window=4096,
+ max_output_tokens=4096,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="meta-llama/llama-4-maverick",
+ display_name="Llama 4 Maverick",
+ provider="open_router",
+ creator="meta",
+ context_window=1048576,
+ max_output_tokens=1000000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=75.0,
+ output_credits_per_1m=116.0,
+ ),
+ ),
+ CatalogModel(
+ slug="meta-llama/llama-4-scout",
+ display_name="Llama 4 Scout",
+ provider="open_router",
+ creator="meta",
+ context_window=131072,
+ max_output_tokens=131072,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=17.0, output_credits_per_1m=51.0
+ ),
+ ),
+ CatalogModel(
+ slug="microsoft/phi-4",
+ display_name="Phi-4",
+ provider="open_router",
+ creator="microsoft",
+ context_window=16384,
+ max_output_tokens=16384,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="mistralai/codestral-2508",
+ display_name="Codestral 2508",
+ provider="open_router",
+ creator="mistral-ai",
+ context_window=256000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=45.0,
+ output_credits_per_1m=135.0,
+ ),
+ ),
+ CatalogModel(
+ slug="mistralai/mistral-large-2512",
+ display_name="Mistral Large 3 2512",
+ provider="open_router",
+ creator="mistral-ai",
+ context_window=262144,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=900.0,
+ ),
+ ),
+ CatalogModel(
+ slug="mistralai/mistral-medium-3.1",
+ display_name="Mistral Medium 3.1",
+ provider="open_router",
+ creator="mistral-ai",
+ context_window=131072,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=60.0,
+ output_credits_per_1m=300.0,
+ ),
+ ),
+ CatalogModel(
+ slug="mistralai/mistral-small-3.2-24b-instruct",
+ display_name="Mistral Small 3.2 24B",
+ provider="open_router",
+ creator="mistral-ai",
+ context_window=131072,
+ max_output_tokens=131072,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=15.0, output_credits_per_1m=45.0
+ ),
+ ),
+ CatalogModel(
+ slug="moonshotai/kimi-k2-thinking",
+ display_name="Kimi K2 Thinking",
+ provider="open_router",
+ creator="moonshot-ai",
+ context_window=262144,
+ max_output_tokens=262144,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=90.0,
+ output_credits_per_1m=375.0,
+ ),
+ ),
+ CatalogModel(
+ slug="moonshotai/kimi-k2.5",
+ display_name="Kimi K2.5",
+ provider="open_router",
+ creator="moonshot-ai",
+ context_window=262144,
+ max_output_tokens=262144,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=66.0,
+ output_credits_per_1m=300.0,
+ ),
+ ),
+ CatalogModel(
+ slug="moonshotai/kimi-k2.6",
+ display_name="Kimi K2.6",
+ provider="open_router",
+ creator="moonshot-ai",
+ context_window=262144,
+ max_output_tokens=262144,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=112.0,
+ output_credits_per_1m=698.0,
+ ),
+ ),
+ CatalogModel(
+ slug="moonshotai/kimi-k3",
+ display_name="Kimi K3",
+ provider="open_router",
+ creator="moonshot-ai",
+ context_window=1048576,
+ # OpenRouter publishes no completion cap for K3 (verified
+ # live: top_provider.max_completion_tokens = null) — None
+ # means "unknown/no published cap"; block runtime falls
+ # back to its own 32K default.
+ max_output_tokens=None,
+ price_tier=3,
+ supports_tools=True,
+ supports_reasoning=True,
+ # Moonshot's premium tier — $3/$15 per Mtok on OpenRouter,
+ # credit rates at the standard 1.5x margin (Sonnet-class
+ # pricing).
+ cost=CatalogModelCost(
+ run_credits=9,
+ input_credits_per_1m=450.0,
+ output_credits_per_1m=2250.0,
+ provider_input_usd_per_1m=3.00,
+ provider_output_usd_per_1m=15.00,
+ ),
+ ),
+ CatalogModel(
+ slug="nousresearch/hermes-3-llama-3.1-405b",
+ display_name="Hermes 3 Llama 3.1 405B",
+ provider="open_router",
+ creator="nous-research",
+ context_window=131000,
+ max_output_tokens=4096,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="nousresearch/hermes-3-llama-3.1-70b",
+ display_name="Hermes 3 Llama 3.1 70B",
+ provider="open_router",
+ creator="nous-research",
+ context_window=12288,
+ max_output_tokens=12288,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="openai/gpt-oss-120b",
+ display_name="GPT-OSS 120B",
+ provider="open_router",
+ creator="openai",
+ context_window=131072,
+ max_output_tokens=131072,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=23.0, output_credits_per_1m=90.0
+ ),
+ ),
+ CatalogModel(
+ slug="openai/gpt-oss-20b",
+ display_name="GPT-OSS 20B",
+ provider="open_router",
+ creator="openai",
+ context_window=131072,
+ max_output_tokens=32768,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=11.0, output_credits_per_1m=45.0
+ ),
+ ),
+ CatalogModel(
+ slug="perplexity/sonar",
+ display_name="Sonar",
+ provider="open_router",
+ creator="perplexity",
+ context_window=127000,
+ max_output_tokens=8000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=150.0,
+ output_credits_per_1m=150.0,
+ ),
+ ),
+ CatalogModel(
+ slug="perplexity/sonar-deep-research",
+ display_name="Sonar Deep Research",
+ provider="open_router",
+ creator="perplexity",
+ context_window=128000,
+ max_output_tokens=16000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=10,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=1200.0,
+ ),
+ ),
+ CatalogModel(
+ slug="perplexity/sonar-pro",
+ display_name="Sonar Pro",
+ provider="open_router",
+ creator="perplexity",
+ context_window=200000,
+ max_output_tokens=8000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=450.0,
+ output_credits_per_1m=2250.0,
+ ),
+ ),
+ CatalogModel(
+ slug="perplexity/sonar-reasoning-pro",
+ display_name="Sonar Reasoning Pro",
+ provider="open_router",
+ creator="perplexity",
+ context_window=128000,
+ max_output_tokens=8000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=1200.0,
+ ),
+ ),
+ CatalogModel(
+ slug="qwen/qwen3-235b-a22b-thinking-2507",
+ display_name="Qwen 3 235B A22B Thinking 2507",
+ provider="open_router",
+ creator="qwen",
+ context_window=262144,
+ max_output_tokens=262144,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="qwen/qwen3-coder",
+ display_name="Qwen 3 Coder",
+ provider="open_router",
+ creator="qwen",
+ context_window=262144,
+ max_output_tokens=262144,
+ price_tier=3,
+ cost=CatalogModelCost(run_credits=9),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-3",
+ display_name="Grok 3",
+ provider="open_router",
+ creator="xai",
+ context_window=131072,
+ max_output_tokens=131072,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=3,
+ input_credits_per_1m=450.0,
+ output_credits_per_1m=2250.0,
+ ),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-4",
+ display_name="Grok 4",
+ provider="open_router",
+ creator="xai",
+ context_window=256000,
+ max_output_tokens=256000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=9,
+ input_credits_per_1m=450.0,
+ output_credits_per_1m=2250.0,
+ ),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-4-fast",
+ display_name="Grok 4 Fast",
+ provider="open_router",
+ creator="xai",
+ context_window=2000000,
+ max_output_tokens=30000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=30.0, output_credits_per_1m=75.0
+ ),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-4.1-fast",
+ display_name="Grok 4.1 Fast",
+ provider="open_router",
+ creator="xai",
+ context_window=2000000,
+ max_output_tokens=30000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=30.0, output_credits_per_1m=75.0
+ ),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-4.20",
+ display_name="Grok 4.20",
+ provider="open_router",
+ creator="xai",
+ context_window=2000000,
+ max_output_tokens=100000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=900.0,
+ ),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-4.20-multi-agent",
+ display_name="Grok 4.20 Multi-Agent",
+ provider="open_router",
+ creator="xai",
+ context_window=2000000,
+ max_output_tokens=100000,
+ price_tier=3,
+ cost=CatalogModelCost(run_credits=5),
+ ),
+ CatalogModel(
+ slug="x-ai/grok-code-fast-1",
+ display_name="Grok Code Fast 1",
+ provider="open_router",
+ creator="xai",
+ context_window=256000,
+ max_output_tokens=10000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=30.0,
+ output_credits_per_1m=225.0,
+ ),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-4.6",
+ display_name="GLM 4.6",
+ provider="open_router",
+ creator="z.ai",
+ context_window=204800,
+ max_output_tokens=204800,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-4.6v",
+ display_name="GLM 4.6V",
+ provider="open_router",
+ creator="z.ai",
+ context_window=131072,
+ max_output_tokens=131072,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-4.7",
+ display_name="GLM 4.7",
+ provider="open_router",
+ creator="z.ai",
+ context_window=202752,
+ max_output_tokens=65535,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-4.7-flash",
+ display_name="GLM 4.7 Flash",
+ provider="open_router",
+ creator="z.ai",
+ context_window=202752,
+ max_output_tokens=202752,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-5",
+ display_name="GLM 5",
+ provider="open_router",
+ creator="z.ai",
+ context_window=80000,
+ max_output_tokens=80000,
+ price_tier=2,
+ cost=CatalogModelCost(run_credits=2),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-5-turbo",
+ display_name="GLM 5 Turbo",
+ provider="open_router",
+ creator="z.ai",
+ context_window=202752,
+ max_output_tokens=131072,
+ price_tier=3,
+ cost=CatalogModelCost(run_credits=4),
+ ),
+ CatalogModel(
+ slug="z-ai/glm-5v-turbo",
+ display_name="GLM 5V Turbo",
+ provider="open_router",
+ creator="z.ai",
+ context_window=202752,
+ max_output_tokens=131072,
+ price_tier=3,
+ cost=CatalogModelCost(run_credits=4),
+ ),
+ # ----- OpenAI -----
+ CatalogModel(
+ slug="gpt-4.1-2025-04-14",
+ display_name="GPT-4.1",
+ provider="openai",
+ creator="openai",
+ context_window=1047576,
+ max_output_tokens=32768,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=1200.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-4.1-mini-2025-04-14",
+ display_name="GPT-4.1 Mini",
+ provider="openai",
+ creator="openai",
+ context_window=1047576,
+ max_output_tokens=32768,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=60.0,
+ output_credits_per_1m=240.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-4o",
+ display_name="GPT-4o",
+ provider="openai",
+ creator="openai",
+ context_window=128000,
+ max_output_tokens=16384,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=3,
+ input_credits_per_1m=375.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-4o-mini",
+ display_name="GPT-4o Mini",
+ provider="openai",
+ creator="openai",
+ context_window=128000,
+ max_output_tokens=16384,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=23.0, output_credits_per_1m=90.0
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5-2025-08-07",
+ display_name="GPT-5",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=188.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5-chat-latest",
+ display_name="GPT-5 Chat Latest",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=16384,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=188.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5-mini-2025-08-07",
+ display_name="GPT-5 Mini",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=38.0,
+ output_credits_per_1m=300.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5-nano-2025-08-07",
+ display_name="GPT-5 Nano",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1, input_credits_per_1m=8.0, output_credits_per_1m=60.0
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.1-2025-11-13",
+ display_name="GPT-5.1",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=188.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.2-2025-12-11",
+ display_name="GPT-5.2",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=6,
+ input_credits_per_1m=263.0,
+ output_credits_per_1m=2100.0,
+ ),
+ ),
+ CatalogModel(
+ slug="o4-mini",
+ display_name="O4 Mini",
+ provider="openai",
+ creator="openai",
+ context_window=200000,
+ max_output_tokens=100000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=165.0,
+ output_credits_per_1m=660.0,
+ ),
+ ),
+ CatalogModel(
+ slug="o3-pro",
+ display_name="O3 Pro",
+ provider="openai",
+ creator="openai",
+ context_window=200000,
+ max_output_tokens=100000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=30,
+ input_credits_per_1m=3000.0,
+ output_credits_per_1m=12000.0,
+ ),
+ ),
+ CatalogModel(
+ slug="o1",
+ display_name="O1",
+ provider="openai",
+ creator="openai",
+ context_window=200000,
+ max_output_tokens=100000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=16,
+ input_credits_per_1m=2250.0,
+ output_credits_per_1m=9000.0,
+ ),
+ ),
+ CatalogModel(
+ slug="o1-mini",
+ display_name="O1 Mini",
+ provider="openai",
+ creator="openai",
+ context_window=128000,
+ max_output_tokens=65536,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=165.0,
+ output_credits_per_1m=660.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.6-sol",
+ display_name="GPT-5.6 Sol",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=10,
+ input_credits_per_1m=750.0,
+ output_credits_per_1m=4500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.6-terra",
+ display_name="GPT-5.6 Terra",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=2,
+ is_recommended=True,
+ cost=CatalogModelCost(
+ run_credits=6,
+ input_credits_per_1m=375.0,
+ output_credits_per_1m=2250.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.6-luna",
+ display_name="GPT-5.6 Luna",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=150.0,
+ output_credits_per_1m=900.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.5-2026-04-23",
+ display_name="GPT-5.5",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=10,
+ input_credits_per_1m=750.0,
+ output_credits_per_1m=4500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.5-pro",
+ display_name="GPT-5.5 Pro",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=60,
+ input_credits_per_1m=4500.0,
+ output_credits_per_1m=27000.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.4-2026-03-05",
+ display_name="GPT-5.4",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=6,
+ input_credits_per_1m=375.0,
+ output_credits_per_1m=2250.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.4-mini-2026-03-17",
+ display_name="GPT-5.4 Mini",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=113.0,
+ output_credits_per_1m=675.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.4-nano-2026-03-17",
+ display_name="GPT-5.4 Nano",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=30.0,
+ output_credits_per_1m=188.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.4-pro",
+ display_name="GPT-5.4 Pro",
+ provider="openai",
+ creator="openai",
+ context_window=1050000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=60,
+ input_credits_per_1m=4500.0,
+ output_credits_per_1m=27000.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.3-chat-latest",
+ display_name="GPT-5.3",
+ provider="openai",
+ creator="openai",
+ context_window=128000,
+ max_output_tokens=16384,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=263.0,
+ output_credits_per_1m=2100.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.3-codex",
+ display_name="GPT-5.3 Codex",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=263.0,
+ output_credits_per_1m=2100.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.2-pro",
+ display_name="GPT-5.2 Pro",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=60,
+ input_credits_per_1m=3150.0,
+ output_credits_per_1m=25200.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5.1-codex",
+ display_name="GPT-5.1 Codex",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=5,
+ input_credits_per_1m=188.0,
+ output_credits_per_1m=1500.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-5-pro",
+ display_name="GPT-5 Pro",
+ provider="openai",
+ creator="openai",
+ context_window=400000,
+ max_output_tokens=128000,
+ price_tier=3,
+ cost=CatalogModelCost(
+ run_credits=60,
+ input_credits_per_1m=2250.0,
+ output_credits_per_1m=18000.0,
+ ),
+ ),
+ CatalogModel(
+ slug="gpt-4.1-nano",
+ display_name="GPT-4.1 Nano",
+ provider="openai",
+ creator="openai",
+ context_window=1047576,
+ max_output_tokens=32768,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=1,
+ input_credits_per_1m=15.0,
+ output_credits_per_1m=60.0,
+ ),
+ ),
+ CatalogModel(
+ slug="o3-2025-04-16",
+ display_name="O3",
+ provider="openai",
+ creator="openai",
+ context_window=200000,
+ max_output_tokens=100000,
+ price_tier=2,
+ cost=CatalogModelCost(
+ run_credits=4,
+ input_credits_per_1m=300.0,
+ output_credits_per_1m=1200.0,
+ ),
+ ),
+ CatalogModel(
+ slug="o3-mini",
+ display_name="O3 Mini",
+ provider="openai",
+ creator="openai",
+ context_window=200000,
+ max_output_tokens=100000,
+ price_tier=1,
+ cost=CatalogModelCost(
+ run_credits=2,
+ input_credits_per_1m=165.0,
+ output_credits_per_1m=660.0,
+ ),
+ ),
+ # ----- v0 by Vercel -----
+ CatalogModel(
+ slug="v0-1.0-md",
+ display_name="v0 1.0 MD",
+ provider="v0",
+ creator="v0",
+ context_window=128000,
+ max_output_tokens=64000,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ CatalogModel(
+ slug="v0-1.5-lg",
+ display_name="v0 1.5 LG",
+ provider="v0",
+ creator="v0",
+ context_window=512000,
+ max_output_tokens=64000,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=2),
+ ),
+ CatalogModel(
+ slug="v0-1.5-md",
+ display_name="v0 1.5 MD",
+ provider="v0",
+ creator="v0",
+ context_window=128000,
+ max_output_tokens=64000,
+ price_tier=1,
+ cost=CatalogModelCost(run_credits=1),
+ ),
+ ],
+ # Routing cells deliberately ship EMPTY. Setting a cell is the
+ # explicit act of moving that (mode, tier)'s control from env vars
+ # into the catalog — shipping populated defaults would silently
+ # shadow the CHAT_*_MODEL env config of any deployment whose env
+ # differs from code defaults, and would flip models during LD
+ # outages. Cells apply on cloud deployments (behave_as=CLOUD) only.
+ routing={},
+ )
diff --git a/autogpt_platform/backend/backend/data/llm_registry/catalog_model.py b/autogpt_platform/backend/backend/data/llm_registry/catalog_model.py
new file mode 100644
index 0000000000..c9737c49d4
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/catalog_model.py
@@ -0,0 +1,131 @@
+"""Catalog schema — the shape of the canonical LLM catalog file.
+
+The catalog file (``catalog.py`` in this package) IS the model database:
+model facts, per-model credit costs, and routing cells, updated by PR and
+propagated by deploy (catalog-as-code). This module defines and validates
+its shape; ``catalog_test.py`` is the referential-integrity guard.
+
+Deliberately excluded: provider credentials (env/settings as always) and
+anything per-install (retirement records live in the DB, see
+``LlmModelMigration``).
+"""
+
+from __future__ import annotations
+
+from datetime import datetime
+from typing import Literal
+
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+
+CATALOG_SCHEMA_VERSION = 1
+
+ModelVisibility = Literal["GA", "EMPLOYEES", "ADMINS", "HIDDEN"]
+
+# Guard against a runaway generator expanding into an unreviewable file.
+MAX_CATALOG_MODELS = 2000
+
+_NAME_PATTERN = r"^[a-z0-9][a-z0-9._-]{0,99}$"
+# Model slugs may be provider-prefixed and contain "/" (e.g. "openai/gpt-4o").
+_SLUG_PATTERN = r"^[a-zA-Z0-9][a-zA-Z0-9/._:-]{0,199}$"
+
+
+class CatalogProvider(BaseModel):
+ model_config = ConfigDict(frozen=True)
+
+ name: str = Field(pattern=_NAME_PATTERN)
+ display_name: str = Field(min_length=1, max_length=200)
+ description: str | None = None
+
+
+class CatalogCreator(BaseModel):
+ model_config = ConfigDict(frozen=True)
+
+ name: str = Field(pattern=_NAME_PATTERN)
+ display_name: str = Field(min_length=1, max_length=200)
+ description: str | None = None
+ website_url: str | None = None
+ logo_url: str | None = None
+
+
+class CatalogModelCost(BaseModel):
+ """Per-model credit pricing.
+
+ ``run_credits`` is the flat credits-per-run tier; the ``*_per_1m``
+ fields are credits per 1,000,000 tokens. These ARE the billing
+ source: ``backend/data/block_cost_config.py`` derives its
+ ``MODEL_COST``/``TOKEN_COST`` lookups from them, and
+ ``catalog_test.py`` pins the values billed at cutover via the
+ pre-catalog snapshot.
+ """
+
+ model_config = ConfigDict(frozen=True)
+
+ run_credits: int | None = Field(default=None, ge=0)
+ input_credits_per_1m: float | None = Field(default=None, ge=0)
+ output_credits_per_1m: float | None = Field(default=None, ge=0)
+ cache_read_credits_per_1m: float | None = Field(default=None, ge=0)
+ cache_creation_credits_per_1m: float | None = Field(default=None, ge=0)
+ # What the PROVIDER charges us (USD list price per 1M tokens), for
+ # in-turn cost estimation — authored per model when it diverges from
+ # the transport module's family default (e.g. copilot/moonshot.py).
+ # Not the billing source; the credits fields above are what users pay.
+ provider_input_usd_per_1m: float | None = Field(default=None, ge=0)
+ provider_output_usd_per_1m: float | None = Field(default=None, ge=0)
+
+ @model_validator(mode="after")
+ def _provider_usd_both_or_neither(self) -> "CatalogModelCost":
+ # A half-authored provider price would silently fall back to the
+ # transport module's (cheaper) family default — underpricing a
+ # premium SKU with no error. Refuse the partial state at authoring.
+ if (self.provider_input_usd_per_1m is None) != (
+ self.provider_output_usd_per_1m is None
+ ):
+ raise ValueError(
+ "provider_input_usd_per_1m and provider_output_usd_per_1m "
+ "must be set together"
+ )
+ return self
+
+
+class CatalogModel(BaseModel):
+ model_config = ConfigDict(frozen=True)
+
+ slug: str = Field(pattern=_SLUG_PATTERN)
+ display_name: str = Field(min_length=1, max_length=200)
+ description: str | None = None
+ provider: str = Field(pattern=_NAME_PATTERN) # FK by CatalogProvider.name
+ creator: str | None = Field(default=None, pattern=_NAME_PATTERN)
+ context_window: int = Field(gt=0)
+ max_output_tokens: int | None = Field(default=None, gt=0)
+ price_tier: Literal[1, 2, 3] = 1
+ is_enabled: bool = True
+ is_recommended: bool = False
+ # Who can SEE the model. Non-GA models are excluded from block picker
+ # metadata today; per-role tiers (EMPLOYEES/ADMINS) get distinct
+ # treatment when the catalog-driven picker ships. Orthogonal to
+ # is_enabled: is_enabled=False is the kill switch (copilot refuses,
+ # picker hides); visibility="HIDDEN" still SERVES when explicitly
+ # routed — the pre-launch testing state.
+ visibility: ModelVisibility = "GA"
+ # Standing replacement pointer: pre-fills the retirement CLI's
+ # replacement and is the hook for future runtime failover.
+ fallback_model_slug: str | None = Field(default=None, pattern=_SLUG_PATTERN)
+ supports_tools: bool = False
+ supports_json_output: bool = False
+ supports_reasoning: bool = False
+ supports_parallel_tool_calls: bool = False
+ cost: CatalogModelCost | None = None
+
+
+class CatalogPayload(BaseModel):
+ model_config = ConfigDict(frozen=True)
+
+ schema_version: int
+ generated_at: datetime
+ providers: list[CatalogProvider]
+ creators: list[CatalogCreator]
+ models: list[CatalogModel] = Field(max_length=MAX_CATALOG_MODELS)
+ # surface -> mode -> tier -> model slug (e.g. routing["copilot"]["fast"]
+ # ["standard"]). The admin-set config layer of model resolution: LD
+ # per-user override above it, ChatConfig env defaults below it.
+ routing: dict[str, dict[str, dict[str, str]]] = {}
diff --git a/autogpt_platform/backend/backend/data/llm_registry/catalog_model_test.py b/autogpt_platform/backend/backend/data/llm_registry/catalog_model_test.py
new file mode 100644
index 0000000000..0cf5899e50
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/catalog_model_test.py
@@ -0,0 +1,93 @@
+"""Validation tests for the catalog schema."""
+
+from __future__ import annotations
+
+from datetime import datetime, timezone
+
+import pydantic
+import pytest
+
+from backend.data.llm_registry.catalog_model import (
+ CATALOG_SCHEMA_VERSION,
+ CatalogModel,
+ CatalogModelCost,
+ CatalogPayload,
+)
+
+
+def _model_kwargs(**overrides):
+ kwargs = dict(
+ slug="openai/gpt-4o",
+ display_name="GPT-4o",
+ provider="openai",
+ context_window=128000,
+ )
+ kwargs.update(overrides)
+ return kwargs
+
+
+def test_valid_model_parses():
+ m = CatalogModel(**_model_kwargs())
+ assert m.price_tier == 1
+ assert m.visibility == "GA"
+ assert m.is_enabled is True
+ assert m.cost is None
+
+
+ "field,value",
+ [
+ ("slug", "has spaces"),
+ ("slug", ""),
+ ("provider", "Has/Slash"),
+ ("context_window", 0),
+ ("context_window", -5),
+ ("max_output_tokens", 0),
+ ("price_tier", 0),
+ ("price_tier", 4),
+ ("display_name", ""),
+ ("visibility", "SECRET"),
+ ],
+)
+def test_invalid_model_fields_rejected(field, value):
+ with pytest.raises(pydantic.ValidationError):
+ CatalogModel(**_model_kwargs(**{field: value}))
+
+
+ "field",
+ [
+ "run_credits",
+ "input_credits_per_1m",
+ "output_credits_per_1m",
+ "cache_read_credits_per_1m",
+ "cache_creation_credits_per_1m",
+ ],
+)
+def test_negative_costs_rejected(field):
+ with pytest.raises(pydantic.ValidationError):
+ CatalogModelCost(**{field: -1})
+
+
+def test_routing_shape_accepts_nested_cells():
+ payload = CatalogPayload(
+ schema_version=CATALOG_SCHEMA_VERSION,
+ generated_at=datetime.now(timezone.utc),
+ providers=[],
+ creators=[],
+ models=[],
+ routing={"copilot": {"fast": {"standard": "openai/gpt-4o"}}},
+ )
+ assert payload.routing["copilot"]["fast"]["standard"] == "openai/gpt-4o"
+
+
+def test_payload_model_count_cap():
+ models = [CatalogModel(**_model_kwargs(slug=f"openai/m{i}")) for i in range(2001)]
+ with pytest.raises(pydantic.ValidationError):
+ CatalogPayload(
+ schema_version=CATALOG_SCHEMA_VERSION,
+ generated_at=datetime.now(timezone.utc),
+ providers=[],
+ creators=[],
+ models=models,
+ )
diff --git a/autogpt_platform/backend/backend/data/llm_registry/catalog_test.py b/autogpt_platform/backend/backend/data/llm_registry/catalog_test.py
new file mode 100644
index 0000000000..52c4357d80
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/catalog_test.py
@@ -0,0 +1,182 @@
+"""Forever-guards for the canonical catalog file.
+
+These tests ARE the review for catalog-only PRs (the /review bot fast-path
+relies on them): if one fails, the catalog edit is wrong — fix the data,
+never the test.
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+from backend.blocks.llm import MODEL_METADATA, LLMModel
+from backend.data.block_cost_config import MODEL_COST, TOKEN_COST
+from backend.data.llm_registry.catalog import get_catalog
+from backend.data.llm_registry.catalog_model import CATALOG_SCHEMA_VERSION
+
+CATALOG = get_catalog()
+
+
+def test_catalog_declares_current_schema_version():
+ assert CATALOG.schema_version == CATALOG_SCHEMA_VERSION
+
+
+def test_catalog_is_not_suspiciously_small():
+ assert len(CATALOG.models) > 50, "catalog lost most of its models"
+
+
+def test_model_slugs_are_unique():
+ slugs = [m.slug for m in CATALOG.models]
+ assert len(slugs) == len(set(slugs)), "duplicate model slugs"
+
+
+def test_provider_and_creator_references_resolve():
+ provider_names = {p.name for p in CATALOG.providers}
+ creator_names = {c.name for c in CATALOG.creators}
+ for m in CATALOG.models:
+ assert m.provider in provider_names, f"{m.slug}: unknown provider {m.provider}"
+ if m.creator:
+ assert m.creator in creator_names, f"{m.slug}: unknown creator {m.creator}"
+
+
+def test_fallback_references_resolve():
+ slugs = {m.slug for m in CATALOG.models}
+ for m in CATALOG.models:
+ if m.fallback_model_slug:
+ assert (
+ m.fallback_model_slug in slugs
+ ), f"{m.slug}: fallback {m.fallback_model_slug} is not in the catalog"
+ assert m.fallback_model_slug != m.slug, f"{m.slug}: fallback is itself"
+
+
+def _resolve_cell(by_slug: dict, value: str):
+ """Slug-tolerant cell resolution mirroring the router's gate: exact,
+ vendor-stripped, and dots-to-dashes forms (cells carry transport
+ spellings, not canonical slugs — see the spelling-convention test)."""
+ candidates = {value, value.split("/", 1)[-1]}
+ candidates |= {c.replace(".", "-") for c in set(candidates)}
+ for c in candidates:
+ if c in by_slug:
+ return by_slug[c]
+ return None
+
+
+def test_routing_cells_reference_enabled_models():
+ by_slug = {m.slug: m for m in CATALOG.models}
+ for surface, modes in CATALOG.routing.items():
+ for mode, tiers in modes.items():
+ for tier, slug in tiers.items():
+ cell = f"routing[{surface}][{mode}][{tier}]"
+ model = _resolve_cell(by_slug, slug)
+ assert model is not None, f"{cell}: unknown model {slug}"
+ assert model.is_enabled, f"{cell}: model {slug} is disabled"
+
+
+# NOTE: there is deliberately no "matrix fully specified" guard. Cells ship
+# empty and get claimed one at a time — an unset cell means env vars keep
+# that (mode, tier), which is the intended rollout-safe default. Cells that
+# DO exist are governed by the reference and spelling tests above/below.
+
+
+_SNAPSHOT_PATH = Path(__file__).parent / "pre_catalog_costs_snapshot.json"
+
+
+def test_billing_matches_pre_catalog_snapshot():
+ """Cutover-parity proof: the catalog-derived cost dicts reproduce the
+ exact prices billed by the hand-maintained literals they replaced
+ (snapshot captured 2026-07-20). New models add entries freely; changing
+ a pre-cutover model's price is a deliberate two-line diff (catalog +
+ snapshot) so review sees old→new explicitly. Deletable once the
+ cutover has soaked a release.
+ """
+ snapshot = json.loads(_SNAPSHOT_PATH.read_text())
+ for slug, credits in snapshot["model_cost"].items():
+ assert MODEL_COST[LLMModel(slug)] == credits, slug
+ for slug, rate in snapshot["token_cost"].items():
+ assert TOKEN_COST[LLMModel(slug)].model_dump() == rate, slug
+ # Absence parity: the cutover itself must not silently move a model
+ # between flat-rate and token billing.
+ pre_cutover = set(snapshot["model_cost"])
+ token_billed = {m.value for m in TOKEN_COST}
+ assert token_billed & pre_cutover == set(snapshot["token_cost"])
+
+
+def test_metadata_matches_pre_catalog_snapshot():
+ """Same parity proof for the block-facing metadata projection.
+
+ DISCLOSED DELTA: 7 display cells in the snapshot were updated to the
+ catalog's names — the cutover intentionally renames them:
+ provider_name "V0" → "v0 by Vercel" and creator_name "V0" →
+ "v0 by Vercel" (v0-1.0-md, v0-1.5-md, v0-1.5-lg), plus provider_name
+ "AI/ML" → "AI/ML API" (meta-llama/Llama-3.3-70B-Instruct-Turbo).
+ Every other cell is the deleted literal, byte-for-byte.
+ """
+ snapshot = json.loads(_SNAPSHOT_PATH.read_text())
+ for slug, fields in snapshot["model_metadata"].items():
+ assert MODEL_METADATA[LLMModel(slug)]._asdict() == fields, slug
+
+
+def test_exactly_one_enabled_recommended_model():
+ """DEFAULT_LLM_MODEL derives from is_recommended in catalog order — a
+ second recommended entry would silently shift the platform default."""
+ recommended = [m.slug for m in CATALOG.models if m.is_recommended and m.is_enabled]
+ assert len(recommended) == 1, recommended
+
+
+def test_kimi_k3_bills_at_authored_rates():
+ """The flagship catalog-native model's billing projections — flat tier
+ and per-1M token rates — must match its authored catalog entry."""
+ k3 = LLMModel("moonshotai/kimi-k3")
+ assert MODEL_COST[k3] == 9
+ assert TOKEN_COST[k3].model_dump() == {
+ "input": 450.0,
+ "output": 2250.0,
+ "cache_read": 0.0,
+ "cache_creation": 0.0,
+ }
+
+
+def test_provider_usd_prices_are_all_or_nothing():
+ """A half-authored provider USD price must refuse to construct — it
+ would silently underprice against the transport family default."""
+ import pytest
+
+ from backend.data.llm_registry.catalog_model import CatalogModelCost
+
+ with pytest.raises(ValueError, match="must be set together"):
+ CatalogModelCost(run_credits=1, provider_input_usd_per_1m=3.0)
+
+
+def test_routing_cells_use_transport_ready_spellings():
+ """Routing cells are sent to providers (nearly) verbatim, so they must use
+ the spelling the transport expects — NOT the catalog's canonical slug.
+
+ Convention: Anthropic cells use the vendor-prefixed DOT form
+ (``anthropic/claude-sonnet-4.6``) — the form OpenRouter serves and the
+ direct-Anthropic normalizer accepts. Bare ``claude-*`` 404s on OpenRouter;
+ dash-form ``anthropic/claude-*-4-6`` exists on no transport. Cells still
+ gate against the catalog via the resolver's slug-tolerant lookup.
+ """
+ slugs = {m.slug for m in CATALOG.models}
+
+ def tolerant_match(value: str) -> bool:
+ candidates = {value, value.split("/", 1)[-1]}
+ candidates |= {c.replace(".", "-") for c in set(candidates)}
+ return any(c in slugs for c in candidates)
+
+ for surface, modes in CATALOG.routing.items():
+ for mode, tiers in modes.items():
+ for tier, cell in tiers.items():
+ where = f"routing[{surface}][{mode}][{tier}] = {cell!r}"
+ assert tolerant_match(cell), f"{where} matches no catalog model"
+ assert not cell.startswith("claude-"), (
+ f"{where}: bare claude-* cells 404 on OpenRouter — use "
+ "the vendor-prefixed dot form (anthropic/claude-…4.6)"
+ )
+ if cell.startswith("anthropic/"):
+ tail = cell.split("/", 1)[1]
+ assert "." in tail, (
+ f"{where}: dash-form anthropic/ cells exist on no "
+ "transport — use the dot form (anthropic/claude-…4.6)"
+ )
diff --git a/autogpt_platform/backend/backend/data/llm_registry/llm_models.py b/autogpt_platform/backend/backend/data/llm_registry/llm_models.py
new file mode 100644
index 0000000000..d9b6c48e31
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/llm_models.py
@@ -0,0 +1,414 @@
+"""Model identity: the ``LLMModel`` enum and its catalog projections.
+
+The catalog file (``catalog.py``) owns every model FACT; this module owns
+the model NAMES — the stable identifiers block schemas serialize — plus the
+projections built from the catalog at import (``MODEL_METADATA``, the
+platform default, rename/alias maps). Lives in ``llm_registry`` so identity
+and facts share one package; ``backend.blocks.llm`` re-exports everything
+for the existing import surface.
+"""
+
+import logging
+import re
+from collections.abc import Mapping
+from enum import Enum, EnumMeta
+from typing import Literal, NamedTuple
+
+from backend.data.llm_registry.catalog import get_catalog
+from backend.data.llm_registry.catalog_model import CatalogPayload
+
+logger = logging.getLogger(__name__)
+
+# Anthropic snapshot-date suffix (claude-haiku-4-5-20251001 → -20251001).
+# Shared by every slug canonicalizer so the pattern can't drift.
+MODEL_DATE_SUFFIX_RE = re.compile(r"-\d{8}$")
+
+
+def transport_slug_candidates(slug: str) -> list[str]:
+ """Spellings a transport-form slug may take in the catalog, in match
+ order: exact, vendor-stripped, dots→dashes. ONE definition shared by
+ every lookup that tolerates transport spellings (the copilot resolver
+ today), so a spelling accepted in one path can't be refused in
+ another. Enum resolution (``_missing_``/aliases) is deliberately
+ separate — enum identity is exact-or-aliased, never fuzzy.
+ """
+ candidates = [slug]
+ if "/" in slug:
+ # ANY vendor prefix (anthropic/, openai/, …) may wrap a catalog
+ # slug that is stored bare — exact match runs first, so slugs
+ # that legitimately carry their prefix (moonshotai/…) still hit
+ # before these fallbacks.
+ tail = slug.split("/", 1)[1]
+ candidates += [tail, tail.replace(".", "-")]
+ elif slug.startswith("claude-"):
+ candidates.append(slug.replace(".", "-"))
+ return candidates
+
+
+class ModelMetadata(NamedTuple):
+ provider: str
+ context_window: int
+ max_output_tokens: int | None
+ display_name: str
+ provider_name: str
+ creator_name: str
+ price_tier: Literal[1, 2, 3]
+
+
+class LLMModelMeta(EnumMeta):
+ pass
+
+
+class LLMModel(str, Enum, metaclass=LLMModelMeta):
+ @classmethod
+ def _missing_(cls, value: object) -> "LLMModel | None":
+ """Resolve provider-prefixed model names.
+
+ Handles two shapes:
+
+ 1. OpenRouter aliases for Anthropic models whose direct-API
+ identifier carries a snapshot suffix that the OpenRouter slug
+ drops — e.g. ``anthropic/claude-haiku-4-5`` ↔ enum value
+ ``claude-haiku-4-5-20251001``. Looked up via
+ ``_OPENROUTER_ALIASES`` (defined below the class so it can hold
+ ``LLMModel`` members directly).
+ 2. Generic provider prefix strip — e.g.
+ ``anthropic/claude-sonnet-4-6`` → ``claude-sonnet-4-6``.
+ """
+ if not isinstance(value, str):
+ return None
+ alias = _OPENROUTER_ALIASES.get(value)
+ if alias is not None:
+ return alias
+ if "/" in value:
+ stripped = value.split("/", 1)[1]
+ try:
+ return cls(stripped)
+ except ValueError:
+ return None
+ return None
+
+ # OpenAI models
+ O3_MINI = "o3-mini"
+ O3 = "o3-2025-04-16"
+ # GPT-5 models
+ GPT5_2 = "gpt-5.2-2025-12-11"
+ GPT5_1 = "gpt-5.1-2025-11-13"
+ GPT5 = "gpt-5-2025-08-07"
+ GPT5_MINI = "gpt-5-mini-2025-08-07"
+ # O-series reasoning models
+ O4_MINI = "o4-mini"
+ O3_PRO = "o3-pro"
+ O1 = "o1"
+ O1_MINI = "o1-mini"
+ # GPT-5.6 models (current flagship, July 2026)
+ GPT5_6_SOL = "gpt-5.6-sol"
+ GPT5_6_TERRA = "gpt-5.6-terra"
+ GPT5_6_LUNA = "gpt-5.6-luna"
+ # GPT-5.5 models
+ GPT5_5 = "gpt-5.5-2026-04-23"
+ GPT5_5_PRO = "gpt-5.5-pro"
+ # GPT-5.4 models (March 2026)
+ GPT5_4 = "gpt-5.4-2026-03-05"
+ GPT5_4_MINI = "gpt-5.4-mini-2026-03-17"
+ GPT5_4_NANO = "gpt-5.4-nano-2026-03-17"
+ GPT5_4_PRO = "gpt-5.4-pro"
+ # GPT-5.3 models
+ GPT5_3 = "gpt-5.3-chat-latest"
+ GPT5_3_CODEX = "gpt-5.3-codex"
+ # Pro/codex variants of existing generations
+ GPT5_2_PRO = "gpt-5.2-pro"
+ GPT5_1_CODEX = "gpt-5.1-codex"
+ GPT5_PRO = "gpt-5-pro"
+ GPT41_NANO = "gpt-4.1-nano"
+ GPT5_NANO = "gpt-5-nano-2025-08-07"
+ GPT5_CHAT = "gpt-5-chat-latest"
+ GPT41 = "gpt-4.1-2025-04-14"
+ GPT41_MINI = "gpt-4.1-mini-2025-04-14"
+ GPT4O_MINI = "gpt-4o-mini"
+ GPT4O = "gpt-4o"
+ # Anthropic models
+ CLAUDE_4_5_OPUS = "claude-opus-4-5-20251101"
+ CLAUDE_4_5_SONNET = "claude-sonnet-4-5-20250929"
+ CLAUDE_4_5_HAIKU = "claude-haiku-4-5-20251001"
+ CLAUDE_4_6_OPUS = "claude-opus-4-6"
+ CLAUDE_4_7_OPUS = "claude-opus-4-7"
+ CLAUDE_4_6_SONNET = "claude-sonnet-4-6"
+ # AI/ML API models
+ AIML_API_LLAMA3_3_70B = "meta-llama/Llama-3.3-70B-Instruct-Turbo"
+ # Groq models
+ LLAMA3_3_70B = "llama-3.3-70b-versatile"
+ LLAMA3_1_8B = "llama-3.1-8b-instant"
+ # Ollama models
+ OLLAMA_LLAMA3_3 = "llama3.3"
+ OLLAMA_LLAMA3_2 = "llama3.2"
+ OLLAMA_LLAMA3_8B = "llama3"
+ OLLAMA_LLAMA3_405B = "llama3.1:405b"
+ OLLAMA_DOLPHIN = "dolphin-mistral:latest"
+ # OpenRouter models
+ OPENAI_GPT_OSS_120B = "openai/gpt-oss-120b"
+ OPENAI_GPT_OSS_20B = "openai/gpt-oss-20b"
+ GEMINI_2_5_PRO = "google/gemini-2.5-pro"
+ GEMINI_3_1_PRO_PREVIEW = "google/gemini-3.1-pro-preview"
+ GEMINI_3_FLASH_PREVIEW = "google/gemini-3-flash-preview"
+ GEMINI_2_5_FLASH = "google/gemini-2.5-flash"
+ GEMINI_2_0_FLASH = "google/gemini-2.0-flash-001"
+ GEMINI_3_1_FLASH_LITE_PREVIEW = "google/gemini-3.1-flash-lite-preview"
+ GEMINI_2_5_FLASH_LITE = "google/gemini-2.5-flash-lite"
+ GEMINI_2_0_FLASH_LITE = "google/gemini-2.0-flash-lite-001"
+ MISTRAL_LARGE_3 = "mistralai/mistral-large-2512"
+ MISTRAL_MEDIUM_3_1 = "mistralai/mistral-medium-3.1"
+ MISTRAL_SMALL_3_2 = "mistralai/mistral-small-3.2-24b-instruct"
+ CODESTRAL = "mistralai/codestral-2508"
+ COHERE_COMMAND_A_03_2025 = "cohere/command-a-03-2025"
+ COHERE_COMMAND_A_TRANSLATE_08_2025 = "cohere/command-a-translate-08-2025"
+ COHERE_COMMAND_A_REASONING_08_2025 = "cohere/command-a-reasoning-08-2025"
+ COHERE_COMMAND_A_VISION_07_2025 = "cohere/command-a-vision-07-2025"
+ DEEPSEEK_CHAT = "deepseek/deepseek-chat" # Actually: DeepSeek V3
+ DEEPSEEK_R1_0528 = "deepseek/deepseek-r1-0528"
+ PERPLEXITY_SONAR = "perplexity/sonar"
+ PERPLEXITY_SONAR_PRO = "perplexity/sonar-pro"
+ PERPLEXITY_SONAR_REASONING_PRO = "perplexity/sonar-reasoning-pro"
+ PERPLEXITY_SONAR_DEEP_RESEARCH = "perplexity/sonar-deep-research"
+ NOUSRESEARCH_HERMES_3_LLAMA_3_1_405B = "nousresearch/hermes-3-llama-3.1-405b"
+ NOUSRESEARCH_HERMES_3_LLAMA_3_1_70B = "nousresearch/hermes-3-llama-3.1-70b"
+ AMAZON_NOVA_LITE_V1 = "amazon/nova-lite-v1"
+ AMAZON_NOVA_MICRO_V1 = "amazon/nova-micro-v1"
+ AMAZON_NOVA_PRO_V1 = "amazon/nova-pro-v1"
+ MICROSOFT_PHI_4 = "microsoft/phi-4"
+ GRYPHE_MYTHOMAX_L2_13B = "gryphe/mythomax-l2-13b"
+ META_LLAMA_4_SCOUT = "meta-llama/llama-4-scout"
+ META_LLAMA_4_MAVERICK = "meta-llama/llama-4-maverick"
+ GROK_3 = "x-ai/grok-3"
+ GROK_4 = "x-ai/grok-4"
+ GROK_4_FAST = "x-ai/grok-4-fast"
+ GROK_4_1_FAST = "x-ai/grok-4.1-fast"
+ GROK_4_20 = "x-ai/grok-4.20"
+ GROK_4_20_MULTI_AGENT = "x-ai/grok-4.20-multi-agent"
+ GROK_CODE_FAST_1 = "x-ai/grok-code-fast-1"
+ KIMI_K2_5 = "moonshotai/kimi-k2.5"
+ KIMI_K2_6 = "moonshotai/kimi-k2.6"
+ KIMI_K2_THINKING = "moonshotai/kimi-k2-thinking"
+ KIMI_K3 = "moonshotai/kimi-k3"
+ QWEN3_235B_A22B_THINKING = "qwen/qwen3-235b-a22b-thinking-2507"
+ QWEN3_CODER = "qwen/qwen3-coder"
+ # Z.ai (Zhipu) models
+ ZAI_GLM_4_6 = "z-ai/glm-4.6"
+ ZAI_GLM_4_6V = "z-ai/glm-4.6v"
+ ZAI_GLM_4_7 = "z-ai/glm-4.7"
+ ZAI_GLM_4_7_FLASH = "z-ai/glm-4.7-flash"
+ ZAI_GLM_5 = "z-ai/glm-5"
+ ZAI_GLM_5_TURBO = "z-ai/glm-5-turbo"
+ ZAI_GLM_5V_TURBO = "z-ai/glm-5v-turbo"
+ # Llama API models
+ LLAMA_API_LLAMA_4_SCOUT = "Llama-4-Scout-17B-16E-Instruct-FP8"
+ LLAMA_API_LLAMA4_MAVERICK = "Llama-4-Maverick-17B-128E-Instruct-FP8"
+ LLAMA_API_LLAMA3_3_8B = "Llama-3.3-8B-Instruct"
+ LLAMA_API_LLAMA3_3_70B = "Llama-3.3-70B-Instruct"
+ # v0 by Vercel models
+ V0_1_5_MD = "v0-1.5-md"
+ V0_1_5_LG = "v0-1.5-lg"
+ V0_1_0_MD = "v0-1.0-md"
+
+ @classmethod
+ def __get_pydantic_json_schema__(cls, schema, handler):
+ json_schema = handler(schema)
+ llm_model_metadata = {}
+ for model in cls:
+ # Kill-switched and non-GA models drop out of the picker
+ # metadata but remain valid enum values — stored graphs
+ # referencing them keep validating and executing.
+ if model.value in _PICKER_HIDDEN_SLUGS:
+ continue
+ model_name = model.value
+ metadata = model.metadata
+ llm_model_metadata[model_name] = {
+ "creator": metadata.creator_name,
+ "creator_name": metadata.creator_name,
+ "title": metadata.display_name,
+ "provider": metadata.provider,
+ "provider_name": metadata.provider_name,
+ "name": model_name,
+ "price_tier": metadata.price_tier,
+ }
+ json_schema["llm_model"] = True
+ json_schema["llm_model_metadata"] = llm_model_metadata
+ return json_schema
+
+ @property
+ def metadata(self) -> ModelMetadata:
+ return MODEL_METADATA[self]
+
+ @property
+ def provider(self) -> str:
+ return self.metadata.provider
+
+ @property
+ def context_window(self) -> int:
+ return self.metadata.context_window
+
+ @property
+ def max_output_tokens(self) -> int | None:
+ return self.metadata.max_output_tokens
+
+
+# OpenRouter exposes Anthropic models under canonical ``anthropic/<model>``
+# slugs that drop the snapshot-date suffix Anthropic's own API uses
+# (``claude-haiku-4-5-20251001`` → ``anthropic/claude-haiku-4-5``). The
+# generic provider-prefix strip in ``_missing_`` can't reverse the date
+# truncation, so map the OpenRouter slugs to ``LLMModel`` members here.
+# Only models whose canonical enum value carries a ``-YYYYMMDD`` snapshot
+# suffix need entries; values without a snapshot (4.6/4.7+) are already
+# covered by the prefix-strip path alone. Stored as ``LLMModel`` instances
+# (not strings) so a rename or snapshot rotation on the enum follows the
+# alias automatically — a stale entry becomes a load-time ``AttributeError``
+# rather than a silent ``_missing_`` miss at runtime.
+_OPENROUTER_ALIASES: Mapping[str, LLMModel] = {
+ "anthropic/claude-haiku-4-5": LLMModel.CLAUDE_4_5_HAIKU,
+ "anthropic/claude-opus-4-5": LLMModel.CLAUDE_4_5_OPUS,
+ "anthropic/claude-sonnet-4-5": LLMModel.CLAUDE_4_5_SONNET,
+ "openai/gpt-5.4": LLMModel.GPT5_4,
+ "openai/gpt-5.4-mini": LLMModel.GPT5_4_MINI,
+ "openai/gpt-5.4-nano": LLMModel.GPT5_4_NANO,
+ "openai/gpt-5.5": LLMModel.GPT5_5,
+}
+
+
+def _build_model_metadata() -> dict["LLMModel", ModelMetadata]:
+ """Project catalog facts into the block-facing metadata shape.
+
+ The catalog file (``backend/data/llm_registry/catalog.py``) is the
+ single source of truth for model facts; this module keeps only the
+ ``LLMModel`` identifiers that block schemas serialize. Catalog models
+ without an enum member (copilot-only models routed by slug) simply
+ don't surface in blocks.
+ """
+ payload = get_catalog()
+ providers = {p.name: p.display_name for p in payload.providers}
+ creators = {c.name: c.display_name for c in payload.creators}
+ members = {m.value: m for m in LLMModel}
+ metadata: dict[LLMModel, ModelMetadata] = {}
+ for model in payload.models:
+ member = members.get(model.slug)
+ if member is None:
+ continue
+ metadata[member] = ModelMetadata(
+ provider=model.provider,
+ context_window=model.context_window,
+ max_output_tokens=model.max_output_tokens,
+ display_name=model.display_name,
+ provider_name=providers.get(model.provider, model.provider),
+ creator_name=(
+ creators.get(model.creator, "Unknown") if model.creator else "Unknown"
+ ),
+ price_tier=model.price_tier,
+ )
+ return metadata
+
+
+MODEL_METADATA = _build_model_metadata()
+
+
+def _picker_hidden_slugs(payload: "CatalogPayload") -> frozenset[str]:
+ """Slugs hidden from the block picker: kill-switched models AND models
+ not yet GA (EMPLOYEES/ADMINS/HIDDEN visibility) — the catalog's
+ documented "who can SEE this" contract. Enum values stay valid either
+ way, so stored graphs referencing a hidden model keep validating and
+ executing.
+
+ DECIDED SCOPE: at the block layer this filter is a UX control, not a
+ safety control. A hand-crafted graph node can still select a
+ kill-switched slug; execution of stored graphs must keep working after
+ a kill (and keep billing), so the block layer deliberately does not
+ veto. The hard-stop for an incident is the retirement CLI, which
+ rewrites the nodes; copilot serving IS vetoed at resolution time.
+ """
+ return frozenset(
+ m.slug for m in payload.models if not m.is_enabled or m.visibility != "GA"
+ )
+
+
+_PICKER_HIDDEN_SLUGS = _picker_hidden_slugs(get_catalog())
+
+
+def _default_model_from_catalog() -> LLMModel:
+ """The platform default IS the catalog's recommended model — one fact,
+ one home. First enabled ``is_recommended`` entry with an enum identifier
+ wins (catalog order); no recommendation is a data error caught at boot.
+
+ The default must also be GA: it is offered to every user and must be
+ picker-selectable, so it has to clear the same ``visibility == "GA"``
+ bar as ``_PICKER_HIDDEN_SLUGS`` — otherwise the default could be a model
+ the picker hides. A non-GA model flagged ``is_recommended`` is skipped
+ here (pre-launch models are not the public default).
+ """
+ members = {m.value for m in LLMModel}
+ first_enabled: LLMModel | None = None
+ for model in get_catalog().models:
+ if (
+ not model.is_enabled
+ or model.visibility != "GA"
+ or model.slug not in members
+ ):
+ continue
+ if model.is_recommended:
+ return LLMModel(model.slug)
+ if first_enabled is None:
+ first_enabled = LLMModel(model.slug)
+ # Killing the recommended model must not crash boot — fall back to the
+ # first enabled block-selectable model (deterministic catalog order).
+ if first_enabled is not None:
+ logger.error(
+ "catalog has no enabled is_recommended model — defaulting to %s",
+ first_enabled.value,
+ )
+ return first_enabled
+ raise ValueError("catalog declares no enabled block-selectable models")
+
+
+DEFAULT_LLM_MODEL = _default_model_from_catalog()
+
+# Family-aware mapping for legacy model values that have been retired from the
+# `LLMModel` enum. Used by both the Prisma migration that rewrites stored graph
+# definitions and by the boot-time safety net (`migrate_llm_models` in
+# backend/data/graph.py) so a Claude Opus user lands on a newer Opus instead of
+# the global GPT default. Keep this in sync with
+# migrations/20260512120000_retire_deprecated_llm_models/migration.sql.
+LEGACY_MODEL_MAPPINGS: dict[str, LLMModel] = {
+ "claude-3-haiku-20240307": LLMModel.CLAUDE_4_5_HAIKU,
+ "claude-opus-4-20250514": LLMModel.CLAUDE_4_7_OPUS,
+ "claude-sonnet-4-20250514": LLMModel.CLAUDE_4_6_SONNET,
+ "claude-opus-4-1-20250805": LLMModel.CLAUDE_4_7_OPUS,
+ "gpt-4-turbo": LLMModel.GPT41,
+ "o1": LLMModel.O3,
+ "o1-mini": LLMModel.O3_MINI,
+ "google/gemini-2.5-pro-preview-03-25": LLMModel.GEMINI_2_5_PRO,
+ "google/gemini-2.5-flash-lite-preview-06-17": LLMModel.GEMINI_2_5_FLASH,
+ "cohere/command-r-08-2024": LLMModel.COHERE_COMMAND_A_03_2025,
+ "cohere/command-r-plus-08-2024": LLMModel.COHERE_COMMAND_A_03_2025,
+ "mistralai/mistral-nemo": LLMModel.MISTRAL_SMALL_3_2,
+ "microsoft/wizardlm-2-8x22b": LLMModel.MICROSOFT_PHI_4,
+ "moonshotai/kimi-k2": LLMModel.KIMI_K2_6,
+ "moonshotai/kimi-k2-0905": LLMModel.KIMI_K2_6,
+ "z-ai/glm-4-32b": LLMModel.ZAI_GLM_4_6,
+ "z-ai/glm-4.5": LLMModel.ZAI_GLM_4_6,
+ "z-ai/glm-4.5-air": LLMModel.ZAI_GLM_4_7_FLASH,
+ "z-ai/glm-4.5-air:free": LLMModel.ZAI_GLM_4_7_FLASH,
+ "z-ai/glm-4.5v": LLMModel.ZAI_GLM_4_6V,
+ # AI/ML API stragglers — no direct same-family successor on AI/ML's current
+ # catalogue, so they all map to the closest open-weight Meta/Llama option
+ # that AI/ML still serves.
+ "Qwen/Qwen2.5-72B-Instruct-Turbo": LLMModel.AIML_API_LLAMA3_3_70B,
+ "nvidia/llama-3.1-nemotron-70b-instruct": LLMModel.AIML_API_LLAMA3_3_70B,
+ "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo": LLMModel.AIML_API_LLAMA3_3_70B,
+ "meta-llama/Llama-3.2-3B-Instruct-Turbo": LLMModel.AIML_API_LLAMA3_3_70B,
+}
+
+
+def _assert_metadata_complete() -> None:
+ for member in LLMModel:
+ if member not in MODEL_METADATA:
+ raise ValueError(f"Missing MODEL_METADATA metadata for model: {member}")
+
+
+_assert_metadata_complete()
diff --git a/autogpt_platform/backend/backend/data/llm_registry/llm_models_test.py b/autogpt_platform/backend/backend/data/llm_registry/llm_models_test.py
new file mode 100644
index 0000000000..d5a740b684
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/llm_models_test.py
@@ -0,0 +1,193 @@
+"""Unit tests for model identity + catalog projections (llm_models.py)."""
+
+from __future__ import annotations
+
+import pydantic
+
+from backend.data.llm_registry import llm_models
+from backend.data.llm_registry.llm_models import DEFAULT_LLM_MODEL, LLMModel
+
+
+def test_default_model_is_the_catalog_recommendation():
+ """DEFAULT_LLM_MODEL derives from the catalog's is_recommended flag —
+ the same fact must not be encoded twice."""
+ assert DEFAULT_LLM_MODEL is LLMModel.GPT5_6_TERRA
+
+
+def _schema_metadata() -> dict:
+ class _M(pydantic.BaseModel):
+ model: LLMModel
+
+ return _M.model_json_schema()["$defs"]["LLMModel"]["llm_model_metadata"]
+
+
+def test_kill_switched_model_hidden_from_picker_metadata(monkeypatch):
+ """is_enabled=False drops a model from the picker metadata while the
+ enum value stays valid, so stored graphs keep validating/executing."""
+ victim = LLMModel.KIMI_K3
+ monkeypatch.setattr(llm_models, "_PICKER_HIDDEN_SLUGS", frozenset({victim.value}))
+ metadata = _schema_metadata()
+ assert victim.value not in metadata
+ assert LLMModel.GPT5_2.value in metadata
+ assert LLMModel(victim.value) is victim
+
+
+def test_all_models_visible_when_none_disabled():
+ metadata = _schema_metadata()
+ assert len(metadata) == len(list(LLMModel))
+
+
+def test_default_model_survives_killing_the_recommended_entry(monkeypatch):
+ """Killing the only is_recommended model must not crash boot — the
+ default falls back to the first enabled block-selectable model."""
+ from backend.data.llm_registry import llm_models
+ from backend.data.llm_registry.catalog import get_catalog
+
+ payload = get_catalog()
+ killed = payload.model_copy(
+ update={
+ "models": [
+ (m.model_copy(update={"is_enabled": False}) if m.is_recommended else m)
+ for m in payload.models
+ ]
+ }
+ )
+ monkeypatch.setattr(llm_models, "get_catalog", lambda: killed)
+ fallback = llm_models._default_model_from_catalog()
+ assert fallback is not None
+ assert not any(m.is_recommended and m.is_enabled for m in killed.models)
+
+
+def test_default_model_skips_non_ga_recommended(monkeypatch):
+ """The default is offered to every user and must be picker-selectable,
+ so it clears the same GA bar as _PICKER_HIDDEN_SLUGS. A model flagged
+ is_recommended but demoted below GA must NOT win the default — otherwise
+ the default would be a model the picker hides."""
+ from backend.data.llm_registry.catalog import get_catalog
+
+ payload = get_catalog()
+ rec_slug = next(m.slug for m in payload.models if m.is_recommended)
+ demoted = payload.model_copy(
+ update={
+ "models": [
+ (
+ m.model_copy(update={"visibility": "EMPLOYEES"})
+ if m.is_recommended
+ else m
+ )
+ for m in payload.models
+ ]
+ }
+ )
+ monkeypatch.setattr(llm_models, "get_catalog", lambda: demoted)
+ default = llm_models._default_model_from_catalog()
+ assert default.value != rec_slug # non-GA recommendation skipped
+ chosen = next(m for m in demoted.models if m.slug == default.value)
+ assert chosen.is_enabled and chosen.visibility == "GA" # picker-selectable
+
+
+def test_non_ga_model_hidden_from_picker_metadata(monkeypatch):
+ """visibility != GA drops a model from picker metadata (the catalog's
+ documented "who can SEE this" contract) while the enum value stays
+ valid for stored graphs."""
+ from backend.data.llm_registry import llm_models
+ from backend.data.llm_registry.llm_models import LLMModel
+
+ victim = LLMModel.KIMI_K3
+ monkeypatch.setattr(llm_models, "_PICKER_HIDDEN_SLUGS", frozenset({victim.value}))
+ metadata = _schema_metadata()
+ assert victim.value not in metadata
+ assert LLMModel(victim.value) is victim
+
+
+class TestAliasResolution:
+ """Stored graphs depend on _missing_'s alias/prefix resolution — the
+ behavior moved with the identity module and keeps direct coverage."""
+
+ def test_openrouter_alias_resolves_date_suffixed_member(self):
+ assert LLMModel("anthropic/claude-haiku-4-5") is LLMModel.CLAUDE_4_5_HAIKU
+
+ def test_generic_vendor_prefix_strips(self):
+ assert LLMModel("anthropic/claude-sonnet-4-6") is LLMModel.CLAUDE_4_6_SONNET
+
+ def test_unknown_slug_raises(self):
+ import pytest
+
+ with pytest.raises(ValueError):
+ LLMModel("someprovider/not-a-model")
+
+
+def test_picker_hidden_derivation_from_catalog_payload():
+ """The real derivation (not a patched frozenset): non-GA and disabled
+ models land in the hidden set; enabled GA models never do. An inverted
+ predicate cannot pass this."""
+ from backend.data.llm_registry.catalog import get_catalog
+ from backend.data.llm_registry.llm_models import _picker_hidden_slugs
+
+ payload = get_catalog()
+ ga, others = payload.models[0], payload.models[1:3]
+ modified = payload.model_copy(
+ update={
+ "models": [
+ ga, # enabled GA — must stay visible
+ others[0].model_copy(update={"visibility": "EMPLOYEES"}),
+ others[1].model_copy(update={"is_enabled": False}),
+ ]
+ }
+ )
+ hidden = _picker_hidden_slugs(modified)
+ assert ga.slug not in hidden
+ assert others[0].slug in hidden
+ assert others[1].slug in hidden
+
+
+def test_non_ga_model_flows_through_derivation_to_hidden_picker(monkeypatch):
+ """End-to-end (not a patched frozenset): a real model marked non-GA lands
+ in the DERIVED hidden set and is therefore absent from picker metadata —
+ chaining _picker_hidden_slugs -> _PICKER_HIDDEN_SLUGS -> schema so an
+ inverted or dropped visibility check would fail here, not just in the
+ isolated derivation test above."""
+ from backend.data.llm_registry.catalog import get_catalog
+ from backend.data.llm_registry.llm_models import _picker_hidden_slugs
+
+ victim = LLMModel.GPT5_2
+ payload = get_catalog()
+ modified = payload.model_copy(
+ update={
+ "models": [
+ (
+ m.model_copy(update={"visibility": "EMPLOYEES"})
+ if m.slug == victim.value
+ else m
+ )
+ for m in payload.models
+ ]
+ }
+ )
+ monkeypatch.setattr(
+ llm_models, "_PICKER_HIDDEN_SLUGS", _picker_hidden_slugs(modified)
+ )
+ metadata = _schema_metadata()
+ assert victim.value not in metadata # non-GA -> hidden, via real derivation
+ assert LLMModel(victim.value) is victim # enum identity unaffected
+
+
+def test_metadata_completeness_guard_fires_on_missing_member(monkeypatch):
+ """_assert_metadata_complete is the boot-time safety net enforcing
+ enum ⊆ catalog (every LLMModel has derived MODEL_METADATA). Prove it
+ actually fires: drop one member's metadata and it must raise — otherwise
+ an enum member with no catalog entry would ship silently."""
+ import pytest
+
+ incomplete = dict(llm_models.MODEL_METADATA)
+ victim = next(iter(incomplete))
+ del incomplete[victim]
+ monkeypatch.setattr(llm_models, "MODEL_METADATA", incomplete)
+ with pytest.raises(ValueError, match="Missing MODEL_METADATA"):
+ llm_models._assert_metadata_complete()
+
+
+def test_metadata_completeness_guard_passes_for_real_catalog():
+ """The shipped enum/catalog pair must satisfy the guard (also enforced at
+ import, asserted here explicitly)."""
+ llm_models._assert_metadata_complete()
diff --git a/autogpt_platform/backend/backend/data/llm_registry/pre_catalog_costs_snapshot.json b/autogpt_platform/backend/backend/data/llm_registry/pre_catalog_costs_snapshot.json
new file mode 100644
index 0000000000..eeae221b90
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/pre_catalog_costs_snapshot.json
@@ -0,0 +1,1170 @@
+{
+ "model_cost": {
+ "Llama-3.3-70B-Instruct": 1,
+ "Llama-3.3-8B-Instruct": 1,
+ "Llama-4-Maverick-17B-128E-Instruct-FP8": 1,
+ "Llama-4-Scout-17B-16E-Instruct-FP8": 1,
+ "amazon/nova-lite-v1": 1,
+ "amazon/nova-micro-v1": 1,
+ "amazon/nova-pro-v1": 1,
+ "claude-haiku-4-5-20251001": 4,
+ "claude-opus-4-5-20251101": 14,
+ "claude-opus-4-6": 14,
+ "claude-opus-4-7": 14,
+ "claude-sonnet-4-5-20250929": 9,
+ "claude-sonnet-4-6": 9,
+ "cohere/command-a-03-2025": 3,
+ "cohere/command-a-reasoning-08-2025": 6,
+ "cohere/command-a-translate-08-2025": 3,
+ "cohere/command-a-vision-07-2025": 3,
+ "deepseek/deepseek-chat": 2,
+ "deepseek/deepseek-r1-0528": 1,
+ "dolphin-mistral:latest": 1,
+ "google/gemini-2.0-flash-001": 1,
+ "google/gemini-2.0-flash-lite-001": 1,
+ "google/gemini-2.5-flash": 1,
+ "google/gemini-2.5-flash-lite": 1,
+ "google/gemini-2.5-pro": 4,
+ "google/gemini-3-flash-preview": 2,
+ "google/gemini-3.1-flash-lite-preview": 1,
+ "google/gemini-3.1-pro-preview": 5,
+ "gpt-4.1-2025-04-14": 2,
+ "gpt-4.1-mini-2025-04-14": 1,
+ "gpt-4o": 3,
+ "gpt-4o-mini": 1,
+ "gpt-5-2025-08-07": 2,
+ "gpt-5-chat-latest": 5,
+ "gpt-5-mini-2025-08-07": 1,
+ "gpt-5-nano-2025-08-07": 1,
+ "gpt-5.1-2025-11-13": 5,
+ "gpt-5.2-2025-12-11": 6,
+ "gryphe/mythomax-l2-13b": 1,
+ "llama-3.1-8b-instant": 1,
+ "llama-3.3-70b-versatile": 1,
+ "llama3": 1,
+ "llama3.1:405b": 1,
+ "llama3.2": 1,
+ "llama3.3": 1,
+ "meta-llama/Llama-3.3-70B-Instruct-Turbo": 1,
+ "meta-llama/llama-4-maverick": 1,
+ "meta-llama/llama-4-scout": 1,
+ "microsoft/phi-4": 1,
+ "mistralai/codestral-2508": 1,
+ "mistralai/mistral-large-2512": 2,
+ "mistralai/mistral-medium-3.1": 2,
+ "mistralai/mistral-small-3.2-24b-instruct": 1,
+ "moonshotai/kimi-k2-thinking": 2,
+ "moonshotai/kimi-k2.5": 1,
+ "moonshotai/kimi-k2.6": 2,
+ "nousresearch/hermes-3-llama-3.1-405b": 1,
+ "nousresearch/hermes-3-llama-3.1-70b": 1,
+ "o3-2025-04-16": 4,
+ "o3-mini": 2,
+ "openai/gpt-oss-120b": 1,
+ "openai/gpt-oss-20b": 1,
+ "perplexity/sonar": 1,
+ "perplexity/sonar-deep-research": 10,
+ "perplexity/sonar-pro": 5,
+ "perplexity/sonar-reasoning-pro": 5,
+ "qwen/qwen3-235b-a22b-thinking-2507": 1,
+ "qwen/qwen3-coder": 9,
+ "v0-1.0-md": 1,
+ "v0-1.5-lg": 2,
+ "v0-1.5-md": 1,
+ "x-ai/grok-3": 3,
+ "x-ai/grok-4": 9,
+ "x-ai/grok-4-fast": 1,
+ "x-ai/grok-4.1-fast": 1,
+ "x-ai/grok-4.20": 5,
+ "x-ai/grok-4.20-multi-agent": 5,
+ "x-ai/grok-code-fast-1": 1,
+ "z-ai/glm-4.6": 1,
+ "z-ai/glm-4.6v": 1,
+ "z-ai/glm-4.7": 1,
+ "z-ai/glm-4.7-flash": 1,
+ "z-ai/glm-5": 2,
+ "z-ai/glm-5-turbo": 4,
+ "z-ai/glm-5v-turbo": 4
+ },
+ "model_metadata": {
+ "Llama-3.3-70B-Instruct": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.3 70B Instruct",
+ "max_output_tokens": 4028,
+ "price_tier": 1,
+ "provider": "llama_api",
+ "provider_name": "Llama API"
+ },
+ "Llama-3.3-8B-Instruct": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.3 8B Instruct",
+ "max_output_tokens": 4028,
+ "price_tier": 1,
+ "provider": "llama_api",
+ "provider_name": "Llama API"
+ },
+ "Llama-4-Maverick-17B-128E-Instruct-FP8": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 4 Maverick 17B 128E Instruct FP8",
+ "max_output_tokens": 4028,
+ "price_tier": 1,
+ "provider": "llama_api",
+ "provider_name": "Llama API"
+ },
+ "Llama-4-Scout-17B-16E-Instruct-FP8": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 4 Scout 17B 16E Instruct FP8",
+ "max_output_tokens": 4028,
+ "price_tier": 1,
+ "provider": "llama_api",
+ "provider_name": "Llama API"
+ },
+ "amazon/nova-lite-v1": {
+ "context_window": 300000,
+ "creator_name": "Amazon",
+ "display_name": "Nova Lite V1",
+ "max_output_tokens": 5120,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "amazon/nova-micro-v1": {
+ "context_window": 128000,
+ "creator_name": "Amazon",
+ "display_name": "Nova Micro V1",
+ "max_output_tokens": 5120,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "amazon/nova-pro-v1": {
+ "context_window": 300000,
+ "creator_name": "Amazon",
+ "display_name": "Nova Pro V1",
+ "max_output_tokens": 5120,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "claude-haiku-4-5-20251001": {
+ "context_window": 200000,
+ "creator_name": "Anthropic",
+ "display_name": "Claude Haiku 4.5",
+ "max_output_tokens": 64000,
+ "price_tier": 2,
+ "provider": "anthropic",
+ "provider_name": "Anthropic"
+ },
+ "claude-opus-4-5-20251101": {
+ "context_window": 200000,
+ "creator_name": "Anthropic",
+ "display_name": "Claude Opus 4.5",
+ "max_output_tokens": 64000,
+ "price_tier": 3,
+ "provider": "anthropic",
+ "provider_name": "Anthropic"
+ },
+ "claude-opus-4-6": {
+ "context_window": 200000,
+ "creator_name": "Anthropic",
+ "display_name": "Claude Opus 4.6",
+ "max_output_tokens": 128000,
+ "price_tier": 3,
+ "provider": "anthropic",
+ "provider_name": "Anthropic"
+ },
+ "claude-opus-4-7": {
+ "context_window": 200000,
+ "creator_name": "Anthropic",
+ "display_name": "Claude Opus 4.7",
+ "max_output_tokens": 128000,
+ "price_tier": 3,
+ "provider": "anthropic",
+ "provider_name": "Anthropic"
+ },
+ "claude-sonnet-4-5-20250929": {
+ "context_window": 200000,
+ "creator_name": "Anthropic",
+ "display_name": "Claude Sonnet 4.5",
+ "max_output_tokens": 64000,
+ "price_tier": 3,
+ "provider": "anthropic",
+ "provider_name": "Anthropic"
+ },
+ "claude-sonnet-4-6": {
+ "context_window": 200000,
+ "creator_name": "Anthropic",
+ "display_name": "Claude Sonnet 4.6",
+ "max_output_tokens": 64000,
+ "price_tier": 3,
+ "provider": "anthropic",
+ "provider_name": "Anthropic"
+ },
+ "cohere/command-a-03-2025": {
+ "context_window": 256000,
+ "creator_name": "Cohere",
+ "display_name": "Command A 03.2025",
+ "max_output_tokens": 8192,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "cohere/command-a-reasoning-08-2025": {
+ "context_window": 256000,
+ "creator_name": "Cohere",
+ "display_name": "Command A Reasoning 08.2025",
+ "max_output_tokens": 32768,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "cohere/command-a-translate-08-2025": {
+ "context_window": 128000,
+ "creator_name": "Cohere",
+ "display_name": "Command A Translate 08.2025",
+ "max_output_tokens": 8192,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "cohere/command-a-vision-07-2025": {
+ "context_window": 128000,
+ "creator_name": "Cohere",
+ "display_name": "Command A Vision 07.2025",
+ "max_output_tokens": 8192,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "deepseek/deepseek-chat": {
+ "context_window": 64000,
+ "creator_name": "DeepSeek",
+ "display_name": "DeepSeek Chat",
+ "max_output_tokens": 2048,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "deepseek/deepseek-r1-0528": {
+ "context_window": 163840,
+ "creator_name": "DeepSeek",
+ "display_name": "DeepSeek R1 0528",
+ "max_output_tokens": 163840,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "dolphin-mistral:latest": {
+ "context_window": 32768,
+ "creator_name": "Mistral AI",
+ "display_name": "Dolphin Mistral Latest",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "ollama",
+ "provider_name": "Ollama"
+ },
+ "google/gemini-2.0-flash-001": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 2.0 Flash 001",
+ "max_output_tokens": 8192,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-2.0-flash-lite-001": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 2.0 Flash Lite 001",
+ "max_output_tokens": 8192,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-2.5-flash": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 2.5 Flash",
+ "max_output_tokens": 65535,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-2.5-flash-lite": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 2.5 Flash Lite",
+ "max_output_tokens": 65535,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-2.5-pro": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 2.5 Pro",
+ "max_output_tokens": 65536,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-3-flash-preview": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 3 Flash Preview",
+ "max_output_tokens": 65536,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-3.1-flash-lite-preview": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 3.1 Flash Lite Preview",
+ "max_output_tokens": 65536,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "google/gemini-3.1-pro-preview": {
+ "context_window": 1048576,
+ "creator_name": "Google",
+ "display_name": "Gemini 3.1 Pro Preview",
+ "max_output_tokens": 65536,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "gpt-4.1-2025-04-14": {
+ "context_window": 1047576,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-4.1",
+ "max_output_tokens": 32768,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-4.1-mini-2025-04-14": {
+ "context_window": 1047576,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-4.1 Mini",
+ "max_output_tokens": 32768,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-4o": {
+ "context_window": 128000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-4o",
+ "max_output_tokens": 16384,
+ "price_tier": 2,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-4o-mini": {
+ "context_window": 128000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-4o Mini",
+ "max_output_tokens": 16384,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-5-2025-08-07": {
+ "context_window": 400000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-5",
+ "max_output_tokens": 128000,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-5-chat-latest": {
+ "context_window": 400000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-5 Chat Latest",
+ "max_output_tokens": 16384,
+ "price_tier": 2,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-5-mini-2025-08-07": {
+ "context_window": 400000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-5 Mini",
+ "max_output_tokens": 128000,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-5-nano-2025-08-07": {
+ "context_window": 400000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-5 Nano",
+ "max_output_tokens": 128000,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-5.1-2025-11-13": {
+ "context_window": 400000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-5.1",
+ "max_output_tokens": 128000,
+ "price_tier": 2,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gpt-5.2-2025-12-11": {
+ "context_window": 400000,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-5.2",
+ "max_output_tokens": 128000,
+ "price_tier": 3,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "gryphe/mythomax-l2-13b": {
+ "context_window": 4096,
+ "creator_name": "Gryphe",
+ "display_name": "MythoMax L2 13B",
+ "max_output_tokens": 4096,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "llama-3.1-8b-instant": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.1 8B Instant",
+ "max_output_tokens": 8192,
+ "price_tier": 1,
+ "provider": "groq",
+ "provider_name": "Groq"
+ },
+ "llama-3.3-70b-versatile": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.3 70B Versatile",
+ "max_output_tokens": 32768,
+ "price_tier": 1,
+ "provider": "groq",
+ "provider_name": "Groq"
+ },
+ "llama3": {
+ "context_window": 8192,
+ "creator_name": "Meta",
+ "display_name": "Llama 3",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "ollama",
+ "provider_name": "Ollama"
+ },
+ "llama3.1:405b": {
+ "context_window": 8192,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.1 405B",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "ollama",
+ "provider_name": "Ollama"
+ },
+ "llama3.2": {
+ "context_window": 8192,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.2",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "ollama",
+ "provider_name": "Ollama"
+ },
+ "llama3.3": {
+ "context_window": 8192,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.3",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "ollama",
+ "provider_name": "Ollama"
+ },
+ "meta-llama/Llama-3.3-70B-Instruct-Turbo": {
+ "context_window": 128000,
+ "creator_name": "Meta",
+ "display_name": "Llama 3.3 70B Instruct Turbo",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "aiml_api",
+ "provider_name": "AI/ML API"
+ },
+ "meta-llama/llama-4-maverick": {
+ "context_window": 1048576,
+ "creator_name": "Meta",
+ "display_name": "Llama 4 Maverick",
+ "max_output_tokens": 1000000,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "meta-llama/llama-4-scout": {
+ "context_window": 131072,
+ "creator_name": "Meta",
+ "display_name": "Llama 4 Scout",
+ "max_output_tokens": 131072,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "microsoft/phi-4": {
+ "context_window": 16384,
+ "creator_name": "Microsoft",
+ "display_name": "Phi-4",
+ "max_output_tokens": 16384,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "mistralai/codestral-2508": {
+ "context_window": 256000,
+ "creator_name": "Mistral AI",
+ "display_name": "Codestral 2508",
+ "max_output_tokens": null,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "mistralai/mistral-large-2512": {
+ "context_window": 262144,
+ "creator_name": "Mistral AI",
+ "display_name": "Mistral Large 3 2512",
+ "max_output_tokens": null,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "mistralai/mistral-medium-3.1": {
+ "context_window": 131072,
+ "creator_name": "Mistral AI",
+ "display_name": "Mistral Medium 3.1",
+ "max_output_tokens": null,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "mistralai/mistral-small-3.2-24b-instruct": {
+ "context_window": 131072,
+ "creator_name": "Mistral AI",
+ "display_name": "Mistral Small 3.2 24B",
+ "max_output_tokens": 131072,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "moonshotai/kimi-k2-thinking": {
+ "context_window": 262144,
+ "creator_name": "Moonshot AI",
+ "display_name": "Kimi K2 Thinking",
+ "max_output_tokens": 262144,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "moonshotai/kimi-k2.5": {
+ "context_window": 262144,
+ "creator_name": "Moonshot AI",
+ "display_name": "Kimi K2.5",
+ "max_output_tokens": 262144,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "moonshotai/kimi-k2.6": {
+ "context_window": 262144,
+ "creator_name": "Moonshot AI",
+ "display_name": "Kimi K2.6",
+ "max_output_tokens": 262144,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "nousresearch/hermes-3-llama-3.1-405b": {
+ "context_window": 131000,
+ "creator_name": "Nous Research",
+ "display_name": "Hermes 3 Llama 3.1 405B",
+ "max_output_tokens": 4096,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "nousresearch/hermes-3-llama-3.1-70b": {
+ "context_window": 12288,
+ "creator_name": "Nous Research",
+ "display_name": "Hermes 3 Llama 3.1 70B",
+ "max_output_tokens": 12288,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "o3-2025-04-16": {
+ "context_window": 200000,
+ "creator_name": "OpenAI",
+ "display_name": "O3",
+ "max_output_tokens": 100000,
+ "price_tier": 2,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "o3-mini": {
+ "context_window": 200000,
+ "creator_name": "OpenAI",
+ "display_name": "O3 Mini",
+ "max_output_tokens": 100000,
+ "price_tier": 1,
+ "provider": "openai",
+ "provider_name": "OpenAI"
+ },
+ "openai/gpt-oss-120b": {
+ "context_window": 131072,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-OSS 120B",
+ "max_output_tokens": 131072,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "openai/gpt-oss-20b": {
+ "context_window": 131072,
+ "creator_name": "OpenAI",
+ "display_name": "GPT-OSS 20B",
+ "max_output_tokens": 32768,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "perplexity/sonar": {
+ "context_window": 127000,
+ "creator_name": "Perplexity",
+ "display_name": "Sonar",
+ "max_output_tokens": 8000,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "perplexity/sonar-deep-research": {
+ "context_window": 128000,
+ "creator_name": "Perplexity",
+ "display_name": "Sonar Deep Research",
+ "max_output_tokens": 16000,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "perplexity/sonar-pro": {
+ "context_window": 200000,
+ "creator_name": "Perplexity",
+ "display_name": "Sonar Pro",
+ "max_output_tokens": 8000,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "perplexity/sonar-reasoning-pro": {
+ "context_window": 128000,
+ "creator_name": "Perplexity",
+ "display_name": "Sonar Reasoning Pro",
+ "max_output_tokens": 8000,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "qwen/qwen3-235b-a22b-thinking-2507": {
+ "context_window": 262144,
+ "creator_name": "Qwen",
+ "display_name": "Qwen 3 235B A22B Thinking 2507",
+ "max_output_tokens": 262144,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "qwen/qwen3-coder": {
+ "context_window": 262144,
+ "creator_name": "Qwen",
+ "display_name": "Qwen 3 Coder",
+ "max_output_tokens": 262144,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "v0-1.0-md": {
+ "context_window": 128000,
+ "creator_name": "v0 by Vercel",
+ "display_name": "v0 1.0 MD",
+ "max_output_tokens": 64000,
+ "price_tier": 1,
+ "provider": "v0",
+ "provider_name": "v0 by Vercel"
+ },
+ "v0-1.5-lg": {
+ "context_window": 512000,
+ "creator_name": "v0 by Vercel",
+ "display_name": "v0 1.5 LG",
+ "max_output_tokens": 64000,
+ "price_tier": 1,
+ "provider": "v0",
+ "provider_name": "v0 by Vercel"
+ },
+ "v0-1.5-md": {
+ "context_window": 128000,
+ "creator_name": "v0 by Vercel",
+ "display_name": "v0 1.5 MD",
+ "max_output_tokens": 64000,
+ "price_tier": 1,
+ "provider": "v0",
+ "provider_name": "v0 by Vercel"
+ },
+ "x-ai/grok-3": {
+ "context_window": 131072,
+ "creator_name": "xAI",
+ "display_name": "Grok 3",
+ "max_output_tokens": 131072,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "x-ai/grok-4": {
+ "context_window": 256000,
+ "creator_name": "xAI",
+ "display_name": "Grok 4",
+ "max_output_tokens": 256000,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "x-ai/grok-4-fast": {
+ "context_window": 2000000,
+ "creator_name": "xAI",
+ "display_name": "Grok 4 Fast",
+ "max_output_tokens": 30000,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "x-ai/grok-4.1-fast": {
+ "context_window": 2000000,
+ "creator_name": "xAI",
+ "display_name": "Grok 4.1 Fast",
+ "max_output_tokens": 30000,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "x-ai/grok-4.20": {
+ "context_window": 2000000,
+ "creator_name": "xAI",
+ "display_name": "Grok 4.20",
+ "max_output_tokens": 100000,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "x-ai/grok-4.20-multi-agent": {
+ "context_window": 2000000,
+ "creator_name": "xAI",
+ "display_name": "Grok 4.20 Multi-Agent",
+ "max_output_tokens": 100000,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "x-ai/grok-code-fast-1": {
+ "context_window": 256000,
+ "creator_name": "xAI",
+ "display_name": "Grok Code Fast 1",
+ "max_output_tokens": 10000,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-4.6": {
+ "context_window": 204800,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 4.6",
+ "max_output_tokens": 204800,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-4.6v": {
+ "context_window": 131072,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 4.6V",
+ "max_output_tokens": 131072,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-4.7": {
+ "context_window": 202752,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 4.7",
+ "max_output_tokens": 65535,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-4.7-flash": {
+ "context_window": 202752,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 4.7 Flash",
+ "max_output_tokens": 202752,
+ "price_tier": 1,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-5": {
+ "context_window": 80000,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 5",
+ "max_output_tokens": 80000,
+ "price_tier": 2,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-5-turbo": {
+ "context_window": 202752,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 5 Turbo",
+ "max_output_tokens": 131072,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ },
+ "z-ai/glm-5v-turbo": {
+ "context_window": 202752,
+ "creator_name": "Z.ai",
+ "display_name": "GLM 5V Turbo",
+ "max_output_tokens": 131072,
+ "price_tier": 3,
+ "provider": "open_router",
+ "provider_name": "OpenRouter"
+ }
+ },
+ "token_cost": {
+ "claude-haiku-4-5-20251001": {
+ "cache_creation": 188.0,
+ "cache_read": 15.0,
+ "input": 150.0,
+ "output": 750.0
+ },
+ "claude-opus-4-5-20251101": {
+ "cache_creation": 938.0,
+ "cache_read": 75.0,
+ "input": 750.0,
+ "output": 3750.0
+ },
+ "claude-opus-4-6": {
+ "cache_creation": 938.0,
+ "cache_read": 75.0,
+ "input": 750.0,
+ "output": 3750.0
+ },
+ "claude-opus-4-7": {
+ "cache_creation": 938.0,
+ "cache_read": 75.0,
+ "input": 750.0,
+ "output": 3750.0
+ },
+ "claude-sonnet-4-5-20250929": {
+ "cache_creation": 563.0,
+ "cache_read": 45.0,
+ "input": 450.0,
+ "output": 2250.0
+ },
+ "claude-sonnet-4-6": {
+ "cache_creation": 563.0,
+ "cache_read": 45.0,
+ "input": 450.0,
+ "output": 2250.0
+ },
+ "cohere/command-a-03-2025": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 375.0,
+ "output": 1500.0
+ },
+ "deepseek/deepseek-chat": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 21.0,
+ "output": 42.0
+ },
+ "deepseek/deepseek-r1-0528": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 21.0,
+ "output": 42.0
+ },
+ "google/gemini-2.0-flash-001": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 15.0,
+ "output": 60.0
+ },
+ "google/gemini-2.0-flash-lite-001": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 11.0,
+ "output": 45.0
+ },
+ "google/gemini-2.5-flash": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 45.0,
+ "output": 375.0
+ },
+ "google/gemini-2.5-flash-lite": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 15.0,
+ "output": 60.0
+ },
+ "google/gemini-2.5-pro": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 188.0,
+ "output": 1500.0
+ },
+ "google/gemini-3-flash-preview": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 75.0,
+ "output": 450.0
+ },
+ "google/gemini-3.1-flash-lite-preview": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 38.0,
+ "output": 225.0
+ },
+ "google/gemini-3.1-pro-preview": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 1800.0
+ },
+ "gpt-4.1-2025-04-14": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 1200.0
+ },
+ "gpt-4.1-mini-2025-04-14": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 60.0,
+ "output": 240.0
+ },
+ "gpt-4o": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 375.0,
+ "output": 1500.0
+ },
+ "gpt-4o-mini": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 23.0,
+ "output": 90.0
+ },
+ "gpt-5-2025-08-07": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 188.0,
+ "output": 1500.0
+ },
+ "gpt-5-chat-latest": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 188.0,
+ "output": 1500.0
+ },
+ "gpt-5-mini-2025-08-07": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 38.0,
+ "output": 300.0
+ },
+ "gpt-5-nano-2025-08-07": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 8.0,
+ "output": 60.0
+ },
+ "gpt-5.1-2025-11-13": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 188.0,
+ "output": 1500.0
+ },
+ "gpt-5.2-2025-12-11": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 263.0,
+ "output": 2100.0
+ },
+ "llama-3.1-8b-instant": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 8.0,
+ "output": 12.0
+ },
+ "llama-3.3-70b-versatile": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 89.0,
+ "output": 119.0
+ },
+ "meta-llama/llama-4-maverick": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 75.0,
+ "output": 116.0
+ },
+ "meta-llama/llama-4-scout": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 17.0,
+ "output": 51.0
+ },
+ "mistralai/codestral-2508": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 45.0,
+ "output": 135.0
+ },
+ "mistralai/mistral-large-2512": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 900.0
+ },
+ "mistralai/mistral-medium-3.1": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 60.0,
+ "output": 300.0
+ },
+ "mistralai/mistral-small-3.2-24b-instruct": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 15.0,
+ "output": 45.0
+ },
+ "moonshotai/kimi-k2-thinking": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 90.0,
+ "output": 375.0
+ },
+ "moonshotai/kimi-k2.5": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 66.0,
+ "output": 300.0
+ },
+ "moonshotai/kimi-k2.6": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 112.0,
+ "output": 698.0
+ },
+ "o3-2025-04-16": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 1200.0
+ },
+ "o3-mini": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 165.0,
+ "output": 660.0
+ },
+ "openai/gpt-oss-120b": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 23.0,
+ "output": 90.0
+ },
+ "openai/gpt-oss-20b": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 11.0,
+ "output": 45.0
+ },
+ "perplexity/sonar": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 150.0,
+ "output": 150.0
+ },
+ "perplexity/sonar-deep-research": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 1200.0
+ },
+ "perplexity/sonar-pro": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 450.0,
+ "output": 2250.0
+ },
+ "perplexity/sonar-reasoning-pro": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 1200.0
+ },
+ "x-ai/grok-3": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 450.0,
+ "output": 2250.0
+ },
+ "x-ai/grok-4": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 450.0,
+ "output": 2250.0
+ },
+ "x-ai/grok-4-fast": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 30.0,
+ "output": 75.0
+ },
+ "x-ai/grok-4.1-fast": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 30.0,
+ "output": 75.0
+ },
+ "x-ai/grok-4.20": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 300.0,
+ "output": 900.0
+ },
+ "x-ai/grok-code-fast-1": {
+ "cache_creation": 0.0,
+ "cache_read": 0.0,
+ "input": 30.0,
+ "output": 225.0
+ }
+ }
+}
diff --git a/autogpt_platform/backend/backend/data/llm_registry/registry.py b/autogpt_platform/backend/backend/data/llm_registry/registry.py
new file mode 100644
index 0000000000..57a01ad458
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/registry.py
@@ -0,0 +1,202 @@
+"""In-process view of the LLM catalog.
+
+``load_catalog()`` builds the L1 lookup structures from the canonical
+``catalog.py`` file at startup; copilot routing reads through the
+functions here, while the block layer projects the same file directly
+(``backend.blocks.llm`` / ``block_cost_config``).
+The file only changes at deploy, so a startup load is the whole cache
+story — no Redis layer, no cross-pod invalidation.
+
+The read interface is deliberately stable: it survived the DB-registry →
+catalog-as-code redesign unchanged so consumers never care where model
+facts live.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Literal
+
+from pydantic import BaseModel, ConfigDict
+
+from backend.data.llm_registry.catalog import get_catalog
+from backend.data.llm_registry.catalog_model import (
+ CatalogModelCost,
+ CatalogPayload,
+ ModelVisibility,
+)
+from backend.data.llm_registry.llm_models import MODEL_DATE_SUFFIX_RE
+
+logger = logging.getLogger(__name__)
+
+
+class RegistryModelMetadata(BaseModel):
+ """Model facts in the shape block schemas expect.
+
+ Field-compatible with ``llm_models.ModelMetadata`` by design — the two
+ shapes must not drift (one is the block-facing NamedTuple projection,
+ this is the router-facing pydantic view of the same catalog facts).
+
+ Standalone rather than reusing the ``ModelMetadata`` NamedTuple from
+ ``backend.blocks.llm`` — the registry must not couple to the hardcoded
+ enum module it will eventually replace, and NamedTuples serialize as
+ JSON arrays, which is wrong for API payloads.
+ """
+
+ model_config = ConfigDict(frozen=True)
+
+ provider: str
+ context_window: int
+ max_output_tokens: int | None
+ display_name: str
+ provider_name: str
+ creator_name: str
+ price_tier: Literal[1, 2, 3]
+
+
+class RegistryModel(BaseModel):
+ """A catalog model joined with its provider/creator display data."""
+
+ model_config = ConfigDict(frozen=True)
+
+ slug: str
+ display_name: str
+ description: str | None = None
+ metadata: RegistryModelMetadata
+ provider_display_name: str
+ # is_enabled is the kill switch for SERVING NEW WORK: copilot refuses the
+ # model and it leaves picker metadata. Existing agent graphs keep
+ # executing (and billing) — hard-stopping users' running agents is a
+ # deliberate separate act (the retire CLI), never a flag side-effect.
+ is_enabled: bool
+ is_recommended: bool = False
+
+ # visibility only controls who SEES the model — HIDDEN still serves
+ # when explicitly routed.
+ visibility: ModelVisibility = "GA"
+ fallback_model_slug: str | None = None
+
+ supports_tools: bool = False
+ supports_json_output: bool = False
+ supports_reasoning: bool = False
+ supports_parallel_tool_calls: bool = False
+
+ cost: CatalogModelCost | None = None
+
+
+# L1 lookup structures — built once per process by load_catalog(). Reference
+# swaps are atomic, and the file can't change under a running process, so no
+# locking is needed.
+_dynamic_models: dict[str, RegistryModel] = {}
+# Date-stripped slug → model (claude-haiku-4-5 → the -20251001 entry), so
+# the router's snapshot-suffix fallback is an O(1) lookup, not a scan.
+_date_stripped_models: dict[str, RegistryModel] = {}
+_routes: dict[tuple[str, str, str], str] = {}
+_loaded = False
+
+
+def _build_models(payload: CatalogPayload) -> dict[str, RegistryModel]:
+ providers = {p.name: p for p in payload.providers}
+ creators = {c.name: c for c in payload.creators}
+ models: dict[str, RegistryModel] = {}
+ for m in payload.models:
+ provider = providers.get(m.provider)
+ provider_display = provider.display_name if provider else m.provider
+ creator = creators.get(m.creator) if m.creator else None
+ models[m.slug] = RegistryModel(
+ slug=m.slug,
+ display_name=m.display_name,
+ description=m.description,
+ metadata=RegistryModelMetadata(
+ provider=m.provider,
+ context_window=m.context_window,
+ # None means "unknown/no published cap" — substituting
+ # context_window would overstate the output limit (a 1M-token
+ # context model does not emit 1M output tokens).
+ max_output_tokens=m.max_output_tokens,
+ display_name=m.display_name,
+ provider_name=provider_display,
+ creator_name=creator.display_name if creator else "Unknown",
+ price_tier=m.price_tier,
+ ),
+ provider_display_name=provider_display,
+ is_enabled=m.is_enabled,
+ is_recommended=m.is_recommended,
+ visibility=m.visibility,
+ fallback_model_slug=m.fallback_model_slug,
+ supports_tools=m.supports_tools,
+ supports_json_output=m.supports_json_output,
+ supports_reasoning=m.supports_reasoning,
+ supports_parallel_tool_calls=m.supports_parallel_tool_calls,
+ cost=m.cost,
+ )
+ return models
+
+
+def load_catalog(payload: CatalogPayload | None = None) -> None:
+ """Build the L1 structures from the catalog file. Called at startup.
+
+ Raises on an inconsistent payload — the catalog is load-bearing, so
+ callers let the failure stop the boot rather than serving with an
+ empty registry (which would silently disable routing cells and
+ serve-time gating).
+ """
+ global _dynamic_models, _date_stripped_models, _routes, _loaded
+ if payload is None:
+ payload = get_catalog()
+ models = _build_models(payload)
+ routes = {
+ (surface, mode, tier): slug
+ for surface, modes in payload.routing.items()
+ for mode, tiers in modes.items()
+ for tier, slug in tiers.items()
+ }
+ _dynamic_models = models
+ _date_stripped_models = {
+ stripped: model
+ for slug, model in models.items()
+ if (stripped := MODEL_DATE_SUFFIX_RE.sub("", slug)) != slug
+ }
+ _routes = routes
+ _loaded = True
+ logger.info(
+ f"LLM catalog loaded: {len(models)} models, {len(routes)} routing cells"
+ )
+
+
+def has_models() -> bool:
+ """O(1) emptiness check — the resolver asks this on every turn."""
+ return bool(_dynamic_models)
+
+
+def is_loaded() -> bool:
+ """Whether load_catalog() has run in this process — distinguishes a
+ deliberately-dormant registry from an entrypoint that forgot to load
+ (the latter silently disables copilot gating and routing cells)."""
+ return _loaded
+
+
+def get_model(slug: str) -> RegistryModel | None:
+ """Get a model by slug from the catalog."""
+ return _dynamic_models.get(slug)
+
+
+def get_model_by_date_stripped_slug(slug: str) -> RegistryModel | None:
+ """Resolve a date-suffix-less spelling to its snapshot-suffixed catalog
+ entry (``claude-haiku-4-5`` → the ``-20251001`` model)."""
+ return _date_stripped_models.get(slug)
+
+
+def get_all_models() -> list[RegistryModel]:
+ """All catalog models, including disabled ones."""
+ return list(_dynamic_models.values())
+
+
+def get_route(surface: str, mode: str, tier: str) -> str | None:
+ """Return the catalog's routing-cell slug, if the cell is set.
+
+ Pure L1 lookup — callers own validation of the returned slug (existence
+ and is_enabled are checked by the resolver so a stale cell degrades to
+ the next routing layer instead of serving a dead model).
+ """
+ return _routes.get((surface, mode, tier))
diff --git a/autogpt_platform/backend/backend/data/llm_registry/registry_test.py b/autogpt_platform/backend/backend/data/llm_registry/registry_test.py
new file mode 100644
index 0000000000..3c43bbf99b
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/registry_test.py
@@ -0,0 +1,138 @@
+"""Unit tests for the in-process catalog view (registry.py)."""
+
+from __future__ import annotations
+
+from datetime import datetime, timezone
+
+import pytest
+
+from backend.data.llm_registry.catalog import get_catalog
+from backend.data.llm_registry.catalog_model import (
+ CATALOG_SCHEMA_VERSION,
+ CatalogCreator,
+ CatalogModel,
+ CatalogModelCost,
+ CatalogPayload,
+ CatalogProvider,
+)
+from backend.data.llm_registry.registry import (
+ RegistryModel,
+ get_all_models,
+ get_model,
+ get_route,
+ load_catalog,
+)
+
+CATALOG = get_catalog()
+
+
+def _payload(models: list[CatalogModel], routing=None) -> CatalogPayload:
+ return CatalogPayload(
+ schema_version=CATALOG_SCHEMA_VERSION,
+ generated_at=datetime(2026, 7, 18, tzinfo=timezone.utc),
+ providers=[CatalogProvider(name="openai", display_name="OpenAI")],
+ creators=[CatalogCreator(name="openai", display_name="OpenAI")],
+ models=models,
+ routing=routing or {},
+ )
+
+
+def _model(slug: str, **overrides) -> CatalogModel:
+ kwargs = dict(
+ slug=slug,
+ display_name=slug.split("/")[-1].upper(),
+ provider="openai",
+ creator="openai",
+ context_window=128000,
+ max_output_tokens=16384,
+ price_tier=2,
+ )
+ kwargs.update(overrides)
+ return CatalogModel(**kwargs)
+
+
[email protected](autouse=True)
+def isolated_l1():
+ """Each test loads its own payload; restore the real catalog after."""
+ yield
+ load_catalog(CATALOG)
+
+
+def test_load_catalog_builds_l1_from_payload():
+ load_catalog(
+ _payload(
+ [_model("openai/gpt-a"), _model("openai/gpt-b", is_enabled=False)],
+ routing={"copilot": {"thinking": {"standard": "openai/gpt-a"}}},
+ )
+ )
+
+ assert {m.slug for m in get_all_models()} == {"openai/gpt-a", "openai/gpt-b"}
+ enabled = [m.slug for m in get_all_models() if m.is_enabled]
+ assert enabled == ["openai/gpt-a"]
+ assert get_route("copilot", "thinking", "standard") == "openai/gpt-a"
+ assert get_route("copilot", "fast", "standard") is None
+
+
+def test_registry_model_carries_joined_display_data():
+ load_catalog(
+ _payload(
+ [
+ _model(
+ "openai/gpt-a",
+ cost=CatalogModelCost(run_credits=3, input_credits_per_1m=100),
+ )
+ ]
+ )
+ )
+ model = get_model("openai/gpt-a")
+
+ assert isinstance(model, RegistryModel)
+ assert model.provider_display_name == "OpenAI"
+ assert model.metadata.provider == "openai"
+ assert model.metadata.price_tier == 2
+ assert model.cost is not None and model.cost.run_credits == 3
+
+
+def test_null_max_output_tokens_is_preserved():
+ """None means "unknown/no published cap" — substituting context_window
+ would overstate the limit and publish wrong data through the catalog
+ endpoint."""
+ load_catalog(_payload([_model("openai/gpt-a", max_output_tokens=None)]))
+ model = get_model("openai/gpt-a")
+ assert model is not None
+ assert model.metadata.max_output_tokens is None
+
+
+def test_unknown_provider_falls_back_to_name():
+ payload = _payload([_model("other/x", provider="other")])
+ load_catalog(payload)
+ model = get_model("other/x")
+ assert model is not None
+ assert model.provider_display_name == "other"
+ assert model.metadata.creator_name == "OpenAI"
+
+
+def test_get_model_not_found():
+ load_catalog(_payload([]))
+ assert get_model("nonexistent/model") is None
+
+
+def test_real_catalog_loads():
+ """The shipped catalog file builds a complete L1."""
+ load_catalog(CATALOG)
+
+ assert len(get_all_models()) > 50
+ assert any(m.is_recommended and m.is_enabled for m in get_all_models())
+ # Cells ship empty (env stays authoritative until a cell is claimed) —
+ # populated-cell behavior is covered by the seeded-payload tests above.
+ assert get_route("copilot", "thinking", "standard") is None
+
+
+def test_registry_metadata_stays_field_compatible_with_block_shape():
+ """RegistryModelMetadata (router view) and ModelMetadata (block
+ projection) are documented as field-compatible by design — enforce it
+ so adding a field to one without the other fails here, not in prod."""
+ from backend.data.llm_registry.llm_models import ModelMetadata
+ from backend.data.llm_registry.registry import RegistryModelMetadata
+
+ assert set(RegistryModelMetadata.model_fields) == set(ModelMetadata._fields)
diff --git a/autogpt_platform/backend/backend/data/llm_registry/retire.py b/autogpt_platform/backend/backend/data/llm_registry/retire.py
new file mode 100644
index 0000000000..bf7567836b
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/retire.py
@@ -0,0 +1,378 @@
+"""Retire catalog models: migrate AgentNodes off a model, revertably.
+
+Retirement is about GRAPH NODES — it does not edit the catalog file. The
+operator flips the model's ``enabled: False`` in a catalog PR separately;
+this module rewrites existing workflow references and records the operation
+in ``LlmModelMigration`` (per-install runtime state) so it can be reverted.
+
+CLI::
+
+ python -m backend.data.llm_registry.retire --usage <slug>
+ python -m backend.data.llm_registry.retire <slug> --replacement <slug> \
+ [--reason "..."] [--yes]
+ python -m backend.data.llm_registry.retire --revert <migration-id>
+ python -m backend.data.llm_registry.retire --list [--include-reverted]
+
+Without ``--yes`` the retire command prints a dry-run summary (usage count,
+replacement validation) and exits non-zero so scripts can't retire by
+accident.
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import logging
+import sys
+from datetime import datetime, timezone
+from typing import Any, LiteralString, cast
+
+import prisma.models
+from prisma.errors import UniqueViolationError
+from pydantic import BaseModel
+
+import backend.data.db
+from backend.data.db import get_database_schema, query_raw_with_schema, transaction
+from backend.data.llm_registry.catalog import get_catalog
+from backend.data.llm_registry.registry import get_model, load_catalog
+
+logger = logging.getLogger(__name__)
+
+
+class MigrationResult(BaseModel):
+ source_model_slug: str
+ target_model_slug: str
+ nodes_migrated: int
+ migration_id: str | None # None when no nodes referenced the model
+
+
+class RevertResult(BaseModel):
+ migration_id: str
+ source_model_slug: str
+ target_model_slug: str
+ nodes_reverted: int
+ # Nodes that were manually changed to a third model after the migration
+ # are left alone (the revert UPDATE is guarded on the migrated value).
+ nodes_already_changed: int
+
+
+def _schema_format(query_template: str) -> LiteralString:
+ """Format a ``{schema_prefix}`` query for transaction-scoped raw SQL.
+
+ ``query_raw_with_schema``/``execute_raw_with_schema`` can't run inside an
+ existing transaction client, so FOR UPDATE reads format the template with
+ the same prefix logic and go through ``tx.query_raw``.
+ """
+ schema = get_database_schema()
+ schema_prefix = f'"{schema}".' if schema != "public" else ""
+ # cast: prisma types raw queries as LiteralString; the template is a
+ # module-level literal and the prefix derives from DATABASE_URL config.
+ return cast(LiteralString, query_template.format(schema_prefix=schema_prefix))
+
+
+# AgentNode.constantInput stores FULL enum values — mixed bare and
+# provider-prefixed forms (``claude-opus-4-7``, ``moonshotai/kimi-k2.5``),
+# exactly the catalog slugs; graph.migrate_llm_models compares against
+# ``LLMModel.value`` unmodified. Node values therefore map to catalog slugs
+# by IDENTITY — stripping the provider prefix (as an earlier revision did)
+# both misses prefixed models and writes out-of-enum values that the
+# startup migration would stomp to the global fallback.
+
+
+def _validate_replacement(slug: str) -> None:
+ """Replacement must be a known, enabled catalog model (kill-switch aware)."""
+ model = get_model(slug)
+ if model is None:
+ raise ValueError(f"Replacement model '{slug}' is not in the catalog")
+ if not model.is_enabled:
+ raise ValueError(
+ f"Replacement model '{slug}' is disabled in the catalog — "
+ f"retiring onto a dead model would strand the nodes again"
+ )
+
+
+async def count_model_usage(slug: str) -> int:
+ """How many AgentNodes currently reference *slug*."""
+ result = await query_raw_with_schema(
+ """
+ SELECT COUNT(*) as count
+ FROM {schema_prefix}"AgentNode"
+ WHERE "constantInput"::jsonb->>'model' = $1
+ """,
+ slug,
+ )
+ return int(result[0]["count"]) if result else 0
+
+
+async def _migrate_nodes_in_tx(tx, source_value: str, target_value: str) -> list[str]:
+ """Lock + rewrite AgentNode model references. Returns migrated node ids."""
+ node_ids_result = await tx.query_raw(
+ _schema_format(
+ """
+ SELECT id
+ FROM {schema_prefix}"AgentNode"
+ WHERE "constantInput"::jsonb->>'model' = $1
+ FOR UPDATE
+ """
+ ),
+ source_value,
+ )
+ migrated_node_ids = [row["id"] for row in node_ids_result or []]
+ if migrated_node_ids:
+ await tx.execute_raw(
+ _schema_format(
+ """
+ UPDATE {schema_prefix}"AgentNode"
+ SET "constantInput" = JSONB_SET(
+ "constantInput"::jsonb,
+ '{{model}}',
+ to_jsonb($1::text)
+ )
+ WHERE id::text IN (
+ SELECT jsonb_array_elements_text($2::jsonb)
+ )
+ """
+ ),
+ target_value,
+ json.dumps(migrated_node_ids),
+ )
+ return migrated_node_ids
+
+
+async def retire_model(
+ slug: str, replacement_slug: str, reason: str | None = None
+) -> MigrationResult:
+ """Rewrite every AgentNode referencing *slug* to *replacement_slug*.
+
+ Records an ``LlmModelMigration`` row when any nodes were touched; the
+ partial unique index rejects a second active migration for the same
+ source (revert the first one before retrying).
+ """
+ if slug == replacement_slug:
+ raise ValueError("Replacement must differ from the retired model")
+ _validate_replacement(replacement_slug)
+ source = get_model(slug)
+ if source is not None and source.is_enabled:
+ logger.warning(
+ "Model '%s' is still enabled in the catalog — remember to flip "
+ "enabled: False in a catalog PR, or new graphs will keep "
+ "selecting it",
+ slug,
+ )
+
+ try:
+ async with transaction() as tx:
+ migrated_node_ids = await _migrate_nodes_in_tx(tx, slug, replacement_slug)
+ migration_id: str | None = None
+ if migrated_node_ids:
+ record = await tx.llmmodelmigration.create(
+ data={
+ "sourceModelSlug": slug,
+ "targetModelSlug": replacement_slug,
+ "reason": reason,
+ "migratedNodeIds": json.dumps(migrated_node_ids),
+ "nodeCount": len(migrated_node_ids),
+ }
+ )
+ migration_id = record.id
+ except UniqueViolationError as exc:
+ raise ValueError(
+ f"An active migration for '{slug}' already exists — revert it "
+ f"first (see --list)"
+ ) from exc
+
+ return MigrationResult(
+ source_model_slug=slug,
+ target_model_slug=replacement_slug,
+ nodes_migrated=len(migrated_node_ids),
+ migration_id=migration_id,
+ )
+
+
+async def revert_model_migration(migration_id: str) -> RevertResult:
+ """Restore nodes touched by a migration to the original model."""
+ migration = await prisma.models.LlmModelMigration.prisma().find_unique(
+ where={"id": migration_id}
+ )
+ if not migration:
+ raise ValueError(f"Migration '{migration_id}' not found")
+ if migration.isReverted:
+ raise ValueError(f"Migration '{migration_id}' has already been reverted")
+
+ migrated_node_ids: list[str] = (
+ migration.migratedNodeIds
+ if isinstance(migration.migratedNodeIds, list)
+ else json.loads(str(migration.migratedNodeIds))
+ )
+ if not migrated_node_ids:
+ raise ValueError("No nodes to revert in this migration")
+
+ async with transaction() as tx:
+ # Atomic claim: the guarded update IS the lock — a concurrent revert
+ # of the same migration matches zero rows and aborts here without
+ # touching any nodes (fixes the check-then-act race on isReverted).
+ claimed = await tx.llmmodelmigration.update_many(
+ where={"id": migration_id, "isReverted": False},
+ data={"isReverted": True, "revertedAt": datetime.now(timezone.utc)},
+ )
+ if claimed == 0:
+ raise ValueError(f"Migration '{migration_id}' has already been reverted")
+ # Guarded on the migrated value: nodes manually repointed at a third
+ # model since the migration are left alone.
+ result = await tx.execute_raw(
+ _schema_format(
+ """
+ UPDATE {schema_prefix}"AgentNode"
+ SET "constantInput" = JSONB_SET(
+ "constantInput"::jsonb,
+ '{{model}}',
+ to_jsonb($1::text)
+ )
+ WHERE id::text IN (
+ SELECT jsonb_array_elements_text($2::jsonb)
+ )
+ AND "constantInput"::jsonb->>'model' = $3
+ """
+ ),
+ migration.sourceModelSlug,
+ json.dumps(migrated_node_ids),
+ migration.targetModelSlug,
+ )
+ nodes_reverted = result if isinstance(result, int) else 0
+
+ if get_model(migration.sourceModelSlug) is None:
+ logger.warning(
+ "Reverted nodes now reference '%s', which is not in the catalog — "
+ "re-add it in a catalog PR if this revert is meant to be permanent",
+ migration.sourceModelSlug,
+ )
+
+ return RevertResult(
+ migration_id=migration_id,
+ source_model_slug=migration.sourceModelSlug,
+ target_model_slug=migration.targetModelSlug,
+ nodes_reverted=nodes_reverted,
+ nodes_already_changed=len(migrated_node_ids) - nodes_reverted,
+ )
+
+
+class MigrationRow(BaseModel):
+ """One retirement record, as listed by the CLI."""
+
+ id: str
+ source_model_slug: str
+ target_model_slug: str
+ reason: str | None
+ node_count: int
+ is_reverted: bool
+ reverted_at: str | None
+ created_at: str
+
+
+async def list_model_migrations(include_reverted: bool = False) -> list[MigrationRow]:
+ """Recent migrations, newest first."""
+ where: Any = None if include_reverted else {"isReverted": False}
+ records = await prisma.models.LlmModelMigration.prisma().find_many(
+ where=where, order={"createdAt": "desc"}
+ )
+ return [
+ MigrationRow(
+ id=r.id,
+ source_model_slug=r.sourceModelSlug,
+ target_model_slug=r.targetModelSlug,
+ reason=r.reason,
+ node_count=r.nodeCount,
+ is_reverted=r.isReverted,
+ reverted_at=r.revertedAt.isoformat() if r.revertedAt else None,
+ created_at=r.createdAt.isoformat(),
+ )
+ for r in records
+ ]
+
+
+def _build_parser() -> argparse.ArgumentParser:
+ parser = argparse.ArgumentParser(
+ prog="python -m backend.data.llm_registry.retire",
+ description=__doc__,
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ )
+ parser.add_argument("slug", nargs="?", help="model slug to retire")
+ parser.add_argument(
+ "--replacement",
+ help="catalog slug nodes migrate to (defaults to the retired model's "
+ "catalog fallback_model_slug when set)",
+ )
+ parser.add_argument("--reason", help="recorded on the migration row")
+ parser.add_argument("--yes", action="store_true", help="execute (no dry-run)")
+ parser.add_argument("--usage", metavar="SLUG", help="print node count and exit")
+ parser.add_argument("--revert", metavar="MIGRATION_ID", help="revert a migration")
+ parser.add_argument("--list", action="store_true", help="list migrations")
+ parser.add_argument("--include-reverted", action="store_true")
+ return parser
+
+
+async def _run_cli(args: argparse.Namespace) -> int:
+ # One action per invocation — a retire positional combined with
+ # --usage/--revert/--list would silently ignore the positional.
+ modes = [bool(args.slug), bool(args.usage), bool(args.revert), args.list]
+ if sum(modes) != 1:
+ print(
+ "Pick exactly one action: <slug> --replacement …, --usage, "
+ "--revert, or --list",
+ file=sys.stderr,
+ )
+ return 2
+
+ await backend.data.db.connect()
+ load_catalog()
+
+ if args.usage:
+ print(f"{args.usage}: {await count_model_usage(args.usage)} node(s)")
+ return 0
+ if args.revert:
+ result = await revert_model_migration(args.revert)
+ print(result.model_dump_json(indent=2))
+ return 0
+ if args.list:
+ for row in await list_model_migrations(args.include_reverted):
+ print(row.model_dump_json())
+ return 0
+ if args.slug and not args.replacement:
+ # Standing replacement pointer: the catalog's fallback_model_slug
+ # pre-fills --replacement so routine retirements are one argument.
+ by_slug = {m.slug: m for m in get_catalog().models}
+ entry = by_slug.get(args.slug)
+ if entry is not None and entry.fallback_model_slug:
+ args.replacement = entry.fallback_model_slug
+ print(
+ f"--replacement defaulted from catalog fallback: "
+ f"'{args.replacement}'"
+ )
+ if not args.slug or not args.replacement:
+ print(
+ "slug and --replacement are required to retire (no catalog "
+ "fallback_model_slug is set for this model)",
+ file=sys.stderr,
+ )
+ return 2
+
+ usage = await count_model_usage(args.slug)
+ if not args.yes:
+ _validate_replacement(args.replacement)
+ print(
+ f"DRY RUN: would migrate {usage} node(s) from '{args.slug}' to "
+ f"'{args.replacement}'. Re-run with --yes to execute."
+ )
+ return 1
+ result = await retire_model(args.slug, args.replacement, reason=args.reason)
+ print(result.model_dump_json(indent=2))
+ print(
+ "Reminder: retirement only rewrites graph nodes — flip "
+ f"'{args.slug}' to enabled=False in a catalog PR to stop new use."
+ )
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(asyncio.run(_run_cli(_build_parser().parse_args())))
diff --git a/autogpt_platform/backend/backend/data/llm_registry/retire_test.py b/autogpt_platform/backend/backend/data/llm_registry/retire_test.py
new file mode 100644
index 0000000000..bae2bb21d0
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/retire_test.py
@@ -0,0 +1,309 @@
+"""Real-DB tests for the model retirement CLI/library.
+
+Seeds minimal User/AgentGraph/AgentNode rows, exercises retire → revert
+against the actual `{schema_prefix}` SQL, and verifies the catalog-based
+validation and the active-migration partial unique index.
+"""
+
+from __future__ import annotations
+
+import uuid
+from unittest.mock import AsyncMock
+
+import prisma.models
+import pytest
+from prisma.errors import UniqueViolationError
+
+import backend.data.llm_registry.registry as reg
+from backend.data.llm_registry.registry import get_all_models, load_catalog
+from backend.data.llm_registry.retire import (
+ count_model_usage,
+ list_model_migrations,
+ retire_model,
+ revert_model_migration,
+)
+
+# Prisma's client is bound to the session-scoped server fixture's loop.
+pytestmark = pytest.mark.asyncio(loop_scope="session")
+
+_TEST_USER_ID = "5e53486c-cf57-477e-ba2a-cb02dc82ret1"
+
+
[email protected](autouse=True)
+async def retirement_env(server):
+ """Load the catalog, clean records, and tear down created graph rows."""
+ load_catalog()
+ await prisma.models.LlmModelMigration.prisma().delete_many()
+ created_graph_ids: list[str] = []
+
+ try:
+ await prisma.models.User.prisma().create(
+ data={
+ "id": _TEST_USER_ID,
+ "email": f"retire-{_TEST_USER_ID}@example.com",
+ "name": "Retire Test",
+ }
+ )
+ except UniqueViolationError:
+ pass
+
+ yield created_graph_ids
+
+ await prisma.models.LlmModelMigration.prisma().delete_many()
+ for graph_id in created_graph_ids:
+ await prisma.models.AgentGraph.prisma().delete_many(where={"id": graph_id})
+
+
+async def _seed_node(created_graph_ids: list[str], model_value: str) -> str:
+ """Create a graph + one node whose constantInput uses *model_value*."""
+ block = await prisma.models.AgentBlock.prisma().find_first()
+ assert block is not None, "blocks must be initialized by the server fixture"
+ graph_id = str(uuid.uuid4())
+ await prisma.models.AgentGraph.prisma().create(
+ data={
+ "id": graph_id,
+ "version": 1,
+ "name": "retire-test",
+ "description": "retire-test",
+ "userId": _TEST_USER_ID,
+ "isActive": True,
+ }
+ )
+ created_graph_ids.append(graph_id)
+ node = await prisma.models.AgentNode.prisma().create(
+ data={
+ "agentBlockId": block.id,
+ "agentGraphId": graph_id,
+ "agentGraphVersion": 1,
+ "constantInput": prisma.Json({"model": model_value}),
+ }
+ )
+ return node.id
+
+
+def _two_catalog_slugs() -> tuple[str, str]:
+ models = [m for m in get_all_models() if m.is_enabled]
+ assert len(models) >= 2
+ return models[0].slug, models[1].slug
+
+
+async def _node_model(node_id: str) -> str:
+ node = await prisma.models.AgentNode.prisma().find_unique(where={"id": node_id})
+ assert node is not None
+ constant_input = dict(node.constantInput or {})
+ return str(constant_input.get("model"))
+
+
+def _node_value(slug: str) -> str:
+ """AgentNodes store FULL enum values — identity with catalog slugs."""
+ return slug
+
+
+async def test_usage_counts_seeded_nodes(retirement_env):
+ source, _ = _two_catalog_slugs()
+ await _seed_node(retirement_env, _node_value(source))
+
+ assert await count_model_usage(source) == 1
+
+
+async def test_retire_rewrites_nodes_and_records(retirement_env):
+ source, replacement = _two_catalog_slugs()
+ node_id = await _seed_node(retirement_env, _node_value(source))
+
+ result = await retire_model(source, replacement, reason="test retirement")
+
+ assert result.nodes_migrated == 1
+ assert result.migration_id is not None
+ assert await _node_model(node_id) == _node_value(replacement)
+
+ record = await prisma.models.LlmModelMigration.prisma().find_unique(
+ where={"id": result.migration_id}
+ )
+ assert record is not None
+ assert record.nodeCount == 1
+ assert node_id in list(record.migratedNodeIds or [])
+ assert record.reason == "test retirement"
+
+
+async def test_revert_restores_nodes(retirement_env):
+ source, replacement = _two_catalog_slugs()
+ node_id = await _seed_node(retirement_env, _node_value(source))
+ retired = await retire_model(source, replacement)
+ assert retired.migration_id is not None
+
+ reverted = await revert_model_migration(retired.migration_id)
+
+ assert reverted.nodes_reverted == 1
+ assert reverted.nodes_already_changed == 0
+ assert await _node_model(node_id) == _node_value(source)
+ record = await prisma.models.LlmModelMigration.prisma().find_unique(
+ where={"id": retired.migration_id}
+ )
+ assert record is not None
+ assert record.isReverted is True
+ assert record.revertedAt is not None
+ assert await list_model_migrations() == [] # active-only view is empty
+
+
+async def test_unknown_replacement_rejected_without_writes(retirement_env):
+ source, _ = _two_catalog_slugs()
+ node_id = await _seed_node(retirement_env, _node_value(source))
+
+ with pytest.raises(ValueError, match="not in the catalog"):
+ await retire_model(source, "no/such-model")
+
+ assert await _node_model(node_id) == _node_value(source)
+ assert await prisma.models.LlmModelMigration.prisma().count() == 0
+
+
+async def test_disabled_replacement_rejected(retirement_env):
+ source, replacement = _two_catalog_slugs()
+ await _seed_node(retirement_env, _node_value(source))
+
+ original = reg._dynamic_models[replacement]
+ reg._dynamic_models[replacement] = original.model_copy(update={"is_enabled": False})
+ try:
+ with pytest.raises(ValueError, match="disabled in the catalog"):
+ await retire_model(source, replacement)
+ finally:
+ reg._dynamic_models[replacement] = original
+
+
+async def test_second_active_migration_rejected(retirement_env):
+ source, replacement = _two_catalog_slugs()
+ await _seed_node(retirement_env, _node_value(source))
+ first = await retire_model(source, replacement)
+ assert first.migration_id is not None
+
+ # New node appears on the retired model (e.g. imported graph) — a second
+ # retire must be rejected while the first migration is still active.
+ await _seed_node(retirement_env, _node_value(source))
+ with pytest.raises(ValueError, match="active migration"):
+ await retire_model(source, replacement)
+
+
+async def test_retire_with_no_referencing_nodes_records_nothing(retirement_env):
+ source, replacement = _two_catalog_slugs()
+
+ result = await retire_model(source, replacement)
+
+ assert result.nodes_migrated == 0
+ assert result.migration_id is None
+ assert await prisma.models.LlmModelMigration.prisma().count() == 0
+
+
+async def test_prefixed_model_nodes_are_found_and_rewritten(retirement_env):
+ """Provider-prefixed enum values (e.g. moonshotai/kimi-k2.5) are stored
+ verbatim on nodes; an earlier revision stripped the prefix and silently
+ no-opped on exactly these models."""
+ source = "moonshotai/kimi-k2.5"
+ replacement = "moonshotai/kimi-k2.6"
+ node_id = await _seed_node(retirement_env, source)
+
+ assert await count_model_usage(source) == 1
+ result = await retire_model(source, replacement, reason="prefix regression")
+ assert result.nodes_migrated == 1
+ assert await _node_model(node_id) == replacement
+
+ revert = await revert_model_migration(result.migration_id)
+ assert revert.nodes_reverted == 1
+ assert await _node_model(node_id) == source
+
+
[email protected](loop_scope="session")
+async def test_cli_dry_run_writes_nothing(retirement_env, mocker):
+ """The destructive CLI defaults to dry-run: without --yes it must
+ report and exit 1 having written no node changes and no record."""
+ from backend.data.llm_registry.retire import _build_parser, _run_cli
+
+ source, replacement = _two_catalog_slugs()
+ node_id = await _seed_node(retirement_env, _node_value(source))
+
+ mocker.patch("backend.data.db.connect", new=AsyncMock())
+ args = _build_parser().parse_args([source, "--replacement", replacement])
+ assert await _run_cli(args) == 1
+
+ assert await _node_model(node_id) == _node_value(source)
+ assert await prisma.models.LlmModelMigration.prisma().count() == 0
+
+
[email protected](loop_scope="session")
+async def test_revert_leaves_manually_repointed_nodes_alone(retirement_env):
+ """The documented revert guard: nodes a user manually repointed at a
+ third model since the migration are NOT touched by the revert, and the
+ result reports them as already-changed."""
+ source, replacement = _two_catalog_slugs()
+ node_a = await _seed_node(retirement_env, _node_value(source))
+ node_b = await _seed_node(retirement_env, _node_value(source))
+
+ result = await retire_model(source, replacement)
+ assert result.nodes_migrated == 2
+
+ third = "user/hand-picked-model"
+ node = await prisma.models.AgentNode.prisma().find_unique(where={"id": node_b})
+ assert node is not None
+ assert isinstance(node.constantInput, dict)
+ ci = dict(node.constantInput)
+ ci["model"] = third
+ await prisma.models.AgentNode.prisma().update(
+ where={"id": node_b}, data={"constantInput": prisma.Json(ci)}
+ )
+
+ migrations = await list_model_migrations()
+ revert = await revert_model_migration(migrations[0].id)
+
+ assert revert.nodes_reverted == 1
+ assert revert.nodes_already_changed == 1
+ assert await _node_model(node_a) == _node_value(source)
+ assert await _node_model(node_b) == third
+
+
[email protected](loop_scope="session")
+async def test_cli_replacement_defaults_from_catalog_fallback(retirement_env, mocker):
+ """--replacement pre-fills from the retired model's fallback_model_slug."""
+ from backend.data.llm_registry import retire as retire_mod
+ from backend.data.llm_registry.retire import _build_parser, _run_cli
+
+ source, replacement = _two_catalog_slugs()
+ node_id = await _seed_node(retirement_env, _node_value(source))
+
+ payload = retire_mod.get_catalog()
+ patched = payload.model_copy(
+ update={
+ "models": [
+ (
+ m.model_copy(update={"fallback_model_slug": replacement})
+ if m.slug == source
+ else m
+ )
+ for m in payload.models
+ ]
+ }
+ )
+ mocker.patch.object(retire_mod, "get_catalog", return_value=patched)
+ mocker.patch("backend.data.db.connect", new=AsyncMock())
+
+ args = _build_parser().parse_args([source]) # no --replacement
+ assert await _run_cli(args) == 1 # dry run banner path
+ assert args.replacement == replacement
+ assert await _node_model(node_id) == _node_value(source) # nothing written
+
+
[email protected](loop_scope="session")
+async def test_double_revert_refused_by_atomic_claim(retirement_env):
+ """The second revert of the same migration must refuse — the guarded
+ in-transaction claim matches zero rows (the TOCTOU race guard)."""
+ source, replacement = _two_catalog_slugs()
+ await _seed_node(retirement_env, _node_value(source))
+ await retire_model(source, replacement)
+ migration_id = (await list_model_migrations())[0].id
+
+ await revert_model_migration(migration_id)
+ with pytest.raises(ValueError, match="already been reverted"):
+ await revert_model_migration(migration_id)
+
+
[email protected](loop_scope="session")
+async def test_revert_unknown_migration_id_refused(retirement_env):
+ with pytest.raises(ValueError, match="not found"):
+ await revert_model_migration("no-such-migration-id")
diff --git a/autogpt_platform/backend/backend/data/llm_registry/stored_graph_models_test.py b/autogpt_platform/backend/backend/data/llm_registry/stored_graph_models_test.py
new file mode 100644
index 0000000000..ca04e14e44
--- /dev/null
+++ b/autogpt_platform/backend/backend/data/llm_registry/stored_graph_models_test.py
@@ -0,0 +1,192 @@
+"""Every LLM model value stored in real AgentNode data resolves via the catalog.
+
+Fixture captured 2026-07-20 with a read-only query against BOTH prod and dev:
+
+ SELECT DISTINCT "constantInput"->>'model' FROM "AgentNode";
+
+Saved graphs keep the model as a plain string in ``constantInput`` forever —
+they are a fossil record of every model identifier we ever shipped, so code
+inspection alone cannot prove that cutting model resolution over to the
+catalog strands no existing graph. This fixture makes that set closed and
+testable: every stored value must either resolve through the catalog
+(directly or via ``LEGACY_MODEL_MAPPINGS``) or be claimed by a non-LLM block
+(image/video/search/embedding blocks share the ``model`` input name).
+
+Re-run the query and refresh ``STORED_MODEL_VALUES`` before any change that
+narrows model resolution (model removal, legacy-mapping cleanup).
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from backend.blocks.llm import LEGACY_MODEL_MAPPINGS, LLMModel
+from backend.data.llm_registry.catalog import get_catalog
+
+# Merged prod + dev distinct values, 2026-07-20. NULL rows (nodes without a
+# ``model`` key) excluded — absent keys fall back to the block default.
+STORED_MODEL_VALUES = [
+ "Flux 1.1 Pro",
+ "Flux 1.1 Pro Ultra",
+ "Flux Kontext Max",
+ "Flux Kontext Pro",
+ "Llama-3.3-70B-Instruct",
+ "Llama-3.3-8B-Instruct",
+ "Llama-4-Maverick-17B-128E-Instruct-FP8",
+ "Llama-4-Scout-17B-16E-Instruct-FP8",
+ "Nano Banana 2",
+ "Nano Banana Pro",
+ "Recraft v3",
+ "Stable Diffusion 3.5 Medium",
+ "amazon/nova-lite-v1",
+ "amazon/nova-micro-v1",
+ "amazon/nova-pro-v1",
+ "claude-haiku-4-5-20251001",
+ "claude-opus-4-5-20251101",
+ "claude-opus-4-6",
+ "claude-opus-4-7",
+ "claude-sonnet-4-5-20250929",
+ "claude-sonnet-4-6",
+ "cohere/command-a-03-2025",
+ "deepseek/deepseek-chat",
+ "deepseek/deepseek-r1-0528",
+ "dolphin-mistral:latest",
+ "exa",
+ "exa-research",
+ "exa-research-pro",
+ "fal-ai/luma-dream-machine",
+ "fal-ai/mochi-v1",
+ "fal-ai/veo3",
+ "google/gemini-2.0-flash-001",
+ "google/gemini-2.0-flash-lite-001",
+ "google/gemini-2.5-flash",
+ "google/gemini-2.5-flash-image",
+ "google/gemini-2.5-pro",
+ "google/gemini-3-flash-preview",
+ "google/gemini-3.1-flash-lite-preview",
+ "google/gemini-3.1-pro-preview",
+ "google/nano-banana",
+ "google/nano-banana-2",
+ "google/nano-banana-pro",
+ "gpt-4.1-2025-04-14",
+ "gpt-4.1-mini-2025-04-14",
+ "gpt-4o",
+ "gpt-4o-mini",
+ "gpt-5-2025-08-07",
+ "gpt-5-chat-latest",
+ "gpt-5-mini-2025-08-07",
+ "gpt-5-nano-2025-08-07",
+ "gpt-5.1-2025-11-13",
+ "gpt-5.1-codex",
+ "gpt-5.2-2025-12-11",
+ "gryphe/mythomax-l2-13b",
+ "jina-embeddings-v2-base-en",
+ "kontext-pro",
+ "llama-3.1-8b-instant",
+ "llama-3.3-70b-versatile",
+ "llama3",
+ "llama3.1:405b",
+ "llama3.2",
+ "llama3.3",
+ "meta-llama/Llama-3.3-70B-Instruct-Turbo",
+ "meta-llama/llama-4-maverick",
+ "meta-llama/llama-4-scout",
+ "microsoft/phi-4",
+ "mistralai/codestral-2508",
+ "mistralai/mistral-large-2512",
+ "mistralai/mistral-medium-3.1",
+ "mistralai/mistral-small-3.2-24b-instruct",
+ "moonshotai/kimi-k2.5",
+ "moonshotai/kimi-k2.6",
+ "nano-banana-pro",
+ "nousresearch/hermes-3-llama-3.1-405b",
+ "nousresearch/hermes-3-llama-3.1-70b",
+ "o3-2025-04-16",
+ "o3-mini",
+ "openai/gpt-oss-120b",
+ "openai/gpt-oss-20b",
+ "perplexity/sonar",
+ "perplexity/sonar-deep-research",
+ "perplexity/sonar-pro",
+ "qwen/qwen3-235b-a22b-thinking-2507",
+ "qwen/qwen3-coder",
+ "v0-1.0-md",
+ "v0-1.5-lg",
+ "v0-1.5-md",
+ "x-ai/grok-4",
+ "x-ai/grok-4.1-fast",
+ "x-ai/grok-4.20-multi-agent",
+ "x-ai/grok-code-fast-1",
+ "z-ai/glm-5v-turbo",
+]
+
+# Stored values owned by non-LLM blocks (they also name their input ``model``),
+# mapped to the block module that serves them. Values marked historical no
+# longer appear in the owning block's current enum; their migration story
+# belongs to that block, not the LLM catalog.
+NON_LLM_BLOCK_VALUES: dict[str, str] = {
+ "Flux 1.1 Pro": "ai_image_generator_block.py",
+ "Flux 1.1 Pro Ultra": "ai_image_generator_block.py",
+ "Flux Kontext Max": "flux_kontext.py",
+ "Flux Kontext Pro": "flux_kontext.py",
+ "Nano Banana 2": "ai_image_generator_block.py",
+ "Nano Banana Pro": "ai_image_generator_block.py",
+ "Recraft v3": "ai_image_generator_block.py",
+ "Stable Diffusion 3.5 Medium": "ai_image_generator_block.py",
+ "exa": "exa/research.py", # historical
+ "exa-research": "exa/research.py",
+ "exa-research-pro": "exa/research.py",
+ "fal-ai/luma-dream-machine": "fal/ai_video_generator.py",
+ "fal-ai/mochi-v1": "fal/ai_video_generator.py",
+ "fal-ai/veo3": "fal/ai_video_generator.py",
+ "google/gemini-2.5-flash-image": "ai_image_generator_block.py", # historical
+ "google/nano-banana": "ai_image_customizer.py",
+ "google/nano-banana-2": "ai_image_generator_block.py",
+ "google/nano-banana-pro": "ai_image_generator_block.py",
+ "gpt-5.1-codex": "codex.py",
+ "jina-embeddings-v2-base-en": "jina/embeddings.py",
+ "kontext-pro": "flux_kontext.py",
+ "nano-banana-pro": "ai_image_customizer.py",
+}
+
+BLOCKS_DIR = Path(__file__).parents[2] / "blocks"
+
+
+def test_every_stored_llm_model_value_resolves_through_the_catalog():
+ # Block runtime resolves via the ENUM, whose identifiers must be catalog-
+ # backed (import-time check) — assert against enum values so a future
+ # copilot-only catalog model can't mask a value blocks would reject.
+ enum_values = {m.value for m in LLMModel}
+ catalog_slugs = {m.slug for m in get_catalog().models}
+ assert enum_values <= catalog_slugs
+ unresolved = []
+ for value in STORED_MODEL_VALUES:
+ if value in NON_LLM_BLOCK_VALUES:
+ continue
+ if value in enum_values:
+ continue
+ legacy = LEGACY_MODEL_MAPPINGS.get(value)
+ if legacy is not None and legacy.value in enum_values:
+ continue
+ unresolved.append(value)
+ assert not unresolved, (
+ "Model values stored in real graphs would no longer resolve after the "
+ f"catalog cutover: {unresolved}. Add the model to the catalog or map "
+ "it in LEGACY_MODEL_MAPPINGS before shipping."
+ )
+
+
+def test_non_llm_exclusions_name_real_block_modules():
+ missing = {
+ value: module
+ for value, module in NON_LLM_BLOCK_VALUES.items()
+ if not (BLOCKS_DIR / module).exists()
+ }
+ assert (
+ not missing
+ ), f"Exclusion entries point at block modules that do not exist: {missing}"
+
+
+def test_exclusion_list_stays_scoped_to_the_fixture():
+ stale = set(NON_LLM_BLOCK_VALUES) - set(STORED_MODEL_VALUES)
+ assert not stale, f"Exclusions for values no longer present in the fixture: {stale}"
diff --git a/autogpt_platform/backend/backend/executor/billing_reconciliation_test.py b/autogpt_platform/backend/backend/executor/billing_reconciliation_test.py
index e4db3f981f..b0adc987e2 100644
--- a/autogpt_platform/backend/backend/executor/billing_reconciliation_test.py
+++ b/autogpt_platform/backend/backend/executor/billing_reconciliation_test.py
@@ -307,19 +307,19 @@ async def test_cost_usd_with_larger_spend_bills_full_delta(tmp_block_costs_overr
async def test_tokens_cost_refunds_when_actual_below_estimate(tmp_block_costs_override):
"""TOKENS pre-flight uses MODEL_COST floor; if real token usage is cheaper,
the user is refunded the overcharge via a negative-delta spend_credits."""
- from backend.blocks.llm import LlmModel
+ from backend.blocks.llm import LLMModel
tmp_block_costs_override(
[
BlockCost(
cost_amount=1,
cost_type=BlockCostType.TOKENS,
- cost_filter={"model": LlmModel.GPT5},
+ cost_filter={"model": LLMModel.GPT5},
)
]
)
exec_entry = _node_exec(SearchTheWebBlock().id)
- exec_entry = exec_entry.model_copy(update={"inputs": {"model": LlmModel.GPT5}})
+ exec_entry = exec_entry.model_copy(update={"inputs": {"model": LLMModel.GPT5}})
# Minimal real usage → post-flight < pre-flight MODEL_COST floor.
stats = NodeExecutionStats(
input_token_count=1,
diff --git a/autogpt_platform/backend/backend/executor/block_usage_cost_test.py b/autogpt_platform/backend/backend/executor/block_usage_cost_test.py
index fe0cf14732..498962e5c1 100644
--- a/autogpt_platform/backend/backend/executor/block_usage_cost_test.py
+++ b/autogpt_platform/backend/backend/executor/block_usage_cost_test.py
@@ -13,7 +13,7 @@ from backend.blocks.code_executor import (
from backend.blocks.exa.search import ExaSearchBlock
from backend.blocks.fal.ai_video_generator import AIVideoGeneratorBlock
from backend.blocks.jina.search import SearchTheWebBlock
-from backend.blocks.llm import AITextGeneratorBlock, LlmModel
+from backend.blocks.llm import AITextGeneratorBlock, LLMModel
from backend.data.block_cost_config import (
BLOCK_COSTS,
MODEL_COST,
@@ -200,7 +200,7 @@ def test_tokens_cost_type_uses_token_rate_table(tmp_block_costs_override, monkey
# 2000 credits/1M output.
monkeypatch.setitem(
TOKEN_COST,
- LlmModel.GPT4O_MINI,
+ LLMModel.GPT4O_MINI,
TokenRate(input=1000, output=2000),
)
block = SearchTheWebBlock()
@@ -210,7 +210,7 @@ def test_tokens_cost_type_uses_token_rate_table(tmp_block_costs_override, monkey
)
cost, _ = block_usage_cost(
block,
- {"model": LlmModel.GPT4O_MINI.value},
+ {"model": LLMModel.GPT4O_MINI.value},
stats=stats,
)
# 0.5 * 1000 + 0.25 * 2000 = 500 + 500 = 1000 credits.
@@ -223,7 +223,7 @@ def test_tokens_falls_back_to_flat_model_cost_when_rate_missing(
tmp_block_costs_override([BlockCost(cost_amount=0, cost_type=BlockCostType.TOKENS)])
block = SearchTheWebBlock()
# Ollama models aren't in TOKEN_COST but are in MODEL_COST.
- ollama_model = LlmModel.OLLAMA_LLAMA3_2
+ ollama_model = LLMModel.OLLAMA_LLAMA3_2
expected = MODEL_COST[ollama_model]
cost, _ = block_usage_cost(
block,
@@ -296,14 +296,14 @@ def test_llm_block_charges_per_token_post_flight():
"""AITextGeneratorBlock with Claude 4.6 Sonnet bills by real token counts."""
block = AITextGeneratorBlock()
input_data = {
- "model": LlmModel.CLAUDE_4_6_SONNET,
+ "model": LLMModel.CLAUDE_4_6_SONNET,
"credentials": {
"id": anthropic_credentials.id,
"provider": anthropic_credentials.provider,
"type": anthropic_credentials.type,
},
}
- rate = TOKEN_COST[LlmModel.CLAUDE_4_6_SONNET]
+ rate = TOKEN_COST[LLMModel.CLAUDE_4_6_SONNET]
stats = NodeExecutionStats(
input_token_count=200_000,
output_token_count=50_000,
@@ -323,7 +323,7 @@ def test_llm_block_pre_flight_falls_back_to_model_cost():
cost, _ = block_usage_cost(
block,
{
- "model": LlmModel.GPT5,
+ "model": LLMModel.GPT5,
"credentials": {
"id": openai_credentials.id,
"provider": openai_credentials.provider,
@@ -331,7 +331,7 @@ def test_llm_block_pre_flight_falls_back_to_model_cost():
},
},
)
- assert cost == MODEL_COST[LlmModel.GPT5]
+ assert cost == MODEL_COST[LLMModel.GPT5]
def test_run_cost_type_remains_unchanged(tmp_block_costs_override):
diff --git a/autogpt_platform/backend/backend/executor/simulator.py b/autogpt_platform/backend/backend/executor/simulator.py
index 26f797471e..b86e1b32ac 100644
--- a/autogpt_platform/backend/backend/executor/simulator.py
+++ b/autogpt_platform/backend/backend/executor/simulator.py
@@ -41,7 +41,7 @@ from openai.types.chat import ChatCompletionMessageParam
from backend.blocks.agent import AgentExecutorBlock
from backend.blocks.io import AgentInputBlock, AgentOutputBlock
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import LLMModel
from backend.blocks.orchestrator import ExecutionMode, OrchestratorBlock
from backend.copilot.token_tracking import persist_and_record_usage
from backend.util.clients import get_openai_client, openrouter_helper_cost_provider
@@ -74,7 +74,7 @@ logger = logging.getLogger(__name__)
# Configurable via ``ChatConfig.simulation_model``
# (``CHAT_SIMULATION_MODEL`` env var); overrides must satisfy
# constraints (1) and (2).
-_DEFAULT_SIMULATOR_MODEL = LlmModel.GEMINI_2_5_FLASH_LITE.value
+_DEFAULT_SIMULATOR_MODEL = LLMModel.GEMINI_2_5_FLASH_LITE.value
# OpenRouter-specific extra_body flag that embeds the real generation cost on
# the response usage object. Same shape used by the baseline copilot service
@@ -522,12 +522,12 @@ def prepare_dry_run(block: Any, input_data: dict[str, Any]) -> dict[str, Any] |
max_iters = min(original, 10) if original != 0 else 0
# Resolve the configured simulator model (typically an OpenRouter
# slug like "anthropic/claude-haiku-4-5") to its canonical
- # ``LlmModel.value`` (e.g. "claude-haiku-4-5-20251001"). We have
+ # ``LLMModel.value`` (e.g. "claude-haiku-4-5-20251001"). We have
# to do this BEFORE injecting into ``input["model"]`` because the
# block runs through ``validate_data`` → ``jsonschema.validate``
# before Pydantic gets the value, and the JSON Schema's ``enum``
- # is the literal list of ``LlmModel.value``s — the
- # ``_OPENROUTER_ALIASES`` map in ``LlmModel._missing_`` does NOT
+ # is the literal list of ``LLMModel.value``s — the
+ # ``_OPENROUTER_ALIASES`` map in ``LLMModel._missing_`` does NOT
# surface in the generated schema, so the OR-slug is rejected
# with ``"'<slug>' is not one of [...]"``. After this
# translation, the OrchestratorBlock receives a model identifier
@@ -539,19 +539,19 @@ def prepare_dry_run(block: Any, input_data: dict[str, Any]) -> dict[str, Any] |
# Fall back to the default if the configured override is malformed
# or unmapped — an invalid ``CHAT_SIMULATION_MODEL`` env value
# shouldn't take out every dry-run. The default is asserted as
- # ``LlmModel``-parseable by ``TestDefaultSimulatorModel``, so the
- # fallback ``LlmModel(_DEFAULT_SIMULATOR_MODEL)`` cannot raise
+ # ``LLMModel``-parseable by ``TestDefaultSimulatorModel``, so the
+ # fallback ``LLMModel(_DEFAULT_SIMULATOR_MODEL)`` cannot raise
# unless someone breaks that pin too.
configured_sim_model = _simulator_model()
try:
- sim_model = LlmModel(configured_sim_model).value
+ sim_model = LLMModel(configured_sim_model).value
except ValueError:
logger.warning(
"Dry-run: invalid simulation model %r; falling back to default %r",
configured_sim_model,
_DEFAULT_SIMULATOR_MODEL,
)
- sim_model = LlmModel(_DEFAULT_SIMULATOR_MODEL).value
+ sim_model = LLMModel(_DEFAULT_SIMULATOR_MODEL).value
# Keep the original credentials dict in input_data so the block's
# JSON schema validation passes (validate_data strips None values,
diff --git a/autogpt_platform/backend/backend/executor/simulator_test.py b/autogpt_platform/backend/backend/executor/simulator_test.py
index fe03354183..3510d8c7a9 100644
--- a/autogpt_platform/backend/backend/executor/simulator_test.py
+++ b/autogpt_platform/backend/backend/executor/simulator_test.py
@@ -19,7 +19,7 @@ from openai.types.chat import ChatCompletion
from openai.types.chat.chat_completion import Choice
from openai.types.chat.chat_completion_message import ChatCompletionMessage
-from backend.blocks.llm import LlmModel
+from backend.blocks.llm import LLMModel
from backend.blocks.orchestrator import ExecutionMode, OrchestratorBlock
from backend.executor.simulator import (
_DEFAULT_SIMULATOR_MODEL,
@@ -172,17 +172,17 @@ class TestPrepareDryRun:
assert result["agent_mode_max_iterations"] == 10
assert result["other"] == "val"
assert result["model"] != "gpt-4o" # overridden to simulation model
- # Simulation model must parse as a real LlmModel so OrchestratorBlock's
+ # Simulation model must parse as a real LLMModel so OrchestratorBlock's
# Pydantic input validation accepts it.
- assert LlmModel(result["model"]) is not None
- # The injected model must be a canonical LlmModel value (string equal
+ assert LLMModel(result["model"]) is not None
+ # The injected model must be a canonical LLMModel value (string equal
# to one of the enum's ``.value``s), not an OpenRouter alias slug —
# OrchestratorBlock.validate_data → jsonschema only accepts literal
- # ``LlmModel.value``s in the schema's ``enum``, and the alias map
- # in ``LlmModel._missing_`` does not surface in the generated
+ # ``LLMModel.value``s in the schema's ``enum``, and the alias map
+ # in ``LLMModel._missing_`` does not surface in the generated
# JSON Schema. Anything else trips
# ``"'<slug>' is not one of [...]"`` at runtime.
- canonical_values = {m.value for m in LlmModel}
+ canonical_values = {m.value for m in LLMModel}
assert result["model"] in canonical_values, (
f"prepare_dry_run injected non-canonical model {result['model']!r}; "
f"jsonschema validation will reject it"
@@ -197,7 +197,7 @@ class TestPrepareDryRun:
) -> None:
"""An invalid ``CHAT_SIMULATION_MODEL`` env value must not crash
``prepare_dry_run`` — fall back to the default so dry-run keeps
- working. Without the guard, ``LlmModel('<garbage>')`` raises
+ working. Without the guard, ``LLMModel('<garbage>')`` raises
``ValueError`` and aborts every Orchestrator dry-run."""
with (
patch(
@@ -213,13 +213,13 @@ class TestPrepareDryRun:
OrchestratorBlock(),
{
"prompt": "test",
- "model": LlmModel.CLAUDE_4_7_OPUS.value,
+ "model": LLMModel.CLAUDE_4_7_OPUS.value,
"agent_mode_max_iterations": 1,
},
)
assert result is not None
# Must land on the default value, not the garbage override.
- assert result["model"] == LlmModel(_DEFAULT_SIMULATOR_MODEL).value, (
+ assert result["model"] == LLMModel(_DEFAULT_SIMULATOR_MODEL).value, (
"Invalid CHAT_SIMULATION_MODEL should fall back to default; "
f"got {result['model']!r}"
)
@@ -251,7 +251,7 @@ class TestPrepareDryRun:
block,
{
"prompt": "test",
- "model": LlmModel.CLAUDE_4_7_OPUS.value,
+ "model": LLMModel.CLAUDE_4_7_OPUS.value,
"execution_mode": ExecutionMode.EXTENDED_THINKING.value,
"agent_mode_max_iterations": 1,
},
@@ -266,14 +266,14 @@ class TestPrepareDryRun:
"""The injected dry-run input must pass OrchestratorBlock.validate_data.
Pinning this prevents the SECRT-2368 follow-up bug class where
- prepare_dry_run injects an OpenRouter slug that LlmModel resolves
+ prepare_dry_run injects an OpenRouter slug that LLMModel resolves
via the alias map at the Pydantic layer, but jsonschema enum
validation (which runs *before* Pydantic) rejects.
"""
block = OrchestratorBlock()
user_input = {
"prompt": "test",
- "model": LlmModel.CLAUDE_4_7_OPUS.value,
+ "model": LLMModel.CLAUDE_4_7_OPUS.value,
"credentials": {
"id": "00000000-0000-0000-0000-000000000000",
"provider": "open_router",
@@ -695,7 +695,7 @@ def _sim_completion(*, content: str, usage: CompletionUsage) -> ChatCompletion:
class TestDefaultSimulatorModel:
"""Pin the default model. Four guards line up with the constraints
laid out next to ``_DEFAULT_SIMULATOR_MODEL`` in ``simulator.py``:
- value pin, ``LlmModel`` parseability, OpenRouter slug shape, and
+ value pin, ``LLMModel`` parseability, OpenRouter slug shape, and
``open_router`` provider routing (so the BUILT_IN orchestrator path
in ``llm.llm_call`` doesn't get routed through ``api.anthropic.com``
with the platform OR key)."""
@@ -704,7 +704,7 @@ class TestDefaultSimulatorModel:
assert _DEFAULT_SIMULATOR_MODEL == "google/gemini-2.5-flash-lite"
def test_default_parses_as_llm_model(self) -> None:
- assert LlmModel(_DEFAULT_SIMULATOR_MODEL) is LlmModel.GEMINI_2_5_FLASH_LITE
+ assert LLMModel(_DEFAULT_SIMULATOR_MODEL) is LLMModel.GEMINI_2_5_FLASH_LITE
def test_default_is_openrouter_slug(self) -> None:
# The LLM-simulation path hits OpenRouter's OpenAI-compat endpoint,
@@ -717,7 +717,7 @@ class TestDefaultSimulatorModel:
# at ``api.anthropic.com`` → 401. Pin ``open_router`` here so
# a future default change that breaks this routing trips at
# unit-test time.
- assert LlmModel(_DEFAULT_SIMULATOR_MODEL).metadata.provider == "open_router"
+ assert LLMModel(_DEFAULT_SIMULATOR_MODEL).metadata.provider == "open_router"
class TestExtractCostUsd:
diff --git a/autogpt_platform/backend/backend/util/llm/__init__.py b/autogpt_platform/backend/backend/util/llm/__init__.py
index eedba1a693..d0b99b38da 100644
--- a/autogpt_platform/backend/backend/util/llm/__init__.py
+++ b/autogpt_platform/backend/backend/util/llm/__init__.py
@@ -9,7 +9,7 @@ flex), or new structured-output handling lands once and every caller
picks it up.
This module is intentionally agnostic to the caller's surrounding
-framework: it doesn't depend on the block layer's ``LlmModel`` enum,
+framework: it doesn't depend on the block layer's ``LLMModel`` enum,
the executor's ``NodeExecutionStats``, or the dream-pass's Pydantic
validation. Callers wrap it with their own typed entry points.
"""
diff --git a/autogpt_platform/backend/backend/util/llm/providers.py b/autogpt_platform/backend/backend/util/llm/providers.py
index cb0060ca2a..85ec502502 100644
--- a/autogpt_platform/backend/backend/util/llm/providers.py
+++ b/autogpt_platform/backend/backend/util/llm/providers.py
@@ -16,7 +16,7 @@ What this module IS responsible for:
``flex`` in later steps)
What this module is NOT responsible for (caller wraps):
- * ``LlmModel``-aware token-budget computation (caller passes
+ * ``LLMModel``-aware token-budget computation (caller passes
pre-computed ``max_tokens``)
* Prompt compression (``compress_context``) — block layer needs it,
dream may not
diff --git a/autogpt_platform/backend/backend/util/prompt.py b/autogpt_platform/backend/backend/util/prompt.py
index d8d872237f..bed7e6e8c9 100644
--- a/autogpt_platform/backend/backend/util/prompt.py
+++ b/autogpt_platform/backend/backend/util/prompt.py
@@ -259,22 +259,22 @@ _CONTEXT_OVERHEAD_RESERVE = 60_000
def get_context_window(model: str) -> int | None:
"""Return the context window size for a model, or None if unknown.
- Looks up the model in the :class:`LlmModel` enum (which already
+ Looks up the model in the :class:`LLMModel` enum (which already
carries ``context_window`` via ``MODEL_METADATA``). Handles
provider-prefixed names (``anthropic/claude-opus-4-6``) and
case-insensitive input automatically.
"""
- from backend.blocks.llm import LlmModel # lazy to avoid circular import
+ from backend.blocks.llm import LLMModel # lazy to avoid circular import
try:
- llm_model = LlmModel(model)
+ llm_model = LLMModel(model)
return llm_model.context_window
except (ValueError, KeyError):
pass
# Retry with lowercase for case-insensitive lookup
try:
- llm_model = LlmModel(model.lower())
+ llm_model = LLMModel(model.lower())
return llm_model.context_window
except (ValueError, KeyError):
return None
diff --git a/autogpt_platform/backend/migrations/20260718120000_add_llm_catalog_runtime_tables/migration.sql b/autogpt_platform/backend/migrations/20260718120000_add_llm_catalog_runtime_tables/migration.sql
new file mode 100644
index 0000000000..e6785419bf
--- /dev/null
+++ b/autogpt_platform/backend/migrations/20260718120000_add_llm_catalog_runtime_tables/migration.sql
@@ -0,0 +1,33 @@
+-- AlterTable
+ALTER TABLE "ChatMessage" ADD COLUMN "model" TEXT,
+ADD COLUMN "routingSource" TEXT;
+
+-- CreateTable
+CREATE TABLE "LlmModelMigration" (
+ "id" TEXT NOT NULL,
+ "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ "updatedAt" TIMESTAMP(3) NOT NULL,
+ "sourceModelSlug" TEXT NOT NULL,
+ "targetModelSlug" TEXT NOT NULL,
+ "reason" TEXT,
+ "migratedNodeIds" JSONB NOT NULL DEFAULT '[]',
+ "nodeCount" INTEGER NOT NULL,
+ "isReverted" BOOLEAN NOT NULL DEFAULT false,
+ "revertedAt" TIMESTAMP(3),
+
+ CONSTRAINT "LlmModelMigration_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateIndex
+CREATE INDEX "LlmModelMigration_targetModelSlug_idx" ON "LlmModelMigration"("targetModelSlug");
+
+-- CreateIndex
+CREATE INDEX "LlmModelMigration_sourceModelSlug_isReverted_idx" ON "LlmModelMigration"("sourceModelSlug", "isReverted");
+
+
+-- CreateIndex (partial unique to prevent multiple active migrations per source)
+CREATE UNIQUE INDEX "LlmModelMigration_active_source_key" ON "LlmModelMigration"("sourceModelSlug") WHERE "isReverted" = false;
+
+-- AddCheckConstraint
+ALTER TABLE "LlmModelMigration"
+ ADD CONSTRAINT "LlmModelMigration_nodeCount_check" CHECK ("nodeCount" >= 0);
diff --git a/autogpt_platform/backend/schema.prisma b/autogpt_platform/backend/schema.prisma
index 0f67c97ce1..2ff3b02d94 100644
--- a/autogpt_platform/backend/schema.prisma
+++ b/autogpt_platform/backend/schema.prisma
@@ -512,6 +512,15 @@ model ChatMessage {
sequence Int
durationMs Int? // Wall-clock milliseconds for this assistant turn
+ // Which LLM served this assistant turn, and which routing layer picked it
+ // ("ld" | "catalog" | "env" | "fallback" — the last when the CLI's
+ // overload fallback served a different model than the routed one).
+ // Product-intelligence mirrors these columns to
+ // segment quality judgments by model — model experiments are unmeasurable
+ // without them. Null on user/tool rows and rows predating this column.
+ model String?
+ routingSource String?
+
// Per-row JSONB metadata bag. Currently holds the dispatcher's
// submit-time payload (file_ids, mode, model, permissions, context,
// request_arrival_at) on the user row that triggered a queued turn,
@@ -2368,3 +2377,30 @@ model BotInstall {
@@unique([platform, platformServerId])
@@index([platform, revokedAt])
}
+
+// Retirement records: which AgentNodes were rewritten when a model was
+// retired via the retirement CLI (backend.data.llm_registry.retire), so the
+// operation is observable and revertable. Slugs are plain strings validated
+// against the catalog file at write time — the catalog is not a DB table.
+// Per-install runtime state, unlike the catalog itself (catalog-as-code).
+model LlmModelMigration {
+ id String @id @default(uuid())
+ createdAt DateTime @default(now())
+ updatedAt DateTime @updatedAt
+
+ sourceModelSlug String // The retired model
+ targetModelSlug String // The model workflows were migrated to
+ reason String?
+
+ // JSON array of rewritten AgentNode ids, for revert
+ migratedNodeIds Json @default("[]")
+ nodeCount Int // DB constraint: >= 0
+
+ isReverted Boolean @default(false)
+ revertedAt DateTime?
+
+ // Note: Partial unique index in migration SQL prevents multiple active
+ // migrations per source: UNIQUE (sourceModelSlug) WHERE isReverted = false
+ @@index([targetModelSlug])
+ @@index([sourceModelSlug, isReverted])
+}
diff --git a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/LlmModelField.tsx b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/LlmModelField.tsx
index c2057fdad0..b58c471f36 100644
--- a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/LlmModelField.tsx
+++ b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/LlmModelField.tsx
@@ -82,6 +82,7 @@ export function LlmModelField(props: FieldProps) {
<LlmModelPicker
models={models}
+ selectedName={selectedName}
selectedModel={selectedModel}
recommendedModel={recommendedModel}
onSelect={(value) => onChange(value, fieldPathId?.path)}
diff --git a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/__tests__/LlmModelPicker.test.tsx b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/__tests__/LlmModelPicker.test.tsx
new file mode 100644
index 0000000000..b4fe5d678f
--- /dev/null
+++ b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/__tests__/LlmModelPicker.test.tsx
@@ -0,0 +1,59 @@
+import { render, screen } from "@testing-library/react";
+import { describe, expect, test, vi } from "vitest";
+import { LlmModelPicker } from "../components/LlmModelPicker";
+import { LlmModelMetadata } from "../types";
+
+const sonnet: LlmModelMetadata = {
+ name: "claude-sonnet-4-6",
+ title: "Claude Sonnet 4.6",
+ creator: "anthropic",
+ creator_name: "Anthropic",
+ provider: "anthropic",
+ provider_name: "Anthropic",
+ price_tier: 3,
+};
+
+describe("LlmModelPicker trigger", () => {
+ test("shows the selected model's display name", () => {
+ render(
+ <LlmModelPicker
+ models={[sonnet]}
+ selectedName={sonnet.name}
+ selectedModel={sonnet}
+ recommendedModel={sonnet}
+ onSelect={vi.fn()}
+ />,
+ );
+ expect(screen.getByText("Claude Sonnet 4.6")).toBeDefined();
+ });
+
+ test("shows the raw stored slug when the model is no longer available", () => {
+ // A hidden/kill-switched model drops out of llm_model_metadata while the
+ // node still stores (and executes) its slug — the trigger must show the
+ // truth, never silently substitute the recommended model.
+ render(
+ <LlmModelPicker
+ models={[sonnet]}
+ selectedName="retired/old-model"
+ selectedModel={undefined}
+ recommendedModel={sonnet}
+ onSelect={vi.fn()}
+ />,
+ );
+ expect(screen.getByText("retired/old-model (unavailable)")).toBeDefined();
+ expect(screen.queryByText("Claude Sonnet 4.6")).toBeNull();
+ });
+
+ test("falls back to the recommended model only when nothing is stored", () => {
+ render(
+ <LlmModelPicker
+ models={[sonnet]}
+ selectedName=""
+ selectedModel={undefined}
+ recommendedModel={sonnet}
+ onSelect={vi.fn()}
+ />,
+ );
+ expect(screen.getByText("Claude Sonnet 4.6")).toBeDefined();
+ });
+});
diff --git a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/components/LlmModelPicker.tsx b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/components/LlmModelPicker.tsx
index c107120f01..d385183dfc 100644
--- a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/components/LlmModelPicker.tsx
+++ b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/LlmModelField/components/LlmModelPicker.tsx
@@ -26,6 +26,7 @@ type MenuView = "creator" | "model" | "provider";
type Props = {
models: LlmModelMetadata[];
+ selectedName?: string;
selectedModel?: LlmModelMetadata;
recommendedModel?: LlmModelMetadata;
onSelect: (value: string) => void;
@@ -34,6 +35,7 @@ type Props = {
export function LlmModelPicker({
models,
+ selectedName,
selectedModel,
recommendedModel,
onSelect,
@@ -116,10 +118,19 @@ export function LlmModelPicker({
[onSelect],
);
- const triggerModel = selectedModel ?? recommendedModel ?? models[0];
- const triggerTitle = triggerModel
- ? getModelDisplayName(triggerModel)
- : "Select model";
+ // A stored slug missing from the metadata map means the model was hidden
+ // or kill-switched after this node was configured. Show the truth (the
+ // raw stored slug) rather than silently displaying the recommended model
+ // while the stored one still executes.
+ const isUnavailableSelection = Boolean(selectedName) && !selectedModel;
+ const triggerModel = isUnavailableSelection
+ ? undefined
+ : (selectedModel ?? recommendedModel ?? models[0]);
+ const triggerTitle = isUnavailableSelection
+ ? `${selectedName} (unavailable)`
+ : triggerModel
+ ? getModelDisplayName(triggerModel)
+ : "Select model";
const triggerCreator = triggerModel?.creator ?? "";
return (
diff --git a/docs/integrations/block-integrations/llm.md b/docs/integrations/block-integrations/llm.md
index 66c0b9f2f4..f0065d1fe6 100644
--- a/docs/integrations/block-integrations/llm.md
+++ b/docs/integrations/block-integrations/llm.md
@@ -65,7 +65,7 @@ The result routes data to yes_output or no_output, enabling intelligent branchin
| condition | A plaintext English description of the condition to evaluate | str | Yes |
| yes_value | (Optional) Value to output if the condition is true. If not provided, input_value will be used. | Yes Value | No |
| no_value | (Optional) Value to output if the condition is false. If not provided, input_value will be used. | No Value | No |
-| model | The language model to use for evaluating the condition. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for evaluating the condition. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
### Outputs
@@ -103,7 +103,7 @@ The block sends the entire conversation history to the chosen LLM, including sys
|-------|-------------|------|----------|
| prompt | The prompt to send to the language model. | str | No |
| messages | List of messages in the conversation. | List[Any] | Yes |
-| model | The language model to use for the conversation. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for the conversation. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
| max_tokens | The maximum number of tokens to generate in the chat completion. | int | No |
| ollama_host | Ollama host for local models | str | No |
@@ -257,7 +257,7 @@ The block formulates a prompt based on the given focus or source data, sends it
|-------|-------------|------|----------|
| focus | The focus of the list to generate. | str | No |
| source_data | The data to generate the list from. | str | No |
-| model | The language model to use for generating the list. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for generating the list. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
| max_retries | Maximum number of retries for generating a valid list. | int | No |
| force_json_output | Whether to force the LLM to produce a JSON-only response. This can increase the block's reliability, but may also reduce the quality of the response because it prohibits the LLM from reasoning before providing its JSON response. | bool | No |
| max_tokens | The maximum number of tokens to generate in the chat completion. | int | No |
@@ -424,7 +424,7 @@ The block sends the input prompt to a chosen LLM, along with any system prompts
| prompt | The prompt to send to the language model. | str | Yes |
| expected_format | Expected format of the response. If provided, the response will be validated against this format. The keys should be the expected fields in the response, and the values should be the description of the field. | Dict[str, str] | Yes |
| list_result | Whether the response should be a list of objects in the expected format. | bool | No |
-| model | The language model to use for answering the prompt. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for answering the prompt. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
| force_json_output | Whether to force the LLM to produce a JSON-only response. This can increase the block's reliability, but may also reduce the quality of the response because it prohibits the LLM from reasoning before providing its JSON response. | bool | No |
| sys_prompt | The system prompt to provide additional context to the model. | str | No |
| conversation_history | The conversation history to provide context for the prompt. | List[Dict[str, Any]] | No |
@@ -464,7 +464,7 @@ The block sends the input prompt to a chosen LLM, processes the response, and re
| Input | Description | Type | Required |
|-------|-------------|------|----------|
| prompt | The prompt to send to the language model. You can use any of the {keys} from Prompt Values to fill in the prompt with values from the prompt values dictionary by putting them in curly braces. | str | Yes |
-| model | The language model to use for answering the prompt. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for answering the prompt. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
| sys_prompt | The system prompt to provide additional context to the model. | str | No |
| retry | Number of times to retry the LLM call if the response does not match the expected format. | int | No |
| prompt_values | Values used to fill in the prompt. The values can be used in the prompt by putting them in a double curly braces, e.g. {{variable_name}}. | Dict[str, str] | No |
@@ -501,7 +501,7 @@ The block splits the input text into smaller chunks, sends each chunk to an LLM
| Input | Description | Type | Required |
|-------|-------------|------|----------|
| text | The text to summarize. | str | Yes |
-| model | The language model to use for summarizing the text. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for summarizing the text. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
| focus | The topic to focus on in the summary | str | No |
| style | The style of the summary to generate. | "concise" \| "detailed" \| "bullet points" \| "numbered list" | No |
| max_tokens | The maximum number of tokens to generate in the chat completion. | int | No |
@@ -721,7 +721,7 @@ _Add technical explanation here._
| Input | Description | Type | Required |
|-------|-------------|------|----------|
| prompt | The prompt to send to the language model. | str | Yes |
-| model | The language model to use for answering the prompt. | "o4-mini" \| "o3-2025-04-16" \| "o3-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-2025-12-11" \| "gpt-5.2-pro" \| "gpt-5.1-2025-11-13" \| "gpt-5.1-codex" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "gpt-5-nano-2025-08-07" \| "gpt-5-pro" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4.1-nano" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
+| model | The language model to use for answering the prompt. | "o3-mini" \| "o3-2025-04-16" \| "gpt-5.2-2025-12-11" \| "gpt-5.1-2025-11-13" \| "gpt-5-2025-08-07" \| "gpt-5-mini-2025-08-07" \| "o4-mini" \| "o3-pro" \| "o1" \| "o1-mini" \| "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.5-2026-04-23" \| "gpt-5.5-pro" \| "gpt-5.4-2026-03-05" \| "gpt-5.4-mini-2026-03-17" \| "gpt-5.4-nano-2026-03-17" \| "gpt-5.4-pro" \| "gpt-5.3-chat-latest" \| "gpt-5.3-codex" \| "gpt-5.2-pro" \| "gpt-5.1-codex" \| "gpt-5-pro" \| "gpt-4.1-nano" \| "gpt-5-nano-2025-08-07" \| "gpt-5-chat-latest" \| "gpt-4.1-2025-04-14" \| "gpt-4.1-mini-2025-04-14" \| "gpt-4o-mini" \| "gpt-4o" \| "claude-opus-4-5-20251101" \| "claude-sonnet-4-5-20250929" \| "claude-haiku-4-5-20251001" \| "claude-opus-4-6" \| "claude-opus-4-7" \| "claude-sonnet-4-6" \| "meta-llama/Llama-3.3-70B-Instruct-Turbo" \| "llama-3.3-70b-versatile" \| "llama-3.1-8b-instant" \| "llama3.3" \| "llama3.2" \| "llama3" \| "llama3.1:405b" \| "dolphin-mistral:latest" \| "openai/gpt-oss-120b" \| "openai/gpt-oss-20b" \| "google/gemini-2.5-pro" \| "google/gemini-3.1-pro-preview" \| "google/gemini-3-flash-preview" \| "google/gemini-2.5-flash" \| "google/gemini-2.0-flash-001" \| "google/gemini-3.1-flash-lite-preview" \| "google/gemini-2.5-flash-lite" \| "google/gemini-2.0-flash-lite-001" \| "mistralai/mistral-large-2512" \| "mistralai/mistral-medium-3.1" \| "mistralai/mistral-small-3.2-24b-instruct" \| "mistralai/codestral-2508" \| "cohere/command-a-03-2025" \| "cohere/command-a-translate-08-2025" \| "cohere/command-a-reasoning-08-2025" \| "cohere/command-a-vision-07-2025" \| "deepseek/deepseek-chat" \| "deepseek/deepseek-r1-0528" \| "perplexity/sonar" \| "perplexity/sonar-pro" \| "perplexity/sonar-reasoning-pro" \| "perplexity/sonar-deep-research" \| "nousresearch/hermes-3-llama-3.1-405b" \| "nousresearch/hermes-3-llama-3.1-70b" \| "amazon/nova-lite-v1" \| "amazon/nova-micro-v1" \| "amazon/nova-pro-v1" \| "microsoft/phi-4" \| "gryphe/mythomax-l2-13b" \| "meta-llama/llama-4-scout" \| "meta-llama/llama-4-maverick" \| "x-ai/grok-3" \| "x-ai/grok-4" \| "x-ai/grok-4-fast" \| "x-ai/grok-4.1-fast" \| "x-ai/grok-4.20" \| "x-ai/grok-4.20-multi-agent" \| "x-ai/grok-code-fast-1" \| "moonshotai/kimi-k2.5" \| "moonshotai/kimi-k2.6" \| "moonshotai/kimi-k2-thinking" \| "moonshotai/kimi-k3" \| "qwen/qwen3-235b-a22b-thinking-2507" \| "qwen/qwen3-coder" \| "z-ai/glm-4.6" \| "z-ai/glm-4.6v" \| "z-ai/glm-4.7" \| "z-ai/glm-4.7-flash" \| "z-ai/glm-5" \| "z-ai/glm-5-turbo" \| "z-ai/glm-5v-turbo" \| "Llama-4-Scout-17B-16E-Instruct-FP8" \| "Llama-4-Maverick-17B-128E-Instruct-FP8" \| "Llama-3.3-8B-Instruct" \| "Llama-3.3-70B-Instruct" \| "v0-1.5-md" \| "v0-1.5-lg" \| "v0-1.0-md" | No |
| multiple_tool_calls | Whether to allow multiple tool calls in a single response. | bool | No |
| sys_prompt | The system prompt to provide additional context to the model. | str | No |
| conversation_history | The conversation history to provide context for the prompt. | List[Dict[str, Any]] | No |
diff --git a/docs/platform/SUMMARY.md b/docs/platform/SUMMARY.md
index 14ec66457a..0e440dd325 100644
--- a/docs/platform/SUMMARY.md
+++ b/docs/platform/SUMMARY.md
@@ -34,3 +34,7 @@
* [API Introduction](integrating/api-guide.md)
* [OAuth & SSO](integrating/oauth-guide.md)
+
+## Contributing
+
+* [Managing LLM Models](contributing/managing-llm-models.md)
diff --git a/docs/platform/contributing/managing-llm-models.md b/docs/platform/contributing/managing-llm-models.md
new file mode 100644
index 0000000000..f62539f34d
--- /dev/null
+++ b/docs/platform/contributing/managing-llm-models.md
@@ -0,0 +1,128 @@
+# Managing LLM Models
+
+## Overview
+
+The platform manages LLM models **catalog-as-code**: one canonical, schema-validated file is the source of truth for model definitions, per-model costs, and AutoPilot (copilot) routing. There is no admin UI and no model database — you change models by editing the catalog and opening a PR, git history is the audit log, and the normal deploy pipeline propagates the change to every environment.
+
+The catalog lives at:
+
+```
+autogpt_platform/backend/backend/data/llm_registry/catalog.py
+```
+
+Its schema is defined in `catalog_model.py` (same directory), and `catalog_test.py` contains the integrity guards — the file must parse, slugs must be unique, every provider/creator/fallback/routing reference must resolve, and costs must stay within bounds. A catalog PR that passes these tests is structurally sound by construction, which is what makes bot-reviewed catalog changes safe.
+
+## Catalog fields
+
+Each `CatalogModel` entry:
+
+| Field | Meaning |
+| --- | --- |
+| `slug` | Canonical model identifier (e.g. `claude-sonnet-4-6`, `gpt-5.2-2025-12-11`, `moonshotai/kimi-k2.5`). Referenced by routing cells and fallbacks. |
+| `display_name` | Human-readable name shown in UIs. |
+| `provider` | Who serves the model (must match a `CatalogProvider.name`). Determines which credential/API key is used. |
+| `creator` | Who trained the model (display metadata; must match a `CatalogCreator.name`). |
+| `context_window` / `max_output_tokens` | Token limits. |
+| `price_tier` | 1 (cheapest) to 3 (most expensive); used for display. |
+| `is_enabled` | **The kill switch.** A disabled model is refused at serve time — even when LaunchDarkly routes to it. |
+| `visibility` | Who may *see* the model: `GA` (everyone), `EMPLOYEES`, `ADMINS`, or `HIDDEN`. `HIDDEN` models still **serve when explicitly routed** — that is the pre-launch testing state. Informational until the catalog-driven picker lands (today a model stays out of block pickers by not having an enum line); the field is the picker's contract. Visibility never overrides `is_enabled`. |
+| `fallback_model_slug` | Standing replacement pointer: the retirement CLI defaults `--replacement` to it, and it is reserved for future automatic failover. |
+| `supports_*` | Capability flags (tools, JSON output, reasoning, parallel tool calls). Informational and authored opportunistically — `False` means *not asserted*, not "unsupported"; nothing consumes them at runtime yet, so only rely on authored `True` values. |
+| `cost` | What users pay: flat `run_credits` and/or per-1M token **credit** rates (billing reads these). Optionally `provider_*_usd_per_1m`: what the provider charges us — the USD list price, used for in-turn cost estimates when a model is priced off its family default (e.g. Kimi K3's $3/$15). |
+
+> **Cost note:** the catalog IS the billing source. `MODEL_METADATA`,
+> `MODEL_COST`, and `TOKEN_COST` still exist as names, but they are
+> **derived from the catalog at import** — there is nothing else to edit.
+> One transitional artifact: `pre_catalog_costs_snapshot.json` pins the
+> prices billed at the cutover, so changing a **pre-cutover** model's price
+> is a deliberate two-line diff (catalog + snapshot) that shows old→new in
+> review. New models never touch the snapshot, and the first legitimate
+> legacy price change may simply delete the snapshot test instead
+> (it is cutover proof, not a permanent fixture).
+
+`CatalogPayload.routing` holds AutoPilot's routing cells — which model serves each `(mode, tier)` combination. **Cells ship empty**: an unset cell means the `CHAT_*_MODEL` env vars keep that combination, and *claiming* a cell is the explicit act of moving its control into the catalog:
+
+```python
+# Claiming thinking.standard — env vars keep the other three cells:
+routing={
+ "copilot": {
+ "thinking": {"standard": "anthropic/claude-sonnet-4.6"},
+ },
+}
+```
+
+Cell values must be **transport-ready slugs** — the exact spelling the
+serving transport accepts (OpenRouter's vendor-prefixed dot forms, as
+above). The catalog's integrity tests enforce this convention.
+
+**Cells apply only on the managed cloud deployment** (`BEHAVE_AS=cloud`).
+Self-hosted installs — cloud transport or local — always resolve
+LaunchDarkly → env: a cell set for the cloud platform travels in the
+shipped file but never overrides a self-hosted operator's
+`CHAT_*_MODEL` configuration.
+
+## Updating the catalog
+
+What each change touches — this is the complete list:
+
+| Change | You edit |
+| --- | --- |
+| Add a **block-selectable** model | Catalog entry + one `LLMModel` name line (`llm_registry/llm_models.py`). An import-time check refuses to boot if they drift. |
+| Add a **copilot-only** model | Catalog entry. |
+| Change a price (post-cutover model) | Catalog entry. |
+| Change a price (pre-cutover model) | Catalog entry + its snapshot line (see cost note). |
+| Kill / visibility / routing cell | Catalog entry. |
+
+1. Edit `catalog.py` (add a model, change a cell, flip a flag).
+2. Open a PR. Catalog-only diffs are reviewed by the `/review` bot — the integrity tests are the review.
+3. Merge. CD propagates the change with the next deploy.
+
+Two lanes:
+
+- **Ordinary changes** (new models, metadata, visibility promotions) target `dev` and ride the normal release train.
+- **Incident-speed changes** (kills, routing swaps) may use a `hotfix/*` branch targeting `master` — the base-branch check permits this — so the change deploys with CD immediately after merge. Reverting is `git revert` on the same lane. **Immediately merge `master` back to `dev` after a catalog hotfix**: until the back-merge lands, the next release train would silently revert your change (an emergency kill un-killing itself is the worst version of this).
+
+Two notes. The file is public: a `HIDDEN` model is hidden from pickers, **not from anyone reading this repository** — genuinely embargoed models cannot ride this mechanism before announcement. And a catalog-only model (no enum line) simply never surfaces in blocks — it may and should still carry `cost`: copilot cost estimation uses it today and block billing picks it up automatically if the model later gains an enum line.
+
+## How AutoPilot picks a model
+
+Each `(mode, tier)` cell resolves through three layers, top wins:
+
+1. **LaunchDarkly `copilot-model-routing`** — per-user JSON flag returning model slugs; used for cohort experiments and rollouts. Optional: when LD is down, resolution falls through and only A/B targeting is lost.
+2. **Catalog routing cell** — the PR-authored default above.
+3. **`CHAT_*_MODEL` environment variables** — the bootstrap floor (see `.env.default`).
+
+On the managed cloud, the catalog is the serve-time gate for layers 1–2: a slug that is unknown to the catalog or has `is_enabled: False` is refused — logged every time, reported to Sentry once per slug — and resolution falls through to the next layer. A typo'd LD slug therefore degrades to the default instead of erroring at users. Self-hosted installs and local transports skip the gate entirely (LD → env, their slugs are their own business). Assistant messages served by the baseline path are stamped with the model that served them and which layer picked it (`ChatMessage.model` / `routingSource`), which is what allows product-intelligence to compare model quality; the SDK path resolves through the same chain (message stamping covers the baseline path today).
+
+## Rolling out a new model
+
+1. Add the model to the catalog with `visibility="HIDDEN"` — registered and routable, invisible in any picker or public listing.
+2. Add an LD targeting rule on `copilot-model-routing` sending your test cohort (e.g. employees) to its slug.
+3. Watch product-intelligence quality scores segmented by the stamped model column.
+4. Graduate: flip `visibility` to `GA` and set the routing cell in a catalog PR; delete the LD rule.
+
+## Retiring a model
+
+Retirement has two halves:
+
+1. **Stop it serving**: a catalog PR setting `is_enabled: False` (kill switch — beats LD routing). Two caveats: if the model is also a `CHAT_*_MODEL` env default, the env floor still serves it (loudly — log + Sentry) until you change that default; and existing agent graphs referencing it keep executing and billing — the kill switch stops NEW serving, step 2 is what stops stored graphs.
+2. **Migrate existing graph nodes** onto a replacement so users' agents keep working:
+
+```bash
+# dry run — prints affected node count and exits 1
+python -m backend.data.llm_registry.retire <slug> --replacement <replacement-slug>
+
+# execute (transactional, recorded, revertable)
+python -m backend.data.llm_registry.retire <slug> --replacement <replacement-slug> --yes
+
+# inspect / undo
+python -m backend.data.llm_registry.retire --usage <slug>
+python -m backend.data.llm_registry.retire --list
+python -m backend.data.llm_registry.retire --revert <migration-id>
+```
+
+The replacement must exist in the catalog and be enabled. Every executed retirement writes a revertable `LlmModelMigration` record; only one active migration per source model is allowed at a time.
+
+## Reading the catalog from clients
+
+There is deliberately no public catalog API: the catalog ships inside the repo, so every deployment and self-hosted install already has the exact model list its code supports. When a frontend surface needs the live list (e.g. a catalog-driven model picker), add a small authenticated route that reads the in-process registry (`backend.data.llm_registry.registry`) — don't reach for an unauthenticated endpoint; the last one existed only to bootstrap DB-seeded installs, a problem the in-repo catalog no longer has.
diff --git a/docs/platform/copilot-local-llm.md b/docs/platform/copilot-local-llm.md
index 03e6d01cd3..6a17d2f725 100644
--- a/docs/platform/copilot-local-llm.md
+++ b/docs/platform/copilot-local-llm.md
@@ -49,6 +49,19 @@ arrives — there is no 500. The frontend toggle should already be hidden
because the `CHAT_MODE_OPTION` LaunchDarkly flag defaults off in
self-hosted deployments.
+On the managed cloud platform (`BEHAVE_AS=cloud`), `CHAT_*_MODEL` env
+vars are the *bottom* layer of model resolution: LaunchDarkly per-user
+override → the LLM catalog's routing cell
+(`backend/data/llm_registry/catalog.py`) → env default, with slugs
+unknown to the catalog or disabled in it refused at serve time.
+
+**On self-hosted installs — any transport — the catalog's routing cells
+are skipped entirely**: they are the cloud deployment's config traveling
+in the shipped file, and they never override your `CHAT_*_MODEL`
+configuration. Resolution here is LaunchDarkly → `CHAT_*_MODEL`, exactly
+the pre-catalog behavior; your env vars stay authoritative. See
+[Managing LLM Models](contributing/managing-llm-models.md).
+
## Required environment variables
In `autogpt_platform/backend/.env`:
diff --git a/docs/platform/ollama.md b/docs/platform/ollama.md
index ecab9b8ae1..46c02ec93e 100644
--- a/docs/platform/ollama.md
+++ b/docs/platform/ollama.md
@@ -178,48 +178,28 @@ Then you can use the same steps above but you need to add the Ollama server's IP
## Add Custom Models (Advanced)
-If you want to use models other than the default ones, you'll need to add them to the model list. Follow these steps:
+Model definitions are centralized in the LLM catalog — see [Managing LLM Models](contributing/managing-llm-models.md) for the full field reference and workflow. To add a custom Ollama model:
-1. **Add the model to the LlmModel enum** in `autogpt_platform/backend/backend/blocks/llm.py`:
-
- Find the Ollama models section (around line 119) and add your model like the other Ollama models:
+1. **Add a catalog entry** in `autogpt_platform/backend/backend/data/llm_registry/catalog.py`, next to the other Ollama models:
```python
- # Ollama models
- OLLAMA_LLAMA3_3 = "llama3.3"
- OLLAMA_LLAMA3_2 = "llama3.2"
- OLLAMA_YOUR_MODEL = "The-model-name-from-ollama" # Add your model here
+ CatalogModel(
+ slug="The-model-name-from-ollama", # bare name, exactly as Ollama serves it
+ display_name="Your Model",
+ provider="ollama",
+ context_window=8192, # adjust for your model
+ cost=CatalogModelCost(run_credits=1),
+ ),
```
+ Ollama models use bare model names (no `vendor/` prefix) — the name must match what `ollama list` shows. A run cost of `1` is fine for local usage; cost tracking is disabled for self-hosted instances.
-2. **Add model metadata** in the same file:
-
- Find the `MODEL_METADATA` dictionary (around line 181) and add your model with its metadata:
- ```python
- # In MODEL_METADATA dictionary, add:
- LlmModel.OLLAMA_YOUR_MODEL: ModelMetadata("ollama", 8192, None),
- ```
-
- Where:
-
- - `"ollama"` = provider name
- - `8192` = max context window (adjust based on your model)
- - `None` = max output tokens (None means no specific limit)
-
-3. **Add model cost configuration** in `autogpt_platform/backend/backend/data/block_cost_config.py`:
-
- Find the `MODEL_COST` dictionary (around line 54) and add your model:
- ```python
- # In MODEL_COST dictionary, add:
- LlmModel.OLLAMA_YOUR_MODEL: 1,
- ```
-
- > **Note**: Setting cost to `1` is fine for local usage as cost tracking is disabled for self-hosted instances.
+2. **Also add one `LLMModel` name line** in `autogpt_platform/backend/backend/data/llm_registry/llm_models.py` (Ollama section) — that identifier is what block schemas serialize; every model FACT (metadata, costs) lives in the catalog entry itself, and an import-time check refuses to boot if an enum name has no catalog entry. The catalog entry is what centralizes the model's metadata and makes it routable for AutoPilot.
-4. **Rebuild the backend**:
+3. **Rebuild the backend**:
```bash
docker compose up -d --build
```
-5. **Pull the model in Ollama**:
+4. **Pull the model in Ollama**:
```bash
ollama pull your-model-name
```
diff --git a/docs/platform/what-is-autogpt-platform.md b/docs/platform/what-is-autogpt-platform.md
index b0d1ff0f74..2daed2d9c2 100644
--- a/docs/platform/what-is-autogpt-platform.md
+++ b/docs/platform/what-is-autogpt-platform.md
@@ -58,7 +58,7 @@ You can learn more under: [Build your own Blocks](new_blocks.md)
## Available Language Models
-The platform comes pre-integrated with cutting-edge LLM providers:
+The full model list — models, capabilities, context windows, and pricing — lives in the model catalog checked into the platform itself (`autogpt_platform/backend/backend/data/llm_registry/catalog.py`), so every install ships with the exact list its code supports. The platform comes pre-integrated with cutting-edge LLM providers:
* OpenAI - <https://openai.com/>
* Anthropic - <https://www.anthropic.com/>