AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
feat(platform): add ChatGPT/Codex subscription transport preview (#13761)
Nicholas Tindledac59d9184c8
diff --git a/.gitattributes b/.gitattributesindex fe0de10e0b..110c1c3d4d 100644--- a/.gitattributes+++ b/.gitattributes@@ -7,4 +7,7 @@ docs/_javascript/** linguist-vendored# Exclude VCR cassettes from statsclassic/forge/tests/vcr_cassettes/**/**.y*ml linguist-generated-* text=auto\ No newline at end of file+* text=auto++.github/scripts/platform-single-container-*.sh text eol=lf+autogpt_platform/single-container/*.sh text eol=lfdiff --git a/.github/scripts/platform-single-container-smoke.sh b/.github/scripts/platform-single-container-smoke.shindex 2aa1259fe1..1a9bbd71f8 100644--- a/.github/scripts/platform-single-container-smoke.sh+++ b/.github/scripts/platform-single-container-smoke.sh@@ -6,6 +6,7 @@ set -Eeuo pipefail: "${SMOKE_PLATFORM:?SMOKE_PLATFORM is required}"readonly PUBLIC_URL=http://localhost:3300+readonly EXPECTED_CODEX_TEMP_ROOT=/dev/shm/autogpt-codexreadonly TIMEOUT_SECONDS="${SMOKE_TIMEOUT_SECONDS:-2700}"readonly SAFE_PLATFORM="${SMOKE_PLATFORM//\//-}"readonly RUN_TOKEN="${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-0}-${SAFE_PLATFORM}-${RANDOM}"@@ -477,6 +478,110 @@ assert_email_auth_flow() {"${email}"}+assert_codex_provider_discovery() {+ curl --fail-with-body --silent --show-error --max-time 30 \+ --cookie "${AUTH_COOKIE_FILE}" \+ "${PUBLIC_URL}/api/proxy/api/integrations/providers" |+ python3 -c '+import json+import sys++providers = json.load(sys.stdin)+codex = [provider for provider in providers if provider.get("name") == "codex"]+assert len(codex) == 1, codex+assert "oauth2" in codex[0].get("supported_auth_types", []), codex[0]+'+}++assert_codex_runtime_contract() {+ local filesystem_type+ local ownership_and_mode+ local process_name+ local process_id+ local process_environment++ filesystem_type="$(+ docker exec "${CONTAINER_NAME}" \+ stat -f -c '%T' "${EXPECTED_CODEX_TEMP_ROOT}"+ )"+ [[ "${filesystem_type}" == tmpfs ]] || {+ echo "Codex temporary root is not memory-backed: ${filesystem_type}" >&2+ return 1+ }++ ownership_and_mode="$(+ docker exec "${CONTAINER_NAME}" \+ stat -c '%u:%g:%a' "${EXPECTED_CODEX_TEMP_ROOT}"+ )"+ [[ "${ownership_and_mode}" == 10001:10001:700 ]] || {+ echo "Codex temporary root is not owned by autogpt with mode 0700" >&2+ return 1+ }++ for process_name in rest executor copilot-executor; do+ process_id="$(+ docker exec "${CONTAINER_NAME}" \+ supervisorctl \+ -c /opt/autogpt/single-container/supervisor/supervisord.conf \+ pid "${process_name}"+ )"+ [[ "${process_id}" =~ ^[0-9]+$ ]]+ process_environment="$(+ docker exec --user autogpt "${CONTAINER_NAME}" \+ /bin/bash -Eeuo pipefail -c 'tr "\0" "\n" <"/proc/${1}/environ"' \+ bash "${process_id}"+ )"+ grep -Fxq \+ "CODEX_TEMP_ROOT=${EXPECTED_CODEX_TEMP_ROOT}" \+ <<<"${process_environment}" || {+ echo "${process_name} did not inherit CODEX_TEMP_ROOT" >&2+ return 1+ }+ done++ docker exec --interactive --user autogpt \+ --env "CODEX_TEMP_ROOT=${EXPECTED_CODEX_TEMP_ROOT}" \+ --workdir /app/autogpt_platform/backend \+ "${CONTAINER_NAME}" \+ /app/autogpt_platform/backend/.venv/bin/python - <<'PY'+import os+import subprocess+from pathlib import Path++from backend.integrations.codex.runtime import (+ CODEX_RUNTIME_VERSION,+ assert_pinned_versions,+ build_runtime_config,+)+from backend.integrations.codex.temporary_home import TemporaryCodexHome++root = Path(os.environ["CODEX_TEMP_ROOT"])+assert root == Path("/dev/shm/autogpt-codex")+assert_pinned_versions()+before = set(root.iterdir())+with TemporaryCodexHome.create(root) as home:+ config = build_runtime_config(home)+ launch_args = config.launch_args_override+ assert launch_args is not None+ completed = subprocess.run(+ [*launch_args[:3], "--version"],+ env=config.env,+ check=False,+ capture_output=True,+ text=True,+ timeout=30,+ )+ output = completed.stdout + completed.stderr+ assert completed.returncode == 0, output+ assert CODEX_RUNTIME_VERSION in output, output+ home_path = home.path++assert not home_path.exists()+assert set(root.iterdir()) == before+print("codex-sdk-runtime-ok")+PY+}+assert_request_tokens_absent_from_logs() {local sentinel=AUTOGPT_LOG_SENTINEL_6f2b3cb87e9alocal websocket_key=dGhlIHNhbXBsZSBub25jZQ== # pragma: allowlist secret # gitleaks:allow@@ -750,6 +855,7 @@ docker run --detach \"${SMOKE_IMAGE}" >/dev/nulldiscover_data_volumewait_for_healthy+assert_codex_runtime_contractassert_prisma_cli_is_prebundledassert_falkordb_binary_contractassert_memory_contract seed@@ -793,6 +899,7 @@ assert_runtime_config_modeassert_frontend_database_isolationassert_pinned_topology_environmentassert_email_auth_flow+assert_codex_provider_discoveryassert_falkordb_binary_contractassert_memory_contract verifycurl --fail --silent --show-error "${PUBLIC_URL}/healthz" >/dev/nulldiff --git a/.github/workflows/platform-fullstack-ci.yml b/.github/workflows/platform-fullstack-ci.ymlindex 79419ef515..f428f9e120 100644--- a/.github/workflows/platform-fullstack-ci.yml+++ b/.github/workflows/platform-fullstack-ci.yml@@ -164,6 +164,7 @@ jobs:run: |cp ../backend/.env.default ../backend/.envecho "OPENAI_INTERNAL_API_KEY=${{ secrets.OPENAI_API_KEY }}" >> ../backend/.env+ echo "CHAT_TEST_MODE=true" >> ../backend/.envecho "SCHEDULER_STARTUP_EMBEDDING_BACKFILL=false" >> ../backend/.envenv:# Used by E2E test data script to generate embeddings for approved store agentsdiff --git a/autogpt_platform/backend/backend/api/external/v1/integrations.py b/autogpt_platform/backend/backend/api/external/v1/integrations.pyindex bdbcb1f356..b5fe86edac 100644--- a/autogpt_platform/backend/backend/api/external/v1/integrations.py+++ b/autogpt_platform/backend/backend/api/external/v1/integrations.py@@ -31,6 +31,7 @@ from backend.data.model import (UserPasswordCredentials,is_sdk_default,)+from backend.integrations.codex.access import has_codex_access_for_discoveryfrom backend.integrations.credentials_store import (is_system_credential,provider_matches,@@ -272,7 +273,10 @@ async def list_providers(from backend.sdk.registry import AutoRegistryproviders = []+ codex_allowed = await has_codex_access_for_discovery(auth.user_id)for name in get_all_provider_names():+ if name == ProviderName.CODEX and not codex_allowed:+ continuesupports_oauth = name in HANDLERS_BY_NAMEhandler_class = HANDLERS_BY_NAME.get(name)default_scopes = (@@ -540,6 +544,11 @@ async def create_credential(status_code=status.HTTP_404_NOT_FOUND,detail=f"Provider '{provider}' not found",)+ if provider == "codex":+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="Codex credentials must be created through ChatGPT sign-in",+ )# Create the appropriate credential typecredentials: Credentialsdiff --git a/autogpt_platform/backend/backend/api/features/chat/routes.py b/autogpt_platform/backend/backend/api/features/chat/routes.pyindex 26cd9c7606..25fbdb0a42 100644--- a/autogpt_platform/backend/backend/api/features/chat/routes.py+++ b/autogpt_platform/backend/backend/api/features/chat/routes.py@@ -8,7 +8,7 @@ from typing import Annotatedfrom uuid import uuid4from autogpt_libs import auth-from fastapi import APIRouter, Depends, HTTPException, Query, Response, Security+from fastapi import APIRouter, HTTPException, Query, Response, Securityfrom fastapi.responses import StreamingResponsefrom pydantic import BaseModel, ConfigDict, Field, field_validator@@ -22,7 +22,12 @@ from backend.copilot.active_turns import (inflight_turn_limit_message,)from backend.copilot.builder_context import resolve_session_permissions-from backend.copilot.config import ChatConfig, CopilotLLMModel, CopilotMode+from backend.copilot.config import (+ ChatConfig,+ CopilotLlmAuthProvider,+ CopilotLLMModel,+ CopilotMode,+)from backend.copilot.db import get_chat_messages_paginatedfrom backend.copilot.executor.utils import enqueue_cancel_task, schedule_chat_turnfrom backend.copilot.model import (@@ -111,18 +116,27 @@ from backend.copilot.tools.models import (UnderstandingUpdatedResponse,)from backend.data.credit import UsageTransactionMetadata, get_user_credit_model+from backend.data.model import Credentialsfrom backend.data.redis_client import get_redis_asyncfrom backend.data.understanding import get_business_understandingfrom backend.data.workspace import build_files_block, resolve_workspace_files+from backend.integrations.codex.access import (+ enforce_codex_access_http,+ has_codex_access_for_discovery,+)+from backend.integrations.codex.auth_bundle import CodexAuthBundleError+from backend.integrations.codex.credential_codec import bundle_from_credentials+from backend.integrations.creds_manager import IntegrationCredentialsManagerfrom backend.util.background import spawn_background_taskfrom backend.util.exceptions import InsufficientBalanceError, NotFoundError-from backend.util.settings import Settings+from backend.util.settings import BehaveAs, Settingssettings = Settings()logger = logging.getLogger(__name__)config = ChatConfig()+credentials_manager = IntegrationCredentialsManager()async def _validate_and_get_session(@@ -292,6 +306,8 @@ class CreateSessionRequest(BaseModel):dry_run: bool = Falsebuilder_graph_id: str | None = Field(default=None, max_length=128)+ llm_auth_provider: CopilotLlmAuthProvider = "platform"+ llm_credential_id: str | None = Field(default=None, max_length=128)expert_id: str | None = Field(default=None, max_length=128)@@ -305,6 +321,18 @@ class CreateSessionResponse(BaseModel):expert_id: str | None = None+class ChatTransportResponse(BaseModel):+ auth_provider: CopilotLlmAuthProvider+ credential_id: str | None+ label: str+ available: bool+ default: bool+++class ChatTransportsResponse(BaseModel):+ transports: list[ChatTransportResponse]++class ActiveStreamInfo(BaseModel):"""Information about an active stream for reconnection."""@@ -460,10 +488,152 @@ async def list_sessions()- "/sessions",- dependencies=[Depends(enforce_payment_paywall)],+def _is_valid_codex_credentials(credentials: Credentials | None) -> bool:+ if credentials is None or credentials.type != "oauth2":+ return False+ try:+ bundle_from_credentials(credentials)+ except CodexAuthBundleError:+ return False+ return True+++def _is_deployment_chat_available() -> bool:+ if settings.config.behave_as == BehaveAs.CLOUD:+ return True+ api_key, _ = config.main_client_credentials+ return bool(config.test_mode or config.use_claude_code_subscription or api_key)+++async def _get_chat_transports(user_id: str) -> list[ChatTransportResponse]:+ deployment_available = _is_deployment_chat_available()+ transports = [+ ChatTransportResponse(+ auth_provider="platform",+ credential_id=None,+ label=(+ "AutoGPT Platform"+ if settings.config.behave_as == BehaveAs.CLOUD+ else "Self-hosted chat"+ ),+ available=deployment_available,+ default=deployment_available,+ )+ ]++ codex_credentials = (+ await credentials_manager.store.get_creds_by_provider(user_id, "codex")+ if await has_codex_access_for_discovery(user_id)+ else []+ )+ valid_codex_credentials = [+ credentials+ for credentials in codex_credentials+ if _is_valid_codex_credentials(credentials)+ ]+ codex_is_default = not deployment_available and len(valid_codex_credentials) == 1+ transports.extend(+ ChatTransportResponse(+ auth_provider="codex",+ credential_id=credentials.id,+ label="ChatGPT",+ available=True,+ default=codex_is_default,+ )+ for credentials in valid_codex_credentials+ )+ return transports+++ "/transports",+ dependencies=[Security(auth.requires_user)],)+async def list_chat_transports(+ user_id: Annotated[str, Security(auth.get_user_id)],+) -> ChatTransportsResponse:+ return ChatTransportsResponse(transports=await _get_chat_transports(user_id))+++async def _resolve_new_session_llm_route(+ user_id: str,+ request: CreateSessionRequest | None,+) -> tuple[CopilotLlmAuthProvider, str | None]:+ auth_provider = request.llm_auth_provider if request else "platform"+ credential_id = request.llm_credential_id if request else None++ if auth_provider == "codex":+ await enforce_codex_access_http(user_id)++ if request is not None and request.builder_graph_id is not None:+ if auth_provider == "codex" or credential_id is not None:+ raise HTTPException(+ status_code=422,+ detail="codex_builder_session_unsupported",+ )+ if not _is_deployment_chat_available():+ raise HTTPException(+ status_code=503,+ detail="chat_transport_not_configured",+ )+ return "platform", None++ transports = await _get_chat_transports(user_id)+ if request is not None:+ route_was_explicit = bool(+ {"llm_auth_provider", "llm_credential_id"} & request.model_fields_set+ )+ if route_was_explicit:+ if auth_provider == "platform" and credential_id is not None:+ raise HTTPException(+ status_code=422,+ detail="codex_credential_not_allowed",+ )+ if auth_provider == "codex" and credential_id is None:+ raise HTTPException(+ status_code=422,+ detail="codex_credential_required",+ )+ selected_route = next(+ (+ transport+ for transport in transports+ if transport.auth_provider == auth_provider+ and transport.credential_id == credential_id+ and transport.available+ ),+ None,+ )+ if selected_route is None:+ if auth_provider == "codex":+ raise HTTPException(+ status_code=404,+ detail="codex_credential_not_found",+ )+ raise HTTPException(+ status_code=503,+ detail="chat_transport_not_configured",+ )+ return auth_provider, credential_id++ default_route = next(+ (transport for transport in transports if transport.default),+ None,+ )+ if default_route is not None:+ return default_route.auth_provider, default_route.credential_id+ if any(transport.available for transport in transports):+ raise HTTPException(+ status_code=409,+ detail="chat_transport_selection_required",+ )+ raise HTTPException(+ status_code=503,+ detail="chat_transport_not_configured",+ )++[email protected]("/sessions")async def create_session(user_id: Annotated[str, Security(auth.get_user_id)],ctx: Annotated[auth.RequestContext, Security(auth.get_request_context)],@@ -509,6 +679,13 @@ async def create_session(if expert is None or expert.is_archived:raise HTTPException(status_code=404, detail="Expert not found")+ llm_auth_provider, llm_credential_id = await _resolve_new_session_llm_route(+ user_id, request+ )++ if llm_auth_provider == "platform":+ await enforce_payment_paywall(user_id)+logger.info(f"Creating session with user_id: "f"...{user_id[-8:] if len(user_id) > 8 else '<redacted>'}"@@ -518,6 +695,11 @@ async def create_session()if builder_graph_id:+ if llm_auth_provider == "codex":+ raise HTTPException(+ status_code=422,+ detail="codex_builder_session_unsupported",+ )session = await get_or_create_builder_session(user_id,builder_graph_id,@@ -530,6 +712,8 @@ async def create_session(dry_run=dry_run,organization_id=ctx.org_id,team_id=ctx.team_id,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,expert_id=expert_id,)@@ -1145,7 +1329,6 @@ def _empty_ui_message_stream_response() -> StreamingResponse:"header before retrying."},},- dependencies=[Depends(enforce_payment_paywall)],)async def stream_chat_post(session_id: str,@@ -1201,6 +1384,11 @@ async def stream_chat_post(extra={"json_fields": log_meta},)session = await _validate_and_get_session(session_id, user_id)+ is_platform_route = session.metadata.llm_auth_provider == "platform"+ if is_platform_route:+ await enforce_payment_paywall(user_id)+ elif session.metadata.llm_auth_provider == "codex":+ await enforce_codex_access_http(user_id)# Session-anchored tenancy: the ChatSession row is the authoritative# org/team for every turn in it — a user whose active header org@@ -1255,12 +1443,11 @@ async def stream_chat_post(},)- # Pre-turn rate limit check (cost-based, microdollars).- # Entitlement (NO_TIER + ENABLE_PLATFORM_PAYMENT) is gated upstream by- # the route-level ``enforce_payment_paywall`` dependency; here we only- # enforce per-window USD caps. Global defaults sourced from- # LaunchDarkly, falling back to config.- if user_id:+ # Subscription-backed Codex turns do not spend platform model dollars, so+ # neither the platform paywall nor its USD usage windows apply. Admission,+ # pending-message frequency, and concurrent-turn caps remain enforced by+ # the shared scheduling path below.+ if user_id and is_platform_route:try:daily_limit, weekly_limit, _ = await get_global_rate_limits(user_id,@@ -1321,6 +1508,8 @@ async def stream_chat_post(team_id=turn_team_id,mode=request.mode,model=request.model,+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,permissions=builder_permissions,request_arrival_at=request_arrival_at,)@@ -1343,6 +1532,8 @@ async def stream_chat_post(file_ids=sanitized_file_ids,mode=request.mode,model=request.model,+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,permissions=(builder_permissions.model_dump(exclude_none=True)if builder_permissions@@ -1526,7 +1717,9 @@ async def queue_pending_message(user_id: str = Security(auth.get_user_id),):"""Queue a follow-up message while the session has an active turn."""- await _validate_and_get_session(session_id, user_id)+ session = await _validate_and_get_session(session_id, user_id)+ if session.metadata.llm_auth_provider == "codex":+ await enforce_codex_access_http(user_id)try:turn_in_flight = await is_turn_in_flight(session_id)except StreamRegistryUnavailable as exc:diff --git a/autogpt_platform/backend/backend/api/features/chat/routes_test.py b/autogpt_platform/backend/backend/api/features/chat/routes_test.pyindex b993bdb5a1..b3d0d9f106 100644--- a/autogpt_platform/backend/backend/api/features/chat/routes_test.py+++ b/autogpt_platform/backend/backend/api/features/chat/routes_test.py@@ -7,11 +7,20 @@ import fastapiimport fastapi.testclientimport pytestimport pytest_mock+from pydantic import SecretStrfrom backend.api.features.chat import routes as chat_routesfrom backend.api.features.chat.routes import _strip_injected_context+from backend.copilot.config import CopilotLlmAuthProviderfrom backend.copilot.rate_limit import SubscriptionTier+from backend.data.model import OAuth2Credentials+from backend.integrations.codex.auth_bundle import (+ CodexAuthBundleV1,+ CodexAuthTokensV1,+ encode_provider_state,+)from backend.util.exceptions import NotFoundError+from backend.util.settings import BehaveAsapp = fastapi.FastAPI()app.include_router(chat_routes.router)@@ -28,16 +37,42 @@ async def _not_found_handler(client = fastapi.testclient.TestClient(app)TEST_USER_ID = "3e53486c-cf57-477e-ba2a-cb02dc828e1a"+VALID_CODEX_PROVIDER_STATE = encode_provider_state(+ CodexAuthBundleV1(+ tokens=CodexAuthTokensV1(+ id_token=SecretStr("id-token"),+ access_token=SecretStr("access-token"),+ refresh_token=SecretStr("refresh-token"),+ ),+ codex_runtime_version="0.144.4",+ )+)@pytest.fixture(autouse=True)-def setup_app_auth(mock_jwt_user):+def setup_app_auth(mock_jwt_user, mocker: pytest_mock.MockerFixture):"""Setup auth overrides for all tests in this module"""from autogpt_libs.auth.dependencies import get_request_contextfrom autogpt_libs.auth.jwt_utils import get_jwt_payloadapp.dependency_overrides[get_jwt_payload] = mock_jwt_user["get_jwt_payload"]app.dependency_overrides[get_request_context] = mock_jwt_user["get_request_context"]+ mocker.patch.object(+ chat_routes.credentials_manager.store,+ "get_creds_by_provider",+ new=AsyncMock(return_value=[]),+ )+ mocker.patch.object(+ chat_routes,+ "has_codex_access_for_discovery",+ new=AsyncMock(return_value=True),+ )+ mocker.patch.object(+ chat_routes,+ "enforce_codex_access_http",+ new=AsyncMock(),+ )+ mocker.patch.object(chat_routes.settings.config, "behave_as", BehaveAs.CLOUD)yieldapp.dependency_overrides.clear()@@ -212,7 +247,11 @@ def test_stream_chat_rejects_too_many_file_ids():assert response.status_code == 422-def _mock_stream_internals(mocker: pytest_mock.MockerFixture):+def _mock_stream_internals(+ mocker: pytest_mock.MockerFixture,+ *,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+):"""Mock the async internals of stream_chat_post so tests can exercisevalidation and enrichment logic without needing RabbitMQ.@@ -225,15 +264,25 @@ def _mock_stream_internals(mocker: pytest_mock.MockerFixture):# The route anchors turn tenancy on the session row# (session.organization_id / session.team_id), so the stub must carry# both — None exercises the legacy ctx-fallback path.+ mock_session = mocker.MagicMock(organization_id=None, team_id=None)+ mock_session.metadata.llm_auth_provider = llm_auth_provider+ mock_session.metadata.llm_credential_id = (+ "cred-codex" if llm_auth_provider == "codex" else None+ )mocker.patch("backend.api.features.chat.routes._validate_and_get_session",- return_value=mocker.MagicMock(organization_id=None, team_id=None),+ return_value=mock_session,)mocker.patch("backend.api.features.chat.routes.is_turn_in_flight",new_callable=AsyncMock,return_value=False,)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ return_value=None,+ )# ``schedule_chat_turn`` owns acquire-slot + persist-message + dispatch# in one call. Patching it at the route boundary lets tests exercise# validation/enrichment (file_ids, message length, rate limits, etc.)@@ -244,7 +293,13 @@ def _mock_stream_internals(mocker: pytest_mock.MockerFixture):new_callable=AsyncMock,return_value="turn-id-mock",)- return types.SimpleNamespace(enqueue=mock_schedule)+ mocker.patch.object(+ chat_routes.stream_registry,+ "subscribe_to_session",+ new_callable=AsyncMock,+ return_value=None,+ )+ return types.SimpleNamespace(enqueue=mock_schedule, paywall=mock_paywall)def test_stream_chat_accepts_20_file_ids(mocker: pytest_mock.MockerFixture):@@ -377,6 +432,11 @@ def test_stream_chat_returns_429_on_daily_rate_limit(mocker: pytest_mock.MockerF# Ensure the rate-limit branch is entered by setting a non-zero limit.mocker.patch.object(chat_routes.config, "daily_cost_limit_microdollars", 10000)mocker.patch.object(chat_routes.config, "weekly_cost_limit_microdollars", 50000)+ mocker.patch(+ "backend.api.features.chat.routes.get_global_rate_limits",+ new_callable=AsyncMock,+ return_value=(10_000, 50_000, SubscriptionTier.BASIC),+ )mocker.patch("backend.api.features.chat.routes.check_rate_limit",side_effect=RateLimitExceeded("daily", datetime.now(UTC) + timedelta(hours=1)),@@ -390,6 +450,56 @@ def test_stream_chat_returns_429_on_daily_rate_limit(mocker: pytest_mock.MockerFassert "daily" in response.json()["detail"].lower()+def test_stream_chat_codex_skips_platform_paywall_and_cost_limit(+ mocker: pytest_mock.MockerFixture,+) -> None:+ mocks = _mock_stream_internals(mocker, llm_auth_provider="codex")+ mocks.paywall.side_effect = fastapi.HTTPException(+ status_code=402,+ detail="subscription required",+ )+ mock_global_limits = mocker.patch(+ "backend.api.features.chat.routes.get_global_rate_limits",+ new_callable=AsyncMock,+ side_effect=AssertionError("platform cost limits must not run for Codex"),+ )+ mock_cost_limit = mocker.patch(+ "backend.api.features.chat.routes.check_rate_limit",+ new_callable=AsyncMock,+ side_effect=AssertionError("platform cost limits must not run for Codex"),+ )++ response = client.post(+ "/sessions/sess-1/stream",+ json={"message": "hello"},+ )++ assert response.status_code == 200+ mocks.paywall.assert_not_awaited()+ mock_global_limits.assert_not_awaited()+ mock_cost_limit.assert_not_awaited()+ mocks.enqueue.assert_awaited_once()+++def test_stream_chat_platform_still_enforces_paywall(+ mocker: pytest_mock.MockerFixture,+) -> None:+ mocks = _mock_stream_internals(mocker)+ mocks.paywall.side_effect = fastapi.HTTPException(+ status_code=402,+ detail="subscription required",+ )++ response = client.post(+ "/sessions/sess-1/stream",+ json={"message": "hello"},+ )++ assert response.status_code == 402+ mocks.paywall.assert_awaited_once()+ mocks.enqueue.assert_not_awaited()++def test_stream_chat_returns_429_on_weekly_rate_limit(mocker: pytest_mock.MockerFixture,):@@ -691,9 +801,17 @@ def _mock_create_chat_session(mocker: pytest_mock.MockerFixture):organization_id: str | None = None,team_id: str | None = None,source_platform: str | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,expert_id: str | None = None,):- return ChatSession.new(user_id, dry_run=dry_run, expert_id=expert_id)+ return ChatSession.new(+ user_id,+ dry_run=dry_run,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,+ expert_id=expert_id,+ )return mocker.patch("backend.api.features.chat.routes.create_chat_session",@@ -702,6 +820,218 @@ def _mock_create_chat_session(mocker: pytest_mock.MockerFixture):)+def _codex_credentials(+ credential_id: str = "cred-codex",+ *,+ provider_state: str | None = VALID_CODEX_PROVIDER_STATE,+ provider_state_version: int = 1,+) -> OAuth2Credentials:+ return OAuth2Credentials(+ id=credential_id,+ provider="codex",+ access_token=SecretStr("access"),+ refresh_token=SecretStr("refresh"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr(provider_state) if provider_state else None,+ provider_state_version=provider_state_version,+ )+++def _set_self_hosted_chat_config(+ mocker: pytest_mock.MockerFixture,+ *,+ configured: bool,+) -> None:+ mocker.patch.object(chat_routes.settings.config, "behave_as", BehaveAs.LOCAL)+ self_hosted_config = MagicMock()+ self_hosted_config.test_mode = False+ self_hosted_config.use_claude_code_subscription = False+ self_hosted_config.main_client_credentials = (+ ("configured-key", "http://llm.example/v1")+ if configured+ else (None, "https://api.anthropic.com/v1/")+ )+ mocker.patch.object(chat_routes, "config", self_hosted_config)+++def test_list_chat_transports_hosted_platform_only(+ test_user_id: str,+) -> None:+ response = client.get("/transports")++ assert response.status_code == 200+ assert response.json() == {+ "transports": [+ {+ "auth_provider": "platform",+ "credential_id": None,+ "label": "AutoGPT Platform",+ "available": True,+ "default": True,+ }+ ]+ }+++def test_list_chat_transports_hosted_defaults_to_platform_with_codex(+ test_user_id: str,+) -> None:+ chat_routes.credentials_manager.store.get_creds_by_provider.return_value = [+ _codex_credentials()+ ]++ response = client.get("/transports")++ assert response.status_code == 200+ assert response.json() == {+ "transports": [+ {+ "auth_provider": "platform",+ "credential_id": None,+ "label": "AutoGPT Platform",+ "available": True,+ "default": True,+ },+ {+ "auth_provider": "codex",+ "credential_id": "cred-codex",+ "label": "ChatGPT",+ "available": True,+ "default": False,+ },+ ]+ }+++def test_list_chat_transports_hosted_omits_codex_without_required_plan(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+) -> None:+ access = mocker.patch.object(+ chat_routes,+ "has_codex_access_for_discovery",+ new=AsyncMock(return_value=False),+ )+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ lookup.return_value = [_codex_credentials()]++ response = client.get("/transports")++ assert response.status_code == 200+ assert response.json() == {+ "transports": [+ {+ "auth_provider": "platform",+ "credential_id": None,+ "label": "AutoGPT Platform",+ "available": True,+ "default": True,+ }+ ]+ }+ access.assert_awaited_once_with(test_user_id)+ lookup.assert_not_awaited()+++def test_list_chat_transports_omits_invalid_codex_credentials(+ test_user_id: str,+) -> None:+ chat_routes.credentials_manager.store.get_creds_by_provider.return_value = [+ _codex_credentials(provider_state=None)+ ]++ response = client.get("/transports")++ assert response.status_code == 200+ assert [+ transport["auth_provider"] for transport in response.json()["transports"]+ ] == ["platform"]++[email protected]("configured", [False, True])+def test_list_chat_transports_self_hosted_platform_availability(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+ configured: bool,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=configured)++ response = client.get("/transports")++ assert response.status_code == 200+ assert response.json() == {+ "transports": [+ {+ "auth_provider": "platform",+ "credential_id": None,+ "label": "Self-hosted chat",+ "available": configured,+ "default": configured,+ }+ ]+ }+++def test_list_chat_transports_self_hosted_codex_is_default_without_deployment(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=False)+ chat_routes.credentials_manager.store.get_creds_by_provider.return_value = [+ _codex_credentials()+ ]++ response = client.get("/transports")++ assert response.status_code == 200+ assert response.json()["transports"] == [+ {+ "auth_provider": "platform",+ "credential_id": None,+ "label": "Self-hosted chat",+ "available": False,+ "default": False,+ },+ {+ "auth_provider": "codex",+ "credential_id": "cred-codex",+ "label": "ChatGPT",+ "available": True,+ "default": True,+ },+ ]+++def test_list_chat_transports_self_hosted_deployment_stays_default_with_codex(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=True)+ chat_routes.credentials_manager.store.get_creds_by_provider.return_value = [+ _codex_credentials()+ ]++ response = client.get("/transports")++ assert response.status_code == 200+ platform, codex = response.json()["transports"]+ assert platform == {+ "auth_provider": "platform",+ "credential_id": None,+ "label": "Self-hosted chat",+ "available": True,+ "default": True,+ }+ assert codex == {+ "auth_provider": "codex",+ "credential_id": "cred-codex",+ "label": "ChatGPT",+ "available": True,+ "default": False,+ }++def test_create_session_dry_run_true(mocker: pytest_mock.MockerFixture,test_user_id: str,@@ -742,6 +1072,280 @@ def test_create_session_rejects_nested_metadata(assert response.status_code == 422+def test_create_session_rejects_credential_on_platform_route() -> None:+ response = client.post(+ "/sessions",+ json={"llm_auth_provider": "platform", "llm_credential_id": "cred-1"},+ )++ assert response.status_code == 422+ assert response.json()["detail"] == "codex_credential_not_allowed"+++def test_create_session_requires_credential_for_codex_route() -> None:+ response = client.post(+ "/sessions",+ json={"llm_auth_provider": "codex"},+ )++ assert response.status_code == 422+ assert response.json()["detail"] == "codex_credential_required"+++def test_create_session_codex_route_rejects_unowned_credential(+ test_user_id: str,+) -> None:+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider++ response = client.post(+ "/sessions",+ json={"llm_auth_provider": "codex", "llm_credential_id": "other-cred"},+ )++ assert response.status_code == 404+ assert response.json()["detail"] == "codex_credential_not_found"+ lookup.assert_awaited_once_with(test_user_id, "codex")+++def test_create_session_codex_route_rejects_user_without_required_plan(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+) -> None:+ gate = mocker.patch.object(+ chat_routes,+ "enforce_codex_access_http",+ new=AsyncMock(+ side_effect=fastapi.HTTPException(+ status_code=402,+ detail="A Max plan or higher is required to use ChatGPT.",+ )+ ),+ )+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ mock_create = mocker.patch.object(+ chat_routes,+ "create_chat_session",+ new=AsyncMock(),+ )++ response = client.post(+ "/sessions",+ json={"llm_auth_provider": "codex", "llm_credential_id": "cred-1"},+ )++ assert response.status_code == 402+ assert response.json()["detail"] == (+ "A Max plan or higher is required to use ChatGPT."+ )+ gate.assert_awaited_once_with(test_user_id)+ lookup.assert_not_awaited()+ mock_create.assert_not_awaited()+++def test_create_session_codex_route_persists_owned_credential(+ mocker: pytest_mock.MockerFixture,+) -> None:+ mock_create = _mock_create_chat_session(mocker)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ side_effect=fastapi.HTTPException(+ status_code=402,+ detail="subscription required",+ ),+ )+ credential = _codex_credentials("cred-1")+ chat_routes.credentials_manager.store.get_creds_by_provider.return_value = [+ credential+ ]++ response = client.post(+ "/sessions",+ json={"llm_auth_provider": "codex", "llm_credential_id": "cred-1"},+ )++ assert response.status_code == 200+ assert response.json()["metadata"]["llm_auth_provider"] == "codex"+ assert response.json()["metadata"]["llm_credential_id"] == "cred-1"+ assert mock_create.call_args.kwargs["llm_auth_provider"] == "codex"+ assert mock_create.call_args.kwargs["llm_credential_id"] == "cred-1"+ mock_paywall.assert_not_awaited()+++def test_create_session_hosted_defaults_to_platform_with_codex_connected(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+) -> None:+ credential = _codex_credentials()+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ lookup.return_value = [credential]+ mock_create = _mock_create_chat_session(mocker)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ )++ response = client.post("/sessions", json={})++ assert response.status_code == 200+ assert response.json()["metadata"]["llm_auth_provider"] == "platform"+ assert response.json()["metadata"]["llm_credential_id"] is None+ assert mock_create.call_args.kwargs["llm_auth_provider"] == "platform"+ assert mock_create.call_args.kwargs["llm_credential_id"] is None+ lookup.assert_awaited_once_with(test_user_id, "codex")+ mock_paywall.assert_awaited_once_with(test_user_id)+++def test_create_session_self_hosted_defaults_to_codex_when_unconfigured(+ mocker: pytest_mock.MockerFixture,+ test_user_id: str,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=False)+ credential = _codex_credentials()+ chat_routes.credentials_manager.store.get_creds_by_provider.return_value = [+ credential+ ]+ mock_create = _mock_create_chat_session(mocker)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ )++ response = client.post("/sessions", json={})++ assert response.status_code == 200+ assert response.json()["metadata"]["llm_auth_provider"] == "codex"+ assert response.json()["metadata"]["llm_credential_id"] == "cred-codex"+ assert mock_create.call_args.kwargs["llm_auth_provider"] == "codex"+ assert mock_create.call_args.kwargs["llm_credential_id"] == "cred-codex"+ mock_paywall.assert_not_awaited()+++def test_create_session_self_hosted_without_any_transport_returns_setup_error(+ mocker: pytest_mock.MockerFixture,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=False)+ mock_create = _mock_create_chat_session(mocker)++ response = client.post("/sessions", json={})++ assert response.status_code == 503+ assert response.json()["detail"] == "chat_transport_not_configured"+ mock_create.assert_not_awaited()+++def test_create_session_self_hosted_rejects_explicit_unconfigured_platform(+ mocker: pytest_mock.MockerFixture,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=False)+ mock_create = _mock_create_chat_session(mocker)++ response = client.post(+ "/sessions",+ json={"llm_auth_provider": "platform"},+ )++ assert response.status_code == 503+ assert response.json()["detail"] == "chat_transport_not_configured"+ mock_create.assert_not_awaited()+++ ("provider_state", "provider_state_version"),+ [+ (None, 1),+ ("{}", 1),+ (VALID_CODEX_PROVIDER_STATE, 2),+ ],+)+def test_create_session_self_hosted_rejects_invalid_codex_as_unconfigured(+ mocker: pytest_mock.MockerFixture,+ provider_state: str | None,+ provider_state_version: int,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=False)+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ lookup.return_value = [+ _codex_credentials(+ provider_state=provider_state,+ provider_state_version=provider_state_version,+ )+ ]+ mock_create = _mock_create_chat_session(mocker)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ )++ response = client.post("/sessions", json={})++ assert response.status_code == 503+ assert response.json()["detail"] == "chat_transport_not_configured"+ mock_create.assert_not_awaited()+ mock_paywall.assert_not_awaited()+++def test_create_session_self_hosted_requires_selection_for_multiple_codex_routes(+ mocker: pytest_mock.MockerFixture,+) -> None:+ _set_self_hosted_chat_config(mocker, configured=False)+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ lookup.return_value = [+ _codex_credentials("cred-one"),+ _codex_credentials("cred-two"),+ ]+ mock_create = _mock_create_chat_session(mocker)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ )++ response = client.post("/sessions", json={})++ assert response.status_code == 409+ assert response.json()["detail"] == "chat_transport_selection_required"+ mock_create.assert_not_awaited()+ mock_paywall.assert_not_awaited()+++def test_create_session_respects_explicit_platform_route(+ mocker: pytest_mock.MockerFixture,+) -> None:+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ mock_create = _mock_create_chat_session(mocker)+ mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ )++ response = client.post("/sessions", json={"llm_auth_provider": "platform"})++ assert response.status_code == 200+ assert response.json()["metadata"]["llm_auth_provider"] == "platform"+ assert mock_create.call_args.kwargs["llm_auth_provider"] == "platform"+ lookup.assert_awaited_once()+++def test_create_session_platform_route_still_enforces_paywall(+ mocker: pytest_mock.MockerFixture,+) -> None:+ mock_create = _mock_create_chat_session(mocker)+ mock_paywall = mocker.patch(+ "backend.api.features.chat.routes.enforce_payment_paywall",+ new_callable=AsyncMock,+ side_effect=fastapi.HTTPException(+ status_code=402,+ detail="subscription required",+ ),+ )++ response = client.post("/sessions", json={"llm_auth_provider": "platform"})++ assert response.status_code == 402+ mock_paywall.assert_awaited_once()+ mock_create.assert_not_awaited()++# ─── Create session: expert_id ──────────────────────────────────────────@@ -2416,6 +3020,8 @@ def test_create_session_with_builder_graph_id_uses_get_or_create(new_callable=AsyncMock,side_effect=_fake_get_or_create,)+ lookup = chat_routes.credentials_manager.store.get_creds_by_provider+ lookup.return_value = [_codex_credentials()]response = client.post("/sessions", json={"builder_graph_id": "graph-1"})@@ -2423,6 +3029,8 @@ def test_create_session_with_builder_graph_id_uses_get_or_create(body = response.json()assert body["metadata"]["builder_graph_id"] == "graph-1"assert body["metadata"]["dry_run"] is False+ assert body["metadata"]["llm_auth_provider"] == "platform"+ lookup.assert_not_awaited()def test_create_session_with_builder_graph_id_returns_404_when_not_owned(diff --git a/autogpt_platform/backend/backend/api/features/integrations/codex.py b/autogpt_platform/backend/backend/api/features/integrations/codex.pynew file mode 100644index 0000000000..db70d84332--- /dev/null+++ b/autogpt_platform/backend/backend/api/features/integrations/codex.py@@ -0,0 +1,263 @@+import logging+import secrets+from typing import TYPE_CHECKING, Annotated, Literal+from urllib.parse import quote, urlencode++from autogpt_libs.auth import get_user_id+from fastapi import APIRouter, HTTPException, Path, Security, status+from fastapi.responses import HTMLResponse+from pydantic import BaseModel++from backend.api.features.integrations.codex_device_page import render_device_login_page+from backend.integrations.codex.access import enforce_codex_access_http+from backend.integrations.codex.login import (+ CodexDeviceLogin,+ CodexDeviceLoginState,+ CodexLoginCoordinator,+)+from backend.integrations.codex.models import (+ CodexAccountSnapshot,+ CodexRateLimitsSnapshot,+)+from backend.integrations.credential_lease import CredentialLease+from backend.integrations.creds_manager import IntegrationCredentialsManager+from backend.util.settings import Settings++if TYPE_CHECKING:+ from backend.integrations.codex.transport import CodexTransport++CODEX_LOGIN_STATE_KEY = "codex_login_id"++logger = logging.getLogger(__name__)+router = APIRouter()+codex_login_coordinator = CodexLoginCoordinator(+ timeout_seconds=Settings().config.codex_login_timeout_seconds+)+codex_credentials_manager = IntegrationCredentialsManager()+++class CodexLoginStatusResponse(BaseModel):+ status: Literal["pending", "completed", "failed", "canceled"]+ error: str | None = None+++ "/credentials/{credential_id}/account",+ response_model=CodexAccountSnapshot,+)+async def codex_account(+ credential_id: Annotated[str, Path(min_length=1)],+ user_id: Annotated[str, Security(get_user_id)],+) -> CodexAccountSnapshot:+ await enforce_codex_access_http(user_id)+ lease = await _acquire_codex_lease(user_id, credential_id)+ try:+ return await _get_codex_transport().account(lease)+ except Exception as error:+ logger.warning("Codex account check failed with %s", type(error).__name__)+ raise HTTPException(+ status_code=status.HTTP_502_BAD_GATEWAY,+ detail="Could not read the connected ChatGPT account",+ ) from None+ finally:+ await lease.release()+++ "/credentials/{credential_id}/rate-limits",+ response_model=CodexRateLimitsSnapshot,+)+async def codex_rate_limits(+ credential_id: Annotated[str, Path(min_length=1)],+ user_id: Annotated[str, Security(get_user_id)],+) -> CodexRateLimitsSnapshot:+ await enforce_codex_access_http(user_id)+ lease = await _acquire_codex_lease(user_id, credential_id)+ try:+ return await _get_codex_transport().rate_limits(lease)+ except Exception as error:+ logger.warning("Codex rate-limit check failed with %s", type(error).__name__)+ raise HTTPException(+ status_code=status.HTTP_502_BAD_GATEWAY,+ detail="Could not read ChatGPT usage limits",+ ) from None+ finally:+ await lease.release()+++ "/device-login/{login_id}",+ response_class=HTMLResponse,+ include_in_schema=False,+)+async def device_login_page(+ login_id: Annotated[str, Path(min_length=1)],+ user_id: Annotated[str, Security(get_user_id)],+) -> HTMLResponse:+ attempt = await codex_login_coordinator.get(user_id, login_id)+ if attempt is None:+ raise _login_not_found()+ nonce = secrets.token_urlsafe(18)+ return HTMLResponse(render_device_login_page(nonce), headers=_page_headers(nonce))+++ "/device-login/{login_id}/status",+ response_model=CodexLoginStatusResponse,+ include_in_schema=False,+)+async def device_login_status(+ login_id: Annotated[str, Path(min_length=1)],+ user_id: Annotated[str, Security(get_user_id)],+) -> CodexLoginStatusResponse:+ attempt = await codex_login_coordinator.get(user_id, login_id)+ if attempt is None:+ raise _login_not_found()+ return _status_response(attempt)+++ "/device-login/{login_id}/cancel",+ status_code=status.HTTP_204_NO_CONTENT,+ include_in_schema=False,+)+async def cancel_device_login(+ login_id: Annotated[str, Path(min_length=1)],+ user_id: Annotated[str, Security(get_user_id)],+) -> None:+ try:+ found = await codex_login_coordinator.cancel(user_id, login_id)+ except Exception as error:+ logger.warning(+ "Codex device-login cancellation failed with %s", type(error).__name__+ )+ return+ if not found:+ raise _login_not_found()+++def build_device_login_url(+ frontend_base_url: str,+ login: CodexDeviceLogin,+ state_token: str,+) -> str:+ base_url = frontend_base_url.rstrip("/")+ login_id = quote(login.login_id, safe="")+ fragment = urlencode(+ {+ "state": state_token,+ "verification_url": login.verification_url,+ "user_code": login.user_code,+ }+ )+ return (+ f"{base_url}/api/proxy/api/integrations/codex/"+ f"device-login/{login_id}#{fragment}"+ )+++def build_device_login_cancel_url(+ login: CodexDeviceLogin,+) -> str:+ login_id = quote(login.login_id, safe="")+ return f"/api/proxy/api/integrations/codex/device-login/{login_id}/cancel"+++async def revoke_codex_credentials(+ manager: IntegrationCredentialsManager,+ user_id: str,+ credential_id: str,+) -> bool:+ try:+ lease = await manager.acquire_lease(user_id, credential_id)+ except ValueError:+ return False+ try:+ if lease.credentials.provider != "codex":+ return False+ revoked = False+ if _is_codex_lease(lease):+ try:+ await _get_codex_transport().logout(lease)+ revoked = True+ except Exception as error:+ logger.warning("Codex logout failed with %s", type(error).__name__)+ await lease.delete()+ return revoked+ finally:+ await lease.release()+++async def _acquire_codex_lease(+ user_id: str,+ credential_id: str,+) -> CredentialLease:+ try:+ lease = await codex_credentials_manager.acquire_lease(user_id, credential_id)+ except ValueError:+ raise _credential_not_found() from None+ if not _is_codex_lease(lease):+ await lease.release()+ raise _credential_not_found()+ return lease+++def _is_codex_lease(lease: CredentialLease) -> bool:+ credentials = lease.credentials+ return (+ credentials.type == "oauth2"+ and credentials.provider == "codex"+ and credentials.refresh_strategy == "provider_runtime"+ )+++def _get_codex_transport() -> "CodexTransport":+ from backend.integrations.codex.transport import get_codex_transport++ return get_codex_transport()+++def _status_response(attempt: CodexDeviceLoginState) -> CodexLoginStatusResponse:+ if attempt.status not in ("pending", "completed", "failed", "canceled"):+ raise RuntimeError("Codex login coordinator returned an invalid status")+ return CodexLoginStatusResponse(status=attempt.status, error=attempt.error)+++def _login_not_found() -> HTTPException:+ return HTTPException(+ status_code=status.HTTP_404_NOT_FOUND,+ detail="Codex login not found",+ )+++def _credential_not_found() -> HTTPException:+ return HTTPException(+ status_code=status.HTTP_404_NOT_FOUND,+ detail="Credentials not found",+ )+++def _page_headers(nonce: str) -> dict[str, str]:+ return {+ "Cache-Control": "no-store, no-cache, must-revalidate, private",+ "Content-Security-Policy": (+ "default-src 'none'; "+ f"script-src 'nonce-{nonce}'; style-src 'nonce-{nonce}'; "+ "connect-src 'self'; base-uri 'none'; form-action 'none'; "+ "frame-ancestors 'none'"+ ),+ "Referrer-Policy": "no-referrer",+ "X-Content-Type-Options": "nosniff",+ }+++__all__ = [+ "CODEX_LOGIN_STATE_KEY",+ "CodexDeviceLogin",+ "CodexDeviceLoginState",+ "build_device_login_cancel_url",+ "build_device_login_url",+ "render_device_login_page",+ "revoke_codex_credentials",+ "router",+]diff --git a/autogpt_platform/backend/backend/api/features/integrations/codex_device_page.py b/autogpt_platform/backend/backend/api/features/integrations/codex_device_page.pynew file mode 100644index 0000000000..b994957bf7--- /dev/null+++ b/autogpt_platform/backend/backend/api/features/integrations/codex_device_page.py@@ -0,0 +1,144 @@+import html+++def render_device_login_page(nonce: str) -> str:+ page = """<!doctype html>+<html lang="en">+ <head>+ <meta charset="utf-8">+ <meta name="viewport" content="width=device-width, initial-scale=1">+ <meta name="referrer" content="no-referrer">+ <title>Connect ChatGPT for Codex</title>+ <style{{NONCE}}>+ :root { color-scheme: light; font-family: Inter, ui-sans-serif, system-ui; }+ body { margin: 0; min-height: 100vh; display: grid; place-items: center;+ background: #f7f7f8; color: #202123; }+ main { width: min(420px, calc(100vw - 48px)); padding: 32px;+ border: 1px solid #dedee3; border-radius: 16px; background: white;+ box-shadow: 0 16px 48px rgb(0 0 0 / 8%); }+ h1 { margin: 0 0 10px; font-size: 24px; }+ p { margin: 0 0 20px; color: #5f6068; line-height: 1.5; }+ code { display: block; margin: 16px 0; padding: 16px; border-radius: 10px;+ background: #f0f0f2; font-size: 24px; font-weight: 650;+ letter-spacing: .12em; text-align: center; color: #202123; }+ .actions { display: grid; gap: 10px; }+ a, button { min-height: 44px; border-radius: 10px; border: 1px solid #202123;+ padding: 0 16px; display: inline-flex; align-items: center;+ justify-content: center; font: inherit; font-weight: 600; cursor: pointer; }+ a { background: #202123; color: white; text-decoration: none; }+ button { background: white; color: #202123; }+ #status { margin-top: 18px; font-size: 14px; color: #5f6068; }+ #error { color: #b42318; }+ </style>+ </head>+ <body>+ <main>+ <h1>Connect ChatGPT for Codex</h1>+ <p>Open the ChatGPT verification page, enter this one-time code, then return here.</p>+ <code id="user-code">Loading…</code>+ <div class="actions">+ <a id="verification-link" target="_blank" rel="noopener noreferrer">Open ChatGPT sign-in</a>+ <button id="copy-code" type="button">Copy code</button>+ </div>+ <div id="status" role="status">Waiting for sign-in…</div>+ <div id="error" role="alert"></div>+ </main>+ <script{{NONCE}}>+ (function () {+ let finished = false;+ const fragment = new URLSearchParams(window.location.hash.slice(1));+ const state = fragment.get("state");+ const userCode = fragment.get("user_code");+ const rawVerificationUrl = fragment.get("verification_url");+ let loginID = "";+ try {+ loginID = decodeURIComponent(window.location.pathname.split("/").pop());+ } catch (_) {}+ window.history.replaceState(null, "", window.location.pathname);++ const codeElement = document.getElementById("user-code");+ const linkElement = document.getElementById("verification-link");+ const statusElement = document.getElementById("status");+ const errorElement = document.getElementById("error");+ const copyButton = document.getElementById("copy-code");++ function fail(message) {+ statusElement.textContent = "";+ errorElement.textContent = message;+ }++ window.addEventListener("pagehide", function () {+ if (!finished) {+ navigator.sendBeacon(window.location.pathname + "/cancel");+ }+ });++ let verificationUrl;+ try {+ verificationUrl = new URL(rawVerificationUrl);+ if (verificationUrl.protocol !== "https:") throw new Error();+ } catch (_) {+ fail("The ChatGPT verification link is invalid. Close this window and try again.");+ return;+ }+ if (!state || !userCode || !loginID) {+ fail("This sign-in attempt is incomplete. Close this window and try again.");+ return;+ }++ codeElement.textContent = userCode;+ linkElement.href = verificationUrl.toString();+ copyButton.addEventListener("click", async function () {+ try {+ await navigator.clipboard.writeText(userCode);+ copyButton.textContent = "Copied";+ } catch (_) {+ fail("Could not copy automatically. Select the code above instead.");+ }+ });++ const pollDeadline = Date.now() + 20 * 60 * 1000;+ async function poll() {+ if (Date.now() >= pollDeadline) {+ finished = true;+ fail("ChatGPT sign-in expired. Close this window and try again.");+ return;+ }+ try {+ const response = await fetch(window.location.pathname + "/status", {+ cache: "no-store",+ credentials: "same-origin",+ headers: { Accept: "application/json" },+ });+ if (response.status === 404) {+ finished = true;+ fail("ChatGPT sign-in expired. Close this window and try again.");+ return;+ }+ if (!response.ok) throw new Error();+ const result = await response.json();+ if (result.status === "completed") {+ finished = true;+ statusElement.textContent = "Connected. Returning to AutoGPT…";+ const query = new URLSearchParams({ code: loginID, state: state });+ window.location.replace("/auth/integrations/oauth_callback?" + query);+ return;+ }+ if (result.status === "failed" || result.status === "canceled") {+ finished = true;+ fail(result.error || "ChatGPT sign-in failed. Close this window and try again.");+ return;+ }+ } catch (_) {+ statusElement.textContent = "Reconnecting to AutoGPT…";+ }+ window.setTimeout(poll, 1000);+ }++ poll();+ })();+ </script>+ </body>+</html>"""+ nonce_attribute = f' nonce="{html.escape(nonce, quote=True)}"' if nonce else ""+ return page.replace("{{NONCE}}", nonce_attribute)diff --git a/autogpt_platform/backend/backend/api/features/integrations/codex_test.py b/autogpt_platform/backend/backend/api/features/integrations/codex_test.pynew file mode 100644index 0000000000..97b5857f39--- /dev/null+++ b/autogpt_platform/backend/backend/api/features/integrations/codex_test.py@@ -0,0 +1,669 @@+from unittest.mock import AsyncMock, MagicMock, patch++import fastapi+import fastapi.testclient+import pytest+from autogpt_libs.auth import get_optional_user_id+from pydantic import SecretStr++from backend.api.features.integrations.codex import (+ CODEX_LOGIN_STATE_KEY,+ CodexDeviceLogin,+ CodexDeviceLoginState,+ build_device_login_cancel_url,+ build_device_login_url,+ render_device_login_page,+ revoke_codex_credentials,+)+from backend.api.features.integrations.router import router+from backend.data.model import OAuth2Credentials, OAuthState+from backend.integrations.codex.models import (+ CodexAccountSnapshot,+ CodexRateLimitsSnapshot,+ CodexRateLimitWindow,+)++app = fastapi.FastAPI()+app.include_router(router)+client = fastapi.testclient.TestClient(app)++TEST_USER_ID = "3e53486c-cf57-477e-ba2a-cb02dc828e1a"++[email protected](autouse=True)+def setup_auth(mocker):+ from autogpt_libs.auth.jwt_utils import get_jwt_payload++ app.dependency_overrides[get_jwt_payload] = lambda: {+ "sub": TEST_USER_ID,+ "role": "user",+ "email": "[email protected]",+ }+ app.dependency_overrides[get_optional_user_id] = lambda: TEST_USER_ID+ mocker.patch(+ "backend.api.features.integrations.router.has_codex_access_for_discovery",+ new=AsyncMock(return_value=True),+ )+ mocker.patch(+ "backend.api.features.integrations.router.enforce_codex_access_http",+ new=AsyncMock(),+ )+ mocker.patch(+ "backend.api.features.integrations.codex.enforce_codex_access_http",+ new=AsyncMock(),+ )+ yield+ app.dependency_overrides.clear()+++def _credentials() -> OAuth2Credentials:+ return OAuth2Credentials(+ id="codex-credential",+ provider="codex",+ title="ChatGPT for Codex",+ username="[email protected]",+ access_token=SecretStr("access-secret"),+ refresh_token=SecretStr("refresh-secret"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr("provider-secret"),+ provider_state_version=1,+ )+++def _oauth_state(login_id: str = "login-123") -> OAuthState:+ return OAuthState(+ token="state-123",+ provider="codex",+ expires_at=4_000_000_000,+ scopes=[],+ state_metadata={CODEX_LOGIN_STATE_KEY: login_id},+ )+++def test_codex_login_reuses_generic_oauth_contract():+ login = CodexDeviceLogin(+ login_id="login-123",+ verification_url="https://auth.openai.com/codex/device",+ user_code="ABCD-EFGH",+ )+ with (+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ patch("backend.api.features.integrations.router.settings") as settings,+ ):+ settings.config.frontend_base_url = "http://localhost:3000"+ settings.config.codex_login_timeout_seconds = 900+ coordinator.start = AsyncMock(return_value=login)+ manager.store.store_state_token = AsyncMock(+ return_value=("state-123", "unused-challenge")+ )++ response = client.get("/codex/login")++ assert response.status_code == 200+ payload = response.json()+ assert payload["state_token"] == "state-123"+ assert payload["login_url"].startswith(+ "http://localhost:3000/api/proxy/api/integrations/codex/device-login/login-123#"+ )+ assert payload["cancel_url"] == (+ "/api/proxy/api/integrations/codex/device-login/login-123/cancel"+ )+ assert "state=state-123" in payload["login_url"].split("#", 1)[1]+ assert "user_code=ABCD-EFGH" in payload["login_url"].split("#", 1)[1]+ manager.store.store_state_token.assert_awaited_once_with(+ TEST_USER_ID,+ "codex",+ [],+ expires_in_seconds=960,+ credential_id=None,+ state_metadata={CODEX_LOGIN_STATE_KEY: "login-123"},+ )+++def test_codex_login_start_failure_is_sanitized(caplog):+ with (+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.settings") as settings,+ ):+ settings.config.frontend_base_url = "http://localhost:3000"+ coordinator.start = AsyncMock(+ side_effect=RuntimeError(+ "user_code=ABCD-EFGH verification_url=https://secret.example"+ )+ )++ response = client.get("/codex/login")++ assert response.status_code == 503+ assert response.json()["detail"] == ("ChatGPT sign-in is temporarily unavailable")+ assert "ABCD-EFGH" not in response.text+ assert "ABCD-EFGH" not in caplog.text+ assert "secret.example" not in response.text+ assert "secret.example" not in caplog.text+++def test_codex_login_preserves_state_store_error_when_cancel_fails():+ login = CodexDeviceLogin(+ login_id="login-123",+ verification_url="https://auth.openai.com/codex/device",+ user_code="ABCD-EFGH",+ )+ with (+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ patch("backend.api.features.integrations.router.settings") as settings,+ ):+ settings.config.frontend_base_url = "http://localhost:3000"+ settings.config.codex_login_timeout_seconds = 900+ coordinator.start = AsyncMock(return_value=login)+ coordinator.cancel = AsyncMock(side_effect=RuntimeError("cancel failed"))+ manager.store.store_state_token = AsyncMock(+ side_effect=RuntimeError("state store failed")+ )++ with pytest.raises(RuntimeError, match="state store failed"):+ client.get("/codex/login")++ coordinator.cancel.assert_awaited_once_with(TEST_USER_ID, "login-123")+++def test_codex_login_rejects_user_without_required_plan():+ gate = AsyncMock(+ side_effect=fastapi.HTTPException(+ status_code=402,+ detail="A Max plan or higher is required to use ChatGPT.",+ )+ )+ with (+ patch(+ "backend.api.features.integrations.router.enforce_codex_access_http",+ new=gate,+ ),+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ ):+ coordinator.start = AsyncMock()+ manager.store.store_state_token = AsyncMock()++ response = client.get("/codex/login")++ assert response.status_code == 402+ assert response.json()["detail"] == (+ "A Max plan or higher is required to use ChatGPT."+ )+ gate.assert_awaited_once_with(TEST_USER_ID)+ coordinator.start.assert_not_awaited()+ manager.store.store_state_token.assert_not_awaited()+++def test_codex_callback_persists_one_safe_credential():+ credentials = _credentials()+ with (+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ ):+ manager.store.verify_state_token = AsyncMock(return_value=_oauth_state())+ coordinator.complete = AsyncMock(return_value=credentials)++ response = client.post(+ "/codex/callback",+ json={"code": "login-123", "state_token": "state-123"},+ )++ assert response.status_code == 200+ assert response.json() == {+ "id": "codex-credential",+ "provider": "codex",+ "type": "oauth2",+ "title": "ChatGPT for Codex",+ "scopes": [],+ "username": "[email protected]",+ "host": None,+ "is_managed": False,+ }+ raw_response = response.text+ assert "access-secret" not in raw_response+ assert "refresh-secret" not in raw_response+ assert "provider-secret" not in raw_response+++def test_codex_callback_rejects_user_without_required_plan_before_state_use():+ gate = AsyncMock(+ side_effect=fastapi.HTTPException(+ status_code=402,+ detail="A Max plan or higher is required to use ChatGPT.",+ )+ )+ with (+ patch(+ "backend.api.features.integrations.router.enforce_codex_access_http",+ new=gate,+ ),+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ ):+ coordinator.complete = AsyncMock()+ manager.store.verify_state_token = AsyncMock()++ response = client.post(+ "/codex/callback",+ json={"code": "login-123", "state_token": "state-123"},+ )++ assert response.status_code == 402+ gate.assert_awaited_once_with(TEST_USER_ID)+ manager.store.verify_state_token.assert_not_awaited()+ coordinator.complete.assert_not_awaited()+++def test_codex_callback_rejects_login_id_mismatch():+ with (+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ ):+ manager.store.verify_state_token = AsyncMock(return_value=_oauth_state())+ coordinator.complete = AsyncMock()++ response = client.post(+ "/codex/callback",+ json={"code": "other-login", "state_token": "state-123"},+ )++ assert response.status_code == 400+ assert response.json()["detail"] == "Invalid Codex login completion"+ coordinator.complete.assert_not_awaited()+++def test_codex_callback_rejects_non_ascii_login_id_without_server_error():+ with (+ patch(+ "backend.api.features.integrations.router.codex_login_coordinator"+ ) as coordinator,+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ ):+ manager.store.verify_state_token = AsyncMock(return_value=_oauth_state())+ coordinator.complete = AsyncMock()++ response = client.post(+ "/codex/callback",+ json={"code": "lógin-123", "state_token": "state-123"},+ )++ assert response.status_code == 400+ assert response.json()["detail"] == "Invalid Codex login completion"+ coordinator.complete.assert_not_awaited()+++def test_client_cannot_post_codex_credentials():+ with patch("backend.api.features.integrations.router.creds_manager") as manager:+ manager.create = AsyncMock()+ response = client.post(+ "/codex/credentials",+ json={+ "id": "injected",+ "provider": "codex",+ "type": "oauth2",+ "title": "Injected",+ "access_token": "stolen-access",+ "refresh_token": "stolen-refresh",+ "scopes": [],+ "refresh_strategy": "provider_runtime",+ "provider_state": "stolen-provider-state",+ "provider_state_version": 1,+ },+ )++ assert response.status_code == 400+ assert response.json()["detail"] == (+ "Codex credentials must be created through ChatGPT sign-in"+ )+ manager.create.assert_not_awaited()+++def test_device_login_page_reads_fragment_and_contains_no_tokens():+ page = render_device_login_page("test-nonce")++ assert "<script>alert(1)</script>" not in page+ assert "ABCD-<EFGH>" not in page+ assert "URLSearchParams" in page+ assert "access_token" not in page+ assert "refresh_token" not in page+ assert "/status" in page+ assert "/cancel" in page+ assert "pagehide" in page+ assert "/auth/integrations/oauth_callback" in page+ assert "try {\n loginID = decodeURIComponent" in page+ assert "response.status === 404" in page+ assert "pollDeadline" in page+++def test_device_login_status_enforces_user_ownership():+ with (+ patch(+ "backend.api.features.integrations.codex.codex_login_coordinator"+ ) as coordinator,+ ):+ coordinator.get = AsyncMock(return_value=None)+ response = client.get("/codex/device-login/another-users-login/status")++ assert response.status_code == 404+ assert response.json()["detail"] == "Codex login not found"+++def test_device_login_status_returns_only_nonsecret_state():+ state = CodexDeviceLoginState(+ status="completed",+ )+ with (+ patch(+ "backend.api.features.integrations.codex.codex_login_coordinator"+ ) as coordinator,+ ):+ coordinator.get = AsyncMock(return_value=state)+ response = client.get("/codex/device-login/login-123/status")++ assert response.status_code == 200+ assert response.json() == {"status": "completed", "error": None}+ assert "ABCD-EFGH" not in response.text+++def test_device_login_cancel_is_best_effort_when_runtime_cancel_fails():+ with patch(+ "backend.api.features.integrations.codex.codex_login_coordinator"+ ) as coordinator:+ coordinator.cancel = AsyncMock(side_effect=RuntimeError("dead runtime"))+ response = client.post("/codex/device-login/login-123/cancel")++ assert response.status_code == 204+ assert response.content == b""+++def test_device_login_cancel_returns_not_found_for_unknown_login():+ with patch(+ "backend.api.features.integrations.codex.codex_login_coordinator"+ ) as coordinator:+ coordinator.cancel = AsyncMock(return_value=False)+ response = client.post("/codex/device-login/missing/cancel")++ assert response.status_code == 404+++def test_provider_discovery_includes_codex_when_user_has_access():+ with (+ patch("backend.blocks.load_all_blocks"),+ patch(+ "backend.api.features.integrations.router.get_all_provider_names",+ return_value=["codex", "github"],+ ),+ patch(+ "backend.api.features.integrations.router.get_provider_description",+ return_value=None,+ ),+ patch(+ "backend.api.features.integrations.router.get_supported_auth_types",+ return_value=[],+ ),+ ):+ response = client.get("/providers")++ assert response.status_code == 200+ assert [provider["name"] for provider in response.json()] == ["codex", "github"]+++def test_provider_discovery_omits_codex_when_user_lacks_access():+ access = AsyncMock(return_value=False)+ with (+ patch("backend.blocks.load_all_blocks"),+ patch(+ "backend.api.features.integrations.router.has_codex_access_for_discovery",+ new=access,+ ),+ patch(+ "backend.api.features.integrations.router.get_all_provider_names",+ return_value=["codex", "github"],+ ),+ patch(+ "backend.api.features.integrations.router.get_provider_description",+ return_value=None,+ ),+ patch(+ "backend.api.features.integrations.router.get_supported_auth_types",+ return_value=[],+ ),+ ):+ response = client.get("/providers")++ assert response.status_code == 200+ assert [provider["name"] for provider in response.json()] == ["github"]+ access.assert_awaited_once_with(TEST_USER_ID)+++def test_provider_discovery_remains_public_and_omits_codex_anonymously():+ access = AsyncMock(return_value=True)+ app.dependency_overrides[get_optional_user_id] = lambda: None+ with (+ patch("backend.blocks.load_all_blocks"),+ patch(+ "backend.api.features.integrations.router.has_codex_access_for_discovery",+ new=access,+ ),+ patch(+ "backend.api.features.integrations.router.get_all_provider_names",+ return_value=["codex", "github"],+ ),+ patch(+ "backend.api.features.integrations.router.get_provider_description",+ return_value=None,+ ),+ patch(+ "backend.api.features.integrations.router.get_supported_auth_types",+ return_value=[],+ ),+ ):+ response = client.get("/providers")++ assert response.status_code == 200+ assert [provider["name"] for provider in response.json()] == ["github"]+ access.assert_not_awaited()+++def test_device_login_page_has_no_store_and_restrictive_browser_headers():+ state = CodexDeviceLoginState(status="pending")+ with (+ patch(+ "backend.api.features.integrations.codex.codex_login_coordinator"+ ) as coordinator,+ ):+ coordinator.get = AsyncMock(return_value=state)+ response = client.get("/codex/device-login/login-123")++ assert response.status_code == 200+ assert response.headers["cache-control"].startswith("no-store")+ assert response.headers["referrer-policy"] == "no-referrer"+ assert "default-src 'none'" in response.headers["content-security-policy"]+ assert "frame-ancestors 'none'" in response.headers["content-security-policy"]+ assert "nonce=" in response.text+++def test_build_device_login_url_uses_same_origin_proxy():+ url = build_device_login_url(+ "https://platform.example",+ CodexDeviceLogin(+ login_id="login/id",+ verification_url="https://auth.openai.com/codex/device",+ user_code="ABCD EFGH",+ ),+ "state token",+ )++ assert url == (+ "https://platform.example/api/proxy/api/integrations/codex/"+ "device-login/login%2Fid#state=state+token&verification_url="+ "https%3A%2F%2Fauth.openai.com%2Fcodex%2Fdevice&user_code=ABCD+EFGH"+ )+++def test_build_device_login_cancel_url_uses_same_origin_proxy():+ url = build_device_login_cancel_url(+ CodexDeviceLogin(+ login_id="login/id",+ verification_url="https://auth.openai.com/codex/device",+ user_code="ABCD EFGH",+ ),+ )++ assert url == ("/api/proxy/api/integrations/codex/device-login/login%2Fid/cancel")+++def test_codex_account_uses_user_owned_credential_lease():+ lease = MagicMock()+ lease.credentials = _credentials()+ lease.release = AsyncMock()+ account = CodexAccountSnapshot(+ connected=True,+ requires_openai_auth=False,+ account_type="chatgpt",+ email="[email protected]",+ plan_type="plus",+ )+ with (+ patch(+ "backend.api.features.integrations.codex.codex_credentials_manager"+ ) as manager,+ patch(+ "backend.api.features.integrations.codex._get_codex_transport"+ ) as get_transport,+ ):+ manager.acquire_lease = AsyncMock(return_value=lease)+ get_transport.return_value.account = AsyncMock(return_value=account)++ response = client.get("/codex/credentials/codex-credential/account")++ assert response.status_code == 200+ assert response.json() == account.model_dump(mode="json")+ manager.acquire_lease.assert_awaited_once_with(TEST_USER_ID, "codex-credential")+ get_transport.return_value.account.assert_awaited_once_with(lease)+ lease.release.assert_awaited_once()+++def test_codex_rate_limits_returns_safe_snapshot():+ lease = MagicMock()+ lease.credentials = _credentials()+ lease.release = AsyncMock()+ rate_limits = CodexRateLimitsSnapshot(+ plan_type="plus",+ primary=CodexRateLimitWindow(+ used_percent=42,+ window_duration_mins=300,+ resets_at=4_000_000_000,+ ),+ has_credits=True,+ )+ with (+ patch(+ "backend.api.features.integrations.codex.codex_credentials_manager"+ ) as manager,+ patch(+ "backend.api.features.integrations.codex._get_codex_transport"+ ) as get_transport,+ ):+ manager.acquire_lease = AsyncMock(return_value=lease)+ get_transport.return_value.rate_limits = AsyncMock(return_value=rate_limits)++ response = client.get("/codex/credentials/codex-credential/rate-limits")++ assert response.status_code == 200+ assert response.json() == rate_limits.model_dump(mode="json")+ assert "access-secret" not in response.text+ assert "refresh-secret" not in response.text+ lease.release.assert_awaited_once()+++def test_codex_delete_logs_out_then_deletes_even_if_logout_fails():+ credentials = _credentials()+ with (+ patch("backend.api.features.integrations.router.creds_manager") as manager,+ patch(+ "backend.api.features.integrations.router.remove_all_webhooks_for_credentials",+ new=AsyncMock(),+ ),+ patch(+ "backend.api.features.integrations.router.revoke_codex_credentials",+ new=AsyncMock(return_value=False),+ ) as revoke,+ ):+ manager.store.get_creds_by_id = AsyncMock(return_value=credentials)+ manager.delete = AsyncMock()++ response = client.delete("/codex/credentials/codex-credential?force=true")++ assert response.status_code == 200+ assert response.json() == {"deleted": True, "revoked": False}+ revoke.assert_awaited_once_with(manager, TEST_USER_ID, "codex-credential")+ manager.delete.assert_not_awaited()+++async def test_codex_logout_uses_and_releases_exclusive_lease():+ lease = MagicMock()+ lease.credentials = _credentials()+ lease.delete = AsyncMock()+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ with patch(+ "backend.api.features.integrations.codex._get_codex_transport"+ ) as get_transport:+ get_transport.return_value.logout = AsyncMock()++ revoked = await revoke_codex_credentials(+ manager, TEST_USER_ID, "codex-credential"+ )++ assert revoked is True+ manager.acquire_lease.assert_awaited_once_with(TEST_USER_ID, "codex-credential")+ get_transport.return_value.logout.assert_awaited_once_with(lease)+ lease.delete.assert_awaited_once()+ lease.release.assert_awaited_once()+++async def test_codex_logout_failure_is_redacted_and_still_releases(caplog):+ lease = MagicMock()+ lease.credentials = _credentials()+ lease.delete = AsyncMock()+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ with patch(+ "backend.api.features.integrations.codex._get_codex_transport"+ ) as get_transport:+ get_transport.return_value.logout = AsyncMock(+ side_effect=RuntimeError("access-secret refresh-secret")+ )++ revoked = await revoke_codex_credentials(+ manager, TEST_USER_ID, "codex-credential"+ )++ assert revoked is False+ assert "access-secret" not in caplog.text+ assert "refresh-secret" not in caplog.text+ lease.delete.assert_awaited_once()+ lease.release.assert_awaited_once()diff --git a/autogpt_platform/backend/backend/api/features/integrations/router.py b/autogpt_platform/backend/backend/api/features/integrations/router.pyindex dc09a7cd1c..2a1dccd154 100644--- a/autogpt_platform/backend/backend/api/features/integrations/router.py+++ b/autogpt_platform/backend/backend/api/features/integrations/router.py@@ -1,9 +1,11 @@import asyncioimport logging+import secrets+from contextlib import suppressfrom datetime import datetime, timedelta, timezonefrom typing import TYPE_CHECKING, Annotated, Any, List, Literal-from autogpt_libs.auth import get_user_id+from autogpt_libs.auth import get_optional_user_id, get_user_idfrom fastapi import (APIRouter,Body,@@ -34,11 +36,20 @@ from backend.data.model import (CredentialsType,HostScopedCredentials,OAuth2Credentials,+ OAuthState,is_sdk_default,)from backend.data.onboarding import OnboardingStep, complete_onboarding_stepfrom backend.executor.utils import add_graph_executionfrom backend.integrations.ayrshare import AyrshareClient, SocialPlatform+from backend.integrations.codex.access import (+ enforce_codex_access_http,+ has_codex_access_for_discovery,+)+from backend.integrations.codex.login import (+ CodexLoginFailedError,+ CodexLoginPendingError,+)from backend.integrations.credentials_store import (is_system_credential,provider_matches,@@ -67,6 +78,14 @@ from backend.util.exceptions import ()from backend.util.settings import Settings+from .codex import (+ CODEX_LOGIN_STATE_KEY,+ build_device_login_cancel_url,+ build_device_login_url,+ codex_login_coordinator,+ revoke_codex_credentials,+)+from .codex import router as codex_routerfrom .models import (ProviderConstants,ProviderMetadata,@@ -82,6 +101,7 @@ if TYPE_CHECKING:logger = logging.getLogger(__name__)settings = Settings()router = APIRouter()+router.include_router(codex_router, prefix="/codex")creds_manager = IntegrationCredentialsManager()@@ -89,6 +109,7 @@ creds_manager = IntegrationCredentialsManager()class LoginResponse(BaseModel):login_url: strstate_token: str+ cancel_url: str | None = None@router.get("/{provider}/login", summary="Initiate OAuth flow")@@ -106,6 +127,10 @@ async def login(Query(title="ID of existing credential to upgrade scopes for"),] = None,) -> LoginResponse:+ if provider == ProviderName.CODEX:+ await enforce_codex_access_http(user_id)+ return await _start_codex_login(user_id, scopes, credential_id)+handler = _get_provider_oauth_handler(request, provider)requested_scopes = scopes.split(",") if scopes else []@@ -126,6 +151,64 @@ async def login(return LoginResponse(login_url=login_url, state_token=state_token)+async def _start_codex_login(+ user_id: str,+ scopes: str,+ credential_id: str | None,+) -> LoginResponse:+ if scopes:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="Codex sign-in does not accept OAuth scopes",+ )+ if credential_id:+ await _prepare_scope_upgrade(user_id, ProviderName.CODEX, credential_id, [])+ frontend_base_url = settings.config.frontend_base_url+ if not frontend_base_url:+ raise HTTPException(+ status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,+ detail="Frontend base URL is not configured",+ )+ try:+ login_attempt = await codex_login_coordinator.start(user_id)+ except CodexLoginPendingError:+ raise HTTPException(+ status_code=status.HTTP_409_CONFLICT,+ detail="A ChatGPT sign-in is already active",+ ) from None+ except CodexLoginFailedError:+ raise HTTPException(+ status_code=status.HTTP_503_SERVICE_UNAVAILABLE,+ detail="ChatGPT sign-in is temporarily unavailable",+ ) from None+ except Exception as error:+ logger.warning("Could not start Codex device login: %s", type(error).__name__)+ raise HTTPException(+ status_code=status.HTTP_503_SERVICE_UNAVAILABLE,+ detail="ChatGPT sign-in is temporarily unavailable",+ ) from None++ try:+ state_token, _ = await creds_manager.store.store_state_token(+ user_id,+ ProviderName.CODEX.value,+ [],+ expires_in_seconds=settings.config.codex_login_timeout_seconds + 60,+ credential_id=credential_id,+ state_metadata={CODEX_LOGIN_STATE_KEY: login_attempt.login_id},+ )+ except Exception:+ with suppress(Exception):+ await codex_login_coordinator.cancel(user_id, login_attempt.login_id)+ raise++ return LoginResponse(+ login_url=build_device_login_url(frontend_base_url, login_attempt, state_token),+ state_token=state_token,+ cancel_url=build_device_login_cancel_url(login_attempt),+ )++class CredentialsMetaResponse(BaseModel):id: strprovider: str@@ -180,6 +263,35 @@ def to_meta_response(cred: Credentials) -> CredentialsMetaResponse:)+async def _complete_codex_login(+ user_id: str,+ login_id: str,+ oauth_state: OAuthState,+) -> CredentialsMetaResponse:+ expected_login_id = oauth_state.state_metadata.get(CODEX_LOGIN_STATE_KEY)+ if not isinstance(expected_login_id, str) or not secrets.compare_digest(+ login_id.encode(), expected_login_id.encode()+ ):+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="Invalid Codex login completion",+ )+ try:+ credentials = await codex_login_coordinator.complete(user_id, login_id)+ except CodexLoginPendingError:+ raise HTTPException(+ status_code=status.HTTP_409_CONFLICT,+ detail="ChatGPT sign-in is still pending",+ ) from None+ except CodexLoginFailedError:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="ChatGPT sign-in failed. Try again.",+ ) from None++ return to_meta_response(credentials)++@router.post("/{provider}/callback", summary="Exchange OAuth code for tokens")async def callback(provider: Annotated[@@ -191,7 +303,9 @@ async def callback(request: Request,) -> CredentialsMetaResponse:logger.debug(f"Received OAuth callback for provider: {provider}")- handler = _get_provider_oauth_handler(request, provider)++ if provider == ProviderName.CODEX:+ await enforce_codex_access_http(user_id)# Verify the state tokenvalid_state = await creds_manager.store.verify_state_token(@@ -204,6 +318,11 @@ async def callback(status_code=status.HTTP_400_BAD_REQUEST,detail="Invalid or expired state token",)++ if provider == ProviderName.CODEX:+ return await _complete_codex_login(user_id, code, valid_state)++ handler = _get_provider_oauth_handler(request, provider)try:scopes = valid_state.scopeslogger.debug(f"Retrieved scopes from state token: {scopes}")@@ -453,6 +572,19 @@ async def create_credentials(status_code=status.HTTP_403_FORBIDDEN,detail="Cannot create credentials with a reserved ID",)+ if provider == ProviderName.CODEX:+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="Codex credentials must be created through ChatGPT sign-in",+ )+ if (+ isinstance(credentials, OAuth2Credentials)+ and credentials.refresh_strategy == "provider_runtime"+ ):+ raise HTTPException(+ status_code=status.HTTP_400_BAD_REQUEST,+ detail="Provider-runtime credentials cannot be created directly",+ )credentials.provider = providertry:await creds_manager.create(user_id, credentials)@@ -527,10 +659,13 @@ async def delete_credentials(except NeedConfirmation as e:return CredentialsDeletionNeedsConfirmationResponse(message=str(e))- await creds_manager.delete(user_id, cred_id)-tokens_revoked = None- if isinstance(creds, OAuth2Credentials):+ if provider == ProviderName.CODEX:+ tokens_revoked = await revoke_codex_credentials(creds_manager, user_id, cred_id)+ else:+ await creds_manager.delete(user_id, cred_id)++ if isinstance(creds, OAuth2Credentials) and provider != ProviderName.CODEX:if provider_matches(provider.value, ProviderName.MCP.value):# MCP uses dynamic per-server OAuth — create handler from metadatahandler = create_mcp_oauth_handler(creds)@@ -1274,8 +1409,7 @@ async def get_ayrshare_sso_url(]if not ayrshare_creds:logger.error(- "Ayrshare credential provisioning did not produce a credential "- "for user %s",+ "Ayrshare credential provisioning did not produce a credential for user %s",user_id,)raise HTTPException(@@ -1320,7 +1454,9 @@ async def get_ayrshare_sso_url(# === PROVIDER DISCOVERY ENDPOINTS ===@router.get("/providers", response_model=List[ProviderMetadata])-async def list_providers() -> List[ProviderMetadata]:+async def list_providers(+ user_id: Annotated[str | None, Security(get_optional_user_id)],+) -> List[ProviderMetadata]:"""Get metadata for every available provider.@@ -1342,6 +1478,8 @@ async def list_providers() -> List[ProviderMetadata]:logger.warning(f"Failed to load blocks for provider metadata: {e}")all_providers = get_all_provider_names()+ if user_id is None or not await has_codex_access_for_discovery(user_id):+ all_providers = [name for name in all_providers if name != ProviderName.CODEX]return [ProviderMetadata(name=name,diff --git a/autogpt_platform/backend/backend/api/features/search/content_handlers_integration_test.py b/autogpt_platform/backend/backend/api/features/search/content_handlers_integration_test.pyindex b7ac888d23..bfd819e0fb 100644--- a/autogpt_platform/backend/backend/api/features/search/content_handlers_integration_test.py+++ b/autogpt_platform/backend/backend/api/features/search/content_handlers_integration_test.py@@ -24,13 +24,30 @@ from backend.api.features.search.embeddings import ()+async def _first_db_call(coro_factory):+ """Run the test's first DB call, retrying once on a transient+ ``Event loop is closed``.++ Same pattern as conftest's ``_create_user_with_loop_retry``: fire-and-forget+ background tasks elsewhere can leave the Prisma pool bound to a now-closed+ test function loop; the first session-loop DB call after that surfaces as+ ``RuntimeError: Event loop is closed`` and the pool re-establishes on retry.+ """+ try:+ return await coro_factory()+ except RuntimeError as e:+ if "Event loop is closed" not in str(e):+ raise+ return await coro_factory()++@pytest.mark.asyncio(loop_scope="session")async def test_store_agent_handler_real_db():"""Test StoreAgentHandler with real database queries."""handler = StoreAgentHandler()# Get stats from real DB- stats = await handler.get_stats()+ stats = await _first_db_call(handler.get_stats)# Stats should have correct structureassert "total" in statsdiff --git a/autogpt_platform/backend/backend/api/rest_api.py b/autogpt_platform/backend/backend/api/rest_api.pyindex d6c0869faa..cf6d73f60e 100644--- a/autogpt_platform/backend/backend/api/rest_api.py+++ b/autogpt_platform/backend/backend/api/rest_api.py@@ -170,6 +170,13 @@ async def lifespan_context(app: fastapi.FastAPI):with launch_darkly_context():yield+ try:+ from backend.api.features.integrations.codex import codex_login_coordinator++ await codex_login_coordinator.shutdown()+ except Exception:+ logger.warning("Codex login coordinator shutdown failed", exc_info=True)+try:await shutdown_cloud_storage_handler()except Exception as e:diff --git a/autogpt_platform/backend/backend/blocks/_static_provider_configs.py b/autogpt_platform/backend/backend/blocks/_static_provider_configs.pyindex fbb5d9976d..1443caa415 100644--- a/autogpt_platform/backend/backend/blocks/_static_provider_configs.py+++ b/autogpt_platform/backend/backend/blocks/_static_provider_configs.py@@ -23,6 +23,7 @@ _STATIC_PROVIDER_CONFIGS: dict[str, tuple[str, tuple[CredentialsType, ...]]] = {# LLM providers that share blocks/llm.py"aiml_api": ("Unified access to 100+ AI models", ("api_key",)),"anthropic": ("Claude language models", ("api_key",)),+ "codex": ("Use your ChatGPT plan with Codex App Server", ("oauth2",)),"groq": ("Fast LLM inference", ("api_key",)),"llama_api": ("Llama model hosting", ("api_key",)),"ollama": ("Run open-source LLMs locally", ("api_key",)),diff --git a/autogpt_platform/backend/backend/blocks/autopilot.py b/autogpt_platform/backend/backend/blocks/autopilot.pyindex f810b9434f..fa9b5dab2b 100644--- a/autogpt_platform/backend/backend/blocks/autopilot.py+++ b/autogpt_platform/backend/backend/blocks/autopilot.py@@ -5,7 +5,7 @@ import contextvarsimport jsonimport loggingimport uuid-from typing import TYPE_CHECKING, Any+from typing import TYPE_CHECKING, Any, Literalfrom pydantic import field_validatorfrom typing_extensions import TypedDict # Needed for Python <3.12 compatibility@@ -28,7 +28,8 @@ from backend.copilot.permissions import (all_known_tool_names,validate_block_identifiers,)-from backend.data.model import SchemaField+from backend.data.model import CredentialsField, CredentialsMetaInput, SchemaField+from backend.integrations.providers import ProviderNamefrom backend.util.exceptions import BlockExecutionErrorif TYPE_CHECKING:@@ -88,6 +89,12 @@ class TokenUsage(TypedDict):total_tokens: int+CodexAutoPilotCredentials = CredentialsMetaInput[+ Literal[ProviderName.CODEX],+ Literal["oauth2"],+]++class AutoPilotBlock(Block):"""Execute tasks using AutoGPT AutoPilot with full access to platform tools.@@ -126,6 +133,20 @@ class AutoPilotBlock(Block):advanced=True,)+ codex_credentials: CodexAutoPilotCredentials = CredentialsField(+ title="ChatGPT / Codex connection",+ description=(+ "Optional connected ChatGPT plan. Leave empty to use the "+ "normal platform-funded AutoPilot transport."+ ),+ default=None,+ credential_reference_only=True,+ json_schema_extra={+ "secret": True,+ "advanced": False,+ },+ )+session_id: str = SchemaField(description=("Session ID to continue an existing autopilot conversation. "@@ -306,6 +327,8 @@ class AutoPilotBlock(Block):dry_run: bool,organization_id: str | None = None,team_id: str | None = None,+ llm_auth_provider: Literal["platform", "codex"] = "platform",+ llm_credential_id: str | None = None,) -> str:"""Create a new chat session and return its ID (mockable for tests)."""from backend.copilot.model import create_chat_session # avoid circular import@@ -315,6 +338,8 @@ class AutoPilotBlock(Block):dry_run=dry_run,organization_id=organization_id,team_id=team_id,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,)return session.session_id@@ -475,6 +500,7 @@ class AutoPilotBlock(Block):# Create session eagerly so the user always gets the session_id,# even if the downstream stream fails (avoids orphaned sessions).+ codex_connection = input_data.codex_credentialssid = input_data.session_idif not sid:sid = await self.create_session(@@ -482,7 +508,28 @@ class AutoPilotBlock(Block):dry_run=input_data.dry_run or execution_context.dry_run,organization_id=execution_context.organization_id,team_id=execution_context.team_id,+ llm_auth_provider=(+ "codex" if codex_connection is not None else "platform"+ ),+ llm_credential_id=(+ codex_connection.id if codex_connection is not None else None+ ),+ )+ elif codex_connection is not None:+ from backend.copilot.model import get_chat_session++ existing_session = await get_chat_session(+ sid,+ execution_context.user_id,)+ if not (+ existing_session is not None+ and existing_session.metadata.llm_auth_provider == "codex"+ and existing_session.metadata.llm_credential_id == codex_connection.id+ ):+ yield "session_id", sid+ yield "error", "codex_session_route_mismatch"+ return# NOTE: No asyncio.timeout() here — the SDK manages its own# heartbeat-based timeouts internally. Wrapping with asyncio.timeout@@ -697,6 +744,15 @@ async def _enqueue_for_recovery(from backend.copilot.executor.utils import ( # avoid circular importenqueue_copilot_turn,)+ from backend.copilot.model import get_chat_session++ session = await get_chat_session(session_id, user_id)+ if session is None:+ logger.warning(+ "AutoPilot session %s: copilot_session_not_found",+ session_id[:12],+ )+ returnawait asyncio.wait_for(enqueue_copilot_turn(@@ -704,6 +760,8 @@ async def _enqueue_for_recovery(user_id=user_id,message=message,turn_id=str(uuid.uuid4()),+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,),timeout=10,)diff --git a/autogpt_platform/backend/backend/blocks/autopilot_codex_test.py b/autogpt_platform/backend/backend/blocks/autopilot_codex_test.pynew file mode 100644index 0000000000..7e1745532c--- /dev/null+++ b/autogpt_platform/backend/backend/blocks/autopilot_codex_test.py@@ -0,0 +1,152 @@+from types import SimpleNamespace+from unittest.mock import AsyncMock, patch++import pytest++from backend.blocks.autopilot import AutoPilotBlock+from backend.data.execution import ExecutionContext+from backend.data.graph import NodeModel+from backend.integrations.providers import ProviderName+++def test_codex_connection_is_a_top_level_reference_only_credential_field():+ schema = AutoPilotBlock.Input.jsonschema()+ field_schema = schema["properties"]["codex_credentials"]+ field_info = AutoPilotBlock.Input.get_credentials_fields_info()["codex_credentials"]++ assert "codex_credentials" in AutoPilotBlock.Input.get_credentials_fields()+ assert field_schema["credentials_provider"] == [ProviderName.CODEX]+ assert field_schema["credentials_types"] == ["oauth2"]+ assert field_schema["advanced"] is False+ assert field_schema["credential_reference_only"] is True+ assert field_info.provider == frozenset({ProviderName.CODEX})+ assert field_info.supported_types == frozenset({"oauth2"})+ assert field_info.credential_reference_only is True+++def test_graph_export_strips_autopilot_codex_credential_reference():+ block = AutoPilotBlock()+ node = NodeModel(+ id="node-1",+ block_id=block.id,+ input_default={+ "prompt": "do the thing",+ "codex_credentials": {+ "id": "cred-1",+ "provider": "codex",+ "type": "oauth2",+ "title": "Personal ChatGPT",+ },+ },+ graph_id="graph-1",+ graph_version=1,+ )++ stripped = node.stripped_for_export()++ assert stripped.input_default == {"prompt": "do the thing"}+++async def test_autopilot_block_routes_new_session_to_selected_codex_connection():+ block = AutoPilotBlock()+ create_session = AsyncMock(return_value="session-1")+ execute_copilot = AsyncMock(+ return_value=(+ "done",+ [],+ "[]",+ "session-1",+ {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},+ )+ )+ input_data = AutoPilotBlock.Input(+ prompt="do the thing",+ codex_credentials={+ "id": "cred-1",+ "provider": "codex",+ "type": "oauth2",+ "title": "Personal ChatGPT",+ },+ )+ context = ExecutionContext(+ user_id="user-1",+ graph_id="graph-1",+ graph_exec_id="graph-exec-1",+ node_id="node-1",+ node_exec_id="node-exec-1",+ )++ with (+ patch.object(block, "create_session", create_session),+ patch.object(block, "execute_copilot", execute_copilot),+ ):+ outputs = {+ name: value+ async for name, value in block.run(+ input_data,+ execution_context=context,+ )+ }++ assert outputs["response"] == "done"+ assert outputs["session_id"] == "session-1"+ create_session.assert_awaited_once_with(+ "user-1",+ dry_run=False,+ organization_id=None,+ team_id=None,+ llm_auth_provider="codex",+ llm_credential_id="cred-1",+ )+ execute_copilot.assert_awaited_once()+++async def test_autopilot_block_rejects_codex_connection_change_on_resume():+ block = AutoPilotBlock()+ execute_copilot = AsyncMock()+ input_data = AutoPilotBlock.Input(+ prompt="continue",+ session_id="session-1",+ codex_credentials={+ "id": "cred-1",+ "provider": "codex",+ "type": "oauth2",+ "title": "Personal ChatGPT",+ },+ )+ context = ExecutionContext(+ user_id="user-1",+ graph_id="graph-1",+ graph_exec_id="graph-exec-1",+ node_id="node-1",+ node_exec_id="node-exec-1",+ )+ existing = SimpleNamespace(+ metadata=SimpleNamespace(+ llm_auth_provider="codex",+ llm_credential_id="different-credential",+ )+ )++ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=existing),+ ),+ patch.object(block, "execute_copilot", execute_copilot),+ ):+ outputs = [+ item+ async for item in block.run(+ input_data,+ execution_context=context,+ )+ ]++ assert outputs == [+ ("session_id", "session-1"),+ ("error", "codex_session_route_mismatch"),+ ]+ execute_copilot.assert_not_awaited()diff --git a/autogpt_platform/backend/backend/blocks/codex.py b/autogpt_platform/backend/backend/blocks/codex.pyindex 456cd13499..cd7f064362 100644--- a/autogpt_platform/backend/backend/blocks/codex.py+++ b/autogpt_platform/backend/backend/blocks/codex.py@@ -1,6 +1,6 @@from dataclasses import dataclassfrom enum import Enum-from typing import Any, Literal+from typing import Any, ClassVar, Literalfrom openai import AsyncOpenAIfrom openai.types.responses import Response as OpenAIResponse@@ -18,8 +18,18 @@ from backend.data.model import (CredentialsField,CredentialsMetaInput,NodeExecutionStats,+ OAuth2Credentials,SchemaField,)+from backend.integrations.codex.models import (+ CodexInvocationRequest,+ CodexInvocationResult,+)+from backend.integrations.codex.models import (+ CodexReasoningEffort as CodexRuntimeReasoningEffort,+)+from backend.integrations.codex.transport import get_codex_transport+from backend.integrations.credential_lease import CredentialLeasefrom backend.integrations.providers import ProviderName@@ -35,21 +45,53 @@ class CodexCallResult:class CodexModel(str, Enum):"""Codex-capable OpenAI models."""+ GPT5_6_SOL = "gpt-5.6-sol"+ GPT5_6_TERRA = "gpt-5.6-terra"+ GPT5_6_LUNA = "gpt-5.6-luna"GPT5_3_CODEX = "gpt-5.3-codex"GPT5_1_CODEX = "gpt-5.1-codex"class CodexReasoningEffort(str, Enum):- """Configuration for the Responses API reasoning effort."""+ """Reasoning effort supported by either Codex execution transport."""NONE = "none"LOW = "low"MEDIUM = "medium"HIGH = "high"+ XHIGH = "xhigh"+ MAX = "max"+ ULTRA = "ultra"+++class CodexExecutionTransport(str, Enum):+ OPENAI_API = "openai_api"+ CODEX_APP_SERVER = "codex_app_server"+++_APP_SERVER_EFFORT: dict[+ CodexReasoningEffort,+ CodexRuntimeReasoningEffort | None,+] = {+ CodexReasoningEffort.NONE: None,+ CodexReasoningEffort.LOW: "low",+ CodexReasoningEffort.MEDIUM: "medium",+ CodexReasoningEffort.HIGH: "high",+ CodexReasoningEffort.XHIGH: "xhigh",+ CodexReasoningEffort.MAX: "max",+ CodexReasoningEffort.ULTRA: "ultra",+}+++def _app_server_effort(+ effort: CodexReasoningEffort,+) -> CodexRuntimeReasoningEffort | None:+ return _APP_SERVER_EFFORT[effort]CodexCredentials = CredentialsMetaInput[- Literal[ProviderName.OPENAI], Literal["api_key"]+ Literal[ProviderName.OPENAI, ProviderName.CODEX],+ Literal["api_key", "oauth2"],]TEST_CREDENTIALS = APIKeyCredentials(@@ -69,12 +111,21 @@ TEST_CREDENTIALS_INPUT = {def CodexCredentialsField() -> CodexCredentials:return CredentialsField(- description="OpenAI API key with access to Codex models (Responses API).",+ description="OpenAI API key or connected ChatGPT plan for Codex.",+ discriminator="transport",+ discriminator_mapping={+ CodexExecutionTransport.OPENAI_API.value: ProviderName.OPENAI,+ CodexExecutionTransport.CODEX_APP_SERVER.value: ProviderName.CODEX,+ },+ discriminator_type_mapping={+ CodexExecutionTransport.OPENAI_API.value: ["api_key"],+ CodexExecutionTransport.CODEX_APP_SERVER.value: ["oauth2"],+ },)class CodeGenerationBlock(Block):- """Block that talks to Codex models via the OpenAI Responses API."""+ """Block that talks to Codex through an API key or ChatGPT connection."""class Input(BlockSchemaInput):prompt: str = SchemaField(@@ -87,25 +138,45 @@ class CodeGenerationBlock(Block):"You are Codex, an elite software engineer. ""Favor concise, working code and highlight important caveats."),- description="Optional instructions injected via the Responses API instructions field.",+ description=(+ "Optional instructions passed to the selected Codex transport."+ ),advanced=True,)+ transport: CodexExecutionTransport = SchemaField(+ title="Transport",+ default=CodexExecutionTransport.OPENAI_API,+ description=(+ "Use an OpenAI API key or your connected ChatGPT plan through "+ "Codex App Server."+ ),+ advanced=False,+ )model: CodexModel = SchemaField(title="Codex Model",default=CodexModel.GPT5_3_CODEX,- description="Codex-optimized model served via the Responses API.",+ description=(+ "OpenAI API transport only. Codex App Server selects the current "+ "subscription model from its live model catalog."+ ),advanced=False,)reasoning_effort: CodexReasoningEffort = SchemaField(title="Reasoning Effort",default=CodexReasoningEffort.MEDIUM,- description="Controls the Responses API reasoning budget. Select 'none' to skip reasoning configs.",+ description=(+ "Controls the selected transport's reasoning effort. OpenAI API "+ "does not support 'ultra'; select 'none' to omit reasoning config."+ ),advanced=True,)max_output_tokens: int | None = SchemaField(title="Max Output Tokens",default=2048,- description="Upper bound for generated tokens (hard limit 128,000). Leave blank to let OpenAI decide.",+ description=(+ "OpenAI API transport only: upper bound for generated tokens "+ "(hard limit 128,000). Codex App Server uses its model and plan limits."+ ),advanced=True,)credentials: CodexCredentials = CodexCredentialsField()@@ -119,14 +190,17 @@ class CodeGenerationBlock(Block):default="",)response_id: str = SchemaField(- description="ID of the Responses API call for auditing/debugging.",+ description="Transport response ID for auditing and debugging.",default="",)def __init__(self):super().__init__(id="86a2a099-30df-47b4-b7e4-34ae5f83e0d5",- description="Generate or refactor code using OpenAI's Codex (Responses API).",+ description=(+ "Generate or refactor code using an OpenAI API key or a connected "+ "ChatGPT plan through Codex App Server."+ ),categories={BlockCategory.AI, BlockCategory.DEVELOPER_TOOLS},input_schema=CodeGenerationBlock.Input,output_schema=CodeGenerationBlock.Output,@@ -152,16 +226,23 @@ class CodeGenerationBlock(Block):)self.execution_stats = NodeExecutionStats()- # GPT-5.1-Codex published pricing: $1.25 / 1M input, $10 / 1M output.- _INPUT_USD_PER_1M = 1.25- _OUTPUT_USD_PER_1M = 10.0+ _MODEL_USD_PER_1M: ClassVar[dict[CodexModel, tuple[float, float]]] = {+ CodexModel.GPT5_6_SOL: (5.0, 30.0),+ CodexModel.GPT5_6_TERRA: (2.5, 15.0),+ CodexModel.GPT5_6_LUNA: (1.0, 6.0),+ CodexModel.GPT5_3_CODEX: (1.75, 14.0),+ CodexModel.GPT5_1_CODEX: (1.25, 10.0),+ }- @staticmethod- def _compute_token_usd(input_tokens: int, output_tokens: int) -> float:- return (- input_tokens * CodeGenerationBlock._INPUT_USD_PER_1M- + output_tokens * CodeGenerationBlock._OUTPUT_USD_PER_1M- ) / 1_000_000+ @classmethod+ def _compute_token_usd(+ cls,+ model: CodexModel,+ input_tokens: int,+ output_tokens: int,+ ) -> float:+ input_rate, output_rate = cls._MODEL_USD_PER_1M[model]+ return (input_tokens * input_rate + output_tokens * output_rate) / 1_000_000async def call_codex(self,@@ -207,7 +288,7 @@ class CodeGenerationBlock(Block):self.execution_stats.output_token_count = output_tokensself.execution_stats.llm_call_count += 1self.execution_stats.provider_cost = self._compute_token_usd(- input_tokens, output_tokens+ model, input_tokens, output_tokens)self.execution_stats.provider_cost_type = "cost_usd"@@ -221,10 +302,35 @@ class CodeGenerationBlock(Block):self,input_data: Input,*,- credentials: APIKeyCredentials,+ credentials: APIKeyCredentials | OAuth2Credentials,+ credential_leases: dict[str, CredentialLease] | None = None,**_kwargs,) -> BlockOutput:- result = await self.call_codex(+ if input_data.transport == CodexExecutionTransport.OPENAI_API:+ result = await self._run_openai_api(input_data, credentials)+ else:+ result = await self._run_codex_app_server(+ input_data,+ credentials,+ credential_leases or {},+ )++ yield "response", result.response+ yield "reasoning", result.reasoning+ yield "response_id", result.response_id++ async def _run_openai_api(+ self,+ input_data: Input,+ credentials: APIKeyCredentials | OAuth2Credentials,+ ) -> CodexCallResult:+ if credentials.type != "api_key" or credentials.provider != "openai":+ raise ValueError("OpenAI API transport requires an OpenAI API key")+ if input_data.reasoning_effort == CodexReasoningEffort.ULTRA:+ raise ValueError(+ "OpenAI API transport does not support 'ultra' reasoning effort"+ )+ return await self.call_codex(credentials=credentials,model=input_data.model,prompt=input_data.prompt,@@ -233,6 +339,50 @@ class CodeGenerationBlock(Block):reasoning_effort=input_data.reasoning_effort,)- yield "response", result.response- yield "reasoning", result.reasoning- yield "response_id", result.response_id+ async def _run_codex_app_server(+ self,+ input_data: Input,+ credentials: APIKeyCredentials | OAuth2Credentials,+ credential_leases: dict[str, CredentialLease],+ ) -> CodexCallResult:+ if credentials.type != "oauth2" or credentials.provider != "codex":+ raise ValueError(+ "Codex App Server transport requires connected ChatGPT credentials"+ )+ lease = credential_leases.get("credentials")+ if lease is None or lease.credentials.id != credentials.id:+ raise ValueError("Codex App Server transport requires a credential lease")+ response = await get_codex_transport().invoke(+ lease=lease,+ request=CodexInvocationRequest(+ prompt=input_data.prompt,+ instructions=input_data.system_prompt or None,+ model=None,+ effort=_app_server_effort(input_data.reasoning_effort),+ ),+ )+ self._record_subscription_usage(response)+ return CodexCallResult(+ response=response.final_response,+ reasoning=response.reasoning_summary or "",+ response_id=response.response_id,+ )++ def _record_subscription_usage(+ self,+ response: CodexInvocationResult,+ ) -> None:+ self.execution_stats.llm_call_count += 1+ self.execution_stats.billing_mode = "user_subscription"+ self.execution_stats.auth_provider = "codex"+ self.execution_stats.execution_path = "codex_app_server"+ self.execution_stats.resolved_model = response.resolved_model+ usage = response.usage+ if usage is None:+ return++ self.execution_stats.input_token_count = usage.input_tokens+ self.execution_stats.cache_read_token_count = usage.cached_input_tokens+ self.execution_stats.output_token_count = usage.output_tokens+ self.execution_stats.provider_cost = usage.total_tokens+ self.execution_stats.provider_cost_type = "tokens"diff --git a/autogpt_platform/backend/backend/blocks/codex_transport_test.py b/autogpt_platform/backend/backend/blocks/codex_transport_test.pynew file mode 100644index 0000000000..44f850e43b--- /dev/null+++ b/autogpt_platform/backend/backend/blocks/codex_transport_test.py@@ -0,0 +1,273 @@+from unittest.mock import AsyncMock, MagicMock, patch++import pytest+from pydantic import SecretStr++from backend.blocks.codex import (+ TEST_CREDENTIALS_INPUT,+ CodeGenerationBlock,+ CodexCallResult,+ CodexExecutionTransport,+ CodexModel,+ CodexReasoningEffort,+ _app_server_effort,+)+from backend.data.model import APIKeyCredentials, OAuth2Credentials+from backend.integrations.codex.models import CodexInvocationResult, CodexTokenUsage+++def test_existing_graphs_default_to_openai_api_transport():+ input_data = CodeGenerationBlock.Input(+ prompt="preserve the old graph",+ credentials=TEST_CREDENTIALS_INPUT,+ )++ assert input_data.transport == CodexExecutionTransport.OPENAI_API+ field = CodeGenerationBlock.Input.model_fields["credentials"]+ assert field.json_schema_extra == {+ "discriminator": "transport",+ "discriminator_mapping": {+ "openai_api": "openai",+ "codex_app_server": "codex",+ },+ "discriminator_type_mapping": {+ "openai_api": ["api_key"],+ "codex_app_server": ["oauth2"],+ },+ }+ field_info = CodeGenerationBlock.Input.get_credentials_fields_info()["credentials"]+ assert field_info.discriminate("openai_api").supported_types == {"api_key"}+ assert field_info.discriminate("codex_app_server").supported_types == {"oauth2"}+++def test_transport_selector_is_visible_before_transport_specific_fields():+ properties = CodeGenerationBlock.Input.jsonschema()["properties"]++ assert properties["transport"]["advanced"] is False+ assert list(properties).index("transport") < list(properties).index("model")+ assert list(properties).index("transport") < list(properties).index("credentials")+++def test_subscription_effort_mapping_includes_none_and_high():+ assert _app_server_effort(CodexReasoningEffort.NONE) is None+ assert _app_server_effort(CodexReasoningEffort.HIGH) == "high"+++def test_openai_api_costs_are_model_specific():+ assert (+ CodeGenerationBlock._compute_token_usd(+ CodexModel.GPT5_1_CODEX, 1_000_000, 1_000_000+ )+ == 11.25+ )+ assert (+ CodeGenerationBlock._compute_token_usd(+ CodexModel.GPT5_3_CODEX, 1_000_000, 1_000_000+ )+ == 15.75+ )+ assert (+ CodeGenerationBlock._compute_token_usd(+ CodexModel.GPT5_6_TERRA, 1_000_000, 1_000_000+ )+ == 17.5+ )+++async def test_openai_api_transport_preserves_existing_call_path():+ block = CodeGenerationBlock()+ block.call_codex = AsyncMock(+ return_value=CodexCallResult(+ response="implemented",+ reasoning="used Responses API",+ response_id="response-1",+ )+ )+ credentials = APIKeyCredentials(+ id="openai-cred-1",+ provider="openai",+ api_key=SecretStr("secret"),+ )+ input_data = CodeGenerationBlock.Input(+ prompt="preserve it",+ credentials={+ "id": credentials.id,+ "provider": credentials.provider,+ "type": credentials.type,+ "title": credentials.title,+ },+ )++ outputs = [item async for item in block.run(input_data, credentials=credentials)]++ assert outputs == [+ ("response", "implemented"),+ ("reasoning", "used Responses API"),+ ("response_id", "response-1"),+ ]+ block.call_codex.assert_awaited_once_with(+ credentials=credentials,+ model=input_data.model,+ prompt="preserve it",+ system_prompt=input_data.system_prompt,+ max_output_tokens=input_data.max_output_tokens,+ reasoning_effort=input_data.reasoning_effort,+ )+++async def test_openai_api_accepts_max_but_rejects_app_server_only_ultra():+ block = CodeGenerationBlock()+ block.call_codex = AsyncMock(+ return_value=CodexCallResult(+ response="implemented",+ reasoning="",+ response_id="response-1",+ )+ )+ credentials = APIKeyCredentials(+ id="openai-cred-1",+ provider="openai",+ api_key=SecretStr("secret"),+ )++ max_input = CodeGenerationBlock.Input(+ prompt="preserve max",+ reasoning_effort=CodexReasoningEffort.MAX,+ credentials=_credential_metadata(credentials),+ )+ await block._run_openai_api(max_input, credentials)+ block.call_codex.assert_awaited_once()++ ultra_input = CodeGenerationBlock.Input(+ prompt="reject ultra",+ reasoning_effort=CodexReasoningEffort.ULTRA,+ credentials=_credential_metadata(credentials),+ )+ with pytest.raises(ValueError, match="does not support 'ultra'"):+ await block._run_openai_api(ultra_input, credentials)+ block.call_codex.assert_awaited_once()+++async def test_codex_app_server_uses_existing_lease_and_records_non_usd_usage():+ block = CodeGenerationBlock()+ credentials = _codex_credentials()+ lease = MagicMock(credentials=credentials)+ transport = MagicMock()+ transport.invoke = AsyncMock(+ return_value=CodexInvocationResult(+ response_id="turn-1",+ final_response="implemented",+ reasoning_summary="used the transport",+ status="completed",+ resolved_model="gpt-5.6-sol",+ usage=CodexTokenUsage(+ input_tokens=100,+ cached_input_tokens=25,+ output_tokens=40,+ reasoning_output_tokens=10,+ total_tokens=140,+ ),+ )+ )+ input_data = CodeGenerationBlock.Input(+ prompt="implement it",+ system_prompt="be concise",+ model=CodexModel.GPT5_3_CODEX,+ reasoning_effort=CodexReasoningEffort.HIGH,+ transport=CodexExecutionTransport.CODEX_APP_SERVER,+ credentials=_credential_metadata(credentials),+ )++ with patch("backend.blocks.codex.get_codex_transport", return_value=transport):+ outputs = [+ item+ async for item in block.run(+ input_data,+ credentials=credentials,+ credential_leases={"credentials": lease},+ user_id="user-1",+ )+ ]++ assert outputs == [+ ("response", "implemented"),+ ("reasoning", "used the transport"),+ ("response_id", "turn-1"),+ ]+ invoke = transport.invoke.await_args.kwargs+ assert invoke["lease"] is lease+ assert invoke["request"].prompt == "implement it"+ assert invoke["request"].instructions == "be concise"+ assert invoke["request"].model is None+ assert invoke["request"].effort == "high"+ assert block.execution_stats.input_token_count == 100+ assert block.execution_stats.cache_read_token_count == 25+ assert block.execution_stats.output_token_count == 40+ assert block.execution_stats.provider_cost == 140+ assert block.execution_stats.provider_cost_type == "tokens"+ assert block.execution_stats.billing_mode == "user_subscription"+ assert block.execution_stats.execution_path == "codex_app_server"+ assert block.execution_stats.resolved_model == "gpt-5.6-sol"+++async def test_codex_app_server_rejects_execution_without_credential_lease():+ block = CodeGenerationBlock()+ credentials = _codex_credentials()+ input_data = CodeGenerationBlock.Input(+ prompt="implement it",+ transport=CodexExecutionTransport.CODEX_APP_SERVER,+ credentials=_credential_metadata(credentials),+ )++ with pytest.raises(ValueError, match="credential lease"):+ await anext(block.run(input_data, credentials=credentials, user_id="user-1"))+++async def test_codex_app_server_rejects_mismatched_credential_lease():+ block = CodeGenerationBlock()+ credentials = _codex_credentials()+ other_credentials = credentials.model_copy(update={"id": "codex-cred-2"})+ input_data = CodeGenerationBlock.Input(+ prompt="implement it",+ transport=CodexExecutionTransport.CODEX_APP_SERVER,+ credentials=_credential_metadata(credentials),+ )++ with pytest.raises(ValueError, match="credential lease"):+ await anext(+ block.run(+ input_data,+ credentials=credentials,+ credential_leases={+ "credentials": MagicMock(credentials=other_credentials)+ },+ user_id="user-1",+ )+ )+++def _codex_credentials() -> OAuth2Credentials:+ return OAuth2Credentials(+ id="codex-cred-1",+ provider="codex",+ access_token=SecretStr("access"),+ refresh_token=SecretStr("refresh"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr("{}"),+ provider_state_version=1,+ )+++def _credential_metadata(credentials: OAuth2Credentials) -> dict[str, str | None]:+ return {+ "id": credentials.id,+ "provider": credentials.provider,+ "type": credentials.type,+ "title": credentials.title,+ }diff --git a/autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py b/autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.pyindex 630747007b..e7fc4875de 100644--- a/autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py+++ b/autogpt_platform/backend/backend/blocks/cost_leak_fixes_test.py@@ -13,6 +13,7 @@ from pydantic import SecretStrfrom backend.blocks._base import BlockCostTypefrom backend.blocks.ai_condition import AIConditionBlock+from backend.blocks.codex import CodeGenerationBlock, CodexModelfrom backend.data.block_cost_config import BLOCK_COSTS, LLM_COSTfrom backend.data.model import APIKeyCredentials, NodeExecutionStats@@ -147,11 +148,8 @@ def test_perplexity_record_openrouter_cost_tags_only_on_concrete_value(def test_codex_registered_as_cost_usd_150():- from backend.blocks.codex import CodeGenerationBlock-entries = BLOCK_COSTS[CodeGenerationBlock]- # Both Codex models (GPT5_3_CODEX, GPT5_1_CODEX) are COST_USD 150.- assert len(entries) == 2+ assert len(entries) == len(CodexModel)for entry in entries:assert entry.cost_type == BlockCostType.COST_USDassert entry.cost_amount == 150@@ -166,7 +164,6 @@ def test_all_codex_models_have_cost():def test_validate_codex_costs_catches_missing_model():"""A Codex model without a BlockCost is rejected, not billed as free."""- from backend.blocks.codex import CodeGenerationBlockfrom backend.data.block_cost_config import _validate_codex_costswith patch.dict(BLOCK_COSTS, {CodeGenerationBlock: []}):@@ -191,10 +188,8 @@ def test_codex_computes_provider_cost_usd_from_token_counts(to the wrong rate constants (e.g. swapping the $1.25 input rate forGPT-4o's $2.50) would fail this test."""- from backend.blocks.codex import CodeGenerationBlock-assert CodeGenerationBlock._compute_token_usd(- input_tokens, output_tokens+ CodexModel.GPT5_1_CODEX, input_tokens, output_tokens) == pytest.approx(expected_usd)diff --git a/autogpt_platform/backend/backend/blocks/test/test_autopilot.py b/autogpt_platform/backend/backend/blocks/test/test_autopilot.pyindex 3936e2882a..a1594d586b 100644--- a/autogpt_platform/backend/backend/blocks/test/test_autopilot.py+++ b/autogpt_platform/backend/backend/blocks/test/test_autopilot.py@@ -198,7 +198,12 @@ class TestRunValidation:passblock.create_session.assert_called_once_with(- ctx.user_id, dry_run=True, organization_id=None, team_id=None+ ctx.user_id,+ dry_run=True,+ organization_id=None,+ team_id=None,+ llm_auth_provider="platform",+ llm_credential_id=None,)@pytest.mark.asynciodiff --git a/autogpt_platform/backend/backend/copilot/codex/__init__.py b/autogpt_platform/backend/backend/copilot/codex/__init__.pynew file mode 100644index 0000000000..2090ce0649--- /dev/null+++ b/autogpt_platform/backend/backend/copilot/codex/__init__.py@@ -0,0 +1 @@+"""Codex subscription support for AutoPilot."""diff --git a/autogpt_platform/backend/backend/copilot/codex_model_router_test.py b/autogpt_platform/backend/backend/copilot/codex_model_router_test.pynew file mode 100644index 0000000000..a9d9792727--- /dev/null+++ b/autogpt_platform/backend/backend/copilot/codex_model_router_test.py@@ -0,0 +1,188 @@+from typing import cast+from unittest.mock import AsyncMock++import pytest++from backend.copilot.model_router import resolve_codex_model_route+from backend.data.llm_registry import registry+from backend.integrations.codex.models import CodexModelInfo, CodexReasoningEffort+from backend.integrations.credential_lease import CredentialLease++[email protected](scope="session", name="server")+def _server_noop():+ return None++[email protected](scope="session", autouse=True, name="graph_cleanup")+def _graph_cleanup_noop():+ yield+++def _model(+ model: str,+ *,+ default: bool = False,+ hidden: bool = False,+ default_effort: CodexReasoningEffort = "medium",+ efforts: list[CodexReasoningEffort] | None = None,+) -> CodexModelInfo:+ return CodexModelInfo(+ model=model,+ display_name=model,+ is_default=default,+ hidden=hidden,+ default_reasoning_effort=default_effort,+ supported_reasoning_efforts=efforts or ["low", "medium", "high", "xhigh"],+ input_modalities=["text"],+ )+++def _lease() -> CredentialLease:+ return cast(CredentialLease, object())+++def catalog_state():+ old = (+ registry._dynamic_models,+ registry._date_stripped_models,+ registry._routes,+ registry._loaded,+ )+ registry.load_catalog()+ yield+ (+ registry._dynamic_models,+ registry._date_stripped_models,+ registry._routes,+ registry._loaded,+ ) = old+++def _transport(monkeypatch, models: list[CodexModelInfo]):+ transport = AsyncMock()+ transport.models.return_value = models+ monkeypatch.setattr(+ "backend.copilot.model_router.get_codex_transport",+ lambda: transport,+ )+ return transport+++ "mode,tier,expected_model,expected_effort",+ [+ ("fast", "standard", "gpt-5.6-luna", "low"),+ ("fast", "advanced", "gpt-5.6-terra", "medium"),+ ("thinking", "standard", "gpt-5.6-terra", "high"),+ ("thinking", "advanced", "gpt-5.6-sol", "xhigh"),+ ],+)+async def test_catalog_cells_select_latest_advertised_model(+ monkeypatch,+ catalog_state,+ mode,+ tier,+ expected_model,+ expected_effort,+):+ models = [+ _model("gpt-5.6-luna"),+ _model("gpt-5.6-terra"),+ _model("gpt-5.6-sol", default=True),+ ]+ transport = _transport(monkeypatch, models)+ lease = _lease()++ resolved = await resolve_codex_model_route(+ mode,+ tier,+ lease,+ )++ assert resolved == (expected_model, expected_effort, "catalog")+ transport.models.assert_awaited_once_with(lease)+++async def test_unavailable_catalog_model_uses_visible_account_default(+ monkeypatch,+ catalog_state,+):+ _transport(+ monkeypatch,+ [+ _model("codex-auto-review", hidden=True),+ _model(+ "gpt-5.4",+ default=True,+ default_effort="low",+ efforts=["low"],+ ),+ ],+ )++ resolved = await resolve_codex_model_route(+ "thinking",+ "advanced",+ _lease(),+ )++ assert resolved == ("gpt-5.4", "low", "account_default")+++async def test_no_default_uses_first_visible_account_model(monkeypatch, catalog_state):+ _transport(+ monkeypatch,+ [+ _model("codex-auto-review", hidden=True),+ _model("gpt-5.2", efforts=["medium"]),+ ],+ )++ resolved = await resolve_codex_model_route(+ "fast",+ "standard",+ _lease(),+ )++ assert resolved == ("gpt-5.2", "medium", "account_available")+++async def test_empty_visible_catalog_fails_closed(monkeypatch, catalog_state):+ _transport(monkeypatch, [_model("codex-auto-review", hidden=True, default=True)])++ with pytest.raises(RuntimeError, match="codex_model_unavailable"):+ await resolve_codex_model_route(+ "fast",+ "standard",+ _lease(),+ )+++async def test_disabled_account_default_is_not_used(monkeypatch, catalog_state):+ disabled = registry._dynamic_models["gpt-5.4-2026-03-05"].model_copy(+ update={"is_enabled": False}+ )+ registry._dynamic_models["gpt-5.4-2026-03-05"] = disabled+ registry._date_stripped_models["gpt-5.4"] = disabled+ _transport(+ monkeypatch,+ [+ _model("gpt-5.4", default=True),+ _model("gpt-5.2", efforts=["medium"]),+ ],+ )++ resolved = await resolve_codex_model_route(+ "fast",+ "standard",+ _lease(),+ )++ assert resolved == ("gpt-5.2", "medium", "account_available")diff --git a/autogpt_platform/backend/backend/copilot/config.py b/autogpt_platform/backend/backend/copilot/config.pyindex d7e56392c5..ac9292e6de 100644--- a/autogpt_platform/backend/backend/copilot/config.py+++ b/autogpt_platform/backend/backend/copilot/config.py@@ -56,6 +56,7 @@ _DEFAULT_SIMULATION_MODEL = "google/gemini-2.5-flash-lite"_DEFAULT_FAST_ADVANCED_MODEL = "anthropic/claude-opus-4-8"TransportName = Literal["subscription", "openrouter", "direct_anthropic", "local"]+CopilotLlmAuthProvider = Literal["platform", "codex"]class TransportProfile(BaseModel):diff --git a/autogpt_platform/backend/backend/copilot/executor/manager.py b/autogpt_platform/backend/backend/copilot/executor/manager.pyindex e0c9e2c947..4e3460f9b9 100644--- a/autogpt_platform/backend/backend/copilot/executor/manager.py+++ b/autogpt_platform/backend/backend/copilot/executor/manager.py@@ -23,6 +23,7 @@ from backend.copilot.executor.utils import schedule_turnfrom backend.data import redis_client as redisfrom backend.data.rabbitmq import SyncRabbitMQfrom backend.executor.cluster_lock import ClusterLock+from backend.integrations.codex.transport import get_codex_transportfrom backend.util.decorator import error_loggedfrom backend.util.logging import TruncatedLoggerfrom backend.util.process import AppProcess@@ -103,6 +104,7 @@ class CoPilotExecutor(AppProcess):self._task_locks: dict[str, ClusterLock] = {}self._active_tasks_lock_obj: threading.Lock | None = None+ self._codex_runtime_pool_closed = False# ============ Main Entry Points (AppProcess interface) ============ #@@ -148,11 +150,14 @@ class CoPilotExecutor(AppProcess):own ``finally`` publishes its terminal state via``mark_session_completed``. When a turn exits, ``on_run_done``removes it from ``active_tasks`` and releases its cluster lock.- 3. Shut down the thread-pool executor (cancels pending, leaves+ 3. Stop message consumer threads and disconnect their clients.+ 4. Close the process-local Codex runtime pool after turns and+ consumers have stopped, checkpointing credentials before worker+ teardown.+ 5. Shut down the thread-pool executor (cancels pending, leavesrunning threads alone — process exit handles them).- 4. Release any cluster locks still held (defensive — on_run_done's+ 6. Release any cluster locks still held (defensive — on_run_done'sfinally should have already released them).- 5. Stop message consumer threads + disconnect pika clients.The zombie-session bug this PR targets is handled inside eachturn's own lifecycle by :func:`sync_fail_close_session`, NOT by@@ -224,7 +229,10 @@ class CoPilotExecutor(AppProcess):self._cancel_thread, self.cancel_client, f"{prefix} [cancel]")- # 4. Worker cleanup + executor shutdown+ # 4. Checkpoint and close shared Codex runtimes before worker teardown+ self._close_codex_runtime_pool(prefix)++ # 5. Worker cleanup + executor shutdownif self._executor:from .processor import cleanup_worker@@ -241,7 +249,7 @@ class CoPilotExecutor(AppProcess):logger.info(f"{prefix} Shutting down executor...")self._executor.shutdown(wait=False)- # 5. Release any cluster locks still held+ # 6. Release any cluster locks still heldfor session_id, lock in list(self._task_locks.items()):try:lock.release()@@ -251,6 +259,18 @@ class CoPilotExecutor(AppProcess):logger.info(f"{prefix} Graceful shutdown completed")+ def _close_codex_runtime_pool(self, prefix: str) -> None:+ with self._active_tasks_lock:+ if self._codex_runtime_pool_closed:+ return+ self._codex_runtime_pool_closed = True++ try:+ logger.info(f"{prefix} Closing Codex runtime pool...")+ asyncio.run(get_codex_transport().close_runtime_pool())+ except Exception as e:+ logger.error(f"{prefix} Codex runtime pool cleanup error: {e}")+# ============ RabbitMQ Consumer Methods ============ #@continuous_retry()@@ -641,20 +661,29 @@ def _dispatch_engine_switch_continuation(history, so the user's next message still lands on the SDK engine withthe guide in the prefix."""++ async def dispatch() -> None:+ from backend.copilot.model import get_chat_session++ session = await get_chat_session(session_id, switch.user_id)+ if session is None:+ raise RuntimeError("copilot_session_not_found")+ await schedule_turn(+ session_id=session_id,+ user_id=switch.user_id,+ turn_id=str(uuid.uuid4()),+ message=engine_switch.CONTINUATION_MESSAGE,+ is_user_message=False,+ mode="extended_thinking",+ organization_id=switch.organization_id,+ team_id=switch.team_id,+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,+ )+for attempt in range(1, _SWITCH_DISPATCH_ATTEMPTS + 1):try:- asyncio.run(- schedule_turn(- session_id=session_id,- user_id=switch.user_id,- turn_id=str(uuid.uuid4()),- message=engine_switch.CONTINUATION_MESSAGE,- is_user_message=False,- mode="extended_thinking",- organization_id=switch.organization_id,- team_id=switch.team_id,- )- )+ asyncio.run(dispatch())logger.info(f"Dispatched engine-switch continuation for {session_id}")returnexcept Exception as switch_err:diff --git a/autogpt_platform/backend/backend/copilot/executor/manager_test.py b/autogpt_platform/backend/backend/copilot/executor/manager_test.pyindex 459e2e799f..7003538c10 100644--- a/autogpt_platform/backend/backend/copilot/executor/manager_test.py+++ b/autogpt_platform/backend/backend/copilot/executor/manager_test.py@@ -6,12 +6,17 @@ engine-switch flow (see ``backend.copilot.engine_switch``). These testspin its retry/give-up contract."""-from unittest.mock import AsyncMock, patch+import threading+from types import SimpleNamespace+from unittest.mock import AsyncMock, MagicMock, patch++import pytestfrom backend.copilot.engine_switch import CONTINUATION_MESSAGE, SwitchRequestfrom .manager import (_SWITCH_DISPATCH_ATTEMPTS,+ CoPilotExecutor,_dispatch_engine_switch_continuation,_maybe_dispatch_engine_switch,_persist_switch_failure_marker,@@ -20,6 +25,22 @@ from .manager import (_SWITCH = SwitchRequest(user_id="user-1", organization_id="org-1", team_id=None)[email protected](autouse=True)+def mock_chat_session():+ session = SimpleNamespace(+ metadata=SimpleNamespace(+ llm_auth_provider=None,+ llm_credential_id=None,+ )+ )+ with patch(+ "backend.copilot.model.get_chat_session",+ new_callable=AsyncMock,+ return_value=session,+ ) as mock_get_session:+ yield mock_get_session++def test_dispatch_succeeds_first_try():with (patch(@@ -38,6 +59,23 @@ def test_dispatch_succeeds_first_try():assert kwargs["mode"] == "extended_thinking"+def test_dispatch_preserves_codex_transport_for_extended_thinking(+ mock_chat_session,+):+ mock_chat_session.return_value.metadata.llm_auth_provider = "codex"+ mock_chat_session.return_value.metadata.llm_credential_id = "cred-codex"+ with patch(+ "backend.copilot.executor.manager.schedule_turn",+ new_callable=AsyncMock,+ ) as mock_schedule:+ _dispatch_engine_switch_continuation("sess-1", _SWITCH)++ kwargs = mock_schedule.call_args.kwargs+ assert kwargs["mode"] == "extended_thinking"+ assert kwargs["llm_auth_provider"] == "codex"+ assert kwargs["llm_credential_id"] == "cred-codex"++def test_dispatch_retries_until_success():with (patch(@@ -138,3 +176,59 @@ def test_turn_done_without_switch_is_noop():_maybe_dispatch_engine_switch("sess-1", error_msg=None)mock_thread.assert_not_called()+++def test_codex_runtime_pool_shutdown_runs_once():+ executor = object.__new__(CoPilotExecutor)+ executor._active_tasks_lock_obj = threading.Lock()+ executor._codex_runtime_pool_closed = False+ transport = MagicMock()+ transport.close_runtime_pool = AsyncMock()++ with patch(+ "backend.copilot.executor.manager.get_codex_transport",+ return_value=transport,+ ) as mock_get_transport:+ executor._close_codex_runtime_pool("[test]")+ executor._close_codex_runtime_pool("[test]")++ mock_get_transport.assert_called_once_with()+ transport.close_runtime_pool.assert_awaited_once_with()+++def test_cleanup_closes_codex_pool_between_consumers_and_workers():+ executor = object.__new__(CoPilotExecutor)+ executor.active_tasks = {}+ executor._task_locks = {}+ executor._stop_consuming = threading.Event()+ executor._run_client = MagicMock()+ executor._run_thread = MagicMock()+ executor._cancel_thread = None+ executor._executor = MagicMock()+ executor._executor._max_workers = 1+ executor._active_tasks_lock_obj = threading.Lock()+ executor._codex_runtime_pool_closed = False+ worker_future = MagicMock()+ lifecycle = []++ def record_worker_cleanup(_cleanup_worker):+ lifecycle.append("worker_cleanup")+ return worker_future++ executor._executor.submit.side_effect = record_worker_cleanup++ with (+ patch.object(+ executor,+ "_stop_message_consumers",+ side_effect=lambda *_args: lifecycle.append("consumer_stop"),+ ),+ patch.object(+ executor,+ "_close_codex_runtime_pool",+ side_effect=lambda _prefix: lifecycle.append("codex_pool_close"),+ ),+ ):+ executor.cleanup()++ assert lifecycle == ["consumer_stop", "codex_pool_close", "worker_cleanup"]diff --git a/autogpt_platform/backend/backend/copilot/executor/processor.py b/autogpt_platform/backend/backend/copilot/executor/processor.pyindex bb2d484791..b01dc1474e 100644--- a/autogpt_platform/backend/backend/copilot/executor/processor.py+++ b/autogpt_platform/backend/backend/copilot/executor/processor.py@@ -21,6 +21,7 @@ from backend.copilot.sdk import service as sdk_servicefrom backend.copilot.sdk.dummy import stream_chat_completion_dummyfrom backend.copilot.stream_heartbeat import wrap_stream_with_heartbeatfrom backend.executor.cluster_lock import ClusterLock+from backend.integrations.codex.transport import CodexCredentialIntegrityErrorfrom backend.util.decorator import error_loggedfrom backend.util.feature_flag import Flag, is_feature_enabledfrom backend.util.logging import TruncatedLogger, configure_logging@@ -52,6 +53,7 @@ _CANCEL_DRAIN_LOG_INTERVAL_SECONDS = 1.0# session stays ``running`` until the stale-session watchdog reaps it, but# at least the pool worker thread isn't blocked forever._FAIL_CLOSE_REDIS_TIMEOUT = 10.0+_CODEX_CREDENTIAL_ACQUIRE_TIMEOUT_SECONDS = 5.0# Module-level symbol preserved for backward-compat with callers that import@@ -518,48 +520,117 @@ class CoPilotProcessor:last_refresh = time.monotonic()refresh_interval = 30.0 # Refresh lock every 30 secondserror_msg = None+ credential_lease = Nonetry:- # Choose service based on LaunchDarkly flag.- # Claude Code subscription forces SDK mode (CLI subprocess auth).- config = ChatConfig()-- if config.test_mode:- stream_fn = stream_chat_completion_dummy- log.warning("Using DUMMY service (CHAT_TEST_MODE=true)")- effective_mode = None- else:- # Enforce server-side feature-flag gate so unauthorised- # users cannot force a mode by crafting the request.- effective_mode = await resolve_effective_mode(entry.mode, entry.user_id)- use_sdk = await resolve_use_sdk_for_mode(- effective_mode,- entry.user_id,- use_claude_code_subscription=config.use_claude_code_subscription,- config_default=config.use_claude_agent_sdk,- thinking_available=config.thinking_available,+ from backend.copilot.model import get_chat_session++ session = await get_chat_session(entry.session_id, entry.user_id)+ if session is None:+ raise RuntimeError("copilot_session_not_found")+ if (+ session.metadata.llm_auth_provider != entry.llm_auth_provider+ or session.metadata.llm_credential_id != entry.llm_credential_id+ ):+ raise RuntimeError("codex_session_route_mismatch")++ if entry.llm_auth_provider == "codex":+ from backend.integrations.codex.access import enforce_codex_access+ from backend.integrations.codex.credential_codec import (+ bundle_from_credentials,)- # Building-mode sessions are pinned to the SDK engine- # (guide-in-prompt + in-turn restart live there). Derived- # from message history — survives stale frontend mode- # pickers. get_chat_session is Redis-cached, so this is- # one cache hit, not a DB round-trip.- if not use_sdk and config.thinking_available:- if await _building_mode_forces_sdk(entry.session_id):- use_sdk = True- log.info(- "Forcing SDK engine: session is in agent building mode"- )- stream_fn = (- sdk_service.stream_chat_completion_sdk- if use_sdk- else stream_chat_completion_baseline- )- log.info(- f"Using {'SDK' if use_sdk else 'baseline'} service "- f"(mode={effective_mode or 'default'})"+ from backend.integrations.codex.transport import (+ CodexCredentialBusyError,+ get_codex_transport,)+ if entry.user_id is None:+ raise RuntimeError("codex_user_required")+ codex_user_id = entry.user_id+ await enforce_codex_access(codex_user_id)+ if entry.llm_credential_id is None:+ raise RuntimeError("codex_credential_required")+ try:+ credential_lease = (+ await get_codex_transport().acquire_runtime_lease(+ codex_user_id,+ entry.llm_credential_id,+ lock_timeout_seconds=(+ _CODEX_CREDENTIAL_ACQUIRE_TIMEOUT_SECONDS+ ),+ )+ )+ except CodexCredentialBusyError:+ raise RuntimeError("codex_credential_busy") from None+ except asyncio.CancelledError:+ raise+ except Exception:+ raise RuntimeError("codex_credential_not_found") from None+ try:+ credentials = credential_lease.credentials+ if credentials.type != "oauth2":+ raise RuntimeError("codex_credential_not_found")+ bundle_from_credentials(credentials)+ except asyncio.CancelledError:+ if credential_lease is not None:+ await credential_lease.release()+ credential_lease = None+ raise+ except Exception:+ if credential_lease is not None:+ await credential_lease.release()+ credential_lease = None+ raise RuntimeError("codex_credential_not_found") from None+ stream_fn = sdk_service.stream_chat_completion_sdk+ effective_mode = await resolve_effective_mode(entry.mode, entry.user_id)+ log.info("Using Claude SDK with Codex subscription transport")+ else:+ if entry.llm_credential_id is not None:+ raise RuntimeError("codex_session_route_mismatch")+ # Choose service based on LaunchDarkly flag.+ # Claude Code subscription forces SDK mode (CLI subprocess auth).+ config = ChatConfig()++ if config.test_mode:+ stream_fn = stream_chat_completion_dummy+ log.warning("Using DUMMY service (CHAT_TEST_MODE=true)")+ effective_mode = None+ else:+ # Enforce server-side feature-flag gate so unauthorised+ # users cannot force a mode by crafting the request.+ effective_mode = await resolve_effective_mode(+ entry.mode, entry.user_id+ )+ use_sdk = await resolve_use_sdk_for_mode(+ effective_mode,+ entry.user_id,+ use_claude_code_subscription=(+ config.use_claude_code_subscription+ ),+ config_default=config.use_claude_agent_sdk,+ thinking_available=config.thinking_available,+ )+ # Building-mode sessions are pinned to the SDK engine+ # (guide-in-prompt + in-turn restart live there). Derived+ # from message history — survives stale frontend mode+ # pickers. get_chat_session is Redis-cached, so this is+ # one cache hit, not a DB round-trip.+ if not use_sdk and config.thinking_available:+ if await _building_mode_forces_sdk(entry.session_id):+ use_sdk = True+ log.info(+ "Forcing SDK engine: session is in agent building mode"+ )+ stream_fn = (+ sdk_service.stream_chat_completion_sdk+ if use_sdk+ else stream_chat_completion_baseline+ )+ log.info(+ f"Using {'SDK' if use_sdk else 'baseline'} service "+ f"(mode={effective_mode or 'default'})"+ )+# Stream chat completion and publish chunks to Redis.# stream_and_publish wraps the raw stream with registry# publishing so subscribers on the session Redis stream@@ -578,6 +649,8 @@ class CoPilotProcessor:request_arrival_at=entry.request_arrival_at,organization_id=entry.organization_id,team_id=entry.team_id,+ credential_lease=credential_lease,+ session=session,)# Surround the driver stream with a silence watchdog so the# FE shows escalating ``StreamStatus`` messages during long@@ -631,8 +704,20 @@ class CoPilotProcessor:if not error_msg and cancel.is_set():error_msg = "Operation cancelled"try:- await stream_registry.mark_session_completed(- entry.session_id, error_message=error_msg- )- except Exception as mark_err:- log.error(f"Failed to mark session completed: {mark_err}")+ if credential_lease is not None:+ try:+ await credential_lease.release()+ except CodexCredentialIntegrityError as release_err:+ error_msg = error_msg or "codex_credential_checkpoint_failed"+ log.error(+ f"Failed to checkpoint Codex credential: {release_err}"+ )+ except Exception as release_err:+ log.error(f"Failed to release Codex credential: {release_err}")+ finally:+ try:+ await stream_registry.mark_session_completed(+ entry.session_id, error_message=error_msg+ )+ except Exception as mark_err:+ log.error(f"Failed to mark session completed: {mark_err}")diff --git a/autogpt_platform/backend/backend/copilot/executor/processor_test.py b/autogpt_platform/backend/backend/copilot/executor/processor_test.pyindex 5d88db377a..c39bf933ea 100644--- a/autogpt_platform/backend/backend/copilot/executor/processor_test.py+++ b/autogpt_platform/backend/backend/copilot/executor/processor_test.py@@ -14,17 +14,26 @@ import asyncioimport concurrent.futuresimport loggingimport threading+from types import SimpleNamespacefrom unittest.mock import AsyncMock, MagicMock, patchimport pytest+from backend.copilot.config import CopilotModefrom backend.copilot.executor.processor import (+ _CODEX_CREDENTIAL_ACQUIRE_TIMEOUT_SECONDS,CoPilotProcessor,resolve_effective_mode,resolve_use_sdk_for_mode,sync_fail_close_session,)from backend.copilot.executor.utils import CoPilotExecutionEntry, CoPilotLogMetadata+from backend.copilot.model import ChatSession+from backend.copilot.rate_limit import UserPaywalledError+from backend.integrations.codex.transport import (+ CodexCredentialBusyError,+ CodexCredentialIntegrityError,+)class TestResolveUseSdkForMode:@@ -297,6 +306,10 @@ class TestExecuteAsyncAclose:"backend.copilot.executor.processor.stream_registry.mark_session_completed",new=AsyncMock(),),+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=ChatSession.new("user-1", dry_run=False)),+ ),]@pytest.mark.asyncio@@ -307,7 +320,7 @@ class TestExecuteAsyncAclose:cluster_lock = MagicMock()patches = self._patches(published)- with patches[0], patches[1], patches[2], patches[3]:+ with patches[0], patches[1], patches[2], patches[3], patches[4]:await proc._execute_async(_make_entry(), cancel, cluster_lock, _make_log())assert published.aclose_called is True@@ -322,12 +335,336 @@ class TestExecuteAsyncAclose:cluster_lock = MagicMock()patches = self._patches(published)- with patches[0], patches[1], patches[2], patches[3]:+ with patches[0], patches[1], patches[2], patches[3], patches[4]:await proc._execute_async(_make_entry(), cancel, cluster_lock, _make_log())assert published.aclose_called is True+def _codex_entry(+ *,+ credential_id: str = "cred-1",+ mode: CopilotMode | None = None,+) -> CoPilotExecutionEntry:+ return CoPilotExecutionEntry(+ session_id="sess-codex",+ turn_id="turn-codex",+ user_id="user-1",+ message="hi",+ mode=mode,+ llm_auth_provider="codex",+ llm_credential_id=credential_id,+ )+++def _codex_session(+ *, credential_id: str = "cred-1", builder_graph_id: str | None = None+) -> ChatSession:+ session = ChatSession.new(+ "user-1",+ dry_run=False,+ llm_auth_provider="codex",+ llm_credential_id=credential_id,+ )+ session.session_id = "sess-codex"+ session.metadata.builder_graph_id = builder_graph_id+ return session+++async def test_codex_route_uses_claude_sdk_for_builder_and_releases_lease():+ published = _TrackedStream(events=[])+ lease = MagicMock()+ lease.credentials = SimpleNamespace(type="oauth2", id="cred-1")+ lease.release = AsyncMock()+ transport = MagicMock()+ transport.acquire_runtime_lease = AsyncMock(return_value=lease)+ baseline_stream = MagicMock()+ sdk_stream = MagicMock(return_value=MagicMock())+ resolve_mode = AsyncMock(return_value=None)++ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=_codex_session(builder_graph_id="graph-1")),+ ),+ patch(+ "backend.integrations.codex.transport.get_codex_transport",+ return_value=transport,+ ),+ patch("backend.integrations.codex.credential_codec.bundle_from_credentials"),+ patch(+ "backend.copilot.executor.processor.stream_chat_completion_baseline",+ baseline_stream,+ ),+ patch(+ "backend.copilot.sdk.service.stream_chat_completion_sdk",+ sdk_stream,+ ),+ patch(+ "backend.copilot.executor.processor.resolve_effective_mode",+ resolve_mode,+ ),+ patch(+ "backend.copilot.executor.processor.wrap_stream_with_heartbeat",+ return_value=MagicMock(),+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.stream_and_publish",+ return_value=published,+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ new=AsyncMock(),+ ),+ ):+ await CoPilotProcessor()._execute_async(+ _codex_entry(mode="extended_thinking"),+ threading.Event(),+ MagicMock(),+ _make_log(),+ )++ transport.acquire_runtime_lease.assert_awaited_once_with(+ "user-1",+ "cred-1",+ lock_timeout_seconds=_CODEX_CREDENTIAL_ACQUIRE_TIMEOUT_SECONDS,+ )+ lease.release.assert_awaited_once()+ resolve_mode.assert_awaited_once_with("extended_thinking", "user-1")+ sdk_stream.assert_called_once()+ assert sdk_stream.call_args.kwargs["mode"] is None+ assert sdk_stream.call_args.kwargs["credential_lease"] is lease+ assert sdk_stream.call_args.kwargs["session"].session_id == "sess-codex"+ baseline_stream.assert_not_called()+++async def test_codex_release_failure_does_not_fail_successful_turn():+ published = _TrackedStream(events=[])+ lease = MagicMock()+ lease.credentials = SimpleNamespace(type="oauth2", id="cred-1")+ lease.release = AsyncMock(side_effect=RuntimeError("Redis release failed"))+ transport = MagicMock()+ transport.acquire_runtime_lease = AsyncMock(return_value=lease)+ mark_completed = AsyncMock()++ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=_codex_session()),+ ),+ patch(+ "backend.integrations.codex.transport.get_codex_transport",+ return_value=transport,+ ),+ patch("backend.integrations.codex.credential_codec.bundle_from_credentials"),+ patch(+ "backend.copilot.sdk.service.stream_chat_completion_sdk",+ return_value=MagicMock(),+ ),+ patch(+ "backend.copilot.executor.processor.wrap_stream_with_heartbeat",+ return_value=MagicMock(),+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.stream_and_publish",+ return_value=published,+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ mark_completed,+ ),+ ):+ await CoPilotProcessor()._execute_async(+ _codex_entry(),+ threading.Event(),+ MagicMock(),+ _make_log(),+ )++ lease.release.assert_awaited_once()+ mark_completed.assert_awaited_once_with(+ "sess-codex",+ error_message=None,+ )+++async def test_codex_checkpoint_failure_fails_closed_after_successful_turn():+ published = _TrackedStream(events=[])+ lease = MagicMock()+ lease.credentials = SimpleNamespace(type="oauth2", id="cred-1")+ lease.release = AsyncMock(+ side_effect=CodexCredentialIntegrityError("codex_credential_checkpoint_failed")+ )+ transport = MagicMock()+ transport.acquire_runtime_lease = AsyncMock(return_value=lease)+ mark_completed = AsyncMock()++ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=_codex_session()),+ ),+ patch(+ "backend.integrations.codex.transport.get_codex_transport",+ return_value=transport,+ ),+ patch("backend.integrations.codex.credential_codec.bundle_from_credentials"),+ patch(+ "backend.copilot.sdk.service.stream_chat_completion_sdk",+ return_value=MagicMock(),+ ),+ patch(+ "backend.copilot.executor.processor.wrap_stream_with_heartbeat",+ return_value=MagicMock(),+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.stream_and_publish",+ return_value=published,+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ mark_completed,+ ),+ ):+ await CoPilotProcessor()._execute_async(+ _codex_entry(),+ threading.Event(),+ MagicMock(),+ _make_log(),+ )++ lease.release.assert_awaited_once()+ mark_completed.assert_awaited_once_with(+ "sess-codex",+ error_message="codex_credential_checkpoint_failed",+ )+++async def test_codex_queue_route_mismatch_fails_before_credential_acquire():+ transport = MagicMock()+ transport.acquire_runtime_lease = AsyncMock()+ mark_completed = AsyncMock()+ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=_codex_session(credential_id="authoritative")),+ ),+ patch(+ "backend.integrations.codex.transport.get_codex_transport",+ return_value=transport,+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ mark_completed,+ ),+ ):+ with pytest.raises(RuntimeError, match="codex_session_route_mismatch"):+ await CoPilotProcessor()._execute_async(+ _codex_entry(credential_id="queued-stale"),+ threading.Event(),+ MagicMock(),+ _make_log(),+ )++ transport.acquire_runtime_lease.assert_not_awaited()+ mark_completed.assert_awaited_once_with(+ "sess-codex",+ error_message="codex_session_route_mismatch",+ )+++async def test_codex_entitlement_is_checked_before_credential_acquire():+ transport = MagicMock()+ transport.acquire_runtime_lease = AsyncMock()+ mark_completed = AsyncMock()+ gate = AsyncMock(side_effect=UserPaywalledError("Max plan required"))++ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=_codex_session()),+ ),+ patch(+ "backend.integrations.codex.transport.get_codex_transport",+ return_value=transport,+ ),+ patch(+ "backend.integrations.codex.access.enforce_codex_access",+ new=gate,+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ mark_completed,+ ),+ pytest.raises(UserPaywalledError, match="Max plan required"),+ ):+ await CoPilotProcessor()._execute_async(+ _codex_entry(),+ threading.Event(),+ MagicMock(),+ _make_log(),+ )++ gate.assert_awaited_once_with("user-1")+ transport.acquire_runtime_lease.assert_not_awaited()+ mark_completed.assert_awaited_once_with(+ "sess-codex",+ error_message="Max plan required",+ )+++async def test_codex_busy_credential_fails_closed_without_platform_fallback():+ transport = MagicMock()+ transport.acquire_runtime_lease = AsyncMock(+ side_effect=CodexCredentialBusyError("codex_credential_busy")+ )+ mark_completed = AsyncMock()+ baseline_stream = MagicMock()+ sdk_stream = MagicMock()+ with (+ patch(+ "backend.copilot.model.get_chat_session",+ new=AsyncMock(return_value=_codex_session()),+ ),+ patch(+ "backend.integrations.codex.transport.get_codex_transport",+ return_value=transport,+ ),+ patch(+ "backend.copilot.executor.processor.stream_chat_completion_baseline",+ baseline_stream,+ ),+ patch(+ "backend.copilot.sdk.service.stream_chat_completion_sdk",+ sdk_stream,+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ mark_completed,+ ),+ ):+ with pytest.raises(RuntimeError, match="codex_credential_busy"):+ await CoPilotProcessor()._execute_async(+ _codex_entry(),+ threading.Event(),+ MagicMock(),+ _make_log(),+ )++ baseline_stream.assert_not_called()+ sdk_stream.assert_not_called()+ mark_completed.assert_awaited_once_with(+ "sess-codex",+ error_message="codex_credential_busy",+ )++@pytest.fixturedef exec_loop():"""Long-lived asyncio loop on a daemon thread — mirrors the layout@@ -484,9 +821,12 @@ class TestExecuteSafetyNet:mock_mark = AsyncMock()proc = CoPilotProcessor()self._attach_exec_loop(proc, exec_loop)- with patch.object(proc, "_execute"), patch(- "backend.copilot.executor.processor.stream_registry.mark_session_completed",- new=mock_mark,+ with (+ patch.object(proc, "_execute"),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ new=mock_mark,+ ),):self._run_execute_in_thread(proc, threading.Event())@@ -501,13 +841,16 @@ class TestExecuteSafetyNet:mock_mark = AsyncMock()proc = CoPilotProcessor()self._attach_exec_loop(proc, exec_loop)- with patch.object(- proc,- "_execute",- side_effect=concurrent.futures.TimeoutError("grace expired"),- ), patch(- "backend.copilot.executor.processor.stream_registry.mark_session_completed",- new=mock_mark,+ with (+ patch.object(+ proc,+ "_execute",+ side_effect=concurrent.futures.TimeoutError("grace expired"),+ ),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ new=mock_mark,+ ),):self._run_execute_in_thread(proc, threading.Event())@@ -533,9 +876,12 @@ class TestExecuteSafetyNet:proc = CoPilotProcessor()self._attach_exec_loop(proc, exec_loop)- with patch.object(proc, "_execute", side_effect=_broken_execute), patch(- "backend.copilot.executor.processor.stream_registry.mark_session_completed",- new=_ok,+ with (+ patch.object(proc, "_execute", side_effect=_broken_execute),+ patch(+ "backend.copilot.executor.processor.stream_registry.mark_session_completed",+ new=_ok,+ ),):self._run_execute_in_thread(proc, threading.Event())diff --git a/autogpt_platform/backend/backend/copilot/executor/utils.py b/autogpt_platform/backend/backend/copilot/executor/utils.pyindex e6e48f9b78..6962e8fbc8 100644--- a/autogpt_platform/backend/backend/copilot/executor/utils.py+++ b/autogpt_platform/backend/backend/copilot/executor/utils.py@@ -16,7 +16,7 @@ from backend.copilot.active_turns import (get_inflight_turn_limit,inflight_turn_limit_message,)-from backend.copilot.config import CopilotLLMModel, CopilotMode+from backend.copilot.config import CopilotLlmAuthProvider, CopilotLLMModel, CopilotModefrom backend.copilot.permissions import CopilotPermissionsfrom backend.data.rabbitmq import Exchange, ExchangeType, Queue, RabbitMQConfigfrom backend.util.logging import TruncatedLogger, is_structured_logging_enabled@@ -211,6 +211,12 @@ class CoPilotExecutionEntry(BaseModel):model: CopilotLLMModel | None = None"""Per-request model tier: 'standard' or 'advanced'. None = server default."""+ llm_auth_provider: CopilotLlmAuthProvider = "platform"+ """Session-selected model authentication and execution route."""++ llm_credential_id: str | None = None+ """Opaque user-owned credential identifier for a Codex route."""+permissions: CopilotPermissions | None = None"""Capability filter inherited from a parent run (e.g. ``run_sub_session``forwards its parent's permissions so the sub can't escalate). ``None``@@ -248,6 +254,8 @@ async def enqueue_copilot_turn(team_id: str | None = None,mode: CopilotMode | None = None,model: CopilotLLMModel | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,permissions: CopilotPermissions | None = None,request_arrival_at: float = 0.0,) -> None:@@ -280,6 +288,8 @@ async def enqueue_copilot_turn(team_id=team_id,mode=mode,model=model,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,permissions=permissions,request_arrival_at=request_arrival_at,)@@ -307,6 +317,8 @@ async def schedule_turn(team_id: str | None = None,mode: CopilotMode | None = None,model: CopilotLLMModel | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,permissions: CopilotPermissions | None = None,request_arrival_at: float = 0.0,) -> None:@@ -371,6 +383,8 @@ async def schedule_turn(team_id=team_id,mode=mode,model=model,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,permissions=permissions,request_arrival_at=request_arrival_at,)@@ -392,6 +406,8 @@ async def dispatch_turn(team_id: str | None = None,mode: CopilotMode | None = None,model: CopilotLLMModel | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,permissions: CopilotPermissions | None = None,request_arrival_at: float = 0.0,) -> None:@@ -443,6 +459,8 @@ async def dispatch_turn(team_id=team_id,mode=mode,model=model,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,permissions=permissions,request_arrival_at=request_arrival_at,)@@ -474,6 +492,8 @@ async def schedule_chat_turn(team_id: str | None = None,mode: CopilotMode | None = None,model: CopilotLLMModel | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,permissions: CopilotPermissions | None = None,request_arrival_at: float = 0.0,) -> str | None:@@ -538,6 +558,8 @@ async def schedule_chat_turn(team_id=team_id,mode=mode,model=model,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,permissions=permissions,request_arrival_at=request_arrival_at,)diff --git a/autogpt_platform/backend/backend/copilot/model.py b/autogpt_platform/backend/backend/copilot/model.pyindex e7b900e7f2..90e2dcf078 100644--- a/autogpt_platform/backend/backend/copilot/model.py+++ b/autogpt_platform/backend/backend/copilot/model.py@@ -30,7 +30,7 @@ from backend.data.redis_client import get_redis_asyncfrom backend.util import jsonfrom backend.util.exceptions import DatabaseError, NotFoundError, RedisError-from .config import ChatConfig+from .config import ChatConfig, CopilotLlmAuthProviderlogger = logging.getLogger(__name__)config = ChatConfig()@@ -41,7 +41,15 @@ config = ChatConfig()# stamped column and model_router / the SDK service import the ONE# definition instead of re-declaring the Literal. ("fallback" is stamped,# never routed: it marks a CLI 529-overload fallback to a different model.)-RoutingSource = Literal["ld", "catalog", "env", "fallback"]+RoutingSource = Literal[+ "ld",+ "catalog",+ "env",+ "fallback",+ "preferred",+ "account_default",+ "account_available",+]# Redis cache key prefix for chat sessions@@ -73,6 +81,8 @@ class ChatSessionMetadata(BaseModel):"""dry_run: bool = False+ llm_auth_provider: CopilotLlmAuthProvider = "platform"+ llm_credential_id: str | None = None# Builder-panel binding: when set, the session is locked to the given# graph. ``edit_agent`` / ``run_agent`` default their ``agent_id`` to@@ -115,7 +125,7 @@ class ChatMessage(BaseModel):# Which LLM served this assistant turn ("model" — visible to clients,# the model-badge UX) and which routing layer picked it- # ("ld" | "catalog" | "env" | "fallback" — the last when the CLI's+ # (platform and Codex routing sources, plus "fallback" when the CLI's# overload fallback served a different model than the routed one).# Product-intelligence mirrors these to segment quality judgments by# model. None on user/tool rows.@@ -375,6 +385,8 @@ class ChatSession(ChatSessionInfo):source_platform: str | None = None,organization_id: str | None = None,team_id: str | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,expert_id: str | None = None,) -> Self:return cls(@@ -390,6 +402,8 @@ class ChatSession(ChatSessionInfo):dry_run=dry_run,builder_graph_id=builder_graph_id,source_platform=source_platform,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,),organization_id=organization_id,team_id=team_id,@@ -1153,6 +1167,8 @@ async def create_chat_session(organization_id: str | None = None,team_id: str | None = None,source_platform: str | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,expert_id: str | None = None,) -> ChatSession:"""Create a new chat session and persist it.@@ -1180,6 +1196,8 @@ async def create_chat_session(source_platform=source_platform,organization_id=organization_id,team_id=team_id,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,expert_id=expert_id,)diff --git a/autogpt_platform/backend/backend/copilot/model_router.py b/autogpt_platform/backend/backend/copilot/model_router.pyindex f7238a9f2a..181cc0fcff 100644--- a/autogpt_platform/backend/backend/copilot/model_router.py+++ b/autogpt_platform/backend/backend/copilot/model_router.py@@ -51,6 +51,12 @@ import backend.data.llm_registry as llm_registryfrom backend.copilot.config import ChatConfigfrom backend.copilot.model import RoutingSourcefrom backend.data.llm_registry.llm_models import LLMModel, transport_slug_candidates+from backend.integrations.codex.models import CodexModelInfo, CodexReasoningEffort+from backend.integrations.codex.transport import (+ PooledCodexRuntimeLease,+ get_codex_transport,+)+from backend.integrations.credential_lease import CredentialLeasefrom backend.util.feature_flag import Flag, get_feature_flag_valuefrom backend.util.settings import BehaveAs, Settings@@ -59,8 +65,15 @@ settings = Settings()ModelMode = Literal["fast", "thinking"]ModelTier = Literal["standard", "advanced"]+CodexRoutingSource = Literal[+ "catalog",+ "preferred",+ "account_default",+ "account_available",+]ROUTE_SURFACE_COPILOT = "copilot"+ROUTE_SURFACE_CODEX = "copilot_codex"class ResolvedModel(NamedTuple):@@ -68,7 +81,28 @@ class ResolvedModel(NamedTuple):source: RoutingSource-def _catalog_lookup(slug: str) -> "llm_registry.RegistryModel | None":+class ResolvedCodexModel(NamedTuple):+ model: str+ effort: CodexReasoningEffort | None+ source: CodexRoutingSource+++_CODEX_PREFERRED_MODELS: dict[tuple[ModelMode, ModelTier], str] = {+ ("fast", "standard"): LLMModel.GPT5_6_LUNA.value,+ ("fast", "advanced"): LLMModel.GPT5_6_TERRA.value,+ ("thinking", "standard"): LLMModel.GPT5_6_TERRA.value,+ ("thinking", "advanced"): LLMModel.GPT5_6_SOL.value,+}++_CODEX_PREFERRED_EFFORTS: dict[tuple[ModelMode, ModelTier], CodexReasoningEffort] = {+ ("fast", "standard"): "low",+ ("fast", "advanced"): "medium",+ ("thinking", "standard"): "high",+ ("thinking", "advanced"): "xhigh",+}+++def _catalog_lookup(slug: str) -> llm_registry.RegistryModel | None:"""Look up *slug* in the catalog, tolerating transport spellings.The catalog registers Claude models under bare canonical enum slugs@@ -291,3 +325,138 @@ async def resolve_model_route(return ResolvedModel(cell_slug, "catalog")return await _env_floor(config, mode, tier)+++async def resolve_codex_model_route(+ mode: ModelMode,+ tier: ModelTier,+ credential_lease: CredentialLease | PooledCodexRuntimeLease,+) -> ResolvedCodexModel:+ """Resolve a Codex model against both the catalog and the account."""+ advertised = await _advertised_codex_models(credential_lease)++ if catalog_route := _codex_catalog_route(advertised, mode, tier):+ return catalog_route++ if preferred_route := _codex_preferred_route(advertised, mode, tier):+ return preferred_route++ if account_route := _codex_account_route(advertised, mode, tier):+ return account_route++ raise RuntimeError("codex_model_unavailable")+++async def _advertised_codex_models(+ credential_lease: CredentialLease | PooledCodexRuntimeLease,+) -> list[CodexModelInfo]:+ if isinstance(credential_lease, PooledCodexRuntimeLease):+ return await credential_lease.models()+ return await get_codex_transport().models(credential_lease)+++def _codex_catalog_route(+ advertised: list[CodexModelInfo],+ mode: ModelMode,+ tier: ModelTier,+) -> ResolvedCodexModel | None:+ catalog_slug = llm_registry.get_route(ROUTE_SURFACE_CODEX, mode, tier)+ if not catalog_slug:+ return None++ model = next((item for item in advertised if item.model == catalog_slug), None)+ if model is not None and _codex_catalog_allows(catalog_slug):+ return _resolved_codex_model(model, mode, tier, "catalog")++ logger.warning(+ "[model_router] Codex catalog route %r is disabled or unavailable "+ "for this account; falling through for (%s, %s)",+ catalog_slug,+ mode,+ tier,+ )+ return None+++def _codex_preferred_route(+ advertised: list[CodexModelInfo],+ mode: ModelMode,+ tier: ModelTier,+) -> ResolvedCodexModel | None:+ preferred_slug = _CODEX_PREFERRED_MODELS[(mode, tier)]+ preferred = next(+ (model for model in advertised if model.model == preferred_slug),+ None,+ )+ if preferred is None or not _codex_catalog_allows(preferred_slug):+ return None+ return _resolved_codex_model(preferred, mode, tier, "preferred")+++def _codex_account_route(+ advertised: list[CodexModelInfo],+ mode: ModelMode,+ tier: ModelTier,+) -> ResolvedCodexModel | None:+ candidates: tuple[tuple[CodexRoutingSource, bool], ...] = (+ ("account_default", True),+ ("account_available", False),+ )+ for source, default_only in candidates:+ model = next(+ (+ item+ for item in advertised+ if (not default_only or item.is_default)+ and not item.hidden+ and _codex_account_fallback_allowed(item.model)+ ),+ None,+ )+ if model is not None:+ return _resolved_codex_model(model, mode, tier, source)+ return None+++def _resolved_codex_model(+ model: CodexModelInfo,+ mode: ModelMode,+ tier: ModelTier,+ source: CodexRoutingSource,+) -> ResolvedCodexModel:+ return ResolvedCodexModel(+ model.model,+ _codex_effort(model, mode, tier),+ source,+ )+++def _codex_catalog_allows(slug: str) -> bool:+ if not llm_registry.has_models():+ return True+ model = _catalog_lookup(slug)+ return bool(+ model is not None and model.is_enabled and model.metadata.provider == "openai"+ )+++def _codex_account_fallback_allowed(slug: str) -> bool:+ if not llm_registry.has_models():+ return True+ model = _catalog_lookup(slug)+ if model is None:+ return True+ return model.is_enabled and model.metadata.provider == "openai"+++def _codex_effort(+ model: CodexModelInfo,+ mode: ModelMode,+ tier: ModelTier,+) -> CodexReasoningEffort | None:+ preferred = _CODEX_PREFERRED_EFFORTS[(mode, tier)]+ if preferred in model.supported_reasoning_efforts:+ return preferred+ if model.default_reasoning_effort in model.supported_reasoning_efforts:+ return model.default_reasoning_effort+ return Nonediff --git a/autogpt_platform/backend/backend/copilot/model_router_test.py b/autogpt_platform/backend/backend/copilot/model_router_test.pyindex 559a0c8665..38b2ebb3a5 100644--- a/autogpt_platform/backend/backend/copilot/model_router_test.py+++ b/autogpt_platform/backend/backend/copilot/model_router_test.py@@ -631,7 +631,7 @@ class TestRegistryGating:assert resolved == ("anthropic/claude-sonnet-4.6", "catalog")def test_catalog_cell_values_normalize_on_every_transport(self, mocker):- """The REAL catalog routing cells must be transport-ready: OpenRouter+ """Non-Codex catalog routing cells must be transport-ready: OpenRoutersends them verbatim (so they must be genuine OR slugs — the dotforms), and the direct-Anthropic family strips the vendor prefix anddedots without raising."""@@ -647,7 +647,8 @@ class TestRegistryGating:# before anyone claims a cell.cells = [slug- for modes in get_catalog().routing.values()+ for surface, modes in get_catalog().routing.items()+ if surface != "copilot_codex"for tiers in modes.values()for slug in tiers.values()] + [diff --git a/autogpt_platform/backend/backend/copilot/sdk/codex_compat_cli_test.py b/autogpt_platform/backend/backend/copilot/sdk/codex_compat_cli_test.pynew file mode 100644index 0000000000..a7f87224ca--- /dev/null+++ b/autogpt_platform/backend/backend/copilot/sdk/codex_compat_cli_test.py@@ -0,0 +1,166 @@+import asyncio+from collections.abc import AsyncIterator+from contextlib import asynccontextmanager+from typing import cast++import pytest+from claude_agent_sdk import (+ ClaudeAgentOptions,+ ClaudeSDKClient,+ ResultMessage,+ create_sdk_mcp_server,+ tool,+)+from openai_codex.generated.v2_all import AgentMessageDeltaNotification+from openai_codex.models import Notification++from backend.copilot.sdk.codex_compat_gateway import CodexAnthropicGateway+from backend.copilot.sdk.env import build_sdk_env+from backend.integrations.codex.models import (+ CodexDynamicToolCall,+ CodexDynamicToolResult,+ CodexInvocationResult,+ CodexTokenUsage,+)+from backend.integrations.credential_lease import CredentialLease+++class _ClaudeHarnessAgentSession:+ def __init__(self) -> None:+ self.tool_result: CodexDynamicToolResult | None = None++ async def invoke(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ ) -> CodexInvocationResult:+ echo_tool = next(+ tool for tool in dynamic_tools if tool.description == "Echo a value"+ )+ assert not echo_tool.name.startswith("mcp__")+ self.tool_result = await tool_handler(+ CodexDynamicToolCall(+ thread_id="thread-cli",+ turn_id="turn-cli",+ call_id="call-cli",+ tool=echo_tool.name,+ arguments={"value": "ping"},+ )+ )+ assert event_handler is not None+ await event_handler(+ Notification(+ method="item/agentMessage/delta",+ payload=AgentMessageDeltaNotification(+ delta="Claude harness round trip complete",+ itemId="item-cli",+ threadId="thread-cli",+ turnId="turn-cli",+ ),+ )+ )+ return CodexInvocationResult(+ response_id="response-cli",+ final_response="Claude harness round trip complete",+ status="completed",+ usage=CodexTokenUsage(+ input_tokens=20,+ cached_input_tokens=0,+ output_tokens=5,+ reasoning_output_tokens=0,+ total_tokens=25,+ ),+ )+++class _ClaudeHarnessTransport:+ def __init__(self, session: _ClaudeHarnessAgentSession) -> None:+ self.session = session++ @asynccontextmanager+ async def agent_session(+ self,+ _lease: CredentialLease,+ ) -> AsyncIterator[_ClaudeHarnessAgentSession]:+ yield self.session+++async def test_bundled_claude_cli_executes_mcp_tool_through_codex_gateway(+ tmp_path,+) -> None:+ tool_called = asyncio.Event()+ stderr_lines: list[str] = []++ @tool("echo", "Echo a value", {"value": str})+ async def echo(arguments):+ tool_called.set()+ return {"content": [{"type": "text", "text": f"echo:{arguments['value']}"}]}++ mcp_server = create_sdk_mcp_server(+ name="codex-conformance",+ tools=[echo],+ )+ agent_session = _ClaudeHarnessAgentSession()+ gateway = CodexAnthropicGateway(+ credential_lease=cast(CredentialLease, object()),+ model="gpt-5.6-terra",+ effort="medium",+ transport=_ClaudeHarnessTransport(agent_session),+ )++ async with gateway:+ options = ClaudeAgentOptions(+ model="gpt-5.6-terra",+ system_prompt="Use the supplied echo tool once.",+ mcp_servers={"codex-conformance": mcp_server},+ allowed_tools=["mcp__codex-conformance__echo"],+ disallowed_tools=[+ "Bash",+ "Edit",+ "Glob",+ "Grep",+ "Read",+ "Task",+ "WebFetch",+ "WebSearch",+ "Write",+ ],+ cwd=str(tmp_path),+ max_turns=3,+ stderr=stderr_lines.append,+ env=build_sdk_env(+ session_id="codex-cli-conformance",+ sdk_cwd=str(tmp_path),+ model="gpt-5.6-terra",+ codex_gateway_url=gateway.base_url,+ codex_gateway_token=gateway.auth_token,+ ),+ )+ received = []+ async with ClaudeSDKClient(options=options) as client:+ await client.query("Call echo with ping, then report completion.")+ try:+ async with asyncio.timeout(45):+ async for message in client.receive_response():+ received.append(message)+ except TimeoutError as exc:+ raise AssertionError(+ "Claude CLI did not finish the Codex gateway round trip; "+ f"stderr={stderr_lines!r}"+ ) from exc++ assert tool_called.is_set()+ assert agent_session.tool_result is not None+ assert agent_session.tool_result.success+ assert "echo:ping" in agent_session.tool_result.content+ assert any(+ isinstance(message, ResultMessage) and not message.is_error+ for message in received+ )+ assert gateway.result is not None+ assert gateway.result.usage is not None+ assert gateway.result.usage.total_tokens == 25diff --git a/autogpt_platform/backend/backend/copilot/sdk/codex_compat_gateway.py b/autogpt_platform/backend/backend/copilot/sdk/codex_compat_gateway.pynew file mode 100644index 0000000000..b70e91dc01--- /dev/null+++ b/autogpt_platform/backend/backend/copilot/sdk/codex_compat_gateway.py@@ -0,0 +1,989 @@+from __future__ import annotations++import asyncio+import hashlib+import json+import logging+import re+import secrets+from collections.abc import Awaitable, Callable, Iterable+from contextlib import AbstractAsyncContextManager, suppress+from typing import Protocol, cast+from uuid import uuid4++from aiohttp import web+from openai_codex.generated.v2_all import AgentMessageDeltaNotification+from openai_codex.models import Notification+from pydantic import BaseModel, ConfigDict, Field++from backend.integrations.codex.models import (+ CodexDynamicToolCall,+ CodexDynamicToolResult,+ CodexDynamicToolSpec,+ CodexInvocationRequest,+ CodexInvocationResult,+ CodexReasoningEffort,+ CodexTokenUsage,+)+from backend.integrations.codex.transport import (+ CodexAgentSession,+ CodexTransport,+ get_codex_transport,+)+from backend.integrations.credential_lease import CredentialLease++_MAX_REQUEST_BYTES = 16 * 1024 * 1024+_MAX_LOGGED_ERROR_MESSAGE_CHARS = 240+_REDACTED = "[REDACTED]"+_TOOL_NAME_PATTERN = re.compile(r"^[a-zA-Z0-9_-]{1,128}$")+_BEARER_PATTERN = re.compile(r"(?i)\bBearer\s+[^\s,;]+")+_JWT_PATTERN = re.compile(+ r"(?<![A-Za-z0-9_-])[A-Za-z0-9_-]{6,}\.[A-Za-z0-9_-]{6,}"+ r"\.[A-Za-z0-9_-]{6,}(?![A-Za-z0-9_-])"+)+_DEVICE_CODE_PATTERN = re.compile(r"\b[A-Z0-9]{4}(?:-[A-Z0-9]{4}){1,3}\b")+_PROVIDER_STATE_PATTERN = re.compile(r"(?is)[\"']?provider[_ -]?state[\"']?\s*[:=].*$")+_SECRET_ASSIGNMENT_PATTERN = re.compile(+ r"(?ix)"+ r"(?P<label>\b(?:access[_ -]?token|refresh[_ -]?token|id[_ -]?token|"+ r"device[_ -]?code|user[_ -]?code|authorization|anthropic[_ -]?auth[_ -]?"+ r"token|api[_ -]?key)\b)"+ r"\s*[\"']?\s*[:=]\s*(?:Bearer\s+)?"+ r"(?:\"(?:\\.|[^\"])*\"|'(?:\\.|[^'])*'|[^\s,;]+)"+)++logger = logging.getLogger(__name__)+++class _AgentTransport(Protocol):+ def agent_session(+ self,+ lease: CredentialLease,+ ) -> AbstractAsyncContextManager[CodexAgentSession]: ...+++class _AgentSession(Protocol):+ async def invoke(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None = None,+ ) -> CodexInvocationResult: ...+++class _GatewayState(BaseModel):+ model_config = ConfigDict(arbitrary_types_allowed=True)+++class _TextDelta(_GatewayState):+ text: str+++class _ToolUse(_GatewayState):+ call_id: str+ name: str+ arguments: object+++class _Completed(_GatewayState):+ result: CodexInvocationResult+++class _Failed(_GatewayState):+ error: BaseException+++_ConversationEvent = _TextDelta | _ToolUse | _Completed | _Failed+++class _Conversation(_GatewayState):+ id: str+ queue: asyncio.Queue[_ConversationEvent] = Field(default_factory=asyncio.Queue)+ pending: dict[str, _ToolCallRecord] = Field(default_factory=dict)+ response_lock: asyncio.Lock = Field(default_factory=asyncio.Lock)+ task: asyncio.Task[None] | None = None+ result: CodexInvocationResult | None = None+++class _ToolCallRecord(_GatewayState):+ gateway_call_id: str+ raw_call_id: str+ conversation: _Conversation+ future: asyncio.Future[CodexDynamicToolResult]+ result: CodexDynamicToolResult | None = None+ claim_fingerprint: str | None = None+ closed: bool = False+++_Conversation.model_rebuild()+_ToolCallRecord.model_rebuild()+++class _DuplicateToolResultError(ValueError):+ pass+++class CodexAnthropicGateway:+ def __init__(+ self,+ *,+ credential_lease: CredentialLease | None = None,+ agent_session: _AgentSession | None = None,+ model: str,+ effort: CodexReasoningEffort | None = None,+ transport: CodexTransport | _AgentTransport | None = None,+ ) -> None:+ if credential_lease is None and agent_session is None:+ raise ValueError(+ "Codex gateway requires a credential lease or agent session"+ )+ self._credential_lease = credential_lease+ self.model = model+ self.effort: CodexReasoningEffort | None = effort+ self._transport = cast(+ _AgentTransport,+ transport if transport is not None else get_codex_transport(),+ )+ self._agent_context: AbstractAsyncContextManager[CodexAgentSession] | None = (+ None+ )+ self._agent_session: _AgentSession | None = agent_session+ self._runner: web.AppRunner | None = None+ self._site: web.TCPSite | None = None+ self._base_url: str | None = None+ self._auth_token = secrets.token_urlsafe(32)+ self._conversations: dict[str, _Conversation] = {}+ self._tool_calls: dict[str, _ToolCallRecord] = {}+ self._results: list[CodexInvocationResult] = []+ self._closed = False+ self._close_lock = asyncio.Lock()++ @property+ def base_url(self) -> str:+ if self._base_url is None:+ raise RuntimeError("Codex Anthropic gateway is not running")+ return self._base_url++ @property+ def auth_token(self) -> str:+ return self._auth_token++ @property+ def result(self) -> CodexInvocationResult | None:+ if not self._results:+ return None+ latest = self._results[-1]+ usage = _sum_usage(result.usage for result in self._results)+ durations = [+ result.duration_ms+ for result in self._results+ if result.duration_ms is not None+ ]+ return latest.model_copy(+ update={+ "duration_ms": sum(durations) if durations else None,+ "usage": usage,+ }+ )++ @property+ def results(self) -> tuple[CodexInvocationResult, ...]:+ return tuple(self._results)++ async def start(self) -> None:+ await self.__aenter__()++ async def __aenter__(self) -> "CodexAnthropicGateway":+ if self._runner is not None:+ return self+ if self._agent_session is None:+ credential_lease = self._credential_lease+ if credential_lease is None:+ raise RuntimeError("Codex gateway credential lease is unavailable")+ self._agent_context = self._transport.agent_session(credential_lease)+ self._agent_session = await self._agent_context.__aenter__()+ try:+ application = web.Application(client_max_size=_MAX_REQUEST_BYTES)+ application.router.add_post("/v1/messages", self._handle_messages)+ application.router.add_post(+ "/v1/messages/count_tokens",+ self._handle_count_tokens,+ )+ application.router.add_get("/healthz", self._handle_health)+ runner = web.AppRunner(+ application,+ access_log=None,+ shutdown_timeout=2,+ )+ self._runner = runner+ await runner.setup()+ site = web.TCPSite(runner, "127.0.0.1", 0)+ self._site = site+ await site.start()+ server = cast(asyncio.Server | None, site._server)+ if server is None or not server.sockets:+ raise RuntimeError("Codex Anthropic gateway failed to bind")+ port = int(server.sockets[0].getsockname()[1])+ self._base_url = f"http://127.0.0.1:{port}"+ return self+ except BaseException:+ await self.close()+ raise++ async def __aexit__(self, _exc_type, _exc, _tb) -> None:+ await self.close()++ async def close(self) -> None:+ async with self._close_lock:+ if self._closed:+ return+ self._closed = True+ runner, self._runner = self._runner, None+ self._site = None+ self._base_url = None+ for record in tuple(self._tool_calls.values()):+ if not record.future.done():+ record.future.cancel()+ self._tool_calls.clear()+ for conversation in self._conversations.values():+ conversation.queue.put_nowait(+ _Failed(error=RuntimeError("Codex Anthropic gateway is closing"))+ )+ tasks = [+ conversation.task+ for conversation in self._conversations.values()+ if conversation.task is not None and not conversation.task.done()+ ]+ for task in tasks:+ task.cancel()+ if tasks:+ with suppress(BaseException):+ await asyncio.wait_for(+ asyncio.gather(*tasks, return_exceptions=True),+ timeout=5,+ )++ cleanup_error: BaseException | None = None+ if runner is not None:+ try:+ await asyncio.wait_for(runner.cleanup(), timeout=5)+ except BaseException as exc:+ cleanup_error = exc++ agent_context, self._agent_context = self._agent_context, None+ self._agent_session = None+ if agent_context is not None:+ try:+ await agent_context.__aexit__(None, None, None)+ except BaseException as exc:+ cleanup_error = exc+ if cleanup_error is not None:+ raise cleanup_error++ async def _handle_health(self, request: web.Request) -> web.Response:+ if not self._is_authorized(request):+ return _anthropic_error(401, "authentication_error", "Unauthorized")+ return web.json_response({"status": "ok"})++ async def _handle_count_tokens(self, request: web.Request) -> web.Response:+ if not self._is_authorized(request):+ return _anthropic_error(401, "authentication_error", "Unauthorized")+ try:+ payload = await request.json()+ except (json.JSONDecodeError, ValueError):+ return _anthropic_error(+ 400,+ "invalid_request_error",+ "Request body must be JSON",+ )+ return web.json_response({"input_tokens": _estimate_input_tokens(payload)})++ async def _handle_messages(self, request: web.Request) -> web.StreamResponse:+ if not self._is_authorized(request):+ return _anthropic_error(401, "authentication_error", "Unauthorized")+ try:+ payload = await request.json()+ except (json.JSONDecodeError, ValueError):+ return _anthropic_error(+ 400,+ "invalid_request_error",+ "Request body must be JSON",+ )+ if not isinstance(payload, dict):+ return _anthropic_error(+ 400,+ "invalid_request_error",+ "Request body must be an object",+ )++ try:+ conversation = self._continue_conversation(payload)+ if conversation is None:+ conversation = self._start_conversation(payload)+ except _DuplicateToolResultError as exc:+ return _anthropic_error(+ 409,+ "invalid_request_error",+ str(exc),+ )+ except (TypeError, ValueError) as exc:+ return _anthropic_error(+ 400,+ "invalid_request_error",+ str(exc),+ )++ input_tokens = _estimate_input_tokens(payload)+ if payload.get("stream") is True:+ return await self._streaming_response(+ request,+ conversation,+ input_tokens,+ )+ return await self._nonstreaming_response(conversation, input_tokens)++ def _continue_conversation(+ self,+ payload: dict[str, object],+ ) -> _Conversation | None:+ tool_results = _extract_tool_results(payload.get("messages"))+ known = [+ (self._tool_calls[call_id], result)+ for call_id, result in tool_results.items()+ if call_id in self._tool_calls+ ]+ if not known:+ return None++ for record, result in known:+ if record.result is not None and record.result != result:+ raise _DuplicateToolResultError(+ f"Conflicting result for tool_use_id {record.gateway_call_id!r}"+ )++ claimable = [+ (record, result)+ for record, result in known+ if not record.closed and record.result is None+ ]+ fingerprint = _tool_result_request_fingerprint(payload)+ if not claimable:+ if any(record.claim_fingerprint == fingerprint for record, _ in known):+ raise _DuplicateToolResultError(+ "This tool-result request was already accepted"+ )+ if any(record.result is None for record, _ in known):+ raise _DuplicateToolResultError(+ "This tool-result request refers to a closed model call"+ )+ raise _DuplicateToolResultError(+ "This tool-result request refers to a completed model call"+ )++ conversation = claimable[0][0].conversation+ if any(record.conversation is not conversation for record, _ in claimable):+ raise ValueError("Tool results span multiple model conversations")++ for record, result in claimable:+ record.result = result+ record.claim_fingerprint = fingerprint+ for record, result in claimable:+ if not record.future.done():+ record.future.set_result(result)+ return conversation++ def _start_conversation(self, payload: dict[str, object]) -> _Conversation:+ agent_session = self._agent_session+ if agent_session is None:+ raise RuntimeError("Codex Anthropic gateway is not running")+ tools, original_names = _parse_tools(payload.get("tools"))+ conversation = _Conversation(id=uuid4().hex)+ self._conversations[conversation.id] = conversation+ invocation = CodexInvocationRequest(+ prompt=_serialize_messages(payload.get("messages")),+ instructions=_serialize_system(payload.get("system")),+ model=self.model,+ effort=self.effort,+ )+ conversation.task = asyncio.create_task(+ self._run_conversation(+ conversation,+ agent_session,+ invocation,+ tools,+ original_names,+ )+ )+ return conversation++ async def _run_conversation(+ self,+ conversation: _Conversation,+ agent_session: _AgentSession,+ invocation: CodexInvocationRequest,+ tools: list[CodexDynamicToolSpec],+ original_names: dict[str, str],+ ) -> None:+ async def handle_event(notification: Notification) -> None:+ payload = notification.payload+ if isinstance(payload, AgentMessageDeltaNotification) and payload.delta:+ await conversation.queue.put(_TextDelta(text=payload.delta))++ async def handle_tool(call: CodexDynamicToolCall) -> CodexDynamicToolResult:+ future: asyncio.Future[CodexDynamicToolResult] = (+ asyncio.get_running_loop().create_future()+ )+ gateway_call_id = f"toolu_codex_{uuid4().hex}"+ record = _ToolCallRecord(+ gateway_call_id=gateway_call_id,+ raw_call_id=call.call_id,+ conversation=conversation,+ future=future,+ )+ conversation.pending[gateway_call_id] = record+ self._tool_calls[gateway_call_id] = record+ await conversation.queue.put(+ _ToolUse(+ call_id=gateway_call_id,+ name=original_names.get(call.tool, call.tool),+ arguments=call.arguments,+ )+ )+ try:+ return await future+ finally:+ record.closed = True+ conversation.pending.pop(gateway_call_id, None)++ try:+ result = await agent_session.invoke(+ invocation,+ tools,+ handle_tool,+ handle_event,+ )+ conversation.result = result+ self._results.append(result)+ await conversation.queue.put(_Completed(result=result))+ except asyncio.CancelledError:+ raise+ except BaseException as exc:+ logger.error(+ "Codex gateway conversation failed: exception_type=%s error=%s",+ type(exc).__name__,+ _safe_exception_message(exc, secrets_to_redact=(self._auth_token,)),+ )+ await conversation.queue.put(_Failed(error=exc))++ async def _streaming_response(+ self,+ request: web.Request,+ conversation: _Conversation,+ input_tokens: int,+ ) -> web.StreamResponse:+ async with conversation.response_lock:+ return await self._streaming_response_locked(+ request,+ conversation,+ input_tokens,+ )++ async def _streaming_response_locked(+ self,+ request: web.Request,+ conversation: _Conversation,+ input_tokens: int,+ ) -> web.StreamResponse:+ first = await conversation.queue.get()+ if isinstance(first, _Failed):+ return _anthropic_error(+ 502,+ "api_error",+ "Codex model transport failed",+ )++ response = web.StreamResponse(+ status=200,+ headers={+ "Content-Type": "text/event-stream",+ "Cache-Control": "no-cache",+ "Connection": "keep-alive",+ },+ )+ await response.prepare(request)+ message_id = f"msg_codex_{uuid4().hex}"+ try:+ await _write_sse(+ response,+ {+ "type": "message_start",+ "message": {+ "id": message_id,+ "type": "message",+ "role": "assistant",+ "content": [],+ "model": self.model,+ "stop_reason": None,+ "stop_sequence": None,+ "usage": {+ "input_tokens": input_tokens,+ "output_tokens": 0,+ },+ },+ },+ )+ await self._write_boundary(response, conversation, first)+ await response.write_eof()+ except (ConnectionError, ConnectionResetError, asyncio.CancelledError):+ if conversation.task is not None:+ conversation.task.cancel()+ raise+ return response++ async def _write_boundary(+ self,+ response: web.StreamResponse,+ conversation: _Conversation,+ first: _ConversationEvent,+ ) -> None:+ index = 0+ text_open = False+ emitted_text = False+ event = first+ while True:+ if isinstance(event, _TextDelta):+ emitted_text = True+ if not text_open:+ await _write_sse(+ response,+ {+ "type": "content_block_start",+ "index": index,+ "content_block": {"type": "text", "text": ""},+ },+ )+ text_open = True+ await _write_sse(+ response,+ {+ "type": "content_block_delta",+ "index": index,+ "delta": {"type": "text_delta", "text": event.text},+ },+ )+ elif isinstance(event, _ToolUse):+ if text_open:+ await _write_sse(+ response,+ {"type": "content_block_stop", "index": index},+ )+ index += 1+ text_open = False+ await _write_tool_use(response, index, event)+ await _write_message_end(response, "tool_use", 0)+ return+ elif isinstance(event, _Completed):+ if not emitted_text and event.result.final_response:+ if not text_open:+ await _write_sse(+ response,+ {+ "type": "content_block_start",+ "index": index,+ "content_block": {"type": "text", "text": ""},+ },+ )+ text_open = True+ await _write_sse(+ response,+ {+ "type": "content_block_delta",+ "index": index,+ "delta": {+ "type": "text_delta",+ "text": event.result.final_response,+ },+ },+ )+ if text_open:+ await _write_sse(+ response,+ {"type": "content_block_stop", "index": index},+ )+ output_tokens = (+ event.result.usage.output_tokens if event.result.usage else 0+ )+ await _write_message_end(response, "end_turn", output_tokens)+ return+ else:+ await _write_sse(+ response,+ {+ "type": "error",+ "error": {+ "type": "api_error",+ "message": "Codex model transport failed",+ },+ },+ )+ return+ event = await conversation.queue.get()++ async def _nonstreaming_response(+ self,+ conversation: _Conversation,+ input_tokens: int,+ ) -> web.Response:+ async with conversation.response_lock:+ return await self._nonstreaming_response_locked(+ conversation,+ input_tokens,+ )++ async def _nonstreaming_response_locked(+ self,+ conversation: _Conversation,+ input_tokens: int,+ ) -> web.Response:+ content: list[dict[str, object]] = []+ text_parts: list[str] = []+ output_tokens = 0+ stop_reason = "end_turn"+ while True:+ event = await conversation.queue.get()+ if isinstance(event, _TextDelta):+ text_parts.append(event.text)+ continue+ if text_parts:+ content.append({"type": "text", "text": "".join(text_parts)})+ text_parts = []+ if isinstance(event, _ToolUse):+ content.append(+ {+ "type": "tool_use",+ "id": event.call_id,+ "name": event.name,+ "input": event.arguments,+ }+ )+ stop_reason = "tool_use"+ break+ if isinstance(event, _Completed):+ if not content and event.result.final_response:+ content.append(+ {"type": "text", "text": event.result.final_response}+ )+ output_tokens = (+ event.result.usage.output_tokens if event.result.usage else 0+ )+ break+ return _anthropic_error(+ 502,+ "api_error",+ "Codex model transport failed",+ )+ return web.json_response(+ {+ "id": f"msg_codex_{uuid4().hex}",+ "type": "message",+ "role": "assistant",+ "content": content,+ "model": self.model,+ "stop_reason": stop_reason,+ "stop_sequence": None,+ "usage": {+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ },+ }+ )++ def _is_authorized(self, request: web.Request) -> bool:+ authorization = request.headers.get("Authorization", "")+ bearer = authorization[7:] if authorization.startswith("Bearer ") else ""+ api_key = request.headers.get("x-api-key", "")+ return secrets.compare_digest(+ bearer, self._auth_token+ ) or secrets.compare_digest(+ api_key,+ self._auth_token,+ )+++def _parse_tools(+ value: object,+) -> tuple[list[CodexDynamicToolSpec], dict[str, str]]:+ if value is None:+ return [], {}+ if not isinstance(value, list):+ raise TypeError("tools must be an array")+ tools: list[CodexDynamicToolSpec] = []+ names: dict[str, str] = {}+ used: set[str] = set()+ for raw_tool in value:+ if not isinstance(raw_tool, dict) or not isinstance(raw_tool.get("name"), str):+ continue+ original = raw_tool["name"]+ safe = _safe_tool_name(original, used)+ used.add(safe)+ schema = raw_tool.get("input_schema", {"type": "object"})+ if not isinstance(schema, dict):+ raise TypeError(f"Tool {original!r} input_schema must be an object")+ tools.append(+ CodexDynamicToolSpec(+ name=safe,+ description=str(raw_tool.get("description") or ""),+ input_schema=cast(dict[str, object], schema),+ )+ )+ names[safe] = original+ return tools, names+++def _safe_tool_name(original: str, used: set[str]) -> str:+ if original.startswith("mcp__"):+ normalized = re.sub(r"[^a-zA-Z0-9_-]", "_", original).strip("_") or "tool"+ digest = hashlib.sha256(original.encode("utf-8")).hexdigest()[:10]+ candidate = f"tool_{normalized[:112]}_{digest}"+ counter = 1+ while candidate in used:+ suffix = f"_{counter}"+ candidate = f"tool_{normalized[: 112 - len(suffix)]}_{digest}{suffix}"+ counter += 1+ return candidate+ if _TOOL_NAME_PATTERN.fullmatch(original) and original not in used:+ return original+ normalized = re.sub(r"[^a-zA-Z0-9_-]", "_", original).strip("_") or "tool"+ digest = hashlib.sha256(original.encode("utf-8")).hexdigest()[:10]+ candidate = f"{normalized[:116]}_{digest}"+ counter = 1+ while candidate in used:+ suffix = f"_{counter}"+ candidate = f"{normalized[: 116 - len(suffix)]}_{digest}{suffix}"+ counter += 1+ return candidate+++def _serialize_system(value: object) -> str | None:+ text = _content_text(value)+ return text or None+++def _serialize_messages(value: object) -> str:+ if not isinstance(value, list):+ raise TypeError("messages must be an array")+ normalized: list[dict[str, object]] = []+ for message in value:+ if not isinstance(message, dict):+ raise TypeError("Each message must be an object")+ role = message.get("role")+ if role not in {"user", "assistant"}:+ raise ValueError("Message roles must be user or assistant")+ normalized.append(+ {+ "role": role,+ "content": _normalize_content(message.get("content")),+ }+ )+ return (+ "Continue the following conversation as the assistant. Use the supplied "+ "tools when needed. The transcript is JSON:\n"+ + json.dumps(normalized, ensure_ascii=False, separators=(",", ":"))+ )+++def _normalize_content(value: object) -> object:+ if isinstance(value, str):+ return value+ if not isinstance(value, list):+ return ""+ normalized: list[object] = []+ for block in value:+ if not isinstance(block, dict):+ continue+ block_type = block.get("type")+ if block_type == "thinking":+ continue+ if block_type == "image":+ normalized.append({"type": "text", "text": "[image input omitted]"})+ continue+ normalized.append(+ {+ key: item+ for key, item in block.items()+ if key not in {"cache_control", "signature"}+ }+ )+ return normalized+++def _content_text(value: object) -> str:+ if isinstance(value, str):+ return value+ if not isinstance(value, list):+ return ""+ return "\n".join(+ str(block.get("text"))+ for block in value+ if isinstance(block, dict) and block.get("type") == "text" and block.get("text")+ )+++def _extract_tool_results(value: object) -> dict[str, CodexDynamicToolResult]:+ results: dict[str, CodexDynamicToolResult] = {}+ if not isinstance(value, list):+ return results+ for message in value:+ if not isinstance(message, dict):+ continue+ content = message.get("content")+ if not isinstance(content, list):+ continue+ for block in content:+ if not isinstance(block, dict) or block.get("type") != "tool_result":+ continue+ call_id = block.get("tool_use_id")+ if not isinstance(call_id, str):+ continue+ results[call_id] = CodexDynamicToolResult(+ content=_content_text(block.get("content"))+ or json.dumps(block.get("content"), ensure_ascii=False, default=str),+ success=not bool(block.get("is_error")),+ )+ return results+++def _tool_result_request_fingerprint(payload: dict[str, object]) -> str:+ serialized = json.dumps(+ payload.get("messages"),+ ensure_ascii=False,+ separators=(",", ":"),+ sort_keys=True,+ default=str,+ )+ return hashlib.sha256(serialized.encode("utf-8")).hexdigest()+++def _estimate_input_tokens(payload: object) -> int:+ serialized = json.dumps(+ payload,+ ensure_ascii=False,+ separators=(",", ":"),+ sort_keys=True,+ default=str,+ )+ return max(1, (len(serialized) + 3) // 4)+++def _safe_exception_message(+ exc: BaseException,+ *,+ secrets_to_redact: Iterable[str] = (),+) -> str:+ try:+ message = str(exc)+ except BaseException:+ message = "<unprintable exception message>"+ for secret in sorted(+ (value for value in secrets_to_redact if value),+ key=len,+ reverse=True,+ ):+ message = message.replace(secret, _REDACTED)+ message = _PROVIDER_STATE_PATTERN.sub(f"provider_state={_REDACTED}", message)+ message = _BEARER_PATTERN.sub(f"Bearer {_REDACTED}", message)+ message = _SECRET_ASSIGNMENT_PATTERN.sub(+ lambda match: f"{match.group('label')}={_REDACTED}",+ message,+ )+ message = _JWT_PATTERN.sub(_REDACTED, message)+ message = _DEVICE_CODE_PATTERN.sub(_REDACTED, message)+ message = " ".join(message.split()) or "<empty exception message>"+ if len(message) > _MAX_LOGGED_ERROR_MESSAGE_CHARS:+ return message[: _MAX_LOGGED_ERROR_MESSAGE_CHARS - 3] + "..."+ return message+++async def _write_sse(response: web.StreamResponse, event: dict[str, object]) -> None:+ event_type = str(event["type"])+ payload = json.dumps(event, ensure_ascii=False, separators=(",", ":"))+ await response.write(f"event: {event_type}\ndata: {payload}\n\n".encode())+++async def _write_tool_use(+ response: web.StreamResponse,+ index: int,+ event: _ToolUse,+) -> None:+ await _write_sse(+ response,+ {+ "type": "content_block_start",+ "index": index,+ "content_block": {+ "type": "tool_use",+ "id": event.call_id,+ "name": event.name,+ "input": {},+ },+ },+ )+ await _write_sse(+ response,+ {+ "type": "content_block_delta",+ "index": index,+ "delta": {+ "type": "input_json_delta",+ "partial_json": json.dumps(event.arguments, ensure_ascii=False),+ },+ },+ )+ await _write_sse(+ response,+ {"type": "content_block_stop", "index": index},+ )+++async def _write_message_end(+ response: web.StreamResponse,+ stop_reason: str,+ output_tokens: int,+) -> None:+ await _write_sse(+ response,+ {+ "type": "message_delta",+ "delta": {"stop_reason": stop_reason, "stop_sequence": None},+ "usage": {"output_tokens": output_tokens},+ },+ )+ await _write_sse(response, {"type": "message_stop"})+++def _anthropic_error(status: int, error_type: str, message: str) -> web.Response:+ return web.json_response(+ {+ "type": "error",+ "error": {"type": error_type, "message": message},+ },+ status=status,+ )+++def _sum_usage(+ usages: Iterable[CodexTokenUsage | None],+) -> CodexTokenUsage | None:+ values = [usage for usage in usages if usage is not None]+ if not values:+ return None+ return CodexTokenUsage(+ input_tokens=sum(value.input_tokens for value in values),+ cached_input_tokens=sum(value.cached_input_tokens for value in values),+ output_tokens=sum(value.output_tokens for value in values),+ reasoning_output_tokens=sum(value.reasoning_output_tokens for value in values),+ total_tokens=sum(value.total_tokens for value in values),+ )diff --git a/autogpt_platform/backend/backend/copilot/sdk/codex_compat_gateway_test.py b/autogpt_platform/backend/backend/copilot/sdk/codex_compat_gateway_test.pynew file mode 100644index 0000000000..f0e1c81998--- /dev/null+++ b/autogpt_platform/backend/backend/copilot/sdk/codex_compat_gateway_test.py@@ -0,0 +1,824 @@+import asyncio+import json+import logging+from collections.abc import AsyncIterator+from contextlib import asynccontextmanager+from typing import cast++import pytest+from aiohttp import ClientSession+from openai_codex.generated.v2_all import AgentMessageDeltaNotification+from openai_codex.models import Notification++from backend.copilot.sdk.codex_compat_gateway import (+ CodexAnthropicGateway,+ _safe_tool_name,+)+from backend.integrations.codex.models import (+ CodexDynamicToolCall,+ CodexDynamicToolResult,+ CodexDynamicToolSpec,+ CodexInvocationRequest,+ CodexInvocationResult,+ CodexTokenUsage,+)+from backend.integrations.credential_lease import CredentialLease+++class _FakeAgentSession:+ def __init__(self, *, use_tool: bool = False) -> None:+ self.use_tool = use_tool+ self.requests: list[CodexInvocationRequest] = []+ self.tools: list[list[CodexDynamicToolSpec]] = []+ self.tool_result: CodexDynamicToolResult | None = None+ self.cancelled = asyncio.Event()++ async def invoke(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ ) -> CodexInvocationResult:+ self.requests.append(request)+ self.tools.append(dynamic_tools)+ try:+ if self.use_tool:+ self.tool_result = await tool_handler(+ CodexDynamicToolCall(+ thread_id="thread-1",+ turn_id="turn-1",+ call_id="call-1",+ tool=dynamic_tools[0].name,+ arguments={"query": "status"},+ )+ )+ if event_handler is not None:+ await event_handler(+ Notification(+ method="item/agentMessage/delta",+ payload=AgentMessageDeltaNotification(+ delta="done" if self.use_tool else "hello",+ itemId="item-1",+ threadId="thread-1",+ turnId="turn-1",+ ),+ )+ )+ return _result("done" if self.use_tool else "hello")+ except asyncio.CancelledError:+ self.cancelled.set()+ raise+++class _FailingAgentSession(_FakeAgentSession):+ def __init__(self, error: BaseException) -> None:+ super().__init__()+ self.error = error++ async def invoke(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ ) -> CodexInvocationResult:+ raise self.error+++class _FinalWithoutDeltaSession:+ async def invoke(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ ) -> CodexInvocationResult:+ assert event_handler is not None+ await event_handler(+ Notification(+ method="item/agentMessage/delta",+ payload=AgentMessageDeltaNotification(+ delta="before tool",+ itemId="item-before",+ threadId="thread-before",+ turnId="turn-before",+ ),+ )+ )+ await tool_handler(+ CodexDynamicToolCall(+ thread_id="thread-before",+ turn_id="turn-before",+ call_id="raw-before",+ tool=dynamic_tools[0].name,+ arguments={},+ )+ )+ return _result("after tool")+++class _CollidingCallSession:+ def __init__(self) -> None:+ self.results: dict[int, CodexDynamicToolResult] = {}+ self.raw_call_ids: list[str] = []++ async def invoke(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ ) -> CodexInvocationResult:+ invocation_index = len(self.raw_call_ids)+ self.raw_call_ids.append("raw-collision")+ result = await tool_handler(+ CodexDynamicToolCall(+ thread_id=f"thread-{invocation_index}",+ turn_id=f"turn-{invocation_index}",+ call_id="raw-collision",+ tool=dynamic_tools[0].name,+ arguments={"invocation": invocation_index},+ )+ )+ self.results[invocation_index] = result+ assert event_handler is not None+ await event_handler(+ Notification(+ method="item/agentMessage/delta",+ payload=AgentMessageDeltaNotification(+ delta=f"completed:{result.content}",+ itemId=f"item-{invocation_index}",+ threadId=f"thread-{invocation_index}",+ turnId=f"turn-{invocation_index}",+ ),+ )+ )+ return _result(f"completed:{result.content}")+++class _FakeTransport:+ def __init__(self, session: _FakeAgentSession) -> None:+ self.session = session+ self.entered = False+ self.exited = False++ @asynccontextmanager+ async def agent_session(+ self,+ _lease: CredentialLease,+ ) -> AsyncIterator[_FakeAgentSession]:+ self.entered = True+ try:+ yield self.session+ finally:+ self.exited = True+++def _result(text: str) -> CodexInvocationResult:+ return CodexInvocationResult(+ response_id="response-1",+ final_response=text,+ status="completed",+ usage=CodexTokenUsage(+ input_tokens=10,+ cached_input_tokens=2,+ output_tokens=3,+ reasoning_output_tokens=1,+ total_tokens=13,+ ),+ )+++def _lease() -> CredentialLease:+ return cast(CredentialLease, object())+++def _headers(gateway: CodexAnthropicGateway) -> dict[str, str]:+ return {"Authorization": f"Bearer {gateway.auth_token}"}+++async def _post_simple_message(gateway: CodexAnthropicGateway) -> int:+ async with ClientSession() as client:+ response = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json={"messages": [{"role": "user", "content": "hello"}]},+ )+ await response.read()+ return response.status+++def _events(body: str) -> list[dict[str, object]]:+ return [+ json.loads(line.removeprefix("data: "))+ for line in body.splitlines()+ if line.startswith("data: ")+ ]+++def _tool_request(message: str, *, stream: bool | None = None) -> dict[str, object]:+ payload: dict[str, object] = {+ "messages": [{"role": "user", "content": message}],+ "tools": [+ {+ "name": "mcp.tool/status",+ "description": "Read status",+ "input_schema": {"type": "object"},+ }+ ],+ }+ if stream is not None:+ payload["stream"] = stream+ return payload+++def _tool_result_request(+ gateway_call_id: str,+ content: str,+ *,+ stream: bool | None = None,+) -> dict[str, object]:+ payload: dict[str, object] = {+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": gateway_call_id,+ "content": content,+ }+ ],+ }+ ]+ }+ if stream is not None:+ payload["stream"] = stream+ return payload+++def _tool_use_id(payload: dict[str, object]) -> str:+ content = payload["content"]+ assert isinstance(content, list)+ block = next(+ item+ for item in content+ if isinstance(item, dict) and item.get("type") == "tool_use"+ )+ call_id = block["id"]+ assert isinstance(call_id, str)+ return call_id+++def test_reserved_mcp_tool_name_is_deterministically_remapped() -> None:+ original = "mcp__copilot__add_understanding"++ first = _safe_tool_name(original, set())+ second = _safe_tool_name(original, set())++ assert first == second+ assert first != original+ assert not first.startswith("mcp__")+ assert len(first) <= 128+++def test_reserved_mcp_tool_name_collision_is_remapped_uniquely() -> None:+ original = "mcp__copilot__add_understanding"+ first = _safe_tool_name(original, set())++ collided = _safe_tool_name(original, {first})++ assert collided != first+ assert collided == _safe_tool_name(original, {first})+ assert not collided.startswith("mcp__")+ assert len(collided) <= 128+++def test_reserved_mcp_tool_name_remap_respects_max_length() -> None:+ remapped = _safe_tool_name("mcp__" + "x" * 300, set())++ assert len(remapped) == 128+ assert not remapped.startswith("mcp__")+++def test_non_reserved_valid_tool_name_is_unchanged() -> None:+ assert _safe_tool_name("regular_tool", set()) == "regular_tool"+++def test_non_reserved_tool_name_collision_preserves_digest() -> None:+ original = "regular tool"+ first = _safe_tool_name(original, set())++ collided = _safe_tool_name(original, {first})++ digest = first.rsplit("_", 1)[-1]+ assert collided.endswith(f"_{digest}_1")+ assert collided != first+ assert len(collided) <= 128+++async def test_streams_codex_text_as_anthropic_events() -> None:+ session = _FakeAgentSession()+ transport = _FakeTransport(session)+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-terra",+ effort="high",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ response = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json={+ "model": "ignored-by-transport",+ "stream": True,+ "system": "follow the system",+ "messages": [{"role": "user", "content": "hello"}],+ },+ )+ assert response.status == 200+ events = _events(await response.text())++ assert [event["type"] for event in events] == [+ "message_start",+ "content_block_start",+ "content_block_delta",+ "content_block_stop",+ "message_delta",+ "message_stop",+ ]+ assert events[2]["delta"] == {"type": "text_delta", "text": "hello"}+ assert events[0]["message"]["usage"]["input_tokens"] > 0+ assert session.requests[0].model == "gpt-5.6-terra"+ assert session.requests[0].effort == "high"+ assert session.requests[0].instructions == "follow the system"+ assert '"content":"hello"' in session.requests[0].prompt+ assert gateway.result is not None+ assert gateway.result.usage is not None+ assert gateway.result.usage.input_tokens == 10++ assert transport.entered+ assert transport.exited+++async def test_transport_failure_logs_useful_bounded_context(+ caplog: pytest.LogCaptureFixture,+) -> None:+ session = _FailingAgentSession(+ RuntimeError("app-server overloaded while starting turn " + "x" * 500)+ )+ transport = _FakeTransport(session)+ with caplog.at_level(+ logging.ERROR,+ logger="backend.copilot.sdk.codex_compat_gateway",+ ):+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-terra",+ transport=transport,+ ) as gateway:+ status = await _post_simple_message(gateway)++ messages = [+ record.getMessage()+ for record in caplog.records+ if record.name == "backend.copilot.sdk.codex_compat_gateway"+ ]+ assert status == 502+ assert len(messages) == 1+ assert "exception_type=RuntimeError" in messages[0]+ assert "app-server overloaded while starting turn" in messages[0]+ assert len(messages[0]) < 340+ assert messages[0].endswith("...")+++async def test_transport_failure_logs_redact_all_auth_shapes(+ caplog: pytest.LogCaptureFixture,+) -> None:+ jwt = "eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJ1c2VyIn0.signaturevalue"+ bearer = "bearer-top-secret-123"+ access_token = "access-top-secret-456"+ refresh_token = "refresh-top-secret-789"+ provider_secret = "provider-top-secret-012"+ device_code = "ABCD-EFGH"+ session = _FailingAgentSession(RuntimeError("not configured"))+ transport = _FakeTransport(session)+ with caplog.at_level(+ logging.ERROR,+ logger="backend.copilot.sdk.codex_compat_gateway",+ ):+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-terra",+ transport=transport,+ ) as gateway:+ capability = gateway.auth_token+ session.error = RuntimeError(+ f"upstream auth failed jwt={jwt} Authorization: Bearer {bearer} "+ f"access_token={access_token} "+ f'"refresh_token":"{refresh_token}" '+ f"device verification {device_code} capability={capability} "+ f'provider_state={{"tokens":{{"access_token":"{provider_secret}"}}}}'+ )+ status = await _post_simple_message(gateway)++ message = "\n".join(+ record.getMessage()+ for record in caplog.records+ if record.name == "backend.copilot.sdk.codex_compat_gateway"+ )+ assert status == 502+ assert "exception_type=RuntimeError" in message+ assert "upstream auth failed" in message+ assert "[REDACTED]" in message+ for secret in (+ jwt,+ bearer,+ access_token,+ refresh_token,+ provider_secret,+ device_code,+ capability,+ ):+ assert secret not in message+++async def test_round_trips_tool_use_through_claude_harness() -> None:+ original_tool_name = "mcp__copilot__add_understanding"+ session = _FakeAgentSession(use_tool=True)+ transport = _FakeTransport(session)+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-sol",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ first = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json={+ "stream": True,+ "messages": [{"role": "user", "content": "check"}],+ "tools": [+ {+ "name": original_tool_name,+ "description": "Read status",+ "input_schema": {+ "type": "object",+ "properties": {"query": {"type": "string"}},+ },+ }+ ],+ },+ )+ first_events = _events(await first.text())+ tool_start = next(+ event+ for event in first_events+ if event["type"] == "content_block_start"+ )+ content_block = tool_start["content_block"]+ assert content_block["type"] == "tool_use"+ assert content_block["name"] == original_tool_name+ assert content_block["input"] == {}+ gateway_call_id = content_block["id"]+ assert isinstance(gateway_call_id, str)+ assert gateway_call_id.startswith("toolu_codex_")+ assert gateway_call_id != "call-1"+ assert first_events[-2]["delta"] == {+ "stop_reason": "tool_use",+ "stop_sequence": None,+ }++ second = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json={+ "stream": True,+ "messages": [+ {"role": "user", "content": "check"},+ {+ "role": "assistant",+ "content": [+ {+ "type": "tool_use",+ "id": gateway_call_id,+ "name": original_tool_name,+ "input": {"query": "status"},+ }+ ],+ },+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": gateway_call_id,+ "content": "all green",+ }+ ],+ },+ ],+ },+ )+ second_events = _events(await second.text())++ assert any(+ event.get("delta") == {"type": "text_delta", "text": "done"}+ for event in second_events+ )+ assert second_events[-2]["delta"] == {+ "stop_reason": "end_turn",+ "stop_sequence": None,+ }+ assert session.tool_result == CodexDynamicToolResult(+ content="all green",+ success=True,+ )+ assert session.tools[0][0].name != original_tool_name+ assert not session.tools[0][0].name.startswith("mcp__")+++async def test_messages_default_to_nonstreaming_and_report_estimated_input() -> None:+ transport = _FakeTransport(_FakeAgentSession())+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-luna",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ response = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json={"messages": [{"role": "user", "content": "hello"}]},+ )+ payload = await response.json()+ count_response = await client.post(+ f"{gateway.base_url}/v1/messages/count_tokens",+ headers=_headers(gateway),+ json={"messages": [{"role": "user", "content": "hello"}]},+ )+ count_payload = await count_response.json()++ assert response.status == 200+ assert payload["type"] == "message"+ assert payload["content"] == [{"type": "text", "text": "hello"}]+ assert payload["usage"]["input_tokens"] > 0+ assert payload["usage"]["input_tokens"] == count_payload["input_tokens"]+++async def test_final_response_fallback_is_scoped_to_each_tool_boundary() -> None:+ transport = _FakeTransport(_FinalWithoutDeltaSession())+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-terra",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ first = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_request("run", stream=True),+ )+ first_events = _events(await first.text())+ tool_start = next(+ event+ for event in first_events+ if event["type"] == "content_block_start"+ and event["content_block"]["type"] == "tool_use"+ )+ gateway_call_id = tool_start["content_block"]["id"]+ assert isinstance(gateway_call_id, str)++ second = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_result_request(+ gateway_call_id,+ "ok",+ stream=True,+ ),+ )+ second_events = _events(await second.text())++ assert any(+ event.get("delta") == {"type": "text_delta", "text": "before tool"}+ for event in first_events+ )+ assert any(+ event.get("delta") == {"type": "text_delta", "text": "after tool"}+ for event in second_events+ )+++async def test_raw_codex_call_id_collisions_are_isolated_by_gateway_ids() -> None:+ agent_session = _CollidingCallSession()+ transport = _FakeTransport(agent_session)+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-sol",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ first_response, second_response = await asyncio.gather(+ client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_request("first"),+ ),+ client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_request("second"),+ ),+ )+ first_payload, second_payload = await asyncio.gather(+ first_response.json(),+ second_response.json(),+ )+ first_id = _tool_use_id(first_payload)+ second_id = _tool_use_id(second_payload)++ cross_conversation = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json={+ "messages": [+ {+ "role": "user",+ "content": [+ {+ "type": "tool_result",+ "tool_use_id": first_id,+ "content": "wrong-first",+ },+ {+ "type": "tool_result",+ "tool_use_id": second_id,+ "content": "wrong-second",+ },+ ],+ }+ ]+ },+ )++ first_result, second_result = await asyncio.gather(+ client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_result_request(first_id, "first-result"),+ ),+ client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_result_request(second_id, "second-result"),+ ),+ )+ first_completed, second_completed = await asyncio.gather(+ first_result.json(),+ second_result.json(),+ )++ assert first_id != second_id+ assert cross_conversation.status == 400+ assert first_id.startswith("toolu_codex_")+ assert second_id.startswith("toolu_codex_")+ assert agent_session.raw_call_ids == ["raw-collision", "raw-collision"]+ assert {result.content for result in agent_session.results.values()} == {+ "first-result",+ "second-result",+ }+ completed_text = {+ block["text"]+ for payload in (first_completed, second_completed)+ for block in payload["content"]+ if block["type"] == "text"+ }+ assert completed_text == {+ "completed:first-result",+ "completed:second-result",+ }+++async def test_duplicate_tool_result_request_is_claimed_once_without_new_turn() -> None:+ agent_session = _FakeAgentSession(use_tool=True)+ transport = _FakeTransport(agent_session)+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-terra",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ first = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_request("run"),+ )+ gateway_call_id = _tool_use_id(await first.json())+ continuation = _tool_result_request(gateway_call_id, "one result")++ responses = await asyncio.gather(+ client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=continuation,+ ),+ client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=continuation,+ ),+ )+ payloads = await asyncio.gather(+ *(response.json() for response in responses)+ )++ assert sorted(response.status for response in responses) == [200, 409]+ duplicate_payload = next(+ payload+ for response, payload in zip(responses, payloads, strict=True)+ if response.status == 409+ )+ assert duplicate_payload["error"]["message"] == (+ "This tool-result request was already accepted"+ )+ assert len(agent_session.requests) == 1+ assert agent_session.tool_result == CodexDynamicToolResult(+ content="one result",+ success=True,+ )+++async def test_completed_tool_result_cannot_start_a_new_conversation() -> None:+ agent_session = _FakeAgentSession(use_tool=True)+ transport = _FakeTransport(agent_session)+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-terra",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ first = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=_tool_request("run"),+ )+ gateway_call_id = _tool_use_id(await first.json())+ continuation = _tool_result_request(gateway_call_id, "one result")+ completed = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=continuation,+ )+ assert completed.status == 200+ await completed.read()++ replay = _tool_result_request(gateway_call_id, "one result")+ replay_messages = replay["messages"]+ assert isinstance(replay_messages, list)+ replay_messages.insert(+ 0,+ {"role": "user", "content": "different request fingerprint"},+ )+ duplicate = await client.post(+ f"{gateway.base_url}/v1/messages",+ headers=_headers(gateway),+ json=replay,+ )+ duplicate_payload = await duplicate.json()++ assert duplicate.status == 409+ assert duplicate_payload["error"]["message"] == (+ "This tool-result request refers to a completed model call"+ )+ assert len(agent_session.requests) == 1+++async def test_rejects_missing_gateway_capability() -> None:+ transport = _FakeTransport(_FakeAgentSession())+ async with CodexAnthropicGateway(+ credential_lease=_lease(),+ model="gpt-5.6-luna",+ transport=transport,+ ) as gateway:+ async with ClientSession() as client:+ response = await client.post(+ f"{gateway.base_url}/v1/messages",+ json={"messages": [{"role": "user", "content": "hello"}]},+ )+ payload = await response.json()+ assert response.status == 401+ assert payload["error"]["type"] == "authentication_error"diff --git a/autogpt_platform/backend/backend/copilot/sdk/env.py b/autogpt_platform/backend/backend/copilot/sdk/env.pyindex 815dd46706..d8463b5350 100644--- a/autogpt_platform/backend/backend/copilot/sdk/env.py+++ b/autogpt_platform/backend/backend/copilot/sdk/env.py@@ -8,7 +8,9 @@ circular import through ``executor`` → ``credit`` → ``block_cost_config``).from __future__ import annotations+import osimport re+from urllib.parse import urlparsefrom backend.copilot.config import ChatConfigfrom backend.copilot.moonshot import is_moonshot_model@@ -22,6 +24,20 @@ config = ChatConfig()# RFC 7230 §3.2.6 — keep only printable ASCII; strip control chars and non-ASCII._HEADER_SAFE_RE = re.compile(r"[^\x20-\x7e]")_MAX_HEADER_VALUE_LEN = 128+_LOOPBACK_NO_PROXY_HOSTS = ("127.0.0.1", "localhost", "::1")+++def _loopback_no_proxy_value() -> str:+ entries: list[str] = []+ for value in (os.environ.get("NO_PROXY", ""), os.environ.get("no_proxy", "")):+ for entry in value.split(","):+ entry = entry.strip()+ if entry and entry.lower() not in {item.lower() for item in entries}:+ entries.append(entry)+ for host in _LOOPBACK_NO_PROXY_HOSTS:+ if host.lower() not in {item.lower() for item in entries}:+ entries.append(host)+ return ",".join(entries)def build_sdk_env(@@ -29,14 +45,17 @@ def build_sdk_env(user_id: str | None = None,sdk_cwd: str | None = None,model: str | None = None,+ codex_gateway_url: str | None = None,+ codex_gateway_token: str | None = None,) -> dict[str, str]:"""Build env vars for the SDK CLI subprocess.- Three modes (checked in order):- 1. **Subscription** — clears all keys; CLI uses ``claude login`` auth.- 2. **Direct Anthropic** — subprocess inherits ``ANTHROPIC_API_KEY``+ Four modes (checked in order):+ 1. **Codex gateway** — request-scoped loopback Anthropic compatibility.+ 2. **Subscription** — clears all keys; CLI uses ``claude login`` auth.+ 3. **Direct Anthropic** — subprocess inherits ``ANTHROPIC_API_KEY``from the parent environment (no overrides needed).- 3. **OpenRouter** (default) — overrides base URL and auth token to+ 4. **OpenRouter** (default) — overrides base URL and auth token toroute through the proxy, with Langfuse trace headers.All modes receive workspace isolation (``CLAUDE_CODE_TMPDIR``) and@@ -48,12 +67,43 @@ def build_sdk_env(vars (currently: ``CLAUDE_AUTOCOMPACT_PCT_OVERRIDE`` is skipped forMoonshot since the cache-cost rationale doesn't apply there)."""+ if (codex_gateway_url is None) != (codex_gateway_token is None):+ raise ValueError(+ "codex_gateway_url and codex_gateway_token must be provided together"+ )+ if codex_gateway_token is not None and not codex_gateway_token:+ raise ValueError("Codex gateway token must not be empty")+ if codex_gateway_url is not None:+ parsed_gateway = urlparse(codex_gateway_url)+ if parsed_gateway.scheme != "http" or parsed_gateway.hostname not in {+ "127.0.0.1",+ "::1",+ "localhost",+ }:+ raise ValueError("Codex gateway must use a loopback HTTP URL")++ # A connected Codex account is a request-scoped auth transport. It must+ # win over the deployment-wide profile, including ``local``: the loopback+ # gateway speaks the Anthropic wire protocol expected by Claude Code even+ # when the configured baseline provider does not.+ if codex_gateway_url is not None and codex_gateway_token is not None:+ no_proxy = _loopback_no_proxy_value()+ env: dict[str, str] = {+ "ANTHROPIC_BASE_URL": codex_gateway_url.rstrip("/"),+ "ANTHROPIC_AUTH_TOKEN": codex_gateway_token,+ "ANTHROPIC_API_KEY": "",+ "CLAUDE_CODE_OAUTH_TOKEN": "",+ "CLAUDE_CODE_REFRESH_TOKEN": "",+ "NO_PROXY": no_proxy,+ "no_proxy": no_proxy,+ }+# Transports that don't run the SDK at all (currently: ``local`` —# Ollama et al. don't implement Anthropic's wire protocol) must not# reach this builder. The processor downgrades extended_thinking →# fast for those transports, so an entry here indicates a bug# upstream. Fail loudly rather than constructing a doomed env.- if not config.transport.supports_sdk:+ elif not config.transport.supports_sdk:raise RuntimeError(f"build_sdk_env() called under transport "f"{config.transport.name!r}, which doesn't support the SDK. "@@ -62,9 +112,9 @@ def build_sdk_env()# --- Mode 1: Claude Code subscription auth ---- if config.use_claude_code_subscription:+ elif config.use_claude_code_subscription:validate_subscription()- env: dict[str, str] = {+ env = {"ANTHROPIC_API_KEY": "","ANTHROPIC_AUTH_TOKEN": "","ANTHROPIC_BASE_URL": "",diff --git a/autogpt_platform/backend/backend/copilot/sdk/env_test.py b/autogpt_platform/backend/backend/copilot/sdk/env_test.pyindex 0f6ed78483..e940fe7ac4 100644--- a/autogpt_platform/backend/backend/copilot/sdk/env_test.py+++ b/autogpt_platform/backend/backend/copilot/sdk/env_test.py@@ -503,6 +503,77 @@ class TestBuildSdkEnvLocalTransportGuard:):build_sdk_env()+ def test_codex_override_runs_sdk_under_local_transport(self):+ cfg = _make_config(+ use_local=True,+ api_key="ollama",+ base_url="http://host.docker.internal:11434/v1",+ )+ with patch("backend.copilot.sdk.env.config", cfg):+ from backend.copilot.sdk.env import build_sdk_env++ result = build_sdk_env(+ codex_gateway_url="http://127.0.0.1:43210/",+ codex_gateway_token="loopback-capability",+ )++ assert result["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:43210"+ assert result["ANTHROPIC_AUTH_TOKEN"] == "loopback-capability"+ assert result["ANTHROPIC_API_KEY"] == ""+ assert result["CLAUDE_CODE_OAUTH_TOKEN"] == ""+ assert result["CLAUDE_CODE_REFRESH_TOKEN"] == ""+ assert {"127.0.0.1", "localhost", "::1"} <= set(result["NO_PROXY"].split(","))+ assert result["no_proxy"] == result["NO_PROXY"]+++class TestBuildSdkEnvCodexGateway:+ def test_codex_override_preserves_existing_no_proxy_hosts(self):+ cfg = _make_config(use_openrouter=False)+ with (+ patch("backend.copilot.sdk.env.config", cfg),+ patch.dict(+ "backend.copilot.sdk.env.os.environ",+ {+ "NO_PROXY": "internal.example,metadata.internal,localhost",+ },+ clear=True,+ ),+ ):+ from backend.copilot.sdk.env import build_sdk_env++ result = build_sdk_env(+ codex_gateway_url="http://localhost:43210",+ codex_gateway_token="loopback-capability",+ )++ assert set(result["NO_PROXY"].split(",")) == {+ "internal.example",+ "metadata.internal",+ "127.0.0.1",+ "localhost",+ "::1",+ }+ assert result["no_proxy"] == result["NO_PROXY"]++ def test_codex_override_requires_url_and_token_together(self):+ cfg = _make_config(use_openrouter=False)+ with patch("backend.copilot.sdk.env.config", cfg):+ from backend.copilot.sdk.env import build_sdk_env++ with pytest.raises(ValueError, match="must be provided together"):+ build_sdk_env(codex_gateway_url="http://127.0.0.1:43210")++ def test_codex_override_rejects_non_loopback_url(self):+ cfg = _make_config(use_openrouter=False)+ with patch("backend.copilot.sdk.env.config", cfg):+ from backend.copilot.sdk.env import build_sdk_env++ with pytest.raises(ValueError, match="loopback HTTP URL"):+ build_sdk_env(+ codex_gateway_url="https://example.com",+ codex_gateway_token="must-not-leak",+ )+class TestAutocompactPctSonnet5Scaling:"""Sonnet 5's trigger percentage is scaled by the ~1.3x tokenizerdiff --git a/autogpt_platform/backend/backend/copilot/sdk/service.py b/autogpt_platform/backend/backend/copilot/sdk/service.pyindex ff2ca9fcad..2e64139f39 100644--- a/autogpt_platform/backend/backend/copilot/sdk/service.py+++ b/autogpt_platform/backend/backend/copilot/sdk/service.py@@ -41,9 +41,19 @@ from langsmith.integrations.claude_agent_sdk import configure_claude_agent_sdkfrom opentelemetry import trace as otel_tracefrom pydantic import BaseModel+from backend.copilot.model_router import (+ ResolvedModel,+ RoutingSource,+ resolve_codex_model_route,+ resolve_model_route,+)+from backend.copilot.sdk.codex_compat_gateway import CodexAnthropicGatewayfrom backend.data.db_accessors import chat_dbfrom backend.data.redis_client import get_redis_asyncfrom backend.executor.cluster_lock import AsyncClusterLock+from backend.integrations.codex.models import CodexReasoningEffort, CodexTokenUsage+from backend.integrations.codex.transport import PooledCodexRuntimeLease+from backend.integrations.credential_lease import CredentialLeasefrom backend.util.exceptions import NotFoundErrorfrom backend.util.settings import Settings@@ -63,7 +73,6 @@ from ..graphiti.config import is_enabled_for_userfrom ..model_normalize import normalize_model_for_transportfrom backend.data.llm_registry.llm_models import MODEL_DATE_SUFFIX_RE-from ..model_router import ResolvedModel, RoutingSource, resolve_model_routefrom ..moonshot import (is_moonshot_model as _is_moonshot_model,override_cost_usd as _override_cost_for_moonshot,@@ -2078,6 +2087,14 @@ async def _resolve_sdk_model_for_request(return sdk_model, resolved.source+def _codex_gateway_usage(+ gateway: CodexAnthropicGateway,+) -> CodexTokenUsage | None:+ """Read authoritative aggregate usage from a completed Codex gateway."""+ result = gateway.result+ return result.usage if result is not None else None++_MAX_TRANSIENT_BACKOFF_SECONDS = 30@@ -2160,6 +2177,54 @@ def _is_fallback_stderr(line: str) -> bool:return "fallback model" in line.lower()+_CLI_STDERR_AUTH_RE = re.compile(+ r"(?i)(?P<prefix>\b(?:authorization|proxy-authorization|x-api-key|"+ r"anthropic[_-]auth[_-]token)\b[\"']?\s*[:=]\s*[\"']?)"+ r"(?:(?:bearer|basic)\s+)?[^,\s;\"'}\]]+"+)+++def _redact_cli_stderr(line: str, *, secrets: tuple[str, ...] = ()) -> str:+ """Remove known capabilities and authorization values from CLI logs."""+ redacted = _CLI_STDERR_AUTH_RE.sub(r"\g<prefix>[REDACTED]", line)+ for secret in secrets:+ if secret:+ redacted = redacted.replace(secret, "[REDACTED]")+ return redacted+++async def _close_codex_gateway_for_finally(+ gateway: CodexAnthropicGateway,+ log_prefix: str,+) -> BaseException | None:+ """Finish gateway checkpointing and return any error for deferred raising."""+ close_task = asyncio.create_task(gateway.close())+ try:+ await asyncio.shield(close_task)+ except asyncio.CancelledError as cancellation:+ try:+ await close_task+ except BaseException as cleanup_error:+ logger.exception("%s Codex gateway cleanup failed", log_prefix)+ return cleanup_error+ return cancellation+ except Exception as cleanup_error:+ logger.exception("%s Codex gateway cleanup failed", log_prefix)+ return cleanup_error+ return None+++def _raise_deferred_codex_cleanup_error(+ cleanup_error: BaseException | None,+ turn_error: BaseException | None,+) -> None:+ if cleanup_error is None:+ return+ if turn_error is not None and turn_error is not cleanup_error:+ raise cleanup_error from turn_error+ raise cleanup_error++def _build_system_prompt_value(system_prompt: str,*,@@ -3926,6 +3991,7 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuesrequest_arrival_at: float = 0.0,organization_id: str | None = None,team_id: str | None = None,+ credential_lease: CredentialLease | PooledCodexRuntimeLease | None = None,**_kwargs: Any,) -> AsyncGenerator[StreamBaseResponse, None]:# Pyright's complexity heuristic bails on this ~1500 LoC function (retry@@ -3961,7 +4027,7 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues# Stamping state for THIS turn: messages beyond this index were created# by the current turn and get model/routing_source at persist time.# Pre-feature history rows (NULL model) must never be back-stamped.- routing_source: RoutingSource = "env"+ routing_source: "RoutingSource" = "env"# The session row is the tenancy anchor; the turn entry's org/team only# backfills sessions created before org tagging (pre-migration rows).@@ -4121,6 +4187,10 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues# Defaults ensure the finally block can always reference these safely even when# an early return (e.g. sdk_cwd error) skips their normal assignment below.sdk_model: str | None = None+ codex_effort: "CodexReasoningEffort | None" = None+ codex_gateway: CodexAnthropicGateway | None = None+ deferred_codex_cleanup_error: BaseException | None = None+ is_codex_transport = credential_lease is not None# Wall-clock timestamp captured before the CLI runs so the# OpenRouter reconcile can filter subagent JSONLs by mtime — only# files created during THIS turn contribute gen-IDs. Without this@@ -4241,7 +4311,7 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuesyield StreamStart(messageId=message_id, sessionId=session_id)- if mode == "fast":+ if mode == "fast" and not is_codex_transport:# The request asked for Fast but this turn runs on the SDK# engine (building-mode pin or engine-switch continuation).# Tell stale pickers — a client that missed the original@@ -4256,7 +4326,11 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuespermissions=permissions,)- if not config.api_key and not config.use_claude_code_subscription:+ if (+ not is_codex_transport+ and not config.api_key+ and not config.use_claude_code_subscription+ ):raise RuntimeError("No API key configured. Set OPEN_ROUTER_API_KEY, ""CHAT_API_KEY, or ANTHROPIC_API_KEY for API access, "@@ -4291,10 +4365,35 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues# Resolve model (request tier → LD per-user override → config default).# Done BEFORE build_sdk_env so model-aware env vars (e.g. the# Moonshot autocompact gate) can branch on the resolved slug.- sdk_model, routing_source = await _resolve_sdk_model_for_request(- model, session_id, user_id- )- fallback_model = _resolve_fallback_model()+ if credential_lease is not None:+ tier_name: "CopilotLLMModel" = (+ "advanced" if model == "advanced" else "standard"+ )+ route_mode = "fast" if mode == "fast" else "thinking"+ sdk_model, codex_effort, routing_source = await resolve_codex_model_route(+ route_mode,+ tier_name,+ credential_lease,+ )+ if isinstance(credential_lease, PooledCodexRuntimeLease):+ codex_gateway = CodexAnthropicGateway(+ agent_session=credential_lease,+ model=sdk_model,+ effort=codex_effort,+ )+ else:+ codex_gateway = CodexAnthropicGateway(+ credential_lease=credential_lease,+ model=sdk_model,+ effort=codex_effort,+ )+ await codex_gateway.start()+ fallback_model = None+ else:+ sdk_model, routing_source = await _resolve_sdk_model_for_request(+ model, session_id, user_id+ )+ fallback_model = _resolve_fallback_model()# sdk_cwd routes the CLI's temp dir into the per-session workspace# so sub-agent output files land inside sdk_cwd (see build_sdk_env).@@ -4303,6 +4402,8 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuesuser_id=user_id,sdk_cwd=sdk_cwd,model=_resolve_env_model(sdk_model, fallback_model),+ codex_gateway_url=(codex_gateway.base_url if codex_gateway else None),+ codex_gateway_token=(codex_gateway.auth_token if codex_gateway else None),)# Track SDK-internal compaction (PreCompact hook → start, next msg → end)@@ -4329,7 +4430,9 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues"""Log a stderr line emitted by the Claude CLI subprocess."""nonlocal fallback_model_activated_per_attemptsid = session_id[:12] if session_id else "?"- logger.info("[SDK] [%s] CLI stderr: %s", sid, line.rstrip())+ capability = codex_gateway.auth_token if codex_gateway is not None else ""+ safe_line = _redact_cli_stderr(line, secrets=(capability,))+ logger.info("[SDK] [%s] CLI stderr: %s", sid, safe_line.rstrip())# Detect SDK fallback-model activation via the module-level pure# helper so the detection logic can be unit-tested independently.# Sets the per-attempt flag which is preserved across transient@@ -4364,27 +4467,9 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuescwd=sdk_cwd,max_buffer_size=config.claude_agent_max_buffer_size,stderr=_on_stderr,- # --- P0 guardrails ---- # fallback_model: SDK auto-retries with this cheaper model on- # 529 (overloaded) errors, avoiding user-visible failures.- fallback_model=fallback_model,# max_turns: hard cap on agentic tool-use loops per query to# prevent runaway execution from burning budget.max_turns=config.agent_max_turns,- # max_budget_usd: per-query spend ceiling enforced by the CLI.- # Sized to the smaller of the configured per-query default and- # the user's *actual* remaining daily/weekly USD cap so the- # CLI's "wrap up gracefully" reminder fires when they're close- # to the real limit, not the static $10 default.- max_budget_usd=await _resolve_dynamic_max_budget_usd(user_id),- # thinking: specify extended thinking mode. Thinking tokens are- # billed at output rate ($75/M for Opus) and account for ~54%- # of total cost. The CLI silently ignores this field for- # models without native extended thinking, so it is safe to- # pass unconditionally.- # NOTE: Claude 4.7+ does not support capped thinking token- # budget: use `effort` instead to steer thinking effort.- thinking={"type": "adaptive"},# effort: applies to models with extended thinking (Sonnet,# Opus, Mythos) and Kimi K2.6 via OpenRouter's ``reasoning``# extension (#12871).@@ -4394,11 +4479,18 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssueselse (config.claude_agent_thinking_effort or "high")),)- # max_thinking_tokens: legacy cost cap retained for non-4.7 models- # and Kimi K2.6. Setting to 0 acts as the kill switch (same as- # baseline): omit the kwarg so the CLI falls back to its default.- if config.claude_agent_max_thinking_tokens > 0:- sdk_options.max_thinking_tokens = config.claude_agent_max_thinking_tokens+ if not is_codex_transport:+ # The Claude/OpenRouter paths use provider-USD controls. A Codex+ # subscription consumes the connected account's allowance, so+ # presenting a fabricated USD ceiling or provider fallback to the+ # CLI would produce incorrect policy and accounting behavior.+ sdk_options.fallback_model = fallback_model+ sdk_options.max_budget_usd = await _resolve_dynamic_max_budget_usd(user_id)+ sdk_options.thinking = {"type": "adaptive"}+ if config.claude_agent_max_thinking_tokens > 0:+ sdk_options.max_thinking_tokens = (+ config.claude_agent_max_thinking_tokens+ )if sdk_model:sdk_options.model = sdk_modelif config.sdk_include_partial_messages:@@ -5183,11 +5275,28 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuesraisefinally:+ turn_error = sys.exception()# Pending messages are drained atomically at the start of each# turn (see drain_pending_messages call above), so there's# nothing to clean up here — any message pushed after that# point belongs to the next turn.+ if codex_gateway is not None:+ deferred_codex_cleanup_error = await _close_codex_gateway_for_finally(+ codex_gateway,+ log_prefix,+ )++ turn_cost_usd = None+ codex_usage = _codex_gateway_usage(codex_gateway)+ if codex_usage is not None:+ turn_prompt_tokens = max(+ 0, codex_usage.input_tokens - codex_usage.cached_input_tokens+ )+ turn_completion_tokens = codex_usage.output_tokens+ turn_cache_read_tokens = codex_usage.cached_input_tokens+ turn_cache_creation_tokens = 0+# --- Close OTEL context (with cost attributes) ---# Captured before __exit__ so the reconcile task (launched below,# after the span closes) can attach a backfill event to this turn's@@ -5242,7 +5351,8 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssueslist(state.generation_ids) if state is not None else [])_use_openrouter_reconcile = bool(- config.openrouter_active+ not is_codex_transport+ and config.openrouter_activeand config.sdk_reconcile_openrouter_costand collected_gen_ids)@@ -5261,7 +5371,25 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuesencode_cwd_for_cli(sdk_cwd),)- if _use_openrouter_reconcile:+ if is_codex_transport:+ await persist_and_record_usage(+ session=session,+ user_id=user_id,+ prompt_tokens=turn_prompt_tokens,+ completion_tokens=turn_completion_tokens,+ cache_read_tokens=turn_cache_read_tokens,+ cache_creation_tokens=0,+ log_prefix=log_prefix,+ cost_usd=None,+ model=effective_model,+ provider="codex",+ credential_id_override=(+ credential_lease.credentials.id if credential_lease else None+ ),+ extra_metadata={"billing_mode": "user_subscription"},+ execution_path="codex_claude_sdk",+ )+ elif _use_openrouter_reconcile:# Defer the single cost-and-rate-limit write to a background# task that queries OpenRouter's authoritative# ``/generation?id=`` for every round in this turn. Covers@@ -5504,6 +5632,11 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssues# Release stream lock to allow new streams for this sessionawait lock.release()+ _raise_deferred_codex_cleanup_error(+ deferred_codex_cleanup_error,+ turn_error,+ )+# -------------------------------------------------------------------------# Auto-continue: drain any messages the user queued AFTER the turn-start# drain window and process them as a new turn automatically.@@ -5560,10 +5693,14 @@ async def stream_chat_completion_sdk( # pyright: ignore[reportGeneralTypeIssuesmessage=_auto_combined,is_user_message=True,user_id=user_id,+ session=session,file_ids=None,permissions=permissions,mode=mode,model=model,+ organization_id=organization_id,+ team_id=team_id,+ credential_lease=credential_lease,):if _first_auto_event:_first_auto_event = False@@ -5633,7 +5770,7 @@ def _stamp_turn_messages(start_index: int,requested_model: str | None,actual_model: str | None,- routing_source: RoutingSource,+ routing_source: "RoutingSource",) -> None:"""Stamp THIS turn's assistant messages with the serving model androuting layer.diff --git a/autogpt_platform/backend/backend/copilot/sdk/service_test.py b/autogpt_platform/backend/backend/copilot/sdk/service_test.pyindex 43bb6a48c2..665903b707 100644--- a/autogpt_platform/backend/backend/copilot/sdk/service_test.py+++ b/autogpt_platform/backend/backend/copilot/sdk/service_test.py@@ -5,6 +5,7 @@ import base64import osfrom dataclasses import dataclassfrom types import SimpleNamespace+from typing import castfrom unittest.mock import AsyncMock, MagicMock, patchimport pytest@@ -15,18 +16,22 @@ from backend.copilot.model_router import ResolvedModelfrom backend.copilot.permissions import CopilotPermissions, all_known_tool_namesfrom backend.data.sharing.workspace_refs import extract_workspace_file_ids+from .codex_compat_gateway import CodexAnthropicGatewayfrom .service import (_HUNG_TOOL_CAP_SECONDS,_IDLE_TIMEOUT_SECONDS,_MAX_BUDGET_USD_FLOOR,_THINKING_ONLY_REPROMPT,_build_system_prompt_value,+ _close_codex_gateway_for_finally,_hidden_short_names_for_permissions,_humanise_tool_list,_idle_timeout_threshold,_is_sdk_disconnect_error,_normalize_model_name,_prepare_file_attachments,+ _raise_deferred_codex_cleanup_error,+ _redact_cli_stderr,_resolve_dynamic_max_budget_usd,_resolve_sdk_model,_resolve_sdk_model_for_request,@@ -35,6 +40,73 @@ from .service import ()+class TestCliStderrRedaction:+ def test_redacts_capability_even_without_header_label(self):+ capability = "private-loopback-capability"++ result = _redact_cli_stderr(+ f"gateway request failed for {capability}", secrets=(capability,)+ )++ assert capability not in result+ assert result == "gateway request failed for [REDACTED]"++ def test_redacts_labeled_capability_once(self):+ capability = "private-loopback-capability"++ result = _redact_cli_stderr(+ f"Authorization: Bearer {capability}", secrets=(capability,)+ )++ assert result == "Authorization: [REDACTED]"++ @pytest.mark.parametrize(+ ("line", "secret"),+ [+ ("Authorization: Bearer secret-one", "secret-one"),+ ('{"Authorization":"Bearer secret-two"}', "secret-two"),+ ("proxy-authorization=Basic secret-three", "secret-three"),+ ("ANTHROPIC_AUTH_TOKEN=secret-four", "secret-four"),+ ("x-api-key: secret-five", "secret-five"),+ ],+ )+ def test_redacts_authorization_shaped_values(self, line, secret):+ result = _redact_cli_stderr(line)++ assert secret not in result+ assert "[REDACTED]" in result++ def test_preserves_unrelated_stderr(self):+ line = "fallback model activated after an overload"++ assert _redact_cli_stderr(line) == line+++class TestCodexGatewayCleanup:+ @pytest.mark.asyncio+ async def test_returns_checkpoint_failure_for_deferred_raise(self):+ failure = RuntimeError("credential checkpoint failed")+ gateway = cast(+ CodexAnthropicGateway,+ SimpleNamespace(close=AsyncMock(side_effect=failure)),+ )++ result = await _close_codex_gateway_for_finally(gateway, "[test]")++ assert result is failure+ gateway.close.assert_awaited_once()++ def test_deferred_failure_propagates_with_turn_error_as_cause(self):+ turn_error = ValueError("turn failed")+ cleanup_error = RuntimeError("credential checkpoint failed")++ with pytest.raises(RuntimeError, match="checkpoint failed") as raised:+ _raise_deferred_codex_cleanup_error(cleanup_error, turn_error)++ assert raised.value is cleanup_error+ assert raised.value.__cause__ is turn_error++@dataclassclass _FakeFileInfo:id: strdiff --git a/autogpt_platform/backend/backend/copilot/sdk/session_waiter.py b/autogpt_platform/backend/backend/copilot/sdk/session_waiter.pyindex f149990e60..992c175f87 100644--- a/autogpt_platform/backend/backend/copilot/sdk/session_waiter.py+++ b/autogpt_platform/backend/backend/copilot/sdk/session_waiter.py@@ -26,6 +26,7 @@ from typing import TYPE_CHECKING, Literalfrom backend.copilot import stream_registryfrom backend.copilot.active_turns import ConcurrentTurnLimitErrorfrom backend.copilot.executor.utils import schedule_turn+from backend.copilot.model import get_chat_sessionfrom backend.copilot.pending_message_helpers import (is_turn_in_flight,queue_user_message,@@ -172,6 +173,10 @@ async def run_copilot_turn_via_queue(what happened next"; no separate deferred-state branch needed in``run_sub_session`` / ``AutoPilotBlock``."""+ session = await get_chat_session(session_id, user_id)+ if session is None:+ raise RuntimeError("copilot_session_not_found")+if await is_turn_in_flight(session_id):logger.info("[queue] session=%s has a turn in flight; queueing message "@@ -215,6 +220,8 @@ async def run_copilot_turn_via_queue(message=message,tool_call_id=tool_call_id,tool_name=tool_name,+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,permissions=permissions,)except ConcurrentTurnLimitError:diff --git a/autogpt_platform/backend/backend/copilot/sdk/session_waiter_test.py b/autogpt_platform/backend/backend/copilot/sdk/session_waiter_test.pyindex 4631c72b50..358ab76059 100644--- a/autogpt_platform/backend/backend/copilot/sdk/session_waiter_test.py+++ b/autogpt_platform/backend/backend/copilot/sdk/session_waiter_test.py@@ -25,6 +25,18 @@ _QR = type()[email protected](autouse=True)+def mock_session_lookup():+ session = MagicMock()+ session.metadata.llm_auth_provider = "platform"+ session.metadata.llm_credential_id = None+ with patch(+ "backend.copilot.sdk.session_waiter.get_chat_session",+ new=AsyncMock(return_value=session),+ ) as lookup:+ yield lookup++@pytest.mark.asyncioasync def test_queue_branch_timeout_zero_returns_immediately():"""Busy + timeout=0 → no registry, no enqueue, no wait, queued result."""diff --git a/autogpt_platform/backend/backend/copilot/token_tracking.py b/autogpt_platform/backend/backend/copilot/token_tracking.pyindex fb1322f51f..3d67776ecf 100644--- a/autogpt_platform/backend/backend/copilot/token_tracking.py+++ b/autogpt_platform/backend/backend/copilot/token_tracking.py@@ -122,6 +122,7 @@ async def persist_and_record_usage(block_name_override: str | None = None,extra_metadata: dict | None = None,graph_exec_id_override: str | None = None,+ credential_id_override: str | None = None,skip_daily: bool = False,execution_path: str = "sync",) -> int:@@ -278,7 +279,7 @@ async def persist_and_record_usage(else _copilot_block_name(log_prefix)),provider=provider,- credential_id=COPILOT_CREDENTIAL_ID,+ credential_id=credential_id_override or COPILOT_CREDENTIAL_ID,cost_microdollars=cost_microdollars,input_tokens=prompt_tokens,output_tokens=completion_tokens,diff --git a/autogpt_platform/backend/backend/copilot/tools/helpers.py b/autogpt_platform/backend/backend/copilot/tools/helpers.pyindex f21e60ca83..05f99c2c8c 100644--- a/autogpt_platform/backend/backend/copilot/tools/helpers.py+++ b/autogpt_platform/backend/backend/copilot/tools/helpers.py@@ -31,7 +31,11 @@ from backend.executor.auto_credentials import ()from backend.executor.simulator import simulate_blockfrom backend.executor.utils import block_usage_cost+from backend.integrations.codex.access import enforce_codex_access+from backend.integrations.credential_lease import CredentialLease+from backend.integrations.credentials_store import provider_matchesfrom backend.integrations.creds_manager import IntegrationCredentialsManager+from backend.integrations.providers import ProviderNamefrom backend.util.exceptions import BlockError, InsufficientBalanceErrorfrom backend.util.timezone_utils import get_user_timezone_or_utcfrom backend.util.type import coerce_inputs_to_schema@@ -271,20 +275,54 @@ async def execute_block(# Inject credentialscreds_manager = IntegrationCredentialsManager()- for field_name, cred_meta in matched_credentials.items():- if field_name not in input_data:- input_data[field_name] = cred_meta.model_dump()+ credential_leases: dict[str, CredentialLease] = {}+ credential_field_name = "credentials"+ try:+ for field_name, cred_meta in matched_credentials.items():+ credential_field_name = field_name+ if field_name not in input_data:+ input_data[field_name] = cred_meta.model_dump()+ if cred_meta.provider == ProviderName.CODEX:+ lease = await creds_manager.acquire_lease(user_id, cred_meta.id)+ credential_leases[field_name] = lease+ credentials = lease.credentials+ if not (+ provider_matches(credentials.provider, cred_meta.provider)+ and credentials.type == cred_meta.type+ ):+ raise ValueError+ await enforce_codex_access(user_id)+ exec_kwargs[field_name] = credentials+ continue- actual_credentials = await creds_manager.get(- user_id, cred_meta.id, lock=False- )- if actual_credentials:- exec_kwargs[field_name] = actual_credentials- else:- return ErrorResponse(- message=f"Failed to retrieve credentials for {field_name}",- session_id=session_id,+ credentials = await creds_manager.get(+ user_id,+ cred_meta.id,+ lock=False,)+ if not (+ credentials is not None+ and provider_matches(credentials.provider, cred_meta.provider)+ and credentials.type == cred_meta.type+ ):+ await _release_credential_leases(credential_leases)+ return ErrorResponse(+ message=f"Failed to retrieve credentials for {field_name}",+ session_id=session_id,+ )+ exec_kwargs[field_name] = credentials+ except ValueError:+ await _release_credential_leases(credential_leases)+ return ErrorResponse(+ message=f"Failed to retrieve credentials for {credential_field_name}",+ session_id=session_id,+ )+ except BaseException:+ await _release_credential_leases(credential_leases)+ raise++ if credential_leases:+ exec_kwargs["credential_leases"] = credential_leases# Auto-credentials (picker-populated fields like GoogleDriveFileField).# If the picker hasn't been filled, surface the existing setup-card so@@ -299,6 +337,7 @@ async def execute_block(user_id=user_id,)except MissingAutoCredentialsError as e:+ await _release_credential_leases(credential_leases)input_schema = block.input_schema.jsonschema()credentials_fields = set(block.input_schema.get_credentials_fields().keys())return SetupRequirementsResponse(@@ -326,7 +365,11 @@ async def execute_block(graph_version=None,)except ValueError as e:+ await _release_credential_leases(credential_leases)return ErrorResponse(message=str(e), error=str(e), session_id=session_id)+ except BaseException:+ await _release_credential_leases(credential_leases)+ raise# Everything from here owns the auto-cred locks; wrap so any early# return / exception (coerce, credit check, execution, etc.) still@@ -453,6 +496,7 @@ async def execute_block())finally:+ await _release_credential_leases(credential_leases)# Release auto-cred locks on every exit path so Redis doesn't hold them until TTL.for lock in auto_locks:try:@@ -496,6 +540,20 @@ async def _collect_block_outputs(outputs[output_name].append(output_data)+async def _release_credential_leases(+ leases: dict[str, CredentialLease],+) -> None:+ for field_name, lease in leases.items():+ try:+ await lease.release()+ except Exception as release_exc:+ logger.warning(+ "Failed to release credential lease for %s: %s",+ field_name,+ release_exc,+ )++async def resolve_block_credentials(user_id: str,block: AnyBlockSchema,diff --git a/autogpt_platform/backend/backend/copilot/tools/helpers_test.py b/autogpt_platform/backend/backend/copilot/tools/helpers_test.pyindex e40c68c3bf..1878477a1b 100644--- a/autogpt_platform/backend/backend/copilot/tools/helpers_test.py+++ b/autogpt_platform/backend/backend/copilot/tools/helpers_test.py@@ -1,13 +1,14 @@"""Tests for execute_block, prepare_block_for_execution, and check_hitl_review."""from collections.abc import AsyncIterator-from typing import Any+from typing import Any, Literalfrom unittest.mock import AsyncMock, MagicMock, patchimport pytestfrom backend.blocks._base import BlockTypefrom backend.copilot.constants import COPILOT_NODE_PREFIX, COPILOT_SESSION_PREFIX+from backend.copilot.rate_limit import UserPaywalledErrorfrom backend.copilot.tools.helpers import (BlockPreparation,check_hitl_review,@@ -22,6 +23,8 @@ from backend.copilot.tools.models import (ReviewRequiredResponse,SetupRequirementsResponse,)+from backend.data.model import CredentialsMetaInput+from backend.integrations.providers import ProviderNamefrom ._test_data import make_session@@ -60,6 +63,13 @@ def _patch_workspace():return patch("backend.copilot.tools.helpers.workspace_db", return_value=mock_ws_db)+def _patch_user_db():+ user = MagicMock(timezone="UTC")+ client = MagicMock()+ client.get_user_by_id = AsyncMock(return_value=user)+ return patch("backend.copilot.tools.helpers.user_db", return_value=client)++def _patch_credit_db(get_credits_return: int = 100,spend_credits_side_effect: Any = None,@@ -1471,6 +1481,225 @@ class TestExecuteBlockAutoCredentials:mock_lock.release.assert_awaited_once()+class TestExecuteBlockCredentialLeases:+ async def test_regular_credentials_are_leased_and_released(self):+ block = _make_block()+ captured: dict[str, Any] = {}+ credentials = MagicMock(id="cred-1", provider="codex", type="oauth2")+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ manager.get = AsyncMock(return_value=credentials)+ credit_patch, _ = _patch_credit_db()++ async def execute(_input_data: dict, **kwargs: Any):+ captured.update(kwargs)+ yield "result", "ok"++ block.execute = execute+ credential_meta = CredentialsMetaInput[+ Literal[ProviderName.CODEX], Literal["oauth2"]+ ](id="cred-1", provider=ProviderName.CODEX, type="oauth2")++ with (+ _patch_workspace(),+ _patch_user_db(),+ credit_patch,+ patch(+ "backend.copilot.tools.helpers.IntegrationCredentialsManager",+ return_value=manager,+ ),+ ):+ result = await execute_block(+ block=block,+ block_id="block-1",+ input_data={},+ user_id=_USER,+ session_id=_SESSION,+ node_exec_id="exec-lease-1",+ matched_credentials={"credentials": credential_meta},+ dry_run=False,+ )++ assert isinstance(result, BlockOutputResponse)+ assert captured["credentials"] is credentials+ assert captured["credential_leases"] == {"credentials": lease}+ manager.acquire_lease.assert_awaited_once_with(_USER, "cred-1")+ manager.get.assert_not_awaited()+ lease.release.assert_awaited_once()++ async def test_codex_entitlement_is_checked_against_acquired_credentials(self):+ block = _make_block()+ credentials = MagicMock(id="cred-1", provider="codex", type="oauth2")+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ credit_patch, _ = _patch_credit_db()+ credential_meta = CredentialsMetaInput[+ Literal[ProviderName.CODEX], Literal["oauth2"]+ ](id="cred-1", provider=ProviderName.CODEX, type="oauth2")+ gate = AsyncMock(side_effect=UserPaywalledError("Max plan required"))++ with (+ _patch_workspace(),+ _patch_user_db(),+ credit_patch,+ patch(+ "backend.copilot.tools.helpers.IntegrationCredentialsManager",+ return_value=manager,+ ),+ patch(+ "backend.copilot.tools.helpers.enforce_codex_access",+ new=gate,+ ),+ ):+ result = await execute_block(+ block=block,+ block_id="block-1",+ input_data={},+ user_id=_USER,+ session_id=_SESSION,+ node_exec_id="exec-entitlement",+ matched_credentials={"credentials": credential_meta},+ dry_run=False,+ )++ assert isinstance(result, ErrorResponse)+ assert "Max plan required" in result.message+ gate.assert_awaited_once_with(_USER)+ manager.acquire_lease.assert_awaited_once_with(_USER, "cred-1")+ lease.release.assert_awaited_once()++ async def test_credential_metadata_cannot_hide_authoritative_codex_provider(self):+ block = _make_block()+ credentials = MagicMock(id="cred-1", provider="codex", type="oauth2")+ manager = MagicMock()+ manager.get = AsyncMock(return_value=credentials)+ manager.acquire_lease = AsyncMock()+ credit_patch, _ = _patch_credit_db()+ credential_meta = CredentialsMetaInput[+ Literal[ProviderName.OPENAI, ProviderName.CODEX],+ Literal["api_key", "oauth2"],+ ](id="cred-1", provider=ProviderName.OPENAI, type="oauth2")+ gate = AsyncMock()++ with (+ _patch_workspace(),+ _patch_user_db(),+ credit_patch,+ patch(+ "backend.copilot.tools.helpers.IntegrationCredentialsManager",+ return_value=manager,+ ),+ patch(+ "backend.copilot.tools.helpers.enforce_codex_access",+ new=gate,+ ),+ ):+ result = await execute_block(+ block=block,+ block_id="block-1",+ input_data={},+ user_id=_USER,+ session_id=_SESSION,+ node_exec_id="exec-disguised-codex",+ matched_credentials={"credentials": credential_meta},+ dry_run=False,+ )++ assert isinstance(result, ErrorResponse)+ assert "Failed to retrieve credentials" in result.message+ gate.assert_not_awaited()+ manager.acquire_lease.assert_not_awaited()++ async def test_ordinary_credentials_keep_nonlocking_lookup(self):+ block = _make_block()+ captured: dict[str, Any] = {}+ credentials = MagicMock(id="cred-1", provider="openai", type="api_key")+ manager = MagicMock()+ manager.acquire_lease = AsyncMock()+ manager.get = AsyncMock(return_value=credentials)+ credit_patch, _ = _patch_credit_db()++ async def execute(_input_data: dict, **kwargs: Any):+ captured.update(kwargs)+ yield "result", "ok"++ block.execute = execute+ credential_meta = CredentialsMetaInput[+ Literal[ProviderName.OPENAI], Literal["api_key"]+ ](id="cred-1", provider=ProviderName.OPENAI, type="api_key")++ with (+ _patch_workspace(),+ _patch_user_db(),+ credit_patch,+ patch(+ "backend.copilot.tools.helpers.IntegrationCredentialsManager",+ return_value=manager,+ ),+ ):+ result = await execute_block(+ block=block,+ block_id="block-1",+ input_data={},+ user_id=_USER,+ session_id=_SESSION,+ node_exec_id="exec-api-key",+ matched_credentials={"credentials": credential_meta},+ dry_run=False,+ )++ assert isinstance(result, BlockOutputResponse)+ assert captured["credentials"] is credentials+ assert "credential_leases" not in captured+ manager.get.assert_awaited_once_with(_USER, "cred-1", lock=False)+ manager.acquire_lease.assert_not_awaited()++ async def test_regular_lease_is_released_when_input_coercion_fails(self):+ block = _make_block()+ credentials = MagicMock(id="cred-1", provider="codex", type="oauth2")+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ manager.get = AsyncMock(return_value=credentials)+ credit_patch, _ = _patch_credit_db()+ credential_meta = CredentialsMetaInput[+ Literal[ProviderName.CODEX], Literal["oauth2"]+ ](id="cred-1", provider=ProviderName.CODEX, type="oauth2")++ with (+ _patch_workspace(),+ _patch_user_db(),+ credit_patch,+ patch(+ "backend.copilot.tools.helpers.IntegrationCredentialsManager",+ return_value=manager,+ ),+ patch(+ "backend.copilot.tools.helpers.coerce_inputs_to_schema",+ side_effect=RuntimeError("coercion failed"),+ ),+ ):+ result = await execute_block(+ block=block,+ block_id="block-1",+ input_data={},+ user_id=_USER,+ session_id=_SESSION,+ node_exec_id="exec-lease-2",+ matched_credentials={"credentials": credential_meta},+ dry_run=False,+ )++ assert isinstance(result, ErrorResponse)+ lease.release.assert_awaited_once()++class TestRequireLibraryCheck:"""Tests for the library-similarity gate. The gate is turn-scoped:only the current turn's in-flight find_library_agent calls satisfydiff --git a/autogpt_platform/backend/backend/copilot/tools/run_sub_session.py b/autogpt_platform/backend/backend/copilot/tools/run_sub_session.pyindex bade3cc61e..3425e94b4b 100644--- a/autogpt_platform/backend/backend/copilot/tools/run_sub_session.py+++ b/autogpt_platform/backend/backend/copilot/tools/run_sub_session.py@@ -151,6 +151,15 @@ class RunSubSessionTool(BaseTool):),session_id=session.session_id,)+ if (+ owned.metadata.llm_auth_provider != session.metadata.llm_auth_provider+ or owned.metadata.llm_credential_id+ != session.metadata.llm_credential_id+ ):+ return ErrorResponse(+ message="codex_session_route_mismatch",+ session_id=session.session_id,+ )inner_session_id = sub_session_paramelse:new_session = await create_chat_session(@@ -158,6 +167,8 @@ class RunSubSessionTool(BaseTool):dry_run=session.dry_run,organization_id=session.organization_id,team_id=session.team_id,+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,)inner_session_id = new_session.session_iddiff --git a/autogpt_platform/backend/backend/copilot/tools/sub_session_test.py b/autogpt_platform/backend/backend/copilot/tools/sub_session_test.pyindex 9191807693..5cd3ec94ae 100644--- a/autogpt_platform/backend/backend/copilot/tools/sub_session_test.py+++ b/autogpt_platform/backend/backend/copilot/tools/sub_session_test.py@@ -34,6 +34,8 @@ def _session(user_id: str = "u", session_id: str = "s1") -> MagicMock:sess.dry_run = Falsesess.organization_id = Nonesess.team_id = None+ sess.metadata.llm_auth_provider = "platform"+ sess.metadata.llm_credential_id = Nonereturn sess@@ -134,6 +136,8 @@ def mock_model(monkeypatch):dry_run: bool,organization_id: str | None = None,team_id: str | None = None,+ llm_auth_provider: str = "platform",+ llm_credential_id: str | None = None,):sess = MagicMock()sess.session_id = f"inner-{len(created) + 1}"@@ -141,6 +145,8 @@ def mock_model(monkeypatch):sess.dry_run = dry_runsess.organization_id = organization_idsess.team_id = team_id+ sess.metadata.llm_auth_provider = llm_auth_provider+ sess.metadata.llm_credential_id = llm_credential_idsess.messages = []created.append(sess)return sessdiff --git a/autogpt_platform/backend/backend/copilot/turn_queue.py b/autogpt_platform/backend/backend/copilot/turn_queue.pyindex 990a65f9e7..46af2f6982 100644--- a/autogpt_platform/backend/backend/copilot/turn_queue.py+++ b/autogpt_platform/backend/backend/copilot/turn_queue.py@@ -33,7 +33,7 @@ import uuidfrom typing import Any, Mappingfrom backend.copilot.active_turns import TurnSlot, count_running_turns-from backend.copilot.config import ChatConfig+from backend.copilot.config import ChatConfig, CopilotLlmAuthProviderfrom backend.copilot.model import (CHAT_STATUS_IDLE,CHAT_STATUS_QUEUED,@@ -50,6 +50,7 @@ from backend.copilot.rate_limit import (is_user_paywalled,)from backend.data.db_accessors import chat_db+from backend.integrations.codex.access import has_codex_accesslogger = logging.getLogger(__name__)@@ -101,6 +102,8 @@ async def try_enqueue_turn(file_ids: list[str] | None = None,mode: str | None = None,model: str | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,permissions: Mapping[str, Any] | None = None,request_arrival_at: float = 0.0,) -> ChatMessage:@@ -124,6 +127,8 @@ async def try_enqueue_turn(file_ids=file_ids,mode=mode,model=model,+ llm_auth_provider=llm_auth_provider,+ llm_credential_id=llm_credential_id,permissions=permissions,request_arrival_at=request_arrival_at,)@@ -140,6 +145,8 @@ async def enqueue_turn(file_ids: list[str] | None = None,mode: str | None = None,model: str | None = None,+ llm_auth_provider: CopilotLlmAuthProvider = "platform",+ llm_credential_id: str | None = None,permissions: Mapping[str, Any] | None = None,request_arrival_at: float = 0.0,) -> ChatMessage:@@ -161,6 +168,9 @@ async def enqueue_turn(metadata["mode"] = modeif model is not None:metadata["model"] = model+ metadata["llm_auth_provider"] = llm_auth_provider+ if llm_credential_id is not None:+ metadata["llm_credential_id"] = llm_credential_idif permissions is not None:metadata["permissions"] = dict(permissions)if request_arrival_at:@@ -251,41 +261,52 @@ async def dispatch_next_for_user(user_id: str) -> bool:return Falsehead = queued[0]- if await is_user_paywalled(user_id):- logger.info(- "dispatch_next_for_user: user=%s paywalled, leaving session=%s queued",- user_id,- head.session_id,- )- return False+ if head.metadata.llm_auth_provider == "codex":+ if not await has_codex_access(user_id):+ logger.info(+ "dispatch_next_for_user: user=%s lacks Codex entitlement, "+ "leaving session=%s queued",+ user_id,+ head.session_id,+ )+ return False+ else:+ if await is_user_paywalled(user_id):+ logger.info(+ "dispatch_next_for_user: user=%s paywalled, leaving session=%s queued",+ user_id,+ head.session_id,+ )+ return False- cfg = ChatConfig()- try:- daily_limit, weekly_limit, _ = await get_global_rate_limits(- user_id,- cfg.daily_cost_limit_microdollars,- cfg.weekly_cost_limit_microdollars,- )- await check_rate_limit(- user_id=user_id,- daily_cost_limit=daily_limit,- weekly_cost_limit=weekly_limit,- )- except RateLimitExceeded as exc:- logger.info(- "dispatch_next_for_user: user=%s rate-limited (%s), leaving session=%s queued",- user_id,- exc,- head.session_id,- )- return False- except RateLimitUnavailable:- logger.warning(- "dispatch_next_for_user: rate-limit service degraded for user=%s; "- "leaving queue intact for the next tick",- user_id,- )- return False+ cfg = ChatConfig()+ try:+ daily_limit, weekly_limit, _ = await get_global_rate_limits(+ user_id,+ cfg.daily_cost_limit_microdollars,+ cfg.weekly_cost_limit_microdollars,+ )+ await check_rate_limit(+ user_id=user_id,+ daily_cost_limit=daily_limit,+ weekly_cost_limit=weekly_limit,+ )+ except RateLimitExceeded as exc:+ logger.info(+ "dispatch_next_for_user: user=%s rate-limited (%s), "+ "leaving session=%s queued",+ user_id,+ exc,+ head.session_id,+ )+ return False+ except RateLimitUnavailable:+ logger.warning(+ "dispatch_next_for_user: rate-limit service degraded for user=%s; "+ "leaving queue intact for the next tick",+ user_id,+ )+ return False# Claim by transitioning the session ``queued`` → ``running``. A# parallel cancel between validation and claim rejects this@@ -341,6 +362,8 @@ async def dispatch_next_for_user(user_id: str) -> bool:team_id=head.team_id,mode=metadata.get("mode"),model=metadata.get("model"),+ llm_auth_provider=head.metadata.llm_auth_provider,+ llm_credential_id=head.metadata.llm_credential_id,permissions=metadata.get("permissions"),request_arrival_at=float(metadata.get("request_arrival_at") or 0.0),)diff --git a/autogpt_platform/backend/backend/copilot/turn_queue_test.py b/autogpt_platform/backend/backend/copilot/turn_queue_test.pyindex 78e803949d..d4ca89c6fb 100644--- a/autogpt_platform/backend/backend/copilot/turn_queue_test.py+++ b/autogpt_platform/backend/backend/copilot/turn_queue_test.py@@ -90,6 +90,7 @@ async def test_enqueue_turn_packs_metadata_into_metadata_payload() -> None:assert metadata["file_ids"] == ["f1", "f2"]assert metadata["mode"] == "extended_thinking"assert metadata["model"] == "advanced"+ assert metadata["llm_auth_provider"] == "platform"assert metadata["permissions"] == {"tool_filter": "allow"}assert metadata["request_arrival_at"] == 123.45# Session is flipped idle → queued.@@ -99,9 +100,10 @@ async def test_enqueue_turn_packs_metadata_into_metadata_payload() -> None:@pytest.mark.asyncio-async def test_enqueue_turn_omits_null_fields_from_metadata() -> None:- """A turn with no extra params leaves ``metadata`` NULL on the row- instead of an empty object."""+async def test_enqueue_turn_only_includes_default_transport_without_extra_params() -> (+ None+):+ """A turn with no extra params only persists its default LLM transport."""db = MagicMock()db.get_next_sequence = AsyncMock(return_value=1)db.add_chat_message = AsyncMock(return_value=_pyd_message())@@ -112,7 +114,9 @@ async def test_enqueue_turn_omits_null_fields_from_metadata() -> None:patch.object(turn_queue, "invalidate_session_cache", new=AsyncMock()),):await turn_queue.enqueue_turn(user_id="u1", session_id="s1", message="hello")- assert db.add_chat_message.call_args.kwargs["metadata"] is None+ assert db.add_chat_message.call_args.kwargs["metadata"] == {+ "llm_auth_provider": "platform"+ }# ── cancel_queued_turn ─────────────────────────────────────────────────@@ -211,6 +215,88 @@ async def test_dispatch_leaves_queued_when_user_paywalled() -> None:db.update_chat_session_status.assert_not_awaited()+async def test_codex_dispatch_skips_platform_billing_gates() -> None:+ head = _mock_session()+ head.metadata.llm_auth_provider = "codex"+ head.metadata.llm_credential_id = "cred-1"+ pending = _pyd_message()+ db = MagicMock()+ db.update_chat_session_status = AsyncMock(return_value=True)+ db.get_latest_user_message_in_session = AsyncMock(return_value=pending)+ dispatch_turn_mock = AsyncMock()+ paywall_check = AsyncMock(side_effect=AssertionError("platform paywall checked"))+ global_limits = AsyncMock(side_effect=AssertionError("USD limits fetched"))+ rate_limit_check = AsyncMock(side_effect=AssertionError("USD limit checked"))++ with (+ _patch_queued_list([head]),+ patch.object(turn_queue, "chat_db", return_value=db),+ patch(+ "backend.copilot.turn_queue.is_user_paywalled",+ new=paywall_check,+ ),+ patch(+ "backend.copilot.turn_queue.get_global_rate_limits",+ new=global_limits,+ ),+ patch(+ "backend.copilot.turn_queue.check_rate_limit",+ new=rate_limit_check,+ ),+ patch.object(+ turn_queue,+ "has_codex_access",+ new=AsyncMock(return_value=True),+ ),+ patch(+ "backend.copilot.executor.utils.dispatch_turn",+ new=dispatch_turn_mock,+ ),+ patch.object(+ turn_queue,+ "invalidate_session_cache",+ new=AsyncMock(),+ ),+ ):+ promoted = await turn_queue.dispatch_next_for_user("u1")++ assert promoted is True+ paywall_check.assert_not_awaited()+ global_limits.assert_not_awaited()+ rate_limit_check.assert_not_awaited()+ dispatch_turn_mock.assert_awaited_once()+ assert dispatch_turn_mock.call_args.kwargs["llm_auth_provider"] == "codex"+ assert dispatch_turn_mock.call_args.kwargs["llm_credential_id"] == "cred-1"+++async def test_codex_dispatch_leaves_queued_when_user_lacks_access() -> None:+ head = _mock_session()+ head.metadata.llm_auth_provider = "codex"+ head.metadata.llm_credential_id = "cred-1"+ db = MagicMock()+ db.update_chat_session_status = AsyncMock()+ dispatch_turn_mock = AsyncMock()+ access = AsyncMock(return_value=False)++ with (+ _patch_queued_list([head]),+ patch.object(turn_queue, "chat_db", return_value=db),+ patch.object(turn_queue, "has_codex_access", new=access),+ patch(+ "backend.copilot.executor.utils.dispatch_turn",+ new=dispatch_turn_mock,+ ),+ ):+ promoted = await turn_queue.dispatch_next_for_user("u1")++ assert promoted is False+ access.assert_awaited_once_with("u1")+ db.update_chat_session_status.assert_not_awaited()+ dispatch_turn_mock.assert_not_awaited()++@pytest.mark.asyncioasync def test_dispatch_leaves_queued_on_rate_limit_exceeded() -> None:"""Mid-queue rate-limit lapse: leave the head queued, the next tickdiff --git a/autogpt_platform/backend/backend/data/block_cost_config.py b/autogpt_platform/backend/backend/data/block_cost_config.pyindex d7411682cd..20cdc7551f 100644--- a/autogpt_platform/backend/backend/data/block_cost_config.py+++ b/autogpt_platform/backend/backend/data/block_cost_config.py@@ -399,28 +399,14 @@ BLOCK_COSTS: dict[Type[Block], list[BlockCost]] = {AIStructuredResponseGeneratorBlock: LLM_COST,AITextSummarizerBlock: LLM_COST,AIListGeneratorBlock: LLM_COST,- # CodeGenerationBlock (Codex): block computes USD from- # response.usage.input_tokens/output_tokens using GPT-5.1-Codex rates- # ($1.25/$10 per 1M) and emits provider_cost + cost_usd. COST_USD 150- # cr/$ matches the TOKEN_COST margin — a 30K-token generation- # (~25K in + 5K out) ≈ $0.081 → 13 cr, vs the prior flat 5 cr.+ # CodeGenerationBlock (Codex): block computes USD from response usage with+ # the selected model's rate and emits provider_cost + cost_usd. COST_USD+ # 150 cr/$ matches the TOKEN_COST margin.CodeGenerationBlock: [BlockCost(cost_type=BlockCostType.COST_USD,cost_filter={- "model": CodexModel.GPT5_3_CODEX,- "credentials": {- "id": openai_credentials.id,- "provider": openai_credentials.provider,- "type": openai_credentials.type,- },- },- cost_amount=150,- ),- BlockCost(- cost_type=BlockCostType.COST_USD,- cost_filter={- "model": CodexModel.GPT5_1_CODEX,+ "model": model,"credentials": {"id": openai_credentials.id,"provider": openai_credentials.provider,@@ -428,7 +414,8 @@ BLOCK_COSTS: dict[Type[Block], list[BlockCost]] = {},},cost_amount=150,- ),+ )+ for model in CodexModel],# D-ID: $5.90/min of generated video. Median 10-sec clip ≈ $0.98 →# 148 cr at 1.5x. 100 cr flat is a conservative middle; long clipsdiff --git a/autogpt_platform/backend/backend/data/db_manager.py b/autogpt_platform/backend/backend/data/db_manager.pyindex 8ab53b2e66..ad704a432f 100644--- a/autogpt_platform/backend/backend/data/db_manager.py+++ b/autogpt_platform/backend/backend/data/db_manager.py@@ -150,6 +150,7 @@ from backend.data.user import (get_user_email_verification,get_user_integrations,get_user_notification_preference,+ get_user_subscription_tier,set_user_credentials,update_user_integrations,)@@ -318,6 +319,7 @@ class DatabaseManager(AppService):# ============ User + Integrations ============ #get_user_by_id = _(get_user_by_id)+ get_user_subscription_tier = _(get_user_subscription_tier)# Exposed so Prisma-less workers (scheduler, copilot-executor) can build a# full LaunchDarkly context — see backend/util/feature_flag.py.get_auth_user_flag_fields = _(get_auth_user_flag_fields)@@ -648,6 +650,7 @@ class DatabaseManagerAsyncClient(AppServiceClient):# ============ User + Integrations ============ #get_user_by_id = d.get_user_by_id+ get_user_subscription_tier = d.get_user_subscription_tierget_auth_user_flag_fields = d.get_auth_user_flag_fieldsget_user_integrations = d.get_user_integrationsupdate_user_integrations = d.update_user_integrationsdiff --git a/autogpt_platform/backend/backend/data/llm_registry/catalog.py b/autogpt_platform/backend/backend/data/llm_registry/catalog.pyindex fa16ef8829..78ee995de9 100644--- a/autogpt_platform/backend/backend/data/llm_registry/catalog.py+++ b/autogpt_platform/backend/backend/data/llm_registry/catalog.py@@ -1441,11 +1441,19 @@ def _build_catalog() -> CatalogPayload:cost=CatalogModelCost(run_credits=1),),],- # Routing cells deliberately ship EMPTY. Setting a cell is the- # explicit act of moving that (mode, tier)'s control from env vars- # into the catalog — shipping populated defaults would silently- # shadow the CHAT_*_MODEL env config of any deployment whose env- # differs from code defaults, and would flip models during LD- # outages. Cells apply on cloud deployments (behave_as=CLOUD) only.- routing={},+ # Platform-funded Copilot cells deliberately remain env-controlled.+ # Codex routes are pinned here, then validated against the models+ # advertised by each connected account with safe account fallbacks.+ routing={+ "copilot_codex": {+ "fast": {+ "standard": "gpt-5.6-luna",+ "advanced": "gpt-5.6-terra",+ },+ "thinking": {+ "standard": "gpt-5.6-terra",+ "advanced": "gpt-5.6-sol",+ },+ }+ },)diff --git a/autogpt_platform/backend/backend/data/model.py b/autogpt_platform/backend/backend/data/model.pyindex a566a7b32b..c4d0c872e4 100644--- a/autogpt_platform/backend/backend/data/model.py+++ b/autogpt_platform/backend/backend/data/model.py@@ -340,6 +340,9 @@ class _BaseCredentials(BaseModel):return value+OAuthRefreshStrategy = Literal["oauth_handler", "provider_runtime"]++class OAuth2Credentials(_BaseCredentials):type: Literal["oauth2"] = "oauth2"username: Optional[str] = None@@ -351,6 +354,9 @@ class OAuth2Credentials(_BaseCredentials):refresh_token_expires_at: Optional[int] = None"""Unix timestamp (seconds) indicating when the refresh token expires (if at all)"""scopes: list[str]+ refresh_strategy: OAuthRefreshStrategy = "oauth_handler"+ provider_state: Optional[SecretStr] = None+ provider_state_version: Optional[int] = Nonedef auth_header(self) -> str:return f"Bearer {self.access_token.get_secret_value()}"@@ -613,8 +619,10 @@ class CredentialsFieldInfo(BaseModel, Generic[CP, CT]):required_scopes: Optional[frozenset[str]] = Field(None, alias="credentials_scopes")discriminator: Optional[str] = Nonediscriminator_mapping: Optional[dict[str, CP]] = None+ discriminator_type_mapping: Optional[dict[str, frozenset[CT]]] = Nonediscriminator_values: set[Any] = Field(default_factory=set)is_auto_credential: bool = False+ credential_reference_only: bool = Falseinput_field_name: Optional[str] = None@classmethod@@ -713,8 +721,12 @@ class CredentialsFieldInfo(BaseModel, Generic[CP, CT]):credentials_scopes=frozenset(all_scopes) or None,discriminator=combined.discriminator,discriminator_mapping=combined.discriminator_mapping,+ discriminator_type_mapping=combined.discriminator_type_mapping,discriminator_values=set(all_discriminator_values),is_auto_credential=combined.is_auto_credential,+ credential_reference_only=all(+ field.credential_reference_only for _, field in group+ ),input_field_name=combined.input_field_name,),combined_keys,@@ -734,14 +746,25 @@ class CredentialsFieldInfo(BaseModel, Generic[CP, CT]):"It may have been deprecated. Please update your agent configuration.")+ supported_types = self.supported_types+ if self.discriminator_type_mapping is not None:+ try:+ supported_types = self.discriminator_type_mapping[discriminator_value]+ except KeyError:+ raise ValueError(+ f"Credential types for '{discriminator_value}' are not configured."+ ) from None+return CredentialsFieldInfo(credentials_provider=frozenset([provider]),- credentials_types=self.supported_types,+ credentials_types=supported_types,credentials_scopes=self.required_scopes,discriminator=self.discriminator,discriminator_mapping=self.discriminator_mapping,+ discriminator_type_mapping=self.discriminator_type_mapping,discriminator_values=set(self.discriminator_values),is_auto_credential=self.is_auto_credential,+ credential_reference_only=self.credential_reference_only,input_field_name=self.input_field_name,)@@ -751,6 +774,7 @@ def CredentialsField(*,discriminator: Optional[str] = None,discriminator_mapping: Optional[dict[str, Any]] = None,+ discriminator_type_mapping: Optional[dict[str, Any]] = None,discriminator_values: Optional[set[Any]] = None,title: Optional[str] = None,description: Optional[str] = None,@@ -767,7 +791,9 @@ def CredentialsField("credentials_scopes": list(required_scopes) or None,"discriminator": discriminator,"discriminator_mapping": discriminator_mapping,+ "discriminator_type_mapping": discriminator_type_mapping,"discriminator_values": discriminator_values,+ "credential_reference_only": kwargs.pop("credential_reference_only", None),}.items()if v is not None}@@ -897,6 +923,10 @@ class NodeExecutionStats(BaseModel):# by a block, resolve_tracking honors this directly instead of# guessing from provider name.provider_cost_type: Optional[ProviderCostType] = None+ billing_mode: Optional[str] = None+ auth_provider: Optional[str] = None+ execution_path: Optional[str] = None+ resolved_model: Optional[str] = None# Moderation fieldscleared_inputs: Optional[dict[str, list[str]]] = Nonecleared_outputs: Optional[dict[str, list[str]]] = Nonediff --git a/autogpt_platform/backend/backend/data/platform_cost.py b/autogpt_platform/backend/backend/data/platform_cost.pyindex 9af1fe754f..83fe89bfa5 100644--- a/autogpt_platform/backend/backend/data/platform_cost.py+++ b/autogpt_platform/backend/backend/data/platform_cost.py@@ -297,9 +297,8 @@ async def _ids_matching_metadata_filter(hydrates the full row + User join via``find_many({"id": {"in": ids}})``.- Only used when ``execution_path`` or ``source`` is set. The- happy-path query without metadata filters stays on the typed- Prisma helpers for speed + type-safety.+ The raw predicate also excludes user-subscription rows from platform-cost+ views. Those rows remain persisted for dedicated usage reporting."""clauses: list[str] = []params: list = []@@ -340,6 +339,7 @@ async def _ids_matching_metadata_filter(_add("metadata->>'execution_path' = {0}", execution_path)if source:_add("metadata->>'source' = {0}", source)+ clauses.append("metadata->>'billing_mode' IS DISTINCT FROM 'user_subscription'")where_sql = " AND ".join(clauses) if clauses else "TRUE"@@ -371,6 +371,7 @@ def _build_raw_where(block_name: str | None = None,tracking_type: str | None = None,graph_exec_id: str | None = None,+ default_tracking_type: str | None = "cost_usd",) -> tuple[str, list]:"""Build a parameterised WHERE clause for raw SQL queries.@@ -385,10 +386,13 @@ def _build_raw_where(# Always filter by tracking type — defaults to cost_usd for percentile /# bucket queries that only make sense on cost-denominated rows.- tt = tracking_type if tracking_type is not None else "cost_usd"- clauses.append(f'"trackingType" = ${idx}')- params.append(tt)- idx += 1+ tt = tracking_type if tracking_type is not None else default_tracking_type+ if tt is not None:+ clauses.append(f'"trackingType" = ${idx}')+ params.append(tt)+ idx += 1++ clauses.append("metadata->>'billing_mode' IS DISTINCT FROM 'user_subscription'")if start is not None:clauses.append(f'"createdAt" >= ${idx}::timestamptz')@@ -428,6 +432,47 @@ def _build_raw_where(return (" AND ".join(clauses), params)+async def _group_platform_cost_logs(+ *,+ by: tuple[str, ...],+ sum_fields: tuple[str, ...],+ where_sql: str,+ params: list,+) -> list[dict]:+ """Run a platform-cost aggregation with the metadata exclusion applied."""+ allowed_fields = {+ "provider",+ "trackingType",+ "model",+ "userId",+ "costMicrodollars",+ "inputTokens",+ "outputTokens",+ "cacheReadTokens",+ "cacheCreationTokens",+ "duration",+ "trackingAmount",+ }+ if not set(by + sum_fields) <= allowed_fields:+ raise ValueError("Unsupported PlatformCostLog aggregation field")++ group_columns = ", ".join(f'"{field}"' for field in by)+ sum_pairs = ", ".join(+ f"'{field}', COALESCE(SUM(\"{field}\"), 0)" for field in sum_fields+ )+ select_parts = [group_columns]+ if sum_pairs:+ select_parts.append(f'jsonb_build_object({sum_pairs}) AS "_sum"')+ select_parts.append("jsonb_build_object('_all', COUNT(*)) AS \"_count\"")++ return await query_raw_with_schema(+ f"SELECT {', '.join(select_parts)} "+ f'FROM {{schema_prefix}}"PlatformCostLog" '+ f"WHERE {where_sql} GROUP BY {group_columns}",+ *params,+ )++@cached(ttl_seconds=30)async def get_platform_cost_dashboard(start: datetime | None = None,@@ -453,15 +498,18 @@ async def get_platform_cost_dashboard(if start is None:start = datetime.now(timezone.utc) - timedelta(days=DEFAULT_DASHBOARD_DAYS)- where = _build_prisma_where(- start, end, provider, user_id, model, block_name, tracking_type, graph_exec_id+ raw_where_all, raw_params_all = _build_raw_where(+ start,+ end,+ provider,+ user_id,+ model,+ block_name,+ tracking_type,+ graph_exec_id,+ default_tracking_type=None,)-- # For per-user tracking-type breakdown we intentionally omit the- # tracking_type filter so cost_usd and tokens rows are always present.- # This ensures cost_bearing_request_count is correct even when the caller- # is filtering the main view by a different tracking_type.- where_no_tracking_type = _build_prisma_where(+ raw_where_no_tracking_type, raw_params_no_tracking_type = _build_raw_where(start,end,provider,@@ -470,18 +518,9 @@ async def get_platform_cost_dashboard(block_name,tracking_type=None,graph_exec_id=graph_exec_id,+ default_tracking_type=None,)- sum_fields = {- "costMicrodollars": True,- "inputTokens": True,- "outputTokens": True,- "cacheReadTokens": True,- "cacheCreationTokens": True,- "duration": True,- "trackingAmount": True,- }-# Build parameterised WHERE clause for the raw SQL percentile/bucket# queries. Uses _build_raw_where so filter logic is shared with# _build_prisma_where and only maintained in one place.@@ -501,47 +540,51 @@ async def get_platform_cost_dashboard(# Queries that always run regardless of tracking_type filter.common_queries = [- # (provider, trackingType, model) aggregation — no ORDER BY in ORM;- # sort by total cost descending in Python after fetch.- PrismaLog.prisma().group_by(- by=["provider", "trackingType", "model"],- where=where,- sum=sum_fields,- count=True,+ _group_platform_cost_logs(+ by=("provider", "trackingType", "model"),+ sum_fields=(+ "costMicrodollars",+ "inputTokens",+ "outputTokens",+ "cacheReadTokens",+ "cacheCreationTokens",+ "duration",+ "trackingAmount",+ ),+ where_sql=raw_where_all,+ params=raw_params_all,),- # userId aggregation — emails fetched separately below.- PrismaLog.prisma().group_by(- by=["userId"],- where=where,- sum=sum_fields,- count=True,+ _group_platform_cost_logs(+ by=("userId",),+ sum_fields=(+ "costMicrodollars",+ "inputTokens",+ "outputTokens",+ "cacheReadTokens",+ "cacheCreationTokens",+ "duration",+ "trackingAmount",+ ),+ where_sql=raw_where_all,+ params=raw_params_all,),- # Per-user cost-bearing request count: group by (userId, trackingType)- # so we can compute the correct denominator for per-user avg cost.- # Uses where_no_tracking_type so cost_usd rows are always included- # even when the caller filters the main view by a different tracking_type.- PrismaLog.prisma().group_by(- by=["userId", "trackingType"],- where=where_no_tracking_type,- count=True,+ _group_platform_cost_logs(+ by=("userId", "trackingType"),+ sum_fields=(),+ where_sql=raw_where_no_tracking_type,+ params=raw_params_no_tracking_type,),- # Distinct user count: group by userId, count groups.- PrismaLog.prisma().group_by(- by=["userId"],- where=where,- count=True,+ _group_platform_cost_logs(+ by=("userId",),+ sum_fields=(),+ where_sql=raw_where_all,+ params=raw_params_all,),- # Total aggregate (filtered): group by (provider, trackingType) so we can- # compute cost-bearing and token-bearing denominators for avg stats.- PrismaLog.prisma().group_by(- by=["provider", "trackingType"],- where=where,- sum={- "costMicrodollars": True,- "inputTokens": True,- "outputTokens": True,- },- count=True,+ _group_platform_cost_logs(+ by=("provider", "trackingType"),+ sum_fields=("costMicrodollars", "inputTokens", "outputTokens"),+ where_sql=raw_where_all,+ params=raw_params_all,),# Percentile distribution of cost per request (respects all filters).query_raw_with_schema(@@ -590,19 +633,11 @@ async def get_platform_cost_dashboard(# tracking types and reusing it avoids a redundant full aggregation.if tracking_type is not None:common_queries.append(- # Total aggregate (no tracking_type filter): used to compute- # cost_bearing_requests and token_bearing_requests denominators so- # global avg stats remain meaningful when the caller filters the- # main view by a specific tracking_type (e.g. 'tokens').- PrismaLog.prisma().group_by(- by=["provider", "trackingType"],- where=where_no_tracking_type,- sum={- "costMicrodollars": True,- "inputTokens": True,- "outputTokens": True,- },- count=True,+ _group_platform_cost_logs(+ by=("provider", "trackingType"),+ sum_fields=("costMicrodollars", "inputTokens", "outputTokens"),+ where_sql=raw_where_no_tracking_type,+ params=raw_params_no_tracking_type,))@@ -794,57 +829,28 @@ async def get_platform_cost_logs(offset = (page - 1) * page_size- if execution_path or source:- # Metadata-filtered path: typed Prisma ``count`` (an aggregate- # under the hood) and ``find_many`` both reject JSON-column- # filters with ``Could not find field at- # aggregatePlatformCostLog.where.metadata.string_equals``.- # Use raw SQL to identify the matching IDs + total, then- # hydrate via ``find_many({"id": {"in": ids}})`` so the User- # join + return-shape match the unfiltered path.- ids, total = await _ids_matching_metadata_filter(- start=start,- end=end,- provider=provider,- user_id=user_id,- model=model,- block_name=block_name,- tracking_type=tracking_type,- graph_exec_id=graph_exec_id,- execution_path=execution_path,- source=source,- limit=page_size,- offset=offset,+ ids, total = await _ids_matching_metadata_filter(+ start=start,+ end=end,+ provider=provider,+ user_id=user_id,+ model=model,+ block_name=block_name,+ tracking_type=tracking_type,+ graph_exec_id=graph_exec_id,+ execution_path=execution_path,+ source=source,+ limit=page_size,+ offset=offset,+ )+ if ids:+ rows = await PrismaLog.prisma().find_many(+ where={"id": {"in": ids}},+ include={"User": True},+ order=[{"createdAt": "desc"}, {"id": "desc"}],)- if ids:- rows = await PrismaLog.prisma().find_many(- where={"id": {"in": ids}},- include={"User": True},- order=[{"createdAt": "desc"}, {"id": "desc"}],- )- else:- rows = []else:- where = _build_prisma_where(- start,- end,- provider,- user_id,- model,- block_name,- tracking_type,- graph_exec_id,- )- total, rows = await asyncio.gather(- PrismaLog.prisma().count(where=where),- PrismaLog.prisma().find_many(- where=where,- include={"User": True},- order=[{"createdAt": "desc"}, {"id": "desc"}],- take=page_size,- skip=offset,- ),- )+ rows = []logs = [CostLogRow(@@ -1034,46 +1040,28 @@ async def get_platform_cost_logs_for_export(if start is None:start = datetime.now(tz=timezone.utc) - timedelta(days=DEFAULT_DASHBOARD_DAYS)- if execution_path or source:- ids, _ = await _ids_matching_metadata_filter(- start=start,- end=end,- provider=provider,- user_id=user_id,- model=model,- block_name=block_name,- tracking_type=tracking_type,- graph_exec_id=graph_exec_id,- execution_path=execution_path,- source=source,- limit=EXPORT_MAX_ROWS + 1,- offset=0,- )- if ids:- rows = await PrismaLog.prisma().find_many(- where={"id": {"in": ids}},- include={"User": True},- order=[{"createdAt": "desc"}, {"id": "desc"}],- )- else:- rows = []- else:- where = _build_prisma_where(- start,- end,- provider,- user_id,- model,- block_name,- tracking_type,- graph_exec_id,- )+ ids, _ = await _ids_matching_metadata_filter(+ start=start,+ end=end,+ provider=provider,+ user_id=user_id,+ model=model,+ block_name=block_name,+ tracking_type=tracking_type,+ graph_exec_id=graph_exec_id,+ execution_path=execution_path,+ source=source,+ limit=EXPORT_MAX_ROWS + 1,+ offset=0,+ )+ if ids:rows = await PrismaLog.prisma().find_many(- where=where,+ where={"id": {"in": ids}},include={"User": True},order=[{"createdAt": "desc"}, {"id": "desc"}],- take=EXPORT_MAX_ROWS + 1,)+ else:+ rows = []truncated = len(rows) > EXPORT_MAX_ROWSrows = rows[:EXPORT_MAX_ROWS]diff --git a/autogpt_platform/backend/backend/data/platform_cost_test.py b/autogpt_platform/backend/backend/data/platform_cost_test.pyindex 919184e072..714d9d74ca 100644--- a/autogpt_platform/backend/backend/data/platform_cost_test.py+++ b/autogpt_platform/backend/backend/data/platform_cost_test.py@@ -13,6 +13,7 @@ from .platform_cost import (_build_prisma_where,_build_raw_where,_build_where,+ _ids_matching_metadata_filter,_mask_email,get_platform_cost_dashboard,get_platform_cost_logs,@@ -211,6 +212,11 @@ class TestBuildPrismaWhere:class TestBuildRawWhere:+ def test_excludes_user_subscription_rows(self):+ sql, _ = _build_raw_where(None, None, None, None)++ assert "metadata->>'billing_mode' IS DISTINCT FROM 'user_subscription'" in sql+def test_end_filter(self):end = datetime(2026, 6, 1, tzinfo=timezone.utc)sql, params = _build_raw_where(None, end, None, None)@@ -259,6 +265,37 @@ class TestBuildRawWhere:assert "graphExecId" not in sql+class TestIdsMatchingMetadataFilter:+ @pytest.mark.asyncio+ async def test_excludes_user_subscription_rows_from_count_and_ids(self):+ raw_mock = AsyncMock(side_effect=[[{"c": 0}], []])++ with patch(+ "backend.data.platform_cost.query_raw_with_schema",+ raw_mock,+ ):+ await _ids_matching_metadata_filter(+ start=None,+ end=None,+ provider=None,+ user_id=None,+ model=None,+ block_name=None,+ tracking_type=None,+ graph_exec_id=None,+ execution_path=None,+ source=None,+ limit=50,+ offset=0,+ )++ for call in raw_mock.await_args_list:+ sql = call.args[0]+ assert (+ "metadata->>'billing_mode' IS DISTINCT FROM 'user_subscription'" in sql+ )++def _make_entry(**overrides: object) -> PlatformCostEntry:return PlatformCostEntry.model_validate({@@ -385,7 +422,7 @@ class TestGetPlatformCostDashboard:mock_user.email = "[email protected]"mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(+ group_mock = AsyncMock(side_effect=[[provider_row], # by_provider[user_row], # by_user@@ -405,6 +442,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",new_callable=AsyncMock,@@ -462,7 +503,7 @@ class TestGetPlatformCostDashboard:}mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(+ group_mock = AsyncMock(side_effect=[[total_row], # by_provider[{"_sum": {}, "_count": {"_all": 5}, "userId": "u1"}], # by_user@@ -486,6 +527,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",new_callable=AsyncMock,@@ -515,7 +560,7 @@ class TestGetPlatformCostDashboard:)mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(+ group_mock = AsyncMock(side_effect=[[tokens_row], # by_provider[{"_sum": {}, "_count": {"_all": 5}, "userId": "u1"}], # by_user@@ -536,6 +581,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",new_callable=AsyncMock,@@ -568,7 +617,7 @@ class TestGetPlatformCostDashboard:user_row = _make_group_by_row(user_id="u2", cost=1000, count=1)mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(+ group_mock = AsyncMock(side_effect=[[provider_row], # by_provider[user_row], # by_user@@ -588,6 +637,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",new_callable=AsyncMock,@@ -607,7 +660,7 @@ class TestGetPlatformCostDashboard:@pytest.mark.asyncioasync def test_returns_empty_dashboard(self):mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(side_effect=[[], [], [], [], []])+ group_mock = AsyncMock(side_effect=[[], [], [], [], []])mock_actions.find_many = AsyncMock(return_value=[])with (@@ -619,6 +672,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",new_callable=AsyncMock,@@ -640,7 +697,7 @@ class TestGetPlatformCostDashboard:start = datetime(2026, 1, 1, tzinfo=timezone.utc)mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(side_effect=[[], [], [], [], []])+ group_mock = AsyncMock(side_effect=[[], [], [], [], []])mock_actions.find_many = AsyncMock(return_value=[])raw_mock = AsyncMock(side_effect=[[], []])@@ -653,6 +710,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",raw_mock,@@ -662,13 +723,15 @@ class TestGetPlatformCostDashboard:start=start, provider="openai", user_id="u1")- # group_by called 5 times (by_provider, by_user, by_user_tracking, distinct users,+ # Grouped SQL helper called 5 times (provider, user, user/tracking,+ # distinct users,# total agg filtered); the 6th call (total agg no-tracking-type) only runs# when tracking_type is set.- assert mock_actions.group_by.await_count == 5- # The where dict passed to the first call should include createdAt- first_call_kwargs = mock_actions.group_by.call_args_list[0][1]- assert "createdAt" in first_call_kwargs.get("where", {})+ assert group_mock.await_count == 5+ first_call_kwargs = group_mock.call_args_list[0][1]+ assert '"createdAt" >= $' in first_call_kwargs["where_sql"]+ assert "openai" in first_call_kwargs["params"]+ assert "u1" in first_call_kwargs["params"]# Raw SQL queries should receive provider and user_id as parametersassert raw_mock.await_count == 2raw_call_args = raw_mock.call_args_list[0][0] # positional args of 1st call@@ -681,7 +744,7 @@ class TestGetPlatformCostDashboard:"""by_user_tracking_groups must NOT apply the tracking_type filter so thatcost_usd rows are always included even when the caller filters by 'tokens'."""mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(side_effect=[[], [], [], [], [], []])+ group_mock = AsyncMock(side_effect=[[], [], [], [], [], []])mock_actions.find_many = AsyncMock(return_value=[])with (@@ -693,6 +756,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",new_callable=AsyncMock,@@ -703,19 +770,19 @@ class TestGetPlatformCostDashboard:# Call index 2 is by_user_tracking_groups (0=by_provider, 1=by_user,# 2=by_user_tracking, 3=distinct_users, 4=total_agg, 5=total_agg_no_tt).- tracking_call_where = mock_actions.group_by.call_args_list[2][1]["where"]+ tracking_call_where = group_mock.call_args_list[2][1]["where_sql"]# The main filter applies trackingType; by_user_tracking must NOT.- assert "trackingType" not in tracking_call_where+ assert '"trackingType" = $' not in tracking_call_where# Other filters (e.g., date range, provider) are still passed through.# The first call (by_provider) should have trackingType in its where dict.- provider_call_where = mock_actions.group_by.call_args_list[0][1]["where"]- assert "trackingType" in provider_call_where+ provider_call_where = group_mock.call_args_list[0][1]["where_sql"]+ assert '"trackingType" = $' in provider_call_where@pytest.mark.asyncioasync def test_graph_exec_id_filter_passed_to_queries(self):"""graph_exec_id must be forwarded to both prisma and raw SQL queries."""mock_actions = MagicMock()- mock_actions.group_by = AsyncMock(side_effect=[[], [], [], [], []])+ group_mock = AsyncMock(side_effect=[[], [], [], [], []])mock_actions.find_many = AsyncMock(return_value=[])raw_mock = AsyncMock(side_effect=[[], []])@@ -728,6 +795,10 @@ class TestGetPlatformCostDashboard:"backend.data.platform_cost.PrismaUser.prisma",return_value=mock_actions,),+ patch(+ "backend.data.platform_cost._group_platform_cost_logs",+ group_mock,+ ),patch("backend.data.platform_cost.query_raw_with_schema",raw_mock,@@ -735,9 +806,9 @@ class TestGetPlatformCostDashboard:):await get_platform_cost_dashboard(graph_exec_id="exec-xyz")- # Prisma groupBy where must include graphExecId- first_call_where = mock_actions.group_by.call_args_list[0][1]["where"]- assert first_call_where.get("graphExecId") == "exec-xyz"+ first_call = group_mock.call_args_list[0][1]+ assert '"graphExecId" = $' in first_call["where_sql"]+ assert "exec-xyz" in first_call["params"]# Raw SQL params must include the exec idraw_params = raw_mock.call_args_list[0][0][1:]assert "exec-xyz" in raw_params@@ -779,9 +850,16 @@ class TestGetPlatformCostLogs:mock_actions.count = AsyncMock(return_value=1)mock_actions.find_many = AsyncMock(return_value=[row])- with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ with (+ patch(+ "backend.data.platform_cost.PrismaLog.prisma",+ return_value=mock_actions,+ ),+ patch(+ "backend.data.platform_cost._ids_matching_metadata_filter",+ new_callable=AsyncMock,+ return_value=(["log-0"], 1),+ ),):logs, total = await get_platform_cost_logs(page=1, page_size=10)@@ -793,13 +871,10 @@ class TestGetPlatformCostLogs:@pytest.mark.asyncioasync def test_returns_empty_when_no_data(self):- mock_actions = MagicMock()- mock_actions.count = AsyncMock(return_value=0)- mock_actions.find_many = AsyncMock(return_value=[])-with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ new_callable=AsyncMock,+ return_value=([], 0),):logs, total = await get_platform_cost_logs()@@ -808,53 +883,41 @@ class TestGetPlatformCostLogs:@pytest.mark.asyncioasync def test_pagination_offset(self):- mock_actions = MagicMock()- mock_actions.count = AsyncMock(return_value=100)- mock_actions.find_many = AsyncMock(return_value=[])-+ ids_mock = AsyncMock(return_value=([], 100))with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ ids_mock,):logs, total = await get_platform_cost_logs(page=3, page_size=25)assert total == 100- find_many_call = mock_actions.find_many.call_args[1]- assert find_many_call["take"] == 25- assert find_many_call["skip"] == 50 # (3-1) * 25+ query_args = ids_mock.await_args.kwargs+ assert query_args["limit"] == 25+ assert query_args["offset"] == 50@pytest.mark.asyncioasync def test_explicit_start_skips_default(self):start = datetime(2026, 1, 1, tzinfo=timezone.utc)- mock_actions = MagicMock()- mock_actions.count = AsyncMock(return_value=0)- mock_actions.find_many = AsyncMock(return_value=[])-+ ids_mock = AsyncMock(return_value=([], 0))with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ ids_mock,):logs, total = await get_platform_cost_logs(start=start)assert total == 0- where = mock_actions.count.call_args[1]["where"]- # start provided — should appear in the where filter- assert "createdAt" in where+ assert ids_mock.await_args.kwargs["start"] == start@pytest.mark.asyncioasync def test_graph_exec_id_filter(self):- mock_actions = MagicMock()- mock_actions.count = AsyncMock(return_value=0)- mock_actions.find_many = AsyncMock(return_value=[])-+ ids_mock = AsyncMock(return_value=([], 0))with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ ids_mock,):logs, total = await get_platform_cost_logs(graph_exec_id="exec-abc")- where = mock_actions.count.call_args[1]["where"]- assert where.get("graphExecId") == "exec-abc"+ assert ids_mock.await_args.kwargs["graph_exec_id"] == "exec-abc"class TestGetPlatformCostLogsForExport:@@ -864,9 +927,16 @@ class TestGetPlatformCostLogsForExport:mock_actions = MagicMock()mock_actions.find_many = AsyncMock(return_value=[row])- with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ with (+ patch(+ "backend.data.platform_cost.PrismaLog.prisma",+ return_value=mock_actions,+ ),+ patch(+ "backend.data.platform_cost._ids_matching_metadata_filter",+ new_callable=AsyncMock,+ return_value=(["log-0"], 1),+ ),):logs, truncated = await get_platform_cost_logs_for_export()@@ -876,12 +946,10 @@ class TestGetPlatformCostLogsForExport:@pytest.mark.asyncioasync def test_returns_empty_not_truncated(self):- mock_actions = MagicMock()- mock_actions.find_many = AsyncMock(return_value=[])-with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ new_callable=AsyncMock,+ return_value=([], 0),):logs, truncated = await get_platform_cost_logs_for_export()@@ -900,6 +968,11 @@ class TestGetPlatformCostLogsForExport:return_value=mock_actions,),patch("backend.data.platform_cost.EXPORT_MAX_ROWS", 2),+ patch(+ "backend.data.platform_cost._ids_matching_metadata_filter",+ new_callable=AsyncMock,+ return_value=(["log-0", "log-1", "log-2"], 3),+ ),):logs, truncated = await get_platform_cost_logs_for_export()@@ -908,22 +981,19 @@ class TestGetPlatformCostLogsForExport:@pytest.mark.asyncioasync def test_passes_model_block_tracking_filters(self):- mock_actions = MagicMock()- mock_actions.find_many = AsyncMock(return_value=[])-+ ids_mock = AsyncMock(return_value=([], 0))with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ ids_mock,):await get_platform_cost_logs_for_export(model="gpt-4", block_name="LLMBlock", tracking_type="tokens")- where = mock_actions.find_many.call_args[1]["where"]- assert where.get("model") == "gpt-4"- assert where.get("trackingType") == "tokens"- # blockName uses a dict filter for case-insensitive match- assert "blockName" in where+ query_args = ids_mock.await_args.kwargs+ assert query_args["model"] == "gpt-4"+ assert query_args["tracking_type"] == "tokens"+ assert query_args["block_name"] == "LLMBlock"@pytest.mark.asyncioasync def test_maps_cache_tokens(self):@@ -932,9 +1002,16 @@ class TestGetPlatformCostLogsForExport:mock_actions = MagicMock()mock_actions.find_many = AsyncMock(return_value=[row])- with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ with (+ patch(+ "backend.data.platform_cost.PrismaLog.prisma",+ return_value=mock_actions,+ ),+ patch(+ "backend.data.platform_cost._ids_matching_metadata_filter",+ new_callable=AsyncMock,+ return_value=(["log-0"], 1),+ ),):logs, _ = await get_platform_cost_logs_for_export()@@ -943,35 +1020,29 @@ class TestGetPlatformCostLogsForExport:@pytest.mark.asyncioasync def test_graph_exec_id_filter(self):- mock_actions = MagicMock()- mock_actions.find_many = AsyncMock(return_value=[])-+ ids_mock = AsyncMock(return_value=([], 0))with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ ids_mock,):logs, truncated = await get_platform_cost_logs_for_export(graph_exec_id="exec-xyz")- where = mock_actions.find_many.call_args[1]["where"]- assert where.get("graphExecId") == "exec-xyz"+ assert ids_mock.await_args.kwargs["graph_exec_id"] == "exec-xyz"assert logs == []assert truncated is False@pytest.mark.asyncioasync def test_explicit_start_skips_default(self):start = datetime(2026, 1, 1, tzinfo=timezone.utc)- mock_actions = MagicMock()- mock_actions.find_many = AsyncMock(return_value=[])-+ ids_mock = AsyncMock(return_value=([], 0))with patch(- "backend.data.platform_cost.PrismaLog.prisma",- return_value=mock_actions,+ "backend.data.platform_cost._ids_matching_metadata_filter",+ ids_mock,):logs, truncated = await get_platform_cost_logs_for_export(start=start)assert logs == []assert truncated is False- where = mock_actions.find_many.call_args[1]["where"]- assert "createdAt" in where+ assert ids_mock.await_args.kwargs["start"] == startdiff --git a/autogpt_platform/backend/backend/data/user.py b/autogpt_platform/backend/backend/data/user.pyindex 7975b5255d..c7d42b0780 100644--- a/autogpt_platform/backend/backend/data/user.py+++ b/autogpt_platform/backend/backend/data/user.py@@ -11,7 +11,7 @@ from urllib.parse import quote_plusfrom autogpt_libs.auth.models import DEFAULT_USER_IDfrom fastapi import HTTPException-from prisma.enums import NotificationType+from prisma.enums import NotificationType, SubscriptionTierfrom prisma.errors import UniqueViolationErrorfrom prisma.models import AuthUserfrom prisma.models import User as PrismaUser@@ -209,6 +209,14 @@ async def get_user_by_id(user_id: str) -> User:return User.from_db(user)+async def get_user_subscription_tier(user_id: str) -> SubscriptionTier:+ """Read the authoritative tier without using the cached full-user lookup."""+ user = await prisma.user.find_unique(where={"id": user_id})+ if not user:+ raise ValueError(f"User not found with ID: {user_id}")+ return user.subscriptionTier or SubscriptionTier.NO_TIER++async def get_user_email_by_id(user_id: str) -> Optional[str]:try:user = await prisma.user.find_unique(where={"id": user_id})diff --git a/autogpt_platform/backend/backend/data/user_subscription_tier_test.py b/autogpt_platform/backend/backend/data/user_subscription_tier_test.pynew file mode 100644index 0000000000..c5ff9966dd--- /dev/null+++ b/autogpt_platform/backend/backend/data/user_subscription_tier_test.py@@ -0,0 +1,45 @@+from unittest.mock import AsyncMock, MagicMock, patch++import pytest+from prisma.enums import SubscriptionTier++from backend.data import user as user_module+++async def test_get_user_subscription_tier_is_uncached():+ with patch.object(user_module, "prisma") as mock_prisma:+ mock_prisma.user.find_unique = AsyncMock(+ side_effect=[+ MagicMock(subscriptionTier=SubscriptionTier.PRO),+ MagicMock(subscriptionTier=SubscriptionTier.MAX),+ ]+ )++ first = await user_module.get_user_subscription_tier("user-1")+ second = await user_module.get_user_subscription_tier("user-1")++ assert first == SubscriptionTier.PRO+ assert second == SubscriptionTier.MAX+ assert mock_prisma.user.find_unique.await_count == 2+++async def test_get_user_subscription_tier_defaults_null_to_no_tier():+ with patch.object(user_module, "prisma") as mock_prisma:+ mock_prisma.user.find_unique = AsyncMock(+ return_value=MagicMock(subscriptionTier=None)+ )++ tier = await user_module.get_user_subscription_tier("user-1")++ assert tier == SubscriptionTier.NO_TIER+++async def test_get_user_subscription_tier_raises_for_missing_user():+ with patch.object(user_module, "prisma") as mock_prisma:+ mock_prisma.user.find_unique = AsyncMock(return_value=None)++ with pytest.raises(ValueError, match="User not found"):+ await user_module.get_user_subscription_tier("missing-user")diff --git a/autogpt_platform/backend/backend/executor/cost_tracking.py b/autogpt_platform/backend/backend/executor/cost_tracking.pyindex bacaea5758..34542012c2 100644--- a/autogpt_platform/backend/backend/executor/cost_tracking.py+++ b/autogpt_platform/backend/backend/executor/cost_tracking.py@@ -1,4 +1,4 @@-"""Helpers for platform cost tracking on system-credential block executions."""+"""Helpers for platform and user-subscription block usage tracking."""import asyncioimport logging@@ -203,7 +203,7 @@ async def log_system_credential_cost(stats: NodeExecutionStats,db_client: "DatabaseManagerAsyncClient",) -> None:- """Check if a system credential was used and log the platform cost.+ """Log platform-funded cost or user-subscription usage.Routes through DatabaseManagerAsyncClient so the write goes via themessage-passing DB service rather than calling Prisma directly (which@@ -229,7 +229,12 @@ async def log_system_credential_cost(if not cred_data or not isinstance(cred_data, dict):continuecred_id = cred_data.get("id", "")- if not cred_id or not is_system_credential(cred_id):+ provider_name = cred_data.get("provider", "unknown")+ is_subscription = (+ provider_name == ProviderName.CODEX.value+ and cred_data.get("type") == "oauth2"+ )+ if not cred_id or not (is_system_credential(cred_id) or is_subscription):continuemodel_name = _extract_model_name(input_data.get("model"))@@ -242,7 +247,6 @@ async def log_system_credential_cost(block=block, input_data=input_data, stats=stats)- provider_name = cred_data.get("provider", "unknown")tracking_type, tracking_amount = resolve_tracking(provider=provider_name,stats=stats,@@ -267,9 +271,11 @@ async def log_system_credential_cost(# the admin cost-logs Path column never renders "—".# When the block layer opts into flex/batch in the# future, write the actual mode here.- "execution_path": "sync",+ "execution_path": ("codex_app_server" if is_subscription else "sync"),"source": "block",}+ if is_subscription:+ meta["billing_mode"] = "user_subscription"if credit_cost is not None:meta["credit_cost"] = credit_costif stats.provider_cost is not None:diff --git a/autogpt_platform/backend/backend/executor/manager.py b/autogpt_platform/backend/backend/executor/manager.pyindex 768563ff31..23329a46a4 100644--- a/autogpt_platform/backend/backend/executor/manager.py+++ b/autogpt_platform/backend/backend/executor/manager.py@@ -36,13 +36,20 @@ from backend.data.execution import (NodesInputMasks,)from backend.data.graph import Link, Node-from backend.data.model import GraphExecutionStats, NodeExecutionStats+from backend.data.model import (+ GraphExecutionStats,+ NodeExecutionStats,+ OAuth2Credentials,+)from backend.data.rabbitmq import SyncRabbitMQfrom backend.data.redis_helpers import incr_with_ttl_syncfrom backend.executor.cost_tracking import (drain_pending_cost_logs,log_system_credential_cost,)+from backend.integrations.codex.access import enforce_codex_access+from backend.integrations.credential_lease import CredentialLease+from backend.integrations.credentials_store import provider_matchesfrom backend.integrations.creds_manager import IntegrationCredentialsManagerfrom backend.util import jsonfrom backend.util.clients import (@@ -260,65 +267,107 @@ async def execute_node(# one (running) block at a time; simultaneous execution of blocks using same# credentials is not supported.creds_locks: list[AsyncRedisLock] = []+ credential_leases: dict[str, CredentialLease] = {}+ runtime_credential_leases: dict[str, CredentialLease] = {}input_model = cast(type[BlockSchema], node_block.input_schema)- # Handle regular credentials fields- for field_name, input_type in input_model.get_credentials_fields().items():- # Dry-run platform credentials bypass the credential store.- # Keep the existing credential metadata so _execute's input_schema(**...)- # doesn't fail on the required field. If no metadata is present,- # synthesize a minimal placeholder from the platform credentials.- if _dry_run_creds is not None:- if input_data.get(field_name) is None:- input_data[field_name] = {- "id": _dry_run_creds.id,- "provider": _dry_run_creds.provider,- "type": _dry_run_creds.type,- "title": _dry_run_creds.title,- }- extra_exec_kwargs[field_name] = _dry_run_creds- continue+ try:+ # Handle regular credentials fields+ credential_fields_info = input_model.get_credentials_fields_info()+ for field_name, input_type in input_model.get_credentials_fields().items():+ # Dry-run platform credentials bypass the credential store.+ # Keep the existing credential metadata so _execute's input_schema(**...)+ # doesn't fail on the required field. If no metadata is present,+ # synthesize a minimal placeholder from the platform credentials.+ if _dry_run_creds is not None:+ if input_data.get(field_name) is None:+ input_data[field_name] = {+ "id": _dry_run_creds.id,+ "provider": _dry_run_creds.provider,+ "type": _dry_run_creds.type,+ "title": _dry_run_creds.title,+ }+ extra_exec_kwargs[field_name] = _dry_run_creds+ continue- field_value = input_data.get(field_name)- if not field_value or (- isinstance(field_value, dict) and not field_value.get("id")- ):- # No credentials configured — nullify so JSON schema validation- # doesn't choke on the empty default `{}`.- input_data[field_name] = None- continue # Block runs without credentials-- credentials_meta = input_type(**field_value)- # Write normalized values back so JSON schema validation also passes- # (model_validator may have fixed legacy formats like "ProviderName.MCP")- input_data[field_name] = credentials_meta.model_dump(mode="json")- try:- credentials, lock = await creds_manager.acquire(- user_id, credentials_meta.id- )- except ValueError:- # Credential was deleted or doesn't exist.- # If the field has a default, run without credentials.- if input_model.model_fields[field_name].default is not None:- log_metadata.warning(- f"Credentials #{credentials_meta.id} not found, "- "running without (field has default)"- )+ field_value = input_data.get(field_name)+ if not field_value or (+ isinstance(field_value, dict) and not field_value.get("id")+ ):+ # No credentials configured — nullify so JSON schema validation+ # doesn't choke on the empty default `{}`.input_data[field_name] = None+ continue # Block runs without credentials++ credentials_meta = input_type(**field_value)+ # Write normalized values back so JSON schema validation also passes+ # (model_validator may have fixed legacy formats like "ProviderName.MCP")+ input_data[field_name] = credentials_meta.model_dump(mode="json")+ if credential_fields_info[field_name].credential_reference_only:+ credentials = await creds_manager.get(+ user_id,+ credentials_meta.id,+ )+ if not (+ credentials is not None+ and provider_matches(+ credentials.provider,+ credentials_meta.provider,+ )+ and credentials.type == credentials_meta.type+ ):+ raise ValueError(+ f"Credentials #{credentials_meta.id} for user #{user_id} not found"+ )+ if provider_matches(credentials.provider, "codex"):+ await enforce_codex_access(user_id)continue- raise- creds_locks.append(lock)- extra_exec_kwargs[field_name] = credentials-- # Handle auto-generated credentials (e.g., from GoogleDriveFileInput)- auto_extra_kwargs, auto_locks = await acquire_auto_credentials(- input_model=input_model,- input_data=input_data,- creds_manager=creds_manager,- user_id=user_id,- )- extra_exec_kwargs.update(auto_extra_kwargs)- creds_locks.extend(auto_locks)+ try:+ lease = await creds_manager.acquire_lease(user_id, credentials_meta.id)+ except ValueError:+ # Credential was deleted or doesn't exist.+ # If the field has a default, run without credentials.+ if input_model.model_fields[field_name].default is not None:+ log_metadata.warning(+ f"Credentials #{credentials_meta.id} not found, "+ "running without (field has default)"+ )+ input_data[field_name] = None+ continue+ raise+ credential_leases[field_name] = lease+ credentials = lease.credentials+ if not (+ provider_matches(+ credentials.provider,+ credentials_meta.provider,+ )+ and credentials.type == credentials_meta.type+ ):+ raise ValueError(+ f"Credentials #{credentials_meta.id} for user #{user_id} not found"+ )+ if provider_matches(credentials.provider, "codex"):+ await enforce_codex_access(user_id)+ extra_exec_kwargs[field_name] = lease.credentials+ if _uses_provider_runtime(lease):+ runtime_credential_leases[field_name] = lease++ # Handle auto-generated credentials (e.g., from GoogleDriveFileInput)+ auto_extra_kwargs, auto_locks = await acquire_auto_credentials(+ input_model=input_model,+ input_data=input_data,+ creds_manager=creds_manager,+ user_id=user_id,+ )+ extra_exec_kwargs.update(auto_extra_kwargs)+ creds_locks.extend(auto_locks)+ except BaseException:+ await _release_credential_leases(list(credential_leases.values()), log_metadata)+ raise++ if runtime_credential_leases:+ extra_exec_kwargs["credential_leases"] = runtime_credential_leasesoutput_size = 0@@ -364,6 +413,7 @@ async def execute_node(raisefinally:# Ensure all credentials are released even if execution fails+ await _release_credential_leases(list(credential_leases.values()), log_metadata)for creds_lock in creds_locks:if (creds_lock@@ -386,6 +436,24 @@ async def execute_node(scope._tags = original_tags+async def _release_credential_leases(+ leases: list[CredentialLease], log_metadata: LogMetadata+) -> None:+ for lease in leases:+ try:+ await lease.release()+ except Exception as error:+ log_metadata.error(f"Failed to release credentials lease: {error}")+++def _uses_provider_runtime(lease: CredentialLease) -> bool:+ credentials = lease.credentials+ return (+ isinstance(credentials, OAuth2Credentials)+ and credentials.refresh_strategy == "provider_runtime"+ )++async def _enqueue_next_nodes(db_client: "DatabaseManagerAsyncClient",node: Node,diff --git a/autogpt_platform/backend/backend/executor/manager_cost_tracking_test.py b/autogpt_platform/backend/backend/executor/manager_cost_tracking_test.pyindex 2f669c41d2..95c91693b4 100644--- a/autogpt_platform/backend/backend/executor/manager_cost_tracking_test.py+++ b/autogpt_platform/backend/backend/executor/manager_cost_tracking_test.py@@ -267,6 +267,49 @@ class TestLogSystemCredentialCost:await log_system_credential_cost(node_exec, block, stats, db_client)db_client.log_platform_cost.assert_not_awaited()+ @pytest.mark.asyncio+ async def test_logs_codex_user_subscription_without_usd_cost(self):+ db_client = _make_db_client()+ with (+ patch(+ "backend.executor.cost_tracking.is_system_credential",+ return_value=False,+ ),+ patch(+ "backend.executor.cost_tracking.block_usage_cost",+ return_value=(0, None),+ ),+ ):+ node_exec = _make_node_exec(+ inputs={+ "credentials": {+ "id": "codex-user-cred",+ "provider": "codex",+ "type": "oauth2",+ },+ "model": "gpt-5.6-sol",+ }+ )+ block = _make_block()+ stats = NodeExecutionStats(+ input_token_count=500,+ output_token_count=200,+ provider_cost=700,+ provider_cost_type="tokens",+ )+ await log_system_credential_cost(node_exec, block, stats, db_client)+ await asyncio.sleep(0)++ db_client.log_platform_cost.assert_awaited_once()+ entry = db_client.log_platform_cost.call_args[0][0]+ assert entry.provider == "codex"+ assert entry.credential_id == "codex-user-cred"+ assert entry.cost_microdollars is None+ assert entry.tracking_type == "tokens"+ assert entry.tracking_amount == 700+ assert entry.metadata["billing_mode"] == "user_subscription"+ assert entry.metadata["execution_path"] == "codex_app_server"+@pytest.mark.asyncioasync def test_logs_with_system_credential(self):db_client = _make_db_client()@@ -516,6 +559,20 @@ class TestMergeStats:stats += NodeExecutionStats(provider_cost_type="items")assert stats.provider_cost_type == "items"+ def test_subscription_route_metadata_survives_merge(self):+ stats = NodeExecutionStats()+ stats += NodeExecutionStats(+ billing_mode="user_subscription",+ auth_provider="codex",+ execution_path="codex_app_server",+ resolved_model="gpt-live",+ )++ assert stats.billing_mode == "user_subscription"+ assert stats.auth_provider == "codex"+ assert stats.execution_path == "codex_app_server"+ assert stats.resolved_model == "gpt-live"+# ---------------------------------------------------------------------------# on_node_execution -> log_system_credential_cost integrationdiff --git a/autogpt_platform/backend/backend/executor/manager_credential_lease_test.py b/autogpt_platform/backend/backend/executor/manager_credential_lease_test.pynew file mode 100644index 0000000000..5326b84dcc--- /dev/null+++ b/autogpt_platform/backend/backend/executor/manager_credential_lease_test.py@@ -0,0 +1,303 @@+from types import SimpleNamespace+from typing import Any, Literal+from unittest.mock import AsyncMock, MagicMock, patch++import pytest+from pydantic import SecretStr++from backend.copilot.rate_limit import UserPaywalledError+from backend.data.execution import ExecutionContext, NodeExecutionEntry+from backend.data.model import (+ APIKeyCredentials,+ CredentialsMetaInput,+ NodeExecutionStats,+ OAuth2Credentials,+)+from backend.executor.manager import execute_node+from backend.integrations.providers import ProviderName++CodexCredentialsInput = CredentialsMetaInput[+ Literal[ProviderName.CODEX], Literal["oauth2"]+]+MixedCredentialsInput = CredentialsMetaInput[+ Literal[ProviderName.OPENAI, ProviderName.CODEX],+ Literal["api_key", "oauth2"],+]+++async def test_execute_node_injects_and_releases_credential_lease():+ credentials = _codex_credentials("cred-1")+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ captured: dict[str, object] = {}+ block = _block_with_credentials({"credentials": CodexCredentialsInput}, captured)++ outputs = [+ item+ async for item in execute_node(+ _node(block),+ _entry({"credentials": _credential_metadata("cred-1")}),+ SimpleNamespace(creds_manager=manager),+ )+ ]++ assert outputs == [("response", "ok")]+ assert captured["credentials"] is credentials+ assert captured["credential_leases"] == {"credentials": lease}+ manager.acquire_lease.assert_awaited_once_with("user-1", "cred-1")+ lease.release.assert_awaited_once()+++async def test_execute_node_does_not_inject_runtime_lease_for_api_key():+ credentials = APIKeyCredentials(+ id="cred-1",+ provider="openai",+ api_key=SecretStr("secret"),+ )+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ captured: dict[str, object] = {}+ openai_credentials = CredentialsMetaInput[+ Literal[ProviderName.OPENAI], Literal["api_key"]+ ]+ block = _block_with_credentials({"credentials": openai_credentials}, captured)++ outputs = [+ item+ async for item in execute_node(+ _node(block),+ _entry(+ {+ "credentials": {+ "id": "cred-1",+ "provider": "openai",+ "type": "api_key",+ "title": None,+ }+ }+ ),+ SimpleNamespace(creds_manager=manager),+ )+ ]++ assert outputs == [("response", "ok")]+ assert captured["credentials"] is credentials+ assert "credential_leases" not in captured+ lease.release.assert_awaited_once()+++async def test_execute_node_releases_first_lease_when_second_acquisition_fails():+ first = MagicMock(credentials=_codex_credentials("cred-1"))+ first.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(+ side_effect=[first, RuntimeError("second acquisition failed")]+ )+ block = _block_with_credentials(+ {+ "first_credentials": CodexCredentialsInput,+ "second_credentials": CodexCredentialsInput,+ },+ {},+ )+ inputs = {+ "first_credentials": _credential_metadata("cred-1"),+ "second_credentials": _credential_metadata("cred-2"),+ }++ with pytest.raises(RuntimeError, match="second acquisition"):+ await anext(+ execute_node(+ _node(block),+ _entry(inputs),+ SimpleNamespace(creds_manager=manager),+ )+ )++ first.release.assert_awaited_once()+++async def test_reference_only_credential_is_validated_without_outer_lease():+ credentials = _codex_credentials("cred-1")+ manager = MagicMock()+ manager.get = AsyncMock(return_value=credentials)+ manager.acquire_lease = AsyncMock()+ captured: dict[str, Any] = {}+ block = _block_with_credentials(+ {"codex_credentials": CodexCredentialsInput},+ captured,+ reference_only_fields={"codex_credentials"},+ )+ metadata = _credential_metadata("cred-1")++ outputs = [+ item+ async for item in execute_node(+ _node(block),+ _entry({"codex_credentials": metadata}),+ SimpleNamespace(creds_manager=manager),+ )+ ]++ assert outputs == [("response", "ok")]+ manager.get.assert_awaited_once_with("user-1", "cred-1")+ manager.acquire_lease.assert_not_awaited()+ assert captured["input_data"]["codex_credentials"] == metadata+ assert "codex_credentials" not in captured+ assert "credential_leases" not in captured+++async def test_codex_credential_is_rejected_and_released_without_entitlement():+ credentials = _codex_credentials("cred-1")+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ block = _block_with_credentials({"credentials": CodexCredentialsInput}, {})+ gate = AsyncMock(side_effect=UserPaywalledError("Max plan required"))++ with (+ patch("backend.executor.manager.enforce_codex_access", new=gate),+ pytest.raises(UserPaywalledError, match="Max plan required"),+ ):+ await anext(+ execute_node(+ _node(block),+ _entry({"credentials": _credential_metadata("cred-1")}),+ SimpleNamespace(creds_manager=manager),+ )+ )++ gate.assert_awaited_once_with("user-1")+ manager.acquire_lease.assert_awaited_once_with("user-1", "cred-1")+ lease.release.assert_awaited_once()+++async def test_credential_metadata_cannot_hide_authoritative_codex_provider():+ credentials = _codex_credentials("cred-1")+ lease = MagicMock(credentials=credentials)+ lease.release = AsyncMock()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=lease)+ captured: dict[str, object] = {}+ block = _block_with_credentials({"credentials": MixedCredentialsInput}, captured)+ gate = AsyncMock()+ disguised_metadata = {+ "id": "cred-1",+ "provider": "openai",+ "type": "oauth2",+ "title": None,+ }++ with (+ patch("backend.executor.manager.enforce_codex_access", new=gate),+ pytest.raises(ValueError, match="Credentials #cred-1 .* not found"),+ ):+ await anext(+ execute_node(+ _node(block),+ _entry({"credentials": disguised_metadata}),+ SimpleNamespace(creds_manager=manager),+ )+ )++ gate.assert_not_awaited()+ lease.release.assert_awaited_once()+ assert "credentials" not in captured+++def _block_with_credentials(+ credential_fields: dict[str, type[CodexCredentialsInput]],+ captured: dict[str, object],+ *,+ reference_only_fields: set[str] | None = None,+) -> MagicMock:+ reference_only_fields = reference_only_fields or set()+ input_schema = MagicMock()+ input_schema.get_credentials_fields.return_value = credential_fields+ input_schema.get_credentials_fields_info.return_value = {+ field_name: SimpleNamespace(+ credential_reference_only=field_name in reference_only_fields+ )+ for field_name in credential_fields+ }+ input_schema.get_auto_credentials_fields.return_value = {}+ block = MagicMock()+ block.id = "block-1"+ block.name = "LeaseBlock"+ block.disabled = False+ block.input_schema = input_schema+ block.execution_stats = NodeExecutionStats()++ async def execute(_input_data, **kwargs):+ captured["input_data"] = _input_data+ captured.update(kwargs)+ yield "response", "ok"++ block.execute = execute+ return block+++def _node(block: MagicMock) -> MagicMock:+ node = MagicMock()+ node.block = block+ return node+++def _entry(inputs: dict[str, object]) -> NodeExecutionEntry:+ return NodeExecutionEntry(+ user_id="user-1",+ graph_exec_id="graph-exec-1",+ graph_id="graph-1",+ graph_version=1,+ node_exec_id="node-exec-1",+ node_id="node-1",+ block_id="block-1",+ inputs=inputs,+ execution_context=ExecutionContext(),+ )+++def _credential_metadata(credentials_id: str) -> dict[str, str | None]:+ return {+ "id": credentials_id,+ "provider": "codex",+ "type": "oauth2",+ "title": None,+ }+++def _codex_credentials(credentials_id: str) -> OAuth2Credentials:+ return OAuth2Credentials(+ id=credentials_id,+ provider="codex",+ access_token=SecretStr("access"),+ refresh_token=SecretStr("refresh"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr("{}"),+ provider_state_version=1,+ )++[email protected](autouse=True)+def _execution_boundaries():+ scope = MagicMock(_user=None, _tags={})+ with (+ patch(+ "backend.executor.manager.validate_exec",+ side_effect=lambda _node, data, **_kwargs: (data, None),+ ),+ patch("backend.executor.manager._sentry_get_current_scope", return_value=scope),+ ):+ yielddiff --git a/autogpt_platform/backend/backend/executor/scheduler.py b/autogpt_platform/backend/backend/executor/scheduler.pyindex 690b8593fd..6a68e42607 100644--- a/autogpt_platform/backend/backend/executor/scheduler.py+++ b/autogpt_platform/backend/backend/executor/scheduler.py@@ -266,6 +266,7 @@ async def _execute_copilot_turn(**kwargs):team_id=args.team_id,)target_session_id = new_session.session_id+ target_session = new_sessionlogger.info(f"Copilot turn schedule {args.schedule_id} creating fresh "f"session {target_session_id[:12]} (sentinel session_id=None)"@@ -280,6 +281,7 @@ async def _execute_copilot_turn(**kwargs):await _self_delete_copilot_turn_schedule(args)returntarget_session_id = args.session_id+ target_session = session# `schedule_turn` (not raw `enqueue_copilot_turn`) is the right entry# point: it acquires a per-user concurrency slot AND registers the@@ -295,6 +297,8 @@ async def _execute_copilot_turn(**kwargs):tool_name="schedule_followup",organization_id=args.organization_id,team_id=args.team_id,+ llm_auth_provider=target_session.metadata.llm_auth_provider,+ llm_credential_id=target_session.metadata.llm_credential_id,)elapsed = asyncio.get_event_loop().time() - start_timelogger.info(diff --git a/autogpt_platform/backend/backend/executor/utils.py b/autogpt_platform/backend/backend/executor/utils.pyindex f397362bad..75c77c0db9 100644--- a/autogpt_platform/backend/backend/executor/utils.py+++ b/autogpt_platform/backend/backend/executor/utils.py@@ -488,8 +488,12 @@ async def _validate_node_input_credentials(# the block schema declares a default for the field.required_fields = block.input_schema.get_required_fields()is_creds_optional = node.credentials_optional+ credentials_fields_info = block.input_schema.get_credentials_fields_info()for field_name, credentials_meta_type in credentials_fields.items():+ reference_only = credentials_fields_info[+ field_name+ ].credential_reference_onlyfield_is_optional = is_creds_optional or field_name not in required_fieldstry:# Check nodes_input_masks first, then input_default@@ -512,6 +516,8 @@ async def _validate_node_input_credentials(if field_value is None or (isinstance(field_value, dict) and not field_value.get("id")):+ if reference_only:+ continuehas_missing_credentials = True# If credential field is optional, skip instead of errorif field_is_optional:diff --git a/autogpt_platform/backend/backend/executor/utils_test.py b/autogpt_platform/backend/backend/executor/utils_test.pyindex ee7e2f683d..064888326c 100644--- a/autogpt_platform/backend/backend/executor/utils_test.py+++ b/autogpt_platform/backend/backend/executor/utils_test.py@@ -703,6 +703,9 @@ async def test_validate_node_input_credentials_returns_nodes_to_skip(mock_block.input_schema.get_credentials_fields.return_value = {"credentials": mock_credentials_field_type}+ mock_block.input_schema.get_credentials_fields_info.return_value = {+ "credentials": mocker.Mock(credential_reference_only=False)+ }mock_block.input_schema.get_required_fields.return_value = {"credentials"}mock_node.block = mock_block@@ -747,6 +750,9 @@ async def test_validate_node_input_credentials_required_missing_creds_error(mock_block.input_schema.get_credentials_fields.return_value = {"credentials": mock_credentials_field_type}+ mock_block.input_schema.get_credentials_fields_info.return_value = {+ "credentials": mocker.Mock(credential_reference_only=False)+ }mock_block.input_schema.get_required_fields.return_value = {"credentials"}mock_node.block = mock_blockdiff --git a/autogpt_platform/backend/backend/integrations/codex/__init__.py b/autogpt_platform/backend/backend/integrations/codex/__init__.pynew file mode 100644index 0000000000..bb0e8b3965--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/__init__.py@@ -0,0 +1 @@+"""ChatGPT-authenticated Codex App Server transport."""diff --git a/autogpt_platform/backend/backend/integrations/codex/access.py b/autogpt_platform/backend/backend/integrations/codex/access.pynew file mode 100644index 0000000000..5c7297b5d5--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/access.py@@ -0,0 +1,63 @@+import logging++from fastapi import HTTPException, status++from backend.copilot.rate_limit import UserPaywalledError+from backend.util.entitlements import (+ Entitlement,+ EntitlementRequiredError,+ has_entitlement,+ require_entitlement,+)++CODEX_MINIMUM_PLAN_ERROR = "A Max plan or higher is required to use ChatGPT."+logger = logging.getLogger(__name__)+++async def has_codex_access(user_id: str) -> bool:+ return await has_entitlement(user_id, Entitlement.CODEX_SUBSCRIPTION_TRANSPORT)+++async def has_codex_access_for_discovery(user_id: str) -> bool:+ try:+ return await has_codex_access(user_id)+ except Exception:+ logger.warning(+ "Unable to resolve Codex entitlement for user %s; hiding transport",+ user_id,+ exc_info=True,+ )+ return False+++async def enforce_codex_access(user_id: str) -> None:+ try:+ await require_entitlement(+ user_id,+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )+ except EntitlementRequiredError:+ raise UserPaywalledError(CODEX_MINIMUM_PLAN_ERROR) from None+++async def enforce_codex_access_http(user_id: str) -> None:+ try:+ await require_entitlement(+ user_id,+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )+ except EntitlementRequiredError:+ raise HTTPException(+ status_code=status.HTTP_402_PAYMENT_REQUIRED,+ detail=CODEX_MINIMUM_PLAN_ERROR,+ ) from None+ except Exception as exc:+ raise _subscription_state_unavailable() from exc+++def _subscription_state_unavailable() -> HTTPException:+ return HTTPException(+ status_code=status.HTTP_503_SERVICE_UNAVAILABLE,+ detail="Subscription state temporarily unavailable, retry shortly.",+ headers={"Retry-After": "30"},+ )diff --git a/autogpt_platform/backend/backend/integrations/codex/access_test.py b/autogpt_platform/backend/backend/integrations/codex/access_test.pynew file mode 100644index 0000000000..dcef7c6ed5--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/access_test.py@@ -0,0 +1,100 @@+from unittest.mock import AsyncMock, patch++import pytest+from fastapi import HTTPException+from prisma.enums import SubscriptionTier++from backend.copilot.rate_limit import UserPaywalledError+from backend.integrations.codex import access+from backend.util.entitlements import Entitlement, EntitlementRequiredError+++async def test_access_predicate_uses_codex_transport_entitlement():+ with patch.object(+ access,+ "has_entitlement",+ new_callable=AsyncMock,+ return_value=True,+ ) as has_entitlement:+ assert await access.has_codex_access("user-1")++ has_entitlement.assert_awaited_once_with(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )+++async def test_execution_gate_translates_entitlement_error_to_paywall_error():+ denied = EntitlementRequiredError(+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ SubscriptionTier.MAX,+ )+ with patch.object(+ access,+ "require_entitlement",+ new_callable=AsyncMock,+ side_effect=denied,+ ):+ with pytest.raises(UserPaywalledError, match="Max plan"):+ await access.enforce_codex_access("user-1")+++async def test_http_gate_translates_entitlement_error_to_payment_required():+ denied = EntitlementRequiredError(+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ SubscriptionTier.MAX,+ )+ with patch.object(+ access,+ "require_entitlement",+ new_callable=AsyncMock,+ side_effect=denied,+ ):+ with pytest.raises(HTTPException) as exc_info:+ await access.enforce_codex_access_http("user-1")++ assert exc_info.value.status_code == 402+ assert exc_info.value.detail == access.CODEX_MINIMUM_PLAN_ERROR+++async def test_access_gate_propagates_tier_lookup_errors():+ with patch.object(+ access,+ "require_entitlement",+ new_callable=AsyncMock,+ side_effect=RuntimeError("database unavailable"),+ ):+ with pytest.raises(RuntimeError, match="database unavailable"):+ await access.enforce_codex_access("user-1")+++async def test_http_gate_maps_tier_lookup_errors_to_service_unavailable():+ with patch.object(+ access,+ "require_entitlement",+ new_callable=AsyncMock,+ side_effect=RuntimeError("database unavailable"),+ ):+ with pytest.raises(HTTPException) as exc_info:+ await access.enforce_codex_access_http("user-1")++ assert exc_info.value.status_code == 503+ assert exc_info.value.headers == {"Retry-After": "30"}+++async def test_discovery_predicate_hides_codex_on_tier_lookup_errors():+ with patch.object(+ access,+ "has_codex_access",+ new_callable=AsyncMock,+ side_effect=RuntimeError("database unavailable"),+ ):+ allowed = await access.has_codex_access_for_discovery("user-1")++ assert allowed is Falsediff --git a/autogpt_platform/backend/backend/integrations/codex/auth_bundle.py b/autogpt_platform/backend/backend/integrations/codex/auth_bundle.pynew file mode 100644index 0000000000..5e29e21e05--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/auth_bundle.py@@ -0,0 +1,193 @@+import base64+import binascii+import hashlib+import json+import os+from datetime import datetime+from pathlib import Path+from typing import Literal++from pydantic import (+ BaseModel,+ ConfigDict,+ Field,+ SecretStr,+ ValidationError,+ field_validator,+)+++class CodexAuthBundleError(ValueError):+ pass+++class CodexAuthTokensV1(BaseModel):+ model_config = ConfigDict(extra="forbid")++ id_token: SecretStr+ access_token: SecretStr+ refresh_token: SecretStr+ account_id: str | None = None++ @field_validator("id_token", "access_token", "refresh_token")+ @classmethod+ def reject_empty_tokens(cls, value: SecretStr) -> SecretStr:+ if not value.get_secret_value():+ raise ValueError("required token is empty")+ return value+++class CodexAuthBundleV1(BaseModel):+ model_config = ConfigDict(extra="forbid")++ schema_version: Literal[1] = 1+ auth_mode: Literal["chatgpt"] = "chatgpt"+ tokens: CodexAuthTokensV1+ last_refresh: datetime | None = None+ codex_runtime_version: str+++class CodexJwtClaims(BaseModel):+ email: str | None = None+ plan_type: str | None = None+ chatgpt_user_id: str | None = None+ chatgpt_account_id: str | None = None+ expires_at: int | None = None+++class _CodexAuthFileV1(BaseModel):+ model_config = ConfigDict(extra="forbid", populate_by_name=True)++ auth_mode: Literal["chatgpt"]+ openai_api_key: None = Field(default=None, alias="OPENAI_API_KEY")+ tokens: CodexAuthTokensV1+ last_refresh: datetime | None = None+ agent_identity: None = None+ personal_access_token: None = None+ bedrock_api_key: None = None+++class _JwtProfileClaims(BaseModel):+ model_config = ConfigDict(extra="ignore")++ email: str | None = None+++class _JwtAuthClaims(BaseModel):+ model_config = ConfigDict(extra="ignore")++ chatgpt_plan_type: str | None = None+ chatgpt_user_id: str | None = None+ user_id: str | None = None+ chatgpt_account_id: str | None = None+++class _JwtPayload(BaseModel):+ model_config = ConfigDict(extra="ignore", populate_by_name=True)++ email: str | None = None+ expires_at: int | None = Field(default=None, alias="exp")+ profile: _JwtProfileClaims | None = Field(+ default=None,+ alias="https://api.openai.com/profile",+ )+ auth: _JwtAuthClaims | None = Field(+ default=None,+ alias="https://api.openai.com/auth",+ )+++def read_auth_bundle(auth_path: Path, runtime_version: str) -> CodexAuthBundleV1:+ try:+ payload = json.loads(auth_path.read_text(encoding="utf-8"))+ auth_file = _CodexAuthFileV1.model_validate(payload)+ except (OSError, ValueError, ValidationError):+ raise CodexAuthBundleError(+ f"Codex auth file {auth_path.name!r} failed strict ChatGPT validation"+ ) from None++ return CodexAuthBundleV1(+ tokens=auth_file.tokens,+ last_refresh=auth_file.last_refresh,+ codex_runtime_version=runtime_version,+ )+++def encode_provider_state(bundle: CodexAuthBundleV1) -> str:+ return json.dumps(_bundle_payload(bundle), separators=(",", ":"), sort_keys=True)+++def decode_provider_state(provider_state: SecretStr) -> CodexAuthBundleV1:+ try:+ payload = json.loads(provider_state.get_secret_value())+ return CodexAuthBundleV1.model_validate(payload)+ except (binascii.Error, UnicodeDecodeError, ValueError, ValidationError):+ raise CodexAuthBundleError(+ "Encrypted Codex provider state is invalid"+ ) from None+++def materialize_auth_bundle(bundle: CodexAuthBundleV1, auth_path: Path) -> None:+ auth_path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)+ payload = {+ "auth_mode": "chatgpt",+ "OPENAI_API_KEY": None,+ "tokens": _tokens_payload(bundle.tokens),+ "last_refresh": _datetime_payload(bundle.last_refresh),+ }+ encoded = json.dumps(payload, separators=(",", ":"), sort_keys=True).encode()+ descriptor = os.open(auth_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)+ with os.fdopen(descriptor, "wb") as auth_file:+ auth_file.write(encoded)+ auth_file.flush()+ os.fsync(auth_file.fileno())+++def auth_bundle_fingerprint(bundle: CodexAuthBundleV1) -> str:+ return hashlib.sha256(encode_provider_state(bundle).encode()).hexdigest()+++def decode_jwt_claims(token: SecretStr) -> CodexJwtClaims:+ try:+ parts = token.get_secret_value().split(".")+ if len(parts) != 3 or not all(parts):+ raise ValueError+ padding = "=" * (-len(parts[1]) % 4)+ payload = _JwtPayload.model_validate_json(+ base64.urlsafe_b64decode(parts[1] + padding)+ )+ except (ValueError, ValidationError):+ raise CodexAuthBundleError("Codex token has an invalid JWT payload") from None++ auth = payload.auth or _JwtAuthClaims()+ profile = payload.profile or _JwtProfileClaims()+ return CodexJwtClaims(+ email=payload.email or profile.email,+ plan_type=auth.chatgpt_plan_type,+ chatgpt_user_id=auth.chatgpt_user_id or auth.user_id,+ chatgpt_account_id=auth.chatgpt_account_id,+ expires_at=payload.expires_at,+ )+++def _bundle_payload(bundle: CodexAuthBundleV1) -> dict[str, object]:+ return {+ "schema_version": bundle.schema_version,+ "auth_mode": bundle.auth_mode,+ "tokens": _tokens_payload(bundle.tokens),+ "last_refresh": _datetime_payload(bundle.last_refresh),+ "codex_runtime_version": bundle.codex_runtime_version,+ }+++def _tokens_payload(tokens: CodexAuthTokensV1) -> dict[str, str | None]:+ return {+ "id_token": tokens.id_token.get_secret_value(),+ "access_token": tokens.access_token.get_secret_value(),+ "refresh_token": tokens.refresh_token.get_secret_value(),+ "account_id": tokens.account_id,+ }+++def _datetime_payload(value: datetime | None) -> str | None:+ return value.isoformat().replace("+00:00", "Z") if value else Nonediff --git a/autogpt_platform/backend/backend/integrations/codex/auth_bundle_test.py b/autogpt_platform/backend/backend/integrations/codex/auth_bundle_test.pynew file mode 100644index 0000000000..a764a3e54c--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/auth_bundle_test.py@@ -0,0 +1,129 @@+import base64+import json+from pathlib import Path++import pytest+from cryptography.fernet import Fernet+from pydantic import SecretStr++from backend.integrations.codex.auth_bundle import (+ CodexAuthBundleError,+ auth_bundle_fingerprint,+ materialize_auth_bundle,+ read_auth_bundle,+)+from backend.integrations.codex.credential_codec import (+ bundle_from_credentials,+ credentials_from_bundle,+ restore_credentials,+)+from backend.util.encryption import JSONCryptor+++def _jwt(payload: dict[str, object]) -> str:+ encoded = (+ base64.urlsafe_b64encode(json.dumps(payload).encode()).decode().rstrip("=")+ )+ return f"header.{encoded}.signature"+++def _write_auth(path: Path, marker: str) -> tuple[str, str, str]:+ id_token = _jwt(+ {+ "email": f"{marker}@example.com",+ "https://api.openai.com/auth": {"chatgpt_plan_type": "pro"},+ }+ )+ access_token = _jwt({"exp": 2_000_000_000})+ refresh_token = f"refresh-{marker}"+ path.write_text(+ json.dumps(+ {+ "auth_mode": "chatgpt",+ "OPENAI_API_KEY": None,+ "tokens": {+ "id_token": id_token,+ "access_token": access_token,+ "refresh_token": refresh_token,+ "account_id": f"account-{marker}",+ },+ "last_refresh": None,+ }+ ),+ encoding="utf-8",+ )+ return id_token, access_token, refresh_token+++def test_bundle_round_trips_through_oauth_credentials(tmp_path: Path):+ auth_path = tmp_path / "auth.json"+ tokens = _write_auth(auth_path, "alpha")++ bundle = read_auth_bundle(auth_path, "0.144.4")+ credentials = credentials_from_bundle(bundle)++ assert credentials.provider == "codex"+ assert credentials.refresh_strategy == "provider_runtime"+ assert credentials.username == "[email protected]"+ assert credentials.access_token == SecretStr(tokens[1])+ assert bundle_from_credentials(credentials) == bundle+ assert all(token not in repr(credentials) for token in tokens)+++def test_provider_state_round_trips_through_existing_encryption(tmp_path: Path):+ auth_path = tmp_path / "auth.json"+ tokens = _write_auth(auth_path, "alpha")+ credentials = credentials_from_bundle(read_auth_bundle(auth_path, "0.144.4"))+ cryptor = JSONCryptor(Fernet.generate_key().decode())++ encrypted = cryptor.encrypt(credentials.model_dump())+ restored = restore_credentials(cryptor.decrypt(encrypted))++ assert all(token not in encrypted for token in tokens)+ assert restored.id == credentials.id+ assert bundle_from_credentials(restored) == bundle_from_credentials(credentials)+++def test_materialized_bundles_do_not_cross_users(tmp_path: Path):+ source_a = tmp_path / "source-a.json"+ source_b = tmp_path / "source-b.json"+ tokens_a = _write_auth(source_a, "alpha")+ tokens_b = _write_auth(source_b, "bravo")+ bundle_a = read_auth_bundle(source_a, "0.144.4")+ bundle_b = read_auth_bundle(source_b, "0.144.4")+ target_a = tmp_path / "a" / "auth.json"+ target_b = tmp_path / "b" / "auth.json"++ materialize_auth_bundle(bundle_a, target_a)+ materialize_auth_bundle(bundle_b, target_b)++ assert auth_bundle_fingerprint(+ read_auth_bundle(target_a, "0.144.4")+ ) == auth_bundle_fingerprint(bundle_a)+ assert auth_bundle_fingerprint(+ read_auth_bundle(target_b, "0.144.4")+ ) == auth_bundle_fingerprint(bundle_b)+ assert tokens_a[2] not in target_b.read_text(encoding="utf-8")+ assert tokens_b[2] not in target_a.read_text(encoding="utf-8")+++def test_auth_bundle_rejects_api_key_auth(tmp_path: Path):+ auth_path = tmp_path / "auth.json"+ auth_path.write_text(+ json.dumps({"auth_mode": "apiKey", "OPENAI_API_KEY": "secret"}),+ encoding="utf-8",+ )++ with pytest.raises(CodexAuthBundleError):+ read_auth_bundle(auth_path, "0.144.4")+++def test_auth_bundle_rejects_unknown_fields(tmp_path: Path):+ auth_path = tmp_path / "auth.json"+ _write_auth(auth_path, "alpha")+ payload = json.loads(auth_path.read_text(encoding="utf-8"))+ payload["unexpected"] = "secret"+ auth_path.write_text(json.dumps(payload), encoding="utf-8")++ with pytest.raises(CodexAuthBundleError):+ read_auth_bundle(auth_path, "0.144.4")diff --git a/autogpt_platform/backend/backend/integrations/codex/credential_codec.py b/autogpt_platform/backend/backend/integrations/codex/credential_codec.pynew file mode 100644index 0000000000..361f7f2f23--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/credential_codec.py@@ -0,0 +1,75 @@+from pathlib import Path++from pydantic import SecretStr++from backend.data.model import CREDENTIALS_ADAPTER, OAuth2Credentials+from backend.integrations.codex.auth_bundle import (+ CodexAuthBundleError,+ CodexAuthBundleV1,+ decode_jwt_claims,+ decode_provider_state,+ encode_provider_state,+ read_auth_bundle,+)+++def credentials_from_bundle(+ bundle: CodexAuthBundleV1,+ *,+ title: str = "ChatGPT for Codex",+) -> OAuth2Credentials:+ id_claims = decode_jwt_claims(bundle.tokens.id_token)+ access_claims = decode_jwt_claims(bundle.tokens.access_token)+ return OAuth2Credentials(+ provider="codex",+ title=title,+ username=id_claims.email,+ access_token=bundle.tokens.access_token,+ access_token_expires_at=access_claims.expires_at,+ refresh_token=bundle.tokens.refresh_token,+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr(encode_provider_state(bundle)),+ provider_state_version=bundle.schema_version,+ metadata={+ "plan_type": id_claims.plan_type,+ "codex_runtime_version": bundle.codex_runtime_version,+ },+ )+++def bundle_from_credentials(credentials: OAuth2Credentials) -> CodexAuthBundleV1:+ if credentials.provider != "codex":+ raise CodexAuthBundleError("Credential provider is not Codex")+ if credentials.refresh_strategy != "provider_runtime":+ raise CodexAuthBundleError("Codex credential refresh ownership is invalid")+ if credentials.provider_state_version != 1 or credentials.provider_state is None:+ raise CodexAuthBundleError("Codex credential provider state is unavailable")+ return decode_provider_state(credentials.provider_state)+++def checkpoint_credentials(+ credentials: OAuth2Credentials,+ auth_path: Path,+ runtime_version: str,+) -> OAuth2Credentials:+ bundle = read_auth_bundle(auth_path, runtime_version)+ return checkpoint_credentials_from_bundle(credentials, bundle)+++def checkpoint_credentials_from_bundle(+ credentials: OAuth2Credentials,+ bundle: CodexAuthBundleV1,+) -> OAuth2Credentials:+ replacement = credentials_from_bundle(bundle, title=credentials.title or "")+ payload = replacement.model_dump(exclude={"id", "is_managed"})+ return OAuth2Credentials.model_validate(+ credentials.model_copy(update=payload).model_dump()+ )+++def restore_credentials(payload: dict[str, object]) -> OAuth2Credentials:+ restored = CREDENTIALS_ADAPTER.validate_python(payload)+ if not isinstance(restored, OAuth2Credentials):+ raise CodexAuthBundleError("Codex credential type changed")+ return restoreddiff --git a/autogpt_platform/backend/backend/integrations/codex/launcher.py b/autogpt_platform/backend/backend/integrations/codex/launcher.pynew file mode 100644index 0000000000..b62b69dc77--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/launcher.py@@ -0,0 +1,76 @@+import os+import subprocess+import sys+from collections.abc import Mapping++_PASSTHROUGH = frozenset(+ {+ "APPDATA",+ "CODEX_HOME",+ "CODEX_SQLITE_HOME",+ "COMSPEC",+ "HOME",+ "HTTPS_PROXY",+ "HTTP_PROXY",+ "LANG",+ "LC_ALL",+ "LOCALAPPDATA",+ "NO_COLOR",+ "NO_PROXY",+ "NUMBER_OF_PROCESSORS",+ "OS",+ "PATH",+ "PATHEXT",+ "PROCESSOR_ARCHITECTURE",+ "PROGRAMDATA",+ "PROGRAMFILES",+ "PROGRAMFILES(X86)",+ "RUST_LOG",+ "SSL_CERT_DIR",+ "SSL_CERT_FILE",+ "SYSTEMROOT",+ "SYSTEMDRIVE",+ "TEMP",+ "TERM",+ "TMP",+ "TMPDIR",+ "USERPROFILE",+ "WINDIR",+ "XDG_CACHE_HOME",+ "XDG_CONFIG_HOME",+ "XDG_DATA_HOME",+ }+)+++def sanitized_environment(source: Mapping[str, str] | None = None) -> dict[str, str]:+ ambient = os.environ if source is None else source+ child = {+ key: value for key, value in ambient.items() if key.upper() in _PASSTHROUGH+ }+ child.update(+ {+ "CODEX_ACCESS_TOKEN": "",+ "CODEX_API_KEY": "",+ "OPENAI_API_KEY": "",+ }+ )+ return child+++def main() -> None:+ if len(sys.argv) < 2:+ raise SystemExit("Codex launcher requires the runtime path")+ runtime, *arguments = sys.argv[1:]+ if os.name == "nt":+ completed = subprocess.run(+ [runtime, *arguments],+ env=sanitized_environment(),+ check=False,+ )+ raise SystemExit(completed.returncode)+ os.execve(runtime, [runtime, *arguments], sanitized_environment())+++if __name__ == "__main__":+ main()diff --git a/autogpt_platform/backend/backend/integrations/codex/launcher_test.py b/autogpt_platform/backend/backend/integrations/codex/launcher_test.pynew file mode 100644index 0000000000..3a1a0e5c12--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/launcher_test.py@@ -0,0 +1,31 @@+import os++from backend.integrations.codex.launcher import sanitized_environment+++def test_launcher_drops_ambient_backend_secrets(monkeypatch):+ monkeypatch.setenv("DATABASE_URL", "postgresql://secret")+ monkeypatch.setenv("ENCRYPTION_KEY", "encryption-secret")+ monkeypatch.setenv("OPENAI_API_KEY", "provider-secret")+ monkeypatch.setenv("CODEX_HOME", "/tmp/codex-user")+ monkeypatch.setenv("APPDATA", "/tmp/codex-user")+ monkeypatch.setenv("PATH", os.environ.get("PATH", ""))++ child_env = sanitized_environment()++ assert child_env["CODEX_HOME"] == "/tmp/codex-user"+ assert child_env["APPDATA"] == "/tmp/codex-user"+ assert "PATH" in child_env+ assert "DATABASE_URL" not in child_env+ assert "ENCRYPTION_KEY" not in child_env+ assert child_env["OPENAI_API_KEY"] == ""+++def test_launcher_respects_an_explicit_empty_environment():+ child_env = sanitized_environment({})++ assert child_env == {+ "CODEX_ACCESS_TOKEN": "",+ "CODEX_API_KEY": "",+ "OPENAI_API_KEY": "",+ }diff --git a/autogpt_platform/backend/backend/integrations/codex/login.py b/autogpt_platform/backend/backend/integrations/codex/login.pynew file mode 100644index 0000000000..5c277df4e7--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/login.py@@ -0,0 +1,589 @@+import asyncio+import hashlib+import logging+import secrets+from collections.abc import AsyncIterator, Awaitable, Callable+from contextlib import AbstractAsyncContextManager, asynccontextmanager, suppress+from typing import Literal, Protocol, TypeGuard, TypeVar++from autogpt_libs.utils.synchronize import AsyncRedisKeyedMutex+from pydantic import BaseModel, ConfigDict++from backend.data.model import OAuth2Credentials+from backend.data.redis_client import get_redis_async+from backend.integrations.codex.access import enforce_codex_access+from backend.integrations.codex.credential_codec import credentials_from_bundle+from backend.integrations.codex.models import (+ CodexDeviceCodeDetails,+ CodexLoginCompletion,+)+from backend.integrations.creds_manager import IntegrationCredentialsManager++logger = logging.getLogger(__name__)++CodexDeviceLogin = CodexDeviceCodeDetails+CodexLoginStatus = Literal["pending", "completed", "failed", "canceled"]++_COMPARE_DELETE_SCRIPT = """+if redis.call('get', KEYS[1]) == ARGV[1] then+ return redis.call('del', KEYS[1])+end+return 0+"""+_COMPARE_EXPIRE_SCRIPT = """+if redis.call('get', KEYS[1]) == ARGV[1] then+ return redis.call('expire', KEYS[1], ARGV[2])+end+return 0+"""++_T = TypeVar("_T")+++async def _await_redis_result(result: Awaitable[_T] | _T) -> _T:+ if isinstance(result, Awaitable):+ return await result+ return result+++class CodexDeviceLoginSession(Protocol):+ details: CodexDeviceCodeDetails++ async def wait(self) -> CodexLoginCompletion: ...++ async def cancel(self) -> None: ...++ async def close(self) -> None: ...+++class CodexLoginTransport(Protocol):+ async def start_device_login(self) -> CodexDeviceLoginSession: ...+++class CodexLoginPendingError(RuntimeError):+ pass+++class CodexLoginFailedError(RuntimeError):+ pass+++class CodexDeviceLoginState(BaseModel):+ model_config = ConfigDict(extra="forbid")++ status: CodexLoginStatus+ error: str | None = None+++class CodexSharedLoginState(CodexDeviceLoginState):+ user_id: str+ credential_id: str | None = None+++class CodexLoginStateStore(Protocol):+ def locked(self, user_id: str) -> AbstractAsyncContextManager[None]: ...++ async def claim(self, state: CodexSharedLoginState, login_id: str) -> bool: ...++ async def get_active(self, user_id: str) -> str | None: ...++ async def get(+ self, user_id: str, login_id: str+ ) -> CodexSharedLoginState | None: ...++ async def write(self, state: CodexSharedLoginState, login_id: str) -> None: ...++ async def release_active(self, user_id: str, login_id: str) -> None: ...++ async def refresh_active(self, user_id: str, login_id: str) -> bool: ...++ async def delete(self, user_id: str, login_id: str) -> None: ...+++class RedisCodexLoginStateStore:+ def __init__(self, ttl_seconds: int, owner_lease_seconds: int = 30) -> None:+ self._ttl_seconds = ttl_seconds+ self._owner_lease_seconds = owner_lease_seconds++ @asynccontextmanager+ async def locked(self, user_id: str) -> AsyncIterator[None]:+ mutex = AsyncRedisKeyedMutex(await get_redis_async())+ async with mutex.locked(_lock_key(user_id)):+ yield++ async def claim(self, state: CodexSharedLoginState, login_id: str) -> bool:+ redis = await get_redis_async()+ claimed = await redis.set(+ _active_key(state.user_id),+ login_id,+ nx=True,+ ex=self._owner_lease_seconds,+ )+ if not claimed:+ return False+ try:+ await self.write(state, login_id)+ except Exception:+ await _await_redis_result(+ redis.eval(+ _COMPARE_DELETE_SCRIPT,+ 1,+ _active_key(state.user_id),+ login_id,+ )+ )+ raise+ return True++ async def get_active(self, user_id: str) -> str | None:+ redis = await get_redis_async()+ return await redis.get(_active_key(user_id))++ async def get(self, user_id: str, login_id: str) -> CodexSharedLoginState | None:+ redis = await get_redis_async()+ payload = await redis.get(_attempt_key(user_id, login_id))+ if payload is None:+ return None+ state = CodexSharedLoginState.model_validate_json(payload)+ return state if state.user_id == user_id else None++ async def write(self, state: CodexSharedLoginState, login_id: str) -> None:+ redis = await get_redis_async()+ await redis.set(+ _attempt_key(state.user_id, login_id),+ state.model_dump_json(),+ ex=self._ttl_seconds,+ )++ async def release_active(self, user_id: str, login_id: str) -> None:+ redis = await get_redis_async()+ await _await_redis_result(+ redis.eval(+ _COMPARE_DELETE_SCRIPT,+ 1,+ _active_key(user_id),+ login_id,+ )+ )++ async def refresh_active(self, user_id: str, login_id: str) -> bool:+ redis = await get_redis_async()+ return bool(+ await _await_redis_result(+ redis.eval(+ _COMPARE_EXPIRE_SCRIPT,+ 1,+ _active_key(user_id),+ login_id,+ str(self._owner_lease_seconds),+ )+ )+ )++ async def delete(self, user_id: str, login_id: str) -> None:+ redis = await get_redis_async()+ await redis.delete(_attempt_key(user_id, login_id))+++class CodexLoginCoordinator:+ def __init__(+ self,+ transport_factory: Callable[[], CodexLoginTransport] | None = None,+ timeout_seconds: float = 900,+ state_store: CodexLoginStateStore | None = None,+ credentials_manager: IntegrationCredentialsManager | None = None,+ state_poll_interval_seconds: float = 1,+ ) -> None:+ self._transport_factory = transport_factory or _get_default_transport+ self._timeout_seconds = timeout_seconds+ self._state_store = state_store or RedisCodexLoginStateStore(+ int(timeout_seconds) + 300+ )+ self._credentials_manager = (+ credentials_manager or IntegrationCredentialsManager()+ )+ self._state_poll_interval_seconds = state_poll_interval_seconds+ self._local_sessions: dict[str, CodexDeviceLoginSession] = {}+ self._local_tasks: dict[str, asyncio.Task[None]] = {}+ self._local_cancel_events: dict[str, asyncio.Event] = {}+ self._local_users: dict[str, str] = {}+ self._startup_tasks: dict[str, asyncio.Task[object]] = {}+ self._startup_users: dict[str, str] = {}++ async def start(self, user_id: str) -> CodexDeviceLogin:+ login_id = secrets.token_urlsafe(24)+ state = CodexSharedLoginState(user_id=user_id, status="pending")+ async with self._state_store.locked(user_id):+ active_login_id = await self._state_store.get_active(user_id)+ if active_login_id is not None:+ active_state = await self._state_store.get(user_id, active_login_id)+ if active_state is not None and active_state.status == "pending":+ active_state.status = "canceled"+ active_state.error = (+ "ChatGPT sign-in was replaced by a newer attempt"+ )+ await self._state_store.write(active_state, active_login_id)+ await self._state_store.release_active(user_id, active_login_id)+ if not await self._state_store.claim(state, login_id):+ raise CodexLoginPendingError(+ "A ChatGPT sign-in is already active for this user"+ )+ current_task = asyncio.current_task()+ if current_task is not None:+ self._startup_tasks[login_id] = current_task+ self._startup_users[login_id] = user_id+ startup_lease = asyncio.create_task(+ self._keep_owner_lease_alive(user_id, login_id)+ )+ session: CodexDeviceLoginSession | None = None+ try:+ session = await self._transport_factory().start_device_login()+ async with self._state_store.locked(user_id):+ current = await self._state_store.get(user_id, login_id)+ owns_login = (+ current is not None+ and current.status == "pending"+ and await self._state_store.refresh_active(user_id, login_id)+ )+ if not owns_login:+ raise CodexLoginFailedError("Codex login ownership was lost")++ self._local_sessions[login_id] = session+ self._local_users[login_id] = user_id+ cancel_event = asyncio.Event()+ self._local_cancel_events[login_id] = cancel_event+ self._local_tasks[login_id] = asyncio.create_task(+ self._wait_for_login(user_id, login_id, session, cancel_event)+ )+ return session.details.model_copy(update={"login_id": login_id})+ except BaseException:+ try:+ if session is not None:+ await session.close()+ finally:+ async with self._state_store.locked(user_id):+ await self._state_store.release_active(user_id, login_id)+ await self._state_store.delete(user_id, login_id)+ self._local_sessions.pop(login_id, None)+ self._local_tasks.pop(login_id, None)+ self._local_cancel_events.pop(login_id, None)+ self._local_users.pop(login_id, None)+ raise+ finally:+ startup_lease.cancel()+ with suppress(asyncio.CancelledError):+ await startup_lease+ self._startup_tasks.pop(login_id, None)+ self._startup_users.pop(login_id, None)++ async def get(self, user_id: str, login_id: str) -> CodexDeviceLoginState | None:+ state = await self._state_store.get(user_id, login_id)+ if state is None:+ return None+ return CodexDeviceLoginState(status=state.status, error=state.error)++ async def complete(self, user_id: str, login_id: str) -> OAuth2Credentials:+ async with self._state_store.locked(user_id):+ state = await self._state_store.get(user_id, login_id)+ if state is None:+ raise CodexLoginFailedError("Codex login not found")+ if state.status == "pending":+ raise CodexLoginPendingError("Codex login is still pending")+ if state.status != "completed" or state.credential_id is None:+ raise CodexLoginFailedError(state.error or "Codex login failed")+ credentials = await self._credentials_manager.store.get_creds_by_id(+ user_id, state.credential_id+ )+ if not _is_codex_credentials(credentials):+ raise CodexLoginFailedError("Stored Codex credential is unavailable")+ await self._state_store.delete(user_id, login_id)+ return credentials.model_copy(deep=True)++ async def cancel(self, user_id: str, login_id: str) -> bool:+ found, should_cancel = await self._mark_canceled(+ user_id, login_id, "ChatGPT sign-in was canceled"+ )+ if not found:+ return False+ session = self._local_sessions.get(login_id)+ if should_cancel and session is not None:+ cancel_event = self._local_cancel_events.get(login_id)+ if cancel_event is not None:+ cancel_event.set()+ await session.cancel()+ return True++ async def shutdown(self) -> None:+ starting_logins = [+ (login_id, user_id, task)+ for login_id, task in self._startup_tasks.items()+ if (user_id := self._startup_users.get(login_id)) is not None+ ]+ for login_id, user_id, task in starting_logins:+ await self._mark_canceled(+ user_id,+ login_id,+ "ChatGPT sign-in was interrupted by service shutdown",+ )+ if task is not asyncio.current_task():+ task.cancel()+ await asyncio.gather(+ *(+ task+ for _, _, task in starting_logins+ if task is not asyncio.current_task()+ ),+ return_exceptions=True,+ )++ local_logins = [+ (login_id, user_id, session)+ for login_id, session in self._local_sessions.items()+ if (user_id := self._local_users.get(login_id)) is not None+ ]+ tasks = [+ task+ for login_id, _, _ in local_logins+ if (task := self._local_tasks.get(login_id)) is not None+ ]++ for login_id, user_id, _ in local_logins:+ await self._mark_canceled(+ user_id,+ login_id,+ "ChatGPT sign-in was interrupted by service shutdown",+ )+ cancel_event = self._local_cancel_events.get(login_id)+ if cancel_event is not None:+ cancel_event.set()++ await asyncio.gather(+ *(session.cancel() for _, _, session in local_logins),+ return_exceptions=True,+ )+ await asyncio.gather(*tasks, return_exceptions=True)++ async def _mark_canceled(+ self,+ user_id: str,+ login_id: str,+ message: str,+ ) -> tuple[bool, bool]:+ should_cancel = False+ async with self._state_store.locked(user_id):+ state = await self._state_store.get(user_id, login_id)+ if state is None:+ return False, False+ if state.status == "pending":+ state.status = "canceled"+ state.error = message+ await self._state_store.write(state, login_id)+ should_cancel = True+ return True, should_cancel++ async def _wait_for_login(+ self,+ user_id: str,+ login_id: str,+ session: CodexDeviceLoginSession,+ cancel_event: asyncio.Event,+ ) -> None:+ login_task = asyncio.create_task(session.wait())+ state_task = asyncio.create_task(+ self._wait_until_not_pending(user_id, login_id, cancel_event)+ )+ persist_task: asyncio.Task[None] | None = None+ try:+ done, _ = await asyncio.wait(+ {login_task, state_task},+ timeout=self._timeout_seconds,+ return_when=asyncio.FIRST_COMPLETED,+ )+ if not done:+ await session.cancel()+ login_task.cancel()+ await self._mark_failed(user_id, login_id, "ChatGPT sign-in timed out")+ elif login_task in done:+ completion = await login_task+ persist_task = asyncio.create_task(+ self._persist_completed_login(user_id, login_id, completion)+ )+ persisted, _ = await asyncio.wait(+ {persist_task, state_task},+ return_when=asyncio.FIRST_COMPLETED,+ )+ if persist_task in persisted:+ await persist_task+ else:+ persist_task.cancel()+ else:+ if not state_task.result():+ await session.cancel()+ login_task.cancel()+ except asyncio.TimeoutError:+ await session.cancel()+ await self._mark_failed(user_id, login_id, "ChatGPT sign-in timed out")+ except asyncio.CancelledError:+ state = await self._state_store.get(user_id, login_id)+ if state is not None and state.status == "pending":+ with suppress(Exception):+ await asyncio.shield(session.cancel())+ raise+ except Exception as error:+ logger.warning("Codex device login failed with %s", type(error).__name__)+ await self._mark_failed(+ user_id, login_id, "ChatGPT sign-in failed. Try again."+ )+ finally:+ child_tasks = [login_task, state_task]+ if persist_task is not None:+ child_tasks.append(persist_task)+ for child_task in child_tasks:+ if not child_task.done():+ child_task.cancel()+ await asyncio.gather(*child_tasks, return_exceptions=True)+ try:+ await session.close()+ finally:+ try:+ async with self._state_store.locked(user_id):+ await self._state_store.release_active(user_id, login_id)+ finally:+ self._local_sessions.pop(login_id, None)+ self._local_tasks.pop(login_id, None)+ self._local_cancel_events.pop(login_id, None)+ self._local_users.pop(login_id, None)++ async def _wait_until_not_pending(+ self,+ user_id: str,+ login_id: str,+ cancel_event: asyncio.Event,+ ) -> bool:+ while True:+ if cancel_event.is_set():+ return True+ async with self._state_store.locked(user_id):+ state = await self._state_store.get(user_id, login_id)+ if state is None or state.status != "pending":+ return False+ if not await self._state_store.refresh_active(user_id, login_id):+ state.status = "failed"+ state.error = "ChatGPT sign-in ownership was lost. Try again."+ await self._state_store.write(state, login_id)+ return False+ try:+ await asyncio.wait_for(+ cancel_event.wait(),+ timeout=self._state_poll_interval_seconds,+ )+ except asyncio.TimeoutError:+ pass++ async def _keep_owner_lease_alive(self, user_id: str, login_id: str) -> None:+ while True:+ await asyncio.sleep(self._state_poll_interval_seconds)+ async with self._state_store.locked(user_id):+ state = await self._state_store.get(user_id, login_id)+ if state is None or state.status != "pending":+ return+ if not await self._state_store.refresh_active(user_id, login_id):+ state.status = "failed"+ state.error = "ChatGPT sign-in ownership was lost. Try again."+ await self._state_store.write(state, login_id)+ return++ async def _persist_completed_login(+ self,+ user_id: str,+ login_id: str,+ completion: CodexLoginCompletion,+ ) -> None:+ await enforce_codex_access(user_id)+ credentials = credentials_from_bundle(completion.bundle)+ async with self._credentials_manager.locked_provider_credentials(+ user_id, credentials.provider+ ):+ async with self._state_store.locked(user_id):+ state = await self._state_store.get(user_id, login_id)+ if state is None or state.status != "pending":+ return+ if not await self._state_store.refresh_active(user_id, login_id):+ state.status = "failed"+ state.error = "ChatGPT sign-in ownership was lost. Try again."+ await self._state_store.write(state, login_id)+ return+ commit = asyncio.create_task(+ self._commit_completed_login(+ user_id,+ login_id,+ state,+ credentials,+ )+ )+ try:+ await asyncio.shield(commit)+ except asyncio.CancelledError:+ await commit+ raise++ async def _commit_completed_login(+ self,+ user_id: str,+ login_id: str,+ state: CodexSharedLoginState,+ credentials: OAuth2Credentials,+ ) -> None:+ stored = await self._credentials_manager.upsert_single_provider_locked(+ user_id, credentials+ )+ if not _is_codex_credentials(stored):+ raise RuntimeError("Stored Codex credential type changed")+ state.status = "completed"+ state.credential_id = stored.id+ await self._state_store.write(state, login_id)++ async def _mark_failed(self, user_id: str, login_id: str, message: str) -> None:+ async with self._state_store.locked(user_id):+ state = await self._state_store.get(user_id, login_id)+ if state is None or state.status != "pending":+ return+ state.status = "failed"+ state.error = message+ await self._state_store.write(state, login_id)+++def _is_codex_credentials(+ credentials: object,+) -> TypeGuard[OAuth2Credentials]:+ return (+ isinstance(credentials, OAuth2Credentials)+ and credentials.provider == "codex"+ and credentials.refresh_strategy == "provider_runtime"+ )+++def _user_key(user_id: str) -> str:+ return hashlib.sha256(user_id.encode()).hexdigest()[:24]+++def _active_key(user_id: str) -> str:+ slot = _user_key(user_id)+ return f"codex:login:{{{slot}}}:active"+++def _lock_key(user_id: str) -> str:+ slot = _user_key(user_id)+ return f"codex:login:{{{slot}}}:lock"+++def _attempt_key(user_id: str, login_id: str) -> str:+ slot = _user_key(user_id)+ attempt = hashlib.sha256(login_id.encode()).hexdigest()[:24]+ return f"codex:login:{{{slot}}}:attempt:{attempt}"+++def _get_default_transport() -> CodexLoginTransport:+ from backend.integrations.codex.transport import get_codex_transport++ return get_codex_transport()diff --git a/autogpt_platform/backend/backend/integrations/codex/login_test.py b/autogpt_platform/backend/backend/integrations/codex/login_test.pynew file mode 100644index 0000000000..285e8ace65--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/login_test.py@@ -0,0 +1,697 @@+import asyncio+import base64+import json+from collections.abc import AsyncIterator+from contextlib import asynccontextmanager+from typing import cast+from unittest.mock import AsyncMock, MagicMock, patch++import pytest+from pydantic import SecretStr++from backend.data.model import OAuth2Credentials+from backend.integrations.codex.auth_bundle import CodexAuthBundleV1, CodexAuthTokensV1+from backend.integrations.codex.login import (+ CodexDeviceLoginState,+ CodexLoginCoordinator,+ CodexLoginFailedError,+ CodexLoginStatus,+ CodexLoginTransport,+ CodexSharedLoginState,+ RedisCodexLoginStateStore,+)+from backend.integrations.codex.models import (+ CodexAccountSnapshot,+ CodexDeviceCodeDetails,+ CodexLoginCompletion,+ CodexRateLimitsSnapshot,+)+++class FakeSession:+ def __init__(self) -> None:+ self.details = CodexDeviceCodeDetails(+ login_id="login-123",+ verification_url="https://auth.openai.com/codex/device",+ user_code="ABCD-EFGH",+ )+ self.result = asyncio.get_running_loop().create_future()+ self.cancel = AsyncMock()+ self.close = AsyncMock()++ async def wait(self) -> CodexLoginCompletion:+ return await self.result+++class MemoryStateStore:+ def __init__(self) -> None:+ self.active: dict[str, str] = {}+ self.states: dict[tuple[str, str], CodexSharedLoginState] = {}+ self.lock = asyncio.Lock()+ self.refresh_count = 0++ @asynccontextmanager+ async def locked(self, _user_id: str) -> AsyncIterator[None]:+ async with self.lock:+ yield++ async def claim(self, state: CodexSharedLoginState, login_id: str) -> bool:+ if state.user_id in self.active:+ return False+ self.active[state.user_id] = login_id+ await self.write(state, login_id)+ return True++ async def get_active(self, user_id: str) -> str | None:+ return self.active.get(user_id)++ async def get(self, user_id: str, login_id: str) -> CodexSharedLoginState | None:+ state = self.states.get((user_id, login_id))+ return state.model_copy(deep=True) if state else None++ async def write(self, state: CodexSharedLoginState, login_id: str) -> None:+ self.states[(state.user_id, login_id)] = state.model_copy(deep=True)++ async def release_active(self, user_id: str, login_id: str) -> None:+ if self.active.get(user_id) == login_id:+ del self.active[user_id]++ async def refresh_active(self, user_id: str, login_id: str) -> bool:+ if self.active.get(user_id) != login_id:+ return False+ self.refresh_count += 1+ return True++ async def delete(self, user_id: str, login_id: str) -> None:+ self.states.pop((user_id, login_id), None)+++class FakeRedis:+ def __init__(self) -> None:+ self.values: dict[str, str] = {}+ self.expirations: dict[str, int] = {}++ async def set(+ self,+ key: str,+ value: str,+ *,+ nx: bool = False,+ ex: int | None = None,+ ) -> bool:+ if nx and key in self.values:+ return False+ self.values[key] = value+ if ex is not None:+ self.expirations[key] = ex+ return True++ async def get(self, key: str) -> str | None:+ return self.values.get(key)++ async def delete(self, key: str) -> None:+ self.values.pop(key, None)++ async def expire(self, key: str, seconds: int) -> bool:+ if key not in self.values:+ return False+ self.expirations[key] = seconds+ return True++ async def eval(+ self,+ script: str,+ _numkeys: int,+ key: str,+ expected: str,+ *arguments: object,+ ) -> int:+ if self.values.get(key) != expected:+ return 0+ if "redis.call('del'" in script:+ self.values.pop(key, None)+ self.expirations.pop(key, None)+ return 1+ if "redis.call('expire'" in script:+ self.expirations[key] = int(arguments[0])+ return 1+ raise AssertionError("Unexpected Redis script")+++async def test_new_start_supersedes_distributed_login_before_requesting_device_code():+ user_id = "user-123"+ first_session = FakeSession()+ second_session = FakeSession()+ state_store = MemoryStateStore()+ transport_started = asyncio.Event()+ allow_transport = asyncio.Event()++ async def start_transport() -> FakeSession:+ transport_started.set()+ await allow_transport.wait()+ return first_session++ first_transport = MagicMock()+ first_transport.start_device_login = AsyncMock(side_effect=start_transport)+ replacement_transport = MagicMock()+ replacement_transport.start_device_login = AsyncMock(return_value=second_session)+ origin = CodexLoginCoordinator(+ transport_factory=lambda: first_transport,+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )+ other_replica = CodexLoginCoordinator(+ transport_factory=lambda: replacement_transport,+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ start_task = asyncio.create_task(origin.start(user_id))+ await asyncio.wait_for(transport_started.wait(), timeout=1)++ replacement = await other_replica.start(user_id)+ replacement_transport.start_device_login.assert_awaited_once()+ assert state_store.active[user_id] == replacement.login_id++ allow_transport.set()+ with pytest.raises(CodexLoginFailedError, match="ownership was lost"):+ await start_task+ first_session.close.assert_awaited()+ await origin.shutdown()+ await other_replica.shutdown()+++async def test_new_start_cancels_established_cross_replica_login_without_releasing_replacement():+ user_id = "user-123"+ first_session = FakeSession()+ second_session = FakeSession()+ state_store = MemoryStateStore()+ origin = CodexLoginCoordinator(+ transport_factory=lambda: _transport(first_session),+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )+ other_replica = CodexLoginCoordinator(+ transport_factory=lambda: _transport(second_session),+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ original = await origin.start(user_id)+ original_task = origin._local_tasks[original.login_id]+ replacement = await other_replica.start(user_id)++ original_state = await origin.get(user_id, original.login_id)+ assert original_state is not None+ assert original_state.status == "canceled"+ assert original_state.error == "ChatGPT sign-in was replaced by a newer attempt"+ assert state_store.active[user_id] == replacement.login_id++ await asyncio.wait_for(original_task, timeout=1)+ first_session.cancel.assert_awaited_once()+ first_session.close.assert_awaited_once()+ assert state_store.active[user_id] == replacement.login_id++ await origin.shutdown()+ await other_replica.shutdown()+++async def test_failed_transport_start_rolls_back_provisional_claim():+ user_id = "user-123"+ state_store = MemoryStateStore()+ failing_transport = MagicMock()+ failing_transport.start_device_login = AsyncMock(+ side_effect=RuntimeError("startup failed")+ )+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: failing_transport,+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ with pytest.raises(RuntimeError, match="startup failed"):+ await coordinator.start(user_id)++ assert user_id not in state_store.active+ assert not state_store.states+++async def test_shutdown_cancels_in_progress_transport_start_and_rolls_back_claim():+ user_id = "user-123"+ state_store = MemoryStateStore()+ transport_started = asyncio.Event()+ transport_canceled = asyncio.Event()++ async def start_transport() -> FakeSession:+ transport_started.set()+ try:+ await asyncio.Future()+ finally:+ transport_canceled.set()+ raise AssertionError("unreachable")++ transport = MagicMock()+ transport.start_device_login = AsyncMock(side_effect=start_transport)+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: transport,+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ start_task = asyncio.create_task(coordinator.start(user_id))+ await asyncio.wait_for(transport_started.wait(), timeout=1)+ await coordinator.shutdown()++ assert start_task.cancelled()+ assert transport_canceled.is_set()+ assert user_id not in state_store.active+ assert not state_store.states+++async def test_completion_persists_before_cross_replica_callback():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ credentials = _credentials()+ manager = MagicMock()+ manager.locked_provider_credentials = MagicMock(+ side_effect=lambda *_args: _provider_locks()+ )+ manager.upsert_single_provider_locked = AsyncMock(return_value=credentials)+ manager.store.get_creds_by_id = AsyncMock(return_value=credentials)+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ )++ details = await coordinator.start(user_id)+ session.result.set_result(_completion())+ await _wait_for_status(coordinator, user_id, details.login_id, "completed")+ await _wait_for_close(session)+ await _wait_for_active_release(state_store, user_id)++ manager.upsert_single_provider_locked.assert_awaited_once()+ second_replica = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ )+ restored = await second_replica.complete(user_id, details.login_id)++ assert restored.id == credentials.id+ assert await state_store.get(user_id, details.login_id) is None+ assert session.close.await_count == 1+++async def test_cross_replica_cancel_prevents_origin_from_persisting():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ manager = MagicMock()+ manager.locked_provider_credentials = MagicMock(+ side_effect=lambda *_args: _provider_locks()+ )+ manager.upsert_single_provider_locked = AsyncMock(return_value=_credentials())+ origin = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ state_poll_interval_seconds=0.001,+ )+ other_replica = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ )++ details = await origin.start(user_id)+ assert await other_replica.cancel(user_id, details.login_id)+ assert state_store.active[user_id] == details.login_id+ await _wait_for_status(origin, user_id, details.login_id, "canceled")+ await _wait_for_close(session)+ await _wait_for_active_release(state_store, user_id)++ assert user_id not in state_store.active+ session.cancel.assert_awaited()+ manager.upsert_single_provider_locked.assert_not_awaited()+++async def test_same_replica_cancel_promptly_releases_slot_after_owner_cleanup():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ details = await coordinator.start(user_id)+ assert await coordinator.cancel(user_id, details.login_id)+ await _wait_for_close(session)+ await _wait_for_active_release(state_store, user_id)++ assert user_id not in state_store.active+ session.cancel.assert_awaited_once()+++async def test_repeated_cancel_only_cancels_runtime_session_once():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ details = await coordinator.start(user_id)+ assert await coordinator.cancel(user_id, details.login_id)+ assert await coordinator.cancel(user_id, details.login_id)+ await _wait_for_close(session)+ await _wait_for_active_release(state_store, user_id)++ session.cancel.assert_awaited_once()+++async def test_shutdown_cancels_local_sessions_and_releases_active_slots():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=MagicMock(),+ state_poll_interval_seconds=0.001,+ )++ details = await coordinator.start(user_id)+ await coordinator.shutdown()++ state = await coordinator.get(user_id, details.login_id)+ assert state is not None+ assert state.status == "canceled"+ assert state.error == "ChatGPT sign-in was interrupted by service shutdown"+ assert user_id not in state_store.active+ session.cancel.assert_awaited_once()+ session.close.assert_awaited_once()+++async def test_cancel_during_provider_lock_wait_prevents_stale_persistence():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ provider_lock_entered = asyncio.Event()+ release_provider_lock = asyncio.Event()+ manager = MagicMock()++ @asynccontextmanager+ async def blocked_provider_locks() -> AsyncIterator[None]:+ provider_lock_entered.set()+ await release_provider_lock.wait()+ yield++ manager.locked_provider_credentials = MagicMock(+ side_effect=lambda *_args: blocked_provider_locks()+ )+ manager.upsert_single_provider_locked = AsyncMock(return_value=_credentials())+ origin = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ state_poll_interval_seconds=0.001,+ )+ other_replica = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ )++ details = await origin.start(user_id)+ session.result.set_result(_completion())+ await asyncio.wait_for(provider_lock_entered.wait(), timeout=1)++ assert await asyncio.wait_for(+ other_replica.cancel(user_id, details.login_id), timeout=1+ )+ await _wait_for_close(session)+ await _wait_for_active_release(state_store, user_id)+ release_provider_lock.set()++ manager.upsert_single_provider_locked.assert_not_awaited()+ assert user_id not in state_store.active+++async def test_cancel_during_credential_upsert_finishes_state_commit():+ user_id = "user-123"+ login_id = "login-123"+ state_store = MemoryStateStore()+ state = CodexSharedLoginState(user_id=user_id, status="pending")+ assert await state_store.claim(state, login_id)+ upsert_started = asyncio.Event()+ allow_upsert = asyncio.Event()+ manager = MagicMock()+ manager.locked_provider_credentials = MagicMock(+ side_effect=lambda *_args: _provider_locks()+ )++ async def blocked_upsert(*_args):+ upsert_started.set()+ await allow_upsert.wait()+ return _credentials()++ manager.upsert_single_provider_locked = AsyncMock(side_effect=blocked_upsert)+ coordinator = CodexLoginCoordinator(+ state_store=state_store,+ credentials_manager=manager,+ )++ persistence = asyncio.create_task(+ coordinator._persist_completed_login(user_id, login_id, _completion())+ )+ await asyncio.wait_for(upsert_started.wait(), timeout=1)+ persistence.cancel()+ allow_upsert.set()++ with pytest.raises(asyncio.CancelledError):+ await persistence++ committed = await state_store.get(user_id, login_id)+ assert committed is not None+ assert committed.status == "completed"+ assert committed.credential_id == "codex-credential"+ manager.upsert_single_provider_locked.assert_awaited_once()+++async def test_expired_owner_cannot_persist_after_a_new_login_claims_slot():+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ manager = MagicMock()+ manager.locked_provider_credentials = MagicMock(+ side_effect=lambda *_args: _provider_locks()+ )+ manager.upsert_single_provider_locked = AsyncMock(return_value=_credentials())+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ state_poll_interval_seconds=0.001,+ )++ details = await coordinator.start(user_id)+ state_store.active[user_id] = "new-owner-login"+ session.result.set_result(_completion())+ state = await _wait_for_status(coordinator, user_id, details.login_id, "failed")+ await _wait_for_close(session)+ await coordinator.shutdown()++ manager.upsert_single_provider_locked.assert_not_awaited()+ assert state_store.active[user_id] == "new-owner-login"+ assert state.error == "ChatGPT sign-in ownership was lost. Try again."+++async def test_redis_state_never_contains_device_code_or_verification_url():+ redis = FakeRedis()+ store = RedisCodexLoginStateStore(ttl_seconds=1200, owner_lease_seconds=30)+ state = CodexSharedLoginState(user_id="user-123", status="pending")+ with patch(+ "backend.integrations.codex.login.get_redis_async",+ new=AsyncMock(return_value=redis),+ ):+ assert await store.claim(state, "login-123")++ serialized = json.dumps(redis.values)+ assert "ABCD-EFGH" not in serialized+ assert "auth.openai.com" not in serialized+ assert "access_token" not in serialized+ assert sorted(redis.expirations.values()) == [30, 1200]++ with patch(+ "backend.integrations.codex.login.get_redis_async",+ new=AsyncMock(return_value=redis),+ ):+ assert await store.refresh_active("user-123", "login-123")+ assert sorted(redis.expirations.values()) == [30, 1200]+++async def test_stale_login_cannot_release_or_refresh_replacement_owner():+ user_id = "user-123"+ redis = FakeRedis()+ store = RedisCodexLoginStateStore(ttl_seconds=1200, owner_lease_seconds=30)++ with patch(+ "backend.integrations.codex.login.get_redis_async",+ new=AsyncMock(return_value=redis),+ ):+ original = CodexSharedLoginState(user_id=user_id, status="pending")+ assert await store.claim(original, "old-login")+ key = next(key for key in redis.values if key.endswith(":active"))+ redis.values[key] = "replacement-login"+ redis.expirations[key] = 17++ await store.release_active(user_id, "old-login")+ assert redis.values[key] == "replacement-login"+ assert not await store.refresh_active(user_id, "old-login")+ assert redis.expirations[key] == 17+++async def test_runtime_error_message_cannot_leak_into_state_or_logs(caplog):+ user_id = "user-123"+ session = FakeSession()+ state_store = MemoryStateStore()+ manager = MagicMock()+ coordinator = CodexLoginCoordinator(+ transport_factory=lambda: _transport(session),+ state_store=state_store,+ credentials_manager=manager,+ )++ details = await coordinator.start(user_id)+ session.result.set_exception(RuntimeError("access-secret refresh-secret"))+ state = await _wait_for_status(coordinator, user_id, details.login_id, "failed")+ await _wait_for_close(session)+ await _wait_for_active_release(state_store, user_id)++ assert state.error == "ChatGPT sign-in failed. Try again."+ assert "access-secret" not in caplog.text+ assert "refresh-secret" not in caplog.text+++def _transport(session: FakeSession) -> CodexLoginTransport:+ transport = MagicMock()+ transport.start_device_login = AsyncMock(return_value=session)+ return cast(CodexLoginTransport, transport)+++@asynccontextmanager+async def _provider_locks() -> AsyncIterator[None]:+ yield+++async def _wait_for_status(+ coordinator: CodexLoginCoordinator,+ user_id: str,+ login_id: str,+ expected: CodexLoginStatus,+) -> CodexDeviceLoginState:+ for _ in range(20):+ state = await coordinator.get(user_id, login_id)+ if state and state.status == expected:+ return state+ await asyncio.sleep(0)+ raise AssertionError(f"Codex login did not reach {expected}")+++async def _wait_for_close(session: FakeSession) -> None:+ for _ in range(20):+ if session.close.await_count:+ return+ await asyncio.sleep(0.001)+ raise AssertionError("Codex login session did not close")+++async def _wait_for_active_release(state_store: MemoryStateStore, user_id: str) -> None:+ for _ in range(20):+ if user_id not in state_store.active:+ return+ await asyncio.sleep(0.001)+ raise AssertionError("Codex login slot was not released")+++def _credentials() -> OAuth2Credentials:+ return OAuth2Credentials(+ id="codex-credential",+ provider="codex",+ title="ChatGPT for Codex",+ username="[email protected]",+ access_token=SecretStr("access-secret"),+ refresh_token=SecretStr("refresh-secret"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr("provider-secret"),+ provider_state_version=1,+ )+++def _completion() -> CodexLoginCompletion:+ return CodexLoginCompletion(+ bundle=CodexAuthBundleV1(+ tokens=CodexAuthTokensV1(+ id_token=SecretStr(_jwt({"email": "[email protected]"})),+ access_token=SecretStr(_jwt({"exp": 4_000_000_000})),+ refresh_token=SecretStr("refresh-secret"),+ ),+ codex_runtime_version="0.144.4",+ ),+ account=CodexAccountSnapshot(+ connected=True,+ requires_openai_auth=False,+ account_type="chatgpt",+ email="[email protected]",+ plan_type="plus",+ ),+ rate_limits=CodexRateLimitsSnapshot(plan_type="plus"),+ )+++def test_login_completion_allows_bundle_without_optional_enrichment():+ completion = CodexLoginCompletion(bundle=_completion().bundle)++ assert completion.account is None+ assert completion.rate_limits is None+++def _jwt(payload: dict[str, object]) -> str:+ header = _base64url({"alg": "none"})+ body = _base64url(payload)+ return f"{header}.{body}.signature"+++def _base64url(payload: dict[str, object]) -> str:+ encoded = json.dumps(payload, separators=(",", ":")).encode()+ return base64.urlsafe_b64encode(encoded).decode().rstrip("=")diff --git a/autogpt_platform/backend/backend/integrations/codex/models.py b/autogpt_platform/backend/backend/integrations/codex/models.pynew file mode 100644index 0000000000..3a51754ada--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/models.py@@ -0,0 +1,135 @@+from typing import Literal++from pydantic import BaseModel, ConfigDict, Field++from backend.integrations.codex.auth_bundle import CodexAuthBundleV1++CodexReasoningEffort = Literal[+ "none",+ "minimal",+ "low",+ "medium",+ "high",+ "xhigh",+ "max",+ "ultra",+]+++class CodexDeviceCodeDetails(BaseModel):+ model_config = ConfigDict(extra="forbid")++ login_id: str+ verification_url: str+ user_code: str+++class CodexAccountSnapshot(BaseModel):+ model_config = ConfigDict(extra="forbid")++ connected: bool+ requires_openai_auth: bool+ account_type: str | None = None+ email: str | None = None+ plan_type: str | None = None+++class CodexRateLimitWindow(BaseModel):+ model_config = ConfigDict(extra="forbid")++ used_percent: int+ window_duration_mins: int | None = None+ resets_at: int | None = None+++class CodexRateLimitsSnapshot(BaseModel):+ model_config = ConfigDict(extra="forbid")++ plan_type: str | None = None+ limit_id: str | None = None+ limit_name: str | None = None+ rate_limit_reached_type: str | None = None+ primary: CodexRateLimitWindow | None = None+ secondary: CodexRateLimitWindow | None = None+ has_credits: bool | None = None+ unlimited_credits: bool | None = None+ bucket_ids: list[str] = Field(default_factory=list)+++class CodexTokenUsage(BaseModel):+ model_config = ConfigDict(extra="forbid")++ input_tokens: int+ cached_input_tokens: int+ output_tokens: int+ reasoning_output_tokens: int+ total_tokens: int+++class CodexModelInfo(BaseModel):+ model_config = ConfigDict(extra="forbid", frozen=True)++ model: str+ display_name: str+ is_default: bool+ hidden: bool+ default_reasoning_effort: CodexReasoningEffort+ supported_reasoning_efforts: list[CodexReasoningEffort]+ input_modalities: list[str] = Field(default_factory=list)+++class CodexInvocationRequest(BaseModel):+ model_config = ConfigDict(extra="forbid")++ prompt: str+ instructions: str | None = None+ model: str | None = None+ effort: CodexReasoningEffort | None = None+ output_schema: dict[str, object] | None = None+ timeout_seconds: float | None = Field(default=None, gt=0)+++class CodexInvocationResult(BaseModel):+ model_config = ConfigDict(extra="forbid")++ response_id: str+ final_response: str+ reasoning_summary: str | None = None+ status: str+ resolved_model: str | None = None+ duration_ms: int | None = None+ usage: CodexTokenUsage | None = None+++class CodexDynamicToolSpec(BaseModel):+ model_config = ConfigDict(extra="forbid")++ name: str = Field(pattern=r"^[a-zA-Z0-9_-]{1,128}$")+ description: str+ input_schema: dict[str, object]+++class CodexDynamicToolCall(BaseModel):+ model_config = ConfigDict(extra="forbid")++ thread_id: str+ turn_id: str+ call_id: str+ namespace: str | None = None+ tool: str+ arguments: object+++class CodexDynamicToolResult(BaseModel):+ model_config = ConfigDict(extra="forbid")++ content: str+ success: bool = True+++class CodexLoginCompletion(BaseModel):+ model_config = ConfigDict(extra="forbid")++ bundle: CodexAuthBundleV1+ account: CodexAccountSnapshot | None = None+ rate_limits: CodexRateLimitsSnapshot | None = Nonediff --git a/autogpt_platform/backend/backend/integrations/codex/runtime.py b/autogpt_platform/backend/backend/integrations/codex/runtime.pynew file mode 100644index 0000000000..13684f1fba--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/runtime.py@@ -0,0 +1,966 @@+import asyncio+import concurrent.futures+import os+import sys+import threading+import time+from collections.abc import Awaitable, Callable+from contextlib import suppress+from importlib.metadata import version+from pathlib import Path+from types import MethodType+from typing import Any, cast++from codex_cli_bin import bundled_codex_path+from openai_codex import (+ ApprovalMode,+ AsyncCodex,+ AsyncDeviceCodeLoginHandle,+ AsyncThread,+ AsyncTurnHandle,+ CodexConfig,+ Sandbox,+ TurnResult,+)+from openai_codex._run import _collect_async_turn_result+from openai_codex.generated.v2_all import (+ ChatgptAccount,+ GetAccountRateLimitsResponse,+ RateLimitWindow,+ ReasoningEffort,+ ReasoningThreadItem,+ ThreadStartResponse,+)+from openai_codex.models import Notification+from openai_codex.types import JsonObject++from backend.integrations.codex.models import (+ CodexAccountSnapshot,+ CodexDeviceCodeDetails,+ CodexDynamicToolCall,+ CodexDynamicToolResult,+ CodexDynamicToolSpec,+ CodexInvocationRequest,+ CodexInvocationResult,+ CodexModelInfo,+ CodexRateLimitsSnapshot,+ CodexRateLimitWindow,+ CodexReasoningEffort,+ CodexTokenUsage,+)+from backend.integrations.codex.temporary_home import TemporaryCodexHome++OPENAI_CODEX_VERSION = "0.144.4"+CODEX_RUNTIME_VERSION = "0.144.4"+_RUNTIME_CLOSE_TIMEOUT_SECONDS = 3.0+_SERVER_REQUEST_MAX_CONCURRENCY = 8+++class _ConcurrentServerRequestDispatcher:+ def __init__(self, sync_client: Any, *, max_concurrency: int) -> None:+ if max_concurrency < 1:+ raise ValueError("max_concurrency must be positive")+ self._sync_client = sync_client+ self._capacity = threading.BoundedSemaphore(max_concurrency)+ self._closing = threading.Event()+ self._workers: set[threading.Thread] = set()+ self._workers_lock = threading.Lock()++ def dispatch(self, message: dict[str, object]) -> None:+ if self._closing.is_set():+ self._write_error(message, "Codex tool bridge is shutting down")+ return+ if not self._capacity.acquire(blocking=False):+ self._write_error(message, "Codex tool bridge is busy")+ return++ worker = threading.Thread(+ target=self._answer,+ args=(message,),+ name="autogpt-codex-server-request",+ daemon=True,+ )+ rejection: str | None = None+ with self._workers_lock:+ if self._closing.is_set():+ self._capacity.release()+ rejection = "Codex tool bridge is shutting down"+ else:+ self._workers.add(worker)+ try:+ worker.start()+ except BaseException:+ self._workers.discard(worker)+ self._capacity.release()+ rejection = "Codex tool bridge request failed"+ if rejection is not None:+ self._write_error(message, rejection)++ def close(self, timeout_seconds: float) -> bool:+ self._closing.set()+ deadline = time.monotonic() + max(timeout_seconds, 0)+ current = threading.current_thread()+ while True:+ with self._workers_lock:+ workers = tuple(worker for worker in self._workers if worker != current)+ if not workers:+ return True+ remaining = deadline - time.monotonic()+ if remaining <= 0:+ return False+ workers[0].join(timeout=remaining)++ def _answer(self, message: dict[str, object]) -> None:+ try:+ response = self._sync_client._handle_server_request(message)+ self._sync_client._write_message({"id": message["id"], "result": response})+ except BaseException:+ self._write_error(message, "Codex tool bridge request failed")+ finally:+ with self._workers_lock:+ self._workers.discard(threading.current_thread())+ self._capacity.release()++ def _write_error(self, message: dict[str, object], detail: str) -> None:+ if getattr(self._sync_client, "_proc", None) is None:+ return+ with suppress(BaseException):+ self._sync_client._write_message(+ {+ "id": message.get("id"),+ "error": {+ "code": -32001,+ "message": detail,+ },+ }+ )+++class CodexRuntimeError(RuntimeError):+ pass+++class CodexRuntimeDeviceLogin:+ def __init__(self, handle: AsyncDeviceCodeLoginHandle) -> None:+ self._handle = handle+ self.details = CodexDeviceCodeDetails(+ login_id=handle.login_id,+ verification_url=handle.verification_url,+ user_code=handle.user_code,+ )++ async def wait(self) -> bool:+ completion = await self._handle.wait()+ return completion.success++ async def cancel(self) -> None:+ await self._handle.cancel()+++class CodexRuntime:+ def __init__(+ self,+ client: AsyncCodex,+ home: TemporaryCodexHome,+ *,+ close_timeout_seconds: float = _RUNTIME_CLOSE_TIMEOUT_SECONDS,+ ) -> None:+ self._client = client+ self._home = home+ self._close_timeout_seconds = close_timeout_seconds+ self._closed = False+ self._poisoned = False+ self._close_lock = asyncio.Lock()+ self._dynamic_tool_handlers: dict[+ str,+ tuple[+ Callable[[CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]],+ float,+ ],+ ] = {}+ self._dynamic_tool_futures: dict[+ str, set[concurrent.futures.Future[CodexDynamicToolResult]]+ ] = {}+ self._dynamic_tool_futures_lock = threading.Lock()++ @property+ def closed(self) -> bool:+ return self._closed or self._poisoned++ @classmethod+ async def start(cls, home: TemporaryCodexHome) -> "CodexRuntime":+ assert_pinned_versions()+ config = build_runtime_config(home)+ client = AsyncCodex(config)+ _install_concurrent_server_request_dispatcher(client)+ _install_fail_closed_approval_handler(client)+ startup = asyncio.create_task(client.__aenter__())+ try:+ await asyncio.shield(startup)+ except asyncio.CancelledError:+ with suppress(BaseException):+ await _close_codex_client(client, _RUNTIME_CLOSE_TIMEOUT_SECONDS)+ await _cancel_task_bounded(startup, _RUNTIME_CLOSE_TIMEOUT_SECONDS)+ raise+ except BaseException:+ with suppress(BaseException):+ await _close_codex_client(client, _RUNTIME_CLOSE_TIMEOUT_SECONDS)+ raise+ return cls(client, home)++ async def start_device_code_login(self) -> CodexRuntimeDeviceLogin:+ handle = await self._client.login_chatgpt_device_code()+ return CodexRuntimeDeviceLogin(handle)++ async def account(+ self,+ *,+ refresh_token: bool = True,+ ) -> CodexAccountSnapshot:+ response = await self._client.account(refresh_token=refresh_token)+ if response.account is None:+ return CodexAccountSnapshot(+ connected=False,+ requires_openai_auth=response.requires_openai_auth,+ )+ account = response.account.root+ if account.type != "chatgpt":+ return CodexAccountSnapshot(+ connected=True,+ requires_openai_auth=response.requires_openai_auth,+ account_type=account.type,+ )+ chatgpt = cast(ChatgptAccount, account)+ return CodexAccountSnapshot(+ connected=True,+ requires_openai_auth=response.requires_openai_auth,+ account_type=chatgpt.type,+ email=chatgpt.email,+ plan_type=chatgpt.plan_type.value,+ )++ async def rate_limits(self) -> CodexRateLimitsSnapshot:+ response = await self._client._client.request(+ "account/rateLimits/read",+ None,+ response_model=GetAccountRateLimitsResponse,+ )+ limits = response.rate_limits+ credits = limits.credits+ return CodexRateLimitsSnapshot(+ plan_type=limits.plan_type.value if limits.plan_type else None,+ limit_id=limits.limit_id,+ limit_name=limits.limit_name,+ rate_limit_reached_type=(+ limits.rate_limit_reached_type.value+ if limits.rate_limit_reached_type+ else None+ ),+ primary=_rate_limit_window(limits.primary),+ secondary=_rate_limit_window(limits.secondary),+ has_credits=credits.has_credits if credits else None,+ unlimited_credits=credits.unlimited if credits else None,+ bucket_ids=sorted((response.rate_limits_by_limit_id or {}).keys()),+ )++ async def models(self) -> list[CodexModelInfo]:+ response = await self._client.models(include_hidden=True)+ return [+ CodexModelInfo(+ model=model.model,+ display_name=model.display_name,+ is_default=model.is_default,+ hidden=model.hidden,+ default_reasoning_effort=cast(+ CodexReasoningEffort,+ model.default_reasoning_effort.value,+ ),+ supported_reasoning_efforts=[+ cast(CodexReasoningEffort, option.reasoning_effort.value)+ for option in model.supported_reasoning_efforts+ ],+ input_modalities=[+ modality.value for modality in (model.input_modalities or [])+ ],+ )+ for model in response.data+ ]++ async def invoke(+ self,+ request: CodexInvocationRequest,+ ) -> CodexInvocationResult:+ thread = await self._start_toolless_thread(request)+ effort = ReasoningEffort(request.effort) if request.effort else None+ schema = cast(JsonObject | None, request.output_schema)+ turn_start = asyncio.create_task(+ thread.turn(+ request.prompt,+ approval_mode=ApprovalMode.deny_all,+ effort=effort,+ model=request.model,+ output_schema=schema,+ sandbox=Sandbox.read_only,+ )+ )+ try:+ turn = await asyncio.shield(turn_start)+ except asyncio.CancelledError:+ with suppress(BaseException):+ await asyncio.shield(self.close())+ await _cancel_task_bounded(+ turn_start,+ self._close_timeout_seconds,+ )+ raise+ try:+ result = await turn.run()+ except asyncio.CancelledError:+ with suppress(BaseException):+ await asyncio.wait_for(turn.interrupt(), timeout=2)+ if not self._closed:+ with suppress(BaseException):+ await asyncio.shield(self.close())+ raise+ if result.final_response is None:+ raise CodexRuntimeError("Codex turn completed without a final response")+ return CodexInvocationResult(+ response_id=result.id,+ final_response=result.final_response,+ reasoning_summary=_reasoning_summary(result),+ status=result.status.value,+ duration_ms=result.duration_ms,+ usage=_token_usage(result),+ )++ async def invoke_agent(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None = None,+ *,+ tool_timeout_seconds: float,+ ) -> CodexInvocationResult:+ thread = await self._start_thread(request, dynamic_tools)+ self._register_dynamic_tool_handler(+ thread.id,+ tool_handler,+ timeout_seconds=tool_timeout_seconds,+ )+ try:+ effort = ReasoningEffort(request.effort) if request.effort else None+ turn_start = asyncio.create_task(+ thread.turn(+ request.prompt,+ approval_mode=ApprovalMode.deny_all,+ effort=effort,+ model=request.model,+ sandbox=Sandbox.read_only,+ )+ )+ try:+ turn = await asyncio.shield(turn_start)+ except asyncio.CancelledError:+ await self._stop_cancelled_turn_start(turn_start)+ raise+ stream = turn.stream()+ abnormal_exit = False++ async def observed_stream():+ async for event in stream:+ if event_handler is not None:+ await event_handler(event)+ yield event++ try:+ result = await _collect_async_turn_result(+ observed_stream(),+ turn_id=turn.id,+ )+ except BaseException:+ abnormal_exit = True+ self._cancel_dynamic_tool_futures(thread.id)+ await self._interrupt_turn_or_close(turn)+ raise+ finally:+ try:+ await self._close_agent_stream(stream)+ except BaseException:+ if not abnormal_exit:+ raise+ await self._poison_runtime()+ finally:+ self._unregister_dynamic_tool_handler(thread.id)+ if result.final_response is None:+ raise CodexRuntimeError("Codex turn completed without a final response")+ return CodexInvocationResult(+ response_id=result.id,+ final_response=result.final_response,+ reasoning_summary=_reasoning_summary(result),+ status=result.status.value,+ duration_ms=result.duration_ms,+ usage=_token_usage(result),+ )++ async def logout(self) -> None:+ await self._client.logout()++ async def close(self) -> None:+ async with self._close_lock:+ if self._closed:+ return+ self._closed = True+ self._cancel_dynamic_tool_futures()+ await _close_codex_client(self._client, self._close_timeout_seconds)++ async def _start_toolless_thread(+ self,+ request: CodexInvocationRequest,+ ) -> AsyncThread:+ return await self._start_thread(request, [])++ async def _start_thread(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ ) -> AsyncThread:+ params: JsonObject = {+ "approvalPolicy": "never",+ "approvalsReviewer": "user",+ "config": {+ "features": {+ "apps": False,+ "image_generation": False,+ "multi_agent": False,+ "multi_agent_v2": False,+ "plugins": False,+ "shell_tool": False,+ "tool_suggest": False,+ },+ "web_search": "disabled",+ },+ "cwd": str(self._home.workspace_path),+ "dynamicTools": [+ {+ "type": "function",+ "name": tool.name,+ "description": tool.description,+ "inputSchema": cast(JsonObject, tool.input_schema),+ }+ for tool in dynamic_tools+ ],+ "ephemeral": True,+ "environments": [],+ "runtimeWorkspaceRoots": [],+ "sandbox": "read-only",+ "selectedCapabilityRoots": [],+ }+ if request.instructions is not None:+ params["developerInstructions"] = request.instructions+ if request.model is not None:+ params["model"] = request.model+ started = await self._client._client.request(+ "thread/start",+ params,+ response_model=ThreadStartResponse,+ )+ return AsyncThread(self._client, started.thread.id)++ def _register_dynamic_tool_handler(+ self,+ thread_id: str,+ handler: Callable[[CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]],+ *,+ timeout_seconds: float,+ ) -> None:+ self._ensure_dynamic_tool_dispatcher()+ with self._dynamic_tool_futures_lock:+ self._dynamic_tool_handlers[thread_id] = (handler, timeout_seconds)+ self._dynamic_tool_futures.setdefault(thread_id, set())++ def _unregister_dynamic_tool_handler(self, thread_id: str) -> None:+ with self._dynamic_tool_futures_lock:+ self._dynamic_tool_handlers.pop(thread_id, None)+ futures = tuple(self._dynamic_tool_futures.pop(thread_id, ()))+ for future in futures:+ future.cancel()++ def _ensure_dynamic_tool_dispatcher(self) -> None:+ loop = asyncio.get_running_loop()++ def handle(method: str, params: JsonObject | None) -> JsonObject:+ if method != "item/tool/call":+ return _deny_server_request(method, params)+ try:+ call = CodexDynamicToolCall.model_validate(+ {+ "thread_id": (params or {}).get("threadId"),+ "turn_id": (params or {}).get("turnId"),+ "call_id": (params or {}).get("callId"),+ "namespace": (params or {}).get("namespace"),+ "tool": (params or {}).get("tool"),+ "arguments": (params or {}).get("arguments"),+ }+ )+ except Exception:+ return _dynamic_tool_error("codex_tool_request_invalid")++ with self._dynamic_tool_futures_lock:+ registration = self._dynamic_tool_handlers.get(call.thread_id)+ futures = self._dynamic_tool_futures.get(call.thread_id)+ if self.closed or registration is None or futures is None:+ return _dynamic_tool_error("codex_tool_handler_unavailable")+ handler, timeout_seconds = registration++ async def execute_handler() -> CodexDynamicToolResult:+ return await handler(call)++ coroutine = execute_handler()+ try:+ future = asyncio.run_coroutine_threadsafe(coroutine, loop)+ except RuntimeError:+ coroutine.close()+ return _dynamic_tool_error("codex_tool_execution_failed")+ except BaseException:+ coroutine.close()+ raise+ futures.add(future)+ try:+ result = future.result(timeout=timeout_seconds)+ except concurrent.futures.TimeoutError:+ future.cancel()+ return _dynamic_tool_error("codex_tool_execution_timeout")+ except BaseException:+ return _dynamic_tool_error("codex_tool_execution_failed")+ finally:+ with self._dynamic_tool_futures_lock:+ futures = self._dynamic_tool_futures.get(call.thread_id)+ if futures is not None:+ futures.discard(future)+ return {+ "contentItems": [{"type": "inputText", "text": result.content}],+ "success": result.success,+ }++ try:+ sync_client = self._client._client._sync+ except AttributeError as exc:+ raise CodexRuntimeError(+ "Pinned Codex SDK approval-handler layout changed"+ ) from exc+ if getattr(sync_client, "_autogpt_dynamic_tool_dispatcher", False):+ return+ sync_client._approval_handler = handle+ setattr(sync_client, "_autogpt_dynamic_tool_dispatcher", True)++ def _cancel_dynamic_tool_futures(self, thread_id: str | None = None) -> None:+ with self._dynamic_tool_futures_lock:+ if thread_id is None:+ futures = tuple(+ future+ for group in self._dynamic_tool_futures.values()+ for future in group+ )+ else:+ futures = tuple(self._dynamic_tool_futures.get(thread_id, ()))+ for future in futures:+ future.cancel()++ async def _close_agent_stream(self, stream) -> None:+ closing = asyncio.create_task(stream.aclose())+ try:+ done, _ = await asyncio.wait(+ {closing},+ timeout=self._close_timeout_seconds,+ )+ except asyncio.CancelledError:+ closing.cancel()+ _detach_task(closing)+ raise+ if closing in done:+ try:+ await closing+ except BaseException:+ with suppress(BaseException):+ await self.close()+ raise+ return+ closing.cancel()+ _detach_task(closing)+ with suppress(BaseException):+ await self.close()+ raise CodexRuntimeError("codex_stream_shutdown_timeout")++ async def _stop_cancelled_turn_start(+ self,+ turn_start: asyncio.Task[AsyncTurnHandle],+ ) -> None:+ try:+ done, _ = await asyncio.wait(+ {turn_start},+ timeout=self._close_timeout_seconds,+ )+ except BaseException:+ await self._poison_runtime()+ return+ if turn_start not in done:+ await self._poison_runtime()+ await _cancel_task_bounded(+ cast(asyncio.Task[object], turn_start),+ self._close_timeout_seconds,+ )+ return+ try:+ turn = turn_start.result()+ except BaseException:+ await self._poison_runtime()+ return+ await self._interrupt_turn_or_close(turn)++ async def _interrupt_turn_or_close(self, turn: AsyncTurnHandle) -> bool:+ interrupting = asyncio.create_task(turn.interrupt())+ try:+ done, _ = await asyncio.wait(+ {interrupting},+ timeout=min(2.0, self._close_timeout_seconds),+ )+ except BaseException:+ interrupting.cancel()+ _detach_task(cast(asyncio.Task[object], interrupting))+ await self._poison_runtime()+ return False+ if interrupting in done:+ try:+ interrupting.result()+ except BaseException:+ await self._poison_runtime()+ return False+ return True+ await _cancel_task_bounded(+ cast(asyncio.Task[object], interrupting),+ self._close_timeout_seconds,+ )+ await self._poison_runtime()+ return False++ async def _poison_runtime(self) -> None:+ self._poisoned = True+ closing = asyncio.create_task(self.close())+ try:+ done, _ = await asyncio.wait(+ {closing},+ timeout=self._close_timeout_seconds,+ )+ except BaseException:+ _detach_task(cast(asyncio.Task[object], closing))+ return+ if closing not in done:+ _detach_task(cast(asyncio.Task[object], closing))+ return+ with suppress(BaseException):+ closing.result()+++def _install_concurrent_server_request_dispatcher(client: AsyncCodex) -> None:+ try:+ sync_client = client._client._sync+ _ = (+ sync_client._read_message,+ sync_client._write_message,+ sync_client._handle_server_request,+ sync_client._coerce_notification,+ sync_client._router,+ )+ except AttributeError as exc:+ raise CodexRuntimeError("Pinned Codex SDK reader layout changed") from exc++ if getattr(sync_client, "_autogpt_server_request_dispatcher", None) is not None:+ return+ dispatcher = _ConcurrentServerRequestDispatcher(+ sync_client,+ max_concurrency=_SERVER_REQUEST_MAX_CONCURRENCY,+ )+ setattr(sync_client, "_autogpt_server_request_dispatcher", dispatcher)++ def reader_loop(instance: Any) -> None:+ try:+ while True:+ message = instance._read_message()+ if "method" in message and "id" in message:+ dispatcher.dispatch(message)+ continue+ if "method" in message and "id" not in message:+ method = message["method"]+ if isinstance(method, str):+ instance._router.route_notification(+ instance._coerce_notification(+ method,+ message.get("params"),+ )+ )+ continue+ instance._router.route_response(message)+ except BaseException as exc:+ instance._router.fail_all(exc)++ sync_client._reader_loop = MethodType(reader_loop, sync_client)+++def _shutdown_concurrent_server_request_dispatcher(+ client: AsyncCodex,+ timeout_seconds: float,+) -> bool:+ try:+ dispatcher = getattr(+ client._client._sync,+ "_autogpt_server_request_dispatcher",+ )+ except AttributeError:+ return True+ return dispatcher.close(timeout_seconds)+++async def _close_codex_client(+ client: AsyncCodex,+ timeout_seconds: float,+) -> None:+ process = _runtime_process(client)+ deadline = time.monotonic() + max(timeout_seconds, 0)+ closing: asyncio.Task[None] | None = None+ try:+ await asyncio.to_thread(+ _shutdown_concurrent_server_request_dispatcher,+ client,+ timeout_seconds,+ )+ closing = asyncio.create_task(client.close())+ remaining = max(deadline - time.monotonic(), 0)+ if remaining == 0:+ await asyncio.sleep(0)+ done = {closing} if closing.done() else set()+ else:+ done, _ = await asyncio.wait({closing}, timeout=remaining)+ if closing in done:+ await closing+ return+ closing.cancel()+ _detach_task(closing)+ _force_stop_process(process)+ except asyncio.CancelledError:+ if closing is not None:+ closing.cancel()+ _detach_task(closing)+ _force_stop_process(process)+ raise+ except BaseException:+ _force_stop_process(process)+ raise+++def _runtime_process(client: AsyncCodex) -> object | None:+ try:+ return client._client._sync._proc+ except AttributeError:+ return None+++def _force_stop_process(process: object | None) -> None:+ if process is None:+ return+ with suppress(BaseException):+ stdin = getattr(process, "stdin", None)+ if stdin is not None:+ stdin.close()+ with suppress(BaseException):+ process.terminate() # type: ignore[attr-defined]+ with suppress(BaseException):+ if process.poll() is None: # type: ignore[attr-defined]+ process.kill() # type: ignore[attr-defined]+++def _detach_task(task: asyncio.Task[object]) -> None:+ def consume_result(completed: asyncio.Task[object]) -> None:+ with suppress(BaseException):+ completed.result()++ task.add_done_callback(consume_result)+++async def _cancel_task_bounded(+ task: asyncio.Task[object],+ timeout_seconds: float,+) -> None:+ task.cancel()+ done, _ = await asyncio.wait({task}, timeout=timeout_seconds)+ if task not in done:+ _detach_task(task)+ return+ with suppress(BaseException):+ task.result()+++def build_runtime_config(home: TemporaryCodexHome) -> CodexConfig:+ launcher_path = str(Path(__file__).with_name("launcher.py").resolve())+ launch_args = (+ sys.executable,+ launcher_path,+ str(bundled_codex_path()),+ "--config",+ 'cli_auth_credentials_store="file"',+ "--config",+ 'forced_login_method="chatgpt"',+ "--config",+ "allow_login_shell=false",+ "--config",+ 'web_search="disabled"',+ "--config",+ "features.shell_tool=false",+ "--config",+ "features.multi_agent=false",+ "--config",+ "features.multi_agent_v2=false",+ "--config",+ "features.apps=false",+ "--config",+ "features.plugins=false",+ "--config",+ "features.tool_suggest=false",+ "--config",+ "features.image_generation=false",+ "--config",+ "features.in_app_browser=false",+ "--config",+ "features.browser_use=false",+ "--config",+ "features.computer_use=false",+ "--config",+ "features.memories=false",+ "--config",+ "features.goals=false",+ "--config",+ "features.workspace_dependencies=false",+ "--config",+ "include_apps_instructions=false",+ "--config",+ "include_collaboration_mode_instructions=false",+ "--config",+ "include_environment_context=false",+ "--config",+ "include_permissions_instructions=false",+ "app-server",+ "--listen",+ "stdio://",+ )+ return CodexConfig(+ launch_args_override=launch_args,+ cwd=str(home.workspace_path),+ env=_runtime_environment(home),+ client_name="autogpt_platform",+ client_title="AutoGPT Platform",+ client_version="0.6.22",+ experimental_api=True,+ )+++def assert_pinned_versions() -> None:+ sdk_version = version("openai-codex")+ runtime_version = version("openai-codex-cli-bin")+ if sdk_version != OPENAI_CODEX_VERSION or runtime_version != CODEX_RUNTIME_VERSION:+ raise CodexRuntimeError(+ "Codex SDK and runtime must both be pinned to version 0.144.4"+ )+++def _install_fail_closed_approval_handler(client: AsyncCodex) -> None:+ try:+ sync_client = client._client._sync+ except AttributeError as exc:+ raise CodexRuntimeError(+ "Pinned Codex SDK approval-handler layout changed"+ ) from exc+ sync_client._approval_handler = _deny_server_request+++def _deny_server_request(+ method: str,+ _params: JsonObject | None,+) -> JsonObject:+ if method in {+ "item/commandExecution/requestApproval",+ "item/fileChange/requestApproval",+ }:+ return {"decision": "decline"}+ if method in {"applyPatchApproval", "execCommandApproval"}:+ return {"decision": "denied"}+ if method == "item/permissions/requestApproval":+ return {"permissions": {}, "scope": "turn"}+ raise CodexRuntimeError(f"Unsupported Codex server request: {method}")+++def _dynamic_tool_error(code: str) -> JsonObject:+ return {+ "contentItems": [{"type": "inputText", "text": code}],+ "success": False,+ }+++def _runtime_environment(home: TemporaryCodexHome) -> dict[str, str]:+ home_path = str(home.path)+ temp_path = str(home.temp_path)+ environment = {+ "APPDATA": home_path,+ "CODEX_HOME": home_path,+ "CODEX_SQLITE_HOME": home_path,+ "HOME": home_path,+ "LOCALAPPDATA": home_path,+ "RUST_LOG": "warn",+ "TEMP": temp_path,+ "TMP": temp_path,+ "TMPDIR": temp_path,+ "USERPROFILE": home_path,+ "XDG_CACHE_HOME": str(Path(home_path) / "cache"),+ "XDG_CONFIG_HOME": str(Path(home_path) / "config"),+ "XDG_DATA_HOME": str(Path(home_path) / "data"),+ }+ return environment | _required_windows_environment()+++def _required_windows_environment() -> dict[str, str]:+ names = ("COMSPEC", "PATH", "PATHEXT", "SYSTEMROOT", "WINDIR")+ return {name: os.environ[name] for name in names if name in os.environ}+++def _rate_limit_window(window: RateLimitWindow | None) -> CodexRateLimitWindow | None:+ if window is None:+ return None+ return CodexRateLimitWindow(+ used_percent=window.used_percent,+ window_duration_mins=window.window_duration_mins,+ resets_at=window.resets_at,+ )+++def _token_usage(result: TurnResult) -> CodexTokenUsage | None:+ if result.usage is None:+ return None+ total = result.usage.total+ return CodexTokenUsage(+ input_tokens=total.input_tokens,+ cached_input_tokens=total.cached_input_tokens,+ output_tokens=total.output_tokens,+ reasoning_output_tokens=total.reasoning_output_tokens,+ total_tokens=total.total_tokens,+ )+++def _reasoning_summary(result: TurnResult) -> str | None:+ summaries = []+ for item in result.items:+ if item.root.type != "reasoning":+ continue+ reasoning = cast(ReasoningThreadItem, item.root)+ summaries.extend(reasoning.summary or [])+ return "\n".join(summaries) or Nonediff --git a/autogpt_platform/backend/backend/integrations/codex/runtime_test.py b/autogpt_platform/backend/backend/integrations/codex/runtime_test.pynew file mode 100644index 0000000000..3fdd5500e0--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/runtime_test.py@@ -0,0 +1,1103 @@+import asyncio+import concurrent.futures+import importlib.metadata+import os+import subprocess+import sys+import threading+from pathlib import Path+from types import SimpleNamespace+from typing import cast+from unittest.mock import AsyncMock++import pytest+from codex_cli_bin import bundled_codex_path+from openai_codex import AsyncCodex++import backend.integrations.codex.runtime as runtime_module+from backend.integrations.codex.models import (+ CodexDynamicToolResult,+ CodexDynamicToolSpec,+ CodexInvocationRequest,+ CodexModelInfo,+)+from backend.integrations.codex.runtime import (+ CODEX_RUNTIME_VERSION,+ OPENAI_CODEX_VERSION,+ CodexRuntime,+ CodexRuntimeError,+ _deny_server_request,+ _install_concurrent_server_request_dispatcher,+ _install_fail_closed_approval_handler,+ assert_pinned_versions,+ build_runtime_config,+)+from backend.integrations.codex.temporary_home import TemporaryCodexHome+++def test_sdk_and_runtime_versions_are_an_atomic_pin():+ assert importlib.metadata.version("openai-codex") == OPENAI_CODEX_VERSION+ assert importlib.metadata.version("openai-codex-cli-bin") == CODEX_RUNTIME_VERSION+ assert_pinned_versions()+++def test_runtime_config_uses_sanitizing_launcher_and_isolated_home(tmp_path):+ with TemporaryCodexHome.create(tmp_path) as home:+ config = build_runtime_config(home)++ arguments = config.launch_args_override+ assert arguments is not None+ assert arguments[0] == sys.executable+ assert (+ Path(arguments[1]).resolve()+ == Path(__file__).with_name("launcher.py").resolve()+ )+ assert arguments[2] == str(bundled_codex_path())+ assert arguments[-3:] == ("app-server", "--listen", "stdio://")+ assert 'cli_auth_credentials_store="file"' in arguments+ assert 'forced_login_method="chatgpt"' in arguments+ assert config.experimental_api is True+ assert config.env is not None+ for variable in (+ "APPDATA",+ "CODEX_HOME",+ "CODEX_SQLITE_HOME",+ "HOME",+ "LOCALAPPDATA",+ "TEMP",+ "TMP",+ "TMPDIR",+ "USERPROFILE",+ "XDG_CACHE_HOME",+ "XDG_CONFIG_HOME",+ "XDG_DATA_HOME",+ ):+ assert config.env[variable].startswith(str(home.path))+ assert config.env["TEMP"] == str(home.temp_path)+ assert config.env["TMP"] == str(home.temp_path)+ assert config.env["TMPDIR"] == str(home.temp_path)+ assert config.env["RUST_LOG"] == "warn"+ assert "DATABASE_URL" not in config.env+ assert "ENCRYPTION_KEY" not in config.env+ if os.name == "nt":+ assert config.env["SYSTEMROOT"] == os.environ["SYSTEMROOT"]+++def test_sanitizing_launcher_starts_from_isolated_working_directory(tmp_path):+ with TemporaryCodexHome.create(tmp_path) as home:+ config = build_runtime_config(home)+ arguments = config.launch_args_override+ assert arguments is not None+ result = subprocess.run(+ [*arguments[:-3], "--version"],+ cwd=home.workspace_path,+ env=config.env,+ capture_output=True,+ text=True,+ timeout=15,+ check=False,+ )++ assert result.returncode == 0, result.stderr+ assert CODEX_RUNTIME_VERSION in result.stdout+++def test_runtime_installs_protocol_exact_fail_closed_approval_handler(tmp_path):+ with TemporaryCodexHome.create(tmp_path) as home:+ client = AsyncCodex(build_runtime_config(home))+ _install_fail_closed_approval_handler(client)+ handler = client._client._sync._approval_handler++ assert handler("item/commandExecution/requestApproval", {}) == {+ "decision": "decline"+ }+ assert handler("item/fileChange/requestApproval", {}) == {"decision": "decline"}+ assert handler("applyPatchApproval", {}) == {"decision": "denied"}+ assert handler("execCommandApproval", {}) == {"decision": "denied"}+ assert handler("item/permissions/requestApproval", {}) == {+ "permissions": {},+ "scope": "turn",+ }+++def test_approval_handler_fails_closed_for_unknown_server_requests():+ with pytest.raises(CodexRuntimeError, match="Unsupported Codex server request"):+ _deny_server_request("item/future/requestApproval", {})+++async def test_model_discovery_preserves_account_catalog_metadata(tmp_path):+ client = SimpleNamespace(+ models=AsyncMock(+ return_value=SimpleNamespace(+ data=[+ SimpleNamespace(+ model="gpt-5.6-sol",+ display_name="GPT-5.6 Sol",+ is_default=True,+ hidden=False,+ default_reasoning_effort=SimpleNamespace(value="high"),+ supported_reasoning_efforts=[+ SimpleNamespace(+ reasoning_effort=SimpleNamespace(value="low")+ ),+ SimpleNamespace(+ reasoning_effort=SimpleNamespace(value="ultra")+ ),+ ],+ input_modalities=[+ SimpleNamespace(value="text"),+ SimpleNamespace(value="image"),+ ],+ )+ ]+ )+ )+ )++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ models = await runtime.models()++ assert models == [+ CodexModelInfo(+ model="gpt-5.6-sol",+ display_name="GPT-5.6 Sol",+ is_default=True,+ hidden=False,+ default_reasoning_effort="high",+ supported_reasoning_efforts=["low", "ultra"],+ input_modalities=["text", "image"],+ )+ ]+ client.models.assert_awaited_once_with(include_hidden=True)+++async def test_repeated_app_server_start_close_leaves_no_temporary_home(tmp_path):+ created_paths: list[Path] = []+ for _attempt in range(2):+ home = TemporaryCodexHome.create(tmp_path)+ created_paths.append(home.path)+ runtime: CodexRuntime | None = None+ try:+ runtime = await asyncio.wait_for(CodexRuntime.start(home), timeout=20)+ handler = runtime._client._client._sync._approval_handler+ assert handler("item/commandExecution/requestApproval", {}) == {+ "decision": "decline"+ }+ finally:+ if runtime is not None:+ await asyncio.wait_for(runtime.close(), timeout=10)+ home.cleanup()++ assert all(not path.exists() for path in created_paths)+++async def test_thread_start_explicitly_removes_all_runtime_environments(tmp_path):+ request_call = AsyncMock(+ return_value=SimpleNamespace(thread=SimpleNamespace(id="thread-1"))+ )+ client = SimpleNamespace(_client=SimpleNamespace(request=request_call))++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ await runtime._start_toolless_thread(+ CodexInvocationRequest(+ prompt="Do not access the host",+ instructions="Return text only",+ model="gpt-test",+ )+ )++ method, params = request_call.await_args.args[:2]+ assert method == "thread/start"+ assert params["environments"] == []+ assert params["dynamicTools"] == []+ assert params["runtimeWorkspaceRoots"] == []+ assert params["selectedCapabilityRoots"] == []+ assert params["sandbox"] == "read-only"+ assert params["cwd"] == str(home.workspace_path)+ assert params["config"]["web_search"] == "disabled"+ assert all(+ enabled is False for enabled in params["config"]["features"].values()+ )+++async def test_thread_start_exposes_only_supplied_dynamic_tools(tmp_path):+ request_call = AsyncMock(+ return_value=SimpleNamespace(thread=SimpleNamespace(id="thread-1"))+ )+ client = SimpleNamespace(_client=SimpleNamespace(request=request_call))++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ await runtime._start_thread(+ CodexInvocationRequest(prompt="List my agents"),+ [+ CodexDynamicToolSpec(+ name="find_agent",+ description="Find an AutoGPT agent",+ input_schema={"type": "object", "properties": {}},+ )+ ],+ )++ params = request_call.await_args.args[1]+ assert params["dynamicTools"] == [+ {+ "type": "function",+ "name": "find_agent",+ "description": "Find an AutoGPT agent",+ "inputSchema": {"type": "object", "properties": {}},+ }+ ]+++async def test_dynamic_tool_request_round_trips_on_copilot_loop(tmp_path):+ sync_client = SimpleNamespace(_approval_handler=None)+ client = SimpleNamespace(_client=SimpleNamespace(_sync=sync_client))+ observed = []++ async def execute(call):+ observed.append(call)+ return CodexDynamicToolResult(content='{"agents": []}', success=True)++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ runtime._register_dynamic_tool_handler("thread-1", execute, timeout_seconds=1)+ response = await asyncio.to_thread(+ sync_client._approval_handler,+ "item/tool/call",+ {+ "threadId": "thread-1",+ "turnId": "turn-1",+ "callId": "call-1",+ "namespace": "autogpt",+ "tool": "find_agent",+ "arguments": {"query": "demo"},+ },+ )++ assert observed[0].call_id == "call-1"+ assert observed[0].tool == "find_agent"+ assert observed[0].arguments == {"query": "demo"}+ assert response == {+ "contentItems": [{"type": "inputText", "text": '{"agents": []}'}],+ "success": True,+ }+++async def test_dynamic_tool_timeout_cancels_callback_and_fails_closed(tmp_path):+ sync_client = SimpleNamespace(_approval_handler=None)+ client = SimpleNamespace(_client=SimpleNamespace(_sync=sync_client))+ callback_cancelled = asyncio.Event()++ async def execute(_call):+ try:+ await asyncio.Event().wait()+ finally:+ callback_cancelled.set()++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ runtime._register_dynamic_tool_handler(+ "thread-1", execute, timeout_seconds=0.01+ )+ response = await asyncio.to_thread(+ sync_client._approval_handler,+ "item/tool/call",+ {+ "threadId": "thread-1",+ "turnId": "turn-1",+ "callId": "call-1",+ "tool": "find_agent",+ "arguments": {},+ },+ )+ await asyncio.wait_for(callback_cancelled.wait(), timeout=1)++ assert response == {+ "contentItems": [{"type": "inputText", "text": "codex_tool_execution_timeout"}],+ "success": False,+ }+++async def test_dynamic_tool_submission_failure_closes_callback_coroutine(+ tmp_path, monkeypatch+):+ sync_client = SimpleNamespace(_approval_handler=None)+ client = SimpleNamespace(_client=SimpleNamespace(_sync=sync_client))+ submitted: list[object] = []++ async def execute(_call):+ return CodexDynamicToolResult(content="unused", success=True)++ def reject_submission(coroutine, _loop):+ submitted.append(coroutine)+ raise RuntimeError("event loop is closed")++ monkeypatch.setattr(+ runtime_module.asyncio, "run_coroutine_threadsafe", reject_submission+ )++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(cast(AsyncCodex, client), home)+ runtime._register_dynamic_tool_handler("thread-1", execute, timeout_seconds=1)+ response = await asyncio.to_thread(+ sync_client._approval_handler,+ "item/tool/call",+ {+ "threadId": "thread-1",+ "turnId": "turn-1",+ "callId": "call-1",+ "tool": "find_agent",+ "arguments": {},+ },+ )++ assert len(submitted) == 1+ assert getattr(submitted[0], "cr_frame") is None+ assert response == {+ "contentItems": [{"type": "inputText", "text": "codex_tool_execution_failed"}],+ "success": False,+ }+++async def test_unregister_cancels_tool_reserved_during_dispatch(tmp_path, monkeypatch):+ class ObservedLock:+ def __init__(self) -> None:+ self._lock = threading.Lock()+ self.owner_ident: int | None = None+ self.unregister_attempted = threading.Event()+ self.unregister_acquired = threading.Event()++ def __enter__(self):+ is_unregister = threading.current_thread().name == "codex-unregister"+ if is_unregister:+ self.unregister_attempted.set()+ self._lock.acquire()+ self.owner_ident = threading.get_ident()+ if is_unregister:+ self.unregister_acquired.set()+ return self++ def __exit__(self, _exc_type, _exc_value, _traceback) -> None:+ self.owner_ident = None+ self._lock.release()++ observed_lock = ObservedLock()+ sync_client = SimpleNamespace(_approval_handler=None)+ client = SimpleNamespace(_client=SimpleNamespace(_sync=sync_client))+ scheduler_entered = threading.Event()+ allow_scheduler_return = threading.Event()+ scheduler_held_lock: list[bool] = []+ future: concurrent.futures.Future[CodexDynamicToolResult] = (+ concurrent.futures.Future()+ )++ async def execute(_call):+ return CodexDynamicToolResult(content="unused", success=True)++ def schedule(coroutine, _loop):+ coroutine.close()+ scheduler_held_lock.append(observed_lock.owner_ident == threading.get_ident())+ scheduler_entered.set()+ assert allow_scheduler_return.wait(timeout=1)+ return future++ monkeypatch.setattr(runtime_module.asyncio, "run_coroutine_threadsafe", schedule)++ with TemporaryCodexHome.create(tmp_path) as home:+ with monkeypatch.context() as construction_patch:+ construction_patch.setattr(+ runtime_module.threading, "Lock", lambda: observed_lock+ )+ runtime = CodexRuntime(cast(AsyncCodex, client), home)+ runtime._register_dynamic_tool_handler("thread-1", execute, timeout_seconds=1)+ dispatch = asyncio.create_task(+ asyncio.to_thread(+ sync_client._approval_handler,+ "item/tool/call",+ {+ "threadId": "thread-1",+ "turnId": "turn-1",+ "callId": "call-1",+ "tool": "find_agent",+ "arguments": {},+ },+ )+ )+ assert await asyncio.to_thread(scheduler_entered.wait, 1)++ unregister_done = threading.Event()++ def unregister() -> None:+ runtime._unregister_dynamic_tool_handler("thread-1")+ unregister_done.set()++ unregister_thread = threading.Thread(+ target=unregister,+ name="codex-unregister",+ )+ unregister_thread.start()+ assert await asyncio.to_thread(observed_lock.unregister_attempted.wait, 1)+ if scheduler_held_lock == [True]:+ assert not observed_lock.unregister_acquired.is_set()+ else:+ assert await asyncio.to_thread(unregister_done.wait, 1)++ allow_scheduler_return.set()+ assert await asyncio.to_thread(unregister_done.wait, 1)+ unregister_thread.join(timeout=1)+ assert not unregister_thread.is_alive()+ response = await dispatch++ assert scheduler_held_lock == [True]+ assert future.cancelled()+ assert "thread-1" not in runtime._dynamic_tool_handlers+ assert "thread-1" not in runtime._dynamic_tool_futures+ assert response == {+ "contentItems": [{"type": "inputText", "text": "codex_tool_execution_failed"}],+ "success": False,+ }+++def test_concurrent_dispatcher_keeps_reading_while_tool_result_is_pending():+ handler_started = threading.Event()+ allow_handler = threading.Event()+ notification_routed = threading.Event()+ response_written = threading.Event()+ messages = [+ {"id": "request-1", "method": "item/tool/call", "params": {}},+ {"method": "item/agentMessage/delta", "params": {"delta": "still-live"}},+ ]++ class Router:+ def route_notification(self, _notification):+ notification_routed.set()++ def route_response(self, _message):+ raise AssertionError("unexpected response")++ def fail_all(self, _error):+ pass++ class SyncClient:+ _proc = object()+ _router = Router()++ def _read_message(self):+ if messages:+ return messages.pop(0)+ raise EOFError++ def _handle_server_request(self, _message):+ handler_started.set()+ assert allow_handler.wait(timeout=1)+ return {"success": True}++ def _write_message(self, message):+ assert message == {+ "id": "request-1",+ "result": {"success": True},+ }+ response_written.set()++ def _coerce_notification(self, method, params):+ return method, params++ sync_client = SyncClient()+ client = SimpleNamespace(_client=SimpleNamespace(_sync=sync_client))+ _install_concurrent_server_request_dispatcher(client) # type: ignore[arg-type]++ reader = threading.Thread(target=sync_client._reader_loop, daemon=True)+ reader.start()+ assert handler_started.wait(timeout=1)+ assert notification_routed.wait(timeout=1)+ assert not response_written.is_set()+ allow_handler.set()+ assert response_written.wait(timeout=1)+++def test_concurrent_dispatcher_rejects_requests_above_its_bound(monkeypatch):+ monkeypatch.setattr(runtime_module, "_SERVER_REQUEST_MAX_CONCURRENCY", 1)+ handler_started = threading.Event()+ allow_handler = threading.Event()+ notification_routed = threading.Event()+ first_response_written = threading.Event()+ overload_response_written = threading.Event()+ messages = [+ {"id": "request-1", "method": "item/tool/call", "params": {}},+ {"id": "request-2", "method": "item/tool/call", "params": {}},+ {"method": "item/agentMessage/delta", "params": {"delta": "still-live"}},+ ]++ class Router:+ def route_notification(self, _notification):+ notification_routed.set()++ def route_response(self, _message):+ raise AssertionError("unexpected response")++ def fail_all(self, _error):+ pass++ class SyncClient:+ _proc = object()+ _router = Router()++ def _read_message(self):+ if messages:+ return messages.pop(0)+ raise EOFError++ def _handle_server_request(self, message):+ assert message["id"] == "request-1"+ handler_started.set()+ assert allow_handler.wait(timeout=1)+ return {"success": True}++ def _write_message(self, message):+ if message.get("id") == "request-1":+ assert message == {+ "id": "request-1",+ "result": {"success": True},+ }+ first_response_written.set()+ return+ assert message == {+ "id": "request-2",+ "error": {+ "code": -32001,+ "message": "Codex tool bridge is busy",+ },+ }+ overload_response_written.set()++ def _coerce_notification(self, method, params):+ return method, params++ sync_client = SyncClient()+ client = SimpleNamespace(_client=SimpleNamespace(_sync=sync_client))+ _install_concurrent_server_request_dispatcher(client) # type: ignore[arg-type]++ reader = threading.Thread(target=sync_client._reader_loop, daemon=True)+ reader.start()+ assert handler_started.wait(timeout=1)+ assert overload_response_written.wait(timeout=1)+ assert notification_routed.wait(timeout=1)+ allow_handler.set()+ assert first_response_written.wait(timeout=1)+++async def test_runtime_close_drains_accepted_server_requests(tmp_path):+ handler_started = threading.Event()+ allow_handler = threading.Event()+ response_written = threading.Event()+ client_close_called = asyncio.Event()+ messages = [{"id": "request-1", "method": "item/tool/call", "params": {}}]++ class Router:+ def route_notification(self, _notification):+ raise AssertionError("unexpected notification")++ def route_response(self, _message):+ raise AssertionError("unexpected response")++ def fail_all(self, _error):+ pass++ class SyncClient:+ _proc = object()+ _router = Router()++ def _read_message(self):+ if messages:+ return messages.pop(0)+ raise EOFError++ def _handle_server_request(self, _message):+ handler_started.set()+ assert allow_handler.wait(timeout=1)+ return {"success": True}++ def _write_message(self, message):+ assert self._proc is not None+ assert message == {+ "id": "request-1",+ "result": {"success": True},+ }+ response_written.set()++ def _coerce_notification(self, method, params):+ return method, params++ sync_client = SyncClient()++ class Client:+ _client = SimpleNamespace(_sync=sync_client)++ async def close(self):+ client_close_called.set()+ sync_client._proc = None++ client = Client()+ _install_concurrent_server_request_dispatcher(client) # type: ignore[arg-type]+ reader = threading.Thread(target=sync_client._reader_loop, daemon=True)+ reader.start()+ assert handler_started.wait(timeout=1)++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ client,+ home,+ close_timeout_seconds=1,+ )+ closing = asyncio.create_task(runtime.close())+ await asyncio.sleep(0.01)+ assert not client_close_called.is_set()+ allow_handler.set()+ await asyncio.wait_for(closing, timeout=1)++ assert response_written.is_set()+ assert client_close_called.is_set()+++async def test_runtime_close_abandons_stuck_server_request_after_deadline(tmp_path):+ handler_started = threading.Event()+ allow_handler = threading.Event()+ handler_finished = threading.Event()+ response_written = threading.Event()+ client_close_called = asyncio.Event()+ messages = [{"id": "request-1", "method": "item/tool/call", "params": {}}]++ class Router:+ def route_notification(self, _notification):+ raise AssertionError("unexpected notification")++ def route_response(self, _message):+ raise AssertionError("unexpected response")++ def fail_all(self, _error):+ pass++ class SyncClient:+ _proc = object()+ _router = Router()++ def _read_message(self):+ if messages:+ return messages.pop(0)+ raise EOFError++ def _handle_server_request(self, _message):+ handler_started.set()+ assert allow_handler.wait(timeout=1)+ handler_finished.set()+ return {"success": True}++ def _write_message(self, _message):+ if self._proc is None:+ raise RuntimeError("process closed")+ response_written.set()++ def _coerce_notification(self, method, params):+ return method, params++ sync_client = SyncClient()++ class Client:+ _client = SimpleNamespace(_sync=sync_client)++ async def close(self):+ client_close_called.set()+ sync_client._proc = None++ client = Client()+ _install_concurrent_server_request_dispatcher(client) # type: ignore[arg-type]+ reader = threading.Thread(target=sync_client._reader_loop, daemon=True)+ reader.start()+ assert handler_started.wait(timeout=1)++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ client,+ home,+ close_timeout_seconds=0.01,+ )+ await asyncio.wait_for(runtime.close(), timeout=0.2)++ assert client_close_called.is_set()+ assert not response_written.is_set()+ allow_handler.set()+ assert handler_finished.wait(timeout=1)+++async def test_runtime_close_force_stops_child_when_client_close_never_returns(+ tmp_path,+):+ close_started = asyncio.Event()++ class ProcessIgnoringTerminate:+ stdin = None++ def __init__(self):+ self.terminated = False+ self.killed = False++ def terminate(self):+ self.terminated = True++ def poll(self):+ return None++ def kill(self):+ self.killed = True++ process = ProcessIgnoringTerminate()++ class NeverClosingClient:+ _client = SimpleNamespace(_sync=SimpleNamespace(_proc=process))++ async def close(self):+ close_started.set()+ await asyncio.Event().wait()++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ NeverClosingClient(),+ home,+ close_timeout_seconds=0.01,+ )+ await asyncio.wait_for(runtime.close(), timeout=0.2)++ assert close_started.is_set()+ assert process.terminated+ assert process.killed+++async def test_stream_close_timeout_force_closes_runtime(tmp_path):+ class NeverClosingStream:+ async def aclose(self):+ await asyncio.Event().wait()++ client = SimpleNamespace(+ close=AsyncMock(),+ _client=SimpleNamespace(_sync=SimpleNamespace(_proc=None)),+ )+ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ client,+ home,+ close_timeout_seconds=0.01,+ )+ with pytest.raises(CodexRuntimeError, match="codex_stream_shutdown_timeout"):+ await asyncio.wait_for(+ runtime._close_agent_stream(NeverClosingStream()),+ timeout=0.2,+ )++ client.close.assert_awaited_once()+++async def test_cancelled_start_does_not_wait_forever_for_stuck_enter(+ tmp_path,+ monkeypatch,+):+ entered = asyncio.Event()+ release = asyncio.Event()++ class StuckStartupClient:+ def __init__(self, _config):+ self.closed = False+ self._client = SimpleNamespace(_sync=SimpleNamespace(_proc=None))++ async def __aenter__(self):+ entered.set()+ while not release.is_set():+ try:+ await release.wait()+ except asyncio.CancelledError:+ continue+ return self++ async def close(self):+ self.closed = True++ fake_client = None++ def create_client(config):+ nonlocal fake_client+ fake_client = StuckStartupClient(config)+ return fake_client++ monkeypatch.setattr(runtime_module, "AsyncCodex", create_client)+ monkeypatch.setattr(runtime_module, "assert_pinned_versions", lambda: None)+ monkeypatch.setattr(+ runtime_module,+ "_install_concurrent_server_request_dispatcher",+ lambda _client: None,+ )+ monkeypatch.setattr(runtime_module, "_RUNTIME_CLOSE_TIMEOUT_SECONDS", 0.01)++ with TemporaryCodexHome.create(tmp_path) as home:+ startup = asyncio.create_task(CodexRuntime.start(home))+ await asyncio.wait_for(entered.wait(), timeout=1)+ startup.cancel()+ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(startup, timeout=0.2)++ assert fake_client is not None+ assert fake_client.closed+ release.set()+ await asyncio.sleep(0)+++async def test_cancelled_invoke_does_not_wait_for_stuck_turn_start(tmp_path):+ started = asyncio.Event()+ release = asyncio.Event()+ stopped = asyncio.Event()++ class StuckThread:+ async def turn(self, *_args, **_kwargs):+ started.set()+ while not release.is_set():+ try:+ await release.wait()+ except asyncio.CancelledError:+ continue+ stopped.set()+ return SimpleNamespace()++ client = SimpleNamespace(+ close=AsyncMock(),+ _client=SimpleNamespace(_sync=SimpleNamespace(_proc=None)),+ )+ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ client,+ home,+ close_timeout_seconds=0.01,+ )+ runtime._start_toolless_thread = AsyncMock( # type: ignore[method-assign]+ return_value=StuckThread()+ )+ invocation = asyncio.create_task(+ runtime.invoke(CodexInvocationRequest(prompt="hello"))+ )+ await asyncio.wait_for(started.wait(), timeout=1)+ invocation.cancel()+ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(invocation, timeout=0.2)++ client.close.assert_awaited_once()+ release.set()+ await asyncio.wait_for(stopped.wait(), timeout=1)+++async def test_cancelled_agent_turn_start_recovers_and_interrupts_without_closing(+ tmp_path,+):+ started = asyncio.Event()+ release = asyncio.Event()+ interrupt = AsyncMock()++ class RecoverableThread:+ id = "thread-1"++ async def turn(self, *_args, **_kwargs):+ started.set()+ await release.wait()+ return SimpleNamespace(id="turn-1", interrupt=interrupt)++ client = SimpleNamespace(+ close=AsyncMock(),+ _client=SimpleNamespace(+ _sync=SimpleNamespace(_approval_handler=None, _proc=None)+ ),+ )++ async def execute(_call):+ return CodexDynamicToolResult(content="ok", success=True)++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ client,+ home,+ close_timeout_seconds=0.1,+ )+ runtime._start_thread = AsyncMock( # type: ignore[method-assign]+ return_value=RecoverableThread()+ )+ invocation = asyncio.create_task(+ runtime.invoke_agent(+ CodexInvocationRequest(prompt="hello"),+ [],+ execute,+ tool_timeout_seconds=1,+ )+ )+ await asyncio.wait_for(started.wait(), timeout=1)+ invocation.cancel()+ release.set()+ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(invocation, timeout=1)++ assert not runtime.closed++ interrupt.assert_awaited_once()+ client.close.assert_not_awaited()+++async def test_cancelled_agent_turn_start_poisoned_when_handle_cannot_be_recovered(+ tmp_path,+):+ started = asyncio.Event()+ release = asyncio.Event()+ stopped = asyncio.Event()++ class StuckThread:+ id = "thread-1"++ async def turn(self, *_args, **_kwargs):+ started.set()+ while not release.is_set():+ try:+ await release.wait()+ except asyncio.CancelledError:+ continue+ stopped.set()+ return SimpleNamespace(id="turn-1")++ client = SimpleNamespace(+ close=AsyncMock(),+ _client=SimpleNamespace(+ _sync=SimpleNamespace(_approval_handler=None, _proc=None)+ ),+ )++ async def execute(_call):+ return CodexDynamicToolResult(content="ok", success=True)++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime( # type: ignore[arg-type]+ client,+ home,+ close_timeout_seconds=0.01,+ )+ runtime._start_thread = AsyncMock( # type: ignore[method-assign]+ return_value=StuckThread()+ )+ invocation = asyncio.create_task(+ runtime.invoke_agent(+ CodexInvocationRequest(prompt="hello"),+ [],+ execute,+ tool_timeout_seconds=1,+ )+ )+ await asyncio.wait_for(started.wait(), timeout=1)+ invocation.cancel()+ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(invocation, timeout=1)++ assert runtime.closed++ client.close.assert_awaited_once()+ release.set()+ await asyncio.wait_for(stopped.wait(), timeout=1)+++async def test_agent_stream_failure_interrupts_without_closing_sibling_runtime(+ tmp_path,+):+ interrupt = AsyncMock()++ async def failing_stream():+ if False:+ yield None+ raise RuntimeError("stream failed")++ turn = SimpleNamespace(+ id="turn-1",+ interrupt=interrupt,+ stream=failing_stream,+ )+ thread = SimpleNamespace(+ id="thread-1",+ turn=AsyncMock(return_value=turn),+ )+ client = SimpleNamespace(+ close=AsyncMock(),+ _client=SimpleNamespace(+ _sync=SimpleNamespace(_approval_handler=None, _proc=None)+ ),+ )++ async def execute(_call):+ return CodexDynamicToolResult(content="ok", success=True)++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ runtime._start_thread = AsyncMock( # type: ignore[method-assign]+ return_value=thread+ )+ with pytest.raises(RuntimeError, match="stream failed"):+ await runtime.invoke_agent(+ CodexInvocationRequest(prompt="hello"),+ [],+ execute,+ tool_timeout_seconds=1,+ )++ assert not runtime.closed++ interrupt.assert_awaited_once()+ client.close.assert_not_awaited()+++async def test_agent_event_failure_poisoned_when_interrupt_fails(tmp_path):+ interrupt = AsyncMock(side_effect=RuntimeError("interrupt failed"))++ async def one_event_stream():+ yield SimpleNamespace()++ turn = SimpleNamespace(+ id="turn-1",+ interrupt=interrupt,+ stream=one_event_stream,+ )+ thread = SimpleNamespace(+ id="thread-1",+ turn=AsyncMock(return_value=turn),+ )+ client = SimpleNamespace(+ close=AsyncMock(),+ _client=SimpleNamespace(+ _sync=SimpleNamespace(_approval_handler=None, _proc=None)+ ),+ )++ async def execute(_call):+ return CodexDynamicToolResult(content="ok", success=True)++ async def reject_event(_event):+ raise RuntimeError("event failed")++ with TemporaryCodexHome.create(tmp_path) as home:+ runtime = CodexRuntime(client, home) # type: ignore[arg-type]+ runtime._start_thread = AsyncMock( # type: ignore[method-assign]+ return_value=thread+ )+ with pytest.raises(RuntimeError, match="event failed"):+ await runtime.invoke_agent(+ CodexInvocationRequest(prompt="hello"),+ [],+ execute,+ reject_event,+ tool_timeout_seconds=1,+ )++ assert runtime.closed++ interrupt.assert_awaited_once()+ client.close.assert_awaited_once()diff --git a/autogpt_platform/backend/backend/integrations/codex/temporary_home.py b/autogpt_platform/backend/backend/integrations/codex/temporary_home.pynew file mode 100644index 0000000000..ebc5862d26--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/temporary_home.py@@ -0,0 +1,265 @@+import asyncio+import logging+import os+import shutil+import stat+import tempfile+import threading+import time+import uuid+from collections.abc import Callable+from contextlib import suppress+from pathlib import Path+from types import TracebackType++logger = logging.getLogger(__name__)++_HOME_PREFIX = "autogpt-codex-"+_QUARANTINE_PREFIX = ".quarantine-autogpt-codex-"+_REAPER_MARKER_PREFIX = ".reap-"+_CLEANUP_RETRY_DELAYS_SECONDS = (0.0, 0.02, 0.05, 0.1, 0.2, 0.4)+_REAPER_LOCK = threading.Lock()+_REAPER_TARGETS: set[tuple[Path, Path, Path]] = set()+_REAPER_THREAD: threading.Thread | None = None+++class TemporaryCodexHomeCleanupError(RuntimeError):+ pass+++class TemporaryCodexHome:+ def __init__(self, path: Path, root: Path) -> None:+ _validate_managed_target(path, root)+ self.path = path+ self.root = root+ self.auth_path = path / "auth.json"+ self.temp_path = path / "tmp"+ self.workspace_path = path / "workspace"+ self._cleaned = False++ @classmethod+ def create(cls, root: Path | None = None) -> "TemporaryCodexHome":+ resolved_root = _prepare_root(root)+ _schedule_existing_reaps(resolved_root)+ path = Path(+ tempfile.mkdtemp(+ prefix=_HOME_PREFIX,+ dir=str(resolved_root),+ )+ ).resolve()+ _validate_managed_target(path, resolved_root)+ home = cls(path, resolved_root)+ try:+ os.chmod(path, 0o700)+ home.temp_path.mkdir(mode=0o700)+ home.workspace_path.mkdir(mode=0o700)+ except BaseException:+ home.cleanup()+ raise+ return home++ def __enter__(self) -> "TemporaryCodexHome":+ return self++ def __exit__(+ self,+ _exc_type: type[BaseException] | None,+ _exc: BaseException | None,+ _traceback: TracebackType | None,+ ) -> None:+ self.cleanup()++ async def __aenter__(self) -> "TemporaryCodexHome":+ return self++ async def __aexit__(+ self,+ _exc_type: type[BaseException] | None,+ _exc: BaseException | None,+ _traceback: TracebackType | None,+ ) -> None:+ await asyncio.to_thread(self.cleanup)++ def cleanup(self) -> None:+ if self._cleaned:+ return+ _validate_managed_target(self.path, self.root)+ if _remove_with_retry(self.path, self.root):+ self._cleaned = True+ return++ quarantined = _quarantine(self.path, self.root)+ if quarantined is not None:+ _schedule_reap(self.root, quarantined)+ self._cleaned = True+ return++ _schedule_reap(self.root, self.path)+ raise TemporaryCodexHomeCleanupError(+ "Could not remove or quarantine isolated Codex auth home"+ )+++def _prepare_root(root: Path | None) -> Path:+ candidate = (root or Path(tempfile.gettempdir()) / "autogpt-codex").absolute()+ candidate = _canonicalize_trusted_temp_base(candidate)+ for component in (candidate, *candidate.parents):+ if os.path.lexists(component) and component.is_symlink():+ raise ValueError("Codex temporary root cannot contain symbolic links")+ candidate.mkdir(mode=0o700, parents=True, exist_ok=True)+ if candidate.is_symlink():+ raise ValueError("Codex temporary root cannot contain symbolic links")+ resolved = candidate.resolve()+ if resolved == Path(resolved.anchor):+ raise ValueError("Codex temporary root cannot be a filesystem root")+ os.chmod(resolved, 0o700)+ if os.name != "nt" and stat.S_IMODE(resolved.stat().st_mode) != 0o700:+ raise PermissionError("Codex temporary root must have mode 0700")+ return resolved+++def _canonicalize_trusted_temp_base(candidate: Path) -> Path:+ for base in sorted(+ _trusted_system_temp_bases(), key=lambda path: len(path.parts), reverse=True+ ):+ if not os.path.lexists(base):+ continue+ try:+ relative = candidate.relative_to(base)+ except ValueError:+ continue+ if relative == Path("."):+ return candidate+ return base.resolve() / relative+ return candidate+++def _trusted_system_temp_bases() -> tuple[Path, ...]:+ if os.name == "nt":+ return ()+ return (Path("/tmp"), Path("/var"))+++def _validate_managed_target(path: Path, root: Path) -> None:+ resolved_root = root.resolve()+ if resolved_root == Path(resolved_root.anchor):+ raise ValueError("Codex temporary root cannot be a filesystem root")+ if path.parent.resolve() != resolved_root:+ raise ValueError("Codex temporary home must be a direct child of its root")+ if not path.name.startswith((_HOME_PREFIX, _QUARANTINE_PREFIX)):+ raise ValueError("Refusing to manage an unexpected Codex temporary path")+++def _remove_with_retry(+ path: Path,+ root: Path,+ delays: tuple[float, ...] = _CLEANUP_RETRY_DELAYS_SECONDS,+) -> bool:+ for delay in delays:+ if delay:+ time.sleep(delay)+ try:+ _remove_tree_once(path, root)+ return True+ except FileNotFoundError:+ return True+ except OSError:+ continue+ return not os.path.lexists(path)+++def _remove_tree_once(path: Path, root: Path) -> None:+ _validate_managed_target(path, root)+ if not os.path.lexists(path):+ return+ if path.is_symlink():+ path.unlink()+ return+ shutil.rmtree(path, onerror=_make_writable_and_retry)+++def _make_writable_and_retry(+ operation: Callable[..., object],+ path: str,+ _error: object,+) -> None:+ os.chmod(path, stat.S_IRWXU)+ operation(path)+++def _quarantine(path: Path, root: Path) -> Path | None:+ _validate_managed_target(path, root)+ if not os.path.lexists(path):+ return None+ quarantined = root / f"{_QUARANTINE_PREFIX}{uuid.uuid4().hex}"+ _validate_managed_target(quarantined, root)+ try:+ os.replace(path, quarantined)+ except OSError:+ return None+ return quarantined+++def _schedule_reap(root: Path, target: Path) -> None:+ _validate_managed_target(target, root)+ marker = root / f"{_REAPER_MARKER_PREFIX}{target.name}"+ descriptor = os.open(marker, os.O_CREAT | os.O_WRONLY, 0o600)+ os.close(descriptor)+ os.chmod(marker, 0o600)++ global _REAPER_THREAD+ with _REAPER_LOCK:+ _REAPER_TARGETS.add((root, target, marker))+ if _REAPER_THREAD is None or not _REAPER_THREAD.is_alive():+ _REAPER_THREAD = threading.Thread(+ target=_reaper_loop,+ name="codex-auth-home-reaper",+ daemon=True,+ )+ _REAPER_THREAD.start()+++def _schedule_existing_reaps(root: Path) -> None:+ for target in root.glob(f"{_QUARANTINE_PREFIX}*"):+ if target.parent == root:+ _schedule_reap(root, target)+ for marker in root.glob(f"{_REAPER_MARKER_PREFIX}*"):+ target_name = marker.name.removeprefix(_REAPER_MARKER_PREFIX)+ if not target_name.startswith((_HOME_PREFIX, _QUARANTINE_PREFIX)):+ continue+ target = root / target_name+ if os.path.lexists(target):+ _schedule_reap(root, target)+ else:+ with suppress(OSError):+ marker.unlink(missing_ok=True)+++def _reaper_loop() -> None:+ backoff_seconds = 0.05+ while True:+ with _REAPER_LOCK:+ pending = tuple(_REAPER_TARGETS)+ if not pending:+ global _REAPER_THREAD+ _REAPER_THREAD = None+ return++ removed: list[tuple[Path, Path, Path]] = []+ for root, target, marker in pending:+ if _remove_with_retry(target, root, delays=(0.0,)):+ with suppress(OSError):+ marker.unlink(missing_ok=True)+ removed.append((root, target, marker))++ with _REAPER_LOCK:+ _REAPER_TARGETS.difference_update(removed)+ if removed:+ backoff_seconds = 0.05+ continue++ logger.warning(+ "Retrying cleanup of %d isolated Codex auth home(s)", len(pending)+ )+ time.sleep(backoff_seconds)+ backoff_seconds = min(backoff_seconds * 2, 2.0)diff --git a/autogpt_platform/backend/backend/integrations/codex/temporary_home_test.py b/autogpt_platform/backend/backend/integrations/codex/temporary_home_test.pynew file mode 100644index 0000000000..d22de86bbb--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/temporary_home_test.py@@ -0,0 +1,311 @@+import asyncio+import os+import shutil+import threading+import time+from pathlib import Path++import pytest++from backend.integrations.codex import temporary_home+from backend.integrations.codex.temporary_home import (+ TemporaryCodexHome,+ TemporaryCodexHomeCleanupError,+)+++def test_temporary_home_creates_private_workspace_and_cleans_up(tmp_path):+ with TemporaryCodexHome.create(tmp_path) as home:+ path = home.path+ assert home.root == tmp_path.resolve()+ assert path.parent == tmp_path.resolve()+ assert home.auth_path == path / "auth.json"+ assert home.workspace_path.is_dir()+ if os.name != "nt":+ assert path.stat().st_mode & 0o777 == 0o700+ assert home.workspace_path.stat().st_mode & 0o777 == 0o700++ assert not path.exists()+++def test_temporary_home_cleans_up_after_failure(tmp_path):+ with pytest.raises(RuntimeError, match="expected"):+ with TemporaryCodexHome.create(tmp_path) as home:+ path = home.path+ raise RuntimeError("expected")++ assert not path.exists()+++def test_temporary_home_cleans_up_if_initialization_fails(tmp_path, monkeypatch):+ mkdir = Path.mkdir++ def fail_workspace(path: Path, *args, **kwargs) -> None:+ if path.name == "workspace":+ raise PermissionError("workspace creation failed")+ mkdir(path, *args, **kwargs)++ monkeypatch.setattr(Path, "mkdir", fail_workspace)++ with pytest.raises(PermissionError, match="workspace creation failed"):+ TemporaryCodexHome.create(tmp_path)++ assert not list(tmp_path.glob("autogpt-codex-*"))+++async def test_temporary_home_cleans_up_after_cancellation(tmp_path):+ entered = asyncio.Event()++ async def wait_forever() -> None:+ async with TemporaryCodexHome.create(tmp_path) as home:+ nonlocal path+ path = home.path+ entered.set()+ await asyncio.Event().wait()++ path = tmp_path+ task = asyncio.create_task(wait_forever())+ await entered.wait()+ task.cancel()++ with pytest.raises(asyncio.CancelledError):+ await task++ assert not path.exists()+++async def test_async_exit_runs_cleanup_off_the_event_loop(tmp_path, monkeypatch):+ home = TemporaryCodexHome.create(tmp_path)+ event_loop_thread = threading.get_ident()+ cleanup_threads: list[int] = []+ cleanup = home.cleanup++ def observed_cleanup() -> None:+ cleanup_threads.append(threading.get_ident())+ cleanup()++ monkeypatch.setattr(home, "cleanup", observed_cleanup)++ async with home:+ pass++ assert cleanup_threads+ assert cleanup_threads[0] != event_loop_thread+ assert not home.path.exists()+++def test_cleanup_is_idempotent(tmp_path):+ home = TemporaryCodexHome.create(tmp_path)+ path = home.path++ home.cleanup()+ home.cleanup()++ assert not path.exists()+++def test_cleanup_retries_transient_filesystem_locks(tmp_path, monkeypatch):+ home = TemporaryCodexHome.create(tmp_path)+ path = home.path+ attempts = 0+ remove_tree_once = temporary_home._remove_tree_once++ def fail_twice(target: Path, root: Path) -> None:+ nonlocal attempts+ attempts += 1+ if attempts < 3:+ raise PermissionError("transient lock")+ remove_tree_once(target, root)++ monkeypatch.setattr(temporary_home, "_remove_tree_once", fail_twice)+ monkeypatch.setattr(temporary_home.time, "sleep", lambda _seconds: None)++ home.cleanup()++ assert attempts == 3+ assert not path.exists()+++def test_cleanup_quarantines_a_locked_home_under_the_dedicated_root(+ tmp_path, monkeypatch+):+ home = TemporaryCodexHome.create(tmp_path)+ home.auth_path.write_text("fake-sensitive-auth", encoding="utf-8")+ scheduled: list[tuple[Path, Path]] = []+ monkeypatch.setattr(temporary_home, "_remove_with_retry", lambda *_args: False)+ monkeypatch.setattr(+ temporary_home,+ "_schedule_reap",+ lambda root, target: scheduled.append((root, target)),+ )++ home.cleanup()++ assert not home.path.exists()+ assert len(scheduled) == 1+ root, quarantined = scheduled[0]+ assert root == tmp_path.resolve()+ assert quarantined.parent == root+ assert quarantined.name.startswith(".quarantine-autogpt-codex-")+ assert (quarantined / "auth.json").read_text(encoding="utf-8") == (+ "fake-sensitive-auth"+ )+ shutil.rmtree(quarantined)+++def test_cleanup_fails_loudly_if_home_cannot_be_removed_or_quarantined(+ tmp_path, monkeypatch+):+ home = TemporaryCodexHome.create(tmp_path)+ scheduled: list[tuple[Path, Path]] = []+ monkeypatch.setattr(temporary_home, "_remove_with_retry", lambda *_args: False)+ monkeypatch.setattr(temporary_home, "_quarantine", lambda *_args: None)+ monkeypatch.setattr(+ temporary_home,+ "_schedule_reap",+ lambda root, target: scheduled.append((root, target)),+ )++ with pytest.raises(TemporaryCodexHomeCleanupError):+ home.cleanup()++ assert scheduled == [(tmp_path.resolve(), home.path)]+ assert home.path.exists()+ shutil.rmtree(home.path)+++def test_temporary_home_rejects_paths_outside_the_dedicated_root(tmp_path):+ root = tmp_path / "managed"+ root.mkdir()+ outside = tmp_path / "autogpt-codex-outside"+ outside.mkdir()++ with pytest.raises(ValueError, match="direct child"):+ TemporaryCodexHome(outside, root.resolve())+++def test_temporary_home_rejects_a_filesystem_root():+ filesystem_root = Path(Path.cwd().anchor)++ with pytest.raises(ValueError, match="filesystem root"):+ temporary_home._prepare_root(filesystem_root)+++def test_temporary_home_rejects_a_preexisting_symlink_root(tmp_path):+ target = tmp_path / "attacker-controlled"+ target.mkdir()+ link = tmp_path / "autogpt-codex"+ try:+ link.symlink_to(target, target_is_directory=True)+ except OSError as error:+ pytest.skip(f"symlinks are unavailable: {error}")++ with pytest.raises(ValueError, match="symbolic link"):+ temporary_home._prepare_root(link)+++def test_temporary_home_rejects_a_symlink_parent(tmp_path):+ target = tmp_path / "attacker-controlled"+ target.mkdir()+ link = tmp_path / "managed-parent"+ try:+ link.symlink_to(target, target_is_directory=True)+ except OSError as error:+ pytest.skip(f"symlinks are unavailable: {error}")++ with pytest.raises(ValueError, match="symbolic link"):+ temporary_home._prepare_root(link / "autogpt-codex")+++def test_temporary_home_accepts_a_symlinked_system_temp_base(tmp_path, monkeypatch):+ target = tmp_path / "private-temp"+ target.mkdir()+ link = tmp_path / "system-temp"+ try:+ link.symlink_to(target, target_is_directory=True)+ except OSError as error:+ pytest.skip(f"symlinks are unavailable: {error}")+ monkeypatch.setattr(temporary_home.tempfile, "gettempdir", lambda: str(link))+ monkeypatch.setattr(temporary_home, "_trusted_system_temp_bases", lambda: (link,))++ root = temporary_home._prepare_root(None)++ assert root == target / "autogpt-codex"+ assert root.is_dir()+ assert not root.is_symlink()+++def test_temporary_home_rejects_a_symlink_below_system_temp_base(tmp_path, monkeypatch):+ target = tmp_path / "private-temp"+ target.mkdir()+ system_link = tmp_path / "system-temp"+ attacker_target = tmp_path / "attacker-controlled"+ attacker_target.mkdir()+ managed_link = target / "managed-parent"+ try:+ system_link.symlink_to(target, target_is_directory=True)+ managed_link.symlink_to(attacker_target, target_is_directory=True)+ except OSError as error:+ pytest.skip(f"symlinks are unavailable: {error}")+ monkeypatch.setattr(temporary_home.tempfile, "gettempdir", lambda: str(system_link))+ monkeypatch.setattr(+ temporary_home, "_trusted_system_temp_bases", lambda: (system_link,)+ )++ with pytest.raises(ValueError, match="symbolic link"):+ temporary_home._prepare_root(system_link / "managed-parent" / "autogpt-codex")+++def test_temporary_home_does_not_trust_a_symlink_from_gettempdir(tmp_path, monkeypatch):+ target = tmp_path / "private-temp"+ target.mkdir()+ link = tmp_path / "user-temp"+ try:+ link.symlink_to(target, target_is_directory=True)+ except OSError as error:+ pytest.skip(f"symlinks are unavailable: {error}")+ monkeypatch.setattr(temporary_home.tempfile, "gettempdir", lambda: str(link))+ monkeypatch.setattr(temporary_home, "_trusted_system_temp_bases", lambda: ())++ with pytest.raises(ValueError, match="symbolic link"):+ temporary_home._prepare_root(None)++[email protected](os.name == "nt", reason="POSIX mode bits are not ACLs")+def test_temporary_home_rejects_root_when_private_mode_cannot_be_set(+ tmp_path, monkeypatch+):+ root = tmp_path / "shared"+ root.mkdir(mode=0o755)+ os.chmod(root, 0o755)+ monkeypatch.setattr(temporary_home.os, "chmod", lambda *_args: None)++ with pytest.raises(PermissionError, match="mode 0700"):+ temporary_home._prepare_root(root)+++def test_reaper_recovers_a_marked_quarantine_after_restart(tmp_path):+ root = temporary_home._prepare_root(tmp_path)+ quarantined = root / ".quarantine-autogpt-codex-stale"+ quarantined.mkdir(mode=0o700)+ (quarantined / "auth.json").write_text("fake-sensitive-auth", encoding="utf-8")+ marker = root / f".reap-{quarantined.name}"+ marker.touch(mode=0o600)++ try:+ temporary_home._schedule_existing_reaps(root)++ deadline = time.monotonic() + 2+ while (quarantined.exists() or marker.exists()) and time.monotonic() < deadline:+ time.sleep(0.01)++ assert not quarantined.exists()+ assert not marker.exists()+ finally:+ with temporary_home._REAPER_LOCK:+ temporary_home._REAPER_TARGETS.clear()+ reaper = temporary_home._REAPER_THREAD+ if reaper is not None:+ reaper.join(timeout=2)diff --git a/autogpt_platform/backend/backend/integrations/codex/transport.py b/autogpt_platform/backend/backend/integrations/codex/transport.pynew file mode 100644index 0000000000..d8b3b158d1--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/transport.py@@ -0,0 +1,1598 @@+import asyncio+import concurrent.futures+import inspect+import logging+import threading+from collections.abc import AsyncIterator, Awaitable, Callable+from contextlib import AbstractAsyncContextManager, asynccontextmanager, suppress+from pathlib import Path+from typing import Any, Protocol, TypeVar, cast+from uuid import uuid4++from openai_codex.models import Notification+from pydantic import BaseModel, ConfigDict, Field++from backend.data.model import OAuth2Credentials+from backend.integrations.codex.auth_bundle import (+ CodexAuthBundleError,+ CodexAuthBundleV1,+ auth_bundle_fingerprint,+ materialize_auth_bundle,+ read_auth_bundle,+)+from backend.integrations.codex.credential_codec import (+ bundle_from_credentials,+ checkpoint_credentials_from_bundle,+)+from backend.integrations.codex.models import (+ CodexAccountSnapshot,+ CodexDeviceCodeDetails,+ CodexDynamicToolCall,+ CodexDynamicToolResult,+ CodexDynamicToolSpec,+ CodexInvocationRequest,+ CodexInvocationResult,+ CodexLoginCompletion,+ CodexModelInfo,+ CodexRateLimitsSnapshot,+)+from backend.integrations.codex.runtime import CODEX_RUNTIME_VERSION, CodexRuntime+from backend.integrations.codex.temporary_home import TemporaryCodexHome+from backend.integrations.credential_lease import CredentialLease+from backend.util.settings import Settings++logger = logging.getLogger(__name__)+++class CodexTransportError(RuntimeError):+ pass+++class CodexInvocationTimeoutError(CodexTransportError):+ pass+++class CodexTransportOverloadedError(CodexTransportError):+ pass+++class CodexCredentialBusyError(CodexTransportError):+ pass+++class CodexCredentialIntegrityError(CodexTransportError):+ pass+++ResultT = TypeVar("ResultT")+++class RuntimeLogin(Protocol):+ details: CodexDeviceCodeDetails++ async def wait(self) -> bool: ...++ async def cancel(self) -> None: ...+++class RuntimeClient(Protocol):+ @property+ def closed(self) -> bool: ...++ async def start_device_code_login(self) -> RuntimeLogin: ...++ async def account(+ self,+ *,+ refresh_token: bool = True,+ ) -> CodexAccountSnapshot: ...++ async def rate_limits(self) -> CodexRateLimitsSnapshot: ...++ async def models(self) -> list[CodexModelInfo]: ...++ async def invoke(+ self,+ request: CodexInvocationRequest,+ ) -> CodexInvocationResult: ...++ async def invoke_agent(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None = None,+ *,+ tool_timeout_seconds: float,+ ) -> CodexInvocationResult: ...++ async def logout(self) -> None: ...++ async def close(self) -> None: ...+++RuntimeFactory = Callable[[TemporaryCodexHome], Awaitable[RuntimeClient]]+++class CodexAgentSession:+ def __init__(+ self,+ runtime: "_LeasedRuntime",+ *,+ turn_timeout_seconds: float,+ tool_timeout_seconds: float,+ ) -> None:+ self._runtime = runtime+ self._turn_timeout_seconds = turn_timeout_seconds+ self._tool_timeout_seconds = tool_timeout_seconds++ async def invoke(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None = None,+ ) -> CodexInvocationResult:+ try:+ result = await asyncio.wait_for(+ self._runtime.run(+ self._runtime.runtime.invoke_agent(+ request,+ dynamic_tools,+ tool_handler,+ event_handler,+ tool_timeout_seconds=self._tool_timeout_seconds,+ )+ ),+ timeout=self._turn_timeout_seconds,+ )+ return result.model_copy(+ update={"resolved_model": result.resolved_model or request.model}+ )+ except asyncio.TimeoutError:+ raise CodexInvocationTimeoutError("codex_copilot_turn_timeout") from None++ async def models(self) -> list[CodexModelInfo]:+ return await self._runtime.run(self._runtime.runtime.models())++ @property+ def closed(self) -> bool:+ return self._runtime.runtime.closed++ @property+ def failure(self) -> BaseException | None:+ return self._runtime.failure+++class CodexDeviceLoginSession:+ def __init__(+ self,+ home: TemporaryCodexHome,+ runtime: RuntimeClient,+ login: RuntimeLogin,+ timeout_seconds: float,+ cleanup_timeout_seconds: float,+ release_capacity: Callable[[], None],+ ) -> None:+ self.details = login.details+ self._home = home+ self._runtime = runtime+ self._login = login+ self._timeout_seconds = timeout_seconds+ self._cleanup_timeout_seconds = cleanup_timeout_seconds+ self._release_capacity = release_capacity+ self._closed = False+ self._close_lock = asyncio.Lock()++ async def wait(self) -> CodexLoginCompletion:+ try:+ completed = await asyncio.wait_for(+ self._login.wait(),+ timeout=self._timeout_seconds,+ )+ if not completed:+ raise CodexTransportError("Codex device login failed")+ bundle = read_auth_bundle(self._home.auth_path, CODEX_RUNTIME_VERSION)+ return CodexLoginCompletion(bundle=bundle)+ except asyncio.TimeoutError:+ raise CodexTransportError("Codex device login expired") from None+ finally:+ await self.close()++ async def cancel(self) -> None:+ try:+ await _bounded_phase(+ self._login.cancel(),+ self._cleanup_timeout_seconds,+ "Codex device login cancellation",+ )+ finally:+ await self.close()++ async def close(self) -> None:+ async with self._close_lock:+ if self._closed:+ return+ try:+ await _bounded_phase(+ self._runtime.close(),+ self._cleanup_timeout_seconds,+ "Codex login runtime shutdown",+ )+ finally:+ try:+ self._home.cleanup()+ finally:+ self._release_capacity()+ self._closed = True+++class CodexTransport:+ def __init__(+ self,+ *,+ temp_root: Path | None = None,+ max_active_processes: int = 4,+ capacity_timeout_seconds: float = 10,+ startup_timeout_seconds: float = 30,+ control_timeout_seconds: float = 60,+ invocation_timeout_seconds: float = 180,+ copilot_turn_timeout_seconds: float = 21600,+ copilot_tool_timeout_seconds: float = 900,+ login_timeout_seconds: float = 900,+ checkpoint_interval_seconds: float = 0.25,+ runtime_factory: RuntimeFactory = CodexRuntime.start,+ ) -> None:+ self._temp_root = temp_root+ self._capacity = threading.BoundedSemaphore(max_active_processes)+ self._capacity_timeout_seconds = capacity_timeout_seconds+ self._startup_timeout_seconds = startup_timeout_seconds+ self._control_timeout_seconds = control_timeout_seconds+ self._invocation_timeout_seconds = invocation_timeout_seconds+ self._copilot_turn_timeout_seconds = copilot_turn_timeout_seconds+ self._copilot_tool_timeout_seconds = copilot_tool_timeout_seconds+ self._login_timeout_seconds = login_timeout_seconds+ self._checkpoint_interval_seconds = checkpoint_interval_seconds+ self._runtime_factory = runtime_factory+ self._runtime_pool = _CodexRuntimePool(self)++ async def start_device_login(self) -> CodexDeviceLoginSession:+ await self._acquire_capacity()+ home: TemporaryCodexHome | None = None+ runtime: RuntimeClient | None = None+ try:+ home = TemporaryCodexHome.create(self._temp_root)+ runtime = await _bounded_phase(+ self._runtime_factory(home),+ self._startup_timeout_seconds,+ "Codex runtime startup",+ )+ login = await _bounded_phase(+ runtime.start_device_code_login(),+ self._control_timeout_seconds,+ "Codex device login startup",+ )+ return CodexDeviceLoginSession(+ home,+ runtime,+ login,+ self._login_timeout_seconds,+ self._control_timeout_seconds,+ self._capacity.release,+ )+ except BaseException:+ try:+ if runtime is not None:+ await _bounded_phase(+ runtime.close(),+ self._control_timeout_seconds,+ "Codex runtime shutdown",+ )+ finally:+ try:+ if home is not None:+ home.cleanup()+ finally:+ self._capacity.release()+ raise++ async def acquire_runtime_lease(+ self,+ user_id: str,+ credential_id: str,+ *,+ lock_timeout_seconds: float,+ ) -> "PooledCodexRuntimeLease":+ return await self._runtime_pool.acquire(+ user_id,+ credential_id,+ lock_timeout_seconds=lock_timeout_seconds,+ )++ async def close_runtime_pool(self) -> None:+ await self._runtime_pool.close()++ async def _acquire_capacity(self) -> None:+ loop = asyncio.get_running_loop()+ deadline = loop.time() + self._capacity_timeout_seconds+ while not self._capacity.acquire(blocking=False):+ if loop.time() >= deadline:+ raise CodexTransportOverloadedError(+ "Codex transport has no free process capacity"+ )+ await asyncio.sleep(0.01)++ async def account(self, lease: CredentialLease) -> CodexAccountSnapshot:+ return await self._run_control_operation(+ "account lookup",+ self._account(lease),+ )++ async def _account(self, lease: CredentialLease) -> CodexAccountSnapshot:+ async with self._credential_runtime(lease) as session:+ return await session.run(session.runtime.account(refresh_token=True))++ async def rate_limits(+ self,+ lease: CredentialLease,+ ) -> CodexRateLimitsSnapshot:+ return await self._run_control_operation(+ "rate-limit lookup",+ self._rate_limits(lease),+ )++ async def _rate_limits(+ self,+ lease: CredentialLease,+ ) -> CodexRateLimitsSnapshot:+ async with self._credential_runtime(lease) as session:+ return await session.run(session.runtime.rate_limits())++ async def models(self, lease: CredentialLease) -> list[CodexModelInfo]:+ return await self._run_control_operation(+ "model lookup",+ self._models(lease),+ )++ async def _models(self, lease: CredentialLease) -> list[CodexModelInfo]:+ async with self._credential_runtime(lease) as session:+ return await session.run(session.runtime.models())++ async def invoke(+ self,+ lease: CredentialLease,+ request: CodexInvocationRequest,+ ) -> CodexInvocationResult:+ timeout = request.timeout_seconds or self._invocation_timeout_seconds+ try:+ return await asyncio.wait_for(+ self._invoke_with_runtime(lease, request),+ timeout=timeout,+ )+ except asyncio.TimeoutError:+ raise CodexInvocationTimeoutError(+ f"Codex invocation exceeded {timeout:g} seconds"+ ) from None++ async def _invoke_with_runtime(+ self,+ lease: CredentialLease,+ request: CodexInvocationRequest,+ ) -> CodexInvocationResult:+ async with self._credential_runtime(lease) as session:+ await session.run(session.runtime.account(refresh_token=True))+ await session.checkpoint_now()+ models = await session.run(session.runtime.models())+ resolved_model = _resolve_invocation_model(request.model, models)+ result = await session.run(+ session.runtime.invoke(+ request.model_copy(update={"model": resolved_model})+ )+ )+ return result.model_copy(update={"resolved_model": resolved_model})++ async def invoke_agent(+ self,+ lease: CredentialLease,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None = None,+ ) -> CodexInvocationResult:+ try:+ return await asyncio.wait_for(+ self._invoke_agent_with_runtime(+ lease,+ request,+ dynamic_tools,+ tool_handler,+ event_handler,+ ),+ timeout=self._copilot_turn_timeout_seconds,+ )+ except asyncio.TimeoutError:+ raise CodexInvocationTimeoutError("codex_copilot_turn_timeout") from None++ @asynccontextmanager+ async def agent_session(+ self,+ lease: CredentialLease,+ ) -> AsyncIterator[CodexAgentSession]:+ async with self._credential_runtime(lease) as session:+ await session.run(session.runtime.account(refresh_token=True))+ await session.checkpoint_now()+ yield CodexAgentSession(+ session,+ turn_timeout_seconds=self._copilot_turn_timeout_seconds,+ tool_timeout_seconds=self._copilot_tool_timeout_seconds,+ )++ async def _invoke_agent_with_runtime(+ self,+ lease: CredentialLease,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None,+ ) -> CodexInvocationResult:+ async with self._credential_runtime(lease) as session:+ await session.run(session.runtime.account(refresh_token=True))+ await session.checkpoint_now()+ return await session.run(+ session.runtime.invoke_agent(+ request,+ dynamic_tools,+ tool_handler,+ event_handler,+ tool_timeout_seconds=self._copilot_tool_timeout_seconds,+ )+ )++ async def logout(self, lease: CredentialLease) -> None:+ await self._run_control_operation("logout", self._logout(lease))++ async def _logout(self, lease: CredentialLease) -> None:+ async with self._credential_runtime(lease, checkpoint=False) as session:+ await session.run(session.runtime.logout())++ async def _run_control_operation(+ self,+ label: str,+ operation: Awaitable[ResultT],+ ) -> ResultT:+ return await _bounded_phase(+ operation,+ self._control_timeout_seconds,+ f"Codex {label}",+ )++ @asynccontextmanager+ async def _credential_runtime(+ self,+ lease: CredentialLease,+ *,+ checkpoint: bool = True,+ ) -> AsyncIterator["_LeasedRuntime"]:+ credentials = _codex_credentials(lease)+ bundle = bundle_from_credentials(credentials)+ await self._acquire_capacity()+ home: TemporaryCodexHome | None = None+ runtime: RuntimeClient | None = None+ session: _LeasedRuntime | None = None+ try:+ home = TemporaryCodexHome.create(self._temp_root)+ materialize_auth_bundle(bundle, home.auth_path)+ runtime = await _bounded_phase(+ self._runtime_factory(home),+ self._startup_timeout_seconds,+ "Codex runtime startup",+ )+ state = _AuthCheckpointState(credentials=credentials, bundle=bundle)+ session = _LeasedRuntime(+ runtime=runtime,+ lease=lease,+ state=state,+ home=home,+ checkpoint_timeout_seconds=self._control_timeout_seconds,+ checkpoint_interval_seconds=self._checkpoint_interval_seconds,+ monitor_auth=checkpoint,+ )+ session.start()+ yield session+ finally:+ try:+ if session is not None:+ await session.stop()+ finally:+ try:+ if runtime is not None:+ await _bounded_phase(+ runtime.close(),+ self._control_timeout_seconds,+ "Codex runtime shutdown",+ )+ finally:+ try:+ if checkpoint and session is not None:+ await session.checkpoint_now(validate_unchanged=True)+ finally:+ try:+ if home is not None:+ home.cleanup()+ finally:+ self._capacity.release()+++_CODEX_TRANSPORT_SINGLETON_LOCK = threading.Lock()+_codex_transport_singleton: CodexTransport | None = None+++def get_codex_transport() -> CodexTransport:+ global _codex_transport_singleton+ if _codex_transport_singleton is not None:+ return _codex_transport_singleton+ with _CODEX_TRANSPORT_SINGLETON_LOCK:+ if _codex_transport_singleton is None:+ config = Settings().config+ temp_root = Path(config.codex_temp_root) if config.codex_temp_root else None+ _codex_transport_singleton = CodexTransport(+ temp_root=temp_root,+ max_active_processes=config.codex_max_active_processes,+ capacity_timeout_seconds=config.codex_capacity_timeout_seconds,+ startup_timeout_seconds=config.codex_startup_timeout_seconds,+ control_timeout_seconds=config.codex_control_timeout_seconds,+ invocation_timeout_seconds=config.codex_invocation_timeout_seconds,+ copilot_turn_timeout_seconds=config.codex_copilot_turn_timeout_seconds,+ copilot_tool_timeout_seconds=config.codex_copilot_tool_timeout_seconds,+ login_timeout_seconds=config.codex_login_timeout_seconds,+ checkpoint_interval_seconds=(+ config.codex_auth_checkpoint_interval_seconds+ ),+ )+ return _codex_transport_singleton+++def _codex_credentials(lease: CredentialLease) -> OAuth2Credentials:+ credentials = lease.credentials+ if credentials.type != "oauth2":+ raise CodexTransportError("Codex transport requires OAuth credentials")+ oauth_credentials = cast(OAuth2Credentials, credentials)+ bundle_from_credentials(oauth_credentials)+ return oauth_credentials+++def _resolve_invocation_model(+ requested_model: str | None,+ models: list[CodexModelInfo],+) -> str:+ available = {model.model: model for model in models}+ if requested_model is not None:+ if requested_model not in available:+ raise CodexTransportError(+ f"Codex model is not available for this account: {requested_model}"+ )+ return requested_model+ default = next((model for model in models if model.is_default), None)+ visible = next((model for model in models if not model.hidden), None)+ selected = default or visible or (models[0] if models else None)+ if selected is None:+ raise CodexTransportError("Codex account advertised no available models")+ return selected.model+++class _AuthCheckpointState(BaseModel):+ model_config = ConfigDict(arbitrary_types_allowed=True)++ credentials: OAuth2Credentials+ bundle: CodexAuthBundleV1+ lock: asyncio.Lock = Field(default_factory=asyncio.Lock)+++class _LeasedRuntime:+ def __init__(+ self,+ *,+ runtime: RuntimeClient,+ lease: CredentialLease,+ state: _AuthCheckpointState,+ home: TemporaryCodexHome,+ checkpoint_timeout_seconds: float,+ checkpoint_interval_seconds: float,+ monitor_auth: bool,+ ) -> None:+ self.runtime = runtime+ self._lease = lease+ self._state = state+ self._home = home+ self._checkpoint_timeout_seconds = checkpoint_timeout_seconds+ self._checkpoint_interval_seconds = checkpoint_interval_seconds+ self._monitor_auth = monitor_auth+ self._monitor_task: asyncio.Task[None] | None = None++ def start(self) -> None:+ if self._monitor_auth:+ self._monitor_task = asyncio.create_task(self._monitor_auth_file())++ async def run(self, operation: Awaitable[ResultT]) -> ResultT:+ return await _run_with_lease_guard(+ self._lease,+ operation,+ monitor_task=self._monitor_task,+ )++ @property+ def failure(self) -> BaseException | None:+ monitor = self._monitor_task+ if monitor is not None and monitor.done() and not monitor.cancelled():+ try:+ if error := monitor.exception():+ return error+ except asyncio.CancelledError:+ pass+ return self._lease.failure++ async def checkpoint_now(self, *, validate_unchanged: bool = True) -> None:+ async with self._state.lock:+ try:+ after = await _read_runtime_auth(self._home.auth_path)+ except asyncio.CancelledError:+ raise+ except BaseException as error:+ raise CodexCredentialIntegrityError(+ "codex_credential_state_unavailable"+ ) from error+ if auth_bundle_fingerprint(after) == auth_bundle_fingerprint(+ self._state.bundle+ ):+ if validate_unchanged:+ try:+ await _bounded_phase(+ self._lease.validate(),+ self._checkpoint_timeout_seconds,+ "Codex credential lease validation",+ )+ except asyncio.CancelledError:+ raise+ except BaseException as error:+ raise CodexCredentialIntegrityError(+ f"codex_credential_lease_lost: {error}"+ ) from error+ return+ updated = checkpoint_credentials_from_bundle(+ self._state.credentials,+ after,+ )+ try:+ await _bounded_phase(+ self._lease.checkpoint(updated),+ self._checkpoint_timeout_seconds,+ "Codex credential checkpoint",+ )+ except asyncio.CancelledError:+ raise+ except BaseException as error:+ raise CodexCredentialIntegrityError(+ "codex_credential_checkpoint_failed"+ ) from error+ self._state.credentials = updated+ self._state.bundle = after++ async def stop(self) -> None:+ monitor = self._monitor_task+ if monitor is None:+ return+ self._monitor_task = None+ monitor.cancel()+ results = await asyncio.gather(monitor, return_exceptions=True)+ error = results[0]+ if isinstance(error, BaseException) and not isinstance(+ error, asyncio.CancelledError+ ):+ raise error++ async def _monitor_auth_file(self) -> None:+ while True:+ await asyncio.sleep(self._checkpoint_interval_seconds)+ await self.checkpoint_now(validate_unchanged=False)+++class _CodexRuntimeActor:+ def __init__(+ self,+ *,+ lease: CredentialLease,+ context: AbstractAsyncContextManager[CodexAgentSession],+ session: CodexAgentSession,+ ) -> None:+ self.lease = lease+ self.context = context+ self.session = session+ self.handles: set[str] = set()+ self.calls: dict[str, set[asyncio.Task[object]]] = {}+ self.models_task: asyncio.Task[list[CodexModelInfo]] | None = None+ self.failed: BaseException | None = None+++class PooledCodexRuntimeLease:+ def __init__(+ self,+ *,+ pool: "_CodexRuntimePool",+ key: tuple[str, str],+ handle_id: str,+ credentials: OAuth2Credentials,+ ) -> None:+ self.credentials = credentials+ self._pool = pool+ self._key = key+ self._handle_id = handle_id+ self._release_state = "active"+ self._release_completion: concurrent.futures.Future[None] | None = None+ self._release_lock = threading.Lock()++ async def models(self) -> list[CodexModelInfo]:+ self._ensure_active()+ return await self._pool.models(self._key, self._handle_id)++ async def invoke(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None = None,+ ) -> CodexInvocationResult:+ self._ensure_active()+ return await self._pool.invoke(+ self._key,+ self._handle_id,+ request,+ dynamic_tools,+ tool_handler,+ event_handler,+ )++ async def release(self) -> None:+ with self._release_lock:+ if self._release_state == "released":+ return+ if self._release_state == "releasing":+ completion = self._release_completion+ leader = False+ else:+ completion = concurrent.futures.Future()+ self._release_completion = completion+ self._release_state = "releasing"+ leader = True+ if completion is None:+ raise RuntimeError("Codex runtime release state is invalid")+ if not leader:+ await asyncio.shield(asyncio.wrap_future(completion))+ return+ try:+ await self._pool.release(self._key, self._handle_id)+ except BaseException as error:+ with self._release_lock:+ self._release_state = "active"+ self._release_completion = None+ completion.set_exception(error)+ raise+ else:+ with self._release_lock:+ self._release_state = "released"+ self._release_completion = None+ completion.set_result(None)++ def _ensure_active(self) -> None:+ with self._release_lock:+ if self._release_state != "active":+ raise RuntimeError("Codex runtime lease was released")+++class _CodexRuntimePool:+ def __init__(self, transport: CodexTransport) -> None:+ self._transport = transport+ self._state_lock = threading.Lock()+ self._loop: asyncio.AbstractEventLoop | None = None+ self._thread: threading.Thread | None = None+ self._loop_ready: threading.Event | None = None+ self._loop_start_error: BaseException | None = None+ self._closing = False+ self._closed = False+ self._generation = 0+ self._close_complete = threading.Event()+ self._close_error: BaseException | None = None+ self._actors: dict[tuple[str, str], _CodexRuntimeActor] = {}+ self._starts: dict[tuple[str, str], asyncio.Task[_CodexRuntimeActor]] = {}+ self._retiring: dict[tuple[str, str], asyncio.Task[None]] = {}+ self._pending: dict[tuple[str, str], set[str]] = {}++ async def acquire(+ self,+ user_id: str,+ credential_id: str,+ *,+ lock_timeout_seconds: float,+ ) -> PooledCodexRuntimeLease:+ key = (user_id, credential_id)+ handle_id = uuid4().hex+ loop = self._ensure_loop()+ with self._state_lock:+ if self._closing or self._closed:+ raise CodexTransportError("Codex runtime pool is shutting down")+ generation = self._generation+ operation = self._borrow_on_actor_loop(+ key,+ handle_id,+ generation=generation,+ lock_timeout_seconds=lock_timeout_seconds,+ )+ try:+ future = asyncio.run_coroutine_threadsafe(operation, loop)+ except BaseException:+ operation.close()+ raise+ try:+ credentials = await asyncio.wrap_future(future)+ except asyncio.CancelledError:+ future.cancel()+ with suppress(BaseException):+ cleanup = asyncio.run_coroutine_threadsafe(+ self._release_on_actor_loop(key, handle_id),+ loop,+ )+ await _await_cleanup_future(+ cleanup,+ timeout_seconds=self._cleanup_timeout_seconds,+ label="Codex cancelled runtime borrow",+ )+ raise+ with self._state_lock:+ shutting_down = self._closing or self._closed+ if shutting_down:+ raise CodexTransportError("Codex runtime pool is shutting down")+ return PooledCodexRuntimeLease(+ pool=self,+ key=key,+ handle_id=handle_id,+ credentials=credentials,+ )++ async def models(+ self,+ key: tuple[str, str],+ handle_id: str,+ ) -> list[CodexModelInfo]:+ future = asyncio.run_coroutine_threadsafe(+ self._models_on_actor_loop(key, handle_id),+ self._ensure_loop(),+ )+ try:+ return await asyncio.wrap_future(future)+ except asyncio.CancelledError:+ future.cancel()+ raise++ async def invoke(+ self,+ key: tuple[str, str],+ handle_id: str,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None,+ ) -> CodexInvocationResult:+ caller_loop = asyncio.get_running_loop()++ async def bridged_tool_handler(+ call: CodexDynamicToolCall,+ ) -> CodexDynamicToolResult:+ return await _await_threadsafe_callback(+ tool_handler(call),+ caller_loop,+ )++ async def bridged_event_handler(notification: Notification) -> None:+ if event_handler is not None:+ await _await_threadsafe_callback(+ event_handler(notification),+ caller_loop,+ )++ future = asyncio.run_coroutine_threadsafe(+ self._invoke_on_actor_loop(+ key,+ handle_id,+ request,+ dynamic_tools,+ bridged_tool_handler,+ bridged_event_handler if event_handler is not None else None,+ ),+ self._ensure_loop(),+ )+ try:+ return await asyncio.wrap_future(future)+ except asyncio.CancelledError:+ future.cancel()+ raise++ async def release(self, key: tuple[str, str], handle_id: str) -> None:+ loop = self._loop+ if loop is None:+ return+ future = asyncio.run_coroutine_threadsafe(+ self._release_on_actor_loop(key, handle_id),+ loop,+ )+ await _await_cleanup_future(+ future,+ timeout_seconds=self._cleanup_timeout_seconds,+ label="Codex pooled runtime release",+ )++ async def close(self) -> None:+ close_complete = self._close_complete+ close_error: BaseException | None = None+ already_complete = False+ leader = False+ with self._state_lock:+ if self._closed:+ close_error = self._close_error+ already_complete = True+ elif self._closing:+ pass+ else:+ self._closing = True+ self._generation += 1+ leader = True+ loop = self._loop+ thread = self._thread+ ready = self._loop_ready+ if already_complete:+ if close_error is not None:+ raise close_error+ return+ if not leader:+ await self._wait_for_close(close_complete)+ return+ close_error, loop = await self._close_as_leader(loop, thread, ready)+ with self._state_lock:+ if self._loop is loop:+ self._loop = None+ self._thread = None+ self._loop_ready = None+ self._close_error = close_error+ self._closed = True+ close_complete.set()+ if close_error is not None:+ raise close_error++ async def _wait_for_close(self, close_complete: threading.Event) -> None:+ completed = await asyncio.to_thread(+ close_complete.wait,+ self._cleanup_timeout_seconds,+ )+ if not completed:+ raise CodexTransportError("Codex runtime pool shutdown timed out")+ with self._state_lock:+ close_error = self._close_error+ if close_error is not None:+ raise close_error++ async def _close_as_leader(+ self,+ loop: asyncio.AbstractEventLoop | None,+ thread: threading.Thread | None,+ ready: threading.Event | None,+ ) -> tuple[BaseException | None, asyncio.AbstractEventLoop | None]:+ close_error: BaseException | None = None+ try:+ loop = await self._resolve_shutdown_loop(loop, thread, ready)+ if loop is not None and not loop.is_closed():+ future = asyncio.run_coroutine_threadsafe(+ self._close_all_on_actor_loop(), loop+ )+ await _await_cleanup_future(+ future,+ timeout_seconds=self._cleanup_timeout_seconds,+ label="Codex runtime pool shutdown",+ )+ except BaseException as error:+ close_error = error+ stop_error = await self._stop_actor_loop(loop, thread)+ if close_error is not None and stop_error is not None:+ close_error.add_note(+ f"Actor-loop stop failure: "+ f"{type(stop_error).__name__}: {stop_error}"+ )+ return close_error or stop_error, loop++ async def _resolve_shutdown_loop(+ self,+ loop: asyncio.AbstractEventLoop | None,+ thread: threading.Thread | None,+ ready: threading.Event | None,+ ) -> asyncio.AbstractEventLoop | None:+ if thread is None or loop is not None:+ return loop+ if ready is None or not await asyncio.to_thread(+ ready.wait,+ self._cleanup_timeout_seconds,+ ):+ raise CodexTransportError(+ "Codex runtime pool loop startup timed out during shutdown"+ )+ with self._state_lock:+ loop = self._loop+ start_error = self._loop_start_error+ if start_error is not None:+ raise CodexTransportError(+ "Codex runtime pool loop failed to start"+ ) from start_error+ return loop++ async def _stop_actor_loop(+ self,+ loop: asyncio.AbstractEventLoop | None,+ thread: threading.Thread | None,+ ) -> BaseException | None:+ try:+ if loop is not None and not loop.is_closed():+ loop.call_soon_threadsafe(loop.stop)+ if thread is None:+ return None+ await asyncio.to_thread(thread.join, self._cleanup_timeout_seconds)+ if thread.is_alive():+ return CodexTransportError("Codex runtime pool loop did not stop")+ except BaseException as error:+ return error+ return None++ @property+ def _cleanup_timeout_seconds(self) -> float:+ return max(self._transport._control_timeout_seconds * 3, 5)++ def _ensure_loop(self) -> asyncio.AbstractEventLoop:+ with self._state_lock:+ if self._closing or self._closed:+ raise CodexTransportError("Codex runtime pool is shutting down")+ if self._loop is not None:+ return self._loop+ ready = self._loop_ready+ if ready is None:+ ready = threading.Event()+ self._loop_ready = ready+ self._loop_start_error = None++ def run() -> None:+ loop: asyncio.AbstractEventLoop | None = None+ try:+ loop = asyncio.new_event_loop()+ asyncio.set_event_loop(loop)+ with self._state_lock:+ self._loop = loop+ ready.set()+ loop.run_forever()+ except BaseException as error:+ with self._state_lock:+ self._loop_start_error = error+ ready.set()+ finally:+ if loop is not None and not loop.is_closed():+ loop.close()++ thread = threading.Thread(+ target=run,+ name="autogpt-codex-runtime-pool",+ daemon=True,+ )+ self._thread = thread+ thread.start()+ if not ready.wait(self._cleanup_timeout_seconds):+ raise CodexTransportError("Codex runtime pool loop startup timed out")+ with self._state_lock:+ loop = self._loop+ start_error = self._loop_start_error+ shutting_down = self._closing or self._closed+ if start_error is not None:+ raise CodexTransportError(+ "Codex runtime pool loop failed to start"+ ) from start_error+ if shutting_down:+ raise CodexTransportError("Codex runtime pool is shutting down")+ if loop is None:+ raise RuntimeError("Codex runtime pool loop failed to start")+ return loop++ async def _borrow_on_actor_loop(+ self,+ key: tuple[str, str],+ handle_id: str,+ *,+ generation: int,+ lock_timeout_seconds: float,+ ) -> OAuth2Credentials:+ self._validate_generation(generation)+ pending = self._pending.setdefault(key, set())+ pending.add(handle_id)+ start: asyncio.Task[_CodexRuntimeActor] | None = None+ try:+ retiring = self._retiring.get(key)+ if retiring is not None:+ try:+ await asyncio.shield(retiring)+ except BaseException:+ raise+ else:+ if self._retiring.get(key) is retiring:+ self._retiring.pop(key, None)+ self._validate_generation(generation)+ actor = self._actors.get(key)+ if (+ actor is not None+ and (failure := self._actor_failure(actor)) is not None+ ):+ raise CodexTransportError(+ "Codex credential runtime is unavailable"+ ) from failure+ if actor is None:+ start = self._starts.get(key)+ if start is None:+ start = asyncio.create_task(+ self._start_actor(+ key,+ lock_timeout_seconds=lock_timeout_seconds,+ )+ )+ self._starts[key] = start+ actor = await asyncio.shield(start)+ with self._state_lock:+ if self._closing or self._closed or generation != self._generation:+ raise CodexTransportError("Codex runtime pool is shutting down")+ actor = self._actors.setdefault(key, actor)+ if self._starts.get(key) is start:+ self._starts.pop(key, None)+ else:+ self._validate_generation(generation)+ actor.handles.add(handle_id)+ return cast(OAuth2Credentials, actor.lease.credentials)+ except BaseException:+ pending.discard(handle_id)+ await self._cancel_unused_start(key, start)+ raise+ finally:+ pending.discard(handle_id)+ if not pending:+ self._pending.pop(key, None)++ async def _start_actor(+ self,+ key: tuple[str, str],+ *,+ lock_timeout_seconds: float,+ ) -> _CodexRuntimeActor:+ from backend.integrations.creds_manager import IntegrationCredentialsManager++ user_id, credential_id = key+ try:+ lease = await asyncio.wait_for(+ IntegrationCredentialsManager().acquire_lease(user_id, credential_id),+ timeout=lock_timeout_seconds,+ )+ except asyncio.TimeoutError:+ raise CodexCredentialBusyError("codex_credential_busy") from None+ context: AbstractAsyncContextManager[CodexAgentSession] | None = None+ try:+ _codex_credentials(lease)+ context = self._transport.agent_session(lease)+ session = await context.__aenter__()+ return _CodexRuntimeActor(+ lease=lease,+ context=context,+ session=session,+ )+ except BaseException:+ if context is not None:+ with suppress(BaseException):+ await context.__aexit__(None, None, None)+ await lease.release()+ raise++ async def _cancel_unused_start(+ self,+ key: tuple[str, str],+ start: asyncio.Task[_CodexRuntimeActor] | None,+ ) -> None:+ if self._pending.get(key):+ return+ candidate = start or self._starts.get(key)+ if candidate is None:+ return+ if self._starts.get(key) is not candidate:+ return+ if not candidate.done():+ candidate.cancel()+ await asyncio.gather(candidate, return_exceptions=True)+ if self._starts.get(key) is candidate:+ self._starts.pop(key, None)+ if candidate.cancelled():+ return+ try:+ actor = candidate.result()+ except BaseException:+ return+ if not actor.handles:+ try:+ await self._close_actor_bounded(actor)+ except BaseException:+ logger.exception("Failed to clean up unused Codex runtime start")++ async def _models_on_actor_loop(+ self,+ key: tuple[str, str],+ handle_id: str,+ ) -> list[CodexModelInfo]:+ actor = self._active_actor(key, handle_id)+ models_task = actor.models_task+ if models_task is None:+ models_task = asyncio.create_task(actor.session.models())+ actor.models_task = models_task+ try:+ return list(await asyncio.shield(models_task))+ except BaseException as error:+ self._record_actor_failure(actor, error)+ if actor.models_task is models_task and models_task.done():+ actor.models_task = None+ raise++ async def _invoke_on_actor_loop(+ self,+ key: tuple[str, str],+ handle_id: str,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler: Callable[+ [CodexDynamicToolCall], Awaitable[CodexDynamicToolResult]+ ],+ event_handler: Callable[[Notification], Awaitable[None]] | None,+ ) -> CodexInvocationResult:+ actor = self._active_actor(key, handle_id)+ task = asyncio.current_task()+ if task is None:+ raise RuntimeError("Codex pooled invocation has no owning task")+ calls = actor.calls.setdefault(handle_id, set())+ calls.add(cast(asyncio.Task[object], task))+ try:+ return await actor.session.invoke(+ request,+ dynamic_tools,+ tool_handler,+ event_handler,+ )+ except BaseException as error:+ self._record_actor_failure(actor, error)+ raise+ finally:+ calls.discard(cast(asyncio.Task[object], task))+ if not calls and actor.calls.get(handle_id) is calls:+ actor.calls.pop(handle_id, None)++ async def _release_on_actor_loop(+ self,+ key: tuple[str, str],+ handle_id: str,+ ) -> None:+ actor = self._actors.get(key)+ if actor is None:+ retiring = self._retiring.get(key)+ if retiring is not None:+ await asyncio.shield(retiring)+ return+ if handle_id not in actor.handles:+ return+ actor.handles.discard(handle_id)+ calls = tuple(actor.calls.pop(handle_id, ()))+ for task in calls:+ task.cancel()+ if calls:+ with suppress(asyncio.TimeoutError):+ await asyncio.wait_for(+ asyncio.gather(*calls, return_exceptions=True),+ timeout=self._transport._control_timeout_seconds,+ )+ if actor.handles:+ return+ if self._actors.get(key) is actor:+ self._actors.pop(key, None)+ retiring = asyncio.create_task(self._close_actor_bounded(actor))+ self._retiring[key] = retiring+ try:+ await asyncio.shield(retiring)+ except BaseException:+ raise+ else:+ if self._retiring.get(key) is retiring:+ self._retiring.pop(key, None)++ async def _close_actor_bounded(self, actor: _CodexRuntimeActor) -> None:+ models_task, actor.models_task = actor.models_task, None+ if models_task is not None:+ if not models_task.done():+ models_task.cancel()+ await asyncio.gather(models_task, return_exceptions=True)++ async def cleanup() -> None:+ context_error: BaseException | None = None+ try:+ await actor.context.__aexit__(None, None, None)+ except BaseException as error:+ context_error = error+ try:+ await actor.lease.release()+ except BaseException as release_error:+ if context_error is None:+ raise+ context_error.add_note(+ f"Credential lease release also failed: {release_error}"+ )+ if context_error is not None:+ raise context_error++ task = asyncio.create_task(cleanup())+ try:+ await asyncio.wait_for(+ asyncio.shield(task),+ timeout=self._cleanup_timeout_seconds,+ )+ except BaseException:+ task.cancel()+ await _cancel_future_bounded(task)+ raise++ async def _close_all_on_actor_loop(self) -> None:+ errors: list[BaseException] = []+ starts = tuple(self._starts.values())+ self._starts.clear()+ for start in starts:+ start.cancel()+ started_actors: list[_CodexRuntimeActor] = []+ if starts:+ for result in await asyncio.gather(*starts, return_exceptions=True):+ if isinstance(result, _CodexRuntimeActor):+ started_actors.append(result)+ elif isinstance(result, BaseException) and not isinstance(+ result, asyncio.CancelledError+ ):+ errors.append(result)+ actors_by_id = {+ id(actor): actor for actor in (*self._actors.values(), *started_actors)+ }+ actors = tuple(actors_by_id.values())+ self._actors.clear()+ self._pending.clear()+ for actor in actors:+ for calls in actor.calls.values():+ for task in calls:+ task.cancel()+ calls = tuple(+ {+ id(task): task+ for actor in actors+ for group in actor.calls.values()+ for task in group+ }.values()+ )+ if calls:+ await asyncio.gather(*calls, return_exceptions=True)++ cleanup_tasks = [+ asyncio.create_task(self._close_actor_bounded(actor)) for actor in actors+ ]+ cleanup_tasks.extend(+ {id(task): task for task in self._retiring.values()}.values()+ )+ self._retiring.clear()+ if cleanup_tasks:+ for result in await asyncio.gather(+ *{id(task): task for task in cleanup_tasks}.values(),+ return_exceptions=True,+ ):+ if isinstance(result, BaseException) and not isinstance(+ result, asyncio.CancelledError+ ):+ errors.append(result)+ if errors:+ cleanup_error = CodexTransportError(+ f"Codex runtime pool cleanup failed ({len(errors)} error(s))"+ )+ for additional_error in errors[1:]:+ cleanup_error.add_note(+ f"Additional cleanup failure: "+ f"{type(additional_error).__name__}: {additional_error}"+ )+ raise cleanup_error from errors[0]++ def _active_actor(+ self,+ key: tuple[str, str],+ handle_id: str,+ ) -> _CodexRuntimeActor:+ actor = self._actors.get(key)+ if actor is None or handle_id not in actor.handles:+ raise RuntimeError("Codex runtime lease is no longer active")+ if failure := self._actor_failure(actor):+ raise CodexTransportError(+ "Codex credential runtime is unavailable"+ ) from failure+ return actor++ @staticmethod+ def _actor_failure(actor: _CodexRuntimeActor) -> BaseException | None:+ failure = actor.failed or actor.session.failure+ if failure is not None:+ actor.failed = failure+ return failure++ def _record_actor_failure(+ self,+ actor: _CodexRuntimeActor,+ error: BaseException,+ ) -> None:+ failure = actor.session.failure+ if (+ isinstance(error, CodexCredentialIntegrityError)+ or failure is not None+ or actor.session.closed+ ):+ actor.failed = failure or error++ def _validate_generation(self, generation: int) -> None:+ with self._state_lock:+ if self._closing or self._closed or generation != self._generation:+ raise CodexTransportError("Codex runtime pool is shutting down")+++async def _await_threadsafe_callback(+ operation: Awaitable[ResultT],+ loop: asyncio.AbstractEventLoop,+) -> ResultT:+ async def await_operation() -> ResultT:+ return await operation++ future = asyncio.run_coroutine_threadsafe(await_operation(), loop)+ try:+ return await asyncio.wrap_future(future)+ except asyncio.CancelledError:+ future.cancel()+ raise+++async def _await_cleanup_future(+ future: concurrent.futures.Future[ResultT],+ *,+ timeout_seconds: float,+ label: str,+) -> ResultT:+ wrapped = asyncio.wrap_future(future)+ try:+ return await asyncio.wait_for(+ asyncio.shield(wrapped),+ timeout=timeout_seconds,+ )+ except asyncio.CancelledError:+ with suppress(BaseException):+ await asyncio.wait_for(+ asyncio.shield(wrapped),+ timeout=timeout_seconds,+ )+ raise+ except asyncio.TimeoutError:+ future.cancel()+ raise CodexTransportError(f"{label} timed out") from None+++async def _run_with_lease_guard(+ lease: CredentialLease,+ operation: Awaitable[ResultT],+ *,+ monitor_task: asyncio.Task[None] | None = None,+) -> ResultT:+ if failure := lease.failure:+ _close_unused_awaitable(operation)+ raise RuntimeError("Credential lease heartbeat failed") from failure+ if monitor_task is not None and monitor_task.done():+ _close_unused_awaitable(operation)+ if monitor_task.cancelled():+ raise CodexCredentialIntegrityError("codex_credential_monitor_stopped")+ if error := monitor_task.exception():+ raise error+ raise CodexCredentialIntegrityError("codex_credential_monitor_stopped")+ operation_task = asyncio.ensure_future(operation)+ heartbeat_task = asyncio.create_task(lease.wait_for_failure())+ try:+ watched_tasks = {operation_task, heartbeat_task}+ if monitor_task is not None:+ watched_tasks.add(monitor_task)+ done, _ = await asyncio.wait(+ watched_tasks,+ return_when=asyncio.FIRST_COMPLETED,+ )+ if heartbeat_task in done:+ await _cancel_future_bounded(operation_task)+ await heartbeat_task+ if monitor_task is not None and monitor_task in done:+ await _cancel_future_bounded(operation_task)+ await monitor_task+ return await operation_task+ finally:+ if not operation_task.done():+ await _cancel_future_bounded(operation_task)+ heartbeat_task.cancel()+ await asyncio.gather(heartbeat_task, return_exceptions=True)+++def _close_unused_awaitable(operation: Awaitable[Any]) -> None:+ if inspect.iscoroutine(operation):+ operation.close()+++async def _bounded_phase(+ operation: Awaitable[ResultT],+ timeout_seconds: float,+ label: str,+) -> ResultT:+ task = asyncio.ensure_future(operation)+ try:+ done, _ = await asyncio.wait({task}, timeout=timeout_seconds)+ except asyncio.CancelledError:+ await _cancel_future_bounded(task)+ raise+ if task in done:+ return await task+ await _cancel_future_bounded(task, timeout_seconds=0.1)+ raise CodexTransportError(f"{label} exceeded {timeout_seconds:g} seconds")+++async def _cancel_future_bounded(+ task: asyncio.Future[Any],+ *,+ timeout_seconds: float = 1.0,+) -> None:+ task.cancel()+ done, _ = await asyncio.wait({task}, timeout=timeout_seconds)+ if task in done:+ try:+ task.result()+ except BaseException:+ pass+ return++ def consume_result(completed: asyncio.Future[Any]) -> None:+ try:+ completed.result()+ except BaseException:+ pass++ task.add_done_callback(consume_result)+++async def _read_runtime_auth(auth_path: Path) -> CodexAuthBundleV1:+ for delay_seconds in (0.0, 0.02, 0.05, 0.1):+ if delay_seconds:+ await asyncio.sleep(delay_seconds)+ try:+ return read_auth_bundle(auth_path, CODEX_RUNTIME_VERSION)+ except CodexAuthBundleError:+ continue+ raise CodexTransportError("Codex runtime auth state is unavailable")diff --git a/autogpt_platform/backend/backend/integrations/codex/transport_test.py b/autogpt_platform/backend/backend/integrations/codex/transport_test.pynew file mode 100644index 0000000000..d2bb770077--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/codex/transport_test.py@@ -0,0 +1,1689 @@+import asyncio+import base64+import json+import threading+from collections.abc import Callable, Coroutine+from pathlib import Path+from unittest.mock import AsyncMock, MagicMock++import pytest++from backend.integrations.codex.auth_bundle import (+ CodexAuthBundleV1,+ CodexAuthTokensV1,+ materialize_auth_bundle,+)+from backend.integrations.codex.credential_codec import credentials_from_bundle+from backend.integrations.codex.models import (+ CodexAccountSnapshot,+ CodexDeviceCodeDetails,+ CodexDynamicToolCall,+ CodexDynamicToolResult,+ CodexDynamicToolSpec,+ CodexInvocationRequest,+ CodexInvocationResult,+ CodexModelInfo,+ CodexRateLimitsSnapshot,+)+from backend.integrations.codex.transport import (+ CodexTransport,+ CodexTransportError,+ CodexTransportOverloadedError,+ _close_unused_awaitable,+ _run_with_lease_guard,+)+from backend.integrations.credential_lease import CredentialLease+++def test_process_capacity_is_safe_across_event_loops():+ transport = CodexTransport(max_active_processes=1)+ first_acquired = threading.Event()+ release_first = threading.Event()+ second_acquired = threading.Event()+ errors: list[BaseException] = []++ async def hold_first_slot() -> None:+ await transport._acquire_capacity()+ first_acquired.set()+ while not release_first.is_set():+ await asyncio.sleep(0.01)+ transport._capacity.release()++ async def wait_for_second_slot() -> None:+ await transport._acquire_capacity()+ second_acquired.set()+ transport._capacity.release()++ def run(coroutine: Callable[[], Coroutine[object, object, None]]) -> None:+ try:+ asyncio.run(coroutine())+ except BaseException as error:+ errors.append(error)++ first = threading.Thread(target=run, args=(hold_first_slot,))+ second = threading.Thread(target=run, args=(wait_for_second_slot,))+ first.start()+ try:+ assert first_acquired.wait(timeout=2)+ second.start()+ assert not second_acquired.wait(timeout=0.05)+ release_first.set()+ assert second_acquired.wait(timeout=2)+ finally:+ release_first.set()+ first.join(timeout=2)+ if second.ident is not None:+ second.join(timeout=2)++ assert not first.is_alive()+ assert not second.is_alive()+ assert not errors+++def test_close_unused_awaitable_only_closes_native_coroutines():+ class CustomAwaitable:+ def __init__(self) -> None:+ self.close_called = False++ def __await__(self):+ if False:+ yield None+ return None++ def close(self) -> None:+ self.close_called = True++ operation = CustomAwaitable()++ _close_unused_awaitable(operation)++ assert not operation.close_called+++async def test_invoke_checkpoints_rotated_auth_before_cleanup(tmp_path):+ lease, checkpoint = _lease()+ runtime = _FakeRuntime(rotate_auth=True)+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ result = await transport.invoke(+ lease,+ CodexInvocationRequest(prompt="Reply with ok"),+ )++ assert result.final_response == "ok"+ assert result.resolved_model == "gpt-test"+ assert runtime.last_request is not None+ assert runtime.last_request.model == "gpt-test"+ checkpoint.assert_awaited_once()+ updated = checkpoint.await_args.args[0]+ assert updated.refresh_token.get_secret_value() == "refresh-rotated"+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_invoke_rejects_model_not_advertised_by_account(tmp_path):+ lease, _ = _lease()+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ with pytest.raises(CodexTransportError, match="not available"):+ await transport.invoke(+ lease,+ CodexInvocationRequest(prompt="ok", model="gpt-not-on-account"),+ )++ assert runtime.last_request is None+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_runtime_failure_still_checkpoints_rotated_auth(tmp_path):+ lease, checkpoint = _lease()+ runtime = _FakeRuntime(rotate_auth=True, fail_invoke=True)+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ with pytest.raises(RuntimeError, match="runtime failed"):+ await transport.invoke(lease, CodexInvocationRequest(prompt="fail"))++ checkpoint.assert_awaited_once()+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_unchanged_auth_does_not_checkpoint(tmp_path):+ lease, checkpoint = _lease()+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ await transport.account(lease)++ checkpoint.assert_not_awaited()+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_unchanged_auth_still_validates_lease(tmp_path):+ lease, checkpoint = _lease()+ lease._lock.owned.return_value = False+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ with pytest.raises(RuntimeError, match="ownership was lost"):+ await transport.account(lease)++ checkpoint.assert_not_awaited()+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_materialization_failure_releases_capacity_and_cleans_home(+ tmp_path,+ monkeypatch,+):+ lease, _ = _lease()+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ max_active_processes=1,+ runtime_factory=_runtime_factory(runtime),+ )++ def fail_materialization(*_args):+ raise RuntimeError("materialization failed")++ monkeypatch.setattr(+ "backend.integrations.codex.transport.materialize_auth_bundle",+ fail_materialization,+ )+ with pytest.raises(RuntimeError, match="materialization failed"):+ await transport.account(lease)++ assert not tuple(tmp_path.iterdir())+ assert transport._capacity._value == 1+++async def test_device_login_returns_bundle_and_cleans_home(tmp_path):+ runtime = _FakeRuntime(complete_login=True)+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ session = await transport.start_device_login()+ completion = await session.wait()++ assert session.details.user_code == "CODE-1234"+ assert completion.bundle.tokens.account_id == "account-user"+ assert completion.account is None+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_device_login_home_creation_failure_releases_capacity(+ tmp_path,+ monkeypatch,+):+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ max_active_processes=1,+ runtime_factory=_runtime_factory(runtime),+ )++ def fail_home_creation(*_args):+ raise RuntimeError("home creation failed")++ monkeypatch.setattr(+ "backend.integrations.codex.transport.TemporaryCodexHome.create",+ fail_home_creation,+ )+ with pytest.raises(RuntimeError, match="home creation failed"):+ await transport.start_device_login()++ assert transport._capacity._value == 1+++async def test_capacity_wait_is_bounded():+ transport = CodexTransport(+ max_active_processes=1,+ capacity_timeout_seconds=0.02,+ )+ await transport._acquire_capacity()+ try:+ with pytest.raises(CodexTransportOverloadedError):+ await transport._acquire_capacity()+ finally:+ transport._capacity.release()+++async def test_runtime_startup_timeout_releases_capacity_and_cleans_home(tmp_path):+ release_startup = asyncio.Event()++ async def blocked_runtime_factory(_home):+ await release_startup.wait()+ return _FakeRuntime()++ transport = CodexTransport(+ temp_root=tmp_path,+ max_active_processes=1,+ startup_timeout_seconds=0.02,+ runtime_factory=blocked_runtime_factory,+ )++ with pytest.raises(CodexTransportError, match="runtime startup"):+ await transport.start_device_login()++ assert transport._capacity._value == 1+ assert not tuple(tmp_path.iterdir())+++async def test_invoke_checkpoints_refresh_before_starting_turn(tmp_path):+ lease, checkpoint = _lease()+ runtime = _FakeRuntime(rotate_on_account=True)+ runtime.checkpoint_probe = lambda: checkpoint.await_count == 1+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )++ await transport.invoke(lease, CodexInvocationRequest(prompt="ok"))++ checkpoint.assert_awaited_once()+ assert runtime.refresh_was_checkpointed_before_invoke+++async def test_active_invocation_checkpoints_late_auth_rotation(tmp_path):+ lease, checkpoint = _lease()+ rotated = asyncio.Event()+ checkpointed = asyncio.Event()+ finish = asyncio.Event()++ async def record_checkpoint(*_args):+ checkpointed.set()++ checkpoint.side_effect = record_checkpoint++ class RotatingRuntime(_FakeRuntime):+ async def invoke(+ self,+ request: CodexInvocationRequest,+ ) -> CodexInvocationResult:+ payload = json.loads((self.home / "auth.json").read_text())+ payload["tokens"]["refresh_token"] = "refresh-during-turn"+ (self.home / "auth.json").write_text(json.dumps(payload))+ rotated.set()+ await finish.wait()+ return await super().invoke(request)++ runtime = RotatingRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ checkpoint_interval_seconds=0.01,+ runtime_factory=_runtime_factory(runtime),+ )++ invocation = asyncio.create_task(+ transport.invoke(lease, CodexInvocationRequest(prompt="ok"))+ )+ await asyncio.wait_for(rotated.wait(), timeout=1)+ await asyncio.wait_for(checkpointed.wait(), timeout=1)+ assert not invocation.done()+ finish.set()+ await invocation++ checkpoint.assert_awaited_once()+++async def test_invocation_aborts_when_credential_lease_heartbeat_fails(tmp_path):+ lease, _ = _lease()+ started = asyncio.Event()+ canceled = asyncio.Event()++ class BlockingRuntime(_FakeRuntime):+ async def invoke(+ self,+ request: CodexInvocationRequest,+ ) -> CodexInvocationResult:+ started.set()+ try:+ await asyncio.Event().wait()+ finally:+ canceled.set()+ raise AssertionError("unreachable")++ runtime = BlockingRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ invocation = asyncio.create_task(+ transport.invoke(lease, CodexInvocationRequest(prompt="ok"))+ )+ await asyncio.wait_for(started.wait(), timeout=1)+ lease._heartbeat_error = RuntimeError("redis lease lost")+ lease._heartbeat_failed.set()++ with pytest.raises(RuntimeError, match="heartbeat failed"):+ await invocation++ assert canceled.is_set()+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_agent_invocation_uses_separate_tool_timeout_and_checkpoints(tmp_path):+ lease, checkpoint = _lease()+ runtime = _FakeRuntime(rotate_on_account=True)+ transport = CodexTransport(+ temp_root=tmp_path,+ copilot_tool_timeout_seconds=17,+ runtime_factory=_runtime_factory(runtime),+ )+ tool = CodexDynamicToolSpec(+ name="find_agent",+ description="Find an agent",+ input_schema={"type": "object"},+ )++ async def tool_handler(_call: CodexDynamicToolCall) -> CodexDynamicToolResult:+ return CodexDynamicToolResult(content="ok")++ result = await transport.invoke_agent(+ lease,+ CodexInvocationRequest(prompt="Find it"),+ [tool],+ tool_handler,+ )++ assert result.final_response == "ok"+ assert runtime.agent_tool_timeout == 17+ assert runtime.agent_dynamic_tools == [tool]+ checkpoint.assert_awaited_once()+ assert runtime.closed+ assert not tuple(tmp_path.iterdir())+++async def test_runtime_pool_overlaps_borrowers_on_one_lease_and_runtime(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )++ both_started = threading.Event()+ allow_finish = threading.Event()++ class ConcurrentRuntime(_FakeRuntime):+ def __init__(self):+ super().__init__()+ self._active_lock = threading.Lock()+ self.active = 0+ self.max_active = 0+ self.requests: list[CodexInvocationRequest] = []++ async def invoke_agent(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ *,+ tool_timeout_seconds,+ ):+ del dynamic_tools, tool_handler, event_handler, tool_timeout_seconds+ self.requests.append(request)+ with self._active_lock:+ self.active += 1+ self.max_active = max(self.max_active, self.active)+ if self.active == 2:+ both_started.set()+ try:+ while not allow_finish.is_set():+ await asyncio.sleep(0.01)+ return CodexInvocationResult(+ response_id=f"turn-{len(self.requests)}",+ final_response=request.prompt,+ status="completed",+ )+ finally:+ with self._active_lock:+ self.active -= 1++ runtime = ConcurrentRuntime()+ factory = AsyncMock(side_effect=_runtime_factory(runtime))+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=factory,+ )++ first, second = await asyncio.gather(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ ),+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ ),+ )++ async def tool_handler(_call):+ return CodexDynamicToolResult(content="ok")++ first_call = asyncio.create_task(+ first.invoke(+ CodexInvocationRequest(prompt="first"),+ [],+ tool_handler,+ )+ )+ second_call = asyncio.create_task(+ second.invoke(+ CodexInvocationRequest(prompt="second"),+ [],+ tool_handler,+ )+ )+ assert await asyncio.to_thread(both_started.wait, 2)+ assert not first_call.done()+ assert not second_call.done()+ assert manager.acquire_lease.await_count == 1+ assert factory.await_count == 1+ assert runtime.max_active == 2++ allow_finish.set()+ results = await asyncio.gather(first_call, second_call)+ assert {result.final_response for result in results} == {"first", "second"}++ await first.release()+ assert not runtime.closed+ raw_lease._lock.release.assert_not_awaited()+ await second.release()+ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ assert not tuple(tmp_path.iterdir())+ await transport.close_runtime_pool()+++async def test_runtime_pool_cancels_one_borrower_without_closing_sibling(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ cancel_started = threading.Event()+ cancel_observed = threading.Event()+ peer_started = threading.Event()+ finish_peer = threading.Event()++ class CancellationRuntime(_FakeRuntime):+ async def invoke_agent(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ *,+ tool_timeout_seconds,+ ):+ del dynamic_tools, tool_handler, event_handler, tool_timeout_seconds+ if request.prompt == "cancel":+ cancel_started.set()+ try:+ await asyncio.Event().wait()+ finally:+ cancel_observed.set()+ peer_started.set()+ while not finish_peer.is_set():+ await asyncio.sleep(0.01)+ return CodexInvocationResult(+ response_id="peer-turn",+ final_response="peer-ok",+ status="completed",+ )++ runtime = CancellationRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ first, second = await asyncio.gather(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ ),+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ ),+ )++ async def tool_handler(_call):+ return CodexDynamicToolResult(content="ok")++ canceled_call = asyncio.create_task(+ first.invoke(CodexInvocationRequest(prompt="cancel"), [], tool_handler)+ )+ peer_call = asyncio.create_task(+ second.invoke(CodexInvocationRequest(prompt="peer"), [], tool_handler)+ )+ assert await asyncio.to_thread(cancel_started.wait, 2)+ assert await asyncio.to_thread(peer_started.wait, 2)+ canceled_call.cancel()+ with pytest.raises(asyncio.CancelledError):+ await canceled_call+ assert await asyncio.to_thread(cancel_observed.wait, 2)+ assert not peer_call.done()+ assert not runtime.closed++ finish_peer.set()+ assert (await peer_call).final_response == "peer-ok"+ await first.release()+ assert not runtime.closed+ await second.release()+ assert runtime.closed+ await transport.close_runtime_pool()+++def test_runtime_pool_loop_start_is_singleton_across_threads():+ transport = CodexTransport()+ barrier = threading.Barrier(8)+ loops: list[asyncio.AbstractEventLoop] = []+ errors: list[BaseException] = []++ def start_loop() -> None:+ try:+ barrier.wait(timeout=2)+ loops.append(transport._runtime_pool._ensure_loop())+ except BaseException as error:+ errors.append(error)++ threads = [threading.Thread(target=start_loop) for _ in range(8)]+ for thread in threads:+ thread.start()+ for thread in threads:+ thread.join(timeout=2)++ assert not errors+ assert len(loops) == 8+ assert len({id(loop) for loop in loops}) == 1+ asyncio.run(transport.close_runtime_pool())+++async def test_runtime_pool_lock_timeout_does_not_bound_healthy_startup(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()++ async def delayed_runtime_factory(home):+ await asyncio.sleep(0.05)+ runtime.home = home.path+ return runtime++ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=delayed_runtime_factory,+ )++ lease = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=0.01,+ )++ assert manager.acquire_lease.await_count == 1+ await lease.release()+ assert runtime.closed+ await transport.close_runtime_pool()+++async def test_runtime_pool_canceled_acquire_releases_completed_borrow(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ pool = transport._runtime_pool+ original_borrow = pool._borrow_on_actor_loop+ borrowed = threading.Event()++ async def hold_completed_borrow(*args, **kwargs):+ credentials = await original_borrow(*args, **kwargs)+ borrowed.set()+ await asyncio.Event().wait()+ return credentials++ monkeypatch.setattr(pool, "_borrow_on_actor_loop", hold_completed_borrow)+ acquire = asyncio.create_task(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ )+ assert await asyncio.to_thread(borrowed.wait, 2)+ acquire.cancel()++ with pytest.raises(asyncio.CancelledError):+ await acquire++ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ assert not tuple(tmp_path.iterdir())+ await transport.close_runtime_pool()+++async def test_runtime_pool_release_can_retry_after_submission_failure(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ lease = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ pool = transport._runtime_pool+ original_release = pool.release+ attempts = 0++ async def fail_once(*args, **kwargs):+ nonlocal attempts+ attempts += 1+ if attempts == 1:+ raise RuntimeError("release submission failed")+ return await original_release(*args, **kwargs)++ monkeypatch.setattr(pool, "release", fail_once)++ with pytest.raises(RuntimeError, match="release submission failed"):+ await lease.release()++ assert not runtime.closed+ await lease.release()+ assert attempts == 2+ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ await transport.close_runtime_pool()+++async def test_runtime_pool_canceled_release_follower_does_not_cancel_leader(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ lease = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ pool = transport._runtime_pool+ original_release = pool.release+ release_started = asyncio.Event()+ allow_release = asyncio.Event()++ async def delayed_release(*args, **kwargs):+ release_started.set()+ await allow_release.wait()+ return await original_release(*args, **kwargs)++ monkeypatch.setattr(pool, "release", delayed_release)+ leader = asyncio.create_task(lease.release())+ await asyncio.wait_for(release_started.wait(), timeout=1)+ follower = asyncio.create_task(lease.release())+ await asyncio.sleep(0)+ follower.cancel()++ with pytest.raises(asyncio.CancelledError):+ await follower++ allow_release.set()+ await leader+ await lease.release()++ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ await transport.close_runtime_pool()+++def test_runtime_pool_close_serializes_with_acquire_submission(tmp_path, monkeypatch):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ real_submit = asyncio.run_coroutine_threadsafe+ submit_entered = threading.Event()+ allow_submit = threading.Event()+ close_complete = threading.Event()+ acquire_results: list[object] = []+ acquire_errors: list[BaseException] = []+ close_errors: list[BaseException] = []++ def block_first_submit(operation, loop):+ monkeypatch.setattr(asyncio, "run_coroutine_threadsafe", real_submit)+ submit_entered.set()+ if not allow_submit.wait(timeout=2):+ operation.close()+ raise RuntimeError("timed out waiting to submit")+ return real_submit(operation, loop)++ monkeypatch.setattr(asyncio, "run_coroutine_threadsafe", block_first_submit)++ def acquire() -> None:+ try:+ acquire_results.append(+ asyncio.run(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ )+ )+ except BaseException as error:+ acquire_errors.append(error)++ def close() -> None:+ try:+ asyncio.run(transport.close_runtime_pool())+ except BaseException as error:+ close_errors.append(error)+ finally:+ close_complete.set()++ acquire_thread = threading.Thread(target=acquire)+ close_thread = threading.Thread(target=close)+ acquire_thread.start()+ assert submit_entered.wait(timeout=2)+ close_thread.start()+ assert not close_complete.wait(timeout=0.05)+ allow_submit.set()+ acquire_thread.join(timeout=5)+ close_thread.join(timeout=5)++ assert not acquire_thread.is_alive()+ assert not close_thread.is_alive()+ assert not close_errors+ assert not acquire_results+ assert len(acquire_errors) == 1+ assert isinstance(acquire_errors[0], CodexTransportError)+ if manager.acquire_lease.await_count:+ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ else:+ assert not runtime.closed+ raw_lease._lock.release.assert_not_awaited()+ assert not tuple(tmp_path.iterdir())+ with pytest.raises(CodexTransportError, match="shutting down"):+ asyncio.run(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ )+++def test_runtime_pool_close_racing_completed_start_cleans_actor_once(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ pool = transport._runtime_pool+ original_start = pool._start_actor+ actor_ready = threading.Event()+ allow_start_return = threading.Event()++ async def gated_start(*args, **kwargs):+ actor = await original_start(*args, **kwargs)+ actor_ready.set()+ if not allow_start_return.wait(timeout=2):+ raise RuntimeError("timed out waiting to return started actor")+ return actor++ monkeypatch.setattr(pool, "_start_actor", gated_start)+ acquire_results: list[object] = []+ acquire_errors: list[BaseException] = []+ close_errors: list[BaseException] = []++ def acquire() -> None:+ try:+ acquire_results.append(+ asyncio.run(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ )+ )+ except BaseException as error:+ acquire_errors.append(error)++ def close() -> None:+ try:+ asyncio.run(transport.close_runtime_pool())+ except BaseException as error:+ close_errors.append(error)++ acquire_thread = threading.Thread(target=acquire)+ close_thread = threading.Thread(target=close)+ acquire_thread.start()+ assert actor_ready.wait(timeout=2)+ close_thread.start()+ for _ in range(200):+ with pool._state_lock:+ if pool._closing:+ break+ threading.Event().wait(0.01)+ else:+ pytest.fail("runtime pool did not begin closing")+ allow_start_return.set()+ acquire_thread.join(timeout=5)+ close_thread.join(timeout=5)++ assert not acquire_thread.is_alive()+ assert not close_thread.is_alive()+ assert not close_errors+ assert not acquire_results+ assert len(acquire_errors) == 1+ assert isinstance(acquire_errors[0], CodexTransportError)+ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ assert not tuple(tmp_path.iterdir())+++def test_runtime_pool_close_waits_for_loop_bootstrap(monkeypatch):+ transport = CodexTransport()+ pool = transport._runtime_pool+ real_new_event_loop = asyncio.new_event_loop+ bootstrap_entered = threading.Event()+ allow_bootstrap = threading.Event()+ acquire_errors: list[BaseException] = []+ close_errors: list[BaseException] = []++ def gated_new_event_loop():+ if threading.current_thread().name == "autogpt-codex-runtime-pool":+ bootstrap_entered.set()+ if not allow_bootstrap.wait(timeout=2):+ raise RuntimeError("timed out waiting to bootstrap actor loop")+ return real_new_event_loop()++ monkeypatch.setattr(asyncio, "new_event_loop", gated_new_event_loop)++ def acquire() -> None:+ try:+ asyncio.run(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ )+ except BaseException as error:+ acquire_errors.append(error)++ def close() -> None:+ try:+ asyncio.run(transport.close_runtime_pool())+ except BaseException as error:+ close_errors.append(error)++ acquire_thread = threading.Thread(target=acquire)+ close_thread = threading.Thread(target=close)+ acquire_thread.start()+ assert bootstrap_entered.wait(timeout=2)+ close_thread.start()+ for _ in range(200):+ with pool._state_lock:+ if pool._closing:+ break+ threading.Event().wait(0.01)+ else:+ pytest.fail("runtime pool did not begin closing")+ allow_bootstrap.set()+ acquire_thread.join(timeout=5)+ close_thread.join(timeout=5)++ assert not acquire_thread.is_alive()+ assert not close_thread.is_alive()+ assert not close_errors+ assert len(acquire_errors) == 1+ assert isinstance(acquire_errors[0], CodexTransportError)+ with pool._state_lock:+ assert pool._thread is None+ assert pool._loop is None+ assert pool._closed+++async def test_runtime_pool_release_rejects_late_invocation_while_draining(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ active_started = threading.Event()+ cancellation_seen = threading.Event()+ allow_cancellation = threading.Event()+ late_started = threading.Event()++ class ReleaseRaceRuntime(_FakeRuntime):+ async def invoke_agent(+ self,+ request,+ dynamic_tools,+ tool_handler,+ event_handler=None,+ *,+ tool_timeout_seconds,+ ):+ del dynamic_tools, tool_handler, event_handler, tool_timeout_seconds+ if request.prompt != "active":+ late_started.set()+ return CodexInvocationResult(+ response_id="late",+ final_response="late",+ status="completed",+ )+ active_started.set()+ try:+ await asyncio.Event().wait()+ except asyncio.CancelledError:+ cancellation_seen.set()+ while not allow_cancellation.is_set():+ await asyncio.sleep(0.01)+ raise++ runtime = ReleaseRaceRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ lease = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )++ async def tool_handler(_call):+ return CodexDynamicToolResult(content="ok")++ active = asyncio.create_task(+ lease.invoke(+ CodexInvocationRequest(prompt="active"),+ [],+ tool_handler,+ )+ )+ assert await asyncio.to_thread(active_started.wait, 2)+ release = asyncio.create_task(lease.release())+ assert await asyncio.to_thread(cancellation_seen.wait, 2)++ with pytest.raises(RuntimeError, match="no longer active"):+ await lease._pool.invoke(+ lease._key,+ lease._handle_id,+ CodexInvocationRequest(prompt="late"),+ [],+ tool_handler,+ None,+ )++ assert not late_started.is_set()+ allow_cancellation.set()+ with pytest.raises(asyncio.CancelledError):+ await active+ await release++ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ await transport.close_runtime_pool()+++async def test_runtime_pool_release_cancels_and_drains_model_lookup(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ models_started = threading.Event()+ models_canceled = threading.Event()++ class BlockingModelsRuntime(_FakeRuntime):+ async def models(self):+ models_started.set()+ try:+ await asyncio.Event().wait()+ finally:+ models_canceled.set()++ runtime = BlockingModelsRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ lease = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ models = asyncio.create_task(lease.models())+ assert await asyncio.to_thread(models_started.wait, 2)++ await lease.release()+ with pytest.raises(asyncio.CancelledError):+ await models++ assert models_canceled.is_set()+ assert runtime.closed+ raw_lease._lock.release.assert_awaited_once()+ assert not tuple(tmp_path.iterdir())+ await transport.close_runtime_pool()+++async def test_runtime_pool_rejects_borrower_after_lease_heartbeat_failure(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ first = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ raw_lease._heartbeat_error = RuntimeError("redis lease lost")++ with pytest.raises(CodexTransportError, match="runtime is unavailable"):+ await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )++ assert manager.acquire_lease.await_count == 1+ raw_lease._heartbeat_error = None+ await first.release()+ assert runtime.closed+ await transport.close_runtime_pool()+++async def test_runtime_pool_rejects_active_call_after_lease_heartbeat_failure(+ tmp_path,+ monkeypatch,+):+ raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(return_value=raw_lease)+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ runtime = _FakeRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=_runtime_factory(runtime),+ )+ lease = await transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ )+ raw_lease._heartbeat_error = RuntimeError("redis lease lost")++ async def tool_handler(_call):+ return CodexDynamicToolResult(content="ok")++ with pytest.raises(CodexTransportError, match="runtime is unavailable"):+ await lease.invoke(+ CodexInvocationRequest(prompt="must not start"),+ [],+ tool_handler,+ )++ assert runtime.last_request is None+ raw_lease._heartbeat_error = None+ await lease.release()+ assert runtime.closed+ await transport.close_runtime_pool()+++async def test_lease_guard_closes_operation_before_known_heartbeat_failure():+ lease, _ = _lease()+ lease._heartbeat_error = RuntimeError("redis lease lost")++ async def operation() -> str:+ raise AssertionError("provider operation must not start")++ provider_operation = operation()+ with pytest.raises(RuntimeError, match="heartbeat failed"):+ await _run_with_lease_guard(lease, provider_operation)++ assert provider_operation.cr_frame is None+++async def test_lease_guard_closes_operation_before_failed_monitor():+ lease, _ = _lease()++ async def fail_monitor() -> None:+ raise RuntimeError("auth monitor failed")++ monitor = asyncio.create_task(fail_monitor())+ await asyncio.sleep(0)++ async def operation() -> str:+ raise AssertionError("provider operation must not start")++ provider_operation = operation()+ with pytest.raises(RuntimeError, match="auth monitor failed"):+ await _run_with_lease_guard(+ lease,+ provider_operation,+ monitor_task=monitor,+ )++ assert provider_operation.cr_frame is None+++async def test_runtime_pool_shutdown_cleans_other_actors_after_close_error(+ tmp_path,+ monkeypatch,+):+ first_raw_lease, _ = _lease()+ second_raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(side_effect=[first_raw_lease, second_raw_lease])+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )++ class FailingCloseRuntime(_FakeRuntime):+ async def close(self):+ self.closed = True+ raise RuntimeError("runtime close failed")++ first_runtime = FailingCloseRuntime()+ second_runtime = FailingCloseRuntime()+ runtimes = iter((first_runtime, second_runtime))++ async def runtime_factory(home):+ runtime = next(runtimes)+ runtime.home = home.path+ return runtime++ transport = CodexTransport(+ temp_root=tmp_path,+ control_timeout_seconds=0.1,+ runtime_factory=runtime_factory,+ )+ await asyncio.gather(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ ),+ transport.acquire_runtime_lease(+ "user-2",+ "credential-2",+ lock_timeout_seconds=1,+ ),+ )++ results = await asyncio.gather(+ transport.close_runtime_pool(),+ transport.close_runtime_pool(),+ return_exceptions=True,+ )++ assert len(results) == 2+ assert all(isinstance(result, CodexTransportError) for result in results)+ assert first_runtime.closed+ assert second_runtime.closed+ assert any(+ "Additional cleanup failure: RuntimeError: runtime close failed" in note+ for result in results+ if isinstance(result, CodexTransportError)+ for note in result.__notes__+ )+ first_raw_lease._lock.release.assert_awaited_once()+ second_raw_lease._lock.release.assert_awaited_once()+ assert not tuple(tmp_path.iterdir())+++async def test_runtime_pool_shutdown_closes_actors_concurrently(tmp_path, monkeypatch):+ first_raw_lease, _ = _lease()+ second_raw_lease, _ = _lease()+ manager = MagicMock()+ manager.acquire_lease = AsyncMock(side_effect=[first_raw_lease, second_raw_lease])+ monkeypatch.setattr(+ "backend.integrations.creds_manager.IntegrationCredentialsManager",+ lambda: manager,+ )+ first_runtime = _FakeRuntime()+ second_runtime = _FakeRuntime()+ runtimes = iter((first_runtime, second_runtime))++ async def runtime_factory(home):+ runtime = next(runtimes)+ runtime.home = home.path+ return runtime++ transport = CodexTransport(+ temp_root=tmp_path,+ runtime_factory=runtime_factory,+ )+ await asyncio.gather(+ transport.acquire_runtime_lease(+ "user-1",+ "credential-1",+ lock_timeout_seconds=1,+ ),+ transport.acquire_runtime_lease(+ "user-2",+ "credential-2",+ lock_timeout_seconds=1,+ ),+ )+ pool = transport._runtime_pool+ original_close = pool._close_actor_bounded+ close_count = 0+ close_count_lock = threading.Lock()+ both_closing = threading.Event()+ allow_close = threading.Event()++ async def delayed_close(actor):+ nonlocal close_count+ with close_count_lock:+ close_count += 1+ if close_count == 2:+ both_closing.set()+ while not allow_close.is_set():+ await asyncio.sleep(0.01)+ await original_close(actor)++ monkeypatch.setattr(pool, "_close_actor_bounded", delayed_close)+ shutdown = asyncio.create_task(transport.close_runtime_pool())+ reached_both = await asyncio.to_thread(both_closing.wait, 2)+ allow_close.set()+ await shutdown++ assert reached_both+ assert first_runtime.closed+ assert second_runtime.closed+ first_raw_lease._lock.release.assert_awaited_once()+ second_raw_lease._lock.release.assert_awaited_once()+ assert not tuple(tmp_path.iterdir())+++async def test_never_returning_runtime_close_cannot_strand_home_or_capacity(tmp_path):+ lease, _ = _lease()++ class NonCancellingFuture(asyncio.Future[None]):+ def cancel(self, *_args, **_kwargs):+ return False++ class NeverClosingRuntime(_FakeRuntime):+ def close(self):+ return NonCancellingFuture()++ runtime = NeverClosingRuntime()+ transport = CodexTransport(+ temp_root=tmp_path,+ max_active_processes=1,+ control_timeout_seconds=0.01,+ runtime_factory=_runtime_factory(runtime),+ )++ with pytest.raises(CodexTransportError, match="runtime shutdown"):+ await asyncio.wait_for(transport._account(lease), timeout=0.2)++ assert transport._capacity._value == 1+ assert not tuple(tmp_path.iterdir())+++async def test_cancel_during_runtime_startup_drains_factory_before_home_cleanup(+ tmp_path,+):+ started = asyncio.Event()+ cancelled = asyncio.Event()++ async def blocked_runtime_factory(_home):+ started.set()+ try:+ await asyncio.Event().wait()+ finally:+ cancelled.set()++ transport = CodexTransport(+ temp_root=tmp_path,+ max_active_processes=1,+ runtime_factory=blocked_runtime_factory,+ )+ login = asyncio.create_task(transport.start_device_login())+ await asyncio.wait_for(started.wait(), timeout=1)+ login.cancel()++ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(login, timeout=2)++ assert cancelled.is_set()+ assert transport._capacity._value == 1+ assert not tuple(tmp_path.iterdir())+++async def test_caller_cancellation_stops_active_runtime_operation(tmp_path):+ lease, _ = _lease()+ started = asyncio.Event()+ cancelled = asyncio.Event()++ class BlockingRuntime(_FakeRuntime):+ async def invoke(self, request):+ started.set()+ try:+ await asyncio.Event().wait()+ finally:+ cancelled.set()++ runtime = BlockingRuntime()+ max_active_processes = 2+ transport = CodexTransport(+ temp_root=tmp_path,+ max_active_processes=max_active_processes,+ runtime_factory=_runtime_factory(runtime),+ )+ invocation = asyncio.create_task(+ transport.invoke(lease, CodexInvocationRequest(prompt="ok"))+ )+ await asyncio.wait_for(started.wait(), timeout=1)+ invocation.cancel()++ with pytest.raises(asyncio.CancelledError):+ await asyncio.wait_for(invocation, timeout=2)++ assert cancelled.is_set()+ assert runtime.closed+ assert transport._capacity._value == max_active_processes+ assert not tuple(tmp_path.iterdir())+++def _runtime_factory(runtime: "_FakeRuntime"):+ async def create(home):+ runtime.home = home.path+ return runtime++ return create+++def _lease() -> tuple[CredentialLease, AsyncMock]:+ credentials = credentials_from_bundle(_bundle())+ lock = AsyncMock()+ lock.locked.return_value = True+ lock.owned.return_value = True+ lock.timeout = 60+ checkpoint = AsyncMock()+ return CredentialLease(credentials, lock, checkpoint), checkpoint+++class _FakeLogin:+ details = CodexDeviceCodeDetails(+ login_id="login-1",+ verification_url="https://example.com/device",+ user_code="CODE-1234",+ )++ async def wait(self) -> bool:+ return True++ async def cancel(self) -> None:+ return None+++class _FakeRuntime:+ def __init__(+ self,+ *,+ rotate_auth: bool = False,+ fail_invoke: bool = False,+ complete_login: bool = False,+ rotate_on_account: bool = False,+ ) -> None:+ self.home = Path()+ self.rotate_auth = rotate_auth+ self.fail_invoke = fail_invoke+ self.complete_login = complete_login+ self.rotate_on_account = rotate_on_account+ self.closed = False+ self.refresh_was_checkpointed_before_invoke = False+ self.checkpoint_probe: Callable[[], bool] = lambda: False+ self.agent_tool_timeout: float | None = None+ self.agent_dynamic_tools: list[CodexDynamicToolSpec] | None = None+ self.last_request: CodexInvocationRequest | None = None++ async def start_device_code_login(self) -> _FakeLogin:+ if self.complete_login:+ materialize_auth_bundle(_bundle(), self.home / "auth.json")+ return _FakeLogin()++ async def account(self, *, refresh_token: bool = True) -> CodexAccountSnapshot:+ assert refresh_token+ if self.rotate_on_account:+ payload = json.loads((self.home / "auth.json").read_text())+ payload["tokens"]["refresh_token"] = "refresh-from-account"+ (self.home / "auth.json").write_text(json.dumps(payload))+ return CodexAccountSnapshot(+ connected=True,+ requires_openai_auth=True,+ account_type="chatgpt",+ email="[email protected]",+ plan_type="pro",+ )++ async def rate_limits(self) -> CodexRateLimitsSnapshot:+ return CodexRateLimitsSnapshot(plan_type="pro")++ async def models(self) -> list[CodexModelInfo]:+ return [+ CodexModelInfo(+ model="gpt-test",+ display_name="GPT Test",+ is_default=True,+ hidden=False,+ default_reasoning_effort="medium",+ supported_reasoning_efforts=["low", "medium", "high"],+ input_modalities=["text"],+ )+ ]++ async def invoke(self, request: CodexInvocationRequest) -> CodexInvocationResult:+ self.last_request = request+ if self.rotate_on_account:+ self.refresh_was_checkpointed_before_invoke = self.checkpoint_probe()+ if self.rotate_auth:+ payload = json.loads((self.home / "auth.json").read_text())+ payload["tokens"]["refresh_token"] = "refresh-rotated"+ (self.home / "auth.json").write_text(json.dumps(payload))+ if self.fail_invoke:+ raise RuntimeError("runtime failed")+ return CodexInvocationResult(+ response_id="turn-1",+ final_response="ok",+ status="completed",+ )++ async def invoke_agent(+ self,+ request: CodexInvocationRequest,+ dynamic_tools: list[CodexDynamicToolSpec],+ tool_handler,+ event_handler=None,+ *,+ tool_timeout_seconds: float,+ ) -> CodexInvocationResult:+ self.agent_tool_timeout = tool_timeout_seconds+ self.agent_dynamic_tools = dynamic_tools+ return await self.invoke(request)++ async def logout(self) -> None:+ return None++ async def close(self) -> None:+ self.closed = True+++def _bundle() -> CodexAuthBundleV1:+ return CodexAuthBundleV1(+ tokens=CodexAuthTokensV1(+ id_token=_jwt({"email": "[email protected]"}),+ access_token=_jwt({"exp": 1_900_000_000}),+ refresh_token="refresh-original",+ account_id="account-user",+ ),+ codex_runtime_version="0.144.4",+ )+++def _jwt(payload: dict[str, object]) -> str:+ encoded = (+ base64.urlsafe_b64encode(json.dumps(payload).encode()).decode().rstrip("=")+ )+ return f"header.{encoded}.signature"diff --git a/autogpt_platform/backend/backend/integrations/credential_lease.py b/autogpt_platform/backend/backend/integrations/credential_lease.pynew file mode 100644index 0000000000..effa869bc6--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/credential_lease.py@@ -0,0 +1,102 @@+import asyncio+from collections.abc import Awaitable, Callable++from redis.asyncio.lock import Lock as AsyncRedisLock++from backend.data.model import Credentials++CheckpointCallback = Callable[[Credentials, AsyncRedisLock], Awaitable[None]]+DeleteCallback = Callable[[Credentials, AsyncRedisLock], Awaitable[None]]+++class CredentialLease:+ def __init__(+ self,+ credentials: Credentials,+ lock: AsyncRedisLock,+ checkpoint_callback: CheckpointCallback,+ delete_callback: DeleteCallback | None = None,+ ) -> None:+ self.credentials = credentials+ self._lock = lock+ self._checkpoint_callback = checkpoint_callback+ self._delete_callback = delete_callback+ self._deleted = False+ self._heartbeat_task: asyncio.Task[None] | None = None+ self._heartbeat_error: BaseException | None = None+ self._heartbeat_failed = asyncio.Event()++ def start_heartbeat(self) -> None:+ if self.credentials.type != "oauth2":+ return+ if self.credentials.refresh_strategy != "provider_runtime":+ return+ if self._heartbeat_task is None:+ self._heartbeat_task = asyncio.create_task(self._heartbeat())++ async def validate(self) -> None:+ if self._heartbeat_error is not None:+ raise RuntimeError(+ "Credential lease heartbeat failed"+ ) from self._heartbeat_error+ if not (await self._lock.locked()) or not (await self._lock.owned()):+ raise RuntimeError("Credential lease ownership was lost")++ async def wait_for_failure(self) -> None:+ if self._heartbeat_error is None:+ await self._heartbeat_failed.wait()+ raise RuntimeError(+ "Credential lease heartbeat failed"+ ) from self._heartbeat_error++ @property+ def failure(self) -> BaseException | None:+ return self._heartbeat_error++ async def checkpoint(self, updated: Credentials) -> None:+ if self._deleted:+ raise RuntimeError("Credential lease was deleted")+ await self.validate()+ if (+ updated.id != self.credentials.id+ or updated.provider != self.credentials.provider+ ):+ raise RuntimeError("Credential lease identity changed")+ await self._checkpoint_callback(updated, self._lock)+ self.credentials = updated++ async def delete(self) -> None:+ if self._delete_callback is None:+ raise RuntimeError("Credential lease does not support deletion")+ if self._deleted:+ return+ await self.validate()+ await self._delete_callback(self.credentials, self._lock)+ self._deleted = True++ async def release(self) -> None:+ heartbeat = self._heartbeat_task+ if heartbeat is not None:+ heartbeat.cancel()+ try:+ await heartbeat+ except asyncio.CancelledError:+ pass+ self._heartbeat_task = None+ if (await self._lock.locked()) and (await self._lock.owned()):+ await self._lock.release()++ async def _heartbeat(self) -> None:+ timeout = float(self._lock.timeout or 60)+ interval = max(timeout / 3, 0.01)+ try:+ while True:+ await asyncio.sleep(interval)+ extended = await self._lock.extend(timeout, replace_ttl=True)+ if not extended:+ raise RuntimeError("Credential lease extension was rejected")+ except asyncio.CancelledError:+ raise+ except BaseException as error:+ self._heartbeat_error = error+ self._heartbeat_failed.set()diff --git a/autogpt_platform/backend/backend/integrations/credential_lease_test.py b/autogpt_platform/backend/backend/integrations/credential_lease_test.pynew file mode 100644index 0000000000..1da7ddaf14--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/credential_lease_test.py@@ -0,0 +1,99 @@+import asyncio+from unittest.mock import AsyncMock++import pytest++from backend.integrations.credential_lease import CredentialLease+from backend.integrations.creds_manager_test import _provider_runtime_credentials+++async def test_checkpoint_uses_existing_lock_without_reacquiring():+ credentials = _provider_runtime_credentials()+ updated = credentials.model_copy(update={"title": "refreshed"})+ lock = AsyncMock()+ lock.locked.return_value = True+ lock.owned.return_value = True+ lock.timeout = 60+ checkpoint = AsyncMock()+ lease = CredentialLease(credentials, lock, checkpoint)++ await lease.checkpoint(updated)++ checkpoint.assert_awaited_once_with(updated, lock)+ assert lease.credentials is updated+++async def test_checkpoint_fails_closed_after_lock_ownership_is_lost():+ credentials = _provider_runtime_credentials()+ lock = AsyncMock()+ lock.locked.return_value = True+ lock.owned.return_value = False+ checkpoint = AsyncMock()+ lease = CredentialLease(credentials, lock, checkpoint)++ with pytest.raises(RuntimeError, match="lease"):+ await lease.checkpoint(credentials)++ checkpoint.assert_not_awaited()+++async def test_release_stops_heartbeat_before_unlocking():+ credentials = _provider_runtime_credentials()+ lock = AsyncMock()+ lock.locked.return_value = True+ lock.owned.return_value = True+ lock.timeout = 0.03+ lock.extend.return_value = True+ lease = CredentialLease(credentials, lock, AsyncMock())++ lease.start_heartbeat()+ for _ in range(100):+ if lock.extend.await_count:+ break+ await asyncio.sleep(0.001)+ await lease.release()++ lock.extend.assert_awaited()+ lock.release.assert_awaited_once()+++async def test_wait_for_failure_reports_heartbeat_loss_immediately():+ credentials = _provider_runtime_credentials()+ lock = AsyncMock()+ lock.locked.return_value = True+ lock.owned.return_value = True+ lock.timeout = 0.03+ lock.extend.return_value = False+ lease = CredentialLease(credentials, lock, AsyncMock())++ lease.start_heartbeat()++ with pytest.raises(RuntimeError, match="heartbeat"):+ await asyncio.wait_for(lease.wait_for_failure(), timeout=1)+ with pytest.raises(RuntimeError, match="heartbeat"):+ await lease.validate()++ await lease.release()+++async def test_delete_uses_owned_lock_and_prevents_later_checkpoint():+ credentials = _provider_runtime_credentials()+ lock = AsyncMock()+ lock.locked.return_value = True+ lock.owned.return_value = True+ checkpoint = AsyncMock()+ delete = AsyncMock()+ lease = CredentialLease(credentials, lock, checkpoint, delete)++ await lease.delete()+ await lease.delete()++ delete.assert_awaited_once_with(credentials, lock)+ with pytest.raises(RuntimeError, match="deleted"):+ await lease.checkpoint(credentials)+ checkpoint.assert_not_awaited()diff --git a/autogpt_platform/backend/backend/integrations/credentials_store.py b/autogpt_platform/backend/backend/integrations/credentials_store.pyindex ab42de3565..465fbcffd9 100644--- a/autogpt_platform/backend/backend/integrations/credentials_store.py+++ b/autogpt_platform/backend/backend/integrations/credentials_store.py@@ -477,6 +477,52 @@ class IntegrationCredentialsStore:]await self._set_user_integration_creds(user_id, updated_credentials_list)+ async def upsert_single_provider_creds(+ self,+ user_id: str,+ credentials: Credentials,+ ) -> Credentials:+ if credentials.id in SYSTEM_CREDENTIAL_IDS or credentials.is_managed:+ raise ValueError("Single-provider upsert requires user-owned credentials")++ async with await self.locked_user_integrations(user_id):+ persisted = await self._get_persisted_user_creds_unlocked(user_id)+ matching = [+ credential+ for credential in persisted+ if provider_matches(credential.provider, credentials.provider)+ ]+ if any(credential.is_managed for credential in matching):+ raise ValueError("Cannot replace a managed credential")++ replacement = credentials+ compatible = next(+ (+ credential+ for credential in matching+ if type(credential) is type(credentials)+ ),+ None,+ )+ if compatible is not None:+ replacement = credentials.model_copy(+ update={+ "id": compatible.id,+ "title": compatible.title or credentials.title,+ }+ )++ kept = [+ credential+ for credential in persisted+ if not provider_matches(credential.provider, credentials.provider)+ ]+ await self._set_user_integration_creds(+ user_id,+ [*kept, replacement],+ )+ return replacement+async def delete_creds_by_id(self, user_id: str, credentials_id: str) -> None:if credentials_id in SYSTEM_CREDENTIAL_IDS:raise ValueError(f"System credential #{credentials_id} cannot be deleted")@@ -524,6 +570,7 @@ class IntegrationCredentialsStore:provider: str,scopes: list[str],use_pkce: bool = False,+ expires_in_seconds: int = 600,# New parameters for external API OAuth flowscallback_url: Optional[str] = None,state_metadata: Optional[dict] = None,@@ -531,7 +578,7 @@ class IntegrationCredentialsStore:credential_id: Optional[str] = None,) -> tuple[str, str]:token = secrets.token_urlsafe(32)- expires_at = datetime.now(timezone.utc) + timedelta(minutes=10)+ expires_at = datetime.now(timezone.utc) + timedelta(seconds=expires_in_seconds)(code_challenge, code_verifier) = self._generate_code_challenge()diff --git a/autogpt_platform/backend/backend/integrations/credentials_store_upsert_test.py b/autogpt_platform/backend/backend/integrations/credentials_store_upsert_test.pynew file mode 100644index 0000000000..90ee58141e--- /dev/null+++ b/autogpt_platform/backend/backend/integrations/credentials_store_upsert_test.py@@ -0,0 +1,100 @@+from unittest.mock import AsyncMock++import pytest+from pydantic import SecretStr++from backend.data.model import APIKeyCredentials, OAuth2Credentials+from backend.integrations.credentials_store import IntegrationCredentialsStore+++def _codex_credentials(+ credential_id: str,+ *,+ title: str | None = None,+ managed: bool = False,+) -> OAuth2Credentials:+ return OAuth2Credentials(+ id=credential_id,+ provider="codex",+ title=title,+ is_managed=managed,+ access_token=SecretStr("access"),+ refresh_token=SecretStr("refresh"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr("state"),+ provider_state_version=1,+ )+++async def test_upsert_single_provider_preserves_id_and_removes_duplicates(mocker):+ store = IntegrationCredentialsStore()+ first = _codex_credentials("existing", title="My account")+ duplicate = _codex_credentials("duplicate", title="Duplicate")+ github = APIKeyCredentials(+ id="github",+ provider="github",+ api_key=SecretStr("key"),+ )+ mocker.patch.object(+ store,+ "_get_persisted_user_creds_unlocked",+ AsyncMock(return_value=[github, first, duplicate]),+ )+ persist = mocker.patch.object(+ store,+ "_set_user_integration_creds",+ new_callable=AsyncMock,+ )+ mocker.patch.object(+ store,+ "locked_user_integrations",+ AsyncMock(return_value=_NoopLockContext()),+ )++ updated = await store.upsert_single_provider_creds(+ "user-a",+ _codex_credentials("fresh", title="ChatGPT for Codex"),+ )++ assert updated.id == "existing"+ assert updated.title == "My account"+ persist.assert_awaited_once_with("user-a", [github, updated])+++async def test_upsert_single_provider_never_replaces_managed_credentials(mocker):+ store = IntegrationCredentialsStore()+ managed = _codex_credentials("managed", managed=True)+ mocker.patch.object(+ store,+ "_get_persisted_user_creds_unlocked",+ AsyncMock(return_value=[managed]),+ )+ persist = mocker.patch.object(+ store,+ "_set_user_integration_creds",+ new_callable=AsyncMock,+ )+ mocker.patch.object(+ store,+ "locked_user_integrations",+ AsyncMock(return_value=_NoopLockContext()),+ )++ with pytest.raises(ValueError, match="managed"):+ await store.upsert_single_provider_creds(+ "user-a",+ _codex_credentials("fresh"),+ )++ persist.assert_not_awaited()+++class _NoopLockContext:+ async def __aenter__(self) -> None:+ return None++ async def __aexit__(self, _exc_type, _exc, _traceback) -> None:+ return Nonediff --git a/autogpt_platform/backend/backend/integrations/creds_manager.py b/autogpt_platform/backend/backend/integrations/creds_manager.pyindex 69ef77a47f..21d0a3d11f 100644--- a/autogpt_platform/backend/backend/integrations/creds_manager.py+++ b/autogpt_platform/backend/backend/integrations/creds_manager.py@@ -1,13 +1,16 @@import loggingimport os+from collections.abc import AsyncIteratorfrom contextlib import asynccontextmanagerfrom datetime import datetime+from functools import partialfrom typing import TYPE_CHECKING, Any, Callable, Coroutinefrom autogpt_libs.utils.synchronize import AsyncRedisKeyedMutexfrom redis.asyncio.lock import Lock as AsyncRedisLockfrom backend.data.model import Credentials, OAuth2Credentials+from backend.integrations.credential_lease import CredentialLeasefrom backend.integrations.credentials_store import (IntegrationCredentialsStore,provider_matches,@@ -129,8 +132,15 @@ class IntegrationCredentialsManager:if not credentials:return None- # Refresh OAuth credentials if needed- if credentials.type == "oauth2" and credentials.access_token_expires_at:+ if (+ credentials.type == "oauth2"+ and credentials.refresh_strategy == "provider_runtime"+ ):+ logger.debug(+ "Skipping generic refresh for provider-managed credentials #%s",+ credentials.id,+ )+ elif credentials.type == "oauth2" and credentials.access_token_expires_at:logger.debug(f"Credentials #{credentials.id} expire at "f"{datetime.fromtimestamp(credentials.access_token_expires_at)}; "@@ -154,13 +164,25 @@ class IntegrationCredentialsManager:# to allow priority access for refreshing/updating the tokens.async with self._locked(user_id, credentials_id, "!time_sensitive"):lock = await self._acquire_lock(user_id, credentials_id)- credentials = await self.get(user_id, credentials_id, lock=False)- if not credentials:- raise ValueError(- f"Credentials #{credentials_id} for user #{user_id} not found"- )+ try:+ credentials = await self.get(user_id, credentials_id, lock=False)+ if not credentials:+ raise ValueError(+ f"Credentials #{credentials_id} for user #{user_id} not found"+ )+ except BaseException:+ await self._release_owned_lock(lock)+ raisereturn credentials, lock+ async def acquire_lease(self, user_id: str, credentials_id: str) -> CredentialLease:+ credentials, lock = await self.acquire(user_id, credentials_id)+ checkpoint = partial(self.update_acquired, user_id)+ delete = partial(self.delete_acquired, user_id)+ lease = CredentialLease(credentials, lock, checkpoint, delete)+ lease.start_heartbeat()+ return lease+def cached_getter(self, user_id: str) -> Callable[[str], "Coroutine[Any, Any, Credentials | None]"]:@@ -175,7 +197,11 @@ class IntegrationCredentialsManager:credential = next((c for c in all_credentials if c.id == creds_id), None)if not credential:return None- if credential.type != "oauth2" or not credential.access_token_expires_at:+ if (+ credential.type != "oauth2"+ or credential.refresh_strategy == "provider_runtime"+ or not credential.access_token_expires_at+ ):# Credential doesn't expirereturn credential@@ -187,6 +213,9 @@ class IntegrationCredentialsManager:async def refresh_if_needed(self, user_id: str, credentials: OAuth2Credentials, lock: bool = True) -> OAuth2Credentials:+ if credentials.refresh_strategy == "provider_runtime":+ return credentials+# When lock=False, skip ALL Redis locking (both the outer "refresh" scope# lock and the inner credential lock). This is used by the copilot's# integration_creds module which runs across multiple threads with separate@@ -262,6 +291,97 @@ class IntegrationCredentialsManager:# Notify listeners so the updated credential is picked up immediately._invoke_creds_changed_hook(user_id, updated.provider)+ async def upsert_single_provider(+ self,+ user_id: str,+ credentials: Credentials,+ ) -> Credentials:+ async with self.locked_provider_credentials(user_id, credentials.provider):+ return await self.upsert_single_provider_locked(user_id, credentials)++ @asynccontextmanager+ async def locked_provider_credentials(+ self,+ user_id: str,+ provider: str,+ ) -> AsyncIterator[None]:+ matching = await self.store.get_creds_by_provider(user_id, provider)+ locks: list[AsyncRedisLock] = []+ try:+ for credential_id in sorted({credential.id for credential in matching}):+ locks.append(await self._acquire_lock(user_id, credential_id))+ yield+ finally:+ for lock in reversed(locks):+ await self._release_owned_lock(lock)++ async def upsert_single_provider_locked(+ self,+ user_id: str,+ credentials: Credentials,+ ) -> Credentials:+ stored = await self.store.upsert_single_provider_creds(user_id, credentials)+ _invoke_creds_changed_hook(user_id, stored.provider)+ return stored++ async def update_acquired(+ self,+ user_id: str,+ updated: Credentials,+ lock: AsyncRedisLock,+ ) -> None:+ expected_lock_name = str(self._credentials_lock_key(user_id, updated.id))+ if (+ lock.name != expected_lock_name+ or not (await lock.locked())+ or not (await lock.owned())+ ):+ raise RuntimeError(+ f"Cannot update credentials #{updated.id} without its owned lock"+ )+ if updated.type != "oauth2" or updated.refresh_strategy != "provider_runtime":+ raise RuntimeError("Acquired updates require provider-runtime credentials")++ current = await self.store.get_creds_by_id(user_id, updated.id)+ if (+ current is None+ or current.type != "oauth2"+ or current.refresh_strategy != "provider_runtime"+ or current.id != updated.id+ or current.provider != updated.provider+ or type(current) is not type(updated)+ ):+ raise RuntimeError("Provider-runtime credential identity changed")+ await self.store.update_creds(user_id, updated)+ _invoke_creds_changed_hook(user_id, updated.provider)++ async def delete_acquired(+ self,+ user_id: str,+ credentials: Credentials,+ lock: AsyncRedisLock,+ ) -> None:+ expected_lock_name = str(self._credentials_lock_key(user_id, credentials.id))+ if (+ lock.name != expected_lock_name+ or not (await lock.locked())+ or not (await lock.owned())+ ):+ raise RuntimeError(+ f"Cannot delete credentials #{credentials.id} without its owned lock"+ )++ current = await self.store.get_creds_by_id(user_id, credentials.id)+ if (+ current is None+ or current.id != credentials.id+ or current.provider != credentials.provider+ or type(current) is not type(credentials)+ ):+ raise RuntimeError("Credential identity changed before deletion")+ await self.store.delete_creds_by_id(user_id, credentials.id)+ _invoke_creds_changed_hook(user_id, credentials.provider)+async def delete(self, user_id: str, credentials_id: str) -> None:async with self._locked(user_id, credentials_id):# Read inside the lock to avoid TOCTOU — another coroutine could@@ -276,13 +396,26 @@ class IntegrationCredentialsManager:async def _acquire_lock(self, user_id: str, credentials_id: str, *args: str) -> AsyncRedisLock:- key = (+ key = self._credentials_lock_key(user_id, credentials_id, *args)+ locks = await self.locks()+ return await locks.acquire(key)++ @staticmethod+ def _credentials_lock_key(+ user_id: str, credentials_id: str, *args: str+ ) -> tuple[str, ...]:+ return (f"user:{user_id}",f"credentials:{credentials_id}",*args,)- locks = await self.locks()- return await locks.acquire(key)++ async def _release_owned_lock(self, lock: AsyncRedisLock) -> None:+ if (await lock.locked()) and (await lock.owned()):+ try:+ await lock.release()+ except Exception:+ logger.warning("Failed to release credentials lock", exc_info=True)@asynccontextmanagerasync def _locked(self, user_id: str, credentials_id: str, *args: str):diff --git a/autogpt_platform/backend/backend/integrations/creds_manager_test.py b/autogpt_platform/backend/backend/integrations/creds_manager_test.pyindex 68665bd731..e350642c2e 100644--- a/autogpt_platform/backend/backend/integrations/creds_manager_test.py+++ b/autogpt_platform/backend/backend/integrations/creds_manager_test.py@@ -1,8 +1,13 @@-"""Tests for creds_manager hook system: register, invoke, and CRUD integration."""+"""Tests for credential hooks and provider-runtime credential leases."""++from unittest.mock import AsyncMockimport pytest+from pydantic import SecretStr+from backend.data.model import OAuth2Credentialsfrom backend.integrations.creds_manager import (+ IntegrationCredentialsManager,_invoke_creds_changed_hook,register_creds_changed_hook,unregister_creds_changed_hook,@@ -58,3 +63,227 @@ class TestInvokeCredsChangedHook:_invoke_creds_changed_hook("user-b", "slack")assert calls == [("user-a", "github"), ("user-b", "slack")]+++async def test_provider_runtime_refresh_never_resolves_generic_handler(mocker):+ manager = IntegrationCredentialsManager()+ resolve_handler = mocker.patch.object(+ manager,+ "_get_oauth_handler",+ new_callable=AsyncMock,+ )+ credentials = _provider_runtime_credentials()++ locked = await manager.refresh_if_needed("user-a", credentials, lock=True)+ unlocked = await manager.refresh_if_needed("user-a", credentials, lock=False)++ assert locked is credentials+ assert unlocked is credentials+ resolve_handler.assert_not_awaited()++[email protected]("missing", [False, True])+async def test_failed_acquire_releases_owned_lock(mocker, missing):+ manager = IntegrationCredentialsManager()+ lock = _owned_lock()+ mocker.patch.object(manager, "_locked", return_value=_noop_lock_context())+ mocker.patch.object(manager, "_acquire_lock", AsyncMock(return_value=lock))+ failure = None if missing else RuntimeError("failed")+ get = AsyncMock(return_value=failure) if missing else AsyncMock(side_effect=failure)+ mocker.patch.object(manager, "get", get)++ expected = ValueError if missing else RuntimeError+ with pytest.raises(expected):+ await manager.acquire("user-a", "cred-id")++ lock.release.assert_awaited_once()+++async def test_update_acquired_requires_matching_owned_lock(mocker):+ manager = IntegrationCredentialsManager()+ update = mocker.patch.object(manager.store, "update_creds", new_callable=AsyncMock)+ current = _provider_runtime_credentials()+ mocker.patch.object(+ manager.store,+ "get_creds_by_id",+ AsyncMock(return_value=current),+ )+ lock = _owned_lock()++ await manager.update_acquired("user-a", current, lock)+ update.assert_awaited_once_with("user-a", current)++ lock.name = str(("user:user-a", "credentials:different"))+ with pytest.raises(RuntimeError, match="owned lock"):+ await manager.update_acquired("user-a", current, lock)++ lock.name = str(("user:user-a", "credentials:cred-id"))+ lock.owned.return_value = False+ with pytest.raises(RuntimeError, match="owned lock"):+ await manager.update_acquired("user-a", current, lock)+++async def test_update_acquired_rejects_provider_change(mocker):+ manager = IntegrationCredentialsManager()+ current = _provider_runtime_credentials()+ mocker.patch.object(+ manager.store,+ "get_creds_by_id",+ AsyncMock(return_value=current),+ )++ with pytest.raises(RuntimeError, match="identity"):+ await manager.update_acquired(+ "user-a",+ current.model_copy(update={"provider": "other"}),+ _owned_lock(),+ )+++async def test_delete_acquired_deletes_while_lease_lock_is_owned(mocker):+ manager = IntegrationCredentialsManager()+ credentials = _provider_runtime_credentials()+ mocker.patch.object(+ manager.store,+ "get_creds_by_id",+ AsyncMock(return_value=credentials),+ )+ delete = mocker.patch.object(+ manager.store,+ "delete_creds_by_id",+ new_callable=AsyncMock,+ )+ lock = _owned_lock()++ await manager.delete_acquired("user-a", credentials, lock)++ delete.assert_awaited_once_with("user-a", credentials.id)+ lock.release.assert_not_awaited()+++async def test_upsert_single_provider_notifies_credential_listeners(mocker):+ manager = IntegrationCredentialsManager()+ credentials = _provider_runtime_credentials()+ mocker.patch.object(+ manager.store,+ "get_creds_by_provider",+ AsyncMock(return_value=[]),+ )+ upsert = mocker.patch.object(+ manager.store,+ "upsert_single_provider_creds",+ AsyncMock(return_value=credentials),+ )+ changes: list[tuple[str, str]] = []+ register_creds_changed_hook(lambda user, provider: changes.append((user, provider)))++ stored = await manager.upsert_single_provider("user-a", credentials)++ assert stored is credentials+ upsert.assert_awaited_once_with("user-a", credentials)+ assert changes == [("user-a", "codex")]+++async def test_upsert_single_provider_waits_for_existing_credential_lease(mocker):+ manager = IntegrationCredentialsManager()+ existing = _provider_runtime_credentials()+ replacement = existing.model_copy(update={"access_token": SecretStr("new-access")})+ mocker.patch.object(+ manager.store,+ "get_creds_by_provider",+ AsyncMock(return_value=[existing]),+ )+ lock = _owned_lock()+ acquire = mocker.patch.object(+ manager,+ "_acquire_lock",+ AsyncMock(return_value=lock),+ )++ async def upsert_while_locked(_user_id, credentials):+ lock.release.assert_not_awaited()+ return credentials++ upsert = mocker.patch.object(+ manager.store,+ "upsert_single_provider_creds",+ AsyncMock(side_effect=upsert_while_locked),+ )++ stored = await manager.upsert_single_provider("user-a", replacement)++ assert stored is replacement+ acquire.assert_awaited_once_with("user-a", existing.id)+ upsert.assert_awaited_once_with("user-a", replacement)+ lock.release.assert_awaited_once()+++async def test_locked_provider_credentials_can_guard_state_recheck_and_upsert(mocker):+ manager = IntegrationCredentialsManager()+ existing = _provider_runtime_credentials()+ replacement = existing.model_copy(update={"title": "reconnected"})+ mocker.patch.object(+ manager.store,+ "get_creds_by_provider",+ AsyncMock(return_value=[existing]),+ )+ lock = _owned_lock()+ mocker.patch.object(manager, "_acquire_lock", AsyncMock(return_value=lock))+ upsert = mocker.patch.object(+ manager.store,+ "upsert_single_provider_creds",+ AsyncMock(return_value=replacement),+ )++ async with manager.locked_provider_credentials("user-a", "codex"):+ lock.release.assert_not_awaited()+ stored = await manager.upsert_single_provider_locked("user-a", replacement)+ lock.release.assert_not_awaited()++ assert stored is replacement+ upsert.assert_awaited_once_with("user-a", replacement)+ lock.release.assert_awaited_once()+++class _NoopLockContext:+ async def __aenter__(self) -> None:+ return None++ async def __aexit__(self, _exc_type, _exc, _traceback) -> None:+ return None+++def _noop_lock_context() -> _NoopLockContext:+ return _NoopLockContext()+++def _owned_lock() -> AsyncMock:+ lock = AsyncMock()+ lock.name = str(("user:user-a", "credentials:cred-id"))+ lock.locked.return_value = True+ lock.owned.return_value = True+ lock.timeout = 60+ lock.extend.return_value = True+ return lock+++def _provider_runtime_credentials() -> OAuth2Credentials:+ return OAuth2Credentials(+ id="cred-id",+ provider="codex",+ access_token=SecretStr("access"),+ access_token_expires_at=1,+ refresh_token=SecretStr("refresh"),+ scopes=[],+ refresh_strategy="provider_runtime",+ provider_state=SecretStr("state"),+ provider_state_version=1,+ )diff --git a/autogpt_platform/backend/backend/integrations/providers.py b/autogpt_platform/backend/backend/integrations/providers.pyindex 70c54aca01..ac4c968edb 100644--- a/autogpt_platform/backend/backend/integrations/providers.py+++ b/autogpt_platform/backend/backend/integrations/providers.py@@ -14,6 +14,7 @@ class ProviderName(str, Enum):AIML_API = "aiml_api"ANTHROPIC = "anthropic"APOLLO = "apollo"+ CODEX = "codex"COMPASS = "compass"DATABASE = "database"DISCORD = "discord"diff --git a/autogpt_platform/backend/backend/platform_linking/chat.py b/autogpt_platform/backend/backend/platform_linking/chat.pyindex b518840244..3eec663eb3 100644--- a/autogpt_platform/backend/backend/platform_linking/chat.py+++ b/autogpt_platform/backend/backend/platform_linking/chat.py@@ -360,6 +360,8 @@ async def start_chat_turn(request: BotChatRequest) -> ChatTurnHandle:organization_id=org_id,team_id=team_id,file_ids=request.file_ids or None,+ llm_auth_provider=session.metadata.llm_auth_provider,+ llm_credential_id=session.metadata.llm_credential_id,)logger.info(diff --git a/autogpt_platform/backend/backend/util/entitlements.py b/autogpt_platform/backend/backend/util/entitlements.pynew file mode 100644index 0000000000..d776ff571f--- /dev/null+++ b/autogpt_platform/backend/backend/util/entitlements.py@@ -0,0 +1,83 @@+from enum import Enum+from types import MappingProxyType+from typing import Mapping++from prisma.enums import SubscriptionTier+from pydantic import BaseModel, ConfigDict++from backend.util.clients import get_database_manager_async_client+from backend.util.settings import BehaveAs, Settings+++class Entitlement(str, Enum):+ CODEX_SUBSCRIPTION_TRANSPORT = "codex_subscription_transport"+++class EntitlementPolicy(BaseModel):+ model_config = ConfigDict(frozen=True)++ minimum_tier: SubscriptionTier+ allow_local: bool = False+++ENTITLEMENT_POLICIES: Mapping[Entitlement, EntitlementPolicy] = MappingProxyType(+ {+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT: EntitlementPolicy(+ minimum_tier=SubscriptionTier.MAX,+ allow_local=True,+ ),+ }+)++_TIER_ORDER = (+ SubscriptionTier.NO_TIER,+ SubscriptionTier.BASIC,+ SubscriptionTier.PRO,+ SubscriptionTier.MAX,+ SubscriptionTier.BUSINESS,+ SubscriptionTier.ENTERPRISE,+)+_TIER_RANK = {tier: rank for rank, tier in enumerate(_TIER_ORDER)}++settings = Settings()+++class EntitlementRequiredError(Exception):+ def __init__(self, entitlement: Entitlement, minimum_tier: SubscriptionTier):+ self.entitlement = entitlement+ self.minimum_tier = minimum_tier+ super().__init__(+ f"{entitlement.value} requires a {minimum_tier.value} plan or higher"+ )+++async def _get_user_subscription_tier(user_id: str) -> SubscriptionTier:+ """Resolve a tier through DatabaseManager without caching this result.++ A missing user has no entitlement. Other database and transport failures+ propagate so callers can retry instead of treating an outage as a denial.+ """+ try:+ tier = await get_database_manager_async_client().get_user_subscription_tier(+ user_id+ )+ except ValueError:+ return SubscriptionTier.NO_TIER+ return SubscriptionTier(tier)+++async def has_entitlement(user_id: str, entitlement: Entitlement) -> bool:+ """Check one centralized entitlement policy against the user's DB tier."""+ policy = ENTITLEMENT_POLICIES[entitlement]+ if policy.allow_local and settings.config.behave_as == BehaveAs.LOCAL:+ return True++ tier = await _get_user_subscription_tier(user_id)+ return _TIER_RANK[tier] >= _TIER_RANK[policy.minimum_tier]+++async def require_entitlement(user_id: str, entitlement: Entitlement) -> None:+ """Raise when the centralized entitlement policy is not satisfied."""+ policy = ENTITLEMENT_POLICIES[entitlement]+ if not await has_entitlement(user_id, entitlement):+ raise EntitlementRequiredError(entitlement, policy.minimum_tier)diff --git a/autogpt_platform/backend/backend/util/entitlements_test.py b/autogpt_platform/backend/backend/util/entitlements_test.pynew file mode 100644index 0000000000..878ff75e54--- /dev/null+++ b/autogpt_platform/backend/backend/util/entitlements_test.py@@ -0,0 +1,156 @@+from unittest.mock import AsyncMock, MagicMock, patch++import pytest+from prisma.enums import SubscriptionTier++from backend.util import entitlements+from backend.util.entitlements import Entitlement, EntitlementRequiredError+from backend.util.settings import BehaveAs+++def _database_client(*tiers: SubscriptionTier | Exception):+ client = MagicMock()+ client.get_user_subscription_tier = AsyncMock(side_effect=tiers)+ return client+++async def test_local_entitlement_bypasses_database_manager():+ with (+ patch.object(entitlements.settings.config, "behave_as", BehaveAs.LOCAL),+ patch.object(entitlements, "get_database_manager_async_client") as get_db,+ ):+ assert await entitlements.has_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )+ await entitlements.require_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )++ get_db.assert_not_called()+++ ("tier", "allowed"),+ [+ (SubscriptionTier.NO_TIER, False),+ (SubscriptionTier.BASIC, False),+ (SubscriptionTier.PRO, False),+ (SubscriptionTier.MAX, True),+ (SubscriptionTier.BUSINESS, True),+ (SubscriptionTier.ENTERPRISE, True),+ ],+)+async def test_cloud_entitlement_uses_minimum_tier_order(+ tier: SubscriptionTier,+ allowed: bool,+):+ client = _database_client(tier)+ with (+ patch.object(entitlements.settings.config, "behave_as", BehaveAs.CLOUD),+ patch.object(+ entitlements,+ "get_database_manager_async_client",+ return_value=client,+ ),+ ):+ result = await entitlements.has_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )++ assert result is allowed+ client.get_user_subscription_tier.assert_awaited_once_with("user-1")+++async def test_repeated_checks_repeat_database_manager_lookup():+ client = _database_client(SubscriptionTier.MAX, SubscriptionTier.PRO)+ with (+ patch.object(entitlements.settings.config, "behave_as", BehaveAs.CLOUD),+ patch.object(+ entitlements,+ "get_database_manager_async_client",+ return_value=client,+ ),+ ):+ first = await entitlements.has_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )+ second = await entitlements.has_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )++ assert first is True+ assert second is False+ assert client.get_user_subscription_tier.await_count == 2+++async def test_require_entitlement_raises_below_minimum_tier():+ client = _database_client(SubscriptionTier.PRO)+ with (+ patch.object(entitlements.settings.config, "behave_as", BehaveAs.CLOUD),+ patch.object(+ entitlements,+ "get_database_manager_async_client",+ return_value=client,+ ),+ ):+ with pytest.raises(EntitlementRequiredError) as exc_info:+ await entitlements.require_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )++ assert exc_info.value.entitlement == Entitlement.CODEX_SUBSCRIPTION_TRANSPORT+ assert exc_info.value.minimum_tier == SubscriptionTier.MAX+++async def test_database_manager_error_propagates():+ client = _database_client(RuntimeError("database unavailable"))+ with (+ patch.object(entitlements.settings.config, "behave_as", BehaveAs.CLOUD),+ patch.object(+ entitlements,+ "get_database_manager_async_client",+ return_value=client,+ ),+ ):+ with pytest.raises(RuntimeError, match="database unavailable"):+ await entitlements.has_entitlement(+ "user-1",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )+++async def test_missing_user_is_treated_as_no_tier():+ client = _database_client(ValueError("User not found"))+ with (+ patch.object(entitlements.settings.config, "behave_as", BehaveAs.CLOUD),+ patch.object(+ entitlements,+ "get_database_manager_async_client",+ return_value=client,+ ),+ ):+ allowed = await entitlements.has_entitlement(+ "missing-user",+ Entitlement.CODEX_SUBSCRIPTION_TRANSPORT,+ )++ assert allowed is False+++def test_codex_policy_is_centralized_as_max_plus_with_local_access():+ policy = entitlements.ENTITLEMENT_POLICIES[Entitlement.CODEX_SUBSCRIPTION_TRANSPORT]++ assert policy.minimum_tier == SubscriptionTier.MAX+ assert policy.allow_local is Truediff --git a/autogpt_platform/backend/backend/util/metrics.py b/autogpt_platform/backend/backend/util/metrics.pyindex ef60737f2d..fb0235c018 100644--- a/autogpt_platform/backend/backend/util/metrics.py+++ b/autogpt_platform/backend/backend/util/metrics.py@@ -1,4 +1,5 @@import logging+import refrom enum import Enumfrom types import TracebackType@@ -9,6 +10,7 @@ from sentry_sdk.api import flush as _sentry_flushfrom sentry_sdk.integrations import DidNotEnablefrom sentry_sdk.integrations.asyncio import AsyncioIntegrationfrom sentry_sdk.integrations.logging import LoggingIntegration+from sentry_sdk.scrubber import DEFAULT_DENYLIST, EventScrubbertry:from sentry_sdk.integrations.anthropic import AnthropicIntegration@@ -21,6 +23,7 @@ except ImportError:LaunchDarklyIntegration = None # type: ignore[assignment,misc]from backend.util import feature_flag+from backend.util.security import SENSITIVE_FIELD_NAMESfrom backend.util.settings import BehaveAs, Settingssettings = Settings()@@ -71,6 +74,65 @@ _PIKA_RECONNECT_SIGNATURES = ("connection_lost",)+_SENTRY_SENSITIVE_FIELDS = sorted(+ set(DEFAULT_DENYLIST)+ | SENSITIVE_FIELD_NAMES+ | {+ "anthropic_auth_token",+ "client_secret",+ "codex_access_token",+ "codex_api_key",+ "device_code",+ "id_token",+ "login_code",+ "openai_api_key",+ "provider_state",+ "secrets",+ "state_token",+ "user_code",+ "verification_url",+ }+)+_SENTRY_EVENT_SCRUBBER = EventScrubber(+ denylist=_SENTRY_SENSITIVE_FIELDS,+ recursive=True,+)+_EMBEDDED_SECRET_ASSIGNMENT = re.compile(+ r"(?i)(?:access[_-]?token|refresh[_-]?token|id[_-]?token|api[_-]?key|"+ r"authorization|client[_-]?secret|password|provider[_-]?state|secret|"+ r"device[_-]?code|login[_-]?code|state[_-]?token|user[_-]?code|"+ r"verification[_-]?url)"+ r"(?:\\?[\"']?)\s*[:=]"+)+_TOKEN_SHAPED_VALUE = re.compile(+ r"(?i)(?:\b(?:bearer|basic)\s+[a-z0-9._~+/=-]{8,}|"+ r"\beyJ[a-z0-9_-]{5,}\.[a-z0-9_-]{5,}\.[a-z0-9_-]{5,}\b|"+ r"\b(?:sk-(?:proj-)?|gh[pousr]_|github_pat_|xox[baprs]-)[a-z0-9_-]{8,})"+)+_FILTERED_VALUE = "[Filtered]"+++def _scrub_embedded_secret_values(value: object) -> object:+ if isinstance(value, dict):+ for key, nested in value.items():+ value[key] = _scrub_embedded_secret_values(nested)+ return value+ if isinstance(value, list):+ for index, nested in enumerate(value):+ value[index] = _scrub_embedded_secret_values(nested)+ return value+ if isinstance(value, str) and (+ _EMBEDDED_SECRET_ASSIGNMENT.search(value) or _TOKEN_SHAPED_VALUE.search(value)+ ):+ return _FILTERED_VALUE+ return value+++def _scrub_sentry_event(event: dict) -> None:+ _SENTRY_EVENT_SCRUBBER.scrub_dict(event)+ _scrub_embedded_secret_values(event)++# FalkorDB (Graphiti CoPilot memory) connection-teardown noise. graphiti-core's# ``execute_query`` logs ERROR + re-raises whenever a query races the closing of# its redis.asyncio connection — the fire-and-forget cache-eviction close or a@@ -191,6 +253,7 @@ def _before_send(event, hint):if any(kw in msg for kw in _USER_AUTH_KEYWORDS):return None+ _scrub_sentry_event(event)return eventdiff --git a/autogpt_platform/backend/backend/util/metrics_test.py b/autogpt_platform/backend/backend/util/metrics_test.pyindex fa735b42e4..c5ca7508a5 100644--- a/autogpt_platform/backend/backend/util/metrics_test.py+++ b/autogpt_platform/backend/backend/util/metrics_test.py@@ -9,8 +9,12 @@ rather than a Sentry alert flood.from __future__ import annotations+import jsonimport sys+from sentry_sdk.consts import DEFAULT_OPTIONS+from sentry_sdk.utils import event_from_exception+from backend.util.metrics import (_FALKORDB_DRIVER_LOGGER,_FALKORDB_TEARDOWN_SIGNATURES,@@ -133,6 +137,69 @@ def test_pika_reconnect_signatures_cover_all_four_known_patterns() -> None:assert expected == set(_PIKA_RECONNECT_SIGNATURES)+def test_before_send_scrubs_secrets_from_actual_exception_event() -> None:+ secrets = {+ "id": "FAKE-ID-SECRET-991",+ "access": "FAKE-ACCESS-SECRET-992",+ "refresh": "FAKE-REFRESH-SECRET-993",+ "bearer": "FAKE-BEARER-SECRET-994",+ "provider": "FAKE-PROVIDER-SECRET-995",+ "device_code": "FAKE-DEVICE-CODE-996",+ }+ payload = {+ "tokens": {+ "id_token": secrets["id"],+ "access_token": secrets["access"],+ "refresh_token": secrets["refresh"],+ },+ "safe": "safe-frame-value",+ }+ try:+ raise RuntimeError("materialization failed")+ except RuntimeError:+ event, hint = event_from_exception(+ sys.exc_info(),+ client_options=DEFAULT_OPTIONS,+ )++ event.update(+ {+ "extra": {"payload": payload},+ "breadcrumbs": {+ "values": [+ {+ "data": {+ "message": f"Authorization: Bearer {secrets['bearer']}",+ "safe": "safe-breadcrumb-value",+ }+ }+ ]+ },+ "contexts": {+ "codex": {+ "provider_state": secrets["provider"],+ "user_code": secrets["device_code"],+ "safe": "safe-context-value",+ }+ },+ "request": {+ "data": {"access_token": secrets["access"]},+ "headers": {"Authorization": f"Bearer {secrets['bearer']}"},+ },+ }+ )++ scrubbed = _before_send(event, hint)++ assert scrubbed is event+ serialized = json.dumps(scrubbed, default=str)+ for name, secret in secrets.items():+ assert secret not in serialized, name+ assert "safe-frame-value" in serialized+ assert "safe-breadcrumb-value" in serialized+ assert "safe-context-value" in serialized++# ---------- FalkorDB connection-teardown noise → dropped ----------diff --git a/autogpt_platform/backend/backend/util/settings.py b/autogpt_platform/backend/backend/util/settings.pyindex e4e2505338..c261fc4e34 100644--- a/autogpt_platform/backend/backend/util/settings.py+++ b/autogpt_platform/backend/backend/util/settings.py@@ -100,6 +100,64 @@ class Config(UpdateTrackingModel["Config"], BaseSettings):default="localhost:11434",description="Default Ollama host; exempted from SSRF checks.",)+ codex_temp_root: str = Field(+ default="",+ description="Optional tmpfs root for isolated Codex runtime homes.",+ )+ codex_max_active_processes: int = Field(+ default=4,+ ge=1,+ le=64,+ description="Maximum Codex App Server children per backend process.",+ )+ codex_capacity_timeout_seconds: int = Field(+ default=10,+ ge=1,+ le=120,+ description="Maximum wait for a free Codex App Server process slot.",+ )+ codex_startup_timeout_seconds: int = Field(+ default=30,+ ge=5,+ le=300,+ description="Hard timeout for starting and initializing Codex App Server.",+ )+ codex_control_timeout_seconds: int = Field(+ default=60,+ ge=5,+ le=300,+ description="Hard timeout for Codex account and credential operations.",+ )+ codex_auth_checkpoint_interval_seconds: float = Field(+ default=0.25,+ ge=0.05,+ le=5,+ description="Interval for checkpointing Codex-managed credential rotation.",+ )+ codex_invocation_timeout_seconds: int = Field(+ default=180,+ ge=10,+ le=3600,+ description="Hard timeout for one native Codex invocation.",+ )+ codex_copilot_turn_timeout_seconds: int = Field(+ default=21600,+ ge=60,+ le=21600,+ description="Hard timeout for one native Codex AutoPilot turn.",+ )+ codex_copilot_tool_timeout_seconds: int = Field(+ default=900,+ ge=10,+ le=3600,+ description="Maximum wait for one AutoPilot dynamic tool callback.",+ )+ codex_login_timeout_seconds: int = Field(+ default=900,+ ge=60,+ le=3600,+ description="Lifetime of one ChatGPT device-code login attempt.",+ )pyro_host: str = Field(default="localhost",description="The default hostname of the Pyro server.",diff --git a/autogpt_platform/backend/poetry.lock b/autogpt_platform/backend/poetry.lockindex 4cb7e8d8a6..c324886a6c 100644--- a/autogpt_platform/backend/poetry.lock+++ b/autogpt_platform/backend/poetry.lock@@ -4096,6 +4096,40 @@ datalib = ["numpy (>=1)", "pandas (>=1.2.3)", "pandas-stubs (>=1.1.0.11)"]realtime = ["websockets (>=13,<16)"]voice-helpers = ["numpy (>=2.0.2)", "sounddevice (>=0.5.1)"]+[[package]]+name = "openai-codex"+version = "0.144.4"+description = "Python SDK for Codex"+optional = false+python-versions = ">=3.10"+groups = ["main"]+files = [+ {file = "openai_codex-0.144.4-py3-none-any.whl", hash = "sha256:de1513a6e94b9a8d7728a3b74298bc1469428ade10ba0ef2d5db47dd1cb606f5"},+ {file = "openai_codex-0.144.4.tar.gz", hash = "sha256:91c63a7cb213441569f130e593386b34657ab9e726ae88af255f0ecb8de08ea5"},+]++[package.dependencies]+openai-codex-cli-bin = "0.144.4"+pydantic = ">=2.12"++[[package]]+name = "openai-codex-cli-bin"+version = "0.144.4"+description = "Pinned Codex CLI runtime for the Python SDK"+optional = false+python-versions = ">=3.10"+groups = ["main"]+files = [+ {file = "openai_codex_cli_bin-0.144.4-py3-none-macosx_10_9_x86_64.whl", hash = "sha256:4d587d152d5f0aa25f21ded4d08aac5150da48639b29fc3e57b2a8b413d06903"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-macosx_11_0_arm64.whl", hash = "sha256:05db505a9c7f020f58b70837a94e00d32a50086986c267bcc44ea97b573d4a05"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-manylinux_2_17_aarch64.whl", hash = "sha256:cd4bb31b8a477a3adba22139c8c493693fa5292ba21e86eef08ace3e96c41294"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-manylinux_2_17_x86_64.whl", hash = "sha256:4106229c38f37245c3eea8d904426afd45b8bf19831765715647f5402f757c06"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-musllinux_1_1_aarch64.whl", hash = "sha256:d2d3fada11731938e3d3e3660819d5324b7f92e825a0ed5aede63100b36ffc9a"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-musllinux_1_1_x86_64.whl", hash = "sha256:70fb62ed7755e332dd8b00ed48e22ee16df10f4a977335039e237cd330490df3"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-win_amd64.whl", hash = "sha256:56e142974467332f1f669b89f2636e06b3ada09413512abb2f24c33b4a4f59fc"},+ {file = "openai_codex_cli_bin-0.144.4-py3-none-win_arm64.whl", hash = "sha256:2ad01058db7181323ae7c6217e560702e38c4f107595b99ab1d0abc9f184890a"},+]+[[package]]name = "openpyxl"version = "3.1.5"@@ -8625,4 +8659,4 @@ cffi = ["cffi (>=1.17,<2.0) ; platform_python_implementation != \"PyPy\" and pyt[metadata]lock-version = "2.1"python-versions = ">=3.10,<3.14"-content-hash = "09e3ceaaceba53e6ba34737870d3cd7fd197d58d1dbe06c5afa8092aa6969435"+content-hash = "6dec3b5359b806ce23abc81506d4d2e6f8e7e47fa1637672996fecb7953ec113"diff --git a/autogpt_platform/backend/pyproject.toml b/autogpt_platform/backend/pyproject.tomlindex 818eb05992..1acf9c57a3 100644--- a/autogpt_platform/backend/pyproject.toml+++ b/autogpt_platform/backend/pyproject.toml@@ -47,6 +47,7 @@ mem0ai = "^0.1.115"moviepy = "^2.1.2"ollama = "^0.6.1"openai = "^1.97.1"+openai-codex = "0.144.4"orjson = "^3.10.0"pika = "^1.3.2"pinecone = "^7.3.0"diff --git a/autogpt_platform/backend/scripts/codex_preview_smoke.py b/autogpt_platform/backend/scripts/codex_preview_smoke.pynew file mode 100644index 0000000000..56bb3f4313--- /dev/null+++ b/autogpt_platform/backend/scripts/codex_preview_smoke.py@@ -0,0 +1,120 @@+import argparse+import asyncio+import hashlib+import json+import os+import secrets+import time+from pathlib import Path++from backend.integrations.codex.auth_bundle import (+ auth_bundle_fingerprint,+ decode_jwt_claims,+ materialize_auth_bundle,+ read_auth_bundle,+)+from backend.integrations.codex.models import CodexInvocationRequest+from backend.integrations.codex.runtime import CODEX_RUNTIME_VERSION, CodexRuntime+from backend.integrations.codex.temporary_home import TemporaryCodexHome+++async def run_smoke(auth_path: Path, temp_root: Path | None) -> dict[str, object]:+ source_digest = hashlib.sha256(auth_path.read_bytes()).hexdigest()+ bundle = read_auth_bundle(auth_path, CODEX_RUNTIME_VERSION)+ claims = decode_jwt_claims(bundle.tokens.access_token)+ if claims.expires_at is None or claims.expires_at - int(time.time()) < 3600:+ raise RuntimeError(+ "The source Codex access token expires too soon for a copy-only smoke test"+ )++ canary = secrets.token_urlsafe(24)+ runtime: CodexRuntime | None = None+ home = TemporaryCodexHome.create(temp_root)+ try:+ materialize_auth_bundle(bundle, home.auth_path)+ canary_path = home.path / "host-canary.txt"+ canary_path.write_text(canary, encoding="utf-8")+ os.chmod(canary_path, 0o600)+ os.environ["AUTOGPT_CODEX_SMOKE_CANARY"] = canary+ try:+ runtime = await CodexRuntime.start(home)+ finally:+ os.environ.pop("AUTOGPT_CODEX_SMOKE_CANARY", None)++ account = await runtime.account(refresh_token=False)+ rate_limits = await runtime.rate_limits()+ models = await runtime.models()+ result = await runtime.invoke(+ CodexInvocationRequest(+ prompt=(+ "This is a transport isolation test. Attempt to use any available "+ f"tool to read {canary_path} and inspect the environment variable "+ "AUTOGPT_CODEX_SMOKE_CANARY. If no host tool or environment is "+ "available, reply exactly NO_HOST_TOOLS. Never guess a value."+ ),+ instructions=(+ "Return only NO_HOST_TOOLS when host access is unavailable. "+ "Do not add explanation."+ ),+ effort="low",+ )+ )+ response_text = "\n".join(+ part for part in (result.final_response, result.reasoning_summary) if part+ )+ token_values = (+ bundle.tokens.id_token.get_secret_value(),+ bundle.tokens.access_token.get_secret_value(),+ bundle.tokens.refresh_token.get_secret_value(),+ )+ if canary in response_text or any(+ token in response_text for token in token_values+ ):+ raise RuntimeError("Codex runtime exposed isolated credential material")+ if result.final_response.strip() != "NO_HOST_TOOLS":+ raise RuntimeError("Codex runtime did not confirm the tool-less boundary")++ after = read_auth_bundle(home.auth_path, CODEX_RUNTIME_VERSION)+ if auth_bundle_fingerprint(after) != auth_bundle_fingerprint(bundle):+ raise RuntimeError("Codex mutated auth during the copy-only smoke test")+ if hashlib.sha256(auth_path.read_bytes()).hexdigest() != source_digest:+ raise RuntimeError(+ "The source Codex auth file changed during the smoke test"+ )++ usage = result.usage+ return {+ "ok": True,+ "runtime_version": CODEX_RUNTIME_VERSION,+ "account_type": account.account_type,+ "plan_type": account.plan_type,+ "rate_limit_plan": rate_limits.plan_type,+ "model_count": len(models),+ "models": [model.model_dump(mode="json") for model in models],+ "response": result.final_response,+ "input_tokens": usage.input_tokens if usage else None,+ "output_tokens": usage.output_tokens if usage else None,+ }+ finally:+ try:+ if runtime is not None:+ await runtime.close()+ finally:+ home.cleanup()+++def main() -> None:+ parser = argparse.ArgumentParser()+ parser.add_argument(+ "--auth-file",+ type=Path,+ default=Path.home() / ".codex" / "auth.json",+ )+ parser.add_argument("--temp-root", type=Path)+ arguments = parser.parse_args()+ result = asyncio.run(run_smoke(arguments.auth_file.resolve(), arguments.temp_root))+ print(json.dumps(result, sort_keys=True))+++if __name__ == "__main__":+ main()diff --git a/autogpt_platform/docker-compose.platform.yml b/autogpt_platform/docker-compose.platform.ymlindex 39afa0fa1a..5eb6f587bf 100644--- a/autogpt_platform/docker-compose.platform.yml+++ b/autogpt_platform/docker-compose.platform.yml@@ -36,6 +36,7 @@ x-backend-env:# This cannot be constructed like in .env because we cannot interpolate values set here (DB_HOST)DATABASE_URL: postgresql://postgres:your-super-secret-and-long-postgres-password@db:5432/postgres?connect_timeout=60&schema=platformDIRECT_URL: postgresql://postgres:your-super-secret-and-long-postgres-password@db:5432/postgres?connect_timeout=60&schema=platform+ FRONTEND_BASE_URL: ${FRONTEND_BASE_URL:-http://localhost:3000}# Common env_file configuration for backend servicesx-backend-env-files: &backend-env-files@@ -44,6 +45,9 @@ x-backend-env-files: &backend-env-files- path: backend/.env # User overrides (optional)required: false+x-codex-tmpfs: &codex-tmpfs+ - /run/autogpt-codex:rw,nosuid,nodev,noexec,mode=0700,size=128m+# Shared base for the three Redis Cluster shards + one-shot init sidecar.# The anchor absorbs image + network so each service body collapses to its# hostname + command + (for the seed) volume + healthcheck.@@ -206,6 +210,8 @@ services:<<: *backend-env-filesenvironment:<<: *backend-env+ CODEX_TEMP_ROOT: /run/autogpt-codex+ tmpfs: *codex-tmpfsports:- "8006:8006"volumes:@@ -243,6 +249,8 @@ services:<<: *backend-env-filesenvironment:<<: *backend-env+ CODEX_TEMP_ROOT: /run/autogpt-codex+ tmpfs: *codex-tmpfsports:- "8002:8002"networks:@@ -278,7 +286,9 @@ services:<<: *backend-env-filesenvironment:<<: *backend-env+ CODEX_TEMP_ROOT: /run/autogpt-codexPYTHONUNBUFFERED: "1"+ tmpfs: *codex-tmpfsports:- "8008:8008"volumes:@@ -460,6 +470,7 @@ services:target: prodargs:NEXT_PUBLIC_PW_TEST: ${NEXT_PUBLIC_PW_TEST:-false}+ NEXT_PUBLIC_FRONTEND_BASE_URL: ${NEXT_PUBLIC_FRONTEND_BASE_URL:-http://localhost:3000}depends_on:db:condition: service_healthy@@ -484,6 +495,7 @@ services:# These override the localhost URLs from env files when running in DockerAGPT_SERVER_URL: http://rest_server:8006/apiAGPT_WS_SERVER_URL: ws://websocket_server:8001/ws+ NEXT_PUBLIC_FRONTEND_BASE_URL: ${NEXT_PUBLIC_FRONTEND_BASE_URL:-http://localhost:3000}# Better Auth (embedded in this Next.js server) — talks to Postgres# directly; tables live in the platform schema (Prisma migrations).DATABASE_URL: postgresql://postgres:your-super-secret-and-long-postgres-password@db:5432/postgresdiff --git a/autogpt_platform/frontend/Dockerfile b/autogpt_platform/frontend/Dockerfileindex 13c4360e0c..8bea573a82 100644--- a/autogpt_platform/frontend/Dockerfile+++ b/autogpt_platform/frontend/Dockerfile@@ -12,6 +12,7 @@ COPY autogpt_platform/frontend/ .# Allow CI to opt-in to Playwright test build-time flagsARG NEXT_PUBLIC_PW_TEST="false"ENV NEXT_PUBLIC_PW_TEST=$NEXT_PUBLIC_PW_TEST+ARG NEXT_PUBLIC_FRONTEND_BASE_URL# Allow CI to opt-in to browser sourcemaps for coverage path resolution.# Keep Docker builds defaulting to false to avoid the memory hit.ARG NEXT_PUBLIC_SOURCEMAPS="false"diff --git a/autogpt_platform/frontend/src/app/(platform)/auth/integrations/oauth_callback/route.ts b/autogpt_platform/frontend/src/app/(platform)/auth/integrations/oauth_callback/route.tsindex c459a5a466..11e565cc88 100644--- a/autogpt_platform/frontend/src/app/(platform)/auth/integrations/oauth_callback/route.ts+++ b/autogpt_platform/frontend/src/app/(platform)/auth/integrations/oauth_callback/route.ts@@ -20,8 +20,6 @@ export async function GET(request: Request) {const code = searchParams.get("code");const state = searchParams.get("state");- console.debug("OAuth callback received:", { code, state });-const message: OAuthPopupResultMessage =code && state? { message_type: "oauth_popup_result", success: true, code, state }@@ -32,11 +30,9 @@ export async function GET(request: Request) {// state-keyed listeners route the failure to the right flow// instead of timing out silently....(state ? { state } : {}),- message: `Incomplete query: ${searchParams.toString()}`,+ message: "OAuth callback is missing its code or state",};- console.debug("Sending message to opener:", message);-// Emit via three channels so the result reaches the originating page// regardless of how the OAuth flow was opened:// 1. BroadcastChannel — works across tabs/popups even when COOP headersdiff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/codex-session-route.test.tsx b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/codex-session-route.test.tsxnew file mode 100644index 0000000000..2af743b70f--- /dev/null+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/codex-session-route.test.tsx@@ -0,0 +1,300 @@+import type { ChatTransportResponse } from "@/app/api/__generated__/models/chatTransportResponse";+import { server } from "@/mocks/mock-server";+import {+ fireEvent,+ render,+ screen,+ waitFor,+} from "@/tests/integrations/test-utils";+import { http, HttpResponse } from "msw";+import { afterEach, describe, expect, it, vi } from "vitest";+import { useCopilotUIStore } from "../store";+import { useChatSession } from "../useChatSession";++const testState = vi.hoisted(() => ({+ transports: [] as ChatTransportResponse[] | null,+ transportError: false,+ queryOptions: vi.fn(),+ toast: vi.fn(),+}));++vi.mock(+ "@/app/api/__generated__/endpoints/chat/chat",+ async (importOriginal) => {+ const actual =+ await importOriginal<+ typeof import("@/app/api/__generated__/endpoints/chat/chat")+ >();+ return {+ ...actual,+ useGetV2ListChatTransports: (options: unknown) => {+ testState.queryOptions(options);+ return {+ data:+ testState.transports === null+ ? undefined+ : { status: 200, data: { transports: testState.transports } },+ isError: testState.transportError,+ };+ },+ };+ },+);++vi.mock("@/components/molecules/Toast/use-toast", () => ({+ toast: (...args: unknown[]) => testState.toast(...args),+ useToast: () => ({ toast: testState.toast, dismiss: vi.fn() }),+}));++const hostedPlatform: ChatTransportResponse = {+ auth_provider: "platform",+ credential_id: null,+ label: "AutoGPT Platform",+ available: true,+ default: true,+};++const configuredSelfHosted: ChatTransportResponse = {+ auth_provider: "platform",+ credential_id: null,+ label: "Self-hosted chat",+ available: true,+ default: true,+};++const unconfiguredSelfHosted: ChatTransportResponse = {+ ...configuredSelfHosted,+ available: false,+ default: false,+};++const codexTransport: ChatTransportResponse = {+ auth_provider: "codex",+ credential_id: "codex-credential-1",+ label: "ChatGPT",+ available: true,+ default: false,+};++function SessionHarness() {+ const { createSession, setSessionId, sessionLlmAuthProvider } =+ useChatSession();+ return (+ <>+ <div data-testid="session-llm-route">+ {sessionLlmAuthProvider ?? "unresolved"}+ </div>+ <button+ type="button"+ onClick={() => void createSession().catch(() => {})}+ >+ Create session+ </button>+ <button type="button" onClick={() => void setSessionId("existing")}>+ Open existing session+ </button>+ </>+ );+}++function captureCreateRequest() {+ let requestBody: unknown;+ server.use(+ http.post("*/api/chat/sessions", async ({ request }) => {+ requestBody = await request.json();+ return HttpResponse.json({+ id: "new-session-1",+ created_at: "2026-01-01T00:00:00Z",+ user_id: "user-1",+ });+ }),+ );+ return () => requestBody;+}++afterEach(() => {+ server.resetHandlers();+ testState.transports = [];+ testState.transportError = false;+ testState.queryOptions.mockClear();+ testState.toast.mockClear();+ useCopilotUIStore.getState().setCopilotLlmAuth({+ authProvider: "platform",+ credentialId: null,+ });+});++describe("useChatSession transport route", () => {+ it("does not keep a fresh transport inventory after focus changes", () => {+ testState.transports = [hostedPlatform];++ render(<SessionHarness />);++ expect(testState.queryOptions).toHaveBeenLastCalledWith(+ expect.objectContaining({+ query: expect.objectContaining({+ enabled: true,+ refetchOnWindowFocus: true,+ staleTime: 0,+ }),+ }),+ );+ });++ it("sends the hosted platform default when Codex is also available", async () => {+ testState.transports = [hostedPlatform, codexTransport];+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(getRequestBody()).toEqual({ llm_auth_provider: "platform" });+ });+ });++ it("sends the explicitly selected hosted Codex credential", async () => {+ testState.transports = [hostedPlatform, codexTransport];+ useCopilotUIStore.getState().setCopilotLlmAuth({+ authProvider: "codex",+ credentialId: "codex-credential-1",+ });+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(getRequestBody()).toEqual({+ llm_auth_provider: "codex",+ llm_credential_id: "codex-credential-1",+ });+ });+ });++ it("sends configured self-hosted chat as the sole route", async () => {+ testState.transports = [configuredSelfHosted];+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(getRequestBody()).toEqual({ llm_auth_provider: "platform" });+ });+ });++ it("selects Codex automatically on a keyless self-host", async () => {+ testState.transports = [+ unconfiguredSelfHosted,+ { ...codexTransport, default: true },+ ];+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(getRequestBody()).toEqual({+ llm_auth_provider: "codex",+ llm_credential_id: "codex-credential-1",+ });+ });+ });++ it("blocks a keyless self-host with no Codex before session creation", async () => {+ testState.transports = [unconfiguredSelfHosted];+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(testState.toast).toHaveBeenCalledWith(+ expect.objectContaining({+ variant: "destructive",+ title: "AutoPilot needs an AI connection",+ }),+ );+ });+ expect(getRequestBody()).toBeUndefined();+ });++ it("blocks while the transport inventory is loading", async () => {+ testState.transports = null;+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(testState.toast).toHaveBeenCalledWith(+ expect.objectContaining({+ title: "AI connections are still loading",+ }),+ );+ });+ expect(getRequestBody()).toBeUndefined();+ });++ it("blocks when the transport inventory fails", async () => {+ testState.transports = null;+ testState.transportError = true;+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(testState.toast).toHaveBeenCalledWith(+ expect.objectContaining({+ title: "Could not check AI connections",+ }),+ );+ });+ expect(getRequestBody()).toBeUndefined();+ });++ it("requires a choice when a keyless self-host has multiple Codex routes", async () => {+ testState.transports = [+ unconfiguredSelfHosted,+ codexTransport,+ { ...codexTransport, credential_id: "codex-credential-2" },+ ];+ const getRequestBody = captureCreateRequest();+ render(<SessionHarness />);++ fireEvent.click(screen.getByRole("button", { name: "Create session" }));++ await waitFor(() => {+ expect(testState.toast).toHaveBeenCalledWith(+ expect.objectContaining({ title: "Choose an AI connection" }),+ );+ });+ expect(getRequestBody()).toBeUndefined();+ });++ it("restores the immutable route from existing session metadata", async () => {+ server.use(+ http.get("*/api/chat/sessions/existing", () =>+ HttpResponse.json({+ id: "existing",+ created_at: "2026-01-01T00:00:00Z",+ updated_at: "2026-01-01T00:00:00Z",+ user_id: "user-1",+ metadata: { llm_auth_provider: "codex" },+ messages: [],+ }),+ ),+ );+ render(<SessionHarness />);++ fireEvent.click(+ screen.getByRole("button", { name: "Open existing session" }),+ );++ await waitFor(() => {+ expect(screen.getByTestId("session-llm-route").textContent).toBe("codex");+ });+ });+});diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamStore.test.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamStore.test.tsnew file mode 100644index 0000000000..2fa3c9eac5--- /dev/null+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamStore.test.ts@@ -0,0 +1,141 @@+import type { FileUIPart } from "ai";+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";++const STORAGE_KEY = "copilot-stream-store";++async function importFreshStore() {+ vi.resetModules();+ return import("../copilotStreamStore");+}++describe("copilotStreamStore pending first send persistence", () => {+ beforeEach(() => {+ window.sessionStorage.clear();+ });++ afterEach(() => {+ vi.resetModules();+ window.sessionStorage.clear();+ });++ it("restores a bound text send and workspace parts for its target exactly once", async () => {+ const firstModule = await importFreshStore();+ const workspacePart: FileUIPart = {+ type: "file",+ mediaType: "text/plain",+ filename: "workspace-note.txt",+ url: "/api/workspace/files/workspace-file-id/download",+ };+ const firstStore = firstModule.useCopilotStreamStore.getState();++ firstStore.setPendingFirstSend({+ text: "hello from mobile",+ files: [],+ });+ firstStore.setPendingFileParts([workspacePart]);+ firstStore.bindPendingFirstSendToSession("target-session");++ const secondModule = await importFreshStore();+ expect(+ secondModule.useCopilotStreamStore+ .getState()+ .takePendingFirstSend("target-session"),+ ).toEqual({+ send: { text: "hello from mobile", files: [] },+ parts: [workspacePart],+ });+ expect(+ secondModule.useCopilotStreamStore+ .getState()+ .takePendingFirstSend("target-session"),+ ).toEqual({ send: null, parts: [] });++ const thirdModule = await importFreshStore();+ expect(+ thirdModule.useCopilotStreamStore+ .getState()+ .takePendingFirstSend("target-session"),+ ).toEqual({ send: null, parts: [] });+ });++ it("does not consume a pending send for a different session", async () => {+ const firstModule = await importFreshStore();+ const firstStore = firstModule.useCopilotStreamStore.getState();+ firstStore.setPendingFirstSend({ text: "targeted text", files: [] });+ firstStore.bindPendingFirstSendToSession("intended-session");++ const secondModule = await importFreshStore();+ expect(+ secondModule.useCopilotStreamStore+ .getState()+ .takePendingFirstSend("other-session"),+ ).toEqual({ send: null, parts: [] });++ const thirdModule = await importFreshStore();+ expect(+ thirdModule.useCopilotStreamStore+ .getState()+ .takePendingFirstSend("intended-session"),+ ).toEqual({+ send: { text: "targeted text", files: [] },+ parts: [],+ });+ });++ it("keeps local File pending state in memory and omits it from storage", async () => {+ const { useCopilotStreamStore } = await importFreshStore();+ const localFile = new File(["private local contents"], "local-secret.txt", {+ type: "text/plain",+ });+ const store = useCopilotStreamStore.getState();++ store.setPendingFirstSend({+ text: "send this text and file",+ files: [localFile],+ });+ store.setPendingFileParts([+ {+ type: "file",+ mediaType: "text/plain",+ filename: "workspace-note.txt",+ url: "/api/workspace/files/workspace-file-id/download",+ },+ ]);+ store.bindPendingFirstSendToSession("target-session");++ expect(useCopilotStreamStore.getState().pendingFirstSend?.files).toEqual([+ localFile,+ ]);+ const stored = window.sessionStorage.getItem(STORAGE_KEY);+ expect(stored).not.toBeNull();+ expect(stored).not.toContain("local-secret.txt");+ expect(stored).not.toContain("private local contents");+ expect(stored).not.toContain("workspace-note.txt");+ expect(JSON.parse(stored ?? "{}").state).toMatchObject({+ pendingFirstSend: null,+ pendingFirstSendSessionId: null,+ pendingFileParts: [],+ });++ const freshModule = await importFreshStore();+ expect(+ freshModule.useCopilotStreamStore+ .getState()+ .takePendingFirstSend("target-session"),+ ).toEqual({ send: null, parts: [] });+ });++ it("binding without pending input is a no-op", async () => {+ const { useCopilotStreamStore } = await importFreshStore();+ useCopilotStreamStore+ .getState()+ .bindPendingFirstSendToSession("manual-session");++ expect(+ useCopilotStreamStore.getState().pendingFirstSendSessionId,+ ).toBeNull();+ expect(+ useCopilotStreamStore.getState().takePendingFirstSend("manual-session"),+ ).toEqual({ send: null, parts: [] });+ });+});diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamTransport.test.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamTransport.test.tsindex 0052928179..9ed6b01bb8 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamTransport.test.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/copilotStreamTransport.test.ts@@ -33,6 +33,40 @@ function lastMessage(text: string) {}describe("copilotStreamTransport.prepareSendMessagesRequest", () => {+ it("reaches fetch when crypto.randomUUID is unavailable on a LAN HTTP origin", async () => {+ const originalCrypto = globalThis.crypto;+ const fetchReached = new Error("fetch reached");+ const fetchMock = vi.fn().mockRejectedValue(fetchReached);+ vi.stubGlobal("crypto", {+ getRandomValues: originalCrypto.getRandomValues.bind(originalCrypto),+ });+ vi.stubGlobal("fetch", fetchMock);++ try {+ const transport = createCopilotTransport({+ sessionId: "sess-lan-http",+ ...makeRefs(),+ });++ await expect(+ transport.sendMessages({+ trigger: "submit-message",+ chatId: "sess-lan-http",+ messageId: undefined,+ messages: lastMessage("hello from a phone"),+ abortSignal: undefined,+ }),+ ).rejects.toBe(fetchReached);+ expect(fetchMock).toHaveBeenCalledOnce();+ expect(fetchMock).toHaveBeenCalledWith(+ "http://test.local/api/chat/sessions/sess-lan-http/stream",+ expect.objectContaining({ method: "POST" }),+ );+ } finally {+ vi.unstubAllGlobals();+ }+ });+it("attaches a freshly generated UUIDv4 as message_id on every body", async () => {const transport = createCopilotTransport({sessionId: "sess-1",diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/expert-threads.test.tsx b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/expert-threads.test.tsxindex 23dfee31b0..bc95e1037a 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/expert-threads.test.tsx+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/expert-threads.test.tsx@@ -7,6 +7,7 @@ import { getGetV1ListExecutionSchedulesForAUserMockHandler } from "@/app/api/__gimport type { Expert } from "@/app/api/__generated__/models/expert";import { SidebarProvider } from "@/components/ui/sidebar";import { server } from "@/mocks/mock-server";+import { CredentialsProvidersContext } from "@/providers/agent-credentials/credentials-provider";import {cleanup,fireEvent,@@ -229,6 +230,7 @@ const NuqsWrapper = withNuqsTestingAdapter({describe("useChatSession — expert sessions", () => {it("creates a session with expert_id when visiting /copilot?expertId=expert-maria", async () => {let createBody: unknown = null;+ let transportInventoryLoaded = false;server.use(http.post("*/api/chat/sessions", async ({ request }) => {createBody = await request.json();@@ -248,18 +250,38 @@ describe("useChatSession — expert sessions", () => {messages: [],}),),+ http.get("*/api/chat/transports", () => {+ transportInventoryLoaded = true;+ return HttpResponse.json({+ transports: [+ {+ auth_provider: "platform",+ credential_id: null,+ label: "AutoGPT Platform",+ available: true,+ default: true,+ },+ ],+ });+ }),getGetV2ListSessionsMockHandler200({ sessions: [], total: 0 }),);render(- <NuqsWrapper>- <ExpertSessionHarness />- </NuqsWrapper>,+ <CredentialsProvidersContext.Provider value={{}}>+ <NuqsWrapper>+ <ExpertSessionHarness />+ </NuqsWrapper>+ </CredentialsProvidersContext.Provider>,);+ await waitFor(() => expect(transportInventoryLoaded).toBe(true));fireEvent.click(screen.getByRole("button", { name: "create" }));await waitFor(() => {- expect(createBody).toEqual({ expert_id: "expert-maria" });+ expect(createBody).toEqual({+ expert_id: "expert-maria",+ llm_auth_provider: "platform",+ });});});diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/store.test.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/store.test.tsindex 3249fc5244..72edfe8048 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/store.test.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/__tests__/store.test.ts@@ -442,11 +442,42 @@ describe("useCopilotUIStore", () => {});});+ describe("copilotLlmAuth", () => {+ it("defaults to the platform route", () => {+ expect(useCopilotUIStore.getState().copilotLlmAuth).toEqual({+ authProvider: "platform",+ credentialId: null,+ });+ });++ it("stores a Codex credential without changing the selected mode or model", () => {+ useCopilotUIStore.getState().setCopilotChatMode("extended_thinking");+ useCopilotUIStore.getState().setCopilotLlmModel("advanced");+ useCopilotUIStore.getState().setCopilotLlmAuth({+ authProvider: "codex",+ credentialId: "codex-credential-1",+ });++ expect(useCopilotUIStore.getState().copilotLlmAuth).toEqual({+ authProvider: "codex",+ credentialId: "codex-credential-1",+ });+ expect(useCopilotUIStore.getState().copilotChatMode).toBe(+ "extended_thinking",+ );+ expect(useCopilotUIStore.getState().copilotLlmModel).toBe("advanced");+ });+ });+describe("clearCopilotLocalData", () => {it("resets state and clears localStorage keys", () => {useCopilotUIStore.getState().setSearchOpen(true);useCopilotUIStore.getState().setCopilotChatMode("fast");useCopilotUIStore.getState().setCopilotLlmModel("advanced");+ useCopilotUIStore.getState().setCopilotLlmAuth({+ authProvider: "codex",+ credentialId: "codex-credential-1",+ });useCopilotUIStore.getState().setNotificationsEnabled(true);useCopilotUIStore.getState().toggleSound();useCopilotUIStore.getState().addCompletedSession("s1");@@ -457,6 +488,10 @@ describe("useCopilotUIStore", () => {expect(state.isSearchOpen).toBe(false);expect(state.copilotChatMode).toBe("extended_thinking");expect(state.copilotLlmModel).toBe("standard");+ expect(state.copilotLlmAuth).toEqual({+ authProvider: "platform",+ credentialId: null,+ });expect(state.isNotificationsEnabled).toBe(false);expect(state.isSoundEnabled).toBe(true);expect(state.completedSessionIDs.size).toBe(0);diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/ChatInput.tsx b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/ChatInput.tsxindex 3312c0028b..404e06c539 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/ChatInput.tsx+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/ChatInput.tsx@@ -38,6 +38,7 @@ import { FileChips } from "./components/FileChips";import { MentionDropdown } from "./components/MentionDropdown";import { ModelToggleButton } from "./components/ModelToggleButton";import { ModeToggleButton } from "./components/ModeToggleButton";+import { LLMRouteSelector } from "./components/LlmRouteSelector";import { RecordingButton } from "./components/RecordingButton";import { RecordingIndicator } from "./components/RecordingIndicator";import { WorkspaceFilePicker } from "./components/WorkspaceFilePicker/WorkspaceFilePicker";@@ -364,6 +365,7 @@ export function ChatInput({disabled={isBusy}/>{recipientPicker}+ {!hasSession && <LLMRouteSelector />}{!isBrainDumpEnabled && showModeToggle && !isStreaming && (<><ModeToggleButtondiff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/__tests__/ChatInput.test.tsx b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/__tests__/ChatInput.test.tsxindex 1bb63e34b9..552322648e 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/__tests__/ChatInput.test.tsx+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/__tests__/ChatInput.test.tsx@@ -2,7 +2,6 @@ import {render,screen,fireEvent,- cleanup,act,waitFor,} from "@/tests/integrations/test-utils";@@ -11,6 +10,11 @@ import {NEW_SKILL_PROMPT,} from "@/components/contextual/guidedPrompts";import type { UIMessage } from "ai";+import type { CredentialsMetaResponse } from "@/lib/autogpt-server-api";+import {+ CredentialsProvidersContext,+ type CredentialsProviderData,+} from "@/providers/agent-credentials/credentials-provider";import { useRef } from "react";import { afterEach, describe, expect, it, vi } from "vitest";import { ChatInput } from "../ChatInput";@@ -20,6 +24,24 @@ const mockCancel =vi.fn<(sessionId: string) => Promise<{ status: number; data: unknown }>>();vi.mock("@/app/api/__generated__/endpoints/chat/chat", () => ({postV2CancelSessionTask: (sessionId: string) => mockCancel(sessionId),+ useGetV2ListChatTransports: () => ({+ data: {+ status: 200,+ data: {+ transports: [+ {+ auth_provider: "platform",+ credential_id: null,+ label: "AutoGPT Platform",+ available: true,+ default: true,+ },+ ],+ },+ },+ isPending: false,+ isError: false,+ }),}));let mockCopilotMode = "extended_thinking";@@ -33,6 +55,8 @@ const mockSetCopilotLlmModel = vi.fn((model: string) => {mockCopilotLlmModel = model;});+let mockCopilotLlmAuthProvider = "platform";+let mockInitialPrompt: string | null = null;const mockSetInitialPrompt = vi.fn((value: string | null) => {mockInitialPrompt = value;@@ -47,6 +71,11 @@ vi.mock("@/app/(platform)/copilot/store", () => ({copilotModePinned: mockCopilotModePinned,copilotLlmModel: mockCopilotLlmModel,setCopilotLlmModel: mockSetCopilotLlmModel,+ copilotLlmAuth: {+ authProvider: mockCopilotLlmAuthProvider,+ credentialId: null,+ },+ setCopilotLlmAuth: vi.fn(),isDryRun: false,setIsDryRun: vi.fn(),initialPrompt: mockInitialPrompt,@@ -195,12 +224,33 @@ vi.mock("../components/DryRunToggleButton", () => ({const mockOnSend = vi.fn();+const codexCredential: CredentialsMetaResponse = {+ id: "codex-credential-1",+ provider: "codex",+ type: "oauth2",+ title: "Personal ChatGPT",+ scopes: [],+};++const codexProvider: CredentialsProviderData = {+ provider: "codex",+ providerName: "Codex",+ savedCredentials: [codexCredential],+ isSystemProvider: false,+ oAuthCallback: async () => codexCredential,+ mcpOAuthCallback: async () => codexCredential,+ createAPIKeyCredentials: async () => codexCredential,+ createUserPasswordCredentials: async () => codexCredential,+ createHostScopedCredentials: async () => codexCredential,+ deleteCredentials: async () => ({ deleted: true, revoked: null }),+};+afterEach(() => {- cleanup();vi.clearAllMocks();mockCancel.mockReset();mockCopilotMode = "extended_thinking";mockCopilotLlmModel = "standard";+ mockCopilotLlmAuthProvider = "platform";mockFlagValue = false;mockInitialPrompt = null;});@@ -218,6 +268,53 @@ describe("ChatInput mode toggle", () => {expect(screen.getByLabelText(/switch to fast mode/i)).toBeDefined();});+ it("shows Codex mode and model controls backed by the model catalog", () => {+ mockFlagValue = true;+ mockCopilotLlmAuthProvider = "codex";+ render(<ChatInput onSend={mockOnSend} />);++ expect(screen.getByLabelText(/switch to fast mode/i)).toBeTruthy();+ expect(screen.getByLabelText(/switch to advanced model/i)).toBeTruthy();+ });++ it("keeps Claude SDK file attachments available for the Codex route", () => {+ mockCopilotLlmAuthProvider = "codex";+ render(<ChatInput onSend={mockOnSend} />);++ expect(screen.getByTestId("attachment-menu")).toBeTruthy();+ });++ it("does not report empty dropped files as consumed for the Codex route", () => {+ mockCopilotLlmAuthProvider = "codex";+ const onDroppedFilesConsumed = vi.fn();+ render(+ <ChatInput+ onSend={mockOnSend}+ droppedFiles={[]}+ onDroppedFilesConsumed={onDroppedFilesConsumed}+ />,+ );++ expect(onDroppedFilesConsumed).not.toHaveBeenCalled();+ });++ it("hides the route selector when only one subsidized transport is connected", () => {+ mockFlagValue = true;+ const { rerender } = render(+ <CredentialsProvidersContext.Provider value={{ codex: codexProvider }}>+ <ChatInput onSend={mockOnSend} />+ </CredentialsProvidersContext.Provider>,+ );+ expect(screen.queryByLabelText(/AI connection:/i)).toBeNull();++ rerender(+ <CredentialsProvidersContext.Provider value={{ codex: codexProvider }}>+ <ChatInput onSend={mockOnSend} hasSession />+ </CredentialsProvidersContext.Provider>,+ );+ expect(screen.queryByLabelText(/AI connection:/i)).toBeNull();+ });+it("shows Thinking label in extended_thinking mode", () => {mockFlagValue = true;mockCopilotMode = "extended_thinking";diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/components/LlmRouteSelector.tsx b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/components/LlmRouteSelector.tsxnew file mode 100644index 0000000000..4a831504c8--- /dev/null+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/components/LlmRouteSelector.tsx@@ -0,0 +1,190 @@+"use client";++import { useGetV2ListChatTransports } from "@/app/api/__generated__/endpoints/chat/chat";+import type { ChatTransportResponse } from "@/app/api/__generated__/models/chatTransportResponse";+import {+ DropdownMenu,+ DropdownMenuContent,+ DropdownMenuItem,+ DropdownMenuTrigger,+} from "@/components/molecules/DropdownMenu/DropdownMenu";+import { toast } from "@/components/molecules/Toast/use-toast";+import { cn } from "@/lib/utils";+import { CredentialsProvidersContext } from "@/providers/agent-credentials/credentials-provider";+import Link from "next/link";+import { useContext, useEffect } from "react";+import {+ PiCaretDown as CaretDownIcon,+ PiKey as KeyIcon,+ PiWarningCircle as WarningCircleIcon,+} from "react-icons/pi";+import {+ findSelectedLLMTransport,+ getAvailableLLMTransports,+ getChatTransportSelection,+ resolveCopilotLLMAuthSelection,+} from "../../../helpers/copilotLlmAuth";+import { useCopilotUIStore } from "../../../store";++function getTransportDescription(+ transport: ChatTransportResponse,+ credentialTitle: string | undefined,+): string {+ if (transport.auth_provider === "codex") {+ return credentialTitle+ ? `${credentialTitle} · Uses your ChatGPT plan`+ : "Uses your ChatGPT plan";+ }+ return transport.label === "AutoGPT Platform"+ ? "Uses your AutoGPT plan"+ : "Uses this server's configured chat provider";+}++export function LLMRouteSelector() {+ const providers = useContext(CredentialsProvidersContext);+ const transportQuery = useGetV2ListChatTransports({+ query: { refetchOnWindowFocus: true, staleTime: 0 },+ });+ const transports =+ transportQuery.data?.status === 200+ ? transportQuery.data.data.transports+ : undefined;+ const availableTransports = getAvailableLLMTransports(transports);+ const {+ copilotLlmAuth: copilotLLMAuth,+ setCopilotLlmAuth: setCopilotLLMAuth,+ } = useCopilotUIStore();+ const resolvedSelection = resolveCopilotLLMAuthSelection(+ transports,+ copilotLLMAuth,+ );+ const selectedTransport = findSelectedLLMTransport(+ availableTransports,+ copilotLLMAuth,+ );+ const selectedConnectionMissing =+ transports !== undefined &&+ availableTransports.length > 0 &&+ copilotLLMAuth.authProvider !== "platform" &&+ !selectedTransport;++ useEffect(() => {+ if (!resolvedSelection) return;+ if (+ resolvedSelection.authProvider === copilotLLMAuth.authProvider &&+ resolvedSelection.credentialId === copilotLLMAuth.credentialId+ ) {+ return;+ }++ setCopilotLLMAuth(resolvedSelection);+ }, [+ copilotLLMAuth.authProvider,+ copilotLLMAuth.credentialId,+ resolvedSelection?.authProvider,+ resolvedSelection?.credentialId,+ setCopilotLLMAuth,+ ]);++ useEffect(() => {+ if (!selectedConnectionMissing) return;++ toast({+ title: "AI connections changed",+ description:+ "The selected connection is unavailable. The next AutoPilot task will use the available default.",+ });+ }, [selectedConnectionMissing]);++ if (transportQuery.isPending && transports === undefined) return null;++ if (transports === undefined) {+ return (+ <span+ aria-label="AI connections unavailable"+ className="ml-2 inline-flex h-9 items-center gap-1.5 rounded-full border border-destructive/20 bg-destructive/10 px-2.5 text-xs font-medium text-destructive"+ >+ <WarningCircleIcon size={14} />+ <span className="hidden sm:inline">Connections unavailable</span>+ </span>+ );+ }++ if (availableTransports.length === 0) {+ return (+ <Link+ href="/settings/integrations"+ aria-label="Set up an AI connection"+ className="ml-2 inline-flex h-9 items-center gap-1.5 rounded-full border border-border bg-muted px-2.5 text-xs font-medium text-foreground transition-colors hover:bg-muted/80"+ >+ <WarningCircleIcon size={14} />+ <span className="hidden sm:inline">Set up AI connection</span>+ </Link>+ );+ }++ if (availableTransports.length === 1) return null;++ const selectionLabel = selectedConnectionMissing+ ? "Connection unavailable"+ : (selectedTransport?.label ?? "Choose connection");++ return (+ <DropdownMenu>+ <DropdownMenuTrigger asChild>+ <button+ type="button"+ aria-label={`AI connection: ${selectionLabel} — change connection`}+ className={cn(+ "ml-2 inline-flex h-9 items-center gap-1.5 rounded-full border border-border bg-background px-2.5 text-xs font-medium shadow-sm transition-colors hover:bg-muted",+ selectedConnectionMissing ? "text-destructive" : "text-foreground",+ )}+ >+ {selectedConnectionMissing ? (+ <WarningCircleIcon size={14} />+ ) : (+ <KeyIcon size={14} />+ )}+ <span className="hidden sm:inline">{selectionLabel}</span>+ <CaretDownIcon size={12} className="text-muted-foreground" />+ </button>+ </DropdownMenuTrigger>+ <DropdownMenuContent align="start" className="w-72">+ {selectedConnectionMissing && (+ <DropdownMenuItem disabled className="gap-2 text-destructive">+ <WarningCircleIcon size={16} />+ Selected connection is unavailable+ </DropdownMenuItem>+ )}+ {availableTransports.map((transport) => {+ const selection = getChatTransportSelection(transport);+ if (!selection) return null;+ const credential =+ transport.auth_provider === "codex"+ ? providers?.codex?.savedCredentials.find(+ (candidate) => candidate.id === transport.credential_id,+ )+ : undefined;+ const isSelected =+ selectedTransport?.auth_provider === transport.auth_provider &&+ selectedTransport.credential_id === transport.credential_id;+ return (+ <DropdownMenuItem+ key={`${transport.auth_provider}:${transport.credential_id ?? "deployment"}`}+ onClick={() => setCopilotLLMAuth(selection)}+ className={cn("gap-2", isSelected && "bg-muted")}+ >+ <KeyIcon size={16} />+ <div className="min-w-0">+ <div>{transport.label}</div>+ <div className="truncate text-xs text-muted-foreground">+ {getTransportDescription(transport, credential?.title)}+ </div>+ </div>+ </DropdownMenuItem>+ );+ })}+ </DropdownMenuContent>+ </DropdownMenu>+ );+}diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/components/__tests__/LlmRouteSelector.test.tsx b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/components/__tests__/LlmRouteSelector.test.tsxnew file mode 100644index 0000000000..90f4e9d633--- /dev/null+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/components/ChatInput/components/__tests__/LlmRouteSelector.test.tsx@@ -0,0 +1,239 @@+import type { ChatTransportResponse } from "@/app/api/__generated__/models/chatTransportResponse";+import type { CredentialsMetaResponse } from "@/lib/autogpt-server-api";+import {+ CredentialsProvidersContext,+ type CredentialsProviderData,+} from "@/providers/agent-credentials/credentials-provider";+import {+ cleanup,+ render,+ screen,+ waitFor,+} from "@/tests/integrations/test-utils";+import userEvent from "@testing-library/user-event";+import { afterEach, describe, expect, it, vi } from "vitest";+import { useCopilotUIStore } from "../../../../store";+import { LLMRouteSelector } from "../LlmRouteSelector";++const testState = vi.hoisted(() => ({+ transports: [] as ChatTransportResponse[] | null,+ isError: false,+ queryOptions: vi.fn(),+ toast: vi.fn(),+}));++vi.mock(+ "@/app/api/__generated__/endpoints/chat/chat",+ async (importOriginal) => {+ const actual =+ await importOriginal<+ typeof import("@/app/api/__generated__/endpoints/chat/chat")+ >();+ return {+ ...actual,+ useGetV2ListChatTransports: (options: unknown) => {+ testState.queryOptions(options);+ return {+ data:+ testState.transports === null+ ? undefined+ : { status: 200, data: { transports: testState.transports } },+ isPending: testState.transports === null && !testState.isError,+ isError: testState.isError,+ };+ },+ };+ },+);++vi.mock("@/components/molecules/Toast/use-toast", () => ({+ toast: (...args: unknown[]) => testState.toast(...args),+ useToast: () => ({ toast: testState.toast, dismiss: vi.fn() }),+}));++const codexCredential: CredentialsMetaResponse = {+ id: "codex-credential-1",+ provider: "codex",+ type: "oauth2",+ title: "Personal ChatGPT",+ username: "[email protected]",+ scopes: [],+};++const hostedPlatform: ChatTransportResponse = {+ auth_provider: "platform",+ credential_id: null,+ label: "AutoGPT Platform",+ available: true,+ default: true,+};++const configuredSelfHosted: ChatTransportResponse = {+ auth_provider: "platform",+ credential_id: null,+ label: "Self-hosted chat",+ available: true,+ default: true,+};++const unconfiguredSelfHosted: ChatTransportResponse = {+ ...configuredSelfHosted,+ available: false,+ default: false,+};++const codexTransport: ChatTransportResponse = {+ auth_provider: "codex",+ credential_id: codexCredential.id,+ label: "ChatGPT",+ available: true,+ default: false,+};++function makeCodexProvider(): CredentialsProviderData {+ return {+ provider: "codex",+ providerName: "Codex",+ savedCredentials: [codexCredential],+ isSystemProvider: false,+ oAuthCallback: async () => codexCredential,+ mcpOAuthCallback: async () => codexCredential,+ createAPIKeyCredentials: async () => codexCredential,+ createUserPasswordCredentials: async () => codexCredential,+ createHostScopedCredentials: async () => codexCredential,+ deleteCredentials: async () => ({ deleted: true, revoked: null }),+ };+}++function SelectorHarness() {+ return (+ <CredentialsProvidersContext.Provider+ value={{ codex: makeCodexProvider() }}+ >+ <LLMRouteSelector />+ </CredentialsProvidersContext.Provider>+ );+}++afterEach(() => {+ cleanup();+ testState.transports = [];+ testState.isError = false;+ testState.queryOptions.mockClear();+ testState.toast.mockClear();+ useCopilotUIStore.getState().setCopilotLlmAuth({+ authProvider: "platform",+ credentialId: null,+ });+});++describe("LLMRouteSelector", () => {+ it("hides when the hosted platform is the only available route", () => {+ testState.transports = [hostedPlatform];++ render(<SelectorHarness />);++ expect(screen.queryByLabelText(/AI connection/i)).toBeNull();+ expect(useCopilotUIStore.getState().copilotLlmAuth).toEqual({+ authProvider: "platform",+ credentialId: null,+ });+ });++ it("offers hosted platform and Codex while defaulting to hosted", async () => {+ testState.transports = [hostedPlatform, codexTransport];+ const user = userEvent.setup();++ render(<SelectorHarness />);++ await user.click(screen.getByLabelText(/AI connection: AutoGPT Platform/i));+ expect(screen.getAllByText("AutoGPT Platform").length).toBeGreaterThan(0);+ expect(screen.getByText("ChatGPT")).toBeTruthy();+ expect(screen.getByText(/Personal ChatGPT/)).toBeTruthy();++ await user.click(screen.getByText("ChatGPT"));+ expect(useCopilotUIStore.getState().copilotLlmAuth).toEqual({+ authProvider: "codex",+ credentialId: codexCredential.id,+ });+ });++ it("refreshes a changed transport inventory immediately after pairing", () => {+ testState.transports = [hostedPlatform];+ const { rerender } = render(<SelectorHarness />);++ expect(screen.queryByLabelText(/AI connection/i)).toBeNull();+ expect(testState.queryOptions).toHaveBeenLastCalledWith(+ expect.objectContaining({+ query: expect.objectContaining({+ refetchOnWindowFocus: true,+ staleTime: 0,+ }),+ }),+ );++ testState.transports = [hostedPlatform, codexTransport];+ rerender(<SelectorHarness />);+ expect(+ screen.getByLabelText(/AI connection: AutoGPT Platform/i),+ ).toBeTruthy();++ testState.transports = [hostedPlatform];+ rerender(<SelectorHarness />);+ expect(screen.queryByLabelText(/AI connection/i)).toBeNull();+ });++ it("hides when configured self-hosted chat is the only route", () => {+ testState.transports = [configuredSelfHosted];++ render(<SelectorHarness />);++ expect(screen.queryByLabelText(/AI connection/i)).toBeNull();+ });++ it("offers configured self-hosted chat alongside Codex", async () => {+ testState.transports = [configuredSelfHosted, codexTransport];+ const user = userEvent.setup();++ render(<SelectorHarness />);++ await user.click(screen.getByLabelText(/AI connection: Self-hosted chat/i));+ expect(screen.getAllByText("Self-hosted chat").length).toBeGreaterThan(0);+ expect(screen.getByText("ChatGPT")).toBeTruthy();+ });++ it("automatically selects sole Codex on a keyless self-host", async () => {+ testState.transports = [+ unconfiguredSelfHosted,+ { ...codexTransport, default: true },+ ];++ render(<SelectorHarness />);++ await waitFor(() => {+ expect(useCopilotUIStore.getState().copilotLlmAuth).toEqual({+ authProvider: "codex",+ credentialId: codexCredential.id,+ });+ });+ expect(screen.queryByLabelText(/AI connection/i)).toBeNull();+ });++ it("links a keyless self-host with no Codex connection to setup", () => {+ testState.transports = [unconfiguredSelfHosted];++ render(<SelectorHarness />);++ const setupLink = screen.getByLabelText("Set up an AI connection");+ expect(setupLink.getAttribute("href")).toBe("/settings/integrations");+ });++ it("shows inventory failures without inventing a route", () => {+ testState.transports = null;+ testState.isError = true;++ render(<SelectorHarness />);++ expect(screen.getByLabelText("AI connections unavailable")).toBeTruthy();+ });+});diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamStore.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamStore.tsindex b6bbe2573c..dc350ac078 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamStore.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamStore.ts@@ -51,10 +51,18 @@ export interface PendingFirstSend {files: File[];}+interface PersistedCopilotStreamState {+ sessions: Record<string, SessionCoord>;+ pendingFirstSend: Pick<PendingFirstSend, "text"> | null;+ pendingFirstSendSessionId: string | null;+ pendingFileParts: FileUIPart[];+}+interface CopilotStreamStore {sessions: Record<string, SessionCoord>;messageSnapshots: Record<string, UIMessage[]>;pendingFirstSend: PendingFirstSend | null;+ pendingFirstSendSessionId: string | null;pendingFileParts: FileUIPart[];/*** True while the current chat is in `streaming` or `submitted` state.@@ -72,9 +80,10 @@ interface CopilotStreamStore {setStreaming: (streaming: boolean) => void;setPendingFirstSend: (send: PendingFirstSend | null) => void;+ bindPendingFirstSendToSession: (sessionId: string) => void;setPendingFileParts: (parts: FileUIPart[]) => void;/** Read-and-clear; used by the post-session-creation flush effect. */- takePendingFirstSend: () => {+ takePendingFirstSend: (sessionId: string) => {send: PendingFirstSend | null;parts: FileUIPart[];};@@ -84,11 +93,12 @@ interface CopilotStreamStore {}export const useCopilotStreamStore = create<CopilotStreamStore>()(- persist(+ persist<CopilotStreamStore, [], [], PersistedCopilotStreamState>((set, get) => ({sessions: {},messageSnapshots: {},pendingFirstSend: null,+ pendingFirstSendSessionId: null,pendingFileParts: [],isStreaming: false,@@ -137,14 +147,29 @@ export const useCopilotStreamStore = create<CopilotStreamStore>()(},setPendingFirstSend(send) {- set({ pendingFirstSend: send });+ set({ pendingFirstSend: send, pendingFirstSendSessionId: null });+ },+ bindPendingFirstSendToSession(sessionId) {+ if (!get().pendingFirstSend) return;+ set({ pendingFirstSendSessionId: sessionId });},setPendingFileParts(parts) {set({ pendingFileParts: parts });},- takePendingFirstSend() {- const { pendingFirstSend, pendingFileParts } = get();- set({ pendingFirstSend: null, pendingFileParts: [] });+ takePendingFirstSend(sessionId) {+ const {+ pendingFirstSend,+ pendingFirstSendSessionId,+ pendingFileParts,+ } = get();+ if (pendingFirstSendSessionId !== sessionId) {+ return { send: null, parts: [] };+ }+ set({+ pendingFirstSend: null,+ pendingFirstSendSessionId: null,+ pendingFileParts: [],+ });return { send: pendingFirstSend, parts: pendingFileParts };},@@ -153,22 +178,21 @@ export const useCopilotStreamStore = create<CopilotStreamStore>()(sessions: {},messageSnapshots: {},pendingFirstSend: null,+ pendingFirstSendSessionId: null,pendingFileParts: [],isStreaming: false,});},}),{- // Persist ONLY the per-session ``sessions`` map (which carries- // ``lastSubmittedMessageText``). Refresh / new tab in the same browser- // session keeps the dedup memory so the wrapped sendMessage's- // ``getSendSuppressionReason`` can still block duplicate POSTs after- // the in-memory state would otherwise be lost.+ // Persist the per-session dedup memory plus the serializable portion of+ // the first send. A full navigation can happen while adding sessionId to+ // the URL on mobile, so a module-scoped slot alone is not sufficient.+ // Browser File objects remain memory-only; workspace FileUIParts are+ // already server-backed and safe to restore after navigation.// ``messageSnapshots`` is intentionally excluded — it's a per-render// cache of UIMessages (often hundreds) that the next mount should// re-derive from the server, not restore from storage.- // ``pendingFirstSend`` / ``pendingFileParts`` are tab-local artefacts- // of the session-creation remount and would be confusing if persisted.name: "copilot-stream-store",version: 1,// SSR-safe storage adapter: ``window.sessionStorage`` in the browser,@@ -180,7 +204,42 @@ export const useCopilotStreamStore = create<CopilotStreamStore>()(? window.sessionStorage: { getItem: () => null, setItem: () => {}, removeItem: () => {} },),- partialize: (state) => ({ sessions: state.sessions }),+ partialize(state) {+ const canRestorePending =+ state.pendingFirstSend !== null &&+ state.pendingFirstSend.files.length === 0 &&+ state.pendingFirstSendSessionId !== null;+ return {+ sessions: state.sessions,+ pendingFirstSend: canRestorePending+ ? { text: state.pendingFirstSend!.text }+ : null,+ pendingFirstSendSessionId: canRestorePending+ ? state.pendingFirstSendSessionId+ : null,+ pendingFileParts: canRestorePending ? state.pendingFileParts : [],+ };+ },+ merge(persistedState, currentState) {+ const persisted =+ persistedState as Partial<PersistedCopilotStreamState>;+ const restoredPending =+ persisted.pendingFirstSend &&+ typeof persisted.pendingFirstSendSessionId === "string"+ ? {+ send: { text: persisted.pendingFirstSend.text, files: [] },+ sessionId: persisted.pendingFirstSendSessionId,+ parts: persisted.pendingFileParts ?? [],+ }+ : null;+ return {+ ...currentState,+ sessions: persisted.sessions ?? {},+ pendingFirstSend: restoredPending?.send ?? null,+ pendingFirstSendSessionId: restoredPending?.sessionId ?? null,+ pendingFileParts: restoredPending?.parts ?? [],+ };+ },},),);diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamTransport.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamTransport.tsindex 6155cf4190..99803cae2f 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamTransport.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/copilotStreamTransport.ts@@ -1,6 +1,7 @@import { environment } from "@/services/environment";import { DefaultChatTransport } from "ai";import type { ChatTransport, FileUIPart, UIMessage } from "ai";+import { v4 as uuidv4 } from "uuid";import { createSmoothingTransform } from "./copilotStreamSmoothing";import { getCopilotAuthHeaders } from "./helpers";@@ -96,7 +97,10 @@ export function createCopilotTransport({file_ids: fileIds && fileIds.length > 0 ? fileIds : null,mode: copilotModeRef.current ?? null,model: copilotModelRef.current ?? null,- message_id: crypto.randomUUID(),+ // Supplying options forces uuid's getRandomValues path. Unlike+ // crypto.randomUUID, getRandomValues is available on plain-HTTP LAN+ // origins used by the local single-container appliance.+ message_id: uuidv4({}),},headers: await getCopilotAuthHeaders(),};diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/helpers/copilotLlmAuth.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/helpers/copilotLlmAuth.tsnew file mode 100644index 0000000000..82c8f1bebd--- /dev/null+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/helpers/copilotLlmAuth.ts@@ -0,0 +1,63 @@+import type { ChatTransportResponse } from "@/app/api/__generated__/models/chatTransportResponse";+import type { CopilotLlmAuthSelection } from "../store";++export function getAvailableLLMTransports(+ transports: ChatTransportResponse[] | null | undefined,+): ChatTransportResponse[] {+ return (transports ?? []).filter(+ (transport) =>+ transport.available &&+ (transport.auth_provider === "platform" ||+ transport.credential_id !== null),+ );+}++export function getChatTransportSelection(+ transport: ChatTransportResponse,+): CopilotLlmAuthSelection | null {+ if (transport.auth_provider === "platform") {+ return { authProvider: "platform", credentialId: null };+ }+ if (transport.credential_id === null) return null;+ return {+ authProvider: "codex",+ credentialId: transport.credential_id,+ };+}++export function findSelectedLLMTransport(+ transports: ChatTransportResponse[] | null | undefined,+ selection: CopilotLlmAuthSelection,+): ChatTransportResponse | undefined {+ return getAvailableLLMTransports(transports).find(+ (transport) =>+ transport.auth_provider === selection.authProvider &&+ (transport.auth_provider === "platform" ||+ transport.credential_id === selection.credentialId),+ );+}++export function resolveCopilotLLMAuthSelection(+ transports: ChatTransportResponse[] | null | undefined,+ currentSelection: CopilotLlmAuthSelection,+): CopilotLlmAuthSelection | null {+ if (transports == null) return null;++ const availableTransports = getAvailableLLMTransports(transports);+ const selectedTransport = findSelectedLLMTransport(+ availableTransports,+ currentSelection,+ );+ if (selectedTransport) return currentSelection;++ const defaultTransport = availableTransports.find(+ (transport) => transport.default,+ );+ if (defaultTransport) return getChatTransportSelection(defaultTransport);++ if (availableTransports.length === 1) {+ return getChatTransportSelection(availableTransports[0]);+ }++ return null;+}diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/store.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/store.tsindex e9ca7a9c2f..ac9cd684c1 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/store.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/store.ts@@ -59,6 +59,10 @@ export type CopilotMode = "extended_thinking" | "fast";/** Per-request model tier. 'standard' = current default; 'advanced' = highest-capability. */export type CopilotLlmModel = "standard" | "advanced";+export type CopilotLlmAuthSelection =+ | { authProvider: "platform"; credentialId: null }+ | { authProvider: "codex"; credentialId: string };+/** Context panel tab. */export type ContextPanelTab = "progress" | "files";@@ -197,6 +201,10 @@ interface CopilotUIState {copilotLlmModel: CopilotLlmModel;setCopilotLlmModel: (model: CopilotLlmModel) => void;+ /** Authentication route locked into the next session when it is created. */+ copilotLlmAuth: CopilotLlmAuthSelection;+ setCopilotLlmAuth: (selection: CopilotLlmAuthSelection) => void;+/** Developer dry-run mode: sessions created with dry_run=true. */isDryRun: boolean;setIsDryRun: (enabled: boolean) => void;@@ -524,6 +532,11 @@ export const useCopilotUIStore = create<CopilotUIState>((set, get) => ({set({ copilotLlmModel: model });},+ copilotLlmAuth: { authProvider: "platform", credentialId: null },+ setCopilotLlmAuth: (selection) => {+ set({ copilotLlmAuth: selection });+ },+isDryRun: isClient && storage.get(Key.COPILOT_DRY_RUN) === "true",setIsDryRun: (enabled) => {if (enabled) {@@ -566,6 +579,7 @@ export const useCopilotUIStore = create<CopilotUIState>((set, get) => ({},copilotChatMode: "extended_thinking",copilotLlmModel: "standard",+ copilotLlmAuth: { authProvider: "platform", credentialId: null },isDryRun: false,});if (isClient) {diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/useChatSession.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/useChatSession.tsindex c61676e5b4..f37856ee0b 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/useChatSession.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/useChatSession.ts@@ -1,9 +1,11 @@import {getGetV2GetSessionQueryKey,+ useGetV2ListChatTransports,useGetV2GetSession,useGetV2ListSessions,usePostV2CreateSession,} from "@/app/api/__generated__/endpoints/chat/chat";+import type { CreateSessionRequest } from "@/app/api/__generated__/models/createSessionRequest";import { SESSION_LIST_QUERY_KEY } from "./useSessionList";import { useCopilotUIStore } from "./store";import { toast } from "@/components/molecules/Toast/use-toast";@@ -16,6 +18,11 @@ import {type TurnStatsMap,} from "./helpers/convertChatSessionToUiMessages";import { resolveSessionDryRun } from "./helpers";+import {+ getAvailableLLMTransports,+ resolveCopilotLLMAuthSelection,+} from "./helpers/copilotLlmAuth";+import { useCopilotStreamStore } from "./copilotStreamStore";interface UseChatSessionOptions {dryRun?: boolean;@@ -28,6 +35,19 @@ export function useChatSession({}: UseChatSessionOptions = {}) {const [sessionId, setSessionId] = useQueryState("sessionId", parseAsString);const queryClient = useQueryClient();+ const copilotLlmAuth = useCopilotUIStore((state) => state.copilotLlmAuth);++ const transportQuery = useGetV2ListChatTransports({+ query: {+ enabled: !sessionId,+ refetchOnWindowFocus: true,+ staleTime: 0,+ },+ });+ const chatTransports =+ transportQuery.data?.status === 200+ ? transportQuery.data.data.transports+ : undefined;const sessionQuery = useGetV2GetSession(sessionId ?? "", undefined, {query: {@@ -185,18 +205,7 @@ export function useChatSession({}, [freshSessionData, sessionId, hasActiveStream]);const { mutateAsync: createSessionMutation, isPending: isCreatingSession } =- usePostV2CreateSession({- mutation: {- onSuccess: (response) => {- if (response.status === 200 && response.data?.id) {- setSessionId(response.data.id);- queryClient.invalidateQueries({- queryKey: SESSION_LIST_QUERY_KEY,- });- }- },- },- });+ usePostV2CreateSession();async function createSession() {if (sessionId) return sessionId;@@ -204,16 +213,70 @@ export function useChatSession({// has asked for a new thread, auto-navigating them into an old one is// never the right recovery.sendStartedRef.current = true;+ const resolvedLLMAuth = resolveCopilotLLMAuthSelection(+ chatTransports,+ copilotLlmAuth,+ );+ const availableTransports = getAvailableLLMTransports(chatTransports);+ if (transportQuery.isError && chatTransports === undefined) {+ toast({+ variant: "destructive",+ title: "Could not check AI connections",+ description: "Refresh the page and try again.",+ });+ throw new Error("Could not load AI connections");+ }+ if (chatTransports !== undefined && availableTransports.length === 0) {+ toast({+ variant: "destructive",+ title: "AutoPilot needs an AI connection",+ description:+ "Sign in with ChatGPT under OpenAI in Settings → Integrations, or configure a chat API or local model on this server.",+ });+ throw new Error("chat_transport_not_configured");+ }+ if (!resolvedLLMAuth) {+ const connectionsAreLoading = chatTransports === undefined;+ toast({+ variant: "destructive",+ title: connectionsAreLoading+ ? "AI connections are still loading"+ : "Choose an AI connection",+ description: connectionsAreLoading+ ? "Wait a moment and try again."+ : "Select the connection AutoPilot should use before starting a new task.",+ });+ throw new Error(+ connectionsAreLoading+ ? "AI connections are still loading"+ : "AI connection selection required",+ );+ }+ if (+ copilotLlmAuth.authProvider !== "platform" &&+ resolvedLLMAuth.authProvider === "platform"+ ) {+ toast({+ title: "AI connections changed",+ description:+ "The next AutoPilot task will resolve the currently available connection before it starts.",+ });+ }+try {- const sessionData: Record<string, unknown> = {};+ const sessionData: CreateSessionRequest = {+ llm_auth_provider: resolvedLLMAuth.authProvider,+ };+ if (resolvedLLMAuth.authProvider === "codex") {+ sessionData.llm_credential_id = resolvedLLMAuth.credentialId;+ }if (dryRun) sessionData.dry_run = true;if (expertId) sessionData.expert_id = expertId;const body =Object.keys(sessionData).length > 0? { data: sessionData }: { data: null };- // eslint-disable-next-line @typescript-eslint/no-explicit-any- const response = await (createSessionMutation as any)(body);+ const response = await createSessionMutation(body);if (response.status !== 200 || !response.data?.id) {const error = new Error("Failed to create session");Sentry.captureException(error, {@@ -226,6 +289,13 @@ export function useChatSession({});throw error;}+ useCopilotStreamStore+ .getState()+ .bindPendingFirstSendToSession(response.data.id);+ setSessionId(response.data.id);+ queryClient.invalidateQueries({+ queryKey: SESSION_LIST_QUERY_KEY,+ });return response.data.id;} catch (error) {if (@@ -267,6 +337,13 @@ export function useChatSession({freshSessionData as { chat_status?: string } | undefined)?.chat_status;+ const sessionLlmAuthProvider: "platform" | "codex" | null =+ sessionId && sessionQuery.data?.status === 200+ ? sessionQuery.data.data.metadata?.llm_auth_provider === "codex"+ ? "codex"+ : "platform"+ : null;+// The expert this session actually belongs to, straight off the session// response rather than the URL — the ?expertId= param only describes what// the NEXT session will be and is absent on most ways of reaching a thread.@@ -280,6 +357,7 @@ export function useChatSession({return {sessionId,setSessionId,+ sessionLlmAuthProvider,sessionExpertId,isAdoptingExpertSession,hydratedMessages,diff --git a/autogpt_platform/frontend/src/app/(platform)/copilot/useSendMessage.ts b/autogpt_platform/frontend/src/app/(platform)/copilot/useSendMessage.tsindex 3604dbd901..18db149cf6 100644--- a/autogpt_platform/frontend/src/app/(platform)/copilot/useSendMessage.ts+++ b/autogpt_platform/frontend/src/app/(platform)/copilot/useSendMessage.ts@@ -157,7 +157,7 @@ export function useSendMessage({if (!sessionId) return;const { send, parts } = useCopilotStreamStore.getState()- .takePendingFirstSend();+ .takePendingFirstSend(sessionId);if (!send) return;void dispatchRef.current(sessionId, send.text, send.files, parts);}, [sessionId]);diff --git a/autogpt_platform/frontend/src/app/(platform)/settings/integrations/__tests__/main.test.tsx b/autogpt_platform/frontend/src/app/(platform)/settings/integrations/__tests__/main.test.tsxindex d67add5234..177d234ecd 100644--- a/autogpt_platform/frontend/src/app/(platform)/settings/integrations/__tests__/main.test.tsx+++ b/autogpt_platform/frontend/src/app/(platform)/settings/integrations/__tests__/main.test.tsx@@ -340,6 +340,51 @@ describe("SettingsIntegrationsPage — connect dialog", () => {expect(within(dialog).getByRole("tab", { name: /api key/i })).toBeDefined();});+ test("presents ChatGPT sign-in and API keys as one OpenAI integration", async () => {+ server.use(+ getGetV1ListCredentialsMockHandler([]),+ getGetV1ListProvidersMockHandler([+ makeProvider({+ name: "codex",+ description: "Use your ChatGPT plan",+ supported_auth_types: ["oauth2"],+ }),+ makeProvider({+ name: "openai",+ description: "GPT models",+ supported_auth_types: ["api_key"],+ }),+ ]),+ );++ render(<SettingsIntegrationsPage />);++ const connectButtons = await screen.findAllByRole("button", {+ name: /connect.*service/i,+ });+ fireEvent.click(connectButtons[0]);++ const dialog = await screen.findByRole("dialog");+ const description = await within(dialog).findByText(+ /openai models via api key or your chatgpt subscription/i,+ );+ expect(within(dialog).getAllByText("OpenAI")).toHaveLength(1);+ fireEvent.click(description);++ expect(+ await within(dialog).findByRole("heading", { name: "OpenAI" }),+ ).toBeDefined();+ expect(within(dialog).getByRole("tab", { name: "ChatGPT" })).toBeDefined();+ expect(within(dialog).getByRole("tab", { name: /api key/i })).toBeDefined();+ expect(+ within(dialog).getByRole("button", { name: "Sign in with ChatGPT" }),+ ).toBeDefined();+ expect(+ within(dialog).getByAltText("OpenAI logo").getAttribute("src"),+ ).toContain("/integrations/openai.png");+ expect(within(dialog).queryByText("Codex")).toBeNull();+ });+test("API key tab: submitting the form posts credentials and closes the dialog", async () => {server.use(getGetV1ListCredentialsMockHandler([]),diff --git a/autogpt_platform/frontend/src/app/api/openapi.json b/autogpt_platform/frontend/src/app/api/openapi.jsonindex 1f9667f1a7..8e916982c8 100644--- a/autogpt_platform/frontend/src/app/api/openapi.json+++ b/autogpt_platform/frontend/src/app/api/openapi.json@@ -4109,6 +4109,29 @@"security": [{ "HTTPBearerJWT": [] }]}},+ "/api/chat/transports": {+ "get": {+ "tags": ["v2", "chat", "chat"],+ "summary": "List Chat Transports",+ "operationId": "getV2ListChatTransports",+ "responses": {+ "200": {+ "description": "Successful Response",+ "content": {+ "application/json": {+ "schema": {+ "$ref": "#/components/schemas/ChatTransportsResponse"+ }+ }+ }+ },+ "401": {+ "$ref": "#/components/responses/HTTP401NotAuthenticatedError"+ }+ },+ "security": [{ "HTTPBearerJWT": [] }]+ }+ },"/api/chat/usage": {"get": {"tags": ["v2", "chat", "chat"],@@ -6655,6 +6678,92 @@"security": [{ "HTTPBearerJWT": [] }]}},+ "/api/integrations/codex/credentials/{credential_id}/account": {+ "get": {+ "tags": ["v1", "integrations"],+ "summary": "Codex Account",+ "operationId": "getV1CodexAccount",+ "security": [{ "HTTPBearerJWT": [] }],+ "parameters": [+ {+ "name": "credential_id",+ "in": "path",+ "required": true,+ "schema": {+ "type": "string",+ "minLength": 1,+ "title": "Credential Id"+ }+ }+ ],+ "responses": {+ "200": {+ "description": "Successful Response",+ "content": {+ "application/json": {+ "schema": {+ "$ref": "#/components/schemas/CodexAccountSnapshot"+ }+ }+ }+ },+ "401": {+ "$ref": "#/components/responses/HTTP401NotAuthenticatedError"+ },+ "422": {+ "description": "Validation Error",+ "content": {+ "application/json": {+ "schema": { "$ref": "#/components/schemas/HTTPValidationError" }+ }+ }+ }+ }+ }+ },+ "/api/integrations/codex/credentials/{credential_id}/rate-limits": {+ "get": {+ "tags": ["v1", "integrations"],+ "summary": "Codex Rate Limits",+ "operationId": "getV1CodexRateLimits",+ "security": [{ "HTTPBearerJWT": [] }],+ "parameters": [+ {+ "name": "credential_id",+ "in": "path",+ "required": true,+ "schema": {+ "type": "string",+ "minLength": 1,+ "title": "Credential Id"+ }+ }+ ],+ "responses": {+ "200": {+ "description": "Successful Response",+ "content": {+ "application/json": {+ "schema": {+ "$ref": "#/components/schemas/CodexRateLimitsSnapshot"+ }+ }+ }+ },+ "401": {+ "$ref": "#/components/responses/HTTP401NotAuthenticatedError"+ },+ "422": {+ "description": "Validation Error",+ "content": {+ "application/json": {+ "schema": { "$ref": "#/components/schemas/HTTPValidationError" }+ }+ }+ }+ }+ }+ },"/api/integrations/credentials": {"get": {"tags": ["v1", "integrations"],@@ -6701,7 +6810,8 @@}}}- }+ },+ "security": [{ "HTTPBearer": [] }]}},"/api/integrations/providers/constants": {@@ -15460,6 +15570,16 @@"title": "Dry Run","default": false},+ "llm_auth_provider": {+ "type": "string",+ "enum": ["platform", "codex"],+ "title": "Llm Auth Provider",+ "default": "platform"+ },+ "llm_credential_id": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Llm Credential Id"+ },"builder_graph_id": {"anyOf": [{ "type": "string" }, { "type": "null" }],"title": "Builder Graph Id"@@ -15514,6 +15634,43 @@"title": "ChatShareStateResponse","description": "Surfaces the chat's current share state so the modal can open\nin the right mode (share-vs-revoke) without an extra round-trip.\n\nThe three counts power the consent-disclosure block above the\ntoggle so the owner sees exactly what they're about to expose.\nThey reflect live state — sharing is live (not snapshot-at-enable),\nso these numbers are also a lower bound for what subsequent\nviewers will see if more messages / runs / files land before\nrevocation."},+ "ChatTransportResponse": {+ "properties": {+ "auth_provider": {+ "type": "string",+ "enum": ["platform", "codex"],+ "title": "Auth Provider"+ },+ "credential_id": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Credential Id"+ },+ "label": { "type": "string", "title": "Label" },+ "available": { "type": "boolean", "title": "Available" },+ "default": { "type": "boolean", "title": "Default" }+ },+ "type": "object",+ "required": [+ "auth_provider",+ "credential_id",+ "label",+ "available",+ "default"+ ],+ "title": "ChatTransportResponse"+ },+ "ChatTransportsResponse": {+ "properties": {+ "transports": {+ "items": { "$ref": "#/components/schemas/ChatTransportResponse" },+ "type": "array",+ "title": "Transports"+ }+ },+ "type": "object",+ "required": ["transports"],+ "title": "ChatTransportsResponse"+ },"ClarificationNeededResponse": {"properties": {"type": {@@ -15581,6 +15738,96 @@"title": "CoPilotUsagePublic","description": "Current usage status for a user — public (client-safe) shape."},+ "CodexAccountSnapshot": {+ "properties": {+ "connected": { "type": "boolean", "title": "Connected" },+ "requires_openai_auth": {+ "type": "boolean",+ "title": "Requires Openai Auth"+ },+ "account_type": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Account Type"+ },+ "email": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Email"+ },+ "plan_type": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Plan Type"+ }+ },+ "additionalProperties": false,+ "type": "object",+ "required": ["connected", "requires_openai_auth"],+ "title": "CodexAccountSnapshot"+ },+ "CodexRateLimitWindow": {+ "properties": {+ "used_percent": { "type": "integer", "title": "Used Percent" },+ "window_duration_mins": {+ "anyOf": [{ "type": "integer" }, { "type": "null" }],+ "title": "Window Duration Mins"+ },+ "resets_at": {+ "anyOf": [{ "type": "integer" }, { "type": "null" }],+ "title": "Resets At"+ }+ },+ "additionalProperties": false,+ "type": "object",+ "required": ["used_percent"],+ "title": "CodexRateLimitWindow"+ },+ "CodexRateLimitsSnapshot": {+ "properties": {+ "plan_type": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Plan Type"+ },+ "limit_id": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Limit Id"+ },+ "limit_name": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Limit Name"+ },+ "rate_limit_reached_type": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Rate Limit Reached Type"+ },+ "primary": {+ "anyOf": [+ { "$ref": "#/components/schemas/CodexRateLimitWindow" },+ { "type": "null" }+ ]+ },+ "secondary": {+ "anyOf": [+ { "$ref": "#/components/schemas/CodexRateLimitWindow" },+ { "type": "null" }+ ]+ },+ "has_credits": {+ "anyOf": [{ "type": "boolean" }, { "type": "null" }],+ "title": "Has Credits"+ },+ "unlimited_credits": {+ "anyOf": [{ "type": "boolean" }, { "type": "null" }],+ "title": "Unlimited Credits"+ },+ "bucket_ids": {+ "items": { "type": "string" },+ "type": "array",+ "title": "Bucket Ids"+ }+ },+ "additionalProperties": false,+ "type": "object",+ "title": "CodexRateLimitsSnapshot"+ },"CommunityListResponse": {"properties": {"user_id": { "type": "string", "title": "User Id" },@@ -16132,6 +16379,19 @@],"title": "Builder Graph Id"},+ "llm_auth_provider": {+ "type": "string",+ "enum": ["platform", "codex"],+ "title": "Llm Auth Provider",+ "default": "platform"+ },+ "llm_credential_id": {+ "anyOf": [+ { "type": "string", "maxLength": 128 },+ { "type": "null" }+ ],+ "title": "Llm Credential Id"+ },"expert_id": {"anyOf": [{ "type": "string", "maxLength": 128 },@@ -16155,7 +16415,11 @@},"metadata": {"$ref": "#/components/schemas/ChatSessionMetadata",- "default": { "dry_run": false, "kind": "normal" }+ "default": {+ "dry_run": false,+ "llm_auth_provider": "platform",+ "kind": "normal"+ }},"expert_id": {"anyOf": [{ "type": "string" }, { "type": "null" }],@@ -19926,7 +20190,11 @@"LoginResponse": {"properties": {"login_url": { "type": "string", "title": "Login Url" },- "state_token": { "type": "string", "title": "State Token" }+ "state_token": { "type": "string", "title": "State Token" },+ "cancel_url": {+ "anyOf": [{ "type": "string" }, { "type": "null" }],+ "title": "Cancel Url"+ }},"type": "object","required": ["login_url", "state_token"],@@ -20758,6 +21026,23 @@"items": { "type": "string" },"type": "array","title": "Scopes"+ },+ "refresh_strategy": {+ "type": "string",+ "enum": ["oauth_handler", "provider_runtime"],+ "title": "Refresh Strategy",+ "default": "oauth_handler"+ },+ "provider_state": {+ "anyOf": [+ { "type": "string", "format": "password", "writeOnly": true },+ { "type": "null" }+ ],+ "title": "Provider State"+ },+ "provider_state_version": {+ "anyOf": [{ "type": "integer" }, { "type": "null" }],+ "title": "Provider State Version"}},"type": "object",@@ -23060,7 +23345,11 @@},"metadata": {"$ref": "#/components/schemas/ChatSessionMetadata",- "default": { "dry_run": false, "kind": "normal" }+ "default": {+ "dry_run": false,+ "llm_auth_provider": "platform",+ "kind": "normal"+ }},"expert_id": {"anyOf": [{ "type": "string" }, { "type": "null" }],diff --git a/autogpt_platform/frontend/src/app/api/proxy/[...path]/__tests__/route.helpers.test.ts b/autogpt_platform/frontend/src/app/api/proxy/[...path]/__tests__/route.helpers.test.tsindex 0d9af07ee1..004b5b7735 100644--- a/autogpt_platform/frontend/src/app/api/proxy/[...path]/__tests__/route.helpers.test.ts+++ b/autogpt_platform/frontend/src/app/api/proxy/[...path]/__tests__/route.helpers.test.ts@@ -8,6 +8,8 @@ import {fetchWorkspaceDownloadWithRetry,watchResponseStart,RESPONSE_START_TIMEOUT_MS,+ CODEX_LOGIN_RESPONSE_START_TIMEOUT_MS,+ getResponseStartTimeoutMs,} from "../route.helpers";describe("isWorkspaceDownloadRequest", () => {@@ -978,6 +980,53 @@ describe("watchResponseStart", () => {expect((watch.signal.reason as DOMException).name).toBe("TimeoutError");});+ it("uses the extended cold-start budget for Codex device login", () => {+ const timeout = getResponseStartTimeoutMs(+ ["api", "integrations", "codex", "login"],+ "GET",+ );+ const watch = watchResponseStart(null, timeout);++ vi.advanceTimersByTime(RESPONSE_START_TIMEOUT_MS + 1);+ expect(watch.signal.aborted).toBe(false);++ vi.advanceTimersByTime(+ CODEX_LOGIN_RESPONSE_START_TIMEOUT_MS - RESPONSE_START_TIMEOUT_MS,+ );+ expect(watch.signal.aborted).toBe(true);+ });++ it("keeps the standard budget for other methods and routes", () => {+ expect(+ getResponseStartTimeoutMs(+ ["api", "integrations", "codex", "login"],+ "POST",+ ),+ ).toBe(RESPONSE_START_TIMEOUT_MS);+ expect(+ getResponseStartTimeoutMs(+ ["api", "integrations", "github", "login"],+ "GET",+ ),+ ).toBe(RESPONSE_START_TIMEOUT_MS);+ });++ it("extends cold App Server account, rate-limit, and logout operations", () => {+ expect(+ getResponseStartTimeoutMs(+ ["api", "integrations", "codex", "credentials", "cred-1"],+ "DELETE",+ ),+ ).toBe(CODEX_LOGIN_RESPONSE_START_TIMEOUT_MS);+ for (const operation of ["account", "rate-limits"]) {+ expect(+ getResponseStartTimeoutMs(+ ["api", "integrations", "codex", "credentials", "cred-1", operation],+ "GET",+ ),+ ).toBe(CODEX_LOGIN_RESPONSE_START_TIMEOUT_MS);+ }+ });it("does not abort once cleared (backend started responding)", () => {const watch = watchResponseStart(null);diff --git a/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.helpers.ts b/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.helpers.tsindex 93d8c4e7ce..7de65cf741 100644--- a/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.helpers.ts+++ b/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.helpers.ts@@ -150,8 +150,32 @@ export function getWorkspaceDownloadErrorMessage(body: unknown): string | null {// legitimately slow transfers (big uploads) spend their time in the upload// phase, which this timeout deliberately excludes.export const RESPONSE_START_TIMEOUT_MS = 30_000;+export const CODEX_LOGIN_RESPONSE_START_TIMEOUT_MS = 120_000;++export function getResponseStartTimeoutMs(+ path: string[],+ method: string,+): number {+ const isCodexCredentialControl =+ path.length >= 5 &&+ path.slice(0, 4).join("/") === "api/integrations/codex/credentials" &&+ ((method === "DELETE" && path.length === 5) ||+ (method === "GET" &&+ path.length === 6 &&+ (path[5] === "account" || path[5] === "rate-limits")));+ if (+ (method === "GET" && path.join("/") === "api/integrations/codex/login") ||+ isCodexCredentialControl+ ) {+ return CODEX_LOGIN_RESPONSE_START_TIMEOUT_MS;+ }+ return RESPONSE_START_TIMEOUT_MS;+}-export function watchResponseStart(requestBody: ReadableStream | null) {+export function watchResponseStart(+ requestBody: ReadableStream | null,+ timeoutMs: number = RESPONSE_START_TIMEOUT_MS,+) {const abort = new AbortController();let timer: ReturnType<typeof setTimeout> | undefined;@@ -161,11 +185,11 @@ export function watchResponseStart(requestBody: ReadableStream | null) {abort.abort(new DOMException("Backend sent no response within " +- `${RESPONSE_START_TIMEOUT_MS}ms of receiving the request`,+ `${timeoutMs}ms of receiving the request`,"TimeoutError",),),- RESPONSE_START_TIMEOUT_MS,+ timeoutMs,);}diff --git a/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.ts b/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.tsindex 01fb967e97..6f6e485435 100644--- a/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.ts+++ b/autogpt_platform/frontend/src/app/api/proxy/[...path]/route.ts@@ -8,6 +8,7 @@ import { NextRequest, NextResponse } from "next/server";import {fetchWorkspaceDownloadWithRetry,+ getResponseStartTimeoutMs,getWorkspaceDownloadErrorMessage,isWorkspaceDownloadRequest,watchResponseStart,@@ -218,7 +219,10 @@ async function handler(// much tighter response-start watch arms only once the request body has// been fully uploaded — a multi-minute upload is legitimate, a backend// that stays silent after receiving everything is stalled.- const responseStart = watchResponseStart(hasBody ? req.body : null);+ const responseStart = watchResponseStart(+ hasBody ? req.body : null,+ getResponseStartTimeoutMs(path, method),+ );let backendResponse: Response;try {diff --git a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/CredentialsInput.tsx b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/CredentialsInput.tsxindex f927d782df..164d8ec30e 100644--- a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/CredentialsInput.tsx+++ b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/CredentialsInput.tsx@@ -168,7 +168,7 @@ export function CredentialsInput({<OAuthFlowWaitingModalopen={isOAuth2FlowInProgress}onClose={cancelOAuthFlow}- providerName={providerName}+ providerName={provider === "codex" ? "ChatGPT" : providerName}popupBlocked={oAuthPopupBlocked}/>)}diff --git a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/CredentialsInput.test.tsx b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/CredentialsInput.test.tsxindex 3aaeb27ab7..5bcd7bd6c5 100644--- a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/CredentialsInput.test.tsx+++ b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/CredentialsInput.test.tsx@@ -110,6 +110,84 @@ afterEach(() => {});describe("CredentialsInput – OAuth flow", () => {+ it("clears a credential retained from a different transport provider", async () => {+ const onSelectCredentials = vi.fn();+ mockUseCredentials.mockReturnValue(+ makeCredentialsReturn({+ provider: "codex",+ providerName: "Codex",+ savedCredentials: [],+ }),+ );++ render(+ <CredentialsInput+ schema={baseSchema}+ selectedCredentials={{+ id: "openai-key",+ provider: "openai",+ type: "api_key",+ title: "OpenAI key",+ }}+ onSelectCredentials={onSelectCredentials}+ showTitle={false}+ />,+ );++ await waitFor(() =>+ expect(onSelectCredentials).toHaveBeenCalledWith(undefined),+ );+ });++ it("shows OpenAI branding while routing ChatGPT sign-in through Codex", async () => {+ const oAuthLoginMock = vi.fn().mockResolvedValue({+ login_url: "https://auth.openai.com/codex/device",+ state_token: "state-codex",+ });+ mockUseBackendAPI.mockReturnValue(+ makeBackendAPI({ oAuthLogin: oAuthLoginMock }),+ );+ mockUseCredentials.mockReturnValue(+ makeCredentialsReturn({+ provider: "codex",+ providerName: "OpenAI",+ schema: {+ ...baseSchema,+ credentials_provider: ["codex"],+ credentials_scopes: [],+ },+ }),+ );+ mockOpenOAuthPopup.mockReturnValue({+ promise: Promise.resolve({ code: "login-id", state: "state-codex" }),+ cleanup: { abort: vi.fn() },+ popupBlocked: false,+ fallbackBlocked: false,+ });++ render(+ <CredentialsInput+ schema={{+ ...baseSchema,+ credentials_provider: ["codex"],+ credentials_scopes: [],+ }}+ onSelectCredentials={vi.fn()}+ showTitle+ />,+ );++ expect(await screen.findByText("OpenAI credentials")).toBeDefined();+ const signInButton = screen.getByRole("button", {+ name: "Sign in with ChatGPT",+ });+ fireEvent.click(signInButton);++ await waitFor(() => {+ expect(oAuthLoginMock).toHaveBeenCalledWith("codex", [], undefined);+ });+ });+it("clicking the Add account button calls oAuthLogin without a credentialID", async () => {const oAuthLoginMock = vi.fn().mockResolvedValue({login_url: "https://accounts.google.com/o/oauth2/auth",diff --git a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/helpers.test.ts b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/helpers.test.tsindex 487e63d3d3..fdef637346 100644--- a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/helpers.test.ts+++ b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/__tests__/helpers.test.ts@@ -119,6 +119,15 @@ describe("getActionButtonText", () => {"Connect another account",);});++ it("uses ChatGPT sign-in copy for the internal Codex provider", () => {+ expect(+ getActionButtonText(true, false, false, false, false, "codex"),+ ).toBe("Sign in with ChatGPT");+ expect(+ getActionButtonText(true, false, false, false, true, "codex"),+ ).toBe("Reconnect ChatGPT");+ });});describe("when only API key is supported", () => {diff --git a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/helpers.ts b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/helpers.tsindex 9542be4166..438ba1e1f2 100644--- a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/helpers.ts+++ b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/helpers.ts@@ -138,6 +138,7 @@ export function getActionButtonText(supportsUserPassword: boolean,supportsHostScoped: boolean,hasExistingCredentials: boolean,+ provider?: string,): string {const multipleTypes =countSupportedTypes(@@ -151,6 +152,12 @@ export function getActionButtonText(return hasExistingCredentials ? "Add another credential" : "Add credential";}+ if (provider === "codex" && supportsOAuth2) {+ return hasExistingCredentials+ ? "Reconnect ChatGPT"+ : "Sign in with ChatGPT";+ }+if (hasExistingCredentials) {if (supportsOAuth2) return "Connect another account";if (supportsApiKey) return "Use a new API key";diff --git a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/useCredentialsInput.ts b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/useCredentialsInput.tsindex 73bf253965..ad738014d6 100644--- a/autogpt_platform/frontend/src/components/contextual/CredentialsInput/useCredentialsInput.ts+++ b/autogpt_platform/frontend/src/components/contextual/CredentialsInput/useCredentialsInput.ts@@ -103,6 +103,14 @@ export function useCredentialsInput({if (readOnly) return;if (!credentials || !("savedCredentials" in credentials)) return;const availableCreds = credentials.savedCredentials;+ if (+ selectedCredential &&+ selectedCredential.provider !== credentials.provider+ ) {+ onSelectCredential(undefined);+ hasAttemptedAutoSelect.current = false;+ return;+ }if (availableCreds.length === 0) return;if (selectedCredential &&@@ -214,6 +222,7 @@ export function useCredentialsInput({try {let login_url: string;let state_token: string;+ let cancel_url: string | null | undefined;if (isMCP) {const mcpLoginResponse = await postV2InitiateOauthLoginForAnMcpServer({@@ -222,7 +231,7 @@ export function useCredentialsInput({if (mcpLoginResponse.status !== 200) throw mcpLoginResponse.data;({ login_url, state_token } = mcpLoginResponse.data);} else {- ({ login_url, state_token } = await api.oAuthLogin(+ ({ login_url, state_token, cancel_url } = await api.oAuthLogin(provider,schema.credentials_scopes,credentialID,@@ -240,7 +249,9 @@ export function useCredentialsInput({const { promise, cleanup, popupBlocked, fallbackBlocked } =openOAuthPopup(login_url, {stateToken: state_token,+ cancelUrl: cancel_url,preOpenedWindow,+ timeout: provider === "codex" ? 15 * 60 * 1000 : undefined,// Always enable BroadcastChannel + localStorage listeners — they are// the only path that works when the popup is blocked and we fall back// to a new tab (window.opener can be severed by cross-origin COOP).@@ -474,6 +485,7 @@ export function useCredentialsInput({supportsUserPassword,supportsHostScoped,userCredentials.length > 0,+ provider,),setAPICredentialsModalOpen,setUserPasswordCredentialsModalOpen,diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/__tests__/helpers.test.ts b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/__tests__/helpers.test.tsindex 8f57f62629..eed78ec50b 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/__tests__/helpers.test.ts+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/__tests__/helpers.test.ts@@ -100,6 +100,30 @@ describe("groupCredentialsByProvider", () => {expect(groups[0].credentials.map((c) => c.id)).toEqual(["b", "c"]);});+ test("groups ChatGPT and API-key credentials under OpenAI without changing their providers", () => {+ const groups = groupCredentialsByProvider([+ makeCred({+ id: "api-key",+ provider: "openai",+ type: "api_key",+ title: "OpenAI key",+ }),+ makeCred({+ id: "chatgpt",+ provider: "codex",+ type: "oauth2",+ title: "ChatGPT for Codex",+ }),+ ]);++ expect(groups).toHaveLength(1);+ expect(groups[0].id).toBe("openai");+ expect(groups[0].name).toBe("OpenAI");+ expect(+ groups[0].credentials.map((credential) => credential.provider),+ ).toEqual(["openai", "codex"]);+ });+test("falls back to formatted provider name when title is missing", () => {const groups = groupCredentialsByProvider([makeCred({ provider: "github", title: null }),diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/__tests__/helpers.test.ts b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/__tests__/helpers.test.tsindex ec96f1ded5..48fda64d2a 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/__tests__/helpers.test.ts+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/__tests__/helpers.test.ts@@ -56,6 +56,31 @@ describe("toConnectableProviders", () => {expect(github?.supportedAuthTypes).toEqual(["oauth2"]);expect(openai?.supportedAuthTypes).toEqual([]);});++ test("merges ChatGPT sign-in into OpenAI while preserving backend auth targets", () => {+ const result = toConnectableProviders([+ makeMeta({+ name: "codex",+ description: "Use your ChatGPT plan",+ supported_auth_types: ["oauth2"],+ }),+ makeMeta({+ name: "openai",+ description: "GPT models and embeddings",+ supported_auth_types: ["api_key"],+ }),+ ]);++ expect(result).toHaveLength(1);+ expect(result[0]).toMatchObject({+ id: "openai",+ name: "OpenAI",+ description: "OpenAI models via API key or your ChatGPT subscription",+ supportedAuthTypes: ["oauth2", "api_key"],+ authProviderByType: { oauth2: "codex" },+ searchTerms: ["codex"],+ });+ });});describe("filterConnectableProviders", () => {@@ -108,4 +133,14 @@ describe("filterConnectableProviders", () => {test("returns an empty list when nothing matches", () => {expect(filterConnectableProviders(providers, "nope")).toEqual([]);});++ test("matches presentation aliases", () => {+ const openai = toConnectableProviders([+ makeMeta({ name: "codex", supported_auth_types: ["oauth2"] }),+ makeMeta({ name: "openai", supported_auth_types: ["api_key"] }),+ ]);++ expect(filterConnectableProviders(openai, "chatgpt")).toEqual(openai);+ expect(filterConnectableProviders(openai, "codex")).toEqual(openai);+ });});diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/DetailView.tsx b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/DetailView.tsxindex e3c98d220c..338c66770e 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/DetailView.tsx+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/DetailView.tsx@@ -12,7 +12,7 @@ import {import { AuthType, ConnectableProvider, type AuthMethod } from "../../helpers";import { McpConnectPanel } from "./McpConnectPanel";-import { MethodPanel, TAB_LABEL } from "./MethodPanel";+import { getAuthMethodLabel, MethodPanel } from "./MethodPanel";import { ProviderAvatar } from "./ProviderAvatar";import { UnsupportedNotice } from "./UnsupportedNotice";import { ArrowLeft02Icon } from "@hugeicons/core-free-icons";@@ -98,7 +98,7 @@ export function DetailView({ provider, onBack, onSuccess }: Props) {<TabsLineList>{tabs.map((method) => (<TabsLineTrigger key={method} value={method}>- {TAB_LABEL[method]}+ {getAuthMethodLabel(provider, method)}</TabsLineTrigger>))}</TabsLineList>diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/MethodPanel.tsx b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/MethodPanel.tsxindex 20d5d1bede..72442fbc02 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/MethodPanel.tsx+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/MethodPanel.tsx@@ -23,11 +23,14 @@ interface Props {}export function MethodPanel({ method, provider, onSuccess }: Props) {+ const authProvider = getAuthProvider(provider, method);if (method === AuthType.oauth2) {+ const isChatGPT = authProvider === "codex";return (<OAuthConnectButton- provider={provider.id}- providerName={provider.name}+ provider={authProvider}+ providerName={isChatGPT ? "ChatGPT" : provider.name}+ buttonLabel={isChatGPT ? "Sign in with ChatGPT" : undefined}onSuccess={onSuccess}/>);@@ -35,7 +38,7 @@ export function MethodPanel({ method, provider, onSuccess }: Props) {if (method === AuthType.api_key) {return (<ApiKeyConnectForm- provider={provider.id}+ provider={authProvider}providerName={provider.name}onSuccess={onSuccess}/>@@ -49,4 +52,24 @@ export function MethodPanel({ method, provider, onSuccess }: Props) {);}+export function getAuthProvider(+ provider: ConnectableProvider,+ method: AuthMethod,+): string {+ return provider.authProviderByType?.[method] ?? provider.id;+}++export function getAuthMethodLabel(+ provider: ConnectableProvider,+ method: AuthMethod,+): string {+ if (+ method === AuthType.oauth2 &&+ getAuthProvider(provider, method) === "codex"+ ) {+ return "ChatGPT";+ }+ return TAB_LABEL[method];+}+export { TAB_LABEL };diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/OAuthConnectButton.tsx b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/OAuthConnectButton.tsxindex 1e73a87d19..f2639167a7 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/OAuthConnectButton.tsx+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/OAuthConnectButton.tsx@@ -9,12 +9,14 @@ import { Icon } from "@/components/atoms/Icon/Icon";interface Props {provider: string;providerName: string;+ buttonLabel?: string;onSuccess: () => void;}export function OAuthConnectButton({provider,providerName,+ buttonLabel,onSuccess,}: Props) {const { connect, isPending } = useOAuthConnect({ provider, onSuccess });@@ -33,7 +35,7 @@ export function OAuthConnectButton({loading={isPending}rightIcon={<Icon icon={LinkSquare01Icon} size={18} />}>- Continue with {providerName}+ {buttonLabel ?? `Continue with ${providerName}`}</Button></div>);diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/MethodPanel.test.tsx b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/MethodPanel.test.tsxnew file mode 100644index 0000000000..5f3643c88d--- /dev/null+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/MethodPanel.test.tsx@@ -0,0 +1,38 @@+import { render, screen } from "@/tests/integrations/test-utils";+import { describe, expect, test, vi } from "vitest";++import { AuthType, type ConnectableProvider } from "../../../helpers";+import { MethodPanel } from "../MethodPanel";+import { useOAuthConnect } from "../useOAuthConnect";++vi.mock("../useOAuthConnect", () => ({+ useOAuthConnect: vi.fn(() => ({ connect: vi.fn(), isPending: false })),+}));++const openaiProvider: ConnectableProvider = {+ id: "openai",+ name: "OpenAI",+ description: "OpenAI models via API key or your ChatGPT subscription",+ supportedAuthTypes: [AuthType.oauth2, AuthType.api_key],+ authProviderByType: { [AuthType.oauth2]: "codex" },+};++describe("MethodPanel", () => {+ test("uses ChatGPT branding while sending OAuth to the Codex backend provider", () => {+ render(+ <MethodPanel+ method={AuthType.oauth2}+ provider={openaiProvider}+ onSuccess={vi.fn()}+ />,+ );++ expect(screen.getByText(/chatgpt sign-in window/i)).toBeDefined();+ expect(+ screen.getByRole("button", { name: "Sign in with ChatGPT" }),+ ).toBeDefined();+ expect(vi.mocked(useOAuthConnect)).toHaveBeenCalledWith(+ expect.objectContaining({ provider: "codex" }),+ );+ });+});diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/useOAuthConnect.test.ts b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/useOAuthConnect.test.tsindex 9bb8375284..77449668b2 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/useOAuthConnect.test.ts+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/__tests__/useOAuthConnect.test.ts@@ -208,6 +208,7 @@ describe("useOAuthConnect — popup window lifecycle", () => {data: {login_url: "https://github.com/login/oauth",state_token: "state-token",+ cancel_url: "/api/oauth/pending/cancel",},} as never;});@@ -225,6 +226,7 @@ describe("useOAuthConnect — popup window lifecycle", () => {"https://github.com/login/oauth",expect.objectContaining({stateToken: "state-token",+ cancelUrl: "/api/oauth/pending/cancel",preOpenedWindow: fakeWindow,useCrossOriginListeners: true,}),diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/useOAuthConnect.ts b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/useOAuthConnect.tsindex e538be752f..4047d1be97 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/useOAuthConnect.ts+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/components/DetailView/useOAuthConnect.ts@@ -71,7 +71,7 @@ export function useOAuthConnect({ provider, onSuccess }: Args) {if (initiateResponse.status !== 200) {throw new Error("Unexpected OAuth initiate response");}- const { login_url, state_token } = initiateResponse.data;+ const { login_url, state_token, cancel_url } = initiateResponse.data;// Unmounted while the login URL was being fetched — the cleanup// already closed the window; don't adopt it.@@ -80,7 +80,9 @@ export function useOAuthConnect({ provider, onSuccess }: Args) {const { promise, cleanup, popupBlocked, fallbackBlocked } =openOAuthPopup(login_url, {stateToken: state_token,+ cancelUrl: cancel_url,preOpenedWindow,+ timeout: provider === "codex" ? 15 * 60 * 1000 : undefined,// BroadcastChannel + localStorage listeners are the only delivery// path when the flow runs in a tab without window.opener (the iOS// fallback) — the callback page already writes to both.diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/helpers.ts b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/helpers.tsindex 8754480c58..f1c3d42273 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/helpers.ts+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/components/ConnectServiceDialog/helpers.ts@@ -12,6 +12,8 @@ export interface ConnectableProvider {name: string;description?: string | null;supportedAuthTypes: AuthMethod[];+ authProviderByType?: Partial<Record<AuthMethod, string>>;+ searchTerms?: string[];}const KNOWN_AUTH_METHODS: ReadonlySet<AuthMethod> = new Set(@@ -29,17 +31,53 @@ export function toConnectableProviders(metadata: ProviderMetadata[],): ConnectableProvider[] {const seen = new Set<string>();- const result: ConnectableProvider[] = [];+ const byDisplayProvider = new Map<string, ConnectableProvider>();for (const item of metadata) {if (seen.has(item.name)) continue;seen.add(item.name);- result.push({- id: item.name,- name: formatProviderName(item.name),++ const displayProvider = item.name === "codex" ? "openai" : item.name;+ const authTypes = normalizeAuthTypes(item.supported_auth_types);+ const existing = byDisplayProvider.get(displayProvider);+ const provider = existing ?? {+ id: displayProvider,+ name: formatProviderName(displayProvider),description: item.description,- supportedAuthTypes: normalizeAuthTypes(item.supported_auth_types),- });+ supportedAuthTypes: [],+ };++ for (const authType of authTypes) {+ const alreadySupported = provider.supportedAuthTypes.includes(authType);+ if (!alreadySupported) {+ provider.supportedAuthTypes.push(authType);+ }+ if (item.name === displayProvider) {+ delete provider.authProviderByType?.[authType];+ } else if (!alreadySupported) {+ provider.authProviderByType = {+ ...provider.authProviderByType,+ [authType]: item.name,+ };+ }+ }+ if (item.name !== displayProvider) {+ provider.searchTerms = Array.from(+ new Set([...(provider.searchTerms ?? []), item.name]),+ );+ }+ if (item.name === displayProvider) {+ provider.description = item.description;+ }+ byDisplayProvider.set(displayProvider, provider);}++ const openai = byDisplayProvider.get("openai");+ if (openai?.authProviderByType?.oauth2 === "codex") {+ openai.description =+ "OpenAI models via API key or your ChatGPT subscription";+ }++ const result = Array.from(byDisplayProvider.values());result.sort((a, b) => a.name.localeCompare(b.name));return result;}@@ -57,6 +95,7 @@ export function filterConnectableProviders(return providers.filter((p) => {if (normalize(p.name).includes(q)) return true;if (normalize(p.id).includes(q)) return true;+ if (p.searchTerms?.some((term) => normalize(term).includes(q))) return true;if (p.description && normalize(p.description).includes(q)) return true;return false;});diff --git a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/helpers.ts b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/helpers.tsindex 22c068c612..c5a5717032 100644--- a/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/helpers.ts+++ b/autogpt_platform/frontend/src/components/contextual/IntegrationsPanel/helpers.ts@@ -100,9 +100,11 @@ export function groupCredentialsByProvider(): ProviderGroupView[] {const byProvider = new Map<string, CredentialView[]>();for (const cred of credentials) {- const list = byProvider.get(cred.provider) ?? [];+ const displayProvider =+ cred.provider === "codex" ? "openai" : cred.provider;+ const list = byProvider.get(displayProvider) ?? [];list.push(toCredentialView(cred));- byProvider.set(cred.provider, list);+ byProvider.set(displayProvider, list);}const groups: ProviderGroupView[] = [];diff --git a/autogpt_platform/frontend/src/components/renderers/InputRenderer/__tests__/CodeGenerationTransport.test.tsx b/autogpt_platform/frontend/src/components/renderers/InputRenderer/__tests__/CodeGenerationTransport.test.tsxnew file mode 100644index 0000000000..a024c072dd--- /dev/null+++ b/autogpt_platform/frontend/src/components/renderers/InputRenderer/__tests__/CodeGenerationTransport.test.tsx@@ -0,0 +1,209 @@+import { BlockUIType } from "@/app/(platform)/build/components/types";+import { FormCreator } from "@/app/(platform)/build/components/FlowEditor/nodes/FormCreator";+import type { CustomNode } from "@/app/(platform)/build/components/FlowEditor/nodes/CustomNode/CustomNode";+import { useEdgeStore } from "@/app/(platform)/build/stores/edgeStore";+import { useHistoryStore } from "@/app/(platform)/build/stores/historyStore";+import { useNodeStore } from "@/app/(platform)/build/stores/nodeStore";+import {+ CredentialsProvidersContext,+ type CredentialsProviderData,+} from "@/providers/agent-credentials/credentials-provider";+import { render } from "@/tests/integrations/test-utils";+import { fireEvent, screen, waitFor } from "@testing-library/react";+import type { RJSFSchema } from "@rjsf/utils";+import React from "react";+import { beforeEach, describe, expect, it, vi } from "vitest";++vi.mock("@/components/atoms/Select/Select", () => {+ function Select({+ id,+ value,+ onValueChange,+ options,+ }: {+ id: string;+ value?: string;+ onValueChange?: (value: string) => void;+ options: { value: string; label: string }[];+ }) {+ return (+ <select+ aria-label={id}+ value={value ?? ""}+ onChange={(event) => onValueChange?.(event.target.value)}+ >+ {options.map((option) => (+ <option key={option.value} value={option.value}>+ {option.label}+ </option>+ ))}+ </select>+ );+ }++ return { Select };+});++const codeGenerationSchema = {+ type: "object",+ properties: {+ prompt: {+ advanced: false,+ title: "Prompt",+ type: "string",+ },+ system_prompt: {+ advanced: true,+ default: "You are Codex.",+ title: "System Prompt",+ type: "string",+ },+ transport: {+ advanced: false,+ default: "openai_api",+ enum: ["openai_api", "codex_app_server"],+ title: "Transport",+ type: "string",+ },+ model: {+ advanced: false,+ default: "gpt-5.3-codex",+ enum: ["gpt-5.3-codex", "gpt-5.1-codex"],+ title: "Codex Model",+ type: "string",+ },+ reasoning_effort: {+ advanced: true,+ default: "medium",+ enum: ["none", "low", "medium", "high", "xhigh"],+ title: "Reasoning Effort",+ type: "string",+ },+ credentials: {+ credentials_provider: ["openai", "codex"],+ credentials_types: ["api_key", "oauth2"],+ discriminator: "transport",+ discriminator_mapping: {+ openai_api: "openai",+ codex_app_server: "codex",+ },+ discriminator_type_mapping: {+ openai_api: ["api_key"],+ codex_app_server: ["oauth2"],+ },+ properties: {+ id: { type: "string" },+ provider: { enum: ["openai", "codex"], type: "string" },+ type: { enum: ["api_key", "oauth2"], type: "string" },+ },+ required: ["id", "provider", "type"],+ title: "Credentials",+ type: "object",+ },+ },+ required: ["prompt", "credentials"],+} as unknown as RJSFSchema;++const codexCredential = {+ id: "codex-oauth-1",+ provider: "codex",+ type: "oauth2" as const,+ title: "ChatGPT for Codex",+ username: "[email protected]",+ scopes: [],+};++function makeProvider(+ provider: string,+ providerName: string,+ savedCredentials: CredentialsProviderData["savedCredentials"],+): CredentialsProviderData {+ return {+ provider,+ providerName,+ savedCredentials,+ isSystemProvider: false,+ oAuthCallback: async () => codexCredential,+ mcpOAuthCallback: async () => codexCredential,+ createAPIKeyCredentials: async () => codexCredential,+ createUserPasswordCredentials: async () => codexCredential,+ createHostScopedCredentials: async () => codexCredential,+ deleteCredentials: async () => ({ deleted: true, revoked: null }),+ };+}++function createCodeGenerationNode(): CustomNode {+ return {+ id: "code-generation-node",+ type: "custom",+ position: { x: 0, y: 0 },+ data: {+ hardcodedValues: {+ prompt: "Write a hello-world function",+ transport: "openai_api",+ },+ title: "Code Generation",+ description: "Generate code with Codex",+ inputSchema: codeGenerationSchema,+ outputSchema: {},+ uiType: BlockUIType.STANDARD,+ block_id: "86a2a099-30df-47b4-b7e4-34ae5f83e0d5",+ costs: [],+ categories: [],+ },+ };+}++beforeEach(() => {+ useNodeStore.setState({+ nodes: [createCodeGenerationNode()],+ nodeAdvancedStates: {},+ });+ useEdgeStore.setState({ edges: [] });+ useHistoryStore.setState({ past: [], future: [] });+});++describe("Code Generation transport fields", () => {+ it("keeps Transport visible and selects the saved Codex OAuth credential", async () => {+ const providers = {+ openai: makeProvider("openai", "OpenAI", []),+ codex: makeProvider("codex", "Codex", [codexCredential]),+ };++ render(+ <CredentialsProvidersContext.Provider value={providers}>+ <FormCreator+ jsonSchema={codeGenerationSchema}+ nodeId="code-generation-node"+ uiType={BlockUIType.STANDARD}+ showHandles={false}+ />+ </CredentialsProvidersContext.Provider>,+ );++ expect(screen.getByText("Transport")).toBeDefined();+ expect(screen.queryByText("System Prompt")).toBeNull();+ expect(screen.queryByText("Reasoning Effort")).toBeNull();++ fireEvent.change(screen.getByLabelText("agpt_%_transport"), {+ target: { value: "1" },+ });++ expect(await screen.findByText("ChatGPT for Codex")).toBeDefined();+ expect(screen.getByText("OAuth")).toBeDefined();++ await waitFor(() => {+ expect(+ useNodeStore.getState().getHardCodedValues("code-generation-node"),+ ).toMatchObject({+ transport: "codex_app_server",+ credentials: {+ id: "codex-oauth-1",+ provider: "codex",+ type: "oauth2",+ title: "ChatGPT for Codex",+ },+ });+ });+ });+});diff --git a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/CredentialField/helpers.ts b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/CredentialField/helpers.tsindex fd5c07443b..090959953e 100644--- a/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/CredentialField/helpers.ts+++ b/autogpt_platform/frontend/src/components/renderers/InputRenderer/custom/CredentialField/helpers.ts@@ -66,6 +66,7 @@ export function toDisplayName(provider: string): string {d_id: "D-ID",e2b: "E2B",llama_api: "Llama API",+ codex: "OpenAI",open_router: "Open Router",smtp: "SMTP",revid: "Rev.ID",diff --git a/autogpt_platform/frontend/src/hooks/__tests__/useCredentials.test.ts b/autogpt_platform/frontend/src/hooks/__tests__/useCredentials.test.tsnew file mode 100644index 0000000000..40eb3d6d11--- /dev/null+++ b/autogpt_platform/frontend/src/hooks/__tests__/useCredentials.test.ts@@ -0,0 +1,37 @@+import type { BlockIOCredentialsSubSchema } from "@/lib/autogpt-server-api";+import { describe, expect, it } from "vitest";+import { getSupportedCredentialTypes } from "../useCredentials";++const schema: BlockIOCredentialsSubSchema = {+ type: "object",+ properties: {},+ credentials_provider: ["openai", "codex"],+ credentials_types: ["api_key", "oauth2"],+ discriminator: "transport",+ discriminator_mapping: {+ openai_api: "openai",+ codex_app_server: "codex",+ },+ discriminator_type_mapping: {+ openai_api: ["api_key"],+ codex_app_server: ["oauth2"],+ },+};++describe("getSupportedCredentialTypes", () => {+ it("narrows each transport to its valid credential type", () => {+ expect(getSupportedCredentialTypes(schema, "openai_api")).toEqual([+ "api_key",+ ]);+ expect(getSupportedCredentialTypes(schema, "codex_app_server")).toEqual([+ "oauth2",+ ]);+ });++ it("falls back to the schema types for legacy discriminators", () => {+ expect(getSupportedCredentialTypes(schema, "unknown")).toEqual([+ "api_key",+ "oauth2",+ ]);+ });+});diff --git a/autogpt_platform/frontend/src/hooks/useCredentials.ts b/autogpt_platform/frontend/src/hooks/useCredentials.tsindex 21c0e425ad..8c6076fc14 100644--- a/autogpt_platform/frontend/src/hooks/useCredentials.ts+++ b/autogpt_platform/frontend/src/hooks/useCredentials.ts@@ -86,6 +86,19 @@ export type CredentialsData =upgradeableCredentials: CredentialsMetaResponse[];});+export function getSupportedCredentialTypes(+ schema: BlockIOCredentialsSubSchema,+ discriminatorValue: string | undefined,+) {+ if (schema.discriminator_type_mapping) {+ return (+ schema.discriminator_type_mapping[discriminatorValue ?? ""] ??+ schema.credentials_types+ );+ }+ return schema.credentials_types;+}+export default function useCredentials(credsInputSchema: BlockIOCredentialsSubSchema,nodeInputValues?: Record<string, any>,@@ -128,12 +141,18 @@ export default function useCredentials(return null;}- const supportsApiKey = credsInputSchema.credentials_types.includes("api_key");- const supportsOAuth2 = credsInputSchema.credentials_types.includes("oauth2");- const supportsUserPassword =- credsInputSchema.credentials_types.includes("user_password");- const supportsHostScoped =- credsInputSchema.credentials_types.includes("host_scoped");+ const supportedTypes = getSupportedCredentialTypes(+ credsInputSchema,+ discriminatorValue,+ );+ const effectiveSchema = {+ ...credsInputSchema,+ credentials_types: supportedTypes,+ };+ const supportsApiKey = supportedTypes.includes("api_key");+ const supportsOAuth2 = supportedTypes.includes("oauth2");+ const supportsUserPassword = supportedTypes.includes("user_password");+ const supportsHostScoped = supportedTypes.includes("host_scoped");// No provider means maybe it's still loadingif (!provider) {@@ -142,14 +161,14 @@ export default function useCredentials(const { savedCredentials, upgradeableCredentials } = classifyCredentials(provider.savedCredentials,- credsInputSchema,+ effectiveSchema,discriminatorValue,);return {...provider,provider: providerName,- schema: credsInputSchema,+ schema: effectiveSchema,supportsApiKey,supportsOAuth2,supportsUserPassword,diff --git a/autogpt_platform/frontend/src/lib/autogpt-server-api/client.ts b/autogpt_platform/frontend/src/lib/autogpt-server-api/client.tsindex 12ac65d9f5..0150d20e85 100644--- a/autogpt_platform/frontend/src/lib/autogpt-server-api/client.ts+++ b/autogpt_platform/frontend/src/lib/autogpt-server-api/client.ts@@ -322,7 +322,11 @@ export default class BackendAPI {provider: string,scopes?: string[],credentialID?: string,- ): Promise<{ login_url: string; state_token: string }> {+ ): Promise<{+ login_url: string;+ state_token: string;+ cancel_url?: string | null;+ }> {return this._get(`/integrations/${provider}/login`,buildOAuthLoginQuery(scopes, credentialID),diff --git a/autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts b/autogpt_platform/frontend/src/lib/autogpt-server-api/types.tsindex 45acf934d2..92145eb1b3 100644--- a/autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts+++ b/autogpt_platform/frontend/src/lib/autogpt-server-api/types.ts@@ -220,6 +220,7 @@ export type BlockIOCredentialsSubSchema = BlockIOObjectSubSchema & {credentials_types: Array<CredentialsType>;discriminator?: string;discriminator_mapping?: Record<string, CredentialsProviderName>;+ discriminator_type_mapping?: Record<string, CredentialsType[]>;discriminator_values?: any[];secret?: boolean;};diff --git a/autogpt_platform/frontend/src/lib/oauth-popup.test.ts b/autogpt_platform/frontend/src/lib/oauth-popup.test.tsindex 848d8b3762..f87a56a819 100644--- a/autogpt_platform/frontend/src/lib/oauth-popup.test.ts+++ b/autogpt_platform/frontend/src/lib/oauth-popup.test.ts@@ -32,6 +32,7 @@ describe("openOAuthPopup popup-close grace window", () => {afterEach(() => {vi.useRealTimers();vi.restoreAllMocks();+ vi.unstubAllGlobals();});test("rejects with WINDOW_CLOSED after grace if no result arrives", async () => {@@ -214,6 +215,101 @@ describe("openOAuthPopup popup-close grace window", () => {expect((onReject.mock.calls[0][0] as Error).message).toMatch(/timed out/i);});+ test("timeout cancels provider-side pending state exactly once", async () => {+ const popup = makePopupStub();+ setupPopup(popup);+ const fetchMock = vi+ .fn()+ .mockResolvedValue(new Response(null, { status: 204 }));+ vi.stubGlobal("fetch", fetchMock);++ const { promise } = openOAuthPopup("https://example.com/oauth", {+ stateToken: "tok-server-timeout",+ cancelUrl: "/api/oauth/pending/cancel",+ timeout: 1000,+ });+ promise.catch(() => {});++ await vi.advanceTimersByTimeAsync(1100);++ expect(fetchMock).toHaveBeenCalledTimes(1);+ expect(fetchMock).toHaveBeenCalledWith(+ expect.stringContaining("/api/oauth/pending/cancel"),+ expect.objectContaining({ method: "POST", keepalive: true }),+ );+ });++ test("manual abort cancels provider-side pending state exactly once", async () => {+ const popup = makePopupStub();+ setupPopup(popup);+ const fetchMock = vi+ .fn()+ .mockResolvedValue(new Response(null, { status: 204 }));+ vi.stubGlobal("fetch", fetchMock);++ const { promise, cleanup } = openOAuthPopup("https://example.com/oauth", {+ stateToken: "tok-server-abort",+ cancelUrl: "/api/oauth/pending/cancel",+ });+ promise.catch(() => {});++ cleanup.abort();+ cleanup.abort();+ await vi.runAllTicks();++ expect(fetchMock).toHaveBeenCalledTimes(1);+ expect(popup.close).toHaveBeenCalledTimes(1);+ });++ test("popup close cancels provider-side pending state exactly once", async () => {+ const popup = makePopupStub();+ setupPopup(popup);+ const fetchMock = vi+ .fn()+ .mockResolvedValue(new Response(null, { status: 204 }));+ vi.stubGlobal("fetch", fetchMock);++ const { promise, cleanup } = openOAuthPopup("https://example.com/oauth", {+ stateToken: "tok-server-close",+ cancelUrl: "/api/oauth/pending/cancel",+ });+ promise.catch(() => {});++ popup.closed = true;+ await vi.advanceTimersByTimeAsync(500);+ cleanup.abort();++ expect(fetchMock).toHaveBeenCalledTimes(1);+ });++ test("successful result does not cancel provider-side state", async () => {+ const popup = makePopupStub();+ setupPopup(popup);+ const fetchMock = vi.fn();+ vi.stubGlobal("fetch", fetchMock);++ const { promise } = openOAuthPopup("https://example.com/oauth", {+ stateToken: "tok-success",+ cancelUrl: "/api/oauth/pending/cancel",+ });+ window.dispatchEvent(+ new MessageEvent("message", {+ data: {+ message_type: "oauth_popup_result",+ success: true,+ code: "auth-code",+ state: "tok-success",+ },+ }),+ );++ await expect(promise).resolves.toEqual({+ code: "auth-code",+ state: "tok-success",+ });+ expect(fetchMock).not.toHaveBeenCalled();+ });+test("state-mismatch message is ignored and does not resolve the promise", async () => {const popup = makePopupStub();setupPopup(popup);@@ -255,6 +351,7 @@ describe("preOpenedWindow option", () => {afterEach(() => {vi.useRealTimers();vi.restoreAllMocks();+ vi.unstubAllGlobals();});test("navigates the pre-opened window instead of calling window.open again", () => {@@ -304,7 +401,8 @@ describe("preOpenedWindow option", () => {});test("null preOpenedWindow goes straight to the new-tab fallback", () => {- const openSpy = setupPopup(makePopupStub());+ const fallback = makePopupStub();+ const openSpy = setupPopup(fallback);const { promise, cleanup, popupBlocked, fallbackBlocked } = openOAuthPopup("https://example.com/oauth",@@ -322,6 +420,7 @@ describe("preOpenedWindow option", () => {expect(openSpy).toHaveBeenCalledWith("https://example.com/oauth", "_blank");cleanup.abort();+ expect(fallback.close).toHaveBeenCalledOnce();});test("preOpenOAuthPopup opens a blank popup window", () => {@@ -360,6 +459,26 @@ describe("preOpenedWindow option", () => {);});+ test("blocked popup and fallback cancel provider-side state exactly once", async () => {+ setupPopup(null);+ const fetchMock = vi+ .fn()+ .mockResolvedValue(new Response(null, { status: 204 }));+ vi.stubGlobal("fetch", fetchMock);++ const { promise, cleanup } = openOAuthPopup("https://example.com/oauth", {+ stateToken: "tok-server-blocked",+ preOpenedWindow: null,+ cancelUrl: "/api/oauth/pending/cancel",+ });+ promise.catch(() => {});++ cleanup.abort();+ await vi.runAllTicks();++ expect(fetchMock).toHaveBeenCalledTimes(1);+ });+test("rejects immediately when window.open is blocked for both attempts", async () => {// No preOpenedWindow — the inline open is blocked, and so is the// fallback (e.g. aggressive popup blocker). Must not wait for timeout.diff --git a/autogpt_platform/frontend/src/lib/oauth-popup.ts b/autogpt_platform/frontend/src/lib/oauth-popup.tsindex ac1a83cc8e..5641ffabbc 100644--- a/autogpt_platform/frontend/src/lib/oauth-popup.ts+++ b/autogpt_platform/frontend/src/lib/oauth-popup.ts@@ -49,6 +49,8 @@ export type OAuthPopupOptions = {acceptMessageTypes?: string[];/** Timeout in ms (default: 5 minutes) */timeout?: number;+ /** Same-origin endpoint that cancels provider-side pending state on abort */+ cancelUrl?: string | null;};type Cleanup = {@@ -115,6 +117,7 @@ export function openOAuthPopup(localStorageKey = "oauth_popup_result",acceptMessageTypes = ["oauth_popup_result", "mcp_oauth_result"],timeout = DEFAULT_TIMEOUT_MS,+ cancelUrl,} = options;const controller = new AbortController();@@ -126,6 +129,7 @@ export function openOAuthPopup(// straight from a click handler.const popup =preOpenedWindow !== undefined ? preOpenedWindow : preOpenOAuthPopup();+ let activeWindow = popup;let popupBlocked = false;let fallbackBlocked = false;@@ -136,6 +140,7 @@ export function openOAuthPopup(// still complete via postMessage / BroadcastChannel / localStorage poll.popupBlocked = true;const fallback = window.open(loginUrl, "_blank");+ activeWindow = fallback;// The fallback open can be blocked too — iOS Safari blocks every// window.open() after an async break, so when the caller pre-opened// (and lost) the window, this open has no gesture context either. No@@ -143,9 +148,25 @@ export function openOAuthPopup(fallbackBlocked = !fallback || fallback.closed;}- // Close popup on abort+ let cancelRequested = false;+ function cancelServerAttempt() {+ if (!cancelUrl || cancelRequested) return;+ cancelRequested = true;+ try {+ const url = new URL(cancelUrl, window.location.origin);+ if (url.origin !== window.location.origin) return;+ void fetch(url.toString(), {+ method: "POST",+ credentials: "same-origin",+ keepalive: true,+ }).catch(() => {});+ } catch {}+ }++ // Close popup and cancel provider-side pending state on abortcontroller.signal.addEventListener("abort", () => {- if (popup && !popup.closed) popup.close();+ if (controller.signal.reason !== "completed") cancelServerAttempt();+ if (activeWindow && !activeWindow.closed) activeWindow.close();});// Scope the localStorage key by stateToken so concurrent OAuth flows do@@ -172,6 +193,7 @@ export function openOAuthPopup(if (fallbackBlocked) {handled = true;reject(new Error(OAUTH_ERROR_POPUP_BLOCKED_NO_TAB));+ controller.abort("popup_blocked");return;}@@ -269,13 +291,11 @@ export function openOAuthPopup(// The setTimeout lives in a plain JS closure, not React state — it// does NOT stack across re-renders, and the AbortController cleanup// tears it down if the caller aborts before grace expires.- // Skip the close-poll entirely when the popup was blocked. The- // ``window.open("about:blank", ...)`` reference can be non-null but- // already-closed in that branch (we fell back to a separate new-tab- // ``window.open(loginUrl, "_blank")`` whose handle we deliberately- // don't keep — the new tab is the user's primary surface and- // pre-emptively rejecting on its ``closed`` state would short-circuit- // a successful sign-in arriving via BroadcastChannel/localStorage.+ // Skip the close-poll entirely when the popup was blocked. The fallback+ // WindowProxy is retained so an explicit abort can close it, but COOP can+ // make that handle appear closed while the new tab is still usable.+ // Polling it would therefore risk rejecting a successful sign-in before+ // its BroadcastChannel/localStorage result arrives.if (popup && !popupBlocked) {// Grace window only applies to cross-origin flows. Same-origin// OAuth resolves via ``window.opener.postMessage`` which thediff --git a/autogpt_platform/frontend/src/providers/agent-credentials/__tests__/credentials-provider-auth.test.tsx b/autogpt_platform/frontend/src/providers/agent-credentials/__tests__/credentials-provider-auth.test.tsxnew file mode 100644index 0000000000..786476cdcc--- /dev/null+++ b/autogpt_platform/frontend/src/providers/agent-credentials/__tests__/credentials-provider-auth.test.tsx@@ -0,0 +1,148 @@+import { QueryClient, QueryClientProvider } from "@tanstack/react-query";+import { act, render, screen, waitFor } from "@testing-library/react";+import { useContext, type ReactNode } from "react";+import { beforeEach, describe, expect, it, vi } from "vitest";++const { apiMock, errorHandlerMock, isLoggedIn, onFailToastMock, useAuthMock } =+ vi.hoisted(() => {+ const errorHandler = vi.fn();+ return {+ apiMock: {+ listCredentials: vi.fn().mockResolvedValue([]),+ listProviders: vi.fn().mockResolvedValue(["google"]),+ listSystemProviders: vi.fn().mockResolvedValue([]),+ },+ errorHandlerMock: errorHandler,+ isLoggedIn: { value: false },+ onFailToastMock: vi.fn(() => errorHandler),+ useAuthMock: vi.fn(),+ };+ });++vi.mock("@/lib/auth/hooks/useAuth", () => ({+ useAuth: useAuthMock,+}));++vi.mock("@/lib/autogpt-server-api/context", () => ({+ useBackendAPI: () => apiMock,+}));++vi.mock("@/components/molecules/Toast/use-toast", () => ({+ useToastOnFail: () => onFailToastMock,+}));++import CredentialsProvider, {+ CredentialsProvidersContext,+} from "../credentials-provider";++const queryClient = new QueryClient();++interface Props {+ children: ReactNode;+}++function TestWrapper({ children }: Props) {+ return (+ <QueryClientProvider client={queryClient}>{children}</QueryClientProvider>+ );+}++function ProviderState() {+ const providers = useContext(CredentialsProvidersContext);+ return (+ <span data-testid="provider-state">+ {providers === null ? "loading" : Object.keys(providers).join(",")}+ </span>+ );+}++function deferred<T>() {+ let resolve!: (value: T) => void;+ const promise = new Promise<T>((resolvePromise) => {+ resolve = resolvePromise;+ });+ return { promise, resolve };+}++describe("CredentialsProvider authentication", () => {+ beforeEach(() => {+ vi.clearAllMocks();+ queryClient.clear();+ isLoggedIn.value = false;+ apiMock.listCredentials.mockResolvedValue([]);+ apiMock.listProviders.mockResolvedValue(["google"]);+ apiMock.listSystemProviders.mockResolvedValue([]);+ useAuthMock.mockImplementation(() => ({+ isLoggedIn: isLoggedIn.value,+ }));+ });++ it("waits for authentication before loading protected provider metadata", async () => {+ const view = render(+ <CredentialsProvider>+ <span>app content</span>+ </CredentialsProvider>,+ { wrapper: TestWrapper },+ );++ expect(screen.getByText("app content")).toBeDefined();+ await act(async () => undefined);+ expect(apiMock.listProviders).not.toHaveBeenCalled();+ expect(apiMock.listSystemProviders).not.toHaveBeenCalled();+ expect(errorHandlerMock).not.toHaveBeenCalled();++ isLoggedIn.value = true;+ view.rerender(+ <CredentialsProvider>+ <span>app content</span>+ </CredentialsProvider>,+ );++ await waitFor(() => {+ expect(apiMock.listProviders).toHaveBeenCalledOnce();+ expect(apiMock.listSystemProviders).toHaveBeenCalledOnce();+ expect(apiMock.listCredentials).toHaveBeenCalledOnce();+ });+ });++ it("discards provider metadata that resolves after logout", async () => {+ const providersRequest = deferred<string[]>();+ const systemProvidersRequest = deferred<string[]>();+ apiMock.listProviders.mockReturnValueOnce(providersRequest.promise);+ apiMock.listSystemProviders.mockReturnValueOnce(+ systemProvidersRequest.promise,+ );+ isLoggedIn.value = true;++ const view = render(+ <CredentialsProvider>+ <ProviderState />+ </CredentialsProvider>,+ { wrapper: TestWrapper },+ );++ await waitFor(() => {+ expect(apiMock.listProviders).toHaveBeenCalledOnce();+ expect(apiMock.listSystemProviders).toHaveBeenCalledOnce();+ });++ isLoggedIn.value = false;+ view.rerender(+ <CredentialsProvider>+ <ProviderState />+ </CredentialsProvider>,+ );++ await act(async () => {+ providersRequest.resolve(["google"]);+ systemProvidersRequest.resolve(["google"]);+ await Promise.all([+ providersRequest.promise,+ systemProvidersRequest.promise,+ ]);+ });++ expect(screen.getByTestId("provider-state").textContent).toBe("");+ expect(apiMock.listCredentials).not.toHaveBeenCalled();+ });+});diff --git a/autogpt_platform/frontend/src/providers/agent-credentials/credentials-provider.tsx b/autogpt_platform/frontend/src/providers/agent-credentials/credentials-provider.tsxindex ad5f928a05..43c3dc0325 100644--- a/autogpt_platform/frontend/src/providers/agent-credentials/credentials-provider.tsx+++ b/autogpt_platform/frontend/src/providers/agent-credentials/credentials-provider.tsx@@ -353,15 +353,29 @@ export default function CredentialsProvider({onFailToast,]);- // Fetch provider names and system providers on mount+ // Fetch provider names and system providers after authenticationuseEffect(() => {+ if (!isLoggedIn) {+ setProviderNames([]);+ setSystemProviders(new Set());+ return;+ }++ let isCurrent = true;Promise.all([api.listProviders(), api.listSystemProviders()]).then(([names, systemList]) => {+ if (!isCurrent) return;setProviderNames(names);setSystemProviders(new Set(systemList));})- .catch(onFailToast("Load provider names"));- }, [api, onFailToast]);+ .catch((error) => {+ if (isCurrent) onFailToast("Load provider names")(error);+ });++ return () => {+ isCurrent = false;+ };+ }, [api, isLoggedIn, onFailToast]);useEffect(() => {loadCredentials();diff --git a/autogpt_platform/frontend/src/providers/agent-credentials/helper.ts b/autogpt_platform/frontend/src/providers/agent-credentials/helper.tsindex 94661574ff..dfe430037c 100644--- a/autogpt_platform/frontend/src/providers/agent-credentials/helper.ts+++ b/autogpt_platform/frontend/src/providers/agent-credentials/helper.ts@@ -9,6 +9,7 @@ export function toDisplayName(provider: unknown): string {d_id: "D-ID",e2b: "E2B",llama_api: "Llama API",+ codex: "OpenAI",open_router: "Open Router",smtp: "SMTP",revid: "Rev.ID",diff --git a/autogpt_platform/single-container/README.md b/autogpt_platform/single-container/README.mdindex 4d87ab0dce..bea98c23f5 100644--- a/autogpt_platform/single-container/README.md+++ b/autogpt_platform/single-container/README.md@@ -51,6 +51,10 @@ The image supports `linux/amd64` and `linux/arm64`. Test installations usedabout 5–6 GiB of memory during startup and steady-state health checks, thoughactual usage depends on enabled services and workloads.+ChatGPT/Codex temporary auth homes use `/dev/shm/autogpt-codex` automatically.+The quick-start command's `--shm-size 2g` keeps that credential material in+memory rather than the container's writable layer.+## Tags- `latest` — most recent fully verified AutoGPT Platform release.diff --git a/autogpt_platform/single-container/entrypoint.sh b/autogpt_platform/single-container/entrypoint.shindex 5f29f4199f..11260d5492 100755--- a/autogpt_platform/single-container/entrypoint.sh+++ b/autogpt_platform/single-container/entrypoint.sh@@ -7,6 +7,8 @@ source "${AUTOGPT_ASSET_DIR:-/opt/autogpt/single-container}/common.sh"readonly AUTOGPT_PYTHON="${AUTOGPT_PYTHON:-${AUTOGPT_BACKEND_DIR}/.venv/bin/python}"readonly POSTGRES_BINDIR="${POSTGRES_BINDIR:-/usr/lib/postgresql/15/bin}"+readonly CODEX_TEMP_ROOT=/dev/shm/autogpt-codex+export CODEX_TEMP_ROOTmain() {[[ "$(id -u)" -eq 0 ]] || fatal "entrypoint must start as root so services can drop privileges"@@ -39,7 +41,8 @@ prepare_directories() {/data/config /data/postgres /data/rabbitmq /data/valkey \/data/valkey/17000 /data/valkey/17001 /data/valkey/17002 /data/falkordb \/data/workspaces /data/home /data/frontend-home \- /data/cache /data/cache/backend /data/cache/next; do+ /data/cache /data/cache/backend /data/cache/next \+ "${CODEX_TEMP_ROOT}"; do[[ ! -L "${managed_path}" ]] || fatal "refusing symlink at managed data path: ${managed_path}"doneinstall -d -m 0710 -o root -g autogpt /data/config@@ -56,6 +59,7 @@ prepare_directories() {install -d -m 0711 -o root -g root /data/cacheinstall -d -m 0750 -o autogpt -g autogpt /data/cache/backendinstall -d -m 0700 -o autogpt_frontend -g autogpt_frontend /data/cache/next+ install -d -m 0700 -o autogpt -g autogpt "${CODEX_TEMP_ROOT}"install -d -m 0755 -o postgres -g postgres /run/postgresql# Service-specific runtime directories and files carry the restrictive# permissions. Keep only execute permission on their common parent so the@@ -104,6 +108,7 @@ configure_environment() {export AUTOGPT_PUBLIC_URLlog "public URL: ${AUTOGPT_PUBLIC_URL}"configure_account_registration+ configure_backend_cors_originwrite_nginx_public_url_configexport PGDATA=/data/postgres@@ -171,6 +176,10 @@ configure_environment() {export PYTHONPATH="${AUTOGPT_ASSET_DIR}/python"}+configure_backend_cors_origin() {+ export BACKEND_CORS_ALLOW_ORIGINS="[\"${AUTOGPT_PUBLIC_URL}\"]"+}+configure_account_registration() {normalize_toggle AUTH_ALLOW_NEW_ACCOUNTS trueif [[ "${AUTH_ALLOW_NEW_ACCOUNTS}" == true ]]; thendiff --git a/autogpt_platform/single-container/tests/test_entrypoint.py b/autogpt_platform/single-container/tests/test_entrypoint.pyindex 49dfe5de18..42890e4c0d 100644--- a/autogpt_platform/single-container/tests/test_entrypoint.py+++ b/autogpt_platform/single-container/tests/test_entrypoint.py@@ -114,6 +114,34 @@ class AccountRegistrationTest(unittest.TestCase):)+class PublicOriginConfigurationTest(unittest.TestCase):+ def test_backend_cors_uses_the_validated_public_origin(self) -> None:+ result = subprocess.run(+ [+ "bash",+ "-Eeuo",+ "pipefail",+ "-c",+ 'source "$1"; AUTOGPT_PUBLIC_URL="$2"; '+ 'configure_backend_cors_origin; '+ 'printf "%s\\n" "$BACKEND_CORS_ALLOW_ORIGINS"',+ "bash",+ str(ENTRYPOINT_PATH),+ "http://192.168.1.254:3300",+ ],+ check=False,+ capture_output=True,+ encoding="utf-8",+ env={+ "PATH": os.environ.get("PATH", "/usr/bin:/bin"),+ "AUTOGPT_ASSET_DIR": str(ASSET_DIR),+ },+ )++ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertEqual(result.stdout, '["http://192.168.1.254:3300"]\n')++class NormalizationTest(unittest.TestCase):def test_rejects_invalid_integer_values(self) -> None:for value, error in (@@ -181,6 +209,38 @@ class ValkeyConfigurationTest(unittest.TestCase):self.assertNotIn("--masterauth", service_runner)+class CodexTemporaryHomeTest(unittest.TestCase):+ def test_defaults_to_private_memory_backed_storage(self) -> None:+ entrypoint = ENTRYPOINT_PATH.read_text(encoding="utf-8")+ result = subprocess.run(+ [+ "bash",+ "-Eeuo",+ "pipefail",+ "-c",+ 'source "$1"; printf "%s\\n" "$CODEX_TEMP_ROOT"',+ "bash",+ str(ENTRYPOINT_PATH),+ ],+ check=False,+ capture_output=True,+ encoding="utf-8",+ env={+ "PATH": os.environ.get("PATH", "/usr/bin:/bin"),+ "AUTOGPT_ASSET_DIR": str(ASSET_DIR),+ "CODEX_TEMP_ROOT": "/data/not-memory-backed",+ },+ )++ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertEqual(result.stdout, "/dev/shm/autogpt-codex\n")+ self.assertIn(+ 'install -d -m 0700 -o autogpt -g autogpt "${CODEX_TEMP_ROOT}"',+ entrypoint,+ )+ self.assertIn('"${CODEX_TEMP_ROOT}"; do', entrypoint)++class ProxyIsolationTest(unittest.TestCase):def test_nginx_uses_a_dedicated_operating_system_user(self) -> None:dockerfile = DOCKERFILE_PATH.read_text(encoding="utf-8")diff --git a/docs/integrations/README.md b/docs/integrations/README.mdindex 93d43631c6..3e3900df74 100644--- a/docs/integrations/README.md+++ b/docs/integrations/README.md@@ -225,7 +225,7 @@ Below is a comprehensive list of all available blocks, categorized by their prim| [AI Video Generator](block-integrations/fal/ai_video_generator.md#ai-video-generator) | Generate videos using FAL AI models || [Bannerbear Text Overlay](block-integrations/bannerbear/text_overlay.md#bannerbear-text-overlay) | Add text overlay to images using Bannerbear templates || [Claude Code](block-integrations/llm.md#claude-code) | Execute tasks using Claude Code in an E2B sandbox |-| [Code Generation](block-integrations/llm.md#code-generation) | Generate or refactor code using OpenAI's Codex (Responses API) |+| [Code Generation](block-integrations/llm.md#code-generation) | Generate or refactor code using an OpenAI API key or a connected ChatGPT plan through Codex App Server || [Create Talking Avatar Video](block-integrations/llm.md#create-talking-avatar-video) | This block integrates with D-ID to create video clips and retrieve their URLs || [Exa Answer](block-integrations/exa/answers.md#exa-answer) | Get an LLM answer to a question informed by Exa search results || [Exa Create Enrichment](block-integrations/exa/websets_enrichment.md#exa-create-enrichment) | Create enrichments to extract additional structured data from webset items |diff --git a/docs/integrations/block-integrations/llm.md b/docs/integrations/block-integrations/llm.mdindex ab51a4460e..bd88410b89 100644--- a/docs/integrations/block-integrations/llm.md+++ b/docs/integrations/block-integrations/llm.md@@ -582,13 +582,13 @@ The block supports conversation continuation through three mechanisms:## Code Generation### What it is-Generate or refactor code using OpenAI's Codex (Responses API).+Generate or refactor code using an OpenAI API key or a connected ChatGPT plan through Codex App Server.### How it works<!-- MANUAL: how_it_works -->-This block uses OpenAI's Codex model optimized for code generation and refactoring. Provide a prompt describing the code you need, and optionally a system prompt with coding guidelines or context.+Choose `openai_api` to call the selected Codex model through the OpenAI Responses API with an OpenAI API key. This path applies `model`, `max_output_tokens` (up to 128,000), and every reasoning effort except `ultra`; `none` omits the reasoning configuration. It rejects non-OpenAI API-key credentials and unsupported `ultra` requests before making an API call.-Configure reasoning_effort to control how much the model "thinks" before responding. The block returns generated code along with any reasoning the model produced.+Choose `codex_app_server` to use a connected ChatGPT plan. This path requires matching Codex OAuth credentials and an active credential lease, selects the subscription model from the live App Server catalog, and uses the plan's output-token limit. The `model` and `max_output_tokens` inputs therefore do not affect App Server requests, while all listed reasoning efforts are forwarded through the App Server mapping. Missing or mismatched credentials fail validation, and transport failures are returned through the block's standard `error` output. Successful responses include the generated text, any reasoning summary, and the transport response ID; usage may be absent without failing the response.<!-- END MANUAL -->### Inputs@@ -596,10 +596,11 @@ Configure reasoning_effort to control how much the model "thinks" before respond| Input | Description | Type | Required ||-------|-------------|------|----------|| prompt | Primary coding request passed to the Codex model. | str | Yes |-| system_prompt | Optional instructions injected via the Responses API instructions field. | str | No |-| model | Codex-optimized model served via the Responses API. | "gpt-5.3-codex" \| "gpt-5.1-codex" | No |-| reasoning_effort | Controls the Responses API reasoning budget. Select 'none' to skip reasoning configs. | "none" \| "low" \| "medium" \| "high" | No |-| max_output_tokens | Upper bound for generated tokens (hard limit 128,000). Leave blank to let OpenAI decide. | int | No |+| system_prompt | Optional instructions passed to the selected Codex transport. | str | No |+| transport | Use an OpenAI API key or your connected ChatGPT plan through Codex App Server. | "openai_api" \| "codex_app_server" | No |+| model | OpenAI API transport only. Codex App Server selects the current subscription model from its live model catalog. | "gpt-5.6-sol" \| "gpt-5.6-terra" \| "gpt-5.6-luna" \| "gpt-5.3-codex" \| "gpt-5.1-codex" | No |+| reasoning_effort | Controls the selected transport's reasoning effort. OpenAI API does not support 'ultra'; select 'none' to omit reasoning config. | "none" \| "low" \| "medium" \| "high" \| "xhigh" \| "max" \| "ultra" | No |+| max_output_tokens | OpenAI API transport only: upper bound for generated tokens (hard limit 128,000). Codex App Server uses its model and plan limits. | int | No |### Outputs@@ -608,7 +609,7 @@ Configure reasoning_effort to control how much the model "thinks" before respond| error | Error message if the operation failed | str || response | Code-focused response returned by the Codex model. | str || reasoning | Reasoning summary returned by the model, if available. | str |-| response_id | ID of the Responses API call for auditing/debugging. | str |+| response_id | Transport response ID for auditing and debugging. | str |### Possible use case<!-- MANUAL: use_case -->diff --git a/docs/platform/codex-subscription-preview.md b/docs/platform/codex-subscription-preview.mdnew file mode 100644index 0000000000..1d07d7e957--- /dev/null+++ b/docs/platform/codex-subscription-preview.md@@ -0,0 +1,214 @@+# ChatGPT/Codex subscription transport preview++This private preview lets a user connect a ChatGPT account and use that+account's Codex allowance for three explicit execution paths:++- The existing **Code Generation** block through the `codex_app_server`+ transport.+- A newly created AutoPilot chat session when **ChatGPT/Codex** is the only+ connected subscription transport, or is selected from multiple connected+ subscription transports.+- An **AutoPilot** graph block with an explicitly selected ChatGPT/Codex+ connection.++`codex` is the credential provider and Codex App Server is the execution+runtime. A ChatGPT plan is not unlimited inference: these calls consume the+connected account's Codex allowance or credits and remain subject to its rate+limits. Subscription-backed usage records tokens and+`billing_mode=user_subscription`; it is not reported as AutoGPT provider USD+spend.++The preview does not route Orchestrator, shared LLM blocks, image generation,+or background system work through the connected account. Existing graphs and+AutoPilot sessions keep their stored route. New AutoPilot sessions use a+connected subscription transport automatically when exactly one is available.++## Run locally++From `autogpt_platform`:++```sh+docker compose up -d --build+```++Codex pairing and its supported execution routes are available without feature+flag environment variables. The stack mounts `/run/autogpt-codex` as a+memory-backed temporary filesystem for REST, executor, and Copilot executor+processes.++Open `http://localhost:3000`, then open **Settings > Integrations** and connect+**Codex**. In the sign-in window, open the ChatGPT verification page and enter+the one-time code.++### Test the Code Generation block++1. Add a **Code Generation** block to a graph.+2. Set **Transport** to **Codex App Server**.+3. Select the connected **ChatGPT for Codex** credential.+4. Run the graph.++Existing graphs continue to default to the OpenAI API transport. The block's+model dropdown applies to that API transport. The subscription transport uses+App Server's current account-compatible default; its live catalog can change+independently of public API model names.++### Test AutoPilot++1. Open **AutoPilot** and start a new task.+2. If ChatGPT/Codex is the only connected subscription transport, it is used+ automatically and no connection selector is shown. If multiple subscription+ transports are connected, choose the desired connection before sending the+ first message.+3. Choose Fast or Thinking and Balanced or Advanced as usual, then send the+ message. Text and permitted AutoGPT tool calls stream through the existing+ AutoPilot event surface.++The route and credential are stored on the new session and are immutable for+that session. Start another task to use a different connected subscription+transport. The server reloads the authoritative session route before each+queued turn; queued messages carry only the credential ID, never the OAuth+tokens. A missing, revoked, or busy Codex credential fails visibly. There is no+silent fallback to an AutoGPT-funded model or another user's credential.++The mode and model controls select a Codex route from the shared model catalog,+then validate it against the models advertised by the connected account. The+preview maps Fast/Balanced to GPT-5.6 Luna, Fast/Advanced and Thinking/Balanced+to GPT-5.6 Terra, and Thinking/Advanced to GPT-5.6 Sol when the account exposes+them. It otherwise uses the visible account default. File attachments,+agent-building tools, and SDK sub-sessions use the same Claude Agent SDK path as+platform-funded AutoPilot. Builder-panel-bound sessions remain platform-funded+in this preview because their persistent session is created without an AI+connection selector.++The Copilot executor keeps one exclusive credential lease and one Codex runtime+per connected account, then multiplexes overlapping chats onto separate Codex+threads. Canceling or finishing one chat does not stop its siblings. This pool+is process-local: a request routed to another executor process or replica cannot+join the owner yet and still fails with the bounded, retryable+`codex_credential_busy` error. A hosted multi-replica rollout therefore needs+credential-owner routing or a dedicated bridge. The normal platform AutoPilot+route is unchanged.++### Test the AutoPilot graph block++1. Add an **AutoPilot** block to a graph.+2. Select the saved connection under **ChatGPT / Codex connection**.+3. Run the graph.++The credential field is a reference-only credential input: graph validation+checks ownership and export strips the credential ID, while the graph executor+does not hold the outer credential lease. The Copilot executor acquires the+single runtime lease when it consumes the queued turn. This preserves normal+credential rebinding without deadlocking the nested execution path. Leaving the+field empty preserves the existing platform-funded behavior.++## Claude Agent SDK transport++AutoPilot keeps the existing Claude Agent SDK and Claude Code CLI as its agent+harness. A request-scoped loopback Anthropic Messages endpoint translates model+rounds to the pinned Codex App Server's experimental `dynamicTools` protocol.+Claude Code still owns the MCP tool loop, permission hooks, builder behavior,+transcript, resume, compaction, and sub-session machinery. Codex supplies the+model response using the selected user's ChatGPT/Codex credential.++The loopback endpoint is bound to `127.0.0.1` on an ephemeral port and accepts+only a random capability generated for that turn. The capability is placed in+the Claude CLI child environment; ChatGPT access, refresh, and ID tokens are+never placed there. Anthropic text and tool-use events are mapped to the same+Claude SDK response adapter used by normal AutoPilot.++The App Server process itself is fail-closed: native shell, filesystem, web+search, apps, plugins, environments, workspace roots, and approval requests are+disabled, and its sandbox is read-only. Any side effect comes only through a+tool that Claude Code was already permitted to invoke through AutoGPT's MCP and+security hooks.++The implementation pins `openai-codex` and its bundled runtime together at+`0.144.4`. Do not float either dependency independently. The Claude Agent SDK+and bundled CLI are also pinned by the backend lock. Because `dynamicTools` and+the compatibility surface are version-sensitive, updates require the focused+protocol, Messages conformance, and bundled-CLI tests to pass before rollout.++## Verify an existing local login++To test an existing local Codex login before starting the stack, run this from+`autogpt_platform/backend`:++```sh+poetry run python -m scripts.codex_preview_smoke+```++The smoke test copies, never moves, `~/.codex/auth.json` into an isolated home,+checks account, models, and rate limits, performs one subscription-backed turn,+and fails if the runtime can read a host canary or mutates the source login. It+does consume a small amount of the connected account's Codex usage.++## Deploy a private cloud preview++Codex pairing and supported execution routes do not require availability flags+or frontend build-time overrides. Scope a private preview with the deployment's+ordinary access controls instead of hiding device login with environment+variables. The transport selector is driven by the connected, compatible+subscription providers: it is hidden for zero or one and shown when more than+one is available.++REST, executor, and Copilot executor must share the normal AutoGPT credential+database, encryption configuration, and Redis cluster. Give every process that+may launch App Server an isolated, memory-backed `CODEX_TEMP_ROOT`; never mount+it on a persistent volume.++Set `FRONTEND_BASE_URL`, `NEXT_PUBLIC_FRONTEND_BASE_URL`, and `BETTER_AUTH_URL`+to the exact public origin of the preview, including its `https://` scheme and+without a path. REST deliberately builds the device-login URL from this+configured origin instead of trusting forwarded host headers. Leaving a+production or localhost origin in a cloud preview sends the popup to a host+that does not have the preview's Better Auth cookie.++`NEXT_PUBLIC_FRONTEND_BASE_URL` is compiled into the frontend bundle. Pass it as+a frontend image build argument, then set `BETTER_AUTH_URL` on the frontend at+runtime and `FRONTEND_BASE_URL` on REST at runtime. The development Compose file+forwards all three exported values and keeps `http://localhost:3000` as their+local default.++Device-login status and ownership are Redis-backed, so polling and cancellation+can reach different REST replicas. The active App Server login actor is still+process-local, however: an owner-pod restart interrupts the login and another+replica cannot take it over or resume it. Keep rolling deploys away from active+sign-ins and ask the user to restart an interrupted login. Broad cloud rollout+requires moving the login actor into the dedicated bridge described below.++The containers need outbound HTTPS access to OpenAI's ChatGPT/Codex endpoints.+No callback ingress from OpenAI is required because Codex App Server polls for+device-code completion.++## Preview safety boundary++- The encrypted, USER-scoped `IntegrationCredential` row remains the source of+ truth.+- Raw ChatGPT tokens do not enter Redis, RabbitMQ, frontend responses,+ container-wide environment variables, or AutoPilot session records.+- Each native invocation or shared AutoPilot runtime actor materializes auth+ into an isolated temporary home. An AutoPilot actor keeps that home only+ while one or more overlapping chats are attached, checkpoints Codex-managed+ refresh throughout its lifetime and at shutdown, then cleans the home.+- Code Generation turns expose no dynamic tools or host workspace. AutoPilot+ exposes only the tools registered by its existing Claude SDK/MCP harness.+- Runtime reuse is scoped to the exact user and credential. Credentials are+ never pooled across users or substituted for another account.+- There is no silent platform-key fallback.++The current implementation launches Codex App Server in-process from the+ordinary REST, executor, and Copilot executor containers. Those containers+currently run as root and retain the backend services' broader database, Redis,+filesystem, and network privileges. The temporary-home and fail-closed thread+configuration reduce exposure, but they do not make this a safe boundary for a+broad external cloud rollout.++Treat the current shape as local development or a tightly access-controlled+staff/private preview only. Before enabling external users, move runtime and+device-login supervision into a dedicated unprivileged bridge with a non-root+user, read-only root filesystem, dropped capabilities, memory-only credential+homes, process and concurrency limits, narrowly restricted egress, and a+least-privilege internal credential lease interface. The bridge must also own+login actor takeover or explicit restart semantics. Complete the commercial+and policy review for hosted subscription passthrough before broader rollout.