Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(vlm): pass Codex reasoning effort (#4244)
Yu Zhang85228eb38dea
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex 54fe00144..e34cd348e 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -119,7 +119,8 @@ Use `openviking-server init` to complete the Codex login/import step, then run `"vlm": {"provider" : "openai-codex","model" : "gpt-5.4",- "api_base" : "https://chatgpt.com/backend-api/codex"+ "api_base" : "https://chatgpt.com/backend-api/codex",+ "reasoning_effort": "xhigh"}}```@@ -633,7 +634,7 @@ Vision Language Model for semantic extraction (L0/L1 generation).| `thinking` | bool | Enable thinking mode for VolcEngine models (default: `false`) || `max_concurrent` | int | Maximum concurrent semantic LLM calls (default: `32`) || `max_retries` | int | Maximum retry attempts for transient VLM provider errors (default: `3`; `0` disables retry) |-| `credentials` | array | Ordered VLM credential/model list, with index 0 having the highest priority. Each item can override `provider`, `model`, `api_key`, `api_base`, `api_version`, `extra_headers`, `extra_request_body`, and `stream` |+| `credentials` | array | Ordered VLM credential/model list, with index 0 having the highest priority. Each item can override `provider`, `model`, `api_key`, `api_base`, `api_version`, `extra_headers`, `extra_request_body`, `stream`, and `reasoning_effort` || `failback_timeout_seconds` | float | Time threshold for attempting a step back toward a higher-priority credential after failover (default: `600`) || `failback_request_count` | int | Successful requests on a lower-priority credential before attempting a step back (default: `50`) || `backup` | object | Optional backup VLM configuration (same shape as `vlm`) for automatic failover when the primary fails with retryable errors such as rate limits, `5xx` responses, or connection/timeout failures. Only one level of failover is supported — the backup itself cannot define a nested `backup` |@@ -641,6 +642,7 @@ Vision Language Model for semantic extraction (L0/L1 generation).| `extra_headers` | object | Custom HTTP headers for compatible HTTP providers. `kimi` also accepts header overrides, but already injects the required subscription headers by default || `extra_request_body` | object | Extra JSON body fields for OpenAI-compatible completion requests, useful for provider-specific options such as Ollama `{"think": false}` || `stream` | bool | Enable streaming mode (for OpenAI-compatible providers, default: `false`) |+| `reasoning_effort` | str | Reasoning effort for OpenAI Codex Responses requests. Leave unset to use the model default || `media` | object | Audio/video runtime controls. Media understanding reuses this VLM's provider, model, credentials, client, timeout, retry, headers, output-token limit, failover, and token accounting || `media.enabled` | bool | Enable audio/video understanding (default: `false`) || `media.max_concurrent` | int | Maximum concurrent audio/video calls (default: `2`) |diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex 4c589392a..7f4aeef93 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -120,7 +120,8 @@ openviking-server doctor"vlm": {"provider" : "openai-codex","model" : "gpt-5.4",- "api_base" : "https://chatgpt.com/backend-api/codex"+ "api_base" : "https://chatgpt.com/backend-api/codex",+ "reasoning_effort": "xhigh"}}```@@ -602,7 +603,7 @@ provider,并设置 `storage.vectordb.sparse_weight > 0`。自托管模型的| `thinking` | bool | 启用思考模式(仅对部分火山模型生效,默认:`false`) || `max_concurrent` | int | 语义处理阶段 LLM 最大并发调用数(默认:`32`) || `max_retries` | int | VLM provider 瞬时错误的最大重试次数(默认:`3`;`0` 表示禁用重试) |-| `credentials` | array | 有序 VLM 凭据/模型列表,索引 0 优先级最高。每项可单独覆盖 `provider`、`model`、`api_key`、`api_base`、`api_version`、`extra_headers`、`extra_request_body` 和 `stream` |+| `credentials` | array | 有序 VLM 凭据/模型列表,索引 0 优先级最高。每项可单独覆盖 `provider`、`model`、`api_key`、`api_base`、`api_version`、`extra_headers`、`extra_request_body`、`stream` 和 `reasoning_effort` || `failback_timeout_seconds` | float | 切换到低优先级 credential 后,尝试逐级切回的时间阈值(默认:`600`) || `failback_request_count` | int | 低优先级 credential 成功处理多少次请求后尝试逐级切回(默认:`50`) || `backup` | object | 可选的备用 VLM 配置(结构与 `vlm` 相同),当主 VLM 遇到限流、`5xx`、超时或连接失败等可重试错误时自动切换。仅支持 1 层备用 — 备用 VLM 本身不能再嵌套 `backup` |@@ -610,6 +611,7 @@ provider,并设置 `storage.vectordb.sparse_weight > 0`。自托管模型的| `extra_headers` | object | 兼容 HTTP provider 的自定义请求头。`kimi` 默认已注入所需订阅请求头,也支持在这里覆盖或扩展 || `extra_request_body` | object | 传给 OpenAI 兼容 completion 请求的额外 JSON body 字段,可用于 Ollama `{"think": false}` 等 provider 专有参数 || `stream` | bool | 启用流式模式(OpenAI 兼容 provider 可用,默认:`false`) |+| `reasoning_effort` | str | OpenAI Codex Responses 请求的推理强度。不设置时使用模型默认值 || `media` | object | 音视频运行参数;音视频理解复用该 VLM 的 provider、模型、凭据、client、超时、重试、请求头、输出 token 限制、故障切换和 token 统计 || `media.enabled` | bool | 启用音视频理解(默认:`false`) || `media.max_concurrent` | int | 音视频调用最大并发数(默认:`2`) |diff --git a/examples/ov.conf.example b/examples/ov.conf.exampleindex e23057c38..c8676a5e1 100644--- a/examples/ov.conf.example+++ b/examples/ov.conf.example@@ -157,6 +157,7 @@"provider": "openai-codex","model": "gpt-5.3-codex","api_base": "https://chatgpt.com/backend-api/codex",+ "reasoning_effort": "xhigh","temperature": 0.0,"max_retries": 2,},diff --git a/openviking/models/vlm/backends/codex_responses_adapter.py b/openviking/models/vlm/backends/codex_responses_adapter.pyindex 1b4febf90..d1bf0e3a7 100644--- a/openviking/models/vlm/backends/codex_responses_adapter.py+++ b/openviking/models/vlm/backends/codex_responses_adapter.py@@ -246,6 +246,9 @@ class CodexCompletionsAdapter:tools = _convert_tools_for_responses(kwargs.get("tools"))if tools:response_kwargs["tools"] = tools+ reasoning_effort = kwargs.get("reasoning_effort")+ if reasoning_effort:+ response_kwargs["reasoning"] = {"effort": reasoning_effort}collected_output_items: List[Any] = []collected_text_deltas: List[str] = []has_function_calls = Falsediff --git a/openviking_cli/utils/config/vlm_config.py b/openviking_cli/utils/config/vlm_config.pyindex 45b5a0082..df555929d 100644--- a/openviking_cli/utils/config/vlm_config.py+++ b/openviking_cli/utils/config/vlm_config.py@@ -46,6 +46,10 @@ class VLMCredential(BaseModel):default=None, description="Extra JSON body fields")stream: Optional[bool] = Field(default=None, description="Enable streaming mode")+ reasoning_effort: Optional[str] = Field(+ default=None,+ description="Reasoning effort for OpenAI-compatible reasoning models",+ )max_tokens: Optional[int] = Field(default=None,gt=0,@@ -163,6 +167,11 @@ class VLMConfig(BaseModel):default=False, description="Enable streaming mode for OpenAI-compatible providers")+ reasoning_effort: Optional[str] = Field(+ default=None,+ description="Reasoning effort for OpenAI-compatible reasoning models",+ )+# New multi-credential configurationcredentials: List[VLMCredential] = Field(default_factory=list,@@ -273,6 +282,7 @@ class VLMConfig(BaseModel):or self.extra_headersor self.extra_request_bodyor self.stream+ or self.reasoning_effortor self.forward_api_key is not None):if self.provider not in self.providers:@@ -295,6 +305,8 @@ class VLMConfig(BaseModel):self.providers[self.provider]["extra_request_body"] = self.extra_request_bodyif self.stream and "stream" not in self.providers[self.provider]:self.providers[self.provider]["stream"] = self.stream+ if self.reasoning_effort and "reasoning_effort" not in self.providers[self.provider]:+ self.providers[self.provider]["reasoning_effort"] = self.reasoning_effortdef _normalize_credentials(self):"""Normalize credentials configuration:@@ -333,6 +345,7 @@ class VLMConfig(BaseModel):if primary_cfg.get("stream") is not Noneelse self.stream),+ reasoning_effort=(primary_cfg.get("reasoning_effort") or self.reasoning_effort),max_tokens=self.max_tokens,)migrated_credentials.append(primary_cred)@@ -362,6 +375,9 @@ class VLMConfig(BaseModel):if backup_cfg.get("stream") is not Noneelse self.backup.stream),+ reasoning_effort=(+ backup_cfg.get("reasoning_effort") or self.backup.reasoning_effort+ ),max_tokens=self.backup.max_tokens,)migrated_credentials.append(backup_cred)@@ -402,6 +418,9 @@ class VLMConfig(BaseModel):if provider_cfg.get("stream") is not Noneelse self.stream),+ reasoning_effort=(+ provider_cfg.get("reasoning_effort") or self.reasoning_effort+ ),))@@ -432,6 +451,8 @@ class VLMConfig(BaseModel):cred.extra_request_body = self.extra_request_bodyif cred.stream is None:cred.stream = self.stream+ if not cred.reasoning_effort:+ cred.reasoning_effort = self.reasoning_effortdef _has_legacy_provider_config(self) -> bool:"""Check if there's legacy provider config (not credentials-based)."""@@ -485,6 +506,8 @@ class VLMConfig(BaseModel):config["extra_request_body"] = self.extra_request_bodyif self.stream and "stream" not in config:config["stream"] = self.stream+ if self.reasoning_effort and "reasoning_effort" not in config:+ config["reasoning_effort"] = self.reasoning_effortreturn configdef _provider_has_usable_credentials(self, provider_name: str, config: Dict[str, Any]) -> bool:@@ -498,6 +521,26 @@ class VLMConfig(BaseModel):return has_codex_auth_available()return False+ def _get_provider_config_from_credential(self, cred: VLMCredential) -> Dict[str, Any]:+ config: Dict[str, Any] = {}+ if cred.api_key:+ config["api_key"] = cred.api_key+ if cred.api_base:+ config["api_base"] = cred.api_base+ if cred.api_version:+ config["api_version"] = cred.api_version+ if cred.forward_api_key is not None:+ config["forward_api_key"] = cred.forward_api_key+ if cred.extra_headers:+ config["extra_headers"] = cred.extra_headers+ if cred.extra_request_body:+ config["extra_request_body"] = cred.extra_request_body+ if cred.stream:+ config["stream"] = cred.stream+ if cred.reasoning_effort:+ config["reasoning_effort"] = cred.reasoning_effort+ return config+def _match_provider(self, model: str | None = None) -> tuple[Dict[str, Any] | None, str | None]:"""Match provider config.@@ -508,18 +551,7 @@ class VLMConfig(BaseModel):# If credentials are configured, use the first oneif self.credentials:cred = self.credentials[0]- return (- {- "api_key": cred.api_key,- "api_base": cred.api_base,- "api_version": cred.api_version,- "forward_api_key": cred.forward_api_key,- "extra_headers": cred.extra_headers,- "extra_request_body": cred.extra_request_body,- "stream": cred.stream,- },- cred.provider,- )+ return self._get_provider_config_from_credential(cred), cred.providerif self.provider:return self._get_provider_config_by_name(self.provider) or {}, self.provider@@ -620,6 +652,8 @@ class VLMConfig(BaseModel):result["extra_headers"] = credential.extra_headersif credential.extra_request_body:result["extra_request_body"] = credential.extra_request_body+ if credential.reasoning_effort:+ result["reasoning_effort"] = credential.reasoning_effortreturn result@@ -656,6 +690,8 @@ class VLMConfig(BaseModel):result["extra_headers"] = config.get("extra_headers")if config.get("extra_request_body"):result["extra_request_body"] = config.get("extra_request_body")+ if config.get("reasoning_effort"):+ result["reasoning_effort"] = config.get("reasoning_effort")return resultdiff --git a/tests/unit/test_codex_vlm.py b/tests/unit/test_codex_vlm.pyindex 35c12c187..1888de593 100644--- a/tests/unit/test_codex_vlm.py+++ b/tests/unit/test_codex_vlm.py@@ -95,6 +95,35 @@ def test_codex_text_completion_uses_responses_api(mock_resolve, mock_openai_clasmock_real_client.responses.stream.assert_not_called()+@patch("openviking.models.vlm.backends.codex_vlm.openai.OpenAI")+@patch("openviking.models.vlm.backends.codex_vlm.resolve_codex_runtime_credentials")+def test_codex_text_completion_passes_reasoning_effort_to_responses(+ mock_resolve, mock_openai_class+):+ mock_resolve.return_value = {+ "api_key": "oauth-token",+ "base_url": "https://chatgpt.com/backend-api/codex",+ }+ mock_real_client = MagicMock()+ mock_real_client.responses.create.return_value = _MockResponsesStream(+ _build_final_response("reasoned answer")+ )+ mock_openai_class.return_value = mock_real_client++ vlm = CodexVLM(+ {+ "provider": "openai-codex",+ "model": "gpt-5.3-codex",+ "reasoning_effort": "xhigh",+ }+ )+ result = vlm.get_completion("hello")++ assert result == "reasoned answer"+ call_kwargs = mock_real_client.responses.create.call_args.kwargs+ assert call_kwargs["reasoning"] == {"effort": "xhigh"}++@patch("openviking.models.vlm.backends.codex_vlm.openai.OpenAI")@patch("openviking.models.vlm.backends.codex_vlm.resolve_codex_runtime_credentials")def test_codex_vision_completion_converts_images(mock_resolve, mock_openai_class):@@ -193,6 +222,17 @@ def test_vlm_config_accepts_codex_without_api_key(_mock_auth_available):assert config.get_vlm_instance().__class__.__name__ == "CodexVLM"+@patch("openviking.models.vlm.backends.codex_auth.has_codex_auth_available", return_value=True)+def test_vlm_config_passes_codex_reasoning_effort(_mock_auth_available):+ config = VLMConfig(+ provider="openai-codex",+ model="gpt-5.3-codex",+ reasoning_effort="high",+ )++ assert config._build_vlm_config_dict()["reasoning_effort"] == "high"++@patch("openviking.models.vlm.backends.codex_auth.has_codex_auth_available", return_value=True)def test_vlm_config_default_provider_resolves_codex(_mock_auth_available):config = VLMConfig(