Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
fix(vlm): make OpenAI keepalive expiry configurable (#4473)
Oneflyaf41ee86c190
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex eb01ffcd7..0d7002852 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -683,11 +683,12 @@ Vision Language Model for semantic extraction (L0/L1 generation).| `thinking` | bool | Enable thinking mode for VolcEngine models (default: `false`) || `max_concurrent` | int | Maximum concurrent semantic LLM calls (default: `32`) || `max_retries` | int | Maximum retry attempts for transient VLM provider errors (default: `3`; `0` disables retry) |-| `credentials` | array | Ordered VLM credential/model list, with index 0 having the highest priority. Each item can override `provider`, `model`, `api_key`, `api_base`, `api_version`, `extra_headers`, `extra_request_body`, and `reasoning_effort` |+| `credentials` | array | Ordered VLM credential/model list, with index 0 having the highest priority. Each item can override `provider`, `model`, `api_key`, `api_base`, `api_version`, `extra_headers`, `extra_request_body`, `reasoning_effort`, and `keepalive_expiry` || `failback_timeout_seconds` | float | Time threshold for attempting a step back toward a higher-priority credential after failover (default: `600`) || `failback_request_count` | int | Successful requests on a lower-priority credential before attempting a step back (default: `50`) || `backup` | object | Optional backup VLM configuration (same shape as `vlm`) for automatic failover when the primary fails with retryable errors such as rate limits, `5xx` responses, or connection/timeout failures. Only one level of failover is supported — the backup itself cannot define a nested `backup` || `timeout` | float | Per-request HTTP timeout in seconds passed to the underlying OpenAI/LiteLLM client. Increase for slow endpoints (e.g., DashScope, local inference). Must be `> 0` (default: `600.0`) |+| `keepalive_expiry` | float | Idle connection lifetime in seconds for OpenAI-compatible VLM clients. Set to `0` to disable idle connection reuse; unset uses the OpenAI SDK default. Must be `>= 0` || `extra_headers` | object | Custom HTTP headers for compatible HTTP providers. `kimi` also accepts header overrides, but already injects the required subscription headers by default || `extra_request_body` | object | Extra JSON body fields for OpenAI-compatible completion requests, useful for provider-specific options such as Ollama `{"think": false}` || `reasoning_effort` | str | Reasoning effort for `openai`, `azure`, `kimi`, `glm`, and `openai-codex`; explicit values are forwarded, and accepted values depend on the model. When unset, GPT-5/o-series names retain `low`; other models omit the field. For Chat Completions, `extra_request_body.reasoning_effort` takes precedence |diff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex e83fc4096..f5eae94de 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -652,11 +652,12 @@ provider,并设置 `storage.vectordb.sparse_weight > 0`。自托管模型的| `thinking` | bool | 启用思考模式(仅对部分火山模型生效,默认:`false`) || `max_concurrent` | int | 语义处理阶段 LLM 最大并发调用数(默认:`32`) || `max_retries` | int | VLM provider 瞬时错误的最大重试次数(默认:`3`;`0` 表示禁用重试) |-| `credentials` | array | 有序 VLM 凭据/模型列表,索引 0 优先级最高。每项可单独覆盖 `provider`、`model`、`api_key`、`api_base`、`api_version`、`extra_headers`、`extra_request_body` 和 `reasoning_effort` |+| `credentials` | array | 有序 VLM 凭据/模型列表,索引 0 优先级最高。每项可单独覆盖 `provider`、`model`、`api_key`、`api_base`、`api_version`、`extra_headers`、`extra_request_body`、`reasoning_effort` 和 `keepalive_expiry` || `failback_timeout_seconds` | float | 切换到低优先级 credential 后,尝试逐级切回的时间阈值(默认:`600`) || `failback_request_count` | int | 低优先级 credential 成功处理多少次请求后尝试逐级切回(默认:`50`) || `backup` | object | 可选的备用 VLM 配置(结构与 `vlm` 相同),当主 VLM 遇到限流、`5xx`、超时或连接失败等可重试错误时自动切换。仅支持 1 层备用 — 备用 VLM 本身不能再嵌套 `backup` || `timeout` | float | 单次 VLM API 请求的 HTTP 超时时间(秒),传递给底层 OpenAI/LiteLLM 客户端。慢端点(如 DashScope、本地推理)可调大。必须 `> 0`(默认:`600.0`) |+| `keepalive_expiry` | float | OpenAI 兼容 VLM 客户端的空闲连接保留秒数。设为 `0` 可禁用空闲连接复用;不设置时使用 OpenAI SDK 默认值。必须 `>= 0` || `extra_headers` | object | 兼容 HTTP provider 的自定义请求头。`kimi` 默认已注入所需订阅请求头,也支持在这里覆盖或扩展 || `extra_request_body` | object | 传给 OpenAI 兼容 completion 请求的额外 JSON body 字段,可用于 Ollama `{"think": false}` 等 provider 专有参数 || `reasoning_effort` | str | `openai`、`azure`、`kimi`、`glm` 和 `openai-codex` 的推理强度,显式配置时发送;可用值由模型决定。不设置时,GPT-5/o 系列名称保留 `low`,其他模型不发送。Chat Completions 请求中,`extra_request_body.reasoning_effort` 优先 |diff --git a/openviking/models/vlm/backends/openai_vlm.py b/openviking/models/vlm/backends/openai_vlm.pyindex ab9a62fbf..5bcec67d3 100644--- a/openviking/models/vlm/backends/openai_vlm.py+++ b/openviking/models/vlm/backends/openai_vlm.py@@ -9,6 +9,8 @@ from pathlib import Pathfrom typing import Any, Dict, List, Optional, Unionfrom urllib.parse import urlparse+import httpx+from openviking.models.network import (create_optional_async_httpx_client,create_optional_sync_httpx_client,@@ -79,6 +81,18 @@ class OpenAIVLM(VLMBase):self._async_client_cache = LoopScopedAsyncClientCache()self.api_version = config.get("api_version")self.reasoning_effort = config.get("reasoning_effort")+ self.keepalive_expiry = config.get("keepalive_expiry")++ def _http_client_kwargs(self) -> Dict[str, Any]:+ kwargs: Dict[str, Any] = {"timeout": self.timeout}+ if self.keepalive_expiry is not None:+ defaults = openai.DEFAULT_CONNECTION_LIMITS+ kwargs["limits"] = httpx.Limits(+ max_connections=defaults.max_connections,+ max_keepalive_connections=defaults.max_keepalive_connections,+ keepalive_expiry=self.keepalive_expiry,+ )+ return kwargsdef get_client(self):"""Get sync client"""@@ -93,11 +107,14 @@ class OpenAIVLM(VLMBase):self.extra_headers,self.timeout,)+ http_kwargs = self._http_client_kwargs()http_client = create_optional_sync_httpx_client(self.api_base,client_cls=openai.DefaultHttpxClient,- timeout=self.timeout,+ **http_kwargs,)+ if http_client is None and self.keepalive_expiry is not None:+ http_client = openai.DefaultHttpxClient(**http_kwargs)if http_client is not None:kwargs["http_client"] = http_clientif self.provider == "azure":@@ -118,11 +135,14 @@ class OpenAIVLM(VLMBase):self.extra_headers,self.timeout,)+ http_kwargs = self._http_client_kwargs()http_client = create_optional_async_httpx_client(self.api_base,client_cls=openai.DefaultAsyncHttpxClient,- timeout=self.timeout,+ **http_kwargs,)+ if http_client is None and self.keepalive_expiry is not None:+ http_client = openai.DefaultAsyncHttpxClient(**http_kwargs)if http_client is not None:kwargs["http_client"] = http_clientif self.provider == "azure":@@ -133,6 +153,12 @@ class OpenAIVLM(VLMBase):"""Get an async client scoped to the current event loop."""return self._async_client_cache.get(self._build_async_client)+ def close(self) -> None:+ """Close clients and HTTP transports owned by this backend."""+ if self._sync_client is not None:+ self._sync_client.close()+ self._async_client_cache.close_all_with_close()+def _supports_enable_thinking(self) -> bool:"""Return True for OpenAI-compatible DashScope endpoints that accept enable_thinking."""if self.provider != "openai":diff --git a/openviking/models/vlm/base.py b/openviking/models/vlm/base.pyindex 99a40281c..fa7bd1b72 100644--- a/openviking/models/vlm/base.py+++ b/openviking/models/vlm/base.py@@ -333,6 +333,9 @@ class VLMBase(ABC):"""Reset token usage"""self._token_tracker.reset()+ def close(self) -> None:+ """Release provider resources, if any."""+def _extract_content_from_response(self, response) -> str:if isinstance(response, str):return response@@ -762,6 +765,11 @@ class FailoverVLM(VLMBase):self.primary.reset_token_usage()self.backup.reset_token_usage()+ def close(self) -> None:+ """Close both provider instances."""+ self.primary.close()+ self.backup.close()+class MultiCredentialVLM(VLMBase):"""VLM wrapper that provides failover across multiple ordered credentials.@@ -1128,3 +1136,8 @@ class MultiCredentialVLM(VLMBase):"""Reset token usage for all credential instances."""for instance in self._vlm_instances:instance.reset_token_usage()++ def close(self) -> None:+ """Close all credential provider instances."""+ for instance in self._vlm_instances:+ instance.close()diff --git a/openviking/service/core.py b/openviking/service/core.pyindex ee9fb37e2..14cc402c9 100644--- a/openviking/service/core.py+++ b/openviking/service/core.py@@ -629,6 +629,9 @@ class OpenVikingService:self._queue_manager = Nonelogger.info("Queue manager stopped")+ self._config.vlm.close()+ await asyncio.sleep(0)+if self._vikingdb_manager:self._vikingdb_manager.mark_closing()diff --git a/openviking_cli/utils/config/vlm_config.py b/openviking_cli/utils/config/vlm_config.pyindex 52a73a81f..cf92cc307 100644--- a/openviking_cli/utils/config/vlm_config.py+++ b/openviking_cli/utils/config/vlm_config.py@@ -64,6 +64,11 @@ class VLMCredential(BaseModel):default=None,description="Reasoning effort for OpenAI-compatible reasoning models",)+ keepalive_expiry: Optional[float] = Field(+ default=None,+ ge=0.0,+ description="Idle HTTP connection lifetime for OpenAI-compatible providers",+ )max_tokens: Optional[int] = Field(default=None,gt=0,@@ -133,6 +138,14 @@ class VLMConfig(BaseModel):"high-latency endpoints (e.g., DashScope, local inference servers)."),)+ keepalive_expiry: Optional[float] = Field(+ default=None,+ ge=0.0,+ description=(+ "Idle HTTP connection lifetime in seconds for OpenAI-compatible VLM clients. "+ "Set to 0 to disable connection reuse; None uses the provider SDK default."+ ),+ )provider: Optional[str] = Field(default=None, description="Provider type")backend: Optional[str] = Field(@@ -308,6 +321,7 @@ class VLMConfig(BaseModel):or self.extra_headersor self.extra_request_bodyor self.reasoning_effort+ or self.keepalive_expiry is not Noneor self.forward_api_key is not None):if self.provider not in self.providers:@@ -330,6 +344,11 @@ class VLMConfig(BaseModel):self.providers[self.provider]["extra_request_body"] = self.extra_request_bodyif self.reasoning_effort and "reasoning_effort" not in self.providers[self.provider]:self.providers[self.provider]["reasoning_effort"] = self.reasoning_effort+ if (+ self.keepalive_expiry is not None+ and "keepalive_expiry" not in self.providers[self.provider]+ ):+ self.providers[self.provider]["keepalive_expiry"] = self.keepalive_expirydef _normalize_credentials(self):"""Normalize credentials configuration:@@ -364,6 +383,11 @@ class VLMConfig(BaseModel):primary_cfg.get("extra_request_body") or self.extra_request_body),reasoning_effort=(primary_cfg.get("reasoning_effort") or self.reasoning_effort),+ keepalive_expiry=(+ primary_cfg.get("keepalive_expiry")+ if primary_cfg.get("keepalive_expiry") is not None+ else self.keepalive_expiry+ ),max_tokens=self.max_tokens,)migrated_credentials.append(primary_cred)@@ -391,6 +415,11 @@ class VLMConfig(BaseModel):reasoning_effort=(backup_cfg.get("reasoning_effort") or self.backup.reasoning_effort),+ keepalive_expiry=(+ backup_cfg.get("keepalive_expiry")+ if backup_cfg.get("keepalive_expiry") is not None+ else self.backup.keepalive_expiry+ ),max_tokens=self.backup.max_tokens,)migrated_credentials.append(backup_cred)@@ -429,6 +458,11 @@ class VLMConfig(BaseModel):reasoning_effort=(provider_cfg.get("reasoning_effort") or self.reasoning_effort),+ keepalive_expiry=(+ provider_cfg.get("keepalive_expiry")+ if provider_cfg.get("keepalive_expiry") is not None+ else self.keepalive_expiry+ ),))@@ -459,6 +493,8 @@ class VLMConfig(BaseModel):cred.extra_request_body = self.extra_request_bodyif not cred.reasoning_effort:cred.reasoning_effort = self.reasoning_effort+ if cred.keepalive_expiry is None:+ cred.keepalive_expiry = self.keepalive_expirydef _has_legacy_provider_config(self) -> bool:"""Check if there's legacy provider config (not credentials-based)."""@@ -512,6 +548,8 @@ class VLMConfig(BaseModel):config["extra_request_body"] = self.extra_request_bodyif self.reasoning_effort and "reasoning_effort" not in config:config["reasoning_effort"] = self.reasoning_effort+ if self.keepalive_expiry is not None and "keepalive_expiry" not in config:+ config["keepalive_expiry"] = self.keepalive_expiryreturn configdef _provider_has_usable_credentials(self, provider_name: str, config: Dict[str, Any]) -> bool:@@ -541,6 +579,8 @@ class VLMConfig(BaseModel):config["extra_request_body"] = cred.extra_request_bodyif cred.reasoning_effort:config["reasoning_effort"] = cred.reasoning_effort+ if cred.keepalive_expiry is not None:+ config["keepalive_expiry"] = cred.keepalive_expiryreturn configdef _match_provider(self, model: str | None = None) -> tuple[Dict[str, Any] | None, str | None]:@@ -627,6 +667,13 @@ class VLMConfig(BaseModel):return self._vlm_instance+ def close(self) -> None:+ """Close and clear the cached VLM instance."""+ instance = self._vlm_instance+ self._vlm_instance = None+ if instance is not None:+ instance.close()+def _build_vlm_config_dict_for_credential(self, credential: VLMCredential) -> Dict[str, Any]:"""Build VLM instance config dict for a specific credential."""result = {@@ -634,6 +681,7 @@ class VLMConfig(BaseModel):"temperature": self.temperature,"max_retries": self.max_retries,"timeout": self.timeout,+ "keepalive_expiry": credential.keepalive_expiry,"provider": credential.provider,"thinking": self.thinking,"max_tokens": (@@ -667,6 +715,11 @@ class VLMConfig(BaseModel):"temperature": self.temperature,"max_retries": self.max_retries,"timeout": self.timeout,+ "keepalive_expiry": (+ config.get("keepalive_expiry")+ if config and config.get("keepalive_expiry") is not None+ else self.keepalive_expiry+ ),"provider": name,"thinking": self.thinking,"max_tokens": self.max_tokens,diff --git a/tests/models/vlm/test_timeout_config.py b/tests/models/vlm/test_timeout_config.pyindex b9dcc0bcb..3e2076273 100644--- a/tests/models/vlm/test_timeout_config.py+++ b/tests/models/vlm/test_timeout_config.py@@ -1,6 +1,6 @@# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.# SPDX-License-Identifier: AGPL-3.0-"""Tests for ``vlm.timeout`` configuration propagation.+"""Tests for VLM HTTP client configuration.Before this was wired through, ``_build_openai_client_kwargs`` exposed a``timeout`` parameter (#1208) but callers never passed it, so the default@@ -9,6 +9,7 @@ These tests lock in that the config value flows through to the underlyingOpenAI and LiteLLM clients."""+import asynciofrom unittest import mockimport pytest@@ -53,21 +54,62 @@ def test_build_openai_client_kwargs_custom_timeout():assert kwargs["timeout"] == 120.0-def test_openai_vlm_propagates_config_timeout():- vlm = OpenAIVLM(- {- "provider": "openai",- "model": "gpt-4o-mini",- "api_key": "sk-x",- "api_base": "https://example.invalid",- "timeout": 120.0,- }- )- assert vlm.timeout == 120.0-- with mock.patch("openviking.models.vlm.backends.openai_vlm.openai.OpenAI") as fake:- vlm.get_client()- assert fake.call_args.kwargs.get("timeout") == 120.0+async def test_openai_vlm_applies_http_config_and_closes_clients(monkeypatch):+ connections = 0+ payload = b'{"choices":[{"message":{"role":"assistant","content":"ok"}}]}'++ async def respond(reader, writer):+ nonlocal connections+ connections += 1+ try:+ while True:+ headers = await reader.readuntil(b"\r\n\r\n")+ content_length = next(+ int(line.split(b":", 1)[1])+ for line in headers.split(b"\r\n")+ if line.lower().startswith(b"content-length:")+ )+ await reader.readexactly(content_length)+ writer.write(+ b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n"+ + f"Content-Length: {len(payload)}\r\n\r\n".encode()+ + payload+ )+ await writer.drain()+ except asyncio.IncompleteReadError:+ pass+ finally:+ writer.close()+ await writer.wait_closed()++ monkeypatch.setenv("NO_PROXY", "127.0.0.1")+ async with await asyncio.start_server(respond, "127.0.0.1", 0) as server:+ config = VLMConfig(+ provider="openai",+ model="gpt-4o-mini",+ api_key="sk-x",+ api_base=f"http://127.0.0.1:{server.sockets[0].getsockname()[1]}/v1",+ timeout=1.0,+ keepalive_expiry=0,+ max_retries=0,+ )+ vlm = config.get_vlm_instance()+ sync_client = vlm.get_client()+ async_client = vlm.get_async_client()+ try:+ assert sync_client.timeout == 1.0+ assert async_client.timeout == 1.0+ assert await asyncio.to_thread(vlm.get_completion, "first") == "ok"+ assert await asyncio.to_thread(vlm.get_completion, "second") == "ok"+ assert await vlm.get_completion_async("first") == "ok"+ assert await vlm.get_completion_async("second") == "ok"+ assert connections == 4+ finally:+ config.close()+ await asyncio.sleep(0)+ assert sync_client.is_closed()+ assert async_client.is_closed()def test_openai_vlm_defaults_to_600_timeout_when_config_omits_it():@@ -84,6 +126,7 @@ def test_openai_vlm_defaults_to_600_timeout_when_config_omits_it():with mock.patch("openviking.models.vlm.backends.openai_vlm.openai.OpenAI") as fake:vlm.get_client()assert fake.call_args.kwargs.get("timeout") == 600.0+ assert "http_client" not in fake.call_args.kwargsdef test_litellm_build_kwargs_includes_timeout():diff --git a/tests/unit/service/test_core_consistency.py b/tests/unit/service/test_core_consistency.pyindex 3a915f539..3548345c0 100644--- a/tests/unit/service/test_core_consistency.py+++ b/tests/unit/service/test_core_consistency.py@@ -12,6 +12,7 @@ from openviking.service.core import OpenVikingServicefrom openviking.storage.index_consistency import IndexConsistencyReportfrom openviking_cli.exceptions import InvalidArgumentErrorfrom openviking_cli.session.user_id import UserIdentifier+from openviking_cli.utils.config.vlm_config import VLMConfigdef test_service_passes_queue_worker_concurrency_to_storage(monkeypatch) -> None:@@ -130,6 +131,8 @@ async def test_close_stops_queue_manager_before_ragfs_binding(monkeypatch) -> Noservice = OpenVikingService.__new__(OpenVikingService)service._resource_service = ResourceService()+ service._config = SimpleNamespace(vlm=VLMConfig())+ service._runtime_config_manager = Noneservice._watch_scheduler = Noneservice._session_auto_commit_scheduler = Noneservice._queue_manager = QueueManager()diff --git a/tests/unit/service/test_core_embedder_lifecycle.py b/tests/unit/service/test_core_embedder_lifecycle.pyindex eade47823..8b43e304d 100644--- a/tests/unit/service/test_core_embedder_lifecycle.py+++ b/tests/unit/service/test_core_embedder_lifecycle.py@@ -1,6 +1,8 @@# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.# SPDX-License-Identifier: AGPL-3.0-"""Service lifecycle coverage for embedding clients."""+"""Service lifecycle coverage for model clients."""++from types import SimpleNamespaceimport pytest@@ -31,6 +33,8 @@ async def test_service_closes_embedder_after_workers_and_storage_stop(monkeypatcevents.append("embedder")service = OpenVikingService.__new__(OpenVikingService)+ service._config = SimpleNamespace(vlm=SimpleNamespace(close=lambda: events.append("vlm")))+ service._runtime_config_manager = Noneservice._resource_service = _ResourceService()service._watch_scheduler = Noneservice._session_auto_commit_scheduler = None@@ -46,6 +50,7 @@ async def test_service_closes_embedder_after_workers_and_storage_stop(monkeypatcassert events == ["background","queue",+ "vlm","storage-mark","storage-close","embedder",diff --git a/tests/unit/service/test_core_encryption_startup.py b/tests/unit/service/test_core_encryption_startup.pyindex 0d0ba20c5..b7dda32ba 100644--- a/tests/unit/service/test_core_encryption_startup.py+++ b/tests/unit/service/test_core_encryption_startup.py@@ -12,6 +12,7 @@ import pytestfrom openviking.service.core import OpenVikingServicefrom openviking.utils.agfs_utils import RagfsBindingConfigfrom openviking.utils.process_lock import LOCK_FILENAME+from openviking_cli.utils.config.vlm_config import VLMConfigclass _FakeCacheConfig:@@ -217,6 +218,8 @@ async def test_close_releases_data_dir_lock_after_successful_cleanup(monkeypatchreleased = []service = OpenVikingService.__new__(OpenVikingService)service._resource_service = _ResourceService()+ service._config = SimpleNamespace(vlm=VLMConfig())+ service._runtime_config_manager = Noneservice._watch_scheduler = Noneservice._session_auto_commit_scheduler = Noneservice._queue_manager = None