Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat: make queuefs backend configurable (#2018)
Jiahui Zhoufa0be9c95810
diff --git a/docs/en/guides/01-configuration.md b/docs/en/guides/01-configuration.mdindex 48938e7f8..a4652b4e3 100644--- a/docs/en/guides/01-configuration.md+++ b/docs/en/guides/01-configuration.md@@ -800,13 +800,79 @@ Storage configuration for context data, including file storage (RAGFS) and vecto|-----------|------|-------------|---------|| `backend` | str | `"local"`, `"s3"`, or `"memory"` | `"local"` || `timeout` | float | Request timeout in seconds | `10.0` |-| `queue_db_path` | str (optional) | Override path of the queuefs sqlite database file. Defaults to `{storage.workspace}/_system/queue/queue.db` when not set. Useful when the workspace volume does not support sqlite (e.g. some network filesystems) | `null` |+| `queuefs` | object | QueueFS configuration. Controls the namespace mode, backend, and runtime options for `/queue` | `{ "mode": "shared", "backend": "sqlite", "recover_stale_sec": 0, "busy_timeout_ms": 5000 }` |+| `queue_db_path` | str (optional) | Legacy compatibility field for QueueFS sqlite DB path. Superseded by `storage.agfs.queuefs.db_path`. Defaults to `{storage.workspace}/_system/queue/queue.db` when not set. Useful when the workspace volume does not support sqlite (e.g. some network filesystems) | `null` || `s3` | object | S3 backend configuration (when backend is 's3') | - |**Configuration Examples**RAGFS uses Rust binding mode by default, directly accessing the file system through the Rust implementation.+##### QueueFS Configuration++| Parameter | Type | Description | Default |+|-----------|------|-------------|---------|+| `mode` | str | QueueFS namespace mode: `"shared"` uses `/queue`; `"worker"` isolates each worker under `/queue/worker-<index\|pid>` | `"shared"` |+| `backend` | str | QueueFS backend: `"memory"`, `"sqlite"`, or `"sqlite3"` | `"sqlite"` |+| `db_path` | str (optional) | SQLite database path for QueueFS when backend is `"sqlite"` or `"sqlite3"` | `null` |+| `recover_stale_sec` | int | Recover `processing` queue messages older than this many seconds on startup. `0` means recover all stale processing messages | `0` |+| `busy_timeout_ms` | int | SQLite busy timeout for QueueFS in milliseconds | `5000` |++Notes:++- QueueFS defaults to `sqlite` even if the main AGFS storage backend is `local`, `s3`, or `memory`.+- `mode=shared` keeps the historical global queue namespace at `/queue`; `mode=worker` isolates each worker under `/queue/worker-<index|pid>`.+- `db_path` is only used when QueueFS backend is `sqlite` or `sqlite3`.+- If both `storage.agfs.queuefs.db_path` and legacy `storage.agfs.queue_db_path` are set, `storage.agfs.queuefs.db_path` wins.+- If QueueFS backend is `memory`, any `db_path` or legacy `queue_db_path` is ignored.++Examples:++```json+{+ "storage": {+ "workspace": "./data",+ "agfs": {+ "backend": "local",+ "queuefs": {+ "mode": "shared",+ "backend": "sqlite",+ "db_path": "./data/_system/queue/custom-queue.db"+ }+ }+ }+}+```++```json+{+ "storage": {+ "workspace": "./data",+ "agfs": {+ "backend": "local",+ "queuefs": {+ "mode": "worker",+ "backend": "memory"+ }+ }+ }+}+```++Legacy compatibility example:++```json+{+ "storage": {+ "workspace": "./data",+ "agfs": {+ "backend": "local",+ "queue_db_path": "./data/_system/queue/queue.db"+ }+ }+}+```+##### S3 Backend Configurationdiff --git a/docs/zh/guides/01-configuration.md b/docs/zh/guides/01-configuration.mdindex b77202ae8..28ea2f765 100644--- a/docs/zh/guides/01-configuration.md+++ b/docs/zh/guides/01-configuration.md@@ -772,7 +772,8 @@ AST 提取支持:Python、JavaScript/TypeScript、Rust、Go、Java、C/C++。|------|------|------|--------|| `backend` | str | `"local"`、`"s3"` 或 `"memory"` | `"local"` || `timeout` | float | 请求超时时间(秒) | `10.0` |-| `queue_db_path` | str(可选)| 覆盖 queuefs sqlite 数据库文件路径。未设置时默认为 `{storage.workspace}/_system/queue/queue.db`。适用于 workspace 卷不支持 sqlite 的场景(例如某些网络文件系统) | `null` |+| `queuefs` | object | QueueFS 配置。控制 `/queue` 的命名空间模式、后端和运行时参数 | `{ "mode": "shared", "backend": "sqlite", "recover_stale_sec": 0, "busy_timeout_ms": 5000 }` |+| `queue_db_path` | str(可选)| 旧版兼容字段,用于覆盖 QueueFS 的 sqlite 数据库文件路径。已被 `storage.agfs.queuefs.db_path` 取代。未设置时默认为 `{storage.workspace}/_system/queue/queue.db`。适用于 workspace 卷不支持 sqlite 的场景(例如某些网络文件系统) | `null` || `s3` | object | S3 backend configuration (when backend is 's3') | - |@@ -780,6 +781,71 @@ AST 提取支持:Python、JavaScript/TypeScript、Rust、Go、Java、C/C++。RAGFS 默认使用 Rust binding 模式,通过 Rust 实现直接访问文件系统。+##### QueueFS 配置++| 参数 | 类型 | 说明 | 默认值 |+|------|------|------|--------|+| `mode` | str | QueueFS 命名空间模式:`"shared"` 使用 `/queue`;`"worker"` 为每个 worker 隔离到 `/queue/worker-<index\|pid>` | `"shared"` |+| `backend` | str | QueueFS 后端:`"memory"`、`"sqlite"` 或 `"sqlite3"` | `"sqlite"` |+| `db_path` | str(可选) | 当 backend 为 `"sqlite"` 或 `"sqlite3"` 时使用的 QueueFS sqlite 数据库路径 | `null` |+| `recover_stale_sec` | int | 启动时恢复超过该秒数的 `processing` 队列消息;`0` 表示恢复全部 stale processing 消息 | `0` |+| `busy_timeout_ms` | int | QueueFS sqlite 的 busy timeout,单位毫秒 | `5000` |++说明:++- 即使主 AGFS 存储后端是 `local`、`s3` 或 `memory`,QueueFS 默认仍使用 `sqlite`。+- `mode=shared` 会继续使用历史上的全局队列命名空间 `/queue`;`mode=worker` 会为每个 worker 隔离到 `/queue/worker-<index|pid>`。+- `db_path` 仅在 QueueFS backend 为 `sqlite` 或 `sqlite3` 时生效。+- 如果同时设置了 `storage.agfs.queuefs.db_path` 和旧字段 `storage.agfs.queue_db_path`,以前者为准。+- 如果 QueueFS backend 为 `memory`,则 `db_path` 和旧字段 `queue_db_path` 都会被忽略。++示例:++```json+{+ "storage": {+ "workspace": "./data",+ "agfs": {+ "backend": "local",+ "queuefs": {+ "mode": "shared",+ "backend": "sqlite",+ "db_path": "./data/_system/queue/custom-queue.db"+ }+ }+ }+}+```++```json+{+ "storage": {+ "workspace": "./data",+ "agfs": {+ "backend": "local",+ "queuefs": {+ "mode": "worker",+ "backend": "memory"+ }+ }+ }+}+```++旧字段兼容示例:++```json+{+ "storage": {+ "workspace": "./data",+ "agfs": {+ "backend": "local",+ "queue_db_path": "./data/_system/queue/queue.db"+ }+ }+}+```+##### S3 后端配置diff --git a/openviking/service/core.py b/openviking/service/core.pyindex 40934e29e..a39f1f42e 100644--- a/openviking/service/core.py+++ b/openviking/service/core.py@@ -29,6 +29,7 @@ from openviking.storage.index_consistency import check_index_consistencyfrom openviking.storage.queuefs.queue_manager import QueueManager, init_queue_managerfrom openviking.storage.transaction import LockManager, init_lock_managerfrom openviking.storage.viking_fs import VikingFS, init_viking_fs+from openviking.utils.agfs_utils import resolve_queuefs_mount_pointfrom openviking.utils.resource_processor import ResourceProcessorfrom openviking.utils.skill_processor import SkillProcessorfrom openviking_cli.exceptions import NotInitializedError@@ -121,9 +122,11 @@ class OpenVikingService:# Initialize QueueManager with agfs_clientif self._agfs_client:+ queue_mount_point = resolve_queuefs_mount_point()self._queue_manager = init_queue_manager(agfs=self._agfs_client,timeout=config.agfs.timeout,+ mount_point=queue_mount_point,max_concurrent_embedding=max_concurrent_embedding,max_concurrent_semantic=max_concurrent_semantic,)diff --git a/openviking/session/compressor.py b/openviking/session/compressor.pyindex 0c04d7825..0938155b8 100644--- a/openviking/session/compressor.py+++ b/openviking/session/compressor.py@@ -137,9 +137,14 @@ class SessionCompressor:changes=changes_dict,telemetry_id=telemetry.telemetry_id,)- await semantic_queue.enqueue(msg)if msg.telemetry_id:request_wait_tracker.register_semantic_root(msg.telemetry_id, msg.id)+ try:+ await semantic_queue.enqueue(msg)+ except Exception as e:+ if msg.telemetry_id:+ request_wait_tracker.mark_semantic_failed(msg.telemetry_id, msg.id, str(e))+ raiselogger.info(f"Enqueued semantic generation for {parent_uri} with changes: "f"added={len(changes['added'])}, modified={len(changes['modified'])}, "@@ -191,19 +196,27 @@ class SessionCompressor:chunk_memory.set_vectorize(Vectorize(text=chunk))chunk_msg = EmbeddingMsgConverter.from_context(chunk_memory)if chunk_msg:- enqueued = await self.vikingdb.enqueue_embedding_msg(chunk_msg)- if enqueued and chunk_msg.telemetry_id:+ if chunk_msg.telemetry_id:request_wait_tracker.register_embedding_root(chunk_msg.telemetry_id, chunk_msg.id)+ enqueued = await self.vikingdb.enqueue_embedding_msg(chunk_msg)+ if not enqueued and chunk_msg.telemetry_id:+ request_wait_tracker.mark_embedding_failed(+ chunk_msg.telemetry_id, chunk_msg.id, "embedding enqueue returned false"+ )# Always enqueue the base record (uses abstract as vector text)embedding_msg = EmbeddingMsgConverter.from_context(memory)- enqueued = await self.vikingdb.enqueue_embedding_msg(embedding_msg)- if enqueued and embedding_msg.telemetry_id:+ if embedding_msg and embedding_msg.telemetry_id:request_wait_tracker.register_embedding_root(embedding_msg.telemetry_id, embedding_msg.id)+ enqueued = await self.vikingdb.enqueue_embedding_msg(embedding_msg)+ if embedding_msg and not enqueued and embedding_msg.telemetry_id:+ request_wait_tracker.mark_embedding_failed(+ embedding_msg.telemetry_id, embedding_msg.id, "embedding enqueue returned false"+ )logger.info(f"Enqueued memory for vectorization: {memory.uri}")self._record_semantic_change(memory.uri, change_type, parent_uri=memory.parent_uri)diff --git a/openviking/session/memory/memory_updater.py b/openviking/session/memory/memory_updater.pyindex e9b427292..6f2004189 100644--- a/openviking/session/memory/memory_updater.py+++ b/openviking/session/memory/memory_updater.py@@ -535,11 +535,17 @@ class MemoryUpdater:# Convert to embedding msg and enqueueembedding_msg = EmbeddingMsgConverter.from_context(memory_context)if embedding_msg:- enqueued = await self._vikingdb.enqueue_embedding_msg(embedding_msg)- if enqueued and embedding_msg.telemetry_id:+ if embedding_msg.telemetry_id:request_wait_tracker.register_embedding_root(embedding_msg.telemetry_id, embedding_msg.id)+ enqueued = await self._vikingdb.enqueue_embedding_msg(embedding_msg)+ if not enqueued and embedding_msg.telemetry_id:+ request_wait_tracker.mark_embedding_failed(+ embedding_msg.telemetry_id,+ embedding_msg.id,+ "embedding enqueue returned false",+ )logger.debug(f"Enqueued memory for vectorization: {uri}")except Exception as e:diff --git a/openviking/session/session.py b/openviking/session/session.pyindex 28c728439..bbe69f694 100644--- a/openviking/session/session.py+++ b/openviking/session/session.py@@ -987,6 +987,25 @@ class Session:user_id=self.ctx.user.user_id,)logger.info(f"Session {self.session_id} memory extraction completed")+ except asyncio.CancelledError as e:+ if redo_enabled and redo_task_id:+ redo_log.mark_done(redo_task_id)+ try:+ await self._write_failed_marker(+ archive_uri,+ stage="memory_extraction",+ error=f"cancelled: {e}",+ )+ except Exception:+ logger.debug("Failed to write cancelled marker for session %s", self.session_id)+ tracker.fail(+ task_id,+ f"cancelled: {e}",+ account_id=self.ctx.account_id,+ user_id=self.ctx.user.user_id,+ )+ logger.warning("Memory extraction cancelled for session %s", self.session_id)+ raiseexcept Exception as e:if redo_enabled and redo_task_id:redo_log.mark_done(redo_task_id)diff --git a/openviking/storage/content_write.py b/openviking/storage/content_write.pyindex 76a612ece..ac9c878c5 100644--- a/openviking/storage/content_write.py+++ b/openviking/storage/content_write.py@@ -396,9 +396,14 @@ class ContentWriteCoordinator:lifecycle_lock_handle_id=lifecycle_lock_handle_id,changes={change_type: [changed_uri]},)- await semantic_queue.enqueue(msg)if msg.telemetry_id:get_request_wait_tracker().register_semantic_root(msg.telemetry_id, msg.id)+ try:+ await semantic_queue.enqueue(msg)+ except Exception as e:+ if msg.telemetry_id:+ get_request_wait_tracker().mark_semantic_failed(msg.telemetry_id, msg.id, str(e))+ raiseasync def _enqueue_memory_refresh(self,@@ -423,9 +428,14 @@ class ContentWriteCoordinator:lifecycle_lock_handle_id=lifecycle_lock_handle_id,changes={"modified": [modified_uri]},)- await semantic_queue.enqueue(msg)if msg.telemetry_id:get_request_wait_tracker().register_semantic_root(msg.telemetry_id, msg.id)+ try:+ await semantic_queue.enqueue(msg)+ except Exception as e:+ if msg.telemetry_id:+ get_request_wait_tracker().mark_semantic_failed(msg.telemetry_id, msg.id, str(e))+ raiseasync def _wait_for_queues(self, *, timeout: Optional[float]) -> Dict[str, Any]:queue_manager = get_queue_manager()diff --git a/openviking/storage/queuefs/queue_manager.py b/openviking/storage/queuefs/queue_manager.pyindex a9d762804..222ec7c2f 100644--- a/openviking/storage/queuefs/queue_manager.py+++ b/openviking/storage/queuefs/queue_manager.py@@ -104,7 +104,7 @@ class QueueManager:for queue in list(self._queues.values()):self._start_queue_worker(queue)- logger.info("[QueueManager] Started")+ logger.info(f"[QueueManager] mount_point={self.mount_point} Started")def setup_standard_queues(self, vector_store: Any, start: bool = True) -> None:"""diff --git a/openviking/utils/agfs_utils.py b/openviking/utils/agfs_utils.pyindex 5788abf38..dfd7ce87a 100644--- a/openviking/utils/agfs_utils.py+++ b/openviking/utils/agfs_utils.py@@ -4,6 +4,7 @@RAGFS Client utilities for creating and configuring RAGFS clients."""+import multiprocessingimport osfrom pathlib import Pathfrom typing import Any, Dict@@ -13,15 +14,70 @@ from openviking_cli.utils.logger import get_loggerlogger = get_logger(__name__)-def _generate_plugin_config(agfs_config: Any, data_path: Path) -> Dict[str, Any]:- """Dynamically generate RAGFS plugin configuration based on backend type."""+def resolve_queuefs_mount_point(config: Any = None) -> str:+ """Resolve QueueFS mount point for the current process.++ `shared` keeps the historical global queue root (`/queue`).+ `worker` isolates each worker under `/queue/worker-<index|pid>`.+ """+ mode = None+ if config is not None:+ storage = getattr(config, "storage", None)+ if storage is None and hasattr(config, "agfs"):+ storage = config+ agfs = getattr(storage, "agfs", None) if storage is not None else None+ queuefs = getattr(agfs, "queuefs", None) if agfs is not None else None+ mode = getattr(queuefs, "mode", None)++ if not mode:+ try:+ from openviking_cli.utils.config import get_openviking_config++ mode = get_openviking_config().storage.agfs.queuefs.mode+ except Exception:+ mode = "shared"++ if mode == "worker":+ identity = getattr(multiprocessing.current_process(), "_identity", ())+ if identity:+ worker_id = str(identity[0] - 1)+ else:+ worker_id = str(os.getpid())+ return f"/queue/worker-{worker_id}"+ return "/queue"+++def _build_queuefs_plugin_config(agfs_config: Any, data_path: Path) -> Dict[str, Any]:+ """Build QueueFS plugin configuration from AGFS config with legacy compatibility."""default_queue_db_path = data_path / "_system" / "queue" / "queue.db"- configured_queue_db_path = getattr(agfs_config, "queue_db_path", None)- if configured_queue_db_path:- queue_db_path = str(Path(configured_queue_db_path).expanduser().resolve())- else:- queue_db_path = str(default_queue_db_path)+ queuefs_config = getattr(agfs_config, "queuefs", None)+ backend = getattr(queuefs_config, "backend", "sqlite") if queuefs_config else "sqlite"+ plugin_config: Dict[str, Any] = {+ "backend": backend,+ "recover_stale_sec": getattr(queuefs_config, "recover_stale_sec", 0),+ "busy_timeout_ms": getattr(queuefs_config, "busy_timeout_ms", 5000),+ }++ if backend in {"sqlite", "sqlite3"}:+ configured_queue_db_path = None+ if queuefs_config is not None:+ configured_queue_db_path = getattr(queuefs_config, "db_path", None)+ if not configured_queue_db_path:+ configured_queue_db_path = getattr(agfs_config, "queue_db_path", None)++ if configured_queue_db_path:+ queue_db_path = str(Path(configured_queue_db_path).expanduser().resolve())+ else:+ queue_db_path = str(default_queue_db_path)++ plugin_config["db_path"] = queue_db_path++ return plugin_config+++def _generate_plugin_config(agfs_config: Any, data_path: Path) -> Dict[str, Any]:+ """Dynamically generate RAGFS plugin configuration based on backend type."""config = {"serverinfofs": {"enabled": True,@@ -33,10 +89,7 @@ def _generate_plugin_config(agfs_config: Any, data_path: Path) -> Dict[str, Any]"queuefs": {"enabled": True,"path": "/queue",- "config": {- "backend": "sqlite",- "db_path": queue_db_path,- },+ "config": _build_queuefs_plugin_config(agfs_config, data_path),},}@@ -136,7 +189,6 @@ def mount_agfs_backend(agfs: Any, agfs_config: Any) -> None:vikingfs_path = data_path / "viking"vikingfs_path.mkdir(parents=True, exist_ok=True)- (data_path / "_system" / "queue").mkdir(parents=True, exist_ok=True)# 1. Mount standard pluginsconfig = _generate_plugin_config(agfs_config, data_path)diff --git a/openviking/utils/skill_processor.py b/openviking/utils/skill_processor.pyindex ad9cb5035..0d1f8d81a 100644--- a/openviking/utils/skill_processor.py+++ b/openviking/utils/skill_processor.py@@ -345,8 +345,14 @@ class SkillProcessor:context.set_vectorize(Vectorize(text=context.abstract))embedding_msg = EmbeddingMsgConverter.from_context(context)if embedding_msg:- enqueued = await self.vikingdb.enqueue_embedding_msg(embedding_msg)- if enqueued and embedding_msg.telemetry_id:+ if embedding_msg.telemetry_id:get_request_wait_tracker().register_embedding_root(embedding_msg.telemetry_id, embedding_msg.id)+ enqueued = await self.vikingdb.enqueue_embedding_msg(embedding_msg)+ if not enqueued and embedding_msg.telemetry_id:+ get_request_wait_tracker().mark_embedding_failed(+ embedding_msg.telemetry_id,+ embedding_msg.id,+ "embedding enqueue returned false",+ )diff --git a/openviking/utils/summarizer.py b/openviking/utils/summarizer.pyindex 4be0acb71..d3fb40a59 100644--- a/openviking/utils/summarizer.py+++ b/openviking/utils/summarizer.py@@ -107,9 +107,16 @@ class Summarizer:lifecycle_lock_handle_id=lifecycle_lock_handle_id,is_code_repo=kwargs.get("is_code_repo", False),)- await semantic_queue.enqueue(msg)if msg.telemetry_id:get_request_wait_tracker().register_semantic_root(msg.telemetry_id, msg.id)+ try:+ await semantic_queue.enqueue(msg)+ except Exception as e:+ if msg.telemetry_id:+ get_request_wait_tracker().mark_semantic_failed(+ msg.telemetry_id, msg.id, str(e)+ )+ raiseenqueued_count += 1logger.info(f"Enqueued semantic generation for: {target_uri} (skip_vectorization={skip_vectorization})"diff --git a/openviking_cli/utils/config/agfs_config.py b/openviking_cli/utils/config/agfs_config.pyindex c6e026e5e..39617addf 100644--- a/openviking_cli/utils/config/agfs_config.py+++ b/openviking_cli/utils/config/agfs_config.py@@ -99,6 +99,52 @@ class S3Config(BaseModel):return self+class QueueFSConfig(BaseModel):+ """Configuration for QueueFS backend."""++ mode: str = Field(+ default="shared",+ description="QueueFS namespace mode: 'shared' | 'worker'",+ )++ backend: str = Field(+ default="sqlite",+ description="QueueFS backend: 'memory' | 'sqlite' | 'sqlite3'",+ )++ db_path: Optional[str] = Field(+ default=None,+ description="SQLite database path for QueueFS when backend is 'sqlite' or 'sqlite3'.",+ )++ recover_stale_sec: int = Field(+ default=0,+ description="Recover processing messages older than this many seconds on startup (0 = recover all).",+ )++ busy_timeout_ms: int = Field(+ default=5000,+ description="SQLite busy timeout for QueueFS in milliseconds.",+ )++ model_config = {"extra": "forbid"}++ @model_validator(mode="after")+ def validate_config(self):+ valid_modes = {"shared", "worker"}+ if self.mode not in valid_modes:+ raise ValueError("queuefs mode must be one of: 'shared', 'worker'")++ valid_backends = {"memory", "sqlite", "sqlite3"}+ if self.backend not in valid_backends:+ raise ValueError("queuefs backend must be one of: 'memory', 'sqlite', 'sqlite3'")+ if self.recover_stale_sec < 0:+ raise ValueError("queuefs recover_stale_sec must be >= 0")+ if self.busy_timeout_ms < 0:+ raise ValueError("queuefs busy_timeout_ms must be >= 0")+ return self++class AGFSConfig(BaseModel):"""Configuration for RAGFS (Rust-based AGFS)."""@@ -150,6 +196,11 @@ class AGFSConfig(BaseModel):"Useful when the workspace volume does not support sqlite (e.g. some network filesystems).",)+ queuefs: QueueFSConfig = Field(+ default_factory=QueueFSConfig,+ description="QueueFS configuration.",+ )+retry_times: Any = Field(default=None,exclude=True,@@ -207,4 +258,17 @@ class AGFSConfig(BaseModel):# Validate S3 configurationself.s3.validate_config()+ if self.queue_db_path is not None and self.queuefs.db_path is None:+ logger.warning(+ "AGFSConfig: 'storage.agfs.queue_db_path' is deprecated; "+ "prefer 'storage.agfs.queuefs.db_path'."+ )++ if self.queuefs.backend == "memory":+ if self.queuefs.db_path is not None or self.queue_db_path is not None:+ logger.warning(+ "AGFSConfig: QueueFS backend is 'memory'; "+ "db_path/queue_db_path will be ignored."+ )+return selfdiff --git a/tests/misc/test_config_validation.py b/tests/misc/test_config_validation.pyindex 1acf2432a..8ea30c318 100644--- a/tests/misc/test_config_validation.py+++ b/tests/misc/test_config_validation.py@@ -6,7 +6,9 @@import sysfrom pathlib import Path-from openviking.utils.agfs_utils import _generate_plugin_config+import pytest++from openviking.utils.agfs_utils import _generate_plugin_config, mount_agfs_backendfrom openviking_cli.utils.config.agfs_config import AGFSConfig, S3Configfrom openviking_cli.utils.config.embedding_config import EmbeddingConfig, EmbeddingModelConfigfrom openviking_cli.utils.config.vectordb_config import VectorDBBackendConfig, VolcengineConfig@@ -95,6 +97,190 @@ def test_agfs_s3_normalize_encoding_chars_is_forwarded_to_ragfs_plugin_config():print(f" Fail: {e}")+def test_agfs_queuefs_defaults_to_sqlite_backend():+ config = AGFSConfig(path="/tmp/ov-test", backend="local")++ assert config.queuefs.mode == "shared"+ assert config.queuefs.backend == "sqlite"+ assert config.queuefs.recover_stale_sec == 0+ assert config.queuefs.busy_timeout_ms == 5000+++def test_agfs_queuefs_accepts_memory_backend():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={"mode": "worker", "backend": "memory"},+ )++ assert config.queuefs.mode == "worker"+ assert config.queuefs.backend == "memory"+++def test_agfs_queuefs_rejects_legacy_process_mode():+ with pytest.raises(ValueError, match="queuefs mode"):+ AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={"mode": "process"},+ )+++def test_agfs_queuefs_rejects_invalid_backend():+ with pytest.raises(ValueError, match="queuefs"):+ AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={"backend": "bogus"},+ )+++def test_agfs_queuefs_rejects_negative_timeouts():+ with pytest.raises(ValueError, match="busy_timeout_ms"):+ AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={"busy_timeout_ms": -1},+ )++ with pytest.raises(ValueError, match="recover_stale_sec"):+ AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={"recover_stale_sec": -1},+ )+++def test_generate_plugin_config_uses_queuefs_memory_backend_without_db_path():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={"backend": "memory"},+ )++ plugins = _generate_plugin_config(config, Path("/tmp/ov-test"))++ assert plugins["queuefs"]["config"]["backend"] == "memory"+ assert "db_path" not in plugins["queuefs"]["config"]+++def test_generate_plugin_config_uses_new_queuefs_db_path_over_legacy_field():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queue_db_path="/tmp/legacy-queue.db",+ queuefs={"backend": "sqlite", "db_path": "/tmp/new-queue.db"},+ )++ plugins = _generate_plugin_config(config, Path("/tmp/ov-test"))++ assert plugins["queuefs"]["config"]["backend"] == "sqlite"+ assert plugins["queuefs"]["config"]["db_path"] == str(Path("/tmp/new-queue.db").resolve())+++def test_generate_plugin_config_supports_legacy_queue_db_path():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queue_db_path="/tmp/legacy-queue.db",+ )++ plugins = _generate_plugin_config(config, Path("/tmp/ov-test"))++ assert plugins["queuefs"]["config"]["backend"] == "sqlite"+ assert plugins["queuefs"]["config"]["db_path"] == str(Path("/tmp/legacy-queue.db").resolve())+++def test_generate_plugin_config_uses_workspace_default_queue_db_path():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ )++ plugins = _generate_plugin_config(config, Path("/tmp/ov-test"))++ assert plugins["queuefs"]["config"]["backend"] == "sqlite"+ assert plugins["queuefs"]["config"]["db_path"] == "/tmp/ov-test/_system/queue/queue.db"+++def test_generate_plugin_config_forwards_queuefs_runtime_options():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queuefs={+ "backend": "sqlite3",+ "recover_stale_sec": 17,+ "busy_timeout_ms": 1234,+ },+ )++ plugins = _generate_plugin_config(config, Path("/tmp/ov-test"))++ assert plugins["queuefs"]["config"]["backend"] == "sqlite3"+ assert plugins["queuefs"]["config"]["recover_stale_sec"] == 17+ assert plugins["queuefs"]["config"]["busy_timeout_ms"] == 1234+++def test_generate_plugin_config_ignores_db_paths_in_memory_mode():+ config = AGFSConfig(+ path="/tmp/ov-test",+ backend="local",+ queue_db_path="/tmp/legacy-queue.db",+ queuefs={"backend": "memory", "db_path": "/tmp/new-queue.db"},+ )++ plugins = _generate_plugin_config(config, Path("/tmp/ov-test"))++ assert plugins["queuefs"]["config"]["backend"] == "memory"+ assert "db_path" not in plugins["queuefs"]["config"]+++class _FakeMountClient:+ def __init__(self):+ self.mount_calls = []++ def mount(self, plugin_name, mount_path, config):+ self.mount_calls.append((plugin_name, mount_path, config))++ def unmount(self, _mount_path):+ return None+++def test_mount_agfs_backend_skips_queue_sqlite_dirs_for_memory_backend(tmp_path):+ config = AGFSConfig(+ path=str(tmp_path),+ backend="local",+ queuefs={"backend": "memory"},+ )+ client = _FakeMountClient()++ mount_agfs_backend(client, config)++ assert (tmp_path / "viking").exists()+ assert not (tmp_path / "_system" / "queue").exists()+ queuefs_mount = next(call for call in client.mount_calls if call[0] == "queuefs")+ assert queuefs_mount[2]["backend"] == "memory"+ assert "db_path" not in queuefs_mount[2]+++def test_mount_agfs_backend_creates_queue_sqlite_dirs_for_sqlite_backend(tmp_path):+ queue_db_path = tmp_path / "custom-queue" / "queue.db"+ config = AGFSConfig(+ path=str(tmp_path),+ backend="local",+ queuefs={"backend": "sqlite", "db_path": str(queue_db_path)},+ )+ client = _FakeMountClient()++ mount_agfs_backend(client, config)++ assert (tmp_path / "viking").exists()+ assert queue_db_path.parent.exists()+ queuefs_mount = next(call for call in client.mount_calls if call[0] == "queuefs")+ assert queuefs_mount[2]["backend"] == "sqlite"+ assert queuefs_mount[2]["db_path"] == str(queue_db_path.resolve())++def test_vectordb_validation():"""Test VectorDB config validation"""print("\n" + "=" * 60)diff --git a/tests/server/test_bootstrap.py b/tests/server/test_bootstrap.pynew file mode 100644index 000000000..04918c2d9--- /dev/null+++ b/tests/server/test_bootstrap.py@@ -0,0 +1,147 @@+# Copyright (c) 2026 Beijing Volcano Engine Technology Co., Ltd.+# SPDX-License-Identifier: AGPL-3.0++from __future__ import annotations++import os+from types import SimpleNamespace++import openviking.server.bootstrap as bootstrap+from openviking.server.config import ServerConfig+from openviking.utils.agfs_utils import resolve_queuefs_mount_point+from openviking_cli.utils.config.storage_config import StorageConfig+++def test_main_keeps_config_host_when_cli_host_is_omitted(monkeypatch):+ config = ServerConfig(host="127.0.0.1", port=1933)+ captured: dict[str, object] = {}++ monkeypatch.setattr(+ bootstrap,+ "load_server_config",+ lambda config_path: config,+ )+ monkeypatch.setattr(+ bootstrap,+ "create_app",+ lambda config: "app",+ )+ monkeypatch.setattr(+ bootstrap,+ "configure_uvicorn_logging",+ lambda: None,+ )+ monkeypatch.setattr(+ bootstrap,+ "OpenVikingConfigSingleton",+ SimpleNamespace(initialize=lambda config_path: None),+ raising=False,+ )+ monkeypatch.setattr(+ bootstrap.argparse.ArgumentParser,+ "parse_args",+ lambda self: SimpleNamespace(+ host=None,+ port=None,+ config=None,+ workers=None,+ bot=False,+ with_bot=False,+ bot_url="http://localhost:18790",+ enable_bot_logging=None,+ bot_log_dir="/tmp/bot-logs",+ ),+ )+ monkeypatch.setattr(+ bootstrap.uvicorn,+ "run",+ lambda app, host, port, log_config=None, **kwargs: captured.update(+ {"app": app, "host": host, "port": port, "log_config": log_config, **kwargs}+ ),+ )++ bootstrap.main()++ assert captured["host"] == "127.0.0.1"+ assert captured["port"] == 1933+++def test_main_coerces_cli_host_all_to_none(monkeypatch):+ config = ServerConfig(host="127.0.0.1", port=1933)+ captured: dict[str, object] = {}++ monkeypatch.setattr(+ bootstrap,+ "load_server_config",+ lambda config_path: config,+ )+ monkeypatch.setattr(+ bootstrap,+ "create_app",+ lambda config: "app",+ )+ monkeypatch.setattr(+ bootstrap,+ "configure_uvicorn_logging",+ lambda: None,+ )+ monkeypatch.setattr(+ bootstrap,+ "OpenVikingConfigSingleton",+ SimpleNamespace(initialize=lambda config_path: None),+ raising=False,+ )+ monkeypatch.setattr(+ bootstrap.argparse.ArgumentParser,+ "parse_args",+ lambda self: SimpleNamespace(+ host="all",+ port=None,+ config=None,+ workers=None,+ bot=False,+ with_bot=False,+ bot_url="http://localhost:18790",+ enable_bot_logging=None,+ bot_log_dir="/tmp/bot-logs",+ ),+ )+ monkeypatch.setattr(+ bootstrap.uvicorn,+ "run",+ lambda app, host, port, log_config=None, **kwargs: captured.update(+ {"app": app, "host": host, "port": port, "log_config": log_config, **kwargs}+ ),+ )++ bootstrap.main()++ assert captured["host"] is None+ assert captured["port"] == 1933+++def test_resolve_queuefs_mount_point_defaults_to_shared():+ config = StorageConfig()++ assert resolve_queuefs_mount_point(config) == "/queue"+++def test_resolve_queuefs_mount_point_worker_mode_uses_process_index(monkeypatch):+ monkeypatch.setattr(+ "openviking.utils.agfs_utils.multiprocessing.current_process",+ lambda: SimpleNamespace(_identity=(3,)),+ )+ config = StorageConfig(agfs={"queuefs": {"mode": "worker"}})++ assert resolve_queuefs_mount_point(config) == "/queue/worker-2"+++def test_resolve_queuefs_mount_point_worker_mode_falls_back_to_pid(monkeypatch):+ monkeypatch.setattr(+ "openviking.utils.agfs_utils.multiprocessing.current_process",+ lambda: SimpleNamespace(_identity=()),+ )+ monkeypatch.setattr(os, "getpid", lambda: 43210)+ config = StorageConfig(agfs={"queuefs": {"mode": "worker"}})++ assert resolve_queuefs_mount_point(config) == "/queue/worker-43210"diff --git a/tests/test_server_config_loader.py b/tests/test_server_config_loader.pyindex abc14e1bd..5ffb145c5 100644--- a/tests/test_server_config_loader.py+++ b/tests/test_server_config_loader.py@@ -60,6 +60,7 @@ def test_load_server_config_preserves_supported_fields(tmp_path):"bot_api_url": "http://localhost:19999","observability": {"metrics": {"exporters": {"prometheus": {"enabled": True}}}},},+ "storage": {"agfs": {"queuefs": {"mode": "worker"}}},"encryption": {"enabled": True},})@@ -77,6 +78,14 @@ def test_load_server_config_preserves_supported_fields(tmp_path):assert config.encryption_enabled is True+def test_load_server_config_rejects_legacy_queuefs_scope(tmp_path):+ config_path = tmp_path / "ov.conf"+ config_path.write_text(json.dumps({"server": {"queuefs_scope": "process"}}))++ with pytest.raises(ValueError, match=r"server\.queuefs_scope"):+ load_server_config(str(config_path))++def test_load_bot_gateway_token_reads_token_from_bot_gateway_section(tmp_path):config_path = tmp_path / "ov.conf"config_path.write_text(json.dumps({"bot": {"gateway": {"token": "gateway-token"}}}))