Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Python ai4sciencedocument-analysisdocxextract-datalayout-analysisocrparserpdfpdf-converterpdf-extractor-llmpdf-extractor-pretrainpdf-extractor-ragpdf-parserpptxpythonxlsx
feat: add VLM client configuration support and update related API server functionality
myhloli3e98a997cbb8
diff --git a/docs/next/cli/mineru-kit-api-server.md b/docs/next/cli/mineru-kit-api-server.mdindex 4611fadc..1ebef0b5 100644--- a/docs/next/cli/mineru-kit-api-server.md+++ b/docs/next/cli/mineru-kit-api-server.md@@ -89,7 +89,47 @@ mineru-kit api-server --tier standard --language en --disable-image-analysisOCR 模式通过每次 `POST /v1/parse/jobs` 请求的 `ocr_mode` 设置,可选 `auto`、`txt`、`ocr`,省略时为 `auto`。启动参数 `--ocr-mode` 已移除,传入会报错;Python `create_app()` 同样不再接受该参数。-模型默认在首次解析时懒加载。`--preload-models` 会在 Basic 或 Standard 服务启动时提前加载所需模型,并在加载失败时让能力接口返回明确错误;Flash 没有本地模型,该参数对 Flash 无操作。Doclib managed parse-server 会自动启用模型预加载。+模型默认在首次解析时懒加载。`--preload-models` 会在 Basic 或 Standard 服务启动时提前初始化所需模型或 VLM 客户端,并在失败时让能力接口返回明确错误;Flash 没有本地模型,该参数对 Flash 无操作。Doclib managed parse-server 会自动启用模型预加载。++### 使用远程 MinerU VLM 服务++```bash+mineru-kit api-server --tier standard --vlm-server-url http://127.0.0.1:30000/v1+mineru-api --vlm-server-url https://vlm.example.com/proxy/v1 --vlm-model mineru-model --preload-models+```++两个命令和 `python -m mineru.parser.api_server` 支持相同的连接参数:++| 参数 | 默认值 | 说明 |+|------|--------|------|+| `--vlm-server-url` | 空 | 远程服务地址,空值选择本地 VLM |+| `--vlm-api-key` | 空 | 远程 VLM 服务的 Bearer Key |+| `--vlm-model` | 空 | 上游模型名;为空时通过 `/v1/models` 唯一发现 |+| `--vlm-http-timeout` | 600 | VLM HTTP 读写超时,单位秒,必须为正整数 |+| `--vlm-max-concurrency` | 100 | VLM 推理请求并发,必须为正整数 |++表中为未配置时的默认值。命令行只覆盖显式传入的字段,其余读取全局 `model.vlm`;+环境变量覆盖 YAML。详见 [远程 VLM 配置](../config.md#远程-vlm)。例如:++```bash+export MINERU_MODEL_VLM_SERVER_URL=http://127.0.0.1:30000/v1+export MINERU_MODEL_VLM_API_KEY=your-vlm-key+mineru-kit api-server --vlm-http-timeout 120 --vlm-max-concurrency 16+# 本次启动覆盖全局地址,恢复本地 VLM:+mineru-kit api-server --vlm-server-url ""+```++`--api-key` 认证解析 API 的调用方,`--vlm-api-key` 认证 API server 对上游 VLM 的请求,+两者独立。远程连接只用于 Standard/Advanced 的 VLM 推理,仍需本地 Hybrid 模型;+启动依赖预检不要求本地 VLM 引擎。Basic/Flash 不会连接远程 VLM。++启用 `--preload-models` 时,Standard 服务连接上游进行模型发现或校验,并预加载本地+Hybrid 模型;否则上游连接推迟到首次 Standard/Advanced 解析。连接、认证或模型校验失败+通过已有预加载错误或解析任务错误返回,不自动回退本地 VLM。推理重试继续使用底层默认设置。++Python 可使用 `create_app(vlm_config=VlmConfig(...))` 完整覆盖全局配置。配置由服务实例+传入其任务,不写回全局设置。此功能不增加 HTTP 请求字段;`/v1/models` 和 `/v1/tiers`+继续发布 MinerU 逻辑模型及质量档位,上游 serving 模型别名仅用于推理请求。启动完成后,HTTP API 不暴露 backend。`GET /v1/tiers` 也不新增 backend 字段;调用方如需推断实现,只能从 `current_model` 做弱推断。@@ -105,6 +145,7 @@ OCR 模式通过每次 `POST /v1/parse/jobs` 请求的 `ocr_mode` 设置,可- no-advanced- preload-models- API key+- vlm-server-url / vlm-api-key / vlm-model / vlm-http-timeout / vlm-max-concurrency### 稳定解析参数diff --git a/docs/next/cli/mineru-kit-vlm-server.md b/docs/next/cli/mineru-kit-vlm-server.mdindex bacedf75..cbfb8256 100644--- a/docs/next/cli/mineru-kit-vlm-server.md+++ b/docs/next/cli/mineru-kit-vlm-server.md@@ -21,10 +21,10 @@## 2. 与 api-server 的边界-`mineru-kit vlm-server` 可作为 `mineru-kit api-server` 的后端,尤其服务于:--- `vlm-http-client`-- `hybrid-http-client`+`mineru-kit vlm-server` 可作为 `mineru-kit api-server` 的远程 VLM 后端。+API 服务通过 `--vlm-server-url` 或全局 `model.vlm.server_url` 连接它,内部使用+`http-client` 执行 Standard/Advanced 的 VLM 推理;本地 Hybrid 处理仍在 API 服务进程中执行。+连接、鉴权及模型名配置见 [api-server](mineru-kit-api-server.md#使用远程-mineru-vlm-服务)。边界:diff --git a/docs/next/config.md b/docs/next/config.mdindex c7f41672..cd57ada3 100644--- a/docs/next/config.md+++ b/docs/next/config.md@@ -24,6 +24,51 @@ MinerU 有两类配置:- 默认 DB、日志和 UDS socket 从 `MINERU_HOME` 派生;默认 `data_dir` 是 `$MINERU_HOME/doclib`,其下包含 `parsed/` 和 `temp/`。- `MINERU_CONFIG` 仍可显式指定配置文件路径;`MINERU_HOME` 只影响默认 home 和默认配置文件位置。+### 远程 VLM++`model.vlm` 是所有本地解析入口的 VLM 默认配置,适用于 API server、Python+`parse()` / `parse_async()` 和 `mineru-kit parse`。Standard/Advanced 将 VLM 推理交给+远程 MinerU OpenAI-compatible 服务,本地继续执行 Hybrid 处理;Flash/Basic 不调用远程 VLM。+上游须部署支持 MinerU 文档理解输出的模型,并提供 `/v1/models` 和 `/v1/chat/completions`。++```yaml+model:+ vlm:+ server_url: http://127.0.0.1:30000/v1+ api_key: ${VLM_API_KEY:-}+ model: ""+ http_timeout: 600+ max_concurrency: 100+```++未配置 `server_url` 或显式设为空字符串时使用本地 VLM。地址可使用 HTTP/HTTPS+服务根地址或以 `/v1` 结尾的地址;反向代理前缀会保留,例如 `https://host/proxy/v1`+对应 `https://host/proxy/v1/chat/completions`。URL 不接受内嵌用户名、密码、query 或 fragment。+`api_key` 可为空;`model` 为空时要求上游模型列表恰好包含一个模型,多模型服务须显式指定模型名。+显式模型名也会通过模型列表校验。`http_timeout` 单位为秒,`max_concurrency` 是 VLM+推理请求并发,两者均为正整数;API 的 `--concurrency` 则控制解析任务并发。++环境变量包括 `MINERU_MODEL_VLM_SERVER_URL`、`MINERU_MODEL_VLM_API_KEY`、+`MINERU_MODEL_VLM_MODEL`、`MINERU_MODEL_VLM_HTTP_TIMEOUT` 和+`MINERU_MODEL_VLM_MAX_CONCURRENCY`。API 启动时的优先级为:显式启动参数 > 环境变量 >+YAML > 默认值。启动参数及示例见 [api-server](cli/mineru-kit-api-server.md)。++Python 的 `create_app()`、`MinerUParser`、`parse()` / `parse_async()` 以及底层分析入口+可传入 `vlm_config: VlmConfig | None`:`None` 使用全局设置;显式对象完整覆盖全局设置。+应用与解析器各自保存配置副本,不修改全局配置或进程环境。++```python+from mineru.config import VlmConfig+from mineru.parser import parse++result = parse("document.pdf", tier="standard") # 使用全局 model.vlm+local_result = parse("document.pdf", vlm_config=VlmConfig()) # 本次使用本地 VLM+```++远程模式不会自动回退本地 VLM。旧底层环境变量 `MINERU_VL_API_KEY`、+`MINERU_VL_MODEL_NAME` 若与有效的新配置冲突,会明确报错;应清除旧变量并使用上述+`MINERU_MODEL_VLM_*` 配置。未配置远程地址时,旧 `MINERU_VL_SERVER` 不会启用远程 VLM。+Markdown render 同样读取启动前 `config.yaml`,当前支持以下配置:```yamldiff --git a/mineru/backend/analysis/pdf/layout.py b/mineru/backend/analysis/pdf/layout.pyindex b6714f60..eb0014a2 100644--- a/mineru/backend/analysis/pdf/layout.py+++ b/mineru/backend/analysis/pdf/layout.py@@ -22,25 +22,6 @@ from .constants import ()-def _load_vlm_runtime() -> dict[str, Any]:- """按需加载 VLM runtime 组件,确保只有 high/extra_high 路径触发 VLM 依赖。"""- from ....model.vlm.runtime import (- ModelSingleton,- _get_model_async,- _maybe_enable_serial_execution,- aio_predictor_execution_guard,- predictor_execution_guard,- )-- return {- "ModelSingleton": ModelSingleton,- "_get_model_async": _get_model_async,- "_maybe_enable_serial_execution": _maybe_enable_serial_execution,- "aio_predictor_execution_guard": aio_predictor_execution_guard,- "predictor_execution_guard": predictor_execution_guard,- }--def _layout_item_to_content_block(layout_item: dict[str, Any], page_size: tuple[int, int]) -> dict | None:"""将本地 layout 小模型检测项转换为 mineru-vl-utils 的 ContentBlock。"""label = layout_item.get("label") or layout_item.get("type")diff --git a/mineru/backend/analysis/pdf/pipeline.py b/mineru/backend/analysis/pdf/pipeline.pyindex 4709b23a..8e5dd52e 100644--- a/mineru/backend/analysis/pdf/pipeline.py+++ b/mineru/backend/analysis/pdf/pipeline.py@@ -7,12 +7,12 @@ import timefrom typing import Any, castfrom ..contracts import AnalysisResult, AnalyzeEffort, ParseMode, ResolvedParseMode+from ....config import VlmConfigfrom ....model.runtime.hybrid import HybridLocalModelContextSingletonfrom ....model.runtime.memory import clean_memory-from ....model.vlm.selector import get_vlm_engine+from ....model.vlm.client import get_vlm_predictorfrom ....model.flash.pdf.document import PDFDocument-from .layout import _load_vlm_runtimefrom .normalization import _normalize_pdf_model_listfrom .window import process_pdf_windows@@ -24,6 +24,7 @@ def analyze_pdf(effort: AnalyzeEffort = "high",parse_mode: ParseMode = "auto",image_analysis: bool = True,+ vlm_config: VlmConfig | None = None,) -> AnalysisResult:"""生产 PDF model-list,并返回最终路由元数据和精确推理耗时。"""# 只在真实 PDF 分析开始时配置 MPS 回退,避免 import backend 修改进程环境。@@ -49,14 +50,7 @@ def analyze_pdf(hybrid_model = hybrid_model_singleton.get_model()if effort in ["high", "xhigh"]:- vlm_runtime = _load_vlm_runtime()- vlm_backend = get_vlm_engine(inference_engine="auto", is_async=False)- vlm_predictor = vlm_runtime["ModelSingleton"]().get_model(- backend=vlm_backend,- model_path=None,- server_url=None,- )- vlm_predictor = vlm_runtime["_maybe_enable_serial_execution"](vlm_predictor, vlm_backend)+ vlm_predictor, _vlm_backend = get_vlm_predictor(vlm_config)else:vlm_predictor = Nonediff --git a/mineru/backend/analyze.py b/mineru/backend/analyze.pyindex 1564b071..077b57bc 100644--- a/mineru/backend/analyze.py+++ b/mineru/backend/analyze.py@@ -9,7 +9,7 @@ from typing import castfrom loguru import loggerfrom .analysis.contracts import AnalyzeEffort, OfficeSuffix, ParseMode-from ..config import config+from ..config import VlmConfig, configfrom ..model.flash.html import HtmlSourceContextfrom ..types import FILE_SUFFIXES, FileSuffix, MiddleJson, ModelJsonfrom ..version import __version__ as mineru_version@@ -34,6 +34,7 @@ def doc_analyze(page_index_map: list[int] | None = None,file_suffix: FileSuffix = "pdf",source_context: HtmlSourceContext | None = None,+ vlm_config: VlmConfig | None = None,) -> tuple[MiddleJson, ModelJson]:"""生产严格 ModelJson,并在统一边界构造严格 MiddleJson。"""if file_suffix not in FILE_SUFFIXES:@@ -51,6 +52,7 @@ def doc_analyze(effort=effort,parse_mode=parse_mode,image_analysis=image_analysis,+ vlm_config=vlm_config,)elif file_suffix == "csv":from .analysis.csv import analyze_csv@@ -99,6 +101,7 @@ async def aio_doc_analyze(page_index_map: list[int] | None = None,file_suffix: FileSuffix = "pdf",source_context: HtmlSourceContext | None = None,+ vlm_config: VlmConfig | None = None,) -> tuple[MiddleJson, ModelJson]:"""在线程中执行统一文档分析,避免阻塞调用方事件循环。"""return await asyncio.to_thread(@@ -110,4 +113,5 @@ async def aio_doc_analyze(page_index_map=page_index_map,file_suffix=file_suffix,source_context=source_context,+ vlm_config=vlm_config,)diff --git a/mineru/config.py b/mineru/config.pyindex bd74d577..a64c04d2 100644--- a/mineru/config.py+++ b/mineru/config.py@@ -11,6 +11,7 @@ import tempfilefrom collections.abc import Sequencefrom dataclasses import dataclassfrom typing import Any, Literal, get_args, get_origin+from urllib.parse import urlsplit, urlunsplitimport yamlfrom filelock import FileLock@@ -422,10 +423,80 @@ class LLMAidedConfig(BaseModel):return self+class VlmConfig(BaseModel):+ """所有本地解析入口共享的远程 VLM 连接配置;空地址表示使用本地引擎。"""++ model_config = {"hide_input_in_errors": True}++ server_url: str = ""+ api_key: str = Field(default="", repr=False)+ model: str = ""+ http_timeout: int = Field(default=600, ge=1)+ max_concurrency: int = Field(default=100, ge=1)++ @field_validator("api_key", "model")+ @classmethod+ def _strip_connection_value(cls, value: str) -> str:+ """清理连接字段首尾空白,保持显式空值可覆盖全局配置。"""+ return value.strip()++ @field_validator("server_url")+ @classmethod+ def _normalize_server_url(cls, value: str) -> str:+ """保留代理前缀并移除末尾 v1,以尾斜线阻止底层客户端丢弃路径。"""+ value = value.strip()+ if not value:+ return ""+ try:+ parts = urlsplit(value)+ valid = (+ parts.scheme in {"http", "https"}+ and bool(parts.hostname)+ and parts.username is None+ and parts.password is None+ and not parts.query+ and not parts.fragment+ and not any(char.isspace() for char in value)+ )+ _ = parts.port+ except ValueError:+ valid = False+ if not valid:+ raise ValueError("model.vlm.server_url must be an HTTP(S) service URL without credentials, query or fragment")+ path = parts.path.rstrip("/")+ if path.endswith("/v1"):+ path = path[:-3]+ return urlunsplit((parts.scheme, parts.netloc, path + "/", "", ""))++ @field_validator("http_timeout", "max_concurrency", mode="before")+ @classmethod+ def _reject_boolean_limit(cls, value: Any) -> Any:+ """拒绝布尔数值,同时允许配置环境变量提供整数文本。"""+ if isinstance(value, bool):+ raise ValueError("VLM timeout and concurrency must be positive integers")+ return value++ def validate_environment(self) -> None:+ """远程推理前拒绝会隐式覆盖显式配置的旧环境变量,错误不包含凭据。"""+ if not self.server_url:+ return+ for env_name, expected, field in (+ ("MINERU_VL_API_KEY", self.api_key, "api_key"),+ ("MINERU_VL_MODEL_NAME", self.model, "model"),+ ):+ # 底层只清理 API Key;模型名按原始环境变量值读取。+ actual = os.getenv(env_name, "")+ if field == "api_key":+ actual = actual.strip()+ if actual and actual != expected:+ raise ValueError(f"{env_name} conflicts with model.vlm.{field}; unset the legacy environment variable")++class ModelConfig(BaseModel):base_dir: str = _default_path("models")source: str = "auto"stack: str = "auto" # "auto" | "light" | "full"+ vlm: VlmConfig = Field(default_factory=VlmConfig)class DoclibConfig(BaseModel):@@ -493,6 +564,7 @@ __all__ = ["ConfigSource","LoadedConfig","ModelConfig",+ "VlmConfig","ModelSource","config","Config",diff --git a/mineru/kit/commands/api_server.py b/mineru/kit/commands/api_server.pyindex 6a8fc287..ad8f2388 100644--- a/mineru/kit/commands/api_server.py+++ b/mineru/kit/commands/api_server.py@@ -49,16 +49,37 @@ def api_server_cmd(help="Hybrid medium OCR language hint; accepted by other efforts for compatibility",),disable_image_analysis: bool = typer.Option(False, "--disable-image-analysis", help="Disable image analysis"),- preload_models: bool = typer.Option(False, "--preload-models", help="Load local models during server startup"),+ preload_models: bool = typer.Option(+ False, "--preload-models", help="Initialize VLM client and local Hybrid models at startup"+ ),api_key: str | None = typer.Option(None, "--api-key", help="Optional fixed API key"),+ vlm_server_url: str | None = typer.Option(None, "--vlm-server-url", help="Remote VLM URL; empty value selects local VLM"),+ vlm_api_key: str | None = typer.Option(None, "--vlm-api-key", help="Bearer key for the remote VLM server"),+ vlm_model: str | None = typer.Option(None, "--vlm-model", help="Remote VLM model name; empty value enables discovery"),+ vlm_http_timeout: int | None = typer.Option(+ None, "--vlm-http-timeout", min=1, help="VLM HTTP timeout in seconds (default: 600)"+ ),+ vlm_max_concurrency: int | None = typer.Option(+ None, "--vlm-max-concurrency", min=1, help="VLM inference concurrency (default: 100)"+ ),) -> None:- """Start the self-hosted MinerU parse API server."""+ """转发显式启动参数,启动 self-hosted MinerU 解析 API 服务。"""try:normalized_language = validate_public_ocr_lang(language)except ValueError as exc:exit_with_message("invalid_request", str(exc), "language")tier_value = _normalize_tier_option(tier)tier_args = ["--tier", tier_value] if tier_value is not None else []+ vlm_args: list[str] = []+ for option, value in (+ ("--vlm-server-url", vlm_server_url),+ ("--vlm-api-key", vlm_api_key),+ ("--vlm-model", vlm_model),+ ("--vlm-http-timeout", vlm_http_timeout),+ ("--vlm-max-concurrency", vlm_max_concurrency),+ ):+ if value is not None:+ vlm_args.extend([option, str(value)])try:parser_api_server.main.main(args=[@@ -75,6 +96,7 @@ def api_server_cmd("--language",normalized_language,*tier_args,+ *vlm_args,*(["--no-flash"] if no_flash else []),*(["--no-advanced"] if no_advanced else []),*(["--allow-local-source"] if allow_local_source else []),diff --git a/mineru/model/vlm/__init__.py b/mineru/model/vlm/__init__.pyindex bba81235..f069a38e 100644--- a/mineru/model/vlm/__init__.py+++ b/mineru/model/vlm/__init__.py@@ -1,4 +1,4 @@# Copyright (c) Opendatalab. All rights reserved."""VLM model runtime helpers."""-__all__ = ["engine_utils", "runtime"]+__all__ = ["client", "engine_utils", "runtime"]diff --git a/mineru/model/vlm/client.py b/mineru/model/vlm/client.pynew file mode 100644index 00000000..82912ebf--- /dev/null+++ b/mineru/model/vlm/client.py@@ -0,0 +1,37 @@+"""解析与预加载共享的 VLM 客户端构造入口。"""++from __future__ import annotations++from typing import TYPE_CHECKING++from ...config import VlmConfig, config++if TYPE_CHECKING:+ from mineru_vl_utils import MinerUClient+++def get_vlm_predictor(vlm_config: VlmConfig | None = None) -> tuple[MinerUClient, str]:+ """按连接配置选择远程客户端或本地引擎,预加载与真实解析复用相同缓存。"""+ settings = (vlm_config if vlm_config is not None else config.model.vlm).model_copy(deep=True)+ settings.validate_environment()+ from .runtime import ModelSingleton++ if settings.server_url:+ predictor = ModelSingleton().get_model(+ backend="http-client",+ model_path=None,+ server_url=settings.server_url,+ model_name=settings.model or None,+ server_headers={"Authorization": f"Bearer {settings.api_key}"} if settings.api_key else {},+ http_timeout=settings.http_timeout,+ max_concurrency=settings.max_concurrency,+ )+ return predictor, "http-client"++ from .selector import get_vlm_engine++ engine = get_vlm_engine("auto", is_async=False)+ return ModelSingleton().get_model(backend=engine, model_path=None, server_url=None), engine+++__all__ = ["get_vlm_predictor"]diff --git a/mineru/model/vlm/runtime.py b/mineru/model/vlm/runtime.pyindex 10c818c2..fbe7b3ee 100644--- a/mineru/model/vlm/runtime.py+++ b/mineru/model/vlm/runtime.py@@ -4,6 +4,7 @@ from __future__ import annotationsimport asyncioimport atexitimport gc+import hashlibimport jsonimport osimport threading@@ -51,11 +52,25 @@ class ModelSingleton:],model_path: str | None,server_url: str | None,+ *,+ model_name: str | None = None,**kwargs: Any,) -> MinerUClient:- key = (backend, model_path, server_url)+ """缓存模型实例,远程客户端按模型、凭据与连接选项隔离。"""+ cache_key = (backend, model_path, server_url)+ if backend == "http-client":+ connection_options = {+ "model_name": model_name,+ "server_headers": kwargs.get("server_headers") or {},+ "http_timeout": kwargs.get("http_timeout", 600),+ "max_concurrency": kwargs.get("max_concurrency", 100),+ "max_retries": kwargs.get("max_retries", 3),+ "retry_backoff_factor": kwargs.get("retry_backoff_factor", 0.5),+ }+ fingerprint = hashlib.sha256(json.dumps(connection_options, sort_keys=True).encode()).hexdigest()+ cache_key = (*cache_key, fingerprint)with self._lock:- if key not in self._models:+ if cache_key not in self._models:start_time = time.time()model = Noneprocessor = None@@ -141,7 +156,7 @@ class ModelSingleton:from mineru_vl_utils.mlx_compat import load_mlx_modelmodel, processor = load_mlx_model(model_path)- else:+ elif backend != "http-client":if os.getenv("OMP_NUM_THREADS") is None:os.environ["OMP_NUM_THREADS"] = "1"@@ -256,6 +271,7 @@ class ModelSingleton:)predictor = MinerUClient(backend=backend,+ model_name=model_name,model=model,processor=processor,lmdeploy_engine=lmdeploy_engine,@@ -283,10 +299,10 @@ class ModelSingleton:"llama_cpp_engine": llama_cpp_engine,}_maybe_enable_serial_execution(predictor, backend)- self._models[key] = predictor+ self._models[cache_key] = predictorelapsed = round(time.time() - start_time, 2)logger.info(f"get {backend} predictor cost: {elapsed}s")- return self._models[key]+ return self._models[cache_key]def shutdown(self) -> None:with self._lock:diff --git a/mineru/parser/__init__.py b/mineru/parser/__init__.pyindex fd722b71..74ca7387 100644--- a/mineru/parser/__init__.py+++ b/mineru/parser/__init__.py@@ -4,6 +4,7 @@ from __future__ import annotationsfrom pathlib import Pathfrom typing import Literal+from ..config import VlmConfigfrom ..model.flash.html import HtmlSourceContextfrom ..types import Tierfrom .api_client import ApiJobStatus, MinerUApiParser@@ -33,9 +34,10 @@ def parse(image_analysis: bool = True,page_range: str = "",source_context: HtmlSourceContext | None = None,+ vlm_config: VlmConfig | None = None,) -> ParseResult:"""同步解析文档;source_context 仅供保留 HTML 原始来源的内部调用方使用。"""- parser = MinerUParser(tier=tier, parse_mode=ocr_mode, image_analysis=image_analysis)+ parser = MinerUParser(tier=tier, parse_mode=ocr_mode, image_analysis=image_analysis, vlm_config=vlm_config)return parser.parse(path, page_range=page_range, source_context=source_context)@@ -47,7 +49,8 @@ async def parse_async(image_analysis: bool = True,page_range: str = "",source_context: HtmlSourceContext | None = None,+ vlm_config: VlmConfig | None = None,) -> ParseResult:"""异步解析文档;source_context 仅供保留 HTML 原始来源的内部调用方使用。"""- parser = MinerUParser(tier=tier, parse_mode=ocr_mode, image_analysis=image_analysis)+ parser = MinerUParser(tier=tier, parse_mode=ocr_mode, image_analysis=image_analysis, vlm_config=vlm_config)return await parser.parse_async(path, page_range=page_range, source_context=source_context)diff --git a/mineru/parser/api_server.py b/mineru/parser/api_server.pyindex e033f874..9e412828 100644--- a/mineru/parser/api_server.py+++ b/mineru/parser/api_server.py@@ -39,6 +39,7 @@ from fastapi.responses import JSONResponsefrom pydantic import BaseModel, ConfigDict, Fieldfrom starlette.datastructures import State+from ..config import VlmConfig, config as mineru_configfrom ..errors import MineruErrorfrom ..filetypes import (PARSEABLE_EXTENSIONS,@@ -1332,7 +1333,9 @@ async def _run_job(max_inline_bytes: int = _MAX_INLINE_BYTES_DEFAULT,allow_http_source: bool = False,flash_enabled: bool = True,+ vlm_config: VlmConfig | None = None,) -> None:+ """执行解析任务,使用所属服务的 VLM 配置并记录每个文件的成功或失败。"""rec.status = "running"rec.started_at = JobStore._now()@@ -1386,6 +1389,7 @@ async def _run_job(image_analysis=image_analysis,page_range=page_range,source_context=source_context,+ vlm_config=vlm_config,)# collect outputs@@ -1844,6 +1848,7 @@ async def create_job(allow_http_source_val: bool = request.app.state.allow_http_sourceflash_enabled_val: bool = request.app.state.flash_enabledimage_analysis_val: bool = request.app.state.image_analysis+ vlm_config_val: VlmConfig = request.app.state.vlm_config# async — fire and forgetasync def _bg_run() -> None:@@ -1858,6 +1863,7 @@ async def create_job(max_inline_bytes=max_inline_bytes_val,allow_http_source=allow_http_source_val,flash_enabled=flash_enabled_val,+ vlm_config=vlm_config_val,)asyncio.create_task(_bg_run())@@ -2022,11 +2028,15 @@ def _model_ids_and_tiers_for_server_tiers(tiers: list[Tier]) -> tuple[list[str],return model_ids, tier_infos-def _preflight_tier_dependencies(tier: ServerTier) -> None:+def _preflight_tier_dependencies(tier: ServerTier, vlm_config: VlmConfig | None = None) -> None:+ """远程 VLM 仅预检本地 Hybrid 依赖,避免要求安装本地 VLM serving 引擎。"""if tier == "flash":return+ settings = vlm_config if vlm_config is not None else mineru_config.model.vlm+ if tier == "standard":+ settings.validate_environment()try:- ensure_tier_runtime_dependencies(tier)+ ensure_tier_runtime_dependencies("basic" if settings.server_url else tier)except TierDependencyError as exc:raise ParseServerStartupError(str(exc)) from exc@@ -2081,17 +2091,15 @@ def _preload_local_models(language: str) -> None:manager.get_atom_model(atom_model_name=AtomicModelName.OCR, lang="seal")-def _preload_server_models(tier: DeploymentTier, *, language: str) -> _ModelPreloadResult:+def _preload_server_models(tier: DeploymentTier, *, language: str, vlm_config: VlmConfig | None = None) -> _ModelPreloadResult:+ """使用与实际解析相同的 VLM 客户端,并预加载本地 Hybrid 模型。"""if tier == "basic":_preload_local_models(language)return _ModelPreloadResult(tier=tier, engine="hybrid-local")- from ..model.vlm.selector import get_vlm_engine+ from ..model.vlm.client import get_vlm_predictor- engine = get_vlm_engine("auto", is_async=True)- from ..model.vlm.runtime import ModelSingleton-- ModelSingleton().get_model(engine, None, None) # type: ignore[arg-type]+ _predictor, engine = get_vlm_predictor(vlm_config)_preload_local_models(language)return _ModelPreloadResult(tier=tier, engine=engine)@@ -2131,6 +2139,7 @@ def create_app(language: str = "ch",image_analysis: bool = True,preload_models: bool = False,+ vlm_config: VlmConfig | None = None,) -> FastAPI:"""Create a FastAPI application implementing the MinerU v1 REST API.@@ -2163,7 +2172,9 @@ def create_app(image_analysis:Whether image analysis is enabled for Hybrid backends.preload_models:- Load the configured local models during server startup. Flash has no local models to preload.+ Initialize the VLM client and local Hybrid models at startup. Ignored for Flash.+ vlm_config:+ Complete VLM connection configuration override; defaults to global model.vlm settings."""upload_dir = upload_dir or ""tier = _normalize_server_tier(tier)@@ -2173,7 +2184,8 @@ def create_app(startup_runtime = runtime_options_for_tier(tier)backend = startup_runtime.backendeffort = effort_for_tier(tier)- _preflight_tier_dependencies(tier)+ vlm_config = (vlm_config if vlm_config is not None else mineru_config.model.vlm).model_copy(deep=True)+ _preflight_tier_dependencies(tier, vlm_config)_api_key: str | None = api_key or None_upload_dir = pathlib.Path(upload_dir) if upload_dir else pathlib.Path(tempfile.mkdtemp(prefix="mineru_"))_upload_dir.mkdir(parents=True, exist_ok=True)@@ -2206,14 +2218,16 @@ def create_app(application.state.effort = effortapplication.state.image_analysis = image_analysisapplication.state.preload_models = preload_models+ application.state.vlm_config = vlm_configapplication.state.model_preload_error = Noneif _preload_tier is not None:- logger.info("Preloading local models for startup tier %s", _preload_tier)+ logger.info("Initializing VLM client and local models for startup tier %s", _preload_tier)try:preload_result = await asyncio.to_thread(_preload_server_models,_preload_tier,language=language,+ vlm_config=vlm_config,)except Exception as exc:error_code, error_msg = _classify_model_preload_error(exc)@@ -2221,10 +2235,10 @@ def create_app(code=error_code,message=error_msg,)- logger.exception("Local model preload failed for startup tier %s", _preload_tier)+ logger.exception("Model initialization failed for startup tier %s", _preload_tier)else:logger.info(- "Local model preload completed for startup tier %s (engine=%s)",+ "Model initialization completed for startup tier %s (engine=%s)",_preload_tier,preload_result.engine,)@@ -2260,6 +2274,7 @@ def create_app(application.state.effort = effortapplication.state.image_analysis = image_analysisapplication.state.preload_models = preload_models+ application.state.vlm_config = vlm_configapplication.state.model_preload_error = NoneFileStore(_upload_dir).install(application.state)JobStore(concurrency=concurrency).install(application.state)@@ -2398,7 +2413,16 @@ def create_app(@click.option("--preload-models",is_flag=True,- help="Load local models during server startup. Ignored for Flash.",+ help="Initialize VLM client and local Hybrid models at startup. Ignored for Flash.",+)[email protected]("--vlm-server-url", default=None, type=str, help="Remote VLM URL; empty value selects local VLM.")[email protected]("--vlm-api-key", default=None, type=str, help="Bearer key for the remote VLM server.")[email protected]("--vlm-model", default=None, type=str, help="Remote VLM model name; empty value enables discovery.")+ "--vlm-http-timeout", default=None, type=click.IntRange(min=1), help="VLM HTTP timeout in seconds (default: 600)."+)+ "--vlm-max-concurrency", default=None, type=click.IntRange(min=1), help="VLM inference concurrency (default: 100).")@click.option("--api-key",@@ -2422,8 +2446,13 @@ def main(disable_image_analysis: bool,preload_models: bool,api_key: str | None,+ vlm_server_url: str | None,+ vlm_api_key: str | None,+ vlm_model: str | None,+ vlm_http_timeout: int | None,+ vlm_max_concurrency: int | None,) -> None:- """Start the MinerU v1 REST API server."""+ """合并显式 VLM 参数后启动 MinerU v1 REST API 服务,不修改全局配置。"""configure_standard_streams()shutdown_requested = threading.Event()server_ref: list[uvicorn.Server | None] = [None]@@ -2444,6 +2473,19 @@ def main(if control_watcher is not None:control_watcher.start()try:+ vlm_overrides = {+ "server_url": vlm_server_url,+ "api_key": vlm_api_key,+ "model": vlm_model,+ "http_timeout": vlm_http_timeout,+ "max_concurrency": vlm_max_concurrency,+ }+ vlm_settings = VlmConfig.model_validate(+ {+ **mineru_config.model.vlm.model_dump(),+ **{key: value for key, value in vlm_overrides.items() if value is not None},+ }+ )application = create_app(upload_dir=upload_dir,tier=tier,@@ -2458,6 +2500,7 @@ def main(language=language,image_analysis=not disable_image_analysis,preload_models=preload_models,+ vlm_config=vlm_settings,)except ParseServerStartupError as exc:if control_watcher is not None:diff --git a/mineru/parser/mineru_parser.py b/mineru/parser/mineru_parser.pyindex a8dfccbd..445ead3c 100644--- a/mineru/parser/mineru_parser.py+++ b/mineru/parser/mineru_parser.py@@ -11,6 +11,7 @@ from pathlib import Pathfrom typing import Any, Literal, castfrom ..backend.analyze import aio_doc_analyze, doc_analyze+from ..config import VlmConfig, configfrom ..errors import InvalidRequestErrorfrom ..filetypes import IMAGE_EXTENSIONS, PAGE_RANGE_PARSE_EXTENSIONSfrom ..model.flash.html import HtmlSourceContext@@ -49,11 +50,14 @@ class MinerUParser(DocumentParser):tier: Tier = "standard",parse_mode: _ParseMode = "auto",image_analysis: bool = True,+ vlm_config: VlmConfig | None = None,) -> None:+ """保存当前解析器的 VLM 配置副本,避免其他应用或调用修改连接设置。"""self.tier: Tier = tierself.effort: _Effort = effort_for_tier(tier) # type: ignore[assignment]self.parse_mode: _ParseMode = parse_modeself.image_analysis: bool = image_analysis+ self.vlm_config = (vlm_config if vlm_config is not None else config.model.vlm).model_copy(deep=True)def parse(self,@@ -100,6 +104,7 @@ class MinerUParser(DocumentParser):return self._build_result(middle_json, model_output)def _run_analysis(self, prepared: _PreparedInput) -> tuple[MiddleJson, ModelJson]:+ """将准备后的输入和实例连接配置传入同步分析入口。"""return doc_analyze(prepared.file_bytes,effort=self.effort,@@ -108,9 +113,11 @@ class MinerUParser(DocumentParser):page_index_map=prepared.retained_page_indices,file_suffix=prepared.file_suffix,source_context=prepared.source_context,+ vlm_config=self.vlm_config,)async def _arun_analysis(self, prepared: _PreparedInput) -> tuple[MiddleJson, ModelJson]:+ """将准备后的输入和实例连接配置传入异步分析入口。"""return await aio_doc_analyze(prepared.file_bytes,effort=self.effort,@@ -119,6 +126,7 @@ class MinerUParser(DocumentParser):page_index_map=prepared.retained_page_indices,file_suffix=prepared.file_suffix,source_context=prepared.source_context,+ vlm_config=self.vlm_config,)def _prepare_input(diff --git a/tests/unittest/test_parser_api_contract.py b/tests/unittest/test_parser_api_contract.pyindex b4c64925..5f858a76 100644--- a/tests/unittest/test_parser_api_contract.py+++ b/tests/unittest/test_parser_api_contract.py@@ -22,6 +22,7 @@ from pydantic import ValidationErrorimport mineru.parser.api_client as api_clientimport mineru.parser.api_server as api_serverimport mineru.parser.tier as parser_tier+from mineru.config import VlmConfigfrom mineru.parser import MIDDLE_JSON_SCHEMA_VERSIONfrom mineru.parser.api_client import MinerUApiParser, _pages_from_middle_json, _parse_result_from_job, should_trust_env_for_urlfrom mineru.parser.api_server import (@@ -2413,7 +2414,10 @@ def test_api_server_model_preload_failure_keeps_health_diagnostics_and_rejects_c) -> None:_stub_api_server_dependency_preflight(monkeypatch)- def _fail_preload(startup_tier: DeploymentTier, *, language: str) -> api_server._ModelPreloadResult:+ def _fail_preload(+ startup_tier: DeploymentTier, *, language: str, vlm_config: VlmConfig | None = None+ ) -> api_server._ModelPreloadResult:+ """模拟包含 VLM 初始化在内的服务预加载失败。"""raise ValueError("CUDA is not available.")monkeypatch.setattr(api_server, "_preload_server_models", _fail_preload)@@ -2437,7 +2441,10 @@ def test_api_server_model_preload_is_opt_in_and_ignored_for_flash(tmp_path: Path_stub_api_server_dependency_preflight(monkeypatch)calls: list[tuple[str, str]] = []- def _preload(startup_tier: DeploymentTier, *, language: str) -> api_server._ModelPreloadResult:+ def _preload(+ startup_tier: DeploymentTier, *, language: str, vlm_config: VlmConfig | None = None+ ) -> api_server._ModelPreloadResult:+ """记录预加载调用,兼容显式 VLM 配置传入。"""calls.append((startup_tier, language))return api_server._ModelPreloadResult(tier=startup_tier, engine="test")diff --git a/tests/unittest/test_parser_model_preload.py b/tests/unittest/test_parser_model_preload.pyindex b073c85f..7e4ecc95 100644--- a/tests/unittest/test_parser_model_preload.py+++ b/tests/unittest/test_parser_model_preload.py@@ -2,10 +2,12 @@ from __future__ import annotationsimport sysimport types+from pathlib import Pathimport pytestimport mineru.parser.api_server as api_server+from mineru.config import VlmConfigdef test_preload_local_models_initializes_conditional_model_families(monkeypatch: pytest.MonkeyPatch) -> None:@@ -74,6 +76,34 @@ def test_preload_standard_models_initializes_platform_engine_and_local_models(moassert calls == [("vlm", "lmdeploy-engine", None, None), ("local", "en")]+def test_local_preload_and_parse_reuse_llama_predictor(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:+ """本地预加载与解析共用同步引擎缓存,llama 参数循环不能覆盖模型缓存键。"""+ from unittest.mock import MagicMock++ from mineru.model.vlm import runtime, selector+ from mineru.model.vlm.client import get_vlm_predictor++ engine_factory = MagicMock()+ predictor_factory = MagicMock()+ engine_module = types.ModuleType("mineru_llama_cpp")+ engine_module.Engine = engine_factory+ monkeypatch.setitem(sys.modules, "mineru_llama_cpp", engine_module)+ monkeypatch.setattr(runtime.ModelSingleton, "_models", {})+ monkeypatch.setattr(runtime, "MinerUClient", predictor_factory)+ monkeypatch.setattr(type(runtime.MINERU_2_5_PRO_2605_1_2B), "ensure", lambda self: tmp_path)+ engine_selector = MagicMock(return_value="llama-cpp-engine")+ monkeypatch.setattr(selector, "get_vlm_engine", engine_selector)+ monkeypatch.setattr(api_server, "_preload_local_models", lambda language: None)++ api_server._preload_server_models("standard", language="ch", vlm_config=VlmConfig())+ predictor, engine = get_vlm_predictor(VlmConfig())+ assert predictor is predictor_factory.return_value+ assert engine == "llama-cpp-engine"+ predictor_factory.assert_called_once()+ engine_factory.assert_called_once()+ assert all(call.kwargs == {"is_async": False} for call in engine_selector.call_args_list)++@pytest.mark.parametrize(("exc", "expected_code"),[diff --git a/tests/unittest/test_pdf_analyze_block_content.py b/tests/unittest/test_pdf_analyze_block_content.pyindex dca7ce40..3581aa1e 100644--- a/tests/unittest/test_pdf_analyze_block_content.py+++ b/tests/unittest/test_pdf_analyze_block_content.py@@ -164,9 +164,6 @@ def test_doc_analyze_converts_vlm_results_before_downstream_processing(vlm_predictor = MagicMock()getattr(vlm_predictor, vlm_method_name).return_value = [source_page]- vlm_singleton = MagicMock()- vlm_singleton.get_model.return_value = vlm_predictor- serial_wrapper = MagicMock(return_value=vlm_predictor)page_image = Image.new("RGB", (64, 96), "white")def fake_process_text_and_formulas(@@ -196,15 +193,9 @@ def test_doc_analyze_converts_vlm_results_before_downstream_processing(monkeypatch.setattr(pipeline, "HybridLocalModelContextSingleton", MagicMock(return_value=hybrid_singleton))monkeypatch.setattr(pipeline,- "_load_vlm_runtime",- MagicMock(- return_value={- "ModelSingleton": MagicMock(return_value=vlm_singleton),- "_maybe_enable_serial_execution": serial_wrapper,- }- ),+ "get_vlm_predictor",+ MagicMock(return_value=(vlm_predictor, "transformers")),)- monkeypatch.setattr(pipeline, "get_vlm_engine", MagicMock(return_value="transformers"))monkeypatch.setattr(window,"load_images_from_pdf_bytes_range",diff --git a/tests/unittest/test_pdf_analyze_visual_blocks.py b/tests/unittest/test_pdf_analyze_visual_blocks.pyindex bcf137cc..741a8cb2 100644--- a/tests/unittest/test_pdf_analyze_visual_blocks.py+++ b/tests/unittest/test_pdf_analyze_visual_blocks.py@@ -1266,6 +1266,7 @@ def test_aio_doc_analyze_runs_sync_entrypoint_in_thread_and_forwards_arguments("page_index_map": [3, 5],"file_suffix": "pptx","source_context": None,+ "vlm_config": None,}assert observed["thread_id"] != caller_thread_id@@ -1476,8 +1477,7 @@ def test_pdf_flash_ocr_uses_local_ocr_without_vlm(hybrid_singleton = MagicMock()hybrid_singleton.get_model.return_value = hybrid_modelprocess_pdf_windows = MagicMock(return_value=[])- load_vlm_runtime = MagicMock()- get_vlm_engine = MagicMock()+ get_vlm_predictor = MagicMock()monkeypatch.setattr(pipeline, "PDFDocument", MagicMock(return_value=fake_document))monkeypatch.setattr(@@ -1487,8 +1487,7 @@ def test_pdf_flash_ocr_uses_local_ocr_without_vlm()monkeypatch.setattr(pipeline, "process_pdf_windows", process_pdf_windows)monkeypatch.setattr(pipeline, "clean_memory", MagicMock())- monkeypatch.setattr(pipeline, "_load_vlm_runtime", load_vlm_runtime)- monkeypatch.setattr(pipeline, "get_vlm_engine", get_vlm_engine)+ monkeypatch.setattr(pipeline, "get_vlm_predictor", get_vlm_predictor)result = pipeline.analyze_pdf(b"ocr-pdf",@@ -1505,8 +1504,7 @@ def test_pdf_flash_ocr_uses_local_ocr_without_vlm(fake_document.classify.assert_called_once_with()else:fake_document.classify.assert_not_called()- load_vlm_runtime.assert_not_called()- get_vlm_engine.assert_not_called()+ get_vlm_predictor.assert_not_called()@pytest.mark.parametrize("parse_mode", ["txt", "auto"])@@ -1518,14 +1516,12 @@ def test_pdf_flash_txt_skips_all_neural_model_loading(fake_document = MagicMock()fake_document.classify.return_value = "txt"hybrid_model_factory = MagicMock()- load_vlm_runtime = MagicMock()- get_vlm_engine = MagicMock()+ get_vlm_predictor = MagicMock()clean_memory = MagicMock()process_pdf_windows = MagicMock(return_value=[])monkeypatch.setattr(pipeline, "PDFDocument", MagicMock(return_value=fake_document))monkeypatch.setattr(pipeline, "HybridLocalModelContextSingleton", hybrid_model_factory)- monkeypatch.setattr(pipeline, "_load_vlm_runtime", load_vlm_runtime)- monkeypatch.setattr(pipeline, "get_vlm_engine", get_vlm_engine)+ monkeypatch.setattr(pipeline, "get_vlm_predictor", get_vlm_predictor)monkeypatch.setattr(pipeline, "clean_memory", clean_memory)monkeypatch.setattr(pipeline, "process_pdf_windows", process_pdf_windows)@@ -1545,8 +1541,7 @@ def test_pdf_flash_txt_skips_all_neural_model_loading(else:fake_document.classify.assert_not_called()hybrid_model_factory.assert_not_called()- load_vlm_runtime.assert_not_called()- get_vlm_engine.assert_not_called()+ get_vlm_predictor.assert_not_called()clean_memory.assert_not_called()@@ -1574,19 +1569,11 @@ def test_pdf_infer_timer_excludes_hybrid_vlm_initialization_and_cleanup(hybrid_singleton.get_model.side_effect = fake_hybrid_get_modelvlm_predictor = MagicMock()- vlm_singleton = MagicMock()- def fake_vlm_get_model(**_kwargs: object) -> MagicMock:+ def fake_vlm_get_model(_config: object) -> tuple[MagicMock, str]:"""记录 VLM predictor 初始化顺序。"""events.append("vlm_init")- return vlm_predictor-- vlm_singleton.get_model.side_effect = fake_vlm_get_model-- def fake_enable_serial_execution(predictor: MagicMock, _backend: str) -> MagicMock:- """记录 VLM predictor 包装顺序并原样返回。"""- events.append("vlm_wrap")- return predictor+ return vlm_predictor, "transformers"perf_counter_values = iter([20.0, 22.0])@@ -1606,15 +1593,7 @@ def test_pdf_infer_timer_excludes_hybrid_vlm_initialization_and_cleanup(monkeypatch.setattr(pipeline, "PDFDocument", MagicMock(return_value=fake_document))monkeypatch.setattr(pipeline, "HybridLocalModelContextSingleton", MagicMock(return_value=hybrid_singleton))- monkeypatch.setattr(- pipeline,- "_load_vlm_runtime",- lambda: {- "ModelSingleton": MagicMock(return_value=vlm_singleton),- "_maybe_enable_serial_execution": fake_enable_serial_execution,- },- )- monkeypatch.setattr(pipeline, "get_vlm_engine", MagicMock(return_value="transformers"))+ monkeypatch.setattr(pipeline, "get_vlm_predictor", fake_vlm_get_model)monkeypatch.setattr(pipeline.time, "perf_counter", fake_perf_counter)monkeypatch.setattr(pipeline, "_normalize_pdf_model_list", fake_normalize_model_list)monkeypatch.setattr(pipeline, "clean_memory", fake_clean_memory)@@ -1635,7 +1614,6 @@ def test_pdf_infer_timer_excludes_hybrid_vlm_initialization_and_cleanup(assert events == ["hybrid_init","vlm_init",- "vlm_wrap","timer_20.0","normalize_model_list","timer_22.0",diff --git a/tests/unittest/test_vlm_config.py b/tests/unittest/test_vlm_config.pynew file mode 100644index 00000000..9b9159fd--- /dev/null+++ b/tests/unittest/test_vlm_config.py@@ -0,0 +1,172 @@+from __future__ import annotations++import os+from pathlib import Path+from typing import Any++import pytest+from click.testing import CliRunner+from fastapi.testclient import TestClient+from pydantic import ValidationError+from typer.testing import CliRunner as TyperRunner++from mineru.config import Config, VlmConfig, _load_effective_config+from mineru.kit.commands import api_server as kit_api+from mineru.kit.main import app as kit_app+from mineru.parser import MinerUParser, api_server+++def test_vlm_config_defaults_and_environment_precedence(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ """验证 YAML 和标准环境变量的字段覆盖、配置来源以及无配置默认值。"""+ assert Config().model.vlm == VlmConfig(server_url="", api_key="", model="", http_timeout=600, max_concurrency=100)+ config_file = tmp_path / "config.yaml"+ config_file.write_text(+ "model:\n vlm:\n server_url: http://localhost:9000/proxy/v1\n"+ " api_key: file-key\n model: file-model\n http_timeout: 30\n max_concurrency: 2\n",+ encoding="utf-8",+ )+ monkeypatch.setenv("MINERU_CONFIG", str(config_file))+ monkeypatch.setenv("MINERU_MODEL_VLM_API_KEY", "env-key")+ monkeypatch.setenv("MINERU_MODEL_VLM_HTTP_TIMEOUT", "45")+ loaded = _load_effective_config()+ assert loaded.config.model.vlm == VlmConfig(+ server_url="http://localhost:9000/proxy/", api_key="env-key", model="file-model", http_timeout=45, max_concurrency=2+ )+ assert loaded.sources[("model", "vlm", "server_url")] == "file"+ assert loaded.sources[("model", "vlm", "api_key")] == "env"+ assert "env-key" not in repr(loaded.config.model.vlm)+++ ("value", "expected"),+ [+ ("", ""),+ (" ", ""),+ ("http://localhost:9000", "http://localhost:9000/"),+ ("https://example.test/v1/", "https://example.test/"),+ ("https://example.test/proxy/v1", "https://example.test/proxy/"),+ ("https://example.test/proxy/", "https://example.test/proxy/"),+ ("http://[::1]:9000/v1", "http://[::1]:9000/"),+ ],+)+def test_vlm_url_normalization(value: str, expected: str) -> None:+ """验证服务根地址、v1、代理前缀和 IPv6 的稳定规范化。"""+ settings = VlmConfig(server_url=value)+ assert settings.server_url == expected+ assert VlmConfig.model_validate(settings.model_dump()) == settings+++ "value",+ [+ "ftp://host",+ "host:9000",+ "http://",+ "http://host:bad",+ "http://user:pass@host",+ "http://host?q=1",+ "http://host/#x",+ "http://bad host",+ ],+)+def test_vlm_rejects_invalid_url(value: str) -> None:+ """非法服务地址在配置边界失败,避免错误地址进入 HTTP 客户端。"""+ with pytest.raises(ValidationError, match="HTTP"):+ VlmConfig(server_url=value)++[email protected]("field", ["http_timeout", "max_concurrency"])[email protected]("value", [0, -1, True, 1.5])+def test_vlm_rejects_invalid_limits(field: str, value: object) -> None:+ """连接超时和推理并发只接受正整数。"""+ with pytest.raises(ValidationError):+ VlmConfig.model_validate({field: value})++[email protected](("env_name", "field"), [("MINERU_VL_API_KEY", "api_key"), ("MINERU_VL_MODEL_NAME", "model")])+def test_vlm_legacy_conflicts_are_explicit_and_redacted(monkeypatch: pytest.MonkeyPatch, env_name: str, field: str) -> None:+ """拒绝旧变量覆盖新配置,异常不包含任一凭据值,本地模式不受影响。"""+ monkeypatch.delenv("MINERU_VL_API_KEY", raising=False)+ monkeypatch.delenv("MINERU_VL_MODEL_NAME", raising=False)+ monkeypatch.setenv(env_name, "legacy-secret")+ with pytest.raises(ValueError, match=env_name) as caught:+ VlmConfig.model_validate({"server_url": "http://localhost:9000", field: "explicit-secret"}).validate_environment()+ assert "secret" not in str(caught.value)+ VlmConfig.model_validate({"server_url": "http://localhost:9000", field: "legacy-secret"}).validate_environment()+ VlmConfig().validate_environment()++[email protected]("standalone", [False, True])+def test_vlm_options_forward_including_empty_values(monkeypatch: pytest.MonkeyPatch, standalone: bool) -> None:+ """新版命令及独立别名保留所有显式 VLM 值,包括用于清空配置的空字符串。"""+ calls: list[list[str]] = []++ def fake_main(*, args: list[str], prog_name: str, standalone_mode: bool) -> None:+ """记录命令桥接参数,不启动服务。"""+ calls.append(args)++ monkeypatch.setattr(api_server.main, "main", fake_main)+ options = [+ "--vlm-server-url",+ "",+ "--vlm-api-key",+ "",+ "--vlm-model",+ "alias",+ "--vlm-http-timeout",+ "12",+ "--vlm-max-concurrency",+ "3",+ ]+ if standalone:+ monkeypatch.setattr("sys.argv", ["mineru-api", *options])+ with pytest.raises(SystemExit) as caught:+ kit_api.main()+ assert caught.value.code == 0+ else:+ assert TyperRunner().invoke(kit_app, ["api-server", *options]).exit_code == 0+ for option, value in zip(options[::2], options[1::2]):+ assert calls[0][calls[0].index(option) + 1] == value++[email protected]("url", ["http://cli.test/proxy/v1", ""])+def test_api_cli_merges_vlm_without_mutating_global(monkeypatch: pytest.MonkeyPatch, url: str) -> None:+ """底层 CLI 按字段覆盖全局设置,并保持 API Key、全局配置和环境变量隔离。"""+ settings = VlmConfig(server_url="http://global.test", api_key="global-key", model="global-model", max_concurrency=7)+ monkeypatch.setattr(api_server.mineru_config.model, "vlm", settings)+ seen: dict[str, Any] = {}++ def capture_app(**kwargs: Any) -> object:+ """记录应用构造参数,用于核验 CLI 覆盖而不创建文件。"""+ seen.update(kwargs)+ return object()++ monkeypatch.setattr(api_server, "create_app", capture_app)+ monkeypatch.setattr(api_server.uvicorn.Server, "run", lambda self: None)+ previous_env = dict(os.environ)+ result = CliRunner().invoke(+ api_server.main, ["--vlm-server-url", url, "--vlm-api-key", "", "--vlm-http-timeout", "9", "--api-key", "inbound-key"]+ )+ assert result.exit_code == 0, result.output+ assert seen["api_key"] == "inbound-key"+ assert seen["vlm_config"] == VlmConfig(server_url=url, api_key="", model="global-model", http_timeout=9, max_concurrency=7)+ assert settings.server_url == "http://global.test/"+ assert settings.api_key == "global-key"+ assert dict(os.environ) == previous_env+++def test_vlm_app_and_parser_snapshot_configuration(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ """应用及解析器捕获独立副本,显式空配置完整覆盖全局远程设置。"""+ monkeypatch.setattr(api_server, "ensure_tier_runtime_dependencies", lambda tier: None)+ settings = VlmConfig(server_url="http://remote.test", model="alias")+ monkeypatch.setattr(api_server.mineru_config.model, "vlm", settings)+ app = api_server.create_app(upload_dir=str(tmp_path))+ parser = MinerUParser()+ local_parser = MinerUParser(vlm_config=VlmConfig())+ settings.model = "changed"+ assert parser.vlm_config.model == app.state.vlm_config.model == "alias"+ assert parser.vlm_config is not app.state.vlm_config+ assert local_parser.vlm_config.server_url == ""+ with TestClient(app) as client:+ assert client.get("/v1/health").status_code == 200+ assert app.state.vlm_config.model == "alias"diff --git a/tests/unittest/test_vlm_remote.py b/tests/unittest/test_vlm_remote.pynew file mode 100644index 00000000..d5e5da26--- /dev/null+++ b/tests/unittest/test_vlm_remote.py@@ -0,0 +1,373 @@+from __future__ import annotations++import asyncio+import json+import threading+import time+from collections.abc import Iterator+from concurrent.futures import ThreadPoolExecutor+from dataclasses import dataclass, field+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer+from pathlib import Path+from typing import Any+from unittest.mock import MagicMock++import httpx+import pytest+from fastapi.testclient import TestClient+from PIL import Image+from reportlab.pdfgen import canvas+from typer.testing import CliRunner++from mineru.config import VlmConfig, config+from mineru.model.vlm.client import get_vlm_predictor+from mineru.parser import api_server, parse, parse_async+++@dataclass+class _OpenAIServer:+ """记录模拟推理服务的协议请求和可控响应。"""++ url: str = ""+ models: list[str] = field(default_factory=lambda: ["test-model"])+ api_key: str = "test-key"+ chat_status: int = 200+ requests: list[tuple[str, str, dict[str, Any] | None]] = field(default_factory=list)+++def openai_server(monkeypatch: pytest.MonkeyPatch) -> Iterator[_OpenAIServer]:+ """启动仅监听 loopback 的模拟 OpenAI 服务,并隔离模型缓存与旧环境变量。"""+ from mineru.model.vlm.runtime import ModelSingleton++ monkeypatch.delenv("MINERU_VL_API_KEY", raising=False)+ monkeypatch.delenv("MINERU_VL_MODEL_NAME", raising=False)+ monkeypatch.setattr(config.model, "vlm", VlmConfig())+ monkeypatch.setattr(config.llm_aided.features, "title_leveling", False)+ monkeypatch.setattr(config.llm_aided.features, "cross_page_table_cell_merge", False)+ monkeypatch.setattr(ModelSingleton, "_models", {})+ state = _OpenAIServer()++ class Handler(BaseHTTPRequestHandler):+ """实现模型发现、认证和最小 Chat Completions 协议。"""++ def log_message(self, format: str, *args: object) -> None:+ """关闭测试 HTTP 日志,避免图片请求污染测试输出。"""++ def _respond(self, status: int, payload: dict[str, Any]) -> None:+ """返回带明确长度的 JSON,使客户端可以正常结束响应读取。"""+ data = json.dumps(payload).encode()+ self.send_response(status)+ self.send_header("Content-Type", "application/json")+ self.send_header("Content-Length", str(len(data)))+ self.end_headers()+ self.wfile.write(data)++ def _authorized(self) -> bool:+ """校验 VLM Key,与解析 API 自身的 Key 独立。"""+ if state.api_key and self.headers.get("Authorization") != f"Bearer {state.api_key}":+ self._respond(401, {"error": {"message": "unauthorized"}})+ return False+ return True++ def do_GET(self) -> None:+ """仅在保留代理前缀的 models 路径返回模型列表。"""+ state.requests.append((self.path, self.headers.get("Authorization", ""), None))+ if not self._authorized():+ return+ if self.path != "/proxy/v1/models":+ self._respond(404, {"error": {"message": "unknown path"}})+ return+ self._respond(200, {"data": [{"id": model} for model in state.models]})++ def do_POST(self) -> None:+ """根据版面或文本提示返回稳定结果,支持失败任务测试。"""+ body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))+ state.requests.append((self.path, self.headers.get("Authorization", ""), body))+ if not self._authorized():+ return+ if self.path != "/proxy/v1/chat/completions":+ self._respond(404, {"error": {"message": "unknown path"}})+ return+ if state.chat_status != 200:+ self._respond(state.chat_status, {"error": {"message": "inference rejected"}})+ return+ content = "Remote VLM text"+ if "Layout Detection:" in json.dumps(body["messages"]):+ content = "<|box_start|>50 50 950 250<|box_end|><|ref_start|>text<|ref_end|>"+ self._respond(200, {"choices": [{"message": {"role": "assistant", "content": content}, "finish_reason": "stop"}]})++ server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)+ state.url = f"http://127.0.0.1:{server.server_port}/proxy/v1"+ thread = threading.Thread(target=server.serve_forever, daemon=True)+ thread.start()+ try:+ yield state+ finally:+ for predictor in ModelSingleton._models.values():+ predictor.client._client.close()+ server.shutdown()+ server.server_close()+ thread.join(timeout=2)+++def _settings(server: _OpenAIServer, **overrides: Any) -> VlmConfig:+ """为模拟服务生成连接配置,允许单个用例覆盖连接字段。"""+ return VlmConfig.model_validate({"server_url": server.url, "api_key": server.api_key, **overrides})+++def test_remote_client_protocol_and_cache(openai_server: _OpenAIServer, monkeypatch: pytest.MonkeyPatch) -> None:+ """真实 HTTP 验证模型发现、显式模型、认证、代理路径、参数及缓存隔离。"""+ from mineru.model.vlm import runtime, selector++ local_engine = MagicMock(side_effect=AssertionError("local VLM engine must not load"))+ local_weights = MagicMock(side_effect=AssertionError("local VLM weights must not download"))+ monkeypatch.setattr(selector, "get_vlm_engine", local_engine)+ monkeypatch.setattr(type(runtime.MINERU_2_5_PRO_2605_1_2B), "ensure", local_weights)+ settings = _settings(openai_server, http_timeout=9, max_concurrency=3)+ predictor, backend = get_vlm_predictor(settings)+ assert backend == "http-client"+ assert predictor.client.model_name == "test-model"+ assert predictor.client.max_concurrency == 3+ assert predictor.client._client.timeout.read == 9+ assert predictor.client.predict(Image.new("RGB", (32, 32)), "Text Recognition:") == "Remote VLM text"+ assert get_vlm_predictor(settings)[0] is predictor+ for patch in ({"model": "test-model"}, {"http_timeout": 10}, {"max_concurrency": 4}):+ assert get_vlm_predictor(VlmConfig.model_validate({**settings.model_dump(), **patch}))[0] is not predictor+ openai_server.api_key = "second-key"+ assert get_vlm_predictor(_settings(openai_server))[0] is not predictor+ assert {request[0] for request in openai_server.requests} == {"/proxy/v1/models", "/proxy/v1/chat/completions"}+ body = next(request[2] for request in openai_server.requests if request[2] is not None)+ assert body["model"] == "test-model"+ assert all(request[1] == "Bearer test-key" for request in openai_server.requests[:-1])+ local_engine.assert_not_called()+ local_weights.assert_not_called()+++def test_remote_models_and_auth_failures(openai_server: _OpenAIServer) -> None:+ """认证失败、多模型自动发现及不存在模型均显式失败,之后可以用有效模型重试。"""+ with pytest.raises(Exception, match="401"):+ get_vlm_predictor(_settings(openai_server, api_key="wrong-key"))+ openai_server.models = []+ with pytest.raises(Exception, match="exactly one model"):+ get_vlm_predictor(_settings(openai_server))+ openai_server.models = ["first", "second"]+ with pytest.raises(Exception, match="exactly one model"):+ get_vlm_predictor(_settings(openai_server))+ with pytest.raises(Exception, match="not found"):+ get_vlm_predictor(_settings(openai_server, model="missing"))+ predictor, _ = get_vlm_predictor(_settings(openai_server, model="second"))+ assert predictor.client.model_name == "second"+++def test_remote_optional_auth_and_concurrent_credentials(openai_server: _OpenAIServer) -> None:+ """允许无鉴权服务,并验证同一服务的不同客户端并发请求不会串用凭据。"""+ openai_server.api_key = ""+ predictor, _ = get_vlm_predictor(_settings(openai_server))+ assert predictor.client.predict(None, "test") == "Remote VLM text"+ assert all(request[1] == "" for request in openai_server.requests)++ def infer(key: str) -> str:+ """使用独立 Key 请求同一上游,返回真实客户端收到的文本。"""+ client, _ = get_vlm_predictor(_settings(openai_server, api_key=key))+ return client.client.predict(None, key)++ with ThreadPoolExecutor(max_workers=2) as executor:+ assert list(executor.map(infer, ["first-key", "second-key"])) == ["Remote VLM text", "Remote VLM text"]+ for _path, header, body in openai_server.requests:+ if body is not None and header:+ assert header.removeprefix("Bearer ") in json.dumps(body["messages"])+++def test_remote_timeout_propagates(openai_server: _OpenAIServer, monkeypatch: pytest.MonkeyPatch) -> None:+ """请求超时保留原始异常,不选择本地引擎重试。"""+ predictor, _ = get_vlm_predictor(_settings(openai_server, http_timeout=1))+ monkeypatch.setattr(predictor.client._client, "post", MagicMock(side_effect=httpx.ReadTimeout("timed out")))+ with pytest.raises(httpx.ReadTimeout, match="timed out"):+ predictor.client.predict(None, "test")+++def test_remote_preflight_and_preload(openai_server: _OpenAIServer, monkeypatch: pytest.MonkeyPatch) -> None:+ """远程预检仅需本地 Hybrid 依赖,预加载和推理复用同一个客户端。"""+ checks: list[str] = []+ local_loads: list[str] = []+ monkeypatch.setattr(api_server, "ensure_tier_runtime_dependencies", checks.append)+ monkeypatch.setattr(api_server, "_preload_local_models", local_loads.append)+ settings = _settings(openai_server)+ api_server._preflight_tier_dependencies("standard", settings)+ result = api_server._preload_server_models("standard", language="en", vlm_config=settings)+ assert result.engine == "http-client"+ assert checks == ["basic"]+ assert local_loads == ["en"]+ request_count = len(openai_server.requests)+ get_vlm_predictor(settings)+ assert len(openai_server.requests) == request_count+ api_server._preload_server_models("basic", language="ch", vlm_config=settings)+ assert len(openai_server.requests) == request_count+++def test_remote_preload_failure_is_reported(+ openai_server: _OpenAIServer, monkeypatch: pytest.MonkeyPatch, tmp_path: Path+) -> None:+ """预加载认证失败通过现有健康和能力错误暴露,未预加载时保持懒连接。"""+ monkeypatch.setattr(api_server, "ensure_tier_runtime_dependencies", lambda tier: None)+ settings = _settings(openai_server, api_key="wrong-key")+ lazy = api_server.create_app(upload_dir=str(tmp_path / "lazy"), vlm_config=settings)+ with TestClient(lazy) as client:+ assert client.get("/v1/health").status_code == 200+ assert openai_server.requests == []+ preloaded = api_server.create_app(upload_dir=str(tmp_path / "preload"), vlm_config=settings, preload_models=True)+ with TestClient(preloaded) as client:+ for endpoint in ("health", "tiers", "models"):+ response = client.get(f"/v1/{endpoint}")+ assert response.status_code == 503+ assert response.json()["error"]["code"] == "model_preload_failed"+++def hybrid_stub(monkeypatch: pytest.MonkeyPatch) -> None:+ """隔离本地神经模型,保留真实 PDF 准备、窗口、HTTP VLM 和 Middle JSON 后处理。"""+ from mineru.backend.analysis.pdf import pipeline, window++ context = MagicMock()+ context.device = "cpu"+ context.layout_model.batch_predict.return_value = [[{"bbox": [20, 20, 250, 80], "label": "text"}]]+ monkeypatch.setattr(+ pipeline,+ "HybridLocalModelContextSingleton",+ MagicMock(return_value=MagicMock(get_model=MagicMock(return_value=context))),+ )+ monkeypatch.setattr(pipeline, "clean_memory", lambda device: None)++ def retain_vlm_text(+ images: object, pages: object, blocks: list[list[dict[str, Any]]], *args: object+ ) -> list[list[dict[str, Any]]]:+ """保留远程正文并模拟 Hybrid 提供的行框,满足 PDF 文本块校验合同。"""+ for page in blocks:+ for block in page:+ if block.get("content"):+ block["lines"] = [{"bbox": block["bbox"]}]+ return blocks++ monkeypatch.setattr(window, "_process_text_and_formulas", retain_vlm_text)+ monkeypatch.setattr(window, "_apply_seal_ocr", lambda *args: None)+ monkeypatch.setattr(api_server, "ensure_tier_runtime_dependencies", lambda tier: None)+++def _pdf_input(tmp_path: Path) -> Path:+ """构造单页 PDF fixture,供同步、异步和 API 解析复用。"""+ path = tmp_path / "input.pdf"+ document = canvas.Canvas(str(path), pagesize=(300, 200))+ document.drawString(20, 160, "Source text")+ document.save()+ return path++[email protected]("async_mode", [False, True])[email protected]("image_input", [False, True])+def test_python_parse_uses_global_remote_vlm(+ openai_server: _OpenAIServer,+ hybrid_stub: None,+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+ async_mode: bool,+ image_input: bool,+) -> None:+ """同步及异步入口读取全局 VLM 配置,PDF 和图片转 PDF 均产生真实 Middle JSON。"""+ monkeypatch.setattr(config.model, "vlm", _settings(openai_server))+ source = _pdf_input(tmp_path)+ if image_input:+ source = tmp_path / "input.png"+ Image.new("RGB", (300, 200), "white").save(source)+ result = asyncio.run(parse_async(source, ocr_mode="ocr")) if async_mode else parse(source, ocr_mode="ocr")+ assert result.middle_json.file_suffix == "pdf"+ assert result.middle_json.effort == "high"+ assert result.middle_json.pages[0].page_idx == 0+ assert "Remote VLM text" in result.markdown()+ assert any(request[2] is not None for request in openai_server.requests)+++def test_kit_parse_uses_global_remote_vlm(+ openai_server: _OpenAIServer, hybrid_stub: None, tmp_path: Path, monkeypatch: pytest.MonkeyPatch+) -> None:+ """实际执行 mineru-kit parse 命令,验证全局 VLM 配置进入解析并生成 Markdown 文件。"""+ from mineru.kit.main import app++ monkeypatch.setattr(config.model, "vlm", _settings(openai_server))+ output = tmp_path / "result.md"+ result = CliRunner().invoke(app, ["parse", str(_pdf_input(tmp_path)), "-o", str(output), "--ocr-mode", "ocr"])+ assert result.exit_code == 0, result.output+ assert "Remote VLM text" in output.read_text(encoding="utf-8")++[email protected]("tier", ["standard", "advanced"])[email protected]("chat_status", [200, 400])+def test_api_job_through_http_vlm_to_middle_json(+ openai_server: _OpenAIServer,+ hybrid_stub: None,+ tmp_path: Path,+ tier: str,+ chat_status: int,+) -> None:+ """API 任务经真实 HTTP VLM 生成 Middle JSON,远程推理错误保持任务失败。"""+ openai_server.chat_status = chat_status+ source = _pdf_input(tmp_path)+ app = api_server.create_app(+ upload_dir=str(tmp_path / "api"),+ allow_local_source=True,+ api_key="inbound-key",+ no_flash=True,+ vlm_config=_settings(openai_server),+ )+ with TestClient(app, headers={"Authorization": "Bearer inbound-key"}) as client:+ submitted = client.post(+ "/v1/parse/jobs",+ json={+ "files": [{"source": {"type": "local", "path": str(source)}}],+ "tier": tier,+ "ocr_mode": "ocr",+ "output_formats": ["middle_json"],+ },+ )+ assert submitted.status_code == 202, submitted.text+ job_id = submitted.json()["job_id"]+ deadline = time.monotonic() + 15+ while True:+ job = client.get(f"/v1/parse/jobs/{job_id}").json()+ if job["status"] in {"completed", "failed", "partial"}:+ break+ assert time.monotonic() < deadline, job+ time.sleep(0.01)+ if chat_status != 200:+ assert job["status"] == "failed", job+ assert job["files"][0]["error"]["code"] == "parse_failed"+ return+ assert job["status"] == "completed", job+ file_id = job["files"][0]["output_files"]["middle_json"]["file_id"]+ output = client.get(f"/v1/files/{file_id}/content")+ assert output.status_code == 200+ assert output.json()["effort"] == ("high" if tier == "standard" else "xhigh")+ assert "Remote VLM text" in output.text+ assert all(request[1] == "Bearer test-key" for request in openai_server.requests)++[email protected]("effort", ["flash", "medium"])+def test_low_efforts_never_create_remote_client(+ openai_server: _OpenAIServer,+ hybrid_stub: None,+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+ effort: str,+) -> None:+ """即使配置了远程服务,Flash 和 Basic 分析也不触发 VLM 客户端构造。"""+ from mineru.backend.analysis.pdf import pipeline++ remote_factory = MagicMock(side_effect=AssertionError("VLM must not initialize"))+ monkeypatch.setattr(pipeline, "get_vlm_predictor", remote_factory)+ monkeypatch.setattr(pipeline, "process_pdf_windows", MagicMock(return_value=[]))+ result = pipeline.analyze_pdf(+ _pdf_input(tmp_path).read_bytes(), effort=effort, parse_mode="txt", vlm_config=_settings(openai_server)+ )+ assert result.effort == effort+ assert openai_server.requests == []+ remote_factory.assert_not_called()