Skip to content

Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

refactor(runtime): reuse small-model device decisions for llama.cpp

Xiaomeng Zhao169cf60f7eb3
diff --git a/docs/next/cli/mineru-kit-api-server.md b/docs/next/cli/mineru-kit-api-server.md
index ea5999fe..fe0374d3 100644
--- a/docs/next/cli/mineru-kit-api-server.md
+++ b/docs/next/cli/mineru-kit-api-server.md
@@ -66,7 +66,7 @@ CPU 数按逻辑 CPU 计算,并考虑进程亲和性和 Linux cgroup 配额。
5.5 GiB 按 `5_905_580_032` 字节比较,不四舍五入、不累加多卡,也不使用 `MINERU_VIRTUAL_VRAM_SIZE`。独显读取专用显存总量;Apple 统一内存使用 Metal 推荐工作集,Windows/Linux 集显分别使用 DXGI 本地预算与 Vulkan `heapBudget`。预算比较使用上限,不减去当前用量;未知预算不以系统总内存替代。容量证据可通过 `mineru-kit hardware detect --json` 的 `gpu_memory` 查看。
-GPU 类型与 OCR、llama.cpp 和硬件诊断使用共享分类;目标运行后端可见且原生身份确认的独显可以免测速,但仍须满足 5.5 GiB 内存门槛才能开放 Standard/Advanced。ggml 的通用 GPU 标志、未知类型及虚拟 GPU 均不直接视为独显;同型号多卡无法唯一关联时保持保守结果。各后端可以选择不同 GPU,共用分类和优先级,不强制跨后端同卡。
+GPU 类型与 OCR 和硬件诊断使用共享分类;小模型后端可见且原生身份确认的独显可以免测速,但仍须满足 5.5 GiB 内存门槛才能开放 Standard/Advanced。未知类型、虚拟 GPU 和无法唯一关联的设备不能直接获得独显免校准资格。推荐不查询 llama.cpp 库存,也不受其专用后端设置影响;llama 自动模式消费小模型的 CPU/GPU 结论,具体显卡由原生引擎选择。
自动推荐先解析有效小模型后端。Torch(包括 auto 最终选择 Torch)验证实际设备可用性及容量:选中 CUDA/MPS/XPU 等设备的显存/预算至少 5.5 GiB 时推荐 advanced;显式 CPU 或设备不可用时按 CPU 条件推荐,不下载 ONNX 校准模型或改写其配置/记录。Torch 推荐不依赖 llama.cpp 库存。
diff --git a/docs/next/cli/mineru-kit-hardware.md b/docs/next/cli/mineru-kit-hardware.md
index 4b50b723..36ca0a66 100644
--- a/docs/next/cli/mineru-kit-hardware.md
+++ b/docs/next/cli/mineru-kit-hardware.md
@@ -33,9 +33,13 @@ ONNX 分支保留既有兼容策略、0.8 阈值和进程缓存。Windows 确认
`onnx_device=gpu` 无论来自用户配置还是之前的自动选型,都不触发测速;实际选中兼容 GPU 后,仅检查该设备的显存/预算是否达到 5.5 GiB。校准记录缺失、损坏、过期或记录 GPU 较慢,均不阻止内存达标的 GPU 开放 Standard/Advanced。此时推荐原因是 `configured_gpu`,校准状态为 `status=skipped, reason=configured_gpu`。内存不足或未知时报告相应内存原因。记录摘要只供诊断,读取失败会附带警告,不改变设备和档位结论。`onnx_device=cpu` 仍遵循 CPU 规则。
-llama.cpp 自动集显路径同样按有效小模型后端分流:Torch 使用加速可用性结果,不再绕回 OCR 校准;ONNX 保留原有性能结果和设备身份匹配。原生独显优先、显式 CPU/GPU 参数、物理绑定和兼容处理保持不变。
+llama.cpp 自动模式只复用小模型后端的 CPU/GPU 结论:Torch 使用加速可用性,ONNX 使用实际选型与缓存。
+显式参数优先,具体显卡由原生引擎选择;MinerU 不在推荐或引擎构造时查询 ggml 库存、匹配身份、绑定或重映射设备。
+自动推荐同样不读取 llama 专用后端设置;VLM 使用其他引擎或远程服务时不会因此探测 llama。
+Linux Intel 串行提交仅在自动 ONNX 路径依据已有适配器信息补充,用户环境值优先;Torch 和手动路径不自动补充。
-OCR、llama.cpp、档位推荐与诊断共用 GPU 分类和双向唯一身份匹配。先按各后端的实际可见性和兼容性过滤,再按独显、集显、虚拟 GPU、未知类型排序;后端之间无需选择同一张卡。ggml 的 GPU/IGPU 标志不再直接作为独显判据,Windows 使用 D3D12 UMA、Linux 使用 Vulkan 原生分类。无法唯一关联的设备保留未知类型,不授予独显免校准或仅凭类型开放 Advanced。
+OCR、档位推荐与诊断保留共享 GPU 分类和双向唯一身份匹配,内存查询必须对应小模型后端选中的设备。
+`hardware info` 可独立探测 llama 的真实可见设备,诊断失败不会影响自动推荐。
JSON 报告保留 `schema_version=1`,包含 `ceiling`、`pdf_image_tiers`、`default_request_tier`、`reason`、`cpu_count`、`small_backend`、`vlm_engine`、`device`、`calibration` 和 `warnings`,并增补 `gpu_memory`。内存证据包含 `device`、`identity`、`memory_bytes`、`kind`(`dedicated`/`budget`/`unknown`)、`source`、`warning`;没有候选设备时为 `null`。校准状态来自执行结果;Torch 使用 `status=skipped, reason=torch_backend`,不将可用性结论标成性能实测通过。
diff --git a/docs/next/llama-xe-lpg-serialization-validation.md b/docs/next/llama-xe-lpg-serialization-validation.md
index cb8db6a3..509827d5 100644
--- a/docs/next/llama-xe-lpg-serialization-validation.md
+++ b/docs/next/llama-xe-lpg-serialization-validation.md
@@ -1,5 +1,9 @@
# Linux Xe-LPG 自动串行提交验证
+> 2026-10-29 策略更新:本文保留历史验证结果。当前仅自动 ONNX 路径依据 ONNX 已选适配器补充串行设置;
+> 已移除 llama 独立选卡、绑定复核及手动 GPU 路径的自动注入。Torch 和手动选卡需显式设置兼容变量。
+> 当前行为见 [llama.cpp 自动模式与手动覆盖](onnx-device.md#llamacpp-自动模式与手动覆盖)。
+
2026-10-22。共享 llama.cpp 设备策略新增 Xe-LPG(PCI 家族 `7dxx`、`b6xx`),
保留 Xe-LP 及更早架构的既有规则。Xe-HPG、Xe2/Xe3 LPG、其他后续架构及未知 PCI 家族不自动开启。
diff --git a/docs/next/model-assets.md b/docs/next/model-assets.md
index 1952f8fd..004654b7 100644
--- a/docs/next/model-assets.md
+++ b/docs/next/model-assets.md
@@ -34,21 +34,22 @@ Layout、OCR-det/rec、公式主批次和 UNet 通过 `model.onnx_device=auto/cp
环境变量为 `MINERU_MODEL_ONNX_DEVICE`。自动模式复用六图 rec 校准与输出检查;Linux 还执行 Intel 架构策略。
其他 ONNX 会话及规定的公式尾批次保持 CPU 路线,具体以日志中的实际 provider 为准。
CPU 算子线程读取 `MINERU_INTRA_OP_NUM_THREADS` 和 `MINERU_INTER_OP_NUM_THREADS`,默认 4/1。
-llama.cpp 自动模式优先使用原生 ggml 实际可见的独显;没有可用独显时,Windows 复用 DirectML、Linux 复用 WebGPU 的 ONNX 设备决策,
-包括已保存的 CPU/GPU 选择、输出一致性检查失败及架构回退。ONNX 决策涉及的 GPU 必须与 llama.cpp 的设备身份匹配,不能用另一张卡的成绩替集显决策。
-因此 Linux 上 Xe-LP 及以前的 Intel 默认随现有 WebGPU 架构策略走 CPU;显式 `MINERU_LLAMA_CPP_BACKEND=vulkan`、GPU 层数或 server `--device` 仍可强制使用 GPU。
-
-解析与 `mineru-kit vlm-server` 的 llama 入口共用设备策略。仅当 Linux Vulkan 实际选中明确 PCI 映射中的 Intel Xe-LP/更早架构,
-或 Xe-LPG(`7dxx`、`b6xx`,包含 `8086:7d51`)时,自动启用 `GGML_VK_SERIALIZE_SUBMISSIONS=1`,
-规避已复现的 i915 请求取消、设备丢失及异常视觉结果。该规则对 standard、advanced 等档位统一生效。
-Xe-HPG、Xe2/Xe3 LPG 和其他后续架构、AMD、NVIDIA 及 Windows 不自动开启该开关。Xe-LPG 的 ONNX WebGPU 自适应校准策略不变。
-多卡机器先确认原生设备,再隔离所选 GPU 并复核身份;不能仅因机器里存在 Intel 集显就给独显开启串行提交。
-自动选择也不会混用需要串行提交的设备和其他设备;显式要求此类混合设备组会报错,因为 ggml 的环境开关无法逐卡设置。
-
-已有的显式 `GGML_VK_SERIALIZE_SUBMISSIONS` 环境值保持不变:该开关检查变量是否存在,设为 `0` 同样会开启,撤销须 `unset`。
-进程内初始化结束后会恢复临时环境;ggml 已缓存设备状态时,变更可见显卡或提交策略须使用新进程。
-server 的 `--offline` 同样约束新增的 ONNX 校准模型读取,缺少本地模型时回退 CPU,不发起下载。
-Torch、其他 VLM 引擎和远程 VLM 不使用这套 llama.cpp 设备策略。
+llama.cpp 自动模式直接复用小模型后端的 CPU/GPU 结论:ONNX 使用现有 DirectML/WebGPU 选型与缓存,
+Torch 使用已验证的加速器可用性。兼容独显优先和集显 OCR 校准由 ONNX 统一处理;不增加 VLM 测速。
+MinerU 不再枚举 ggml、匹配跨后端身份、过滤显卡或重映射设备名;GPU 模式下由原生引擎选择具体显卡,
+不要求与 ONNX 同卡。显式后端、GPU 层数、server 设备和投影参数优先,支持主模型 CPU、投影 GPU 的组合。
+macOS 保持原生 Metal/CPU 路径。
+
+Engine 和 `mineru-kit vlm-server` 的 llama 入口共用这套决策。仅自动 ONNX 路径在 Linux 选择 WebGPU,
+且已有适配器信息明确属于 Intel legacy/Xe-LP/Xe-LPG 时,才补充 `GGML_VK_SERIALIZE_SUBMISSIONS=1`。
+该规避沿用已验证的架构范围,未重新探测 llama 实际设备;Windows、其他架构、Torch 和手动选卡不自动补充。
+手动使用受影响 Intel GPU 时须自行设置该变量。已有的显式环境值保持不变:原生开关检查变量是否存在,
+设为 `0` 同样会开启,撤销须 `unset`。进程内初始化结束后恢复临时环境;变更已缓存的 Vulkan 环境须使用新进程。
+
+server 的 `--offline` 同样约束 ONNX 校准模型读取,缺少本地模型时沿用 CPU 回退,不发起下载。
+`hardware info` 保留独立 llama 诊断;诊断失败不会降低推荐档位或改变运行设备。
+其他 VLM 引擎和远程 VLM 不使用 llama.cpp 的运行策略,也不会因自动档位推荐而触发 llama 探测。
+详细规则见 [ONNX 设备选择](onnx-device.md#llamacpp-自动模式与手动覆盖)。
## 资源结构
diff --git a/docs/next/onnx-device.md b/docs/next/onnx-device.md
index 50847621..6acfaa66 100644
--- a/docs/next/onnx-device.md
+++ b/docs/next/onnx-device.md
@@ -9,22 +9,24 @@ model:
```
环境变量 `MINERU_MODEL_ONNX_DEVICE` 高于文件配置。`small_backend=auto` 仍按原有规则选择
-Torch 或 ONNX;此选项不强制切换模型栈,也不改变 Torch / VLM 的设备。
+Torch 或 ONNX;此选项不强制切换模型栈,也不改变 Torch 的设备。小模型使用 ONNX 时,
+llama.cpp 自动模式复用这里的 CPU/GPU 结论。
### 共享 GPU 分类与选型
-OCR、llama.cpp、auto tier 和硬件诊断共用 `discrete / integrated / virtual / unknown` 分类。
+OCR、auto tier 和硬件诊断共用 `discrete / integrated / virtual / unknown` 分类。
Windows 的分类依据为 D3D12 UMA,Linux 为 Vulkan 原生设备类型,macOS 复用现有系统显示设备信息。
统一内存属性单独保存,不将 Apple Silicon 的集显分类解释为低性能设备。
-各后端先应用显式配置、设备可见性及兼容限制,再按“确认独显 → 确认集显 → 虚拟 GPU → 类型未知”排序;
-同类设备保留 Windows 高性能排序、Vulkan 原生顺序或 llama.cpp 的单后端多设备分组规则。
-OCR 与 llama.cpp 可以因支持范围不同选择不同显卡;不在目标后端候选中的独显不影响当前设备的校准或推荐。
+ONNX 先应用显式配置、设备可见性及兼容限制,再按“确认独显 → 确认集显 → 虚拟 GPU → 类型未知”排序;
+同类设备保留 Windows 高性能排序或 Vulkan 原生顺序。显存证据只取 ONNX 实际选中的设备。
+身份匹配优先使用 LUID、PCI 地址,再使用双方唯一的厂商/设备型号;名称只用于原生库存的唯一关联。
+重复库存或原生查询失败时保留未知类型,不据此授予独显免校准资格。
-ggml 的 GPU/IGPU 枚举仅用于确定 llama.cpp 可见的 GPU 库存,分类来自与平台原生设备的唯一关联。
-身份匹配优先使用双方已有的 LUID、PCI 地址,再使用双方唯一的厂商/设备型号;名称只用于原生库存的唯一关联,
-不用于猜测类型。重复库存、同型号多卡无法唯一关联或原生查询失败时保留未知,不凭此开放 Advanced 或免校准。
-本实现不增加 UUID、PCI 地址等身份采集。Torch 的加速可用性推荐和 Apple Silicon/MPS 特例保持不变。
+llama.cpp 的 CPU/GPU 自动决策直接复用小模型后端,具体显卡交给原生引擎。
+MinerU 不在启动推荐或模型构造时枚举 ggml、匹配 ONNX/llama 显卡、绑定 Vulkan 编号或重映射设备名。
+`hardware info` 仍可显式运行隔离的 llama 诊断;Windows 诊断读取有效 Vulkan LUID 与 DXGI 关联,
+但诊断库存不参与推荐或推理决策。
| 值 | Windows DML | Linux WebGPU |
|---|---|---|
@@ -97,7 +99,7 @@ MinerU 的 WebGPU 运行时锁覆盖过滤与会话初始化,后续每次会
本机双卡及仅 Intel ICD 的验证见 [Intel 架构过滤报告](onnx-intel-architecture-policy-validation.md)。
-通常在首次实际使用 ONNX 模型时选型。API 使用 `--tier auto`(或省略),有效小模型后端为 ONNX,且可用逻辑 CPU 至少 4 核或存在显存/预算达到 5.5 GiB 的兼容 GPU 候选时,Windows/Linux 会提前同步调用相同的设备选择流程,并直接采用本次设备选型结果,不依赖 `--preload-models`。实际选中 GPU 且内存达标时开放 Standard/Advanced;auto tier 不增加独立测速,也不额外检查历史校准是否胜出。`onnx_device=gpu` 无论来自用户偏好还是自动选型写入,均免测速,校准记录缺失、损坏、过期或 GPU 较慢不阻止内存达标的已选 GPU 开放高级档位。CPU 不足 4 核且无内存达标 GPU 候选、显式启动档位或 macOS 不增加这项启动选型。有效小模型后端为 Torch 时采用实际设备可用性和内存证据并跳过 OCR 校准,不修改 ONNX 设备偏好或证据;llama.cpp 自动集显路径也遵循这处分流。
+通常在首次实际使用 ONNX 模型时选型。API 使用 `--tier auto`(或省略),有效小模型后端为 ONNX,且可用逻辑 CPU 至少 4 核或存在显存/预算达到 5.5 GiB 的兼容 GPU 候选时,Windows/Linux 会提前同步调用相同的设备选择流程,并直接采用本次设备选型结果,不依赖 `--preload-models`。实际选中 GPU 且内存达标时开放 Standard/Advanced;auto tier 不增加独立测速,也不额外检查历史校准是否胜出。`onnx_device=gpu` 无论来自用户偏好还是自动选型写入,均免测速,校准记录缺失、损坏、过期或 GPU 较慢不阻止内存达标的已选 GPU 开放高级档位。CPU 不足 4 核且无内存达标 GPU 候选、显式启动档位或 macOS 不增加这项启动选型。有效小模型后端为 Torch 时采用实际设备可用性和内存证据并跳过 OCR 校准,不修改 ONNX 设备偏好或证据;llama.cpp 自动模式也遵循这处分流,且不要求另一次显卡匹配。
`mineru-kit hardware detect` 与 API 自动启动共享推荐逻辑;`hardware info` 只读报告环境与校准记录,不测速。界面启动本身、导入 SDK、查询配置和模型列表不会运行测速;界面托管的 API 服务仍遵循其启动参数。
自动决策成功后原子更新有效配置文件中的 `model.onnx_device`,保留其余键;进程间通过文件锁协调。
@@ -112,6 +114,30 @@ Linux 插件缺失、没有可用设备、物理绑定失败、校准超时或
要重新测量,将配置改回 `auto` 并重启应用。修改程序阈值也不主动覆盖已保存的选择。
+## llama.cpp 自动模式与手动覆盖
+
+Windows/Linux 的 Engine 和 `llama-server` 复用同一 CPU/GPU 决策:
+
+- 小模型为 ONNX:使用 `resolve_onnx_device()` 的结果和进程缓存。兼容独显免测速;集显或类型未知按既有 OCR 校准决定。
+- 小模型为 Torch:使用已核验的 Torch 加速器可用性,不调用 ONNX、不下载 OCR 校准模型。
+- CPU:模型、视觉投影均使用 CPU,server 同时关闭额外算子卸载。
+- GPU:允许原生运行时按默认规则选择显卡;不要求它与 ONNX 使用同一物理卡,不新增 VLM 测速。
+- macOS:保持原生 Metal/CPU 路径。
+
+显式 `MINERU_LLAMA_CPP_BACKEND`、GPU 层数、server 设备或投影设备参数优先于自动模式。
+原生设备名称与用户提供的可见设备环境变量保持原样;“主模型 CPU、投影 GPU”等显式组合继续支持。
+原生引擎负责实际显卡选择及参数合法性;其自动 GPU 不可用回退保持原样,显式后端不可用及推理错误直接报告,不另行重试。
+`llama.cpp device policy` 日志说明共享决策来源;原生 Engine 的 `verbosity=LOG_LEVEL_INFO` 可查看实际设备。
+
+仅自动 ONNX 路径在 Linux 选择 WebGPU、已选适配器为 Intel legacy/Xe-LP/Xe-LPG 架构时,
+MinerU 才在初始化期间补充 `GGML_VK_SERIALIZE_SUBMISSIONS=1`。依据来自 ONNX 已有适配器信息,
+不再探测 llama 的实际显卡。用户已设置的值(包括 `0`)优先;Torch 和手动选卡路径需自行设置该兼容变量。
+临时环境设置使用互斥锁并在成功或异常后恢复;不生成 `GGML_VK_VISIBLE_DEVICES`。
+离线 server 仍禁止为自动选型下载校准模型,缺少本地权重时沿用现有 ONNX CPU 回退。
+
+档位推荐只看小模型后端、其实际选型及显存预算,不受 llama 专用后端参数或诊断失败影响。
+CPU 核数、5.5 GiB 显存门槛及默认 Standard 的规则保持不变。
+
## 模型预设
| 模型 | Windows DML | Linux WebGPU |
diff --git a/mineru/kit/vlm_server/llama_cpp_server.py b/mineru/kit/vlm_server/llama_cpp_server.py
index 375f782e..59d882c6 100644
--- a/mineru/kit/vlm_server/llama_cpp_server.py
+++ b/mineru/kit/vlm_server/llama_cpp_server.py
@@ -12,7 +12,6 @@ llama-server 的全部服务能力由 llama.cpp 提供,本模块只补默认
from __future__ import annotations
import os
-from dataclasses import replace
import struct
import sys
import typing
@@ -210,13 +209,6 @@ def _exec_server(args: list[str], selection: LlamaSelection | None = None) -> ty
print(f"start llama.cpp server: {binary} {' '.join(_redact_secrets(args))}")
changes = dict(selection.environment) if selection is not None else {}
- if selection is not None and selection.device != "native":
- # Native env values are parsed before CLI arguments and can contain names
- # invalid after physical filtering; remap those too, before exec.
- for name, flags in (("LLAMA_ARG_DEVICE", ("-dev", "--device")), ("MTMD_BACKEND_DEVICE", ("-mmdev", "--mmproj-device"))):
- value = _flag_value(args, *flags)
- if name in os.environ and value is not None:
- changes[name] = value
previous = {name: os.environ.get(name) for name in changes}
try:
os.environ.update(changes)
@@ -241,36 +233,19 @@ def _device_policy(args: list[str]) -> LlamaSelection:
names = tuple(devices.split(",")) if devices else None
offline = _has_arg(args, "--offline") or _has_env("LLAMA_ARG_OFFLINE")
options = {"allow_download": False} if offline else {}
- model_cpu = count == 0 or names == ("none",) or os.environ.get("MINERU_LLAMA_CPP_BACKEND") == "cpu"
projector = _flag_value(args, "-mmdev", "--mmproj-device") or os.environ.get("MTMD_BACKEND_DEVICE")
projector_enabled = _projector_enabled(args)
projector_gpu = projector != "none" and (
projector_enabled is True or (projector is not None and projector_enabled is not False)
)
- if model_cpu and projector_gpu:
- selected = resolve_llama_device(
- n_gpu_layers=99, device_names=(projector,) if projector else None, backend="vulkan", **options
- )
- return replace(selected, cpu_model=True, model_devices=())
- selected = resolve_llama_device(n_gpu_layers=count, device_names=names, **options)
- if selected.device == "cpu" and projector_gpu:
- selected = resolve_llama_device(
- n_gpu_layers=99, device_names=(projector,) if projector else None, backend="vulkan", **options
- )
- return replace(selected, cpu_model=True, model_devices=())
- if selected.device == "gpu" and projector_gpu and projector:
- model_names = tuple(old for old, _new in selected.device_aliases)
- if projector not in model_names:
- selected = resolve_llama_device(device_names=(*model_names, projector), **options)
- aliases = dict(selected.device_aliases)
- selected = replace(selected, model_devices=tuple(aliases[name] for name in model_names))
- return selected
+ if projector_gpu:
+ # 显式投影 GPU 可以与 CPU 主模型共存;设备名和组合合法性由原生 parser 处理。
+ return LlamaSelection(reason="explicit_projector")
+ return resolve_llama_device(n_gpu_layers=count, device_names=names, **options)
def _apply_device_policy(args: list[str], selection: LlamaSelection) -> None:
- """将经过物理绑定的设备名称转交原生 server,不让投影落到其他显卡。"""
- if selection.device == "native":
- return
+ """自动 CPU 完整关闭卸载;GPU 及显式设备名交给原生运行时。"""
if selection.device == "cpu":
_set_arg(args, "0", "--n-gpu-layers", "-ngl", "--gpu-layers")
_set_arg(args, "none", "--device", "-dev")
@@ -278,21 +253,19 @@ def _apply_device_policy(args: list[str], selection: LlamaSelection) -> None:
args.append("--no-op-offload")
if not _has_arg(args, "--mmproj-offload", "--no-mmproj-offload") and not _has_env("LLAMA_ARG_MMPROJ_OFFLOAD"):
args.append("--no-mmproj-offload")
- return
- if selection.cpu_model:
- _set_arg(args, "0", "--n-gpu-layers", "-ngl", "--gpu-layers")
- args.append("--no-op-offload")
- model_devices = selection.model_devices if selection.model_devices else tuple(d.name for d in selection.devices)
- _set_arg(args, ",".join(model_devices), "--device", "-dev")
- projector = _flag_value(args, "-mmdev", "--mmproj-device") or os.environ.get("MTMD_BACKEND_DEVICE")
- if projector and projector != "none":
- if _projector_enabled(args) is False:
- _set_arg(args, "none", "--mmproj-device", "-mmdev")
- return
- alias = dict(selection.device_aliases).get(projector)
- if alias is None:
- raise ValueError("mmproj device must belong to the selected llama.cpp GPU group")
- _set_arg(args, alias, "--mmproj-device", "-mmdev")
+ elif selection.reason == "explicit_projector":
+ layers = _flag_value(args, "-ngl", "--gpu-layers", "--n-gpu-layers") or os.environ.get("LLAMA_ARG_N_GPU_LAYERS")
+ devices = _flag_value(args, "-dev", "--device") or os.environ.get("LLAMA_ARG_DEVICE")
+ try:
+ cpu_layers = int(layers) == 0 if layers is not None else False
+ except ValueError:
+ cpu_layers = False
+ if cpu_layers or devices == "none" or os.environ.get("MINERU_LLAMA_CPP_BACKEND") == "cpu":
+ _set_arg(args, "0", "--n-gpu-layers", "-ngl", "--gpu-layers")
+ args.append("--no-op-offload")
+ # 明确指定投影设备时,后续 ngl=0 默认值不能再关闭投影 GPU。
+ if not _has_arg(args, "--mmproj-offload", "--no-mmproj-offload") and not _has_env("LLAMA_ARG_MMPROJ_OFFLOAD"):
+ args.append("--mmproj-offload")
def main() -> None:
diff --git a/mineru/model/runtime/llama_device.py b/mineru/model/runtime/llama_device.py
index d7ef8eef..307740c1 100644
--- a/mineru/model/runtime/llama_device.py
+++ b/mineru/model/runtime/llama_device.py
@@ -1,14 +1,12 @@
# Copyright (c) Opendatalab. All rights reserved.
-"""llama.cpp 实际设备选择、集显 ONNX 决策复用与 Intel Vulkan 串行提交策略。"""
+"""复用小模型后端的 CPU/GPU 结论,具体显卡交由 llama.cpp 原生运行时选择。"""
from __future__ import annotations
-from collections.abc import Iterator, Mapping
+from collections.abc import Iterator
from contextlib import contextmanager
-from dataclasses import dataclass, replace
-import json
+from dataclasses import dataclass
import os
-import subprocess
import sys
import threading
from typing import Literal
@@ -16,8 +14,6 @@ from typing import Literal
from loguru import logger
from .intel_gpu import INTEL_VENDOR_ID, intel_architecture
-from .gpu import GPU_KIND_ORDER, is_discrete_gpu, match_gpu_identities
-from .llama_probe import LlamaDevice, PROBE_MARKER
from .vulkan import vulkan_environment_lock
SERIALIZE_ENV = "GGML_VK_SERIALIZE_SUBMISSIONS"
@@ -36,168 +32,68 @@ _BACKEND_ENV = (
@dataclass(frozen=True)
class LlamaSelection:
- """可审查的选择计划;environment 只在创建引擎或启动子进程时应用。"""
+ """只决定运行模式和兼容环境,不携带显卡库存或原生设备名称。"""
device: Literal["native", "cpu", "gpu"] = "native"
- devices: tuple[LlamaDevice, ...] = ()
reason: str = "native"
environment: tuple[tuple[str, str], ...] = ()
- device_aliases: tuple[tuple[str, str], ...] = ()
- model_devices: tuple[str, ...] | None = None
- cpu_model: bool = False
-
-
-def discover_llama_devices(environment: Mapping[str, str] | None = None) -> tuple[LlamaDevice, ...]:
- """隔离原生注册与驱动错误,返回 ggml 过滤后真正可选的设备。"""
- with vulkan_environment_lock():
- snapshot = dict(os.environ if environment is None else environment)
- result = subprocess.run(
- [sys.executable, "-m", "mineru.model.runtime.llama_probe"],
- env=snapshot,
- capture_output=True,
- text=True,
- timeout=15,
- check=True,
- )
- payload = next(
- (line.removeprefix(PROBE_MARKER) for line in result.stdout.splitlines() if line.startswith(PROBE_MARKER)), None
- )
- if payload is None:
- raise RuntimeError("llama.cpp device probe returned no inventory")
- return tuple(LlamaDevice(**item) for item in json.loads(payload))
-
-
-def needs_serialized_submissions(device: LlamaDevice) -> bool:
- """仅 Linux Vulkan 的 Xe-LPG、Xe-LP 及更早 PCI 架构触发规避,不推断后续架构。"""
- return (
- sys.platform == "linux"
- and device.backend == "vulkan"
- and device.vendor_id == INTEL_VENDOR_ID
- and device.device_id is not None
- and intel_architecture(device.device_id) in {"legacy", "xe_lp", "xe_lpg"}
- )
-
-
-def _preferred(devices: tuple[LlamaDevice, ...], backend: str) -> tuple[LlamaDevice, ...]:
- """与包内 Engine 的独显优先、单后端多设备规则保持一致。"""
- for kind in GPU_KIND_ORDER:
- for name in ("metal", "vulkan"):
- if backend not in ("auto", name):
- continue
- group = tuple(d for d in devices if d.kind == kind and d.backend == name)
- if group:
- return group
- return ()
-
-
-def _identity(device: LlamaDevice) -> tuple[str, str, int | None, int | None, str]:
- """不使用可能因可见设备过滤而变化的 VulkanN 名称作为物理身份。"""
- return device.description, device.kind, device.vendor_id, device.device_id, device.luid or ""
-
-
-def _bind(selection: LlamaSelection) -> LlamaSelection:
- """隔离选中的物理设备并用原生 ggml 再核验,防止环境开关污染其他 GPU。"""
- selected = selection.devices
- if not selected or any(d.backend != "vulkan" or d.vulkan_index is None for d in selected):
- if any(needs_serialized_submissions(d) for d in selected):
- raise RuntimeError(
- "Cannot safely bind the Intel Vulkan device requiring serialization; use a verified device filter"
- )
- return replace(selection, device_aliases=tuple((d.name, d.name) for d in selected))
- changes = dict(selection.environment)
- changes["GGML_VK_VISIBLE_DEVICES"] = ",".join(str(d.vulkan_index) for d in selected)
- with vulkan_environment_lock():
- environment = {**os.environ, **changes}
- bound = discover_llama_devices(environment)
- actual = tuple(d for d in bound if d.backend == "vulkan")
- if sorted(map(_identity, actual)) != sorted(map(_identity, selected)):
- raise RuntimeError("llama.cpp Vulkan device binding changed the selected hardware")
- aliases = tuple((old.name, next(d.name for d in actual if _identity(d) == _identity(old))) for old in selected)
- return replace(selection, devices=actual, environment=tuple(changes.items()), device_aliases=aliases)
def resolve_llama_device(
*,
n_gpu_layers: int | None = None,
device_names: tuple[str, ...] | None = None,
- backend: str | None = None,
allow_download: bool = True,
) -> LlamaSelection:
- """自动集显按小模型后端复用 Torch 可用性或 ONNX 选型;显式设备设置优先。"""
+ """显式参数优先;自动模式复用 ONNX 选型或 Torch 可用性,不探测 ggml。"""
with _lock:
- selection = _resolve(
- n_gpu_layers=n_gpu_layers, device_names=device_names, backend=backend, allow_download=allow_download
- )
- target = [d.description for d in selection.devices] if selection.devices else selection.device
- logger.info("llama.cpp device policy: {} ({})", target, selection.reason)
+ selection = _resolve(n_gpu_layers=n_gpu_layers, device_names=device_names, allow_download=allow_download)
+ logger.info("llama.cpp device policy: {} ({})", selection.device, selection.reason)
return selection
-def _resolve(
- *, n_gpu_layers: int | None, device_names: tuple[str, ...] | None, backend: str | None, allow_download: bool
-) -> LlamaSelection:
- """在原生环境互斥保护下构造设备计划,校准仍走既有 ONNX 生命周期。"""
- requested = backend if backend is not None else os.environ.get("MINERU_LLAMA_CPP_BACKEND", "auto")
+def _resolve(*, n_gpu_layers: int | None, device_names: tuple[str, ...] | None, allow_download: bool) -> LlamaSelection:
+ requested = os.environ.get("MINERU_LLAMA_CPP_BACKEND", "auto")
if sys.platform not in {"linux", "win32"}:
return LlamaSelection(reason="platform_native")
if requested not in {"auto", "cpu", "vulkan", "metal"}:
raise ValueError("MINERU_LLAMA_CPP_BACKEND must be auto, cpu, vulkan or metal")
if n_gpu_layers == 0 or requested == "cpu" or device_names == ("none",):
return LlamaSelection("cpu", reason="explicit_cpu")
- try:
- inventory = discover_llama_devices()
- except (OSError, subprocess.SubprocessError, RuntimeError, ValueError, TypeError) as error:
- logger.warning("llama.cpp device discovery unavailable; preserving native selection: {}", error)
- return LlamaSelection(reason="discovery_unavailable")
- if device_names is not None:
- selected = tuple(next((d for d in inventory if d.name == name), None) for name in device_names)
- if not selected or any(d is None for d in selected):
- raise ValueError(f"Unknown llama.cpp device selection: {device_names}")
- selected = tuple(d for d in selected if d is not None)
- else:
- selected = _preferred(inventory, requested)
- if not selected:
- if requested != "auto" or device_names is not None:
- raise RuntimeError(f"Requested llama.cpp {requested} GPU is unavailable")
- return LlamaSelection("cpu", reason="no_gpu")
- manual = requested != "auto" or n_gpu_layers is not None or device_names is not None
- reason = "explicit_gpu" if manual else "discrete_gpu"
- if not manual and not any(is_discrete_gpu(d) for d in selected):
- from .device import resolve_small_model_backend, resolve_torch_device
-
- if resolve_small_model_backend() == "torch":
- torch_choice = resolve_torch_device()
- if not torch_choice.accelerated:
- return LlamaSelection("cpu", reason=torch_choice.reason)
- # Torch 只提供加速可用性结论,物理设备仍由当前原生库存和绑定校验决定。
- reason = "torch_accelerator"
- else:
- from .onnx_device import resolve_onnx_device
-
- choice = resolve_onnx_device() if allow_download else resolve_onnx_device(allow_download=False)
- if choice.device != "gpu" or choice.adapter is None:
- return LlamaSelection("cpu", reason=f"onnx_{choice.reason}")
- matched = match_gpu_identities((choice.adapter.identity,), [d.identity for d in selected])[0]
- if matched is None:
- return LlamaSelection("cpu", reason="onnx_gpu_identity_unconfirmed")
- selected, reason = (selected[matched],), f"onnx_{choice.backend}_{choice.reason}"
- serialized_devices = tuple(d for d in selected if needs_serialized_submissions(d))
- if serialized_devices and len(serialized_devices) != len(selected):
- if manual:
- raise ValueError("Selected GPUs need different Vulkan submission policies; select one compatible device group")
- # The global ggml switch cannot be restricted per device. Keep only the
- # selected GPUs that do not require serialization when policies differ.
- selected = tuple(d for d in selected if not needs_serialized_submissions(d))
- serialized_devices = ()
- changes = {SERIALIZE_ENV: "1"} if serialized_devices and SERIALIZE_ENV not in os.environ else {}
- return _bind(LlamaSelection("gpu", selected, reason, tuple(changes.items())))
+ if requested != "auto" or n_gpu_layers is not None or device_names is not None:
+ return LlamaSelection(reason="explicit_gpu")
+
+ from .device import resolve_small_model_backend, resolve_torch_device
+
+ if resolve_small_model_backend() == "torch":
+ choice = resolve_torch_device()
+ if choice.warning:
+ logger.warning("llama.cpp follows Torch device decision: {}", choice.warning)
+ return LlamaSelection("gpu" if choice.accelerated else "cpu", reason=choice.reason)
+
+ from .onnx_device import resolve_onnx_device
+
+ choice = resolve_onnx_device() if allow_download else resolve_onnx_device(allow_download=False)
+ adapter = choice.adapter
+ environment = ()
+ if (
+ choice.device == "gpu"
+ and sys.platform == "linux"
+ and choice.backend == "webgpu"
+ and adapter is not None
+ and adapter.vendor_id == INTEL_VENDOR_ID
+ and intel_architecture(adapter.device_id) in {"legacy", "xe_lp", "xe_lpg"}
+ and SERIALIZE_ENV not in os.environ
+ ):
+ environment = ((SERIALIZE_ENV, "1"),)
+ return LlamaSelection(choice.device, reason=f"onnx_{choice.reason}", environment=environment)
@contextmanager
def llama_engine_scope(selection: LlamaSelection) -> Iterator[None]:
- """临时应用进程环境并还原;已缓存的原生后端不得被换卡或修改提交策略。"""
+ """串行初始化并恢复临时设置;手动 GPU 也不能读到并发 CPU 构造的环境。"""
global _runtime_environment
- if selection.device == "native":
+ if selection.device == "native" and sys.platform not in {"linux", "win32"}:
yield
return
with _lock, vulkan_environment_lock():
@@ -224,10 +120,4 @@ def llama_engine_scope(selection: LlamaSelection) -> Iterator[None]:
os.environ[name] = value
-__all__ = [
- "LlamaSelection",
- "discover_llama_devices",
- "llama_engine_scope",
- "needs_serialized_submissions",
- "resolve_llama_device",
-]
+__all__ = ["LlamaSelection", "llama_engine_scope", "resolve_llama_device"]
diff --git a/mineru/model/runtime/llama_probe.py b/mineru/model/runtime/llama_probe.py
index 31508b84..86df1610 100644
--- a/mineru/model/runtime/llama_probe.py
+++ b/mineru/model/runtime/llama_probe.py
@@ -48,7 +48,11 @@ def _classify_devices(devices: tuple[LlamaDevice, ...], physical: tuple[VulkanDe
continue
native = physical[matched]
result[index] = replace(
- result[index], vendor_id=native.vendor_id, device_id=native.device_id, vulkan_index=native.index
+ result[index],
+ vendor_id=native.vendor_id,
+ device_id=native.device_id,
+ vulkan_index=native.index,
+ luid=native.luid or result[index].luid,
)
if sys.platform == "linux":
result[index] = with_gpu_classification(result[index], native)
@@ -92,21 +96,27 @@ def _probe_library(root: Path) -> tuple[LlamaDevice, ...]:
if path is None:
raise RuntimeError("Cannot find the packaged ggml device registry library")
library = ct.CDLL(str(path))
+ device_library = library
+ if sys.platform == "win32":
+ # Windows does not resolve exports through dependent DLLs. Device and
+ # backend accessors belong to ggml-base, while the registry is in ggml.
+ base_path = path.with_name(path.name.replace("ggml", "ggml-base"))
+ device_library = ct.CDLL(str(base_path))
load = library.ggml_backend_load_all_from_path
load.argtypes, load.restype = [ct.c_char_p], None
count = library.ggml_backend_dev_count
count.argtypes, count.restype = [], ct.c_size_t
get = library.ggml_backend_dev_get
get.argtypes, get.restype = [ct.c_size_t], ct.c_void_p
- kind = library.ggml_backend_dev_type
+ kind = device_library.ggml_backend_dev_type
kind.argtypes, kind.restype = [ct.c_void_p], ct.c_int
- name = library.ggml_backend_dev_name
+ name = device_library.ggml_backend_dev_name
name.argtypes, name.restype = [ct.c_void_p], ct.c_char_p
- description = library.ggml_backend_dev_description
+ description = device_library.ggml_backend_dev_description
description.argtypes, description.restype = [ct.c_void_p], ct.c_char_p
- registry = library.ggml_backend_dev_backend_reg
+ registry = device_library.ggml_backend_dev_backend_reg
registry.argtypes, registry.restype = [ct.c_void_p], ct.c_void_p
- backend = library.ggml_backend_reg_name
+ backend = device_library.ggml_backend_reg_name
backend.argtypes, backend.restype = [ct.c_void_p], ct.c_char_p
load(str(root / "bin").encode())
try:
diff --git a/mineru/model/runtime/tier_recommendation.py b/mineru/model/runtime/tier_recommendation.py
index 17f87168..e8f1bd6f 100644
--- a/mineru/model/runtime/tier_recommendation.py
+++ b/mineru/model/runtime/tier_recommendation.py
@@ -3,7 +3,6 @@
from __future__ import annotations
-import os
import sys
from dataclasses import dataclass, replace
from typing import TYPE_CHECKING
@@ -13,7 +12,7 @@ from loguru import logger
from ...types import TIERS, Tier, select_default_task_tier
from .cpu_threads import effective_cpu_count
from .device import resolve_small_model_backend, resolve_torch_device
-from .gpu import is_discrete_gpu, match_gpu_identities
+from .gpu import is_discrete_gpu
from .gpu_memory import GpuMemory, native_gpu_memory, torch_gpu_memory
from .onnx_calibration_cache import CalibrationInfo, inspect_calibration_record
@@ -92,11 +91,8 @@ def _onnx_gpu_hardware(count: int, choice: OnnxSelection | None = None) -> TierH
if choice is not None and (choice.device != "gpu" or choice.adapter is None):
return TierHardware(count, reason=choice.reason)
- backend = os.getenv("MINERU_LLAMA_CPP_BACKEND", "auto")
- if backend == "cpu":
- return TierHardware(count, reason="configured_cpu")
if sys.platform == "darwin":
- if backend in {"auto", "metal"} and _apple_metal_available():
+ if _apple_metal_available():
return _gpu_hardware(count, True, "apple_silicon_metal", torch_gpu_memory("mps"))
return TierHardware(count)
if sys.platform not in {"linux", "win32"}:
@@ -104,12 +100,6 @@ def _onnx_gpu_hardware(count: int, choice: OnnxSelection | None = None) -> TierH
if choice is None and get_config().model.onnx_device == "cpu":
return TierHardware(count, reason="configured_cpu")
- from .llama_device import discover_llama_devices
-
- devices = tuple(d for d in discover_llama_devices() if backend == "auto" or d.backend == backend)
- if not devices:
- return TierHardware(count, reason="no_visible_gpu")
-
if choice is not None:
adapter = choice.adapter
elif sys.platform == "win32":
@@ -122,9 +112,6 @@ def _onnx_gpu_hardware(count: int, choice: OnnxSelection | None = None) -> TierH
adapter = select_webgpu_device(discover_webgpu_devices())
if adapter is None:
return TierHardware(count, reason="no_eligible_gpu")
- matched = match_gpu_identities((adapter.identity,), [device.identity for device in devices])[0]
- if matched is None:
- return TierHardware(count, reason="gpu_identity_unconfirmed")
memory = native_gpu_memory(adapter.identity, device=f"{choice.backend if choice else 'onnx'}:gpu")
if choice is not None:
# 性能判断属于 ONNX 自动选型;已选 GPU 仅需满足内存门槛,不再要求历史校准证据。
@@ -141,7 +128,7 @@ def _onnx_gpu_hardware(count: int, choice: OnnxSelection | None = None) -> TierH
threshold = ONNX_GPU_TIME_RATIO_THRESHOLDS["webgpu" if sys.platform == "linux" else "dml"]
if threshold is None:
return TierHardware(count, reason="no_eligible_gpu")
- discrete = (is_discrete_gpu(adapter) or is_discrete_gpu(devices[matched])) and (
+ discrete = is_discrete_gpu(adapter) and (
not isinstance(adapter, WebGPUDevice) or adapter.vulkan_device == (adapter.vendor_id, adapter.device_id)
)
calibrated = calibrated_gpu_is_faster(adapter, threshold)
@@ -238,8 +225,6 @@ def recommend_tiers(*, vlm_config: VlmConfig | None = None) -> TierRecommendatio
warnings.append("GPU discovery unavailable; using CPU recommendation rules")
if hardware.gpu_memory is not None and hardware.gpu_memory.warning:
warnings.append(hardware.gpu_memory.warning)
- if hardware.reason == "gpu_identity_unconfirmed":
- warnings.append("GPU device identity could not be uniquely matched; using CPU recommendation rules")
logger.info(
"Recommended API tiers: {} (available CPUs: {}, reason: {})",
",".join(tier for tier in TIERS if tier in tiers),
diff --git a/mineru/model/runtime/vulkan.py b/mineru/model/runtime/vulkan.py
index d57795e3..4feeaee4 100644
--- a/mineru/model/runtime/vulkan.py
+++ b/mineru/model/runtime/vulkan.py
@@ -1,7 +1,8 @@
# Copyright (c) Opendatalab. All rights reserved.
"""No-compute Vulkan enumeration and scoped physical-device restrictions for Dawn.
-Uses the stable Vulkan 1.0 C ABI. No logical device, shader or GPU work is created.
+Uses Vulkan 1.0 properties and Vulkan 1.1 identity/budget queries.
+No logical device, shader or GPU work is created.
Callers must serialize scopes with the WebGPU runtime lock.
"""
@@ -33,11 +34,12 @@ class VulkanDevice(GpuClassification):
dedicated_memory_bytes: int | None = None
memory_budget_bytes: int | None = None
memory_error: str | None = None
+ luid: str | None = None
@property
def identity(self) -> GpuIdentity:
- """返回现有 PCI 型号和原生名称,不把枚举序号当作跨后端身份。"""
- return GpuIdentity(self.vendor_id, self.device_id, name=self.name)
+ """携带 Windows 原生 LUID,避免同型号 DXGI 适配器记录造成身份歧义。"""
+ return GpuIdentity(self.vendor_id, self.device_id, luid=self.luid, name=self.name)
class _ApplicationInfo(ct.Structure):
@@ -83,6 +85,35 @@ class _PropertiesStorage(ct.Union):
_fields_ = [("prefix", _PropertiesPrefix), ("storage", ct.c_uint64 * 128)]
+class _Properties2(ct.Structure):
+ _fields_ = [("sType", ct.c_uint32), ("pNext", ct.c_void_p), ("properties", _PropertiesStorage)]
+
+
+class _DeviceIDProperties(ct.Structure):
+ _fields_ = [
+ ("sType", ct.c_uint32),
+ ("pNext", ct.c_void_p),
+ ("deviceUUID", ct.c_uint8 * 16),
+ ("driverUUID", ct.c_uint8 * 16),
+ ("deviceLUID", ct.c_uint8 * 8),
+ ("deviceNodeMask", ct.c_uint32),
+ ("deviceLUIDValid", ct.c_uint32),
+ ]
+
+
+def _device_luid(library: Any, handle: ct.c_void_p) -> str | None:
+ """只接受 Vulkan 声明有效的 Windows LUID,并按 DXGI 的 high:low 格式编码。"""
+ identity = _DeviceIDProperties(sType=1000071004)
+ info = _Properties2(sType=1000059001, pNext=ct.cast(ct.pointer(identity), ct.c_void_p))
+ properties = library.vkGetPhysicalDeviceProperties2
+ properties.argtypes, properties.restype = [ct.c_void_p, ct.POINTER(_Properties2)], None
+ properties(handle, ct.byref(info))
+ if not identity.deviceLUIDValid:
+ return None
+ value = bytes(identity.deviceLUID)
+ return f"{int.from_bytes(value[4:], 'little'):08x}:{int.from_bytes(value[:4], 'little'):08x}"
+
+
class _MemoryType(ct.Structure):
_fields_ = [("flags", ct.c_uint32), ("heap_index", ct.c_uint32)]
@@ -203,6 +234,7 @@ def enumerate_vulkan_devices(*, include_memory: bool = False) -> tuple[VulkanDev
total,
budget,
memory_error,
+ luid=_device_luid(library, handle) if sys.platform == "win32" else None,
),
classify_vulkan_gpu(value.deviceType),
)
diff --git a/tests/unittest/test_gpu_policy.py b/tests/unittest/test_gpu_policy.py
index 0abb123d..e8910768 100644
--- a/tests/unittest/test_gpu_policy.py
+++ b/tests/unittest/test_gpu_policy.py
@@ -12,7 +12,7 @@ from unittest.mock import Mock
import pytest
-from mineru.model.runtime import hardware, llama_device, llama_probe, tier_recommendation, vulkan, webgpu, windows_gpu
+from mineru.model.runtime import hardware, llama_probe, tier_recommendation, vulkan, webgpu, windows_gpu
from mineru.model.runtime.gpu_memory import GpuMemory
from mineru.model.runtime.gpu import (
GPU_KIND_ORDER,
@@ -68,7 +68,6 @@ def test_native_classification_agrees_across_runtime_and_report(
monkeypatch.setattr(windows_gpu, "discover_gpus", lambda: (windows,))
monkeypatch.setattr(webgpu, "discover_webgpu_devices", lambda: (webgpu.WebGPUDevice(0x1002, 1),))
actual = llama_probe._classify_devices((llama_probe.LlamaDevice("Vulkan0", "GPU", "vulkan"),), (native,))
- monkeypatch.setattr(llama_device, "discover_llama_devices", lambda: actual)
ocr = (
windows_gpu.preferred_gpu((windows,))
if platform == "win32"
@@ -90,21 +89,15 @@ def test_native_classification_agrees_across_runtime_and_report(
@pytest.mark.parametrize("order", list(itertools.permutations(GPU_KIND_ORDER)))
-def test_backends_share_kind_order_with_native_tiebreakers(order: tuple[GpuKind, ...]) -> None:
- """Windows 性能排序和 llama 后端分组都只能在统一类型优先级之后生效。"""
+def test_windows_kind_order_with_native_tiebreakers(order: tuple[GpuKind, ...]) -> None:
+ """Windows 性能排序只能在统一类型优先级之后生效。"""
windows = tuple(
windows_gpu.GpuAdapter(index, str(index), kind, 0x1002, index, index, kind=kind) for index, kind in enumerate(order)
)
- llama = tuple(
- llama_probe.LlamaDevice(f"Vulkan{index}", kind, "vulkan", 0x1002, index, index, kind=kind)
- for index, kind in enumerate(order)
- )
for expected in GPU_KIND_ORDER:
selected = windows_gpu.preferred_gpu(windows)
assert selected is not None and selected.kind == expected
- assert llama_device._preferred(llama, "auto")[0].kind == expected
windows = tuple(device for device in windows if device.kind != expected)
- llama = tuple(device for device in llama if device.kind != expected)
def test_identity_matching_requires_mutual_uniqueness() -> None:
@@ -148,6 +141,28 @@ def test_ambiguous_native_mapping_remains_unknown_everywhere(monkeypatch: pytest
assert len(report) == 2 and all(device.kind == "unknown" for device in report)
[email protected]("luid", ["selected", "other", "missing", None])
[email protected]("reverse", [False, True])
+def test_windows_vulkan_luid_disambiguates_same_model_dxgi_adapters(
+ monkeypatch: pytest.MonkeyPatch, luid: str | None, reverse: bool
+) -> None:
+ """同型号 DXGI 库存必须按 Vulkan 的有效 LUID 关联,缺失或冲突时不猜测。"""
+ monkeypatch.setattr(llama_probe, "sys", SimpleNamespace(platform="win32"))
+ adapters = (
+ windows_gpu.GpuAdapter(0, "selected", "GPU", 0x10DE, 1, 0, kind="discrete", kind_source="d3d12"),
+ windows_gpu.GpuAdapter(2, "other", "GPU", 0x10DE, 1, 3, kind="integrated", kind_source="d3d12"),
+ )
+ monkeypatch.setattr(windows_gpu, "discover_gpus", lambda: adapters[::-1] if reverse else adapters)
+ result = llama_probe._classify_devices(
+ (llama_probe.LlamaDevice("Vulkan0", "GPU", "vulkan"),),
+ (vulkan.VulkanDevice(0x10DE, 1, "GPU", 0, luid=luid),),
+ )
+ expected = {"selected": "discrete", "other": "integrated"}.get(luid, "unknown")
+ assert result[0].kind == expected
+ assert result[0].identity.luid == luid
+ assert result[0].vulkan_index == 0
+
+
def test_windows_native_query_failure_does_not_use_ggml_or_vulkan_type(monkeypatch: pytest.MonkeyPatch) -> None:
"""Windows 的分类查询失败时保留未知,不把 ggml/Vulkan 的 GPU 标签升级为独显。"""
monkeypatch.setattr(llama_probe, "sys", SimpleNamespace(platform="win32"))
@@ -186,6 +201,38 @@ def test_ggml_gpu_enum_only_controls_inventory(monkeypatch: pytest.MonkeyPatch,
llama_probe.LlamaDevice("Vulkan0", "GPU", "vulkan", integrated=True)
[email protected]("folder", ["bin", "lib"])
[email protected]("prefix", ["", "lib"])
+def test_windows_probe_loads_device_accessors_from_base_dll(
+ monkeypatch: pytest.MonkeyPatch, tmp_path: Path, folder: str, prefix: str
+) -> None:
+ """Windows DLL 导出不沿依赖解析;严格分离注册与设备接口以覆盖真实 wheel 布局。"""
+ registry_path = tmp_path / folder / f"{prefix}ggml.dll"
+ registry_path.parent.mkdir()
+ registry_path.touch()
+ base_path = registry_path.with_name(f"{prefix}ggml-base.dll")
+ base_path.touch()
+ registry_library = SimpleNamespace(
+ ggml_backend_load_all_from_path=Mock(),
+ ggml_backend_dev_count=Mock(return_value=1),
+ ggml_backend_dev_get=Mock(return_value=1),
+ )
+ base_library = SimpleNamespace(
+ ggml_backend_dev_type=Mock(return_value=1),
+ ggml_backend_dev_name=Mock(return_value=b"Vulkan0"),
+ ggml_backend_dev_description=Mock(return_value=b"GPU"),
+ ggml_backend_dev_backend_reg=Mock(return_value=1),
+ ggml_backend_reg_name=Mock(return_value=b"Vulkan"),
+ )
+ libraries = {str(registry_path): registry_library, str(base_path): base_library}
+ monkeypatch.setattr(llama_probe, "sys", SimpleNamespace(platform="win32"))
+ monkeypatch.setattr(llama_probe.ct, "CDLL", libraries.__getitem__)
+ monkeypatch.setattr(llama_probe, "enumerate_vulkan_devices", lambda: ())
+ monkeypatch.setattr(windows_gpu, "discover_gpus", lambda: ())
+
+ assert llama_probe._probe_library(tmp_path) == (llama_probe.LlamaDevice("Vulkan0", "GPU", "vulkan"),)
+
+
def test_apple_classification_preserves_unified_memory() -> None:
"""Apple Silicon 的集显分类保留统一内存事实,不改变 MPS 的可用性推荐。"""
apple = classify_macos_gpu("Apple", "arm64", integrated=False)
diff --git a/tests/unittest/test_hardware_recommendation.py b/tests/unittest/test_hardware_recommendation.py
index 10a17f60..88f5bb03 100644
--- a/tests/unittest/test_hardware_recommendation.py
+++ b/tests/unittest/test_hardware_recommendation.py
@@ -18,7 +18,6 @@ from mineru.config import VlmConfig, get_config
from mineru.kit.commands import hardware as hardware_cmd
from mineru.kit.main import app as kit_app
from mineru.model.runtime import device, llama_device, onnx_calibration_cache, onnx_device, tier_recommendation
-from mineru.model.runtime.llama_probe import LlamaDevice
from mineru.parser import api_server
from mineru.types import TIERS
@@ -196,22 +195,13 @@ def test_remote_vlm_and_api_override_match_detect(
def test_torch_llama_integrated_route_does_not_return_to_onnx(
torch_environment: Mock, monkeypatch: pytest.MonkeyPatch, accelerated: bool
) -> None:
- """llama 自动集显复用 Torch 可用性,后续绑定及 Intel 兼容策略继续执行。"""
+ """llama 自动模式仅复用 Torch 可用性,不探测显卡或注入 Intel 兼容设置。"""
monkeypatch.setattr(llama_device, "sys", SimpleNamespace(platform="linux"))
monkeypatch.delenv("MINERU_LLAMA_CPP_BACKEND", raising=False)
monkeypatch.delenv("GGML_VK_SERIALIZE_SUBMISSIONS", raising=False)
monkeypatch.setenv("MINERU_DEVICE_MODE", "xpu" if accelerated else "cpu")
_fake_torch(monkeypatch, "xpu", accelerated)
- native = LlamaDevice("Vulkan0", "Arc", "vulkan", 0x8086, 0x7D51, 0, kind="integrated", kind_source="vulkan")
- monkeypatch.setattr(llama_device, "discover_llama_devices", lambda: (native,))
- bind = Mock(side_effect=lambda selection: selection)
- monkeypatch.setattr(llama_device, "_bind", bind)
result = llama_device.resolve_llama_device()
assert result.device == ("gpu" if accelerated else "cpu")
- if accelerated:
- assert result.devices == (native,)
- assert dict(result.environment)["GGML_VK_SERIALIZE_SUBMISSIONS"] == "1"
- bind.assert_called_once()
- else:
- bind.assert_not_called()
+ assert result.environment == ()
torch_environment.assert_not_called()
diff --git a/tests/unittest/test_llama_device_policy.py b/tests/unittest/test_llama_device_policy.py
index 44789a4a..9b0d1b5e 100644
--- a/tests/unittest/test_llama_device_policy.py
+++ b/tests/unittest/test_llama_device_policy.py
@@ -1,304 +1,202 @@
-"""实际可见设备、跨平台集显决策与进程级 Vulkan 环境隔离。"""
+"""小模型后端统一决定 CPU/GPU;llama 不枚举、匹配或绑定显卡。"""
from __future__ import annotations
+from concurrent.futures import ThreadPoolExecutor
from dataclasses import replace
-import json
import os
from pathlib import Path
import subprocess
import sys
+import threading
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
-from mineru.model.runtime.gpu import classify_d3d12_gpu, with_gpu_classification
-from mineru.model.runtime import llama_device as policy
-from mineru.model.runtime.llama_probe import LlamaDevice, PROBE_MARKER
+from mineru.config import get_config
+from mineru.model.runtime import device, llama_device as policy, llama_probe, onnx_device
+from mineru.model.runtime.device import TorchDeviceSelection
from mineru.model.runtime.onnx_device import OnnxSelection
from mineru.model.runtime.webgpu import WebGPUDevice
-from mineru.model.runtime.windows_gpu import GpuAdapter
-
-
-def gpu(vendor: int, chip: int, *, integrated: bool = True, index: int = 0) -> LlamaDevice:
- """构造有稳定物理索引的原生设备。"""
- return LlamaDevice(
- f"Vulkan{index}",
- f"GPU {vendor:04x}:{chip:04x}",
- "vulkan",
- vendor,
- chip,
- index,
- kind="integrated" if integrated else "discrete",
- kind_source="vulkan",
- )
@pytest.fixture(autouse=True)
def isolated(monkeypatch: pytest.MonkeyPatch) -> None:
- """单测不触发真实设备探测、模型加载或持久化校准。"""
- monkeypatch.setattr(policy, "sys", SimpleNamespace(platform="linux", executable="python"))
+ monkeypatch.setattr(policy, "sys", SimpleNamespace(platform="linux"))
monkeypatch.setattr(policy, "_runtime_environment", None)
- from mineru.model.runtime import device
-
monkeypatch.setattr(device, "resolve_small_model_backend", lambda: "onnx")
for name in (*policy._BACKEND_ENV, "MINERU_LLAMA_CPP_BACKEND"):
monkeypatch.delenv(name, raising=False)
+ forbidden = Mock(side_effect=AssertionError("llama runtime must not probe devices"))
+ monkeypatch.setattr(llama_probe, "probe_devices", forbidden)
+ monkeypatch.setattr(llama_probe, "_probe_library", forbidden)
+ monkeypatch.setattr(subprocess, "run", forbidden)
-def inventory(monkeypatch: pytest.MonkeyPatch, devices: tuple[LlamaDevice, ...]) -> Mock:
- """模拟 ggml 可见设备过滤和过滤后 VulkanN 的重新编号。"""
+def onnx(monkeypatch: pytest.MonkeyPatch, selection: OnnxSelection) -> Mock:
+ function = Mock(return_value=selection)
+ monkeypatch.setattr(onnx_device, "resolve_onnx_device", function)
+ return function
- def discover(environment: dict[str, str] | None = None) -> tuple[LlamaDevice, ...]:
- if environment is None or "GGML_VK_VISIBLE_DEVICES" not in environment:
- return devices
- indices = [int(i) for i in environment["GGML_VK_VISIBLE_DEVICES"].split(",")]
- return tuple(
- replace(next(d for d in devices if d.vulkan_index == index), name=f"Vulkan{i}") for i, index in enumerate(indices)
- )
- function = Mock(side_effect=discover)
- monkeypatch.setattr(policy, "discover_llama_devices", function)
- return function
[email protected]("platform", ["win32", "linux"])
+ "decision,reason",
+ [
+ ("gpu", "discrete_gpu"),
+ ("gpu", "rec_calibration"),
+ ("gpu", "configured_gpu"),
+ ("cpu", "rec_calibration"),
+ ("cpu", "configured_cpu"),
+ ("cpu", "no_hardware_gpu"),
+ ("cpu", "selection_failed"),
+ ("cpu", "webgpu_architecture_policy"),
+ ],
+)
+def test_automatic_route_consumes_onnx_result(
+ monkeypatch: pytest.MonkeyPatch, platform: str, decision: str, reason: str
+) -> None:
+ monkeypatch.setattr(policy.sys, "platform", platform)
+ # CPU/GPU 结论本身即可驱动 llama;不要求携带或匹配另一套显卡库存。
+ select = onnx(monkeypatch, OnnxSelection(decision, reason=reason))
+ result = policy.resolve_llama_device()
+ assert result == policy.LlamaSelection(decision, reason=f"onnx_{reason}")
+ select.assert_called_once_with()
-def onnx(monkeypatch: pytest.MonkeyPatch, selection: OnnxSelection) -> Mock:
- """替换共享 ONNX 决策,单独核验是否应当调用。"""
- from mineru.model.runtime import onnx_device
+def test_repeated_resolution_reuses_onnx_cache(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(get_config().model, "onnx_device", "gpu")
+ select = Mock(return_value=OnnxSelection("gpu", reason="configured_gpu"))
+ monkeypatch.setattr(onnx_device, "_select_and_save", select)
+ assert policy.resolve_llama_device() == policy.resolve_llama_device()
+ select.assert_called_once()
- function = Mock(return_value=selection)
- monkeypatch.setattr(onnx_device, "resolve_onnx_device", function)
- return function
+
+def test_offline_resolution_preserves_download_constraint(monkeypatch: pytest.MonkeyPatch) -> None:
+ select = onnx(monkeypatch, OnnxSelection(reason="selection_failed"))
+ assert policy.resolve_llama_device(allow_download=False).device == "cpu"
+ select.assert_called_once_with(allow_download=False)
+
+
[email protected]("accelerated", [False, True])
+def test_torch_decision_does_not_use_onnx_or_intel_workaround(monkeypatch: pytest.MonkeyPatch, accelerated: bool) -> None:
+ monkeypatch.setattr(device, "resolve_small_model_backend", lambda: "torch")
+ monkeypatch.setattr(
+ device, "resolve_torch_device", lambda: TorchDeviceSelection("xpu" if accelerated else "cpu", "torch_result")
+ )
+ select = onnx(monkeypatch, OnnxSelection())
+ result = policy.resolve_llama_device()
+ assert result == policy.LlamaSelection("gpu" if accelerated else "cpu", reason="torch_result")
+ select.assert_not_called()
[email protected]("platform", ["linux", "win32"])
@pytest.mark.parametrize(
- "vendor,chip,expected",
+ "vendor,chip,affected",
[
(0x8086, 0x4680, True),
(0x8086, 0x3E92, True),
- (0x8086, 0x4905, True),
(0x8086, 0x7D51, True),
- (0x8086, 0x7D55, True),
(0x8086, 0xB640, True),
- (0x8086, 0x4F80, False),
(0x8086, 0x56A0, False),
- (0x8086, 0x6420, False),
- (0x8086, 0xE202, False),
- (0x8086, 0xB080, False),
- (0x8086, 0xFD80, False),
+ (0x8086, 0x64A0, False),
(0x8086, 0xFFFF, False),
- (0x1002, 0x4680, False),
- (0x10DE, 0x4680, False),
- (0x1002, 0x7D51, False),
(0x10DE, 0x7D51, False),
],
)
-def test_explicit_architecture_mapping(vendor: int, chip: int, expected: bool) -> None:
- assert policy.needs_serialized_submissions(gpu(vendor, chip)) is expected
-
-
[email protected]("chip", [0x4680, 0x7D51, 0xB640])
[email protected]("platform", ["win32", "darwin"])
-def test_workaround_is_linux_vulkan_only(monkeypatch: pytest.MonkeyPatch, chip: int, platform: str) -> None:
- intel = gpu(0x8086, chip)
- assert not policy.needs_serialized_submissions(replace(intel, backend="metal"))
+def test_intel_compatibility_uses_existing_onnx_adapter(
+ monkeypatch: pytest.MonkeyPatch, platform: str, vendor: int, chip: int, affected: bool
+) -> None:
monkeypatch.setattr(policy.sys, "platform", platform)
- assert not policy.needs_serialized_submissions(intel)
+ onnx(monkeypatch, OnnxSelection("gpu", WebGPUDevice(vendor, chip), "rec_calibration", "webgpu"))
+ result = policy.resolve_llama_device()
+ assert result.environment == (((policy.SERIALIZE_ENV, "1"),) if platform == "linux" and affected else ())
+ assert "GGML_VK_VISIBLE_DEVICES" not in dict(result.environment)
-def test_missing_pci_identity_does_not_guess_from_name() -> None:
- intel = gpu(0x8086, 0x7D51)
- assert not policy.needs_serialized_submissions(replace(intel, device_id=None, description="Intel Arc Xe-LPG"))
- assert not policy.needs_serialized_submissions(replace(intel, vendor_id=None))
[email protected]("value", ["0", "1"])
+def test_existing_serialization_setting_wins(monkeypatch: pytest.MonkeyPatch, value: str) -> None:
+ monkeypatch.setenv(policy.SERIALIZE_ENV, value)
+ onnx(monkeypatch, OnnxSelection("gpu", WebGPUDevice(0x8086, 0x7D51), "configured_gpu", "webgpu"))
+ result = policy.resolve_llama_device()
+ assert not result.environment
+ with policy.llama_engine_scope(result):
+ assert os.environ[policy.SERIALIZE_ENV] == value
+ assert os.environ[policy.SERIALIZE_ENV] == value
[email protected]("platform", ["linux", "win32"])
[email protected]("vendor,chip", [(0x10DE, 0x2489), (0x1002, 0x73BF), (0x8086, 0x56A0)])
[email protected]("igpu_chip", [0x4680, 0x7D51])
-def test_discrete_gpu_ignores_cpu_onnx_choice(
- monkeypatch: pytest.MonkeyPatch, platform: str, vendor: int, chip: int, igpu_chip: int
-) -> None:
- monkeypatch.setattr(policy.sys, "platform", platform)
- intel, discrete = gpu(0x8086, igpu_chip), gpu(vendor, chip, integrated=False, index=1)
- inventory(monkeypatch, (intel, discrete))
- calibrate = onnx(monkeypatch, OnnxSelection(reason="configured_cpu"))
- selected = policy.resolve_llama_device()
- assert selected.device == "gpu" and selected.devices[0].device_id == chip
- assert dict(selected.environment) == {"GGML_VK_VISIBLE_DEVICES": "1"}
- assert selected.devices[0].name == "Vulkan0"
- calibrate.assert_not_called()
-
-
[email protected]("platform,backend", [("linux", "webgpu"), ("win32", "dml")])
[email protected]("decision", ["cpu", "gpu"])
-def test_integrated_only_reuses_platform_decision(
- monkeypatch: pytest.MonkeyPatch, platform: str, backend: str, decision: str
+ "backend,layers,names,expected",
+ [
+ ("auto", 0, None, "cpu"),
+ ("cpu", None, None, "cpu"),
+ ("auto", None, ("none",), "cpu"),
+ ("vulkan", None, None, "native"),
+ ("metal", None, None, "native"),
+ ("auto", 99, None, "native"),
+ ("auto", -1, None, "native"),
+ ("auto", None, ("Vulkan99",), "native"),
+ ],
+)
+def test_explicit_overrides_never_select_or_validate_inventory(
+ monkeypatch: pytest.MonkeyPatch, backend: str, layers: int | None, names: tuple[str, ...] | None, expected: str
) -> None:
+ monkeypatch.setenv("MINERU_LLAMA_CPP_BACKEND", backend)
+ select = onnx(monkeypatch, OnnxSelection())
+ result = policy.resolve_llama_device(n_gpu_layers=layers, device_names=names)
+ assert result.device == expected and not result.environment
+ select.assert_not_called()
+
+
+def test_invalid_backend_is_rejected_even_with_cpu_layers(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setenv("MINERU_LLAMA_CPP_BACKEND", "invalid")
+ with pytest.raises(ValueError, match="MINERU_LLAMA_CPP_BACKEND"):
+ policy.resolve_llama_device(n_gpu_layers=0)
+
+
[email protected]("platform", ["darwin", "other"])
+def test_other_platforms_keep_native_routing(monkeypatch: pytest.MonkeyPatch, platform: str) -> None:
monkeypatch.setattr(policy.sys, "platform", platform)
- device = gpu(0x8086, 0x7D51)
- inventory(monkeypatch, (device,))
- adapter = (
- WebGPUDevice(0x8086, 0x7D51)
- if backend == "webgpu"
- else with_gpu_classification(GpuAdapter(0, "luid", "Arc", 0x8086, 0x7D51, 0), classify_d3d12_gpu(True))
- )
- calibrate = onnx(monkeypatch, OnnxSelection(decision, adapter if decision == "gpu" else None, "rec_calibration", backend))
- result = policy.resolve_llama_device()
- assert result.device == decision
- expected = "1" if platform == "linux" and decision == "gpu" else None
- assert dict(result.environment).get(policy.SERIALIZE_ENV) == expected
- calibrate.assert_called_once_with()
-
-
-def test_old_intel_architecture_fallback_and_forced_vulkan(monkeypatch: pytest.MonkeyPatch) -> None:
- inventory(monkeypatch, (gpu(0x8086, 0x4680),))
- calibrate = onnx(monkeypatch, OnnxSelection(reason="webgpu_architecture_policy", backend="webgpu"))
- assert policy.resolve_llama_device().device == "cpu"
- forced = policy.resolve_llama_device(backend="vulkan")
- assert forced.device == "gpu" and dict(forced.environment)[policy.SERIALIZE_ENV] == "1"
- assert calibrate.call_count == 1
-
-
-def test_failed_or_mismatched_calibration_cannot_select_gpu(monkeypatch: pytest.MonkeyPatch) -> None:
- inventory(monkeypatch, (gpu(0x1002, 0x150E),))
- onnx(monkeypatch, OnnxSelection(reason="selection_failed"))
- assert policy.resolve_llama_device().device == "cpu"
- onnx(monkeypatch, OnnxSelection("gpu", WebGPUDevice(0x10DE, 0x2489), "rec_calibration", "webgpu"))
- assert policy.resolve_llama_device().reason == "onnx_gpu_identity_unconfirmed"
-
-
-def test_unselected_backend_discrete_does_not_exempt_integrated(monkeypatch: pytest.MonkeyPatch) -> None:
- """其他后端的独显不在自动候选中时,实际选中的 Vulkan 集显仍需复用校准决策。"""
- unavailable = replace(gpu(0x10DE, 1, integrated=False), backend="other")
- integrated = gpu(0x1002, 0x150E, index=1)
- inventory(monkeypatch, (unavailable, integrated))
- calibrate = onnx(monkeypatch, OnnxSelection(reason="configured_cpu"))
- result = policy.resolve_llama_device()
- assert result.device == "cpu" and result.reason == "onnx_configured_cpu"
- calibrate.assert_called_once()
+ select = onnx(monkeypatch, OnnxSelection())
+ assert policy.resolve_llama_device() == policy.LlamaSelection(reason="platform_native")
+ select.assert_not_called()
-def test_no_gpu_and_explicit_cpu_do_not_calibrate(monkeypatch: pytest.MonkeyPatch) -> None:
- discover = inventory(monkeypatch, ())
- calibrate = onnx(monkeypatch, OnnxSelection())
- assert policy.resolve_llama_device(n_gpu_layers=0).device == "cpu"
- discover.assert_not_called()
- assert policy.resolve_llama_device().device == "cpu"
- calibrate.assert_not_called()
- with pytest.raises(RuntimeError, match="unavailable"):
- policy.resolve_llama_device(backend="vulkan")
+def test_explicit_native_initialization_waits_for_temporary_cpu_environment() -> None:
+ """自动 CPU 的临时后端变量不能让并发的手动 GPU 构造落到 CPU。"""
+ attempted, entered = threading.Event(), threading.Event()
+ def initialize_native() -> None:
+ attempted.set()
+ with policy.llama_engine_scope(policy.LlamaSelection(reason="explicit_gpu")):
+ assert "MINERU_LLAMA_CPP_BACKEND" not in os.environ
+ entered.set()
[email protected]("chip", [0x4680, 0x7D51, 0xB640])
[email protected]("other_vendor,other_chip", [(0x10DE, 0x2489), (0x1002, 0x73BF), (0x8086, 0x56A0)])
-def test_explicit_target_on_multi_gpu_is_bound_before_workaround(
- monkeypatch: pytest.MonkeyPatch, chip: int, other_vendor: int, other_chip: int
-) -> None:
- discrete = gpu(other_vendor, other_chip, integrated=False)
- intel = gpu(0x8086, chip, index=1)
- inventory(monkeypatch, (discrete, intel))
- calibrate = onnx(monkeypatch, OnnxSelection())
- result = policy.resolve_llama_device(device_names=("Vulkan1",))
- assert result.devices[0].device_id == chip
- assert dict(result.environment) == {policy.SERIALIZE_ENV: "1", "GGML_VK_VISIBLE_DEVICES": "1"}
- assert result.device_aliases == (("Vulkan1", "Vulkan0"),)
- calibrate.assert_not_called()
-
-
[email protected]("chip", [0x4905, 0x7D51])
[email protected]("other_vendor,other_chip", [(0x10DE, 0x2489), (0x8086, 0x56A0)])
-def test_mixed_submission_policies_do_not_slow_other_gpu(
- monkeypatch: pytest.MonkeyPatch, chip: int, other_vendor: int, other_chip: int
-) -> None:
- inventory(
- monkeypatch,
- (gpu(0x8086, chip, integrated=chip == 0x7D51), gpu(other_vendor, other_chip, integrated=False, index=1)),
- )
- selected = policy.resolve_llama_device()
- assert selected.devices[0].device_id == other_chip
- assert policy.SERIALIZE_ENV not in dict(selected.environment)
- with pytest.raises(ValueError, match="different Vulkan submission policies"):
- policy.resolve_llama_device(device_names=("Vulkan0", "Vulkan1"))
-
-
[email protected]("chip", [0x4680, 0x7D51, 0x56A0])
-def test_manual_environment_is_preserved_including_zero(monkeypatch: pytest.MonkeyPatch, chip: int) -> None:
- inventory(monkeypatch, (gpu(0x8086, chip, integrated=chip != 0x56A0),))
- monkeypatch.setenv(policy.SERIALIZE_ENV, "0")
- result = policy.resolve_llama_device(n_gpu_layers=99)
- assert policy.SERIALIZE_ENV not in dict(result.environment)
- with policy.llama_engine_scope(result):
- assert os.environ[policy.SERIALIZE_ENV] == "0"
- assert os.environ[policy.SERIALIZE_ENV] == "0"
- assert "GGML_VK_VISIBLE_DEVICES" not in os.environ
+ with ThreadPoolExecutor(max_workers=1) as executor:
+ with policy.llama_engine_scope(policy.LlamaSelection("cpu")):
+ future = executor.submit(initialize_native)
+ assert attempted.wait(3)
+ assert not entered.wait(0.05)
+ future.result(timeout=3)
+ assert entered.is_set()
[email protected]("chip", [0x4680, 0x7D51])
-def test_environment_restored_on_failure_and_cached_backend_change_rejected(monkeypatch: pytest.MonkeyPatch, chip: int) -> None:
- inventory(monkeypatch, (gpu(0x8086, chip),))
- result = policy.resolve_llama_device(n_gpu_layers=99)
- with pytest.raises(ValueError, match="constructor"):
+def test_environment_restored_and_cached_vulkan_setting_cannot_change() -> None:
+ result = policy.LlamaSelection("gpu", environment=((policy.SERIALIZE_ENV, "1"),))
+ with pytest.raises(RuntimeError, match="constructor"):
with policy.llama_engine_scope(result):
assert os.environ[policy.SERIALIZE_ENV] == "1"
- raise ValueError("constructor")
- assert policy.SERIALIZE_ENV not in os.environ and "GGML_VK_VISIBLE_DEVICES" not in os.environ
+ raise RuntimeError("constructor")
+ assert policy.SERIALIZE_ENV not in os.environ
with policy.llama_engine_scope(result):
assert os.environ[policy.SERIALIZE_ENV] == "1"
with pytest.raises(RuntimeError, match="fresh process"):
- with policy.llama_engine_scope(replace(result, environment=(("GGML_VK_VISIBLE_DEVICES", "0"),))):
+ with policy.llama_engine_scope(replace(result, environment=())):
pass
assert policy.SERIALIZE_ENV not in os.environ
[email protected]("chip", [0x4680, 0x7D51])
-def test_binding_verification_rejects_wrong_physical_gpu(monkeypatch: pytest.MonkeyPatch, chip: int) -> None:
- intel, other = gpu(0x8086, chip), gpu(0x10DE, 0x2489, integrated=False)
- monkeypatch.setattr(policy, "discover_llama_devices", Mock(side_effect=[(intel,), (other,)]))
- with pytest.raises(RuntimeError, match="binding changed"):
- policy.resolve_llama_device(backend="vulkan")
-
-
[email protected]("chip", [0x4680, 0x7D51])
-def test_missing_device_index_cannot_apply_global_workaround(monkeypatch: pytest.MonkeyPatch, chip: int) -> None:
- inventory(monkeypatch, (replace(gpu(0x8086, chip), vulkan_index=None),))
- with pytest.raises(RuntimeError, match="Cannot safely bind"):
- policy.resolve_llama_device(backend="vulkan")
- assert policy.SERIALIZE_ENV not in os.environ
-
-
-def test_probe_protocol_uses_native_inventory_without_parsing_log_names(monkeypatch: pytest.MonkeyPatch) -> None:
- native = [
- {
- "name": "Vulkan0",
- "description": "device",
- "backend": "vulkan",
- "kind": "integrated",
- "kind_source": "vulkan",
- "vendor_id": 0x8086,
- "device_id": 0x4680,
- "vulkan_index": 1,
- }
- ]
- run = Mock(return_value=SimpleNamespace(stdout="native log\n" + PROBE_MARKER + json.dumps(native) + "\n"))
- monkeypatch.setattr(policy.subprocess, "run", run)
- assert policy.discover_llama_devices()[0].vulkan_index == 1
- assert run.call_args.kwargs["env"] == dict(os.environ)
- assert "onnxruntime" not in run.call_args.args[0]
-
-
-def test_other_platforms_do_not_probe_or_calibrate(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setattr(policy.sys, "platform", "darwin")
- probe = inventory(monkeypatch, ())
- assert policy.resolve_llama_device().device == "native"
- probe.assert_not_called()
-
-
-def test_discovery_error_is_contained(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setattr(policy, "discover_llama_devices", Mock(side_effect=subprocess.CalledProcessError(1, "probe")))
- assert policy.resolve_llama_device().reason == "discovery_unavailable"
-
-
@pytest.mark.parametrize(
"chip,forced,calibrated_gpu",
[(0x4680, False, False), (0x4680, True, False), (0x7D51, False, False), (0x7D51, True, False), (0x7D51, False, True)],
@@ -308,7 +206,6 @@ def test_real_model_factory_applies_plan_at_engine_construction(
) -> None:
from mineru.model.vlm import runtime
- inventory(monkeypatch, (gpu(0x8086, chip),))
choice = (
OnnxSelection("gpu", WebGPUDevice(0x8086, chip), "rec_calibration", "webgpu")
if calibrated_gpu
@@ -330,6 +227,6 @@ def test_real_model_factory_applies_plan_at_engine_construction(
kwargs, serialize, backend = captured[0]
use_gpu = forced or calibrated_gpu
assert kwargs.get("n_gpu_layers") == (99 if forced else None if use_gpu else 0)
- assert serialize == ("1" if use_gpu else None)
+ assert serialize == ("1" if calibrated_gpu and not forced else None)
assert backend == (None if use_gpu else "cpu")
assert policy.SERIALIZE_ENV not in os.environ and "MINERU_LLAMA_CPP_BACKEND" not in os.environ
diff --git a/tests/unittest/test_llama_server_device_policy.py b/tests/unittest/test_llama_server_device_policy.py
index bcded302..4f5feef1 100644
--- a/tests/unittest/test_llama_server_device_policy.py
+++ b/tests/unittest/test_llama_server_device_policy.py
@@ -1,29 +1,39 @@
-"""server 与进程内引擎遵循同一设备计划,参数别名与环境不泄漏。"""
+"""Engine/server 共用 CPU/GPU 决策,显式设备参数原样交给原生运行时。"""
from __future__ import annotations
import os
from pathlib import Path
+import subprocess
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
from mineru.kit.vlm_server import llama_cpp_server as server
-from mineru.model.runtime import llama_device as policy
+from mineru.model.runtime import device, llama_device as policy, llama_probe, onnx_device
from mineru.model.runtime.llama_device import LlamaSelection, SERIALIZE_ENV
-from mineru.model.runtime.llama_probe import LlamaDevice
+from mineru.model.runtime.onnx_device import OnnxSelection
+from mineru.model.runtime.webgpu import WebGPUDevice
-def plan() -> LlamaSelection:
- """显式选择原来的 Vulkan1 后,隔离原生设备索引会变为 Vulkan0。"""
- return LlamaSelection(
- "gpu",
- (LlamaDevice("Vulkan0", "Intel", "vulkan", 0x8086, 0x4680, 1, kind="integrated", kind_source="vulkan"),),
- "explicit_gpu",
- ((SERIALIZE_ENV, "1"), ("GGML_VK_VISIBLE_DEVICES", "1")),
- (("Vulkan1", "Vulkan0"),),
- )
[email protected](autouse=True)
+def isolated(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(policy, "sys", SimpleNamespace(platform="linux"))
+ monkeypatch.setattr(device, "resolve_small_model_backend", lambda: "onnx")
+ for name in (
+ *policy._BACKEND_ENV,
+ "MINERU_LLAMA_CPP_BACKEND",
+ "LLAMA_ARG_DEVICE",
+ "LLAMA_ARG_N_GPU_LAYERS",
+ "MTMD_BACKEND_DEVICE",
+ "LLAMA_ARG_MMPROJ_OFFLOAD",
+ "LLAMA_ARG_OFFLINE",
+ ):
+ monkeypatch.delenv(name, raising=False)
+ forbidden = Mock(side_effect=AssertionError("server must not probe llama devices"))
+ monkeypatch.setattr(llama_probe, "probe_devices", forbidden)
+ monkeypatch.setattr(subprocess, "run", forbidden)
@pytest.mark.parametrize(
@@ -39,8 +49,6 @@ def plan() -> LlamaSelection:
def test_raw_selection_arguments_are_read_before_defaults(
monkeypatch: pytest.MonkeyPatch, args: list[str], layers: int | None, names: tuple[str, ...] | None
) -> None:
- monkeypatch.delenv("LLAMA_ARG_DEVICE", raising=False)
- monkeypatch.delenv("LLAMA_ARG_N_GPU_LAYERS", raising=False)
resolve = Mock(return_value=LlamaSelection())
monkeypatch.setattr(server, "resolve_llama_device", resolve)
server._device_policy(args)
@@ -57,24 +65,35 @@ def test_native_environment_selectors(monkeypatch: pytest.MonkeyPatch) -> None:
def test_offline_prohibits_calibration_download(monkeypatch: pytest.MonkeyPatch) -> None:
- for name in ("LLAMA_ARG_N_GPU_LAYERS", "LLAMA_ARG_DEVICE", "MTMD_BACKEND_DEVICE", "LLAMA_ARG_MMPROJ_OFFLOAD"):
- monkeypatch.delenv(name, raising=False)
resolve = Mock(return_value=LlamaSelection())
monkeypatch.setattr(server, "resolve_llama_device", resolve)
server._device_policy(["--offline"])
resolve.assert_called_once_with(n_gpu_layers=None, device_names=None, allow_download=False)
-def test_cpu_model_with_explicit_gpu_projector_still_gets_workaround(monkeypatch: pytest.MonkeyPatch) -> None:
- resolve = Mock(return_value=plan())
- monkeypatch.setattr(server, "resolve_llama_device", resolve)
- args = ["-ngl", "0", "--mmproj-device", "Vulkan1", "--mmproj-offload"]
[email protected]("source", ["cli", "env"])
[email protected]("cpu_model", [False, True])
+def test_explicit_projector_preserves_native_names_and_mixed_routing(
+ monkeypatch: pytest.MonkeyPatch, source: str, cpu_model: bool
+) -> None:
+ args = ["-ngl", "0"] if cpu_model else ["--device", "Vulkan1"]
+ if source == "cli":
+ args += ["--mmproj-device", "Vulkan9"]
+ else:
+ monkeypatch.setenv("MTMD_BACKEND_DEVICE", "Vulkan9")
+ select = Mock(side_effect=AssertionError("explicit projector must bypass ONNX"))
+ monkeypatch.setattr(onnx_device, "resolve_onnx_device", select)
selected = server._device_policy(args)
- assert selected.cpu_model
+ assert selected.device == "native" and not selected.environment
server._apply_device_policy(args, selected)
- assert server._flag_value(args, "--n-gpu-layers") == "0"
- assert server._flag_value(args, "--mmproj-device") == "Vulkan0"
- assert dict(selected.environment)[SERIALIZE_ENV] == "1"
+ assert (server._flag_value(args, "--mmproj-device") or os.environ.get("MTMD_BACKEND_DEVICE")) == "Vulkan9"
+ assert "--mmproj-offload" in args
+ if cpu_model:
+ assert server._flag_value(args, "--n-gpu-layers", "-ngl") == "0"
+ assert "--no-op-offload" in args
+ else:
+ assert server._flag_value(args, "--device") == "Vulkan1"
+ select.assert_not_called()
def test_disabled_projector_does_not_activate_gpu(monkeypatch: pytest.MonkeyPatch) -> None:
@@ -84,109 +103,78 @@ def test_disabled_projector_does_not_activate_gpu(monkeypatch: pytest.MonkeyPatc
resolve.assert_called_once_with(n_gpu_layers=0, device_names=None)
-def test_cpu_policy_disables_model_projector_and_op_offload(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.delenv("LLAMA_ARG_MMPROJ_OFFLOAD", raising=False)
+def test_cpu_policy_disables_model_projector_and_op_offload() -> None:
args = []
server._apply_device_policy(args, LlamaSelection("cpu", reason="onnx_rec_calibration"))
assert server._flag_value(args, "--n-gpu-layers") == "0"
assert server._flag_value(args, "--device") == "none"
+ assert server._flag_value(args, "--mmproj-device") == "none"
assert "--no-mmproj-offload" in args and "--no-op-offload" in args
-def test_projector_device_remaps_with_model(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.delenv("MTMD_BACKEND_DEVICE", raising=False)
- args = ["--device", "Vulkan1", "--mmproj-device", "Vulkan1"]
- server._apply_device_policy(args, plan())
- assert server._flag_value(args, "--device") == "Vulkan0"
- assert server._flag_value(args, "--mmproj-device") == "Vulkan0"
- assert "Vulkan1" not in args
- assert args.count("--device") == 1 and args.count("--mmproj-device") == 1
- with pytest.raises(ValueError, match="GPU group"):
- server._apply_device_policy(["--mmproj-device", "Vulkan2"], plan())
+def test_gpu_policy_does_not_inject_device_names() -> None:
+ args = ["--model", "model.gguf"]
+ original = list(args)
+ server._apply_device_policy(args, LlamaSelection("gpu", reason="onnx_configured_gpu"))
+ assert args == original
@pytest.mark.parametrize("fails", [False, True])
-def test_exec_scopes_workaround_and_restores_on_return(monkeypatch: pytest.MonkeyPatch, tmp_path: Path, fails: bool) -> None:
+def test_exec_scopes_compatibility_and_preserves_user_filters(
+ monkeypatch: pytest.MonkeyPatch, tmp_path: Path, fails: bool
+) -> None:
binary = tmp_path / "llama-server"
binary.touch()
monkeypatch.setattr(server, "llama_server_binary", lambda: binary)
- monkeypatch.delenv(SERIALIZE_ENV, raising=False)
- monkeypatch.delenv("GGML_VK_VISIBLE_DEVICES", raising=False)
+ monkeypatch.setenv("GGML_VK_VISIBLE_DEVICES", "1")
+ monkeypatch.setenv("LLAMA_ARG_DEVICE", "Vulkan1")
+ monkeypatch.setenv("MTMD_BACKEND_DEVICE", "Vulkan2")
def execute(path: str, args: list[str]) -> None:
- assert os.environ[SERIALIZE_ENV] == "1" and os.environ["GGML_VK_VISIBLE_DEVICES"] == "1"
+ assert os.environ[SERIALIZE_ENV] == "1"
+ assert os.environ["GGML_VK_VISIBLE_DEVICES"] == "1"
+ assert os.environ["LLAMA_ARG_DEVICE"] == "Vulkan1"
+ assert os.environ["MTMD_BACKEND_DEVICE"] == "Vulkan2"
if fails:
raise OSError("exec failed")
monkeypatch.setattr(server.os, "execv", execute)
+ selection = LlamaSelection("gpu", environment=((SERIALIZE_ENV, "1"),))
if fails:
with pytest.raises(OSError, match="exec failed"):
- server._exec_server([], plan())
+ server._exec_server([], selection)
else:
- server._exec_server([], plan())
- assert SERIALIZE_ENV not in os.environ and "GGML_VK_VISIBLE_DEVICES" not in os.environ
-
+ server._exec_server([], selection)
+ assert SERIALIZE_ENV not in os.environ
+ assert os.environ["GGML_VK_VISIBLE_DEVICES"] == "1"
-def test_exec_remaps_environment_before_native_parser_validates_it(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
- binary = tmp_path / "llama-server"
- binary.touch()
- monkeypatch.setattr(server, "llama_server_binary", lambda: binary)
- monkeypatch.setenv("LLAMA_ARG_DEVICE", "Vulkan1")
- monkeypatch.setenv("MTMD_BACKEND_DEVICE", "Vulkan1")
- args = []
- server._apply_device_policy(args, plan())
- def execute(path: str, argv: list[str]) -> None:
- assert os.environ["LLAMA_ARG_DEVICE"] == "Vulkan0"
- assert os.environ["MTMD_BACKEND_DEVICE"] == "Vulkan0"
- assert "Vulkan1" not in argv
-
- monkeypatch.setattr(server.os, "execv", execute)
- server._exec_server(args, plan())
- assert os.environ["LLAMA_ARG_DEVICE"] == "Vulkan1"
- assert os.environ["MTMD_BACKEND_DEVICE"] == "Vulkan1"
-
-
- "platform,chip,serialize",
- [("linux", 0x7D51, True), ("linux", 0xB640, True), ("linux", 0x56A0, False), ("win32", 0x7D51, False)],
-)
[email protected]("projector_only", [False, True])
-def test_actual_device_policy_reaches_server_environment(
- monkeypatch: pytest.MonkeyPatch, tmp_path: Path, platform: str, chip: int, serialize: bool, projector_only: bool
[email protected]("platform", ["linux", "win32"])
[email protected]("explicit", [False, True])
+def test_only_automatic_onnx_route_adds_intel_compatibility(
+ monkeypatch: pytest.MonkeyPatch, platform: str, explicit: bool
) -> None:
- """复用真实设备策略,验证模型或仅视觉编码选中 Xe-LPG 时的启动环境。"""
- for name in (
- *policy._BACKEND_ENV,
- "MINERU_LLAMA_CPP_BACKEND",
- "LLAMA_ARG_DEVICE",
- "LLAMA_ARG_N_GPU_LAYERS",
- "MTMD_BACKEND_DEVICE",
- "LLAMA_ARG_MMPROJ_OFFLOAD",
- ):
- monkeypatch.delenv(name, raising=False)
- monkeypatch.setattr(policy, "sys", SimpleNamespace(platform=platform))
- device = LlamaDevice(
- "Vulkan0", "Intel", "vulkan", 0x8086, chip, 0, kind="integrated" if chip != 0x56A0 else "discrete", kind_source="vulkan"
- )
- monkeypatch.setattr(policy, "discover_llama_devices", Mock(return_value=(device,)))
- args = ["-ngl", "0", "--mmproj-device", "Vulkan0", "--mmproj-offload"] if projector_only else ["--device", "Vulkan0"]
+ monkeypatch.setattr(policy.sys, "platform", platform)
+ select = Mock(return_value=OnnxSelection("gpu", WebGPUDevice(0x8086, 0x7D51), "configured_gpu", "webgpu"))
+ monkeypatch.setattr(onnx_device, "resolve_onnx_device", select)
+ args = ["--device", "Vulkan1"] if explicit else []
selected = server._device_policy(args)
- assert selected.cpu_model is projector_only
- assert dict(selected.environment).get(SERIALIZE_ENV) == ("1" if serialize else None)
+ assert dict(selected.environment).get(SERIALIZE_ENV) == ("1" if platform == "linux" and not explicit else None)
server._apply_device_policy(args, selected)
-
- binary = tmp_path / "llama-server"
- binary.touch()
- monkeypatch.setattr(server, "llama_server_binary", lambda: binary)
-
- def execute(path: str, argv: list[str]) -> None:
- assert os.environ.get(SERIALIZE_ENV) == ("1" if serialize else None)
- assert os.environ["GGML_VK_VISIBLE_DEVICES"] == "0"
- if projector_only:
- assert server._flag_value(argv, "--n-gpu-layers") == "0"
- assert server._flag_value(argv, "--mmproj-device") == "Vulkan0"
-
- monkeypatch.setattr(server.os, "execv", execute)
- server._exec_server(args, selected)
- assert SERIALIZE_ENV not in os.environ and "GGML_VK_VISIBLE_DEVICES" not in os.environ
+ assert args == (["--device", "Vulkan1"] if explicit else [])
+ assert select.call_count == int(not explicit)
+
+
[email protected]("decision", ["cpu", "gpu"])
+def test_server_main_applies_automatic_mode_before_gpu_defaults(monkeypatch: pytest.MonkeyPatch, decision: str) -> None:
+ monkeypatch.setattr(onnx_device, "resolve_onnx_device", lambda: OnnxSelection(decision, reason="configured_" + decision))
+ monkeypatch.setattr(server.sys, "argv", ["llama-server", "-m", "model.gguf", "--mmproj", "mmproj.gguf"])
+ monkeypatch.setattr(server, "_read_gguf_context_length", lambda path: 0)
+ execute = Mock()
+ monkeypatch.setattr(server, "_exec_server", execute)
+ server.main()
+ args, selection = execute.call_args.args
+ assert selection.device == decision
+ assert server._flag_value(args, "--n-gpu-layers") == ("0" if decision == "cpu" else "99")
+ assert ("--no-mmproj-offload" in args) is (decision == "cpu")
+ assert ("--no-op-offload" in args) is (decision == "cpu")
diff --git a/tests/unittest/test_tier_recommendation.py b/tests/unittest/test_tier_recommendation.py
index 68a66cbd..40d23325 100644
--- a/tests/unittest/test_tier_recommendation.py
+++ b/tests/unittest/test_tier_recommendation.py
@@ -17,7 +17,6 @@ from mineru.model.runtime import onnx_calibration_cache as cache
from mineru.model.runtime import tier_recommendation as recommendation
from mineru.model.runtime.gpu import GpuIdentity, classify_d3d12_gpu, classify_vulkan_gpu, with_gpu_classification
from mineru.model.runtime.gpu_memory import GpuMemory
-from mineru.model.runtime.llama_probe import LlamaDevice
from mineru.model.runtime.webgpu import WebGPUDevice
from mineru.model.runtime.windows_gpu import GpuAdapter
from mineru.types import TIERS
@@ -26,6 +25,19 @@ from mineru.types import TIERS
@pytest.fixture(autouse=True)
def onnx_recommendation_backend(monkeypatch: pytest.MonkeyPatch) -> None:
"""本文件验证 ONNX 既有推荐规则,隔离宿主可能自动选择的 Torch 后端。"""
+ from mineru.model.runtime import llama_probe
+ import subprocess
+
+ forbidden = Mock(side_effect=AssertionError("recommendation must not probe llama devices"))
+ monkeypatch.setattr(llama_probe, "probe_devices", forbidden)
+ monkeypatch.setattr(llama_probe, "_probe_library", forbidden)
+ run = subprocess.run
+
+ def guarded_run(args: list[str], **kwargs: Any) -> Any:
+ assert "mineru.model.runtime.llama_probe" not in args
+ return run(args, **kwargs)
+
+ monkeypatch.setattr(subprocess, "run", guarded_run)
monkeypatch.setattr(recommendation, "resolve_small_model_backend", lambda: "onnx")
from mineru.config import get_config
@@ -165,7 +177,7 @@ def test_unwritable_record_does_not_break_calibration(evidence_path: Path, monke
@pytest.mark.parametrize(("platform", "metal"), [("darwin", True), ("darwin", False), ("other", False)])
def test_apple_metal_recommendation(monkeypatch: pytest.MonkeyPatch, platform: str, metal: bool) -> None:
"""Apple Silicon 按 Metal 可用性推荐,不要求 Windows/Linux 校准记录。"""
- from mineru.model.runtime import llama_device, onnx_device
+ from mineru.model.runtime import onnx_device
monkeypatch.delenv("MINERU_LLAMA_CPP_BACKEND", raising=False)
monkeypatch.setattr(recommendation, "sys", SimpleNamespace(platform=platform))
@@ -174,7 +186,6 @@ def test_apple_metal_recommendation(monkeypatch: pytest.MonkeyPatch, platform: s
monkeypatch.setattr(
recommendation, "torch_gpu_memory", lambda device: GpuMemory(device, GpuIdentity(), 8 * 1024**3, "budget", "test")
)
- monkeypatch.setattr(llama_device, "discover_llama_devices", lambda: ())
selection = Mock(side_effect=AssertionError("unsupported platform must not select ONNX device"))
monkeypatch.setattr(onnx_device, "resolve_onnx_device", selection)
assert frozenset(recommendation.recommend_tiers().pdf_image_tiers) == frozenset(TIERS if metal else TIERS[:2])
@@ -182,41 +193,25 @@ def test_apple_metal_recommendation(monkeypatch: pytest.MonkeyPatch, platform: s
@pytest.mark.parametrize("platform", ["win32", "linux"])
[email protected]("kind", ["discrete", "calibrated", "slow", "unknown", "mismatch", "hidden", "failure"])
-def test_gpu_discovery_uses_only_existing_evidence(
[email protected]("kind", ["discrete", "calibrated", "slow", "unknown", "hidden", "failure"])
+def test_gpu_discovery_uses_only_existing_onnx_evidence(
evidence_path: Path, monkeypatch: pytest.MonkeyPatch, platform: str, kind: str
) -> None:
- """硬件初选只读取真实证据,校准由达到门槛后的启动编排触发。"""
- from mineru.model.runtime import llama_device, onnx_device, webgpu, windows_gpu
+ """初选仅查询 ONNX 候选和旧证据,不提前校准或依赖 llama 库存。"""
+ from mineru.model.runtime import onnx_device, webgpu, windows_gpu
- monkeypatch.delenv("MINERU_LLAMA_CPP_BACKEND", raising=False)
monkeypatch.setattr(recommendation, "sys", SimpleNamespace(platform=platform))
monkeypatch.setattr(recommendation, "effective_cpu_count", lambda: 4)
monkeypatch.setattr(onnx_device, "resolve_onnx_device", Mock(side_effect=AssertionError("must not calibrate")))
- descriptor = LlamaDevice(
- "Vulkan0",
- "test",
- "vulkan",
- 0x8086,
- 0x7D51,
- kind="integrated" if kind != "discrete" else "discrete",
- kind_source="vulkan",
- )
- if kind == "mismatch":
- descriptor = replace(descriptor, device_id=1)
- inventory = () if kind == "hidden" else (descriptor,)
- monkeypatch.setattr(
- llama_device,
- "discover_llama_devices",
- Mock(
- return_value=inventory,
- side_effect=RuntimeError("driver") if kind == "failure" else None,
- ),
- )
adapter = _adapter() if platform == "win32" else WebGPUDevice(0x8086, 0x7D51, vulkan_device=(0x8086, 0x7D51))
- monkeypatch.setattr(windows_gpu, "discover_gpus", lambda: (adapter,))
- monkeypatch.setattr(webgpu, "discover_webgpu_devices", lambda: (adapter,))
- monkeypatch.setattr(webgpu, "select_webgpu_device", lambda devices: devices[0])
+ if kind == "discrete":
+ adapter = replace(adapter, kind="discrete")
+ discover = Mock(
+ return_value=() if kind == "hidden" else (adapter,), side_effect=RuntimeError("driver") if kind == "failure" else None
+ )
+ monkeypatch.setattr(windows_gpu, "discover_gpus", discover)
+ monkeypatch.setattr(webgpu, "discover_webgpu_devices", discover)
+ monkeypatch.setattr(webgpu, "select_webgpu_device", lambda devices: devices[0] if devices else None)
if kind != "unknown":
cache.save_calibration_record(adapter, _measurement(gpu_elapsed=3.0 if kind == "slow" else 1.0))
before = evidence_path.read_bytes() if evidence_path.exists() else None
@@ -228,7 +223,9 @@ def test_gpu_discovery_uses_only_existing_evidence(
def test_explicit_cpu_and_discovery_failure(monkeypatch: pytest.MonkeyPatch) -> None:
"""显式 CPU 禁止 GPU 推荐,CPU 探测失败时只推荐基础档位。"""
- monkeypatch.setenv("MINERU_LLAMA_CPP_BACKEND", "cpu")
+ from mineru.config import get_config
+
+ monkeypatch.setattr(get_config().model, "onnx_device", "cpu")
monkeypatch.setattr(recommendation, "effective_cpu_count", Mock(side_effect=OSError("affinity")))
assert recommendation.recommend_tiers().pdf_image_tiers == ("flash",)
@@ -241,7 +238,7 @@ def startup_calibration(
import onnxruntime
from mineru.model.download import ModelPath
- from mineru.model.runtime import llama_device, onnx_calibration, onnx_device, vulkan, webgpu, windows_gpu
+ from mineru.model.runtime import onnx_calibration, onnx_device, vulkan, webgpu, windows_gpu
settings = importlib.import_module("mineru.config")
config_file = evidence_path.with_name("settings.yaml")
@@ -264,11 +261,6 @@ def startup_calibration(
monkeypatch.setattr(module, "discover_gpus", lambda **kwargs: (_adapter(),))
for module in (onnx_device, webgpu):
monkeypatch.setattr(module, "discover_webgpu_devices", lambda: (WebGPUDevice(0x8086, 0x7D51),))
- monkeypatch.setattr(
- llama_device,
- "discover_llama_devices",
- lambda: (LlamaDevice("Vulkan0", "test", "vulkan", 0x8086, 0x7D51, kind="integrated", kind_source="vulkan"),),
- )
# 只替换权重准备,避免服务惰性导入的模块永久持有伪造的仓库对象。
monkeypatch.setattr(ModelPath, "ensure", Mock(return_value=Path("rec.onnx")))
measurement = Mock(return_value=_measurement())
@@ -359,14 +351,9 @@ def test_low_cpu_advanced_recommendation_still_selects_device(
startup_calibration: tuple[Path, Mock], monkeypatch: pytest.MonkeyPatch
) -> None:
"""门槛按初选档位判断,低 CPU 配额的独显设备仍按平台规则提前选型。"""
- from mineru.model.runtime import llama_device, onnx_device, vulkan
+ from mineru.model.runtime import onnx_device, vulkan
monkeypatch.setattr(recommendation, "effective_cpu_count", lambda: 2)
- monkeypatch.setattr(
- llama_device,
- "discover_llama_devices",
- lambda: (LlamaDevice("Vulkan0", "test", "vulkan", 0x8086, 0x7D51, kind="discrete", kind_source="vulkan"),),
- )
monkeypatch.setattr(
onnx_device, "discover_gpus", lambda **kwargs: (with_gpu_classification(_adapter(), classify_d3d12_gpu(False)),)
)
@@ -522,19 +509,15 @@ def test_configured_gpu_does_not_require_calibration_threshold(
configured_gpu[1].assert_not_called()
[email protected]("inventory", ["hidden", "mismatch", "ambiguous"])
-def test_configured_gpu_still_requires_visible_matching_device(
- configured_gpu: tuple[Path, Mock], monkeypatch: pytest.MonkeyPatch, inventory: str
[email protected]("llama_backend", ["auto", "cpu", "vulkan", "metal"])
+def test_configured_gpu_recommendation_is_independent_of_llama(
+ configured_gpu: tuple[Path, Mock], monkeypatch: pytest.MonkeyPatch, llama_backend: str
) -> None:
- """配置 GPU 不能借用不可见、另一张卡或身份不唯一的显存来开放高级档位。"""
- from mineru.model.runtime import llama_device
-
- device = LlamaDevice("Vulkan0", "test", "vulkan", 0x8086, 1 if inventory == "mismatch" else 0x7D51)
- devices = () if inventory == "hidden" else (device, device) if inventory == "ambiguous" else (device,)
- monkeypatch.setattr(llama_device, "discover_llama_devices", lambda: devices)
+ """VLM 后端设置、缺失或有歧义的 ggml 库存不参与 ONNX 档位推荐。"""
+ monkeypatch.setenv("MINERU_LLAMA_CPP_BACKEND", llama_backend)
report = recommendation.recommend_tiers()
- assert report.device.endswith(":gpu") and report.pdf_image_tiers == TIERS[:2]
- assert report.reason == ("no_visible_gpu" if inventory == "hidden" else "gpu_identity_unconfirmed")
+ assert report.device.endswith(":gpu") and report.pdf_image_tiers == TIERS
+ assert report.reason == "configured_gpu"
configured_gpu[1].assert_not_called()
diff --git a/tests/unittest/test_webgpu_architecture.py b/tests/unittest/test_webgpu_architecture.py
index 3ed5349c..68cec8da 100644
--- a/tests/unittest/test_webgpu_architecture.py
+++ b/tests/unittest/test_webgpu_architecture.py
@@ -368,8 +368,10 @@ def test_bound_descriptor_roundtrips_through_worker_json(monkeypatch: pytest.Mon
@pytest.mark.parametrize("error", [False, True])
@pytest.mark.parametrize("native_type", [0, 1, 2, 3])
[email protected]("platform", ["linux", "win32"])
[email protected]("luid_valid", [False, True])
def test_vulkan_query_reads_pci_identity_without_creating_logical_device_and_cleans_up(
- monkeypatch: pytest.MonkeyPatch, error: bool, native_type: int
+ monkeypatch: pytest.MonkeyPatch, error: bool, native_type: int, platform: str, luid_valid: bool
) -> None:
"""原生类型在适配边界统一转换,未知硬件仍保留,而软件 CPU 不进入 GPU 库存。"""
@@ -391,19 +393,35 @@ def test_vulkan_query_reads_pci_identity_without_creating_logical_device_and_cle
(0x10DE, 0x2489, native_type, b"NVIDIA") if handle == 100 else (0x10005, 0, 4, b"llvmpipe")
)
+ def properties2(handle: object, storage: object) -> None:
+ assert handle == 100
+ assert storage._obj.sType == 1000059001
+ identity = ct.cast(storage._obj.pNext, ct.POINTER(vulkan._DeviceIDProperties)).contents
+ assert identity.sType == 1000071004
+ identity.deviceLUID[:] = bytes.fromhex("67452301efcdab89")
+ identity.deviceLUIDValid = int(luid_valid)
+
lib = SimpleNamespace(
vkCreateInstance=Mock(side_effect=create),
vkDestroyInstance=Mock(),
vkEnumeratePhysicalDevices=Mock(side_effect=enumerate_devices),
vkGetPhysicalDeviceProperties=Mock(side_effect=properties),
+ vkGetPhysicalDeviceProperties2=Mock(side_effect=properties2),
)
- monkeypatch.setattr(vulkan.ct, "CDLL", lambda name: lib)
+ monkeypatch.setattr(vulkan, "sys", SimpleNamespace(platform=platform))
+ monkeypatch.setattr(vulkan.ct, "WinDLL" if platform == "win32" else "CDLL", lambda name: lib, raising=False)
if error:
with pytest.raises(RuntimeError, match="count devices"):
vulkan.enumerate_vulkan_devices()
else:
- assert vulkan.enumerate_vulkan_devices() == (
- with_gpu_classification(vulkan.VulkanDevice(0x10DE, 0x2489, "NVIDIA", 0), classify_vulkan_gpu(native_type)),
+ luid = "89abcdef:01234567" if platform == "win32" and luid_valid else None
+ result = vulkan.enumerate_vulkan_devices()
+ assert result == (
+ with_gpu_classification(
+ vulkan.VulkanDevice(0x10DE, 0x2489, "NVIDIA", 0, luid=luid), classify_vulkan_gpu(native_type)
+ ),
)
+ assert result[0].identity.luid == luid
+ assert lib.vkGetPhysicalDeviceProperties2.call_count == int(platform == "win32" and not error)
lib.vkDestroyInstance.assert_called_once()
assert ct.sizeof(vulkan._PropertiesStorage) >= 824