Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Python ai4sciencedocument-analysisdocxextract-datalayout-analysisocrparserpdfpdf-converterpdf-extractor-llmpdf-extractor-pretrainpdf-extractor-ragpdf-parserpptxpythonxlsx
feat(onnx): unify thread configuration for ONNX sessions with environment variable fallback
myhloli2bbaf6efa8f7
diff --git a/docs/en/usage/cli_tools.md b/docs/en/usage/cli_tools.mdindex 155f5413..16c37257 100644--- a/docs/en/usage/cli_tools.md+++ b/docs/en/usage/cli_tools.md@@ -94,7 +94,7 @@ mineru-kit vlm-server --engine auto --port 30000- `MINERU_PDF_RENDER_TIMEOUT` / `MINERU_PDF_RENDER_THREADS`: PDF rendering timeout and worker count.- `MINERU_PROCESSING_WINDOW_SIZE`: processing window size used for large documents. Flash native text parsing renders only pages with visual blocks, preserving this window boundary, rendering timeout, and worker settings. A 32 MiB pixel budget can split batches further. OCR and other tiers still render entire windows.- `MINERU_MALLOC_TRIM`: disabled by default; only `1/true/yes/on` enable it, ignoring case and surrounding whitespace. On Linux, when available, it attempts to return the current process's free glibc heap pages after inference windows and PDF document cleanup. Flash native parsing only attempts trimming at document completion. Unsupported platforms or missing symbols are a no-op. It does not release live objects, GPU memory, or render-worker memory. See [memory benchmarking](./pdf_memory.md).-- `MINERU_INTRA_OP_NUM_THREADS` / `MINERU_INTER_OP_NUM_THREADS`: ONNX operator thread settings.+- `MINERU_INTRA_OP_NUM_THREADS` / `MINERU_INTER_OP_NUM_THREADS`: Per-session ONNX intra-op / inter-op thread counts, defaulting to `4` / `1`, including CPU fallback from table CUDA sessions. Explicit positive arguments or session thread settings take precedence over environment variables; missing, non-integer, or non-positive environment values use the defaults. These settings do not cap the process's total thread count or automatically detect container CPU quotas.Prefer each command's `--help` output and [model source documentation](./model_source.md) for current defaults.diff --git a/docs/zh/usage/cli_tools.md b/docs/zh/usage/cli_tools.mdindex 88868adf..e9d2de75 100644--- a/docs/zh/usage/cli_tools.md+++ b/docs/zh/usage/cli_tools.md@@ -94,7 +94,7 @@ mineru-kit vlm-server --engine auto --port 30000- `MINERU_PDF_RENDER_TIMEOUT` / `MINERU_PDF_RENDER_THREADS`:PDF 渲染超时和 worker 数量。- `MINERU_PROCESSING_WINDOW_SIZE`:大文档处理窗口大小。Flash 原生文本路径仅渲染有视觉块的页面,仍遵守该窗口边界、渲染超时和 worker 配置,并按 32 MiB 像素预算进一步分批。OCR 和其他档位保持整窗口渲染。- `MINERU_MALLOC_TRIM`:默认关闭;仅 `1/true/yes/on` 启用,忽略大小写和首尾空白。Linux 上可用时,在推理窗口及 PDF 文档清理后尝试归还当前进程的 glibc 空闲堆页;Flash 原生路径仅在文档结束时尝试。其他平台或缺少符号时为空操作,不释放存活对象、GPU 显存或渲染子进程内存。参见[内存压测方法](./pdf_memory.md)。-- `MINERU_INTRA_OP_NUM_THREADS` / `MINERU_INTER_OP_NUM_THREADS`:ONNX 算子线程配置。+- `MINERU_INTRA_OP_NUM_THREADS` / `MINERU_INTER_OP_NUM_THREADS`:每个 ONNX 会话的算子内 / 算子间线程数,默认分别为 `4` / `1`,包括表格 CUDA 的 CPU 回退路径。显式正数参数或会话线程配置优先于环境变量;环境变量缺失、非整数或非正数时使用默认值。这不是整个进程的线程总数上限,也不会自动检测容器 CPU 配额。当前默认值以各命令的 `--help` 和[模型源说明](./model_source.md)为准。diff --git a/mineru/model/runtime/onnx.py b/mineru/model/runtime/onnx.pyindex 34d5c12e..c0d54e1b 100644--- a/mineru/model/runtime/onnx.py+++ b/mineru/model/runtime/onnx.py@@ -21,22 +21,34 @@ def get_op_num_threads(env_name: str) -> int:return value if value > 0 else -1+def configure_ort_threads(+ opts: ort.SessionOptions,+ *,+ intra_op_num_threads: int = 0,+ inter_op_num_threads: int = 0,+) -> None:+ """统一设置线程数:显式正数及已有会话配置优先,其次环境变量,最后回退为 4/1。"""+ for attr, configured, env_name, default in (+ ("intra_op_num_threads", intra_op_num_threads, "MINERU_INTRA_OP_NUM_THREADS", 4),+ ("inter_op_num_threads", inter_op_num_threads, "MINERU_INTER_OP_NUM_THREADS", 1),+ ):+ threads = configured if configured > 0 else getattr(opts, attr)+ if threads <= 0:+ threads = get_op_num_threads(env_name)+ setattr(opts, attr, threads if threads > 0 else default)++def ort_providers(device: str | None = None) -> list[tuple[str, dict[str, object]]]:"""通用 ONNX 模型固定使用 CPU;表格工厂独立选择 CUDA provider。"""return [("CPUExecutionProvider", {"arena_extend_strategy": "kSameAsRequested"})]def ort_session(model_path: str, device: str | None = None, intra_op_num_threads: int = 0) -> ort.InferenceSession:- """创建 CPU 会话;显式线程数优先,否则使用统一环境配置。"""+ """创建 CPU 会话;显式线程数优先,其次环境配置,默认使用 4/1 线程。"""opts = ort.SessionOptions()opts.log_severity_level = 3opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL- intra_threads = intra_op_num_threads or get_op_num_threads("MINERU_INTRA_OP_NUM_THREADS")- inter_threads = get_op_num_threads("MINERU_INTER_OP_NUM_THREADS")- if intra_threads > 0:- opts.intra_op_num_threads = intra_threads- if inter_threads > 0:- opts.inter_op_num_threads = inter_threads+ configure_ort_threads(opts, intra_op_num_threads=intra_op_num_threads)return ort.InferenceSession(model_path, sess_options=opts, providers=ort_providers(device))@@ -48,13 +60,7 @@ def table_ort_session(model_path: str, *, sess_options: ort.SessionOptions | Nonopts = sess_options if sess_options is not None else ort.SessionOptions()opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALLopts.log_severity_level = 3- for attr, env_name in (- ("intra_op_num_threads", "MINERU_INTRA_OP_NUM_THREADS"),- ("inter_op_num_threads", "MINERU_INTER_OP_NUM_THREADS"),- ):- threads = get_op_num_threads(env_name)- if threads > 0:- setattr(opts, attr, threads)+ configure_ort_threads(opts)profile_dir = os.getenv("MINERU_ONNX_PROFILE_DIR")if profile_dir:Path(profile_dir).mkdir(parents=True, exist_ok=True)@@ -101,4 +107,4 @@ def table_ort_session(model_path: str, *, sess_options: ort.SessionOptions | Nonreturn session-__all__ = ["get_op_num_threads", "ort_providers", "ort_session", "table_ort_session"]+__all__ = ["configure_ort_threads", "get_op_num_threads", "ort_providers", "ort_session", "table_ort_session"]diff --git a/mineru/model/table/rec/slanet_plus/table_structure.py b/mineru/model/table/rec/slanet_plus/table_structure.pyindex cfd8a20a..c36c61db 100644--- a/mineru/model/table/rec/slanet_plus/table_structure.py+++ b/mineru/model/table/rec/slanet_plus/table_structure.py@@ -16,7 +16,6 @@ from typing import Any, Dict, List, Tupleimport numpy as np-from ....runtime.onnx import get_op_num_threadsfrom .table_structure_utils import (OrtInferSession,TableLabelDecode,@@ -30,9 +29,6 @@ class TableStructurer:self.preprocess_op = TablePreprocess()self.batch_preprocess_op = BatchTablePreprocess()- config["intra_op_num_threads"] = get_op_num_threads("MINERU_INTRA_OP_NUM_THREADS")- config["inter_op_num_threads"] = get_op_num_threads("MINERU_INTER_OP_NUM_THREADS")-self.session = OrtInferSession(config)self.character = self.session.get_metadata()diff --git a/mineru/model/table/rec/slanet_plus/table_structure_utils.py b/mineru/model/table/rec/slanet_plus/table_structure_utils.pyindex 404956a8..883cff3d 100644--- a/mineru/model/table/rec/slanet_plus/table_structure_utils.py+++ b/mineru/model/table/rec/slanet_plus/table_structure_utils.py@@ -11,7 +11,6 @@# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.-import osimport tracebackfrom pathlib import Pathfrom typing import Any, Dict, List, Tuple, Union@@ -21,7 +20,7 @@ import numpy as npfrom loguru import loggerfrom onnxruntime import GraphOptimizationLevel, SessionOptions-from ....runtime.onnx import table_ort_session+from ....runtime.onnx import configure_ort_threads, table_ort_sessionclass OrtInferSession:@@ -39,19 +38,17 @@ class OrtInferSession:@staticmethoddef _init_sess_opts(config: Dict[str, Any]) -> SessionOptions:+ """保留表格内存选项,线程数交给共享策略处理。"""sess_opt = SessionOptions()sess_opt.log_severity_level = 4sess_opt.enable_cpu_mem_arena = Falsesess_opt.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL- cpu_nums = os.cpu_count()- intra_op_num_threads = config.get("intra_op_num_threads", -1)- if intra_op_num_threads != -1 and 1 <= intra_op_num_threads <= cpu_nums:- sess_opt.intra_op_num_threads = intra_op_num_threads-- inter_op_num_threads = config.get("inter_op_num_threads", -1)- if inter_op_num_threads != -1 and 1 <= inter_op_num_threads <= cpu_nums:- sess_opt.inter_op_num_threads = inter_op_num_threads+ configure_ort_threads(+ sess_opt,+ intra_op_num_threads=config.get("intra_op_num_threads", 0),+ inter_op_num_threads=config.get("inter_op_num_threads", 0),+ )return sess_optdiff --git a/mineru/model/table/rec/unet_table/table_structure_unet.py b/mineru/model/table/rec/unet_table/table_structure_unet.pyindex baf3b585..14f0f6fb 100644--- a/mineru/model/table/rec/unet_table/table_structure_unet.py+++ b/mineru/model/table/rec/unet_table/table_structure_unet.py@@ -6,7 +6,6 @@ from typing import Optional, Dict, Any, Tupleimport cv2import numpy as np-from ....runtime.onnx import get_op_num_threadsfrom .utils import OrtInferSession, resize_imgfrom .utils_table_line_rec import (get_table_line,@@ -31,9 +30,6 @@ class TSRUnet:self.inp_height = 1024self.inp_width = 1024- config["intra_op_num_threads"] = get_op_num_threads("MINERU_INTRA_OP_NUM_THREADS")- config["inter_op_num_threads"] = get_op_num_threads("MINERU_INTER_OP_NUM_THREADS")-self.session = OrtInferSession(config)def __call__(diff --git a/mineru/model/table/rec/unet_table/utils.py b/mineru/model/table/rec/unet_table/utils.pyindex 61e25978..6a01aa0e 100644--- a/mineru/model/table/rec/unet_table/utils.py+++ b/mineru/model/table/rec/unet_table/utils.py@@ -11,7 +11,7 @@ import numpy as npfrom onnxruntime import GraphOptimizationLevel, SessionOptionsfrom PIL import Image, UnidentifiedImageError-from ....runtime.onnx import table_ort_session+from ....runtime.onnx import configure_ort_threads, table_ort_sessionroot_dir = Path(__file__).resolve().parentInputType = Union[str, np.ndarray, bytes, Path]@@ -30,19 +30,17 @@ class OrtInferSession:@staticmethoddef _init_sess_opts(config: Dict[str, Any]) -> SessionOptions:+ """保留表格内存选项,线程数交给共享策略处理。"""sess_opt = SessionOptions()sess_opt.log_severity_level = 4sess_opt.enable_cpu_mem_arena = Falsesess_opt.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL- cpu_nums = os.cpu_count()- intra_op_num_threads = config.get("intra_op_num_threads", -1)- if intra_op_num_threads != -1 and 1 <= intra_op_num_threads <= cpu_nums:- sess_opt.intra_op_num_threads = intra_op_num_threads-- inter_op_num_threads = config.get("inter_op_num_threads", -1)- if inter_op_num_threads != -1 and 1 <= inter_op_num_threads <= cpu_nums:- sess_opt.inter_op_num_threads = inter_op_num_threads+ configure_ort_threads(+ sess_opt,+ intra_op_num_threads=config.get("intra_op_num_threads", 0),+ inter_op_num_threads=config.get("inter_op_num_threads", 0),+ )return sess_optdiff --git a/tests/unittest/test_table_onnx_runtime.py b/tests/unittest/test_table_onnx_runtime.pyindex bd811e03..e2b41e80 100644--- a/tests/unittest/test_table_onnx_runtime.py+++ b/tests/unittest/test_table_onnx_runtime.py@@ -9,6 +9,77 @@ import pytestfrom mineru.model.runtime import onnx[email protected](autouse=True)+def clear_thread_environment(monkeypatch: pytest.MonkeyPatch) -> None:+ """隔离开发机线程配置,确保默认值与覆盖行为可以独立验证。"""+ monkeypatch.delenv("MINERU_INTRA_OP_NUM_THREADS", raising=False)+ monkeypatch.delenv("MINERU_INTER_OP_NUM_THREADS", raising=False)++[email protected]("table", [False, True])+ ("intra", "inter", "expected"),+ [+ (None, None, (4, 1)),+ ("", "invalid", (4, 1)),+ ("0", "-1", (4, 1)),+ ("-2", "1.5", (4, 1)),+ ("8", "2", (8, 2)),+ ("2", None, (2, 1)),+ (None, "3", (4, 3)),+ ],+)+def test_session_thread_defaults_and_environment(+ monkeypatch: pytest.MonkeyPatch,+ table: bool,+ intra: str | None,+ inter: str | None,+ expected: tuple[int, int],+) -> None:+ """两个会话入口都必须显式设置线程,非法环境值不能退回 ORT 自动扩张。"""+ monkeypatch.setenv("MINERU_TABLE_DEVICE", "cpu")+ if intra is not None:+ monkeypatch.setenv("MINERU_INTRA_OP_NUM_THREADS", intra)+ if inter is not None:+ monkeypatch.setenv("MINERU_INTER_OP_NUM_THREADS", inter)+ factory = Mock(return_value=SimpleNamespace(get_providers=lambda: ["CPUExecutionProvider"]))+ monkeypatch.setattr(onnx.ort, "InferenceSession", factory)+ if table:+ onnx.table_ort_session("table.onnx")+ else:+ onnx.ort_session("model.onnx")+ opts = factory.call_args.kwargs["sess_options"]+ assert (opts.intra_op_num_threads, opts.inter_op_num_threads) == expected+ assert opts.execution_mode == onnx.ort.ExecutionMode.ORT_SEQUENTIAL++[email protected](("explicit", "expected"), [(2, 2), (0, 8), (-1, 8)])+def test_explicit_intra_threads_take_precedence(monkeypatch: pytest.MonkeyPatch, explicit: int, expected: int) -> None:+ """显式正数优先于环境变量,非正数参数沿用统一回退规则。"""+ monkeypatch.setenv("MINERU_INTRA_OP_NUM_THREADS", "8")+ factory = Mock()+ monkeypatch.setattr(onnx.ort, "InferenceSession", factory)+ onnx.ort_session("model.onnx", intra_op_num_threads=explicit)+ opts = factory.call_args.kwargs["sess_options"]+ assert (opts.intra_op_num_threads, opts.inter_op_num_threads) == (expected, 1)+++def test_explicit_table_session_options_are_preserved(monkeypatch: pytest.MonkeyPatch) -> None:+ """表格工厂保留显式线程和内存配置,仅补齐未指定的线程设置。"""+ monkeypatch.setenv("MINERU_TABLE_DEVICE", "cpu")+ monkeypatch.setenv("MINERU_INTRA_OP_NUM_THREADS", "8")+ monkeypatch.setenv("MINERU_INTER_OP_NUM_THREADS", "3")+ opts = onnx.ort.SessionOptions()+ opts.intra_op_num_threads = 2+ opts.enable_cpu_mem_arena = False+ factory = Mock(return_value=SimpleNamespace(get_providers=lambda: ["CPUExecutionProvider"]))+ monkeypatch.setattr(onnx.ort, "InferenceSession", factory)+ onnx.table_ort_session("table.onnx", sess_options=opts)+ assert factory.call_args.kwargs["sess_options"] is opts+ assert (opts.intra_op_num_threads, opts.inter_op_num_threads) == (2, 3)+ assert opts.enable_cpu_mem_arena is False++def test_table_default_uses_cpu_without_cuda_provider(monkeypatch: pytest.MonkeyPatch) -> None:"""CPU ORT 包保持 CPU 执行,Torch 设备不参与表格 provider 决策。"""monkeypatch.delenv("MINERU_TABLE_DEVICE", raising=False)@@ -28,8 +99,6 @@ def test_cuda_selection_and_other_onnx_models_stay_cpu(monkeypatch: pytest.Monkemonkeypatch.delenv("MINERU_TABLE_DEVICE", raising=False)else:monkeypatch.setenv("MINERU_TABLE_DEVICE", device)- monkeypatch.setenv("MINERU_INTRA_OP_NUM_THREADS", "4")- monkeypatch.setenv("MINERU_INTER_OP_NUM_THREADS", "1")monkeypatch.setattr(onnx.ort, "get_available_providers", lambda: ["CUDAExecutionProvider", "CPUExecutionProvider"])factory = Mock(return_value=SimpleNamespace(get_providers=lambda: ["CUDAExecutionProvider", "CPUExecutionProvider"]))monkeypatch.setattr(onnx.ort, "InferenceSession", factory)@@ -70,6 +139,9 @@ def test_cuda_failure_is_reported_and_cpu_remains_available(monkeypatch: pytest.monkeypatch.setattr(onnx.logger, "warning", warning)assert onnx.table_ort_session("table.onnx") is cpuassert warning.called == (mode != "unavailable" or device == "cuda")+ for call in factory.call_args_list:+ opts = call.kwargs["sess_options"]+ assert (opts.intra_op_num_threads, opts.inter_op_num_threads) == (4, 1)if mode in {"unavailable", "exception"}:assert factory.call_args.kwargs["providers"] == onnx.ort_providers()@@ -100,3 +172,36 @@ def test_all_table_wrappers_use_shared_factory(monkeypatch: pytest.MonkeyPatch,utils.OrtInferSession({"model_path": str(path)})assert factory.call_count == 3assert all(call.args == (str(path),) for call in factory.call_args_list)++[email protected]("explicit", [False, True])+def test_table_structure_entrypoints_use_shared_threads(+ monkeypatch: pytest.MonkeyPatch, tmp_path: Path, explicit: bool+) -> None:+ """真实结构模型入口保留配置字典和显式线程,两个包装器共用环境与默认策略。"""+ from mineru.model.table.rec.slanet_plus.table_structure import TableStructurer+ from mineru.model.table.rec.unet_table.table_structure_unet import TSRUnet++ monkeypatch.setenv("MINERU_TABLE_DEVICE", "cpu")+ monkeypatch.setenv("MINERU_INTRA_OP_NUM_THREADS", "8")+ monkeypatch.setenv("MINERU_INTER_OP_NUM_THREADS", "3")+ model = tmp_path / "table.onnx"+ model.touch()+ config = {"model_path": str(model)}+ if explicit:+ config.update(intra_op_num_threads=2, inter_op_num_threads=1)+ original = config.copy()+ session = SimpleNamespace(+ get_providers=lambda: ["CPUExecutionProvider"],+ get_modelmeta=lambda: SimpleNamespace(custom_metadata_map={"character": "<td>\n</td>"}),+ )+ factory = Mock(return_value=session)+ monkeypatch.setattr(onnx.ort, "InferenceSession", factory)+ TableStructurer(config)+ TSRUnet(config)+ assert config == original+ assert factory.call_count == 2+ for call in factory.call_args_list:+ opts = call.kwargs["sess_options"]+ assert (opts.intra_op_num_threads, opts.inter_op_num_threads) == ((2, 1) if explicit else (8, 3))+ assert opts.enable_cpu_mem_arena is False