Skip to content

Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

fix: update page range syntax in documentation and codebase for compatibility with new format

myhloli82965669bd49
diff --git a/README.md b/README.md
index e4a2ef0d..821ae002 100644
--- a/README.md
+++ b/README.md
@@ -498,7 +498,7 @@ Use JSON when you need structured status, tier, content, and continuation:
mineru parse "report.pdf" --json
```
-PDF page selection is shared across CLI, Doclib, API, Gradio and Python. See the [page-range contract and breaking upgrade instructions](docs/next/page-ranges.md). The old `~` and negative page-number syntax is no longer supported.
+PDF page selection is shared across CLI, Doclib, API, Gradio and Python. See the [page-range syntax and historical result compatibility](docs/next/page-ranges.md). New requests use the current syntax; stored positive page ranges using ASCII `~` remain readable without rebuilding Doclib caches. Fullwidth `~` and negative page-number notation are not supported.
For a specific page range (1-based, inclusive; `r1` is the last page, `all` selects every page):
diff --git a/docs/en/usage/cli_tools.md b/docs/en/usage/cli_tools.md
index 180cf215..7edf6467 100644
--- a/docs/en/usage/cli_tools.md
+++ b/docs/en/usage/cli_tools.md
@@ -80,6 +80,7 @@ Use `--pages "1-5,8,r3-r1"`: page numbers start at 1, ranges include both endpoi
and `r1` means the last page. Use `all` for every page. Results are sorted and deduplicated;
partially out-of-bounds ranges select their valid intersection. Reversed or empty selections
fail with `page_range_invalid`. Without `--pages`, `mineru parse` starts with the first 10 pages;
-`mineru-kit parse`, Python and Gradio select all pages. The old `~` and negative-number syntax
-is no longer accepted. Upgrade clients and servers together and rebuild old Doclib caches
-in a fresh data directory. See the [shared page-range contract](../../next/page-ranges.md).
+`mineru-kit parse`, Python and Gradio select all pages. New requests use the current syntax. Historical positive result ranges using ASCII `~`
+remain readable without rebuilding Doclib caches; result responses and new cache entries use `-`.
+Fullwidth `~` and negative page-number notation are not supported.
+See [page-range syntax and historical result compatibility](../../next/page-ranges.md).
diff --git a/docs/en/usage/quick_usage.md b/docs/en/usage/quick_usage.md
index 997ec2e4..13cc6c83 100644
--- a/docs/en/usage/quick_usage.md
+++ b/docs/en/usage/quick_usage.md
@@ -113,6 +113,7 @@ Use `--pages "1-5,8,r3-r1"`: page numbers start at 1, ranges include both endpoi
and `r1` means the last page. Use `all` for every page. Results are sorted and deduplicated;
partially out-of-bounds ranges select their valid intersection. Reversed or empty selections
fail with `page_range_invalid`. Without `--pages`, `mineru parse` starts with the first 10 pages;
-`mineru-kit parse`, Python and Gradio select all pages. The old `~` and negative-number syntax
-is no longer accepted. Upgrade clients and servers together and rebuild old Doclib caches
-in a fresh data directory. See the [shared page-range contract](../../next/page-ranges.md).
+`mineru-kit parse`, Python and Gradio select all pages. New requests use the current syntax. Historical positive result ranges using ASCII `~`
+remain readable without rebuilding Doclib caches; result responses and new cache entries use `-`.
+Fullwidth `~` and negative page-number notation are not supported.
+See [page-range syntax and historical result compatibility](../../next/page-ranges.md).
diff --git a/docs/next/README.md b/docs/next/README.md
index 83dd91b3..a8c136a7 100644
--- a/docs/next/README.md
+++ b/docs/next/README.md
@@ -1,6 +1,6 @@
# Next MinerU 文档体系
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](page-ranges.md)。
状态: Implemented
读者: 项目核心开发者、SDK/API 设计参与者、需要理解下一代能力边界的外部开发者
diff --git a/docs/next/api/parse-jobs.md b/docs/next/api/parse-jobs.md
index 2cc11a01..3ac7ef5b 100644
--- a/docs/next/api/parse-jobs.md
+++ b/docs/next/api/parse-jobs.md
@@ -1,6 +1,6 @@
# Parse Jobs
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Draft
读者: API 使用者、服务端开发者、SDK 开发者
diff --git a/docs/next/cli/mineru-kit-gradio.md b/docs/next/cli/mineru-kit-gradio.md
index 2777f25c..34f720b6 100644
--- a/docs/next/cli/mineru-kit-gradio.md
+++ b/docs/next/cli/mineru-kit-gradio.md
@@ -1,6 +1,6 @@
# mineru-kit gradio
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Implemented
diff --git a/docs/next/cli/mineru-kit-parse.md b/docs/next/cli/mineru-kit-parse.md
index bf59a78e..3401fd0a 100644
--- a/docs/next/cli/mineru-kit-parse.md
+++ b/docs/next/cli/mineru-kit-parse.md
@@ -1,6 +1,6 @@
# mineru-kit parse
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Draft
读者: 批处理开发者、解析内核开发者
diff --git a/docs/next/cli/mineru-parse.md b/docs/next/cli/mineru-parse.md
index 3e619f3a..a309aa53 100644
--- a/docs/next/cli/mineru-parse.md
+++ b/docs/next/cli/mineru-parse.md
@@ -1,6 +1,6 @@
# mineru parse
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Draft
读者: Agent skill 作者、CLI 使用者、核心开发者
diff --git a/docs/next/config.md b/docs/next/config.md
index 7261869b..c7f41672 100644
--- a/docs/next/config.md
+++ b/docs/next/config.md
@@ -1,6 +1,6 @@
# 配置体系
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](page-ranges.md)。
状态: Draft
读者: 想了解 doclib / mineru 配置的用户、实现配置能力的核心开发者
diff --git a/docs/next/page-naming.md b/docs/next/page-naming.md
index 907e2f57..14b5947c 100644
--- a/docs/next/page-naming.md
+++ b/docs/next/page-naming.md
@@ -1,6 +1,6 @@
# Page Naming Convention
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](page-ranges.md)。
状态: Implemented
读者: 核心开发者、编程 Agent、SDK/API/CLI 设计参与者、文档作者
diff --git a/docs/next/page-ranges.md b/docs/next/page-ranges.md
index 907a9fc2..96f48f81 100644
--- a/docs/next/page-ranges.md
+++ b/docs/next/page-ranges.md
@@ -84,15 +84,16 @@ parsing rules 和排队中的 V1 请求尚未取得总页数时允许保留 `rN`
表示空集合,不表示用户输入的“未指定”。总页数未知的缓存内容导出可使用绝对页码或全部缓存,
不能用已有缓存的最大页码推测 `r1`。
-## 不兼容升级
+## 历史结果兼容
-本次不支持旧输入 `1~5`、`-1`、`-5~-1`,也不提供旧页码缓存迁移或旧服务端回退。
-更新调用方、SDK、Doclib 和 Parse Server,重新创建使用旧语法的 parsing rules。
+旧 Doclib 中使用半角 `~` 的正整数结果范围(例如 `1~5,8`)可以直接读取,
+并可与新格式结果共存。内容读取、缓存命中、覆盖统计和 compaction 共用此兼容规则;
+无需因页码分隔符变化而重建数据库、重新解析或更换数据目录。
-已有 Doclib 数据可保留备份。升级后使用新的数据目录重新入库、解析:
+读取旧缓存文件时仍使用数据库记录中的原始范围和原文件名,例如 `1~5_时间戳.json`。
+对外返回的结果范围、新解析记录及正常 compaction 输出统一使用连字符,例如 `1-5,8`。
+读取和结果展示不会原地修改旧记录或重命名旧缓存文件。
-1. 停止旧 Doclib 实例,在独立配置文件中把 `doclib.data_dir` 配置为新的空目录。
-2. 按[配置说明](config.md)同步配置独立 DB/endpoint 路径,以新配置启动 Doclib;客户端使用同一配置。
-3. 重新添加 watch / parsing rules,重新扫描或执行 `mineru parse`,生成新格式记录与缓存。
-
-程序不会在此次升级中自动删除旧数据。不要把旧数据库或旧 parsed 目录复制到新实例继续使用。
+兼容仅适用于已求值的历史结果及元数据,不放宽新请求输入,也不扩展旧 parsing rules 或旧待处理任务的执行兼容。
+新请求仍使用 `1-5`、`r1`、`all`。全角 `~`、负号倒数页码 `-1` / `-5~-1` 不受支持;
+已求值结果中也不能包含未经展开的 `rN` 或 `all`。
diff --git a/docs/next/sdk/doclib-client.md b/docs/next/sdk/doclib-client.md
index 1ab00d8b..ab09e619 100644
--- a/docs/next/sdk/doclib-client.md
+++ b/docs/next/sdk/doclib-client.md
@@ -1,6 +1,6 @@
# Doclib SDK: `DoclibClient`
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Draft
读者: SDK 开发者、`mineru` CLI 开发者、MCP/桌面端集成方
diff --git a/docs/next/sdk/migration.md b/docs/next/sdk/migration.md
index e7c8b5f6..14076481 100644
--- a/docs/next/sdk/migration.md
+++ b/docs/next/sdk/migration.md
@@ -1,6 +1,6 @@
# 迁移路径
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Draft
读者: 核心开发者、SDK 开发者
diff --git a/docs/next/sdk/parser.md b/docs/next/sdk/parser.md
index cc927fd6..45a5f0ca 100644
--- a/docs/next/sdk/parser.md
+++ b/docs/next/sdk/parser.md
@@ -1,6 +1,6 @@
# Tool SDK: `mineru.parser`
-页码表达式、默认范围及不兼容升级要求统一见 [PDF 页码范围规范](../page-ranges.md)。
+页码规范与历史结果兼容说明见 [PDF 页码范围规范](../page-ranges.md)。
状态: Draft
读者: SDK 开发者、`mineru-kit` 开发者、核心开发者
diff --git a/docs/zh/usage/cli_tools.md b/docs/zh/usage/cli_tools.md
index 0ce6d910..9e141259 100644
--- a/docs/zh/usage/cli_tools.md
+++ b/docs/zh/usage/cli_tools.md
@@ -79,5 +79,6 @@ mineru-kit vlm-server --engine auto --port 30000
使用 `--pages "1-5,8,r3-r1"`:页码从 1 开始,包含区间两端,`r1` 表示最后一页,`all` 表示全部。
结果去重并按原页序排列;部分越界取有效交集,倒序或选不到页面时返回 `page_range_invalid`。
省略页码时 `mineru parse` 默认前 10 页,`mineru-kit parse`、Python 和 Gradio 默认全部。
-不再接受 `~` 和负数页码;升级调用方和服务端后,在新 Doclib 数据目录重建解析缓存。
-完整说明见 [PDF 页码范围规范](../../next/page-ranges.md)。
+新请求使用新规范;历史正整数半角 `~` 结果可直接读取,无需重建 Doclib 缓存。
+结果返回值和新缓存使用 `-`,全角 `~` 及负号倒数页码不受支持。
+完整说明见 [页码规范与历史结果兼容](../../next/page-ranges.md)。
diff --git a/docs/zh/usage/quick_usage.md b/docs/zh/usage/quick_usage.md
index 58def7b0..0f2a7972 100644
--- a/docs/zh/usage/quick_usage.md
+++ b/docs/zh/usage/quick_usage.md
@@ -112,5 +112,6 @@ model:
使用 `--pages "1-5,8,r3-r1"`:页码从 1 开始,包含区间两端,`r1` 表示最后一页,`all` 表示全部。
结果去重并按原页序排列;部分越界取有效交集,倒序或选不到页面时返回 `page_range_invalid`。
省略页码时 `mineru parse` 默认前 10 页,`mineru-kit parse`、Python 和 Gradio 默认全部。
-不再接受 `~` 和负数页码;升级调用方和服务端后,在新 Doclib 数据目录重建解析缓存。
-完整说明见 [PDF 页码范围规范](../../next/page-ranges.md)。
+新请求使用新规范;历史正整数半角 `~` 结果可直接读取,无需重建 Doclib 缓存。
+结果返回值和新缓存使用 `-`,全角 `~` 及负号倒数页码不受支持。
+完整说明见 [页码规范与历史结果兼容](../../next/page-ranges.md)。
diff --git a/mineru/doclib/server.py b/mineru/doclib/server.py
index 19b2c22f..53bf357f 100644
--- a/mineru/doclib/server.py
+++ b/mineru/doclib/server.py
@@ -28,6 +28,7 @@ from ..parser.page_range import (
expand_page_range as _expand_page_range,
format_page_range as _page_numbers_to_range_str,
normalize_page_range_input,
+ normalize_result_page_range,
)
from ..parser.tier import TierDependencyError, ensure_tier_runtime_dependencies
from ..render._internal.markdown.assets import build_markdown_image
@@ -2327,12 +2328,15 @@ def _doc_info(row: DocRow, *, files: list[FileInfo] | None = None) -> DocInfo:
def _parse_info(row: ParseRow, *, coverage: ParseCoverage | None = None) -> ParseInfo:
data = dict(row)
+ data["page_range"] = normalize_result_page_range(row["page_range"])
data["coverage"] = coverage
return ParseInfo.model_validate(data)
def _tier_parse_info(row: ParseRow) -> TierParseInfo:
- return TierParseInfo.model_validate(row)
+ data = dict(row)
+ data["page_range"] = normalize_result_page_range(row["page_range"])
+ return TierParseInfo.model_validate(data)
def _watch_info(row: WatchTargetRow) -> WatchInfo:
diff --git a/mineru/doclib/services/parse_svc.py b/mineru/doclib/services/parse_svc.py
index 85e4e80a..6ee4944f 100644
--- a/mineru/doclib/services/parse_svc.py
+++ b/mineru/doclib/services/parse_svc.py
@@ -32,6 +32,7 @@ from ...parser.page_range import (
expand_page_range,
format_page_range as _page_numbers_to_range_str,
normalize_page_range_input,
+ normalize_result_page_range,
parse_page_range_set,
)
from ...types import QUALITY_TIERS, TIER_ORDER, PageInfo, Tier, select_default_quality_tier, select_parsing_rule_tier
@@ -1547,7 +1548,7 @@ def _parse_record_response(row: ParseRow) -> dict:
"id": row["id"],
"sha256": row["sha256"],
"tier": row["tier"],
- "page_range": row["page_range"],
+ "page_range": normalize_result_page_range(row["page_range"]),
"status": row["status"],
"done_at": row.get("done_at"),
"created_at": row.get("created_at"),
diff --git a/mineru/parser/page_range.py b/mineru/parser/page_range.py
index 113a85de..ac7959a0 100644
--- a/mineru/parser/page_range.py
+++ b/mineru/parser/page_range.py
@@ -125,15 +125,21 @@ def expand_page_range(raw: str | None, page_count: int) -> str:
def _absolute_intervals(raw: str) -> list[tuple[int, int]]:
- """读取已求值的覆盖范围;空文本表示空集合,禁止未经求值的 all/rN。"""
+ """读取已求值范围,兼容历史半角 ~;空文本为空集合,禁止 all/rN。"""
if not raw.strip():
return []
- segments = _parse_segments(raw)
+ # 只在结果读取边界兼容旧分隔符,新请求及全角波浪号仍由严格语法拒绝。
+ segments = _parse_segments(raw.replace("~", "-"))
if segments is None or any(start < 0 or end < 0 for start, end in segments):
raise _invalid_range(raw, "page count is required to resolve all/rN")
return _merge_intervals(segments)
+def normalize_result_page_range(raw: str) -> str:
+ """将已求值的新旧结果范围规范化为连字符格式,不修改持久化记录或文件名。"""
+ return _format_intervals(_absolute_intervals(raw))
+
+
def parse_page_range_set(raw: str) -> set[int]:
"""把已求值的范围转换为 1-based 页码集合,供缓存覆盖与内容过滤使用。"""
return {page_no for start, end in _absolute_intervals(raw) for page_no in range(start, end + 1)}
@@ -156,6 +162,7 @@ __all__ = [
"format_page_range",
"get_end_page_id",
"normalize_page_range_input",
+ "normalize_result_page_range",
"parse_page_range",
"parse_page_range_set",
]
diff --git a/tests/unittest/test_doclib_legacy_page_ranges.py b/tests/unittest/test_doclib_legacy_page_ranges.py
new file mode 100644
index 00000000..f1f24f9a
--- /dev/null
+++ b/tests/unittest/test_doclib_legacy_page_ranges.py
@@ -0,0 +1,213 @@
+from __future__ import annotations
+
+import asyncio
+import json
+from collections.abc import AsyncIterator, Iterable
+from contextlib import asynccontextmanager
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+
+from mineru.doclib.background import compaction as compaction_module
+from mineru.doclib.background.compaction import Compaction
+from mineru.doclib.core.db import DatabaseManager
+from mineru.doclib.core.fts import FTSManager
+from mineru.doclib.server import DoclibServer, _parse_info, _tier_parse_info
+from mineru.doclib.services.config_svc import ConfigService
+from mineru.doclib.services.parse_svc import (
+ ParseService,
+ _parse_record_response,
+ load_pages_from_done_batches,
+ parse_batch_json_path,
+)
+from mineru.parser.base import MIDDLE_JSON_SCHEMA_VERSION
+from mineru.version import __version__
+
+_SHA256 = "a" * 64
+
+
+def _cache_path(root: Path, page_range: str, done_at: int) -> Path:
+ """按数据库原始范围定位批次文件,保留旧半角波浪号文件名。"""
+ return Path(parse_batch_json_path(str(root), _SHA256, "flash", page_range, done_at))
+
+
+async def _add_result(
+ db: DatabaseManager, root: Path, page_range: str, done_at: int, page_numbers: Iterable[int], label: str
+) -> int:
+ """写入真实历史或新格式批次及可辨识正文,构造混合缓存。"""
+ path = _cache_path(root, page_range, done_at)
+ path.parent.mkdir(parents=True, exist_ok=True)
+ payload = {
+ "schema_version": MIDDLE_JSON_SCHEMA_VERSION,
+ "is_full_document": False,
+ "file_suffix": "pdf",
+ "effort": "flash",
+ "parse_mode": "txt",
+ "mineru_version": __version__,
+ "pages": [
+ {
+ "page_idx": page_no - 1,
+ "blocks": [
+ {
+ "type": "text",
+ "index": 0,
+ "bbox": [0, 0, 1, 1],
+ "content": [{"type": "text", "content": f"{label} page {page_no}"}],
+ }
+ ],
+ }
+ for page_no in page_numbers
+ ],
+ }
+ path.write_text(json.dumps(payload), encoding="utf-8")
+ return await db.execute_insert(
+ "INSERT INTO parses (sha256, tier, page_range, status, privacy, done_at, created_at, updated_at) "
+ "VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
+ (_SHA256, "flash", page_range, "done", "local", done_at, done_at, done_at),
+ )
+
+
+@asynccontextmanager
+async def _legacy_store(root: Path) -> AsyncIterator[tuple[DatabaseManager, ParseService, DoclibServer]]:
+ """建立已入库的源文件元数据、旧名称缓存及真实 SQLite 服务,退出时关闭连接。"""
+ source = root / "document.pdf"
+ source.write_bytes(b"%PDF-1.7\n")
+ stat = source.stat()
+ db = DatabaseManager(str(root / "doclib.db"))
+ await db.initialize()
+ try:
+ await db.execute(
+ "INSERT INTO docs (sha256, short_id, size_bytes, file_type, page_count, first_seen_at, updated_at) "
+ "VALUES (?, ?, ?, ?, ?, ?, ?)",
+ (_SHA256, "aaaaaaa", stat.st_size, "pdf", 10, 1000, 1000),
+ )
+ await db.execute(
+ "INSERT INTO files (path, filename, ext, size_bytes, mtime_ms, sha256, status, first_seen_at, updated_at) "
+ "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
+ (str(source), source.name, "pdf", stat.st_size, int(stat.st_mtime * 1000), _SHA256, "active", 1000, 1000),
+ )
+ await _add_result(db, root, "1~5", 1000, range(1, 6), "old")
+ service = ParseService(
+ db=db, fts=FTSManager(db), config_svc=ConfigService(db), data_dir=str(root), parse_lock_timeout_sec=1800
+ )
+ server = DoclibServer(SimpleNamespace(db=db, parse_svc=service, data_dir=str(root)))
+ yield db, service, server
+ finally:
+ await db.close()
+
+
+def test_old_cache_hits_exports_and_serializes_without_mutating_storage(tmp_path: Path) -> None:
+ """旧结果正常命中、导出和查询,返回新格式而文件与数据库始终保留旧名称。"""
+
+ async def verify() -> None:
+ """读取各个结果出口,确认规范化不破坏原始缓存定位。"""
+ async with _legacy_store(tmp_path) as (db, service, server):
+ old_path = _cache_path(tmp_path, "1~5", 1000)
+ original_bytes = old_path.read_bytes()
+ row = await db.fetchone("SELECT * FROM parses WHERE sha256=?", (_SHA256,))
+ original_row = dict(row)
+ result = await service.request_parse(str(tmp_path / "document.pdf"), tier="flash", page_range="1-5")
+ assert result.cache_hit and result.status == "done"
+ assert result.page_range == "1-5"
+ assert result.created_parse_ids == []
+ listing = await server.list_parses(doc_ref="aaaaaaa", tier="flash", page_range="1-10")
+ assert listing.parses[0].page_range == "1-5"
+ assert listing.coverage.done_page_range == "1-5"
+ assert listing.coverage.missing_page_range == "6-10"
+ assert (await server.get_parse(row["id"])).page_range == "1-5"
+ assert (await service.get_parse_record(row["id"]))["page_range"] == "1-5"
+ assert (await service.list_parse_records(sha256=_SHA256))["parses"][0]["page_range"] == "1-5"
+ public_row = dict(row, short_id="aaaaaaa")
+ assert _parse_info(public_row).page_range == "1-5"
+ assert _tier_parse_info(public_row).page_range == "1-5"
+ assert _parse_record_response(public_row)["page_range"] == "1-5"
+ assert public_row["page_range"] == "1~5"
+ exported = await server._render_doc_content(
+ _SHA256, tier="flash", page_range="1-5", format="markdown", no_marker=True
+ )
+ assert "old page 1" in exported and "old page 5" in exported
+ content = await server.get_doc_content("aaaaaaa", tier="flash", page_range="1-5")
+ assert content.request_scope.page_range == "1-5"
+ assert "old page 3" in content.content
+ assert row == original_row
+ assert (await db.fetchone("SELECT * FROM parses WHERE id=?", (row["id"],)))["page_range"] == "1~5"
+ assert old_path.read_bytes() == original_bytes
+ assert not _cache_path(tmp_path, "1-5", 1000).exists()
+
+ asyncio.run(verify())
+
+
+def test_old_cache_only_schedules_uncovered_pages(tmp_path: Path) -> None:
+ """旧 1~5 批次覆盖前五页,新请求只创建 6-10,重复请求复用新任务。"""
+
+ async def verify() -> None:
+ """检查真实任务入队和原记录内容。"""
+ async with _legacy_store(tmp_path) as (db, service, _server):
+ result = await service.request_parse(str(tmp_path / "document.pdf"), tier="flash", page_range="1-10")
+ assert not result.cache_hit and len(result.created_parse_ids) == 1
+ rows = await db.fetchall("SELECT page_range, status FROM parses ORDER BY id")
+ assert rows == [{"page_range": "1~5", "status": "done"}, {"page_range": "6-10", "status": "pending"}]
+ repeated = await service.request_parse(str(tmp_path / "document.pdf"), tier="flash", page_range="1-10")
+ assert repeated.reused_parse_ids == result.created_parse_ids
+ assert repeated.created_parse_ids == []
+ assert _cache_path(tmp_path, "1~5", 1000).is_file()
+
+ asyncio.run(verify())
+
+
+def test_mixed_cache_compaction_keeps_latest_pages_and_writes_new_ranges(tmp_path: Path) -> None:
+ """新旧缓存合并时重复页取最新正文,并生成连字符记录与文件名。"""
+
+ async def verify() -> None:
+ """压缩前后均核对页序与重复页正文。"""
+ async with _legacy_store(tmp_path) as (db, _service, server):
+ await _add_result(db, tmp_path, "4-7", 2000, range(4, 8), "new")
+ rows = await db.fetchall("SELECT * FROM parses ORDER BY done_at DESC")
+ pages = load_pages_from_done_batches(str(tmp_path), _SHA256, "flash", rows)
+ assert [page.page_idx for page in pages] == list(range(7))
+ assert pages[3].blocks[0].content[0].content == "new page 4"
+ assert pages[4].blocks[0].content[0].content == "new page 5"
+ compaction = Compaction(db=db, interval_sec=600, data_dir=str(tmp_path))
+ assert await compaction._compact_doc_tier(_SHA256, "flash") == 1
+ assert await db.fetchall("SELECT page_range, done_at FROM parses") == [{"page_range": "1-7", "done_at": 2000}]
+ path = _cache_path(tmp_path, "1-7", 2000)
+ assert sorted(p.name for p in path.parent.glob("*.json")) == [path.name]
+ exported = await server._render_doc_content(
+ _SHA256, tier="flash", page_range="1-7", format="markdown", no_marker=True
+ )
+ assert "old page 3" in exported and "new page 4" in exported
+ assert "old page 4" not in exported
+
+ asyncio.run(verify())
+
+
[email protected]("failure", ["missing_source", "write_failure"])
+def test_failed_mixed_cache_compaction_preserves_old_records(
+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, failure: str
+) -> None:
+ """源文件缺失或新文件提升失败时保留旧记录与现存缓存。"""
+
+ async def verify() -> None:
+ """在真实缓存与数据库上触发失败并检查无破坏性替换。"""
+ async with _legacy_store(tmp_path) as (db, _service, _server):
+ await _add_result(db, tmp_path, "4-7", 2000, range(4, 8), "new")
+ old_path = _cache_path(tmp_path, "1~5", 1000)
+ new_path = _cache_path(tmp_path, "4-7", 2000)
+ if failure == "missing_source":
+ new_path.unlink()
+ else:
+
+ def fail_replace(source: str, target: str) -> None:
+ """模拟压缩结果提升到目标文件时失败。"""
+ raise OSError("simulated cache write failure")
+
+ monkeypatch.setattr(compaction_module.os, "replace", fail_replace)
+ records = await db.fetchall("SELECT * FROM parses ORDER BY id")
+ files = {path.name: path.read_bytes() for path in old_path.parent.iterdir()}
+ compaction = Compaction(db=db, interval_sec=600, data_dir=str(tmp_path))
+ assert await compaction._compact_doc_tier(_SHA256, "flash") == 0
+ assert await db.fetchall("SELECT * FROM parses ORDER BY id") == records
+ assert {path.name: path.read_bytes() for path in old_path.parent.iterdir()} == files
+
+ asyncio.run(verify())
diff --git a/tests/unittest/test_pdf_page_id.py b/tests/unittest/test_pdf_page_id.py
index 28a08417..70611a5e 100644
--- a/tests/unittest/test_pdf_page_id.py
+++ b/tests/unittest/test_pdf_page_id.py
@@ -29,6 +29,7 @@ from mineru.parser.page_range import (
expand_page_range,
format_page_range,
normalize_page_range_input,
+ normalize_result_page_range,
parse_page_range,
parse_page_range_set,
)
@@ -49,6 +50,7 @@ VALID_RANGES = [
("5-5", [5], "5"),
]
INVALID_RANGES = [
+ "1~5",
"1~5",
"-1",
"-5~-1",
@@ -365,3 +367,33 @@ def test_doclib_relative_selection_coverage_and_export(tmp_path: Path) -> None:
await db.close()
asyncio.run(verify())
+
+
+ ("raw", "canonical", "pages"),
+ [
+ ("1~5,8", "1-5,8", {1, 2, 3, 4, 5, 8}),
+ ("1~3,5-7", "1-3,5-7", {1, 2, 3, 5, 6, 7}),
+ (" 5 , 1 ~ 3 , 2-5 ", "1-5", {1, 2, 3, 4, 5}),
+ ("5~5", "5", {5}),
+ ("", "", set()),
+ (" ", "", set()),
+ ],
+)
+def test_historical_result_ranges_are_read_without_relaxing_inputs(raw: str, canonical: str, pages: set[int]) -> None:
+ """历史半角波浪号仅在已求值结果读取中生效,规范输出仍使用连字符。"""
+ assert normalize_result_page_range(raw) == canonical
+ assert parse_page_range_set(raw) == pages
+ assert count_pages_in_range(raw) == len(pages)
+ if "~" in raw:
+ with pytest.raises(InvalidRequestError):
+ normalize_page_range_input(raw)
+
+
[email protected]("raw", ["1~5", "-1", "-5~-1", "1~-1", "r1", "r5-r1", "all", "5~3", "1~~5", "0~3"])
+def test_result_readers_reject_fullwidth_negative_or_unresolved_ranges(raw: str) -> None:
+ """历史结果读取不支持全角波浪号、负号倒数页码、符号端点或非法区间。"""
+ for reader in (normalize_result_page_range, parse_page_range_set, count_pages_in_range):
+ with pytest.raises(InvalidRequestError) as error:
+ reader(raw)
+ assert error.value.code == "page_range_invalid"