Skip to content

Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

feat(gradio): cancel remote tasks when conversions are invalidated

myhloliec6c23cff904
diff --git a/docs/next/cli/mineru-kit-webui.md b/docs/next/cli/mineru-kit-webui.md
index fb96ab71..074aa2f1 100644
--- a/docs/next/cli/mineru-kit-webui.md
+++ b/docs/next/cli/mineru-kit-webui.md
@@ -105,7 +105,9 @@ Gradio 首先发现 `/v1/health` 和 `/v1/tiers`,然后通过 `MinerUApiParser
左下角使用与 3.4.5 一致的两列步骤卡片,展示准备请求、检查服务、提交任务、排队、解析中、下载结果、整理输出和完成。文件准备状态来自 V1 的 `waiting-file` 响应;排队状态来自本地任务槽或 V1 的 `pending` 响应,圆点每秒循环更新;不显示前方任务数。每个界面服务仍一次执行一个转换,等待中的会话也会立即显示排队动画。
-收到 V1 的 `running` 后开始解析计时,由浏览器本地刷新;收到任务终态时冻结。API 路径的最终完成文案优先使用响应中已有的 `files[0].parse.duration_ms`,包含服务端解析和结果导出/打包,不包含排队、客户端下载和 WebUI 整理输出,因此即使轮询跳过 `running` 也能显示亚秒耗时。HF 等直接调用解析函数的适配层继续使用实际阶段通知之间的计时;缺少 API 耗时字段时也保留这种计时回退。完成数值按两位小数四舍五入,最低显示 `0.01 秒`。两种路径均不承诺该值等同于日志中的纯 `infer` 耗时。界面动画不增加 API 请求,任务轮询仍为每秒一次。清除或更换文件会停止当前会话的状态流并释放本地等待位置,不向 V1 发送远端取消请求。
+收到 V1 的 `running` 后开始解析计时,由浏览器本地刷新;收到任务终态时冻结。API 路径的最终完成文案优先使用响应中已有的 `files[0].parse.duration_ms`,包含服务端解析和结果导出/打包,不包含排队、客户端下载和 WebUI 整理输出,因此即使轮询跳过 `running` 也能显示亚秒耗时。HF 等直接调用解析函数的适配层继续使用实际阶段通知之间的计时;缺少 API 耗时字段时也保留这种计时回退。完成数值按两位小数四舍五入,最低显示 `0.01 秒`。两种路径均不承诺该值等同于日志中的纯 `infer` 耗时。界面动画不增加 API 请求,任务轮询仍为每秒一次。
+
+清除、更换文件、同会话重新提交或正常关闭页面会立即停止旧任务的界面状态流,并向该任务所属 V1 服务发送 `POST /v1/tasks/{task_id}/cancel`。每个任务最多发起一次取消,本地执行槽在取消清理结束后释放;已知服务端任务终结时不会再次取消,也不删除已完成产物。创建回执尚未返回时先等待同一次创建取得 ID(整体最多 120 秒),随后取消请求最多等待 5 秒;网络故障可能使远端取消无法确认,此时记录日志。页面关闭联动依赖服务端收到 Gradio unload 事件。HF 进程内适配层没有 HTTP Tasks,仍等待已进入同步 CPU/ZeroGPU 的工作安全结束后释放执行槽。
`MinerUApiParser.parse()` 和 `parse_async()` 提供可选的 `status_callback` 参数,接收公开的 `ApiTaskStatus`(`waiting-file`、`pending`、`running`、`completed`、`partial`、`failed`、`canceled`)。回调在任务提交响应及每次轮询响应后同步调用;回调异常只记录日志。这里的 `completed` 表示服务端任务完成,解析方法仍需下载并构造 `ParseResult` 后才返回。可选的 `duration_callback` 在最终成功响应之后、开始下载之前接收既有文件耗时(浮点毫秒);缺失或非法耗时不触发回调,回调异常不影响解析结果。该值属于本次调用,不存放在共享 parser 实例或 `ParseResult` 中。
diff --git a/docs/next/sdk/api-parser.md b/docs/next/sdk/api-parser.md
index 04c7b698..ff49ce4f 100644
--- a/docs/next/sdk/api-parser.md
+++ b/docs/next/sdk/api-parser.md
@@ -32,6 +32,8 @@ HTTP 输入只有上传和 URL;SDK 的路径参数始终代表客户端文件
状态回调在创建和查询后触发,回调异常只记录日志。服务端完成表示开始下载,不等于 SDK 已经返回。
单文件成功后的 `parse` 包含实际 `tier`、`duration_ms` 和 `parser_version`。`duration_callback` 在开始下载前接收服务端单文件 `parse.duration_ms`,包含解析和导出;缺失或非法值不触发回调。
+`parse_async(..., cancel_on_cancel=True)` 将协程取消联动到 `POST /v1/tasks/{task_id}/cancel`,默认关闭,普通 SDK、Doclib 和 file-parse 的既有行为不变。Gradio 显式开启此选项。取消发生在创建期间时,会等待同一次创建回执以取得服务端 ID,创建整体最多等待 120 秒,不重复提交;拿到未终结任务后,最多用 5 秒发送一次取消。404 和 `409 task_already_terminal` 视为已无可取消任务,其他失败记录日志,最终仍传播原始 `CancelledError`。网络失败且没有取得 ID 时无法确认远端取消。已知任务终结后的下载或本地输出取消不会调用远端取消,也不删除产物。
+
创建请求没有幂等保证,不自动重试。上传响应丢失后先查状态:进入 pending/running/completed 后不重传;waiting-file 且 未返回 upload 时仅轮询;出现上传目标后才在原重试预算内发送。PUT 成功后等待确认,不因状态仍是 waiting-file 而重复上传。查询、传输和下载有界重试;任务查询和用量查询对 408/429/500/502/503/504 最多请求 3 次,支持 `Retry-After` 秒数和 HTTP 日期。单次建议等待超过 60 秒时返回原错误,不提前重试;无有效建议时使用指数退避。下载签名失效后重新查询任务。密钥仅附加到 API 同源请求,每次重定向重新判断来源。
结构化业务错误保留 code/message/param。传输失败保留 `_APITransportError` 阶段及超时信息,Doclib 继续区分 `remote_timeout`、`remote_unreachable` 与业务 `parse_failed`。
diff --git a/mineru/kit/gradio/client.py b/mineru/kit/gradio/client.py
index 79409661..a8acc7ae 100644
--- a/mineru/kit/gradio/client.py
+++ b/mineru/kit/gradio/client.py
@@ -231,7 +231,11 @@ class V1ArtifactClient:
try:
result = await parser.parse_async(
- path, page_range=page_range, status_callback=on_job_status, duration_callback=on_job_duration
+ path,
+ page_range=page_range,
+ status_callback=on_job_status,
+ duration_callback=on_job_duration,
+ cancel_on_cancel=True,
)
except Exception as exc:
emit(f"Failed: {exc}")
diff --git a/mineru/parser/api_client.py b/mineru/parser/api_client.py
index a26f88b4..767f374b 100644
--- a/mineru/parser/api_client.py
+++ b/mineru/parser/api_client.py
@@ -40,6 +40,8 @@ from .page_range import normalize_page_range_input
_POLL_INTERVAL_SECONDS = 1
_POLL_MAX_ATTEMPTS = 3600
+_TASK_CREATE_TIMEOUT_SECONDS = 120.0
+_TASK_CANCEL_TIMEOUT_SECONDS = 5.0
_TERMINAL_JOB_STATUSES = {"completed", "partial", "failed", "canceled"}
_TRANSPORT_MAX_ATTEMPTS = 3
_TRANSPORT_RETRY_BASE_DELAY_SECONDS = 0.25
@@ -405,13 +407,67 @@ class MinerUApiParser(DocumentParser):
page_range: str = "",
status_callback: Callable[[ApiTaskStatus], None] | None = None,
duration_callback: Callable[[float], None] | None = None,
+ cancel_on_cancel: bool = False,
) -> ParseResult:
- """创建单文件任务、按需上传并下载完整 ZIP,回调对应真实服务端阶段。"""
+ """创建单文件任务并下载结果;显式开启时将协程取消联动到服务端任务。"""
file_path = Path(path)
payload = await asyncio.to_thread(self._payload, file_path, normalize_page_range_input(page_range))
async with httpx.AsyncClient(
timeout=httpx.Timeout(120, connect=30), trust_env=self._trust_env, follow_redirects=False
) as cli:
+ task: dict[str, Any] | None = None
+ creation: asyncio.Task[dict[str, Any]] | None = None
+ try:
+ if cancel_on_cancel:
+ creation = asyncio.create_task(self._async_create_task(cli, payload, timeout=_TASK_CREATE_TIMEOUT_SECONDS))
+ task = await asyncio.shield(creation)
+ else:
+ task = await self._async_create_task(cli, payload)
+ _notify_job_status(task, status_callback)
+ created_status = task["status"]
+ task = await self._async_query(cli, task)
+ if task["status"] != created_status:
+ _notify_job_status(task, status_callback)
+ upload_state = _UploadState()
+ task = await self._async_upload_input(
+ cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state
+ )
+ for _ in range(_POLL_MAX_ATTEMPTS):
+ if task["status"] in _TERMINAL_JOB_STATUSES:
+ break
+ await asyncio.sleep(_POLL_INTERVAL_SECONDS)
+ task = await self._async_query(cli, task)
+ _notify_job_status(task, status_callback)
+ task = await self._async_upload_input(
+ cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state
+ )
+ if task["status"] not in _TERMINAL_JOB_STATUSES:
+ raise _V1APIError("timeout", f"Task {task['task_id']} did not complete within timeout")
+ _raise_for_terminal_job_error(task)
+ if task["files"][0].get("status") != "completed":
+ raise _V1APIError("parse_failed", "Task did not complete its input file")
+ _notify_job_duration(task, duration_callback)
+ with tempfile.TemporaryDirectory(prefix="mineru-api-client-") as directory:
+ target = Path(directory) / "result.zip"
+ await self._async_download_task(cli, task, target)
+ return _parse_result_from_zip_bytes(target, include_images=self.include_images)
+ except asyncio.CancelledError:
+ if cancel_on_cancel:
+ cleanup = asyncio.create_task(self._async_cancel_parse(cli, task, creation))
+ # 清除、换文件和 unload 可能重复取消同一协程,不能中断唯一的远端清理。
+ while not cleanup.done():
+ try:
+ await asyncio.shield(cleanup)
+ except asyncio.CancelledError:
+ continue
+ cleanup.result()
+ raise
+
+ async def _async_create_task(
+ self, cli: httpx.AsyncClient, payload: dict[str, Any], *, timeout: float | None = None
+ ) -> dict[str, Any]:
+ """一次性提交并校验创建回执,可为受保护的提交设置整体截止时间。"""
+ async with asyncio.timeout(timeout):
response = await _async_request_once(
cli,
"POST",
@@ -420,35 +476,47 @@ class MinerUApiParser(DocumentParser):
headers=self._headers(),
json_body=payload,
)
- task = self._created_task(response, payload)
- _notify_job_status(task, status_callback)
- created_status = task["status"]
- task = await self._async_query(cli, task)
- if task["status"] != created_status:
- _notify_job_status(task, status_callback)
- upload_state = _UploadState()
- task = await self._async_upload_input(
- cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state
- )
- for _ in range(_POLL_MAX_ATTEMPTS):
- if task["status"] in _TERMINAL_JOB_STATUSES:
- break
- await asyncio.sleep(_POLL_INTERVAL_SECONDS)
- task = await self._async_query(cli, task)
- _notify_job_status(task, status_callback)
- task = await self._async_upload_input(
- cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state
+ return self._created_task(response, payload)
+
+ async def _async_cancel_parse(
+ self,
+ cli: httpx.AsyncClient,
+ task: dict[str, Any] | None,
+ creation: asyncio.Task[dict[str, Any]] | None,
+ ) -> None:
+ """恢复被取消提交的身份并有界取消未终结任务,清理失败不覆盖调用方的取消。"""
+ try:
+ if task is None and creation is not None:
+ task = await creation
+ if task is None:
+ logger.warning("Task cancellation could not be confirmed: no task ID was received")
+ return
+ if task["status"] in _TERMINAL_JOB_STATUSES:
+ return
+ async with asyncio.timeout(_TASK_CANCEL_TIMEOUT_SECONDS):
+ response = await _async_request_once(
+ cli,
+ "POST",
+ task_status_url(self._base_url, task["task_id"]) + "/cancel",
+ stage="task cancellation",
+ headers=self._headers(),
)
- if task["status"] not in _TERMINAL_JOB_STATUSES:
- raise _V1APIError("timeout", f"Task {task['task_id']} did not complete within timeout")
- _raise_for_terminal_job_error(task)
- if task["files"][0].get("status") != "completed":
- raise _V1APIError("parse_failed", "Task did not complete its input file")
- _notify_job_duration(task, duration_callback)
- with tempfile.TemporaryDirectory(prefix="mineru-api-client-") as directory:
- target = Path(directory) / "result.zip"
- await self._async_download_task(cli, task, target)
- return _parse_result_from_zip_bytes(target, include_images=self.include_images)
+ if response.status_code == 404:
+ return
+ if (
+ response.status_code == 409
+ and (_structured_error(response.json()) or {}).get("code") == "task_already_terminal"
+ ):
+ return
+ canceled = self._check(response)
+ if canceled.get("task_id") != task["task_id"] or canceled.get("status") != "canceled":
+ raise _V1APIError("invalid_response", "Inconsistent Tasks cancellation response")
+ except (Exception, asyncio.CancelledError) as exc:
+ logger.warning(
+ "Task cancellation could not be confirmed for %s (%s)",
+ task["task_id"] if task is not None else "unknown task",
+ type(exc).__name__,
+ )
async def _async_query(self, cli: httpx.AsyncClient, task: dict[str, Any]) -> dict[str, Any]:
"""按配置的 API 基址和任务 ID 查询,保留元数据请求的有限重试。"""
diff --git a/tests/unittest/test_gradio_task_cancellation.py b/tests/unittest/test_gradio_task_cancellation.py
new file mode 100644
index 00000000..20e91234
--- /dev/null
+++ b/tests/unittest/test_gradio_task_cancellation.py
@@ -0,0 +1,151 @@
+"""真实 Gradio 事件经 SDK 联动 HTTP Tasks 取消的回归。"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any
+
+import httpx
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+
+from mineru.kit.gradio.app import build_gradio_app
+from mineru.kit.gradio.client import V1ArtifactClient, V1ServerCapabilities
+from tests.unittest.test_tasks_sdk import bind_transport
+from tests.unittest.test_task_api import service as _service_fixture
+
+# 复用真实 API/双 worker Router,客户端不能绕过服务端取消与资源清理。
+service = _service_fixture
+
+
[email protected]("trigger", ["clear", "file_change", "resubmit", "unload"])
+def test_ui_invalidation_cancels_only_its_upstream_task(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, trigger: str) -> None:
+ """通过真实组件回调和 HTTPX 传输验证四类事件,旧票据不能取消替换后的任务。"""
+ source = tmp_path / "source.csv"
+ source.write_text("a,b\n1,2\n")
+ client = V1ArtifactClient(api_url="https://api.test")
+ capabilities = V1ServerCapabilities(client.base_url, ("flash",), ("default",), ("upload",))
+ client._capabilities = capabilities
+ demo = build_gradio_app(client, capabilities, output_root=tmp_path / "output", enable_example=False)
+ callbacks = {function.name: function.fn for function in demo.fns.values()}
+ request = SimpleNamespace(session_hash="browser-session")
+
+ async def scenario() -> None:
+ """让任务停在真实 SDK 查询中,界面事件必须发送一次对应服务端 ID 的 cancel。"""
+ waiting: asyncio.Queue[str] = asyncio.Queue()
+ canceled: list[str] = []
+ created = 0
+
+ async def handle(message: httpx.Request) -> httpx.Response:
+ """创建独立的服务端身份,任何错误路由或旧 DELETE 都会失败。"""
+ nonlocal created
+ assert message.url.host == "api.test"
+ if message.method == "POST" and message.url.path == "/v1/tasks":
+ created += 1
+ return httpx.Response(
+ 202,
+ json={
+ "task_id": f"remote-{created}",
+ "status": "pending",
+ "files": [{"file_id": f"file-{created}", "file_name": source.name, "status": "pending"}],
+ },
+ )
+ remote = message.url.path.split("/")[3]
+ if message.method == "POST" and message.url.path.endswith("/cancel"):
+ canceled.append(remote)
+ return httpx.Response(200, json={"task_id": remote, "status": "canceled"})
+ assert message.method == "GET"
+ waiting.put_nowait(remote)
+ await asyncio.Future()
+ raise AssertionError("unreachable")
+
+ bind_transport(monkeypatch, handle)
+ tickets = [json.dumps({"run_id": f"{index:032x}", "revision": index}) for index in (1, 2)]
+ first = asyncio.create_task(callbacks["convert_ui"](str(source), 0, "", False, tickets[0], request))
+ second = None
+ try:
+ assert await asyncio.wait_for(waiting.get(), 5) == "remote-1"
+ if trigger == "resubmit":
+ second = asyncio.create_task(callbacks["convert_ui"](str(source), 0, "", False, tickets[1], request))
+ assert await asyncio.wait_for(waiting.get(), 5) == "remote-2"
+ elif trigger == "unload":
+ await callbacks["unload_session"](request)
+ else:
+ await callbacks["cancel_ui_conversion"](tickets[0], request)
+ assert await asyncio.wait_for(first, 5) == ""
+ assert canceled == ["remote-1"]
+ # 重复的清除通知不能再次取消原任务,也不能误伤替换任务。
+ await callbacks["cancel_ui_conversion"](tickets[0], request)
+ assert canceled == ["remote-1"]
+ if second is not None:
+ assert not second.done()
+ await callbacks["cancel_ui_conversion"](tickets[1], request)
+ assert await asyncio.wait_for(second, 5) == ""
+ assert canceled == ["remote-1", "remote-2"]
+ finally:
+ first.cancel()
+ if second is not None:
+ second.cancel()
+ await asyncio.gather(first, *([second] if second is not None else []), return_exceptions=True)
+
+ try:
+ asyncio.run(scenario())
+ finally:
+ demo.close()
+
+
+def test_gradio_cancel_reaches_real_api_and_router(
+ service: tuple[TestClient, FastAPI], tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """只替换推理工作,贯通真正的提交、上传、取消路由及 worker 协程回收。"""
+ import threading
+
+ from mineru.kit.gradio import client as gradio_client
+ from mineru.parser import api_client, api_server
+ from tests.unittest.test_kit_gradio import _httpx_proxy_for_test_client
+
+ http_client, app = service
+ started, stopped = threading.Event(), threading.Event()
+ source = tmp_path / "real.html"
+ source.write_text("<h1>cancel actual task</h1>")
+
+ async def parse(*_args: Any, **_kwargs: Any) -> None:
+ """保持实际解析作业运行,取消必须最终到达该生产工作协程。"""
+ started.set()
+ try:
+ await asyncio.Future()
+ finally:
+ stopped.set()
+
+ monkeypatch.setattr(api_server, "parse_async", parse)
+ calls = []
+ proxy = _httpx_proxy_for_test_client(http_client, calls)
+ monkeypatch.setattr(gradio_client, "httpx", proxy)
+ monkeypatch.setattr(api_client, "httpx", proxy)
+ monkeypatch.setattr(api_client, "_POLL_INTERVAL_SECONDS", 0.01)
+ client = V1ArtifactClient(api_url="http://testserver")
+
+ async def scenario() -> None:
+ """在真实解析开始后取消等待,服务端必须保留 canceled 终态且清理作业。"""
+ operation = asyncio.create_task(client.parse_file(source, tier="flash", page_range=""))
+ try:
+ assert await asyncio.to_thread(started.wait, 5)
+ operation.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await asyncio.wait_for(operation, 5)
+ assert await asyncio.to_thread(stopped.wait, 5)
+ finally:
+ operation.cancel()
+ await asyncio.gather(operation, return_exceptions=True)
+
+ asyncio.run(scenario())
+ cancellations = [path for method, path, _ in calls if method == "POST" and path.endswith("/cancel")]
+ assert len(cancellations) == 1
+ response = http_client.get(cancellations[0].removesuffix("/cancel"))
+ assert response.status_code == 200
+ assert response.json()["status"] == "canceled"
+ assert response.json()["files"][0]["status"] == "canceled"
diff --git a/tests/unittest/test_kit_gradio.py b/tests/unittest/test_kit_gradio.py
index 1315c6ac..fa41351b 100644
--- a/tests/unittest/test_kit_gradio.py
+++ b/tests/unittest/test_kit_gradio.py
@@ -663,7 +663,13 @@ def test_v1_client_preserves_v1_error_code(monkeypatch: pytest.MonkeyPatch, tmp_
"""忽略构造参数。"""
async def parse_async(
- self, _path: Path, *, page_range: str, status_callback: Any = None, duration_callback: Any = None
+ self,
+ _path: Path,
+ *,
+ page_range: str,
+ status_callback: Any = None,
+ duration_callback: Any = None,
+ cancel_on_cancel: bool = False,
) -> ParseResult:
"""抛出与正式 API client 一致的错误形态。"""
error = RuntimeError(f"bad range: {page_range}")
@@ -696,11 +702,18 @@ def test_v1_client_parse_uses_api_parser_zip_contract(
calls.update(kwargs)
async def parse_async(
- self, path: Path, *, page_range: str, status_callback: Any = None, duration_callback: Any = None
+ self,
+ path: Path,
+ *,
+ page_range: str,
+ status_callback: Any = None,
+ duration_callback: Any = None,
+ cancel_on_cancel: bool = False,
) -> ParseResult:
"""返回最小解析结果。"""
calls["path"] = path
calls["page_range"] = page_range
+ assert cancel_on_cancel is True
assert statuses == ["Preparing request...", "Submitting task..."]
assert status_callback is not None
status_callback("pending")
diff --git a/tests/unittest/test_task_cancellation.py b/tests/unittest/test_task_cancellation.py
new file mode 100644
index 00000000..d12c16b7
--- /dev/null
+++ b/tests/unittest/test_task_cancellation.py
@@ -0,0 +1,226 @@
+"""协程取消到 Tasks 取消接口的身份、竞争和有界清理回归。"""
+
+from __future__ import annotations
+
+import asyncio
+import copy
+import logging
+from pathlib import Path
+
+import httpx
+import pytest
+
+from mineru.parser import api_client
+from mineru.parser.api_client import MinerUApiParser
+from tests.unittest.test_tasks_sdk import bind_transport, task_snapshot
+
+
[email protected]("enabled", [False, True])
[email protected]("stage", ["create", "query", "upload", "pending", "running", "download"])
+def test_cancel_targets_original_task_once(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, enabled: bool, stage: str) -> None:
+ """从真实异步入口取消各个传输阶段,验证默认不取消、前缀鉴权和重复取消隔离。"""
+ source = tmp_path / "file.pdf"
+ source.write_bytes(b"pdf")
+
+ async def scenario() -> None:
+ """用可控网络响应阻塞指定阶段,并保证所有受保护任务在退出前收尾。"""
+ entered, release, cancel_entered, cancel_release = (asyncio.Event() for _ in range(4))
+ requests: list[httpx.Request] = []
+ queries = 0
+ snapshot = task_snapshot("waiting-file" if stage == "upload" else "pending", upload=stage == "upload")
+ snapshot["task_id"] = "task one"
+
+ async def handle(request: httpx.Request) -> httpx.Response:
+ """只阻塞选中的请求;取消响应保留到第二次本地取消之后。"""
+ nonlocal queries
+ requests.append(request)
+ if request.url.path.endswith("/cancel"):
+ assert request.url.raw_path == b"/gateway/api/v1/tasks/task%20one/cancel"
+ assert request.headers["authorization"] == "Bearer secret"
+ cancel_entered.set()
+ await cancel_release.wait()
+ return httpx.Response(200, json={"task_id": "task one", "status": "canceled"})
+ if request.method == "POST":
+ if stage == "create":
+ entered.set()
+ await release.wait()
+ return httpx.Response(202, json=copy.deepcopy(snapshot))
+ if request.method == "PUT" or request.url.host == "storage.test":
+ entered.set()
+ await release.wait()
+ raise AssertionError("canceled transfer resumed")
+ queries += 1
+ if stage == "query" or (stage in {"pending", "running"} and queries > 1):
+ entered.set()
+ await release.wait()
+ raise AssertionError("canceled query resumed")
+ current = copy.deepcopy(snapshot)
+ if stage in {"running", "download"}:
+ current = task_snapshot("completed" if stage == "download" else "running")
+ current["task_id"] = "task one"
+ return httpx.Response(200, json=current)
+
+ bind_transport(monkeypatch, handle)
+ operation = asyncio.create_task(
+ MinerUApiParser(api_url="https://api.test/gateway/api", api_key="secret").parse_async(
+ source, cancel_on_cancel=enabled
+ )
+ )
+ try:
+ await asyncio.wait_for(entered.wait(), 3)
+ operation.cancel()
+ await asyncio.sleep(0)
+ if enabled and stage == "create":
+ assert not operation.done()
+ release.set()
+ if enabled and stage != "download":
+ await asyncio.wait_for(cancel_entered.wait(), 3)
+ operation.cancel()
+ await asyncio.sleep(0)
+ assert not operation.done()
+ cancel_release.set()
+ with pytest.raises(asyncio.CancelledError):
+ await asyncio.wait_for(operation, 3)
+ assert sum(request.method == "POST" and not request.url.path.endswith("/cancel") for request in requests) == 1
+ cancels = [request for request in requests if request.url.path.endswith("/cancel")]
+ assert len(cancels) == int(enabled and stage != "download")
+ assert all(request.method != "DELETE" for request in requests)
+ finally:
+ release.set()
+ cancel_release.set()
+ operation.cancel()
+ await asyncio.gather(operation, return_exceptions=True)
+
+ asyncio.run(scenario())
+
+
[email protected]("outcome", ["404", "409", "bad409", "500", "timeout", "wrong_identity"])
+def test_cancel_failure_preserves_original_cancellation(
+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture, outcome: str
+) -> None:
+ """确认取消仅一次、短超时生效,正常竞争不告警,失败不会替换原始 CancelledError。"""
+ source = tmp_path / "file.pdf"
+ source.write_bytes(b"pdf")
+ monkeypatch.setattr(api_client, "_TASK_CANCEL_TIMEOUT_SECONDS", 0.03)
+
+ async def scenario() -> None:
+ """先取得合法任务身份,再在首次查询时取消。"""
+ entered = asyncio.Event()
+ cancellations = 0
+
+ async def handle(request: httpx.Request) -> httpx.Response:
+ """注入取消接口的终态竞争、错误和网络无响应。"""
+ nonlocal cancellations
+ if request.url.path.endswith("/cancel"):
+ cancellations += 1
+ if outcome == "timeout":
+ await asyncio.Future()
+ if outcome == "wrong_identity":
+ return httpx.Response(200, json={"task_id": "another", "status": "canceled"})
+ code = "other_conflict" if outcome == "bad409" else "task_already_terminal"
+ return httpx.Response(409 if outcome == "bad409" else int(outcome), json={"error": {"code": code}})
+ if request.method == "POST":
+ return httpx.Response(202, json=task_snapshot("pending"))
+ entered.set()
+ await asyncio.Future()
+ raise AssertionError("unreachable")
+
+ bind_transport(monkeypatch, handle)
+ operation = asyncio.create_task(MinerUApiParser(api_url="https://api.test").parse_async(source, cancel_on_cancel=True))
+ await asyncio.wait_for(entered.wait(), 3)
+ operation.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await asyncio.wait_for(operation, 1)
+ assert cancellations == 1
+
+ with caplog.at_level(logging.WARNING, logger="mineru.api_client"):
+ asyncio.run(scenario())
+ assert ("could not be confirmed" in caplog.text) == (outcome not in {"404", "409"})
+
+
[email protected]("failure", ["timeout", "transport", "invalid_response"])
+def test_cancel_during_failed_creation_has_no_orphan_cleanup(
+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture, failure: str
+) -> None:
+ """创建失败或截止后无法取得身份时记录未确认,不重建任务且回收受保护的提交。"""
+ source = tmp_path / "file.pdf"
+ source.write_bytes(b"pdf")
+ monkeypatch.setattr(api_client, "_TASK_CREATE_TIMEOUT_SECONDS", 0.05)
+
+ async def scenario() -> None:
+ """让调用方先取消,再触发原创建请求的失败。"""
+ entered, release = asyncio.Event(), asyncio.Event()
+ requests = []
+
+ async def handle(request: httpx.Request) -> httpx.Response:
+ """阻止重试,并观察超时确实结束原来的传输。"""
+ requests.append(request)
+ entered.set()
+ await release.wait()
+ if failure == "transport":
+ raise httpx.ReadError("lost receipt", request=request)
+ return httpx.Response(202, json={})
+
+ bind_transport(monkeypatch, handle)
+ operation = asyncio.create_task(MinerUApiParser(api_url="https://api.test").parse_async(source, cancel_on_cancel=True))
+ await asyncio.wait_for(entered.wait(), 3)
+ operation.cancel()
+ if failure != "timeout":
+ release.set()
+ with pytest.raises(asyncio.CancelledError):
+ await asyncio.wait_for(operation, 1)
+ assert len(requests) == 1
+ assert not [task for task in asyncio.all_tasks() if task is not asyncio.current_task() and not task.done()]
+
+ asyncio.run(scenario())
+ assert "could not be confirmed for unknown task" in caplog.text
+
+
+def test_concurrent_calls_on_same_parser_keep_separate_task_ids(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """同一解析器的两个并发调用只能取消各自服务端任务,不能复用实例上的旧身份。"""
+ sources = [tmp_path / name for name in ("a.pdf", "b.pdf")]
+ for source in sources:
+ source.write_bytes(b"pdf")
+
+ async def scenario() -> None:
+ """分别阻塞两个已提交任务,先取消一个并确认另一个仍可独立等待。"""
+ waiting: asyncio.Queue[str] = asyncio.Queue()
+ canceled = []
+
+ async def handle(request: httpx.Request) -> httpx.Response:
+ """用文件名生成不同公共任务 ID,避免用调度顺序推测调用身份。"""
+ if request.url.path.endswith("/cancel"):
+ task_id = request.url.path.split("/")[-2]
+ canceled.append(task_id)
+ return httpx.Response(200, json={"task_id": task_id, "status": "canceled"})
+ if request.method == "POST":
+ import json
+
+ file_name = json.loads(request.content)["files"][0]["file_name"]
+ snapshot = task_snapshot("pending", file_name=file_name)
+ snapshot["task_id"] = "task-" + Path(file_name).stem
+ return httpx.Response(202, json=snapshot)
+ waiting.put_nowait(request.url.path.split("/")[-1])
+ await asyncio.Future()
+ raise AssertionError("unreachable")
+
+ bind_transport(monkeypatch, handle)
+ parser = MinerUApiParser(api_url="https://api.test")
+ operations = [asyncio.create_task(parser.parse_async(source, cancel_on_cancel=True)) for source in sources]
+ try:
+ assert {await asyncio.wait_for(waiting.get(), 3) for _ in operations} == {"task-a", "task-b"}
+ operations[0].cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await asyncio.wait_for(operations[0], 3)
+ assert canceled == ["task-a"]
+ assert not operations[1].done()
+ operations[1].cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await asyncio.wait_for(operations[1], 3)
+ assert canceled == ["task-a", "task-b"]
+ finally:
+ for operation in operations:
+ operation.cancel()
+ await asyncio.gather(*operations, return_exceptions=True)
+
+ asyncio.run(scenario())