Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Python ai4sciencedocument-analysisdocxextract-datalayout-analysisocrparserpdfpdf-converterpdf-extractor-llmpdf-extractor-pretrainpdf-extractor-ragpdf-parserpptxpythonxlsx
feat(api): add recoverable synchronous file parsing service
myhlolif59acf1ffa79
diff --git a/docs/next/api.md b/docs/next/api.mdindex 4bc4395e..e74a708d 100644--- a/docs/next/api.md+++ b/docs/next/api.md@@ -23,6 +23,7 @@ Unified API 文档描述两类部署形态下的同一套 API:4. [Uploads 与 Files](api/uploads-files.md): 上传生命周期、文件对象、产物下载。5. [Parse Jobs](api/parse-jobs.md): 解析任务、轮询。[V1 tasks](api/tasks.md): JSON 清单、逐文件上传、独立结果下载链接和 Router 调度。+ [同步 file_parse](api/file-parse.md): 独立包装服务,单文件直传、同步附件和超时恢复。6. [Chat 与 Responses](api/chat.md): OpenAI-compatible 文档对话接口。7. [Webhooks](api/webhooks.md): 异步回调协议。8. [Usage 与 Limits](api/usage-limits.md): access level、限流、用量查询。diff --git a/docs/next/api/README.md b/docs/next/api/README.mdindex e545afb7..bfbef375 100644--- a/docs/next/api/README.md+++ b/docs/next/api/README.md@@ -68,6 +68,8 @@ ID 使用 OpenAI 风格的前缀加随机串。官方 API 使用 24 字符 base6JSON、逐文件上传、独立结果链接与 Router 负载规则见 [V1 tasks](tasks.md)。SaaS 可以独立实现同一契约,并返回 OSS 直链;客户端不要求其实现本地 Files 下载路由。+需要单文件直传和同步附件响应时,可另外启动 [file_parse 包装服务](file-parse.md),它提供自己的 `/v1/file_parse`、查询和下载接口,并通过上面的 Tasks 协议连接任意部署。+## 认证官方 API 使用 Bearer Token:diff --git a/docs/next/api/file-parse.md b/docs/next/api/file-parse.mdnew file mode 100644index 00000000..1b05ed9d--- /dev/null+++ b/docs/next/api/file-parse.md@@ -0,0 +1,121 @@+# 单文件同步解析:file_parse++`mineru-kit file-parse-server` 提供独立的同步 HTTP 包装服务:接收一个文件,创建 Tasks、上传、等待,完成后直接返回 ZIP、DOCX 或 HTML 附件。它连接本地 API、Router 或 SaaS;切换部署只修改上游地址和认证。++## 启动++```bash+mineru-kit file-parse-server --api-url http://127.0.0.1:8000++# 上游基地址可以带路径前缀;不要附加 /v1/tasks。+MINERU_API_URL=https://mineru.net/api MINERU_API_KEY=your-key \+ mineru-kit file-parse-server+```++| 启动参数 | 默认值 | 含义 |+|---|---|---|+| `--api-url` | `MINERU_API_URL`,否则必填 | 上游统一 Tasks API 基地址。 |+| `--api-key` | `MINERU_API_KEY`,否则为空 | 上游 Bearer Token。 |+| `--host`、`--port` | `127.0.0.1`、`8003` | 包装服务监听地址。 |+| `--state-dir` | `${MINERU_HOME:-~/.mineru}/file-parse` | 恢复记录及传输临时目录。 |+| `--retention-seconds` | `86400` | 首次观察终态后保留恢复记录的秒数;`0` 禁用回收。 |++包装入口不提供认证,面向本机或可信网络;部署方可以在反向代理设置访问认证。`--api-key` 只用于连接上游。请求不能覆盖上游地址或认证,入站 Authorization 不会转发到上游。++服务只执行 HTTP 上传、查询、下载,不初始化解析模型。OpenAPI 位于 `/openapi.json`,交互文档位于 `/docs`。这些路由属于独立包装服务,不注册到原 API Server 或 Router。++## 上传与等待++`POST /v1/file_parse` 仅接受 `multipart/form-data`,业务参数放在表单字段中;不接受查询参数、未知字段或重复标量字段。++| 字段 | 默认值 | 说明 |+|---|---|---|+| `file` | 必填 | 一个非空文件,保留扩展名;不接受多文件。 |+| `tier` | 上游默认 | `flash/basic/standard/advanced`。 |+| `ocr_mode` | `auto` | `auto/txt/ocr`。 |+| `page_range` | 整本 | 与 Tasks 一致,PDF 支持 `1-5`、`r1`、`all`;格式支持范围由上游校验。 |+| `output_formats` | `default` | `default/docx/html/latex`;重复同名字段表达列表,重复值按首次出现去重。 |+| `wait_timeout` | `3600` | 整数秒,范围 `0–3600`。 |+| `request_id` | 自动生成 | 1–64 个字母、数字、下划线或短横线。提前指定并保存它,可在连接中断后查询。 |++```bash+# 成功时 response.body 是附件;超时或错误时是 JSON,请同时检查 response.headers。+curl --max-time 7200 -D response.headers -o response.body \+ http://127.0.0.1:8003/v1/file_parse \+ -F '[email protected]' \+ -F 'request_id=report-20261022-001' \+ -F 'tier=flash' -F 'ocr_mode=txt' \+ -F 'output_formats=default' -F 'output_formats=docx'+```++`output_formats` 的列表语义与 [Tasks](tasks.md) 一致;multipart 中不使用 JSON 字符串或逗号分隔字符串。++| 选择 | HTTP 成功附件 |+|---|---|+| 单选 `default` | ZIP,包含 Markdown、中间协议、Structured Content、外置图片。 |+| 单选 `docx` | 图片内嵌的 DOCX。 |+| 单选 `html` | 图片内嵌的 HTML。 |+| 单选 `latex` | ZIP,包含 TeX 和外置图片。 |+| 多个不同格式 | 上游生成的单个 ZIP。 |++成功返回 `200`、正确的 Content-Type 和 `Content-Disposition: attachment`,文件名为 `<输入文件主名>.<扩展名>`,并携带 `X-Request-ID`、`X-Task-ID`。包装服务原样返回验证过的上游产物,不重新渲染或打包。++每次调用在确认上游文件就绪后开始计时,等待包含排队、解析与导出。接收文件、上游上传和结果下载不占此预算,独立使用传输空闲超时。缓存命中跳过上传。++- 正常轮询间隔为 2 秒。+- 连接、元数据请求、传输空闲超时分别为 10、30、600 秒。+- `wait_timeout=0` 在文件就绪后直接返回 `202` 查询信息;即使已经完成也不下载附件。+- 等待到期返回 `202` JSON 和 `Location: <status_url>`,上游继续执行。+- HTTP 客户端和反向代理的读取超时应覆盖上传、同步等待和结果传输;例如客户端配置 7200 秒,代理按业务文件大小配置相应超时。++## 超时与断连后查询++```json+{+ "request_id": "report-20261022-001",+ "task_id": "task_example",+ "status": "running",+ "status_url": "http://127.0.0.1:8003/v1/file_parse/report-20261022-001",+ "result_url": null,+ "output_formats": ["default", "docx"],+ "error": null+}+```++`GET /v1/file_parse/{request_id}` 查询状态,完成后 `result_url` 指向包装服务自己的 `/v1/file_parse/{request_id}/result`。调用者不需要上游密钥。++```bash+curl http://127.0.0.1:8003/v1/file_parse/report-20261022-001+curl --fail-with-body -o report.zip \+ http://127.0.0.1:8003/v1/file_parse/report-20261022-001/result+```++结果未就绪时下载接口返回 `409 result_not_ready`。下载中断可以再次 GET;每次会查询最新下载链接。签名失效时重新查询原任务,最多尝试 3 次;上传失败同样先查询状态,确认已上传成功便不重传。++状态包括接收完成后的 `prepared/submitting`,以及单文件 Tasks 的 `waiting_upload/queued/running/completed/failed/canceled`。`submission_unknown` 表示创建请求可能已被上游接受,但包装服务没有获得有效任务 ID:不会自动重放创建请求,不能承诺找回这个不确定窗口内的任务。++错误响应使用 `error.code/message/upstream_status`。已接受请求的错误还包含请求 ID 和查询地址。参数错误返回 `400`,内容类型错误返回 `415`,ID 冲突或结果未就绪返回 `409`,记录不存在或已过保留期返回 `404`;上游调用或解析失败通常返回 `502` 并保留上游错误码。等待预算耗尽返回 `202`,不是解析失败。++## 去重与持久化边界++完整接收并持久化源文件后,包装服务才接受请求并托管上游上传。此后连接中断不取消上传或解析;文件尚未完整接收时中断不会创建任务。++同一 `request_id`、相同文件内容、逻辑文件名及归一化解析参数复用原任务;不同输入返回 `409 request_id_conflict`。`wait_timeout` 不参与身份比较。没有预先提供 ID 且尚未收到响应的调用者,可能无法自行找回任务。++请求映射使用独立 SQLite 数据库,按上游地址和认证隔离。数据库不保存上游 API Key。单个状态目录只允许一个包装服务进程;服务重启后恢复已知任务,尚待上传且链接有效时继续上传。上游任务记录如果因重启或过期而消失,不会自动重建。++源文件在上游确认就绪后删除,终结失败也会清理源文件。结果只临时下载,发送完成或中断即删除;重启清理遗留传输文件。恢复记录的保留期限不延长上游结果有效期,查询也不会为上游上传链接续期。结果及请求过期后应使用新的请求 ID 发起新的解析。++## Python HTTP 示例++仓库提供 `scripts/http_file_parse_example.py`,依赖 `httpx`,提前打印并保存请求 ID;成功时原子保存附件,`202` 时打印查询信息并以 `124` 退出。++```bash+python scripts/http_file_parse_example.py report.pdf --output results/report.docx \+ --output-format docx --request-id report-20261022-002++python scripts/http_file_parse_example.py report.pdf --wait-timeout 0 \+ --request-id report-20261022-003+```++后续查询和下载仍使用上面的包装接口,不需要安装 MinerU SDK 或重建 `ParseResult`。diff --git a/docs/next/api/tasks.md b/docs/next/api/tasks.mdindex b51f3c11..1545043c 100644--- a/docs/next/api/tasks.md+++ b/docs/next/api/tasks.md@@ -221,5 +221,6 @@ Router 使用有界、可续租的内部结果租约保护 worker 产物,上本次协议不保留旧表单、直传或 `markdown/middle_json/structured_content/zip` Tasks 格式别名。客户端应改用 JSON、`files[].result_url` 及逐文件保存方式。独立 `/v1/uploads`、`/v1/parse/jobs` 与 SDK 的现有请求及输出协议保持不变。-`/v1/file_parse`、`/v1/tasks/{task_id}/result` 不恢复。+API Server 和 Router 不提供 `/v1/file_parse`、`/v1/tasks/{task_id}/result`。+需要同步单文件调用时,独立启动 [file_parse 包装服务](file-parse.md),由它调用本页的共同 Tasks 契约。SaaS Go 服务和对象存储配置由部署方按此共同契约实现,不属于本地 Python 服务代码。diff --git a/docs/next/cli/mineru-kit.md b/docs/next/cli/mineru-kit.mdindex 4d0dee53..794624da 100644--- a/docs/next/cli/mineru-kit.md+++ b/docs/next/cli/mineru-kit.md@@ -25,6 +25,7 @@| `mineru-kit parse` | 无状态文件/目录批处理解析 | [mineru-kit parse](mineru-kit-parse.md) || `mineru-kit webui` | 基于 V1 API 的本地文档解析 Web UI | [mineru-kit webui](mineru-kit-webui.md) || `mineru-kit api-server` | 启动兼容统一 API 的本地解析服务 | [mineru-kit api-server](mineru-kit-api-server.md) |+| `mineru-kit file-parse-server` | 单文件直传、同步附件和超时恢复,连接既有 Tasks 服务 | [file_parse](../api/file-parse.md) || `mineru-kit vlm-server` | 本地 VLM 服务,兼容 OpenAI Chat Completions 协议 | [mineru-kit vlm-server](mineru-kit-vlm-server.md) || `mineru-kit router` | 启动 V1 路由服务,转发到已有 upstream 或管理本地 worker | 仅暴露 `/v1/*` |@@ -50,7 +51,7 @@ mineru-api --host 127.0.0.1 --port 8000 --tier standard| 能力 | `mineru` | `mineru-kit` ||------|----------|--------------|-| 本地数据库 | 有 | 无 |+| 本地文档库 | 有 | 无;file-parse-server 仅保存独立请求恢复记录 || SHA256 缓存 | 有 | 无 || STDOUT 默认阅读 | 有 | 不是默认重点 || 目录批处理 | 非首要 | 核心能力 |diff --git a/mineru/kit/commands/__init__.py b/mineru/kit/commands/__init__.pyindex 01f43505..38d00a1e 100644--- a/mineru/kit/commands/__init__.py+++ b/mineru/kit/commands/__init__.py@@ -1,5 +1,5 @@"""mineru-kit command modules."""-from . import api_server, models, parse, router, vlm_server, webui+from . import api_server, file_parse_server, models, parse, router, vlm_server, webui-__all__ = ["api_server", "models", "parse", "router", "vlm_server", "webui"]+__all__ = ["api_server", "file_parse_server", "models", "parse", "router", "vlm_server", "webui"]diff --git a/mineru/kit/commands/file_parse_server.py b/mineru/kit/commands/file_parse_server.pynew file mode 100644index 00000000..78fcce8f--- /dev/null+++ b/mineru/kit/commands/file_parse_server.py@@ -0,0 +1,39 @@+"""启动只调用共同 Tasks 契约的同步包装服务。"""++from __future__ import annotations++from pathlib import Path++import typer++from ...utils.i18n import t+from ..errors import exit_with_message+++def file_parse_server_cmd(+ api_url: str = typer.Option(..., "--api-url", envvar="MINERU_API_URL", help=t("Upstream Tasks API base URL")),+ api_key: str | None = typer.Option(None, "--api-key", envvar="MINERU_API_KEY", help=t("Upstream Bearer API key")),+ host: str = typer.Option("127.0.0.1", "--host", help=t("Server host")),+ port: int = typer.Option(8003, "--port", min=1, max=65535, help=t("Server port")),+ state_dir: Path | None = typer.Option(None, "--state-dir", help=t("File parse recovery state directory")),+ retention_seconds: int = typer.Option(+ 86400, "--retention-seconds", min=0, help=t("Terminal retention seconds (default 86400; 0 disables collection)")+ ),+) -> None:+ """惰性加载包装应用,不启动本地解析模型或继承调用者的认证头。"""+ import uvicorn+ from filelock import Timeout++ from ..file_parse import FileParseSettings, create_app++ try:+ settings = FileParseSettings(api_url, api_key, state_dir, retention_seconds)+ except ValueError as exc:+ exit_with_message("invalid_request", str(exc))+ try:+ uvicorn.run(create_app(settings), host=host, port=port)+ except Timeout:+ exit_with_message("invalid_request", "Another file-parse-server is using this state directory")+++__all__ = ["file_parse_server_cmd"]diff --git a/mineru/kit/file_parse/__init__.py b/mineru/kit/file_parse/__init__.pynew file mode 100644index 00000000..ea0af740--- /dev/null+++ b/mineru/kit/file_parse/__init__.py@@ -0,0 +1,6 @@+"""独立单文件同步解析服务的显式入口。"""++from .app import create_app+from .settings import FileParseSettings++__all__ = ["FileParseSettings", "create_app"]diff --git a/mineru/kit/file_parse/app.py b/mineru/kit/file_parse/app.pynew file mode 100644index 00000000..b5e0784c--- /dev/null+++ b/mineru/kit/file_parse/app.py@@ -0,0 +1,308 @@+"""单文件同步 HTTP 包装层;成功返回附件,超时返回可恢复的请求链接。"""++from __future__ import annotations++import asyncio+import hashlib+import os+import re+import tempfile+import uuid+from collections.abc import AsyncIterator+from contextlib import asynccontextmanager+from pathlib import Path+from typing import Any++import httpx+from fastapi import FastAPI, Request, Response+from fastapi.responses import FileResponse, JSONResponse+from pydantic import BaseModel, ValidationError+from starlette.datastructures import UploadFile+from starlette.exceptions import HTTPException+from starlette.requests import ClientDisconnect+from starlette.types import ASGIApp, Receive, Scope, Send++from ...task_types import TASK_OUTPUT_FORMATS+from .models import GatewayError, ParseOptions, Record+from .service import FileParseService+from .settings import FileParseSettings++REQUEST_ID = re.compile(r"^[A-Za-z0-9_-]{1,64}$")+MIME_TYPES = {+ "zip": "application/zip",+ "docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",+ "html": "text/html",+}+++class StatusResponse(BaseModel):+ """查询与超时响应共享结构,创建尚未确认时 task_id 为空。"""++ request_id: str+ task_id: str | None+ status: str+ status_url: str+ result_url: str | None+ output_formats: list[str]+ error: dict[str, Any] | None+++class ReceiveTimeoutMiddleware:+ """对接收请求体的每次等待设置独立空闲超时。"""++ def __init__(self, app: ASGIApp, timeout: float) -> None:+ """记录传输预算,不限制正常持续传输大文件的总时长。"""+ self.app = app+ self.timeout = timeout++ async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:+ """仅约束 POST 请求体,响应等待期间的断连观察不应用传输超时。"""+ body_finished = False++ async def timed_receive() -> dict[str, Any]:+ """在最后一块请求体到达后恢复普通 receive 行为。"""+ nonlocal body_finished+ try:+ if body_finished:+ return await receive()+ message = await asyncio.wait_for(receive(), self.timeout)+ except asyncio.TimeoutError as exc:+ raise GatewayError("upload_receive_timeout", "Receiving the file timed out", 408) from exc+ if message["type"] == "http.request" and not message.get("more_body", False):+ body_finished = True+ return message++ await self.app(scope, timed_receive if scope["type"] == "http" and scope.get("method") == "POST" else receive, send)+++class TemporaryFileResponse(FileResponse):+ """无论发送完成、客户端断连还是取消,都释放已验证的结果临时文件。"""++ async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:+ """用 finally 兜底清理,避免只依赖成功发送后的后台回调。"""+ try:+ await super().__call__(scope, receive, send)+ finally:+ Path(self.path).unlink(missing_ok=True)+++def _request_schema() -> dict[str, Any]:+ """为手动严格解析的 multipart 请求提供完整 OpenAPI 描述。"""+ properties = ParseOptions.model_json_schema()["properties"]+ properties.update(+ {+ "file": {"type": "string", "format": "binary"},+ "request_id": {"type": "string", "pattern": REQUEST_ID.pattern},+ "wait_timeout": {"type": "integer", "minimum": 0, "maximum": 3600, "default": 3600},+ }+ )+ properties["output_formats"] = {+ "type": "array",+ "items": {"type": "string", "enum": list(TASK_OUTPUT_FORMATS)},+ "minItems": 1,+ "default": ["default"],+ }+ return {+ "requestBody": {+ "required": True,+ "content": {+ "multipart/form-data": {+ "schema": {"type": "object", "properties": properties, "required": ["file"], "additionalProperties": False},+ "encoding": {"output_formats": {"style": "form", "explode": True}},+ }+ },+ }+ }+++async def _receive_file(request: Request, service: FileParseService) -> tuple[Record, int]:+ """严格接收单文件 multipart 并持久化源字节,完整接受后才启动上游工作。"""+ if request.query_params:+ raise GatewayError("invalid_request", "Parameters must be supplied as multipart fields", 400)+ if request.headers.get("content-type", "").split(";", 1)[0].strip().lower() != "multipart/form-data":+ raise GatewayError("unsupported_media_type", "Expected multipart/form-data", 415)+ allowed = {"file", "request_id", "wait_timeout", "tier", "ocr_mode", "page_range", "output_formats"}+ source: Path | None = None+ accepted = False+ try:+ async with request.form(max_files=1, max_fields=32, max_part_size=65536) as form:+ if set(form) - allowed or any(len(form.getlist(key)) != 1 for key in form if key != "output_formats"):+ raise GatewayError("invalid_request", "Unknown or repeated scalar multipart fields", 400)+ upload = form.get("file")+ if not isinstance(upload, UploadFile) or any(+ not isinstance(value, str) for key, value in form.multi_items() if key != "file"+ ):+ raise GatewayError("invalid_request", "Exactly one file and textual parameters are required", 400)+ filename = (upload.filename or "").replace("\\", "/").rsplit("/", 1)[-1]+ if filename in {"", ".", ".."} or len(filename) > 1024 or any(ord(char) < 32 for char in filename):+ raise GatewayError("invalid_request", "A valid filename is required", 400)+ request_id = str(form.get("request_id", uuid.uuid4().hex))+ if not REQUEST_ID.fullmatch(request_id):+ raise GatewayError("invalid_request", "Invalid request_id", 400)+ request.state.request_id = request_id+ try:+ wait_timeout = int(str(form.get("wait_timeout", "3600")))+ if not 0 <= wait_timeout <= 3600:+ raise ValueError("wait_timeout out of range")+ options = ParseOptions.model_validate(+ {+ **{key: form[key] for key in ("tier", "ocr_mode", "page_range") if key in form},+ "output_formats": form.getlist("output_formats") if "output_formats" in form else ["default"],+ }+ )+ except (ValueError, ValidationError) as exc:+ raise GatewayError("invalid_request", "Invalid parsing parameters or wait_timeout", 400) from exc+ descriptor, name = tempfile.mkstemp(prefix="source-", dir=service.store.sources)+ source = Path(name)+ size, hasher = 0, hashlib.sha256()+ with os.fdopen(descriptor, "wb") as destination:+ while chunk := await upload.read(1024 * 1024):+ size += len(chunk)+ hasher.update(chunk)+ await asyncio.to_thread(destination.write, chunk)+ destination.flush()+ await asyncio.to_thread(os.fsync, destination.fileno())+ if size == 0:+ raise GatewayError("invalid_request", "The file must not be empty", 400)+ operation = asyncio.create_task(service.accept(request_id, filename, size, hasher.hexdigest(), options, source))+ try:+ record = await asyncio.shield(operation)+ except asyncio.CancelledError:+ await operation+ accepted = True+ raise+ accepted = True+ return record, wait_timeout+ except HTTPException as exc:+ raise GatewayError("invalid_request", "Invalid multipart upload or more than one file", 400) from exc+ except ClientDisconnect as exc:+ raise GatewayError("upload_interrupted", "Client disconnected before the file was accepted", 400) from exc+ except OSError as exc:+ raise GatewayError("storage_error", "Cannot retain the uploaded file", 500) from exc+ finally:+ if source is not None and not accepted:+ source.unlink(missing_ok=True)+++async def _attachment(service: FileParseService, record: Record) -> TemporaryFileResponse:+ """获取已验证产物并设置可直接下载的文件名、类型和任务标识。"""+ try:+ path = await service.artifact(record)+ except OSError as exc:+ raise GatewayError("storage_error", "Cannot save the downloaded result", 500) from exc+ formats = record.options["output_formats"]+ extension = formats[0] if len(formats) == 1 and formats[0] in {"docx", "html"} else "zip"+ task = record.task+ assert task is not None+ return TemporaryFileResponse(+ path,+ filename=f"{Path(record.filename).stem}.{extension}",+ media_type=MIME_TYPES[extension],+ headers={"X-Request-ID": record.request_id, "X-Task-ID": task.task_id},+ )+++async def _finish_request(request: Request, service: FileParseService, record: Record, timeout: int) -> Response:+ """等待预算只包围排队、解析及导出,结果下载使用独立传输预算。"""+ if await service.wait(record, timeout):+ return await _attachment(service, record)+ payload = service.payload(record, str(request.base_url))+ headers = {"Location": payload["status_url"], "X-Request-ID": record.request_id, "Retry-After": "2"}+ if payload["task_id"] is not None:+ headers["X-Task-ID"] = payload["task_id"]+ return JSONResponse(payload, status_code=202, headers=headers)+++async def _until_disconnect(request: Request, operation: asyncio.Task[Response]) -> Response:+ """客户端离开时只停止该连接的等待或下载,不传播到托管上传任务。"""++ async def disconnected() -> None:+ """在请求体读取完毕后等待 ASGI 的真实断连消息。"""+ while (await request.receive())["type"] != "http.disconnect":+ pass++ watcher = asyncio.create_task(disconnected())+ try:+ done, _ = await asyncio.wait({operation, watcher}, return_when=asyncio.FIRST_COMPLETED)+ if operation in done:+ return await operation+ return Response(status_code=499)+ finally:+ watcher.cancel()+ if not operation.done():+ operation.cancel()+ await asyncio.gather(watcher, operation, return_exceptions=True)+++def create_app(settings: FileParseSettings, *, transport: httpx.AsyncBaseTransport | None = None) -> FastAPI:+ """构建独立包装服务,生命周期内创建数据库和 HTTP 客户端。"""++ @asynccontextmanager+ async def lifespan(application: FastAPI) -> AsyncIterator[None]:+ """启动恢复、后台观察和关闭清理均由应用生命周期管理。"""+ service = FileParseService(settings, transport)+ application.state.service = service+ try:+ await service.start()+ yield+ finally:+ await service.close()++ application = FastAPI(title="MinerU File Parse", version="1", lifespan=lifespan)+ application.add_middleware(ReceiveTimeoutMiddleware, timeout=settings.transfer_timeout)++ @application.exception_handler(GatewayError)+ async def gateway_error(request: Request, exc: GatewayError) -> JSONResponse:+ """错误响应保留已接受请求的查询链接,但不暴露上游凭据及签名链接。"""+ payload: dict[str, Any] = {}+ service = request.app.state.service+ request_id = getattr(request.state, "request_id", None)+ record = service.records.get(request_id)+ if record is not None:+ payload.update(service.payload(record, str(request.base_url)))+ payload["error"] = exc.detail()+ return JSONResponse(payload, status_code=exc.status_code)++ @application.post(+ "/v1/file_parse",+ openapi_extra=_request_schema(),+ response_model=None,+ response_class=Response,+ responses={+ 200: {+ "description": "Result attachment",+ "content": {mime: {"schema": {"type": "string", "format": "binary"}} for mime in MIME_TYPES.values()},+ },+ 202: {"model": StatusResponse, "description": "Accepted; query status_url to continue"},+ },+ )+ async def file_parse(request: Request) -> Response:+ """接收一个文件并同步等待,调用方可通过 request_id 恢复已有任务。"""+ service = request.app.state.service+ record, timeout = await _receive_file(request, service)+ return await _until_disconnect(request, asyncio.create_task(_finish_request(request, service, record, timeout)))++ @application.get("/v1/file_parse/{request_id}", response_model=StatusResponse)+ async def status(request_id: str, request: Request) -> dict[str, Any]:+ """查询已有请求,创建结果未知时返回明确状态而不发起新的任务。"""+ request.state.request_id = request_id+ service = request.app.state.service+ record = service.get(request_id)+ if record.task is not None:+ await service.refresh(record)+ return service.payload(record, str(request.base_url))++ @application.get("/v1/file_parse/{request_id}/result", response_class=FileResponse)+ async def result(request_id: str, request: Request) -> Response:+ """使用当前有效的上游链接获取结果,断连时释放本次下载临时文件。"""+ request.state.request_id = request_id+ service = request.app.state.service+ record = service.get(request_id)+ if record.task is None:+ service.raise_record_error(record)+ return await _until_disconnect(request, asyncio.create_task(_attachment(service, record)))++ return application+++__all__ = ["StatusResponse", "create_app"]diff --git a/mineru/kit/file_parse/client.py b/mineru/kit/file_parse/client.pynew file mode 100644index 00000000..d1654bae--- /dev/null+++ b/mineru/kit/file_parse/client.py@@ -0,0 +1,290 @@+"""只通过共同 Tasks HTTP 契约工作的流式客户端。"""++from __future__ import annotations++import asyncio+import json+import time+import zipfile+from collections.abc import AsyncIterator, Mapping+from pathlib import Path+from typing import Any+from urllib.parse import urljoin++import httpx+from pydantic import ValidationError++from .models import GatewayError, Record, RemoteTask+from .settings import FileParseSettings, origin++CHUNK_BYTES = 1024 * 1024+MAX_JSON_BYTES = 4 * 1024 * 1024+RETRYABLE_HTTP = frozenset({408, 429, 500, 502, 503, 504})+REDIRECTS = frozenset({301, 302, 303, 307, 308})+++async def file_chunks(path: Path) -> AsyncIterator[bytes]:+ """从磁盘分块读取上传内容,不在事件循环中执行大块文件 I/O。"""+ with path.open("rb") as source:+ while chunk := await asyncio.to_thread(source.read, CHUNK_BYTES):+ yield chunk+++def validate_artifact(path: Path, formats: list[str]) -> None:+ """校验下载的基本格式和 ZIP 完整性,不解压文件或重新生成产物。"""+ try:+ if path.stat().st_size == 0:+ raise ValueError("Empty artifact")+ if formats == ["html"]:+ with path.open("r", encoding="utf-8-sig") as source:+ prefix = source.read(65536).lower()+ if "<html" not in prefix:+ raise ValueError("Expected an HTML document")+ else:+ with zipfile.ZipFile(path) as archive:+ if archive.testzip() is not None:+ raise ValueError("Corrupt ZIP member")+ if formats == ["docx"] and "word/document.xml" not in archive.namelist():+ raise ValueError("Expected a DOCX document")+ except (ValueError, UnicodeError, zipfile.BadZipFile, RuntimeError) as exc:+ raise GatewayError("invalid_result", "Upstream returned an invalid or incomplete result", retryable=True) from exc+++class TaskClient:+ """所有请求显式选择认证;创建任务始终只发送一次。"""++ def __init__(self, settings: FileParseSettings, transport: httpx.AsyncBaseTransport | None = None) -> None:+ """创建轻量连接池,测试可以注入传输而不替换业务实现。"""+ self.settings = settings+ self.origin = origin(settings.api_url)+ self.client = httpx.AsyncClient(+ timeout=httpx.Timeout(settings.request_timeout, connect=settings.connect_timeout),+ follow_redirects=False,+ transport=transport,+ )+ self.transfer_timeout = httpx.Timeout(settings.transfer_timeout, connect=settings.connect_timeout)+ self.retry_delay = 1.0++ async def close(self) -> None:+ """释放上游连接池,停止服务时不发送任务取消请求。"""+ await self.client.aclose()++ def resolve(self, url: str, *, base: str | None = None, metadata: bool = False) -> str:+ """保留签名查询串,元数据查询必须留在已配置的 API 同源地址。"""+ resolved = urljoin(base or self.settings.api_url + "/", url)+ try:+ target = origin(resolved)+ except ValueError as exc:+ raise GatewayError("upstream_protocol_error", "Upstream returned an invalid HTTP URL") from exc+ if metadata and target != self.origin:+ raise GatewayError("upstream_protocol_error", "Task status URL must use the configured API origin")+ return resolved++ def headers(self, url: str, supplied: Mapping[str, str] | None = None) -> dict[str, str]:+ """只在 API 同源请求中添加密钥,保留存储服务显式要求的上传请求头。"""+ try:+ headers = httpx.Headers(supplied or {})+ except (UnicodeError, ValueError) as exc:+ raise GatewayError("upstream_protocol_error", "Upstream returned invalid upload headers") from exc+ if any("\r" in key + value or "\n" in key + value for key, value in headers.items()):+ raise GatewayError("upstream_protocol_error", "Upstream returned invalid upload headers")+ token = f"Bearer {self.settings.api_key}" if self.settings.api_key else None+ if origin(url) == self.origin:+ if token is not None and "authorization" not in headers:+ headers["Authorization"] = token+ elif token is not None and headers.get("Authorization") == token:+ raise GatewayError("upstream_protocol_error", "Refusing to forward the API key to another origin")+ return dict(headers)++ async def _read_metadata(self, response: httpx.Response) -> bytes:+ """限制元数据响应的总大小,避免意外把产物当成 JSON 缓冲。"""+ content = bytearray()+ async for chunk in response.aiter_bytes():+ content.extend(chunk)+ if len(content) > MAX_JSON_BYTES:+ raise GatewayError("upstream_protocol_error", "Upstream metadata response is too large")+ return bytes(content)++ def _http_error(self, status: int, body: bytes) -> GatewayError:+ """保留上游结构化错误码,同时区分配置认证失败和可重试响应。"""+ code, message = "upstream_http_error", f"Upstream returned HTTP {status}"+ try:+ payload = json.loads(body)+ detail = payload.get("error") if isinstance(payload, dict) else None+ if isinstance(detail, dict):+ code = str(detail.get("code") or code)+ message = str(detail.get("message") or message)+ except (ValueError, UnicodeError):+ pass+ return GatewayError(code, message, retryable=status in RETRYABLE_HTTP, upstream_status=status)++ def _snapshot(self, body: bytes, expected: RemoteTask | None, formats: list[str]) -> RemoteTask:+ """校验任务 ID、单文件映射和输出格式,拒绝不匹配的任务响应。"""+ try:+ task = RemoteTask.model_validate_json(body)+ if expected is not None and task.task_id != expected.task_id:+ raise ValueError("Task identity changed")+ if list(dict.fromkeys(task.output_formats)) != formats:+ raise ValueError("Task output formats changed")+ task.status_url = self.resolve(task.status_url, metadata=True)+ for link in task.uploads:+ link.upload_url = self.resolve(link.upload_url)+ if task.files[0].result_url:+ task.files[0].result_url = self.resolve(task.files[0].result_url)+ return task+ except (ValidationError, ValueError) as exc:+ raise GatewayError("upstream_protocol_error", "Upstream returned inconsistent task metadata") from exc++ async def _json_request(+ self,+ method: str,+ url: str,+ formats: list[str],+ *,+ expected: RemoteTask | None = None,+ payload: dict[str, Any] | None = None,+ ) -> RemoteTask:+ """安全查询有界重试,创建响应丢失或内容不可用则标记结果不确定。"""+ for attempt in range(1 if method == "POST" else 3):+ try:+ async with self.client.stream(method, url, headers=self.headers(url), json=payload) as response:+ body = await self._read_metadata(response)+ if response.status_code not in ({202} if method == "POST" else {200}):+ error = self._http_error(response.status_code, body)+ if method == "POST" and (response.status_code >= 500 or response.status_code == 408):+ raise GatewayError("submission_unknown", "Task creation outcome is unknown; not resubmitting")+ raise error+ return self._snapshot(body, expected, formats)+ except (httpx.HTTPError, GatewayError) as exc:+ error = (+ exc+ if isinstance(exc, GatewayError)+ else GatewayError("upstream_unreachable", "Cannot communicate with the upstream service", retryable=True)+ )+ if method == "POST":+ if isinstance(exc, httpx.HTTPError) or error.code == "upstream_protocol_error":+ error = GatewayError("submission_unknown", "Task creation outcome is unknown; not resubmitting")+ raise error from exc+ if attempt == 2 or not error.retryable:+ raise error from exc+ await asyncio.sleep(self.retry_delay * (attempt + 1))+ raise AssertionError("unreachable")++ async def create(self, record: Record) -> RemoteTask:+ """只提交一条清单,hash 用于上游缓存预检及上传完整性检查。"""+ options = {key: value for key, value in record.options.items() if value is not None}+ payload = {+ **options,+ "manifest": [+ {+ "client_file_id": "file",+ "filename": record.filename,+ "bytes": record.size,+ "sha256sum": record.sha256sum,+ }+ ],+ }+ return await self._json_request(+ "POST", self.settings.api_url + "/v1/tasks", record.options["output_formats"], payload=payload+ )++ async def query(self, task: RemoteTask) -> RemoteTask:+ """按服务给出的地址查询原任务,失败不会触发创建操作。"""+ return await self._json_request("GET", task.status_url, task.output_formats, expected=task)++ async def upload(self, task: RemoteTask, path: Path, size: int) -> RemoteTask:+ """上传失败先查询确认,最多重传三次,绝不续期或创建替代任务。"""+ for attempt in range(3):+ item = task.files[0]+ if item.upload_status == "ready" or item.status in {"completed", "failed", "canceled"}:+ return task+ if not task.uploads:+ raise GatewayError("upstream_protocol_error", "Pending file has no upload link")+ link = task.uploads[0]+ if link.expires_at <= time.time():+ raise GatewayError("upload_expired", "The task upload link has expired", 410)+ try:+ headers = self.headers(link.upload_url, link.upload_headers)+ headers["content-length"] = str(size)+ async with self.client.stream(+ link.upload_method,+ link.upload_url,+ headers=headers,+ content=file_chunks(path),+ timeout=self.transfer_timeout,+ ) as response:+ body = await self._read_metadata(response)+ if not 200 <= response.status_code < 300:+ raise self._http_error(response.status_code, body)+ return await self.query(task)+ except (httpx.HTTPError, GatewayError) as exc:+ error = (+ exc+ if isinstance(exc, GatewayError)+ else GatewayError("upload_interrupted", "Upstream upload was interrupted", retryable=True)+ )+ task = await self.query(task)+ item = task.files[0]+ if item.upload_status == "ready" or item.status in {"completed", "failed", "canceled"}:+ return task+ if attempt == 2 or not error.retryable:+ raise error from exc+ await asyncio.sleep(self.retry_delay * (attempt + 1))+ raise AssertionError("unreachable")++ async def _download_once(self, url: str, path: Path) -> None:+ """逐跳重新决定认证并流式写盘,在完整接收前不向调用者发送成功响应。"""+ for _ in range(6):+ async with self.client.stream(+ "GET",+ url,+ headers=self.headers(url),+ timeout=self.transfer_timeout,+ ) as response:+ if response.status_code in REDIRECTS:+ location = response.headers.get("location")+ if not location:+ raise GatewayError("upstream_protocol_error", "Download redirect has no Location")+ url = self.resolve(location, base=url)+ continue+ if response.status_code != 200:+ raise self._http_error(response.status_code, await self._read_metadata(response))+ size = 0+ with path.open("wb") as target:+ async for chunk in response.aiter_bytes(CHUNK_BYTES):+ await asyncio.to_thread(target.write, chunk)+ size += len(chunk)+ length = response.headers.get("content-length")+ if length is not None and response.headers.get("content-encoding", "identity") == "identity":+ try:+ complete = int(length) == size+ except ValueError:+ complete = False+ if not complete:+ raise GatewayError("download_incomplete", "Result download is incomplete", retryable=True)+ return+ raise GatewayError("upstream_protocol_error", "Too many result download redirects")++ async def download(self, task: RemoteTask, path: Path) -> RemoteTask:+ """下载错误或签名失效时查询原任务,再对可恢复错误执行有界重试。"""+ for attempt in range(3):+ if task.files[0].status != "completed" or not task.files[0].result_url:+ raise GatewayError("result_not_ready", "The result is not ready", 409)+ try:+ await self._download_once(task.files[0].result_url, path)+ await asyncio.to_thread(validate_artifact, path, task.output_formats)+ return task+ except (httpx.HTTPError, GatewayError) as exc:+ error = (+ exc+ if isinstance(exc, GatewayError)+ else GatewayError("download_interrupted", "Result download was interrupted", retryable=True)+ )+ if attempt == 2 or (not error.retryable and error.upstream_status not in {401, 403, 404}):+ raise error from exc+ task = await self.query(task)+ await asyncio.sleep(self.retry_delay * (attempt + 1))+ raise AssertionError("unreachable")+++__all__ = ["TaskClient", "validate_artifact"]diff --git a/mineru/kit/file_parse/models.py b/mineru/kit/file_parse/models.pynew file mode 100644index 00000000..ec2aef0a--- /dev/null+++ b/mineru/kit/file_parse/models.py@@ -0,0 +1,136 @@+"""同步包装服务的请求、远端快照和持久化记录。"""++from __future__ import annotations++import hashlib+import json+from dataclasses import asdict, dataclass+from typing import Any, Literal++from pydantic import BaseModel, Field, model_validator++from ...task_types import TaskOutputFormat++TERMINAL = frozenset({"completed", "failed", "canceled", "submission_unknown"})+FileStatus = Literal["waiting_upload", "queued", "running", "completed", "failed", "canceled"]+++class GatewayError(Exception):+ """保留结构化错误,避免把含签名地址或密钥的传输异常直接返回。"""++ def __init__(+ self,+ code: str,+ message: str,+ status_code: int = 502,+ *,+ retryable: bool = False,+ upstream_status: int | None = None,+ ) -> None:+ """区分调用参数、远端业务错误与可安全重试的传输错误。"""+ super().__init__(message)+ self.code = code+ self.message = message+ self.status_code = status_code+ self.retryable = retryable+ self.upstream_status = upstream_status++ def detail(self) -> dict[str, Any]:+ """生成可持久化且不包含底层请求对象的错误信息。"""+ return {"code": self.code, "message": self.message, "upstream_status": self.upstream_status}+++class ParseOptions(BaseModel):+ """保持与 Tasks 一致的解析参数,等待时间不属于解析身份。"""++ tier: Literal["flash", "basic", "standard", "advanced"] | None = None+ ocr_mode: Literal["auto", "txt", "ocr"] = "auto"+ page_range: str = ""+ output_formats: list[TaskOutputFormat] = Field(default_factory=lambda: ["default"], min_length=1)++ @model_validator(mode="after")+ def normalize(self) -> ParseOptions:+ """按首次出现去重格式,并统一整本页范围的等价写法。"""+ self.output_formats = list(dict.fromkeys(self.output_formats))+ self.page_range = self.page_range.strip()+ if self.page_range.lower() == "all":+ self.page_range = ""+ return self+++class RemoteFile(BaseModel):+ """仅消费共同 Tasks 契约,不依赖本地 Files 或 Job 实现。"""++ client_file_id: str+ status: FileStatus+ upload_status: Literal["pending", "ready", "not_required", "expired"]+ result_url: str | None = None+ error: dict[str, Any] | None = None+++class UploadLink(BaseModel):+ """保留原始上传地址、请求头和固定截止时间。"""++ client_file_id: str+ upload_url: str+ upload_method: Literal["PUT"]+ upload_headers: dict[str, str]+ expires_at: int+++class RemoteTask(BaseModel):+ """校验单文件任务的身份和产物契约,允许上游添加其他元数据。"""++ task_id: str = Field(min_length=1)+ status: Literal["waiting_upload", "queued", "running", "completed", "partial", "failed", "canceled"]+ status_url: str = Field(min_length=1)+ files: list[RemoteFile] = Field(min_length=1, max_length=1)+ uploads: list[UploadLink] = Field(default_factory=list, max_length=1)+ output_formats: list[TaskOutputFormat] = Field(min_length=1)++ @model_validator(mode="after")+ def validate_file(self) -> RemoteTask:+ """确保返回的文件和链接仍对应本次唯一清单项。"""+ item = self.files[0]+ if item.client_file_id != "file" or any(link.client_file_id != "file" for link in self.uploads):+ raise ValueError("Unexpected client_file_id")+ if item.status == "completed" and not item.result_url:+ raise ValueError("Completed file has no result_url")+ if item.status != "completed" and item.result_url is not None:+ raise ValueError("Unfinished file has a result_url")+ return self+++@dataclass+class Record:+ """保存恢复所需的最小状态;密钥仅来自启动配置,不写入记录。"""++ request_id: str+ filename: str+ size: int+ sha256sum: str+ options: dict[str, Any]+ source_path: str | None+ fingerprint: str+ status: str = "prepared"+ remote: dict[str, Any] | None = None+ error: dict[str, Any] | None = None+ terminal_at: float | None = None++ @property+ def task(self) -> RemoteTask | None:+ """读取已校验远端快照,空值表示尚未确认创建结果。"""+ return RemoteTask.model_validate(self.remote) if self.remote is not None else None++ def to_json(self) -> str:+ """把完整记录序列化为 SQLite 单行快照。"""+ return json.dumps(asdict(self), ensure_ascii=False)+++def fingerprint(filename: str, size: int, sha256sum: str, options: ParseOptions) -> str:+ """用文件内容、逻辑名称与归一化参数区分同 ID 的冲突提交。"""+ value = {"filename": filename, "size": size, "sha256sum": sha256sum, "options": options.model_dump()}+ return hashlib.sha256(json.dumps(value, ensure_ascii=False, sort_keys=True).encode()).hexdigest()+++__all__ = ["GatewayError", "ParseOptions", "Record", "RemoteTask", "TERMINAL", "UploadLink", "fingerprint"]diff --git a/mineru/kit/file_parse/service.py b/mineru/kit/file_parse/service.pynew file mode 100644index 00000000..fb63d298--- /dev/null+++ b/mineru/kit/file_parse/service.py@@ -0,0 +1,332 @@+"""把请求连接与持久化的 Tasks 上传、查询生命周期分离。"""++from __future__ import annotations++import asyncio+import logging+import os+import tempfile+import time+from dataclasses import replace+from pathlib import Path+from typing import Any++import httpx++from .client import TaskClient+from .models import TERMINAL, GatewayError, ParseOptions, Record, RemoteTask, fingerprint+from .settings import FileParseSettings+from .store import RecordStore++logger = logging.getLogger(__name__)+++class FileParseService:+ """单进程管理恢复记录;每个请求只有一个独立的上传准备任务。"""++ def __init__(self, settings: FileParseSettings, transport: httpx.AsyncBaseTransport | None = None) -> None:+ """初始化无副作用的管理状态,网络连接在应用启动后使用。"""+ self.settings = settings+ self.store = RecordStore(settings)+ self.client = TaskClient(settings, transport)+ self.records: dict[str, Record] = {}+ self.preparations: dict[str, asyncio.Task[None]] = {}+ self.changes: dict[str, asyncio.Event] = {}+ self.query_locks: dict[str, asyncio.Lock] = {}+ self.state_locks: dict[str, asyncio.Lock] = {}+ self._accept_lock = asyncio.Lock()+ self._query_limit = asyncio.Semaphore(16)+ self._monitor: asyncio.Task[None] | None = None++ async def start(self) -> None:+ """恢复已有任务;曾经开始但未确认的创建请求不能自动重发。"""+ for record in await self.store.open():+ self.records[record.request_id] = record+ if record.status == "submitting" and record.remote is None:+ await self._remember_error(+ record, GatewayError("submission_unknown", "Task creation outcome is unknown; not resubmitting")+ )+ elif record.status not in TERMINAL:+ self._prepare_in_background(record)+ await self.collect()+ self._monitor = asyncio.create_task(self._monitor_records(), name="file-parse-monitor")++ async def close(self) -> None:+ """停止本地协程并保存已确认的任务状态,不取消上游解析。"""+ tasks = [*self.preparations.values()]+ if self._monitor is not None:+ tasks.append(self._monitor)+ for task in tasks:+ task.cancel()+ await asyncio.gather(*tasks, return_exceptions=True)+ await self.client.close()+ await self.store.close()++ def _changed(self, record: Record) -> None:+ """唤醒正在等待该请求状态变化的连接。"""+ self.changes.setdefault(record.request_id, asyncio.Event()).set()++ def _prepare_in_background(self, record: Record) -> None:+ """把任务创建及上传托管到应用,连接取消不会传播到这里。"""+ previous = self.preparations.get(record.request_id)+ if previous is None or previous.done():+ self.preparations[record.request_id] = asyncio.create_task(+ self._prepare(record), name=f"file-parse-prepare-{record.request_id}"+ )++ async def accept(+ self,+ request_id: str,+ filename: str,+ size: int,+ sha256sum: str,+ options: ParseOptions,+ source: Path,+ ) -> Record:+ """原子接受或复用请求,同 ID 不同内容不会替换原任务或源文件。"""+ identity = fingerprint(filename, size, sha256sum, options)+ async with self._accept_lock:+ existing = self.records.get(request_id)+ if existing is not None and self._expired(existing):+ await self._remove_record(existing)+ existing = None+ if existing is not None:+ source.unlink(missing_ok=True)+ if existing.fingerprint != identity:+ raise GatewayError("request_id_conflict", "request_id already identifies a different request", 409)+ if existing.status not in TERMINAL:+ self._prepare_in_background(existing)+ return existing+ record = Record(request_id, filename, size, sha256sum, options.model_dump(), str(source), identity)+ await self.store.save(record)+ self.records[request_id] = record+ self._prepare_in_background(record)+ return record++ def get(self, request_id: str) -> Record:+ """仅在当前上游命名空间内查找请求;过期记录等同于不存在。"""+ record = self.records.get(request_id)+ if record is None or self._expired(record):+ raise GatewayError("request_not_found", "Request was not found or its recovery record expired", 404)+ return record++ def _expired(self, record: Record) -> bool:+ """终态首次观察时间决定保留期限,重复查询不续期。"""+ return bool(+ self.settings.retention_seconds+ and record.terminal_at is not None+ and record.terminal_at + self.settings.retention_seconds <= time.time()+ )++ async def _save_and_cleanup(self, record: Record, updated: Record, *, remove_source: bool) -> None:+ """持久化和清理完成后才发布新状态,避免并发查询提前观察完成状态。"""+ source = updated.source_path if remove_source else None+ if remove_source:+ updated.source_path = None+ await self.store.save(updated)+ try:+ if source:+ Path(source).unlink(missing_ok=True)+ finally:+ # 即使文件清理失败,也不能丢掉已经持久化的远端 task_id。+ record.remote = updated.remote+ record.status = updated.status+ record.source_path = updated.source_path+ record.error = updated.error+ record.terminal_at = updated.terminal_at+ self._changed(record)++ async def _apply(self, record: Record, task: RemoteTask) -> None:+ """更新已确认任务快照,并避免旧轮询覆盖较新的终态。"""+ async with self.state_locks.setdefault(record.request_id, asyncio.Lock()):+ if self.records.get(record.request_id) is not record:+ return+ item = task.files[0]+ if record.remote is not None and record.status in TERMINAL and item.status not in TERMINAL:+ return+ updated = replace(record, remote=task.model_dump(), status=item.status)+ if item.upload_status == "ready" or item.status in TERMINAL:+ updated.error = item.error+ if item.status in {"failed", "canceled"} and updated.error is None:+ updated.error = {"code": f"task_{item.status}", "message": f"Upstream task {item.status}"}+ if updated.status in TERMINAL and updated.terminal_at is None:+ updated.terminal_at = time.time()+ await self._save_and_cleanup(+ record, updated, remove_source=item.upload_status == "ready" or item.status in TERMINAL+ )++ async def _remember_error(self, record: Record, error: GatewayError) -> None:+ """无任务 ID 的失败是本地终态;已知任务的传输错误保留后续查询能力。"""+ async with self.state_locks.setdefault(record.request_id, asyncio.Lock()):+ if self.records.get(record.request_id) is not record:+ return+ updated = replace(record, error={**error.detail(), "http_status": error.status_code})+ remove_source = False+ if updated.remote is None:+ updated.status = "submission_unknown" if error.code == "submission_unknown" else "failed"+ updated.terminal_at = updated.terminal_at or time.time()+ remove_source = True+ elif error.upstream_status in {404, 410}:+ updated.status = "failed"+ updated.terminal_at = updated.terminal_at or time.time()+ remove_source = True+ await self._save_and_cleanup(record, updated, remove_source=remove_source)++ async def _prepare(self, record: Record) -> None:+ """创建和上传最多执行一条流水线,上传阶段错误仍允许查询已有任务。"""+ try:+ record.error = None+ if record.remote is None:+ record.status = "submitting"+ await self.store.save(record)+ task = await self.client.create(record)+ await self._apply(record, task)+ else:+ task = await self.refresh(record)+ if task.files[0].upload_status == "pending" and task.files[0].status not in TERMINAL:+ if record.source_path is None or not Path(record.source_path).is_file():+ raise GatewayError("source_unavailable", "The retained source is unavailable")+ task = await self.client.upload(task, Path(record.source_path), record.size)+ await self._apply(record, task)+ except GatewayError as exc:+ await self._remember_error(record, exc)+ except OSError:+ await self._remember_error(record, GatewayError("storage_error", "Cannot access the local source storage", 500))+ finally:+ self._changed(record)++ async def refresh(self, record: Record) -> RemoteTask:+ """序列化同一请求的状态查询,限制后台查询总并发。"""+ async with self.query_locks.setdefault(record.request_id, asyncio.Lock()):+ task = record.task+ if task is None:+ raise GatewayError("task_not_created", "No confirmed upstream task is available", 409)+ async with self._query_limit:+ current = await self.client.query(task)+ await self._apply(record, current)+ return current++ async def _refresh_safely(self, record: Record) -> None:+ """后台查询临时失败只更新错误,不使其他请求的轮询停止。"""+ try:+ await self.refresh(record)+ except GatewayError as exc:+ await self._remember_error(record, exc)++ async def _monitor_records(self) -> None:+ """持续观察已提交任务,使断连任务也能进入终态和开始保留期。"""+ while True:+ await asyncio.sleep(self.settings.poll_interval)+ records = [+ record+ for record in self.records.values()+ if record.remote is not None+ and record.status not in TERMINAL+ and (record.request_id not in self.preparations or self.preparations[record.request_id].done())+ ]+ try:+ results = await asyncio.gather(*(self._refresh_safely(record) for record in records), return_exceptions=True)+ for result in results:+ if isinstance(result, Exception):+ logger.error("File parse reconciliation error: %s", type(result).__name__)+ await self.collect()+ except Exception:+ logger.exception("File parse record reconciliation failed")++ async def collect(self) -> None:+ """回收超过保留期的终态记录及残留源文件,不触碰上游资源。"""+ async with self._accept_lock:+ for request_id, record in list(self.records.items()):+ operation = self.preparations.get(request_id)+ if not self._expired(record) or (operation is not None and not operation.done()):+ continue+ await self._remove_record(record)++ async def _remove_record(self, record: Record) -> None:+ """与状态提交串行回收记录,迟到的查询不能重新插入已过期的旧任务。"""+ request_id = record.request_id+ async with self.state_locks.setdefault(request_id, asyncio.Lock()):+ await self.store.delete(request_id)+ if record.source_path:+ Path(record.source_path).unlink(missing_ok=True)+ self.records.pop(request_id, None)+ self.preparations.pop(request_id, None)+ self.changes.pop(request_id, None)+ self.query_locks.pop(request_id, None)+ self.state_locks.pop(request_id, None)++ async def wait(self, record: Record, timeout: int) -> bool:+ """上游文件准备完成后才启动单调时钟预算,连接取消不取消上传任务。"""+ preparation = self.preparations.get(record.request_id)+ if preparation is not None:+ await asyncio.shield(preparation)+ changed = self.changes.setdefault(record.request_id, asyncio.Event())+ while True:+ changed.clear()+ self.raise_record_error(record)+ task = record.task+ if record.status in TERMINAL or (task is not None and task.files[0].upload_status == "ready"):+ break+ # 对象存储 PUT 成功后,网关可能稍后才确认 ready;这段等待仍属于上传准备。+ await changed.wait()+ if timeout == 0:+ return False+ deadline = asyncio.get_running_loop().time() + timeout+ while True:+ changed.clear()+ self.raise_record_error(record)+ if record.status == "completed":+ return True+ remaining = deadline - asyncio.get_running_loop().time()+ if remaining <= 0:+ return False+ try:+ await asyncio.wait_for(changed.wait(), remaining)+ except asyncio.TimeoutError:+ return record.status == "completed"++ def raise_record_error(self, record: Record) -> None:+ """把已知失败或传输异常转成结构化 HTTP 错误,保留记录供查询。"""+ if record.error is not None:+ default_status = 409 if record.status in {"canceled", "submission_unknown"} else 502+ raise GatewayError(+ str(record.error.get("code") or "task_failed"),+ str(record.error.get("message") or "Upstream task failed"),+ int(record.error.get("http_status") or default_status),+ upstream_status=record.error.get("upstream_status"),+ )++ def payload(self, record: Record, base_url: str) -> dict[str, Any]:+ """只返回包装服务的公共链接,隐藏上游上传地址和签名下载地址。"""+ status_url = f"{base_url.rstrip('/')}/v1/file_parse/{record.request_id}"+ task = record.task+ return {+ "request_id": record.request_id,+ "task_id": task.task_id if task else None,+ "status": record.status,+ "status_url": status_url,+ "result_url": status_url + "/result" if record.status == "completed" else None,+ "output_formats": record.options["output_formats"],+ "error": record.error,+ }++ async def artifact(self, record: Record) -> Path:+ """获取最新下载链接并验证临时产物,失败时立即清理临时文件。"""+ task = await self.refresh(record)+ if task.files[0].status in {"failed", "canceled"}:+ self.raise_record_error(record)+ if task.files[0].status != "completed":+ raise GatewayError("result_not_ready", "The result is not ready", 409)+ descriptor, name = tempfile.mkstemp(prefix="result-", dir=self.store.results)+ os.close(descriptor)+ path = Path(name)+ try:+ task = await self.client.download(task, path)+ await self._apply(record, task)+ return path+ except BaseException:+ path.unlink(missing_ok=True)+ raise+++__all__ = ["FileParseService"]diff --git a/mineru/kit/file_parse/settings.py b/mineru/kit/file_parse/settings.pynew file mode 100644index 00000000..1e5d7d1e--- /dev/null+++ b/mineru/kit/file_parse/settings.py@@ -0,0 +1,62 @@+"""同步包装服务的显式启动配置。"""++from __future__ import annotations++import hashlib+import math+import os+from dataclasses import dataclass+from pathlib import Path+from urllib.parse import urlsplit+++def origin(url: str) -> tuple[str, str, int]:+ """严格比较协议、主机和端口,并拒绝 URL 内嵌凭据。"""+ parts = urlsplit(url)+ if parts.scheme not in {"http", "https"} or not parts.hostname or parts.username or parts.password:+ raise ValueError("Expected an HTTP(S) URL without embedded credentials")+ return parts.scheme.lower(), parts.hostname.lower(), parts.port or (443 if parts.scheme == "https" else 80)+++@dataclass(frozen=True)+class FileParseSettings:+ """只在创建配置实例时读取默认目录,不在模块导入时读取环境。"""++ api_url: str+ api_key: str | None = None+ state_dir: Path | None = None+ retention_seconds: int = 86400+ poll_interval: float = 2.0+ connect_timeout: float = 10.0+ request_timeout: float = 30.0+ transfer_timeout: float = 600.0++ def __post_init__(self) -> None:+ """校验地址和预算,禁止静默选择官方远程上游。"""+ normalized = self.api_url.rstrip("/")+ origin(normalized)+ parts = urlsplit(normalized)+ if parts.query or parts.fragment:+ raise ValueError("api_url must not contain a query or fragment")+ if self.api_key and any(character in self.api_key for character in "\r\n"):+ raise ValueError("api_key must not contain line breaks")+ object.__setattr__(self, "api_url", normalized)+ if self.state_dir is None:+ home = Path(os.environ.get("MINERU_HOME") or "~/.mineru").expanduser()+ object.__setattr__(self, "state_dir", home / "file-parse")+ else:+ object.__setattr__(self, "state_dir", Path(self.state_dir).expanduser().resolve())+ if type(self.retention_seconds) is not int or self.retention_seconds < 0:+ raise ValueError("retention_seconds must be a non-negative integer")+ for name in ("poll_interval", "connect_timeout", "request_timeout", "transfer_timeout"):+ value = getattr(self, name)+ if not math.isfinite(value) or value <= 0:+ raise ValueError(f"{name} must be positive and finite")++ @property+ def namespace(self) -> str:+ """按上游地址和认证隔离恢复记录,不存储明文密钥。"""+ return hashlib.sha256(f"{self.api_url}\0{self.api_key or ''}".encode()).hexdigest()+++__all__ = ["FileParseSettings", "origin"]diff --git a/mineru/kit/file_parse/store.py b/mineru/kit/file_parse/store.pynew file mode 100644index 00000000..822ff251--- /dev/null+++ b/mineru/kit/file_parse/store.py@@ -0,0 +1,85 @@+"""独立于 Doclib 的 SQLite 恢复记录与临时文件管理。"""++from __future__ import annotations++import asyncio+import json++import aiosqlite+from filelock import FileLock++from .models import Record+from .settings import FileParseSettings+++class RecordStore:+ """每个状态目录只允许一个进程使用,各上游配置拥有独立数据空间。"""++ def __init__(self, settings: FileParseSettings) -> None:+ """只保存路径和锁对象,实际创建资源由应用启动阶段负责。"""+ assert settings.state_dir is not None+ self.root = settings.state_dir+ self.directory = self.root / settings.namespace+ self.sources = self.directory / "sources"+ self.results = self.directory / "results"+ self.lock = FileLock(str(self.root / ".lock"), thread_local=False)+ self.connection: aiosqlite.Connection | None = None+ self._writes = asyncio.Lock()++ async def open(self) -> list[Record]:+ """获取独占锁并加载记录,启动时清理上次进程遗留的结果临时文件。"""+ self.root.mkdir(parents=True, exist_ok=True)+ self.lock.acquire(timeout=0)+ try:+ self.sources.mkdir(parents=True, exist_ok=True)+ self.results.mkdir(parents=True, exist_ok=True)+ for path in self.results.iterdir():+ if path.is_file():+ path.unlink()+ self.connection = await aiosqlite.connect(self.directory / "requests.sqlite3")+ await self.connection.execute("PRAGMA journal_mode=WAL")+ await self.connection.execute("PRAGMA synchronous=FULL")+ await self.connection.execute(+ "CREATE TABLE IF NOT EXISTS requests (request_id TEXT PRIMARY KEY, data TEXT NOT NULL)"+ )+ await self.connection.commit()+ async with self.connection.execute("SELECT data FROM requests") as cursor:+ records = [Record(**json.loads(row[0])) for row in await cursor.fetchall()]+ retained = {record.source_path for record in records if record.source_path}+ for path in self.sources.iterdir():+ if path.is_file() and str(path) not in retained:+ path.unlink()+ return records+ except BaseException:+ await self.close()+ raise++ async def save(self, record: Record) -> None:+ """原子提交单个记录快照,任务 ID 确认后立即落盘。"""+ assert self.connection is not None+ data = record.to_json()+ async with self._writes:+ await self.connection.execute(+ "INSERT INTO requests(request_id, data) VALUES (?, ?) ON CONFLICT(request_id) DO UPDATE SET data=excluded.data",+ (record.request_id, data),+ )+ await self.connection.commit()++ async def delete(self, request_id: str) -> None:+ """回收过期恢复记录,保留其他请求及其他上游空间。"""+ assert self.connection is not None+ async with self._writes:+ await self.connection.execute("DELETE FROM requests WHERE request_id=?", (request_id,))+ await self.connection.commit()++ async def close(self) -> None:+ """关闭数据库后释放目录独占锁,允许下次启动恢复。"""+ try:+ if self.connection is not None:+ await self.connection.close()+ self.connection = None+ finally:+ self.lock.release()+++__all__ = ["RecordStore"]diff --git a/mineru/kit/main.py b/mineru/kit/main.pyindex 57ebf608..48c59f4a 100644--- a/mineru/kit/main.py+++ b/mineru/kit/main.py@@ -10,12 +10,13 @@ from ..cli.version_command import show_version, version_cmdfrom ..utils.i18n import tfrom ..utils.logger import configure_global_log_levelfrom ..utils.stdio import configure_standard_streams-from .commands import api_server, models, parse, router, vlm_server, webui+from .commands import api_server, file_parse_server, models, parse, router, vlm_server, webuiTOP_LEVEL_COMMAND_ORDER = ["parse","webui","api-server",+ "file-parse-server","vlm-server","router","models",@@ -63,6 +64,9 @@ app.command(help=t("Start the local VLM server with OpenAI-compatible chat completions."),)(vlm_server.vlm_server_cmd)app.command("router", help=t("Start a standalone Router service exposing only the MinerU V1 API."))(router.router_cmd)+app.command("file-parse-server", help=t("Start the synchronous single-file Tasks wrapper service."))(+ file_parse_server.file_parse_server_cmd+)app.command("version", help=t("Print MinerU and Python versions."))(version_cmd)diff --git a/mineru/parser/api_server.py b/mineru/parser/api_server.pyindex 0f200036..fc385db9 100644--- a/mineru/parser/api_server.py+++ b/mineru/parser/api_server.py@@ -80,7 +80,7 @@ from .tier import (runtime_options_for_tier,)from .writer import DataWriter-from .task_outputs import TASK_OUTPUT_FORMATS, TaskOutputFormat+from ..task_types import TASK_OUTPUT_FORMATS, TaskOutputFormat_DEFAULT_API_SERVER_TIER: ServerTier = "standard"_MAX_INLINE_BYTES_DEFAULT = 1024 * 1024diff --git a/mineru/parser/task_api.py b/mineru/parser/task_api.pyindex 57e8e974..9966f863 100644--- a/mineru/parser/task_api.py+++ b/mineru/parser/task_api.py@@ -22,6 +22,7 @@ from starlette.requests import ClientDisconnectfrom ..errors import MineruErrorfrom ..filetypes import is_flash_only_parse_extension+from ..task_types import TaskOutputFormatfrom ..types import Tierfrom ..utils.async_utils import drain_future, run_syncfrom .api_server import (@@ -44,7 +45,6 @@ from .api_server import (submit_parse_job,)from .page_range import normalize_page_range_input-from .task_outputs import TaskOutputFormatMAX_TASK_FILES = 100TASK_UPLOAD_TTL = 3600diff --git a/mineru/parser/task_outputs.py b/mineru/parser/task_outputs.pyindex b66894a0..57cf914e 100644--- a/mineru/parser/task_outputs.py+++ b/mineru/parser/task_outputs.py@@ -5,13 +5,10 @@ from __future__ import annotationsimport jsonimport zipfilefrom pathlib import Path-from typing import Literal+from ..task_types import TASK_OUTPUT_FORMATS, TaskOutputFormatfrom .base import ParseResult-TaskOutputFormat = Literal["default", "docx", "html", "latex"]-TASK_OUTPUT_FORMATS: tuple[TaskOutputFormat, ...] = ("default", "docx", "html", "latex")-def build_task_output(result: ParseResult, formats: list[TaskOutputFormat], directory: Path) -> Path:"""一次物化并渲染所选格式,全部成功后才返回主产物路径。"""diff --git a/mineru/task_types.py b/mineru/task_types.pynew file mode 100644index 00000000..4995fb6b--- /dev/null+++ b/mineru/task_types.py@@ -0,0 +1,8 @@+"""Tasks 客户端与服务端共用的轻量格式定义,不导入解析或渲染实现。"""++from typing import Literal++TaskOutputFormat = Literal["default", "docx", "html", "latex"]+TASK_OUTPUT_FORMATS: tuple[TaskOutputFormat, ...] = ("default", "docx", "html", "latex")++__all__ = ["TASK_OUTPUT_FORMATS", "TaskOutputFormat"]diff --git a/mineru/utils/translations.py b/mineru/utils/translations.pyindex 133c6048..fc37fb68 100644--- a/mineru/utils/translations.py+++ b/mineru/utils/translations.py@@ -489,6 +489,10 @@ ZH_MESSAGES: dict[str, str] = {"Server host": "服务监听主机","Server port": "服务监听端口","Upload directory": "上传目录",+ "Start the synchronous single-file Tasks wrapper service.": "启动单文件同步 Tasks 包装服务。",+ "Upstream Tasks API base URL": "上游 Tasks API 基础 URL",+ "Upstream Bearer API key": "上游 Bearer API 密钥",+ "File parse recovery state directory": "同步解析请求恢复目录","Server capability tier: flash, basic, or standard": "服务能力档位: flash、basic 或 standard","Disable Flash tier advertisement and execution": "禁用 Flash 档位的广播与执行","Disable Advanced tier advertisement and execution": "禁用 Advanced 档位的广播与执行",diff --git a/scripts/http_file_parse_example.py b/scripts/http_file_parse_example.pynew file mode 100644index 00000000..58af2a04--- /dev/null+++ b/scripts/http_file_parse_example.py@@ -0,0 +1,78 @@+#!/usr/bin/env python3+"""同步 HTTP 包装示例:提前记录请求 ID,成功保存附件,超时返回查询链接。"""++from __future__ import annotations++import argparse+import json+import os+import tempfile+import uuid+from pathlib import Path++import httpx+++def main() -> int:+ """演示标准 multipart 列表编码及二进制成功响应与 JSON 状态响应的区分。"""+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("file", type=Path)+ parser.add_argument("--base-url", default="http://127.0.0.1:8003")+ parser.add_argument("--output", type=Path)+ parser.add_argument("--output-format", action="append", choices=("default", "docx", "html", "latex"))+ parser.add_argument("--request-id", default=uuid.uuid4().hex)+ parser.add_argument("--wait-timeout", type=int, default=3600)+ parser.add_argument("--tier", choices=("flash", "basic", "standard", "advanced"))+ parser.add_argument("--ocr-mode", choices=("auto", "txt", "ocr"), default="auto")+ parser.add_argument("--page-range", default="")+ args = parser.parse_args()+ if not args.file.is_file():+ parser.error("Input file does not exist")+ formats = list(dict.fromkeys(args.output_format or ["default"]))+ extension = formats[0] if formats in (["docx"], ["html"]) else "zip"+ output = args.output or Path("results") / f"{args.file.stem}.{extension}"+ if output.exists():+ parser.error("Output already exists; choose another --output path")+ print(f"request_id={args.request_id}", flush=True)+ print(f"status_url={args.base_url.rstrip('/')}/v1/file_parse/{args.request_id}", flush=True)+ fields = [+ ("request_id", (None, args.request_id)),+ ("wait_timeout", (None, str(args.wait_timeout))),+ ("ocr_mode", (None, args.ocr_mode)),+ ("page_range", (None, args.page_range)),+ *(("output_formats", (None, value)) for value in formats),+ ]+ if args.tier is not None:+ fields.append(("tier", (None, args.tier)))+ temporary: Path | None = None+ try:+ with args.file.open("rb") as source, httpx.Client(timeout=httpx.Timeout(7200, connect=10, write=600)) as client:+ with client.stream(+ "POST", args.base_url.rstrip("/") + "/v1/file_parse", files=[("file", (args.file.name, source)), *fields]+ ) as response:+ if response.status_code != 200:+ payload = response.read()+ try:+ print(json.dumps(json.loads(payload), ensure_ascii=False, indent=2))+ except ValueError:+ print(f"HTTP {response.status_code}: {payload[:4096].decode(errors='replace')}")+ return 124 if response.status_code == 202 else 1+ output.parent.mkdir(parents=True, exist_ok=True)+ descriptor, name = tempfile.mkstemp(prefix=f".{output.name}.", suffix=".part", dir=output.parent)+ temporary = Path(name)+ with os.fdopen(descriptor, "wb") as destination:+ for chunk in response.iter_bytes(1024 * 1024):+ destination.write(chunk)+ temporary.replace(output)+ print(f"Saved {output}")+ return 0+ except (httpx.HTTPError, OSError) as exc:+ print(f"Transfer failed ({type(exc).__name__}); use the printed status_url to recover")+ return 1+ finally:+ if temporary is not None:+ temporary.unlink(missing_ok=True)+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/tests/unittest/test_file_parse.py b/tests/unittest/test_file_parse.pynew file mode 100644index 00000000..2397ed6e--- /dev/null+++ b/tests/unittest/test_file_parse.py@@ -0,0 +1,654 @@+"""同步包装服务的参数、恢复、认证和传输故障契约。"""++from __future__ import annotations++import asyncio+import io+import itertools+import json+import subprocess+import sys+import threading+import time+import zipfile+from concurrent.futures import ThreadPoolExecutor+from pathlib import Path+from typing import Any++import httpx+import pytest+from fastapi.testclient import TestClient+from filelock import Timeout+from typer.testing import CliRunner++from mineru.kit.file_parse import FileParseSettings, create_app+from mineru.kit.file_parse.app import TemporaryFileResponse+from mineru.kit.file_parse.client import TaskClient+from mineru.kit.file_parse.models import ParseOptions, Record, fingerprint+from mineru.kit.file_parse.settings import origin+from mineru.kit.file_parse.store import RecordStore+++def artifact_bytes(formats: list[str]) -> bytes:+ """构造可校验的不同格式附件,下载字节必须原样返回。"""+ if formats == ["html"]:+ return b"<!doctype html><html><body>file parse result</body></html>"+ output = io.BytesIO()+ with zipfile.ZipFile(output, "w") as archive:+ archive.writestr("word/document.xml" if formats == ["docx"] else "markdown.md", "file parse result")+ return output.getvalue()+++class Upstream:+ """模拟带路径前缀的 SaaS API 和独立对象存储,不提供 Jobs 或 Files 接口。"""++ def __init__(self) -> None:+ """初始化可控故障点及全部 HTTP 调用记录。"""+ self.tasks: dict[str, dict[str, Any]] = {}+ self.api_url = "https://api.test/api"+ self.storage_url = "https://store.test"+ self.calls: list[tuple[str, str, str | None]] = []+ self.complete = True+ self.cache_hit = False+ self.create_failure: str | None = None+ self.upload_response_lost = False+ self.upload_failures = 0+ self.upload_delay = 0.0+ self.ready_delay = 0.0+ self.complete_delay = 0.0+ self.download_expired = False+ self.download_corrupt = False+ self.download_interrupted = False+ self.redirect = False+ self.gone = False+ self.uploads = 0+ self.downloads = 0++ def payload(self, task_id: str) -> dict[str, Any]:+ """返回共同协议快照,签名中的转义字符用于校验地址未被重新拼装。"""+ task = self.tasks[task_id]+ ready = task["ready"] and time.monotonic() >= task["uploaded_at"] + self.ready_delay+ complete = ready and self.complete and time.monotonic() >= task["uploaded_at"] + self.complete_delay+ expired = not ready and task["expires_at"] <= time.time()+ status = "failed" if expired else "completed" if complete else "running" if ready else "waiting_upload"+ result_url = f"{self.storage_url}/result/{task_id}?signature=a%2Fb&version={task['version']}" if complete else None+ if result_url and self.redirect:+ result_url = f"{self.api_url}/redirect/{task_id}"+ return {+ "task_id": task_id,+ "status": status,+ "status_url": f"{self.api_url}/v1/tasks/{task_id}",+ "output_formats": task["options"]["output_formats"],+ "files": [+ {+ "client_file_id": "file",+ "status": status,+ "upload_status": "expired" if expired else "ready" if ready else "pending",+ "result_url": result_url,+ "error": {"code": "upload_expired"} if expired else None,+ }+ ],+ "uploads": []+ if ready or expired+ else [+ {+ "client_file_id": "file",+ "upload_method": "PUT",+ "upload_url": f"{self.storage_url}/upload/{task_id}",+ "upload_headers": {"x-upload-token": "storage-token"},+ "expires_at": task["expires_at"],+ }+ ],+ }++ async def handle(self, request: httpx.Request) -> httpx.Response:+ """执行清单、上传、查询和下载,并在每一跳检查认证边界。"""+ self.calls.append((request.method, str(request.url), request.headers.get("authorization")))+ if origin(str(request.url)) == origin(self.api_url):+ assert request.headers.get("authorization") == "Bearer secret"+ else:+ assert origin(str(request.url)) == origin(self.storage_url)+ assert request.headers.get("authorization") is None+ if request.method == "POST":+ assert request.url.path == "/api/v1/tasks"+ if self.create_failure == "auth":+ return httpx.Response(401, json={"error": {"code": "invalid_api_key", "message": "Bad upstream key"}})+ body = json.loads(await request.aread())+ assert len(body["manifest"]) == 1+ task_id = f"task_{len(self.tasks) + 1}"+ self.tasks[task_id] = {+ "options": body,+ "ready": self.cache_hit,+ "uploaded_at": time.monotonic(),+ "version": 0,+ "expires_at": int(time.time()) + 3600,+ }+ if self.create_failure == "lost":+ raise httpx.ReadError("creation response lost")+ payload = self.payload(task_id)+ if self.create_failure == "malformed":+ payload["task_id"] = None+ return httpx.Response(202, json=payload)+ task_id = request.url.path.rsplit("/", 1)[-1]+ task = self.tasks[task_id]+ if request.method == "PUT":+ self.uploads += 1+ assert request.headers["x-upload-token"] == "storage-token"+ content = await request.aread()+ assert len(content) == task["options"]["manifest"][0]["bytes"]+ if self.upload_failures:+ self.upload_failures -= 1+ return httpx.Response(503)+ await asyncio.sleep(self.upload_delay)+ task["ready"] = True+ task["uploaded_at"] = time.monotonic()+ return httpx.Response(503 if self.upload_response_lost else 200)+ if "/v1/tasks/" in request.url.path:+ return httpx.Response(404) if self.gone else httpx.Response(200, json=self.payload(task_id))+ if "/redirect/" in request.url.path:+ return httpx.Response(302, headers={"Location": f"{self.storage_url}/result/{task_id}?signature=a%2Fb"})+ self.downloads += 1+ assert "signature=a%2Fb" in str(request.url)+ if self.download_expired:+ self.download_expired = False+ task["version"] += 1+ return httpx.Response(403)+ if self.download_corrupt:+ self.download_corrupt = False+ return httpx.Response(200, content=b"corrupt result")+ if self.download_interrupted:+ self.download_interrupted = False+ return httpx.Response(200, stream=InterruptedStream())+ return httpx.Response(200, content=artifact_bytes(task["options"]["output_formats"]))+++class InterruptedStream(httpx.AsyncByteStream):+ """模拟结果传输到一半时上游断开连接。"""++ async def __aiter__(self) -> Any:+ """发出部分 ZIP 头后报错,重试必须从头覆盖临时文件。"""+ yield b"PK\x03\x04partial"+ raise httpx.ReadError("download disconnected")+++def application(tmp_path: Path, upstream: Upstream, **settings: Any) -> Any:+ """用真实包装层和可控 HTTP 传输构造测试应用。"""+ return create_app(+ FileParseSettings("https://api.test/api", "secret", tmp_path, poll_interval=0.01, **settings),+ transport=httpx.MockTransport(upstream.handle),+ )+++def submit(client: TestClient, *, request_id: str = "request_1", content: bytes = b"source", **options: Any) -> httpx.Response:+ """使用真实 multipart 编码,列表选项通过重复同名字段发送。"""+ fields: list[tuple[str, Any]] = [("file", ("report.pdf", content, "application/pdf")), ("request_id", (None, request_id))]+ for name, value in options.items():+ for item in value if isinstance(value, list) else [value]:+ fields.append((name, (None, str(item))))+ return client.post("/v1/file_parse", files=fields)+++FORMATS = [+ list(values) for size in range(1, 5) for values in itertools.combinations(("default", "docx", "html", "latex"), size)+]++[email protected]("formats", FORMATS)+def test_formats_are_returned_unchanged(tmp_path: Path, formats: list[str]) -> None:+ """覆盖全部十五种格式组合及附件字节、文件名、MIME 和临时文件释放。"""+ upstream = Upstream()+ app = application(tmp_path, upstream)+ with TestClient(app) as client:+ response = submit(client, output_formats=formats)+ assert response.status_code == 200, response.text+ assert response.content == artifact_bytes(formats)+ extension = formats[0] if formats in (["docx"], ["html"]) else "zip"+ assert f"report.{extension}" in response.headers["content-disposition"]+ assert response.headers["x-request-id"] == "request_1"+ assert response.headers["x-task-id"] == "task_1"+ assert not list(app.state.service.store.sources.iterdir())+ assert not list(app.state.service.store.results.iterdir())+++def test_timeout_recovery_deduplication_and_conflict(tmp_path: Path) -> None:+ """零等待返回包装地址,同 ID 复用任务,参数冲突不会替换原任务。"""+ upstream = Upstream()+ upstream.complete = False+ app = application(tmp_path, upstream)+ with TestClient(app, root_path="/gateway") as client:+ response = submit(client, wait_timeout=0, output_formats=["html", "html"])+ assert response.status_code == 202+ payload = response.json()+ assert response.headers["location"] == payload["status_url"]+ assert payload["status_url"] == "http://testserver/gateway/v1/file_parse/request_1"+ assert payload["output_formats"] == ["html"]+ assert payload["result_url"] is None+ assert client.get(payload["status_url"] + "/result").status_code == 409+ assert submit(client, wait_timeout=0, output_formats=["html"]).status_code == 202+ assert len(upstream.tasks) == 1+ assert submit(client, content=b"different", wait_timeout=0).status_code == 409+ upstream.complete = True+ response = client.get(payload["status_url"])+ assert response.json()["status"] == "completed"+ assert client.get(response.json()["result_url"]).content == artifact_bytes(["html"])+ assert not list(app.state.service.store.results.iterdir())+++def test_wait_budget_excludes_upload(tmp_path: Path) -> None:+ """慢速上游上传不占一秒等待预算,解析完成仍能直接返回附件。"""+ upstream = Upstream()+ upstream.upload_delay = 1.1+ upstream.complete_delay = 0.15+ with TestClient(application(tmp_path, upstream)) as client:+ started = time.monotonic()+ response = submit(client, wait_timeout=1)+ assert response.status_code == 200, response.text+ assert time.monotonic() - started >= 1.1+++def test_expired_wait_does_not_cancel_upstream(tmp_path: Path) -> None:+ """等待预算到期返回 202,后台继续观察任务直至完成。"""+ upstream = Upstream()+ upstream.complete = False+ with TestClient(application(tmp_path, upstream)) as client:+ started = time.monotonic()+ response = submit(client, wait_timeout=1)+ assert response.status_code == 202+ assert 0.9 <= time.monotonic() - started < 3+ assert not any(method == "DELETE" for method, _, _ in upstream.calls)+ upstream.complete = True+ assert client.get(response.json()["status_url"]).json()["status"] == "completed"+++def test_wait_budget_excludes_storage_notification_delay(tmp_path: Path) -> None:+ """存储已经接收字节但网关尚未确认 ready 时,不消耗同步解析等待预算。"""+ upstream = Upstream()+ upstream.ready_delay = 1.1+ upstream.complete_delay = 1.2+ with TestClient(application(tmp_path, upstream)) as client:+ response = submit(client, wait_timeout=1)+ assert response.status_code == 200, response.text+ assert upstream.uploads == 1++[email protected]("failure", ["lost", "malformed"])+def test_unknown_creation_is_never_replayed(tmp_path: Path, failure: str) -> None:+ """上游创建成功但响应丢失或身份缺失时,重试和重启都不能再次 POST。"""+ upstream = Upstream()+ upstream.create_failure = failure+ with TestClient(application(tmp_path, upstream)) as client:+ response = submit(client)+ assert response.status_code == 502+ assert response.json()["status"] == "submission_unknown"+ assert response.json()["task_id"] is None+ assert submit(client).status_code in {409, 502}+ with TestClient(application(tmp_path, upstream)) as client:+ assert client.get("/v1/file_parse/request_1").json()["status"] == "submission_unknown"+ assert submit(client).status_code in {409, 502}+ assert len(upstream.tasks) == 1+++def test_auth_failure_cleans_source_without_retry(tmp_path: Path) -> None:+ """上游认证错误是配置错误,不向无认证的包装入口伪报 401。"""+ upstream = Upstream()+ upstream.create_failure = "auth"+ app = application(tmp_path, upstream)+ with TestClient(app) as client:+ response = submit(client)+ assert response.status_code == 502+ assert response.json()["error"]["code"] == "invalid_api_key"+ assert response.json()["status_url"]+ assert not list(app.state.service.store.sources.iterdir())+ assert len(upstream.calls) == 1+++def test_upload_response_loss_is_reconciled(tmp_path: Path) -> None:+ """上传已成功但响应丢失时先查 ready,避免再次发送文件。"""+ upstream = Upstream()+ upstream.upload_response_lost = True+ with TestClient(application(tmp_path, upstream)) as client:+ assert submit(client).status_code == 200+ assert upstream.uploads == 1+++def test_upload_retry_budget_and_explicit_resume(tmp_path: Path) -> None:+ """自动上传最多尝试三次;调用者重传相同请求可以继续原任务而不再次创建。"""+ upstream = Upstream()+ upstream.upload_failures = 3+ app = application(tmp_path, upstream)+ with TestClient(app) as client:+ app.state.service.client.retry_delay = 0+ response = submit(client)+ assert response.status_code == 502+ assert response.json()["task_id"] == "task_1"+ assert upstream.uploads == 3+ assert list(app.state.service.store.sources.iterdir())+ assert submit(client).status_code == 200+ assert not list(app.state.service.store.sources.iterdir())+ assert upstream.uploads == 4 and len(upstream.tasks) == 1++[email protected]("fault", ["download_expired", "download_corrupt", "download_interrupted", "redirect"])+def test_download_retries_and_cross_origin_headers(tmp_path: Path, fault: str) -> None:+ """签名失效和损坏结果可以恢复;重定向每跳都验证未向存储发送 API Key。"""+ upstream = Upstream()+ setattr(upstream, fault, True)+ app = application(tmp_path, upstream)+ with TestClient(app) as client:+ app.state.service.client.retry_delay = 0+ response = submit(client)+ assert response.status_code == 200+ assert response.content == artifact_bytes(["default"])+ assert upstream.downloads == (1 if fault == "redirect" else 2)+ assert not list(app.state.service.store.results.iterdir())+++def test_cache_hit_avoids_upload(tmp_path: Path) -> None:+ """源文件 hash 缓存命中时直接等待,不尝试构造或访问上传链接。"""+ upstream = Upstream()+ upstream.cache_hit = True+ with TestClient(application(tmp_path, upstream)) as client:+ assert submit(client).status_code == 200+ assert upstream.uploads == 0+++def test_concurrent_duplicate_requests_share_one_task(tmp_path: Path) -> None:+ """并发提交相同请求 ID 只创建、上传一次,但每个连接可分别下载。"""+ upstream = Upstream()+ upstream.upload_delay = 0.1+ with TestClient(application(tmp_path, upstream)) as client, ThreadPoolExecutor(max_workers=4) as executor:+ responses = list(executor.map(lambda _: submit(client), range(4)))+ assert all(response.status_code == 200 for response in responses)+ assert len(upstream.tasks) == 1+ assert upstream.uploads == 1+++def test_restart_uses_existing_task_and_isolates_upstream(tmp_path: Path) -> None:+ """进程重启后查询同一任务;更换上游配置不会访问旧记录。"""+ upstream = Upstream()+ upstream.complete = False+ with TestClient(application(tmp_path, upstream)) as client:+ response = submit(client, wait_timeout=0)+ assert response.status_code == 202+ upstream.complete = True+ with TestClient(application(tmp_path, upstream)) as client:+ assert client.get("/v1/file_parse/request_1").json()["status"] == "completed"+ assert client.get("/v1/file_parse/request_1/result").status_code == 200+ assert len(upstream.tasks) == 1+ app = create_app(FileParseSettings("https://different.test", state_dir=tmp_path))+ with TestClient(app) as client:+ assert client.get("/v1/file_parse/request_1").status_code == 404++[email protected]("stage", ["prepared", "pending", "expired", "submitting"])+def test_restart_at_upload_boundaries(tmp_path: Path, stage: str) -> None:+ """覆盖崩溃前尚未创建、已创建待上传、上传过期和提交不确定四个恢复边界。"""+ upstream = Upstream()+ settings = FileParseSettings("https://api.test/api", "secret", tmp_path, poll_interval=0.01)++ async def seed() -> None:+ """直接落盘一个崩溃时刻的真实恢复记录,不伪造运行中的任务对象。"""+ store = RecordStore(settings)+ await store.open()+ source = store.sources / "retained-source"+ source.write_bytes(b"source")+ options = ParseOptions()+ record = Record(+ "request_1",+ "report.pdf",+ 6,+ "a" * 64,+ options.model_dump(),+ str(source),+ fingerprint("report.pdf", 6, "a" * 64, options),+ )+ if stage in {"pending", "expired"}:+ remote = TaskClient(settings, httpx.MockTransport(upstream.handle))+ try:+ task = await remote.create(record)+ record.remote = task.model_dump()+ record.status = "waiting_upload"+ if stage == "expired":+ upstream.tasks[task.task_id]["expires_at"] = int(time.time()) - 1+ finally:+ await remote.close()+ elif stage == "submitting":+ record.status = "submitting"+ await store.save(record)+ await store.close()++ asyncio.run(seed())+ app = application(tmp_path, upstream)+ with TestClient(app) as client:+ deadline = time.monotonic() + 3+ while True:+ response = client.get("/v1/file_parse/request_1")+ status = response.json()["status"]+ if status in {"completed", "failed", "submission_unknown"} or time.monotonic() > deadline:+ break+ time.sleep(0.01)+ assert status == ("submission_unknown" if stage == "submitting" else "failed" if stage == "expired" else "completed")+ assert len(upstream.tasks) == (0 if stage == "submitting" else 1)+ assert upstream.uploads == (1 if stage in {"prepared", "pending"} else 0)+ assert not list(app.state.service.store.sources.iterdir())+++def test_disconnect_does_not_cancel_upload(tmp_path: Path) -> None:+ """真实 ASGI 断连消息只能取消连接等待,已经接收的源文件继续上传。"""+ upstream = Upstream()+ upstream.upload_delay = 0.1+ app = application(tmp_path, upstream)++ async def exercise() -> None:+ """发送完整 multipart 后立即断连,再通过同一请求 ID 检查完成结果。"""+ async with app.router.lifespan_context(app):+ encoded = httpx.Request(+ "POST",+ "http://wrapper/v1/file_parse",+ files={+ "file": ("report.pdf", b"source"),+ "request_id": (None, "request_1"),+ },+ )+ body = encoded.read()+ received = False++ async def receive() -> dict[str, Any]:+ """第一次返回完整文件,后续接收返回真实断连事件。"""+ nonlocal received+ if not received:+ received = True+ return {"type": "http.request", "body": body, "more_body": False}+ return {"type": "http.disconnect"}++ async def send(message: dict[str, Any]) -> None:+ """模拟断开的客户端,不消费任何响应内容。"""+ assert message["type"] in {"http.response.start", "http.response.body"}++ scope = {+ "type": "http",+ "method": "POST",+ "path": "/v1/file_parse",+ "root_path": "",+ "raw_path": b"/v1/file_parse",+ "query_string": b"",+ "headers": [(key.lower(), value) for key, value in encoded.headers.raw],+ "server": ("wrapper", 80),+ "client": ("client", 123),+ "scheme": "http",+ "http_version": "1.1",+ "asgi": {"version": "3.0", "spec_version": "2.4"},+ }+ await app(scope, receive, send)+ for _ in range(100):+ record = app.state.service.records["request_1"]+ if record.status == "completed":+ break+ await asyncio.sleep(0.01)+ assert record.status == "completed"+ assert upstream.uploads == 1 and len(upstream.tasks) == 1+ assert not list(app.state.service.store.sources.iterdir())++ asyncio.run(exercise())+++def test_attachment_cleanup_after_send_disconnect(tmp_path: Path) -> None:+ """附件发送中断也执行清理,而非只依赖发送成功后的后台回调。"""+ path = tmp_path / "result.zip"+ path.write_bytes(artifact_bytes(["default"]))++ async def exercise() -> None:+ """在响应已经开始后断开下游,验证 finally 释放结果文件。"""++ async def receive() -> dict[str, Any]:+ """提供断连事件,兼容 ASGI 响应实现。"""+ return {"type": "http.disconnect"}++ async def send(message: dict[str, Any]) -> None:+ """在文件主体发送时模拟客户端 socket 关闭。"""+ if message["type"] == "http.response.body":+ raise OSError("client disconnected")++ with pytest.raises(OSError):+ await TemporaryFileResponse(path)(+ {"type": "http", "method": "GET", "headers": [], "asgi": {"spec_version": "2.4"}}, receive, send+ )++ asyncio.run(exercise())+ assert not path.exists()+++def test_retention_and_state_directory_lock(tmp_path: Path) -> None:+ """同目录第二进程无法接管;恢复记录按首次终态时间回收。"""+ upstream = Upstream()+ app = application(tmp_path, upstream, retention_seconds=1)+ with TestClient(app) as client:+ assert submit(client).status_code == 200+ with pytest.raises(Timeout), TestClient(application(tmp_path, upstream)):+ pass+ app.state.service.records["request_1"].terminal_at = time.time() - 2+ assert client.get("/v1/file_parse/request_1").status_code == 404+ client.portal.call(app.state.service.collect)+ assert "request_1" not in app.state.service.records+++def test_late_query_cannot_replace_reused_request_id(tmp_path: Path) -> None:+ """记录过期并复用 ID 后,旧任务的迟到查询不能覆盖新任务的持久化映射。"""+ upstream = Upstream()+ started, release = threading.Event(), asyncio.Event()+ pause = False++ async def delayed(request: httpx.Request) -> httpx.Response:+ """仅延迟旧任务查询,保证新任务的创建和下载仍可正常完成。"""+ if pause and request.method == "GET" and request.url.path.endswith("/v1/tasks/task_1"):+ started.set()+ await release.wait()+ return await upstream.handle(request)++ app = create_app(+ FileParseSettings("https://api.test/api", "secret", tmp_path, retention_seconds=1, poll_interval=0.01),+ transport=httpx.MockTransport(delayed),+ )+ with TestClient(app) as client, ThreadPoolExecutor(max_workers=1) as executor:+ assert submit(client).status_code == 200+ pause = True+ pending = executor.submit(client.get, "/v1/file_parse/request_1")+ assert started.wait(2)+ app.state.service.records["request_1"].terminal_at = time.time() - 2+ response = submit(client)+ assert response.status_code == 200+ assert response.headers["x-task-id"] == "task_2"+ client.portal.call(release.set)+ assert pending.result(timeout=2).status_code == 200+ assert app.state.service.records["request_1"].task.task_id == "task_2"+ with TestClient(application(tmp_path, upstream)) as client:+ assert client.get("/v1/file_parse/request_1").json()["task_id"] == "task_2"+++ "options",+ [+ {"wait_timeout": -1},+ {"wait_timeout": 3601},+ {"wait_timeout": "1.5"},+ {"request_id": "../bad"},+ {"output_formats": ["markdown"]},+ {"output_formats": [""]},+ {"tier": "wrong"},+ {"ocr_mode": "wrong"},+ {"api_url": "https://different.test"},+ ],+)+def test_invalid_parameters_do_not_create_tasks(tmp_path: Path, options: dict[str, Any]) -> None:+ """所有本地可校验参数错误在创建上游任务前返回。"""+ upstream = Upstream()+ with TestClient(application(tmp_path, upstream)) as client:+ assert submit(client, **options).status_code == 400+ assert not upstream.tasks+++def test_single_file_content_type_and_scalar_duplicates(tmp_path: Path) -> None:+ """拒绝 JSON、空文件、重复文件和重复标量,格式列表保持唯一例外。"""+ upstream = Upstream()+ with TestClient(application(tmp_path, upstream)) as client:+ assert client.post("/v1/file_parse", json={}).status_code == 415+ assert submit(client, content=b"").status_code == 400+ assert client.post("/v1/file_parse", files=[("file", ("a.pdf", b"a")), ("file", ("b.pdf", b"b"))]).status_code == 400+ assert (+ client.post(+ "/v1/file_parse",+ files=[+ ("file", ("a.pdf", b"a")),+ ("tier", (None, "flash")),+ ("tier", (None, "flash")),+ ],+ ).status_code+ == 400+ )+ assert not upstream.tasks+++def test_openapi_and_light_imports(tmp_path: Path) -> None:+ """OpenAPI 说明格式数组及附件,独立导入包装应用不加载解析运行时。"""+ schema = application(tmp_path, Upstream()).openapi()+ post = schema["paths"]["/v1/file_parse"]["post"]+ request = post["requestBody"]["content"]["multipart/form-data"]["schema"]+ assert request["properties"]["wait_timeout"]["maximum"] == 3600+ assert request["properties"]["output_formats"]["items"]["enum"] == ["default", "docx", "html", "latex"]+ assert "application/zip" in post["responses"]["200"]["content"]+ result = subprocess.run(+ [+ sys.executable,+ "-c",+ (+ "import sys; from mineru.kit.file_parse import create_app; "+ "assert not any(x in sys.modules for x in ('torch', 'transformers', 'mineru.parser.api_server'))"+ ),+ ],+ capture_output=True,+ text=True,+ )+ assert result.returncode == 0, result.stderr+++def test_cli_configuration_and_environment(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ """真实 CLI 解析环境变量及显式覆盖,把上游配置交给独立应用。"""+ from mineru.kit.main import app++ captured: list[tuple[Any, dict[str, Any]]] = []++ def run(application: Any, **options: Any) -> None:+ """捕获启动参数,通过应用 lifespan 验证最终配置。"""+ captured.append((application, options))+ with TestClient(application):+ assert application.state.service.settings.api_url == "https://explicit.test/api"+ assert application.state.service.settings.api_key == "environment-key"++ monkeypatch.setattr("uvicorn.run", run)+ monkeypatch.setenv("MINERU_API_URL", "https://environment.test")+ monkeypatch.setenv("MINERU_API_KEY", "environment-key")+ result = CliRunner().invoke(+ app, ["file-parse-server", "--api-url", "https://explicit.test/api", "--state-dir", str(tmp_path)]+ )+ assert result.exit_code == 0, result.output+ assert captured[0][1] == {"host": "127.0.0.1", "port": 8003}diff --git a/tests/unittest/test_file_parse_http.py b/tests/unittest/test_file_parse_http.pynew file mode 100644index 00000000..045ed8ff--- /dev/null+++ b/tests/unittest/test_file_parse_http.py@@ -0,0 +1,230 @@+"""本地 API、Router、SaaS/对象存储的真实 TCP 同步包装验收。"""++from __future__ import annotations++import io+import socket+import subprocess+import sys+import time+import zipfile+from collections.abc import Iterator+from contextlib import contextmanager+from pathlib import Path++import httpx+from fastapi import FastAPI, Request, Response+from PIL import Image, ImageDraw+from reportlab.lib.utils import ImageReader+from reportlab.pdfgen.canvas import Canvas+from test_file_parse import FORMATS, Upstream, artifact_bytes+from test_http_api_example_script import _Server+from test_task_http import HTML, _LiveService, live_service as tasks_live_service++from mineru.kit.file_parse import FileParseSettings, create_app++live_service = tasks_live_service+++def assert_clean(directory: Path) -> None:+ """TCP 客户端可能先收到末块数据,给服务器的响应 finally 一个有界收尾窗口。"""+ deadline = time.monotonic() + 2+ while list(directory.iterdir()) and time.monotonic() < deadline:+ time.sleep(0.01)+ assert not list(directory.iterdir())+++@contextmanager+def wrapper(tmp_path: Path, base_url: str, key: str | None) -> Iterator[tuple[httpx.Client, FastAPI]]:+ """启动真正的包装 HTTP 服务,统一测试只改变上游地址和认证。"""+ app = create_app(FileParseSettings(base_url, key, tmp_path, poll_interval=0.02))+ server = _Server(app)+ server.start()+ try:+ with httpx.Client(base_url=server.url, timeout=90, trust_env=False) as client:+ yield client, app+ finally:+ server.stop()+++def parse_file(+ client: httpx.Client,+ request_id: str,+ filename: str,+ content: bytes,+ formats: list[str],+) -> httpx.Response:+ """从调用者端发送单文件 multipart,请求格式遵循标准重复字段编码。"""+ return client.post(+ "/v1/file_parse",+ files=[+ ("file", (filename, content)),+ ("request_id", (None, request_id)),+ ("tier", (None, "flash")),+ ("ocr_mode", (None, "txt")),+ *(("output_formats", (None, item)) for item in formats),+ ],+ )+++def test_api_and_router_all_formats(live_service: _LiveService, tmp_path: Path) -> None:+ """真实解析与导出全部格式组合,包装返回的字节必须和上游保存的产物相同。"""+ with wrapper(tmp_path / "wrapper", live_service.url, "test-key") as (client, app):+ with httpx.Client(headers={"Authorization": "Bearer test-key"}, timeout=90, trust_env=False) as upstream:+ for index, formats in enumerate(FORMATS):+ response = parse_file(client, f"formats_{index}", "source.html", HTML, formats)+ assert response.status_code == 200, response.text+ task_id = response.headers["x-task-id"]+ task = upstream.get(f"{live_service.url}/v1/tasks/{task_id}").json()+ direct = upstream.get(task["files"][0]["result_url"])+ assert direct.status_code == 200+ assert response.content == direct.content+ assert response.headers["content-type"] == direct.headers["content-type"]+ assert_clean(app.state.service.store.results)+ assert_clean(app.state.service.store.sources)+++def cloud_bridge(upstream: Upstream) -> FastAPI:+ """通过独立网络端口暴露模拟 SaaS 协议,数据传输仍走真实 HTTP。"""+ app = FastAPI()++ @app.api_route("/{path:path}", methods=["GET", "POST", "PUT"])+ async def relay(request: Request) -> Response:+ """只桥接 HTTP,不为包装客户端增加 Jobs、Files 或存储路径假设。"""+ response = await upstream.handle(+ httpx.Request(+ request.method,+ str(request.url),+ headers=request.headers,+ content=await request.body(),+ )+ )+ return Response(response.content, status_code=response.status_code, headers=dict(response.headers))++ return app+++def test_saas_storage_all_formats_and_recovery(tmp_path: Path) -> None:+ """API 路径前缀、独立存储端口、重定向和签名刷新均使用同一包装实现。"""+ state = Upstream()+ api, storage = _Server(cloud_bridge(state)), _Server(cloud_bridge(state))+ state.api_url, state.storage_url = api.url + "/api", storage.url+ api.start()+ storage.start()+ try:+ with wrapper(tmp_path, state.api_url, "secret") as (client, app):+ app.state.service.client.retry_delay = 0+ state.redirect = True+ for index, formats in enumerate(FORMATS):+ state.download_expired = True+ response = parse_file(client, f"cloud_{index}", "source.pdf", b"source", formats)+ assert response.status_code == 200, response.text+ assert response.content == artifact_bytes(formats)+ assert_clean(app.state.service.store.results)+ assert_clean(app.state.service.store.sources)+ source = tmp_path / "example.pdf"+ source.write_bytes(b"source")+ output = tmp_path / "example.docx"+ script = Path(__file__).resolve().parents[2] / "scripts" / "http_file_parse_example.py"+ common = [sys.executable, str(script), str(source), "--base-url", str(client.base_url)]+ completed = subprocess.run(+ [*common, "--output-format", "docx", "--output", str(output), "--request-id", "example-success"],+ capture_output=True,+ text=True,+ timeout=10,+ )+ assert completed.returncode == 0, completed.stdout + completed.stderr+ assert output.read_bytes() == artifact_bytes(["docx"])+ state.complete = False+ waiting = subprocess.run(+ [*common, "--wait-timeout", "0", "--request-id", "example-waiting", "--output", str(tmp_path / "pending.zip")],+ capture_output=True,+ text=True,+ timeout=10,+ )+ assert waiting.returncode == 124, waiting.stdout + waiting.stderr+ assert "status_url" in waiting.stdout and not (tmp_path / "pending.zip").exists()+ assert len(state.tasks) == 17+ assert all(method != "DELETE" for method, _, _ in state.calls)+ finally:+ storage.stop()+ api.stop()+++def illustrated_pdf() -> bytes:+ """生成包含文本、原生图片和图注的真实 PDF,验证附件传输不改变解析产物。"""+ picture = Image.new("RGB", (240, 140), "white")+ drawing = ImageDraw.Draw(picture)+ drawing.rectangle((10, 20, 100, 120), fill="navy")+ drawing.ellipse((130, 20, 230, 120), fill="orange")+ buffer = io.BytesIO()+ canvas = Canvas(buffer, pagesize=(595, 842))+ canvas.setFont("Helvetica", 18)+ canvas.drawString(72, 760, "File Parse round trip")+ canvas.setFont("Helvetica", 12)+ canvas.drawString(72, 720, "The same document must reach every download client unchanged.")+ canvas.drawImage(ImageReader(picture), 100, 420, width=360, height=210)+ canvas.drawString(100, 395, "Figure 1. A blue rectangle and an orange circle.")+ canvas.drawString(72, 350, "The figure is embedded in the PDF as a native image.")+ canvas.save()+ return buffer.getvalue()+++def test_real_pdf_attachment_matches_upstream(live_service: _LiveService, tmp_path: Path) -> None:+ """真实 PDF 经 API 与 Router 导出四格式后,附件和上游产物逐字节一致且包含图片。"""+ pdf = illustrated_pdf()+ (tmp_path / "illustrated.pdf").write_bytes(pdf)+ with wrapper(tmp_path / "wrapper", live_service.url, "test-key") as (client, _):+ response = parse_file(client, "illustrated", "illustrated.pdf", pdf, ["default", "docx", "html", "latex"])+ assert response.status_code == 200, response.text+ (tmp_path / "wrapper-result.zip").write_bytes(response.content)+ with httpx.Client(headers={"Authorization": "Bearer test-key"}, timeout=90, trust_env=False) as upstream:+ task = upstream.get(f"{live_service.url}/v1/tasks/{response.headers['x-task-id']}").json()+ direct = upstream.get(task["files"][0]["result_url"])+ assert response.content == direct.content+ with zipfile.ZipFile(io.BytesIO(response.content)) as archive:+ assert "File Parse round trip" in archive.read("markdown.md").decode()+ assert {"document.docx", "document.html", "document.tex"}.issubset(archive.namelist())+ assert any(name.startswith("images/") and not name.endswith("/") for name in archive.namelist())+ assert b"data:image/" in archive.read("document.html")+++def test_tcp_disconnect_keeps_accepted_request(tmp_path: Path) -> None:+ """调用者关闭真实 socket 后,完整收到的文件仍上传并可通过请求 ID 找回。"""+ state = Upstream()+ state.upload_delay = 0.2+ api, storage = _Server(cloud_bridge(state)), _Server(cloud_bridge(state))+ state.api_url, state.storage_url = api.url + "/api", storage.url+ api.start()+ storage.start()+ try:+ with wrapper(tmp_path, state.api_url, "secret") as (client, app):+ request = httpx.Request(+ "POST",+ str(client.base_url).rstrip("/") + "/v1/file_parse",+ files={+ "file": ("source.pdf", b"source"),+ "request_id": (None, "disconnected"),+ },+ )+ content = request.read()+ headers = "\r\n".join(f"{key}: {value}" for key, value in request.headers.items()).encode()+ address = client.base_url+ with socket.create_connection((address.host, address.port)) as sock:+ sock.sendall(b"POST /v1/file_parse HTTP/1.1\r\n" + headers + b"\r\n\r\n" + content)+ deadline = time.monotonic() + 3+ while "disconnected" not in app.state.service.records and time.monotonic() < deadline:+ time.sleep(0.01)+ assert "disconnected" in app.state.service.records+ deadline = time.monotonic() + 5+ while True:+ response = client.get("/v1/file_parse/disconnected")+ if response.json().get("status") == "completed" or time.monotonic() > deadline:+ break+ time.sleep(0.02)+ assert response.json()["status"] == "completed"+ assert client.get(response.json()["result_url"]).content == artifact_bytes(["default"])+ assert state.uploads == 1 and len(state.tasks) == 1+ finally:+ storage.stop()+ api.stop()diff --git a/tests/unittest/test_kit_commands.py b/tests/unittest/test_kit_commands.pyindex 7060850e..5b91ee3a 100644--- a/tests/unittest/test_kit_commands.py+++ b/tests/unittest/test_kit_commands.py@@ -400,6 +400,7 @@ def test_kit_root_commands_keep_product_order() -> None:"parse","webui","api-server",+ "file-parse-server","vlm-server","router","models",