Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
Python ai4sciencedocument-analysisdocxextract-datalayout-analysisocrparserpdfpdf-converterpdf-extractor-llmpdf-extractor-pretrainpdf-extractor-ragpdf-parserpptxpythonxlsx
refactor(api): unify task file states and response contracts
myhloli619765f4d89d
diff --git a/docs/en/usage/http_api.md b/docs/en/usage/http_api.mdindex 7f646d9f..c785c4ba 100644--- a/docs/en/usage/http_api.md+++ b/docs/en/usage/http_api.md@@ -12,19 +12,19 @@ OUTPUT_FORMATS=default,docx MINERU_API_URL=http://127.0.0.1:8000 bash scripts/htSend application/json to POST `/v1/tasks`. Each `files` entry requires `file_name` and, for uploaded files, `bytes`; `sha256sum`, `page_range`, and `client_file_id` are optional. Omitted or null IDs are assigned the smallest available `fN`, starting at `f1`, in input order after reserving all explicit IDs. IDs remain stable in task responses and upload links. URL entries may omit bytes. Options are tier, ocr_mode, page_range and output_formats.-The response contains task_id, status_url and pending_uploads descriptors for missing files. Uploads always use PUT; upload_method is not returned. Follow each returned PUT URL and its headers; no completion request is needed. Each ready file queues independently. Waiting for uploads consumes no parsing slot.+The response contains task_id and an upload object on each file that still needs uploading. files[].upload is null otherwise; its fields are url, headers and expires_at. Uploads always use PUT; upload_method is not returned. Follow each returned PUT URL and its headers; no completion request is needed. Each ready file enters pending independently. waiting-file covers upload, server-side download and validation; only a non-null upload object asks the client to send bytes. Input preparation consumes no parsing slot.Loopback requests follow the same protocol. The SDK computes SHA-256 before submission; completed, unexpired cached inputs are not uploaded again. Concurrent first uploads may transfer duplicate bytes, but source storage is deduplicated. Default retention is 24 hours with collection every 5 minutes; active task and transfer references protect their bytes.-Upload links expire one hour after task creation. Queries do not renew them. Uploads accepted before the deadline may finish. After an uncertain upload, query first and skip ready files; retry pending uploads from the beginning while their links remain valid.+Upload links expire one hour after task creation. Queries do not renew them. Uploads accepted before the deadline may finish. After an uncertain upload, query first. pending/running/completed need no upload; waiting-file with upload=null requires polling only. Retry only when an upload target is available, within the transfer retry budget. A successful PUT is not repeated while waiting for confirmation.## Status and results-Poll status_url and match files by client_file_id. Parent states are waiting_upload, queued, running, completed, partial, failed and canceled. Each completed file has one files[].result_url; there is no top-level result_url or combined batch ZIP. Completed files can be downloaded while other files are still pending.+Poll GET `/v1/tasks/{task_id}` using the configured API base, preserving its path prefix and URL-encoding the task ID. Match files by client_file_id. Parent states are waiting-file, pending, running, completed, partial, failed and canceled. Files use the same states except partial; upload_status has been removed. Each completed file has one files[].result_url; there is no top-level result_url or combined batch ZIP. Completed files can be downloaded while other files are still pending.Outputs are default (Markdown, Middle JSON, Structured Content and images in a ZIP), docx, html, or latex (TEX and images in a ZIP). Multiple formats share one flat ZIP. DOCX and HTML embed images. ModelJson stays inside parsing and postprocessing; API, SDK, CLI and Gradio results do not include model output. The removed include_model_output option is rejected with HTTP 400, including false.-Task and file listing endpoints have been removed. DELETE `/v1/tasks/{task_id}` cancels unfinished files. Files metadata, output download and deletion by ID remain available. Per-file parse statistics contain only duration_ms and parser_version; tier remains at task level.+Task and file listing endpoints have been removed. DELETE `/v1/tasks/{task_id}` cancels unfinished files. Files metadata, output download and deletion by ID remain available. Before completion, files[].parse is null. Successful files report their actual tier, duration_ms and parser_version; there is no response-level tier. Responses no longer echo per-file sha256sum, page_range, file_id or name. Request parameters remain available.Use returned result URLs directly. Local servers and Router return Files URLs; SaaS may return signed storage URLs. Attach the API key only to same-origin API requests and reevaluate credentials at every redirect. Query the task again when a result signature expires.diff --git a/docs/next/api/changes.md b/docs/next/api/changes.mdindex 057828a8..3e9780ed 100644--- a/docs/next/api/changes.md+++ b/docs/next/api/changes.md@@ -15,9 +15,14 @@| `include_model_output` 与模型输出文件 | 参数已删除,HTTP 提交返回 400;ModelJson 仅在内部解析使用,不进入 API、SDK、CLI 或 Gradio 结果。 || Health `output_formats` | `task_output_formats`;sources 为 upload/url;删除 `task_model_output`。 || GET 任务列表 / 文件列表 | 已移除;合法认证下 `/v1/tasks` 返回 405,`/v1/files` 返回 404。 |-| 响应 `uploads` | `pending_uploads`,无旧字段别名;上传路由路径仍含 `/uploads/`。 |+| 响应 `uploads/pending_uploads` | 改为各个 `files[].upload`,子字段为 `url/headers/expires_at`,不重复携带文件 ID;上传路由路径仍含 `/uploads/`。 |+| Tasks `status_url` | 删除,客户端使用配置的 API 基址和编码后的 task_id 构造固定查询路由。 |+| 响应顶层 `tier` | 改为成功文件的 `files[].parse.tier`,取实际结果元数据;请求 tier 仍保留。 |+| 响应 `files[].sha256sum/page_range/file_id/name` | 删除;请求 hash 和页范围不变,源引用只在内部管理。 |+| `upload_status` | 删除;仅保留 files[].status。waiting-file 统一输入准备,pending 表示已就绪后的解析排队。 |+| 状态 `waiting_upload/queued` | 改为 `waiting-file/pending`,不提供旧值别名;等待文件且 upload=null 时只轮询。 || `upload_method` | 字段删除,客户端固定使用 PUT,并原样使用 URL 和请求头。 |-| `files[].parse.model_used` | 删除;仅保留 `duration_ms/parser_version`,不重复返回任务顶层 `tier`。 |+| `files[].parse.model_used` | 删除;成功后的 parse 包含 `tier/duration_ms/parser_version`,完成前为 null。 |`--allow-local-source` 和 `--max-inline-bytes` 同时删除。Files 按 ID 查询、下载与删除、Health、Tiers、Models、Usage 保留;同步 `file_parse` 继续作为独立包装服务运行。SDK 保留 `MinerUApiParser` 方法和返回类型,状态类型改为 `ApiTaskStatus`,保留 `ApiJobStatus` Python 别名。直接本地 parser 与 HF CPU/ZeroGPU 内嵌解析不经过 HTTP。@@ -25,4 +30,6 @@ SDK 保留 `MinerUApiParser` 方法和返回类型,状态类型改为 `ApiTask停收旧请求、等待在途任务完成,再协调升级 worker、Router、SDK、file_parse 包装服务和应用。SaaS Go 服务需部署方按同一契约配套更新;进程内任务和缓存索引不保证跨重启恢复。Router 通过候选 worker 的 `/v1/usage` 校验认证,不再依赖 Files 列表。-file_parse 恢复状态 namespace 增加 `tasks-v2-` 前缀,旧记录和目录保留但不加载;新空间内继续支持 request_id 幂等和重启恢复。不提供旧客户端或旧 worker 的协议回退。+file_parse 恢复状态 namespace 增加 `tasks-v4-` 前缀,旧 `tasks-v3-`、`tasks-v2-` 及无前缀记录目录保留但不加载;新空间内继续支持 request_id 幂等和重启恢复。不提供旧客户端或旧 worker 的协议回退。++Router 源缓存与下游源 Files ID 解耦,仍保留结果 Files 映射和结果租约。file_parse 自身基于 request_id 的 status_url 不属于 Tasks 响应,本次保持不变。diff --git a/docs/next/api/examples.md b/docs/next/api/examples.mdindex 19f0e11b..a4e41758 100644--- a/docs/next/api/examples.md+++ b/docs/next/api/examples.md@@ -14,7 +14,7 @@ TASK_ID=job_... MINERU_API_URL=http://127.0.0.1:8000 bash scripts/http_task_exam{"files":[{"client_file_id":"report","file_name":"report.pdf","url":"https://example.com/report.pdf"}],"output_formats":["default"]}```-对 `/v1/tasks` 执行 POST,保存返回的 `task_id/status_url`。本地文件清单包含 `bytes` 和可选 `sha256sum`;仅对响应中的待上传项使用返回的 PUT 地址。-完成后下载对应 `files[].result_url`。通过返回的 `status_url` 查询任务,取消使用 `DELETE /v1/tasks/{task_id}`;不提供任务列表。+对 `/v1/tasks` 执行 POST,保存返回的 `task_id`。本地文件清单包含 `bytes` 和可选 `sha256sum`;只对 `files[].upload` 非空的文件,使用其中的 `url/headers` 发送 PUT。+完成后下载对应 `files[].result_url`。使用 API 基址和编码后的 task_id 构造 `GET /v1/tasks/{task_id}` 查询任务,取消使用 `DELETE /v1/tasks/{task_id}`;不提供任务列表。详见 [Tasks 契约](tasks.md)、[SDK](../sdk/api-parser.md) 和 [同步包装服务](file-parse.md)。diff --git a/docs/next/api/file-parse.md b/docs/next/api/file-parse.mdindex 4ba89b8e..12a8cb7d 100644--- a/docs/next/api/file-parse.md+++ b/docs/next/api/file-parse.md@@ -100,7 +100,7 @@ X-Task-ID: job_0123456789abcdef01234567### StatusResponse:查询与 202 的完整 JSON 对象-以下七个顶层字段均会出现。状态对象不返回 Tasks 的 `files/pending_uploads/progress`,不暴露上游上传链接或签名下载地址。+以下七个顶层字段均会出现。状态对象不返回 Tasks 的 `files/progress`,不暴露上游上传链接或签名下载地址。| 字段 | JSON 类型 | 可为 null | 含义 ||---|---|---|---|@@ -116,8 +116,8 @@ X-Task-ID: job_0123456789abcdef01234567|---|---|| `prepared` | 本地源文件和恢复记录已持久化,尚未开始提交。 || `submitting` | 正在向上游创建任务,尚无确认后的 `task_id`。 |-| `waiting_upload` | 上游源文件尚未确认就绪。 |-| `queued` | 已进入上游执行队列。 |+| `waiting-file` | 上游仍在接收、下载、校验或确认输入;upload=null 不表示已经就绪。 |+| `pending` | 已进入上游执行队列。 || `running` | 上游正在解析或导出。 || `completed` | 上游完整产物已生成,可通过包装服务下载。 || `failed` | 已知失败,读取 `error`。 |@@ -287,7 +287,7 @@ Content-Type: application/json## 等待、超时与断连后查询-每次调用在确认上游文件就绪后开始计时,等待包含排队、解析与导出。接收文件、上游上传和结果下载不占此预算,独立使用传输空闲超时。缓存命中跳过上传。+每次调用在确认上游文件就绪后开始计时,等待包含排队、解析与导出。接收文件、上游上传、文件下载与校验准备及结果下载不占此预算,独立使用传输空闲超时。缓存命中跳过上传;只有进入 pending/running/completed 或失败/取消终态后,才清理本地保留的源文件。- 正常轮询间隔为 2 秒。- 连接、元数据请求、传输空闲超时分别为 10、30、600 秒。@@ -328,4 +328,4 @@ python scripts/http_file_parse_example.py report.pdf --wait-timeout 0 \## 升级与恢复记录本次协议升级要求 API Server、Router、SDK、Gradio 与包装服务同步更新,先停收请求并排空旧任务。-恢复记录目录使用 `tasks-v2-` 加既有上游地址与认证摘要,不加载、不迁移也不自动删除旧目录。旧 request_id 不再恢复;新空间内继续支持幂等和进程重启恢复。+恢复记录目录使用 `tasks-v4-` 加既有上游地址与认证摘要,不加载、不迁移也不自动删除旧 `tasks-v3-`、`tasks-v2-` 或无前缀目录。包装服务自身的 `status_url` 与 Location 头保持不变,上游 Tasks 查询改用固定路径。旧 request_id 不再恢复;新空间内继续支持幂等和进程重启恢复。diff --git a/docs/next/api/files.md b/docs/next/api/files.mdindex 089d6c0c..b917913e 100644--- a/docs/next/api/files.md+++ b/docs/next/api/files.md@@ -43,7 +43,7 @@ Files 的时间字段为 Unix 秒级整数,与 Tasks 的 ISO-8601 时间字符}```-示例中的 ID、大小、时间和 hash 为示意值。源文件元数据使用相同结构,`purpose` 为 `parse`,`file_name` 为输入文件名。+示例中的 ID、大小、时间和 hash 为示意值。本地源文件元数据使用相同结构,`purpose` 为 `parse`,`file_name` 为输入文件名。Tasks 不公开源 Files ID;Router 仅登记结果的公共 Files 映射。## DELETE /v1/files/{file_id}:删除文件视图@@ -91,4 +91,4 @@ JSON 字段使用 `file_name`;HTTP 附件头的标准参数仍是 `filename`API Server 与 Router 默认保留 86400 秒,每 300 秒扫描。`--retention-seconds` 优先于 `MINERU_API_RETENTION_SECONDS`,0 禁用正常保留期清理。上传链接固定的一小时窗口不随该设置改变。清理只删除过期视图和无引用字节。排队、执行、上传、下载持有的引用阻止物理删除;结果保留期从父任务完成资源清理后开始。-Router 映射自身 Files 地址,按调用方隔离缓存与元数据,后台维持 worker 结果租约。SaaS 可以提供对象存储签名地址,不要求实现本地 Files 下载路由。+Router 映射结果 Files 地址,源缓存由按调用方隔离的私有引用管理,不依赖下游源 Files ID。父任务清理后开始私有源缓存保留期,后台独立回收并维持 worker 结果租约。SaaS 可以提供对象存储签名地址,不要求实现本地 Files 下载路由。diff --git a/docs/next/api/tasks.md b/docs/next/api/tasks.mdindex 34cbbb27..393942e6 100644--- a/docs/next/api/tasks.md+++ b/docs/next/api/tasks.md@@ -1,7 +1,7 @@# V1 Tasks:JSON 清单、逐文件上传与下载`/v1/tasks` 是本地 API Server、Router 和 SaaS 的共同异步客户端契约。-客户端用 JSON 创建任务,按返回链接上传缺失文件,再轮询 `status_url`。+客户端用 JSON 创建任务,按 `files[].upload` 上传缺失文件,再通过固定路径 `GET /v1/tasks/{task_id}` 轮询。每个文件就绪后立即执行,完成后在 `files[].result_url` 返回独立下载链接。客户端不拼接存储路径,也不要求 SaaS 实现本地 Files API。@@ -12,13 +12,13 @@| 方法与路径 | 行为 ||---|---|| `POST /v1/tasks` | 仅接受 `application/json`,返回 `202` 和稳定 `task_id`。 |-| `PUT` 返回的 `upload_url` | 每个链接上传一个文件;本地使用 `PUT /v1/tasks/{task_id}/uploads/{client_file_id}`。 |+| `PUT` 返回的 `files[].upload.url` | 每个链接上传一个文件;本地使用 `PUT /v1/tasks/{task_id}/uploads/{client_file_id}`。 || `GET /v1/tasks/{task_id}` | 返回 `200`,包含总状态、逐文件状态、尚有效的上传链接和已完成文件的下载链接。 || `DELETE /v1/tasks/{task_id}` | 返回 `200` 和取消后的完整任务对象,保留已经完成文件的下载链接。 |创建请求不再接受表单、multipart 或文件字节,其他 Content-Type 返回 `415`。业务参数必须放在 JSON 主体中,POST 查询参数、未知字段、非法值返回 `400 invalid_request`。-上传文件的 PUT 仍发送原始字节,使用返回的 `upload_headers`;签名 URL 必须原样使用。+上传文件的 PUT 仍发送原始字节,使用返回的 `files[].upload.headers`;签名 URL 必须原样使用。Task ID 是父任务标识;内部逐文件执行 ID 不属于公共契约。| JSON 字段 | 默认值 | 说明 |@@ -43,7 +43,7 @@ ModelJson 仅用于内部解析与后处理,不属于 HTTP 或 SDK 结果。| `page_range` | 否 | 逐文件 PDF 页范围,未提供时继承全局值;非 PDF 仍只支持整本解析。 |服务端先预留全部显式 `client_file_id`,再按缺省项的输入顺序,从 `f1` 开始分配最小未占用的 `fN`。例如显式占用了 `f2`,两项缺省 ID 分别生成 `f1`、`f3`。空字符串、非法 ID 和重复显式 ID 返回 400。-创建响应的 `files` 顺序与输入一致,`files[]` 和 `pending_uploads[]` 始终返回非空 ID;生成后上传、查询及 Router 转发均保持不变。+创建响应的 `files` 顺序与输入一致,`files[]` 始终返回非空 ID;上传描述直接位于对应文件的 `upload`,不重复携带 ID。生成后的文件标识在上传、查询及 Router 转发中保持不变。先从真实文件生成 JSON 请求(本例省略 ID,由服务端补全):@@ -85,23 +85,21 @@ curl --fail-with-body "$MINERU_API_URL/v1/tasks" \JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列出的字段在当前本地 API Server 和 Router 响应中均会出现;可空字段输出 `null`,空集合输出 `[]` 或 `{}`,不以省略字段代替。客户端应忽略未来新增的响应字段。时间字符串使用 ISO-8601 UTC 格式,例如 `2026-10-23T04:00:00Z`;上传链接的 `expires_at` 单独使用 Unix 秒级整数。-请求中的 `ocr_mode` 和顶层 `page_range` 不在响应顶层回显;逐文件页范围通过 `files[].page_range` 返回。+请求中的 `tier/ocr_mode/page_range` 与逐文件 `sha256sum/page_range` 不作响应回显;实际执行档位在成功文件的 `parse.tier` 中提供。+客户端使用配置的 API 基址(包含代理前缀)追加 `/v1/tasks/{URL 编码后的 task_id}` 查询;响应不包含 `status_url`。### 顶层字段| 字段 | JSON 类型 | 可为 null | 含义 ||---|---|---|---|| `task_id` | string | 否 | 稳定的父任务 ID;是不透明标识,不代表内部逐文件执行 ID。 |-| `status` | string | 否 | 父任务状态:`waiting_upload/queued/running/completed/partial/failed/canceled`。 |-| `status_url` | string | 否 | 查询本任务的完整 URL;使用返回地址,不自行拼接。 |+| `status` | string | 否 | 父任务状态:`waiting-file/pending/running/completed/partial/failed/canceled`。 || `created_at` | string | 否 | 父任务创建时间。 || `started_at` | string | 是 | 已知的最早逐文件执行开始时间;尚无执行开始记录时为 `null`。 || `finished_at` | string | 是 | 父任务进入终态的时间;非终态时为 `null`,取消成功时也会填写。 |-| `tier` | string | 是 | 任务级档位,非空时为 `flash/basic/standard/advanced`;文件实际采用的流程还受格式约束。 || `output_formats` | array of string | 否 | 本次任务采用的输出格式,按首次出现去重。 || `progress` | object | 否 | 文件计数,结构见下表。 || `files` | array of object | 否 | 完整逐文件快照,顺序始终与创建请求一致,结构见下表。 |-| `pending_uploads` | array of object | 否 | 仍有效、尚未确认就绪的上传描述;无需上传或任务取消后为 `[]`。 || `error` | object | 是 | 父任务级错误,结构为 `ErrorDetail`;逐文件失败应检查 `files[].error`。 |### progress:文件计数@@ -122,37 +120,46 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列| `file_name` | string | 否 | 创建时提交的逻辑文件名,保留扩展名;同名文件通过 ID 区分。 || `bytes` | integer | 是 | 上传项的声明大小;URL 项可暂为 `null`,完整下载并校验后补齐实际字节数。 || `url` | string | 是 | URL 输入保留原始下载地址;客户端上传项为 `null`。 |-| `sha256sum` | string | 是 | 创建时声明的小写 SHA-256;URL 完整下载并校验后回填实际值,未声明 hash 的上传项可保持 `null`。 |-| `page_range` | string | 是 | 归一化后的逐文件页范围,初始继承请求设置,整本初始为 `""`;执行后可回填实际页号范围。 |-| `upload_status` | string | 否 | `pending/ready/not_required/expired`,与解析状态分别表达。 |-| `file_id` | string | 是 | 本地/Router 上传项的源 Files ID,准备中可能为 `null`;URL 项为 `null`。这是源文件 ID,产物使用 `result_url` 下载。 |-| `name` | string | 否 | 执行层回传的文件名;当前本地/Router 与 `file_name` 相同,仍保留此字段。 |-| `status` | string | 否 | 单文件状态:`waiting_upload/queued/running/completed/failed/canceled`;没有 `partial`。 |-| `parse` | object | 是 | 解析统计,尚无统计时为 `null`;非空结构见下表。 |+| `upload` | object | 是 | 当前可执行的客户端上传动作;服务端接收、下载、校验、输入已就绪或文件终结时为 `null`,结构见下表。 |+| `status` | string | 否 | 单文件状态:`waiting-file/pending/running/completed/failed/canceled`;没有 `partial`。 |+| `parse` | object | 是 | 解析成功完成后返回实际档位和统计;完成前、失败或取消的文件为 `null`。 || `result_url` | string | 是 | 结果完整保存后发布的唯一下载 URL;非 `completed` 文件为 `null`。 || `error` | object | 是 | 该文件的错误详情;无错误时为 `null`。 |-`upload_status` 的具体含义:`pending` 表示上传尚未确认,`ready` 表示源文件已经就绪,`not_required` 表示 URL 输入不需要客户端上传,`expired` 表示尚未完成上传的条目已过上传期限。-取消尚未上传的文件后,其 `status` 为 `canceled`,`upload_status` 仍可能是 `pending`,但不再提供上传链接。+逐文件仅使用一套 `status`:++| 状态 | 含义 |+|---|---|+| `waiting-file` | 文件准备中,包含等待客户端上传、接收文件、对象存储确认、URL 下载及完整性校验。 |+| `pending` | 输入已就绪,等待解析资源。 |+| `running` | 解析、导出及保存结果中。 |+| `completed` | 结果已保存,可通过 result_url 下载。 |+| `failed` | 本次处理失败,原因见 error。 |+| `canceled` | 已取消。 |++仅在 `waiting-file` 且 `upload` 非空时执行上传;`waiting-file` 且 `upload=null` 表示当前由服务端准备文件,客户端只轮询。+成功 PUT 后即使仍为 `waiting-file` 也不重复传输。只有上传失败、查询确认仍需上传且链接有效时,才进行有界重试。+缓存命中可以直接进入 `pending`。上传过期使用 `failed` 和 `error.code=upload_expired`,不另设过期状态;取消时 upload 为 null。### files[].parse:解析统计-仅包含耗时和解析器版本;档位保留在任务顶层 `tier`,逐文件统计不重复提供。+成功完成后包含实际执行档位、耗时和解析器版本。档位来自结果中的 MinerU 元数据,不以创建请求的 tier 代替;响应顶层不再有 tier。+PDF 与图片沿用多档位规则,HTML、Office 等原生文档使用 flash。同一批次可出现不同的逐文件档位。| 字段 | JSON 类型 | 可为 null | 含义 ||---|---|---|---|-| `duration_ms` | integer | 是 | 毫秒级文件执行耗时;当前实现包含源读取/下载、解析和导出,不含等待上传及排队时间。 |+| `tier` | string | 否 | 实际执行档位:`flash/basic/standard/advanced`。 |+| `duration_ms` | integer | 是 | 毫秒级文件执行耗时;当前实现包含源读取/下载、校验、解析和导出,不含等待客户端上传、准备名额及解析队列的时间。 || `parser_version` | string | 是 | 执行 worker 的 MinerU 版本,不保证与 Router 的版本一致。 |-### pending_uploads[]:待上传描述+### files[].upload:待上传描述-每个条目固定使用 **PUT** 上传原始文件字节;响应不包含 `upload_method`。客户端原样使用 `upload_url` 和 `upload_headers`。+该对象固定使用 **PUT** 上传原始文件字节;不包含 `client_file_id` 或 `upload_method`。客户端原样使用 `url` 和 `headers`,按外层文件标识关联输入。查询时仍返回当前有效的上传描述,便于断线恢复。| 字段 | JSON 类型 | 含义 ||---|---|---|-| `client_file_id` | string | 对应 `files[]` 中同 ID 的输入。 |-| `upload_url` | string | 上传一个文件的完整 URL;签名地址必须原样使用。 |-| `upload_headers` | object | 字符串到字符串的映射;上传时发送这些请求头,不假定总是固定头集合。 |+| `url` | string | 上传一个文件的完整 URL;签名地址必须原样使用。 |+| `headers` | object | 字符串到字符串的映射;上传时发送这些请求头,不假定总是固定头集合。 || `expires_at` | integer | Unix 秒级截止时间,创建任务后固定,查询不续期。 |### ErrorDetail:父任务或逐文件错误@@ -174,7 +181,7 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列### POST /v1/tasks:创建任务成功返回 **`202 Accepted` + `application/json`**,正文为 `TaskStatusResponse`。`202` 只表示任务已接单,不表示文件已经解析成功。-初始状态通常为 `waiting_upload`;URL 或缓存命中项可以直接进入 `queued`。`pending_uploads` 只列出当前需要客户端上传的文件。+初始状态通常为 `waiting-file`;URL 项在下载和校验后进入 `pending`,缓存命中项可直接进入 `pending`。需要客户端上传的文件具有非空 `upload`,其他文件的 `upload` 为 `null`。请求示例:@@ -194,12 +201,10 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列```json{"task_id": "job_0123456789abcdef01234567",- "status": "waiting_upload",- "status_url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567",+ "status": "waiting-file","created_at": "2026-10-23T04:00:00Z","started_at": null,"finished_at": null,- "tier": "flash","output_formats": ["default"],@@ -214,12 +219,14 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列"file_name": "report.html","bytes": 64,"url": null,- "sha256sum": null,- "page_range": "",- "upload_status": "pending",- "file_id": null,- "name": "report.html",- "status": "waiting_upload",+ "upload": {+ "url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567/uploads/f1",+ "headers": {+ "Content-Type": "application/octet-stream"+ },+ "expires_at": 1792731600+ },+ "status": "waiting-file","parse": null,"result_url": null,"error": null@@ -229,35 +236,19 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列"file_name": "notes.html","bytes": 64,"url": null,- "sha256sum": null,- "page_range": "",- "upload_status": "pending",- "file_id": null,- "name": "notes.html",- "status": "waiting_upload",+ "upload": {+ "url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567/uploads/f2",+ "headers": {+ "Content-Type": "application/octet-stream"+ },+ "expires_at": 1792731600+ },+ "status": "waiting-file","parse": null,"result_url": null,"error": null}],- "pending_uploads": [- {- "client_file_id": "f1",- "upload_url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567/uploads/f1",- "upload_headers": {- "Content-Type": "application/octet-stream"- },- "expires_at": 1792731600- },- {- "client_file_id": "f2",- "upload_url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567/uploads/f2",- "upload_headers": {- "Content-Type": "application/octet-stream"- },- "expires_at": 1792731600- }- ],"error": null}```@@ -265,17 +256,15 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列### GET /v1/tasks/{task_id}:查询任务成功返回 **`200 OK` + `application/json`**,正文始终为 `TaskStatusResponse`,包括任务仍在运行、等待上传、部分失败、全部失败或已取消时。-以下快照中,第一个文件已完成,可立即下载;第二个文件仍等待上传,所以父任务为 `waiting_upload`,`finished_at` 仍为 `null`。+以下快照中,第一个文件已完成,可立即下载;第二个文件仍等待上传,所以父任务为 `waiting-file`,`finished_at` 仍为 `null`。```json{"task_id": "job_0123456789abcdef01234567",- "status": "waiting_upload",- "status_url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567",+ "status": "waiting-file","created_at": "2026-10-23T04:00:00Z","started_at": "2026-10-23T04:00:01Z","finished_at": null,- "tier": "flash","output_formats": ["default"],@@ -290,13 +279,10 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列"file_name": "report.html","bytes": 64,"url": null,- "sha256sum": null,- "page_range": "1",- "upload_status": "ready",- "file_id": "file-source0123456789abcdef",- "name": "report.html",+ "upload": null,"status": "completed","parse": {+ "tier": "flash","duration_ms": 125,"parser_version": "4.0.12"},@@ -308,34 +294,26 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列"file_name": "notes.html","bytes": 64,"url": null,- "sha256sum": null,- "page_range": "",- "upload_status": "pending",- "file_id": null,- "name": "notes.html",- "status": "waiting_upload",+ "upload": {+ "url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567/uploads/f2",+ "headers": {+ "Content-Type": "application/octet-stream"+ },+ "expires_at": 1792731600+ },+ "status": "waiting-file","parse": null,"result_url": null,"error": null}],- "pending_uploads": [- {- "client_file_id": "f2",- "upload_url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567/uploads/f2",- "upload_headers": {- "Content-Type": "application/octet-stream"- },- "expires_at": 1792731600- }- ],"error": null}```### DELETE /v1/tasks/{task_id}:取消任务-成功返回 **`200 OK` + `application/json`**,正文为取消后的完整 `TaskStatusResponse`,`status` 为 `canceled`,`pending_uploads` 为 `[]`。+成功返回 **`200 OK` + `application/json`**,正文为取消后的完整 `TaskStatusResponse`,`status` 为 `canceled`,所有文件的 `upload` 均为 `null`。这个接口取消未完成文件,不删除 Task 或已完成的结果;返回时后台不可中断计算或资源清理可能仍在收尾。已经成功或失败的文件保持原状态,成功文件继续提供 `result_url`。对任意已终结的父任务再次取消,返回 `409 task_already_terminal`。@@ -343,11 +321,9 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列{"task_id": "job_0123456789abcdef01234567","status": "canceled",- "status_url": "http://127.0.0.1:8000/v1/tasks/job_0123456789abcdef01234567","created_at": "2026-10-23T04:00:00Z","started_at": "2026-10-23T04:00:01Z","finished_at": "2026-10-23T04:00:03Z",- "tier": "flash","output_formats": ["default"],@@ -362,13 +338,10 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列"file_name": "report.html","bytes": 64,"url": null,- "sha256sum": null,- "page_range": "1",- "upload_status": "ready",- "file_id": "file-source0123456789abcdef",- "name": "report.html",+ "upload": null,"status": "completed","parse": {+ "tier": "flash","duration_ms": 125,"parser_version": "4.0.12"},@@ -380,18 +353,13 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列"file_name": "notes.html","bytes": 64,"url": null,- "sha256sum": null,- "page_range": "",- "upload_status": "pending",- "file_id": null,- "name": "notes.html",+ "upload": null,"status": "canceled","parse": null,"result_url": null,"error": null}],- "pending_uploads": [],"error": null}```@@ -399,11 +367,11 @@ JSON 响应使用 `Content-Type: application/json`,正文为 UTF-8。下表列### PUT /v1/tasks/{task_id}/uploads/{client_file_id}:本地上传响应本地 API Server 和 Router 的上传成功返回 **`200 OK` + `application/json`**,正文仍为上面定义的完整 `TaskStatusResponse`。-本次上传文件的 `upload_status` 为 `ready`,通常从 `pending_uploads` 中移除并进入 `queued/running`;其他文件和已完成结果也包含在响应中。+本次文件完成保存及校验后,`upload` 变为 `null` 并进入 `pending/running`;其他文件和已完成结果也包含在响应中。有效期内重复 PUT 已确认文件同样返回 `200` 和当前快照;上传链接到期后返回 `410`,不会重新读取或覆盖源字节。SaaS 返回的上传目标可能是独立对象存储:其成功状态码和响应体由该存储服务决定,不要求返回 Tasks JSON。-通用客户端应接受目标服务的成功响应,随后查询 `status_url` 确认就绪情况,不把对象存储上传响应直接作为任务状态解析。+通用客户端应接受目标服务的成功响应,随后通过 `GET /v1/tasks/{task_id}` 确认就绪情况,不把对象存储上传响应直接作为任务状态解析。## 各接口错误响应@@ -452,25 +420,25 @@ URL 输入同样先创建 JSON 任务:}```-- URL 项的 `upload_status` 为 `not_required`,不生成上传链接、不接受 PUT;源 `file_id` 为 `null`。创建后即进入执行队列,不等待其他上传。-- URL 保留原始签名地址,使用逻辑 `file_name` 识别类型;不要求 HEAD 或 `Content-Length`。下载后返回实际 `bytes` 和 `sha256sum`,提供的大小/hash 必须匹配。URL 每次下载,不用 hash 缓存跳过下载。-- 上传项 hash 命中时为 `ready`,不返回上传链接,立即排队;未命中或无 hash 时返回该文件的链接。上传后仍计算 hash、校验大小并对源字节去重。+- URL 项保持 `waiting-file` 和 `upload=null`,由服务端下载和校验;确认有效后进入 `pending`。不生成客户端上传链接,不接受 PUT,不等待其他文件上传。+- URL 保留原始签名地址,使用逻辑 `file_name` 识别类型;不要求 HEAD 或 `Content-Length`。下载后回填实际 `bytes`;请求提供的大小/hash 必须匹配,但响应不回显 hash。URL 每次下载,不用 hash 缓存跳过下载。+- 上传项 hash 命中时直接进入 `pending`,不返回上传链接;未命中或无 hash 时返回该文件的链接。上传后仍计算 hash、校验大小并对源字节去重。- 缓存只复用源文件,不复用不匹配的解析结果;每个文件使用本次解析参数独立执行。-- 每个上传项校验完成即排队,不需要客户端发送完成确认。等待上传不占解析并发,也不预占 worker。+- 每个上传项校验完成即排队,不需要客户端发送完成确认。等待客户端上传不预占 worker;worker 上 URL 下载与校验使用受控准备阶段,不占解析并发。### 一小时上传窗口所有上传链接的 `expires_at` 固定为创建时起 3600 秒,采用 Unix 秒。查询不会延长或换发链接。到期后新 PUT 返回 `410 upload_expired`,包括对已经确认文件的重复 PUT。-截止前已受理的上传可以继续传完并校验,该文件仍处于 `waiting_upload`,不会被过期扫描中止。-尚未受理上传的文件到期后变为 `status=failed`、`upload_status=expired`,逐文件错误码为 `upload_expired`。+截止前已受理的上传可以继续传完并校验,该文件仍处于 `waiting-file`,不会被过期扫描中止。+尚未受理上传的文件到期后变为 `status=failed`,逐文件错误码为 `upload_expired`。URL 下载不受客户端上传截止时间影响,沿用来源下载超时。在途上传中断且已经过期时,同样进入失败终态;补交须创建新任务。上传截止时间不限制已上传文件的排队、解析或导出时间。有效期内,一个文件中断或校验失败可以从头重传,其他文件的状态不受影响。重复 PUT 已确认文件不读取或覆盖源字节,并发请求也只提交一次解析。-上传响应丢失时先查询:已经 `ready` 则跳过;仍待上传且链接有效时再重试。+上传响应丢失时先查询:进入 `pending/running/completed` 后跳过上传;仍为 `waiting-file` 且 upload 为 null 时只轮询;有有效上传目标时才有限重试。不支持单文件分块断点续传。## 总状态与逐文件状态@@ -481,21 +449,21 @@ URL 输入同样先创建 JSON 任务:完整字段、类型和可空约束见 [TaskStatusResponse](#taskstatusresponse),下面说明状态聚合规则。Task 顶层没有 `result_url`;`files[]` 不暴露底层 `output_files`。-已完成文件可以在父任务仍为 `running/queued/waiting_upload` 时下载。+已完成文件可以在父任务仍为 `running/pending/waiting-file` 时下载。总状态优先级如下:| 条件 | `status` ||---|---|| 存在运行中的文件,包括导出阶段 | `running` |-| 没有运行文件,存在排队文件 | `queued` |-| 剩余未完成文件全部等待上传 | `waiting_upload` |+| 没有运行文件,存在排队文件 | `pending` |+| 剩余未完成文件全部处于文件准备阶段 | `waiting-file` || 所有文件成功 | `completed` || 所有文件终结,成功与失败并存 | `partial` || 所有文件终结,无成功文件 | `failed` || 用户取消任务 | `canceled` |-因此,第一个文件解析完成而第二个文件尚未上传时,总状态可以由 `running` 回到 `waiting_upload`,第一个文件的下载链接仍然有效。+因此,第一个文件解析完成而第二个文件尚未上传时,总状态可以由 `running` 回到 `waiting-file`,第一个文件的下载链接仍然有效。取消只停止未完成的文件,已经成功或失败的文件保持原状态;重复取消终态任务返回 `409`。## 输出格式与下载@@ -552,7 +520,7 @@ TASK_ID=job_... MINERU_API_URL=http://127.0.0.1:8000 \| `MINERU_API_URL` | 必填 | API 或 Router 基地址。 || `MINERU_API_KEY` | 空 | 服务认证。 || `SEND_HASH` | `1` | `0` 时不计算或提交 hash。 |-| `TASK_ID` | 空 | 恢复已有任务,跳过 ready 文件。 |+| `TASK_ID` | 空 | 恢复已有任务,只处理仍有上传动作的文件。 || `MINERU_TIER`、`OCR_MODE`、`PAGE_RANGE` | 服务默认 / `auto` / 整本 | 解析参数。 || `OUTPUT_FORMATS` | `default` | 逗号分隔,转为 JSON 数组。 || `RESULT_DIR` | `results` | 逐文件结果目录。 |@@ -571,13 +539,14 @@ URL 下载沿用服务的来源策略:默认 HTTPS;HTTP 需启用 `--allow-hRouter 按就绪文件独立选择 worker 并预占负载,同一父 Task 可使用多个 worker。URL 交给 worker 下载,客户端上传字节按 worker 的缓存预检结果传输。+Router 的源缓存使用按调用方隔离的私有引用,不等待或转发 worker 的源 Files ID;结果仍通过 Files 映射下载。私有源引用在父任务完成清理后开始保留期,过期独立回收,活动传输继续保护字节。健康接口的 `features.task_output_formats` 声明 Tasks 格式,`sources` 为 `upload/url`。Router 创建任务前使用候选 worker 的受保护 `/v1/usage` 校验认证,不调用列表接口,不预占解析名额。不声明新 Tasks 能力的旧 worker 不参与 Tasks 调度,更新时应先更新 worker,再更新 Router 和客户端。Router 使用有界、可续租的内部结果租约保护 worker 产物,上传窗口不受结果租约影响;Router 丢失后租约会自行结束。已完成任务按 worker 并行续租,每轮网络等待最多 10 秒,避免失联 worker 阻塞其他 worker 的结果保留。创建请求仅接受 `files` 和 `file_name`,不兼容 `manifest`、`filename`;新旧字段混传同样返回 400。Tasks 响应及 Files 详情统一使用 `file_name`。-任务和文件列表接口已移除;使用创建响应中的 `task_id/status_url` 恢复任务,按 `files[].result_url` 下载。合法认证下 `GET /v1/tasks` 返回 405,`GET /v1/files` 返回 404。+任务和文件列表接口已移除;使用创建响应中的 `task_id` 和固定查询路由恢复任务,按 `files[].result_url` 下载。合法认证下 `GET /v1/tasks` 返回 405,`GET /v1/files` 返回 404。这是同一 `/v1` 下的破坏性变更:先停收请求并排空在途任务,再协调更新 worker、Router、SDK、file_parse 包装服务及其他客户端;SaaS 由部署方同步对齐。本次协议不保留旧表单、直传或 `markdown/middle_json/structured_content/zip` Tasks 格式别名。diff --git a/docs/next/api/webhooks.md b/docs/next/api/webhooks.mdindex 54be6aca..fa36178b 100644--- a/docs/next/api/webhooks.md+++ b/docs/next/api/webhooks.md@@ -1,5 +1,5 @@# 任务通知-当前 Tasks 契约采用轮询,不接受 `callback`,也不提供 SSE 或 Webhook。客户端使用返回的 `status_url` 查询;批量任务的单个文件完成即可下载,无需等待整个批次终结。+当前 Tasks 契约采用轮询,不接受 `callback`,也不提供 SSE 或 Webhook。客户端使用配置的 API 基址与 `GET /v1/tasks/{task_id}` 查询;批量任务的单个文件完成即可下载,无需等待整个批次终结。SDK 提供进程内 `status_callback` 与 `duration_callback`,它们不是服务端 Webhook。未来增加推送能力需要另行定义认证、重试和幂等契约。diff --git a/docs/next/cli/mineru-kit-api-server.md b/docs/next/cli/mineru-kit-api-server.mdindex 14336c8e..bfae0fab 100644--- a/docs/next/cli/mineru-kit-api-server.md+++ b/docs/next/cli/mineru-kit-api-server.md@@ -239,3 +239,9 @@ Torch 小模型首次初始化时限制 CPU 线程,并传播到后续工作线配置优先,否则依次使用 `MINERU_CPU_NUM_THREADS`、`MINERU_INTRA_OP_NUM_THREADS`、8。按 CPU 亲和性及 cgroup 配额裁剪,且不调高已有更低值。单独设置 OpenBLAS 不视为 Torch线程限制。纯 ONNX/Flash 的启动路径保持惰性加载。++### 文件准备与解析排队++Tasks 使用 waiting-file 表示等待上传、服务端下载或校验,准备完成后进入 pending,取得解析名额后进入 running。+URL 输入由独立准备阶段处理,其并发上限沿用 concurrency;等待解析名额的已准备输入也占准备预算,防止源字节无界缓冲。准备阶段不占解析名额,已上传就绪的文件可以正常执行。+文件 duration_ms 包含实际下载、校验、解析和导出耗时,不包含准备或解析队列等待时间。diff --git a/docs/next/cli/mineru-kit-webui.md b/docs/next/cli/mineru-kit-webui.mdindex f6b3a20f..fb96ab71 100644--- a/docs/next/cli/mineru-kit-webui.md+++ b/docs/next/cli/mineru-kit-webui.md@@ -103,11 +103,11 @@ Gradio 转换事件保留 `raw_page_range` 字符串接口,仍接受 V1 `page_Gradio 首先发现 `/v1/health` 和 `/v1/tiers`,然后通过 `MinerUApiParser` 提交 hash 清单到 `/v1/tasks`、按需上传、轮询父任务并下载逐文件默认 ZIP 结果。解析结果保存为严格 Middle JSON 2.0。-左下角使用与 3.4.5 一致的两列步骤卡片,展示准备请求、检查服务、提交任务、排队、解析中、下载结果、整理输出和完成。排队状态来自本地任务槽或 V1 的 `queued` 响应,圆点每秒循环更新;不显示前方任务数。每个界面服务仍一次执行一个转换,等待中的会话也会立即显示排队动画。+左下角使用与 3.4.5 一致的两列步骤卡片,展示准备请求、检查服务、提交任务、排队、解析中、下载结果、整理输出和完成。文件准备状态来自 V1 的 `waiting-file` 响应;排队状态来自本地任务槽或 V1 的 `pending` 响应,圆点每秒循环更新;不显示前方任务数。每个界面服务仍一次执行一个转换,等待中的会话也会立即显示排队动画。收到 V1 的 `running` 后开始解析计时,由浏览器本地刷新;收到任务终态时冻结。API 路径的最终完成文案优先使用响应中已有的 `files[0].parse.duration_ms`,包含服务端解析和结果导出/打包,不包含排队、客户端下载和 WebUI 整理输出,因此即使轮询跳过 `running` 也能显示亚秒耗时。HF 等直接调用解析函数的适配层继续使用实际阶段通知之间的计时;缺少 API 耗时字段时也保留这种计时回退。完成数值按两位小数四舍五入,最低显示 `0.01 秒`。两种路径均不承诺该值等同于日志中的纯 `infer` 耗时。界面动画不增加 API 请求,任务轮询仍为每秒一次。清除或更换文件会停止当前会话的状态流并释放本地等待位置,不向 V1 发送远端取消请求。-`MinerUApiParser.parse()` 和 `parse_async()` 提供可选的 `status_callback` 参数,接收公开的 `ApiTaskStatus`(`waiting_upload`、`queued`、`running`、`completed`、`partial`、`failed`、`canceled`)。回调在任务提交响应及每次轮询响应后同步调用;回调异常只记录日志。这里的 `completed` 表示服务端任务完成,解析方法仍需下载并构造 `ParseResult` 后才返回。可选的 `duration_callback` 在最终成功响应之后、开始下载之前接收既有文件耗时(浮点毫秒);缺失或非法耗时不触发回调,回调异常不影响解析结果。该值属于本次调用,不存放在共享 parser 实例或 `ParseResult` 中。+`MinerUApiParser.parse()` 和 `parse_async()` 提供可选的 `status_callback` 参数,接收公开的 `ApiTaskStatus`(`waiting-file`、`pending`、`running`、`completed`、`partial`、`failed`、`canceled`)。回调在任务提交响应及每次轮询响应后同步调用;回调异常只记录日志。这里的 `completed` 表示服务端任务完成,解析方法仍需下载并构造 `ParseResult` 后才返回。可选的 `duration_callback` 在最终成功响应之后、开始下载之前接收既有文件耗时(浮点毫秒);缺失或非法耗时不触发回调,回调异常不影响解析结果。该值属于本次调用,不存放在共享 parser 实例或 `ParseResult` 中。## 结果与下载@@ -145,4 +145,4 @@ PDF 和图片会生成与解析范围一致的 `origin.pdf`;布局预览使用`mineru-webui` 提供为 `mineru-kit webui` 的独立命令名,参数与行为完全相同;旧 HTTP 协议和旧 Gradio 专属参数不再提供。-Gradio 使用 Tasks 能力发现和 `pending_uploads`,固定使用 PUT 上传;仅保存 Middle JSON 和素材,下载菜单从保存的 Middle JSON 按需导出。+Gradio 使用 Tasks 能力发现和逐文件 `upload` 描述,固定使用 PUT 上传;仅保存 Middle JSON 和素材,下载菜单从保存的 Middle JSON 按需导出。diff --git a/docs/next/sdk/api-parser.md b/docs/next/sdk/api-parser.mdindex 1f0edff0..7dfff3d0 100644--- a/docs/next/sdk/api-parser.md+++ b/docs/next/sdk/api-parser.md@@ -19,8 +19,8 @@ print(result.markdown())1. 校验文件及页范围,计算大小和 SHA-256,包括同机 HTTP 请求。2. 向 `/v1/tasks` 提交一个 `client_file_id="file"` 的 files 数组,固定请求 `output_formats=["default"]`。-3. 缓存命中不上传;未命中遍历 `pending_uploads`,固定使用 PUT 向返回的链接按 headers 流式上传,并按清单大小设置 `Content-Length`。无需完成确认。-4. 按返回的同源 `status_url` 每秒查询,轮询预算 3600 次;用 `client_file_id` 关联结果。+3. 缓存命中不上传;未命中遍历 `files[]` 中非空的 `upload`,固定使用 PUT 向返回的链接按 headers 流式上传,并按清单大小设置 `Content-Length`。无需完成确认。+4. 以配置的 API 基址追加 `/v1/tasks/{编码后的 task_id}` 每秒查询,保留代理路径前缀,轮询预算 3600 次;用 `client_file_id` 关联结果。5. 从该文件的 `result_url` 流式下载并校验 ZIP,读取 `middle_json.json`。6. `include_images=True` 恢复图片素材。`ParseResult` 只持有 MiddleJson,不携带或导出 ModelJson。@@ -28,11 +28,11 @@ HTTP 输入只有上传和 URL;SDK 的路径参数始终代表客户端文件## 状态和错误-`ApiTaskStatus` 包含 `waiting_upload/queued/running/completed/partial/failed/canceled`;`ApiJobStatus` 是保留的 Python 类型别名。+`ApiTaskStatus` 包含 `waiting-file/pending/running/completed/partial/failed/canceled`;`ApiJobStatus` 是保留的 Python 类型别名。状态回调在创建和查询后触发,回调异常只记录日志。服务端完成表示开始下载,不等于 SDK 已经返回。-`duration_callback` 在开始下载前接收服务端单文件 `parse.duration_ms`,包含解析和导出;缺失或非法值不触发回调。+单文件成功后的 `parse` 包含实际 `tier`、`duration_ms` 和 `parser_version`。`duration_callback` 在开始下载前接收服务端单文件 `parse.duration_ms`,包含解析和导出;缺失或非法值不触发回调。-创建请求没有幂等保证,不自动重试。上传响应丢失后先查状态:已 ready 则继续,仍 pending 才重传。查询、传输和下载有界重试;任务查询和用量查询对 408/429/500/502/503/504 最多请求 3 次,支持 `Retry-After` 秒数和 HTTP 日期。单次建议等待超过 60 秒时返回原错误,不提前重试;无有效建议时使用指数退避。下载签名失效后重新查询任务。密钥仅附加到 API 同源请求,每次重定向重新判断来源。+创建请求没有幂等保证,不自动重试。上传响应丢失后先查状态:进入 pending/running/completed 后不重传;waiting-file 且 upload=null 时仅轮询;出现上传目标后才在原重试预算内发送。PUT 成功后等待确认,不因状态仍是 waiting-file 而重复上传。查询、传输和下载有界重试;任务查询和用量查询对 408/429/500/502/503/504 最多请求 3 次,支持 `Retry-After` 秒数和 HTTP 日期。单次建议等待超过 60 秒时返回原错误,不提前重试;无有效建议时使用指数退避。下载签名失效后重新查询任务。密钥仅附加到 API 同源请求,每次重定向重新判断来源。结构化业务错误保留 code/message/param。传输失败保留 `_APITransportError` 阶段及超时信息,Doclib 继续区分 `remote_timeout`、`remote_unreachable` 与业务 `parse_failed`。diff --git a/docs/zh/usage/http_api.md b/docs/zh/usage/http_api.mdindex d48195ee..b950e98c 100644--- a/docs/zh/usage/http_api.md+++ b/docs/zh/usage/http_api.md@@ -14,14 +14,14 @@ Tasks 与 Files JSON 统一使用 `file_name`;旧字段 `manifest`、`filename向 `POST /v1/tasks` 提交 `application/json`:`files` 每项包含必填逻辑 `file_name`、上传文件的 `bytes` 和可选 `client_file_id/sha256sum/page_range`。`client_file_id` 省略或为 `null` 时,先预留全部显式 ID,再按输入顺序从 `f1` 开始分配最小未占用的 `fN`;创建后上传和查询均保持不变。也可以提供 URL,此时 bytes 可省略。解析选项为 `tier/ocr_mode/page_range/output_formats`。-创建返回稳定的 `task_id/status_url` 和 `pending_uploads` 待上传描述。客户端固定使用 PUT 上传缺失文件,使用响应指定的 PUT 地址及请求头,无需 complete。每个文件就绪即独立排队,其他文件等待上传不会阻塞它。+创建返回稳定的 `task_id`;每个文件的 `upload` 为待上传描述或 `null`,包含 `url/headers/expires_at`。客户端固定使用 PUT 上传缺失文件,使用响应指定的 PUT 地址及请求头,无需 complete。每个文件就绪即独立排队,其他文件等待上传不会阻塞它。同机也使用相同流程。SDK 总是计算 hash;已完成上传的有效缓存可以免传。并发首次请求允许重复传输,但源字节去重。缓存默认保留 24 小时、每 5 分钟扫描,正在执行及传输的引用受保护。-上传链接固定有效一小时,查询不会续期。截止前已受理上传可继续完成。中断后先查询,ready 文件跳过,其余在有效期内从头重传;过期项需创建新任务。+上传链接固定有效一小时,查询不会续期。截止前已受理上传可继续完成。中断后先查询,进入 pending/running/completed 后跳过上传;waiting-file 且 upload=null 时只等待,有有效上传目标时才有限重试。过期文件进入 failed,错误码为 upload_expired。## 状态与结果-使用返回的 `status_url` 轮询。父任务状态为 `waiting_upload/queued/running/completed/partial/failed/canceled`;通过 `client_file_id` 对应输入。+使用配置的 API 基址(保留代理前缀)和编码后的 task_id 构造 `GET /v1/tasks/{task_id}` 轮询。父任务状态为 `waiting-file/pending/running/completed/partial/failed/canceled`;通过 `client_file_id` 对应输入。逐文件使用同一组状态但没有 partial,不再返回 upload_status。waiting-file 包括上传、服务端下载和校验,pending 表示输入已就绪并排队等待解析。每个成功文件在 `files[].result_url` 提供唯一下载地址,即使父任务尚未完成也可下载。顶层没有 result_url,不生成跨输入文件的总 ZIP。| output_formats | 内容 |@@ -33,7 +33,7 @@ Tasks 与 Files JSON 统一使用 `file_name`;旧字段 `manifest`、`filename| 多种 | 对应文件平铺在一个 ZIP 中 |ModelJson 仅在解析内部使用,API、SDK、CLI 和 Gradio 均不输出模型文件。`include_model_output` 已删除,提交该字段返回 400。-`DELETE /v1/tasks/{task_id}` 取消未完成文件。任务和文件列表已移除;Files 的按 ID 元数据、结果下载和删除保留。逐文件 `parse` 仅包含耗时与解析器版本,档位位于任务顶层 `tier`。+`DELETE /v1/tasks/{task_id}` 取消未完成文件。任务和文件列表已移除;Files 的按 ID 元数据、结果下载和删除保留。逐文件 `parse` 在完成前为 `null`,成功后包含实际 `tier`、耗时和解析器版本;响应顶层不再有 tier。逐文件响应不回显 hash、页范围、源 file_id 或重复的 name;请求参数仍保留。客户端原样使用结果链接:本地/Router 返回 Files 地址,SaaS 可返回对象存储签名地址。只为 API 同源请求附加 API Key,每次重定向重新判断;签名失效可重新查询结果链接。diff --git a/mineru/kit/file_parse/client.py b/mineru/kit/file_parse/client.pyindex cab33446..2b1abe8d 100644--- a/mineru/kit/file_parse/client.py+++ b/mineru/kit/file_parse/client.py@@ -14,6 +14,7 @@ from urllib.parse import urljoinimport httpxfrom pydantic import ValidationError+from ...task_types import task_status_urlfrom .models import GatewayError, Record, RemoteTaskfrom .settings import FileParseSettings, origin@@ -126,9 +127,13 @@ class TaskClient:raise ValueError("Task identity changed")if list(dict.fromkeys(task.output_formats)) != formats:raise ValueError("Task output formats changed")- task.status_url = self.resolve(task.status_url, metadata=True)- for link in task.pending_uploads:- link.upload_url = self.resolve(link.upload_url)+ if expected is not None and (+ task.files[0].file_name != expected.files[0].file_name or task.files[0].bytes != expected.files[0].bytes+ ):+ raise ValueError("Task input identity changed")+ for item in task.files:+ if item.upload is not None:+ item.upload.url = self.resolve(item.upload.url)if task.files[0].result_url:task.files[0].result_url = self.resolve(task.files[0].result_url)return task@@ -189,48 +194,53 @@ class TaskClient:)async def query(self, task: RemoteTask) -> RemoteTask:- """按服务给出的地址查询原任务,失败不会触发创建操作。"""- return await self._json_request("GET", task.status_url, task.output_formats, expected=task)+ """保留部署前缀并按稳定任务 ID 查询,失败不会触发创建操作。"""+ return await self._json_request(+ "GET", task_status_url(self.settings.api_url, task.task_id), task.output_formats, expected=task+ )async def upload(self, task: RemoteTask, path: Path, size: int) -> RemoteTask:- """上传失败先查询确认,最多重传三次,绝不续期或创建替代任务。"""- for attempt in range(3):- item = task.files[0]- if item.upload_status == "ready" or item.status in {"completed", "failed", "canceled"}:- return task- if not task.pending_uploads:- raise GatewayError("upstream_protocol_error", "Pending file has no upload link")- link = task.pending_uploads[0]+ """文件准备期间只轮询,明确出现上传目标后才发送,最多重试三次。"""+ attempts = 0+ uploaded = False+ last_error: GatewayError | None = None+ while task.files[0].status == "waiting-file":+ link = task.files[0].upload+ if link is None or uploaded:+ await asyncio.sleep(self.settings.poll_interval)+ task = await self.query(task)+ continue+ if attempts >= 3:+ raise last_error or GatewayError("upload_failed", "Upload retry budget exhausted")if link.expires_at <= time.time():raise GatewayError("upload_expired", "The task upload link has expired", 410)+ attempts += 1try:- headers = self.headers(link.upload_url, link.upload_headers)+ headers = self.headers(link.url, link.headers)headers["content-length"] = str(size)async with self.client.stream(- "PUT",- link.upload_url,- headers=headers,- content=file_chunks(path),- timeout=self.transfer_timeout,+ "PUT", link.url, headers=headers, content=file_chunks(path), timeout=self.transfer_timeout) as response:body = await self._read_metadata(response)if not 200 <= response.status_code < 300:raise self._http_error(response.status_code, body)- return await self.query(task)+ uploaded = True+ task = await self.query(task)except (httpx.HTTPError, GatewayError) as exc:- error = (+ last_error = (excif isinstance(exc, GatewayError)- else GatewayError("upload_interrupted", "Upstream upload was interrupted", retryable=True)+ else GatewayError("upload_interrupted", "The task upload was interrupted", retryable=True))task = await self.query(task)- item = task.files[0]- if item.upload_status == "ready" or item.status in {"completed", "failed", "canceled"}:+ if task.files[0].status != "waiting-file":return task- if attempt == 2 or not error.retryable:- raise error from exc- await asyncio.sleep(self.retry_delay * (attempt + 1))- raise AssertionError("unreachable")+ if task.files[0].upload is None:+ continue+ if attempts == 3 or not last_error.retryable:+ raise last_error from exc+ await asyncio.sleep(self.retry_delay * attempts)+ return taskasync def _download_once(self, url: str, path: Path) -> None:"""逐跳重新决定认证并流式写盘,在完整接收前不向调用者发送成功响应。"""diff --git a/mineru/kit/file_parse/models.py b/mineru/kit/file_parse/models.pyindex 3aedcbee..8ec8a9e4 100644--- a/mineru/kit/file_parse/models.py+++ b/mineru/kit/file_parse/models.py@@ -9,10 +9,10 @@ from typing import Any, Literalfrom pydantic import BaseModel, ConfigDict, Field, model_validator-from ...task_types import TaskOutputFormat+from ...task_types import TaskFileState, TaskOutputFormat, TaskStatus, task_status_urlTERMINAL = frozenset({"completed", "failed", "canceled", "submission_unknown"})-FileStatus = Literal["waiting_upload", "queued", "running", "completed", "failed", "canceled"]+FileStatus = TaskFileStateclass GatewayError(Exception):@@ -60,41 +60,43 @@ class ParseOptions(BaseModel):return self+class UploadLink(BaseModel):+ """保留文件内嵌上传地址、请求头和固定截止时间。"""++ url: str+ headers: dict[str, str]+ expires_at: int++class RemoteFile(BaseModel):"""仅消费共同 Tasks 契约,不依赖本地 Files 或 Job 实现。"""client_file_id: str+ file_name: str+ bytes: intstatus: FileStatus- upload_status: Literal["pending", "ready", "not_required", "expired"]+ upload: UploadLink | Noneresult_url: str | None = Noneerror: dict[str, Any] | None = None-class UploadLink(BaseModel):- """保留原始上传地址、请求头和固定截止时间。"""-- client_file_id: str- upload_url: str- upload_headers: dict[str, str]- expires_at: int--class RemoteTask(BaseModel):"""校验单文件任务的身份和产物契约,允许上游添加其他元数据。"""task_id: str = Field(min_length=1)- status: Literal["waiting_upload", "queued", "running", "completed", "partial", "failed", "canceled"]- status_url: str = Field(min_length=1)+ status: TaskStatusfiles: list[RemoteFile] = Field(min_length=1, max_length=1)- pending_uploads: list[UploadLink] = Field(max_length=1)output_formats: list[TaskOutputFormat] = Field(min_length=1)@model_validator(mode="after")def validate_file(self) -> RemoteTask:"""确保返回的文件和链接仍对应本次唯一清单项。"""item = self.files[0]- if item.client_file_id != "file" or any(link.client_file_id != "file" for link in self.pending_uploads):+ task_status_url("", self.task_id)+ if item.client_file_id != "file":raise ValueError("Unexpected client_file_id")+ if item.upload is not None and item.status != "waiting-file":+ raise ValueError("Only waiting-file accepts uploads")if item.status == "completed" and not item.result_url:raise ValueError("Completed file has no result_url")if item.status != "completed" and item.result_url is not None:diff --git a/mineru/kit/file_parse/service.py b/mineru/kit/file_parse/service.pyindex fb63d298..53b564eb 100644--- a/mineru/kit/file_parse/service.py+++ b/mineru/kit/file_parse/service.py@@ -145,15 +145,13 @@ class FileParseService:if record.remote is not None and record.status in TERMINAL and item.status not in TERMINAL:returnupdated = replace(record, remote=task.model_dump(), status=item.status)- if item.upload_status == "ready" or item.status in TERMINAL:+ if item.status != "waiting-file":updated.error = item.errorif item.status in {"failed", "canceled"} and updated.error is None:updated.error = {"code": f"task_{item.status}", "message": f"Upstream task {item.status}"}if updated.status in TERMINAL and updated.terminal_at is None:updated.terminal_at = time.time()- await self._save_and_cleanup(- record, updated, remove_source=item.upload_status == "ready" or item.status in TERMINAL- )+ await self._save_and_cleanup(record, updated, remove_source=item.status != "waiting-file")async def _remember_error(self, record: Record, error: GatewayError) -> None:"""无任务 ID 的失败是本地终态;已知任务的传输错误保留后续查询能力。"""@@ -183,7 +181,7 @@ class FileParseService:await self._apply(record, task)else:task = await self.refresh(record)- if task.files[0].upload_status == "pending" and task.files[0].status not in TERMINAL:+ if task.files[0].status == "waiting-file":if record.source_path is None or not Path(record.source_path).is_file():raise GatewayError("source_unavailable", "The retained source is unavailable")task = await self.client.upload(task, Path(record.source_path), record.size)@@ -265,9 +263,9 @@ class FileParseService:changed.clear()self.raise_record_error(record)task = record.task- if record.status in TERMINAL or (task is not None and task.files[0].upload_status == "ready"):+ if record.status in TERMINAL or (task is not None and task.files[0].status != "waiting-file"):break- # 对象存储 PUT 成功后,网关可能稍后才确认 ready;这段等待仍属于上传准备。+ # 文件准备包括上传确认与服务端下载;进入 pending 后才启动解析等待预算。await changed.wait()if timeout == 0:return Falsediff --git a/mineru/kit/file_parse/settings.py b/mineru/kit/file_parse/settings.pyindex 6e7083fe..0903a353 100644--- a/mineru/kit/file_parse/settings.py+++ b/mineru/kit/file_parse/settings.py@@ -56,7 +56,7 @@ class FileParseSettings:@propertydef namespace(self) -> str:"""按新任务协议、上游和认证隔离记录,保留但不加载旧协议状态。"""- return "tasks-v2-" + hashlib.sha256(f"{self.api_url}\0{self.api_key or ''}".encode()).hexdigest()+ return "tasks-v4-" + hashlib.sha256(f"{self.api_url}\0{self.api_key or ''}".encode()).hexdigest()__all__ = ["FileParseSettings", "origin"]diff --git a/mineru/kit/gradio/client.py b/mineru/kit/gradio/client.pyindex 7343a0c8..79409661 100644--- a/mineru/kit/gradio/client.py+++ b/mineru/kit/gradio/client.py@@ -31,7 +31,7 @@ from .status import (STATUS_PROCESSING_OUTPUT,STATUS_QUEUED_ON_SERVER,STATUS_SUBMITTING_TASK,- STATUS_UPLOADING_FILE,+ STATUS_PREPARING_FILE,ParseStatusUpdate,)@@ -215,8 +215,8 @@ class V1ArtifactClient:def on_job_status(status: ApiTaskStatus) -> None:"""将真实 V1 状态映射到卡片阶段,服务端完成仅代表开始下载。"""messages: dict[ApiTaskStatus, str] = {- "waiting_upload": STATUS_UPLOADING_FILE,- "queued": STATUS_QUEUED_ON_SERVER,+ "waiting-file": STATUS_PREPARING_FILE,+ "pending": STATUS_QUEUED_ON_SERVER,"running": STATUS_PROCESSING_ON_SERVER,"completed": STATUS_DOWNLOADING_RESULT,"partial": STATUS_DOWNLOADING_RESULT,diff --git a/mineru/kit/gradio/i18n.py b/mineru/kit/gradio/i18n.pyindex 04c957e7..cc2a5b22 100644--- a/mineru/kit/gradio/i18n.py+++ b/mineru/kit/gradio/i18n.py@@ -122,7 +122,7 @@ MESSAGES: dict[str, tuple[str, str]] = {"status_step_failed": ("Failed", "失败"),"preparing_request": ("Preparing request...", "正在准备请求…"),"checking_server": ("Checking server status...", "正在检查服务状态…"),- "uploading_file": ("Uploading file...", "正在上传文件…"),+ "preparing_file": ("Preparing file...", "正在准备文件…"),"submitting_task": ("Submitting task...", "正在提交任务…"),"queued_locally": ("Queued locally", "本地排队中"),"queued_on_server": ("Queued on server", "服务端排队中"),diff --git a/mineru/kit/gradio/status.py b/mineru/kit/gradio/status.pyindex 9ec1e53c..b82ec1b8 100644--- a/mineru/kit/gradio/status.py+++ b/mineru/kit/gradio/status.py@@ -14,7 +14,7 @@ DEFAULT_STATUS = "Upload a file and start conversion."STATUS_PREPARING_REQUEST = "Preparing request..."STATUS_CHECKING_SERVER = "Checking server status..."STATUS_SUBMITTING_TASK = "Submitting task..."-STATUS_UPLOADING_FILE = "Uploading file..."+STATUS_PREPARING_FILE = "Preparing file..."STATUS_QUEUED_LOCALLY = "Queued locally"STATUS_QUEUED_ON_SERVER = "Queued on server"STATUS_PROCESSING_ON_SERVER = "Processing on server..."@@ -27,7 +27,7 @@ _MESSAGE_STEPS = {STATUS_PREPARING_REQUEST: 0,STATUS_CHECKING_SERVER: 1,STATUS_SUBMITTING_TASK: 2,- STATUS_UPLOADING_FILE: 2,+ STATUS_PREPARING_FILE: 2,STATUS_QUEUED_LOCALLY: 3,STATUS_QUEUED_ON_SERVER: 3,STATUS_PROCESSING_ON_SERVER: 4,@@ -159,7 +159,7 @@ __all__ = ["STATUS_QUEUED_LOCALLY","STATUS_QUEUED_ON_SERVER","STATUS_SUBMITTING_TASK",- "STATUS_UPLOADING_FILE",+ "STATUS_PREPARING_FILE","ParseStatusUpdate","StatusPanelState","status_html",diff --git a/mineru/kit/router/app.py b/mineru/kit/router/app.pyindex 17cec6e8..88e8bd2a 100644--- a/mineru/kit/router/app.py+++ b/mineru/kit/router/app.py@@ -226,6 +226,7 @@ def create_app(while True:await asyncio.sleep(RETENTION_SCAN_INTERVAL_SECONDS)registry.collect_expired(source_store)+ source_store.collect_expired()try:if resolved_settings.retention_seconds:diff --git a/mineru/kit/router/resources.py b/mineru/kit/router/resources.pyindex cdc0526f..76e9133b 100644--- a/mineru/kit/router/resources.py+++ b/mineru/kit/router/resources.py@@ -59,7 +59,8 @@ class SourceFileStore:self._temp_dir = tempfile.TemporaryDirectory(prefix="mineru-v1-router-sources-")self._root = Path(self._temp_dir.name)self.retention_seconds = retention_seconds- self._file_expiry: dict[str, int | None] = {}+ self._file_expiry: dict[str, float | None] = {}+ self._active_references: set[str] = set()self._uploads: dict[str, StoredSourceFile] = {}self._files: dict[str, StoredSourceFile] = {}self._bound_uploads: set[str] = set()@@ -75,7 +76,7 @@ class SourceFileStore:mime_type: str,max_bytes: int | None = None,) -> StoredSourceFile:- """流式写入一个公共 Upload 的输入字节,并计算实际大小与 SHA256。"""+ """流式写入私有暂存输入,并计算实际大小与 SHA256。"""if upload_id in self._bound_uploads:raise ValueError(f"Upload {upload_id} is already bound to a completed file")path = self._root / "uploads" / upload_id@@ -111,14 +112,14 @@ class SourceFileStore:def bind_file(self, upload_id: str, file_id: str, *, owner_scope: str = "anonymous", expires_at: int | None = None) -> StoredSourceFile:- """把已完成 Upload 的暂存输入绑定到 Router 公共 File。"""+ """把已完成上传的暂存输入绑定到 Router 私有源引用。"""stored = self._uploads.pop(upload_id)self.bind_source(file_id, stored, owner_scope=owner_scope, expires_at=expires_at)self._bound_uploads.add(upload_id)return storeddef bind_source(self, file_id: str, stored: StoredSourceFile, *, owner_scope: str, expires_at: int | None = None) -> None:- """为公共文件增加缓存引用,并建立调用方隔离的哈希索引。"""+ """增加私有源缓存引用,并建立调用方隔离的哈希索引。"""previous = self._files.get(file_id)if previous is not None:previous_scope = self._file_scopes.get(file_id, "anonymous")@@ -138,6 +139,27 @@ class SourceFileStore:if previous is not None:self._release_path(previous.path)+ def hold_reference(self, reference_id: str) -> None:+ """准备和执行阶段固定源引用,父任务清理前不开始保留期。"""+ if reference_id not in self._files:+ raise KeyError(reference_id)+ self._active_references.add(reference_id)+ self._file_expiry[reference_id] = None++ def release_reference(self, reference_id: str, finished: float) -> None:+ """首次解除任务引用时开始缓存保留期,重复释放不延长期限。"""+ if reference_id not in self._active_references:+ return+ self._active_references.remove(reference_id)+ self._file_expiry[reference_id] = finished + self.retention_seconds if self.retention_seconds else None++ def collect_expired(self) -> None:+ """独立回收过期私有引用,传输中的字节仍由 pin 保护。"""+ now = time.time()+ for reference_id, deadline in tuple(self._file_expiry.items()):+ if reference_id not in self._active_references and deadline is not None and deadline <= now:+ self.delete_file(reference_id)+def find_hash(self, owner_scope: str, sha256sum: str) -> StoredSourceFile | None:"""只查当前调用方仍持有的源字节,不使用 worker 的哈希声明。"""for file_id in sorted(self._hash_files.get((owner_scope, sha256sum), set())):@@ -168,7 +190,7 @@ class SourceFileStore:path.unlink(missing_ok=True)def find_file(self, file_id: str) -> StoredSourceFile | None:- """读取公共 File 对应的私有暂存输入,不存在时返回 None。"""+ """按私有源引用读取缓存输入,不存在时返回 None。"""return self._files.get(file_id)def discard_upload(self, upload_id: str) -> None:@@ -181,7 +203,8 @@ class SourceFileStore:self._release_path(stored.path)def delete_file(self, file_id: str) -> None:- """删除公共 File 绑定的私有暂存输入。"""+ """删除私有源引用,只有最后一个引用及传输租约释放后才删除字节。"""+ self._active_references.discard(file_id)self._file_expiry.pop(file_id, None)stored = self._files.pop(file_id, None)if stored is not None:@@ -200,6 +223,7 @@ class SourceFileStore:self._bound_uploads.clear()self._file_scopes.clear()self._file_expiry.clear()+ self._active_references.clear()self._hash_files.clear()self._pins.clear()self._temp_dir.cleanup()diff --git a/mineru/kit/router/tasks.py b/mineru/kit/router/tasks.pyindex 2293af9b..6cbe0409 100644--- a/mineru/kit/router/tasks.py+++ b/mineru/kit/router/tasks.py@@ -39,7 +39,7 @@ from .proxy import (require_worker_generation,rewrite_file_payload,)-from .resources import ResourceRegistry, ResourceRoute, SourceFileStore, StoredSourceFile, utc_now_iso+from .resources import ResourceRegistry, ResourceRoute, SourceFileStore, utc_now_isofrom .workers import WorkerPool, WorkerStateFinalizeJob = Callable[[ResourceRoute, dict[str, Any], Request | None], Awaitable[None]]@@ -113,18 +113,14 @@ def _worker_path(worker: WorkerState, url: str) -> str:def _job_payload(payload: dict[str, Any], upstream_id: str) -> dict[str, Any]:"""把 worker Tasks 转换为私有执行快照,不包含旧 Jobs 公开协议。"""- file_keys = ("file_id", "name", "page_range", "status", "parse", "error")+ file_keys = ("status", "parse", "error")files = [{key: item.get(key) for key in file_keys} for item in payload["files"]]- for file in files:- if file["status"] in {"waiting_upload", "canceled"}:- file["status"] = "queued"return {"job_id": upstream_id,- "status": "queued" if payload["status"] == "waiting_upload" else payload["status"],+ "status": payload["status"],"created_at": payload["created_at"],"started_at": payload.get("started_at"),"finished_at": payload.get("finished_at"),- "tier": payload["tier"],"progress": payload["progress"],"files": files,}@@ -211,19 +207,15 @@ async def read_task_execution(if result_route is not None:result_route.metadata["task_result_held"] = bool(route.metadata.get("task_result_held"))rewritten = _job_payload(payload, route.public_id)- aliases = route.metadata.get("input_aliases") or {}- for item in rewritten["files"]:- if item.get("file_id") in aliases:- item["file_id"] = aliases[item["file_id"]]if rewritten["status"] in TERMINAL_TASK_STATUSES:release_job_load(route, pool)route.metadata["payload"] = copy.deepcopy(rewritten)route.metadata["task_result_file_id"] = result_file_iderror = ErrorDetail.model_validate(payload["error"]) if payload.get("error") else Noneinput_metadata = {- index: _JobInputMetadata(item["bytes"], item["sha256sum"])+ index: _JobInputMetadata(item["bytes"])for index, item in enumerate(payload["files"])- if item.get("url") is not None and item["bytes"] is not None and item["sha256sum"] is not None+ if item.get("url") is not None and item["bytes"] is not None}route.metadata["task_input_metadata"] = input_metadatareturn TaskExecution(rewritten, result_file_id, error, input_metadata)@@ -241,7 +233,8 @@ class RouterTaskBackend:self.registry: ResourceRegistry = request.app.state.registryself.pool: WorkerPool = request.app.state.worker_poolself._pins = ExitStack()- self._owned_files: list[str] = []+ self._prepared_sources: list[str] = []+ self._task_sources: list[str] = []def fork(self) -> RouterTaskBackend:"""每个输入文件拥有独立暂存引用和 worker 调度上下文。"""@@ -249,6 +242,10 @@ class RouterTaskBackend:async def release_result(self, task_id: str, finished: float) -> None:"""父任务完成后记录公开保留期限,后台续租直到该期限再释放 worker。"""+ # 私有源缓存独立于 worker 路由,即使 worker 已丢失也必须开始回收倒计时。+ for reference_id in self._task_sources:+ self.sources.release_reference(reference_id, finished)+ self._task_sources.clear()route = self.registry.find("job", task_id)if route is None:return@@ -301,7 +298,7 @@ class RouterTaskBackend:"""为已校验文件建立准备引用,同一调用方相同内容复用现有暂存字节。"""async with self.request.app.state.task_store.storage_lock:private_id = "task-source-" + secrets.token_hex(12)- self._owned_files.append(private_id)+ self._prepared_sources.append(private_id)stored = self.sources.find_hash(self.owner_scope, upload.sha256sum)if stored is None:staging_id = "task-" + secrets.token_hex(12)@@ -312,20 +309,22 @@ class RouterTaskBackend:self.sources.discard_upload(staging_id)else:self.sources.bind_source(private_id, stored, owner_scope=self.owner_scope)+ self.sources.hold_reference(private_id)self._pins.enter_context(self.sources.pin(stored))return replace(upload, path=stored.path, file_id=None)def release_uploads(self, *, rollback: bool) -> None:- """关闭准备引用;成功提交后公开 File 绑定继续持有可复用源字节。"""- for file_id in self._owned_files:- self.sources.delete_file(file_id)- self._owned_files.clear()+ """回滚未交接的源引用并释放传输租约,成功提交的缓存由父任务持有。"""+ if rollback:+ for reference_id in self._prepared_sources:+ self.sources.delete_file(reference_id)+ self._prepared_sources.clear()self._pins.close()- async def _wait_for_input_files(+ async def _wait_for_uploads_ready(self, worker: WorkerState, upstream_id: str, generation: int, inputs: list[TaskFileItem]) -> dict[str, Any]:- """等待已上传项生成公共 File;级联 Router 的上传就绪可能早于下游引用交接。"""+ """只等待上传确认,不要求下游暴露源 File 或完成内部引用交接。"""while True:response = await _worker_request(self.pool,@@ -337,7 +336,7 @@ class RouterTaskBackend:)require_worker_generation(worker, generation)ready = TaskStatusResponse.model_validate(_checked_json(response)).model_dump()- if ready["task_id"] != upstream_id or len(ready["files"]) != len(inputs) or ready["status"] == "waiting_upload":+ if ready["task_id"] != upstream_id or len(ready["files"]) != len(inputs):_raise_api_error(502, error_type="api_error", code="invalid_upstream_response", message="Worker inputs are not ready")@@ -345,13 +344,15 @@ class RouterTaskBackend:_raise_api_error(502, error_type="api_error", code="invalid_upstream_response", message="Worker input order changed")- pending = any(- source.url is None- and item["upload_status"] == "ready"- and item["file_id"] is None- and item["status"] in {"queued", "running"}- for item, source in zip(ready["files"], inputs)- )+ pending = False+ for item, source in zip(ready["files"], inputs):+ if source.url is not None:+ if item["upload"] is not None:+ _raise_api_error(+ 502, error_type="api_error", code="invalid_upstream_response", message="URL input requests upload"+ )+ continue+ pending = pending or item["status"] == "waiting-file"if not pending:return ready# 只等待合法的中间状态;取消、worker replacement 和终态仍走原有清理边界。@@ -376,7 +377,6 @@ class RouterTaskBackend:worker = reservation.workerupstream_id: str | None = Noneroute: ResourceRoute | None = None- registered_inputs: list[str] = []transferred = Falsetry:worker_inputs = [@@ -415,17 +415,24 @@ class RouterTaskBackend:active_counted=True,reservation=reservation,upstream_headers={"authorization": authorization} if authorization else {},- payload={"job_id": task_id, "status": "queued", "files": []},+ payload={"job_id": task_id, "status": "pending", "files": []},)- by_client_id = {item.client_file_id: index for index, item in enumerate(inputs)}- for link in payload["pending_uploads"]:- index = by_client_id.get(link["client_file_id"])+ if len(payload["files"]) != len(inputs) or any(+ item["client_file_id"] != source.client_file_id for item, source in zip(payload["files"], inputs)+ ):+ _raise_api_error(+ 502, error_type="api_error", code="invalid_upstream_response", message="Worker input order changed"+ )+ for index, item in enumerate(payload["files"]):+ link = item["upload"]+ if link is None:+ continueif index not in uploads:_raise_api_error(502, error_type="api_error", code="invalid_upstream_response", message="Unexpected worker upload")- path = _worker_path(worker, link["upload_url"])- expected_path = f"/v1/tasks/{quote(upstream_id, safe='')}/uploads/{quote(link['client_file_id'], safe='')}"+ path = _worker_path(worker, link["url"])+ expected_path = f"/v1/tasks/{quote(upstream_id, safe='')}/uploads/{quote(item['client_file_id'], safe='')}"if path != expected_path:_raise_api_error(502, error_type="api_error", code="invalid_upstream_response", message="Invalid worker upload path"@@ -438,7 +445,7 @@ class RouterTaskBackend:request=self.request,content=file_chunks(uploads[index].path),headers={- **link["upload_headers"],+ **link["headers"],"content-length": str(uploads[index].bytes),"x-mineru-task-lease-seconds": "120",},@@ -446,46 +453,10 @@ class RouterTaskBackend:if response.status_code >= 400:_checked_json(response)require_worker_generation(worker, reservation.generation)- ready = await self._wait_for_input_files(worker, upstream_id, reservation.generation, inputs)- aliases: dict[str, str] = {}- for index, item in enumerate(ready["files"]):- if inputs[index].url is not None:- if item["upload_status"] != "not_required":- _raise_api_error(- 502, error_type="api_error", code="invalid_upstream_response", message="Unexpected URL upload"- )- continue- upstream_file_id = item.get("file_id")- if not isinstance(upstream_file_id, str):- _raise_api_error(- 502, error_type="api_error", code="invalid_upstream_response", message="Missing worker source File"- )- upload = uploads[index]- rewritten = rewrite_file_payload(- {- "id": upstream_file_id,- "object": "file",- "file_name": upload.filename,- "bytes": upload.bytes,- "sha256sum": upload.sha256sum,- "purpose": "parse",- "created_at": int(time.time()),- "expires_at": None,- },- worker,- self.registry,- self.owner_scope,- )- public_id = rewritten["id"]- registered_inputs.append(public_id)- self.sources.bind_source(- public_id,- StoredSourceFile(upload.path, upload.bytes, upload.sha256sum, upload.mime_type),- owner_scope=self.owner_scope,- )- aliases[upstream_file_id] = public_id- route.metadata["input_aliases"] = aliases+ await self._wait_for_uploads_ready(worker, upstream_id, reservation.generation, inputs)result = await self.get(task_id)+ self._task_sources.extend(self._prepared_sources)+ self._prepared_sources.clear()transferred = Truereturn resultfinally:@@ -500,9 +471,6 @@ class RouterTaskBackend:route.metadata["task_result_retain_until"] = time.time()else:reservation.release()- for public_id in registered_inputs:- self.registry.remove("file", public_id)- self.sources.delete_file(public_id)async def _cancel_remote(self, worker: WorkerState, upstream_id: str) -> None:"""尽力取消已经创建的 worker 任务,清理失败不覆盖最初的提交错误。"""diff --git a/mineru/parser/api_client.py b/mineru/parser/api_client.pyindex 0c3da23e..e781b389 100644--- a/mineru/parser/api_client.py+++ b/mineru/parser/api_client.py@@ -24,6 +24,7 @@ import timeimport tempfileimport zipfilefrom collections.abc import Callable, Iterator, AsyncIterator+from dataclasses import dataclassfrom email.utils import parsedate_to_datetimefrom pathlib import Pathfrom typing import Any, Literal, cast@@ -32,6 +33,7 @@ from urllib.parse import unquote, urlparse, urljoinimport httpxfrom docvortex.assets import validate_image_sidecar_path+from ..task_types import TaskStatus, task_status_urlfrom ..types import PageInfo, Tier, validate_tierfrom .base import DocumentParser, ParseResultfrom .page_range import normalize_page_range_input@@ -55,9 +57,17 @@ _HTML_IMAGE_SOURCE_RE = re.compile(logger = logging.getLogger("mineru.api_client")_RequestMethod = Literal["GET", "POST", "PUT"]-ApiTaskStatus = Literal["waiting_upload", "queued", "running", "completed", "partial", "failed", "canceled"]+ApiTaskStatus = TaskStatusApiJobStatus = ApiTaskStatus-_JOB_STATUSES = {"waiting_upload", "queued", "running", *_TERMINAL_JOB_STATUSES}+_JOB_STATUSES = {"waiting-file", "pending", "running", *_TERMINAL_JOB_STATUSES}+++@dataclass+class _UploadState:+ """跨轮询保留上传成功标记与重试次数,等待服务确认时不重复发送文件。"""++ attempts: int = 0+ accepted: bool = Falsedef _notify_job_status(job: dict[str, Any], callback: Callable[[ApiJobStatus], None] | None) -> None:@@ -187,7 +197,6 @@ class MinerUApiParser(DocumentParser):or not isinstance(task.get("status"), str)or task.get("status") not in _JOB_STATUSESor task.get("output_formats") != ["default"]- or not isinstance(task.get("pending_uploads"), list)or not isinstance(files, list)or len(files) != 1or not isinstance(files[0], dict)@@ -196,18 +205,25 @@ class MinerUApiParser(DocumentParser):):raise _V1APIError("invalid_response", "Inconsistent Tasks response")item = files[0]- if item.get("upload_status") not in ("pending", "ready", "expired") or item.get("status") not in (- "waiting_upload",- "queued",+ if item.get("status") not in (+ "waiting-file",+ "pending","running","completed","failed","canceled",):raise _V1APIError("invalid_response", "Invalid task file state")- if not isinstance(task.get("status_url"), str):- raise _V1APIError("invalid_response", "Task status URL is missing")- task["status_url"] = self._resolve(task["status_url"], metadata=True)+ if "upload" not in item or (item["upload"] is not None and not isinstance(item["upload"], dict)):+ raise _V1APIError("invalid_response", "Invalid task upload descriptor")+ if item["upload"] is not None and item["status"] != "waiting-file":+ raise _V1APIError("invalid_response", "Only waiting-file accepts uploads")+ if expected is not None and any(item.get(key) != expected["files"][0].get(key) for key in ("file_name", "bytes")):+ raise _V1APIError("invalid_response", "Task input identity changed")+ try:+ task_status_url(self._base_url, task["task_id"])+ except ValueError as exc:+ raise _V1APIError("invalid_response", str(exc)) from excif files[0].get("status") == "completed" and not isinstance(files[0].get("result_url"), str):raise _V1APIError("invalid_response", "Completed file is missing result_url")return task@@ -234,13 +250,19 @@ class MinerUApiParser(DocumentParser):)task = self._snapshot(response)_notify_job_status(task, status_callback)- task = self._upload_input(cli, task, file_path, status_callback, size=payload["files"][0]["bytes"])+ upload_state = _UploadState()+ task = self._upload_input(+ cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state+ )for _ in range(_POLL_MAX_ATTEMPTS):if task["status"] in _TERMINAL_JOB_STATUSES:breaktime.sleep(_POLL_INTERVAL_SECONDS)task = self._query(cli, task)_notify_job_status(task, status_callback)+ task = self._upload_input(+ cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state+ )if task["status"] not in _TERMINAL_JOB_STATUSES:raise _V1APIError("timeout", f"Task {task['task_id']} did not complete within timeout")_raise_for_terminal_job_error(task)@@ -253,11 +275,11 @@ class MinerUApiParser(DocumentParser):return _parse_result_from_zip_bytes(target, include_images=self.include_images)def _query(self, cli: httpx.Client, task: dict[str, Any]) -> dict[str, Any]:- """按服务返回的同源状态链接查询,保留元数据请求的有限重试。"""+ """按配置的 API 基址和任务 ID 查询,保留元数据请求的有限重试。"""response = _request_with_retry(cli,"GET",- task["status_url"],+ task_status_url(self._base_url, task["task_id"]),stage="task polling",headers=self._headers(),)@@ -271,47 +293,46 @@ class MinerUApiParser(DocumentParser):callback: Callable[[ApiTaskStatus], None] | None,*,size: int,+ upload_state: _UploadState,) -> dict[str, Any]:- """首次上传失败先核实是否已确认,再重新打开文件流;缓存命中完全不传字节。"""- for attempt in range(1, _TRANSPORT_MAX_ATTEMPTS + 1):- if task["files"][0].get("upload_status") != "pending":+ """只有明确待上传时才发 PUT,上传成功及等待确认均不消耗重复传输。"""+ if task["files"][0]["status"] != "waiting-file":+ upload_state.accepted = True+ if upload_state.accepted or task["files"][0]["upload"] is None:+ return task+ while upload_state.attempts < _TRANSPORT_MAX_ATTEMPTS:+ link = task["files"][0]["upload"]+ if link is None:return task- links = task["pending_uploads"]- link = next((item for item in links if isinstance(item, dict) and item.get("client_file_id") == "file"), None)- if not link:- raise _V1APIError("invalid_response", "Missing task upload link")- url = self._resolve(link["upload_url"])- headers = _same_origin_upload_headers(self._base_url, url, link.get("upload_headers", {}), self._headers())+ url = self._resolve(link["url"])+ headers = _same_origin_upload_headers(self._base_url, url, link.get("headers", {}), self._headers())headers = dict(httpx.Headers(headers))headers.pop("transfer-encoding", None)headers["content-length"] = str(size)+ upload_state.attempts += 1try:- response = _request_once(- cli,- "PUT",- url,- stage="upload content",- headers=headers,- content=_file_chunks(path),- )+ response = _request_once(cli, "PUT", url, stage="upload content", headers=headers, content=_file_chunks(path))if response.status_code >= 400:self._check(response)except (_APITransportError, _V1APIError) as exc:task = self._query(cli, task)_notify_job_status(task, callback)- if task["files"][0].get("upload_status") != "pending":+ if task["files"][0]["status"] != "waiting-file":+ upload_state.accepted = True+ return task+ if task["files"][0]["upload"] is None:return taskif isinstance(exc, _V1APIError) and response.status_code not in {408, 429, 500, 502, 503, 504}:raise- if attempt == _TRANSPORT_MAX_ATTEMPTS:+ if upload_state.attempts == _TRANSPORT_MAX_ATTEMPTS:raise- time.sleep(_transport_retry_delay(attempt))+ time.sleep(_transport_retry_delay(upload_state.attempts))continue+ upload_state.accepted = Truetask = self._query(cli, task)_notify_job_status(task, callback)- if task["files"][0].get("upload_status") != "pending":- return task- raise _V1APIError("upload_failed", "Upload was not confirmed")+ return task+ raise _V1APIError("upload_failed", "Upload retry budget exhausted")def _download_task(self, cli: httpx.Client, task: dict[str, Any], target: Path) -> None:"""流式下载和校验完整包;重定向逐跳选择认证,签名失效后重新查询任务。"""@@ -389,13 +410,19 @@ class MinerUApiParser(DocumentParser):)task = self._snapshot(response)_notify_job_status(task, status_callback)- task = await self._async_upload_input(cli, task, file_path, status_callback, size=payload["files"][0]["bytes"])+ upload_state = _UploadState()+ task = await self._async_upload_input(+ cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state+ )for _ in range(_POLL_MAX_ATTEMPTS):if task["status"] in _TERMINAL_JOB_STATUSES:breakawait asyncio.sleep(_POLL_INTERVAL_SECONDS)task = await self._async_query(cli, task)_notify_job_status(task, status_callback)+ task = await self._async_upload_input(+ cli, task, file_path, status_callback, size=payload["files"][0]["bytes"], upload_state=upload_state+ )if task["status"] not in _TERMINAL_JOB_STATUSES:raise _V1APIError("timeout", f"Task {task['task_id']} did not complete within timeout")_raise_for_terminal_job_error(task)@@ -408,11 +435,11 @@ class MinerUApiParser(DocumentParser):return _parse_result_from_zip_bytes(target, include_images=self.include_images)async def _async_query(self, cli: httpx.AsyncClient, task: dict[str, Any]) -> dict[str, Any]:- """按服务返回的同源状态链接查询,保留元数据请求的有限重试。"""+ """按配置的 API 基址和任务 ID 查询,保留元数据请求的有限重试。"""response = await _async_request_with_retry(cli,"GET",- task["status_url"],+ task_status_url(self._base_url, task["task_id"]),stage="task polling",headers=self._headers(),)@@ -426,47 +453,48 @@ class MinerUApiParser(DocumentParser):callback: Callable[[ApiTaskStatus], None] | None,*,size: int,+ upload_state: _UploadState,) -> dict[str, Any]:- """首次上传失败先核实是否已确认,再重新打开文件流;缓存命中完全不传字节。"""- for attempt in range(1, _TRANSPORT_MAX_ATTEMPTS + 1):- if task["files"][0].get("upload_status") != "pending":+ """只有明确待上传时才发 PUT,上传成功及等待确认均不消耗重复传输。"""+ if task["files"][0]["status"] != "waiting-file":+ upload_state.accepted = True+ if upload_state.accepted or task["files"][0]["upload"] is None:+ return task+ while upload_state.attempts < _TRANSPORT_MAX_ATTEMPTS:+ link = task["files"][0]["upload"]+ if link is None:return task- links = task["pending_uploads"]- link = next((item for item in links if isinstance(item, dict) and item.get("client_file_id") == "file"), None)- if not link:- raise _V1APIError("invalid_response", "Missing task upload link")- url = self._resolve(link["upload_url"])- headers = _same_origin_upload_headers(self._base_url, url, link.get("upload_headers", {}), self._headers())+ url = self._resolve(link["url"])+ headers = _same_origin_upload_headers(self._base_url, url, link.get("headers", {}), self._headers())headers = dict(httpx.Headers(headers))headers.pop("transfer-encoding", None)headers["content-length"] = str(size)+ upload_state.attempts += 1try:response = await _async_request_once(- cli,- "PUT",- url,- stage="upload content",- headers=headers,- content=_async_file_chunks(path),+ cli, "PUT", url, stage="upload content", headers=headers, content=_async_file_chunks(path))if response.status_code >= 400:self._check(response)except (_APITransportError, _V1APIError) as exc:task = await self._async_query(cli, task)_notify_job_status(task, callback)- if task["files"][0].get("upload_status") != "pending":+ if task["files"][0]["status"] != "waiting-file":+ upload_state.accepted = True+ return task+ if task["files"][0]["upload"] is None:return taskif isinstance(exc, _V1APIError) and response.status_code not in {408, 429, 500, 502, 503, 504}:raise- if attempt == _TRANSPORT_MAX_ATTEMPTS:+ if upload_state.attempts == _TRANSPORT_MAX_ATTEMPTS:raise- await asyncio.sleep(_transport_retry_delay(attempt))+ await asyncio.sleep(_transport_retry_delay(upload_state.attempts))continue+ upload_state.accepted = Truetask = await self._async_query(cli, task)_notify_job_status(task, callback)- if task["files"][0].get("upload_status") != "pending":- return task- raise _V1APIError("upload_failed", "Upload was not confirmed")+ return task+ raise _V1APIError("upload_failed", "Upload retry budget exhausted")async def _async_download_task(self, cli: httpx.AsyncClient, task: dict[str, Any], target: Path) -> None:"""流式下载和校验完整包;重定向逐跳选择认证,签名失效后重新查询任务。"""@@ -939,7 +967,7 @@ def _same_origin_upload_headers(if not isinstance(upload_headers, dict) or any(not isinstance(key, str) or not isinstance(value, str) for key, value in upload_headers.items()):- raise _V1APIError("invalid_response", "upload_headers must be a string mapping")+ raise _V1APIError("invalid_response", "upload.headers must be a string mapping")headers = dict(upload_headers)if _http_origin(base_url) == _http_origin(upload_url):headers.update(auth_headers)diff --git a/mineru/parser/api_server.py b/mineru/parser/api_server.pyindex 85024fbe..98edd4eb 100644--- a/mineru/parser/api_server.py+++ b/mineru/parser/api_server.py@@ -42,6 +42,7 @@ from starlette.types import Receive, Scope, Sendfrom ..config import VlmConfigfrom ..config import config as mineru_configfrom ..errors import MineruError+from ..integrations.docvortex import MinerUMetadatafrom ..filetypes import (PARSEABLE_EXTENSIONS,batch_effective_parse_tier,@@ -74,7 +75,7 @@ from .tier import (ensure_tier_runtime_dependencies,runtime_options_for_tier,)-from ..task_types import TASK_OUTPUT_FORMATS, TaskOutputFormat+from ..task_types import TASK_OUTPUT_FORMATS, TaskFileState, TaskOutputFormat, TaskStatus_DEFAULT_API_SERVER_TIER: ServerTier = "standard"_MAX_FILES_PER_JOB_DEFAULT = 100@@ -89,10 +90,10 @@ class ParseServerStartupError(RuntimeError):# ── literal type aliases ────────────────────────────────────────────-JobStatus = Literal["queued", "running", "completed", "partial", "failed", "canceled"]+JobStatus = TaskStatus"""Job lifecycle states."""-FileStatus = Literal["queued", "running", "completed", "failed"]+FileStatus = TaskFileState"""Per-file status within a job."""AccessLevel = Literal["anonymous", "registered"]@@ -370,6 +371,7 @@ class TaskProgress(BaseModel):class FileParseInfo(BaseModel):model_config = _PYDANTIC_CONFIG+ tier: Tierduration_ms: int | None = Noneparser_version: str | None = None@@ -804,13 +806,12 @@ class _JobInputMetadata:"""记录完整下载且校验通过的实际输入信息,不扩展公开 Jobs 协议。"""bytes: int- sha256sum: str@dataclassclass _JobRecord:id: str- status: JobStatus = "queued"+ status: JobStatus = "pending"created_at: str = ""started_at: str | None = Nonefinished_at: str | None = None@@ -820,6 +821,7 @@ class _JobRecord:input_blobs: dict[str, str] = field(default_factory=dict)input_specs: dict[int, _JobInputSpec] = field(default_factory=dict)input_metadata: dict[int, _JobInputMetadata] = field(default_factory=dict)+ prepared_sources: dict[int, _PreparedInput] = field(default_factory=dict)file_store: FileStore | None = Noneusage_counted: bool = Falsetask_result_file_id: str | None = None@@ -843,6 +845,7 @@ class JobStore:self._jobs: dict[str, _JobRecord] = {}self._concurrency = max(1, concurrency)self._semaphore = asyncio.Semaphore(self._concurrency)+ self._preparation_semaphore = asyncio.Semaphore(self._concurrency)self._started_at = JobStore._now()self._tasks: dict[str, asyncio.Task[None]] = {}self._completion_events: dict[str, asyncio.Event] = {}@@ -874,7 +877,13 @@ class JobStore:self.fail_engine(message)_raise_api_error(503, error_type="engine_error", code="engine_dead", message=message)- def start_task(self, rec: _JobRecord, operation: Callable[[], Awaitable[None]]) -> None:+ def start_task(+ self,+ rec: _JobRecord,+ operation: Callable[[], Awaitable[None]],+ *,+ prepare: Callable[[], Awaitable[None]] | None = None,+ ) -> None:"""保存真实后台任务及所属应用租约,排队取消也不会开始解析。"""self.require_ready()if self._closing:@@ -882,17 +891,32 @@ class JobStore:self._completion_events[rec.id] = asyncio.Event()async def run_owned() -> None:- """在整个任务期间传播应用租约,并直到清理完成才释放 job 名额。"""+ """准备输入不占解析名额,等待解析时仍限制准备任务数以控制内存。"""token = runtime_owner.set(self.runtime_owner)+ acquired = Falsetry:- async with self._semaphore:- if rec.status == "queued":- await operation()+ if prepare is not None:+ async with self._preparation_semaphore:+ if rec.status not in {"canceled", "failed"}:+ await prepare()+ if rec.status != "pending":+ return+ # 取得解析名额后才允许下一份输入准备,避免待解析字节无限堆积。+ await self._semaphore.acquire()+ acquired = True+ else:+ await self._semaphore.acquire()+ acquired = True+ if rec.status == "pending":+ await operation()except asyncio.CancelledError:if rec.status != "failed":rec.status = "canceled"raisefinally:+ rec.prepared_sources.clear()+ if acquired:+ self._semaphore.release()runtime_owner.reset(token)def finished(task: asyncio.Task[None]) -> None:@@ -970,7 +994,7 @@ class JobStore:now = self._now()rec = _JobRecord(id=job_id,- status="queued",+ status="waiting-file" if any(isinstance(entry.source, _UrlSource) for entry in req.files) else "pending",created_at=now,tier=req.tier,task_output_formats=req.output_formats,@@ -983,7 +1007,7 @@ class JobStore:file_id=entry.source.file_id if isinstance(entry.source, _StoredSource) else None,name=name,page_range=entry.page_range or "",- status="queued",+ status="waiting-file" if isinstance(entry.source, _UrlSource) else "pending",))rec.progress.total = len(rec.files)@@ -1153,6 +1177,74 @@ class _ExtractedSource:media_type: str | None = None+@dataclass(frozen=True, slots=True)+class _PreparedInput:+ """暂存已校验的 URL 输入及准备耗时,等待解析队列不计入文件执行耗时。"""++ source: _ExtractedSource+ duration_seconds: float+++def _validate_url_input(rec: _JobRecord, index: int, extracted: _ExtractedSource) -> None:+ """下载完成后验证大小和 hash,只有有效输入才能进入解析队列。"""+ data = extracted.data+ if not data:+ raise MineruError("invalid_request", "Empty files are not supported")+ spec = rec.input_specs.get(index)+ if spec is not None:+ if spec.bytes is not None and len(data) != spec.bytes:+ raise MineruError("bytes_mismatch", "Declared size does not match downloaded bytes")+ if spec.sha256sum is not None and hashlib.sha256(data).hexdigest() != spec.sha256sum:+ raise MineruError("file_hash_mismatch", "SHA-256 mismatch")+ rec.input_metadata[index] = _JobInputMetadata(bytes=len(data))+++async def _prepare_job_inputs(+ rec: _JobRecord,+ req: _ParseRequest,+ file_store: FileStore,+ *,+ url_timeout: int,+ allow_http_source: bool,+ max_url_bytes: int,+) -> None:+ """在独立受控阶段下载 URL,保留 HTML 来源信息并逐文件收敛准备失败。"""+ if rec.status in {"canceled", "failed"}:+ return+ rec.status = "waiting-file"+ rec.started_at = rec.started_at or JobStore._now()+ for index, entry in enumerate(req.files):+ if rec.status in {"canceled", "failed"}:+ return+ if not isinstance(entry.source, _UrlSource):+ continue+ file = rec.files[index]+ file.status = "waiting-file"+ started = time.monotonic()+ try:+ extracted = await _extract_bytes(+ entry.source,+ file_store,+ url_timeout=url_timeout,+ allow_http_source=allow_http_source,+ max_url_bytes=max_url_bytes,+ )+ _validate_url_input(rec, index, extracted)+ rec.prepared_sources[index] = _PreparedInput(extracted, time.monotonic() - started)+ file.status = "pending"+ except Exception as exc:+ logger.exception("Parse-server input preparation failed: job_id=%s file=%r", rec.id, file.name)+ file.status = "failed"+ file.error = (+ ErrorDetail(type=exc.type, code=exc.code, message=str(exc), param=exc.param)+ if isinstance(exc, MineruError)+ else ErrorDetail(type="engine_error", code="parse_failed", message=str(exc))+ )+ rec.progress.failed += 1+ if rec.status not in {"canceled", "failed"}:+ rec.status = "failed" if rec.progress.failed == rec.progress.total else "pending"++async def _extract_bytes(source: _ExecutionSource,file_store: FileStore,@@ -1220,35 +1312,34 @@ async def _run_job(if rec.status == "canceled":returnrec.status = "running"- rec.started_at = JobStore._now()+ rec.started_at = rec.started_at or JobStore._now()with tempfile.TemporaryDirectory(prefix="mineru_job_") as tmpdir:for i, entry in enumerate(req.files):if rec.status in {"canceled", "failed"}:breakfr = rec.files[i]+ if fr.status in {"failed", "canceled"}:+ continue+ prepared = rec.prepared_sources.pop(i, None)try:- file_started = time.monotonic()+ file_started = time.monotonic() - (prepared.duration_seconds if prepared is not None else 0)fr.status = "running"- extracted = await _extract_bytes(- entry.source,- file_store,- url_timeout=url_timeout,- allow_http_source=allow_http_source,- max_url_bytes=max_url_bytes,- input_blobs=rec.input_blobs,+ extracted = (+ prepared.source+ if prepared is not None+ else await _extract_bytes(+ entry.source,+ file_store,+ url_timeout=url_timeout,+ allow_http_source=allow_http_source,+ max_url_bytes=max_url_bytes,+ input_blobs=rec.input_blobs,+ ))data = extracted.data- input_spec = rec.input_specs.get(i)- if input_spec is not None and isinstance(entry.source, _UrlSource):- if not data:- raise MineruError("invalid_request", "Empty files are not supported")- if input_spec.bytes is not None and len(data) != input_spec.bytes:- raise MineruError("bytes_mismatch", "Declared size does not match downloaded bytes")- sha = hashlib.sha256(data).hexdigest()- if input_spec.sha256sum is not None and sha != input_spec.sha256sum:- raise MineruError("file_hash_mismatch", "SHA-256 mismatch")- rec.input_metadata[i] = _JobInputMetadata(bytes=len(data), sha256sum=sha)+ if prepared is None and isinstance(entry.source, _UrlSource):+ _validate_url_input(rec, i, extracted)suffix = pathlib.Path(fr.name).suffixstype = _suffix_type(fr.name)if not stype and not suffix and isinstance(entry.source, _UrlSource) and extracted.media_type == "text/html":@@ -1288,6 +1379,7 @@ async def _run_job(if rec.status in {"canceled", "failed"}:break+ actual_metadata = MinerUMetadata.model_validate(result.middle_json.extensions.get("mineru"))from .task_outputs import build_task_outputtry:@@ -1312,6 +1404,7 @@ async def _run_job(fr.page_range = _page_range_from_result_pages(result.pages)fr.parse = FileParseInfo(+ tier=actual_metadata.tier,duration_ms=int((time.monotonic() - file_started) * 1000),parser_version=__version__,)@@ -1657,8 +1750,21 @@ async def submit_execution(job_store=job_store,)+ async def _bg_prepare() -> None:+ """复用本次来源配置准备 URL 输入,只有准备成功的文件进入解析队列。"""+ await _prepare_job_inputs(+ rec,+ body,+ file_store,+ url_timeout=url_timeout_val,+ allow_http_source=allow_http_source_val,+ max_url_bytes=max_url_bytes_val,+ )+try:- job_store.start_task(rec, _bg_run)+ job_store.start_task(+ rec, _bg_run, prepare=_bg_prepare if any(isinstance(entry.source, _UrlSource) for entry in body.files) else None+ )except BaseException:file_store.release_inputs(rec.input_blobs)job_store._jobs.pop(rec.id, None)diff --git a/mineru/parser/task_api.py b/mineru/parser/task_api.pyindex 5468ad13..bc54f77d 100644--- a/mineru/parser/task_api.py+++ b/mineru/parser/task_api.py@@ -22,7 +22,7 @@ from starlette.requests import ClientDisconnectfrom ..errors import MineruErrorfrom ..filetypes import is_flash_only_parse_extension-from ..task_types import TaskOutputFormat+from ..task_types import TaskFileState, TaskOutputFormat, TaskStatusfrom ..types import Tierfrom ..utils.async_utils import drain_future, run_syncfrom .api_server import (@@ -48,8 +48,6 @@ from .page_range import PAGE_RANGE_DESCRIPTION, normalize_page_range_inputMAX_TASK_FILES = 100TASK_UPLOAD_TTL = 3600TERMINAL_TASK_STATUSES = frozenset({"completed", "partial", "failed", "canceled"})-TaskStatus = Literal["waiting_upload", "queued", "running", "completed", "partial", "failed", "canceled"]-TaskFileState = Literal["waiting_upload", "queued", "running", "completed", "failed", "canceled"]logger = logging.getLogger(__name__)@@ -100,38 +98,43 @@ class CreateTaskRequest(UploadTaskOptions):class TaskUploadLink(BaseModel):"""描述一个文件的上传目标,客户端原样使用服务端给出的地址和请求头。"""- client_file_id: str = Field(pattern=r"^[A-Za-z0-9_-]{1,64}$")- upload_url: str- upload_headers: dict[str, str]+ url: str+ headers: dict[str, str]expires_at: int-class TaskFileStatus(TaskFileItem):- """同时提供上传准备情况与逐文件解析状态,禁止内联解析正文。"""+class TaskFileStatus(BaseModel):+ """用统一状态描述文件准备和解析,上传对象仅表示客户端当前可执行的动作。"""- upload_status: Literal["pending", "ready", "not_required", "expired"]- file_id: str | None = None- name: str+ client_file_id: str = Field(pattern=r"^[A-Za-z0-9_-]{1,64}$")+ file_name: str+ bytes: int | None+ url: str | None+ upload: TaskUploadLink | Nonestatus: TaskFileStateparse: FileParseInfo | None = Noneresult_url: str | None = Noneerror: ErrorDetail | None = None+ @model_validator(mode="after")+ def validate_upload_action(self) -> TaskFileStatus:+ """只有等待客户端文件的条目才能提供上传目标,服务端准备阶段可以为空。"""+ if self.upload is not None and (self.status != "waiting-file" or self.url is not None):+ raise ValueError("Only waiting-file without a URL source accepts uploads")+ return self+class TaskStatusResponse(BaseModel):"""任务状态只携带元数据;每个成功文件提供自己的下载链接。"""task_id: strstatus: TaskStatus- status_url: strcreated_at: strstarted_at: str | None = Nonefinished_at: str | None = None- tier: Tier | None = Noneoutput_formats: list[TaskOutputFormat]progress: TaskProgressfiles: list[TaskFileStatus]- pending_uploads: list[TaskUploadLink]error: ErrorDetail | None = None@@ -345,7 +348,7 @@ class _TaskFileRecord:backend: TaskBackendexecution_id: str = field(default_factory=JobStore._new_job_id)- status: TaskFileState = "waiting_upload"+ status: TaskFileState = "waiting-file"upload: TaskUpload | None = Noneupload_lock: asyncio.Lock = field(default_factory=asyncio.Lock)upload_active: bool = False@@ -370,7 +373,7 @@ class _TaskRecord:expires_at: intfile_states: list[_TaskFileRecord]created_at: str = field(default_factory=JobStore._now)- status: TaskStatus = "waiting_upload"+ status: TaskStatus = "waiting-file"refresh_lock: asyncio.Lock = field(default_factory=asyncio.Lock)changed: asyncio.Event = field(default_factory=asyncio.Event)operation: asyncio.Task[None] | None = None@@ -413,10 +416,10 @@ class TaskStore:states = [file.status for file in record.file_states]if "running" in states:record.status = "running"- elif "queued" in states:- record.status = "queued"- elif "waiting_upload" in states:- record.status = "waiting_upload"+ elif "pending" in states:+ record.status = "pending"+ elif "waiting-file" in states:+ record.status = "waiting-file"else:completed = states.count("completed")record.status = "completed" if completed == len(states) else ("partial" if completed else "failed")@@ -430,8 +433,8 @@ class TaskStore:"""只终结尚未受理上传的文件;截止前已经受理的流允许继续完成。"""if record.expires_at > time.time() or record.status == "canceled":return- for file in record.file_states:- if file.status == "waiting_upload" and not file.upload_active:+ for item, file in zip(record.inputs, record.file_states):+ if item.url is None and file.upload is None and file.status == "waiting-file" and not file.upload_active:file.expired = Truefile.status = "failed"file.error = ErrorDetail(type="invalid_request_error", code="upload_expired", message="Upload expired")@@ -511,9 +514,9 @@ class TaskStore:def _start_file(self, record: _TaskRecord, index: int) -> None:"""事件循环内同步登记提交句柄,重复 PUT 不能创建第二个子作业。"""file = record.file_states[index]- if file.operation is not None or file.status != "waiting_upload" or record.status == "canceled":+ if file.operation is not None or file.status != "waiting-file" or record.status == "canceled":return- file.status = "queued"+ file.status = "pending" if file.upload is not None else "waiting-file"record.changed.set()file.operation = asyncio.create_task(self._dispatch_file(record, index), name=f"mineru-task-file-{file.execution_id}")self._aggregate(record)@@ -523,7 +526,7 @@ class TaskStore:if file.status in TERMINAL_TASK_STATUSES:returnstatus = execution.job.get("status")- if status not in {"queued", "running", "completed", "failed", "canceled"}:+ if status not in {"waiting-file", "pending", "running", "completed", "failed", "canceled"}:_invalid("Invalid file status from execution backend", code="invalid_response", status_code=502)if status == "completed" and execution.result_file_id is None:_invalid("Completed file has no result", code="invalid_response", status_code=502)@@ -669,7 +672,7 @@ class TaskStore:_invalid("Task was canceled", status_code=409, code="task_already_terminal")if file.upload is not None:return True- if file.status != "waiting_upload":+ if file.status != "waiting-file":_invalid("File no longer accepts uploads", status_code=409, code="file_already_terminal")length = request.headers.get("content-length")if length is not None:@@ -755,56 +758,54 @@ def _url(request: Request, path: str) -> str:def task_payload(record: _TaskRecord, request: Request) -> dict[str, Any]:"""逐文件合并上传与执行元数据,成功文件立即给出独立下载链接。"""files: list[dict[str, Any]] = []- links: list[dict[str, Any]] = []starts: list[str] = []for item, state in zip(record.inputs, record.file_states):- file = item.model_dump()- upload = state.upload- file.update(- name=item.file_name,- upload_status="not_required"- if item.url is not None- else ("ready" if upload is not None else ("expired" if state.expired else "pending")),- status=state.status,- file_id=upload.file_id if upload else None,- result_url=None,- )+ file: dict[str, Any] = {+ "client_file_id": item.client_file_id,+ "file_name": item.file_name,+ "bytes": item.bytes,+ "url": item.url,+ "status": state.status,+ "upload": None,+ "parse": None,+ "result_url": None,+ "error": None,+ }if state.execution is not None:job = state.execution.jobresults = job.get("files") or []if results:- file.update({key: results[0].get(key) for key in ("name", "file_id", "page_range", "parse", "error")})+ file["error"] = results[0].get("error")+ if state.status == "completed":+ file["parse"] = results[0].get("parse")if job.get("started_at"):starts.append(job["started_at"])if (metadata := state.execution.input_metadata.get(0)) is not None:- file.update(bytes=metadata.bytes, sha256sum=metadata.sha256sum)+ file["bytes"] = metadata.bytesif state.status == "completed" and state.execution.result_file_id:file["result_url"] = _url(request, f"/v1/files/{quote(state.execution.result_file_id, safe='')}/content")if state.error is not None:file["error"] = state.error.model_dump()- files.append(file)if (item.url is None- and state.status == "waiting_upload"+ and state.upload is None+ and not state.upload_active+ and state.status == "waiting-file"and record.status != "canceled"and record.expires_at > time.time()):- links.append(- {- "client_file_id": item.client_file_id,- "upload_url": _url(request, f"/v1/tasks/{record.id}/uploads/{quote(item.client_file_id, safe='')}"),- "upload_headers": {"Content-Type": "application/octet-stream"},- "expires_at": record.expires_at,- }- )+ file["upload"] = {+ "url": _url(request, f"/v1/tasks/{record.id}/uploads/{quote(item.client_file_id, safe='')}"),+ "headers": {"Content-Type": "application/octet-stream"},+ "expires_at": record.expires_at,+ }+ files.append(file)payload = {"task_id": record.id,"status": record.status,- "status_url": _url(request, f"/v1/tasks/{record.id}"),"created_at": record.created_at,"started_at": min(starts) if starts else None,"finished_at": record.finished_at,- "tier": record.options.tier,"output_formats": record.options.output_formats,"progress": {"completed": sum(file.status == "completed" for file in record.file_states),@@ -812,7 +813,6 @@ def task_payload(record: _TaskRecord, request: Request) -> dict[str, Any]:"total": len(files),},"files": files,- "pending_uploads": links,"error": record.error.model_dump() if record.error else None,}return TaskStatusResponse.model_validate(payload).model_dump()diff --git a/mineru/task_types.py b/mineru/task_types.pyindex 4995fb6b..2f44b119 100644--- a/mineru/task_types.py+++ b/mineru/task_types.py@@ -1,8 +1,19 @@-"""Tasks 客户端与服务端共用的轻量格式定义,不导入解析或渲染实现。"""+"""Tasks 共用的轻量格式与查询路由,不导入解析或渲染实现。"""from typing import Literal+from urllib.parse import quoteTaskOutputFormat = Literal["default", "docx", "html", "latex"]TASK_OUTPUT_FORMATS: tuple[TaskOutputFormat, ...] = ("default", "docx", "html", "latex")+TaskStatus = Literal["waiting-file", "pending", "running", "completed", "partial", "failed", "canceled"]+TaskFileState = Literal["waiting-file", "pending", "running", "completed", "failed", "canceled"]-__all__ = ["TASK_OUTPUT_FORMATS", "TaskOutputFormat"]++def task_status_url(api_url: str, task_id: str) -> str:+ """保留 API 部署前缀并编码任务标识,拒绝会改变固定路由的点路径。"""+ if not task_id or task_id in {".", ".."}:+ raise ValueError("Invalid task_id")+ return api_url.rstrip("/") + "/v1/tasks/" + quote(task_id, safe="")+++__all__ = ["TASK_OUTPUT_FORMATS", "TaskFileState", "TaskOutputFormat", "TaskStatus", "task_status_url"]diff --git a/scripts/http_task_example.py b/scripts/http_task_example.pyindex fae615f6..9b9601b9 100644--- a/scripts/http_task_example.py+++ b/scripts/http_task_example.py@@ -20,7 +20,7 @@ from typing import Anyfrom urllib.parse import quote, urljoin, urlsplitTERMINAL = frozenset({"completed", "partial", "failed", "canceled"})-STATUSES = TERMINAL | {"waiting_upload", "queued", "running"}+STATUSES = TERMINAL | {"waiting-file", "pending", "running"}RETRYABLE_HTTP = frozenset({408, 429, 500, 502, 503, 504})REDIRECTS = frozenset({301, 302, 303, 307, 308})CHUNK_BYTES = 1024 * 1024@@ -113,8 +113,9 @@ class TaskClient:if not 0 <= self.poll_interval < float("inf"):raise ClientError("POLL_INTERVAL must be a non-negative finite number")self.task_id: str | None = None- self.status_url: str | None = Noneself.saved_ids: set[str] = set()+ self.uploaded_ids: set[str] = set()+ self.upload_attempts: dict[str, int] = {}def resolve(self, url: str) -> str:"""解析服务返回的绝对或相对链接,保留 OSS 签名查询串。"""@@ -236,12 +237,15 @@ class TaskClient:def validate_task(self, payload: dict[str, Any]) -> dict[str, Any]:"""校验任务身份和文件映射,恢复请求不能误把文件上传给另一项。"""task_id = payload.get("task_id")- if not isinstance(task_id, str) or not task_id or (self.task_id is not None and task_id != self.task_id):+ if (+ not isinstance(task_id, str)+ or not task_id+ or task_id in {".", ".."}+ or (self.task_id is not None and task_id != self.task_id)+ ):raise ClientError("Missing or inconsistent task_id")if payload.get("status") not in STATUSES:raise ClientError("Unknown task status")- if not isinstance(payload.get("status_url"), str) or not payload["status_url"]:- raise ClientError("Missing status_url")files = payload.get("files")if not isinstance(files, list) or len(files) != len(self.files):raise ClientError("Task file count does not match local inputs")@@ -256,30 +260,19 @@ class TaskClient:local = self.by_id[client_id]if item.get("file_name") != local.path.name or item.get("bytes") != local.size:raise ClientError("Task files differ from local inputs; use the original file list and order")- if local.sha256sum is not None and item.get("sha256sum") not in {None, local.sha256sum}:- raise ClientError("Task hash differs from the local file")- if item.get("upload_status") not in {"pending", "ready", "expired"}:- raise ClientError("Unknown upload_status")- uploads = payload.get("pending_uploads")- if not isinstance(uploads, list):- raise ClientError("Missing pending_uploads list")- seen_uploads: set[str] = set()- pending = {item["client_file_id"] for item in files if item["upload_status"] == "pending"}- for link in uploads:+ if "upload" not in item:+ raise ClientError("Missing file upload descriptor")+ link = item["upload"]+ if link is None:+ continueif not isinstance(link, dict):raise ClientError("Invalid upload descriptor")- client_id = link.get("client_file_id")- if not isinstance(client_id, str) or client_id not in self.by_id or client_id in seen_uploads:- raise ClientError("Invalid or duplicate upload client_file_id")- seen_uploads.add(client_id)- if not isinstance(link.get("upload_url"), str) or not link["upload_url"]:+ if not isinstance(link.get("url"), str) or not link["url"]:raise ClientError("Invalid upload URL")- if not isinstance(link.get("upload_headers"), dict):- raise ClientError("Invalid upload headers")- if not seen_uploads.issubset(pending):- raise ClientError("Upload descriptors do not match pending files")- if payload["status"] in TERMINAL and seen_uploads:- raise ClientError("Terminal task still returns upload URLs")+ if not isinstance(link.get("headers"), dict) or type(link.get("expires_at")) is not int:+ raise ClientError("Invalid upload headers or expiry")+ if item["status"] != "waiting-file" or payload["status"] in TERMINAL:+ raise ClientError("Upload descriptor does not match file state")formats = payload.get("output_formats")if (not isinstance(formats, list)@@ -288,7 +281,7 @@ class TaskClient:):raise ClientError("Invalid task output_formats")for item in files:- if item.get("status") not in {"waiting_upload", "queued", "running", "completed", "failed", "canceled"}:+ if item.get("status") not in {"waiting-file", "pending", "running", "completed", "failed", "canceled"}:raise ClientError("Unknown file status")if item["status"] == "completed":if not isinstance(item.get("result_url"), str) or not item["result_url"]:@@ -296,7 +289,6 @@ class TaskClient:elif item.get("result_url") is not None:raise ClientError("Unfinished file has a download URL")self.task_id = task_id- self.status_url = self.resolve(payload["status_url"])return payloaddef create(self) -> dict[str, Any]:@@ -321,9 +313,9 @@ class TaskClient:def refresh(self) -> dict[str, Any]:"""只查询已有任务,任何查询失败都不能触发创建重试。"""- if self.status_url is None:- raise ClientError("No task status URL")- return self.json_request("GET", self.status_url)+ if self.task_id is None or self.task_id in {".", ".."}:+ raise ClientError("No valid task ID")+ return self.json_request("GET", self.base + "/v1/tasks/" + quote(self.task_id, safe=""))def refresh_after_transfer_error(self) -> dict[str, Any]:"""传输结果不确定时先恢复状态查询,无法确认时不盲目重复发送文件。"""@@ -337,12 +329,19 @@ class TaskClient:raise AssertionError("unreachable")def upload_missing(self, task: dict[str, Any]) -> dict[str, Any]:- """逐文件上传与恢复,已经 ready 的文件永远不会重新发送。"""- for original in task["pending_uploads"]:+ """逐文件上传与恢复,成功 PUT 或已进入后续阶段的文件不重复发送。"""+ for original in task["files"]:client_id = original["client_file_id"]local = self.by_id[client_id]- link = original- for attempt in range(3):+ link = original["upload"]+ if original["status"] != "waiting-file":+ self.uploaded_ids.add(client_id)+ if link is None or client_id in self.uploaded_ids:+ continue+ if self.upload_attempts.get(client_id, 0) >= 3:+ raise ClientError(f"Upload failed after three attempts: {local.path.name}")+ for attempt in range(self.upload_attempts.get(client_id, 0), 3):+ self.upload_attempts[client_id] = attempt + 1def upload_body() -> Generator[bytes, None, None]:"""每次重试重新打开同一个文件,单文件从头重传。"""@@ -351,13 +350,14 @@ class TaskClient:try:with self.request("PUT",- link["upload_url"],- headers=link["upload_headers"],+ link["url"],+ headers=link["headers"],body=upload_body,length=local.size,) as response:if not 200 <= response.status < 300:raise self._http_error(response)+ self.uploaded_ids.add(client_id)breakexcept ClientError as exc:if not exc.retryable and exc.status not in {403, 410}:@@ -365,16 +365,13 @@ class TaskClient:# 请求可能已经落盘但响应丢失;先看状态,再决定是否重传。current = self.refresh_after_transfer_error()remote = next(item for item in current["files"] if item["client_file_id"] == client_id)- if remote["upload_status"] == "ready":- break- if remote["status"] in {"failed", "canceled"}:+ if remote["status"] != "waiting-file":+ self.uploaded_ids.add(client_id)break- replacement = next(- (item for item in current["pending_uploads"] if item["client_file_id"] == client_id), None- )+ replacement = remote["upload"]if replacement is None:- raise ClientError("Pending file has no upload URL") from exc- if exc.status == 403 and replacement["upload_url"] == link["upload_url"]:+ break+ if exc.status == 403 and replacement["url"] == link["url"]:raiseif attempt == 2:raise ClientError(f"Upload failed after three attempts: {local.path.name}") from exc@@ -461,17 +458,16 @@ class TaskClient:resume = os.environ.get("TASK_ID", "")if resume:self.task_id = resume- self.status_url = self.base + "/v1/tasks/" + quote(resume, safe="")task = self.refresh()else:task = self.create()print(f"task_id={self.task_id}", file=sys.stderr)self.download_completed(task, output)- for link in list(task["pending_uploads"]):- current_links = [item for item in task["pending_uploads"] if item["client_file_id"] == link["client_file_id"]]- if not current_links:+ for original in list(task["files"]):+ current = next(item for item in task["files"] if item["client_file_id"] == original["client_file_id"])+ if current["upload"] is None:continue- task = self.upload_missing({**task, "pending_uploads": current_links})+ task = self.upload_missing({**task, "files": [current]})self.download_completed(task, output)polls = 0while task["status"] not in TERMINAL:@@ -486,6 +482,13 @@ class TaskClient:if not exc.retryable:raisecontinue+ if any(+ item["status"] == "waiting-file"+ and item["upload"] is not None+ and item["client_file_id"] not in self.uploaded_ids+ for item in task["files"]+ ):+ task = self.upload_missing(task)self.download_completed(task, output)if task["status"] in {"failed", "canceled"}:print(diff --git a/tests/unittest/task_test_utils.py b/tests/unittest/task_test_utils.pyindex 65b96e79..0f43d693 100644--- a/tests/unittest/task_test_utils.py+++ b/tests/unittest/task_test_utils.py@@ -8,12 +8,24 @@ import zipfilefrom pathlib import Pathfrom typing import Any+import httpxfrom fastapi.testclient import TestClient+from mineru.task_types import task_status_urlfrom mineru.parser.api_server import FileStore, _JobRecordfrom mineru.parser.base import ParseResultfrom mineru.parser.task_outputs import build_task_output+def task_url(client: TestClient | httpx.Client | httpx.AsyncClient, task: dict[str, Any]) -> str:+ """从客户端部署地址和任务 ID 构造固定查询路径,不依赖响应链接。"""+ return task_status_url(str(client.base_url), task["task_id"])+++def pending_files(task: dict[str, Any]) -> list[dict[str, Any]]:+ """从真实逐文件快照选取待上传项,缺少新 upload 字段时直接失败。"""+ return [item for item in task["files"] if item["upload"] is not None]++def stored_source(store: FileStore, path: Path) -> dict[str, str]:"""为执行层单元测试登记已校验源字节,不创建公开上传资源。"""sha = hashlib.sha256(path.read_bytes()).hexdigest()@@ -52,8 +64,8 @@ def submit_task(client: TestClient, path: Path, **options: Any) -> dict[str, Any)assert response.status_code == 202, response.texttask = response.json()- for link in task["pending_uploads"]:- response = client.put(link["upload_url"], content=data, headers=link["upload_headers"])+ for link in pending_files(task):+ response = client.put(link["upload"]["url"], content=data, headers=link["upload"]["headers"])assert response.status_code < 400, response.textreturn task@@ -62,7 +74,7 @@ def wait_task(client: TestClient, task: dict[str, Any], timeout: float = 5) -> d"""等待父任务终态,避免使用内部执行 ID 代替公开 task_id。"""deadline = time.monotonic() + timeoutwhile time.monotonic() < deadline:- response = client.get(task["status_url"])+ response = client.get(task_url(client, task))assert response.status_code == 200, response.texttask = response.json()if task["status"] in {"completed", "partial", "failed", "canceled"}:diff --git a/tests/unittest/test_engine_dead_lifecycle.py b/tests/unittest/test_engine_dead_lifecycle.pyindex ba129dbc..649586a2 100644--- a/tests/unittest/test_engine_dead_lifecycle.py+++ b/tests/unittest/test_engine_dead_lifecycle.py@@ -2,6 +2,8 @@from __future__ import annotations+from task_test_utils import task_url+from task_test_utils import submit_task, wait_taskimport asyncio@@ -60,11 +62,11 @@ def test_api_engine_failure_drains_running_and_queued_jobs(tmp_path: Path, monkeclient.portal.call(make_gate)jobs = []- for name in ("fatal", "running", "queued", "user-cancel"):+ for name in ("fatal", "running", "pending", "user-cancel"):source = tmp_path / f"{name}.pdf"source.write_bytes(name.encode())jobs.append(submit_task(client, source, tier="flash"))- assert client.delete(jobs[-1]["status_url"]).status_code == 200+ assert client.delete(task_url(client, jobs[-1])).status_code == 200client.portal.call(gate.set)for job in jobs:wait_task(client, job)@@ -83,7 +85,7 @@ def test_api_engine_failure_drains_running_and_queued_jobs(tmp_path: Path, monkeclient.portal.call(drain_tasks)assert entered == ["fatal", "running"]assert set(cleanup) == {"fatal", "running"}- assert [client.get(job["status_url"]).json()["status"] for job in jobs] == [+ assert [client.get(task_url(client, job)).json()["status"] for job in jobs] == ["failed","failed","failed",@@ -186,7 +188,7 @@ def test_old_generation_failure_snapshot_remains_queryable() -> None:route.metadata["payload"] = {"job_id": route.public_id,"status": "running",- "files": [{"status": "queued"}],+ "files": [{"status": "pending"}],"progress": {"total": 1, "completed": 0},}client.portal.call(pool.refresh, state)diff --git a/tests/unittest/test_file_parse.py b/tests/unittest/test_file_parse.pyindex 0b5bf360..89b30e52 100644--- a/tests/unittest/test_file_parse.py+++ b/tests/unittest/test_file_parse.py@@ -72,34 +72,31 @@ class Upstream:ready = task["ready"] and time.monotonic() >= task["uploaded_at"] + self.ready_delaycomplete = ready and self.complete and time.monotonic() >= task["uploaded_at"] + self.complete_delayexpired = not ready and task["expires_at"] <= time.time()- status = "failed" if expired else "completed" if complete else "running" if ready else "waiting_upload"+ status = "failed" if expired else "completed" if complete else "running" if ready else "waiting-file"result_url = f"{self.storage_url}/result/{task_id}?signature=a%2Fb&version={task['version']}" if complete else Noneif result_url and self.redirect:result_url = f"{self.api_url}/redirect/{task_id}"return {"task_id": task_id,"status": status,- "status_url": f"{self.api_url}/v1/tasks/{task_id}","output_formats": task["options"]["output_formats"],"files": [{"client_file_id": "file",+ "file_name": task["options"]["files"][0]["file_name"],+ "bytes": task["options"]["files"][0]["bytes"],"status": status,- "upload_status": "expired" if expired else "ready" if ready else "pending",+ "upload": None+ if task["ready"] or expired+ else {+ "url": f"{self.storage_url}/upload/{task_id}",+ "headers": {"x-upload-token": "storage-token"},+ "expires_at": task["expires_at"],+ },"result_url": result_url,"error": {"code": "upload_expired"} if expired else None,}],- "pending_uploads": []- if ready or expired- else [- {- "client_file_id": "file",- "upload_url": f"{self.storage_url}/upload/{task_id}",- "upload_headers": {"x-upload-token": "storage-token"},- "expires_at": task["expires_at"],- }- ],}async def handle(self, request: httpx.Request) -> httpx.Response:@@ -386,8 +383,8 @@ def test_new_protocol_preserves_but_does_not_restore_legacy_records(tmp_path: Paupstream = Upstream()settings = FileParseSettings(upstream.api_url, "secret", tmp_path)legacy_namespace = hashlib.sha256(f"{settings.api_url}\0secret".encode()).hexdigest()- assert settings.namespace == "tasks-v2-" + legacy_namespace- legacy = tmp_path / legacy_namespace+ assert settings.namespace == "tasks-v4-" + legacy_namespace+ legacy = tmp_path / ("tasks-v3-" + legacy_namespace)source = legacy / "sources" / "old-input"source.parent.mkdir(parents=True)source.write_bytes(b"old source")@@ -457,7 +454,7 @@ def test_restart_at_upload_boundaries(tmp_path: Path, stage: str) -> None:try:task = await remote.create(record)record.remote = task.model_dump()- record.status = "waiting_upload"+ record.status = "waiting-file"if stage == "expired":upstream.tasks[task.task_id]["expires_at"] = int(time.time()) - 1finally:diff --git a/tests/unittest/test_kit_gradio.py b/tests/unittest/test_kit_gradio.pyindex e6f20d87..1315c6ac 100644--- a/tests/unittest/test_kit_gradio.py+++ b/tests/unittest/test_kit_gradio.py@@ -703,7 +703,7 @@ def test_v1_client_parse_uses_api_parser_zip_contract(calls["page_range"] = page_rangeassert statuses == ["Preparing request...", "Submitting task..."]assert status_callback is not None- status_callback("queued")+ status_callback("pending")if observe_running:status_callback("running")status_callback("completed")diff --git a/tests/unittest/test_parser_api_contract.py b/tests/unittest/test_parser_api_contract.pyindex 71eb3ccc..12b679d9 100644--- a/tests/unittest/test_parser_api_contract.py+++ b/tests/unittest/test_parser_api_contract.py@@ -720,16 +720,17 @@ def test_api_server_logs_traceback_when_job_file_fails(def test_parse_info_uses_parser_version_not_backend_version() -> None:- """解析统计仅保留耗时和解析器版本,不新增逐文件档位。"""- parse = FileParseInfo(duration_ms=12, parser_version="3.2.1")+ """解析统计包含实际档位、耗时和解析器版本,不接受已删除的模型字段。"""+ parse = FileParseInfo(tier="basic", duration_ms=12, parser_version="3.2.1")assert parse.model_dump(exclude_none=True) == {+ "tier": "basic","duration_ms": 12,"parser_version": "3.2.1",}- for removed in ("backend_version", "model_used", "tier"):+ for removed in ("backend_version", "model_used"):with pytest.raises(ValidationError):- FileParseInfo.model_validate({removed: "unused"})+ FileParseInfo.model_validate({"tier": "basic", removed: "unused"})@pytest.mark.parametrize("field", ["backend_version", "parser_version"])diff --git a/tests/unittest/test_parser_api_ocr.py b/tests/unittest/test_parser_api_ocr.pyindex db9c697e..cf5749b6 100644--- a/tests/unittest/test_parser_api_ocr.py+++ b/tests/unittest/test_parser_api_ocr.py@@ -40,7 +40,7 @@ def test_api_parser_serializes_request_ocr_mode(return httpx.Response(200, content=zip_output())assert request.method == "POST" and request.url.path == "/v1/tasks"payloads.append(json.loads(request.content))- return httpx.Response(202, json={**task_snapshot(), "status_url": "http://localhost:8000/v1/tasks/task-one"})+ return httpx.Response(202, json=task_snapshot())transport = httpx.MockTransport(respond)sync_client, async_client = httpx.Client, httpx.AsyncClientdiff --git a/tests/unittest/test_parser_api_status.py b/tests/unittest/test_parser_api_status.pyindex 5f8aafe8..aa1ebe01 100644--- a/tests/unittest/test_parser_api_status.py+++ b/tests/unittest/test_parser_api_status.py@@ -23,12 +23,13 @@ from mineru.types import MiddleJson@pytest.mark.parametrize("statuses",[- ("queued", "running", "running", "completed"),+ ("waiting-file", "pending", "running", "completed"),+ ("pending", "running", "running", "completed"),("completed",),- ("queued", "completed"),- ("queued", "partial"),- ("queued", "running", "failed"),- ("queued", "canceled"),+ ("pending", "completed"),+ ("pending", "partial"),+ ("pending", "running", "failed"),+ ("pending", "canceled"),],)def test_parse_notifies_submission_and_polls_before_result_building(@@ -73,13 +74,12 @@ def test_parse_notifies_submission_and_polls_before_result_building(200,json={"task_id": "task_test",- "status_url": "/v1/tasks/task_test","status": status,"output_formats": ["default"],"files": [{"client_file_id": "file",- "upload_status": "ready",+ "upload": None,"status": "completed" if status == "partial" else status,"parse": {"duration_ms": 250},"result_url": "/result",diff --git a/tests/unittest/test_resource_retention.py b/tests/unittest/test_resource_retention.pyindex 225695ad..1579086e 100644--- a/tests/unittest/test_resource_retention.py+++ b/tests/unittest/test_resource_retention.py@@ -355,7 +355,7 @@ def test_external_worker_lost_task_releases_load_and_keeps_snapshot() -> None:"job", owner_scope="anonymous", worker_id=reservation.worker.worker_id, upstream_id="lost-task")route.metadata.update(- active_counted=True, reservation=reservation, payload={"status": "running", "files": [{"status": "queued"}]}+ active_counted=True, reservation=reservation, payload={"status": "running", "files": [{"status": "pending"}]})client.portal.call(_reconcile_jobs_once, app.state.registry, pool)assert reservation.worker.active_jobs == 0diff --git a/tests/unittest/test_router_balancing.py b/tests/unittest/test_router_balancing.pyindex 2a79e506..8492d5a6 100644--- a/tests/unittest/test_router_balancing.py+++ b/tests/unittest/test_router_balancing.py@@ -2,6 +2,8 @@from __future__ import annotations+from task_test_utils import pending_files+import asyncioimport threadingimport time@@ -125,7 +127,7 @@ def test_cached_tasks_still_balance(tmp_path: Path, monkeypatch: pytest.MonkeyPaassert [worker.active_jobs for worker in app.state.worker_pool.workers] == [0, 0]_upload(client, task, [("source.html", HTML)])else:- assert task["pending_uploads"] == []+ assert pending_files(task) == []tasks.append(task)deadline = time.monotonic() + 5while not app.state.task_store._records[task["task_id"]].file_states[0].submitted:diff --git a/tests/unittest/test_router_cascade.py b/tests/unittest/test_router_cascade.pyindex 89381f70..b3c1654d 100644--- a/tests/unittest/test_router_cascade.py+++ b/tests/unittest/test_router_cascade.py@@ -2,6 +2,8 @@from __future__ import annotations+from task_test_utils import pending_files, task_url+import asyncioimport hashlibimport io@@ -103,7 +105,7 @@ async def _terminal(client: httpx.AsyncClient, task: dict[str, Any]) -> dict[str"""按真实公共链接等待终态,超时不允许测试永久挂起。"""async with asyncio.timeout(10):while True:- response = await client.get(task["status_url"])+ response = await client.get(task_url(client, task))assert response.status_code == 200, response.textresult = response.json()if result["status"] in {"completed", "partial", "failed", "canceled"}:@@ -120,31 +122,37 @@ async def _check_download(client: httpx.AsyncClient, result: dict[str, Any]) ->assert b"Preserved content" in archive.read("markdown.md")-def test_cascade_waits_for_source_file_and_reuses_cached_upload(tmp_path: Path) -> None:- """上传 ready 但 file_id 尚空时继续等待,交接后正常下载并按 hash 复用源文件。"""+def test_cascade_accepts_ready_upload_without_source_file_and_reuses_cache(tmp_path: Path) -> None:+ """下游尚未提交解析也可确认上传,级联只依赖任务身份和私有缓存。"""async def scenario() -> None:"""通过提交等待固定竞争窗口,验证内外层均不提前判错。"""async with _cascade(tmp_path, block_submission=True) as (client, outer, inner, worker, transport):task = await _submit(client)- response = await client.put(task["pending_uploads"][0]["upload_url"], content=HTML)+ response = await client.put(pending_files(task)[0]["upload"]["url"], content=HTML)assert response.status_code == 200, response.textawait asyncio.wait_for(transport.submission_started.wait(), timeout=5)inner_record = next(iter(inner.state.task_store._records.values()))inner_route = outer.state.registry.list("job")[0]ready = await outer.state.worker_pool.client.get(f"http://inner/v1/tasks/{inner_route.upstream_id}")- assert ready.json()["files"][0]["upload_status"] == "ready"- assert ready.json()["files"][0]["file_id"] is None- pending = (await client.get(task["status_url"])).json()- assert pending["files"][0]["status"] == "queued", pending- assert inner_record.status == "queued"+ assert ready.json()["files"][0]["status"] == "pending"+ assert ready.json()["files"][0]["upload"] is None+ assert "file_id" not in ready.json()["files"][0]+ async with asyncio.timeout(5):+ while not outer.state.task_store._records[task["task_id"]].file_states[0].submitted:+ await asyncio.sleep(0.01)+ assert not outer.state.registry.list("file")+ assert outer.state.source_store._active_references+ pending = (await client.get(task_url(client, task))).json()+ assert pending["files"][0]["status"] == "pending", pending+ assert inner_record.status == "pending"transport.allow_submission.set()result = await _terminal(client, task)assert result["status"] == "completed", result- assert result["files"][0]["file_id"] is not None+ assert "file_id" not in result["files"][0]await _check_download(client, result)cached = await _submit(client, filename="renamed.html")- assert cached["pending_uploads"] == []+ assert pending_files(cached) == []result = await _terminal(client, cached)assert result["status"] == "completed", resultawait _check_download(client, result)@@ -154,17 +162,17 @@ def test_cascade_waits_for_source_file_and_reuses_cached_upload(tmp_path: Path)asyncio.run(scenario())-def test_cascade_can_cancel_while_waiting_for_source_file(tmp_path: Path) -> None:- """等待源 File 的轮询可被取消,内外层预占和准备引用都正常释放。"""+def test_cascade_can_cancel_while_downstream_submission_is_blocked(tmp_path: Path) -> None:+ """下游提交阻塞时仍可取消,内外层预占和准备引用都正常释放。"""async def scenario() -> None:"""阻塞实际任务提交后取消外层父任务,确认没有遗留解析或缓存租约。"""async with _cascade(tmp_path, block_submission=True) as (client, outer, inner, worker, transport):task = await _submit(client)- response = await client.put(task["pending_uploads"][0]["upload_url"], content=HTML)+ response = await client.put(pending_files(task)[0]["upload"]["url"], content=HTML)assert response.status_code == 200, response.textawait asyncio.wait_for(transport.submission_started.wait(), timeout=5)- response = await client.delete(task["status_url"])+ response = await client.delete(task_url(client, task))assert response.status_code == 200, response.textassert response.json()["status"] == "canceled"async with asyncio.timeout(5):diff --git a/tests/unittest/test_router_source_cache.py b/tests/unittest/test_router_source_cache.pynew file mode 100644index 00000000..9c57ecae--- /dev/null+++ b/tests/unittest/test_router_source_cache.py@@ -0,0 +1,104 @@+"""私有源缓存不依赖下游 Files 标识,引用和过期由本层任务生命周期管理。"""++from __future__ import annotations++import hashlib+import time+from pathlib import Path++import pytest+from fastapi.testclient import TestClient+from task_test_utils import task_url+from test_task_api import HTML, _WorkerTransport, _submit, _terminal, _upload++from mineru.kit.router import RouterSettings, create_app+from mineru.kit.router.resources import SourceFileStore, StoredSourceFile+from mineru.parser import api_server+++def test_private_cache_reference_lifetime_and_transfer_pins(monkeypatch: pytest.MonkeyPatch) -> None:+ """独立回收只移除到期引用,活跃任务和并发下载均保护同一份字节。"""+ now = [1000.0]+ monkeypatch.setattr(time, "time", lambda: now[0])+ sources = SourceFileStore(retention_seconds=10)+ try:+ path = sources._root / "shared"+ path.write_bytes(b"source")+ stored = StoredSourceFile(path, 6, hashlib.sha256(b"source").hexdigest(), "application/pdf")+ for reference in ("task-a", "task-b"):+ sources.bind_source(reference, stored, owner_scope="owner")+ sources.hold_reference(reference)+ assert sources.find_hash("stranger", stored.sha256sum) is None+ now[0] += 1000+ sources.collect_expired()+ assert sources.find_hash("owner", stored.sha256sum) is stored+ sources.release_reference("task-a", now[0])+ deadline = sources._file_expiry["task-a"]+ now[0] += 11+ sources.release_reference("task-a", now[0])+ assert sources._file_expiry["task-a"] == deadline+ sources.collect_expired()+ assert set(sources._files) == {"task-b"} and path.exists()+ with sources.pin(stored):+ sources.release_reference("task-b", now[0])+ now[0] += 11+ sources.collect_expired()+ assert not sources._files and not sources._active_references+ assert sources.find_hash("owner", stored.sha256sum) is None+ assert path.exists()+ assert not path.exists() and not sources._pins+ finally:+ sources.close()+++def test_private_cache_disabled_retention_and_rollback(monkeypatch: pytest.MonkeyPatch) -> None:+ """禁用保留期仍保留已提交源缓存,准备失败回滚只删除自己的引用。"""+ sources = SourceFileStore(retention_seconds=0)+ try:+ path = sources._root / "source"+ path.write_bytes(b"source")+ stored = StoredSourceFile(path, 6, hashlib.sha256(b"source").hexdigest(), "application/pdf")+ for reference in ("committed", "prepared"):+ sources.bind_source(reference, stored, owner_scope="owner")+ sources.hold_reference(reference)+ sources.delete_file("prepared")+ sources.delete_file("prepared")+ sources.release_reference("committed", 1000)+ monkeypatch.setattr(time, "time", lambda: 10_000_000)+ sources.collect_expired()+ assert sources.find_hash("owner", stored.sha256sum) is stored+ assert set(sources._files) == {"committed"} and not sources._active_references+ finally:+ sources.close()+++def test_parent_cleanup_releases_sources_after_worker_route_disappears(tmp_path: Path) -> None:+ """执行路由已移除时,父任务仍须结束私有源引用并允许独立缓存回收。"""+ worker = api_server.create_app(upload_dir=str(tmp_path / "worker"), tier="flash")+ app = create_app(+ RouterSettings(upstream_urls=("http://worker",), local_gpus="none", worker_refresh_interval_seconds=0),+ transport=_WorkerTransport({"worker": worker}),+ )+ with TestClient(app) as client:+ task = _submit(client, [("source.html", HTML)], headers={"x-mineru-task-lease-seconds": "120"})+ _upload(client, task, [("source.html", HTML)])+ assert _terminal(client, task)["status"] == "completed"+ record = app.state.task_store._records[task["task_id"]]+ sources = app.state.source_store+ assert sources._active_references+ client.portal.call(app.state.registry.remove, "job", record.file_states[0].execution_id)+ assert client.get(task_url(client, task), headers={"x-mineru-task-lease-seconds": "0"}).status_code == 200++ async def drain() -> None:+ """等待真实父任务清理,确保检查时释放钩子已经执行。"""+ await record.operation++ client.portal.call(drain)+ assert not sources._active_references and not sources._pins+ assert sources._files+ for reference in sources._files:+ assert sources._file_expiry[reference] is not None+ sources._file_expiry[reference] = 0+ paths = {stored.path for stored in sources._files.values()}+ client.portal.call(sources.collect_expired)+ assert not sources._files and all(not path.exists() for path in paths)diff --git a/tests/unittest/test_task_api.py b/tests/unittest/test_task_api.pyindex 9dcfb0fa..8519f291 100644--- a/tests/unittest/test_task_api.py+++ b/tests/unittest/test_task_api.py@@ -2,6 +2,8 @@from __future__ import annotations+from task_test_utils import pending_files, task_url+import asyncioimport hashlibimport io@@ -56,6 +58,26 @@ def _execution_apps(app: FastAPI) -> list[FastAPI]:return list(app.state.test_transport.apps.values()) if hasattr(app.state, "test_transport") else [app]+def _source_references(app: FastAPI, task: dict[str, Any]) -> set[str]:+ """从内部存储定位源引用,公共 Tasks 响应不再暴露源文件标识。"""+ upload = app.state.task_store._records[task["task_id"]].file_states[0].upload+ assert upload is not None+ if hasattr(app.state, "source_store"):+ return {key for key, stored in app.state.source_store._files.items() if stored.path == upload.path}+ assert upload.file_id is not None+ return {upload.file_id}+++def _delete_source_references(client: TestClient, app: FastAPI, references: set[str]) -> None:+ """删除旧源引用以复现共享字节竞争,Router 私有引用不经过 Files HTTP。"""+ assert references+ for reference in references:+ if hasattr(app.state, "source_store"):+ client.portal.call(app.state.source_store.delete_file, reference)+ else:+ assert client.delete(f"/v1/files/{reference}").status_code == 200++def _url_manifest(**metadata: Any) -> dict[str, Any]:"""构造与下载地址名称无关的逻辑文件条目,默认不声明大小或 hash。"""return {@@ -135,10 +157,10 @@ def _upload(client: TestClient, task: dict[str, Any], inputs: list[tuple[str, bytes]], *, headers: dict[str, str] | None = None) -> None:"""仅上传返回链接对应的缺失文件,按 client_file_id 保持同名文件的关联。"""- for link in task["pending_uploads"]:+ for link in pending_files(task):index = int(link["client_file_id"][1:]) - 1response = client.put(- link["upload_url"], content=inputs[index][1], headers={**link["upload_headers"], **(headers or {})}+ link["upload"]["url"], content=inputs[index][1], headers={**link["upload"]["headers"], **(headers or {})})assert response.status_code == 200, response.text@@ -147,7 +169,7 @@ def _terminal(client: TestClient, task: dict[str, Any], *, headers: dict[str, st"""只轮询状态接口;未完成的文件不能发布 result_url。"""deadline = time.monotonic() + 20while time.monotonic() < deadline:- response = client.get(task["status_url"], headers=headers)+ response = client.get(task_url(client, task), headers=headers)assert response.status_code == 200, response.textpayload = response.json()if payload["status"] in task_api.TERMINAL_TASK_STATUSES:@@ -161,7 +183,7 @@ def _file_done(client: TestClient, task: dict[str, Any], index: int) -> dict[str"""只等待指定文件完成,父任务可继续等待其他上传。"""deadline = time.monotonic() + 20while time.monotonic() < deadline:- payload = client.get(task["status_url"]).json()+ payload = client.get(task_url(client, task)).json()if payload["files"][index]["status"] in {"completed", "failed", "canceled"}:return payloadtime.sleep(0.01)@@ -191,13 +213,13 @@ def test_manifest_uploads_and_zip_only_status(service: tuple[TestClient, FastAPIclient, app = serviceinputs = [("同名 文档.html", HTML), ("同名 文档.html", SECOND)]task = _submit(client, inputs, hashes=hashes, output_formats=["default"])- assert task["status"] == "waiting_upload" and "result_url" not in task- assert len(task["pending_uploads"]) == 2 and _job_count(app) == 0- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200+ assert task["status"] == "waiting-file" and "result_url" not in task+ assert len(pending_files(task)) == 2 and _job_count(app) == 0+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200ready = _file_done(client, task, 0)- assert ready["status"] == "waiting_upload"- assert [file["upload_status"] for file in ready["files"]] == ["ready", "pending"]- assert [link["client_file_id"] for link in ready["pending_uploads"]] == ["f2"]+ assert ready["status"] == "waiting-file"+ assert [file["status"] for file in ready["files"]] == ["completed", "waiting-file"]+ assert [link["client_file_id"] for link in pending_files(ready)] == ["f2"]assert _job_count(app) == 1assert b"Preserved content" in _archive(client, ready)["markdown.md"]_upload(client, ready, inputs)@@ -213,7 +235,7 @@ def test_manifest_uploads_and_zip_only_status(service: tuple[TestClient, FastAPIassert client.get(f"/v1/tasks/{task['task_id']}/result").status_code == 404assert client.post("/v1/file_parse").status_code == 404assert client.post("/tasks").status_code == 404- assert client.get(f"/v1/files/{result['files'][0]['file_id']}/content").status_code == 403+ assert all({"file_id", "name", "sha256sum", "page_range"}.isdisjoint(file) for file in result["files"])def test_direct_and_form_submissions_are_rejected(service: tuple[TestClient, FastAPI]) -> None:@@ -243,11 +265,12 @@ def test_mixed_cache_hit_survives_source_deletion(service: tuple[TestClient, Fasinitial = _submit(client, [("original.html", HTML)])_upload(client, initial, [("original.html", HTML)])original = _terminal(client, initial)+ old_references = _source_references(app, original)inputs = [("renamed.html", HTML), ("new.html", SECOND)]task = _submit(client, inputs)- assert [link["client_file_id"] for link in task["pending_uploads"]] == ["f2"]- assert task["files"][0]["upload_status"] == "ready"- assert client.delete(f"/v1/files/{original['files'][0]['file_id']}").status_code == 200+ assert [link["client_file_id"] for link in pending_files(task)] == ["f2"]+ assert task["files"][0]["status"] == "pending" and task["files"][0]["upload"] is None+ _delete_source_references(client, app, old_references)duplicate = client.put(f"/v1/tasks/{task['task_id']}/uploads/f1",content=_UnreadableBody(),@@ -256,9 +279,9 @@ def test_mixed_cache_hit_survives_source_deletion(service: tuple[TestClient, Fasassert duplicate.status_code == 200 and duplicate.headers["connection"] == "close"_upload(client, task, inputs)result = _terminal(client, task)- assert result["status"] == "completed" and result["files"][0]["name"] == "renamed.html"+ assert result["status"] == "completed" and result["files"][0]["file_name"] == "renamed.html"cached = _submit(client, inputs)- assert cached["pending_uploads"] == [] and cached["status"] == "queued"+ assert pending_files(cached) == [] and cached["status"] == "pending"assert _terminal(client, cached)["status"] == "completed"assert _job_count(app) == 5@@ -270,22 +293,22 @@ def test_no_hash_still_deduplicates_source_storage(service: tuple[TestClient, Fa_upload(client, first, [("first.html", HTML)])assert _terminal(client, first)["status"] == "completed"second = _submit(client, [("second.html", HTML)], hashes=False)- assert len(second["pending_uploads"]) == 1+ assert len(pending_files(second)) == 1_upload(client, second, [("second.html", HTML)])assert _terminal(client, second)["status"] == "completed"records = app.state.task_store._recordsassert records[first["task_id"]].file_states[0].upload.path == records[second["task_id"]].file_states[0].upload.path-def test_waiting_upload_retains_deleted_source_views(service: tuple[TestClient, FastAPI]) -> None:+def test_waiting_file_retains_deleted_source_views(service: tuple[TestClient, FastAPI]) -> None:"""删除已解析源视图不影响已生成结果,也不阻止其他文件继续上传。"""client, app = serviceinputs = [("a.html", HTML), ("b.html", SECOND)]task = _submit(client, inputs, hashes=False)- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200ready = _file_done(client, task, 0)- assert client.delete(f"/v1/files/{ready['files'][0]['file_id']}").status_code == 200- assert client.put(task["pending_uploads"][1]["upload_url"], content=SECOND).status_code == 200+ _delete_source_references(client, app, _source_references(app, ready))+ assert client.put(pending_files(task)[1]["upload"]["url"], content=SECOND).status_code == 200result = _terminal(client, task)assert result["status"] == "completed", resultassert b"Preserved content" in _archive(client, result)["markdown.md"]@@ -296,14 +319,15 @@ def test_failed_upload_can_retry_without_reuploading_ready_file(service: tuple[Tclient, app = serviceinputs = [("a.html", HTML), ("b.html", SECOND)]task = _submit(client, inputs)- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200- failed = client.put(task["pending_uploads"][1]["upload_url"], content=SECOND.replace(b"Second", b"Broken"))+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200+ failed = client.put(pending_files(task)[1]["upload"]["url"], content=SECOND.replace(b"Second", b"Broken"))assert failed.status_code == 400 and failed.json()["error"]["code"] == "file_hash_mismatch"- mismatch = client.put(task["pending_uploads"][1]["upload_url"], content=SECOND + b"x")+ mismatch = client.put(pending_files(task)[1]["upload"]["url"], content=SECOND + b"x")assert mismatch.status_code == 400 and mismatch.json()["error"]["code"] == "upload_size_mismatch"pending = _file_done(client, task, 0)- assert pending["status"] == "waiting_upload" and _job_count(app) == 1- assert [file["upload_status"] for file in pending["files"]] == ["ready", "pending"]+ assert pending["status"] == "waiting-file" and _job_count(app) == 1+ assert pending["files"][0]["status"] in {"pending", "running", "completed"}+ assert pending["files"][1]["status"] == "waiting-file"assert list(app.state.task_store._records[task["task_id"]].directory.iterdir()) == []_upload(client, pending, inputs)assert _terminal(client, task)["status"] == "completed" and _job_count(app) == 2@@ -318,9 +342,9 @@ def test_concurrent_last_uploads_and_duplicate_put_submit_once(service: tuple[Te"""在服务所属事件循环同时发起两个文件及同一文件的重复请求。"""async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app), base_url="http://testserver") as asynchronous:return await asyncio.gather(- asynchronous.put(task["pending_uploads"][0]["upload_url"], content=HTML),- asynchronous.put(task["pending_uploads"][1]["upload_url"], content=SECOND),- asynchronous.put(task["pending_uploads"][1]["upload_url"], content=SECOND),+ asynchronous.put(pending_files(task)[0]["upload"]["url"], content=HTML),+ asynchronous.put(pending_files(task)[1]["upload"]["url"], content=SECOND),+ asynchronous.put(pending_files(task)[1]["upload"]["url"], content=SECOND),)responses = client.portal.call(upload_all)@@ -332,29 +356,29 @@ def test_concurrent_last_uploads_and_duplicate_put_submit_once(service: tuple[Te@pytest.mark.parametrize("expired", [False, True])-def test_cancel_and_expire_waiting_upload_release_inputs(service: tuple[TestClient, FastAPI], expired: bool) -> None:+def test_cancel_and_expire_waiting_file_release_inputs(service: tuple[TestClient, FastAPI], expired: bool) -> None:"""取消或上传过期只终结未完成文件,已生成结果继续可下载。"""client, app = servicetask = _submit(client, [("a.html", HTML), ("b.html", SECOND)])- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200ready = _file_done(client, task, 0)record = app.state.task_store._records[task["task_id"]]if expired:record.expires_at = int(time.time()) - 1- response = client.get(task["status_url"])+ response = client.get(task_url(client, task))assert response.json()["files"][1]["error"]["code"] == "upload_expired"- assert response.json()["files"][1]["upload_status"] == "expired"+ assert response.json()["files"][1]["status"] == "failed"else:- response = client.delete(task["status_url"])+ response = client.delete(task_url(client, task))assert response.status_code == 200assert response.json()["status"] == ("partial" if expired else "canceled")- assert response.json()["pending_uploads"] == [] and _job_count(app) == 1+ assert pending_files(response.json()) == [] and _job_count(app) == 1assert response.json()["files"][0]["result_url"] == ready["files"][0]["result_url"]assert b"Preserved content" in _archive(client, response.json())["markdown.md"]- rejected = client.put(task["pending_uploads"][1]["upload_url"], content=SECOND)+ rejected = client.put(pending_files(task)[1]["upload"]["url"], content=SECOND)assert rejected.status_code == (410 if expired else 409)if expired:- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 410+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 410async def wait_cleanup() -> None:"""等待父任务持有的清理句柄,避免按定时猜测引用是否已释放。"""@@ -362,7 +386,11 @@ def test_cancel_and_expire_waiting_upload_release_inputs(service: tuple[TestClieclient.portal.call(wait_cleanup)assert not record.directory.exists()- assert all(file.backend._owned_files == [] for file in record.file_states)+ if hasattr(app.state, "source_store"):+ assert all(not file.backend._prepared_sources and not file.backend._task_sources for file in record.file_states)+ assert not app.state.source_store._active_references+ else:+ assert all(file.backend._owned_files == [] for file in record.file_states)def test_cancel_running_task_releases_execution_slot(@@ -381,10 +409,10 @@ def test_cancel_running_task_releases_execution_slot(task = _submit(client, [("a.html", HTML)])_upload(client, task, [("a.html", HTML)])assert started.wait(5)- response = client.delete(task["status_url"])+ response = client.delete(task_url(client, task))assert response.status_code == 200, response.textassert response.json()["status"] == "canceled"- assert client.get(task["status_url"]).json()["files"][0]["result_url"] is None+ assert client.get(task_url(client, task)).json()["files"][0]["result_url"] is Nonedeadline = time.monotonic() + 5operation = app.state.task_store._records[task["task_id"]].operationwhile operation is not None and not operation.done():@@ -424,7 +452,7 @@ def test_status_never_reads_or_rebuilds_zip(service: tuple[TestClient, FastAPI],monkeypatch.setattr(api_server.FileStore, "read_blob", forbid_read)monkeypatch.setattr(api_server.FileStore, "read_file_data", forbid_read)for _ in range(3):- assert client.get(task["status_url"]).json()["files"] == result["files"]+ assert client.get(task_url(client, task)).json()["files"] == result["files"]assert "markdown.md" in _archive(client, result)assert "markdown.md" in _archive(client, result, 1)assert calls == 2@@ -505,10 +533,10 @@ def test_input_validation_and_openapi(service: tuple[TestClient, FastAPI]) -> Noassert "file_name" in files_schema["items"]["required"]assert "client_file_id" not in files_schema["items"]["required"]assert "filename" not in files_schema["items"]["properties"]- for model in ("TaskFileStatus", "TaskUploadLink"):- id_schema = spec["components"]["schemas"][model]- assert "client_file_id" in id_schema["required"]- assert id_schema["properties"]["client_file_id"]["type"] == "string"+ id_schema = spec["components"]["schemas"]["TaskFileStatus"]+ assert "client_file_id" in id_schema["required"]+ assert id_schema["properties"]["client_file_id"]["type"] == "string"+ assert set(spec["components"]["schemas"]["TaskUploadLink"]["properties"]) == {"url", "headers", "expires_at"}if hasattr(app.state, "job_store"):metadata_schema = spec["components"]["schemas"]["FileObjectModel"]assert "file_name" in metadata_schema["required"]@@ -544,12 +572,12 @@ def test_optional_file_ids_survive_upload_and_router_forwarding(task = response.json()for _ in range(2):assert [item["client_file_id"] for item in task["files"]] == expected_ids- assert [item["client_file_id"] for item in task["pending_uploads"]] == expected_ids+ assert [item["client_file_id"] for item in pending_files(task)] == expected_idsassert all(item["file_name"] == "same.html" and "filename" not in item for item in task["files"])- task = client.get(task["status_url"]).json()- for link in task["pending_uploads"]:+ task = client.get(task_url(client, task)).json()+ for link in pending_files(task):for _ in range(2):- assert client.put(link["upload_url"], content=HTML, headers=link["upload_headers"]).status_code == 200+ assert client.put(link["upload"]["url"], content=HTML, headers=link["upload"]["headers"]).status_code == 200completed = _terminal(client, task)assert completed["status"] == "completed", completedassert [item["client_file_id"] for item in completed["files"]] == expected_ids@@ -580,20 +608,25 @@ def test_invalid_explicit_file_ids_fail_before_creation(service: tuple[TestCliendef test_files_metadata_and_attachment_keep_logical_names(service: tuple[TestClient, FastAPI]) -> None:"""Tasks 和 Files 只返回新文件名字段,源文件与结果附件仍使用各自逻辑名称。"""- client, _ = service+ client, app = serviceinputs = [("report.v2.html", HTML)]task = _submit(client, inputs)_upload(client, task, inputs)completed = _terminal(client, task)item = completed["files"][0]- assert item["file_name"] == item["name"] == "report.v2.html" and "filename" not in item- source = client.get(f"/v1/files/{item['file_id']}").json()+ assert item["file_name"] == "report.v2.html" and "name" not in item and "filename" not in itemresult = client.get(item["result_url"].removesuffix("/content")).json()- assert source["file_name"] == "report.v2.html" and "filename" not in source+ if hasattr(app.state, "file_store"):+ source_id = next(iter(_source_references(app, completed)))+ source = client.get(f"/v1/files/{source_id}").json()+ assert source["file_name"] == "report.v2.html" and "filename" not in source+ assert client.get(f"/v1/files/{source_id}/content").status_code == 403+ else:+ assert all(route.metadata["payload"]["purpose"] == "parse_output" for route in app.state.registry.list("file"))assert result["file_name"] == "report.v2.zip" and "filename" not in result- assert set(item["parse"]) == {"duration_ms", "parser_version"}- assert completed["tier"] == "flash"- assert completed["pending_uploads"] == []+ assert set(item["parse"]) == {"tier", "duration_ms", "parser_version"}+ assert item["parse"]["tier"] == "flash" and "tier" not in completed+ assert pending_files(completed) == []assert "uploads" not in completed and "include_model_output" not in completeddownload = client.get(item["result_url"])assert download.status_code == 200@@ -625,16 +658,16 @@ def test_url_inputs_download_in_worker_and_report_actual_metadata(task = response.json()expected_id = item["client_file_id"] or "f1"assert task["files"][0]["client_file_id"] == expected_id- assert task["status"] == "queued" and task["pending_uploads"] == []+ assert task["status"] == "waiting-file" and pending_files(task) == []assert task["files"][0]["bytes"] == metadata.get("bytes")- assert task["files"][0]["upload_status"] == "not_required"+ assert task["files"][0]["status"] == "waiting-file" and task["files"][0]["upload"] is Noneresult = _terminal(client, task)assert result["status"] == "completed"file = result["files"][0]assert file["client_file_id"] == expected_id- assert file["file_name"] == file["name"] == "logical.html"- assert file["bytes"] == len(HTML) and file["sha256sum"] == hashlib.sha256(HTML).hexdigest()- assert file["upload_status"] == "not_required" and file["file_id"] is None+ assert file["file_name"] == "logical.html" and "name" not in file+ assert file["bytes"] == len(HTML) and "sha256sum" not in file+ assert file["status"] == "completed" and "file_id" not in file and file["upload"] is Noneassert file["url"] == item["url"]assert b"Preserved content" in _archive(client, result)["markdown.md"]assert len(url_downloads) == 1 and str(url_downloads[0].url) == item["url"]@@ -661,7 +694,7 @@ def test_url_manifest_validation(service: tuple[TestClient, FastAPI], url_downloassert response.status_code == 415assert _job_count(app) == 0 and url_downloads == []schema = client.get("/openapi.json").json()- assert "not_required" in schema["components"]["schemas"]["TaskFileStatus"]["properties"]["upload_status"]["enum"]+ assert "upload_status" not in schema["components"]["schemas"]["TaskFileStatus"]["properties"]def test_mixed_url_upload_partial_and_no_early_reservation(@@ -677,12 +710,12 @@ def test_mixed_url_upload_partial_and_no_early_reservation(response = client.post("/v1/tasks", json={"files": manifest})assert response.status_code == 202, response.texttask = response.json()- assert task["status"] == "queued"- assert [link["client_file_id"] for link in task["pending_uploads"]] == ["local"]+ assert task["status"] == "waiting-file"+ assert [link["client_file_id"] for link in pending_files(task)] == ["local"]ready = _file_done(client, task, 0)assert b"Preserved content" in _archive(client, ready)["markdown.md"]- assert ready["files"][1]["status"] == "waiting_upload"- assert client.put(task["pending_uploads"][0]["upload_url"], content=SECOND).status_code == 200+ assert ready["files"][1]["status"] == "waiting-file"+ assert client.put(pending_files(task)[0]["upload"]["url"], content=SECOND).status_code == 200result = _terminal(client, task)assert result["status"] == "partial" and result["progress"] == {"completed": 2, "failed": 1, "total": 3}assert result["files"][0]["bytes"] == len(HTML)@@ -794,7 +827,7 @@ def test_url_logical_filename_drives_flash_fallback(assert response.status_code == 202, response.textresult = _terminal(client, response.json())assert result["status"] == "completed", result- assert result["files"][0]["name"] == "logical.html"+ assert result["files"][0]["file_name"] == "logical.html"assert b"Preserved content" in _archive(client, result)["markdown.md"]assert len(url_downloads) == 1@@ -836,7 +869,7 @@ def test_cancel_url_download_closes_stream_and_releases_worker(task = response.json()assert started.wait(5)before = time.monotonic()- canceled = client.delete(task["status_url"])+ canceled = client.delete(task_url(client, task))assert canceled.status_code == 200 and canceled.json()["status"] == "canceled"assert time.monotonic() - before < 2assert closed.wait(5)@@ -848,7 +881,7 @@ def test_cancel_url_download_closes_stream_and_releases_worker(while any(worker.active_jobs for worker in app.state.worker_pool.workers):assert time.monotonic() < deadline, "Canceled URL task retained its worker reservation"time.sleep(0.01)- final = client.get(task["status_url"]).json()+ final = client.get(task_url(client, task)).json()assert final["status"] == "canceled" and final["files"][0]["result_url"] is Noneassert final["files"][0]["bytes"] is None@@ -865,26 +898,28 @@ def test_router_private_hash_cache_and_task_scope(tmp_path: Path) -> None:task = _submit(client, [("a.html", HTML)], headers=owner)_upload(client, task, [("a.html", HTML)], headers=owner)result = _terminal(client, task, headers=owner)- assert client.get(task["status_url"], headers=stranger).status_code == 404+ assert client.get(task_url(client, task), headers=stranger).status_code == 404assert client.get(result["files"][0]["result_url"], headers=stranger).status_code == 404other = _submit(client, [("a.html", HTML)], headers=stranger)- assert len(other["pending_uploads"]) == 1- assert client.delete(other["status_url"], headers=stranger).status_code == 200+ assert len(pending_files(other)) == 1+ assert client.delete(task_url(client, other), headers=stranger).status_code == 200owned = _submit(client, [("a.html", HTML)], headers=owner)- assert not owned["pending_uploads"]+ assert not pending_files(owned)assert _terminal(client, owned, headers=owner)["status"] == "completed"@pytest.mark.parametrize("root", ["", "/mineru"])def test_task_links_respect_proxy_root(tmp_path: Path, root: str) -> None:- """状态、上传与结果链接全部携带部署前缀,客户端不用重建任何地址。"""+ """固定查询路由保留客户端部署前缀,上传和结果仍使用服务返回的链接。"""app = api_server.create_app(upload_dir=str(tmp_path), tier="flash", api_key="secret")- with TestClient(app, root_path=root, headers={"Authorization": "Bearer secret"}) as client:+ with TestClient(+ app, base_url="http://testserver" + root, root_path=root, headers={"Authorization": "Bearer secret"}+ ) as client:manifest = [{"client_file_id": "f1", "file_name": "a.html", "bytes": len(HTML)}]- response = client.post(root + "/v1/tasks", json={"files": manifest, "tier": "flash"})+ response = client.post("/v1/tasks", json={"files": manifest, "tier": "flash"})assert response.status_code == 202task = response.json()- assert task["status_url"].startswith("http://testserver" + root + "/v1/tasks/")+ assert task_url(client, task).startswith("http://testserver" + root + "/v1/tasks/")_upload(client, task, [("a.html", HTML)])result = _terminal(client, task)assert result["files"][0]["result_url"].startswith("http://testserver" + root + "/v1/files/")@@ -899,9 +934,9 @@ def test_inflight_upload_may_finish_after_deadline(service: tuple[TestClient, Faclient, app = servicetask = _submit(client, [("a.html", HTML), ("b.html", SECOND)], hashes=False)record = app.state.task_store._records[task["task_id"]]- expires_at = task["pending_uploads"][0]["expires_at"]+ expires_at = pending_files(task)[0]["upload"]["expires_at"]assert 3598 <= expires_at - time.time() <= 3600- assert client.get(task["status_url"]).json()["pending_uploads"][0]["expires_at"] == expires_at+ assert pending_files(client.get(task_url(client, task)).json())[0]["upload"]["expires_at"] == expires_atasync def cross_deadline() -> httpx.Response:"""让真实 ASGI 上传跨过时限,并在受理流中途并发查询和尝试新 PUT。"""@@ -912,17 +947,17 @@ def test_inflight_upload_may_finish_after_deadline(service: tuple[TestClient, Fayield HTML[:10]assert record.file_states[0].upload_activerecord.expires_at = int(time.time()) - 1- state = (await transport.get(task["status_url"])).json()- assert state["files"][0]["status"] == "waiting_upload"+ state = (await transport.get(task_url(client, task))).json()+ assert state["files"][0]["status"] == "waiting-file"assert state["files"][1]["status"] == "failed"- assert state["pending_uploads"] == []- rejected = await transport.put(task["pending_uploads"][1]["upload_url"], content=SECOND)+ assert pending_files(state) == []+ rejected = await transport.put(pending_files(task)[1]["upload"]["url"], content=SECOND)assert rejected.status_code == 410if interrupted:raise ClientDisconnectyield HTML[10:]- return await transport.put(task["pending_uploads"][0]["upload_url"], content=body())+ return await transport.put(pending_files(task)[0]["upload"]["url"], content=body())response = client.portal.call(cross_deadline)result = _terminal(client, task)@@ -934,7 +969,7 @@ def test_inflight_upload_may_finish_after_deadline(service: tuple[TestClient, Faassert response.status_code == 200assert result["status"] == "partial"assert b"Preserved content" in _archive(client, result)["markdown.md"]- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 410+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 410def test_expired_upload_rejects_before_reading_body(service: tuple[TestClient, FastAPI]) -> None:@@ -942,12 +977,13 @@ def test_expired_upload_rejects_before_reading_body(service: tuple[TestClient, Fclient, app = servicetask = _submit(client, [("a.html", HTML)], hashes=False)app.state.task_store._records[task["task_id"]].expires_at = int(time.time()) - 1- response = client.put(task["pending_uploads"][0]["upload_url"], content=_UnreadableBody())+ response = client.put(pending_files(task)[0]["upload"]["url"], content=_UnreadableBody())assert response.status_code == 410 and response.json()["error"]["code"] == "upload_expired"for _ in range(2):- state = client.get(task["status_url"]).json()- assert state["status"] == "failed" and state["pending_uploads"] == []- assert state["files"][0]["upload_status"] == "expired"+ state = client.get(task_url(client, task)).json()+ assert state["status"] == "failed" and pending_files(state) == []+ assert state["files"][0]["status"] == "failed"+ assert state["files"][0]["error"]["code"] == "upload_expired"assert _job_count(app) == 0@@ -962,16 +998,16 @@ def test_completed_child_retention_starts_after_parent_finishes(service: tuple[Tworker.state.job_store.retention_seconds = 1worker.state.file_store.retention_seconds = 1task = _submit(client, [("a.html", HTML), ("b.html", SECOND)], hashes=False)- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200ready = _file_done(client, task, 0)result_url = ready["files"][0]["result_url"]time.sleep(1.05)for worker in _execution_apps(app):worker.state.job_store.collect_expired()worker.state.file_store.collect_expired()- assert client.get(task["status_url"]).json()["files"][0]["status"] == "completed"+ assert client.get(task_url(client, task)).json()["files"][0]["status"] == "completed"assert client.get(result_url).status_code == 200- assert client.put(task["pending_uploads"][1]["upload_url"], content=SECOND).status_code == 200+ assert client.put(pending_files(task)[1]["upload"]["url"], content=SECOND).status_code == 200assert _terminal(client, task)["status"] == "completed"record = app.state.task_store._records[task["task_id"]]@@ -982,7 +1018,7 @@ def test_completed_child_retention_starts_after_parent_finishes(service: tuple[Tclient.portal.call(finish_retention)assert client.get(result_url).status_code == 200time.sleep(1.05)- assert client.get(task["status_url"]).status_code == 404+ assert client.get(task_url(client, task)).status_code == 404assert client.get(result_url).status_code == 404@@ -1030,7 +1066,7 @@ def test_worker_upload_renews_result_lease_after_long_transfer(tmp_path: Path) -async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app), base_url="http://testserver") as transport:response = await transport.put(- task["pending_uploads"][0]["upload_url"], content=content(), headers={"x-mineru-task-lease-seconds": "120"}+ pending_files(task)[0]["upload"]["url"], content=content(), headers={"x-mineru-task-lease-seconds": "120"})assert response.status_code == 200assert record.lease_expires_at > time.time() + 110@@ -1039,7 +1075,7 @@ def test_worker_upload_renews_result_lease_after_long_transfer(tmp_path: Path) -result = _terminal(client, task)assert result["status"] == "completed" and record.retention_epoch is Noneassert client.get(result["files"][0]["result_url"]).status_code == 200- assert client.get(task["status_url"], headers={"x-mineru-task-lease-seconds": "0"}).status_code == 200+ assert client.get(task_url(client, task), headers={"x-mineru-task-lease-seconds": "0"}).status_code == 200def test_router_ends_local_result_lease_even_if_worker_rejects_release(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:@@ -1089,13 +1125,13 @@ def test_removed_lists_preserve_task_creation_query_and_cancel(service: tuple[Teassert client.get("/v1/tasks").status_code == 405assert client.get("/v1/files").status_code == 404assert "uploads" not in task and "include_model_output" not in task- assert task["tier"] == "flash"- assert set(task["pending_uploads"][0]) == {"client_file_id", "upload_url", "upload_headers", "expires_at"}- assert "/uploads/" in task["pending_uploads"][0]["upload_url"]- assert client.get(task["status_url"]).json()["pending_uploads"] == task["pending_uploads"]- canceled = client.delete(task["status_url"])+ assert {"tier", "status_url", "pending_uploads"}.isdisjoint(task)+ assert set(pending_files(task)[0]["upload"]) == {"url", "headers", "expires_at"}+ assert "/uploads/" in pending_files(task)[0]["upload"]["url"]+ assert pending_files(client.get(task_url(client, task)).json()) == pending_files(task)+ canceled = client.delete(task_url(client, task))assert canceled.status_code == 200- assert canceled.json()["status"] == "canceled" and canceled.json()["pending_uploads"] == []+ assert canceled.json()["status"] == "canceled" and pending_files(canceled.json()) == []def test_tasks_only_routes_and_health(service: tuple[TestClient, FastAPI]) -> None:@@ -1123,8 +1159,10 @@ def test_tasks_only_routes_and_health(service: tuple[TestClient, FastAPI]) -> Noassert "/v1/files" not in schema["paths"]assert {"TaskListItem", "TaskListResponse", "FileListResponse"}.isdisjoint(schema["components"]["schemas"])status_properties = schema["components"]["schemas"]["TaskStatusResponse"]["properties"]- assert "pending_uploads" in status_properties- assert {"uploads", "include_model_output"}.isdisjoint(status_properties)+ assert {"status_url", "tier", "pending_uploads", "uploads", "include_model_output"}.isdisjoint(status_properties)+ file_properties = schema["components"]["schemas"]["TaskFileStatus"]["properties"]+ assert {"sha256sum", "page_range", "file_id", "name", "upload_status"}.isdisjoint(file_properties)+ assert "upload" in schema["components"]["schemas"]["TaskFileStatus"]["required"]assert "upload_method" not in schema["components"]["schemas"]["TaskUploadLink"]["properties"]@@ -1215,5 +1253,5 @@ def test_router_lease_timeout_does_not_expire_other_worker_results(tmp_path: Patassert worker_record.retention_epoch is Noneclient.portal.call(check_renewal)- assert client.get(result["status_url"]).json()["status"] == "completed"+ assert client.get(task_url(client, result)).json()["status"] == "completed"assert client.get(result["files"][0]["result_url"]).status_code == 200diff --git a/tests/unittest/test_task_client_file_states.py b/tests/unittest/test_task_client_file_states.pynew file mode 100644index 00000000..408abefa--- /dev/null+++ b/tests/unittest/test_task_client_file_states.py@@ -0,0 +1,88 @@+"""SDK 和同步包装在等待文件但没有上传动作时只轮询,避免重复传输或提前清理。"""++from __future__ import annotations++import time+from concurrent.futures import ThreadPoolExecutor+from pathlib import Path++import httpx+import pytest+from fastapi.testclient import TestClient+from test_file_parse import Upstream, application, submit+from test_tasks_sdk import bind_transport, parse, task_snapshot, zip_output++from mineru.parser import MinerUApiParser++[email protected]("asynchronous", [False, True])[email protected]("outcome", ["success", "lost_accepted", "lost_retry"])+def test_sdk_waits_without_upload_and_preserves_retry_budget(+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, asynchronous: bool, outcome: str+) -> None:+ """上传目标延后出现、成功后延迟确认和响应丢失均不触发重复创建或无限上传。"""+ source = tmp_path / "file.pdf"+ source.write_bytes(b"pdf")+ puts = 0+ queries = 0+ creates = 0++ def handle(request: httpx.Request) -> httpx.Response:+ """按阶段提供无上传动作和待重传快照,所有结果仍由正式 SDK 消费。"""+ nonlocal puts, queries, creates+ if request.method == "POST":+ creates += 1+ return httpx.Response(202, json=task_snapshot("waiting-file"))+ if request.method == "PUT":+ puts += 1+ queries = 0+ assert request.content == b"pdf" and request.headers["content-length"] == "3"+ if puts == 1 and outcome != "success":+ raise httpx.ReadError("upload response lost", request=request)+ return httpx.Response(200)+ if request.url.host == "storage.test":+ return httpx.Response(200, content=zip_output())+ queries += 1+ if puts == 0:+ snapshot = task_snapshot("waiting-file", upload=queries >= 2)+ elif outcome == "lost_retry" and puts == 1:+ snapshot = task_snapshot("waiting-file", upload=queries >= 3)+ else:+ snapshot = task_snapshot("waiting-file" if queries < 3 else "pending" if queries == 3 else "completed")+ return httpx.Response(200, json=snapshot)++ bind_transport(monkeypatch, handle)+ result = parse(MinerUApiParser(api_url="https://api.test"), source, asynchronous)+ assert len(result.pages) == 1 and creates == 1+ assert puts == (2 if outcome == "lost_retry" else 1)+++def test_file_parse_keeps_source_until_input_is_ready(tmp_path: Path) -> None:+ """缓存命中后的服务端准备期间保留本地源文件,不把 upload=null 当成已就绪。"""+ upstream = Upstream()+ upstream.cache_hit = True+ upstream.ready_delay = 1000+ app = application(tmp_path, upstream)+ with TestClient(app) as client, ThreadPoolExecutor(max_workers=1) as executor:+ pending = executor.submit(submit, client, wait_timeout=0)+ try:+ deadline = time.monotonic() + 3+ while True:+ record = app.state.service.records.get("request_1")+ if record is not None and record.remote is not None:+ break+ assert time.monotonic() < deadline+ time.sleep(0.01)+ state = client.get("/v1/file_parse/request_1").json()+ assert state["status"] == "waiting-file"+ assert record.task.files[0].upload is None+ retained = Path(record.source_path)+ assert retained.is_file() and upstream.uploads == 0+ assert not pending.done()+ finally:+ upstream.ready_delay = 0+ response = pending.result(timeout=3)+ assert response.status_code in {200, 202}+ assert client.get("/v1/file_parse/request_1").json()["status"] == "completed"+ assert record.source_path is None and not retained.exists()+ assert upstream.uploads == 0 and len(upstream.tasks) == 1diff --git a/tests/unittest/test_task_file_states.py b/tests/unittest/test_task_file_states.pynew file mode 100644index 00000000..45601f13--- /dev/null+++ b/tests/unittest/test_task_file_states.py@@ -0,0 +1,181 @@+"""统一文件状态的准备、排队、取消和耗时边界,使用真实 API 生命周期验证。"""++from __future__ import annotations++import asyncio+import threading+import time+from collections.abc import AsyncIterator+from pathlib import Path+from types import SimpleNamespace+from typing import Any++import httpx+import pytest+from fastapi.testclient import TestClient+from task_test_utils import task_url+from test_task_api import HTML, _archive, _submit, _terminal, _upload+from test_tasks_sdk import middle++from mineru.parser import api_server+from mineru.parser.base import ParseResult+++def _install_downloads(monkeypatch: pytest.MonkeyPatch, handler: Any) -> None:+ """只隔离源站下载,保留客户端到 API 的真实协议和任务调度。"""++ def client(**kwargs: Any) -> httpx.AsyncClient:+ """为每次源下载创建独立 HTTPX 客户端,不修改全局 HTTPX 对象。"""+ return httpx.AsyncClient(transport=httpx.MockTransport(handler), **kwargs)++ monkeypatch.setattr(api_server, "httpx", SimpleNamespace(AsyncClient=client))+++def _url_task(client: TestClient, name: str = "remote") -> dict[str, Any]:+ """创建无需客户端上传的 URL 文件,初始必须处于文件准备阶段。"""+ response = client.post(+ "/v1/tasks", json={"tier": "flash", "files": [{"file_name": name + ".html", "url": "https://source.test/" + name}]}+ )+ assert response.status_code == 202, response.text+ task = response.json()+ assert task["status"] == "waiting-file"+ assert task["files"][0]["upload"] is None and "upload_status" not in task["files"][0]+ return task++[email protected]("cancel", [False, True])+def test_url_preparation_does_not_block_ready_files_or_use_upload_expiry(+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, cancel: bool+) -> None:+ """单解析名额下慢下载不阻塞已上传文件,URL 准备不受客户端上传截止时间影响。"""+ started, release, closed = threading.Event(), threading.Event(), threading.Event()++ class Source(httpx.AsyncByteStream):+ """提供可取消的源下载流,确认取消真正关闭了网络输入。"""++ async def __aiter__(self) -> AsyncIterator[bytes]:+ """等待测试放行后输出有效 HTML,取消必须中断等待。"""+ started.set()+ while not release.is_set():+ await asyncio.sleep(0.01)+ yield HTML++ async def aclose(self) -> None:+ """记录源流关闭,区别取消和正常放行。"""+ closed.set()++ _install_downloads(monkeypatch, lambda _: httpx.Response(200, stream=Source()))+ app = api_server.create_app(upload_dir=str(tmp_path), tier="flash", concurrency=1)+ try:+ with TestClient(app) as client:+ remote = _url_task(client)+ assert started.wait(3)+ app.state.task_store._records[remote["task_id"]].expires_at = 0+ snapshot = client.get(task_url(client, remote)).json()+ assert snapshot["status"] == "waiting-file" and snapshot["files"][0]["error"] is None+ assert snapshot["files"][0]["parse"] is None+ local = _submit(client, [("local.html", HTML)])+ _upload(client, local, [("local.html", HTML)])+ assert _terminal(client, local)["status"] == "completed"+ assert not release.is_set()+ if cancel:+ assert client.delete(task_url(client, remote)).json()["status"] == "canceled"+ assert closed.wait(3)+ else:+ release.set()+ completed = _terminal(client, remote)+ assert completed["status"] == "completed"+ assert completed["files"][0]["bytes"] == len(HTML)+ assert b"Preserved content" in _archive(client, completed)["markdown.md"]+ assert all(not job.prepared_sources for job in app.state.job_store._jobs.values())+ finally:+ release.set()++[email protected]("cancel_ready", [False, True])+def test_ready_url_inputs_wait_in_pending_with_bounded_preparation(+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, cancel_ready: bool+) -> None:+ """文件下载完成后排队,准备缓冲有界,排队取消会释放缓冲及准备名额。"""+ parsing, release = threading.Event(), threading.Event()+ downloads: list[str] = []+ original = api_server.parse_async++ async def infer(path: str, **kwargs: Any) -> ParseResult:+ """仅阻塞占位文件的解析,让后续 URL 输入稳定停在排队阶段。"""+ if b"blocker" in Path(path).read_bytes():+ parsing.set()+ while not release.is_set():+ await asyncio.sleep(0.01)+ return await original(path, **kwargs)++ def download(request: httpx.Request) -> httpx.Response:+ """记录实际开始的下载,尚未取得准备名额时不应访问源站。"""+ downloads.append(request.url.path)+ return httpx.Response(200, content=HTML)++ monkeypatch.setattr(api_server, "parse_async", infer)+ _install_downloads(monkeypatch, download)+ app = api_server.create_app(upload_dir=str(tmp_path), tier="flash", concurrency=1)+ try:+ with TestClient(app) as client:+ blocking = [("blocker.html", b"<html><body>blocker</body></html>")]+ first = _submit(client, blocking)+ _upload(client, first, blocking)+ assert parsing.wait(3)+ second = _url_task(client, "second")+ deadline = time.monotonic() + 3+ while client.get(task_url(client, second)).json()["status"] != "pending":+ assert time.monotonic() < deadline+ time.sleep(0.01)+ record = app.state.task_store._records[second["task_id"]]+ job = app.state.job_store._jobs[record.file_states[0].execution_id]+ assert job.prepared_sources and job.files[0].parse is None+ third = _url_task(client, "third")+ assert client.get(task_url(client, third)).json()["status"] == "waiting-file"+ assert downloads == ["/second"]+ if cancel_ready:+ assert client.delete(task_url(client, second)).json()["status"] == "canceled"+ deadline = time.monotonic() + 3+ while "/third" not in downloads:+ assert time.monotonic() < deadline+ time.sleep(0.01)+ assert not job.prepared_sources+ release.set()+ assert _terminal(client, first)["status"] == "completed"+ assert _terminal(client, second)["status"] == ("canceled" if cancel_ready else "completed")+ assert _terminal(client, third)["status"] == "completed"+ assert all(not item.prepared_sources for item in app.state.job_store._jobs.values())+ finally:+ release.set()+++def test_file_duration_includes_preparation_but_excludes_queue_wait(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ """确定性时钟区分两秒准备、千秒排队和三秒解析,最终耗时应为五秒。"""+ clock = [0.0]+ monkeypatch.setattr(api_server, "time", SimpleNamespace(time=time.time, monotonic=lambda: clock[0]))++ def download(request: httpx.Request) -> httpx.Response:+ """源下载推进两秒执行时钟,真实准备逻辑仍负责完整性检查。"""+ clock[0] += 2+ return httpx.Response(200, content=HTML)++ async def infer(path: str, **kwargs: Any) -> ParseResult:+ """解析推进三秒时钟,保留真实导出和统计发布。"""+ clock[0] += 3+ return ParseResult(middle())++ _install_downloads(monkeypatch, download)+ monkeypatch.setattr(api_server, "parse_async", infer)+ app = api_server.create_app(upload_dir=str(tmp_path), tier="flash", concurrency=1)+ with TestClient(app) as client:+ client.portal.call(app.state.job_store._semaphore.acquire)+ task = _url_task(client)+ deadline = time.monotonic() + 3+ while client.get(task_url(client, task)).json()["status"] != "pending":+ assert time.monotonic() < deadline+ time.sleep(0.01)+ clock[0] += 1000+ client.portal.call(app.state.job_store._semaphore.release)+ result = _terminal(client, task)+ assert result["files"][0]["parse"]["duration_ms"] == 5000diff --git a/tests/unittest/test_task_http.py b/tests/unittest/test_task_http.pyindex 3644c5ff..55227040 100644--- a/tests/unittest/test_task_http.py+++ b/tests/unittest/test_task_http.py@@ -2,6 +2,8 @@from __future__ import annotations+from task_test_utils import pending_files, task_url+import asyncioimport hashlibimport io@@ -149,7 +151,7 @@ def _wait(client: httpx.Client, task: dict[str, Any], timeout: float = 60) -> di"""有界轮询任务元数据,不能调用已删除的 result 路由。"""deadline = time.monotonic() + timeoutwhile time.monotonic() < deadline:- response = client.get(task["status_url"])+ response = client.get(task_url(client, task))assert response.status_code == 200, response.textpayload = response.json()if payload["status"] in TERMINAL:@@ -185,10 +187,10 @@ def test_pdf_without_client_id_keeps_metadata_and_cached_results(live_service: _response = client.post("/v1/tasks", json=payload)assert response.status_code == 202, response.texttask = response.json()- assert task["files"][0]["client_file_id"] == task["pending_uploads"][0]["client_file_id"] == "f1"- link = task["pending_uploads"][0]+ assert task["files"][0]["client_file_id"] == pending_files(task)[0]["client_file_id"] == "f1"+ link = pending_files(task)[0]for _ in range(2):- upload = client.put(link["upload_url"], content=data, headers=link["upload_headers"])+ upload = client.put(link["upload"]["url"], content=data, headers=link["upload"]["headers"])assert upload.status_code == 200, upload.textcompleted = _wait(client, task)assert completed["status"] == "completed", completed@@ -207,8 +209,8 @@ def test_pdf_without_client_id_keeps_metadata_and_cached_results(live_service: _cached = client.post("/v1/tasks", json=payload)assert cached.status_code == 202, cached.textcached_task = cached.json()- assert cached_task["pending_uploads"] == [] and cached_task["files"][0]["client_file_id"] == "f1"- assert cached_task["files"][0]["upload_status"] == "ready"+ assert pending_files(cached_task) == [] and cached_task["files"][0]["client_file_id"] == "f1"+ assert cached_task["files"][0]["status"] == "pending" and cached_task["files"][0]["upload"] is Nonecached_result = _wait(client, cached_task)assert cached_result["status"] == "completed", cached_resultcached_download = client.get(cached_result["files"][0]["result_url"])@@ -247,15 +249,16 @@ def test_router_keeps_tls_proxy_headers_on_the_public_hop_only(tmp_path: Path) -) as client:for index in range(2):task = _submit(client, [(f"document-{index}.html", HTML)])- assert urlsplit(task["status_url"]).scheme == "https"- assert len(task["pending_uploads"]) == (1 if index == 0 else 0)- for upload in task["pending_uploads"]:+ assert "status_url" not in task+ assert all(urlsplit(item["upload"]["url"]).scheme == "https" for item in pending_files(task))+ assert len(pending_files(task)) == (1 if index == 0 else 0)+ for upload in pending_files(task):# 以本地 HTTP 模拟 TLS 终止代理到 Router 的后段连接,保留真实入站代理头。- response = client.put(urlsplit(upload["upload_url"]).path, content=HTML)+ response = client.put(urlsplit(upload["upload"]["url"]).path, content=HTML)assert response.status_code == 200, response.textdeadline = time.monotonic() + 10while time.monotonic() < deadline:- response = client.get(urlsplit(task["status_url"]).path)+ response = client.get(urlsplit(task_url(client, task)).path)assert response.status_code == 200, response.textresult = response.json()if result["status"] in TERMINAL:@@ -342,18 +345,18 @@ def test_hash_preflight_sends_only_missing_file_bytes(live_service: _LiveService"""真实网络计数证明混合命中仅上传缺失文件,全命中不发送任何文件 PUT。"""with _client(live_service) as client:first = _submit(client, [("a.html", HTML)])- assert client.put(first["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200+ assert client.put(pending_files(first)[0]["upload"]["url"], content=HTML).status_code == 200assert _wait(client, first)["status"] == "completed"second_data = b"<h1>second</h1>"mixed = _submit(client, [("renamed.html", HTML), ("b.html", second_data)])- assert [link["client_file_id"] for link in mixed["pending_uploads"]] == ["f2"]- assert client.put(mixed["pending_uploads"][0]["upload_url"], content=second_data).status_code == 200+ assert [link["client_file_id"] for link in pending_files(mixed)] == ["f2"]+ assert client.put(pending_files(mixed)[0]["upload"]["url"], content=second_data).status_code == 200assert _wait(client, mixed)["status"] == "completed"base = f"/v1/tasks/{mixed['task_id']}/uploads/"assert live_service.counts.get(base + "f1", 0) == 0assert live_service.counts[base + "f2"] == len(second_data)hit = _submit(client, [("a.html", HTML), ("b.html", second_data)])- assert hit["pending_uploads"] == []+ assert pending_files(hit) == []assert _wait(client, hit)["status"] == "completed"assert not any(hit["task_id"] in path for path in live_service.counts)denied = client.post(@@ -376,8 +379,8 @@ def test_disconnect_then_resume_keeps_uploaded_files(live_service: _LiveService,second_path.write_bytes(second)with _client(live_service) as client:task = _submit(client, [(first_path.name, HTML), (second_path.name, second)])- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200- target = urlsplit(task["pending_uploads"][1]["upload_url"])+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200+ target = urlsplit(pending_files(task)[1]["upload"]["url"])connection = socket.create_connection((target.hostname, target.port), timeout=5)prefix = second[:4096]try:@@ -392,17 +395,18 @@ def test_disconnect_then_resume_keeps_uploaded_files(live_service: _LiveService,assert live_service.counts[target.path] == len(prefix)finally:connection.close()- pending = client.get(task["status_url"]).json()- assert pending["status"] in {"waiting_upload", "queued", "running"}- assert [item["upload_status"] for item in pending["files"]] == ["ready", "pending"]+ pending = client.get(task_url(client, task)).json()+ assert pending["status"] in {"waiting-file", "pending", "running"}+ assert pending["files"][0]["status"] in {"pending", "running", "completed"}+ assert pending["files"][1]["status"] == "waiting-file"result = _run_script(_script_env(tmp_path, live_service.url, TASK_ID=task["task_id"]), first_path, second_path)assert result.returncode == 0, result.stdout + result.stderr- assert live_service.counts[urlsplit(task["pending_uploads"][0]["upload_url"]).path] == len(HTML)+ assert live_service.counts[urlsplit(pending_files(task)[0]["upload"]["url"]).path] == len(HTML)assert live_service.counts[target.path] == len(prefix) + len(second)assert len(live_service.app.state.task_store._records) == 1members = _members((tmp_path / "results/f2/b.zip").read_bytes())assert b"Resumed file" in members["markdown.md"]- final = client.get(task["status_url"]).json()+ final = client.get(task_url(client, task)).json()assert final["task_id"] == task["task_id"] and final["status"] == "completed"@@ -422,12 +426,12 @@ def test_cancel_response_does_not_wait_for_native_thread_cleanup(try:with _client(live_service) as client:task = _submit(client, [("a.html", HTML)])- assert client.put(task["pending_uploads"][0]["upload_url"], content=HTML).status_code == 200+ assert client.put(pending_files(task)[0]["upload"]["url"], content=HTML).status_code == 200assert entered.wait(5)- canceled = client.delete(task["status_url"], timeout=2)+ canceled = client.delete(task_url(client, task), timeout=2)assert canceled.status_code == 200 and canceled.json()["status"] == "canceled"assert not release.is_set()- assert client.get(task["status_url"]).json()["files"][0]["result_url"] is None+ assert client.get(task_url(client, task)).json()["files"][0]["result_url"] is Nonefinally:release.set()@@ -458,13 +462,13 @@ def test_upload_over_200_mib_is_accepted(live_service: _LiveService, tmp_path: Ptask = response.json()with source.open("rb") as stream:uploaded = client.put(- task["pending_uploads"][0]["upload_url"], content=stream, headers={"Content-Length": str(size)}+ pending_files(task)[0]["upload"]["url"], content=stream, headers={"Content-Length": str(size)})assert uploaded.status_code == 200, uploaded.text- assert live_service.counts[urlsplit(task["pending_uploads"][0]["upload_url"]).path] == size+ assert live_service.counts[urlsplit(pending_files(task)[0]["upload"]["url"]).path] == sizeassert started.wait(60)- assert client.get(task["status_url"]).json()["files"][0]["upload_status"] == "ready"- assert client.delete(task["status_url"]).json()["status"] == "canceled"+ assert client.get(task_url(client, task)).json()["files"][0]["status"] in {"pending", "running", "completed"}+ assert client.delete(task_url(client, task)).json()["status"] == "canceled"@pytest.mark.parametrize("page_count", [601, 1001])@@ -478,7 +482,7 @@ def test_large_page_count_pdf_is_not_rejected(live_service: _LiveService, page_cdata = buffer.getvalue()with _client(live_service) as client:task = _submit(client, [("many-pages.pdf", data)])- assert client.put(task["pending_uploads"][0]["upload_url"], content=data).status_code == 200+ assert client.put(pending_files(task)[0]["upload"]["url"], content=data).status_code == 200result = _wait(client, task, timeout=90)assert result["status"] == "completed", resultdownload = client.get(result["files"][0]["result_url"])@@ -514,7 +518,7 @@ def test_real_documents_match_local_parser_outputs(live_service: _LiveService, dexpected = build_task_output(baseline, ["default"], Path(directory) / "outputs").read_bytes()with _client(live_service) as client:task = _submit(client, [(name, data)], page_range=page_range, output_formats=["default"])- assert client.put(task["pending_uploads"][0]["upload_url"], content=data).status_code == 200+ assert client.put(pending_files(task)[0]["upload"]["url"], content=data).status_code == 200result = _wait(client, task)assert result["status"] == "completed", resultoutput = client.get(result["files"][0]["result_url"])@@ -576,11 +580,11 @@ def test_url_task_over_http_downloads_chunked_source(live_service: _LiveService,)assert response.status_code == 202, response.texttask = response.json()- assert task["pending_uploads"] == [] and task["files"][0]["upload_status"] == "not_required"+ assert pending_files(task) == [] and task["files"][0]["status"] == "waiting-file"result = _wait(client, task)assert result["status"] == "completed", resultassert result["files"][0]["bytes"] == len(data)- assert result["files"][0]["sha256sum"] == hashlib.sha256(data).hexdigest()+ assert "sha256sum" not in result["files"][0]members = _members(client.get(result["files"][0]["result_url"]).content)assert live_service.counts == {}assert len(requests) == 1 and requests[0].method == "GET"@@ -595,7 +599,7 @@ def test_url_task_over_http_downloads_chunked_source(live_service: _LiveService,baseline_url = baseline_result["files"][0]["result_url"]else:baseline = _submit(client, [(item["file_name"], data)], page_range="2")- assert client.put(baseline["pending_uploads"][0]["upload_url"], content=data).status_code == 200+ assert client.put(pending_files(baseline)[0]["upload"]["url"], content=data).status_code == 200baseline_result = _wait(client, baseline)assert baseline_result["status"] == "completed"baseline_url = baseline_result["files"][0]["result_url"]@@ -636,7 +640,7 @@ class _CloudState:files = []for index, item in enumerate(record["files"]):client_id = item["client_file_id"]- status = "completed" if client_id in ready else "waiting_upload"+ status = "completed" if client_id in ready else "waiting-file"if client_id in expired:status = "failed"if client_id in ready and self.forced_status in {"failed", "running"}:@@ -650,9 +654,17 @@ class _CloudState:result_url = f"{self.gateway_url}/redirect/{task_id}/{client_id}"files.append({- **item,- "name": item["file_name"],- "upload_status": "ready" if client_id in ready else ("expired" if client_id in expired else "pending"),+ "client_file_id": client_id,+ "file_name": item["file_name"],+ "bytes": item["bytes"],+ "url": item.get("url"),+ "upload": None+ if client_id in ready | expired+ else {+ "url": f"{self.storage_url}/objects/{task_id}/{client_id}?signature=0",+ "headers": {"Content-Type": "application/octet-stream", "X-Upload-Token": "storage-token"},+ "expires_at": record["expires_at"],+ },"status": status,"result_url": result_url,}@@ -660,30 +672,18 @@ class _CloudState:states = [item["status"] for item in files]if "running" in states:status = "running"- elif "waiting_upload" in states:- status = "waiting_upload"+ elif "waiting-file" in states:+ status = "waiting-file"else:status = ("completed"if all(value == "completed" for value in states)else ("partial" if "completed" in states else "failed"))- uploads = [- {- "client_file_id": item["client_file_id"],- "upload_url": f"{self.storage_url}/objects/{task_id}/{item['client_file_id']}?signature=0",- "upload_headers": {"Content-Type": "application/octet-stream", "X-Upload-Token": "storage-token"},- "expires_at": record["expires_at"],- }- for item in record["files"]- if item["client_file_id"] not in ready | expired- ]payload = {"task_id": task_id,"status": status,- "status_url": f"{self.gateway_url}/v1/tasks/{task_id}","files": files,- "pending_uploads": uploads,"error": None,"output_formats": record["output_formats"],}diff --git a/tests/unittest/test_task_outputs.py b/tests/unittest/test_task_outputs.pyindex 355a9915..8ed8c1c2 100644--- a/tests/unittest/test_task_outputs.py+++ b/tests/unittest/test_task_outputs.py@@ -74,6 +74,7 @@ def _illustrated_result() -> tuple[ParseResult, bytes]:],is_full_document=True,metadata={"file_suffix": "docx", "producer": {"name": "mineru", "version": "test"}},+ extensions={"mineru": {"tier": "flash", "parse_mode": "txt"}},)return ParseResult(middle_json=middle), pngdiff --git a/tests/unittest/test_task_response_contract.py b/tests/unittest/test_task_response_contract.pynew file mode 100644index 00000000..96d710a7--- /dev/null+++ b/tests/unittest/test_task_response_contract.py@@ -0,0 +1,139 @@+"""跨部署验证逐文件响应与实际档位,阻止内部请求字段再次进入公共协议。"""++from __future__ import annotations++import io+import json+from pathlib import Path+from typing import Any++import pytest+from docx import Document+from fastapi import FastAPI+from fastapi.testclient import TestClient+from PIL import Image+from reportlab.pdfgen.canvas import Canvas+from test_task_api import _WorkerTransport, _archive, _submit, _terminal, _upload++from mineru.integrations.docvortex import build_metadata+from mineru.kit.router import RouterSettings, create_app as create_router+from mineru.parser import api_server+from mineru.parser.base import ParseResult+from mineru.types import MiddleJson, PageInfo, Tier+++def _mixed_inputs() -> list[tuple[str, bytes]]:+ """生成有效 PDF、HTML、DOCX 和 PNG,原生格式执行真实解析和导出。"""+ pdf = io.BytesIO()+ canvas = Canvas(pdf)+ canvas.drawString(60, 720, "Mixed task PDF")+ canvas.save()+ word = io.BytesIO()+ document = Document()+ document.add_paragraph("Mixed task DOCX")+ document.save(word)+ image = io.BytesIO()+ Image.new("RGB", (20, 20), "white").save(image, "PNG")+ return [+ ("input.pdf", pdf.getvalue()),+ ("input.html", b"<html><body><p>Mixed task HTML</p></body></html>"),+ ("input.docx", word.getvalue()),+ ("input.png", image.getvalue()),+ ]+++def _service(tmp_path: Path, deployment: str) -> FastAPI:+ """创建完整本地或双 worker Router 服务,复用真实上传和后台执行流程。"""+ if deployment == "api":+ return api_server.create_app(upload_dir=str(tmp_path / "api"), tier="standard")+ workers = {+ name: api_server.create_app(upload_dir=str(tmp_path / name), tier="standard") for name in ("worker-a", "worker-b")+ }+ return create_router(+ RouterSettings(+ upstream_urls=tuple(f"http://{name}" for name in workers), local_gpus="none", worker_refresh_interval_seconds=0+ ),+ transport=_WorkerTransport(workers),+ )++[email protected]("deployment", ["api", "router"])+ "requested,actual",+ [("flash", "flash"), ("basic", "basic"), ("standard", "standard"), ("advanced", "advanced"), ("advanced", "basic")],+)+def test_mixed_file_tiers_come_from_completed_results(+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, deployment: str, requested: Tier, actual: Tier+) -> None:+ """混合任务逐文件展示实际档位,最后一组刻意让推理结果与请求不同。"""+ monkeypatch.setattr(api_server, "_preflight_tier_dependencies", lambda *_args, **_kwargs: None)+ original = api_server.parse_async+ calls: list[tuple[str, Tier]] = []++ async def infer(path: str, **kwargs: Any) -> ParseResult:+ """仅替代 PDF 和图片推理,原生文档仍走真实解析,记录实际路由参数。"""+ suffix = Path(path).suffix+ calls.append((suffix, kwargs["tier"]))+ if suffix in {".html", ".docx"}:+ return await original(path, **kwargs)+ effort = {"flash": "flash", "basic": "medium", "standard": "high", "advanced": "xhigh"}[actual]+ return ParseResult(+ MiddleJson(+ pages=[PageInfo(page_idx=0)],+ is_full_document=True,+ metadata={"file_suffix": "pdf", "producer": {"name": "mineru", "version": "test"}},+ extensions=build_metadata(effort=effort, parse_mode="txt"),+ )+ )++ monkeypatch.setattr(api_server, "parse_async", infer)+ inputs = _mixed_inputs()+ with TestClient(_service(tmp_path, deployment)) as client:+ task = _submit(client, inputs, tier=requested)+ assert {"status_url", "tier", "pending_uploads"}.isdisjoint(task)+ for item in task["files"]:+ assert set(item) == {+ "client_file_id",+ "file_name",+ "bytes",+ "url",+ "upload",+ "status",+ "parse",+ "result_url",+ "error",+ }+ assert item["parse"] is None+ assert set(item["upload"]) == {"url", "headers", "expires_at"}+ _upload(client, task, inputs)+ completed = _terminal(client, task)+ assert completed["status"] == "completed", completed+ assert [item["parse"]["tier"] for item in completed["files"]] == [actual, "flash", "flash", actual]+ assert dict(calls) == {".pdf": requested, ".html": "flash", ".docx": "flash", ".png": requested}+ for index, item in enumerate(completed["files"]):+ assert item["upload"] is None+ assert set(item["parse"]) == {"tier", "duration_ms", "parser_version"}+ assert item["parse"]["duration_ms"] >= 0+ middle = json.loads(_archive(client, completed, index)["middle_json.json"])+ assert middle["extensions"]["mineru"]["tier"] == item["parse"]["tier"]+ assert b"Mixed task HTML" in _archive(client, completed, 1)["markdown.md"]+ assert b"Mixed task DOCX" in _archive(client, completed, 2)["markdown.md"]+++def test_missing_actual_metadata_does_not_fabricate_requested_tier(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:+ """结果缺少真实档位时解析失败,不能以请求默认值伪造完成统计。"""++ async def infer(path: str, **kwargs: Any) -> ParseResult:+ """返回缺少产品扩展的结果,覆盖 worker 生产者违反契约的边界。"""+ return ParseResult(+ MiddleJson(pages=[], metadata={"file_suffix": "html", "producer": {"name": "test", "version": "test"}})+ )++ monkeypatch.setattr(api_server, "parse_async", infer)+ with TestClient(api_server.create_app(upload_dir=str(tmp_path), tier="flash")) as client:+ inputs = [("input.html", b"<p>missing metadata</p>")]+ task = _submit(client, inputs)+ _upload(client, task, inputs)+ result = _terminal(client, task)+ assert result["status"] == "failed"+ assert result["files"][0]["parse"] is None and result["files"][0]["result_url"] is Nonediff --git a/tests/unittest/test_tasks_sdk.py b/tests/unittest/test_tasks_sdk.pyindex 0fde29a5..25726793 100644--- a/tests/unittest/test_tasks_sdk.py+++ b/tests/unittest/test_tasks_sdk.py@@ -43,33 +43,28 @@ def zip_output() -> bytes:def task_snapshot(status: str = "completed", *, upload: bool = False) -> dict[str, Any]:- """提供可独立调整状态和上传准备情况的单文件任务快照。"""+ """提供逐文件内嵌上传描述,不伪造已移除的查询链接或顶层档位。"""return {"task_id": "task-one",- "status_url": "https://api.test/v1/tasks/task-one","status": status,"output_formats": ["default"],"files": [{"client_file_id": "file",+ "file_name": "file.pdf",+ "bytes": 3,"status": status,- "upload_status": "pending" if upload else "ready",- "parse": {"duration_ms": 250},+ "upload": {+ "url": "https://storage.test/input",+ "headers": {},+ "expires_at": 9999999999,+ }+ if upload+ else None,+ "parse": {"tier": "flash", "duration_ms": 250, "parser_version": "test"} if status == "completed" else None,"result_url": "https://storage.test/result?signature=original" if status == "completed" else None,}],- "pending_uploads": (- [- {- "client_file_id": "file",- "upload_url": "https://storage.test/input",- "upload_headers": {},- "expires_at": 9999999999,- }- ]- if upload- else []- ),}@@ -88,6 +83,50 @@ def parse(parser: MinerUApiParser, source: Path, asynchronous: bool, **kwargs: Areturn asyncio.run(parser.parse_async(source, **kwargs)) if asynchronous else parser.parse(source, **kwargs)[email protected]("asynchronous", [False, True])[email protected]("prefix", ["", "/gateway/api"])+def test_task_query_uses_configured_prefix_without_status_url(+ tmp_path: Path, monkeypatch: pytest.MonkeyPatch, asynchronous: bool, prefix: str+) -> None:+ """响应不含查询链接时仍能轮询,代理前缀和编码后的任务 ID 均不丢失。"""+ source = tmp_path / "file.pdf"+ source.write_bytes(b"pdf")+ queries: list[str] = []++ def handle(request: httpx.Request) -> httpx.Response:+ """区分固定 API 路径和跨源结果下载,禁止向对象存储泄漏凭据。"""+ if request.url.host == "storage.test":+ assert request.headers.get("authorization") is None+ return httpx.Response(200, content=zip_output())+ assert request.headers.get("authorization") == "Bearer secret"+ if request.method == "POST":+ assert request.url.path == prefix + "/v1/tasks"+ snapshot = task_snapshot("pending")+ else:+ queries.append(request.url.raw_path.decode())+ assert request.url.raw_path.decode() == prefix + "/v1/tasks/task%20one"+ snapshot = task_snapshot()+ snapshot["task_id"] = "task one"+ assert "status_url" not in snapshot+ return httpx.Response(200, json=snapshot)++ bind_transport(monkeypatch, handle)+ result = parse(MinerUApiParser(api_url="https://api.test" + prefix, api_key="secret"), source, asynchronous)+ assert len(result.pages) == 1 and len(queries) == 1++[email protected]("asynchronous", [False, True])+def test_tasks_sdk_rejects_missing_nested_upload(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, asynchronous: bool) -> None:+ """缺少新 upload 字段的旧响应不能被当作缓存命中或触发旧字段回退。"""+ source = tmp_path / "file.pdf"+ source.write_bytes(b"pdf")+ snapshot = task_snapshot("waiting-file", upload=True)+ del snapshot["files"][0]["upload"]+ bind_transport(monkeypatch, lambda _: httpx.Response(202, json=snapshot))+ with pytest.raises(api_client._V1APIError, match="upload descriptor"):+ parse(MinerUApiParser(api_url="https://api.test"), source, asynchronous)++@pytest.mark.parametrize("asynchronous", [False, True])@pytest.mark.parametrize("cached", [False, True])@pytest.mark.parametrize("lost_upload_response", [False, True])@@ -98,7 +137,7 @@ def test_tasks_sdk_upload_cache_and_lost_response(source = tmp_path / "report.pdf"source.write_bytes(b"%PDF-example")calls: list[httpx.Request] = []- state = task_snapshot("completed" if cached else "waiting_upload", upload=not cached)+ state = task_snapshot("completed" if cached else "waiting-file", upload=not cached)def handle(request: httpx.Request) -> httpx.Response:"""记录线上协议,不接受任何独立上传或 Jobs 路由。"""@@ -138,7 +177,7 @@ def test_tasks_sdk_upload_cache_and_lost_response(assert len(result.pages) == 1 and durations == [250.0]assert sum(r.method == "POST" for r in calls) == 1assert sum(r.method == "PUT" for r in calls) == (0 if cached else 1)- assert statuses == (["completed"] if cached else ["waiting_upload", "completed"])+ assert statuses == (["completed"] if cached else ["waiting-file", "completed"])@pytest.mark.parametrize("asynchronous", [False, True])@@ -152,7 +191,7 @@ def test_tasks_sdk_transport_recovery(source = tmp_path / "file.pdf"source.write_bytes(b"pdf")counts = {"POST": 0, "PUT": 0, "query": 0, "download": 0}- state = task_snapshot("waiting_upload" if failure == "upload" else "queued", upload=failure == "upload")+ state = task_snapshot("waiting-file" if failure == "upload" else "pending", upload=failure == "upload")def handle(request: httpx.Request) -> httpx.Response:"""在指定阶段只注入一次故障,观测是否重复创建或泄露认证。"""@@ -204,9 +243,8 @@ def test_tasks_sdk_transport_recovery("field,value",[("task_id", "other"),- ("status_url", "https://evil.test/metadata"),+ ("task_id", ".."),("output_formats", ["html"]),- ("pending_uploads", None),],)def test_tasks_sdk_rejects_changed_identity(@@ -218,7 +256,7 @@ def test_tasks_sdk_rejects_changed_identity(def handle(request: httpx.Request) -> httpx.Response:"""提交合法,后续查询注入字段变化。"""- response = task_snapshot("queued") if request.method == "POST" else {**task_snapshot(), field: value}+ response = task_snapshot("pending") if request.method == "POST" else {**task_snapshot(), field: value}return httpx.Response(200, json=response)bind_transport(monkeypatch, handle)@@ -282,7 +320,7 @@ def test_tasks_sdk_retries_are_bounded(def handle(request: httpx.Request) -> httpx.Response:"""按阶段产生持续故障,创建请求仍只提交一次。"""if request.method == "POST":- return httpx.Response(202, json=task_snapshot("queued" if failure == "query" else "completed"))+ return httpx.Response(202, json=task_snapshot("pending" if failure == "query" else "completed"))if request.url.host == "api.test":if failure == "query":attempts.append(request)@@ -302,9 +340,7 @@ def test_tasks_sdk_retries_are_bounded(@pytest.mark.parametrize("asynchronous", [False, True])- "field,value", [("task_id", None), ("status", {}), ("files", []), ("status_url", "file:///tmp/result")]-)[email protected]("field,value", [("task_id", None), ("status", {}), ("files", []), ("task_id", ".")])def test_tasks_sdk_rejects_malformed_creation(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, asynchronous: bool, field: str, value: Any) -> None:@@ -360,7 +396,7 @@ def test_tasks_sdk_retries_transient_poll_responses("""首次创建成功,按预算注入轮询故障,之后返回可读取的完整结果。"""if request.method == "POST":counts["post"] += 1- return httpx.Response(202, json=task_snapshot("queued"))+ return httpx.Response(202, json=task_snapshot("pending"))if request.url.host == "api.test":counts["query"] += 1if counts["query"] <= failures: