Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

fix(media): 修复尾部索引 MP4 的播放头与旧失败缓存

simon15614e950fb2f23d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 07889f5b1..1c42dbaee 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,7 @@ Removed / Fixed / Security。
### 修复
+- **播放**:缺少显示时间戳表的 MP4 可在索引位于文件末尾时重建播放头;旧版失败缓存会重新检查这类布局,无法判断编辑起点的文件继续转码。
- **配置**:同源检查统一识别 HTTP、HTTPS 的默认端口,避免地址显式包含 `:80` 或 `:443` 时被误拒绝。
- **馆藏**:处理失败后的线程清理完成前保留停止中状态,避免立即重试撞上仍持有的处理锁;起步失败时显示本次错误,不再退回上一轮结果。
- **采集**:云下载能力未取得时,在本机诊断记录保留令牌、挂载点或目录探测阶段及失败分类,便于区分配置与网络问题。
diff --git a/docs/REUSE.md b/docs/REUSE.md
index 80ed67580..0109dcca5 100644
--- a/docs/REUSE.md
+++ b/docs/REUSE.md
@@ -815,8 +815,9 @@ A 盘垃圾隔离使用短名称 `peach-purge-<32 位 UUID>.peach-quarantine`,
- 远程媒体持久缓存复用 HTTPX 0.28.1 的流式响应、Starlette 1.7.0 的文件 Range、filelock 4.0.12 和 `fsutil.atomic_path`;缓存身份、失败保留与按需取得由 Peach 管理。HLS 清单解析用 [m3u8 6.0.0](https://pypi.org/project/m3u8/)(MIT,Python ≥3.7,纯 Python、24.1 KB wheel、无传递依赖),已读取固定版 `model.py` 并验证嵌套清单、密钥、初始化段和字节范围。Hishel 1.4.0(BSD-3-Clause)已读取 `_sync_cache.py` 与 Range 判据,通用 HTTP 缓存按来源 URL、缓存头和重新验证工作;此处需按稳定条目身份避开签名 URL 更新、永久保留成功媒体,因此复用现有持久文件路径。规则及验收见 [媒体缓存](MEDIA_CACHE.md)。
- 账本路径在 Windows 只做 `abspath`,不 `resolve()`:PikPak 的 A: 是 WinFsp 网络驱动器,`resolve()` 得到 UNC 形态,之后每个经 `file_for()` 的播放请求都慢几个数量级;115 的 B: 不受影响。
- 不兼容片源(HEVC、mp3 以外的音轨、非 MP4 容器)按 6 秒片重编码给 HLS,账本没记时长或为负时用 ffprobe 的时长切,探测也拿不到才回 Range。分片与整片共享硬件后端探测及回退链,见[硬件转码后端](#硬件转码后端),同一分片只起一个 FFmpeg。Range 响应按 1 MiB 读文件,`BufferedFileResponse` 自己盯 `http.disconnect`(uvicorn 断开后 `send()` 静默返回,Starlette `FileResponse` 不监听),否则拖一次进度条就留下一个把整部片拉完的幽灵读者。
-- 有 B 帧却没有 `ctts` 的 MP4 是时间戳错乱,浏览器丢掉倒着走的帧。这类片源不重编码:`mp4repair.py` 重建一份 `moov`(游程编码的 `ctts`、编辑列表补整体平移、`stco`/`co64` 按头长差平移)存成 `transcode_root` 里的 `.mp4hdr` 边车,`/stream` 用「边车的头 + 原文件那段 mdat」按 Range 发。显示顺序取自 `ffprobe -ignore_editlist 1 -show_entries frame=pts`,`pkt_dts` 不能用。边车算出来前走 HLS 转码,同一部片后台只算一次,算不出就跳过。
- 编辑列表的半帧舍入判据采用 peach-patcher 的 [PR #175](https://github.com/peach-mitao/peach/pull/175)(`4106d30cc740863cd1d44a207f80c5f75291141f`),按最短正帧间距判断已有补偿,保留真实裁切、空编辑和段速率。新版边车标识为 `PEACHMP4` 第 2 版;第 1 版在读取时按偏移游程的加权均值还原整体平移,仅修正重复补偿,不读原片、不展开整片时间轴、不重写边车或媒体。已有无效边车可在所选媒体库的修复任务中重建,其旧失败结论不再遮蔽这项修复;正常边车、正常扫描结论和无产物的既有解码失败继续复用。临时旧边车、批处理和 Range 拼接均有回归;未新增依赖。
+- 有 B 帧却没有 `ctts` 的 MP4 是时间戳错乱,浏览器丢掉倒着走的帧。这类片源不重编码:`mp4repair.py` 重建一份 `moov`(游程编码的 `ctts`、编辑列表补整体平移、`stco`/`co64` 按头长差平移)存成 `transcode_root` 里的 `.mp4hdr` 边车,`/stream` 用「边车的头 + 原文件那段 mdat」按 Range 发。`moov` 位于媒体前后都支持;位于末尾时只读索引尾段,把它放到媒体前,媒体后的其他盒子原样保留。显示顺序取自 `ffprobe -ignore_editlist 1 -show_entries frame=pts`,`pkt_dts` 不能用。边车算出来前走 HLS 转码,同一部片后台只算一次,算不出就跳过。
+- 尾部索引布局采用 simon1561 的 [PR #157](https://github.com/peach-mitao/peach/pull/157)(`e67ead5a35a63fc36aa8153e5d62998a001779db`),补充扩展盒头、单个媒体盒和索引读取边界。非零编辑起点超过重建补偿范围时无法区分预卷与裁剪,保留转码;临时 B 帧视频已验证适用样本 50 帧逐帧一致、歧义样本拒修。扫描缓存第 1 版的失败结论按尾部布局重新诊断,第 2 版保留当前失败结论,不清空正常缓存或自动运行媒体修复。
- 「只采集在线资料」对齐全的行(番号已落库、字段有着落或已在候选表、封面在位)不碰磁盘;文件在不在看目录列表、同目录只列一次,扫描用目录列表自带的大小与时间、不逐个 stat;候选 CSV 每 5 秒写盘一次,被打断也在收尾写全。
- 一个控件在两页出现时,选中态也归它:首页、资料页与关注页的「全部/没看过/稍后看/已标记」共用 `use-view-glide.ts` 那块滑动玻璃与 `[data-view-glide]`,填充只由玻璃给,各排自己不铺底。
- 本地详情与关注详情的「接着看」共用 `RelatedSection`、横滚接线与卡片 `next` 版式,数量读同一设置。空结果复用 `EmptyState`,读取失败复用 `RetryNote`,可原位重试;关闭推荐时不取数。关注推荐复用 `/api/follow` 分页及 TanStack Query 缓存,最多读取三页;优先同创作者和共同标签,排除当前合集、已看、忽略与不同媒体类型。创作者头像链接复用卡片的 hover,下载链接复用舞台操作键几何;未新增依赖,推荐读取不写 ledger。
diff --git a/src/peach/mp4repair.py b/src/peach/mp4repair.py
index 0f21f3f74..cc94d6ff4 100644
--- a/src/peach/mp4repair.py
+++ b/src/peach/mp4repair.py
@@ -297,11 +297,22 @@ def _shift_chunk_offsets(box: _Box, delta: int) -> None:
_Span = tuple[bytes, int, int, int]
-def _layout(source: Path) -> tuple[bytes, _Span, _Span]:
- """读出 `mdat` 之前那一整段,并确认这个布局拼得回去。
+@dataclass(frozen=True)
+class _Layout:
+ before: bytes
+ mdat_header: bytes
+ suffix: bytes
+ mdat: _Span
+ moov_box: bytes
+ moov_body: int
+ moov_before_start: int | None
+
+
+def _layout(source: Path) -> _Layout:
+ """读出 `mdat` 和 `moov`,返回拼回虚拟文件所需的几段。
- 拼接只换头、原样接上 `mdat`,所以 `mdat` 必须是最后一个盒子,前面的东西也得
- 一次读得完。不满足就抛 `RepairUnavailable`,让调用方回到转码那条路。
+ faststart 文件把 `moov` 放在媒体前面,尾部只允许没有盒子;下载器常见的布局则把
+ `moov` 放在末尾。后一种只多读媒体之后的索引段,媒体仍按原字节流发送,不复制原片。
"""
from .smb_media import SmbResource
with (source.open() if isinstance(source, SmbResource) else open(source, 'rb')) as handle:
@@ -309,22 +320,44 @@ def _layout(source: Path) -> tuple[bytes, _Span, _Span]:
kinds = {kind for kind, *_ in boxes}
if not {b"moov", b"mdat"} <= kinds:
raise RepairUnavailable("顶层缺 moov 或 mdat")
+ if any(sum(box[0] == kind for box in boxes) != 1 for kind in (b"moov", b"mdat")):
+ raise RepairUnavailable("顶层必须各有一个 moov 和 mdat")
mdat = next(box for box in boxes if box[0] == b"mdat")
moov = next(box for box in boxes if box[0] == b"moov")
- if moov[1] > mdat[1]:
- raise RepairUnavailable("moov 在 mdat 之后,这个布局还没支持")
- if any(box[1] > mdat[1] for box in boxes):
- raise RepairUnavailable("mdat 后面还有盒子,这个布局还没支持")
if mdat[3] - mdat[2] <= 0 or moov[3] - moov[1] > HEADER_LIMIT:
raise RepairUnavailable("moov 太大或 mdat 是空的")
+ if mdat[2] > HEADER_LIMIT:
+ raise RepairUnavailable("媒体前的索引段太大")
+ if moov[1] < mdat[1]:
+ if any(box[1] > mdat[1] for box in boxes):
+ raise RepairUnavailable("mdat 后面还有盒子,这个布局还没支持")
+ handle.seek(moov[1])
+ moov_box = handle.read(moov[3] - moov[1])
+ handle.seek(0)
+ before = handle.read(mdat[1])
+ handle.seek(mdat[1])
+ mdat_header = handle.read(mdat[2] - mdat[1])
+ return _Layout(before, mdat_header, b"", mdat, moov_box, moov[2] - moov[1], moov[1])
+ after_size = handle.seek(0, 2) - mdat[3]
+ if after_size > HEADER_LIMIT - mdat[2]:
+ raise RepairUnavailable("媒体后的索引段太大")
handle.seek(0)
- return handle.read(mdat[2]), moov, mdat
+ before = handle.read(mdat[1])
+ handle.seek(mdat[1])
+ mdat_header = handle.read(mdat[2] - mdat[1])
+ handle.seek(mdat[3])
+ after = handle.read(after_size)
+ start = moov[1] - mdat[3]
+ stop = moov[3] - mdat[3]
+ return _Layout(before, mdat_header, after[:start] + after[stop:], mdat,
+ after[start:stop], moov[2] - moov[1], None)
def repaired_header(source: Path, ffprobe: Path) -> RepairedHeader:
"""算出修好的头。修不了就抛 `RepairUnavailable`。"""
- head, moov, mdat = _layout(source)
- tree = _parse(head, moov[2], moov[3])
+ layout = _layout(source)
+ mdat = layout.mdat
+ tree = _parse(layout.moov_box, layout.moov_body, len(layout.moov_box))
root = _Box(b"moov", None, tree)
traks = [child for child in root.children if child.kind == b"trak"]
video = next((trak for trak in traks if _is_video(trak)), None)
@@ -346,6 +379,13 @@ def repaired_header(source: Path, ffprobe: Path) -> RepairedHeader:
stbl.children.insert(stbl.children.index(stts) + 1, _Box(b"ctts", _ctts(offsets), []))
edts = video.find(b"edts")
elst = edts.find(b"elst") if edts else None
+ if layout.moov_before_start is None and elst and elst.payload:
+ def checked_start(value: int) -> int:
+ if value > shift + _half_frame(decode_times):
+ raise RepairUnavailable("尾部索引的编辑起点无法区分预卷与裁剪")
+ return value
+
+ _edit_times(elst.payload, checked_start)
payload = _retimed_edit_list(
elst.payload if elst and elst.payload else None, shift, _track_duration(root, video),
_half_frame(decode_times))
@@ -356,14 +396,31 @@ def repaired_header(source: Path, ffprobe: Path) -> RepairedHeader:
else:
video.children.insert(1, _Box(b"edts", None, [_Box(b"elst", payload, [])]))
+ def assemble(rebuilt: bytes) -> bytes:
+ if layout.moov_before_start is not None:
+ stop = layout.moov_before_start + len(layout.moov_box)
+ return (layout.before[:layout.moov_before_start] + rebuilt
+ + layout.before[stop:] + layout.mdat_header)
+ return layout.before + rebuilt + layout.mdat_header
+
rebuilt = root.serialize()
- delta = len(rebuilt) - (moov[3] - moov[1])
+ delta = len(assemble(rebuilt)) - mdat[2]
_shift_chunk_offsets(root, delta)
rebuilt = root.serialize()
- prefix = head[:moov[1]] + rebuilt + head[moov[3]:]
+ prefix = assemble(rebuilt)
if len(prefix) - mdat[2] != delta:
raise RepairUnavailable("重建后的头长度对不上")
- return RepairedHeader(prefix=prefix, payload_start=mdat[2], payload_end=mdat[3])
+ return RepairedHeader(prefix=prefix, payload_start=mdat[2], payload_end=mdat[3], suffix=layout.suffix)
+
+
+def has_tail_moov(source: Path) -> bool:
+ """只读顶层盒子位置,识别旧布局判据会跳过的尾部索引。"""
+ from .smb_media import SmbResource
+ with (source.open() if isinstance(source, SmbResource) else open(source, 'rb')) as handle:
+ boxes = _top_level(handle)
+ moovs = [box for box in boxes if box[0] == b'moov']
+ mdats = [box for box in boxes if box[0] == b'mdat']
+ return len(moovs) == len(mdats) == 1 and moovs[0][1] > mdats[0][1]
def write_sidecar(path: Path, header: RepairedHeader) -> None:
diff --git a/src/peach/web_media_repair.py b/src/peach/web_media_repair.py
index e275e5507..ed218b890 100644
--- a/src/peach/web_media_repair.py
+++ b/src/peach/web_media_repair.py
@@ -24,7 +24,7 @@ import subprocess
import time
from pathlib import Path, PureWindowsPath
-from . import media_libraries, media_probe, mp4recover, settings_file
+from . import media_libraries, media_probe, mp4recover, mp4repair, settings_file
from .mp4repair import RepairUnavailable
from .platform import translate_ledger_path
@@ -34,6 +34,7 @@ SCAN_CACHE_NAME = "mp4-repair-scan.json"
#: 文件身份对应的判定缓存;修复产物由边车或新文件核验。
CLEAN = "clean"
UNAVAILABLE = "unavailable"
+LEGACY_UNAVAILABLE = "unavailable-layout-v1"
#: 缺时间戳表,修的是边车。
HEADER = "header"
@@ -53,7 +54,10 @@ def _load_cache(contract) -> dict[str, str]:
except (OSError, ValueError):
return {}
entries = payload.get("entries") if isinstance(payload, dict) else None
- return {str(key): str(value) for key, value in entries.items()} if isinstance(entries, dict) else {}
+ if not isinstance(entries, dict):
+ return {}
+ return {str(key): LEGACY_UNAVAILABLE if value == UNAVAILABLE and payload.get('version', 1) == 1
+ else str(value) for key, value in entries.items()}
def _save_cache(contract, entries: dict[str, str]) -> None:
@@ -61,7 +65,7 @@ def _save_cache(contract, entries: dict[str, str]) -> None:
try:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + ".tmp")
- temporary.write_text(json.dumps({"version": 1, "entries": entries}), encoding="utf-8")
+ temporary.write_text(json.dumps({"version": 2, "entries": entries}), encoding="utf-8")
temporary.replace(path)
except OSError:
# 结论缓存丢了只是下次重扫一遍,不该让整轮任务失败。
@@ -117,6 +121,15 @@ def _survey(contract, job, job_id: str, rows, cache: dict[str, str]) -> list[tup
_advance(job, job_id, index, skipped=True)
continue
kind = cache.get(key)
+ if kind == LEGACY_UNAVAILABLE:
+ try:
+ kind = None if mp4repair.has_tail_moov(source) else UNAVAILABLE
+ except OSError:
+ _advance(job, job_id, index, skipped=True)
+ continue
+ if kind == UNAVAILABLE:
+ cache[key] = kind
+ _save_cache(contract, cache)
if kind == UNAVAILABLE and store.has_invalid_sidecar(asset_id, source):
kind = HEADER
if kind in (CLEAN, UNAVAILABLE):
diff --git a/tests/test_mp4repair.py b/tests/test_mp4repair.py
index b6ddcb963..bd57265ec 100644
--- a/tests/test_mp4repair.py
+++ b/tests/test_mp4repair.py
@@ -81,6 +81,20 @@ def legacy_sidecar(path: Path, rows: list[tuple[int, int, int]], *, version=0) -
return header
+def tail_mp4(**options) -> bytes:
+ """把正常布局里的 moov 搬到媒体后,并把块偏移改到新的媒体起点。"""
+ blob = minimal_mp4(timescale=1000, sample_delta=400, samples=len(DECODE_TO_DISPLAY),
+ keyframe_every=6, **options)
+ moov_span = next((body - 8, stop) for kind, body, stop in _iter_boxes(blob, 0, len(blob))
+ if kind == b"moov")
+ start, stop = moov_span
+ moov = blob[start:stop]
+ tree = mp4repair._parse(moov, 8, len(moov))
+ root = mp4repair._Box(b"moov", None, tree)
+ mp4repair._shift_chunk_offsets(root, -len(moov))
+ return blob[:start] + blob[stop:] + root.serialize()
+
+
class Mp4HeaderRepairTests(unittest.TestCase):
def setUp(self) -> None:
self._temporary = TemporaryDirectory()
@@ -132,6 +146,87 @@ class Mp4HeaderRepairTests(unittest.TestCase):
self.assertEqual(after[-1][0],
header.size - SAMPLE_SIZE, "最后一个块没落在文件末尾那一段上")
+ def test_a_tail_moov_is_moved_ahead_of_the_media_without_rewriting_samples(self):
+ source = self.root / "tail.mp4"
+ source.write_bytes(tail_mp4())
+ header = self.repair(source)
+ blob = source.read_bytes()
+
+ self.assertEqual(blob[header.payload_start:header.payload_end],
+ sample_bytes(len(DECODE_TO_DISPLAY)))
+ self.assertEqual(header.suffix, b"")
+ moov = moov_of(header.prefix)
+ self.assertEqual(next(_iter_boxes(moov, 0, len(moov)))[0], b"mvhd")
+
+ def test_tail_layout_chunk_offsets_point_into_the_spliced_media(self):
+ source = self.root / "tail.mp4"
+ source.write_bytes(tail_mp4())
+ header = self.repair(source)
+ after = entries(table(moov_of(header.prefix), b"stco"), ">I")
+
+ self.assertEqual(after[0][0], len(header.prefix))
+ self.assertEqual(after[-1][0], header.size - SAMPLE_SIZE)
+
+ def test_other_tail_boxes_stay_after_the_media_when_moov_moves(self):
+ source = self.root / "tail.mp4"
+ media = tail_mp4()
+ moov_start = media.rfind(b"moov") - 4
+ source.write_bytes(media[:moov_start] + b"\x00\x00\x00\x10freeABCDEFGH" + media[moov_start:])
+ header = self.repair(source)
+
+ self.assertEqual(header.suffix, b"\x00\x00\x00\x10freeABCDEFGH")
+
+ def test_tail_layout_moves_64_bit_chunk_offsets_with_the_media(self):
+ blob = tail_mp4()
+ start = blob.rfind(b'moov') - 4
+ root = mp4repair._Box(b'moov', None, mp4repair._parse(blob[start:], 8, len(blob) - start))
+ video = next(child for child in root.children if child.kind == b'trak')
+ stco = mp4repair._sample_table(video).find(b'stco')
+ offsets = entries(stco.payload, '>I')
+ stco.kind = b'co64'
+ stco.payload = stco.payload[:8] + b''.join(struct.pack('>Q', value) for value, in offsets)
+ source = self.root / 'co64.mp4'
+ source.write_bytes(blob[:start] + root.serialize())
+
+ header = self.repair(source)
+
+ self.assertEqual(entries(table(moov_of(header.prefix), b'co64'), '>Q'),
+ [(len(header.prefix) + index * SAMPLE_SIZE,)
+ for index in range(len(DECODE_TO_DISPLAY))])
+
+ def test_extended_size_tail_moov_uses_its_actual_payload_start(self):
+ blob = tail_mp4()
+ start = blob.rfind(b'moov') - 4
+ moov = blob[start:]
+ source = self.root / 'extended.mp4'
+ source.write_bytes(blob[:start] + struct.pack('>I4sQ', 1, b'moov', len(moov) + 8) + moov[8:])
+ header = self.repair(source)
+ self.assertEqual(entries(table(moov_of(header.prefix), b'stco'), '>I')[0][0], len(header.prefix))
+ normal = self.broken_mp4(self.root / 'normal.mp4')
+ self.assertEqual(composition_times(moov_of(header.prefix), 400),
+ composition_times(moov_of(self.repair(normal).prefix), 400))
+
+ def test_tail_layout_requires_one_media_box_and_bounded_metadata(self):
+ source = self.root / 'tail.mp4'
+ source.write_bytes(tail_mp4() + b'\x00\x00\x00\x0cmdatDATA')
+ with self.assertRaisesRegex(RepairUnavailable, '各有一个'):
+ self.repair(source)
+ source.write_bytes(tail_mp4())
+ with patch.object(mp4repair, 'HEADER_LIMIT', 24):
+ with self.assertRaises(RepairUnavailable):
+ self.repair(source)
+
+ def test_ambiguous_tail_edit_start_keeps_the_transcode_fallback(self):
+ source = self.root / 'tail.mp4'
+ source.write_bytes(tail_mp4(edit_list=(5200, 800)))
+ with self.assertRaisesRegex(RepairUnavailable, '预卷与裁剪'):
+ self.repair(source)
+ for start in (0, 400):
+ with self.subTest(start=start):
+ source.write_bytes(tail_mp4(edit_list=(5200, start)))
+ self.assertEqual(entries(edit_list_of(self.repair(source)), '>IiI'),
+ [(5200, 400, 1 << 16)])
+
def test_an_edit_list_that_already_compensates_the_shift_is_left_alone(self):
"""封装器为 ctts 补过的那一刀要原样留着,再补一次会真的切掉开头那两帧。"""
source = self.broken_mp4(self.root / "broken.mp4", edit_list=(5200, 400))
diff --git a/tests/test_streaming.py b/tests/test_streaming.py
index 15bb7f115..a42520d73 100644
--- a/tests/test_streaming.py
+++ b/tests/test_streaming.py
@@ -15,7 +15,7 @@ from peach.streaming import CancellableFileResponse, StreamSessionRegistry
from peach.mp4index import segment_plan
from peach.routes_media import hls_segment
from peach.segments import HlsSegmentService, build_hls_playlist
-from test_mp4repair import edit_list_of, entries, legacy_sidecar
+from test_mp4repair import DECODE_TO_DISPLAY, edit_list_of, entries, legacy_sidecar, tail_mp4
async def _collect(response, requested: bytes | None, method: str = "GET"):
@@ -329,6 +329,28 @@ class StreamSessionRegistryTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(status_code, 416)
self.assertEqual(headers["content-range"], f"bytes */{len(whole)}")
+ async def test_tail_moov_repair_serves_both_seams_and_the_saved_suffix(self):
+ from peach import mp4repair
+ from peach.streaming import SplicedMp4Response
+
+ with tempfile.TemporaryDirectory() as tmp:
+ path = Path(tmp) / 'tail.mp4'
+ path.write_bytes(tail_mp4() + b'\x00\x00\x00\x10freeABCDEFGH')
+ original = path.read_bytes()
+ with patch.object(mp4repair, '_display_order', return_value=list(DECODE_TO_DISPLAY)):
+ header = mp4repair.repaired_header(path, Path('ffprobe'))
+ sidecar = Path(tmp) / 'repair.mp4hdr'
+ mp4repair.write_sidecar(sidecar, header)
+ header = mp4repair.read_sidecar(sidecar)
+ whole = header.prefix + original[header.payload_start:header.payload_end] + header.suffix
+ for seam in (len(header.prefix), len(whole) - len(header.suffix)):
+ status, headers, body = await _collect(
+ SplicedMp4Response(path, header), f'bytes={seam - 4}-{seam + 4}'.encode())
+ self.assertEqual(status, 206)
+ self.assertEqual(body, whole[seam - 4:seam + 5])
+ self.assertEqual(headers['content-length'], '9')
+ self.assertEqual(path.read_bytes(), original)
+
async def test_a_spliced_head_request_reports_the_size_without_a_body(self):
import tempfile
from pathlib import Path
diff --git a/tests/test_web_media_repair.py b/tests/test_web_media_repair.py
index e2ea8be59..7d4e35e50 100644
--- a/tests/test_web_media_repair.py
+++ b/tests/test_web_media_repair.py
@@ -4,6 +4,7 @@
修好之后文件和账本落成什么样。重建头的算术在 `test_mp4repair` 里,重建索引的判据在
`test_mp4recover` 里。
"""
+import json
import sqlite3
import tempfile
import unittest
@@ -18,6 +19,7 @@ from peach.mp4repair import RepairUnavailable
from peach import mp4repair
from support.ledger import fresh_ledger
+from test_mp4repair import tail_mp4
SEP = chr(92)
#: 顶层只有 `ftyp` 和写到文件末尾的 `mdat`:下载没写完、缺整个索引的那一类。
@@ -264,6 +266,40 @@ class MediaRepairRunTests(unittest.TestCase):
saved = rm_repair._load_cache(self.contract)
self.assertEqual(saved[rm_repair._key(3, self.media / 'fine.mp4')], rm_repair.CLEAN)
+ def test_old_failed_tail_layout_is_rechecked_without_rescanning_other_files(self):
+ (self.media / 'favourite.mp4').write_bytes(tail_mp4())
+ entries = {rm_repair._key(asset_id, self.media / name): verdict
+ for asset_id, name, verdict in (
+ (1, 'seldom.mp4', rm_repair.UNAVAILABLE),
+ (2, 'favourite.mp4', rm_repair.UNAVAILABLE),
+ (3, 'fine.mp4', rm_repair.CLEAN),
+ (6, 'cut.mp4', rm_repair.UNAVAILABLE))}
+ path = rm_repair._cache_path(self.contract)
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(json.dumps({'version': 1, 'entries': entries}), encoding='utf-8')
+
+ state = self.run_once()
+
+ self.assertEqual(self.transcodes.asked, ['favourite.mp4'])
+ self.assertEqual(self.store.repaired, [2])
+ self.assertEqual((state['repaired'], state['failed']), (1, 0))
+ saved = json.loads(path.read_text(encoding='utf-8'))
+ self.assertEqual(saved['version'], 2)
+ self.assertEqual(saved['entries'][rm_repair._key(1, self.media / 'seldom.mp4')],
+ rm_repair.UNAVAILABLE)
+
+ def test_a_current_failed_tail_layout_stays_cached(self):
+ (self.media / 'favourite.mp4').write_bytes(tail_mp4())
+ cache = {rm_repair._key(asset_id, self.media / name): rm_repair.UNAVAILABLE
+ for asset_id, name in ((1, 'seldom.mp4'), (2, 'favourite.mp4'),
+ (3, 'fine.mp4'), (6, 'cut.mp4'))}
+ rm_repair._save_cache(self.contract, cache)
+
+ self.run_once()
+
+ self.assertEqual(self.transcodes.asked, [])
+ self.assertEqual(self.store.repaired, [])
+
def test_a_changed_file_is_judged_again(self):
"""结论跟着文件的大小和改动时间走:换了内容,上一轮的结论就不作数。"""
self.run_once()