Peach: Local-first video metadata scraper and media manager.
fix(storage): 按挂载类型执行 rclone 永久删除
simon156152dd489d626f
diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.mdindex ac8d375f6..6289c8e27 100644--- a/docs/OPERATIONS.md+++ b/docs/OPERATIONS.md@@ -503,6 +503,8 @@ Peach 配置「媒体库与入库 → 视频预览」中的「抽帧解码方式- 移入回收站的条目可以在回收站里「还原」。- **在回收站里「彻底删除」选中项,或者「清空回收站」,都会永久删除媒体文件和对应的馆藏记录,无法恢复。**+- rclone FUSE 挂载上的媒体会直接删除,避免部分后端的远端改名退化为复制。若媒体已删但账本事务失败,回收站记录会保留;再次清空可移除这条残留记录,无法回滚已完成的远端文件删除。+- 飞牛 OS 文件管理器的原生 115 远程挂载可在 Linux 挂载表中表现为 `fuse.rclone`;此时适用上述行为。这里的 rclone 是该环境的底层文件系统类型,不表示用户另外部署了 rclone、CloudDrive2 或 OpenList,也不代表其他挂载方式已经验证。### 垃圾文件diff --git a/docs/REUSE.md b/docs/REUSE.mdindex ca971c5a4..d17281fc3 100644--- a/docs/REUSE.md+++ b/docs/REUSE.md@@ -157,7 +157,9 @@ DMM / MGS 的只读商品号识别复用 `code_key`、`release_code_from_text`STRM 采用文本指针、播放时解析的通用行为,参考 SmartStrm 文档、OpenList 的 `drivers/strm` 与 Jellyfin 的 STRM 探测入口;固定 revision、许可证与实际读取位置见 [取证记录](reference-snapshots/strm-source-evidence.md)。SmartStrm 核心闭源,源码未取得。Peach 独立实现受限 HTTP(S) 指针,复用 HTTPX、现有 Range 传输、`m3u8` 解析与播放器计划;无新增运行依赖,也不复制上游应用代码。-光鸭、OneDrive 与其他网盘复用声明挂载根、路径翻译、扫描、文件整理与回滚、隔离删除和空目录清理。所有挂载来源统一开放读写,底层权限或不支持的操作由挂载工具报错;网盘登录与平台 API 由 CloudDrive2 或 OpenList 等挂载工具提供,不新增 SDK 或依赖。临时文件与账本回归覆盖三个来源的整理、回滚、删除、权限拒绝和空目录清理;真实账号兼容性见待办。挂载目录可单独设置代理流量保护,成本投影与批处理使用同一规则。STRM、ISO 跳过后台媒体读取,普通媒体保留按来源策略执行的探测;九宫格和时间轴共用本机视频预览开关。+光鸭、OneDrive 与其他网盘复用声明挂载根、路径翻译、扫描、文件整理与回滚、隔离删除和空目录清理。所有挂载来源统一开放读写,底层权限或不支持的操作由挂载工具报错;网盘登录与平台 API 由飞牛 OS 原生远程挂载、CloudDrive2 或 OpenList 等挂载工具提供,不新增 SDK 或依赖。本地文件先隔离再提交账本;rclone FUSE 通过挂载表识别并直接删除,避免部分后端的远端改名触发复制,账本事务失败时保留可重试的回收站记录。飞牛 OS 原生挂载 115 的实际文件系统可报告为 `fuse.rclone`,不等于用户单独部署 rclone;其他挂载方式仍须分别验证。临时文件与账本回归覆盖三个来源的整理、回滚、删除、权限拒绝和空目录清理;真实账号兼容性见待办。挂载目录可单独设置代理流量保护,成本投影与批处理使用同一规则。STRM、ISO 跳过后台媒体读取,普通媒体保留按来源策略执行的探测;九宫格和时间轴共用本机视频预览开关。++回收站复用 `platform.translate_ledger_path`、声明来源根与 Python 标准库 `stat`,只将 `FileNotFoundError` 视为文件缺失,权限或 I/O 错误保留记录。Linux 挂载判定读取原生 `/proc/self/mountinfo`,字段、父挂载关系与文件系统类型按 [Linux man-pages 6.19](https://man7.org/linux/man-pages/man5/proc_pid_mountinfo.5.html);采用最深挂载点,覆盖挂载按父子关系选择,不依赖文本行顺序。转义解码与路径比较属于 Peach 的删除策略,没有复制外部实现。rclone 挂载能力参考其[官方文档](https://rclone.org/commands/rclone_mount/),不新增 rclone 运行服务;现有 psutil 7.2.2 仅是开发依赖,不为读取一份 Linux 原生挂载表增加运行依赖。未知挂载信息保留本地隔离流程,临时文件回归覆盖带空格挂载点、父目录路径、覆盖挂载、删除失败与混合批次事务失败。Actions 工作流静态检查复用 actionlint 1.7.12(MIT,`rhysd/actionlint:1.7.12` 官方 Docker 镜像),由本地 Docker action 使用,独立 Docker 目录纳入 Dependabot 与接管入口。GitHub Actions 生态不更新 `docker://` 镜像引用。Windows 官方二进制按发布校验和完成本地 POC;CI 在 Linux 容器检查语法、表达式、任务依赖与脚本。Python 与浏览器回归继续使用现有统一入口及 Playwright;六片浏览器权重来自 [已通过的 CI](https://github.com/peach-mitao/peach/actions/runs/38981261189) 的 69 份 TAP,记录在 `scripts/ci-browser-timings.json`,没有新增应用运行依赖。diff --git a/src/peach/web_batch.py b/src/peach/web_batch.pyindex bbbae1940..e61f0bd25 100644--- a/src/peach/web_batch.py+++ b/src/peach/web_batch.py@@ -3,14 +3,15 @@`junk` 提供各流程共用的证据与隔离规则;本模块将评分转成页面数据,并执行用户的批量决定。自动隔离只进入回收站,永久删除需要独立的确认与取证流程。-物理删除要清的引用表只写在 `personal_records.ASSET_REFERENCE_TABLES` 一处;文件隔离加数据库失败回滚-(`_restore_staged_media`)是这个模块最不能出错的部分:删错的文件找不回来。+物理删除要清的引用表只写在 `personal_records.ASSET_REFERENCE_TABLES` 一处。本地文件先隔离,+账本失败时可以恢复;rclone FUSE 挂载直接删除,账本失败时保留回收站行供重试。"""from __future__ import annotationsimport hashlibimport jsonimport os+import reimport statimport shutilimport tempfile@@ -56,6 +57,8 @@ from .junk import (promo_residue as promo_residue,)+# Linux 容器可从 mountinfo 识别媒体实际所在的 FUSE 文件系统。+MOUNTINFO_PATH = Path("/proc/self/mountinfo")#: 批量操作在活动页上的名字。表里存 `operation`,人看的是这一列。BATCH_LABELS = {@@ -155,6 +158,47 @@ def _restore_staged_media(staged):os.rename(quarantine, original)+def _mountinfo_field(value: str) -> str:+ """解码 mountinfo 中空白字符和反斜杠的八进制转义。"""+ return re.sub(r"\\([0-7]{3})", lambda match: chr(int(match.group(1), 8)), value)+++def _is_rclone_mount(path: Path) -> bool:+ """按最深挂载点与覆盖关系识别 rclone 文件系统;不确定时保留隔离流程。"""+ try:+ lines = MOUNTINFO_PATH.read_text(encoding="utf-8", errors="surrogateescape").splitlines()+ except OSError:+ return False++ target = Path(os.path.abspath(path))+ matches = []+ for line in lines:+ before, separator, after = line.partition(" - ")+ if not separator:+ continue+ fields = before.split()+ filesystem = after.split(maxsplit=1)+ if len(fields) < 5 or not filesystem:+ continue+ try:+ mount_id, parent_id = int(fields[0]), int(fields[1])+ except ValueError:+ continue+ mountpoint = Path(os.path.abspath(_mountinfo_field(fields[4])))+ try:+ target.relative_to(mountpoint)+ except ValueError:+ continue+ matches.append((len(mountpoint.parts), mount_id, parent_id, filesystem[0]))+ if not matches:+ return False+ depth = max(item[0] for item in matches)+ deepest = [item for item in matches if item[0] == depth]+ covered = {item[2] for item in deepest if item[1] != item[2]}+ visible = [item for item in deepest if item[1] not in covered]+ return len(visible) == 1 and visible[0][3] == "fuse.rclone"++def _quarantine_media(original: Path, *, location: str | None = None) -> Path:"""同目录改名不支持时,小文件先保留本地完整备份再删除原文件。"""key = uuid.uuid4().hex@@ -288,11 +332,10 @@ def _finish_purge(outcome):def purge_assets(connection, rows, *, missing_only: bool = False):- """Quarantine media, delete ledger rows, and leave final removal to the caller.+ """删除媒体和账本行,调用方提交后清理隔离文件。- Renaming beside the source is reversible and stays on the same filesystem. The- caller restores the quarantined names if commit fails, then permanently removes- them only after the SQLite transaction has committed.+ 本地文件同目录隔离,账本事务失败时恢复。rclone FUSE 直接删除,避免远端改名+ 触发复制;账本事务失败时保留记录供重试,已经删除的远端媒体无法恢复。``missing_only`` 删的只是账本行:文件还在(或答不上在不在)的行进 ``blocked``。"""@@ -341,8 +384,11 @@ def purge_assets(connection, rows, *, missing_only: bool = False):if media_stat is not None and not stat.S_ISREG(media_stat.st_mode):raise OSError("not a regular file")if media_stat is not None:- quarantine = _quarantine_media(original, location=row['location'] if 'location' in row.keys() else None)- staged.append((original, quarantine))+ if _is_rclone_mount(original):+ original.unlink()+ else:+ quarantine = _quarantine_media(original, location=row['location'] if 'location' in row.keys() else None)+ staged.append((original, quarantine))except OSError as error:blocked.append({"id": row["id"], "path": media,"reason": error.strerror or str(error)})diff --git a/tests/test_rm_web.py b/tests/test_rm_web.pyindex 5c3152993..5e1c5bbd1 100644--- a/tests/test_rm_web.py+++ b/tests/test_rm_web.py@@ -1801,6 +1801,110 @@ class WebDataTests(unittest.TestCase):self.assertTrue(kept.exists(), "不在回收站的媒体一个都不能碰")self.assertIsNotNone(self.row(2))+ def test_rclone_mount_detection_uses_the_most_specific_mountpoint(self):+ mountinfo = Path(self.tmp.name) / "mountinfo"+ mountinfo.write_text(+ "36 25 0:41 / /media rw - ext4 /dev/root rw\n"+ "37 36 0:62 / /media/115 rw - fuse.rclone remote:library rw\n"+ "38 37 0:63 / /media/115/local rw - ext4 /dev/data rw\n",+ encoding="utf-8",+ )++ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):+ self.assertTrue(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115-old/movie.mp4")))+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/local/movie.mp4")))++ def test_rclone_mount_detection_handles_escapes_and_normalizes_parent_segments(self):+ mountinfo = Path(self.tmp.name).resolve() / "mountinfo"+ mountinfo.write_text(+ "36 25 0:41 / /media rw - ext4 /dev/root rw\n"+ "37 36 0:62 / /media/cloud\\040drive rw - fuse.rclone remote:library rw\n",+ encoding="utf-8")+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):+ self.assertTrue(web_batch._is_rclone_mount(Path("/media/cloud drive/movie.mp4")))+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/cloud drive/../local/movie.mp4")))++ def test_rclone_mount_detection_uses_mount_parents_for_stacked_mounts(self):+ mountinfo = Path(self.tmp.name).resolve() / "mountinfo"+ rows = ["36 25 0:41 / /media rw - ext4 /dev/root rw\n",+ "38 37 0:63 / /media/115 rw - ext4 /dev/data rw\n",+ "37 36 0:62 / /media/115 rw - fuse.rclone remote:library rw\n"]+ for lines in (rows, list(reversed(rows))):+ mountinfo.write_text("".join(lines), encoding="utf-8")+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))++ def test_unknown_mount_information_keeps_the_local_isolation_policy(self):+ mountinfo = Path(self.tmp.name).resolve() / "mountinfo"+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))+ mountinfo.write_text("invalid mount entry\n", encoding="utf-8")+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))++ def test_empty_trash_unlinks_115_rclone_media_without_remote_rename(self):+ media = self.stage_media(2, "rclone-115.mp4")+ media.write_bytes(b"x" * (1024 * 1024 + 1))+ rm_web.w_feedback(self.contract, {"id": 2, "kind": "dispose"})++ with mock.patch.object(web_batch, "_is_rclone_mount", return_value=True), \+ mock.patch.object(web_batch.os, "rename") as rename:+ result = rm_web.w_empty_trash(self.contract)++ self.assertEqual((result["purged"], result["blocked"]), (1, []))+ self.assertFalse(media.exists())+ self.assertIsNone(self.row(2))+ rename.assert_not_called()++ def test_rclone_unlink_database_failure_leaves_a_retryable_trash_row(self):+ local = self.stage_media(1, "local-commit-failure.mp4")+ media = self.stage_media(2, "rclone-commit-failure.mp4")+ rm_web.w_feedback(self.contract, {"id": 1, "kind": "dispose"})+ rm_web.w_feedback(self.contract, {"id": 2, "kind": "dispose"})++ @contextmanager+ def fail_after_body():+ connection = self.contract.database.connect(write=True)+ try:+ yield connection+ connection.rollback()+ raise sqlite3.OperationalError("simulated commit failure")+ finally:+ connection.close()++ with mock.patch.object(web_batch, "_is_rclone_mount", side_effect=lambda path: path == media), \+ mock.patch.object(self.contract, "write_transaction", fail_after_body):+ with self.assertRaisesRegex(sqlite3.OperationalError, "commit failure"):+ rm_web.w_empty_trash(self.contract)++ self.assertFalse(media.exists(), "远端删除已完成,数据库失败时不能假称文件已恢复")+ self.assertTrue(local.is_file())+ self.assertEqual(self.row(1)["disposal"], "trash")+ self.assertEqual(self.row(2)["disposal"], "trash")++ retry = rm_web.w_empty_trash(self.contract)+ self.assertEqual((retry["purged"], retry["blocked"]), (2, []))+ self.assertFalse(local.exists())+ self.assertIsNone(self.row(2), "重试应只清理已不存在媒体对应的回收站记录")++ def test_rclone_delete_failure_keeps_media_and_a_retryable_trash_row(self):+ media = self.stage_media(2, "rclone-delete-failure.mp4")+ rm_web.w_feedback(self.contract, {"id": 2, "kind": "dispose"})+ original_unlink = Path.unlink++ def unlink(path, *args, **kwargs):+ if path == media:+ raise PermissionError("remote delete denied")+ return original_unlink(path, *args, **kwargs)++ with mock.patch.object(web_batch, "_is_rclone_mount", return_value=True), \+ mock.patch.object(Path, "unlink", unlink):+ result = rm_web.w_empty_trash(self.contract)+ self.assertEqual(result["purged"], 0)+ self.assertEqual([item["id"] for item in result["blocked"]], [2])+ self.assertTrue(media.is_file())+ self.assertEqual(self.row(2)["disposal"], "trash")+def test_undeletable_media_keeps_its_row_in_the_bin_instead_of_orphaning_the_file(self):"""文件删不掉时保留账本行:留一条能重试的回收站条目,好过留一个没人认领的文件。"""blocked_path = Path(self.tmp.name) / "locked"