Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

fix(storage): 按挂载类型执行 rclone 永久删除

simon156152dd489d626f
diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md
index ac8d375f6..6289c8e27 100644
--- a/docs/OPERATIONS.md
+++ b/docs/OPERATIONS.md
@@ -503,6 +503,8 @@ Peach 配置「媒体库与入库 → 视频预览」中的「抽帧解码方式
- 移入回收站的条目可以在回收站里「还原」。
- **在回收站里「彻底删除」选中项,或者「清空回收站」,都会永久删除媒体文件和对应的馆藏记录,无法恢复。**
+- rclone FUSE 挂载上的媒体会直接删除,避免部分后端的远端改名退化为复制。若媒体已删但账本事务失败,回收站记录会保留;再次清空可移除这条残留记录,无法回滚已完成的远端文件删除。
+- 飞牛 OS 文件管理器的原生 115 远程挂载可在 Linux 挂载表中表现为 `fuse.rclone`;此时适用上述行为。这里的 rclone 是该环境的底层文件系统类型,不表示用户另外部署了 rclone、CloudDrive2 或 OpenList,也不代表其他挂载方式已经验证。
### 垃圾文件
diff --git a/docs/REUSE.md b/docs/REUSE.md
index ca971c5a4..d17281fc3 100644
--- a/docs/REUSE.md
+++ b/docs/REUSE.md
@@ -157,7 +157,9 @@ DMM / MGS 的只读商品号识别复用 `code_key`、`release_code_from_text`
STRM 采用文本指针、播放时解析的通用行为,参考 SmartStrm 文档、OpenList 的 `drivers/strm` 与 Jellyfin 的 STRM 探测入口;固定 revision、许可证与实际读取位置见 [取证记录](reference-snapshots/strm-source-evidence.md)。SmartStrm 核心闭源,源码未取得。Peach 独立实现受限 HTTP(S) 指针,复用 HTTPX、现有 Range 传输、`m3u8` 解析与播放器计划;无新增运行依赖,也不复制上游应用代码。
-光鸭、OneDrive 与其他网盘复用声明挂载根、路径翻译、扫描、文件整理与回滚、隔离删除和空目录清理。所有挂载来源统一开放读写,底层权限或不支持的操作由挂载工具报错;网盘登录与平台 API 由 CloudDrive2 或 OpenList 等挂载工具提供,不新增 SDK 或依赖。临时文件与账本回归覆盖三个来源的整理、回滚、删除、权限拒绝和空目录清理;真实账号兼容性见待办。挂载目录可单独设置代理流量保护,成本投影与批处理使用同一规则。STRM、ISO 跳过后台媒体读取,普通媒体保留按来源策略执行的探测;九宫格和时间轴共用本机视频预览开关。
+光鸭、OneDrive 与其他网盘复用声明挂载根、路径翻译、扫描、文件整理与回滚、隔离删除和空目录清理。所有挂载来源统一开放读写,底层权限或不支持的操作由挂载工具报错;网盘登录与平台 API 由飞牛 OS 原生远程挂载、CloudDrive2 或 OpenList 等挂载工具提供,不新增 SDK 或依赖。本地文件先隔离再提交账本;rclone FUSE 通过挂载表识别并直接删除,避免部分后端的远端改名触发复制,账本事务失败时保留可重试的回收站记录。飞牛 OS 原生挂载 115 的实际文件系统可报告为 `fuse.rclone`,不等于用户单独部署 rclone;其他挂载方式仍须分别验证。临时文件与账本回归覆盖三个来源的整理、回滚、删除、权限拒绝和空目录清理;真实账号兼容性见待办。挂载目录可单独设置代理流量保护,成本投影与批处理使用同一规则。STRM、ISO 跳过后台媒体读取,普通媒体保留按来源策略执行的探测;九宫格和时间轴共用本机视频预览开关。
+
+回收站复用 `platform.translate_ledger_path`、声明来源根与 Python 标准库 `stat`,只将 `FileNotFoundError` 视为文件缺失,权限或 I/O 错误保留记录。Linux 挂载判定读取原生 `/proc/self/mountinfo`,字段、父挂载关系与文件系统类型按 [Linux man-pages 6.19](https://man7.org/linux/man-pages/man5/proc_pid_mountinfo.5.html);采用最深挂载点,覆盖挂载按父子关系选择,不依赖文本行顺序。转义解码与路径比较属于 Peach 的删除策略,没有复制外部实现。rclone 挂载能力参考其[官方文档](https://rclone.org/commands/rclone_mount/),不新增 rclone 运行服务;现有 psutil 7.2.2 仅是开发依赖,不为读取一份 Linux 原生挂载表增加运行依赖。未知挂载信息保留本地隔离流程,临时文件回归覆盖带空格挂载点、父目录路径、覆盖挂载、删除失败与混合批次事务失败。
Actions 工作流静态检查复用 actionlint 1.7.12(MIT,`rhysd/actionlint:1.7.12` 官方 Docker 镜像),由本地 Docker action 使用,独立 Docker 目录纳入 Dependabot 与接管入口。GitHub Actions 生态不更新 `docker://` 镜像引用。Windows 官方二进制按发布校验和完成本地 POC;CI 在 Linux 容器检查语法、表达式、任务依赖与脚本。Python 与浏览器回归继续使用现有统一入口及 Playwright;六片浏览器权重来自 [已通过的 CI](https://github.com/peach-mitao/peach/actions/runs/38981261189) 的 69 份 TAP,记录在 `scripts/ci-browser-timings.json`,没有新增应用运行依赖。
diff --git a/src/peach/web_batch.py b/src/peach/web_batch.py
index bbbae1940..e61f0bd25 100644
--- a/src/peach/web_batch.py
+++ b/src/peach/web_batch.py
@@ -3,14 +3,15 @@
`junk` 提供各流程共用的证据与隔离规则;本模块将评分转成页面数据,并执行用户的
批量决定。自动隔离只进入回收站,永久删除需要独立的确认与取证流程。
-物理删除要清的引用表只写在 `personal_records.ASSET_REFERENCE_TABLES` 一处;文件隔离加数据库失败回滚
-(`_restore_staged_media`)是这个模块最不能出错的部分:删错的文件找不回来。
+物理删除要清的引用表只写在 `personal_records.ASSET_REFERENCE_TABLES` 一处。本地文件先隔离,
+账本失败时可以恢复;rclone FUSE 挂载直接删除,账本失败时保留回收站行供重试。
"""
from __future__ import annotations
import hashlib
import json
import os
+import re
import stat
import shutil
import tempfile
@@ -56,6 +57,8 @@ from .junk import (
promo_residue as promo_residue,
)
+# Linux 容器可从 mountinfo 识别媒体实际所在的 FUSE 文件系统。
+MOUNTINFO_PATH = Path("/proc/self/mountinfo")
#: 批量操作在活动页上的名字。表里存 `operation`,人看的是这一列。
BATCH_LABELS = {
@@ -155,6 +158,47 @@ def _restore_staged_media(staged):
os.rename(quarantine, original)
+def _mountinfo_field(value: str) -> str:
+ """解码 mountinfo 中空白字符和反斜杠的八进制转义。"""
+ return re.sub(r"\\([0-7]{3})", lambda match: chr(int(match.group(1), 8)), value)
+
+
+def _is_rclone_mount(path: Path) -> bool:
+ """按最深挂载点与覆盖关系识别 rclone 文件系统;不确定时保留隔离流程。"""
+ try:
+ lines = MOUNTINFO_PATH.read_text(encoding="utf-8", errors="surrogateescape").splitlines()
+ except OSError:
+ return False
+
+ target = Path(os.path.abspath(path))
+ matches = []
+ for line in lines:
+ before, separator, after = line.partition(" - ")
+ if not separator:
+ continue
+ fields = before.split()
+ filesystem = after.split(maxsplit=1)
+ if len(fields) < 5 or not filesystem:
+ continue
+ try:
+ mount_id, parent_id = int(fields[0]), int(fields[1])
+ except ValueError:
+ continue
+ mountpoint = Path(os.path.abspath(_mountinfo_field(fields[4])))
+ try:
+ target.relative_to(mountpoint)
+ except ValueError:
+ continue
+ matches.append((len(mountpoint.parts), mount_id, parent_id, filesystem[0]))
+ if not matches:
+ return False
+ depth = max(item[0] for item in matches)
+ deepest = [item for item in matches if item[0] == depth]
+ covered = {item[2] for item in deepest if item[1] != item[2]}
+ visible = [item for item in deepest if item[1] not in covered]
+ return len(visible) == 1 and visible[0][3] == "fuse.rclone"
+
+
def _quarantine_media(original: Path, *, location: str | None = None) -> Path:
"""同目录改名不支持时,小文件先保留本地完整备份再删除原文件。"""
key = uuid.uuid4().hex
@@ -288,11 +332,10 @@ def _finish_purge(outcome):
def purge_assets(connection, rows, *, missing_only: bool = False):
- """Quarantine media, delete ledger rows, and leave final removal to the caller.
+ """删除媒体和账本行,调用方提交后清理隔离文件。
- Renaming beside the source is reversible and stays on the same filesystem. The
- caller restores the quarantined names if commit fails, then permanently removes
- them only after the SQLite transaction has committed.
+ 本地文件同目录隔离,账本事务失败时恢复。rclone FUSE 直接删除,避免远端改名
+ 触发复制;账本事务失败时保留记录供重试,已经删除的远端媒体无法恢复。
``missing_only`` 删的只是账本行:文件还在(或答不上在不在)的行进 ``blocked``。
"""
@@ -341,8 +384,11 @@ def purge_assets(connection, rows, *, missing_only: bool = False):
if media_stat is not None and not stat.S_ISREG(media_stat.st_mode):
raise OSError("not a regular file")
if media_stat is not None:
- quarantine = _quarantine_media(original, location=row['location'] if 'location' in row.keys() else None)
- staged.append((original, quarantine))
+ if _is_rclone_mount(original):
+ original.unlink()
+ else:
+ quarantine = _quarantine_media(original, location=row['location'] if 'location' in row.keys() else None)
+ staged.append((original, quarantine))
except OSError as error:
blocked.append({"id": row["id"], "path": media,
"reason": error.strerror or str(error)})
diff --git a/tests/test_rm_web.py b/tests/test_rm_web.py
index 5c3152993..5e1c5bbd1 100644
--- a/tests/test_rm_web.py
+++ b/tests/test_rm_web.py
@@ -1801,6 +1801,110 @@ class WebDataTests(unittest.TestCase):
self.assertTrue(kept.exists(), "不在回收站的媒体一个都不能碰")
self.assertIsNotNone(self.row(2))
+ def test_rclone_mount_detection_uses_the_most_specific_mountpoint(self):
+ mountinfo = Path(self.tmp.name) / "mountinfo"
+ mountinfo.write_text(
+ "36 25 0:41 / /media rw - ext4 /dev/root rw\n"
+ "37 36 0:62 / /media/115 rw - fuse.rclone remote:library rw\n"
+ "38 37 0:63 / /media/115/local rw - ext4 /dev/data rw\n",
+ encoding="utf-8",
+ )
+
+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):
+ self.assertTrue(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))
+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115-old/movie.mp4")))
+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/local/movie.mp4")))
+
+ def test_rclone_mount_detection_handles_escapes_and_normalizes_parent_segments(self):
+ mountinfo = Path(self.tmp.name).resolve() / "mountinfo"
+ mountinfo.write_text(
+ "36 25 0:41 / /media rw - ext4 /dev/root rw\n"
+ "37 36 0:62 / /media/cloud\\040drive rw - fuse.rclone remote:library rw\n",
+ encoding="utf-8")
+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):
+ self.assertTrue(web_batch._is_rclone_mount(Path("/media/cloud drive/movie.mp4")))
+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/cloud drive/../local/movie.mp4")))
+
+ def test_rclone_mount_detection_uses_mount_parents_for_stacked_mounts(self):
+ mountinfo = Path(self.tmp.name).resolve() / "mountinfo"
+ rows = ["36 25 0:41 / /media rw - ext4 /dev/root rw\n",
+ "38 37 0:63 / /media/115 rw - ext4 /dev/data rw\n",
+ "37 36 0:62 / /media/115 rw - fuse.rclone remote:library rw\n"]
+ for lines in (rows, list(reversed(rows))):
+ mountinfo.write_text("".join(lines), encoding="utf-8")
+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):
+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))
+
+ def test_unknown_mount_information_keeps_the_local_isolation_policy(self):
+ mountinfo = Path(self.tmp.name).resolve() / "mountinfo"
+ with mock.patch.object(web_batch, "MOUNTINFO_PATH", mountinfo):
+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))
+ mountinfo.write_text("invalid mount entry\n", encoding="utf-8")
+ self.assertFalse(web_batch._is_rclone_mount(Path("/media/115/movie.mp4")))
+
+ def test_empty_trash_unlinks_115_rclone_media_without_remote_rename(self):
+ media = self.stage_media(2, "rclone-115.mp4")
+ media.write_bytes(b"x" * (1024 * 1024 + 1))
+ rm_web.w_feedback(self.contract, {"id": 2, "kind": "dispose"})
+
+ with mock.patch.object(web_batch, "_is_rclone_mount", return_value=True), \
+ mock.patch.object(web_batch.os, "rename") as rename:
+ result = rm_web.w_empty_trash(self.contract)
+
+ self.assertEqual((result["purged"], result["blocked"]), (1, []))
+ self.assertFalse(media.exists())
+ self.assertIsNone(self.row(2))
+ rename.assert_not_called()
+
+ def test_rclone_unlink_database_failure_leaves_a_retryable_trash_row(self):
+ local = self.stage_media(1, "local-commit-failure.mp4")
+ media = self.stage_media(2, "rclone-commit-failure.mp4")
+ rm_web.w_feedback(self.contract, {"id": 1, "kind": "dispose"})
+ rm_web.w_feedback(self.contract, {"id": 2, "kind": "dispose"})
+
+ @contextmanager
+ def fail_after_body():
+ connection = self.contract.database.connect(write=True)
+ try:
+ yield connection
+ connection.rollback()
+ raise sqlite3.OperationalError("simulated commit failure")
+ finally:
+ connection.close()
+
+ with mock.patch.object(web_batch, "_is_rclone_mount", side_effect=lambda path: path == media), \
+ mock.patch.object(self.contract, "write_transaction", fail_after_body):
+ with self.assertRaisesRegex(sqlite3.OperationalError, "commit failure"):
+ rm_web.w_empty_trash(self.contract)
+
+ self.assertFalse(media.exists(), "远端删除已完成,数据库失败时不能假称文件已恢复")
+ self.assertTrue(local.is_file())
+ self.assertEqual(self.row(1)["disposal"], "trash")
+ self.assertEqual(self.row(2)["disposal"], "trash")
+
+ retry = rm_web.w_empty_trash(self.contract)
+ self.assertEqual((retry["purged"], retry["blocked"]), (2, []))
+ self.assertFalse(local.exists())
+ self.assertIsNone(self.row(2), "重试应只清理已不存在媒体对应的回收站记录")
+
+ def test_rclone_delete_failure_keeps_media_and_a_retryable_trash_row(self):
+ media = self.stage_media(2, "rclone-delete-failure.mp4")
+ rm_web.w_feedback(self.contract, {"id": 2, "kind": "dispose"})
+ original_unlink = Path.unlink
+
+ def unlink(path, *args, **kwargs):
+ if path == media:
+ raise PermissionError("remote delete denied")
+ return original_unlink(path, *args, **kwargs)
+
+ with mock.patch.object(web_batch, "_is_rclone_mount", return_value=True), \
+ mock.patch.object(Path, "unlink", unlink):
+ result = rm_web.w_empty_trash(self.contract)
+ self.assertEqual(result["purged"], 0)
+ self.assertEqual([item["id"] for item in result["blocked"]], [2])
+ self.assertTrue(media.is_file())
+ self.assertEqual(self.row(2)["disposal"], "trash")
+
def test_undeletable_media_keeps_its_row_in_the_bin_instead_of_orphaning_the_file(self):
"""文件删不掉时保留账本行:留一条能重试的回收站条目,好过留一个没人认领的文件。"""
blocked_path = Path(self.tmp.name) / "locked"