Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

feat(ledger): 账本立即备份与从备份恢复

longmeidao81903bf57412
diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md
index 8cb9d7077..2e50bae1f 100644
--- a/docs/OPERATIONS.md
+++ b/docs/OPERATIONS.md
@@ -424,6 +424,15 @@ Peach 配置页地址是 `/configuration`。从管理菜单或设置弹窗的「
源码运行时,默认位置是项目旁边的 `peach-data`;原位置留下的 `state/data-root.txt` 让默认位置和显式的 `PEACH_DATA_ROOT` 都跟到新位置。直接运行 `peach serve` 时没有看守进程接手,不能在设置页迁移:改用托盘或无桌面服务启动,或退出 Peach,手动移动数据目录,再把 `PEACH_DATA_ROOT` 指向新位置。
+### 账本备份与恢复
+
+配置页「应用与维护 → 账本备份」列出 `database/` 与 `generated/organize-backups/` 里所有能恢复的备份,新的在前,每行写明时间、来源(手动,或自动及其用途)和大小。取舍见 [ADR-0134](adr/0134-user-triggered-ledger-backup-and-restore.md)。
+
+- **立即备份**:后台复制当前账本,进度同时显示在任务中心。复制完先做完整性检查,通过才命名为 `ledger.pre-manual-<时间>.db`;没通过的那份直接删掉。手动备份和自动备份同一套保留规则(见「迁移、备份与运维脚本」)。
+- **从备份恢复**:只能在写入端执行,且数据目录里没有别的任务在跑。确认后依次检查备份的完整性和结构(来自更新版本的备份拒收,较旧的先迁到当前结构),把当前账本另存为 `ledger.pre-restore-<时间>.db`,再把备份写进账本。恢复期间不能启动新任务;任务中心的历史保留恢复前的记录。完成后刷新页面查看恢复后的资料。
+- **中断处理**:写入账本前会在账本目录留下 `ledger.restore.json`。恢复被打断(断电、强制结束)后,下次启动服务或运行 `peach migrate upgrade` 时先处理它:当前账本完整就保留;不完整就用已经迁好的暂存 `ledger.restore-staging.db` 补完恢复;暂存也不能用就退回恢复前另存的那份。结果显示在「账本备份」里,同时写入 `ledger.restore-result.json`。
+- 账本损坏、服务读不出资料时,「账本备份」照常可用,恢复在后台进行,进度只在这一页显示。
+
容器的数据目录是宿主机挂载的卷,在容器里迁移会离开卷,重建容器时数据丢失,所以设置页不提供迁移,改为列出在宿主机上迁移的步骤和命令,见 [Docker 部署](DOCKER.md#迁移数据目录)。
### 卸载
@@ -816,7 +825,7 @@ Windows 托盘在取得单实例锁后启动独立监视器。监视器用 Win32
- 真实迁移前依次执行:SQLite 备份、asset/tag 计数、`PRAGMA integrity_check`、迁移版本检查、服务冒烟测试。已应用与待应用的迁移只认 `peach migrate status` 的输出。
- 已应用的迁移文件不能修改,任何后续变更都要新增一个版本。已应用的迁移被改写会造成校验和漂移,只能用迁移前备份重放、逐条比对差异为 0 后再校正 `schema_migration`。
-- 数据库备份自动清退:`ledger.pre-*.db` 由 `peach.ledger_backups` 管理,「最近 5 份」「24 小时内」「比 `ledger.db` 新」三类全部保留,其余连同 `-wal`/`-shm` 删除。Windows 托盘每次启动自动执行;`scripts/prune_ledger_backups.py` 手动运行时默认只列计划,`--apply` 才删除;数据库 `integrity_check` 不是 ok 时拒绝清退,退出码 2。每份备份一百多 MB,5 天不清就是 7 GB。
+- 数据库备份自动清退:`ledger.pre-*.db`(含设置页的手动备份 `pre-manual` 与恢复前另存的 `pre-restore`)由 `peach.ledger_backups` 管理,「最近 5 份」「24 小时内」「比 `ledger.db` 新」三类全部保留,其余连同 `-wal`/`-shm` 删除。Windows 托盘每次启动自动执行;`scripts/prune_ledger_backups.py` 手动运行时默认只列计划,`--apply` 才删除;数据库 `integrity_check` 不是 ok 时拒绝清退,退出码 2。每份备份一百多 MB,5 天不清就是 7 GB。
- 整理备份自动清退:下载后整理与手动整理写入前各留一份整库备份,放在 `<Peach 数据目录>/generated/organize-backups/`,文件名是 `ledger.pre-download-<任务>-<时间>.db` 与 `ledger.pre-organize-<时间>-<任务>.db`。每次写备份后按同一套规则清退这个目录,只留最近 5 份(`ORGANIZE_KEEP_RECENT`),不按年龄保留(`ORGANIZE_MIN_AGE` 为零):一波下载一小时能整理四十个文件,24 小时窗口会攒下几十 GB。正在被整理引用的那份由 `ledger_backups.holding` 登记,Web 服务进程里的清退都跳过;目录里的其他文件名不动。手动查看或清理用 `scripts/prune_ledger_backups.py --organize`,同样默认只列计划,`--apply` 才删除;登记不跨进程,整理进行中不要给 `--keep 0`。
- `<Peach 数据目录>/logs` 里的 `*.log` 统一保留半年,不限大小(`peach.log_retention.sweep`,托盘在启动任何子进程之前运行):半年没写过的文件整份删除;还在写的文件按自然月分段,上次写入在更早月份的改名成 `<名字>.until-<最后写入日期>.log`,再过半年删除。子进程直接追加 stdout,不经过 `logging`,所以按文件处理,不按行处理。
- 更新与打包产物自动清退:托盘每次启动只保留最近 2 份 `dist/Peach/Peach.pre-source-sync-*.exe` 备份,并删掉 `<Peach 数据目录>/state/source-sync-build/` 里不属于待应用记录的暂存构建(`WindowsUpdateInstaller.sweep_artifacts`);`build_windows.ps1` 成功后删掉 PyInstaller 工作目录 `build/windows/app`;单文件托盘每次启动删掉 `%TEMP%` 里超过一天、不属于自己的 `_MEI*` 解压残留(`sweep_onefile_extractions`,托盘被结束进程时 PyInstaller 不会自己清理,每份 40–80 MB)。自动清理只认这几种命名:`Peach.exe` 本体、手工放进 `dist/` 的目录、`build/release-*`、`attic/` 都不碰,手工构建的残留要自己删。
diff --git a/docs/adr/0134-user-triggered-ledger-backup-and-restore.md b/docs/adr/0134-user-triggered-ledger-backup-and-restore.md
new file mode 100644
index 000000000..ebfc5f92e
--- /dev/null
+++ b/docs/adr/0134-user-triggered-ledger-backup-and-restore.md
@@ -0,0 +1,28 @@
+# ADR-0134:用户可触发的账本备份与恢复
+
+- 状态:已采纳
+- 日期:2026-11-04
+- 相关:ADR-0040、ADR-0052、ADR-0091
+
+## 背景
+
+账本是唯一真相源。此前只有写入前的自动备份(`ledger.pre-<用途>-<时间>.db`)与清退规则,设置页只能清理旧备份。想回到某一份备份,只能停服务、手工换文件,还要自己处理 `-wal`/`-shm`、结构版本和任务记录。
+
+恢复是第一个由用户在界面上触发、整份替换账本的动作,需要定下替换方式、前置条件和中断后的处理。参考了 javm(MIT)`docs/decisions/0026-library-recovery.md` 的行为:Backup API 分页复制、通过检查才转为正式备份、恢复前另存当前库、恢复中断留标记。
+
+## 决策
+
+1. **立即备份**:后台任务(`ledger-backup`,进任务中心)用 SQLite Backup API 分页复制到 `ledger.pre-manual-<时间>.db.part`,`integrity_check` 为 `ok` 才改成正式文件名,没通过就删掉。命名与保留沿用自动备份:同在账本目录,按 `peach.ledger_backups` 的「最近 5 份、24 小时内、比账本新」三条规则清退。
+2. **恢复的前置条件**:只在写入端执行;数据目录里没有别的任务在跑(`update_tasks.running`,与安装更新同一判据);恢复期间用安装预约挡住新任务。备份先过完整性检查,复制到暂存后按当前程序的迁移目录检查结构:来自更新版本的备份拒收,较旧的备份在暂存里迁到当前结构,再查一次完整性。这些检查都在动账本之前完成。
+3. **先另存再替换**:当前账本另存为 `ledger.pre-restore-<时间>.db`,同一套保留规则。
+4. **替换方式**:不换文件,而是经 Backup API 把暂存整库写进在用的账本。服务的连接都是按请求开关的;写入期间账本上是一个写事务,别的连接照常读旧内容,进程中途退出时事务回滚,账本仍是原来那份。Windows 上换文件会撞上打开的句柄,WAL 模式下还会丢掉 `-wal` 里已提交的内容。当前账本已经坏到 SQLite 打不开时才退回整份换文件。
+5. **运行记录不随备份回退**:替换后用另存的那份覆盖 `task_run`(任务历史和这次恢复自己那一行),`ledger_revision` 取两边较大值再加一,聚合缓存不会撞上旧版本号;随后清掉服务的读缓存。
+6. **中断标记**:写进账本之前在账本目录留下 `ledger.restore.json`,写完才删。服务启动与 `peach migrate upgrade` 在打开账本前先处理它:当前账本完整就保留;不完整或不在了,用暂存里迁好的备份补完;暂存也不能用,就退回另存的那份。账本不在时绝不交给后面的步骤,否则会被当成新装建出空账本。结果写进 `ledger.restore-result.json`,设置页显示。
+7. **坏账本也能进入恢复**:账本读不出来时任务中心那张表也读不出来,恢复改用一个不进任务中心的后台任务,查忙只看登记文件与锁。
+
+## 代价
+
+- 恢复期间进来的写请求会等写锁,拿到后写进恢复后的账本;定时任务撞上预约会排队,文案沿用「等待安装完成后继续」。
+- 恢复后,依赖账本的确定性补修(升级补修)在下次启动时才按版本补上。
+- 页面大小与当前账本不同的备份不能在服务运行时恢复,会直接拒收。
+- 双机复制时,恢复后的账本照常由写入端发布,读取端随下一次同步拿到;读取端不能恢复。
diff --git a/src/peach/api.py b/src/peach/api.py
index f966d315a..8a6481655 100644
--- a/src/peach/api.py
+++ b/src/peach/api.py
@@ -35,7 +35,7 @@ from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.datastructures import MutableHeaders
from starlette.middleware.gzip import DEFAULT_EXCLUDED_CONTENT_TYPES, GZipMiddleware
-from . import (__version__, browser_transport, feeds, tunnel, web_contract,
+from . import (__version__, browser_transport, feeds, ledger_backups, tunnel, web_contract,
web_feeds, web_follow, web_settings)
from . import routes_api, routes_auth, routes_configuration, routes_jav_browser, routes_media, routes_pages, routes_subtitles
from .buildinfo import frozen_build
@@ -207,6 +207,11 @@ WARM_AGGREGATES = (("/api/taste", {"window": "all"}), ("/api/review", {}), ("/ap
def _start_ledger_services(settings: PeachSettings, contract, services) -> bool:
"""只读核对当前账本后,恢复任务并启动依赖账本的后台服务。"""
+ try:
+ # 上次从备份恢复没走完:先把账本收拾到确定的状态,再判它能不能用(ADR-0134)。
+ ledger_backups.recover(settings.db_path)
+ except Exception:
+ LOGGER.exception('账本恢复标记未能处理')
status = inspect_database(settings.db_path)
repairs = getattr(contract, 'upgrade_repairs', None)
if status['schema'] == 'pending' and repairs is not None and repairs.available:
diff --git a/src/peach/cli.py b/src/peach/cli.py
index 77802fe39..7a74c054c 100644
--- a/src/peach/cli.py
+++ b/src/peach/cli.py
@@ -210,6 +210,11 @@ def _build_sync(args: argparse.Namespace, settings: PeachSettings) -> LedgerSync
def _migrate(args: argparse.Namespace) -> int:
_require_readable_settings()
+ if args.action == "upgrade":
+ # 托盘起服务前先跑这一步;恢复中断后账本可能暂时不在,不收拾就会迁出一个空账本。
+ from .ledger_backups import recover
+ if (settled := recover(args.db)) is not None:
+ print(f"restore marker: {settled['message']}")
all_migrations, pending = plan(args.db, MIGRATIONS_DIR)
print(f"database: {args.db}")
print(f"migrations: {len(all_migrations)}, pending: {len(pending)}")
diff --git a/src/peach/jobs.py b/src/peach/jobs.py
index 422461e0e..f610e40fe 100644
--- a/src/peach/jobs.py
+++ b/src/peach/jobs.py
@@ -349,6 +349,13 @@ class BackgroundJob:
value = getattr(getattr(self.runs, 'database', None), 'db_path', None)
return Path(value) if isinstance(value, (str, Path)) else None
+ def registration(self, job_id: str) -> tuple[tuple[Path, ...], tuple[int, ...]]:
+ """这一轮自己的任务登记文件与 `task_run` 行。要等别的任务都停下的那一轮查忙时,把自己除外。"""
+ activity = self._activity_files.get(job_id)
+ run_id = self._run_ids.get(job_id)
+ return ((activity,) if activity is not None else (),
+ (run_id,) if run_id is not None else ())
+
def update(self, job_id: str, **fields: object) -> bool:
"""状态仍属于 `job_id` 时改字段;已经被顶掉就什么都不做并返回 False。"""
with self.editing(job_id) as state:
diff --git a/src/peach/ledger_backups.py b/src/peach/ledger_backups.py
index 84b2f6799..872678e33 100644
--- a/src/peach/ledger_backups.py
+++ b/src/peach/ledger_backups.py
@@ -15,20 +15,29 @@
一小时里整理四十个文件,按年龄保留在这里等于不设上限,所以这一类只按份数留
(`ORGANIZE_KEEP_RECENT`,`ORGANIZE_MIN_AGE` 为零);整理进行中的那份由 `holding`
登记,同一进程里的清退都不碰它。登记不跨进程:手动脚本给 `--keep 0` 时只靠份数保护。
+
+设置页的「立即备份」与「从备份恢复」也在这里(ADR-0134)。手动备份叫
+`ledger.pre-manual-<时间>.db`,恢复前另存的当前账本叫 `ledger.pre-restore-<时间>.db`,
+和自动备份同目录、同一套保留规则。恢复不换文件,而是经 SQLite Backup API 写进在用的账本,
+写入前留下 `ledger.restore.json`;启动时看到它说明上次恢复没走完,由 `recover` 收拾。
"""
from __future__ import annotations
+import json
import logging
import re
+import shutil
import sqlite3
import threading
+import time
from collections.abc import Iterator
-from contextlib import contextmanager
+from contextlib import closing, contextmanager
from dataclasses import dataclass
from datetime import datetime, timedelta
from pathlib import Path
-from .migrations import sqlite_backup
+from .fsutil import atomic_write_text
+from .migrations import plan as migration_plan, sqlite_backup, upgrade as migration_upgrade
log = logging.getLogger(__name__)
@@ -37,6 +46,24 @@ SIDECAR_SUFFIXES = ("-wal", "-shm")
KEEP_RECENT = 5
MIN_AGE = timedelta(hours=24)
+#: 设置页「立即备份」与恢复前另存当前账本的用途名,和自动备份同一套命名与保留规则。
+MANUAL_PURPOSE = "manual"
+RESTORE_PURPOSE = "restore"
+#: 复制中的文件后缀。不合 `BACKUP_PATTERN`,列表与清退都看不见它。
+PART_SUFFIX = ".part"
+#: 分页复制每步的页数:4 KiB 一页时每步 16 MiB,一百多 MB 的账本十来步,进度条看得见在走。
+COPY_PAGES = 4096
+#: 账本目录里的恢复标记、暂存与上次恢复的结果。标记在写进账本之前落下,写完才删。
+MARKER_NAME = "ledger.restore.json"
+STAGING_NAME = "ledger.restore-staging.db"
+RESULT_NAME = "ledger.restore-result.json"
+#: 备份列表里「用途」一栏;不在表里的写入前备份统称「写入前自动备份」。
+PURPOSE_LABELS = {
+ MANUAL_PURPOSE: "手动备份", RESTORE_PURPOSE: "恢复前另存",
+ "migrate": "迁移前", "schema": "迁移前", "upgrade": "自动修复前",
+ "organize": "整理前", "download": "下载后整理前", "tag-management": "标签管理前",
+}
+
ORGANIZE_DIRNAME = "organize-backups"
ORGANIZE_PATTERN = re.compile(r"^ledger\.pre-(?:download|organize)-.+\.db$")
ORGANIZE_KEEP_RECENT = 5
@@ -218,3 +245,448 @@ def prune(
except OSError as exc:
log.warning("数据库备份未能删除 %s: %s", target, exc)
return decided
+
+
+# ---------------------------------------------------------------------------
+# 手动备份与从备份恢复(ADR-0134)
+
+
+class BackupRejected(ValueError):
+ """这份备份或这次恢复不能做,原因写给人看;账本没有被改动。"""
+
+
+def _free_name(directory: Path, purpose: str, now: datetime | None = None) -> Path:
+ stamp = (now or datetime.now()).strftime("%Y%m%d-%H%M%S")
+ candidate = directory / f"ledger.pre-{purpose}-{stamp}.db"
+ serial = 2
+ while candidate.exists() or candidate.with_name(candidate.name + PART_SUFFIX).exists():
+ candidate = directory / f"ledger.pre-{purpose}-{stamp}-{serial}.db"
+ serial += 1
+ return candidate
+
+
+def _unlink_all(path: Path) -> None:
+ for target in (path, *(path.with_name(path.name + s) for s in SIDECAR_SUFFIXES)):
+ target.unlink(missing_ok=True)
+
+
+def _pages(progress, stage: str):
+ """把 `Connection.backup` 的进度回调翻成 `progress(stage=, done=, total=)`。"""
+ if progress is None:
+ return None
+ return lambda _status, remaining, total: progress(stage=stage, done=total - remaining, total=total)
+
+
+def _copy(live: Path, destination: Path, progress, stage: str) -> None:
+ """分页复制成一个单文件备份。
+
+ 副本带着账本的 WAL 标记,之后只读打开它做完整性检查,会在旁边留下 `-wal`/`-shm`;改成回滚日志模式,
+ 备份就只有一个文件。写回账本时不受影响:Backup API 保留目标库自己的日志模式。
+ """
+ sqlite_backup(live, destination, pages=COPY_PAGES, progress=_pages(progress, stage))
+ with closing(sqlite3.connect(destination)) as connection:
+ connection.execute("PRAGMA journal_mode=DELETE")
+
+
+def create(live: Path, *, purpose: str = MANUAL_PURPOSE, progress=None, now: datetime | None = None) -> Path:
+ """写一份整库备份:先写 `.part`,完整性检查通过才改成正式文件名,再按保留规则清退。
+
+ `.part` 不合 `BACKUP_PATTERN`,列表与清退都看不见它;没通过检查的那份当场删掉。
+ """
+ if not live.is_file():
+ raise BackupRejected("账本不存在")
+ target = _free_name(live.parent, purpose, now)
+ part = target.with_name(target.name + PART_SUFFIX)
+ try:
+ _copy(live, part, progress, "复制账本")
+ if progress is not None:
+ progress(stage="检查备份完整性")
+ check = integrity(part)
+ if check != "ok":
+ raise BackupRejected(f"备份没有通过完整性检查:{check}")
+ part.replace(target)
+ finally:
+ _unlink_all(part)
+ try:
+ prune(live, apply=True)
+ except Exception:
+ log.warning("账本备份清退失败", exc_info=True)
+ return target
+
+
+def purpose_of(name: str) -> str:
+ """`ledger.pre-<用途>-<时间>.db` 里的用途,如 `manual`、`migrate`、`alias-merge`。"""
+ rest = name.removeprefix("ledger.pre-").removesuffix(".db")
+ matched = re.match(r"(.+?)-\d", rest)
+ return matched.group(1) if matched else rest
+
+
+def describe(path: Path, location: str) -> dict:
+ purpose = purpose_of(path.name)
+ stat = path.stat()
+ return {
+ "id": f"{location}/{path.name}", "name": path.name,
+ "bytes": stat.st_size, "modified_at": stat.st_mtime,
+ "source": "manual" if purpose == MANUAL_PURPOSE else "auto",
+ "purpose": PURPOSE_LABELS.get(purpose, "写入前自动备份"),
+ }
+
+
+def catalog(live: Path, organize_directory: Path) -> list[dict]:
+ """两处备份目录里能恢复的备份,新的在前。"""
+ found = [(path, location) for location, directory, pattern in
+ (("database", live.parent, BACKUP_PATTERN), ("organize", organize_directory, ORGANIZE_PATTERN))
+ for path in list_backups(directory, pattern)]
+ entries = []
+ for path, location in found:
+ try:
+ entries.append(describe(path, location))
+ except OSError:
+ continue
+ return sorted(entries, key=lambda entry: entry["modified_at"], reverse=True)
+
+
+def resolve(live: Path, organize_directory: Path, ident: object) -> Path:
+ """页面传回的备份 id 换成文件;只认两处目录里合规则的文件名。"""
+ location, _, name = str(ident or "").partition("/")
+ directory, pattern = {"database": (live.parent, BACKUP_PATTERN),
+ "organize": (organize_directory, ORGANIZE_PATTERN)}.get(location, (None, None))
+ if directory is None or not pattern.match(name) or Path(name).name != name:
+ raise BackupRejected("请选择列表里的一份备份")
+ path = directory / name
+ if not path.is_file():
+ raise BackupRejected("这份备份已经不在了,请刷新列表")
+ return path
+
+
+@dataclass(frozen=True)
+class RestoreOutcome:
+ backup: Path
+ saved: Path | None # 恢复前另存的当前账本;当前账本不存在时为 None
+ migrated: tuple[str, ...] = ()
+
+
+def _marker(live: Path) -> Path:
+ return live.with_name(MARKER_NAME)
+
+
+def _staging(live: Path) -> Path:
+ return live.with_name(STAGING_NAME)
+
+
+def read_result(live: Path) -> dict | None:
+ """上一次恢复的结果:中断后启动时的处理,或恢复失败后的退回。成功的恢复看任务中心。"""
+ try:
+ data = json.loads(live.with_name(RESULT_NAME).read_text(encoding="utf-8"))
+ except (OSError, ValueError):
+ return None
+ return data if isinstance(data, dict) else None
+
+
+def _record(live: Path, **fields) -> dict:
+ data = {**fields, "finished_at": time.time()}
+ try:
+ atomic_write_text(live.with_name(RESULT_NAME), json.dumps(data, ensure_ascii=False))
+ except OSError:
+ log.warning("账本恢复结果未能写入", exc_info=True)
+ return data
+
+
+def _page_size(path: Path) -> int:
+ with closing(sqlite3.connect(f"file:{path.as_posix()}?mode=ro", uri=True)) as connection:
+ return int(connection.execute("PRAGMA page_size").fetchone()[0])
+
+
+def _copy_into(source: Path, live: Path) -> None:
+ """把 `source` 整库写进 `live`:走 SQLite Backup API,一步写完。
+
+ 写入期间 `live` 上是一个写事务,别的连接照常读到旧内容;进程中途退出时这个事务回滚,
+ `live` 还是原来的账本,不会只剩半份。这是不直接换文件的原因:Windows 上服务的连接还开着,
+ WAL 模式下 `-wal` 里也可能有没合进主文件的已提交内容。
+ """
+ src = sqlite3.connect(f"file:{source.as_posix()}?mode=ro", uri=True)
+ dst = sqlite3.connect(live, timeout=30)
+ try:
+ src.backup(dst)
+ finally:
+ dst.close()
+ src.close()
+
+
+def _replace_file(source: Path, live: Path) -> None:
+ """当前账本坏到 SQLite 打不开时的退路:整份文件换掉,连同旧的 `-wal`/`-shm`。"""
+ for suffix in SIDECAR_SUFFIXES:
+ live.with_name(live.name + suffix).unlink(missing_ok=True)
+ shutil.copy2(source, live.with_name(live.name + PART_SUFFIX))
+ live.with_name(live.name + PART_SUFFIX).replace(live)
+
+
+def _carry_operational(live: Path, saved: Path) -> None:
+ """恢复后把运行记录接回来:任务历史用恢复前的,账本版本号只增不减。
+
+ `task_run` 记的是这台机器上跑过什么、正在跑什么(包括这次恢复自己),不是馆藏数据;
+ 退回备份里的旧行,正在跑的这一轮就没有行可结算,旧的 `running` 行还会占着互斥键。
+ `ledger_revision` 是聚合缓存的键,退回旧值后再写几次就会撞上恢复前缓存过的版本号。
+ """
+ connection = sqlite3.connect(live.resolve().as_uri(), uri=True, timeout=30, isolation_level=None)
+ try:
+ connection.execute("ATTACH DATABASE ? AS saved", (saved.resolve().as_uri() + "?mode=ro",))
+ def columns(schema: str, table: str) -> list[str]:
+ return [row[1] for row in connection.execute(f"PRAGMA {schema}.table_info({table})")]
+
+ connection.execute("BEGIN IMMEDIATE")
+ shared = [name for name in columns("main", "task_run") if name in set(columns("saved", "task_run"))]
+ if shared:
+ names = ", ".join(f'"{name}"' for name in shared)
+ connection.execute("DELETE FROM main.task_run")
+ connection.execute(f"INSERT INTO main.task_run ({names}) SELECT {names} FROM saved.task_run")
+ if columns("main", "ledger_revision") and columns("saved", "ledger_revision"):
+ connection.execute(
+ "UPDATE main.ledger_revision SET n = max(n, coalesce((SELECT s.n FROM saved.ledger_revision s "
+ "WHERE s.tbl = main.ledger_revision.tbl), 0)) + 1")
+ connection.execute("COMMIT")
+ except sqlite3.Error:
+ if connection.in_transaction:
+ connection.execute("ROLLBACK")
+ raise
+ finally:
+ connection.close()
+
+
+def restore(live: Path, backup: Path, *, migrations_dir: Path, progress=None,
+ now: datetime | None = None) -> RestoreOutcome:
+ """用 `backup` 替换当前账本。
+
+ 顺序是先查后动:备份完整性 → 复制到暂存 → 迁到当前结构 → 暂存完整性;全部通过才另存当前
+ 账本、留下中断标记、写进 `live`。写进之前的任何一步失败,账本都没被碰过。写进之后完整性
+ 不过,用另存的那份退回去。
+ """
+ report = progress or (lambda **_: None)
+ report(stage="检查备份完整性")
+ check = integrity(backup)
+ if check != "ok":
+ raise BackupRejected(f"这份备份没有通过完整性检查:{check}")
+ staging = _staging(live)
+ _unlink_all(staging)
+ try:
+ sqlite_backup(backup, staging, pages=COPY_PAGES, progress=_pages(progress, "复制备份"))
+ try:
+ _, pending = migration_plan(staging, migrations_dir)
+ except RuntimeError as exc:
+ raise BackupRejected(f"这份备份的结构与当前程序不符:{exc}") from exc
+ if pending:
+ report(stage="把备份迁到当前结构")
+ migration_upgrade(staging, migrations_dir)
+ check = integrity(staging)
+ if check != "ok":
+ raise BackupRejected(f"备份迁到当前结构后没有通过完整性检查:{check}")
+ healthy = live.is_file() and integrity(live) == "ok"
+ if healthy and _page_size(live) != _page_size(staging):
+ raise BackupRejected("这份备份的页面大小与当前账本不同,不能在服务运行时恢复")
+ report(stage="另存当前账本")
+ saved = None
+ if live.is_file():
+ saved = _free_name(live.parent, RESTORE_PURPOSE, now)
+ if healthy:
+ _copy(live, saved, progress, "另存当前账本")
+ if integrity(saved) != "ok":
+ _unlink_all(saved)
+ raise BackupRejected("当前账本另存后没有通过完整性检查,没有恢复")
+ else:
+ # 坏账本 SQLite 未必读得出来,按文件原样留一份,连同 `-wal`。
+ for suffix in ("", "-wal"):
+ source = live.with_name(live.name + suffix)
+ if source.is_file():
+ shutil.copy2(source, saved.with_name(saved.name + suffix))
+ marker = _marker(live)
+ atomic_write_text(marker, json.dumps({
+ "state": "copying", "backup": str(backup), "saved": str(saved) if saved else "",
+ "started_at": time.time()}, ensure_ascii=False))
+ report(stage="写入账本")
+ try:
+ if healthy:
+ _copy_into(staging, live)
+ else:
+ atomic_write_text(marker, json.dumps({
+ "state": "replacing", "backup": str(backup), "saved": str(saved) if saved else "",
+ "started_at": time.time()}, ensure_ascii=False))
+ _replace_file(staging, live)
+ check = integrity(live)
+ if check != "ok":
+ raise BackupRejected(f"恢复后的账本没有通过完整性检查:{check}")
+ except Exception as exc:
+ settled = settle(live, reason=f"恢复没有完成({exc})", state="failed")
+ raise BackupRejected(settled["message"]) from exc
+ if healthy and saved is not None:
+ try:
+ _carry_operational(live, saved)
+ except sqlite3.Error:
+ log.warning("恢复后未能接回任务历史与账本版本号", exc_info=True)
+ marker.unlink(missing_ok=True)
+ live.with_name(RESULT_NAME).unlink(missing_ok=True)
+ finally:
+ _unlink_all(staging)
+ try:
+ prune(live, apply=True)
+ except Exception:
+ log.warning("账本备份清退失败", exc_info=True)
+ return RestoreOutcome(backup, saved, tuple(migration.version for migration in pending))
+
+
+def settle(live: Path, *, reason: str = "上次从备份恢复被中断", state: str = "interrupted") -> dict:
+ """恢复留下了标记却没走完时,把账本收拾到一个确定的状态,并记下结果。
+
+ 启动时(服务与 `peach migrate upgrade` 打开账本之前)调用,恢复中途出错时也调用。依次:
+ 当前账本完整就留着它;不完整或不在了,用暂存里迁好的备份补完这次恢复;暂存也不能用,就退回
+ 恢复前另存的那份。绝不让账本不在的情况落到后面去——那会被当成新装、建出一个空账本。
+ """
+ marker = _marker(live)
+ try:
+ data = json.loads(marker.read_text(encoding="utf-8"))
+ except (OSError, ValueError):
+ data = {}
+ saved = Path(data["saved"]) if data.get("saved") else None
+ staging = _staging(live)
+ if live.is_file() and integrity(live) == "ok":
+ outcome = "kept"
+ message = f"{reason};当前账本完整性检查通过,保持不变"
+ elif staging.is_file() and integrity(staging) == "ok":
+ _replace_file(staging, live)
+ outcome = "completed"
+ message = f"{reason};已用迁好的备份完成恢复"
+ elif saved is not None and saved.is_file() and integrity(saved) == "ok":
+ if live.is_file():
+ try:
+ _copy_into(saved, live)
+ except sqlite3.Error:
+ _replace_file(saved, live)
+ else:
+ shutil.copy2(saved, live)
+ outcome = "rolled-back"
+ message = f"{reason};已退回恢复前的账本"
+ else:
+ outcome = "unresolved"
+ message = f"{reason};当前账本和恢复前的另存都没有通过完整性检查,请从备份列表另选一份恢复"
+ if saved is not None and outcome != "rolled-back":
+ message += f",恢复前的账本另存在 {saved.name}"
+ _unlink_all(staging)
+ result = _record(live, state=state, outcome=outcome, message=message,
+ backup=Path(data.get("backup") or "").name)
+ marker.unlink(missing_ok=True)
+ log.warning("账本恢复标记处理:%s", message)
+ return result
+
+
+def recover(live: Path) -> dict | None:
+ """有中断标记就处理它;没有返回 None。"""
+ if not _marker(live).is_file():
+ return None
+ return settle(live)
+
+
+class LedgerBackups:
+ """设置页「账本备份」:立即备份与从备份恢复各一个后台任务,都进任务中心。
+
+ 当前账本坏了时任务中心那张表也读不出来,恢复改用一个不进任务中心的任务,进度只在设置页看。
+ """
+
+ def __init__(self, config, *, runs=None, after_restore=None, migrations_dir: Path | None = None):
+ from .jobs import BackgroundJob
+ self.config = config
+ self.live = config.directory("database") / "ledger.db"
+ self.organize = organize_dir(config.directory("generated"))
+ self.migrations_dir = migrations_dir
+ self.after_restore = after_restore
+ self.backup_job = BackgroundJob("PeachLedgerBackup", task_key="ledger-backup", runs=runs)
+ self.restore_job = BackgroundJob("PeachLedgerRestore", task_key="ledger-restore", runs=runs)
+ self.offline_restore = BackgroundJob("PeachLedgerRestore", task_key="ledger-restore")
+ self.lock = threading.Lock()
+ self._restoring = self.restore_job
+
+ def _busy(self) -> bool:
+ return any((job.snapshot() or {}).get("status") == "running"
+ for job in (self.backup_job, self._restoring))
+
+ def snapshot(self) -> dict:
+ from .runtime_prepare import configured_ledger_writer
+ writer, reason = configured_ledger_writer(self.config)
+ return {
+ "backups": catalog(self.live, self.organize),
+ "backup": self.backup_job.snapshot() or {"status": "idle"},
+ "restore": self._restoring.snapshot() or {"status": "idle"},
+ "writer": writer, "writer_reason": "" if writer else reason,
+ "last_restore": read_result(self.live),
+ }
+
+ def start_backup(self) -> dict:
+ with self.lock:
+ if self._busy():
+ raise BackupRejected("备份或恢复正在进行,请等它完成")
+ job = self.backup_job
+
+ def work(identity):
+ path = create(self.live, progress=lambda **values: job.update(identity, **values))
+ job.update(identity, status="complete", stage="备份完成", name=path.name,
+ bytes=path.stat().st_size)
+
+ job.start(work, restart=True, initial={"stage": "准备备份", "done": 0, "total": 0})
+ return self.snapshot()
+
+ def start_restore(self, ident: object) -> dict:
+ from .health import inspect_database
+ from .runtime_prepare import configured_ledger_writer
+ from . import update_tasks
+ writer, reason = configured_ledger_writer(self.config)
+ if not writer:
+ raise BackupRejected(f"只有写入端能恢复账本:{reason}")
+ backup = resolve(self.live, self.organize, ident)
+ readable = bool(inspect_database(self.live, self.migrations_dir)["readable"])
+ with self.lock:
+ if self._busy():
+ raise BackupRejected("备份或恢复正在进行,请等它完成")
+ busy = update_tasks.running(self.live, task_rows=readable)
+ if busy:
+ raise BackupRejected(f"还有后台任务在运行:{'、'.join(busy)}。请等它们结束或停止后再恢复")
+ job = self.restore_job if readable else self.offline_restore
+ self._restoring = job
+ migrations_dir = self.migrations_dir
+
+ def work(identity):
+ from .config import MIGRATIONS_DIR
+ token = update_tasks.reserve(self.live, busy="正在从备份恢复账本,暂时不能启动新任务。")
+ try:
+ own_files, own_runs = job.registration(identity)
+ late = update_tasks.running(self.live, own_files=own_files, own_runs=own_runs,
+ task_rows=readable)
+ if late:
+ raise BackupRejected(f"还有后台任务在运行:{'、'.join(late)}。没有恢复")
+ outcome = restore(self.live, backup, migrations_dir=migrations_dir or MIGRATIONS_DIR,
+ progress=lambda **values: job.update(identity, **values))
+ finally:
+ update_tasks.release(self.live, token)
+ if self.after_restore is not None:
+ try:
+ self.after_restore()
+ except Exception:
+ log.warning("恢复后清理缓存失败", exc_info=True)
+ job.update(identity, status="complete", stage="恢复完成", backup=backup.name,
+ saved=outcome.saved.name if outcome.saved else "",
+ migrated=len(outcome.migrated))
+
+ job.start(work, restart=True, initial={"stage": "准备恢复", "backup": backup.name,
+ "done": 0, "total": 0})
+ return self.snapshot()
+
+
+_MANAGER_LOCK = threading.Lock()
+
+
+def manager(request) -> LedgerBackups:
+ from . import settings_file
+ with _MANAGER_LOCK:
+ state = request.app.state
+ if not hasattr(state, "ledger_backups"):
+ contract = state.web_contract
+ state.ledger_backups = LedgerBackups(settings_file.load_config(), runs=contract.task_runs,
+ after_restore=contract.cache_bust)
+ return state.ledger_backups
diff --git a/src/peach/migrations.py b/src/peach/migrations.py
index 0b89d9c8b..fb039a169 100644
--- a/src/peach/migrations.py
+++ b/src/peach/migrations.py
@@ -75,14 +75,15 @@ def plan(db_path: Path, directory: Path) -> tuple[list[Migration], list[Migratio
return all_migrations, [m for m in all_migrations if m.version not in done]
-def sqlite_backup(source: Path, destination: Path) -> None:
+def sqlite_backup(source: Path, destination: Path, *, pages: int = -1, progress=None) -> None:
+ """`pages` 与 `progress` 原样交给 `Connection.backup`:分页复制时每页之间放开源库的读锁,并报进度。"""
destination.parent.mkdir(parents=True, exist_ok=True)
if destination.exists():
raise FileExistsError(destination)
src = sqlite3.connect(source)
dst = sqlite3.connect(destination)
try:
- src.backup(dst)
+ src.backup(dst, pages=pages, progress=progress)
finally:
dst.close()
src.close()
diff --git a/src/peach/routes_configuration.py b/src/peach/routes_configuration.py
index b8e6bd8df..a76cadfda 100644
--- a/src/peach/routes_configuration.py
+++ b/src/peach/routes_configuration.py
@@ -24,8 +24,9 @@ from fastapi import APIRouter, Body, Depends, HTTPException, Request
from fastapi.responses import JSONResponse, Response
from filelock import FileLock, Timeout
-from . import (access, data_storage, distribution, entry_links, error_reporting, folder_picker, media_libraries, onboarding,
- push_discovery, settings_file, media_configuration, tunnel, web_downloads)
+from . import (access, data_storage, distribution, entry_links, error_reporting, folder_picker, ledger_backups,
+ media_libraries, onboarding, push_discovery, settings_file, media_configuration, tunnel, web_downloads)
+from .jobs import JobAlreadyRunning
from .routes_auth import require_auth, require_authenticated_client, same_origin
from .web_entry import runtime_fact_entries
from .web_local_access import local_client, managed_configuration
@@ -721,6 +722,42 @@ def relocate_storage(request: Request, body: dict[str, Any] = Body(default_facto
raise HTTPException(409, "迁移请求正在处理") from exc
+def _ledger_backups(request: Request):
+ _storage_config()
+ return ledger_backups.manager(request)
+
+
[email protected]("/api/configuration/ledger-backups")
+def read_ledger_backups(request: Request, _args=Depends(require_auth)):
+ """「账本备份」:可恢复的备份列表、立即备份与恢复两个任务的状态、上次恢复的结果。"""
+ local_only(request)
+ return _ledger_backups(request).snapshot()
+
+
[email protected]("/api/configuration/ledger-backups/backup")
+def backup_ledger(request: Request, _args=Depends(require_auth)):
+ local_only(request)
+ same_origin(request)
+ try:
+ return _ledger_backups(request).start_backup()
+ except (ValueError, JobAlreadyRunning) as exc:
+ raise HTTPException(409, str(exc)) from exc
+
+
[email protected]("/api/configuration/ledger-backups/restore")
+def restore_ledger(request: Request, body: dict[str, Any] = Body(default_factory=dict),
+ _args=Depends(require_auth)):
+ """用选中的备份替换当前账本;页面二次确认后才发,`confirm` 必须为真。"""
+ local_only(request)
+ same_origin(request)
+ if body.get("confirm") is not True:
+ raise HTTPException(400, "恢复会替换当前账本,请先确认")
+ try:
+ return _ledger_backups(request).start_restore(body.get("id"))
+ except (ValueError, JobAlreadyRunning) as exc:
+ raise HTTPException(409, str(exc)) from exc
+
+
@router.get("/api/configuration/updates")
def check_updates(request: Request, _args=Depends(require_auth)):
local_only(request)
diff --git a/src/peach/task_runs.py b/src/peach/task_runs.py
index 864ebebf1..0d5d7f049 100644
--- a/src/peach/task_runs.py
+++ b/src/peach/task_runs.py
@@ -96,6 +96,8 @@ TASK_LABELS = {
"mp4-header-repair": "播放文件头修复",
'jav-discovery-metadata': '发现页资料读取',
'diagnostic-export': '诊断导出',
+ "ledger-backup": "账本备份",
+ "ledger-restore": "从备份恢复账本",
"upgrade-repairs": "自动修复",
"upgrade-library": "资料自动修复",
"organize": "按模板整理",
diff --git a/src/peach/update_tasks.py b/src/peach/update_tasks.py
index 1d6a2c689..05e8d2e83 100644
--- a/src/peach/update_tasks.py
+++ b/src/peach/update_tasks.py
@@ -74,8 +74,8 @@ def admission(db: Path):
"""持锁完成任务登记;安装取得预约后不接受新任务。"""
root = state_for(db)
with _mutex(root):
- if _maintenance(root):
- raise UpdateBusy("正在准备安装更新,暂时不能启动新任务。请等待安装完成。")
+ if data := _maintenance(root):
+ raise UpdateBusy(data.get("busy") or "正在准备安装更新,暂时不能启动新任务。请等待安装完成。")
yield
@@ -99,15 +99,19 @@ def activity(db: Path, label: str):
path.unlink(missing_ok=True)
-def reserve(db: Path, *, owner: int | None = None) -> str:
- """接受安装预约并通知所有服务保存进度、暂停任务。"""
+def reserve(db: Path, *, owner: int | None = None, busy: str | None = None) -> str:
+ """接受安装预约并通知所有服务保存进度、暂停任务。
+
+ `busy` 是预约期间拒绝新任务时给人看的那句话;账本恢复也借这把闸门挡新任务。
+ """
root = state_for(db)
with _mutex(root):
- if _maintenance(root):
- raise UpdateBusy("已有更新安装正在进行,请等待完成。")
+ if data := _maintenance(root):
+ raise UpdateBusy(data.get("busy") or "已有更新安装正在进行,请等待完成。")
token = uuid.uuid4().hex
atomic_write_text(root / "install.json", json.dumps({
- "pid": owner if owner is not None else os.getpid(), "token": token}))
+ "pid": owner if owner is not None else os.getpid(), "token": token,
+ **({"busy": busy} if busy else {})}, ensure_ascii=False))
from .job_recovery import request_pause
request_pause(db)
return token
@@ -125,23 +129,37 @@ def check_pause(db: Path | None) -> None:
raise JobPaused('任务已暂停,安装完成后自动继续。')
-def _running(db: Path) -> list[str]:
+def running(db: Path, *, own_files=(), own_runs=(), task_rows: bool = True) -> list[str]:
+ """数据目录里还在跑的任务,给人看的名字。
+
+ `own_files`/`own_runs` 是调用方自己那一轮的登记文件与 `task_run` 行,不算在内。
+ `task_rows` 为假时不读 `task_run`:账本坏了读不出来,那时只看登记文件与锁。
+ """
+ return _running(db, own_files=own_files, own_runs=own_runs, task_rows=task_rows)
+
+
+def _running(db: Path, *, own_files=(), own_runs=(), task_rows: bool = True) -> list[str]:
from .jobs import process_alive
from .runtime_prepare import task_gate
root = state_for(db)
+ own = {Path(path).name for path in own_files}
with _mutex(root):
labels = []
for path in (root / "active").glob("*.json"):
+ if path.name in own:
+ continue
data = _read(path)
if _alive(data):
labels.append(str(data.get("label") or "后台任务"))
else:
path.unlink(missing_ok=True)
- gate = task_gate(db, host=socket.gethostname(), alive=process_alive)
- if gate.error:
- labels.append('任务状态未取得')
- labels.extend(run.describe() for run in (*gate.blocking, *gate.resumable)
- if run.status == "running" and (run.pid is None or process_alive(run.pid)))
+ if task_rows:
+ gate = task_gate(db, host=socket.gethostname(), alive=process_alive)
+ if gate.error:
+ labels.append('任务状态未取得')
+ labels.extend(run.describe() for run in (*gate.blocking, *gate.resumable)
+ if run.status == "running" and run.id not in own_runs
+ and (run.pid is None or process_alive(run.pid)))
# 旧版 CLI 已有 PID 锁,预检同时兼容这些仍在运行的批次。
legacy = [root.parent / name for name in ('.probe.lock', '.sheets.lock', '.creator-boards.lock')]
for path in legacy:
diff --git a/tests/test_ledger_backups.py b/tests/test_ledger_backups.py
index 7a596a7ba..763a09e85 100644
--- a/tests/test_ledger_backups.py
+++ b/tests/test_ledger_backups.py
@@ -3,14 +3,18 @@ import importlib.util
import io
import json
import os
+import shutil
import sqlite3
import tempfile
import unittest
from datetime import datetime, timedelta
from pathlib import Path
from types import SimpleNamespace
+from unittest.mock import patch
-from peach import ledger_backups, web_organize
+from peach import ledger_backups, migrations, settings_file, update_tasks, web_organize
+from peach.task_runs import TaskRunStore
+from support.ledger import MIGRATIONS_DIR, fresh_ledger
ROOT = Path(__file__).resolve().parents[1]
@@ -333,5 +337,301 @@ class OrganizeBackupRetentionTests(unittest.TestCase):
self.assertTrue(path.exists(), "缺省是 dry-run")
+def note(db: Path, value: str) -> None:
+ """往账本里记一笔可辨认的内容,用来分辨恢复后是哪一份。"""
+ with contextlib.closing(sqlite3.connect(db)) as connection:
+ connection.execute("CREATE TABLE IF NOT EXISTS probe(v TEXT)")
+ connection.execute("INSERT INTO probe VALUES (?)", (value,))
+ connection.commit()
+
+
+def notes(db: Path) -> list[str]:
+ with contextlib.closing(sqlite3.connect(db)) as connection:
+ return [row[0] for row in connection.execute("SELECT v FROM probe ORDER BY rowid")]
+
+
+def task_row(db: Path, task_key: str, status: str = "succeeded") -> int:
+ with contextlib.closing(sqlite3.connect(db)) as connection:
+ finished = None if status in ("pending", "running") else "2026-11-04T00:00:00+00:00"
+ cursor = connection.execute(
+ "INSERT INTO task_run(task_key, trigger, status, finished_at) VALUES (?, 'manual', ?, ?)",
+ (task_key, status, finished))
+ connection.commit()
+ return int(cursor.lastrowid)
+
+
+class LedgerRestoreTests(unittest.TestCase):
+ def setUp(self):
+ temporary = tempfile.TemporaryDirectory()
+ self.addCleanup(temporary.cleanup)
+ self.root = Path(temporary.name).resolve()
+ self.database = self.root / "database"
+ self.database.mkdir()
+ self.live = fresh_ledger(self.database)
+ note(self.live, "before")
+
+ def restore(self, backup: Path, **kwargs):
+ return ledger_backups.restore(self.live, backup, migrations_dir=MIGRATIONS_DIR, **kwargs)
+
+ def test_manual_backup_is_checked_before_it_gets_a_backup_name(self):
+ seen = []
+ backup = ledger_backups.create(self.live, progress=lambda **values: seen.append(values))
+
+ self.assertRegex(backup.name, r"^ledger\.pre-manual-\d{8}-\d{6}\.db$")
+ self.assertTrue(ledger_backups.BACKUP_PATTERN.match(backup.name), "和自动备份同一套清退规则")
+ self.assertEqual(notes(backup), ["before"])
+ self.assertEqual(seen[-1], {"stage": "检查备份完整性"})
+ copied = [values for values in seen if values.get("stage") == "复制账本"]
+ self.assertTrue(copied and copied[-1]["done"] == copied[-1]["total"] > 0)
+ self.assertEqual([path.name for path in self.database.iterdir() if path.name.endswith(".part")], [])
+
+ def test_backup_failing_integrity_check_is_discarded(self):
+ original = ledger_backups.integrity
+ with patch.object(ledger_backups, "integrity",
+ side_effect=lambda db: "page 3 is never used" if db.name.endswith(".part") else original(db)):
+ with self.assertRaisesRegex(ledger_backups.BackupRejected, "完整性"):
+ ledger_backups.create(self.live)
+
+ self.assertEqual(sorted(path.name for path in self.database.iterdir()), ["ledger.db"])
+
+ def test_catalog_tells_manual_from_automatic_and_resolve_only_takes_listed_names(self):
+ organize = self.root / "generated" / "organize-backups"
+ organize.mkdir(parents=True)
+ auto = self.database / "ledger.pre-migrate-20261101-000000.db"
+ shutil.copyfile(self.live, auto)
+ stamp(auto, datetime.now() - timedelta(days=2))
+ manual = ledger_backups.create(self.live)
+ organized = organize / "ledger.pre-organize-20261102-000000-1.db"
+ shutil.copyfile(self.live, organized)
+ stamp(organized, datetime.now() - timedelta(days=1))
+
+ listed = ledger_backups.catalog(self.live, organize)
+
+ self.assertEqual([(entry["id"], entry["source"], entry["purpose"]) for entry in listed], [
+ (f"database/{manual.name}", "manual", "手动备份"),
+ ("organize/ledger.pre-organize-20261102-000000-1.db", "auto", "整理前"),
+ ("database/ledger.pre-migrate-20261101-000000.db", "auto", "迁移前"),
+ ])
+ self.assertEqual(ledger_backups.resolve(self.live, organize, listed[1]["id"]), organized)
+ for wrong in ("database/ledger.db", "database/../ledger.pre-x.db", "elsewhere/" + manual.name,
+ "organize/" + manual.name, None):
+ with self.assertRaises(ledger_backups.BackupRejected, msg=wrong):
+ ledger_backups.resolve(self.live, organize, wrong)
+
+ def test_restore_saves_the_current_ledger_first_and_keeps_task_history(self):
+ backup = ledger_backups.create(self.live)
+ note(self.live, "after")
+ running = task_row(self.live, "ledger-restore", "running")
+ with contextlib.closing(sqlite3.connect(self.live)) as connection:
+ connection.execute("PRAGMA journal_mode=WAL")
+ connection.execute("UPDATE ledger_revision SET n = 50 WHERE tbl = 'asset'")
+ connection.commit()
+
+ outcome = self.restore(backup)
+
+ self.assertEqual(notes(self.live), ["before"])
+ with contextlib.closing(sqlite3.connect(self.live)) as connection:
+ self.assertEqual(connection.execute("PRAGMA journal_mode").fetchone(), ("wal",), "账本保留自己的日志模式")
+ self.assertEqual(sorted(path.name for path in self.database.glob(outcome.saved.name + "*")),
+ [outcome.saved.name], "另存的备份是单个文件,旁边没有 -wal/-shm")
+ self.assertEqual(outcome.saved.parent, self.database)
+ self.assertRegex(outcome.saved.name, r"^ledger\.pre-restore-")
+ self.assertEqual(notes(outcome.saved), ["before", "after"], "恢复前的账本完整另存")
+ self.assertEqual(outcome.migrated, ())
+ with contextlib.closing(sqlite3.connect(self.live)) as connection:
+ self.assertEqual(connection.execute("SELECT status FROM task_run WHERE id = ?", (running,)).fetchone(),
+ ("running",), "这次恢复自己那一行还在,结算才有地方写")
+ self.assertEqual(connection.execute("SELECT n FROM ledger_revision WHERE tbl = 'asset'").fetchone(),
+ (51,), "版本号只增不减,聚合缓存不会撞上旧值")
+ self.assertFalse((self.database / ledger_backups.MARKER_NAME).exists())
+ self.assertFalse((self.database / ledger_backups.STAGING_NAME).exists())
+ self.assertEqual(ledger_backups.integrity(self.live), "ok")
+
+ def test_restore_refuses_a_damaged_backup_without_touching_the_ledger(self):
+ damaged = self.database / "ledger.pre-manual-20261101-000000.db"
+ damaged.write_bytes(b"SQLite format 3\x00" + b"\x00" * 200)
+
+ with self.assertRaisesRegex(ledger_backups.BackupRejected, "完整性"):
+ self.restore(damaged)
+
+ self.assertEqual(notes(self.live), ["before"])
+ self.assertEqual(sorted(path.name for path in self.database.iterdir()), sorted(["ledger.db", damaged.name]),
+ "没有另存、没有标记、没有暂存")
+
+ def test_restore_migrates_an_older_backup_before_it_reaches_the_ledger(self):
+ older = self.root / "older-migrations"
+ older.mkdir()
+ files = sorted(MIGRATIONS_DIR.glob("*.sql"))
+ for path in files[:-1]:
+ shutil.copyfile(path, older / path.name)
+ backup = self.database / "ledger.pre-migrate-20261101-000000.db"
+ migrations.upgrade(backup, older)
+ note(backup, "older")
+
+ outcome = self.restore(backup)
+
+ self.assertEqual(outcome.migrated, (files[-1].name[:4],))
+ self.assertEqual(notes(self.live), ["older"])
+ _, pending = migrations.plan(self.live, MIGRATIONS_DIR)
+ self.assertEqual(pending, [])
+
+ def test_restore_refuses_a_backup_from_a_newer_program(self):
+ backup = ledger_backups.create(self.live)
+ with contextlib.closing(sqlite3.connect(backup)) as connection:
+ connection.execute("INSERT INTO schema_migration(version, name, checksum, applied_at) VALUES ('9999', 'future', 'x', '')")
+ connection.commit()
+ note(self.live, "after")
+
+ with self.assertRaisesRegex(ledger_backups.BackupRejected, "结构"):
+ self.restore(backup)
+
+ self.assertEqual(notes(self.live), ["before", "after"])
+ self.assertEqual(len(list(self.database.glob("ledger.pre-restore-*"))), 0)
+
+ def test_failure_while_writing_leaves_the_ledger_as_it_was(self):
+ backup = ledger_backups.create(self.live)
+ note(self.live, "after")
+
+ with patch.object(ledger_backups, "_copy_into", side_effect=sqlite3.OperationalError("database is locked")):
+ with self.assertRaisesRegex(ledger_backups.BackupRejected, "保持不变"):
+ self.restore(backup)
+
+ self.assertEqual(notes(self.live), ["before", "after"])
+ self.assertFalse((self.database / ledger_backups.MARKER_NAME).exists())
+ self.assertEqual(ledger_backups.read_result(self.live)["state"], "failed")
+
+ def interrupted(self, *, saved: Path | None, staging: bool) -> None:
+ if staging:
+ shutil.copyfile(self.live, self.database / ledger_backups.STAGING_NAME)
+ note(self.database / ledger_backups.STAGING_NAME, "staged")
+ (self.database / ledger_backups.MARKER_NAME).write_text(json.dumps({
+ "state": "replacing", "backup": str(self.database / "ledger.pre-manual-1.db"),
+ "saved": str(saved) if saved else ""}), encoding="utf-8")
+
+ def test_startup_keeps_an_intact_ledger_after_an_interrupted_restore(self):
+ self.assertIsNone(ledger_backups.recover(self.live), "没有标记什么都不做")
+ self.interrupted(saved=None, staging=True)
+
+ result = ledger_backups.recover(self.live)
+
+ self.assertEqual((result["state"], result["outcome"]), ("interrupted", "kept"))
+ self.assertEqual(notes(self.live), ["before"])
+ self.assertFalse((self.database / ledger_backups.MARKER_NAME).exists())
+ self.assertFalse((self.database / ledger_backups.STAGING_NAME).exists())
+ self.assertEqual(ledger_backups.read_result(self.live)["outcome"], "kept")
+
+ def test_startup_never_leaves_the_ledger_missing_after_an_interrupted_restore(self):
+ saved = self.database / "ledger.pre-restore-20261104-000000.db"
+ shutil.copyfile(self.live, saved)
+ self.interrupted(saved=saved, staging=True)
+ self.live.unlink()
+
+ self.assertEqual(ledger_backups.recover(self.live)["outcome"], "completed")
+ self.assertEqual(notes(self.live), ["before", "staged"], "迁好的暂存补完这次恢复")
+
+ self.interrupted(saved=saved, staging=False)
+ self.live.unlink()
+ self.assertEqual(ledger_backups.recover(self.live)["outcome"], "rolled-back")
+ self.assertEqual(notes(self.live), ["before"], "没有暂存就退回恢复前另存的那份")
+
+ def test_migrate_upgrade_settles_the_marker_before_creating_anything(self):
+ saved = self.database / "ledger.pre-restore-20261104-000000.db"
+ shutil.copyfile(self.live, saved)
+ self.interrupted(saved=saved, staging=False)
+ self.live.unlink()
+ from peach import cli
+ with patch.object(cli, "_require_readable_settings"), contextlib.redirect_stdout(io.StringIO()):
+ cli._migrate(SimpleNamespace(action="upgrade", db=self.live, yes=True))
+
+ self.assertEqual(notes(self.live), ["before"])
+
+
+class LedgerBackupManagerTests(unittest.TestCase):
+ def setUp(self):
+ temporary = tempfile.TemporaryDirectory()
+ self.addCleanup(temporary.cleanup)
+ self.root = Path(temporary.name).resolve()
+ (self.root / "config.toml").write_text("", encoding="utf-8")
+ (self.root / "database").mkdir()
+ self.live = fresh_ledger(self.root / "database")
+ note(self.live, "before")
+ self.config = settings_file.load_config(environ={settings_file.DATA_ROOT_ENV: str(self.root)})
+ self.runs = TaskRunStore(self.live)
+ self.busted = []
+ self.manager = ledger_backups.LedgerBackups(self.config, runs=self.runs, migrations_dir=MIGRATIONS_DIR,
+ after_restore=lambda: self.busted.append(True))
+
+ def finish(self, job) -> dict:
+ job.thread.join(30)
+ self.assertFalse(job.thread.is_alive())
+ return job.snapshot()
+
+ def finished_runs(self, task_key: str) -> list[str]:
+ with contextlib.closing(sqlite3.connect(self.live)) as connection:
+ return [row[0] for row in connection.execute(
+ "SELECT status FROM task_run WHERE task_key = ? ORDER BY id", (task_key,))]
+
+ def test_backup_and_restore_run_as_background_tasks_in_the_task_center(self):
+ self.manager.start_backup()
+ state = self.finish(self.manager.backup_job)
+ self.assertEqual(state["status"], "complete", state)
+ listed = self.manager.snapshot()["backups"]
+ self.assertEqual([entry["name"] for entry in listed], [state["name"]])
+ note(self.live, "after")
+
+ self.manager.start_restore(listed[0]["id"])
+ state = self.finish(self.manager.restore_job)
+
+ self.assertEqual(state["status"], "complete", state)
+ self.assertEqual(notes(self.live), ["before"])
+ self.assertEqual(self.finished_runs("ledger-backup"), ["succeeded"])
+ self.assertEqual(self.finished_runs("ledger-restore"), ["succeeded"],
+ "自己那一行接回恢复后的账本并正常结算;查忙时没把自己算进去")
+ self.assertEqual(self.busted, [True])
+ self.assertFalse(update_tasks.is_pausing(self.live), "挡新任务的闸门已放开")
+ self.assertEqual(notes(self.root / "database" / state["saved"]), ["before", "after"])
+
+ def test_restore_is_refused_while_other_tasks_run(self):
+ backup = ledger_backups.create(self.live)
+ with patch.object(update_tasks, "running", return_value=["扫描与采集"]):
+ with self.assertRaisesRegex(ledger_backups.BackupRejected, "扫描与采集"):
+ self.manager.start_restore(f"database/{backup.name}")
+ self.assertEqual(self.manager.snapshot()["restore"], {"status": "idle"})
+
+ def test_restore_is_refused_on_a_reading_device(self):
+ backup = ledger_backups.create(self.live)
+ with patch("peach.runtime_prepare.configured_ledger_writer", return_value=(False, "这台电脑是只读端")):
+ with self.assertRaisesRegex(ledger_backups.BackupRejected, "写入端"):
+ self.manager.start_restore(f"database/{backup.name}")
+
+ def test_new_tasks_are_held_off_while_restoring(self):
+ token = update_tasks.reserve(self.live, busy="正在从备份恢复账本,暂时不能启动新任务。")
+ self.addCleanup(update_tasks.release, self.live, token)
+ with self.assertRaisesRegex(update_tasks.UpdateBusy, "正在从备份恢复账本"):
+ with update_tasks.admission(self.live):
+ pass
+
+ def test_routes_require_confirmation_and_report_refusals(self):
+ from fastapi import FastAPI
+ from fastapi.testclient import TestClient
+ from peach import routes_configuration
+ from peach.routes_auth import require_auth
+ app = FastAPI()
+ app.include_router(routes_configuration.router)
+ app.dependency_overrides[require_auth] = lambda: None
+ app.state.ledger_backups = self.manager
+ with TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 123)) as client, \
+ patch.object(settings_file, "load_config", return_value=self.config):
+ self.assertEqual(client.get("/api/configuration/ledger-backups").json()["backups"], [])
+ unconfirmed = client.post("/api/configuration/ledger-backups/restore", json={"id": "database/x"})
+ self.assertEqual(unconfirmed.status_code, 400)
+ missing = client.post("/api/configuration/ledger-backups/restore",
+ json={"id": "database/ledger.pre-manual-1.db", "confirm": True})
+ self.assertEqual(missing.status_code, 409)
+ self.assertIn("已经不在", missing.json()["detail"])
+ self.assertEqual(client.post("/api/configuration/ledger-backups/backup",
+ headers={"origin": "https://other.invalid"}).status_code, 403)
+
+
if __name__ == "__main__":
unittest.main()