Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

fix(packaging): 接通随包采集环境并隔离系统依赖

longmeidaoa1ff00051d5f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a1d9aa956..aec596dd9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -21,6 +21,10 @@ Removed / Fixed / Security。
## [未发布]
+### 修复
+
+- **采集**:Windows、macOS 与 Docker 随包分发刮削运行环境,使用者无需另装 uv、Python 或 Git;安装包缺少采集工具或版本不匹配时提示重新安装。
+
## [0.48.2] - 2026-10-24
已安装的 Windows 版请手动下载并运行一次完整安装包:人脸模型改随安装包分发,旧版自动更新器跨不过这次升级,之后照常自动更新。
diff --git a/README.en.md b/README.en.md
index bf3819d08..3fe79bede 100644
--- a/README.en.md
+++ b/README.en.md
@@ -96,6 +96,8 @@ Try search, filters and profiles in the [read-only demo](https://demo.peach.vide
The full installer includes media tools for offline installation. The smaller lite installer downloads tools as needed during setup.
+Windows and macOS packages include the scraping runtime; users do not need to install uv, Python, or Git. Reinstall Peach if the scraping tools are missing or their version does not match.
+
Test builds are unsigned. If Windows says “Windows protected your PC”, confirm the file came from this project's Releases, then choose “More info → Run anyway”. See the [Windows guide](docs/TESTING_DESKTOP.md) for installation, updates and troubleshooting.
### Build on macOS
diff --git a/README.md b/README.md
index 4dfa19df3..52d35c87a 100644
--- a/README.md
+++ b/README.md
@@ -96,6 +96,8 @@ https://github.com/user-attachments/assets/a5610874-e611-41bb-be22-7d99f0d64b62
完整安装包携带媒体工具,可离线安装;轻量安装包体积更小,安装时按需下载工具。
+Windows 与 macOS 安装包均自带采集运行环境,使用者无需另装 uv、Python 或 Git。采集工具缺失或版本不匹配时,请重新安装 Peach。
+
测试版尚未签名。若 Windows 提示「已保护你的电脑」,确认来自本项目的 Release,再点「更多信息 → 仍要运行」。安装、更新和故障排查见 [Windows 使用说明](docs/TESTING_DESKTOP.md)。
### macOS 本机构建
diff --git a/docs/REUSE.md b/docs/REUSE.md
index f94fbe0a5..e44813023 100644
--- a/docs/REUSE.md
+++ b/docs/REUSE.md
@@ -724,7 +724,7 @@ A 盘垃圾隔离使用短名称 `peach-purge-<32 位 UUID>.peach-quarantine`,
| 厂牌字标名录 | 发行商与平台自己的厂牌名录,入口在 `harvest_maker_directories.DIRECTORIES`:MGStage `/ppv/makers.php`、Prestige `/api/maker`、KMP `/label`(大半是 SVG)与 jae.tokyo 展会名录 | slug↔账本对账(四路判据,空罗马字形不可比)、按形状分三张指定表(方标装大位、字标烤方两位共用、方标只管小位)、改地址后靠 provenance 边车重新收人、复核 CSV 与安装闸门。存入口不存图片地址(KMP 文件名带时间戳),不推导 URL、不猜名字 |
| JAV 元数据查询 | 每个站只有一个归属(ADR-0048):自写解析器持有 r18.dev、DMM/FANZA、一本道、FC2、fc2cmadb、FC2PPV-DB、JAVten、JavArchive、AVBase、JavBus、javdb,片商官网与转载站经 amane 桥(ADR-0043、ADR-0044);MetaTube SDK `6a5e6128c725187aeaf921d48ed7d9cd9f30671b`(Apache-2.0)只作来源身份与丰富字段模型参考;DMM 的 GraphQL 接口地址、`ppvContent` 与 `legacySearchPPV` 两条查询的取法参照 OpenAver(MIT)`core/scrapers/dmm.py`,cid 匹配与身份核对是 Peach 自己的(ADR-0059) | 只发规范番号;Peach 管来源链(`metadata_routes`)、`provider_id`/`content_id`、逐字段优先级、原始与目录证据、健康统计、候选复核与 ledger 投影。amane 的 POC 判据与字段缺口见 [取证](reference-snapshots/amane-crawlers-poc.md) |
| Javinizer-Go 历史快照 | `sources/metadata/javinizer-go/<番号>/<来源>.json`,由 Javinizer-Go v1.5.x(MIT,`dd56998328d078c9baf68ff4fde2e6fcaa2a691a`)在 2026-09 之前取回;二进制与快照留在磁盘,没有调用路径(ADR-0044) | 只作离线证据:封面层读它的 `cover_url` 与 `content_id`,别名解析读企划名义,账本里 `javinizer:<站>:<字段>` 的 provenance 按 `metadata_policy.HISTORICAL_SOURCES` 认级别。`scrape_codes` 写进同一目录的新快照 `provider` 记解析器名、`provider_version` 记 Peach 版本 |
-| amane 刮削站点(官方档 makers、prestige、faleno、dahlia、mgstage;转载站 fc2club、freejavbt、airav、avsox) | amane `cc42cee6ecf290cf05af1caa347d2dc2d5aeffcd`(v0.20.0,GPL-3.0),经 `tools/amane-bridge/` 薄桥子进程接入(ADR-0043、ADR-0048;`makers` 是 amane 的 `official` 模块):独立 `pyproject.toml`/`uv.lock` 钉 sha,venv 建在 `<Peach 数据目录>/tools/amane-bridge/.venv`,桥只用 `amane.crawlers.sites.<站>` 与 `amane.net.*`,不碰 `aggregate` | GPL-3.0 与 Peach 的 AGPL-3.0-or-later 以进程边界相接,仓库与分发件只含清单、锁与桥脚本,amane 源码由 uv 按锁下载。字段、失败分档与无写入 POC 见 [上游复核](reference-snapshots/upstream-review.md)。Prestige 封套与正封分别投影封面和海报,配信日不作发行日;AIRAV 不解析试看时长。上游 `WebClient` 以 `verify=False` 发公开请求、不带凭据;403/406 可另轮换三次指纹,Peach 保持 60 秒子进程总时限。Peach 管来源链位置、身份核对、冷却、候选与结算 |
+| amane 刮削站点(官方档 makers、prestige、faleno、dahlia、mgstage;转载站 fc2club、freejavbt、airav、avsox) | amane `cc42cee6ecf290cf05af1caa347d2dc2d5aeffcd`(v0.20.0,GPL-3.0),经 `tools/amane-bridge/` 薄桥子进程接入(ADR-0043、ADR-0048;`makers` 是 amane 的 `official` 模块):独立 `pyproject.toml`/`uv.lock` 钉 sha,桌面包通过主锁固定的 PyInstaller 分发独立可执行文件,Docker 携带独立 venv;源码部署的 venv 建在 `<Peach 数据目录>/tools/amane-bridge/.venv`。桥只用 `amane.crawlers.sites.<站>` 与 `amane.net.*`,不碰 `aggregate` | GPL-3.0 与 Peach 的 AGPL-3.0-or-later 以进程边界相接;随包附许可证与完整对应源码。运行时核对版本、revision 和程序路径,构建与制品验收清空 PATH 后离线加载全部站点类;安装包缺失或不匹配时要求重新安装。字段、失败分档与无写入 POC 见 [上游复核](reference-snapshots/upstream-review.md)。Prestige 封套与正封分别投影封面和海报,配信日不作发行日;AIRAV 不解析试看时长。上游 `WebClient` 以 `verify=False` 发公开请求、不带凭据;403/406 可另轮换三次指纹,Peach 保持 60 秒子进程总时限。Peach 管来源链位置、身份核对、冷却、候选与结算 |
| 已确认厂牌的目录归位 | Javinizer-Go v1.5.2 organizer(MIT)只作冲突预检、模板化目录和回滚边界的协议参考,不调用它,不让它持有 Peach ledger | `rehome_unknown_jav.py` 只消费人工确认映射;先出逐文件 CSV,拒绝扁平化重名与厂牌冲突,SQLite 备份后移动文件并同步 Peach 路径/实体 provenance |
| FC2 目录元数据与跨号证据 | 已缓存的 fc2cmadb Inertia `article`/评论;Javinizer-Go v1.5.2 的 FC2 解析器(MIT)只作商品页字段边界参考 | 旧文章仍有标题、原始标签、日期、时长、卖家、FC2 CDN 封面与 `comments`(2026-08-31 登录态实测);无歧义标签译成现有词表,标题/标签进 `/review`,`w1200` 封面过尺寸与解码门槛落生成产物;pair、合集/分片保护、hash/时长/尺寸佐证、库外 evidence、健康统计与复核归 Peach,不依赖 FC2-Leak-Detector/JavSP,镜像候选不直写 ledger |
| 缺索引 MP4 重建 | untrunc(anthwlock `9d86ec9`,GPL-2.0-or-later)与兼容的 FFmpeg 8.1.3 静态构建,固定来源及哈希见 `scripts/untrunc-sources.json`。Windows 安装包与免安装包携带 `tools/untrunc`,同一 Release 提供完整对应源码;原生样片实测覆盖尾部 `moov` 缺失后的 H.264/AAC 恢复、时长与全片解码。显式 `PEACH_UNTRUNC` 优先,其后依次查随包目录、数据工具与 PATH | `src/peach/mp4recover.py` 挑参照(同目录文件名最近的先试,最多 4 部)、验收(解码错误 ≤3 行、音视频时长差 ≤5%)、换回原路径并把坏原件改名成同目录的 `.文件名.peach-original`;缺整个 `moov` 的片子哪里都打不开,所以结果替换原文件,不像缺 `ctts` 那样另存头。实测见 [取证](SOURCING.md#媒体与关注实测) |
diff --git a/docs/adr/0043-amane-sites-via-thin-bridge-subprocess.md b/docs/adr/0043-amane-sites-via-thin-bridge-subprocess.md
index 3ba53243c..80557aff6 100644
--- a/docs/adr/0043-amane-sites-via-thin-bridge-subprocess.md
+++ b/docs/adr/0043-amane-sites-via-thin-bridge-subprocess.md
@@ -1,6 +1,6 @@
# ADR-0043:amane 刮削站点以薄桥子进程接入
-- 状态:Accepted
+- 状态:Accepted;桌面与 Docker 的决策三由 [ADR-0120](0120-packaged-scraping-runtime.md) 取代。
- 日期:2026-09-22
## 背景
diff --git a/docs/adr/0120-packaged-scraping-runtime.md b/docs/adr/0120-packaged-scraping-runtime.md
new file mode 100644
index 000000000..c359292b5
--- /dev/null
+++ b/docs/adr/0120-packaged-scraping-runtime.md
@@ -0,0 +1,21 @@
+# ADR-0120:安装包携带独立采集运行环境
+
+- 状态:Accepted
+- 日期:2026-10-24
+- 取代:[ADR-0043](0043-amane-sites-via-thin-bridge-subprocess.md) 的决策三在桌面与 Docker 分发中的运行环境定位;源码部署仍使用数据目录的独立 venv。
+
+## 背景
+
+桌面应用和 Docker 面向直接安装使用。采集来源若在运行时依赖用户的 uv、Python、Git 或开发工具 PATH,应用即使能够启动,也可能无法取得资料。桌面进程继承的 PATH 还可能不包含终端中的 Homebrew 路径。
+
+## 决策
+
+1. 保留 ADR-0043 的子进程边界、来源链和独立依赖锁。Windows 与 macOS 通过主锁固定的 PyInstaller 构建独立 console 桥,Docker 在构建阶段准备独立 venv。
+2. 所有桌面应用制品均保留完整采集运行环境,包括轻量安装包和应用更新包。版本、完整 revision 与程序相对路径写入随包清单;许可证、固定版本完整对应源码和依赖许可证一起分发。
+3. 运行时优先使用匹配清单的随包环境。桌面或 Docker 的运行环境缺失、越界或不匹配时明确要求重新安装,不调用系统 uv,也不借用数据目录中的旧环境。
+4. 桥子进程清除父应用的 Python 与 PyInstaller 内部环境,独立可执行文件按 PyInstaller 的新应用语义启动。
+5. 构建自检清空 PATH,离线加载全部开放站点类。桌面制品验收同时核对运行环境自检与实际服务 API 返回的程序路径、版本和 revision;Docker 在断网容器执行对应检查。
+
+## 后果
+
+用户不需要准备采集依赖。安装包增加独立运行目录,构建时需要网络取得固定依赖与对应源码;来源站点的网络可达性仍由实际连接检查验证。源码部署继续按独立锁准备数据目录中的 venv,不将 amane 并入 Peach 主依赖树。
diff --git a/scripts/build_amane_runtime.py b/scripts/build_amane_runtime.py
index 7cc7e707e..7b6040359 100644
--- a/scripts/build_amane_runtime.py
+++ b/scripts/build_amane_runtime.py
@@ -94,7 +94,7 @@ def verify(destination: Path, *, script: Path = BRIDGE / "bridge.py") -> dict:
env = {key: value for key, value in os.environ.items()
if key not in {"PYTHONPATH", "PYTHONHOME", "VIRTUAL_ENV", "_PYI_APPLICATION_HOME_DIR",
"_PYI_ARCHIVE_FILE", "_PYI_PARENT_PROCESS_LEVEL"}}
- env.update(PYTHONUTF8="1", PYTHONIOENCODING="utf-8", PYINSTALLER_RESET_ENVIRONMENT="1")
+ env.update(PATH="", PYTHONUTF8="1", PYTHONIOENCODING="utf-8", PYINSTALLER_RESET_ENVIRONMENT="1")
result = subprocess.run([*command, "--self-check"], check=True, capture_output=True,
text=True, encoding="utf-8", cwd=destination, env=env, timeout=120)
report = json.loads(result.stdout)
diff --git a/scripts/smoke_desktop.py b/scripts/smoke_desktop.py
index d893c396e..de8044093 100644
--- a/scripts/smoke_desktop.py
+++ b/scripts/smoke_desktop.py
@@ -64,6 +64,19 @@ def assert_public_resources(application: Path) -> None:
assert expected == hashlib.sha256(content.encode('utf-8')).hexdigest(), "公共实体种子摘要不一致"
+def assert_amane_runtime(opener, base: str, runtime: Path) -> None:
+ """空数据目录使用随包采集工具,实际服务返回相同的固定版本与程序路径。"""
+ from build_amane_runtime import verify
+ verify(runtime)
+ manifest = json.loads((runtime / 'manifest.json').read_text(encoding='utf-8'))
+ with opener.open(base + '/api/scraping/amane-bridge', timeout=15) as response:
+ facts = json.load(response)
+ assert facts['installed'] and not facts['problem'], facts
+ assert facts['revision'] == manifest['revision'], facts
+ assert facts['installed_version'] == manifest['version'], facts
+ assert Path(facts['python']).resolve() == (runtime / manifest['executable']).resolve(), facts
+
+
def assert_bundled_media_tools(application: Path) -> None:
"""在没有系统媒体工具的机器上验证随包转码、探测和抽帧。"""
tools = application / "tools" / "ffmpeg"
@@ -442,6 +455,7 @@ def main() -> int:
headers={"Origin": base})
with opener.open(login, timeout=10) as response:
assert response.status == 200
+ assert_amane_runtime(opener, base, executable.parent / '_internal/tools/amane-runtime')
for path in ("/", "/app.css", "/dist/peach-app.js", "/dist/peach-app.css", "/api/items"):
with opener.open(base + path, timeout=10) as response:
assert response.status == 200, path
@@ -465,7 +479,7 @@ def main() -> int:
assert (data / "config.previous.toml").is_file()
assert (data / "state" / "configuration-reload.request").is_file()
print(json.dumps({"ok": True, "version": health.get("version"),
- "checks": ["oobe", "migrations", "lan-bind", "mdns", "password-login", "unauthenticated-api", "pages", "island", "items", "clouddrive-configuration", "cloudflared-sidecar", "bundled-media-tools", "public-resources"]}))
+ "checks": ["oobe", "migrations", "lan-bind", "mdns", "password-login", "unauthenticated-api", "pages", "island", "items", "clouddrive-configuration", "cloudflared-sidecar", "bundled-media-tools", "bundled-amane-runtime", "public-resources"]}))
finally:
process.terminate()
process.wait(timeout=15)
diff --git a/scripts/smoke_macos.py b/scripts/smoke_macos.py
index 8045a37fa..639380e87 100644
--- a/scripts/smoke_macos.py
+++ b/scripts/smoke_macos.py
@@ -14,7 +14,7 @@ import urllib.error
import urllib.parse
import urllib.request
-from smoke_desktop import assert_requires_login, complete_first_run, smoke_environment, wait_health
+from smoke_desktop import assert_amane_runtime, assert_requires_login, complete_first_run, smoke_environment, wait_health
def main() -> int:
@@ -57,6 +57,7 @@ def main() -> int:
headers={"Origin": base})
with opener.open(request, timeout=10) as response:
assert response.status == 200
+ assert_amane_runtime(opener, base, executable.parent.parent / 'Resources/tools/amane-runtime')
for path in ("/", "/dist/peach-app.js", "/dist/peach-app.css", "/api/items",
"/healthz?ready=1"):
with opener.open(base + path, timeout=30) as response:
@@ -75,7 +76,7 @@ def main() -> int:
assert (data / "config.previous.toml").is_file()
print(json.dumps({"ok": True, "version": health.get("version"),
"checks": ["relocated-app", "no-development-tools", "first-run", "migrations",
- "standalone-ledger", "login", "frontend", "configuration-write"]}))
+ "standalone-ledger", "login", "frontend", "bundled-amane-runtime", "configuration-write"]}))
except BaseException as error:
if isinstance(error, urllib.error.HTTPError):
print(error.read().decode("utf-8", "replace"))
diff --git a/scripts/test_runner.py b/scripts/test_runner.py
index 3f5f3698b..23e874c30 100644
--- a/scripts/test_runner.py
+++ b/scripts/test_runner.py
@@ -310,7 +310,7 @@ AUTO_SCOPE_PREFIXES: tuple[tuple[str, str | tuple[str, ...]], ...] = (
("scripts/merge_studio_name_variants.py", ("metadata", "tooling")),
("scripts/harvest_", ("metadata", "tooling")),
# amane 桥(ADR-0043):脚本由 metadata 域的单测装载,清单与锁由 checks 域的依赖策略核。
- ("tools/amane-bridge/", ("metadata", "checks")),
+ ("tools/amane-bridge/", ("metadata", "packaging", "checks")),
("scripts/sync_brand_marks.py", ("catalog", "tooling")),
("scripts/detect_cover_faces.py", ("web", "tooling")),
("scripts/demo_dataset.py", ("web", "tooling")),
diff --git a/src/peach/metadata_amane.py b/src/peach/metadata_amane.py
index c734a4282..c2dd1e826 100644
--- a/src/peach/metadata_amane.py
+++ b/src/peach/metadata_amane.py
@@ -1,8 +1,8 @@
"""amane 刮削站点的 Peach 侧接入:起桥子进程、读一行 JSON、把结果翻成 Peach 候选(ADR-0043)。
桥本体在 `tools/amane-bridge/bridge.py`,运行环境由同目录的 `pyproject.toml` / `uv.lock`
-钉死,venv 建在数据目录 `<tools>/amane-bridge/.venv`,不进 Peach 主 venv。这里只做三件事:
-找到 venv 里的解释器并起子进程;把 amane 的 `MediaMetadata` 套进 `peach.sources` 契约的
+钉死。桌面与 Docker 使用随包独立运行环境;源码部署的 venv 建在数据目录的工具区。
+本模块定位运行环境并起子进程,把 amane 的 `MediaMetadata` 套进 `peach.sources` 契约的
`SiteRecord`,再投影成 `extract_peach_fields` 认得的 payload;把 amane 的 `FailureReason` 一对一映到
契约的 `FailureReason`,由它翻成 `MetadataProviderError` 现有的 `auth` / `unavailable` / `not_found`
三档,上游原样的 reason 留在 `detail` 里给冷却与快照回溯用。链上 `LibraryMetadataProvider` 拿到的
@@ -20,6 +20,7 @@ import os
import re
import shutil
import subprocess
+import sys
import time
import tomllib
from dataclasses import dataclass
@@ -147,6 +148,36 @@ def bridge_python(tools_root: Path) -> Path:
return home / ("Scripts/python.exe" if os.name == "nt" else "bin/python")
+def packaged_runtime(*, root: Path | None = None) -> tuple[Path, str] | None:
+ """定位与桥清单版本、revision 一致的随包运行环境。"""
+ bridge = root if root is not None else BRIDGE_ROOT
+ runtime = Path(os.environ.get("PEACH_AMANE_RUNTIME") or bridge.parent / "amane-runtime")
+ try:
+ data = json.loads((runtime / "manifest.json").read_text(encoding="utf-8"))
+ if not isinstance(data, dict) or not isinstance(data.get("executable"), str):
+ return None
+ relative = Path(data["executable"])
+ if (data.get("revision") != pinned_revision(bridge) or data.get("version") != locked_version(bridge)
+ or data.get("kind") not in {"python", "executable"}
+ or relative.is_absolute() or relative.drive or ".." in relative.parts):
+ return None
+ program = runtime / relative
+ # Python venv 可以使用系统解释器软链,独立可执行文件须完全位于随包目录内。
+ boundary = program.parent.resolve() if data["kind"] == "python" else program.resolve()
+ if boundary.is_relative_to(runtime.resolve()) and program.is_file():
+ return program, data["kind"]
+ except (OSError, ValueError, TypeError, KeyError):
+ pass
+ return None
+
+
+def _requires_packaged_runtime() -> bool:
+ return bool(getattr(sys, "frozen", False)) or os.environ.get("PEACH_DISTRIBUTION") == "docker"
+
+
+PACKAGED_RUNTIME_PROBLEM = "安装包中的采集工具缺失或版本不匹配,请重新安装 Peach"
+
+
def find_uv() -> Path | None:
"""建 venv 用的 uv。托盘进程的 PATH 未必带 WinGet 的 Links 目录,所以多看一眼那里。"""
explicit = os.environ.get("PEACH_UV", "").strip()
@@ -181,6 +212,8 @@ def bundle_problem(root: Path = BRIDGE_ROOT) -> str:
pinned_revision(root)
except (OSError, ValueError, KeyError, TypeError) as exc:
return f"amane 桥的清单读不出来:{exc}"
+ if _requires_packaged_runtime() and not packaged_runtime(root=root):
+ return PACKAGED_RUNTIME_PROBLEM
return ""
@@ -189,6 +222,9 @@ def rebuild(tools_root: Path, *, runner: Runner = subprocess.run, timeout: int =
"""重建桥 venv,返回给设置页的回执。首次要下载约 98 MB,之后只校验。"""
if problem := bundle_problem(root):
return {"ok": False, "error": problem}
+ if packaged_runtime(root=root):
+ return {"ok": True, "result": "采集工具已随 Peach 安装,无需另行准备",
+ **describe(tools_root, root=root)}
uv = find_uv()
if uv is None:
return {"ok": False, "error": "未找到 uv,请先安装 uv 再重建 amane 桥"}
@@ -213,7 +249,8 @@ def describe(tools_root: Path, *, root: Path = BRIDGE_ROOT) -> dict:
清单读不全时 `bundled` 为假、`problem` 写原因,sha 与锁版本留空,卡片据此停用安装。
"""
- python = bridge_python(tools_root)
+ runtime = packaged_runtime(root=root)
+ python = runtime[0] if runtime else bridge_python(tools_root)
problem = bundle_problem(root)
return {
"repository": AMANE_REPOSITORY,
@@ -230,18 +267,21 @@ def describe(tools_root: Path, *, root: Path = BRIDGE_ROOT) -> dict:
def installed(tools_root: Path) -> bool:
- """解释器和 amane 的包元数据都在才算装好。
-
- `uv sync` 先建 venv 再下载依赖,拉不到 GitHub 时会留下一个只有解释器的空壳;
- 只看解释器在不在,这种半截环境会被当成已安装。
- """
- return bridge_python(tools_root).is_file() and bool(installed_version(tools_root))
+ """发行包须有匹配的独立运行环境;源码环境须有解释器和 amane 包元数据。"""
+ if packaged_runtime():
+ return True
+ return (not _requires_packaged_runtime() and bridge_python(tools_root).is_file()
+ and bool(installed_version(tools_root)))
def installed_version(tools_root: Path) -> str:
"""从独立运行环境的包元数据读取版本,不导入上游或启动子进程。"""
from importlib.metadata import distributions
+ if packaged_runtime():
+ return locked_version()
+ if _requires_packaged_runtime():
+ return ""
venv = bridge_home(tools_root) / ".venv"
paths = [venv / "Lib" / "site-packages", *venv.glob("lib/python*/site-packages")]
for distribution in distributions(path=[str(path) for path in paths if path.is_dir()]):
@@ -292,13 +332,18 @@ class AmaneBridge:
"""起一次桥子进程:一次 `--output json`、stdout 一行 JSON;可注入 runner,单测不起真进程。"""
python: Path
- script: Path = BRIDGE_SCRIPT
+ script: Path | None = BRIDGE_SCRIPT
timeout: int = DEFAULT_TIMEOUT
runner: Runner = subprocess.run
@classmethod
def create(cls, tools_root: Path, *, timeout: int = DEFAULT_TIMEOUT,
runner: Runner = subprocess.run) -> "AmaneBridge":
+ runtime = packaged_runtime()
+ if runtime:
+ return cls(runtime[0], BRIDGE_SCRIPT if runtime[1] == "python" else None, timeout, runner)
+ if _requires_packaged_runtime():
+ raise MetadataProviderError(PACKAGED_RUNTIME_PROBLEM)
python = bridge_python(tools_root)
if not python.is_file():
raise MetadataProviderError(
@@ -327,8 +372,13 @@ class AmaneBridge:
direct_sites=direct_sites, timeout=timeout)
checkpoint('network')
proxy_args, env = proxy_argument(proxy_options or {"trust_env": True})
+ for key in list(env):
+ if key.startswith('_PYI_') or key in {'PYTHONHOME', 'PYTHONPATH'}:
+ env.pop(key)
+ if self.script is None:
+ env['PYINSTALLER_RESET_ENVIRONMENT'] = '1'
env["PYTHONIOENCODING"] = "utf-8"
- command = [str(self.python), "-X", "utf8", str(self.script), "--number", number,
+ command = [str(self.python), *(["-X", "utf8", str(self.script)] if self.script else []), "--number", number,
"--sites", ",".join(names), "--language", language, "--output", "json",
*proxy_args]
if direct_sites:
diff --git a/tests/test_metadata_amane.py b/tests/test_metadata_amane.py
index 3041deb53..e57d0aba8 100644
--- a/tests/test_metadata_amane.py
+++ b/tests/test_metadata_amane.py
@@ -262,6 +262,108 @@ class ProxyArgumentTests(unittest.TestCase):
self.assertNotIn("HTTPS_PROXY", env)
+class PackagedRuntimeTests(unittest.TestCase):
+ def setUp(self):
+ temporary = tempfile.TemporaryDirectory()
+ self.addCleanup(temporary.cleanup)
+ self.root = Path(temporary.name).resolve()
+ self.runtime = self.root / 'runtime'
+ self.runtime.mkdir()
+ self.tools = self.root / 'tools'
+ self.manifest = self.runtime / 'manifest.json'
+ self.environment = patch.dict(os.environ, {'PEACH_AMANE_RUNTIME': str(self.runtime), 'PATH': ''})
+ self.environment.start()
+ self.addCleanup(self.environment.stop)
+
+ def write_runtime(self, **changes):
+ facts = {'revision': metadata_amane.pinned_revision(), 'version': metadata_amane.locked_version(),
+ 'kind': 'executable', 'executable': 'amane-bridge.exe'}
+ facts.update(changes)
+ executable = self.runtime / facts['executable']
+ if executable.is_relative_to(self.runtime) and '..' not in executable.parts:
+ executable.parent.mkdir(parents=True, exist_ok=True)
+ executable.write_bytes(b'fixture')
+ self.manifest.write_text(json.dumps(facts), encoding='utf-8')
+ return executable
+
+ def test_packaged_installation_and_rebuild_need_no_uv_or_data_environment(self):
+ executable = self.write_runtime()
+ with patch.object(sys, 'frozen', True, create=True), \
+ patch.object(metadata_amane, 'find_uv', side_effect=AssertionError('无需 uv')):
+ facts = metadata_amane.describe(self.tools)
+ rebuilt = metadata_amane.rebuild(self.tools)
+ self.assertTrue(facts['installed'])
+ self.assertEqual(facts['installed_version'], metadata_amane.locked_version())
+ self.assertEqual(facts['python'], str(executable))
+ self.assertTrue(rebuilt['ok'])
+ self.assertFalse(self.tools.exists())
+
+ def test_packaged_commands_use_the_runtime_and_clear_parent_python_environment(self):
+ for kind in ('executable', 'python'):
+ with self.subTest(kind=kind):
+ executable = self.write_runtime(kind=kind)
+ calls = []
+ def run(command, **kwargs):
+ calls.append((command, kwargs))
+ return completed(report(fc2club=FOUND))
+ with patch.dict(os.environ, {'PYTHONPATH': 'parent-source', 'PYTHONHOME': 'parent-python',
+ '_PYI_PARENT_PROCESS_LEVEL': '1'}):
+ bridge = metadata_amane.AmaneBridge.create(self.tools, runner=run)
+ result = bridge.query('FC2-PPV-1234567', ['fc2club'], proxy_options={'trust_env': False})
+ self.assertEqual(result['sites']['fc2club']['status'], 'found')
+ command, kwargs = calls[0]
+ self.assertEqual(command[0], str(executable))
+ self.assertEqual('-X' in command, kind == 'python')
+ for key in ('PYTHONPATH', 'PYTHONHOME', '_PYI_PARENT_PROCESS_LEVEL'):
+ self.assertNotIn(key, kwargs['env'])
+ if kind == 'executable':
+ self.assertEqual(kwargs['env']['PYINSTALLER_RESET_ENVIRONMENT'], '1')
+
+ def test_invalid_packaged_runtime_requests_reinstallation_without_uv_fallback(self):
+ # 数据目录的解释器不能顶替发行包要求的固定运行环境。
+ python = metadata_amane.bridge_python(self.tools)
+ python.parent.mkdir(parents=True)
+ python.touch()
+ write_amane_metadata(self.tools, metadata_amane.locked_version())
+ for change in ({'revision': '0' * 40}, {'version': 'unknown'}, {'kind': 'shell'},
+ {'executable': '../outside.exe'}, {'executable': str(self.root / 'outside.exe')}):
+ with self.subTest(change=change):
+ self.write_runtime(**change)
+ with patch.object(sys, 'frozen', True, create=True), \
+ patch.object(metadata_amane, 'find_uv', side_effect=AssertionError('无需 uv')):
+ facts = metadata_amane.describe(self.tools)
+ rebuilt = metadata_amane.rebuild(self.tools)
+ with self.assertRaisesRegex(MetadataProviderError, '重新安装'):
+ metadata_amane.AmaneBridge.create(self.tools)
+ self.assertFalse(facts['installed'])
+ self.assertFalse(facts['bundled'])
+ self.assertIn('重新安装', facts['problem'])
+ self.assertFalse(rebuilt['ok'])
+ self.assertIn('重新安装', rebuilt['error'])
+
+ def test_missing_or_malformed_docker_runtime_requests_reinstallation(self):
+ for content in (None, '[]', 'null', '{', '{}'):
+ with self.subTest(content=content):
+ if content is None:
+ self.manifest.unlink(missing_ok=True)
+ else:
+ self.manifest.write_text(content, encoding='utf-8')
+ with patch.dict(os.environ, {'PEACH_DISTRIBUTION': 'docker'}), \
+ patch.object(metadata_amane, 'find_uv', side_effect=AssertionError('无需 uv')):
+ result = metadata_amane.rebuild(self.tools)
+ self.assertFalse(result['ok'])
+ self.assertIn('重新安装', result['error'])
+
+ def test_desktop_runtime_is_discovered_beside_the_bridge_resources(self):
+ self.write_runtime()
+ with patch.dict(os.environ, {'PEACH_AMANE_RUNTIME': ''}), \
+ patch.object(metadata_amane, 'BRIDGE_ROOT', self.root / 'amane-bridge'):
+ self.runtime.rename(self.root / 'amane-runtime')
+ with patch.object(metadata_amane, 'pinned_revision', return_value=metadata_amane.pinned_revision()), \
+ patch.object(metadata_amane, 'locked_version', return_value=metadata_amane.locked_version()):
+ self.assertTrue(metadata_amane.installed(self.tools))
+
+
class BridgeQueryTests(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory()
diff --git a/tests/test_metadata_amane_bridge.py b/tests/test_metadata_amane_bridge.py
index 626d0060b..cc7ad994c 100644
--- a/tests/test_metadata_amane_bridge.py
+++ b/tests/test_metadata_amane_bridge.py
@@ -266,8 +266,28 @@ class RuntimePackagingTests(unittest.TestCase):
self.assertEqual(command[-1], "--self-check")
self.assertEqual("-X" in command, kind == "python")
self.assertNotIn("PYTHONPATH", run.call_args.kwargs["env"])
+ self.assertEqual(run.call_args.kwargs["env"]["PATH"], "")
self.assertEqual(run.call_args.kwargs["env"]["PYINSTALLER_RESET_ENVIRONMENT"], "1")
+ def test_desktop_smoke_requires_the_service_to_use_the_bundled_runtime(self):
+ from scripts import smoke_desktop
+ manifest = self.manifest('executable')
+ facts = {'installed': True, 'problem': '', 'revision': manifest['revision'],
+ 'installed_version': manifest['version'], 'python': str(self.root / manifest['executable'])}
+ opener = mock.Mock()
+ with mock.patch.dict(sys.modules, {'build_amane_runtime': self.builder}), \
+ mock.patch.object(self.builder, 'verify') as verify:
+ opener.open.return_value = io.StringIO(json.dumps(facts))
+ smoke_desktop.assert_amane_runtime(opener, 'http://127.0.0.1:8900', self.root)
+ verify.assert_called_once_with(self.root)
+ opener.open.assert_called_once_with('http://127.0.0.1:8900/api/scraping/amane-bridge', timeout=15)
+ for change in ({'installed': False}, {'revision': 'foreign'}, {'installed_version': 'unknown'},
+ {'python': str(self.root / 'user-python')}, {'problem': '缺少运行环境'}):
+ with self.subTest(change=change):
+ opener.open.return_value = io.StringIO(json.dumps({**facts, **change}))
+ with self.assertRaises(AssertionError):
+ smoke_desktop.assert_amane_runtime(opener, 'http://127.0.0.1:8900', self.root)
+
def test_runtime_self_check_rejects_wrong_versions_and_process_failures(self):
self.manifest("executable")
with mock.patch.object(self.builder.subprocess, "run", return_value=mock.Mock(