Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

feat(diagnostics): 版本统计区分分发方式并附本机量级,Docker 镜像带上 Sentry DSN

longmeidao98b19fa750f2
diff --git a/.dockerignore b/.dockerignore
index 44a8b2ea2..51aaae610 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -16,5 +16,6 @@
!tools/amane-bridge/bridge.py
!tools/amane-bridge/pyproject.toml
!tools/amane-bridge/uv.lock
+!scripts/write_reporting_config.py
**/__pycache__/
**/*.pyc
diff --git a/Dockerfile b/Dockerfile
index 21e3be127..abe9b40d8 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -12,7 +12,12 @@ COPY migrations ./migrations
COPY web ./web
COPY resources ./resources
COPY tools/amane-bridge ./tools/amane-bridge
+COPY scripts/write_reporting_config.py ./scripts/
RUN uv sync --locked --no-dev --no-editable
+# 公开 DSN 写进 wheel 的资源目录;发行流水线传 PEACH_REQUIRE_REPORTING=1,缺失时构建失败。
+ARG PEACH_SENTRY_DSN="" PEACH_LIBRARY_SENTRY_DSN="" PEACH_REQUIRE_REPORTING=""
+RUN target="$(.venv/bin/python -c 'import pathlib, peach; print(pathlib.Path(peach.__file__).parent / "_resources" / "sentry.json")')" \
+ && .venv/bin/python scripts/write_reporting_config.py --output "$target" ${PEACH_REQUIRE_REPORTING:+--require}
FROM python:3.14.8-slim-trixie
LABEL org.opencontainers.image.title="Peach" \
@@ -24,7 +29,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg openssl
&& mkdir /data && chown peach:peach /data
COPY --from=build /app/.venv /app/.venv
ENV PATH="/app/.venv/bin:$PATH" PEACH_DATA_ROOT=/data PYTHONUNBUFFERED=1 \
- PYTHONIOENCODING=utf-8 PYTHONUTF8=1
+ PYTHONIOENCODING=utf-8 PYTHONUTF8=1 PEACH_DISTRIBUTION=docker
WORKDIR /data
USER peach
EXPOSE 8900
diff --git a/docs/DIAGNOSTICS.md b/docs/DIAGNOSTICS.md
index 8a716aee7..e733c9493 100644
--- a/docs/DIAGNOSTICS.md
+++ b/docs/DIAGNOSTICS.md
@@ -31,8 +31,8 @@
| 类别 | 发送内容与筛选 |
| --- | --- |
-| 版本统计 | Peach 版本、操作系统类型、运行环境、源码/独立包形态与公开构建提交号,每机 UTC 自然日最多一次;在进程启动时尝试,没有安装标识,不能据此统计独立用户 |
-| Python 错误 | 版本、平台、Python、CPU 架构、故障环节、分层异常链、Peach 与白名单框架文件名/函数名/行号、结构化错误码;错误文本、变量、源码上下文与用户目录不发送 |
+| 版本统计 | Peach 版本、分发方式(`windows-app`、`macos-app`、`docker`、`wheel`、`source`)、操作系统类型与版本、CPU 架构与逻辑核数、内存档位、Python 版本、运行环境、源码/独立包形态与公开构建提交号,每机 UTC 自然日最多一次;在进程启动时尝试,没有安装标识,不能据此统计独立用户。内存取最近的常见容量档;Docker 内的系统版本是宿主内核,Docker Desktop 的宿主系统由内核名称区分 |
+| Python 错误 | 版本、分发方式、平台、Python、CPU 架构、故障环节、分层异常链、Peach 与白名单框架文件名/函数名/行号、结构化错误码;错误文本、变量、源码上下文与用户目录不发送 |
| HTTP 5xx | 追加方法、路由模板与关联错误编号;不含实例参数、查询参数、正文、鉴权头或用户网址 |
| 媒体工具 | 退出码,以及确定匹配的 AV1 解码器缺失、非法媒体、权限、磁盘空间或未知解码器类别;stderr 与媒体名称留在本机 |
| 浏览器错误 | 白名单异常类型、Peach 正式脚本的行列位置与公开构建映射 ID;没有可定位堆栈的错误与扩展脚本不发送 |
@@ -57,7 +57,7 @@
两项均通过公开 DSN 发送到 Sentry 托管服务,接收服务会接触连接 IP。客户端不发送 IP 字段、用户身份、设备标识、观看记录、媒体信息、凭据、完整日志或诊断 ZIP。SDK 发送队列不落盘,本机聚合文件仅存技术投影和次数;后台发送前复查本机同意版本。关闭或重新选择会清除未提交聚合,并让排队旧事件失效;已在途或已接收的请求无法撤回。本机 ZIP 导出独立可用。
-接收端由维护者配置:源码与 Docker 使用 `PEACH_SENTRY_DSN`;Windows 与 macOS 打包由 `scripts/write_reporting_config.py` 从同名环境变量写入包根 `sentry.json`。发行流水线从 GitGud 项目 CI/CD 变量 `PEACH_SENTRY_DSN` 与 `PEACH_LIBRARY_SENTRY_DSN` 读取,两者为受保护变量、环境作用域 `release`;缺失或不合法时发行构建失败,本地构建写空值,安装后显示上传服务未配置。只接受 Sentry 的公开 HTTPS DSN(含 US / DE 区域),不需要 API token。没有 DSN 时不会创建 SDK 客户端。SDK 自动框架集成、日志、性能追踪和指标均关闭;版本统计作为普通事件计入 Sentry 错误事件额度。账号套餐和总量以 [Sentry 订阅页](https://sentry.io/settings/) 为准。
+接收端由维护者配置:源码运行使用 `PEACH_SENTRY_DSN` 或检出根的 `sentry.json`;Windows、macOS 打包与 Docker 镜像由 `scripts/write_reporting_config.py` 从同名环境变量(Docker 为同名构建参数)写入包根 `sentry.json`,镜像里位于 wheel 的 `peach/_resources/`,容器运行时设置的同名环境变量优先。发行流水线从 GitGud 项目 CI/CD 变量 `PEACH_SENTRY_DSN` 与 `PEACH_LIBRARY_SENTRY_DSN` 读取,两者为受保护变量、环境作用域 `release`;缺失或不合法时发行构建失败,本地构建写空值,安装后显示上传服务未配置。只接受 Sentry 的公开 HTTPS DSN(含 US / DE 区域),不需要 API token。没有 DSN 时不会创建 SDK 客户端。SDK 自动框架集成、日志、性能追踪和指标均关闭;版本统计作为普通事件计入 Sentry 错误事件额度。账号套餐和总量以 [Sentry 订阅页](https://sentry.io/settings/) 为准。
回归验收使用临时 Peach 数据目录和 SDK 内存接收端。实际接收与发行制品的验收记录保存在仓库外。技术诊断选择见 [ADR-0110](adr/0110-recommended-diagnostics-choice.md)。安装辅助进程只在本机取证,不初始化远程客户端或发送事件;应用进程按用户选择发送。
diff --git a/scripts/ci_windows.ps1 b/scripts/ci_windows.ps1
index c574f2b2c..ab387a946 100644
--- a/scripts/ci_windows.ps1
+++ b/scripts/ci_windows.ps1
@@ -63,10 +63,13 @@ if ($Mode -eq 'docker') {
$Candidate = "peach-ci:$Version-$env:CI_PIPELINE_ID"
$Revision = (git rev-parse HEAD).Trim()
Assert-Exit '无法读取提交'
- docker build --pull --label "org.opencontainers.image.version=$Version" --label "org.opencontainers.image.revision=$Revision" -t $Candidate .
+ docker build --pull --label "org.opencontainers.image.version=$Version" --label "org.opencontainers.image.revision=$Revision" `
+ --build-arg PEACH_SENTRY_DSN --build-arg PEACH_LIBRARY_SENTRY_DSN --build-arg PEACH_REQUIRE_REPORTING=1 -t $Candidate .
Assert-Exit '镜像构建失败'
& (Get-Command python.exe).Source -X utf8 scripts/smoke_docker.py $Candidate
Assert-Exit '镜像验收失败'
+ docker run --rm --entrypoint python $Candidate -c 'from peach import library_sharing, remote_reporting, reporting_identity; assert remote_reporting.dsn() and library_sharing.target() and reporting_identity.distribution() == ''docker'''
+ Assert-Exit '镜像内 Sentry DSN 或分发标记缺失'
# 登录写进作业专用配置目录,不改动宿主用户的 Docker 登录。
$SavedConfig, $SavedHost = $env:DOCKER_CONFIG, $env:DOCKER_HOST
$env:DOCKER_HOST = $Endpoint
diff --git a/src/peach/remote_reporting.py b/src/peach/remote_reporting.py
index 33f2d6886..eb18e8428 100644
--- a/src/peach/remote_reporting.py
+++ b/src/peach/remote_reporting.py
@@ -15,11 +15,11 @@ from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlsplit
-from .reporting_identity import event_metadata
+from .reporting_identity import event_metadata, machine_facts
FIELDS = {
- "usage": "Peach 版本、操作系统类型、运行环境与公开构建编号;每天最多一次,不含安装标识。",
- "errors": "Peach 版本、运行环境与公开构建编号、系统类型、Python 版本、CPU 架构、故障环节、异常链、Peach 与白名单框架的代码文件名/函数名/行号、结构化错误码、媒体工具故障类别、浏览器白名单异常类型与正式脚本构建映射、HTTP 方法/路由模板/关联错误编号、同类故障次数与统计日期;首次立即上传,重复次数每小时或退出时合并,每天最多 20 批。",
+ "usage": "Peach 版本、分发方式(Windows/macOS 应用、Docker、wheel 或源码)、操作系统类型与版本、CPU 架构与核数、内存档位、Python 版本、运行环境与公开构建编号;每天最多一次,不含安装标识。",
+ "errors": "Peach 版本、分发方式、运行环境与公开构建编号、系统类型、Python 版本、CPU 架构、故障环节、异常链、Peach 与白名单框架的代码文件名/函数名/行号、结构化错误码、媒体工具故障类别、浏览器白名单异常类型与正式脚本构建映射、HTTP 方法/路由模板/关联错误编号、同类故障次数与统计日期;首次立即上传,重复次数每小时或退出时合并,每天最多 20 批。",
}
NOTICE = "发送至 Sentry。不上报错误原文、完整日志、诊断 ZIP、媒体名称、路径、网址、观看记录或凭据。接收服务会接触连接 IP;关闭后停止新发送,已发送的信息无法撤回。"
_lock = threading.RLock()
@@ -265,7 +265,8 @@ def send(kind, event, config=None):
def usage(config=None):
- send("usage", {"message": "Peach started", "level": "info", "tags": {"os": platform.system()}}, config)
+ send("usage", {"message": "Peach started", "level": "info",
+ "tags": {"os": platform.system(), **machine_facts()}}, config)
flush_counts(config)
try:
if not _disabled and consent(config)["errors"] and dsn():
diff --git a/src/peach/reporting_identity.py b/src/peach/reporting_identity.py
index 7619635d9..fc071d682 100644
--- a/src/peach/reporting_identity.py
+++ b/src/peach/reporting_identity.py
@@ -1,7 +1,9 @@
-"""远程事件的有限用途、运行环境与构建身份。"""
+"""远程事件的有限用途、运行环境、分发方式与构建身份。"""
from __future__ import annotations
+import math
import os
+import platform
import re
import subprocess
import sys
@@ -11,6 +13,57 @@ from . import __version__
from .buildinfo import frozen_build
PURPOSES = {"usage": "version-statistics", "errors": "technical-diagnostics", "research": "population-research"}
+# 内存按常见容量取最近一档,只说明机器量级,不上报精确字节数。
+MEMORY_BUCKETS_GB = (2, 4, 6, 8, 12, 16, 24, 32, 48, 64, 96, 128, 192, 256)
+
+
+def distribution():
+ """安装来源:桌面应用、Docker 镜像、wheel 安装或源码检出。"""
+ if getattr(sys, "frozen", False):
+ return {"Windows": "windows-app", "Darwin": "macos-app"}.get(platform.system(), "packaged")
+ if os.environ.get("PEACH_DISTRIBUTION") == "docker":
+ return "docker"
+ from .settings_file import PROJECT_ROOT
+ return "wheel" if PROJECT_ROOT.name == "_resources" else "source"
+
+
+def _memory_bytes():
+ if os.name == "nt":
+ import ctypes
+
+ class MemoryStatus(ctypes.Structure):
+ _fields_ = [("length", ctypes.c_ulong), ("load", ctypes.c_ulong),
+ *((name, ctypes.c_ulonglong) for name in (
+ "total", "available", "page_total", "page_available",
+ "virtual_total", "virtual_available", "extended_available"))]
+
+ status = MemoryStatus()
+ status.length = ctypes.sizeof(MemoryStatus)
+ return status.total if ctypes.windll.kernel32.GlobalMemoryStatusEx(ctypes.byref(status)) else None
+ return os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES")
+
+
+def memory_bucket(total):
+ gigabytes = total / 2 ** 30
+ if gigabytes >= MEMORY_BUCKETS_GB[-1] * 1.25:
+ return f"{MEMORY_BUCKETS_GB[-1]}+"
+ return str(min(MEMORY_BUCKETS_GB, key=lambda bucket: abs(math.log(gigabytes / bucket))))
+
+
+def machine_facts():
+ """版本统计附带的本机量级信息;Docker 内的系统版本是宿主内核。"""
+ system = platform.system()
+ version = (platform.mac_ver()[0] if system == "Darwin"
+ else platform.version() if system == "Windows" else platform.release())
+ facts = {"os_version": version, "architecture": platform.machine(),
+ "python": platform.python_version(), "cpu_cores": str(os.cpu_count() or "")}
+ try:
+ total = _memory_bytes()
+ if total:
+ facts["memory_gb"] = memory_bucket(total)
+ except (OSError, ValueError, AttributeError):
+ pass
+ return {key: value[:200] for key, value in facts.items() if value}
@lru_cache(maxsize=1)
@@ -35,7 +88,7 @@ def event_metadata(kind):
build = frozen_build() if runtime == "packaged" else None
commit = build.commit if build else (_source_commit() if runtime == "source" else None)
result = {"release": __version__, "environment": environment,
- "tags": {"purpose": PURPOSES[kind], "runtime": runtime}}
+ "tags": {"purpose": PURPOSES[kind], "runtime": runtime, "distribution": distribution()}}
if isinstance(commit, str) and re.fullmatch(r"[a-f0-9]{40}", commit):
result["dist"] = commit
return result
diff --git a/tests/test_remote_reporting.py b/tests/test_remote_reporting.py
index 9e5e03ec3..e746b5b4b 100644
--- a/tests/test_remote_reporting.py
+++ b/tests/test_remote_reporting.py
@@ -117,6 +117,43 @@ class RemoteReportingTests(unittest.TestCase):
self.assertEqual(len(payload["records"]), 20)
self.assertTrue(all(row['total'] == 2 for row in payload['records'].values()))
+ def test_usage_counts_distribution_and_coarse_machine_facts(self):
+ from peach import reporting_identity as identity
+ reporting.save({"usage": True, "errors": True})
+ client = mock.Mock()
+ with mock.patch.object(reporting, "_new_client", return_value=client), \
+ mock.patch.object(identity, "_memory_bytes", return_value=15.8 * 2 ** 30):
+ reporting.usage()
+ tags = client.capture_event.call_args.args[0]["tags"]
+ self.assertEqual(tags["distribution"], "source")
+ self.assertEqual(tags["memory_gb"], "16")
+ self.assertEqual(tags["cpu_cores"], str(os.cpu_count()))
+ for key in ("os", "os_version", "architecture", "python"):
+ self.assertTrue(tags[key], key)
+
+ def test_distribution_tells_desktop_apps_docker_wheel_and_source_apart(self):
+ from peach import reporting_identity as identity
+ cases = (
+ ("windows-app", {"frozen": True, "system": "Windows"}),
+ ("macos-app", {"frozen": True, "system": "Darwin"}),
+ ("docker", {"environ": {"PEACH_DISTRIBUTION": "docker"}}),
+ ("wheel", {"root": Path("site-packages/peach/_resources")}),
+ ("source", {"root": Path("peach-app")}),
+ )
+ for expected, case in cases:
+ with self.subTest(expected), \
+ mock.patch.object(identity.sys, "frozen", case.get("frozen", False), create=True), \
+ mock.patch.object(identity.platform, "system", return_value=case.get("system", "Linux")), \
+ mock.patch.dict(os.environ, case.get("environ", {"PEACH_DISTRIBUTION": ""})), \
+ mock.patch("peach.settings_file.PROJECT_ROOT", case.get("root", Path("peach-app"))):
+ self.assertEqual(identity.distribution(), expected)
+
+ def test_memory_is_reported_as_the_nearest_common_capacity(self):
+ from peach.reporting_identity import memory_bucket
+ for gigabytes, bucket in ((7.6, "8"), (15.8, "16"), (23.4, "24"), (31.9, "32"), (0.9, "2"), (1024, "256+")):
+ with self.subTest(gigabytes=gigabytes):
+ self.assertEqual(memory_bucket(gigabytes * 2 ** 30), bucket)
+
def test_frequency_batches_send_disjoint_counts_and_keep_group_identity(self):
from peach import diagnostic_frequency as frequency
reporting.save({"usage": True, "errors": True})
diff --git a/tests/test_reporting_transport.py b/tests/test_reporting_transport.py
index 29a861557..8de94ad24 100644
--- a/tests/test_reporting_transport.py
+++ b/tests/test_reporting_transport.py
@@ -107,7 +107,8 @@ class ReportingTransportTests(unittest.TestCase):
metadata = reporting_identity.event_metadata("errors")
self.assertEqual(metadata["environment"], "development")
self.assertEqual(metadata["dist"], "a" * 40)
- self.assertEqual(metadata["tags"], {"purpose": "technical-diagnostics", "runtime": "source"})
+ self.assertEqual(metadata["tags"], {"purpose": "technical-diagnostics", "runtime": "source",
+ "distribution": "source"})
self.assertNotIn("private", str(metadata))
with mock.patch.object(reporting_identity.sys, "frozen", True, create=True), \
mock.patch.object(reporting_identity, "frozen_build", return_value=SimpleNamespace(commit="bad")), \