Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

feat(diagnostics): 版本统计与故障计数改走日志和会话

longmeidao269f16b6bbef
diff --git a/docs/DIAGNOSTICS.md b/docs/DIAGNOSTICS.md
index 2bbfd8f34..783cf74a0 100644
--- a/docs/DIAGNOSTICS.md
+++ b/docs/DIAGNOSTICS.md
@@ -76,7 +76,7 @@
| 类别 | 发送内容与筛选 |
| --- | --- |
-| 版本统计 | Peach 版本、分发方式(`windows-app`、`macos-app`、`docker`、`wheel`、`source`)、操作系统类型与版本、CPU 架构与逻辑核数、内存档位、Python 版本、运行环境、源码/独立包形态与公开构建提交号,每机 UTC 自然日最多一次;在进程启动时尝试,没有安装标识,不能据此统计独立用户。内存取最近的常见容量档;Docker 内的系统版本是宿主内核,Docker Desktop 的宿主系统由内核名称区分 |
+| 版本统计 | Peach 版本、分发方式(`windows-app`、`macos-app`、`docker`、`wheel`、`source`)、操作系统类型与版本、CPU 架构与逻辑核数、内存档位、Python 版本、运行环境、源码/独立包形态与公开构建提交号,每机 UTC 自然日最多一次,以一条日志和一条会话汇总发送;在进程启动时尝试,没有安装标识,不能据此统计独立用户。内存取最近的常见容量档;Docker 内的系统版本是宿主内核,Docker Desktop 的宿主系统由内核名称区分 |
| Python 错误 | 版本、分发方式、平台、Python、CPU 架构、故障环节、分层异常链、Peach 与白名单框架文件名/函数名/行号、结构化错误码;错误文本、变量、源码上下文与用户目录不发送 |
| HTTP 5xx | 追加方法、路由模板与关联错误编号;不含实例参数、查询参数、正文、鉴权头或用户网址 |
| 媒体工具 | 退出码,以及确定匹配的 AV1 解码器缺失、非法媒体、权限、磁盘空间或未知解码器类别;stderr 与媒体名称留在本机 |
@@ -84,9 +84,9 @@
SQLite 错误复用本机与远程的结构化错误码;能够确定时附缺表、缺列、锁定、只读、损坏或无法打开的固定类别,不发送表名、列名或 SQL。媒体故障在路径脱敏前提取固定类别,子进程异常本身携带的 stderr 也参与分类。系统不支持文件夹对话框时保留本机状态;支持的平台上对话框启动失败仍进入技术诊断。
-远程事件数表示接收端存储的上报批次,`occurrence_count` 表示该批代表的发生次数。每日预算、关闭上传、断网和筛选会造成观测缺失;未收到事件不能解释为没有故障。版本统计每日一次且不含安装标识,不能当作独立用户数、日活或故障率分母。错误事件附带的是服务主机环境,不能据此推断访问页面的客户端系统或浏览器。研究快照与版本统计均是普通事件,仍占用接收平台的事件配额;分项目不隔离组织配额。
+错误事件表示某台机器在某个版本里第一次遇到这个诊断键,发生次数在计数日志里。每日额度、关闭上传、断网和筛选会造成观测缺失;未收到事件不能解释为没有故障。版本统计每日一次且不含安装标识,不能当作独立用户数、日活或故障率分母。错误事件附带的是服务主机环境,不能据此推断访问页面的客户端系统或浏览器。研究快照是普通事件,与错误事件共用组织的事件额度;版本统计与计数日志按日志字节计量,会话汇总不计费([ADR-0132](adr/0132-diagnostics-counts-via-logs-and-sessions.md))。
-事件的 `purpose` 区分 `version-statistics`、`technical-diagnostics` 与独立研究通道的 `population-research`。故障视图使用 `level:error purpose:technical-diagnostics`,生产故障可再加 `environment:production`。没有用途标签的历史事件须单独审阅,不按缺失字段推断来源。启动统计是普通 info 事件,应排除在故障视图与告警条件之外。
+`purpose` 区分 `version-statistics`、`technical-diagnostics` 与独立研究通道的 `population-research`。故障视图使用 `level:error purpose:technical-diagnostics`,生产故障可再加 `environment:production`。版本统计在日志中查询 `purpose:version-statistics`,各版本占比也显示在接收端的 Releases 页;故障次数在日志中查询正文 `Peach diagnostic count`,按 `diagnostic_key`、`occurrence_day` 或 `sentry.release` 分组对 `occurrence_count` 求和。没有用途标签的历史事件须单独审阅,不按缺失字段推断来源;ADR-0132 之前的版本仍以 info 事件 `Peach started` 发送版本统计,应排除在故障视图与告警条件之外。
`environment` 只接受 `production`、`development`、`test`;独立包与普通源码部署默认 `production`,Git 隔离工作树默认 `development`,可用 `PEACH_REPORTING_ENVIRONMENT` 明确指定。`runtime` 为 `packaged` 或 `source`;`dist` 使用构建提交号,不发送本机目录、主机名或自由文本标识。标准测试入口关闭本进程遥测,并向子进程传递 `PEACH_REPORTING_TEST_MODE=1`,在最终 HTTP 请求边界阻止两条通道真实外发;内存接收端仍可验证最终载荷。
@@ -98,15 +98,15 @@ SQLite 错误复用本机与远程的结构化错误码;能够确定时附缺
正常取消、来源未命中/暂不可用、HTTP 4xx、互斥、普通 warning 和无堆栈的泛化日志不发送,也不计入故障次数。后台任务的纯文本失败回执仍保留在本机;有异常链的意外失败可发送。SDK 内部日志不进入远程钩子。
-同类故障首次在额度内立即上传,重复次数每小时或正常退出时合并提交。`occurrence_count` 是本批增量次数;例如先上传 1 次,随后又发生 9 次,下一批上传 9,合计 10。`occurrence_day` 是实际发生的 UTC 日期,`diagnostic_key` 由脱敏技术字段计算,包含版本但排除每次 HTTP 错误编号;相同代码故障可跨机器对应,不是安装标识。Sentry 的事件数代表上传批次,频率需按统计日期汇总 `occurrence_count`,不能把最新一批当累计总数。普通标签不提供 Sentry 原生的数值求和;汇总需读取事件字段后计算,不能把原生事件趋势图当成真实触发趋势。
+`diagnostic_key` 由脱敏技术字段计算,包含版本但排除每次 HTTP 错误编号;相同代码故障可跨机器对应,不是安装标识。新的诊断键在额度内立即上传一条错误事件,代表 1 次发生;之后同一台机器上的重复发生只累计在本机,每 6 小时、跨 UTC 日或正常退出时合并成计数日志。计数日志的 `occurrence_count` 是整数增量,例如首次事件代表 1 次,随后又发生 9 次,下一条计数日志记 9,合计 10;`occurrence_day` 是实际发生的 UTC 日期。错误事件数不代表发生频率,接收端的问题趋势图也不是真实触发趋势,频率以计数日志求和为准。
-每机每天最多接收 20 类故障的本机计数,远程每天最多提交 20 批,重复补报与首次诊断共用批次上限。托盘与子服务使用同一文件锁及聚合文件 `state/remote-reporting-frequency.json`,保留至多 7 个 UTC 日期的技术样本、累计数和已入队数;不读取历史日志。额度不足的待提交次数可在后续启动或定时检查时继续提交,超过保留期失效。SDK 接受入队后更新已入队数;连接失败、服务端丢弃、强制退出、锁争用或超过种类及保留上限均可能造成缺口,因此这是尽力交付的故障计数,不是完整运行审计。SDK 尚未接受的次数保留在聚合文件中。
+每机每天最多登记 20 个新的诊断键,最多上传 5 条首次错误事件;额度不足的首次事件推迟到后续定时检查或次日补传,统计日期保持实际发生日。托盘与子服务使用同一文件锁及聚合文件 `state/remote-reporting-frequency.json`,保存技术样本、首次事件是否已上传、当日累计数和已提交数;不读取历史日志。诊断键连续 7 个 UTC 日没有再发生即过期,再次发生时重新上传一条错误事件。SDK 接受入队后更新已提交数;计数日志提交失败时,跨日前的旧次数不再保留。连接失败、服务端丢弃、强制退出、锁争用或超过登记与保留上限均可能造成缺口,因此这是尽力交付的故障计数,不是完整运行审计。
-`state/remote-reporting-delivery.json` 保存各技术用途最后一批的本机回执,配置 API 的 `remote_reporting.delivery` 提供相同的脱敏状态。`queued` 仅表示准备入队;`accepted` 表示传输取得 HTTP 2xx,不保证接收端长期保留;`failed` 表示明确拒绝或本机队列/限流阻断;`unknown` 表示未取得可靠结果,网络异常也可能发生在接收之后。HTTP 2xx 同时带限流头时按未知处理。回执不保存服务端正文、请求网址或载荷;旧事件与旧同意版本的回执不能覆盖当前记录。传输回执不把入队计数改写成接收计数,也不触发补传。
+`state/remote-reporting-delivery.json` 保存各技术用途最后一批(错误事件、计数日志或版本统计)的本机回执,配置 API 的 `remote_reporting.delivery` 提供相同的脱敏状态。`queued` 仅表示准备入队;`accepted` 表示传输取得 HTTP 2xx,不保证接收端长期保留;`failed` 表示明确拒绝或本机队列/限流阻断;`unknown` 表示未取得可靠结果,网络异常也可能发生在接收之后。HTTP 2xx 同时带限流头时按未知处理。回执不保存服务端正文、请求网址或载荷;旧事件与旧同意版本的回执不能覆盖当前记录。传输回执不把入队计数改写成接收计数,也不触发补传。
-两项均通过公开 DSN 发送到 Sentry 托管服务,接收服务会接触连接 IP。客户端不发送 IP 字段、用户身份、设备标识、观看记录、媒体信息、凭据、完整日志或诊断 ZIP。SDK 发送队列不落盘,本机聚合文件仅存技术投影和次数;后台发送前复查本机同意版本。关闭或重新选择会清除未提交聚合,并让排队旧事件失效;已在途或已接收的请求无法撤回。本机 ZIP 导出独立可用。
+两项均通过公开 DSN 发送到 Sentry 托管服务,接收服务会接触连接 IP。客户端不发送 IP 字段、用户身份、设备标识、观看记录、媒体信息、凭据、完整日志或诊断 ZIP。日志与会话条目由 Peach 自行组装,不经过 SDK 的日志与会话通道,主机名、SDK 名称与追踪上下文不进入载荷;传输闸门只放行带当前同意凭据的这两类条目。SDK 发送队列不落盘,本机聚合文件仅存技术投影和次数;后台发送前复查本机同意版本。关闭或重新选择会清除未提交聚合,并让排队旧事件失效;已在途或已接收的请求无法撤回。本机 ZIP 导出独立可用。
-接收端由维护者配置:源码运行使用 `PEACH_SENTRY_DSN` 或检出根的 `sentry.json`;Windows、macOS 打包与 Docker 镜像由 `scripts/write_reporting_config.py` 从同名环境变量(Docker 为同名构建参数)写入包根 `sentry.json`,镜像里位于 wheel 的 `peach/_resources/`,容器运行时设置的同名环境变量优先。发行流水线从 GitGud 项目 CI/CD 变量 `PEACH_SENTRY_DSN` 与 `PEACH_LIBRARY_SENTRY_DSN` 读取,两者为受保护变量、环境作用域 `release`;缺失或不合法时发行构建失败,本地构建写空值,安装后显示上传服务未配置。只接受 Sentry 的公开 HTTPS DSN(含 US / DE 区域),不需要 API token。没有 DSN 时不会创建 SDK 客户端。SDK 自动框架集成、日志、性能追踪和指标均关闭;版本统计作为普通事件计入 Sentry 错误事件额度。账号套餐和总量以 [Sentry 订阅页](https://sentry.io/settings/) 为准。
+接收端由维护者配置:源码运行使用 `PEACH_SENTRY_DSN` 或检出根的 `sentry.json`;Windows、macOS 打包与 Docker 镜像由 `scripts/write_reporting_config.py` 从同名环境变量(Docker 为同名构建参数)写入包根 `sentry.json`,镜像里位于 wheel 的 `peach/_resources/`,容器运行时设置的同名环境变量优先。发行流水线从 GitGud 项目 CI/CD 变量 `PEACH_SENTRY_DSN` 与 `PEACH_LIBRARY_SENTRY_DSN` 读取,两者为受保护变量、环境作用域 `release`;缺失或不合法时发行构建失败,本地构建写空值,安装后显示上传服务未配置。只接受 Sentry 的公开 HTTPS DSN(含 US / DE 区域),不需要 API token。没有 DSN 时不会创建 SDK 客户端。SDK 自动框架集成、自带的日志与会话采集、性能追踪和指标均关闭。账号套餐和总量以 [Sentry 订阅页](https://sentry.io/settings/) 为准。
回归验收使用临时 Peach 数据目录和 SDK 内存接收端。实际接收与发行制品的验收记录保存在仓库外。技术诊断选择见 [ADR-0110](adr/0110-recommended-diagnostics-choice.md)。安装辅助进程只在本机取证,不初始化远程客户端或发送事件;应用进程按用户选择发送。
diff --git a/docs/REUSE.md b/docs/REUSE.md
index a89aa47bf..f33e98bff 100644
--- a/docs/REUSE.md
+++ b/docs/REUSE.md
@@ -503,9 +503,9 @@ Peach 播放器皮肤直接装饰 Video.js 10 的 light DOM:右侧控制组用
启动取证复用 Python 3.14 标准库的 `sys.excepthook`、`threading.excepthook`、`sys.unraisablehook`、`logging.Handler`、`faulthandler` 和 `zipfile`(PSF 许可,无额外安装体积)。`error_reporting` 在配置与第三方依赖之前加载,`support_bundle` 仅收集选定日志和系统字段;Peach 保留脱敏、备用目录、本机权限与报告内容策略。已有诊断报告用于页面和运行期导出。临时目录与独立子进程覆盖无控制台、导入失败、线程异常、备用目录、配置损坏和脱敏。
-可选远程诊断复用官方 [sentry-sdk 2.72.0](https://github.com/getsentry/sentry-python/tree/2.72.0)(MIT,维护中,支持 Python 3.14);固定版源码通过 `opensrc path pypi:sentry-sdk` 核对。SDK 负责后台 HTTP、TLS、Sentry 信封和服务端限流;Peach 负责本机选择、技术字段投影、同类错误聚合与每机每日上限。版本统计与错误诊断共用默认勾选的开关,关闭时不构造 SDK;SDK 全局自动集成、追踪、回放、日志、会话与指标采集均关闭。`HttpTransport._send_envelope` 的窄覆盖在真正发送前复查本机选择版本并移除本机传输元数据;它是固定版本的内部接口,升级必须通过真实 SDK 信封回归。无真实外发的内存接收端检查最终字节、停用后队列失效与断网不影响本机取证。新增依赖仅 `sentry-sdk`,复用已声明的 urllib3 / certifi;体积以发行构建为准。未采用自写 Sentry 协议或 Workers 收集后台,避免另行维护传输、去重和查看界面。没有真实 ledger 写入。
+可选远程诊断复用官方 [sentry-sdk 2.72.0](https://github.com/getsentry/sentry-python/tree/2.72.0)(MIT,维护中,支持 Python 3.14);固定版源码通过 `opensrc path pypi:sentry-sdk` 核对。SDK 负责后台 HTTP、TLS、Sentry 信封和服务端限流;Peach 负责本机选择、技术字段投影、同类错误聚合与每机每日上限。版本统计与错误诊断共用默认勾选的开关,关闭时不构造 SDK;SDK 全局自动集成、追踪、回放、日志、会话与指标采集均关闭;版本统计与计数日志、会话汇总由 Peach 用 SDK 的 `Envelope`、`Item` 与 `PayloadRef` 组装,经同一传输发送(ADR-0132)。`HttpTransport._send_envelope` 的窄覆盖在真正发送前复查本机选择版本并移除本机传输元数据;它是固定版本的内部接口,升级必须通过真实 SDK 信封回归。无真实外发的内存接收端检查最终字节、停用后队列失效与断网不影响本机取证。新增依赖仅 `sentry-sdk`,复用已声明的 urllib3 / certifi;体积以发行构建为准。未采用自写 Sentry 协议或 Workers 收集后台,避免另行维护传输、去重和查看界面。没有真实 ledger 写入。
-故障频率复用已固定的 `filelock`、`fsutil.atomic_write_text` 和 SDK 事件标签,保留 Peach 的同意版本、技术投影、跨进程计数与每日批次策略。`diagnostic_frequency` 保存有界技术样本及待提交增量,首次、每小时和正常退出通过同一 SDK 提交;关闭会清除聚合。Sentry 默认事件数统计批次,不会将自定义次数标签自动求和;汇总按事件的 `occurrence_day` 与 `occurrence_count` 计算。`reporting_transport` 复用官方 `HttpTransport` 的 `_handle_response`、`_handle_request_error` 与队列丢弃回调,取得 HTTP 状态且不重复实现请求、TLS 或重试;固定内部接口以真实 SDK 和模拟 HTTP 回归。`reporting_delivery` 只保存本机有限回执;SDK 入队、HTTP 接收与接收端留存分别解释。用途、环境与公开构建号复用 Sentry 原生字段,测试在同一传输边界拦截外发。未开启 SDK metrics 或新增收集服务:独立指标通道会增加同意边界、额度与交付口径。计数只覆盖已筛选技术故障,发送及保留限制见 [诊断核验](DIAGNOSTICS.md)。没有新增 Python 依赖或真实 ledger 写入。
+故障频率复用已固定的 `filelock`、`fsutil.atomic_write_text` 和 SDK 事件标签,保留 Peach 的同意版本、技术投影、跨进程计数与每日批次策略。`diagnostic_frequency` 保存有界技术样本及待提交增量;每个诊断键只上传一次错误事件,之后的次数每 6 小时、跨日或正常退出时合并成带整数 `occurrence_count` 的计数日志,接收端日志查询可直接求和;关闭会清除聚合。`reporting_transport` 复用官方 `HttpTransport` 的 `_handle_response`、`_handle_request_error` 与队列丢弃回调,取得 HTTP 状态且不重复实现请求、TLS 或重试;固定内部接口以真实 SDK 和模拟 HTTP 回归。`reporting_delivery` 只保存本机有限回执;SDK 入队、HTTP 接收与接收端留存分别解释。用途、环境与公开构建号复用 Sentry 原生字段,测试在同一传输边界拦截外发。未开启 SDK metrics 或新增收集服务;计数改走日志而不是指标,是因为日志条目能带上与错误事件相同的诊断键和白名单标签。计数只覆盖已筛选技术故障,发送及保留限制见 [诊断核验](DIAGNOSTICS.md)。没有新增 Python 依赖或真实 ledger 写入。
Windows 连接清理复用 [CPython 3.14.7 的 Proactor 对象协议](https://github.com/python/cpython/blob/v3.14.7/Lib/asyncio/proactor_events.py)(PSF 许可,无新增依赖)。精确识别纯标准库清理回调中的 `WinError 10054`,只补完 socket 关闭、服务端解除登记和关闭状态,不重复通知协议,不全局修改标准库类或替换事件循环。它使用 `_sock`、`_server._detach` 与 `_called_connection_lost` 私有接口,升级 Python 时须核验。真实 socket 和 `asyncio.Server` 配合合成故障验证资源释放;收尾自身失败保留原始异常链并上报。33 次回环实验未复现原始竞争时序,不据此认定客户端来源。协议类别只写本机日志;来源、任务、媒体和浏览器的远程筛选复用现有领域状态,见 [诊断核验](DIAGNOSTICS.md#远程错误筛选核验)。
diff --git a/docs/adr/0109-opt-in-diagnostics-and-population-research.md b/docs/adr/0109-opt-in-diagnostics-and-population-research.md
index c1073392e..96b3c8782 100644
--- a/docs/adr/0109-opt-in-diagnostics-and-population-research.md
+++ b/docs/adr/0109-opt-in-diagnostics-and-population-research.md
@@ -1,6 +1,6 @@
# ADR-0109:明确同意的远程诊断与群体偏好研究
-- 状态:已采纳;技术诊断的选择与默认值见 [ADR-0110](0110-recommended-diagnostics-choice.md),馆藏明细契约见 [ADR-0112](0112-complete-library-sharing.md)
+- 状态:已采纳;技术诊断的选择与默认值见 [ADR-0110](0110-recommended-diagnostics-choice.md),馆藏明细契约见 [ADR-0112](0112-complete-library-sharing.md),版本统计与故障计数的发送方式见 [ADR-0132](0132-diagnostics-counts-via-logs-and-sessions.md)
- 日期:2026-10-18
- 相关:ADR-0023、ADR-0092
diff --git a/docs/adr/0132-diagnostics-counts-via-logs-and-sessions.md b/docs/adr/0132-diagnostics-counts-via-logs-and-sessions.md
new file mode 100644
index 000000000..471abc425
--- /dev/null
+++ b/docs/adr/0132-diagnostics-counts-via-logs-and-sessions.md
@@ -0,0 +1,34 @@
+# ADR-0132:技术诊断的计数改走日志与会话
+
+- 状态:已采纳
+- 日期:2026-11-03
+- 相关:ADR-0109、ADR-0110
+
+## 背景
+
+ADR-0109 让版本统计和故障计数都以 Sentry 错误事件发送。托管服务的错误事件按条计费,免费档有月度上限。实际运行中,上限在计费周期开始两天内就被耗尽,之后的事件全部被拒收,新故障也看不到了。事件的主要来源有三类:
+
+- 每台机器每天一条版本统计;
+- 同一台机器上的同一个故障,每个 UTC 日都重新上传一次「首次」,之后每小时再补一批增量次数;
+- 设计上应留在本机的预期失败,因为筛选判据有漏项而被上传。
+
+免费档不提供按版本或消息的入站过滤、按密钥限流或数据转发,接收端没有可用的开关,只能由客户端少发。
+
+## 决策
+
+错误事件只承载需要定位的诊断:带异常链和代码位置、按问题归组的那一条。计数和统计改用不按条计费的两种数据:
+
+1. **版本统计**:每天一次,改发一条 Sentry 日志(`log` 条目,属性沿用原版本统计字段),再加一条会话汇总(`sessions` 条目,只含版本与运行环境,记为一次正常结束)。会话汇总让接收端的 Releases 页显示各版本占比。会话不在托管服务的计费类别里;日志按字节计费,额度与错误事件分开。
+2. **故障计数**:同一台机器上同一个诊断键(脱敏技术字段的哈希,包含版本)只上传一次错误事件;之后的发生次数累计在本机,每 6 小时或正常退出时合并成计数日志,属性带 `diagnostic_key`、`occurrence_day` 和整数 `occurrence_count`,接收端可以直接按日期和版本求和。诊断键连续 7 天没有再发生,本机记录过期;再次发生时重新上传一次错误事件。每机每天最多上传 5 条首次错误事件,新出现的诊断键每天最多登记 20 个。
+3. **筛选判据补漏**:坏媒体与读写错误、文件锁争用、SQLite 磁盘与锁错误、媒体子进程超时、来源连接超时、路由主动返回的 502–504,以及读取方已断开时的管道写入失败,都留在本机。
+
+日志和会话条目由 Peach 自己组装成信封,不经过 SDK 的日志批处理器和会话追踪,因此 SDK 默认附加的主机名、SDK 名称和追踪上下文不会进入载荷。两类条目和错误事件走同一个传输闸门:发送前复查本机同意版本,关闭后排队的条目失效;测试模式下同样在最终 HTTP 边界阻止外发。
+
+## 代价
+
+- 接收端的问题列表不再反映发生次数;频率要在日志查询里按 `diagnostic_key` 汇总。
+- 日志条目不归组、不触发问题告警,免费档只保留约 30 天。
+- 同一故障在一台机器上每个版本只报一次,修复验证要看计数日志是否归零,不能只看问题是否出现新事件。
+- 已安装的旧版本仍按旧节奏发送,直到用户升级。必要时可以在新版本换用新的接收密钥,待新版普及后停用旧密钥。
+
+同意范围不变:两类数据仍由「版本统计与错误诊断」开关控制,字段白名单不扩大。[日志与诊断核验](../DIAGNOSTICS.md#可选远程诊断)同步改写了节奏和查询口径。
diff --git a/src/peach/diagnostic_frequency.py b/src/peach/diagnostic_frequency.py
index 5a36f7c76..1ae19e129 100644
--- a/src/peach/diagnostic_frequency.py
+++ b/src/peach/diagnostic_frequency.py
@@ -1,4 +1,7 @@
-"""跨进程聚合已筛选的技术诊断;不读取日志或账本。"""
+"""跨进程聚合已筛选的技术诊断;不读取日志或账本。
+
+同一诊断键在本机只上传一次错误事件,之后的发生次数按间隔合并成计数日志(ADR-0132)。
+"""
from __future__ import annotations
import hashlib
@@ -9,8 +12,9 @@ from filelock import FileLock
from .fsutil import atomic_write_text
-INTERVAL = 3600
-DAILY_LIMIT = 20
+INTERVAL = 6 * 3600
+EVENT_LIMIT = 5
+NEW_KEY_LIMIT = 20
RETENTION_DAYS = 7
@@ -22,6 +26,16 @@ def _path(config):
return config.directory("state") / "remote-reporting-frequency.json"
+def _valid(key, row, cutoff, day):
+ return (isinstance(key, str) and len(key) == 64 and isinstance(row, dict)
+ and isinstance(row.get("day"), str) and cutoff <= row["day"] <= day
+ and isinstance(row.get("first"), str) and row["first"] <= row["day"]
+ and isinstance(row.get("event"), dict) and type(row.get("sent")) is bool
+ and type(row.get("total")) is int and type(row.get("submitted")) is int
+ and 0 <= row["submitted"] <= row["total"]
+ and type(row.get("last_logged")) in (int, float))
+
+
def _load(path, revision, now):
try:
state = json.loads(path.read_text(encoding="utf-8"))
@@ -31,32 +45,27 @@ def _load(path, revision, now):
state = {"revision": revision, "records": {}}
day = now.date().isoformat()
if state.get("day") != day:
- state.update(day=day, batches=0)
+ state.update(day=day, events=0)
cutoff = (now.date() - timedelta(days=RETENTION_DAYS - 1)).isoformat()
records = state.get("records", {})
if not isinstance(records, dict):
records = {}
- state["records"] = {key: row for key, row in records.items()
- if isinstance(key, str) and len(key) == 75 and isinstance(row, dict)
- and isinstance(row.get("day"), str) and cutoff <= row["day"] <= day
- and isinstance(row.get("event"), dict)
- and type(row.get("total")) is int and type(row.get("submitted")) is int
- and 0 <= row["submitted"] <= row["total"]
- and type(row.get("last_sent")) in (int, float)}
- if type(state.get("batches")) is not int or state["batches"] < 0:
- state["batches"] = 0
+ state["records"] = {key: row for key, row in records.items() if _valid(key, row, cutoff, day)}
+ if type(state.get("events")) is not int or state["events"] < 0:
+ state["events"] = 0
return state
-def _emit(state, row, key, now, emit, *, force=False, event_id=None):
- pending = row["total"] - row["submitted"]
- if pending <= 0 or state["batches"] >= DAILY_LIMIT:
- return
- if not force and row["submitted"] and now.timestamp() - row["last_sent"] < INTERVAL:
+def _tags(row, key, count):
+ return {**row["event"].get("tags", {}), "diagnostic_key": key,
+ "occurrence_day": row["day"], "occurrence_count": str(count)}
+
+
+def _emit_event(state, row, key, emit, *, event_id=None):
+ """诊断键的首次错误事件只携带一次发生,其余次数交给计数日志。"""
+ if row["sent"] or row["total"] - row["submitted"] <= 0 or state["events"] >= EVENT_LIMIT:
return
- event = {**row["event"], "tags": {**row["event"].get("tags", {}),
- "diagnostic_key": key.split(":", 1)[1], "occurrence_day": row["day"],
- "occurrence_count": str(pending)}}
+ event = {**row["event"], "tags": _tags(row, key, 1)}
if event_id:
event["event_id"] = event_id
try:
@@ -64,34 +73,62 @@ def _emit(state, row, key, now, emit, *, force=False, event_id=None):
except Exception:
accepted = False
if accepted:
- row.update(submitted=row["total"], last_sent=now.timestamp())
- state["batches"] += 1
+ row.update(sent=True, submitted=row["submitted"] + 1)
+ state["events"] += 1
+
+
+def _emit_counts(rows, now, log, *, force=False):
+ """已上传首次事件的诊断键,把未提交次数合并成一批计数日志。"""
+ due = [(key, row) for key, row in rows
+ if row["sent"] and row["total"] > row["submitted"]
+ and (force or row["day"] != now.date().isoformat()
+ or now.timestamp() - row["last_logged"] >= INTERVAL)]
+ if not due:
+ return
+ counts = [{**row["event"], "tags": _tags(row, key, row["total"] - row["submitted"])} for key, row in due]
+ try:
+ accepted = log(counts)
+ except Exception:
+ accepted = False
+ if accepted:
+ for _key, row in due:
+ row.update(submitted=row["total"], last_logged=now.timestamp())
+
+
+def _roll(row, key, now, log):
+ """跨 UTC 日先提交前一天的次数,计数日志的统计日期是实际发生日;提交失败的旧次数不再保留。"""
+ day = now.date().isoformat()
+ if row["day"] == day or not row["sent"]:
+ return
+ _emit_counts([(key, row)], now, log, force=True)
+ row.update(day=day, total=0, submitted=0, last_logged=now.timestamp())
-def record(config, revision, event, emit):
- """记录一次筛选后的故障;首次立即提交,重复故障合并提交增量次数。"""
+def record(config, revision, event, emit, log):
+ """记录一次筛选后的故障;新诊断键立即上传错误事件,重复发生只累计次数。"""
path = _path(config)
path.parent.mkdir(parents=True, exist_ok=True)
with FileLock(str(path) + ".lock", timeout=.1):
now = _now()
state = _load(path, revision, now)
sample = {key: value for key, value in event.items() if key != "event_id"}
- fingerprint = hashlib.sha256(json.dumps(sample, sort_keys=True).encode()).hexdigest()
+ key = hashlib.sha256(json.dumps(sample, sort_keys=True).encode()).hexdigest()
day = now.date().isoformat()
- key = day + ":" + fingerprint
records = state["records"]
if key not in records:
- if sum(row["day"] == day for row in records.values()) >= DAILY_LIMIT:
+ if sum(row["first"] == day for row in records.values()) >= NEW_KEY_LIMIT:
return
- records[key] = {"day": day, "event": sample, "total": 0, "submitted": 0, "last_sent": 0}
+ records[key] = {"event": sample, "first": day, "day": day, "sent": False,
+ "total": 0, "submitted": 0, "last_logged": now.timestamp()}
row = records[key]
+ _roll(row, key, now, log)
row["total"] += 1
- _emit(state, row, key, now, emit, event_id=event.get("event_id") if row["total"] == 1 else None)
+ _emit_event(state, row, key, emit, event_id=event.get("event_id"))
atomic_write_text(path, json.dumps(state))
-def flush(config, revision, emit, *, force=False):
- """按发生日期提交未提交次数,退出时可提前提交,仍遵守每日批次上限。"""
+def flush(config, revision, emit, log, *, force=False):
+ """补传额度不足时推迟的首次事件,并按间隔提交计数日志;退出时可提前提交。"""
path = _path(config)
if not path.exists():
return
@@ -99,7 +136,8 @@ def flush(config, revision, emit, *, force=False):
now = _now()
state = _load(path, revision, now)
for key, row in state["records"].items():
- _emit(state, row, key, now, emit, force=force)
+ _emit_event(state, row, key, emit)
+ _emit_counts(state["records"].items(), now, log, force=force)
atomic_write_text(path, json.dumps(state))
diff --git a/src/peach/remote_reporting.py b/src/peach/remote_reporting.py
index f14617416..06e049b4c 100644
--- a/src/peach/remote_reporting.py
+++ b/src/peach/remote_reporting.py
@@ -19,8 +19,8 @@ from .reporting_identity import event_metadata, machine_facts
from .diagnostic_exceptions import media_details
FIELDS = {
- "usage": "Peach 版本、分发方式(Windows/macOS 应用、Docker、wheel 或源码)、操作系统类型与版本、CPU 架构与核数、内存档位、Python 版本、运行环境与公开构建编号;每天最多一次,不含安装标识。",
- "errors": "Peach 版本、分发方式、运行环境与公开构建编号、系统类型、Python 版本、CPU 架构、故障环节、异常链、Peach 与白名单框架的代码文件名/函数名/行号、结构化错误码、媒体工具故障类别、浏览器白名单异常类型与正式脚本构建映射、HTTP 方法/路由模板/关联错误编号、同类故障次数与统计日期;首次立即上传,重复次数每小时或退出时合并,每天最多 20 批。",
+ "usage": "Peach 版本、分发方式(Windows/macOS 应用、Docker、wheel 或源码)、操作系统类型与版本、CPU 架构与核数、内存档位、Python 版本、运行环境与公开构建编号;每天最多一条日志和一条会话汇总,不含安装标识。",
+ "errors": "Peach 版本、分发方式、运行环境与公开构建编号、系统类型、Python 版本、CPU 架构、故障环节、异常链、Peach 与白名单框架的代码文件名/函数名/行号、结构化错误码、媒体工具故障类别、浏览器白名单异常类型与正式脚本构建映射、HTTP 方法/路由模板/关联错误编号、同类故障次数与统计日期;同一版本的同类故障只上传一次完整诊断,每天最多 5 条,之后的次数每 6 小时或退出时合并为计数日志。",
}
NOTICE = "发送至 Sentry。不上报错误原文、完整日志、诊断 ZIP、媒体名称、路径、网址、观看记录或凭据。接收服务会接触连接 IP;关闭后停止新发送,已发送的信息无法撤回。"
_lock = threading.RLock()
@@ -29,6 +29,7 @@ _client_dsn = ""
_disabled = False
_frequency_timer = None
_frequency_configs = {}
+TELEMETRY_ITEMS = frozenset({"log", "sessions"})
MEDIA_STAGES = frozenset({"frame-capture", "frame-tile", "media-probe", "preview", "library-preview"})
LOCAL_EXCEPTION_TYPES = frozenset({
("asyncio.exceptions", "CancelledError"), ("concurrent.futures._base", "CancelledError"),
@@ -121,7 +122,7 @@ def save(body, config=None):
def _reserve(config, kind, event):
- """跨托盘与子服务共享每日上限;额度文件只存指纹。"""
+ """版本统计跨托盘与子服务每天只发一次;额度文件只存指纹。"""
from filelock import FileLock
from .fsutil import atomic_write_text
path = config.directory("state") / "remote-reporting-budget.json"
@@ -135,7 +136,7 @@ def _reserve(config, kind, event):
return False
fingerprint = hashlib.sha256(json.dumps({key: value for key, value in event.items()
if key != "event_id"}, sort_keys=True).encode()).hexdigest()
- if fingerprint in seen or len(seen) >= (1 if kind == "usage" else 20):
+ if seen:
return False
value[kind] = [*seen, fingerprint]
atomic_write_text(path, json.dumps(value))
@@ -146,21 +147,29 @@ def _new_client(target, *, envelope_gate=None, delivery_callback=None):
from sentry_sdk import Client
from .reporting_transport import transport_class
+ def permitted(permit):
+ try:
+ current = _consent(Path(permit["consent"]))
+ return all(current.get(key) is True for key in FIELDS) and current["revision"] == permit["revision"]
+ except Exception:
+ return False
+
def gate(envelope):
if _disabled:
return False
if envelope_gate is not None:
return envelope_gate(envelope)
+ # 日志与会话汇总没有 extra,同意凭据在信封头;只有 Peach 自己组装、带凭据的信封能放行这两类条目。
+ telemetry = "peach_permit" in envelope.headers
+ if telemetry and not permitted(envelope.headers.pop("peach_permit")):
+ return False
for item in envelope.items:
- event = item.get_event()
- if event is None:
+ if (item.type in TELEMETRY_ITEMS) != telemetry:
return False
- permit = event.pop("extra", {})
- try:
- current = _consent(Path(permit["consent"]))
- if not all(current.get(key) is True for key in FIELDS) or current["revision"] != permit["revision"]:
- return False
- except Exception:
+ if telemetry:
+ continue
+ event = item.get_event()
+ if event is None or not permitted(event.pop("extra", {})):
return False
for key in tuple(envelope.headers):
if key not in {"event_id", "sent_at"}:
@@ -186,19 +195,67 @@ def _new_client(target, *, envelope_gate=None, delivery_callback=None):
before_send=final_event)
-def _submit(kind, event, config, current, target):
+def _connect(target):
global _client, _client_dsn
+ if _client is None or _client_dsn != target:
+ if _client is not None:
+ _client.close(timeout=0)
+ _client, _client_dsn = _new_client(target), target
+ return _client
+
+
+def _permit(kind, config, current):
+ return {"consent": str(config.directory("state") / "remote-reporting.json"),
+ "kind": kind, "revision": current["revision"]}
+
+
+def _attribute(value):
+ kind = {bool: "boolean", int: "integer", float: "double"}.get(type(value), "string")
+ return {"value": value if kind != "string" else str(value)[:200], "type": kind}
+
+
+def _log_item(body, event, extra=None):
+ """事件的版本、环境与白名单标签转成日志属性;不经过 SDK,避免附加主机名与追踪上下文。"""
+ attributes = {"sentry.release": event.get("release", ""), "sentry.environment": event.get("environment", ""),
+ **({"sentry.dist": event["dist"]} if event.get("dist") else {}),
+ **event.get("tags", {}), **(extra or {})}
+ return {"timestamp": datetime.now(timezone.utc).timestamp(), "trace_id": uuid.uuid4().hex,
+ "level": "info", "body": body,
+ "attributes": {key: _attribute(value) for key, value in attributes.items() if value != ""}}
+
+
+def _submit_telemetry(kind, items, config, current, target):
+ """日志与会话汇总组成一个信封交给同一传输;同意凭据只在信封头,闸门发送前移除。"""
+ from sentry_sdk.envelope import Envelope, Item, PayloadRef
+ from sentry_sdk.utils import format_timestamp
with _lock:
- if _client is None or _client_dsn != target:
- if _client is not None:
- _client.close(timeout=0)
- _client, _client_dsn = _new_client(target), target
+ client = _connect(target)
from .reporting_delivery import record
- permit = {"consent": str(config.directory("state") / "remote-reporting.json"),
- "kind": kind, "revision": current["revision"]}
+ receipt = uuid.uuid4().hex
+ permit = {**_permit(kind, config, current), "receipt": receipt}
+ envelope = Envelope(headers={"sent_at": format_timestamp(datetime.now(timezone.utc)), "peach_permit": permit})
+ logs = [item for item in items if "body" in item]
+ if logs:
+ envelope.add_item(Item(type="log", content_type="application/vnd.sentry.items.log+json",
+ headers={"item_count": len(logs)},
+ payload=PayloadRef(json={"version": 2, "items": logs})))
+ for session in (item for item in items if "aggregates" in item):
+ envelope.add_item(Item(type="sessions", payload=PayloadRef(json=session)))
+ record(permit, receipt, {"status": "queued"})
+ if client.transport is None:
+ return False
+ client.transport.capture_envelope(envelope)
+ return True
+
+
+def _submit(kind, event, config, current, target):
+ with _lock:
+ client = _connect(target)
+ from .reporting_delivery import record
+ permit = _permit(kind, config, current)
event_id = event.get("event_id") or uuid.uuid4().hex
record(permit, event_id, {"status": "queued"})
- accepted = _client.capture_event({**event, "event_id": event_id,
+ accepted = client.capture_event({**event, "event_id": event_id,
"platform": event.get("platform", "python"), "extra": permit})
if not accepted:
record(permit, event_id, {"status": "failed", "reason": "queue_full"})
@@ -227,6 +284,19 @@ def _frequency_tick():
_schedule_frequency(next(iter(_frequency_configs.values())))
+def _count_logs(config, current, target):
+ """计数日志带整数 `occurrence_count`,接收端可按诊断键、日期与版本直接求和。"""
+ def submit(counts):
+ items = []
+ for event in counts:
+ values = event.get("exception", {}).get("values", [])
+ extra = {"occurrence_count": int(event["tags"]["occurrence_count"]),
+ **({"exception_type": values[-1].get("type", "")} if values else {})}
+ items.append(_log_item("Peach diagnostic count", event, extra))
+ return _submit_telemetry("errors", items, config, current, target)
+ return submit
+
+
def flush_counts(config=None, *, force=False):
try:
from . import diagnostic_frequency
@@ -235,7 +305,8 @@ def flush_counts(config=None, *, force=False):
if _disabled or not current["errors"] or not target:
return
diagnostic_frequency.flush(config, current["revision"],
- lambda event: _submit("errors", event, config, current, target), force=force)
+ lambda event: _submit("errors", event, config, current, target),
+ _count_logs(config, current, target), force=force)
except Exception:
pass
@@ -255,10 +326,15 @@ def send(kind, event, config=None):
if kind == "errors":
from . import diagnostic_frequency
diagnostic_frequency.record(config, current["revision"], event,
- lambda batch: _submit(kind, batch, config, current, target))
+ lambda batch: _submit(kind, batch, config, current, target),
+ _count_logs(config, current, target))
_schedule_frequency(config)
elif _reserve(config, kind, event):
- _submit(kind, event, config, current, target)
+ # 版本统计是一条日志加一条会话汇总,不占错误事件额度(ADR-0132)。
+ day = datetime.now(timezone.utc).date().isoformat()
+ session = {"attrs": {"release": event["release"], "environment": event["environment"]},
+ "aggregates": [{"started": f"{day}T00:00:00Z", "exited": 1}]}
+ _submit_telemetry(kind, [_log_item(event.get("message", ""), event), session], config, current, target)
return True
except Exception:
# 不能把遥测自身故障写回错误钩子而递归发送。
diff --git a/src/peach/reporting_transport.py b/src/peach/reporting_transport.py
index b5e8f33dd..a7d0f23c9 100644
--- a/src/peach/reporting_transport.py
+++ b/src/peach/reporting_transport.py
@@ -19,7 +19,13 @@ def transport_class(gate, callback):
super().__init__(options)
def _context(self, envelope):
- event = next((item.get_event() for item in envelope.items if item.get_event() is not None), {})
+ event = next((item.get_event() for item in envelope.items if item.get_event() is not None), None)
+ if event is None:
+ # 日志与会话汇总的同意凭据和回执编号在信封头;闸门移除前先取出。
+ permit = envelope.headers.get("peach_permit", {})
+ self._delivery_context.permit = permit
+ self._delivery_context.event_id = permit.get("receipt") if isinstance(permit, dict) else None
+ return
self._delivery_context.permit = event.get("extra", {})
self._delivery_context.event_id = event.get("event_id")
diff --git a/tests/test_remote_reporting.py b/tests/test_remote_reporting.py
index 5a66edf02..6c00c68d8 100644
--- a/tests/test_remote_reporting.py
+++ b/tests/test_remote_reporting.py
@@ -53,13 +53,15 @@ class RemoteReportingTests(unittest.TestCase):
self.assertTrue(reporting.snapshot()["errors"])
reporting.usage()
reporting.capture("unhandled-main", ValueError("private"))
- self.assertEqual(client.capture_event.call_count, 2)
+ self.assertEqual(client.capture_event.call_count, 1)
+ self.assertEqual(client.transport.capture_envelope.call_count, 1)
reporting.save({"usage": False, "errors": False})
self.assertFalse(reporting.snapshot()["usage"])
self.assertFalse(reporting.snapshot()["errors"])
reporting.usage()
reporting.capture("unhandled-main", ValueError("private"))
- self.assertEqual(client.capture_event.call_count, 2)
+ self.assertEqual(client.capture_event.call_count, 1)
+ self.assertEqual(client.transport.capture_envelope.call_count, 1)
def test_consent_requires_joint_choice_and_missing_endpoint_is_visible(self):
for invalid in ({"usage": "false", "errors": False}, {"usage": True}, [],
@@ -102,6 +104,25 @@ class RemoteReportingTests(unittest.TestCase):
with self.subTest(value=value), mock.patch.dict(os.environ, {"PEACH_SENTRY_DSN": value}):
self.assertEqual(reporting.dsn(), "")
+ @staticmethod
+ def _telemetry(client):
+ """内存客户端收到的日志与会话条目,按信封顺序展开。"""
+ items = []
+ for call in client.transport.capture_envelope.call_args_list:
+ for item in call.args[0].items:
+ payload = item.payload.json
+ if item.type == 'log':
+ items.extend(('log', log) for log in payload['items'])
+ else:
+ items.append((item.type, payload))
+ return items
+
+ @staticmethod
+ def _counts(client):
+ return [{key: value['value'] for key, value in log['attributes'].items()}
+ for kind, log in RemoteReportingTests._telemetry(client)
+ if kind == 'log' and log['body'] == 'Peach diagnostic count']
+
def test_daily_budget_is_shared_deduplicated_and_bounded(self):
reporting.save({"usage": True, "errors": True})
client = mock.Mock()
@@ -112,24 +133,32 @@ class RemoteReportingTests(unittest.TestCase):
event = {"message": str(index)}
reporting.send("errors", event)
reporting.send("errors", event)
- self.assertEqual(client.capture_event.call_count, 21)
+ self.assertEqual(client.capture_event.call_count, 5)
+ self.assertEqual(client.transport.capture_envelope.call_count, 1)
payload = json.loads((self.config.directory("state") / "remote-reporting-frequency.json").read_text())
self.assertEqual(len(payload["records"]), 20)
self.assertTrue(all(row['total'] == 2 for row in payload['records'].values()))
- def test_usage_counts_distribution_and_coarse_machine_facts(self):
+ def test_usage_is_one_log_and_session_without_error_events(self):
from peach import reporting_identity as identity
reporting.save({"usage": True, "errors": True})
client = mock.Mock()
with mock.patch.object(reporting, "_new_client", return_value=client), \
mock.patch.object(identity, "_memory_bytes", return_value=15.8 * 2 ** 30):
reporting.usage()
- tags = client.capture_event.call_args.args[0]["tags"]
- self.assertEqual(tags["distribution"], "source")
- self.assertEqual(tags["memory_gb"], "16")
- self.assertEqual(tags["cpu_cores"], str(os.cpu_count()))
- for key in ("os", "os_version", "architecture", "python"):
- self.assertTrue(tags[key], key)
+ client.capture_event.assert_not_called()
+ (kind, log), (session_kind, session) = self._telemetry(client)
+ self.assertEqual((kind, session_kind, log['body']), ('log', 'sessions', 'Peach started'))
+ attributes = {key: value['value'] for key, value in log['attributes'].items()}
+ self.assertEqual(attributes["purpose"], "version-statistics")
+ self.assertEqual(attributes["distribution"], "source")
+ self.assertEqual(attributes["memory_gb"], "16")
+ self.assertEqual(attributes["cpu_cores"], str(os.cpu_count()))
+ self.assertEqual(attributes["sentry.release"], session['attrs']['release'])
+ for key in ("os", "os_version", "architecture", "python", "sentry.environment"):
+ self.assertTrue(attributes[key], key)
+ self.assertEqual(session['aggregates'][0]['exited'], 1)
+ self.assertEqual(set(session['attrs']), {'release', 'environment'})
def test_distribution_tells_desktop_apps_docker_wheel_and_source_apart(self):
from peach import reporting_identity as identity
@@ -154,7 +183,7 @@ class RemoteReportingTests(unittest.TestCase):
with self.subTest(gigabytes=gigabytes):
self.assertEqual(memory_bucket(gigabytes * 2 ** 30), bucket)
- def test_frequency_batches_send_disjoint_counts_and_keep_group_identity(self):
+ def test_frequency_sends_one_event_then_count_logs_with_the_same_key(self):
from peach import diagnostic_frequency as frequency
reporting.save({"usage": True, "errors": True})
client = mock.Mock()
@@ -168,12 +197,17 @@ class RemoteReportingTests(unittest.TestCase):
self.assertEqual(first['tags']['occurrence_count'], '1')
clock.return_value = now + timedelta(hours=1)
reporting.flush_counts()
- second = client.capture_event.call_args.args[0]
- self.assertEqual(second['tags']['occurrence_count'], '4')
- self.assertEqual(second['tags']['diagnostic_key'], first['tags']['diagnostic_key'])
- self.assertEqual(second['tags']['occurrence_day'], '2026-10-18')
+ self.assertEqual(self._counts(client), [])
+ clock.return_value = now + timedelta(hours=6)
+ reporting.flush_counts()
reporting.flush_counts(force=True)
- self.assertEqual(client.capture_event.call_count, 2)
+ [count] = self._counts(client)
+ self.assertEqual(count['occurrence_count'], 4)
+ self.assertEqual(count['diagnostic_key'], first['tags']['diagnostic_key'])
+ self.assertEqual(count['occurrence_day'], '2026-10-18')
+ self.assertEqual(count['exception_type'], 'ValueError')
+ self.assertEqual(count['purpose'], 'technical-diagnostics')
+ self.assertEqual(client.capture_event.call_count, 1)
payload = (self.config.directory('state') / 'remote-reporting-frequency.json').read_text()
self.assertNotIn('private title', payload)
self.assertNotIn('installation', payload)
@@ -188,11 +222,10 @@ class RemoteReportingTests(unittest.TestCase):
self.assertEqual(first['event_id'], 'a' * 32)
reporting.close()
reporting.flush_counts(force=True)
- second = client.capture_event.call_args.args[0]
- self.assertNotEqual(second['event_id'], first['event_id'])
- self.assertEqual(second['tags']['occurrence_count'], '2')
- self.assertEqual(first['tags']['diagnostic_key'], second['tags']['diagnostic_key'])
- self.assertEqual(client.capture_event.call_count, 2)
+ [count] = self._counts(client)
+ self.assertEqual(count['occurrence_count'], 2)
+ self.assertEqual(count['diagnostic_key'], first['tags']['diagnostic_key'])
+ self.assertEqual(client.capture_event.call_count, 1)
def test_frequency_pending_counts_expire_with_consent_and_expected_states_are_excluded(self):
import asyncio
@@ -208,28 +241,62 @@ class RemoteReportingTests(unittest.TestCase):
reporting.save({"usage": True, "errors": True})
reporting.flush_counts(force=True)
self.assertEqual(client.capture_event.call_count, 1)
+ self.assertEqual(self._counts(client), [])
reporting.capture('unhandled-main', ValueError('private'))
+ self.assertEqual(client.capture_event.call_count, 2)
self.assertEqual(client.capture_event.call_args.args[0]['tags']['occurrence_count'], '1')
- def test_frequency_daily_limit_defers_counts_and_preserves_occurrence_day(self):
+ def test_frequency_daily_event_limit_defers_first_events_and_keeps_their_day(self):
from peach import diagnostic_frequency as frequency
reporting.save({"usage": True, "errors": True})
client = mock.Mock()
now = datetime(2026, 10, 18, 12, tzinfo=timezone.utc)
with mock.patch.object(reporting, '_new_client', return_value=client), \
mock.patch.object(frequency, '_now', return_value=now) as clock:
- for index in range(20):
+ for index in range(7):
reporting.send('errors', {'message': str(index)})
reporting.send('errors', {'message': '0'})
reporting.flush_counts(force=True)
- self.assertEqual(client.capture_event.call_count, 20)
+ self.assertEqual(client.capture_event.call_count, frequency.EVENT_LIMIT)
+ self.assertEqual([count['occurrence_count'] for count in self._counts(client)], [1])
clock.return_value = now + timedelta(days=1)
reporting.flush_counts(force=True)
- self.assertEqual(client.capture_event.call_count, 21)
+ self.assertEqual(client.capture_event.call_count, 7)
self.assertEqual(client.capture_event.call_args.args[0]['tags']['occurrence_count'], '1')
self.assertEqual(client.capture_event.call_args.args[0]['tags']['occurrence_day'], '2026-10-18')
- def test_frequency_sdk_rejection_keeps_pending_counts_for_later_batch(self):
+ def test_frequency_counts_roll_over_by_occurrence_day(self):
+ from peach import diagnostic_frequency as frequency
+ reporting.save({"usage": True, "errors": True})
+ client = mock.Mock()
+ now = datetime(2026, 10, 18, 23, tzinfo=timezone.utc)
+ with mock.patch.object(reporting, '_new_client', return_value=client), \
+ mock.patch.object(frequency, '_now', return_value=now) as clock:
+ for _ in range(3):
+ reporting.capture('unhandled-main', ValueError('private'))
+ clock.return_value = now + timedelta(hours=2)
+ reporting.capture('unhandled-main', ValueError('private'))
+ reporting.flush_counts(force=True)
+ self.assertEqual(client.capture_event.call_count, 1)
+ self.assertEqual([(count['occurrence_day'], count['occurrence_count']) for count in self._counts(client)],
+ [('2026-10-18', 2), ('2026-10-19', 1)])
+
+ def test_frequency_resends_one_event_after_the_key_goes_quiet(self):
+ from peach import diagnostic_frequency as frequency
+ reporting.save({"usage": True, "errors": True})
+ client = mock.Mock()
+ now = datetime(2026, 10, 18, 12, tzinfo=timezone.utc)
+ with mock.patch.object(reporting, '_new_client', return_value=client), \
+ mock.patch.object(frequency, '_now', return_value=now) as clock:
+ reporting.capture('unhandled-main', ValueError('private'))
+ clock.return_value = now + timedelta(days=frequency.RETENTION_DAYS - 1)
+ reporting.capture('unhandled-main', ValueError('private'))
+ self.assertEqual(client.capture_event.call_count, 1)
+ clock.return_value = now + timedelta(days=2 * frequency.RETENTION_DAYS)
+ reporting.capture('unhandled-main', ValueError('private'))
+ self.assertEqual(client.capture_event.call_count, 2)
+
+ def test_frequency_sdk_rejection_keeps_the_first_event_for_later(self):
reporting.save({"usage": True, "errors": True})
client = mock.Mock()
client.capture_event.return_value = None
@@ -240,6 +307,7 @@ class RemoteReportingTests(unittest.TestCase):
self.assertEqual(client.capture_event.call_args.args[0]['tags']['occurrence_count'], '1')
reporting.flush_counts(force=True)
self.assertEqual(client.capture_event.call_count, 2)
+ self.assertEqual(self._counts(client), [])
def test_frequency_shared_file_serializes_independent_processes(self):
from peach import diagnostic_frequency as frequency
@@ -248,7 +316,8 @@ class RemoteReportingTests(unittest.TestCase):
program = ('from peach import diagnostic_frequency as f; '
'from peach.settings_file import active; '
'from peach.remote_reporting import consent; '
- 'f.record(active(), consent()["revision"], {"release":"test", "tags":{"stage":"cli"}}, lambda e: True)')
+ 'f.record(active(), consent()["revision"], {"release":"test", "tags":{"stage":"cli"}}, '
+ 'lambda e: True, lambda c: True)')
env = {**os.environ, 'PYTHONPATH': str(root / 'src'), 'PEACH_SENTRY_DSN': ''}
processes = [subprocess.Popen([sys.executable, '-X', 'utf8', '-c', program], env=env,
stdout=subprocess.PIPE, stderr=subprocess.PIPE) for _ in range(2)]
@@ -260,10 +329,11 @@ class RemoteReportingTests(unittest.TestCase):
if process.poll() is None:
process.kill()
process.communicate()
- sent = []
- frequency.flush(self.config, reporting.consent()['revision'], lambda e: sent.append(e) or True, force=True)
- self.assertEqual(len(sent), 1)
- self.assertEqual(sent[0]['tags']['occurrence_count'], '1')
+ events, counts = [], []
+ frequency.flush(self.config, reporting.consent()['revision'], lambda e: events.append(e) or True,
+ lambda batch: counts.extend(batch) or True, force=True)
+ self.assertEqual(events, [])
+ self.assertEqual([count['tags']['occurrence_count'] for count in counts], ['1'])
def test_frequency_shutdown_submits_pending_counts_and_stops_timer(self):
reporting.save({"usage": True, "errors": True})
@@ -272,8 +342,8 @@ class RemoteReportingTests(unittest.TestCase):
for _ in range(3):
reporting.capture('unhandled-main', ValueError('private'))
reporting._shutdown()
- self.assertEqual(client.capture_event.call_count, 2)
- self.assertEqual(client.capture_event.call_args.args[0]['tags']['occurrence_count'], '2')
+ self.assertEqual(client.capture_event.call_count, 1)
+ self.assertEqual([count['occurrence_count'] for count in self._counts(client)], [2])
self.assertIsNone(reporting._frequency_timer)
self.assertFalse(reporting._frequency_configs)
@@ -294,10 +364,10 @@ class RemoteReportingTests(unittest.TestCase):
for _ in range(4):
reporting.capture('unhandled-main', ValueError('private'))
callback = timer.call_args.args[1]
- clock.return_value = now + timedelta(hours=1)
+ clock.return_value = now + timedelta(seconds=frequency.INTERVAL)
callback()
- self.assertEqual(client.capture_event.call_count, 2)
- self.assertEqual(client.capture_event.call_args.args[0]['tags']['occurrence_count'], '3')
+ self.assertEqual(client.capture_event.call_count, 1)
+ self.assertEqual([count['occurrence_count'] for count in self._counts(client)], [3])
def test_actionable_faults_keep_codes_and_omit_private_text(self):
with mock.patch.object(reporting, "send") as send:
@@ -514,6 +584,50 @@ class RemoteReportingTests(unittest.TestCase):
client.transport._send_envelope(envelope)
network.assert_not_called()
+ def test_telemetry_envelopes_need_a_current_permit_and_events_cannot_carry_them(self):
+ from sentry_sdk.transport import HttpTransport
+ from sentry_sdk.envelope import Envelope, Item, PayloadRef
+ reporting.save({"usage": True, "errors": True})
+ current = reporting.consent()
+ client = reporting._new_client(DSN)
+ self.addCleanup(lambda: client.close(timeout=0))
+ permit = {"consent": str(self.config.directory('state') / 'remote-reporting.json'),
+ "kind": "usage", "revision": current['revision'], "receipt": 'r' * 32}
+
+ def envelope(headers, *items):
+ built = Envelope(headers=headers)
+ for kind in items:
+ if kind == 'event':
+ built.add_event({"message": "fixture", "extra": dict(permit)})
+ else:
+ built.add_item(Item(type=kind, payload=PayloadRef(json={"items": []})))
+ return built
+
+ sent = []
+ with mock.patch.object(HttpTransport, "_send_envelope", side_effect=sent.append):
+ client.transport._send_envelope(envelope({"peach_permit": dict(permit)}, 'log', 'sessions'))
+ self.assertEqual(len(sent), 1)
+ self.assertNotIn('peach_permit', sent[0].headers)
+ for blocked in (envelope({}, 'log'), envelope({"peach_permit": dict(permit)}, 'log', 'event'),
+ envelope({}, 'event', 'sessions'),
+ envelope({"peach_permit": {**permit, "revision": "old"}}, 'log')):
+ client.transport._send_envelope(blocked)
+ self.assertEqual(len(sent), 1)
+
+ def test_usage_wire_payload_has_no_host_sdk_or_consent_path(self):
+ from sentry_sdk.transport import HttpTransport
+ sent = []
+ reporting.save({"usage": True, "errors": True})
+ with mock.patch.object(HttpTransport, "_send_envelope", side_effect=lambda envelope: sent.append(envelope.serialize())):
+ reporting.usage()
+ reporting._client.flush(timeout=2)
+ self.assertEqual(len(sent), 1)
+ wire = sent[0].decode()
+ for value in (str(self.root), 'server.address', 'sentry.sdk', 'consent', 'peach_permit', 'receipt'):
+ self.assertNotIn(value, wire)
+ self.assertIn('"type":"log"', wire.replace(' ', ''))
+ self.assertIn('"aggregates"', wire)
+
def test_sdk_failure_does_not_break_local_error_recording(self):
from peach import error_reporting
reporting.save({"usage": True, "errors": True})