Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

fix(diagnostics): 完成断连资源收尾并筛选预期状态

longmeidao566267635063
diff --git a/docs/DIAGNOSTICS_AUDIT.md b/docs/DIAGNOSTICS_AUDIT.md
index f45ee347a..ea7ac0e49 100644
--- a/docs/DIAGNOSTICS_AUDIT.md
+++ b/docs/DIAGNOSTICS_AUDIT.md
@@ -43,7 +43,27 @@
回归验收使用临时数据根和 SDK 内存接收端。本机已配置独立的 `peach-errors` 与 `peach-library-research` 接收项目,合成错误 `PEACH-ERRORS-1` 已在 Sentry 核对代码位置;没有发送真实馆藏。发布工作流的两个公开 DSN 仓库变量已配置并回读核对,构建包从变量取得接收地址;配置变量不触发发布。技术诊断选择见 [ADR-0110](adr/0110-recommended-diagnostics-choice.md)。安装辅助进程只在本机取证,不初始化远程客户端或发送事件;应用进程按用户选择发送。
-`asyncio` 连接清理只在同时满足以下条件时保留为本机诊断:`ConnectionResetError` 的 `winerror` 为 `10054`,回调为标准库 `_ProactorBasePipeTransport._call_connection_lost`,调用栈只含该回调与 `asyncio.events._run`,且没有关联异常。该分类不发送至 Sentry;其他错误码、回调、业务栈与关联异常继续按技术故障规则处理。本机记录保留异常与标准库帧,并补充回调、协议类、可识别的服务端类别和 TLS 标志;这些上下文字段不进入远程载荷,不访问连接地址、请求内容或对象文本。默认事件循环处理器仍执行,底层连接清理行为沿用 Python 标准库。
+`asyncio` 连接清理只在同时满足以下条件时识别为连接重置收尾:`ConnectionResetError` 的 `winerror` 为 `10054`,回调为标准库 `_ProactorBasePipeTransport._call_connection_lost`,调用栈只含该回调与 `asyncio.events._run`,且没有关联异常。处理器核对真实 Proactor transport 处于清理状态,关闭 socket、解除服务端登记并完成关闭状态;不重复通知协议。成功记录 `classification=connection-cleanup` 与 `cleanup_recovered=true`,仅保留本机证据。收尾自身失败则保留原始异常链、上报技术故障并交给默认处理器;不能确认对象状态时也沿用默认处理器。没有全局修改 Python 类或替换事件循环。
+
+本机记录保留异常与标准库帧,并补充回调、协议类、可识别的服务端类别和 TLS 标志;这些上下文字段不进入远程载荷,不访问连接地址、请求内容或对象文本。资源收尾使用 CPython 3.14.7 的私有对象协议,升级需验证实际 socket、`asyncio.Server`、重复关闭与收尾失败。原始竞争时序在 33 次回环实验中未复现;合成故障验证了资源实际关闭与服务端释放,不能据此推断原始客户端来源。
+
+### 远程错误筛选核验
+
+核验覆盖异常钩子、日志、后台任务、来源、HTTP、媒体、浏览器、启动与更新入口。筛选依据结构化状态和实际异常继承关系,不按错误原文、任意同名类或所有网络异常统一过滤。
+
+| 状态 | 本机与远程处理 |
+| --- | --- |
+| 任务取消、暂停、互斥、计费与磁盘策略门槛 | 保留本机异常和任务回执,不占用错误上传额度 |
+| 来源冷却、动作预算、授权、未命中、明确的临时故障 | 复用来源异常与分类,保留本机状态;来源解析错误和未知失败继续上报 |
+| HTTP 客户端错误 | 识别直接状态码、httpx 响应及 urllib HTTPError;4xx 留在本机,未分类的 5xx 继续上报;`0` 不作为 HTTP 状态码 |
+| warning 与重试日志 | 低于 error 的日志留在本机;error/critical 携带 Peach 栈时上报 |
+| 媒体输入与环境条件 | 坏媒体、输入缺失、权限、空间与媒体工具超时留在本机;工具执行文件缺失、解码器能力缺失和未知技术错误继续上报;预览前置条件保留提示和可重试问题项 |
+| 浏览器取消与扩展 | 主动取消请求的 AbortError 不发起记录请求;后端收到该类型仍仅写本机;扩展 URL 帧不上传,Peach 带构建参数的脚本帧保留行列位置 |
+| 客户端中途断开 | Starlette 的 ClientDisconnect 留在本机,不当作服务端程序故障 |
+| 启动、CLI、托盘、更新与回滚 | 未处理的技术故障继续上报;安装辅助进程沿用仅本机诊断边界 |
+| SDK、无堆栈日志与不透明浏览器错误 | 不递归上报 SDK 自身问题;无法取得 Peach 代码位置的日志和浏览器错误仅保留本机 |
+
+本机策略不清除已有 Sentry 事件,也不控制邮件;邮件摘要与即时通知分别在接收端设置。
### 馆藏与口味群体研究
@@ -89,11 +109,11 @@ ZIP 不包含 ledger、配置文件、私钥、登录 Cookie 或媒体文件。
| 影响面 | 适用性 |
| --- | --- |
| 数据层 | 本机独立同意与限额文件;研究只读账本,复用派生偏好;没有迁移或真实 ledger 写入 |
-| API/契约 | 诊断选择读写、完整研究预览与同意写入;本机、鉴权、同源限制 |
+| API/契约 | 诊断选择、研究预览与同意;浏览器接收可选异常类型用于取消识别;本机、鉴权、同源限制 |
| 页面 | 配置页「应用与维护」、首次设置;Windows 安装器默认勾选推荐的诊断选项;其他馆藏页面不适用 |
| 反向状态 | 诊断与研究分别关闭;关闭使排队载荷失效,研究重新开启须预览并同意 |
| 视口 | 桌面和 390×844 手机验证明细折行、内部滚动、同意与关闭;无横向溢出 |
-| 测试 | 诊断默认开启、取消与持久关闭、研究默认关闭、脱敏、白名单、限额、SDK 内存传输、权限及界面回归 |
+| 测试 | 诊断与研究选择、SDK 内存传输、脱敏与权限;实际 socket 和服务端清理、收尾失败、预期状态、解析错误、浏览器取消与构建参数堆栈 |
| 文档 | 中英文 README、STATUS、REUSE、待办、本记录与 ADR-0109 |
| 智能体入口 | 没有新增规则、技能或入口 |
| 生产 | 本机源码运行态按托盘入口重启并严格校验 HTTPS;外部测试包发布独立验收 |
diff --git a/docs/REUSE.md b/docs/REUSE.md
index 5ab75e4d3..4e3f9b88a 100644
--- a/docs/REUSE.md
+++ b/docs/REUSE.md
@@ -150,7 +150,7 @@ Peach 播放器皮肤直接装饰 Video.js 10 的 light DOM:右侧控制组用
可选远程诊断复用官方 [sentry-sdk 2.72.0](https://github.com/getsentry/sentry-python/tree/2.72.0)(MIT,维护中,支持 Python 3.14);固定版源码通过 `opensrc path pypi:sentry-sdk` 核对。SDK 负责后台 HTTP、TLS、Sentry 信封和服务端限流;Peach 负责本机选择、技术字段投影、同类错误去重与每机每日上限。版本统计与错误诊断共用默认勾选的开关,关闭时不构造 SDK;SDK 全局自动集成、追踪、回放、日志、会话与指标采集均关闭。`HttpTransport._send_envelope` 的窄覆盖在真正发送前复查本机选择版本并移除本机传输元数据;它是固定版本的内部接口,升级必须通过真实 SDK 信封回归。无真实外发的内存接收端检查最终字节、停用后队列失效与断网不影响本机取证。新增依赖仅 `sentry-sdk`,复用已声明的 urllib3 / certifi;体积以发行构建为准。未采用自写 Sentry 协议或 Workers 收集后台,避免另行维护传输、去重和查看界面。没有真实 ledger 写入。
-Windows 连接清理诊断复用 CPython 3.14.7 的 `asyncio` 回调与异常栈(PSF 许可,无新增依赖)。只将纯标准库 `_call_connection_lost` 回调中的 `WinError 10054` 分类为本机连接清理证据,保留其他网络故障。核对本机 `proactor_events.py` 源码与重启前后日志;33 次回环实验未复现原始竞争时序,不据此认定客户端来源。合成异常通过真实 `asyncio.Handle` 分派验证筛选与默认处理器,协议类别只写本机日志。没有修改或复制传输实现、全局替换事件循环,或引入另一套日志 SDK。
+Windows 连接清理复用 [CPython 3.14.7 的 Proactor 对象协议](https://github.com/python/cpython/blob/v3.14.7/Lib/asyncio/proactor_events.py)(PSF 许可,无新增依赖)。精确识别纯标准库清理回调中的 `WinError 10054`,只补完 socket 关闭、服务端解除登记和关闭状态,不重复通知协议,不全局修改标准库类或替换事件循环。它使用 `_sock`、`_server._detach` 与 `_called_connection_lost` 私有接口,升级 Python 时须核验。真实 socket 和 `asyncio.Server` 配合合成故障验证资源释放;收尾自身失败保留原始异常链并上报。33 次回环实验未复现原始竞争时序,不据此认定客户端来源。协议类别只写本机日志;来源、任务、媒体和浏览器的远程筛选复用现有领域状态,见 [诊断核验](DIAGNOSTICS_AUDIT.md#远程错误筛选核验)。
馆藏与口味群体研究复用 `LedgerDatabase.read_connection`、`taste_preferences.preference_profile`、`error_reporting.redact` 与同一固定版 Sentry 的内存附件 API;偏好权重与身份口径不另写一份。研究使用独立项目、独立同意文件和字段版本;默认关闭,预览只在用户点击时读取,后台读取要求有效同意与接收端。没有复制浏览器资料、原始历史或媒体,没有真实 ledger 写入。SDK 本机安装目录含缓存约 3.1 MB,Windows 构建复制 MIT 许可证元数据;压缩后的发行增量未单独测量。
diff --git a/frontend/src/error-reporting.ts b/frontend/src/error-reporting.ts
index 46ae40559..1c6e07a3e 100644
--- a/frontend/src/error-reporting.ts
+++ b/frontend/src/error-reporting.ts
@@ -8,6 +8,8 @@ export function installErrorReporting(): void {
let count = 0;
const seen = new Set<string>();
const send = (kind: string, cause: unknown, fallback = '', source?: ErrorEvent) => {
+ const errorName = cause instanceof Error || cause instanceof DOMException ? cause.name : '';
+ if (errorName === 'AbortError') return;
if (Date.now() - start >= 60_000) {
start = Date.now(); count = 0; seen.clear();
}
@@ -20,7 +22,7 @@ export function installErrorReporting(): void {
seen.add(key); count += 1;
void fetch('/api/diagnostics/client-error', {
method: 'POST', headers: { 'Content-Type': 'application/json' },
- body: JSON.stringify({ kind, message, stack, page: window.location.pathname,
+ body: JSON.stringify({ kind, message, stack, error_name: errorName, page: window.location.pathname,
source_file: source?.filename?.split('/').pop()?.split('?')[0],
line: source?.lineno, column: source?.colno }), credentials: 'same-origin', keepalive: true,
}).catch(() => {});
diff --git a/frontend/test/error-reporting.test.ts b/frontend/test/error-reporting.test.ts
index 1ea957376..db6c21d4b 100644
--- a/frontend/test/error-reporting.test.ts
+++ b/frontend/test/error-reporting.test.ts
@@ -27,3 +27,21 @@ it('浏览器异常有上限、去重,网络失败不会触发递归上报', a
await Promise.resolve();
expect(fetch).toHaveBeenCalledTimes(10);
});
+
+it('主动取消请求不生成错误记录,真实异常保留类型', async () => {
+ const listeners = new Map<string, (event: unknown) => void>();
+ vi.spyOn(window, 'addEventListener').mockImplementation((name, callback) => {
+ listeners.set(name, callback as (event: unknown) => void);
+ });
+ const fetch = vi.fn().mockResolvedValue({ ok: true });
+ vi.stubGlobal('fetch', fetch);
+ const { installErrorReporting } = await import('../src/error-reporting');
+ installErrorReporting();
+ const cancelled = new DOMException('request cancelled', 'AbortError');
+ listeners.get('unhandledrejection')!({ reason: cancelled });
+ listeners.get('error')!({ error: cancelled, message: cancelled.message });
+ expect(fetch).not.toHaveBeenCalled();
+ listeners.get('unhandledrejection')!({ reason: new TypeError('render failed') });
+ expect(fetch).toHaveBeenCalledTimes(1);
+ expect(JSON.parse(fetch.mock.calls[0]![1].body)).toMatchObject({ error_name: 'TypeError' });
+});
diff --git a/src/peach/error_reporting.py b/src/peach/error_reporting.py
index 5fa4cf962..d85aa0032 100644
--- a/src/peach/error_reporting.py
+++ b/src/peach/error_reporting.py
@@ -293,10 +293,35 @@ def is_asyncio_connection_cleanup(error, fields) -> bool:
return last == ("asyncio.proactor_events", "_call_connection_lost")
+def _finish_asyncio_connection_cleanup(context) -> bool:
+ """完成 CPython 清理回调中 shutdown 之后的资源收尾,不重复通知协议。"""
+ from asyncio.proactor_events import _ProactorBasePipeTransport
+ callback = getattr(context.get("handle"), "_callback", None)
+ transport = getattr(callback, "__self__", None)
+ if (not isinstance(transport, _ProactorBasePipeTransport) or not transport._closing
+ or transport._called_connection_lost or transport._sock is None):
+ return False
+ transport._sock.close()
+ transport._sock = None
+ if transport._server is not None:
+ transport._server._detach(transport)
+ transport._server = None
+ transport._called_connection_lost = True
+ return True
+
+
def asyncio_handler(loop, context) -> None:
error = context.get("exception")
fields = _asyncio_fields(context)
if is_asyncio_connection_cleanup(error, fields):
fields["classification"] = "connection-cleanup"
+ try:
+ fields["cleanup_recovered"] = _finish_asyncio_connection_cleanup(context)
+ except Exception as cleanup_error:
+ cleanup_error.__cause__ = error
+ error = cleanup_error
+ fields.update(classification="connection-cleanup-failed", cleanup_recovered=False)
+ context = {**context, "exception": error}
event("unhandled-asyncio", error, message=context.get("message", ""), **fields)
- loop.default_exception_handler(context)
+ if not fields.get("cleanup_recovered"):
+ loop.default_exception_handler(context)
diff --git a/src/peach/library_processing.py b/src/peach/library_processing.py
index 92584b50b..197f5564b 100644
--- a/src/peach/library_processing.py
+++ b/src/peach/library_processing.py
@@ -1833,6 +1833,7 @@ def _generate_preview(config, row, database, resolver, *, progress):
"""复用九宫格抽帧原语,产物可用后在共享写锁内登记。"""
from . import frame_capture, media_probe
from .fsutil import atomic_path
+ from .previews import PreviewUnavailable
root = (config.directory('generated') / 'snapshots').resolve()
previous = row.get('snapshot_path')
@@ -1842,19 +1843,19 @@ def _generate_preview(config, row, database, resolver, *, progress):
return False
choice = resolver.ffmpeg()
if choice is None:
- raise ValueError('未找到 FFmpeg,无法生成预览图,请在诊断页检查工具状态')
+ raise PreviewUnavailable('未找到 FFmpeg,无法生成预览图,请在诊断页检查工具状态')
video = translate_ledger_path(row['path'])
if not video.is_file():
- raise ValueError('媒体文件不可访问')
+ raise PreviewUnavailable('媒体文件不可访问')
duration = row.get('duration') or 0
if duration <= 0:
probe = resolver.ffprobe()
if probe is None:
- raise ValueError('未找到 ffprobe,无法读取视频时长')
+ raise PreviewUnavailable('未找到 ffprobe,无法读取视频时长')
measured = media_probe.measure(str(probe.path), row['id'], str(video))
duration = measured[0]
if duration <= 0:
- raise ValueError('视频时长未取得,请检查媒体文件后重试')
+ raise PreviewUnavailable('视频时长未取得,请检查媒体文件后重试')
progress()
with database.write_transaction() as connection:
connection.execute(media_probe.UPDATE, measured)
@@ -1896,6 +1897,7 @@ def _generate_preview(config, row, database, resolver, *, progress):
def _generate_previews(config, rows, database, *, update, issue, guard,
completed=(), checkpoint=lambda row: None):
"""只读本机视频;来源失败与抽帧失败各自保留可重试项目。"""
+ from .previews import PreviewUnavailable
if database is None:
return 0
rows = [row for row in rows if row['location'] == 'local' and not skip_background_read(row['path'])]
@@ -1918,10 +1920,10 @@ def _generate_previews(config, rows, database, *, update, issue, guard,
checkpoint(row)
except InterruptedError:
raise
- except (OSError, ValueError, subprocess.SubprocessError) as error:
+ except (OSError, ValueError, subprocess.SubprocessError, PreviewUnavailable) as error:
from .error_reporting import event
event('library-preview', error, asset_id=row['id'])
- message = (str(error) if isinstance(error, ValueError) else
+ message = (str(error) if isinstance(error, (ValueError, PreviewUnavailable)) else
'生成预览图超时,请重试' if isinstance(error, subprocess.SubprocessError) else
'生成预览图失败,请检查 FFmpeg 与目录权限后重试')
details = {'diagnostics': error.diagnostics} if isinstance(error, _PreviewCaptureError) else {}
diff --git a/src/peach/remote_reporting.py b/src/peach/remote_reporting.py
index 2a8327650..0078a9857 100644
--- a/src/peach/remote_reporting.py
+++ b/src/peach/remote_reporting.py
@@ -2,11 +2,13 @@
from __future__ import annotations
import atexit
+import errno
import hashlib
import json
import os
import platform
import re
+import subprocess
import threading
import uuid
from datetime import datetime, timezone
@@ -24,6 +26,17 @@ _lock = threading.RLock()
_client = None
_client_dsn = ""
_disabled = False
+MEDIA_STAGES = frozenset({"frame-capture", "frame-tile", "media-probe", "preview", "library-preview"})
+LOCAL_EXCEPTION_TYPES = frozenset({
+ ("asyncio.exceptions", "CancelledError"), ("concurrent.futures._base", "CancelledError"),
+ ("builtins", "InterruptedError"), ("peach.jobs", "JobPolicyError"),
+ ("peach.jav_cover_fetch", "Unavailable"), ("peach.jav_cover_fetch", "DeadlineExceeded"),
+ ("peach.scraping_access", "SourcePaused"), ("peach.previews", "PreviewUnavailable"),
+ ("peach.transcodes", "TranscodeCancelled"), ("peach.segments", "SegmentCancelled"),
+ ("peach.browser_transport", "BrowserUnavailable"), ("peach.browser_transport", "ChallengeUnsolved"),
+ ("peach.face_detect", "FaceModelUnavailable"), ("peach.avatar_watermark", "WatermarkModelUnavailable"),
+ ("starlette.requests", "ClientDisconnect"),
+})
def _config(config=None):
@@ -194,16 +207,28 @@ def usage(config=None):
send("usage", {"message": "Peach started", "level": "info", "tags": {"os": platform.system()}}, config)
+def _http_status(error):
+ status = getattr(error, "status_code", None)
+ if type(status) is not int or not 100 <= status <= 599:
+ status = getattr(getattr(error, "response", None), "status_code", None)
+ if type(status) is not int and any(cls.__module__ == "urllib.error" and cls.__name__ == "HTTPError"
+ for cls in type(error).__mro__):
+ status = getattr(error, "code", None)
+ return status if type(status) is int and 100 <= status <= 599 else None
+
+
def _exception_details(error):
frames, codes, chain = [], {}, []
current, visited = error, set()
while current is not None and id(current) not in visited and len(chain) < 8:
visited.add(id(current))
chain.append(type(current).__name__)
- for key in ("errno", "winerror", "returncode", "status_code"):
+ for key in ("errno", "winerror", "returncode"):
value = getattr(current, key, None)
if type(value) is int:
codes[key] = value
+ if (status := _http_status(current)) is not None:
+ codes["status_code"] = status
tb = current.__traceback__
while tb is not None and len(frames) < 40:
filename = Path(tb.tb_frame.f_code.co_filename)
@@ -217,17 +242,45 @@ def _exception_details(error):
def _media_details(error, fields, codes):
detail = getattr(error, "diagnostics", {}) if error is not None else {}
+ detail = detail if isinstance(detail, dict) else {}
if type(detail.get("returncode")) is int:
codes["returncode"] = detail["returncode"]
raw = str(fields.get("stderr", "")) + str(detail.get("stderr", ""))
for marker, code in (("Your platform doesn't support hardware accelerated AV1 decoding", "av1_decoder_unavailable"),
+ ("Unknown decoder", "decoder_unavailable"),
("Invalid data found", "invalid_media"), ("Permission denied", "permission_denied"),
- ("No space left", "disk_full"), ("Unknown decoder", "decoder_unavailable")):
+ ("No space left", "disk_full"), ("No such file or directory", "missing_media")):
if marker in raw:
return code
return ""
+def _expected_failure(stage, error, fields):
+ if stage == "logging" and fields.get("level", "ERROR") not in {"ERROR", "CRITICAL"}:
+ return True
+ if stage == "browser" and fields.get("error_name") == "AbortError":
+ return True
+ kinds = {(cls.__module__, cls.__name__) for cls in type(error).__mro__}
+ if kinds & LOCAL_EXCEPTION_TYPES:
+ return True
+ if ("peach.sources.base", "SourceFailure") in kinds:
+ return str(error.reason) in {"cloudflare_challenge", "ip_banned", "geo_restricted", "auth_required",
+ "not_found", "gone", "no_usable_metadata", "rate_limited", "timeout", "server_error", "network"}
+ if ("peach.metadata", "MetadataProviderError") in kinds and error.detail != "parse_error":
+ return error.kind in {"auth", "not_found"} or error.retryable is True or error.temporary is True
+ if stage in MEDIA_STAGES:
+ detail = getattr(error, "diagnostics", {})
+ if isinstance(error, subprocess.TimeoutExpired) or isinstance(detail, dict) and detail.get("reason") == "timeout":
+ return True
+ if isinstance(error, OSError) and error.errno == errno.ENOSPC:
+ return True
+ if isinstance(error, (FileNotFoundError, PermissionError, IsADirectoryError, NotADirectoryError)):
+ filename = getattr(error, "filename", None)
+ if filename and Path(filename).name != Path(str(fields.get("tool", ""))).name:
+ return True
+ return False
+
+
def _actionable(stage, error, fields):
"""仅允许技术故障入口;预期领域状态和 SDK 自身日志留在本机。"""
if str(fields.get("logger", "")).startswith(("sentry_sdk", "urllib3")):
@@ -242,11 +295,11 @@ def _actionable(stage, error, fields):
"preview", "standalone-update-install", "standalone-update-rollback", "logging"}
if stage not in stages or isinstance(error, (SystemExit, KeyboardInterrupt)):
return False
- if error is not None and type(error).__name__ in {"CancelledError", "JobCancelled", "NotFound", "Unavailable"}:
+ if _expected_failure(stage, error, fields):
return False
if stage == "http-request" and fields.get("status_code", 500) < 500:
return False
- if error is not None and type(getattr(error, "status_code", None)) is int and error.status_code < 500:
+ if (status := _http_status(error)) is not None and 400 <= status < 500:
return False
return True
@@ -266,17 +319,27 @@ def _http_details(stage, fields):
return tags, error_id
+def _browser_frames(stack):
+ frames = []
+ pattern = r"(peach-(?:app|pages)\.js)(?:\?[^\s):]*)?:(\d+):(\d+)"
+ for line in str(stack).splitlines()[:80]:
+ if "chrome-extension://" in line or "moz-extension://" in line:
+ continue
+ for name, row, column in re.findall(pattern, line):
+ frames.append({"filename": name, "lineno": int(row), "colno": int(column)})
+ return frames[:20]
+
+
def capture(stage, error=None, **fields):
"""只投影代码位置和错误类别,不发送异常文本或日志文本。"""
if not _actionable(stage, error, fields):
return
- media_stage = stage in {"frame-capture", "frame-tile", "media-probe", "preview", "library-preview"}
+ media_stage = stage in MEDIA_STAGES
if error is None and stage != "browser" and not media_stage:
return
frames, codes, chain = _exception_details(error)
if error is None and stage == "browser":
- for name, line, column in re.findall(r"(peach-(?:app|pages)\.js):(\d+):(\d+)", str(fields.get("stack", "")))[:20]:
- frames.append({"filename": name, "lineno": int(line), "colno": int(column)})
+ frames = _browser_frames(fields.get("stack", ""))
if not frames:
return
if stage == "logging" and not frames:
@@ -287,6 +350,8 @@ def capture(stage, error=None, **fields):
reason = ""
if media_stage:
reason = _media_details(error, fields, codes)
+ if reason in {"invalid_media", "permission_denied", "disk_full", "missing_media"}:
+ return
if not reason and not codes and error is None:
return
category = type(error).__name__ if error is not None else ("BrowserError" if stage == "browser" else "LogError")
diff --git a/src/peach/routes_configuration.py b/src/peach/routes_configuration.py
index ba5a990ba..152d1d203 100644
--- a/src/peach/routes_configuration.py
+++ b/src/peach/routes_configuration.py
@@ -157,7 +157,7 @@ async def client_error(request: Request):
raise HTTPException(400, "错误记录格式无效") from None
if not isinstance(data, dict) or not isinstance(data.get("message"), str):
raise HTTPException(400, "错误记录缺少原因")
- fields = {key: value[:6000] for key in ("message", "stack", "kind", "page", "source_file")
+ fields = {key: value[:6000] for key in ("message", "stack", "kind", "page", "source_file", "error_name")
if isinstance(value := data.get(key), str)}
fields.update({key: value for key in ('line', 'column')
if type(value := data.get(key)) is int and value >= 0})
diff --git a/tests/test_error_reporting.py b/tests/test_error_reporting.py
index ba9f942a1..e4e9a2a56 100644
--- a/tests/test_error_reporting.py
+++ b/tests/test_error_reporting.py
@@ -291,6 +291,13 @@ class ErrorReportingTests(unittest.TestCase):
self.assertEqual(browser["stack"], "at render")
self.assertEqual(browser['line'], 21)
self.assertEqual(browser['source_file'], 'app-abcd.js')
+ from peach import remote_reporting
+ with mock.patch.object(remote_reporting, 'send') as send:
+ response = client.post(path, json={'message': 'request cancelled', 'error_name': 'AbortError',
+ 'stack': 'at peach-app.js:12:34'})
+ self.assertEqual(response.status_code, 200)
+ self.assertEqual(self.records()[-1]['error_name'], 'AbortError')
+ send.assert_not_called()
with mock.patch("peach.diagnostics.report", side_effect=RuntimeError("diagnostics unavailable")):
response = client.get("/api/diagnostics/bundle")
self.assertEqual(response.status_code, 200)
@@ -395,26 +402,82 @@ class ErrorReportingTests(unittest.TestCase):
loop.default_exception_handler.assert_called_once()
def asyncio_callback(self, error, *, module='asyncio.proactor_events',
- name='_call_connection_lost', business=False, cleared=False):
+ name='_call_connection_lost', business=False, cleared=False,
+ recoverable=False, close_error=None, socket_override=None, server_override=None):
namespace = {'__name__': module, 'failure': error}
+ from asyncio.proactor_events import _ProactorBasePipeTransport
+ namespace['Base'] = _ProactorBasePipeTransport if recoverable else object
if business:
exec(compile('def fail():\n raise failure\n', 'peach/network_fixture.py', 'exec'), namespace)
operation = 'fail()' if business else 'raise failure'
- exec(compile(f'class _ProactorBasePipeTransport:\n def {name}(self):\n {operation}\n',
+ if recoverable:
+ operation = ('if self._called_connection_lost:\n return\n'
+ ' self._protocol.connection_lost(None)\n raise failure')
+ exec(compile(f'class _ProactorBasePipeTransport(Base):\n def {name}(self):\n {operation}\n',
'asyncio/proactor_events.py', 'exec'), namespace)
- transport = namespace['_ProactorBasePipeTransport']()
+ transport = object.__new__(namespace['_ProactorBasePipeTransport'])
protocol_type = type('H11Protocol', (), {'__module__': 'uvicorn.protocols.http.h11_impl'})
ssl_type = type('SSLProtocol', (), {'__module__': 'asyncio.sslproto'})
transport._protocol = ssl_type()
transport._protocol._app_protocol = None if cleared else protocol_type()
+ transport._protocol.connection_lost = mock.Mock()
+ if recoverable:
+ transport._sock = socket_override if socket_override is not None else mock.Mock()
+ if socket_override is None:
+ transport._sock.close.side_effect = close_error
+ transport._server = server_override if server_override is not None else mock.Mock()
+ if server_override is not None:
+ transport._server._attach(transport)
+ transport._closing = True
+ transport._called_connection_lost = False
+ self.addCleanup(lambda: setattr(transport, '_sock', None))
transport.private_url = 'https://private.example/?token=private-token'
transport.private_path = 'D:/private/movie.mp4'
loop = mock.Mock()
loop.get_debug.return_value = False
+ loop.diagnostic_transport = transport
+ if recoverable:
+ loop.diagnostic_socket = transport._sock
+ loop.diagnostic_server = transport._server
loop.call_exception_handler.side_effect = lambda context: error_reporting.asyncio_handler(loop, context)
asyncio.Handle(getattr(transport, name), (), loop)._run()
return loop
+ def test_connection_cleanup_completes_socket_close_and_server_detach_once(self):
+ from peach import remote_reporting
+ error = ConnectionResetError(10054, 'connection reset')
+ error.winerror = 10054
+ with mock.patch.object(remote_reporting, 'send') as send:
+ loop = self.asyncio_callback(error, recoverable=True)
+ transport = loop.diagnostic_transport
+ self.assertIsNone(transport._sock)
+ self.assertIsNone(transport._server)
+ self.assertTrue(transport._called_connection_lost)
+ self.assertTrue(self.records()[-1]['cleanup_recovered'])
+ loop.diagnostic_socket.close.assert_called_once()
+ loop.diagnostic_server._detach.assert_called_once_with(transport)
+ self.assertEqual(transport._protocol.connection_lost.call_count, 1)
+ asyncio.Handle(transport._call_connection_lost, (), loop)._run()
+ self.assertEqual(len(self.records()), 1)
+ self.assertEqual(transport._protocol.connection_lost.call_count, 1)
+ loop.default_exception_handler.assert_not_called()
+ send.assert_not_called()
+
+ def test_socket_cleanup_failure_keeps_the_original_exception_and_reports(self):
+ from peach import remote_reporting
+ error = ConnectionResetError(10054, 'connection reset')
+ error.winerror = 10054
+ close_error = OSError(5, 'close failed')
+ with mock.patch.object(remote_reporting, 'send') as send:
+ loop = self.asyncio_callback(error, recoverable=True, close_error=close_error)
+ send.assert_called_once()
+ self.assertFalse(loop.diagnostic_transport._called_connection_lost)
+ record = self.records()[-1]
+ self.assertEqual(record['classification'], 'connection-cleanup-failed')
+ self.assertFalse(record['cleanup_recovered'])
+ self.assertIs(close_error.__cause__, error)
+ loop.default_exception_handler.assert_called_once()
+
def test_windows_asyncio_connection_cleanup_stays_local_with_safe_context(self):
from peach import remote_reporting
error = ConnectionResetError(10054, 'connection reset')
@@ -435,6 +498,31 @@ class ErrorReportingTests(unittest.TestCase):
self.assertNotIn(value, json.dumps(record))
loop.default_exception_handler.assert_called_once()
+ def test_connection_cleanup_closes_real_socket_and_releases_asyncio_server(self):
+ import socket
+ from peach import remote_reporting
+
+ async def run():
+ server = await asyncio.get_running_loop().create_server(asyncio.Protocol, '127.0.0.1', 0)
+ accepted, peer = socket.socketpair()
+ try:
+ error = ConnectionResetError(10054, 'connection reset')
+ error.winerror = 10054
+ with mock.patch.object(remote_reporting, 'send') as send:
+ loop = self.asyncio_callback(error, recoverable=True,
+ socket_override=accepted, server_override=server)
+ self.assertEqual(accepted.fileno(), -1)
+ self.assertEqual(len(server._clients), 0)
+ loop.default_exception_handler.assert_not_called()
+ send.assert_not_called()
+ finally:
+ accepted.close()
+ peer.close()
+ server.close()
+ await asyncio.wait_for(server.wait_closed(), timeout=2)
+
+ asyncio.run(run())
+
def test_asyncio_tls_cleanup_keeps_protocol_class_after_application_detaches(self):
from peach import remote_reporting
error = ConnectionResetError(10054, 'connection reset')
diff --git a/tests/test_remote_reporting.py b/tests/test_remote_reporting.py
index 307ce6f35..3d4b84d75 100644
--- a/tests/test_remote_reporting.py
+++ b/tests/test_remote_reporting.py
@@ -141,6 +141,137 @@ class RemoteReportingTests(unittest.TestCase):
self.assertEqual(send.call_args.args[1]['exception']['values'][0]['stacktrace']['frames'],
[{'filename': 'peach-app.js', 'lineno': 12, 'colno': 34}])
+ def test_control_and_source_states_keep_local_records_without_remote_events(self):
+ import asyncio
+ from concurrent.futures import CancelledError
+ from peach import error_reporting
+ from peach.browser_transport import BrowserUnavailable, ChallengeUnsolved
+ from peach.jav_cover_fetch import CoverConnectError, DeadlineExceeded
+ from peach.jobs import JobPaused, MeteredSourceDenied, DiskSpaceDenied, TaskRunConflict
+ from peach.metadata import MetadataProviderError
+ from peach.previews import PreviewUnavailable
+ from peach.scraping_access import SourcePaused
+ from peach.segments import SegmentCancelled
+ from peach.sources.base import SourceFailure, FailureReason
+ from peach.transcodes import TranscodeCancelled
+ from starlette.requests import ClientDisconnect
+ errors = [asyncio.CancelledError(), CancelledError(), InterruptedError(), JobPaused(),
+ MeteredSourceDenied(), DiskSpaceDenied(), TaskRunConflict('processing', 1),
+ SourcePaused(), DeadlineExceeded(), CoverConnectError(), PreviewUnavailable(),
+ SegmentCancelled(), TranscodeCancelled(), BrowserUnavailable(), ChallengeUnsolved(),
+ ClientDisconnect(),
+ MetadataProviderError('auth', kind='auth'),
+ MetadataProviderError('absent', kind='not_found'),
+ MetadataProviderError('retry', kind='unavailable', retryable=True)]
+ errors += [SourceFailure(reason, 'source state') for reason in FailureReason
+ if reason != FailureReason.PARSE_ERROR]
+ with mock.patch.object(reporting, 'send') as send:
+ for error in errors:
+ with self.subTest(error=type(error).__name__):
+ path = error_reporting.event('background-job', error)
+ self.assertTrue(path.is_file())
+ send.assert_not_called()
+
+ def test_source_parser_and_unknown_provider_failures_are_actionable(self):
+ from peach.metadata import MetadataProviderError
+ from peach.sources.base import SourceFailure, FailureReason
+ errors = [SourceFailure(FailureReason.PARSE_ERROR, 'parser failure'),
+ MetadataProviderError('parser failure', kind='unavailable', detail='parse_error'),
+ MetadataProviderError('unknown failure')]
+ for error in errors:
+ with self.subTest(error=type(error).__name__), mock.patch.object(reporting, 'send') as send:
+ reporting.capture('background-job', error)
+ send.assert_called_once()
+
+ def test_unrelated_exception_names_do_not_acquire_domain_state_semantics(self):
+ for name in ('CancelledError', 'NotFound', 'Unavailable'):
+ with self.subTest(name=name), mock.patch.object(reporting, 'send') as send:
+ error = type(name, (RuntimeError,), {'__module__': 'peach.fixture'})('unexpected failure')
+ reporting.capture('background-job', error)
+ send.assert_called_once()
+
+ def test_source_client_http_statuses_are_local_and_server_failures_are_reported(self):
+ import httpx
+ from urllib.error import HTTPError
+ for code in (400, 401, 403, 404, 410, 429, 500, 503):
+ response = httpx.Response(code, request=httpx.Request('GET', 'https://example.test/'))
+ errors = [httpx.HTTPStatusError('http state', request=response.request, response=response),
+ HTTPError('https://example.test/', code, 'http state', {}, None)]
+ for error in errors:
+ with self.subTest(code=code, error=type(error).__name__), \
+ mock.patch.object(reporting, 'send') as send:
+ reporting.capture('background-job', error)
+ if code < 500:
+ send.assert_not_called()
+ else:
+ send.assert_called_once()
+ self.assertEqual(send.call_args.args[1]['tags']['status_code'], str(code))
+
+ def test_warning_exceptions_do_not_consume_remote_error_budget(self):
+ namespace = {}
+ exec(compile('def fail():\n raise ValueError("retrying")\n', 'peach/fixture.py', 'exec'), namespace)
+ try:
+ namespace['fail']()
+ except ValueError as error:
+ with mock.patch.object(reporting, 'send') as send:
+ reporting.capture('logging', error, level='WARNING', logger='peach.fixture')
+ send.assert_not_called()
+ reporting.capture('logging', error, level='ERROR', logger='peach.fixture')
+ send.assert_called_once()
+
+ def test_expected_media_input_conditions_are_local_and_decoder_faults_are_reported(self):
+ for marker in ('Invalid data found', 'Permission denied', 'No space left', 'No such file or directory'):
+ with self.subTest(marker=marker), mock.patch.object(reporting, 'send') as send:
+ reporting.capture('frame-capture', returncode=1, stderr=marker)
+ send.assert_not_called()
+ with mock.patch.object(reporting, 'send') as send:
+ import errno
+ reporting.capture('media-probe', OSError(errno.ENOSPC, 'disk full'), tool='ffprobe')
+ send.assert_not_called()
+ reporting.capture('media-probe', subprocess.TimeoutExpired('ffprobe', 20), tool='ffprobe')
+ send.assert_not_called()
+ reporting.capture('media-probe', FileNotFoundError(2, 'missing input', 'movie.mp4'), tool='ffprobe')
+ send.assert_not_called()
+ reporting.capture('media-probe', FileNotFoundError(2, 'missing tool', 'ffprobe'), tool='ffprobe')
+ send.assert_called_once()
+ send.reset_mock()
+ reporting.capture('frame-capture', returncode=69,
+ stderr="Your platform doesn't support hardware accelerated AV1 decoding")
+ send.assert_called_once()
+ send.reset_mock()
+ reporting.capture('frame-capture', returncode=1,
+ stderr='Unknown decoder; Invalid data found')
+ send.assert_called_once()
+
+ def test_browser_abort_and_extension_frames_stay_local_and_bundle_versions_keep_positions(self):
+ with mock.patch.object(reporting, 'send') as send:
+ reporting.capture('browser', error_name='AbortError', stack='peach-app.js:12:34')
+ reporting.capture('browser', stack='at chrome-extension://extension/peach-app.js:12:34')
+ reporting.capture('browser', stack='at moz-extension://extension/peach-pages.js:12:34')
+ send.assert_not_called()
+ reporting.capture('browser', stack='at https://example.test/dist/peach-app.js?v=build:12:34')
+ send.assert_called_once()
+ frames = send.call_args.args[1]['exception']['values'][0]['stacktrace']['frames']
+ self.assertEqual(frames, [{'filename': 'peach-app.js', 'lineno': 12, 'colno': 34}])
+
+ def test_preview_environment_conditions_preserve_guidance_and_local_issue(self):
+ from peach import library_processing
+ from peach.previews import PreviewUnavailable
+ resolver = mock.Mock()
+ resolver.ffmpeg.return_value = None
+ row = {'id': 1, 'path': str(self.root / 'missing.mp4'), 'location': 'local', 'name': 'missing.mp4'}
+ with self.assertRaises(PreviewUnavailable) as raised:
+ library_processing._generate_preview(self.config, row, mock.Mock(), resolver, progress=lambda: None)
+ issue = mock.Mock()
+ with mock.patch.object(reporting, 'send') as send, \
+ mock.patch.object(library_processing, '_generate_preview', side_effect=raised.exception):
+ made = library_processing._generate_previews(self.config, [row], mock.Mock(),
+ update=mock.Mock(), issue=issue, guard=mock.Mock())
+ self.assertEqual(made, 0)
+ self.assertIn('FFmpeg', issue.call_args.args[1])
+ self.assertTrue(issue.call_args.kwargs['retryable'])
+ send.assert_not_called()
+
def test_sdk_wire_payload_has_no_local_context_or_consent_path(self):
from sentry_sdk.transport import HttpTransport
sent = []
diff --git a/web/dist/peach-app.js b/web/dist/peach-app.js
index 33c0f906a..cf79ad970 100644
--- a/web/dist/peach-app.js
+++ b/web/dist/peach-app.js
@@ -20,17 +20,19 @@ function d() {
if (u || typeof window > "u") return;
u = !0;
let e = Date.now(), t = 0, n = /* @__PURE__ */ new Set(), r = (r, i, a = "", o) => {
- if (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10) return;
- let s = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);
- if (!s) return;
- let c = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", l = `${r}:${s}:${c}`;
- n.has(l) || (n.add(l), t += 1, fetch("/api/diagnostics/client-error", {
+ let s = i instanceof Error || i instanceof DOMException ? i.name : "";
+ if (s === "AbortError" || (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10)) return;
+ let c = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);
+ if (!c) return;
+ let l = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", u = `${r}:${c}:${l}`;
+ n.has(u) || (n.add(u), t += 1, fetch("/api/diagnostics/client-error", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
kind: r,
- message: s,
- stack: c,
+ message: c,
+ stack: l,
+ error_name: s,
page: window.location.pathname,
source_file: o?.filename?.split("/").pop()?.split("?")[0],
line: o?.lineno,
diff --git a/web/dist/peach-pages.js b/web/dist/peach-pages.js
index 07d08e62f..91ed22beb 100644
--- a/web/dist/peach-pages.js
+++ b/web/dist/peach-pages.js
@@ -13,17 +13,19 @@ function u() {
if (l || typeof window > "u") return;
l = !0;
let e = Date.now(), t = 0, n = /* @__PURE__ */ new Set(), r = (r, i, a = "", o) => {
- if (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10) return;
- let s = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);
- if (!s) return;
- let c = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", l = `${r}:${s}:${c}`;
- n.has(l) || (n.add(l), t += 1, fetch("/api/diagnostics/client-error", {
+ let s = i instanceof Error || i instanceof DOMException ? i.name : "";
+ if (s === "AbortError" || (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10)) return;
+ let c = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);
+ if (!c) return;
+ let l = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", u = `${r}:${c}:${l}`;
+ n.has(u) || (n.add(u), t += 1, fetch("/api/diagnostics/client-error", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
kind: r,
- message: s,
- stack: c,
+ message: c,
+ stack: l,
+ error_name: s,
page: window.location.pathname,
source_file: o?.filename?.split("/").pop()?.split("?")[0],
line: o?.lineno,