Peach: Local-first video metadata scraper and media manager.
fix(diagnostics): 完成断连资源收尾并筛选预期状态
longmeidao566267635063
diff --git a/docs/DIAGNOSTICS_AUDIT.md b/docs/DIAGNOSTICS_AUDIT.mdindex f45ee347a..ea7ac0e49 100644--- a/docs/DIAGNOSTICS_AUDIT.md+++ b/docs/DIAGNOSTICS_AUDIT.md@@ -43,7 +43,27 @@回归验收使用临时数据根和 SDK 内存接收端。本机已配置独立的 `peach-errors` 与 `peach-library-research` 接收项目,合成错误 `PEACH-ERRORS-1` 已在 Sentry 核对代码位置;没有发送真实馆藏。发布工作流的两个公开 DSN 仓库变量已配置并回读核对,构建包从变量取得接收地址;配置变量不触发发布。技术诊断选择见 [ADR-0110](adr/0110-recommended-diagnostics-choice.md)。安装辅助进程只在本机取证,不初始化远程客户端或发送事件;应用进程按用户选择发送。-`asyncio` 连接清理只在同时满足以下条件时保留为本机诊断:`ConnectionResetError` 的 `winerror` 为 `10054`,回调为标准库 `_ProactorBasePipeTransport._call_connection_lost`,调用栈只含该回调与 `asyncio.events._run`,且没有关联异常。该分类不发送至 Sentry;其他错误码、回调、业务栈与关联异常继续按技术故障规则处理。本机记录保留异常与标准库帧,并补充回调、协议类、可识别的服务端类别和 TLS 标志;这些上下文字段不进入远程载荷,不访问连接地址、请求内容或对象文本。默认事件循环处理器仍执行,底层连接清理行为沿用 Python 标准库。+`asyncio` 连接清理只在同时满足以下条件时识别为连接重置收尾:`ConnectionResetError` 的 `winerror` 为 `10054`,回调为标准库 `_ProactorBasePipeTransport._call_connection_lost`,调用栈只含该回调与 `asyncio.events._run`,且没有关联异常。处理器核对真实 Proactor transport 处于清理状态,关闭 socket、解除服务端登记并完成关闭状态;不重复通知协议。成功记录 `classification=connection-cleanup` 与 `cleanup_recovered=true`,仅保留本机证据。收尾自身失败则保留原始异常链、上报技术故障并交给默认处理器;不能确认对象状态时也沿用默认处理器。没有全局修改 Python 类或替换事件循环。++本机记录保留异常与标准库帧,并补充回调、协议类、可识别的服务端类别和 TLS 标志;这些上下文字段不进入远程载荷,不访问连接地址、请求内容或对象文本。资源收尾使用 CPython 3.14.7 的私有对象协议,升级需验证实际 socket、`asyncio.Server`、重复关闭与收尾失败。原始竞争时序在 33 次回环实验中未复现;合成故障验证了资源实际关闭与服务端释放,不能据此推断原始客户端来源。++### 远程错误筛选核验++核验覆盖异常钩子、日志、后台任务、来源、HTTP、媒体、浏览器、启动与更新入口。筛选依据结构化状态和实际异常继承关系,不按错误原文、任意同名类或所有网络异常统一过滤。++| 状态 | 本机与远程处理 |+| --- | --- |+| 任务取消、暂停、互斥、计费与磁盘策略门槛 | 保留本机异常和任务回执,不占用错误上传额度 |+| 来源冷却、动作预算、授权、未命中、明确的临时故障 | 复用来源异常与分类,保留本机状态;来源解析错误和未知失败继续上报 |+| HTTP 客户端错误 | 识别直接状态码、httpx 响应及 urllib HTTPError;4xx 留在本机,未分类的 5xx 继续上报;`0` 不作为 HTTP 状态码 |+| warning 与重试日志 | 低于 error 的日志留在本机;error/critical 携带 Peach 栈时上报 |+| 媒体输入与环境条件 | 坏媒体、输入缺失、权限、空间与媒体工具超时留在本机;工具执行文件缺失、解码器能力缺失和未知技术错误继续上报;预览前置条件保留提示和可重试问题项 |+| 浏览器取消与扩展 | 主动取消请求的 AbortError 不发起记录请求;后端收到该类型仍仅写本机;扩展 URL 帧不上传,Peach 带构建参数的脚本帧保留行列位置 |+| 客户端中途断开 | Starlette 的 ClientDisconnect 留在本机,不当作服务端程序故障 |+| 启动、CLI、托盘、更新与回滚 | 未处理的技术故障继续上报;安装辅助进程沿用仅本机诊断边界 |+| SDK、无堆栈日志与不透明浏览器错误 | 不递归上报 SDK 自身问题;无法取得 Peach 代码位置的日志和浏览器错误仅保留本机 |++本机策略不清除已有 Sentry 事件,也不控制邮件;邮件摘要与即时通知分别在接收端设置。### 馆藏与口味群体研究@@ -89,11 +109,11 @@ ZIP 不包含 ledger、配置文件、私钥、登录 Cookie 或媒体文件。| 影响面 | 适用性 || --- | --- || 数据层 | 本机独立同意与限额文件;研究只读账本,复用派生偏好;没有迁移或真实 ledger 写入 |-| API/契约 | 诊断选择读写、完整研究预览与同意写入;本机、鉴权、同源限制 |+| API/契约 | 诊断选择、研究预览与同意;浏览器接收可选异常类型用于取消识别;本机、鉴权、同源限制 || 页面 | 配置页「应用与维护」、首次设置;Windows 安装器默认勾选推荐的诊断选项;其他馆藏页面不适用 || 反向状态 | 诊断与研究分别关闭;关闭使排队载荷失效,研究重新开启须预览并同意 || 视口 | 桌面和 390×844 手机验证明细折行、内部滚动、同意与关闭;无横向溢出 |-| 测试 | 诊断默认开启、取消与持久关闭、研究默认关闭、脱敏、白名单、限额、SDK 内存传输、权限及界面回归 |+| 测试 | 诊断与研究选择、SDK 内存传输、脱敏与权限;实际 socket 和服务端清理、收尾失败、预期状态、解析错误、浏览器取消与构建参数堆栈 || 文档 | 中英文 README、STATUS、REUSE、待办、本记录与 ADR-0109 || 智能体入口 | 没有新增规则、技能或入口 || 生产 | 本机源码运行态按托盘入口重启并严格校验 HTTPS;外部测试包发布独立验收 |diff --git a/docs/REUSE.md b/docs/REUSE.mdindex 5ab75e4d3..4e3f9b88a 100644--- a/docs/REUSE.md+++ b/docs/REUSE.md@@ -150,7 +150,7 @@ Peach 播放器皮肤直接装饰 Video.js 10 的 light DOM:右侧控制组用可选远程诊断复用官方 [sentry-sdk 2.72.0](https://github.com/getsentry/sentry-python/tree/2.72.0)(MIT,维护中,支持 Python 3.14);固定版源码通过 `opensrc path pypi:sentry-sdk` 核对。SDK 负责后台 HTTP、TLS、Sentry 信封和服务端限流;Peach 负责本机选择、技术字段投影、同类错误去重与每机每日上限。版本统计与错误诊断共用默认勾选的开关,关闭时不构造 SDK;SDK 全局自动集成、追踪、回放、日志、会话与指标采集均关闭。`HttpTransport._send_envelope` 的窄覆盖在真正发送前复查本机选择版本并移除本机传输元数据;它是固定版本的内部接口,升级必须通过真实 SDK 信封回归。无真实外发的内存接收端检查最终字节、停用后队列失效与断网不影响本机取证。新增依赖仅 `sentry-sdk`,复用已声明的 urllib3 / certifi;体积以发行构建为准。未采用自写 Sentry 协议或 Workers 收集后台,避免另行维护传输、去重和查看界面。没有真实 ledger 写入。-Windows 连接清理诊断复用 CPython 3.14.7 的 `asyncio` 回调与异常栈(PSF 许可,无新增依赖)。只将纯标准库 `_call_connection_lost` 回调中的 `WinError 10054` 分类为本机连接清理证据,保留其他网络故障。核对本机 `proactor_events.py` 源码与重启前后日志;33 次回环实验未复现原始竞争时序,不据此认定客户端来源。合成异常通过真实 `asyncio.Handle` 分派验证筛选与默认处理器,协议类别只写本机日志。没有修改或复制传输实现、全局替换事件循环,或引入另一套日志 SDK。+Windows 连接清理复用 [CPython 3.14.7 的 Proactor 对象协议](https://github.com/python/cpython/blob/v3.14.7/Lib/asyncio/proactor_events.py)(PSF 许可,无新增依赖)。精确识别纯标准库清理回调中的 `WinError 10054`,只补完 socket 关闭、服务端解除登记和关闭状态,不重复通知协议,不全局修改标准库类或替换事件循环。它使用 `_sock`、`_server._detach` 与 `_called_connection_lost` 私有接口,升级 Python 时须核验。真实 socket 和 `asyncio.Server` 配合合成故障验证资源释放;收尾自身失败保留原始异常链并上报。33 次回环实验未复现原始竞争时序,不据此认定客户端来源。协议类别只写本机日志;来源、任务、媒体和浏览器的远程筛选复用现有领域状态,见 [诊断核验](DIAGNOSTICS_AUDIT.md#远程错误筛选核验)。馆藏与口味群体研究复用 `LedgerDatabase.read_connection`、`taste_preferences.preference_profile`、`error_reporting.redact` 与同一固定版 Sentry 的内存附件 API;偏好权重与身份口径不另写一份。研究使用独立项目、独立同意文件和字段版本;默认关闭,预览只在用户点击时读取,后台读取要求有效同意与接收端。没有复制浏览器资料、原始历史或媒体,没有真实 ledger 写入。SDK 本机安装目录含缓存约 3.1 MB,Windows 构建复制 MIT 许可证元数据;压缩后的发行增量未单独测量。diff --git a/frontend/src/error-reporting.ts b/frontend/src/error-reporting.tsindex 46ae40559..1c6e07a3e 100644--- a/frontend/src/error-reporting.ts+++ b/frontend/src/error-reporting.ts@@ -8,6 +8,8 @@ export function installErrorReporting(): void {let count = 0;const seen = new Set<string>();const send = (kind: string, cause: unknown, fallback = '', source?: ErrorEvent) => {+ const errorName = cause instanceof Error || cause instanceof DOMException ? cause.name : '';+ if (errorName === 'AbortError') return;if (Date.now() - start >= 60_000) {start = Date.now(); count = 0; seen.clear();}@@ -20,7 +22,7 @@ export function installErrorReporting(): void {seen.add(key); count += 1;void fetch('/api/diagnostics/client-error', {method: 'POST', headers: { 'Content-Type': 'application/json' },- body: JSON.stringify({ kind, message, stack, page: window.location.pathname,+ body: JSON.stringify({ kind, message, stack, error_name: errorName, page: window.location.pathname,source_file: source?.filename?.split('/').pop()?.split('?')[0],line: source?.lineno, column: source?.colno }), credentials: 'same-origin', keepalive: true,}).catch(() => {});diff --git a/frontend/test/error-reporting.test.ts b/frontend/test/error-reporting.test.tsindex 1ea957376..db6c21d4b 100644--- a/frontend/test/error-reporting.test.ts+++ b/frontend/test/error-reporting.test.ts@@ -27,3 +27,21 @@ it('浏览器异常有上限、去重,网络失败不会触发递归上报', aawait Promise.resolve();expect(fetch).toHaveBeenCalledTimes(10);});++it('主动取消请求不生成错误记录,真实异常保留类型', async () => {+ const listeners = new Map<string, (event: unknown) => void>();+ vi.spyOn(window, 'addEventListener').mockImplementation((name, callback) => {+ listeners.set(name, callback as (event: unknown) => void);+ });+ const fetch = vi.fn().mockResolvedValue({ ok: true });+ vi.stubGlobal('fetch', fetch);+ const { installErrorReporting } = await import('../src/error-reporting');+ installErrorReporting();+ const cancelled = new DOMException('request cancelled', 'AbortError');+ listeners.get('unhandledrejection')!({ reason: cancelled });+ listeners.get('error')!({ error: cancelled, message: cancelled.message });+ expect(fetch).not.toHaveBeenCalled();+ listeners.get('unhandledrejection')!({ reason: new TypeError('render failed') });+ expect(fetch).toHaveBeenCalledTimes(1);+ expect(JSON.parse(fetch.mock.calls[0]![1].body)).toMatchObject({ error_name: 'TypeError' });+});diff --git a/src/peach/error_reporting.py b/src/peach/error_reporting.pyindex 5fa4cf962..d85aa0032 100644--- a/src/peach/error_reporting.py+++ b/src/peach/error_reporting.py@@ -293,10 +293,35 @@ def is_asyncio_connection_cleanup(error, fields) -> bool:return last == ("asyncio.proactor_events", "_call_connection_lost")+def _finish_asyncio_connection_cleanup(context) -> bool:+ """完成 CPython 清理回调中 shutdown 之后的资源收尾,不重复通知协议。"""+ from asyncio.proactor_events import _ProactorBasePipeTransport+ callback = getattr(context.get("handle"), "_callback", None)+ transport = getattr(callback, "__self__", None)+ if (not isinstance(transport, _ProactorBasePipeTransport) or not transport._closing+ or transport._called_connection_lost or transport._sock is None):+ return False+ transport._sock.close()+ transport._sock = None+ if transport._server is not None:+ transport._server._detach(transport)+ transport._server = None+ transport._called_connection_lost = True+ return True++def asyncio_handler(loop, context) -> None:error = context.get("exception")fields = _asyncio_fields(context)if is_asyncio_connection_cleanup(error, fields):fields["classification"] = "connection-cleanup"+ try:+ fields["cleanup_recovered"] = _finish_asyncio_connection_cleanup(context)+ except Exception as cleanup_error:+ cleanup_error.__cause__ = error+ error = cleanup_error+ fields.update(classification="connection-cleanup-failed", cleanup_recovered=False)+ context = {**context, "exception": error}event("unhandled-asyncio", error, message=context.get("message", ""), **fields)- loop.default_exception_handler(context)+ if not fields.get("cleanup_recovered"):+ loop.default_exception_handler(context)diff --git a/src/peach/library_processing.py b/src/peach/library_processing.pyindex 92584b50b..197f5564b 100644--- a/src/peach/library_processing.py+++ b/src/peach/library_processing.py@@ -1833,6 +1833,7 @@ def _generate_preview(config, row, database, resolver, *, progress):"""复用九宫格抽帧原语,产物可用后在共享写锁内登记。"""from . import frame_capture, media_probefrom .fsutil import atomic_path+ from .previews import PreviewUnavailableroot = (config.directory('generated') / 'snapshots').resolve()previous = row.get('snapshot_path')@@ -1842,19 +1843,19 @@ def _generate_preview(config, row, database, resolver, *, progress):return Falsechoice = resolver.ffmpeg()if choice is None:- raise ValueError('未找到 FFmpeg,无法生成预览图,请在诊断页检查工具状态')+ raise PreviewUnavailable('未找到 FFmpeg,无法生成预览图,请在诊断页检查工具状态')video = translate_ledger_path(row['path'])if not video.is_file():- raise ValueError('媒体文件不可访问')+ raise PreviewUnavailable('媒体文件不可访问')duration = row.get('duration') or 0if duration <= 0:probe = resolver.ffprobe()if probe is None:- raise ValueError('未找到 ffprobe,无法读取视频时长')+ raise PreviewUnavailable('未找到 ffprobe,无法读取视频时长')measured = media_probe.measure(str(probe.path), row['id'], str(video))duration = measured[0]if duration <= 0:- raise ValueError('视频时长未取得,请检查媒体文件后重试')+ raise PreviewUnavailable('视频时长未取得,请检查媒体文件后重试')progress()with database.write_transaction() as connection:connection.execute(media_probe.UPDATE, measured)@@ -1896,6 +1897,7 @@ def _generate_preview(config, row, database, resolver, *, progress):def _generate_previews(config, rows, database, *, update, issue, guard,completed=(), checkpoint=lambda row: None):"""只读本机视频;来源失败与抽帧失败各自保留可重试项目。"""+ from .previews import PreviewUnavailableif database is None:return 0rows = [row for row in rows if row['location'] == 'local' and not skip_background_read(row['path'])]@@ -1918,10 +1920,10 @@ def _generate_previews(config, rows, database, *, update, issue, guard,checkpoint(row)except InterruptedError:raise- except (OSError, ValueError, subprocess.SubprocessError) as error:+ except (OSError, ValueError, subprocess.SubprocessError, PreviewUnavailable) as error:from .error_reporting import eventevent('library-preview', error, asset_id=row['id'])- message = (str(error) if isinstance(error, ValueError) else+ message = (str(error) if isinstance(error, (ValueError, PreviewUnavailable)) else'生成预览图超时,请重试' if isinstance(error, subprocess.SubprocessError) else'生成预览图失败,请检查 FFmpeg 与目录权限后重试')details = {'diagnostics': error.diagnostics} if isinstance(error, _PreviewCaptureError) else {}diff --git a/src/peach/remote_reporting.py b/src/peach/remote_reporting.pyindex 2a8327650..0078a9857 100644--- a/src/peach/remote_reporting.py+++ b/src/peach/remote_reporting.py@@ -2,11 +2,13 @@from __future__ import annotationsimport atexit+import errnoimport hashlibimport jsonimport osimport platformimport re+import subprocessimport threadingimport uuidfrom datetime import datetime, timezone@@ -24,6 +26,17 @@ _lock = threading.RLock()_client = None_client_dsn = ""_disabled = False+MEDIA_STAGES = frozenset({"frame-capture", "frame-tile", "media-probe", "preview", "library-preview"})+LOCAL_EXCEPTION_TYPES = frozenset({+ ("asyncio.exceptions", "CancelledError"), ("concurrent.futures._base", "CancelledError"),+ ("builtins", "InterruptedError"), ("peach.jobs", "JobPolicyError"),+ ("peach.jav_cover_fetch", "Unavailable"), ("peach.jav_cover_fetch", "DeadlineExceeded"),+ ("peach.scraping_access", "SourcePaused"), ("peach.previews", "PreviewUnavailable"),+ ("peach.transcodes", "TranscodeCancelled"), ("peach.segments", "SegmentCancelled"),+ ("peach.browser_transport", "BrowserUnavailable"), ("peach.browser_transport", "ChallengeUnsolved"),+ ("peach.face_detect", "FaceModelUnavailable"), ("peach.avatar_watermark", "WatermarkModelUnavailable"),+ ("starlette.requests", "ClientDisconnect"),+})def _config(config=None):@@ -194,16 +207,28 @@ def usage(config=None):send("usage", {"message": "Peach started", "level": "info", "tags": {"os": platform.system()}}, config)+def _http_status(error):+ status = getattr(error, "status_code", None)+ if type(status) is not int or not 100 <= status <= 599:+ status = getattr(getattr(error, "response", None), "status_code", None)+ if type(status) is not int and any(cls.__module__ == "urllib.error" and cls.__name__ == "HTTPError"+ for cls in type(error).__mro__):+ status = getattr(error, "code", None)+ return status if type(status) is int and 100 <= status <= 599 else None++def _exception_details(error):frames, codes, chain = [], {}, []current, visited = error, set()while current is not None and id(current) not in visited and len(chain) < 8:visited.add(id(current))chain.append(type(current).__name__)- for key in ("errno", "winerror", "returncode", "status_code"):+ for key in ("errno", "winerror", "returncode"):value = getattr(current, key, None)if type(value) is int:codes[key] = value+ if (status := _http_status(current)) is not None:+ codes["status_code"] = statustb = current.__traceback__while tb is not None and len(frames) < 40:filename = Path(tb.tb_frame.f_code.co_filename)@@ -217,17 +242,45 @@ def _exception_details(error):def _media_details(error, fields, codes):detail = getattr(error, "diagnostics", {}) if error is not None else {}+ detail = detail if isinstance(detail, dict) else {}if type(detail.get("returncode")) is int:codes["returncode"] = detail["returncode"]raw = str(fields.get("stderr", "")) + str(detail.get("stderr", ""))for marker, code in (("Your platform doesn't support hardware accelerated AV1 decoding", "av1_decoder_unavailable"),+ ("Unknown decoder", "decoder_unavailable"),("Invalid data found", "invalid_media"), ("Permission denied", "permission_denied"),- ("No space left", "disk_full"), ("Unknown decoder", "decoder_unavailable")):+ ("No space left", "disk_full"), ("No such file or directory", "missing_media")):if marker in raw:return codereturn ""+def _expected_failure(stage, error, fields):+ if stage == "logging" and fields.get("level", "ERROR") not in {"ERROR", "CRITICAL"}:+ return True+ if stage == "browser" and fields.get("error_name") == "AbortError":+ return True+ kinds = {(cls.__module__, cls.__name__) for cls in type(error).__mro__}+ if kinds & LOCAL_EXCEPTION_TYPES:+ return True+ if ("peach.sources.base", "SourceFailure") in kinds:+ return str(error.reason) in {"cloudflare_challenge", "ip_banned", "geo_restricted", "auth_required",+ "not_found", "gone", "no_usable_metadata", "rate_limited", "timeout", "server_error", "network"}+ if ("peach.metadata", "MetadataProviderError") in kinds and error.detail != "parse_error":+ return error.kind in {"auth", "not_found"} or error.retryable is True or error.temporary is True+ if stage in MEDIA_STAGES:+ detail = getattr(error, "diagnostics", {})+ if isinstance(error, subprocess.TimeoutExpired) or isinstance(detail, dict) and detail.get("reason") == "timeout":+ return True+ if isinstance(error, OSError) and error.errno == errno.ENOSPC:+ return True+ if isinstance(error, (FileNotFoundError, PermissionError, IsADirectoryError, NotADirectoryError)):+ filename = getattr(error, "filename", None)+ if filename and Path(filename).name != Path(str(fields.get("tool", ""))).name:+ return True+ return False++def _actionable(stage, error, fields):"""仅允许技术故障入口;预期领域状态和 SDK 自身日志留在本机。"""if str(fields.get("logger", "")).startswith(("sentry_sdk", "urllib3")):@@ -242,11 +295,11 @@ def _actionable(stage, error, fields):"preview", "standalone-update-install", "standalone-update-rollback", "logging"}if stage not in stages or isinstance(error, (SystemExit, KeyboardInterrupt)):return False- if error is not None and type(error).__name__ in {"CancelledError", "JobCancelled", "NotFound", "Unavailable"}:+ if _expected_failure(stage, error, fields):return Falseif stage == "http-request" and fields.get("status_code", 500) < 500:return False- if error is not None and type(getattr(error, "status_code", None)) is int and error.status_code < 500:+ if (status := _http_status(error)) is not None and 400 <= status < 500:return Falsereturn True@@ -266,17 +319,27 @@ def _http_details(stage, fields):return tags, error_id+def _browser_frames(stack):+ frames = []+ pattern = r"(peach-(?:app|pages)\.js)(?:\?[^\s):]*)?:(\d+):(\d+)"+ for line in str(stack).splitlines()[:80]:+ if "chrome-extension://" in line or "moz-extension://" in line:+ continue+ for name, row, column in re.findall(pattern, line):+ frames.append({"filename": name, "lineno": int(row), "colno": int(column)})+ return frames[:20]++def capture(stage, error=None, **fields):"""只投影代码位置和错误类别,不发送异常文本或日志文本。"""if not _actionable(stage, error, fields):return- media_stage = stage in {"frame-capture", "frame-tile", "media-probe", "preview", "library-preview"}+ media_stage = stage in MEDIA_STAGESif error is None and stage != "browser" and not media_stage:returnframes, codes, chain = _exception_details(error)if error is None and stage == "browser":- for name, line, column in re.findall(r"(peach-(?:app|pages)\.js):(\d+):(\d+)", str(fields.get("stack", "")))[:20]:- frames.append({"filename": name, "lineno": int(line), "colno": int(column)})+ frames = _browser_frames(fields.get("stack", ""))if not frames:returnif stage == "logging" and not frames:@@ -287,6 +350,8 @@ def capture(stage, error=None, **fields):reason = ""if media_stage:reason = _media_details(error, fields, codes)+ if reason in {"invalid_media", "permission_denied", "disk_full", "missing_media"}:+ returnif not reason and not codes and error is None:returncategory = type(error).__name__ if error is not None else ("BrowserError" if stage == "browser" else "LogError")diff --git a/src/peach/routes_configuration.py b/src/peach/routes_configuration.pyindex ba5a990ba..152d1d203 100644--- a/src/peach/routes_configuration.py+++ b/src/peach/routes_configuration.py@@ -157,7 +157,7 @@ async def client_error(request: Request):raise HTTPException(400, "错误记录格式无效") from Noneif not isinstance(data, dict) or not isinstance(data.get("message"), str):raise HTTPException(400, "错误记录缺少原因")- fields = {key: value[:6000] for key in ("message", "stack", "kind", "page", "source_file")+ fields = {key: value[:6000] for key in ("message", "stack", "kind", "page", "source_file", "error_name")if isinstance(value := data.get(key), str)}fields.update({key: value for key in ('line', 'column')if type(value := data.get(key)) is int and value >= 0})diff --git a/tests/test_error_reporting.py b/tests/test_error_reporting.pyindex ba9f942a1..e4e9a2a56 100644--- a/tests/test_error_reporting.py+++ b/tests/test_error_reporting.py@@ -291,6 +291,13 @@ class ErrorReportingTests(unittest.TestCase):self.assertEqual(browser["stack"], "at render")self.assertEqual(browser['line'], 21)self.assertEqual(browser['source_file'], 'app-abcd.js')+ from peach import remote_reporting+ with mock.patch.object(remote_reporting, 'send') as send:+ response = client.post(path, json={'message': 'request cancelled', 'error_name': 'AbortError',+ 'stack': 'at peach-app.js:12:34'})+ self.assertEqual(response.status_code, 200)+ self.assertEqual(self.records()[-1]['error_name'], 'AbortError')+ send.assert_not_called()with mock.patch("peach.diagnostics.report", side_effect=RuntimeError("diagnostics unavailable")):response = client.get("/api/diagnostics/bundle")self.assertEqual(response.status_code, 200)@@ -395,26 +402,82 @@ class ErrorReportingTests(unittest.TestCase):loop.default_exception_handler.assert_called_once()def asyncio_callback(self, error, *, module='asyncio.proactor_events',- name='_call_connection_lost', business=False, cleared=False):+ name='_call_connection_lost', business=False, cleared=False,+ recoverable=False, close_error=None, socket_override=None, server_override=None):namespace = {'__name__': module, 'failure': error}+ from asyncio.proactor_events import _ProactorBasePipeTransport+ namespace['Base'] = _ProactorBasePipeTransport if recoverable else objectif business:exec(compile('def fail():\n raise failure\n', 'peach/network_fixture.py', 'exec'), namespace)operation = 'fail()' if business else 'raise failure'- exec(compile(f'class _ProactorBasePipeTransport:\n def {name}(self):\n {operation}\n',+ if recoverable:+ operation = ('if self._called_connection_lost:\n return\n'+ ' self._protocol.connection_lost(None)\n raise failure')+ exec(compile(f'class _ProactorBasePipeTransport(Base):\n def {name}(self):\n {operation}\n','asyncio/proactor_events.py', 'exec'), namespace)- transport = namespace['_ProactorBasePipeTransport']()+ transport = object.__new__(namespace['_ProactorBasePipeTransport'])protocol_type = type('H11Protocol', (), {'__module__': 'uvicorn.protocols.http.h11_impl'})ssl_type = type('SSLProtocol', (), {'__module__': 'asyncio.sslproto'})transport._protocol = ssl_type()transport._protocol._app_protocol = None if cleared else protocol_type()+ transport._protocol.connection_lost = mock.Mock()+ if recoverable:+ transport._sock = socket_override if socket_override is not None else mock.Mock()+ if socket_override is None:+ transport._sock.close.side_effect = close_error+ transport._server = server_override if server_override is not None else mock.Mock()+ if server_override is not None:+ transport._server._attach(transport)+ transport._closing = True+ transport._called_connection_lost = False+ self.addCleanup(lambda: setattr(transport, '_sock', None))transport.private_url = 'https://private.example/?token=private-token'transport.private_path = 'D:/private/movie.mp4'loop = mock.Mock()loop.get_debug.return_value = False+ loop.diagnostic_transport = transport+ if recoverable:+ loop.diagnostic_socket = transport._sock+ loop.diagnostic_server = transport._serverloop.call_exception_handler.side_effect = lambda context: error_reporting.asyncio_handler(loop, context)asyncio.Handle(getattr(transport, name), (), loop)._run()return loop+ def test_connection_cleanup_completes_socket_close_and_server_detach_once(self):+ from peach import remote_reporting+ error = ConnectionResetError(10054, 'connection reset')+ error.winerror = 10054+ with mock.patch.object(remote_reporting, 'send') as send:+ loop = self.asyncio_callback(error, recoverable=True)+ transport = loop.diagnostic_transport+ self.assertIsNone(transport._sock)+ self.assertIsNone(transport._server)+ self.assertTrue(transport._called_connection_lost)+ self.assertTrue(self.records()[-1]['cleanup_recovered'])+ loop.diagnostic_socket.close.assert_called_once()+ loop.diagnostic_server._detach.assert_called_once_with(transport)+ self.assertEqual(transport._protocol.connection_lost.call_count, 1)+ asyncio.Handle(transport._call_connection_lost, (), loop)._run()+ self.assertEqual(len(self.records()), 1)+ self.assertEqual(transport._protocol.connection_lost.call_count, 1)+ loop.default_exception_handler.assert_not_called()+ send.assert_not_called()++ def test_socket_cleanup_failure_keeps_the_original_exception_and_reports(self):+ from peach import remote_reporting+ error = ConnectionResetError(10054, 'connection reset')+ error.winerror = 10054+ close_error = OSError(5, 'close failed')+ with mock.patch.object(remote_reporting, 'send') as send:+ loop = self.asyncio_callback(error, recoverable=True, close_error=close_error)+ send.assert_called_once()+ self.assertFalse(loop.diagnostic_transport._called_connection_lost)+ record = self.records()[-1]+ self.assertEqual(record['classification'], 'connection-cleanup-failed')+ self.assertFalse(record['cleanup_recovered'])+ self.assertIs(close_error.__cause__, error)+ loop.default_exception_handler.assert_called_once()+def test_windows_asyncio_connection_cleanup_stays_local_with_safe_context(self):from peach import remote_reportingerror = ConnectionResetError(10054, 'connection reset')@@ -435,6 +498,31 @@ class ErrorReportingTests(unittest.TestCase):self.assertNotIn(value, json.dumps(record))loop.default_exception_handler.assert_called_once()+ def test_connection_cleanup_closes_real_socket_and_releases_asyncio_server(self):+ import socket+ from peach import remote_reporting++ async def run():+ server = await asyncio.get_running_loop().create_server(asyncio.Protocol, '127.0.0.1', 0)+ accepted, peer = socket.socketpair()+ try:+ error = ConnectionResetError(10054, 'connection reset')+ error.winerror = 10054+ with mock.patch.object(remote_reporting, 'send') as send:+ loop = self.asyncio_callback(error, recoverable=True,+ socket_override=accepted, server_override=server)+ self.assertEqual(accepted.fileno(), -1)+ self.assertEqual(len(server._clients), 0)+ loop.default_exception_handler.assert_not_called()+ send.assert_not_called()+ finally:+ accepted.close()+ peer.close()+ server.close()+ await asyncio.wait_for(server.wait_closed(), timeout=2)++ asyncio.run(run())+def test_asyncio_tls_cleanup_keeps_protocol_class_after_application_detaches(self):from peach import remote_reportingerror = ConnectionResetError(10054, 'connection reset')diff --git a/tests/test_remote_reporting.py b/tests/test_remote_reporting.pyindex 307ce6f35..3d4b84d75 100644--- a/tests/test_remote_reporting.py+++ b/tests/test_remote_reporting.py@@ -141,6 +141,137 @@ class RemoteReportingTests(unittest.TestCase):self.assertEqual(send.call_args.args[1]['exception']['values'][0]['stacktrace']['frames'],[{'filename': 'peach-app.js', 'lineno': 12, 'colno': 34}])+ def test_control_and_source_states_keep_local_records_without_remote_events(self):+ import asyncio+ from concurrent.futures import CancelledError+ from peach import error_reporting+ from peach.browser_transport import BrowserUnavailable, ChallengeUnsolved+ from peach.jav_cover_fetch import CoverConnectError, DeadlineExceeded+ from peach.jobs import JobPaused, MeteredSourceDenied, DiskSpaceDenied, TaskRunConflict+ from peach.metadata import MetadataProviderError+ from peach.previews import PreviewUnavailable+ from peach.scraping_access import SourcePaused+ from peach.segments import SegmentCancelled+ from peach.sources.base import SourceFailure, FailureReason+ from peach.transcodes import TranscodeCancelled+ from starlette.requests import ClientDisconnect+ errors = [asyncio.CancelledError(), CancelledError(), InterruptedError(), JobPaused(),+ MeteredSourceDenied(), DiskSpaceDenied(), TaskRunConflict('processing', 1),+ SourcePaused(), DeadlineExceeded(), CoverConnectError(), PreviewUnavailable(),+ SegmentCancelled(), TranscodeCancelled(), BrowserUnavailable(), ChallengeUnsolved(),+ ClientDisconnect(),+ MetadataProviderError('auth', kind='auth'),+ MetadataProviderError('absent', kind='not_found'),+ MetadataProviderError('retry', kind='unavailable', retryable=True)]+ errors += [SourceFailure(reason, 'source state') for reason in FailureReason+ if reason != FailureReason.PARSE_ERROR]+ with mock.patch.object(reporting, 'send') as send:+ for error in errors:+ with self.subTest(error=type(error).__name__):+ path = error_reporting.event('background-job', error)+ self.assertTrue(path.is_file())+ send.assert_not_called()++ def test_source_parser_and_unknown_provider_failures_are_actionable(self):+ from peach.metadata import MetadataProviderError+ from peach.sources.base import SourceFailure, FailureReason+ errors = [SourceFailure(FailureReason.PARSE_ERROR, 'parser failure'),+ MetadataProviderError('parser failure', kind='unavailable', detail='parse_error'),+ MetadataProviderError('unknown failure')]+ for error in errors:+ with self.subTest(error=type(error).__name__), mock.patch.object(reporting, 'send') as send:+ reporting.capture('background-job', error)+ send.assert_called_once()++ def test_unrelated_exception_names_do_not_acquire_domain_state_semantics(self):+ for name in ('CancelledError', 'NotFound', 'Unavailable'):+ with self.subTest(name=name), mock.patch.object(reporting, 'send') as send:+ error = type(name, (RuntimeError,), {'__module__': 'peach.fixture'})('unexpected failure')+ reporting.capture('background-job', error)+ send.assert_called_once()++ def test_source_client_http_statuses_are_local_and_server_failures_are_reported(self):+ import httpx+ from urllib.error import HTTPError+ for code in (400, 401, 403, 404, 410, 429, 500, 503):+ response = httpx.Response(code, request=httpx.Request('GET', 'https://example.test/'))+ errors = [httpx.HTTPStatusError('http state', request=response.request, response=response),+ HTTPError('https://example.test/', code, 'http state', {}, None)]+ for error in errors:+ with self.subTest(code=code, error=type(error).__name__), \+ mock.patch.object(reporting, 'send') as send:+ reporting.capture('background-job', error)+ if code < 500:+ send.assert_not_called()+ else:+ send.assert_called_once()+ self.assertEqual(send.call_args.args[1]['tags']['status_code'], str(code))++ def test_warning_exceptions_do_not_consume_remote_error_budget(self):+ namespace = {}+ exec(compile('def fail():\n raise ValueError("retrying")\n', 'peach/fixture.py', 'exec'), namespace)+ try:+ namespace['fail']()+ except ValueError as error:+ with mock.patch.object(reporting, 'send') as send:+ reporting.capture('logging', error, level='WARNING', logger='peach.fixture')+ send.assert_not_called()+ reporting.capture('logging', error, level='ERROR', logger='peach.fixture')+ send.assert_called_once()++ def test_expected_media_input_conditions_are_local_and_decoder_faults_are_reported(self):+ for marker in ('Invalid data found', 'Permission denied', 'No space left', 'No such file or directory'):+ with self.subTest(marker=marker), mock.patch.object(reporting, 'send') as send:+ reporting.capture('frame-capture', returncode=1, stderr=marker)+ send.assert_not_called()+ with mock.patch.object(reporting, 'send') as send:+ import errno+ reporting.capture('media-probe', OSError(errno.ENOSPC, 'disk full'), tool='ffprobe')+ send.assert_not_called()+ reporting.capture('media-probe', subprocess.TimeoutExpired('ffprobe', 20), tool='ffprobe')+ send.assert_not_called()+ reporting.capture('media-probe', FileNotFoundError(2, 'missing input', 'movie.mp4'), tool='ffprobe')+ send.assert_not_called()+ reporting.capture('media-probe', FileNotFoundError(2, 'missing tool', 'ffprobe'), tool='ffprobe')+ send.assert_called_once()+ send.reset_mock()+ reporting.capture('frame-capture', returncode=69,+ stderr="Your platform doesn't support hardware accelerated AV1 decoding")+ send.assert_called_once()+ send.reset_mock()+ reporting.capture('frame-capture', returncode=1,+ stderr='Unknown decoder; Invalid data found')+ send.assert_called_once()++ def test_browser_abort_and_extension_frames_stay_local_and_bundle_versions_keep_positions(self):+ with mock.patch.object(reporting, 'send') as send:+ reporting.capture('browser', error_name='AbortError', stack='peach-app.js:12:34')+ reporting.capture('browser', stack='at chrome-extension://extension/peach-app.js:12:34')+ reporting.capture('browser', stack='at moz-extension://extension/peach-pages.js:12:34')+ send.assert_not_called()+ reporting.capture('browser', stack='at https://example.test/dist/peach-app.js?v=build:12:34')+ send.assert_called_once()+ frames = send.call_args.args[1]['exception']['values'][0]['stacktrace']['frames']+ self.assertEqual(frames, [{'filename': 'peach-app.js', 'lineno': 12, 'colno': 34}])++ def test_preview_environment_conditions_preserve_guidance_and_local_issue(self):+ from peach import library_processing+ from peach.previews import PreviewUnavailable+ resolver = mock.Mock()+ resolver.ffmpeg.return_value = None+ row = {'id': 1, 'path': str(self.root / 'missing.mp4'), 'location': 'local', 'name': 'missing.mp4'}+ with self.assertRaises(PreviewUnavailable) as raised:+ library_processing._generate_preview(self.config, row, mock.Mock(), resolver, progress=lambda: None)+ issue = mock.Mock()+ with mock.patch.object(reporting, 'send') as send, \+ mock.patch.object(library_processing, '_generate_preview', side_effect=raised.exception):+ made = library_processing._generate_previews(self.config, [row], mock.Mock(),+ update=mock.Mock(), issue=issue, guard=mock.Mock())+ self.assertEqual(made, 0)+ self.assertIn('FFmpeg', issue.call_args.args[1])+ self.assertTrue(issue.call_args.kwargs['retryable'])+ send.assert_not_called()+def test_sdk_wire_payload_has_no_local_context_or_consent_path(self):from sentry_sdk.transport import HttpTransportsent = []diff --git a/web/dist/peach-app.js b/web/dist/peach-app.jsindex 33c0f906a..cf79ad970 100644--- a/web/dist/peach-app.js+++ b/web/dist/peach-app.js@@ -20,17 +20,19 @@ function d() {if (u || typeof window > "u") return;u = !0;let e = Date.now(), t = 0, n = /* @__PURE__ */ new Set(), r = (r, i, a = "", o) => {- if (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10) return;- let s = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);- if (!s) return;- let c = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", l = `${r}:${s}:${c}`;- n.has(l) || (n.add(l), t += 1, fetch("/api/diagnostics/client-error", {+ let s = i instanceof Error || i instanceof DOMException ? i.name : "";+ if (s === "AbortError" || (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10)) return;+ let c = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);+ if (!c) return;+ let l = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", u = `${r}:${c}:${l}`;+ n.has(u) || (n.add(u), t += 1, fetch("/api/diagnostics/client-error", {method: "POST",headers: { "Content-Type": "application/json" },body: JSON.stringify({kind: r,- message: s,- stack: c,+ message: c,+ stack: l,+ error_name: s,page: window.location.pathname,source_file: o?.filename?.split("/").pop()?.split("?")[0],line: o?.lineno,diff --git a/web/dist/peach-pages.js b/web/dist/peach-pages.jsindex 07d08e62f..91ed22beb 100644--- a/web/dist/peach-pages.js+++ b/web/dist/peach-pages.js@@ -13,17 +13,19 @@ function u() {if (l || typeof window > "u") return;l = !0;let e = Date.now(), t = 0, n = /* @__PURE__ */ new Set(), r = (r, i, a = "", o) => {- if (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10) return;- let s = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);- if (!s) return;- let c = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", l = `${r}:${s}:${c}`;- n.has(l) || (n.add(l), t += 1, fetch("/api/diagnostics/client-error", {+ let s = i instanceof Error || i instanceof DOMException ? i.name : "";+ if (s === "AbortError" || (Date.now() - e >= 6e4 && (e = Date.now(), t = 0, n.clear()), t >= 10)) return;+ let c = (i instanceof Error ? i.message : typeof i == "string" ? i : a).slice(0, 2e3);+ if (!c) return;+ let l = i instanceof Error ? (i.stack ?? "").slice(0, 6e3) : "", u = `${r}:${c}:${l}`;+ n.has(u) || (n.add(u), t += 1, fetch("/api/diagnostics/client-error", {method: "POST",headers: { "Content-Type": "application/json" },body: JSON.stringify({kind: r,- message: s,- stack: c,+ message: c,+ stack: l,+ error_name: s,page: window.location.pathname,source_file: o?.filename?.split("/").pop()?.split("?")[0],line: o?.lineno,