Peach: Local-first video metadata scraper and media manager.
ci(test): 前端检查通过后运行完整回归
longmeidao3ef0e5b4a23e
diff --git a/.github/workflows/test.yml b/.github/workflows/test.ymlindex ef326699a..483f9c890 100644--- a/.github/workflows/test.yml+++ b/.github/workflows/test.yml@@ -62,9 +62,23 @@ jobs:BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before || github.sha }}run: uv run --locked --python 3.14 scripts/ci_plan.py --event "$EVENT_NAME" --base "$BASE_SHA"+ frontend-ready:+ name: 前端检查结果+ needs: [plan, web-bundle, web-vendor]+ if: ${{ always() && needs.plan.result == 'success' }}+ runs-on: ubuntu-latest+ timeout-minutes: 5+ steps:+ - uses: actions/checkout@v7+ - name: 所选前端检查均成功+ env:+ RESULTS: ${{ toJSON(needs) }}+ REQUIRED_JOBS: ${{ needs.plan.outputs.required_jobs }}+ run: python scripts/ci_plan.py --verify-results --phase frontend+wheel-build:- needs: plan- if: needs.plan.outputs.wheel == 'true'+ needs: [plan, frontend-ready]+ if: ${{ !cancelled() && needs.frontend-ready.result == 'success' && needs.plan.outputs.wheel == 'true' }}runs-on: ubuntu-latesttimeout-minutes: 15steps:@@ -89,7 +103,7 @@ jobs:wheel-smoke:name: wheel 最小安装 / ${{ matrix.os }} / ${{ matrix.python }}needs: [plan, wheel-build]- if: needs.plan.outputs.wheel == 'true'+ if: ${{ !cancelled() && needs.wheel-build.result == 'success' && needs.plan.outputs.wheel == 'true' }}strategy:matrix: ${{ fromJSON(needs.plan.outputs.wheel_matrix) }}runs-on: ${{ matrix.os }}@@ -113,7 +127,8 @@ jobs:python "$GITHUB_WORKSPACE/scripts/smoke_wheel.py"python:name: ${{ matrix.os }} / Python ${{ matrix.python }} / ${{ matrix.scope }} / ${{ matrix.shard_index }}- needs: plan+ needs: [plan, frontend-ready]+ if: ${{ !cancelled() && needs.frontend-ready.result == 'success' }}runs-on: ${{ matrix.os }}timeout-minutes: 60strategy:@@ -227,15 +242,15 @@ jobs:- name: 核对 web/dist 与源码一致run: |git add --intent-to-add -- web/dist- git diff --exit-code -- web/dist+ git diff --exit-code --stat -- web/dist- name: npm audit (frontend)run: python scripts/npm_audit.py --prefix frontend# 每片拥有独立服务与演示库;文件内的浏览器交互保持串行。web-e2e:name: 浏览器回归 / ${{ matrix.index }}- needs: [plan]- if: needs.plan.outputs.browser == 'true'+ needs: [plan, frontend-ready]+ if: ${{ !cancelled() && needs.frontend-ready.result == 'success' && needs.plan.outputs.browser == 'true' }}runs-on: windows-latesttimeout-minutes: 40strategy:@@ -313,7 +328,7 @@ jobs:verified:name: 完整验证结果if: always()- needs: [plan, workflow-lint, python, wheel-build, wheel-smoke, web-bundle, web-e2e, web-vendor]+ needs: [plan, workflow-lint, frontend-ready, python, wheel-build, wheel-smoke, web-bundle, web-e2e, web-vendor]runs-on: ubuntu-latesttimeout-minutes: 5steps:diff --git a/docs/REUSE.md b/docs/REUSE.mdindex 1152e2fb5..2b9b1250d 100644--- a/docs/REUSE.md+++ b/docs/REUSE.md@@ -22,7 +22,7 @@ Peach 保留页面粗提取与身份冲突提示,服务端复用 `catalog_rule资料采集与关注检查复用已有字段判据、事务和候选复核。全部本地 NFO 先读取并按影片提交;在线资料按来源返回时提交,候选 CSV 仍按时间合并落盘。关注列表的安全 partial 行与逐项详情分别保存,历史边界和整页游标沿用连接器规则。中断保留已提交结果,停止后不再落库。-显示层复用 SQLite `ledger_revision` 触发器、候选与图片目录版本,以及 TanStack Query 5.104.2(MIT)查询失效和任务轮询;不新增推送服务或依赖。应用根按结果版本刷新相关读缓存,任务心跳不使资料失效;慢请求期间的新版本排队,保留当前内容。资源同步扫描同时显示已完成来源,清理仍需完整扫描和确认。隔离账本、暂停的来源调用与手动放行的 Promise 验证中间结果可见及中断保留;无真实抓取或真实 ledger 写入。+显示层复用 SQLite `ledger_revision` 触发器、候选与图片目录版本,以及 TanStack Query 5.104.3(MIT)查询失效和任务轮询;不新增推送服务或依赖。应用根按结果版本刷新相关读缓存,任务心跳不使资料失效;慢请求期间的新版本排队,保留当前内容。资源同步扫描同时显示已完成来源,清理仍需完整扫描和确认。隔离账本、暂停的来源调用与手动放行的 Promise 验证中间结果可见及中断保留;无真实抓取或真实 ledger 写入。## STRM 与挂载来源@@ -258,7 +258,7 @@ OneJAV、Free JAV Torrent 只给种子链接。这类条目经索引器同源的## 独立测试包在线更新-- 任务起跑与安装预约复用已固定的 filelock 4.1.1(MIT,Windows/macOS/Linux,无额外依赖)和原子 JSON。`update_tasks` 在同一数据目录内串行登记任务和预约安装,覆盖各服务进程、任务中心、后继领取和 `job_main` 批处理。安装请求触发采集任务在检查点保存并暂停,任务中心保留 `pending` 行;新采集请求进入等待。安装入口等在途步骤保存进度,随后停止自己的服务,不要求用户等待整轮任务结束。源码批处理保留 PID 锁,写完在途批次后等待安装,结束后由原进程继续。+- 任务起跑与安装预约复用已固定的 filelock 4.2.0(MIT,Windows/macOS/Linux,无额外依赖)和原子 JSON。`update_tasks` 在同一数据目录内串行登记任务和预约安装,覆盖各服务进程、任务中心、后继领取和 `job_main` 批处理。安装请求触发采集任务在检查点保存并暂停,任务中心保留 `pending` 行;新采集请求进入等待。安装入口等在途步骤保存进度,随后停止自己的服务,不要求用户等待整轮任务结束。源码批处理保留 PID 锁,写完在途批次后等待安装,结束后由原进程继续。- 安装器在临时目录展开随包运行时,以 `--installer-prepare` 承接馆藏处理状态并暂停任务;预约归安装器持有,完成或取消时释放。在线更新将预约交给更新助手,切换目录和迁移完成后释放,失败回滚也释放。恢复轮询复用 APScheduler,安装取消时现有服务也会接回任务。不采用新任务队列或更新框架:任务语义、PyInstaller 目录切换和 Inno Setup 生命周期继续由已有实现承担。- `job_recovery` 保存馆藏处理范围、阶段、指定影片、时间轴密度、单部封面、关注、订阅和想要资料的请求;服务启动或安装结束后只领取未完成记录。暂停任务复用任务中心的同一行,已完成、失败和主动停止的任务不循环重试。馆藏预览在产物登记后保存影片检查点,元数据检查点跟随候选 CSV 成功落盘;未取得结果的项目不记完成。时间轴复用已有索引判定。恢复领取和新任务起跑共用安装闸门,数据仍留在本机 state 目录,不引入 ledger 迁移。diff --git a/docs/TESTING.md b/docs/TESTING.mdindex 02b2e7436..467b15aee 100644--- a/docs/TESTING.md+++ b/docs/TESTING.md@@ -34,6 +34,8 @@ uv sync --locked --extra buildDependabot 的 PR 一律在本地接管,不在网页上合:+依赖升级须同时提交派生产物。只改 npm 清单与锁文件时,vendor 版本记录或构建结果可能与 Git 中的文件不一致,相关门槛会拒绝;这本身不代表依赖存在兼容性问题。Test 工作流先完成所选前端检查,通过后才启动 Python、浏览器与 wheel 回归。缺少产物时先本地接管重算,不能跳过一致性检查后直接合并。+- npm 的两份清单各有一层派生产物:根 `package.json` 对应 `web/vendor/**` 与 `web/index.html` 的版本注释,`frontend/package.json` 对应 `web/dist/` 的主界面与独立页面包。Dependabot 算不出这些,它的 workflow 拿到的 token 又是只读的,推不回 `dependabot/**`,所以 `npm run check:vendor` 或前端产物检查会失败。- uv 与 github-actions 没有派生产物,但 master 在本机集成、通常领先 origin。在网页上合会让两边分叉,回并要在主检出 master 上 merge,被 `scripts/githooks/` 拒收。@@ -82,7 +84,7 @@ Test 工作流的 `web-bundle` 在 Ubuntu 上统一执行 tsc、lint、Vitest、浏览器用例由 `web-e2e` 的六个 Windows runner 分担,每片有自己的服务、演示库与日志。`ci_plan.py` 按 `ci-browser-timings.json` 的实测秒数分配文件,新文件使用默认权重;每个文件恰好执行一次。每片经项目入口的 `-Focus` 运行分配的浏览器文件,首启文件始终归首次设置服务。它装 Node 24、前端依赖、ffmpeg,并经 `PEACH_E2E_CHROME` 指定 Chrome。本机浏览器入口与并发语义保持完整。-PR 计划按改动选择前端、vendor、wheel 与浏览器任务,文档 PR 只执行适用检查;主线、依赖、未知影响与手动验证包含全部任务。`verified` 根据计划核对必需任务,缺失、失败、取消及必需任务跳过均判失败。Actions 工作流本身由固定版 actionlint 检查语法、表达式、依赖与脚本。PR 更新取消同 PR 的过期运行,主线 Test 按提交各自保留结果。+PR 计划按改动选择前端、vendor、wheel 与浏览器任务,文档 PR 只执行适用检查;主线、依赖、未知影响与手动验证包含全部任务。`frontend-ready` 汇总所选前端检查,失败时不启动后续完整回归;未选择前端任务的计划可正常执行。产物差异以文件统计显示。`verified` 根据计划核对必需任务,缺失、失败、取消及必需任务跳过均判失败。Actions 工作流本身由固定版 actionlint 检查语法、表达式、依赖与脚本。PR 更新取消同 PR 的过期运行,主线 Test 按提交各自保留结果。Python 分片上传完整测试输出;浏览器分片上传 TAP、服务日志及已脱敏网络日志,保留 14 天。npm 审计遇到 high 或 critical 漏洞直接失败;只有临时网络错误最多请求三次,在前两次失败后分别等待 15 与 30 秒,未知响应不当成通过。diff --git a/frontend/e2e/orphan-table.test.ts b/frontend/e2e/orphan-table.test.tsindex 6b2ec9d74..546ab6e16 100644--- a/frontend/e2e/orphan-table.test.ts+++ b/frontend/e2e/orphan-table.test.ts@@ -48,11 +48,11 @@ describe('孤儿记录数据表', () => {await track.waitFor({ state: 'visible' });const before = await track.locator('.ovthumb').evaluate(node => getComputedStyle(node).transform);await scroller.evaluate(node => { node.scrollLeft = node.scrollWidth });- await page.waitForFunction(() => {- const node = document.querySelector('#orphan-records [data-overlay-scrollbar]')!;- return node.scrollLeft > 0;- });- await settle(page);+ await page.waitForFunction(previous => {+ const node = document.querySelector('#orphan-records [data-overlay-scrollbar]');+ const thumb = document.querySelector('#orphan-records [data-board-data-table] > .ov-x .ovthumb');+ return !!node && !!thumb && node.scrollLeft > 0 && getComputedStyle(thumb).transform !== previous;+ }, before);const after = await track.locator('.ovthumb').evaluate(node => getComputedStyle(node).transform);assert.notEqual(after, before, '横向滑块没有跟随表格滚动');await scroller.evaluate(node => { node.scrollLeft = 0 });diff --git a/scripts/ci_plan.py b/scripts/ci_plan.pyindex 35f8278cc..817986ed9 100644--- a/scripts/ci_plan.py+++ b/scripts/ci_plan.py@@ -32,9 +32,16 @@ def browser_matrix(root: Path = ROOT, count: int = 6) -> dict:for index, group in enumerate(groups)]}-def validate_results(results: dict, required: list[str]) -> None:+def validate_results(results: dict, required: list[str], *, phase: str = "complete") -> None:"""所选任务全部成功;未选择的任务可以跳过,不能隐藏失败。"""- if not {"plan", "python", "workflow-lint"}.issubset(required):+ if phase == "frontend":+ required = [name for name in required if name in {"plan", "web-bundle", "web-vendor"}]+ mandatory = {"plan"}+ elif phase == "complete":+ mandatory = {"plan", "python", "workflow-lint", "frontend-ready"}+ else:+ raise ValueError(f"未知验证阶段:{phase}")+ if not mandatory.issubset(required):raise ValueError("验证计划缺少必需任务")bad = {name: results.get(name, {}).get("result", "missing")for name in required if results.get(name, {}).get("result") != "success"}@@ -69,7 +76,7 @@ def plan(event: str, paths: list[str], contents: dict | None = None) -> dict:"scripts/vendor_web_dependencies.mjs"} for path in paths)wheel = shared or "packaging" in scopes or any(path.startswith(("src/", "migrations/", "web/", "resources/"))for path in paths)- required = ["plan", "python", "workflow-lint"]+ required = ["plan", "python", "workflow-lint", "frontend-ready"]required += [name for enabled, names in ((browser, ["web-e2e"]), (frontend, ["web-bundle"]),(vendor, ["web-vendor"]), (wheel, ["wheel-build", "wheel-smoke"]))if enabled for name in names]@@ -83,9 +90,11 @@ def main() -> None:parser.add_argument("--event", choices=("push", "pull_request", "workflow_dispatch"))parser.add_argument("--base")parser.add_argument("--verify-results", action="store_true")+ parser.add_argument("--phase", choices=("complete", "frontend"), default="complete")args = parser.parse_args()if args.verify_results:- validate_results(json.loads(os.environ["RESULTS"]), json.loads(os.environ["REQUIRED_JOBS"]))+ validate_results(json.loads(os.environ["RESULTS"]), json.loads(os.environ["REQUIRED_JOBS"]),+ phase=args.phase)returnif not args.event or not args.base:parser.error("生成计划需要 --event 与 --base")diff --git a/tests/test_frontend_build.py b/tests/test_frontend_build.pyindex a658c0534..eb98341be 100644--- a/tests/test_frontend_build.py+++ b/tests/test_frontend_build.py@@ -293,7 +293,7 @@ class FrontendManifestTests(unittest.TestCase):"npm --prefix frontend test","npm --prefix frontend run build","git add --intent-to-add -- web/dist",- "git diff --exit-code -- web/dist"):+ "git diff --exit-code --stat -- web/dist"):self.assertIn(step, workflow, f"CI 缺了 {step}")def test_the_bundle_is_committed_and_the_toolchain_is_not(self):diff --git a/tests/test_test_planning.py b/tests/test_test_planning.pyindex 7659efd01..a1645a79f 100644--- a/tests/test_test_planning.py+++ b/tests/test_test_planning.py@@ -96,6 +96,11 @@ class CIFrontendOwnershipTests(unittest.TestCase):class TestPlanningTests(unittest.TestCase):+ def setUp(self):+ environment = patch.dict(os.environ, {'PEACH_TEST_FOCUS': '', 'PEACH_E2E_FILES': ''})+ environment.start()+ self.addCleanup(environment.stop)+def test_every_supplementary_test_id_names_a_real_test(self):"""域的补充用例按名字装载:被引用的用例改名后,装进来的是一个必然报错的占位用例,选到这一域的 auto 都签不出记录。在这里按名字装一遍,改名当场就红。"""@@ -188,7 +193,7 @@ class TestPlanningTests(unittest.TestCase):def test_document_changes_avoid_optional_jobs_and_unknown_changes_require_them(self):documents = ci_plan.plan('pull_request', ['docs/TESTING.md'])- self.assertEqual(documents['required_jobs'], ['plan', 'python', 'workflow-lint'])+ self.assertEqual(documents['required_jobs'], ['plan', 'python', 'workflow-lint', 'frontend-ready'])unknown = ci_plan.plan('pull_request', ['unknown.file'])self.assertLessEqual({'web-e2e', 'web-bundle', 'web-vendor', 'wheel-smoke'},set(unknown['required_jobs']))@@ -198,7 +203,7 @@ class TestPlanningTests(unittest.TestCase):set(mainline['required_jobs']))def test_result_gate_rejects_missing_failed_cancelled_or_required_skipped_jobs(self):- required = ['plan', 'python', 'workflow-lint', 'web-e2e']+ required = ['plan', 'python', 'workflow-lint', 'frontend-ready', 'web-e2e']passed = {name: {'result': 'success'} for name in required}ci_plan.validate_results({**passed, 'web-vendor': {'result': 'skipped'}}, required)for result in ('failure', 'cancelled', 'skipped'):@@ -209,6 +214,33 @@ class TestPlanningTests(unittest.TestCase):with self.assertRaises(ValueError):ci_plan.validate_results(passed, [])+ def test_frontend_gate_requires_only_the_selected_frontend_jobs(self):+ required = ci_plan.plan('push', ['frontend/package-lock.json'])['required_jobs']+ passed = {name: {'result': 'success'} for name in ('plan', 'web-bundle', 'web-vendor')}+ ci_plan.validate_results(passed, required, phase='frontend')+ for name in ('web-bundle', 'web-vendor'):+ for result in ('missing', 'failure', 'cancelled', 'skipped'):+ results = dict(passed)+ if result == 'missing':+ results.pop(name)+ else:+ results[name] = {'result': result}+ with self.subTest(name=name, result=result), self.assertRaises(ValueError):+ ci_plan.validate_results(results, required, phase='frontend')++ def test_frontend_gate_allows_document_plans_and_rejects_an_invalid_plan(self):+ required = ci_plan.plan('pull_request', ['docs/TESTING.md'])['required_jobs']+ results = {'plan': {'result': 'success'}, 'web-bundle': {'result': 'skipped'},+ 'web-vendor': {'result': 'skipped'}}+ ci_plan.validate_results(results, required, phase='frontend')+ for result in ('failure', 'cancelled', 'skipped'):+ with self.subTest(result=result), self.assertRaises(ValueError):+ ci_plan.validate_results({**results, 'plan': {'result': result}}, required, phase='frontend')+ with self.assertRaises(ValueError):+ ci_plan.validate_results(results, [], phase='frontend')+ with self.assertRaises(ValueError):+ ci_plan.validate_results(results, required, phase='unknown')+def test_dependency_and_manual_runs_cover_both_systems(self):for event, paths in [('push', ['uv.lock']), ('pull_request', ['migrations/0025.sql']),('workflow_dispatch', [])]: