Skip to content

armgit / peach

Public Imported from GitHub
0

Peach: Local-first video metadata scraper and media manager.

feat(follow): 支持 Iwara 视频作者订阅与按需播放

longmeidao8429bb039673
diff --git a/docs/REUSE.md b/docs/REUSE.md
index d17281fc3..12a6b220d 100644
--- a/docs/REUSE.md
+++ b/docs/REUSE.md
@@ -648,6 +648,8 @@ A 盘垃圾隔离使用短名称 `peach-purge-<32 位 UUID>.peach-quarantine`,
- 共用网站的厂牌专页通过 `site_logos.logo_images(owner=...)` 核对图片的明确归属,只接收名称匹配的标识,不沿用平台页头、推荐厂牌或全站社媒账号;独立官网继续复用站点声明与页头标识。
- 关注的作者头像与来源图标是元数据:`follow_assets` 取回落在 `generated/follow-assets/`,经 `/follow-avatar`、`/source-icon` 给页面;地址只从固定表或固定主机拼,字节先认成图再写盘,到期重取失败继续用旧的并退避一小时,缓存有效期与 `/link-mark` 共用 `web_settings.metadata_refresh_seconds`。视频与图片不存本机。官方头像先认 FANBOX,没有时 `follow_avatar.profile_avatar_tiers` 取名片上的 X 与 Patreon,`follow_assets.largest_image` 留像素最大的那张。
- 厂牌标识由契约位 `has_logo` 决定出不出图:没装标识的厂牌一个 `<img>` 都不发,改用首字母底板,不靠 404 摘。标识 198 张随仓库分发(ADR-0026)。
+- Iwara 视频作者订阅复用 [yt-dlp 的 Iwara 提取器](https://github.com/yt-dlp/yt-dlp/blob/9b58cb33d0cd3e823c8d8732f6adce8faa1f37e3/yt_dlp/extractor/iwara.py)(Unlicense,固定 revision `9b58cb33d0cd3e823c8d8732f6adce8faa1f37e3`)的 profile→用户 ID→videos 分页模型与 `X-Version` 文件接口签名;[gallery-dl 的 Iwara 提取器](https://github.com/mikf/gallery-dl/blob/882c8d0bb5c5a3ed4e88726d2f42a1cb7db59bf2/gallery_dl/extractor/iwara.py)(GPL-2.0,固定 revision `882c8d0bb5c5a3ed4e88726d2f42a1cb7db59bf2`)用于交叉核对 `apiq.iwara.tv`、视频文件字段与格式清单,不复制其实现。两者在 2026-10-23 获取时均维护中,Python 客户端可跨平台运行;完整下载器包含 Peach 不使用的下载、账户缓存与大量站点能力,因此只复用公开协议模型和 Unlicense 签名算法,不新增依赖。HTTP 复用现有 `curl_cffi`:匿名只读 POC 中普通 HTTP 对 API 返回 403,Chrome 传输取得作者资料、视频列表两页、详情与三档格式,五次请求约 4.9 KB、4.7 秒,未读取凭据或下载媒体。Peach 保留 DTO、关注状态、历史游标、凭据隔离及媒体代理;可选 `access_token` 仅发送到固定 API,不同步。列表只保存稳定视频页地址,短效文件地址与清晰度仅在播放缓存中保留;外部播放器、权限不足、空历史页、限流与结构错误分别处理。用法见 [Iwara 视频作者订阅](SOURCING.md#iwara-视频作者订阅)。
+- Iwara 封面复用 `/follow-image` 的同源缓存,官方缩略图经正式 HTTPX 传输实测取得 200 `image/jpeg`;来源图标采用官网声明且实测为 PNG 的 `/logo.png`。官方头像专用解析路径未取得,作者头像沿用首字母回退;作者主页的用户名与署名复用既有官方别名学习。部分迁移视频没有可用时长或原画分辨率,字段保留未知,播放仍优先选择 `Source` 原画。
- 关注检查分两阶段:列表阶段落 partial 行,详情补全按 provider 额度只补新行和未补齐行。「补齐过」由连接器的 `ENRICHED_MARK` 声明、判据登记在 `follow_store._ENRICHED_PREDICATES`。
- 存量行重抓(连接器新学到图片宽高、封面这类字段时,旧行不会自己补上)分两步:把 `ENRICHED_MARK` 换成新字段落库后才有的键并登记判据;再 `POST /api/follow/check` 发 `{"older":true,"backfill_all":true,"rewind":true,"background":true}`(可加 `"sources":[id,…]`),从第 1 页走到站点说没有更多,进度读 `GET /api/follow/check`。`backfill_all`、`rewind` 离开 `older` 被拒收、不进界面,ledger 的回填游标只进不退;长跑任务,重启前按 `peach-batch-jobs` 先查。
- 图片的固有宽高只问文件头:`follow_image_dims.probe_image_dims` 发 `Range: bytes=0-65535`,`dims_from_header` 认 PNG/GIF/WebP(VP8、VP8L、VP8X)/JPEG(跳过 EXIF 到 SOF);连接器、`backfill_follow_image_dims.py` 与 `/api/follow/image-dims` 落库前都经 `positive_dims` 归一,`FollowStore.set_image_dims` 只补空缺。宽高只为图片视图的卡面预留比例,回填每个条目只问卡面那张;原文件主机拦脚本(pawchive 的 `file.` 子域挂 ddos-guard)时按缩略图量,详情原图取不到也退回缩略图。
diff --git a/docs/SOURCING.md b/docs/SOURCING.md
index e8c6d64aa..a5e01c0c6 100644
--- a/docs/SOURCING.md
+++ b/docs/SOURCING.md
@@ -36,6 +36,12 @@
网盘媒体的探测、九宫格、时间轴和播放会读取视频数据;挂载路径的网络连接由 CloudDrive2、OpenList 对应挂载工具负责。Peach 的目录「媒体读取策略」限制后台读取和确认边界,不修改挂载工具的代理。联网采集作品文字和封面使用上述来源设置,与视频所在网盘分别配置。
+## Iwara 视频作者订阅
+
+在关注管理粘贴 Iwara 作者主页或作者的视频列表链接,或用 `provider=iwara` 与作者用户名登记来源。订阅按发布时间读取作者视频,支持检查更新和抓取更早内容;图片、播放列表和全站视频列表不属于这个来源。条目保留标题、作者、时间、标签、时长和封面;使用外部播放器的条目提示打开来源页面。
+
+公开作者视频无需登录。有访问限制的视频须由具备访问权的账号提供有效 `access_token`,在「来源与订阅」设置中填写。令牌只保存在本机凭据文件,不进账本或同步;失效时重新登录并按设置页说明复制。Peach 不求解浏览器验证。播放时按需解析短效视频地址,默认选择最高分辨率,可切换来源提供的清晰度;私有或已删除视频、接口结构变化与限流会显示具体失败。没有更早内容是正常的历史终点。
+
## 采集判据
采集结果须保留资料来源与判断依据。代码规则能够确定、且可按来源整批撤回的资料按既有规则保存;LLM 输出、来源冲突和多种可能结果保留为待确认资料。不能只凭站名判断资料一定正确。
diff --git a/frontend/src/react/follow-manage/credential-providers.json b/frontend/src/react/follow-manage/credential-providers.json
index f8afef306..53ac9e919 100644
--- a/frontend/src/react/follow-manage/credential-providers.json
+++ b/frontend/src/react/follow-manage/credential-providers.json
@@ -3,6 +3,7 @@
{"provider":"f95zone","label":"F95zone","requirement":"optional","fields":["cookie"],"why":"发现更新不需要登录;只有取附件和 masked 下载链接才需要会话。","where":"https://f95zone.to/","howto":"登录后从浏览器复制整条 Cookie 请求头,写进凭据文件的 cookie 字段。"},
{"provider":"fanbox","label":"FANBOX","requirement":"optional","fields":["cookie"],"why":"公开列表不需要登录;帖子详情被 FANBOX 验证页拦住时,可用浏览器会话取得正文、多图和外部资源链接。","where":"https://www.fanbox.cc/","howto":"登录 FANBOX 后,从一次成功的 api.fanbox.cc/post.info 请求复制整条 Cookie 请求头。"},
{"provider":"gofile","label":"Gofile","requirement":"optional","fields":["api_token"],"why":"用于展开 Gofile 文件页,取得其中的图片和视频列表;Gofile 当前要求 Premium 才能读取 contents API,不配置仍会保留文件页链接。","where":"https://gofile.io/myprofile","howto":"登录 Premium Gofile 账户后,在个人资料页复制 API token。"},
+ {"provider":"iwara","label":"Iwara","requirement":"optional","fields":["access_token"],"why":"公开作者视频无需登录;有访问限制的视频需要账号访问权及有效登录凭据。","where":"https://www.iwara.tv/","howto":"登录 Iwara 后,从成功的 apiq.iwara.tv/video 请求复制 Authorization 中 Bearer 后的值,填写 access_token。令牌过期后需重新复制。"},
{"provider":"kemono","label":"Kemono","requirement":"none"},
{"provider":"patreon","label":"Patreon","requirement":"none"},
{"provider":"pawchive","label":"Pawchive","requirement":"none"},
diff --git a/frontend/src/react/follow-manage/follow-manage.ts b/frontend/src/react/follow-manage/follow-manage.ts
index fa6a4cdd8..95d6bf1b7 100644
--- a/frontend/src/react/follow-manage/follow-manage.ts
+++ b/frontend/src/react/follow-manage/follow-manage.ts
@@ -668,7 +668,7 @@ export const checkedText = (source: FollowSource): string =>
/** 有站点图标的来源。图标由服务端按 `follow_assets.SOURCE_ICON_URLS` 取回并存在本机,
* 页面只认这份名单:没登记的来源直接不出 `<img>`。 */
export const SOURCE_ICON_PROVIDERS = new Set([
- 'fanbox', 'patreon', 'subscribestar', 'kemono', 'coomer', 'pawchive',
+ 'fanbox', 'patreon', 'subscribestar', 'iwara', 'kemono', 'coomer', 'pawchive',
'rule34video', 'rule34xxx', 'rule34paheal', 'gofile', 'f95zone', 'simpcity',
]);
diff --git a/src/peach/api.py b/src/peach/api.py
index 34ea69807..034076e97 100644
--- a/src/peach/api.py
+++ b/src/peach/api.py
@@ -454,6 +454,7 @@ def create_app(
await tunnel_startup
await asyncio.to_thread(tunnel_manager.stop)
http_transport.close()
+ follow_media_resolver.close()
hls_plan_executor.shutdown(wait=False, cancel_futures=True)
app = FastAPI(
diff --git a/src/peach/follow_assets.py b/src/peach/follow_assets.py
index aca688fc9..77934da3e 100644
--- a/src/peach/follow_assets.py
+++ b/src/peach/follow_assets.py
@@ -43,6 +43,7 @@ RETRY_SECONDS = 3600
SOURCE_ICON_URLS: dict[str, str] = {
"fanbox": "https://www.fanbox.cc/favicon.ico",
"patreon": "https://www.patreon.com/favicon.ico",
+ "iwara": "https://www.iwara.tv/logo.png",
"subscribestar": "https://assets.subscribestar.com/assets/public/images/favicons/"
"favicon-32x32-b9aa1e7e5bab6cb1b28b5161e16f9d42.png",
"kemono": "https://kemono.cr/assets/favicon-CPB6l7kH.ico",
diff --git a/src/peach/follow_providers.py b/src/peach/follow_providers.py
index 8de72b1a4..8564225c5 100644
--- a/src/peach/follow_providers.py
+++ b/src/peach/follow_providers.py
@@ -77,6 +77,9 @@ PROVIDERS: dict[str, ProviderSpec] = {
priority=2, official_identity=True),
ProviderSpec("patreon", "Patreon", source_url="https://www.patreon.com/cw/{ref}",
url_hosts=("patreon.com",), priority=3, official_identity=True),
+ ProviderSpec("iwara", "Iwara", source_url="https://www.iwara.tv/profile/{ref}",
+ hosts=("iwara.tv",), url_hosts=("iwara.tv",),
+ priority=4, backfill=True, official_identity=True),
# 归档站:同一套代码的姊妹站,支持真实历史分页所以可回填。
ProviderSpec("kemono", "Kemono", source_url="https://kemono.cr/{ref}",
hosts=("kemono.cr",), url_hosts=("kemono.cr",),
diff --git a/src/peach/follow_secrets.py b/src/peach/follow_secrets.py
index 490a8821a..6b04dbab7 100644
--- a/src/peach/follow_secrets.py
+++ b/src/peach/follow_secrets.py
@@ -203,6 +203,14 @@ CREDENTIAL_GUIDE: dict[str, dict] = {
"coomer": {"requirement": "none"},
"pawchive": {"requirement": "none"},
"rule34video": {"requirement": "none"},
+ "iwara": {
+ "requirement": "optional",
+ "fields": ["access_token"],
+ "syncable": [],
+ "why": "公开作者视频无需登录;有访问限制的视频需要账号访问权及有效登录凭据。",
+ "where": "https://www.iwara.tv/",
+ "howto": "登录 Iwara 后,从成功的 apiq.iwara.tv/video 请求复制 Authorization 中 Bearer 后的值,填写 access_token。令牌过期后需重新复制。",
+ },
"rule34xxx": {
"requirement": "required",
"fields": ["user_id", "api_key"],
diff --git a/src/peach/follow_sources.py b/src/peach/follow_sources.py
index b1ae37e61..1ef1df208 100644
--- a/src/peach/follow_sources.py
+++ b/src/peach/follow_sources.py
@@ -10,6 +10,7 @@ from __future__ import annotations
import base64
import html
+import hashlib
import json
import re
import time
@@ -401,15 +402,9 @@ _PAHEAL_LIST_RE = re.compile(r"^/post/list/([^/]+)/\d+$")
def canonical_source_ref(provider: str, ref: str) -> str:
- """Return the provider's stable source identity.
-
- Most provider ids are case-sensitive opaque values. rule34.xxx tags are not:
- the API returns the same feed for ``LazyProcrastinator`` and
- ``lazyprocrastinator``. Keeping the pasted spelling in the unique key therefore
- creates duplicate subscriptions when the same author is added in two batches.
- """
+ """来源的稳定身份;booru 标签与 Iwara 用户名忽略大小写。"""
value = str(ref or "").strip()
- return value.casefold() if provider in {"rule34xxx", "rule34paheal"} else value
+ return value.casefold() if provider in {"rule34xxx", "rule34paheal", "iwara"} else value
#: 线程 slug 里从这个 token 起就不是作品名了:f95 的惯例是
@@ -775,6 +770,188 @@ class _BaseConnector:
).expand(links, labels=labels)
+class IwaraConnector(_BaseConnector):
+ """Iwara 作者视频列表及播放格式;协议依据见 docs/REUSE.md。"""
+
+ provider = "iwara"
+ API_ROOT = "https://apiq.iwara.tv"
+ PAGE_SIZE = 32
+ _HANDLE_RE = re.compile(r"^[A-Za-z0-9_-]{1,80}$")
+ _VIDEO_ID_RE = re.compile(r"^[A-Za-z0-9]{1,80}$")
+
+ def __init__(self, **kwargs):
+ if kwargs.get("transport") is None:
+ kwargs["transport"] = CurlCffiTransport(impersonate="chrome")
+ super().__init__(**kwargs)
+
+ @classmethod
+ def parse_url(cls, provider, parsed, host):
+ matched = re.fullmatch(r"/profile/([A-Za-z0-9_-]{1,80})(?:/videos)?/?",
+ parsed.path)
+ if host not in {"iwara.tv", "ecchi.iwara.tv"} or not matched:
+ raise FollowSourceError("Iwara 请粘贴作者主页,例如 https://www.iwara.tv/profile/username")
+ ref = matched.group(1).casefold()
+ return ParsedSource(provider, ref, f"https://www.iwara.tv/profile/{ref}", ref)
+
+ @classmethod
+ def profile_handle(cls, ref):
+ value = str(ref or "").strip()
+ return value if cls._HANDLE_RE.fullmatch(value) else ""
+
+ def _headers(self):
+ headers = {**super()._headers(), "Accept": "application/json",
+ "Origin": "https://www.iwara.tv", "Referer": "https://www.iwara.tv/",
+ "X-Site": "www.iwara.tv"}
+ if self.credential and self.credential.values.get("access_token"):
+ headers["Authorization"] = "Bearer " + self.credential.values["access_token"]
+ return headers
+
+ def _check_status(self, response):
+ if response.status in (401, 403):
+ raise FollowSourceError(
+ f"Iwara 拒绝访问(HTTP {response.status}):登录凭据可能已过期,"
+ "或站点要求浏览器验证,请核对来源与订阅设置")
+ if response.status == 404:
+ raise FollowSourceError("Iwara 作者或视频未取得:可能已删除或需要有访问权的账号")
+ super()._check_status(response)
+
+ def _object(self, response):
+ payload = self.parse_json(response)
+ if not isinstance(payload, dict):
+ raise FollowSourceError("Iwara 接口响应格式不符")
+ if payload.get("message"):
+ code = str(payload["message"])
+ if code in {"errors.privateVideo", "errors.notFound", "errors.unauthorized"}:
+ raise FollowSourceError("Iwara 内容未取得:请确认账号有访问权并更新登录凭据")
+ raise FollowSourceError("Iwara 接口拒绝请求,请在来源页面核对内容状态")
+ return payload
+
+ def fetch(self, ref, *, etag=None, last_modified=None, page=0):
+ handle = self.profile_handle(ref).casefold()
+ if not handle:
+ raise FollowSourceError("Iwara 的 ref 必须是作者用户名")
+ if int(page) < 0:
+ raise FollowSourceError("Iwara 页码不能小于 0")
+ profile = self.fetch_json(f"{self.API_ROOT}/profile/{handle}")
+ if not isinstance(profile, dict) or not isinstance(profile.get("user"), dict):
+ raise FollowSourceError("Iwara 作者资料格式不符")
+ user = profile["user"]
+ user_id = str(user.get("id") or "")
+ if not re.fullmatch(r"[A-Za-z0-9-]{1,80}", user_id):
+ raise FollowSourceError("Iwara 作者 ID 未取得")
+ url = self.API_ROOT + "/videos?" + urllib.parse.urlencode({
+ "user": user_id, "page": int(page), "limit": self.PAGE_SIZE,
+ "sort": "date", "rating": "all"})
+ common, response = self._request(url, ref=handle, page=page,
+ etag=etag, last_modified=last_modified)
+ if response is None:
+ return SourceFetch(not_modified=True, **common)
+ payload = self._object(response)
+ rows = payload.get("results")
+ if not isinstance(rows, list):
+ raise FollowSourceError("Iwara 视频列表格式不符")
+ if page and not rows:
+ raise FollowHistoryEnd("没有更多历史内容")
+ candidates = []
+ seen = set()
+ for row in rows[:self.max_items]:
+ candidate = self._candidate(row, user, handle)
+ if candidate.external_id in seen:
+ continue
+ seen.add(candidate.external_id)
+ candidates.append(candidate)
+ return SourceFetch(candidates=tuple(candidates), raw_body=response.body, **common)
+
+ def _candidate(self, row, user, handle):
+ """把列表字段映射为稳定 DTO,不保存短效文件地址。"""
+ if not isinstance(row, dict) or not self._VIDEO_ID_RE.fullmatch(str(row.get("id") or "")):
+ raise FollowSourceError("Iwara 视频条目缺少有效 ID")
+ video_id = row["id"]
+ file = row.get("file") if isinstance(row.get("file"), dict) else {}
+ file_id = str(file.get("id") or "")
+ thumbnail = None
+ if re.fullmatch(r"[A-Za-z0-9-]{1,80}", file_id):
+ thumbnail = f"https://files.iwara.tv/image/thumbnail/{file_id}/thumbnail-00.jpg"
+ url = f"https://www.iwara.tv/video/{video_id}"
+ tags = row.get("tags") if isinstance(row.get("tags"), list) else []
+ extra = {"media_kind": "video", "tags": [
+ tag["id"] for tag in tags
+ if isinstance(tag, dict) and isinstance(tag.get("id"), str)]}
+ if row.get("embedUrl"):
+ extra["media_error"] = "Iwara 此条目使用外部播放器,请打开来源页面观看"
+ return FollowCandidate(
+ provider=self.provider, external_id=video_id,
+ title=plain_text(str(row.get("title") or "")) or video_id,
+ url=url, media_url=None if row.get("embedUrl") else url,
+ thumb_url=thumbnail, published_at=_iso_from_text(row.get("createdAt")),
+ duration=self._duration(file.get("duration")),
+ author=plain_text(str(user.get("name") or handle)),
+ summary=plain_text(str(row.get("body") or "")) or None, extra=extra)
+
+ @staticmethod
+ def _duration(value):
+ try:
+ seconds = float(value)
+ return seconds if 0 < seconds < 10 ** 7 else None
+ except (TypeError, ValueError, OverflowError):
+ return None
+
+ @staticmethod
+ def media_url_allowed(url):
+ try:
+ parsed = urllib.parse.urlsplit(url)
+ return (parsed.scheme == "https" and parsed.port in {None, 443}
+ and not parsed.username and not parsed.password
+ and (parsed.hostname == "iwara.tv"
+ or str(parsed.hostname or "").endswith(".iwara.tv")))
+ except ValueError:
+ return False
+
+ def playback_formats(self, video_id):
+ """播放时取得短效格式清单;凭据只发送到固定 API。"""
+ if not self._VIDEO_ID_RE.fullmatch(str(video_id or "")):
+ raise FollowSourceError("Iwara 视频 ID 无效")
+ response = self._get(f"{self.API_ROOT}/video/{video_id}")
+ self._check_status(response)
+ detail = self._object(response)
+ file_url = str(detail.get("fileUrl") or "")
+ if not file_url:
+ raise FollowSourceError("Iwara 正片地址未取得,请打开来源页面核对访问权")
+ if not self.media_url_allowed(file_url):
+ raise FollowSourceError("Iwara 返回了不受信任的文件接口地址")
+ parsed = urllib.parse.urlsplit(file_url)
+ expires = urllib.parse.parse_qs(parsed.query).get("expires", [""])[0]
+ if not expires.isdecimal():
+ raise FollowSourceError("Iwara 文件接口缺少有效期限")
+ # yt-dlp IwaraIE 的公开文件接口签名,固定 revision 见 docs/REUSE.md。
+ key = "_".join((parsed.path.rstrip("/").rsplit("/", 1)[-1], expires,
+ "mSvL05GfEmeEmsEYfGCnVpEjYgTJraJN"))
+ response = self._get(file_url, connector_headers=False,
+ headers={"X-Version": hashlib.sha1(key.encode()).hexdigest()})
+ self._check_status(response)
+ rows = self.parse_json(response)
+ if not isinstance(rows, list):
+ raise FollowSourceError("Iwara 播放格式清单不符")
+ formats = []
+ file = detail.get("file") if isinstance(detail.get("file"), dict) else {}
+ source_height = file.get("height")
+ for row in rows:
+ if not isinstance(row, dict) or not isinstance(row.get("src"), dict):
+ continue
+ value = str(row["src"].get("view") or row["src"].get("download") or "")
+ value = "https:" + value if value.startswith("//") else value
+ if not self.media_url_allowed(value):
+ continue
+ name = str(row.get("name") or "")
+ level = int(name) if name.isdecimal() else (
+ int(source_height) if name == "Source" and str(source_height).isdecimal() else 0)
+ formats.append((level, value, name == "Source"))
+ if not formats:
+ raise FollowSourceError("Iwara 正片格式未取得")
+ return tuple((level, value) for level, value, _ in sorted(
+ formats, key=lambda entry: (entry[2], entry[0]), reverse=True))
+
+
class KemonoConnector(_BaseConnector):
"""kemono.cr / coomer.st / pawchive.pw 共用同一套公开 JSON API。
@@ -2046,8 +2223,15 @@ _PROFILE_PATH_STOPWORDS = frozenset({
_PROFILE_HANDLE_RE = re.compile(r"^[A-Za-z0-9._-]{2,64}$")
_PIXIV_USER_RE = re.compile(r"(?:^|/)users/(\d{1,20})(?:$|/)")
_XENFORO_MEMBER_RE = re.compile(r"^members/([^/.]+)\.(\d{1,20})/?$")
-_BSKY_PROFILE_RE = re.compile(r"^profile/([^/]+)$")
+_BSKY_PROFILE_RE = re.compile(r"^profile/([A-Za-z0-9._-]{2,64})$")
_FANTIA_FANCLUB_RE = re.compile(r"^fanclubs/(\d{1,20})(?:$|/)")
+_PROFILE_PATH_IDENTITIES = {
+ "pixiv.net": ("pixiv", _PIXIV_USER_RE),
+ "bsky.app": ("bsky", _BSKY_PROFILE_RE),
+ "fantia.jp": ("fantia", _FANTIA_FANCLUB_RE),
+ "iwara.tv": ("iwara", re.compile(r"^profile/([A-Za-z0-9_-]{1,80})(?:/videos)?$")),
+ "ecchi.iwara.tv": ("iwara", re.compile(r"^profile/([A-Za-z0-9_-]{1,80})(?:/videos)?$")),
+}
def profile_link_identity(url: str, *, forum_host: str = "") -> tuple[str, str] | None:
@@ -2073,19 +2257,14 @@ def profile_link_identity(url: str, *, forum_host: str = "") -> tuple[str, str]
if host == "fanbox.cc" and first.startswith("@"):
handle = first[1:]
return ("fanbox", handle) if _PROFILE_HANDLE_RE.match(handle) else None
- if host == "pixiv.net":
- matched = _PIXIV_USER_RE.search(path)
- return ("pixiv", matched.group(1)) if matched else None
+ identity = _PROFILE_PATH_IDENTITIES.get(host)
+ if identity:
+ service, pattern = identity
+ matched = pattern.search(path)
+ return (service, canonical_source_ref(service, matched.group(1))) if matched else None
if host.endswith(".itch.io"):
handle = host.removesuffix(".itch.io")
return ("itchio", handle) if _PROFILE_HANDLE_RE.match(handle) else None
- if host == "bsky.app":
- matched = _BSKY_PROFILE_RE.match(path)
- handle = matched.group(1) if matched else ""
- return ("bsky", handle) if _PROFILE_HANDLE_RE.match(handle) else None
- if host == "fantia.jp":
- matched = _FANTIA_FANCLUB_RE.match(path)
- return ("fantia", matched.group(1)) if matched else None
if forum_host and host == forum_host.lower().removeprefix("www."):
matched = _XENFORO_MEMBER_RE.match(path)
return (forum_host.split(".")[0], matched.group(1)) if matched else None
@@ -3057,7 +3236,7 @@ def parse_source_url(raw_url: str) -> ParsedSource:
_CONNECTOR_CLASSES: tuple[type, ...] = (
FanboxConnector, PatreonConnector, SubscribeStarConnector, KemonoConnector,
Rule34VideoConnector, Rule34XxxConnector, Rule34PahealConnector,
- F95ZoneConnector, SimpCityConnector,
+ F95ZoneConnector, SimpCityConnector, IwaraConnector,
)
CONNECTORS: dict[str, type] = {
diff --git a/src/peach/follow_stream.py b/src/peach/follow_stream.py
index dba34f071..cc78229ca 100644
--- a/src/peach/follow_stream.py
+++ b/src/peach/follow_stream.py
@@ -1,8 +1,4 @@
-"""Proxy targets for playing a follow candidate inside Peach.
-
-The browser receives only a Peach URL. Upstream URLs stay in the ledger/source layer, and
-Rule34Video's expiring signed URL is resolved only when the user explicitly presses play.
-"""
+"""关注媒体的播放代理;Rule34Video 与 Iwara 的短效地址按需解析。"""
from __future__ import annotations
import html
@@ -17,10 +13,11 @@ from typing import Callable, Mapping
import httpx
from . import follow_providers
-from .follow_sources import USER_AGENT, archive_file_url, f95_attachment_media_items
+from .follow_sources import USER_AGENT, IwaraConnector, archive_file_url, f95_attachment_media_items
+from .follow import FollowSourceError
from .follow_secrets import Credential
from .follow_store import FollowItemRow
-from .http import HttpRequest, HttpResponse, HttpTransport, public_https_url, resolves_publicly
+from .http import HttpRequest, HttpResponse, HttpTransport, HttpxTransport, public_https_url, resolves_publicly
LOGGER = logging.getLogger(__name__)
@@ -41,8 +38,8 @@ class ResolvedFollowMedia:
url: str
referer: str | None = None
headers: Mapping[str, str] | None = None
- #: (高度, URL) 从高到低。只有 rule34video 给多档;其余来源留空,
- #: 播放器据此只显示「原画」。
+ #: (高度, URL) 按播放优先级排列;Iwara 的 Source 原画可以没有已知高度。
+ #: 没有多档的来源留空,播放器据此只显示「原画」。
qualities: tuple[tuple[int, str], ...] = ()
#: 这条媒体允许落到哪些主机后缀。解析时已经按它校验过 `url`,代理层跟重定向
#: 时还要再按它校验每一跳:`headers` 里可能带 Cookie / Bearer token,一旦上游
@@ -114,7 +111,7 @@ def proxyable(provider: str, media_url: str | None) -> bool:
class FollowMediaResolver:
"""Resolve stable ledger candidates into short-lived upstream playback URLs."""
- #: 缓存里最多留多少条解析结果。只有 rule34video 会进这个缓存,一次列表几百条,
+ #: 缓存里最多留多少条解析结果。Rule34Video 与 Iwara 的短效地址共用此缓存,
#: 用户挨个点开就会挨个留下一条;只按 TTL 过期的话,进程活多久它就长多久。
#: 上限按「一屏反复点开也够用」定,超了先丢已过期的,再丢最早写进来的。
MAX_CACHE_ENTRIES = 256
@@ -133,6 +130,8 @@ class FollowMediaResolver:
max_cache_entries: int | None = None,
clock: Callable[[], float] = time.monotonic):
self.transport = transport
+ self._iwara_transport = (IwaraConnector().transport
+ if isinstance(transport, HttpxTransport) else transport)
self.ttl = ttl
self.timeout = timeout
self.max_bytes = max_bytes
@@ -150,8 +149,13 @@ class FollowMediaResolver:
with self._lock:
return max(0.0, self._cooldowns.get(provider, 0.0) - self.clock())
+ def close(self) -> None:
+ """关闭解析器自己持有的 Iwara 浏览器传输。"""
+ if self._iwara_transport is not self.transport:
+ self._iwara_transport.close()
+
def fetch(self, provider: str, request: HttpRequest, timeout: float,
- max_bytes: int) -> HttpResponse:
+ max_bytes: int, *, transport: HttpTransport | None = None) -> HttpResponse:
"""经来源冷却闸门发一次请求;来源回 403 / 429 就让整站进入冷却。
播放解析与卡面 poster 共用这一道闸:同一个站,一边被限流、另一边还在请求,
@@ -160,7 +164,7 @@ class FollowMediaResolver:
remaining = self.cooling(provider)
if remaining:
raise FollowMediaUnavailable(f"{provider} 限流冷却中,{int(remaining) + 1} 秒后再试")
- response = self.transport(request, timeout, max_bytes)
+ response = (transport or self.transport)(request, timeout, max_bytes)
if response.status in self.REFUSED_STATUSES:
headers = {key.lower(): value for key, value in response.headers.items()}
try:
@@ -168,8 +172,8 @@ class FollowMediaResolver:
except ValueError:
retry_after = None
seconds = self.pause(provider, retry_after)
- LOGGER.warning("%s refused %s with HTTP %s; pausing upstream requests for %ds",
- provider, request.url, response.status, seconds)
+ LOGGER.warning("%s refused upstream request with HTTP %s; pausing for %ds",
+ provider, response.status, seconds)
return response
def pause(self, provider: str, retry_after: float | None = None) -> float:
@@ -205,7 +209,7 @@ class FollowMediaResolver:
missed = self._misses.get(item_id)
if missed is not None:
if missed > now:
- raise FollowMediaUnavailable("Rule34Video 详情页刚刚未取得,稍后再试")
+ raise FollowMediaUnavailable("来源媒体刚刚未取得,稍后再试")
del self._misses[item_id]
return None
@@ -288,10 +292,17 @@ class FollowMediaResolver:
def resolve(self, item: FollowItemRow, media_index: int | None = None,
height: int | None = None) -> ResolvedFollowMedia:
- """解析可播地址。`height` 指定清晰度,只有 rule34video 有多档可选;
+ """解析可播地址。`height` 指定 Rule34Video 或 Iwara 的清晰度;
要的那档不存在就退回默认(最高档),不报错——签名 URL 会过期,
为一次选档失败中断播放不值得。"""
media_items = item.metadata.get("media_items")
+ if item.provider == "iwara":
+ with self._item_lock(item.id):
+ with self._lock:
+ cached = self._cached(item.id)
+ if cached:
+ return _pick_quality(cached, height)
+ return _pick_quality(self._fetch_iwara(item), height)
if (not isinstance(media_items, list) or not media_items) \
and item.provider == "f95zone":
media_items = f95_attachment_media_items(item.metadata)
@@ -325,6 +336,30 @@ class FollowMediaResolver:
return _pick_quality(cached, height)
return _pick_quality(self._fetch_rule34video(item), height)
+ def _fetch_iwara(self, item: FollowItemRow) -> ResolvedFollowMedia:
+ """Iwara 的短效地址只进播放缓存,访问令牌只交给固定 API。"""
+ if not item.url or not _allowed("iwara", item.url):
+ raise FollowMediaUnavailable("Iwara 视频地址不可用")
+ loader = getattr(self, "_credential_loader", None)
+ credential = loader("iwara") if loader else None
+
+ def transport(request, timeout, max_bytes):
+ return self.fetch("iwara", request, timeout, max_bytes,
+ transport=self._iwara_transport)
+
+ connector = IwaraConnector(transport=transport, credential=credential,
+ timeout=self.timeout, max_bytes=self.max_bytes)
+ try:
+ qualities = connector.playback_formats(item.external_id)
+ except FollowSourceError as exc:
+ self._remember_miss(item.id)
+ raise FollowMediaUnavailable(str(exc)) from exc
+ resolved = ResolvedFollowMedia(qualities[0][1], item.url, qualities=qualities,
+ allowed_hosts=("iwara.tv",))
+ with self._lock:
+ self._remember(item.id, resolved)
+ return resolved
+
def _fetch_rule34video(self, item: FollowItemRow) -> ResolvedFollowMedia:
"""抓一次详情页,解析出各档正片地址并写进缓存;没取到记一次失败。"""
response = self.fetch(
diff --git a/tests/test_follow_providers.py b/tests/test_follow_providers.py
index c7e4af218..d5c562769 100644
--- a/tests/test_follow_providers.py
+++ b/tests/test_follow_providers.py
@@ -55,9 +55,11 @@ class ProviderRegistryTests(unittest.TestCase):
values = list(PROVIDER_PRIORITY.values())
self.assertEqual(len(values), len(set(values)), "优先级重复会让选主条目结果不确定")
- def test_official_identity_providers_do_not_proxy_media(self):
- """官方三家走各自详情接口取媒体,不经本地代理,所以不该有主机白名单。"""
- for key in _OFFICIAL_IDENTITY_PROVIDERS:
+ def test_official_identity_preserves_each_sources_media_boundary(self):
+ """官方作者身份与媒体传输独立;Iwara 的签名媒体由本机代理。"""
+ self.assertIn("iwara", _OFFICIAL_IDENTITY_PROVIDERS)
+ self.assertEqual(_PROVIDER_HOSTS["iwara"], ("iwara.tv",))
+ for key in _OFFICIAL_IDENTITY_PROVIDERS - {"iwara"}:
self.assertEqual(follow_providers.PROVIDERS[key].hosts, ())
self.assertNotIn(key, _PROVIDER_HOSTS)
diff --git a/tests/test_follow_sources.py b/tests/test_follow_sources.py
index ce574405c..4a22be8f1 100644
--- a/tests/test_follow_sources.py
+++ b/tests/test_follow_sources.py
@@ -5,6 +5,7 @@
thread 50685),不是凭记忆构造的形状。测试本身不联网:transport 全部注入。
"""
import json
+import hashlib
import os
import tempfile
import unittest
@@ -17,7 +18,7 @@ from peach import follow_sources
from peach.follow import FollowHistoryEnd, FollowSourceError, FollowSourceRateLimited
from peach.follow_secrets import Credential, CredentialError, CredentialStore
from peach.follow_sources import (
- USER_AGENT, F95ZoneConnector, FanboxConnector, KemonoConnector,
+ USER_AGENT, F95ZoneConnector, FanboxConnector, IwaraConnector, KemonoConnector,
PatreonConnector, Rule34PahealConnector, Rule34VideoConnector, Rule34XxxConnector,
SimpCityConnector, SubscribeStarConnector, build_connector,
_iso_from_relative, origin_group_key, parse_source_url,
@@ -2062,6 +2063,121 @@ class ParseSourceUrlTests(unittest.TestCase):
parse_source_url(url)
+class IwaraConnectorTests(unittest.TestCase):
+ PROFILE = {"user": {"id": "user-id", "name": "Test Creator", "username": "creator"}}
+ VIDEO = {"id": "Video123", "title": "Test animation", "body": "Description",
+ "createdAt": "2026-10-01T12:00:00Z", "tags": [{"id": "animation"}],
+ "file": {"id": "file-id", "height": 1080, "duration": 20.5}}
+
+ def connector(self, payloads, **kwargs):
+ requests = []
+ queue = list(payloads)
+
+ def transport(request, timeout, max_bytes):
+ requests.append(request)
+ payload = queue.pop(0)
+ return payload if isinstance(payload, HttpResponse) else HttpResponse(
+ 200, {"ETag": "list-version"}, json.dumps(payload).encode())
+
+ return IwaraConnector(transport=transport, **kwargs), requests
+
+ def test_author_urls_and_refs_have_one_identity(self):
+ for url in ("https://www.iwara.tv/profile/Creator", "iwara.tv/profile/creator/videos",
+ "https://ecchi.iwara.tv/profile/creator/"):
+ parsed = parse_source_url(url)
+ self.assertEqual((parsed.provider, parsed.ref), ("iwara", "creator"))
+ self.assertEqual(parsed.url, "https://www.iwara.tv/profile/creator")
+ self.assertEqual(follow_sources.canonical_source_ref("iwara", "Creator"), "creator")
+ self.assertEqual(follow_sources.profile_link_identity(
+ "https://www.iwara.tv/profile/Creator/videos"), ("iwara", "creator"))
+ for path in ("/video/abc", "/videos", "/profile/creator/images", "/profile/a/b"):
+ with self.assertRaises(FollowSourceError):
+ parse_source_url("https://www.iwara.tv" + path)
+
+ def test_list_projects_stable_media_metadata_and_no_short_lived_addresses(self):
+ connector, requests = self.connector([self.PROFILE, {"results": [self.VIDEO]}])
+ result = connector.fetch("Creator", etag="old-version")
+ item = result.candidates[0]
+ self.assertEqual(result.ref, "creator")
+ self.assertEqual(item.media_url, "https://www.iwara.tv/video/Video123")
+ self.assertEqual(item.thumb_url,
+ "https://files.iwara.tv/image/thumbnail/file-id/thumbnail-00.jpg")
+ self.assertEqual((item.author, item.duration), ("Test Creator", 20.5))
+ self.assertEqual(item.published_at, "2026-10-01T12:00:00Z")
+ self.assertEqual(item.extra, {"media_kind": "video", "tags": ["animation"]})
+ self.assertNotIn("If-None-Match", requests[0].headers)
+ self.assertEqual(requests[1].headers["If-None-Match"], "old-version")
+ query = urllib.parse.parse_qs(urllib.parse.urlsplit(requests[1].url).query)
+ self.assertEqual(query, {"user": ["user-id"], "page": ["0"], "limit": ["32"],
+ "sort": ["date"], "rating": ["all"]})
+
+ def test_paging_and_history_end_use_upstream_results(self):
+ connector, requests = self.connector([self.PROFILE, {"results": []}])
+ with self.assertRaises(FollowHistoryEnd):
+ connector.fetch("creator", page=2, etag="first-page")
+ self.assertIn("page=2", requests[1].url)
+ self.assertNotIn("If-None-Match", requests[1].headers)
+ connector, _ = self.connector([self.PROFILE, {"results": []}])
+ self.assertEqual(connector.fetch("creator").candidates, ())
+
+ def test_unchanged_list_keeps_the_conditional_receipt(self):
+ connector, _ = self.connector([self.PROFILE, HttpResponse(304, {"ETag": "same"}, b"")])
+ result = connector.fetch("creator", etag="same")
+ self.assertTrue(result.not_modified)
+ self.assertEqual(result.etag, "same")
+
+ def test_external_player_has_an_explicit_state(self):
+ video = {**self.VIDEO, "embedUrl": "https://example.org/video/123"}
+ connector, _ = self.connector([self.PROFILE, {"results": [video]}])
+ item = connector.fetch("creator").candidates[0]
+ self.assertIsNone(item.media_url)
+ self.assertIn("外部播放器", item.extra["media_error"])
+
+ def test_invalid_list_and_access_failures_are_not_empty_successes(self):
+ for payload in ({}, {"results": {}}, {"results": [{"title": "missing id"}]},
+ {"message": "errors.privateVideo"}):
+ connector, _ = self.connector([self.PROFILE, payload])
+ with self.assertRaises(FollowSourceError):
+ connector.fetch("creator")
+ for status in (401, 403, 404, 429):
+ connector, _ = self.connector([HttpResponse(status, {}, b"")])
+ with self.assertRaises(FollowSourceError):
+ connector.fetch("creator")
+
+ def test_playback_signs_the_file_request_and_keeps_credentials_on_the_api(self):
+ file_url = "https://files.iwara.tv/file/file-id?expires=2000000000"
+ formats = [{"name": "540", "src": {"view": "//hime.iwara.tv/540.mp4"}},
+ {"name": "Source", "src": {"download": "//mikoto.iwara.tv/source.mp4"}},
+ {"name": "1080", "src": {"view": "https://iwara.tv.evil.org/stolen.mp4"}}]
+ connector, requests = self.connector(
+ [{**self.VIDEO, "fileUrl": file_url}, formats],
+ credential=Credential("iwara", {"access_token": "secret-token"}))
+ self.assertEqual(connector.playback_formats("Video123"), (
+ (1080, "https://mikoto.iwara.tv/source.mp4"), (540, "https://hime.iwara.tv/540.mp4")))
+ self.assertEqual(requests[0].headers["Authorization"], "Bearer secret-token")
+ self.assertNotIn("Authorization", requests[1].headers)
+ self.assertEqual(requests[1].headers["X-Version"], hashlib.sha1(
+ b"file-id_2000000000_mSvL05GfEmeEmsEYfGCnVpEjYgTJraJN").hexdigest())
+ self.assertNotIn("secret-token", requests[0].url)
+
+ def test_playback_refuses_untrusted_or_incomplete_file_interfaces(self):
+ for url in ("http://files.iwara.tv/file/x?expires=1", "https://evil.org/file/x?expires=1",
+ "https://files.iwara.tv/file/x", "https://user:[email protected]/file/x?expires=1",
+ "https://files.iwara.tv:8443/file/x?expires=1"):
+ connector, requests = self.connector([{**self.VIDEO, "fileUrl": url}])
+ with self.assertRaises(FollowSourceError):
+ connector.playback_formats("Video123")
+ self.assertEqual(len(requests), 1)
+
+ def test_source_format_is_preferred_when_original_dimensions_are_unknown(self):
+ connector, _ = self.connector([
+ {"fileUrl": "https://files.iwara.tv/file/id?expires=2000000000", "file": {}},
+ [{"name": "540", "src": {"view": "//hime.iwara.tv/540.mp4"}},
+ {"name": "Source", "src": {"view": "//mikoto.iwara.tv/original.mp4"}}]])
+ self.assertEqual(connector.playback_formats("Video123")[0],
+ (0, "https://mikoto.iwara.tv/original.mp4"))
+
+
class BuildConnectorTests(unittest.TestCase):
def test_registry_maps_the_three_kemono_hosts_to_distinct_providers(self):
for provider in ("kemono", "coomer", "pawchive"):
diff --git a/tests/test_follow_store.py b/tests/test_follow_store.py
index e0941543b..e86913574 100644
--- a/tests/test_follow_store.py
+++ b/tests/test_follow_store.py
@@ -1087,6 +1087,13 @@ class AuthorIdentityTests(_StoreCase):
"kemono", "fanbox/30917150",
(_candidate("1", "a", provider="kemono", author="Initiala"),)))
+ def test_iwara_author_profile_provides_a_trusted_handle_alias(self):
+ learned = self.store.learn_official_author_alias(
+ "iwara", "user_creator",
+ (_candidate("1", "Work", provider="iwara", author="Test Creator"),))
+ self.assertEqual(learned["alias"], "user_creator")
+ self.assertEqual(learned["source"], "official:iwara")
+
class SourceRemovalTests(_StoreCase):
def test_removing_a_source_takes_it_out_of_the_listing(self):
diff --git a/tests/test_follow_stream.py b/tests/test_follow_stream.py
index 135cf7b3d..41ce072bc 100644
--- a/tests/test_follow_stream.py
+++ b/tests/test_follow_stream.py
@@ -1,4 +1,5 @@
import threading
+import json
import unittest
from types import SimpleNamespace
from unittest import mock
@@ -16,6 +17,56 @@ from peach.http import HttpResponse
class FollowMediaResolverTests(unittest.TestCase):
+ def test_iwara_transport_cleanup_preserves_the_callers_transport(self):
+ owned = mock.Mock()
+ upstream = peach_http.HttpxTransport()
+ with mock.patch("peach.follow_stream.IwaraConnector", return_value=SimpleNamespace(transport=owned)):
+ resolver = FollowMediaResolver(upstream)
+ resolver.close()
+ owned.close.assert_called_once_with()
+ upstream.close()
+ injected = mock.Mock()
+ FollowMediaResolver(injected).close()
+ injected.close.assert_not_called()
+
+ def test_iwara_playback_uses_shared_cache_quality_selection_and_no_media_credentials(self):
+ requests = []
+
+ def transport(request, timeout, max_bytes):
+ requests.append(request)
+ payload = ({"fileUrl": "https://files.iwara.tv/file/id?expires=2000000000",
+ "file": {"height": 1080}}
+ if "/video/" in request.url else [
+ {"name": "Source", "src": {"view": "//hime.iwara.tv/1080.mp4"}},
+ {"name": "540", "src": {"view": "//mikoto.iwara.tv/540.mp4"}}])
+ return HttpResponse(200, {}, json.dumps(payload).encode())
+
+ item = SimpleNamespace(id=90, provider="iwara", external_id="Video123",
+ url="https://www.iwara.tv/video/Video123", metadata={},
+ media_url="https://www.iwara.tv/video/Video123")
+ resolver = FollowMediaResolver(transport).with_credential_loader(
+ lambda _: Credential("iwara", {"access_token": "private-token"}))
+ self.assertEqual(resolver.resolve(item).url, "https://hime.iwara.tv/1080.mp4")
+ lower = resolver.resolve(item, height=540)
+ self.assertEqual(lower.url, "https://mikoto.iwara.tv/540.mp4")
+ self.assertEqual(len(requests), 2)
+ self.assertEqual(lower.allowed_hosts, ("iwara.tv",))
+ self.assertIsNone(lower.headers)
+ self.assertNotIn("Authorization", requests[1].headers)
+ self.assertTrue(proxyable("iwara", item.media_url))
+
+ def test_iwara_access_failure_is_cached_and_rate_limit_pauses_the_source(self):
+ transport = mock.Mock(return_value=HttpResponse(429, {"Retry-After": "30"}, b""))
+ resolver = FollowMediaResolver(transport, clock=lambda: 100)
+ item = SimpleNamespace(id=90, provider="iwara", external_id="Video123",
+ url="https://www.iwara.tv/video/Video123", metadata={})
+ with self.assertRaises(FollowMediaUnavailable):
+ resolver.resolve(item)
+ with self.assertRaises(FollowMediaUnavailable):
+ resolver.resolve(item)
+ self.assertEqual(transport.call_count, 1)
+ self.assertEqual(resolver.cooling("iwara"), 30)
+
def test_rule34video_resolves_the_current_signed_url_and_caches_it(self):
requests = []