You're reading this. The skill predicted it. A workflow that turns every post into a calibrated experiment—score, blind-predict, retro, evolve. The future doesn't reward effort, it rewards those who see the pattern first. 1M followers in a month — not luck, system.
Python
feat: xhs-explore adapter (Xiaohongshu perf-data) (#27)
Zongrong(Jasper) Lidefce470d941
diff --git a/.gitignore b/.gitignoreindex 84801f2..48b285d 100644--- a/.gitignore+++ b/.gitignore@@ -37,6 +37,7 @@ content.db# adapter 跑过留下的.auth/+.auth-xhs/.debug/# Editor / IDEdiff --git a/adapters/perf-data/douyin-session/README.md b/adapters/perf-data/douyin-session/README.mdindex a7f2b31..6d38d9b 100644--- a/adapters/perf-data/douyin-session/README.md+++ b/adapters/perf-data/douyin-session/README.md@@ -117,6 +117,6 @@ adapters/perf-data/douyin-session/- `youtube-data-api`(待)— YouTube 用官方 API,不需要 Playwright,更轻- `bilibili-stat`(待)— B 站官方 stat 接口公开,也不需要 Playwright-- `xhs-explore`(待)— 小红书也走 Playwright 路线,可借鉴本 adapter+- `xhs-explore` — 小红书,已实现(同样走 Playwright 被动拦截路线,照搬自本 adapter)如果你做多平台内容,**只装你实际用的 adapter**——不需要全装。diff --git a/adapters/perf-data/xhs-explore/README.md b/adapters/perf-data/xhs-explore/README.mdnew file mode 100644index 0000000..6f60cd8--- /dev/null+++ b/adapters/perf-data/xhs-explore/README.md@@ -0,0 +1,136 @@+# Adapter: xhs-explore(小红书爬取)++被 `/cheat-retro` 在 `state.data_collection=adapter` + `platform=xhs` 时自动调用。++> **来源**:照搬 `douyin-session` adapter 的架构(Playwright 持久化登录态 + 被动拦截 XHR),+> 接口路径与字段参考 NanmiCoder/MediaCrawler 与 ReaJason/xhs。已在真实创作者账号端到端验证(2026-05)。++---++## 这个 adapter 是干嘛的++小红书反爬靠 `x-s`/`x-t`/`x-s-common` 签名 + `xsec_token`——纯 HTTP / requests 拿不到数据。++xhs-explore 用 **Playwright + 持久化 Chromium context** 模拟真实浏览器,**不逆向签名、不伪造请求**,+让页面自己发带签名的请求,我们只被动拦截返回的 JSON:++- 首次扫码登录创作者中心,cookie 存在**你的内容项目根目录** `.auth-xhs/`+- 之后每次抓取复用 cookie+- 抓两路数据:+ 1. **创作者中心 galaxy 接口**(`/api/galaxy/v2/creator/note/user/posted`)— 你**自己**笔记的运营数据:观看/点赞/收藏/评论/分享,**不需要 xsec_token**,最稳。列表每条自带 `xsec_token`,抓自己的笔记无需手动粘带 token 的链接+ 2. **前台 web API**(`/api/sns/web/v1/feed` + `/api/sns/web/v2/comment/page`)— 拿确认字段的点赞/收藏数 + 评论文本++输出写到**你的内容项目** `videos/<...>/report.md`。+调试产物(URL dump / 截图 / galaxy 原始 JSON)写到 `.cheat-cache/xhs-explore-debug/`。++## 字段映射(已校准)++galaxy `note/user/posted` 的真实字段名已用真实账号确认并写死在 `crawler.py` 的 `_normalize_note()`:+观看 `view_count`、点赞 `likes`、收藏 `collected_count`、评论 `comments_count`、分享 `shared_count`、+发布时间 `visible_time`(unix 秒)/ `time`(本地时间串)、单篇 `xsec_token`。++万一小红书改版导致某项显示 0:打开 `videos/<...>/report.md` 末尾的"galaxy 原始字段"JSON,+把新 key 加进 `_normalize_note()` 对应的 `_first(v, ...)` 候选列表即可(多候选兜底仍保留)。++## 安装(一次性)++```bash+# 1. 进你的内容项目根目录+cd ~/Documents/my-channel++# 2. 建虚拟环境+python3 -m venv .venv+source .venv/bin/activate # Windows: .venv\Scripts\activate++# 3. 装 Playwright + Chromium+pip install playwright>=1.44+playwright install chromium++# 4. 首次扫码登录小红书创作者中心+ADAPTER=$(find ~/cheat-on-content -name "xhs-explore" -type d 2>/dev/null | head -1)+python "$ADAPTER/crawler.py" login+# → 弹出 Chromium 窗口,用小红书 App 扫码+# → 登录成功后窗口自动关闭,cookie 存在 当前目录/.auth-xhs/+```++> Windows 提示:adapter 在 `~/cheat-on-content/adapters/perf-data/xhs-explore`(克隆源码处),+> 不在 `~/.claude/skills`(install.sh 只复制 15 个 skill,不复制 adapter)。++## 用法++cheat-retro 自动调用。手动测试:++```bash+cd ~/Documents/my-channel+source .venv/bin/activate++# 列最近笔记(验证登录态没失效)+python "$ADAPTER/review.py" list++# 抓特定笔记+python "$ADAPTER/review.py" note <note_id> [script.txt]++# 输出在 当前目录/videos/<日期>_<标题>/report.md+```++## 怎么拿到 note_id++小红书笔记 URL:+- `https://www.xiaohongshu.com/explore/66f1a2b3c4d5e6f700112233?xsec_token=...` → `note_id = 66f1a2b3c4d5e6f700112233`+- `https://xhslink.com/xxxxx` → 短链,cheat-publish 会 resolve++cheat-publish 登记发布时把 note_id 存到 prediction header,cheat-retro 启动时读这个字段。++## report.md 输出格式++由 `renderer.py` 生成:+- 笔记元信息(标题、发布时间、链接、IP 归属)+- 数据快照(曝光/浏览、点赞、收藏、评论、分享 + 派生比率:赞曝比 / 藏曝比 / 评曝比 / 分曝比 + 涨粉)+- galaxy 原始字段 JSON(debug / 接口改版时核对字段用)+- Top 评论(按赞数排序,含文本 + 赞数 + IP)++## 失败模式(按概率从高到低)++| 症状 | 原因 | 处理 |+|---|---|---|+| 曝光显示 0 但 JSON 有数 | galaxy 字段 key 不在候选列表 | 看 report.md 里 galaxy JSON,把真实 key 加进 `_normalize_note` |+| `ensure_login` 超时 | cookie 过期 | 重跑 `python crawler.py login` |+| 笔记列表为空 | 创作者中心改了 galaxy 接口路径 | 看 `.cheat-cache/xhs-explore-debug/creator_urls.txt`,更新 `GALAXY_*_KEYS` |+| 评论抓不到 | xsec_token 缺失 / 评论被关 | 看 `frontend_urls.txt`;拿不到就降级 manual 粘评论(cheat-retro 会提示) |+| Chromium 崩溃 | 内存不足 | 关其他 Chromium;`playwright install chromium --force` |++**关键现实**:小红书接口 path(`/feed`、`/comment/page`、galaxy)相对稳定,但签名和 xsec_token 机制常变。+本 adapter 用被动拦截规避签名风险——最易抖动处是 **galaxy 创作者接口**和**评论回复**。++## 稳定性等级++★★ — Playwright + 登录态能扛比纯 HTTP 强得多的反爬,但仍受小红书前端改版影响。+评论路径(依赖 xsec_token)比创作者数据路径脆,必要时降级 manual。++## 风险提示++- **冷启动用户慎装**:Playwright + Chromium ~500MB+- **TOS 风险**:用自己的 cookie 抓自己后台数据是个人用途;别滥用、别高频+- **不要把 `.auth-xhs/` 提交到 git**:cookie 含会话凭据,泄露 = 他人能登录你的小红书账号+- `.cheat-cache/xhs-explore-debug/` 也不应提交(含调试截图 / 接口 URL / 原始 JSON)++## 文件清单++```+adapters/perf-data/xhs-explore/+├── README.md # 本文件+├── requirements.txt # playwright>=1.44+├── crawler.py # 抓取核心(galaxy 笔记列表+数据 / 前台 interact+评论)+├── review.py # CLI 入口(login / list / note <note_id>)+├── renderer.py # 把抓回的 JSON 渲染成 report.md+├── paths.py # 项目根 / .auth-xhs / debug 路径解析+└── run.sh # cheat-retro 调用的 wrapper+```++## 与其他 adapter 的关系++- `douyin-session` — 抖音,本 adapter 的架构来源+- `youtube-data-api`(待)— YouTube 官方 API,更轻+- `bilibili-stat`(待)— B 站官方 stat 接口++如果你做多平台内容,**只装你实际用的 adapter**。diff --git a/adapters/perf-data/xhs-explore/crawler.py b/adapters/perf-data/xhs-explore/crawler.pynew file mode 100644index 0000000..e87473c--- /dev/null+++ b/adapters/perf-data/xhs-explore/crawler.py@@ -0,0 +1,426 @@+"""小红书创作者中心 + 前台评论抓取。++登录一次后,Cookie 持久化在 .auth-xhs/,之后直接复用。+一次抓取共享一个 Chromium 会话,稳定性优于每步一个进程。++设计原则(和 douyin-session 一致):+- 不逆向 x-s / x-t 签名、不伪造请求——用登录态浏览器,让页面自己发带签名的请求,+ 我们只被动拦截返回的 JSON。+- 创作者**自己的**笔记数据走 galaxy 接口(不需要 xsec_token),是最稳的主路。+- 评论走前台 web API(需要 xsec_token),让页面自己导航触发带 token 的请求;+ 拿不到就优雅降级(report.md 标 comments_unavailable,cheat-retro 回落到 manual 粘评论)。+"""+from __future__ import annotations++import asyncio+import json+from pathlib import Path+from typing import Any++from playwright.async_api import BrowserContext, Page, Response, async_playwright+from paths import auth_dir, debug_dir++CREATOR_HOME = "https://creator.xiaohongshu.com/new/home"+CREATOR_NOTE_MANAGER = "https://creator.xiaohongshu.com/new/note-manager"+# galaxy 接口路径片段——宽松匹配,接口偶有版本号变化+GALAXY_NOTE_LIST_KEYS = (+ # 松匹配后缀,兼容 /api/galaxy/creator/... 与 /api/galaxy/v2/creator/...(实测是 v2)+ "/creator/note/user/posted",+)+GALAXY_NOTE_STATS_KEYS = (+ "/api/galaxy/creator/data/note_stats",+ "/api/galaxy/creator/data/note_detail",+)+# 前台 web API+FEED_KEY = "/api/sns/web/v1/feed"+COMMENT_KEY = "/api/sns/web/v2/comment/page"+++class Session:+ """单浏览器会话,按顺序跑多步抓取。"""++ def __init__(self, ctx: BrowserContext, pw: Any) -> None:+ self.ctx = ctx+ self.pw = pw++ @classmethod+ async def open(cls, headless: bool = False) -> "Session":+ pw = await async_playwright().start()+ auth_path = auth_dir()+ auth_path.mkdir(parents=True, exist_ok=True)+ ctx = await pw.chromium.launch_persistent_context(+ user_data_dir=str(auth_path),+ headless=headless,+ viewport={"width": 1440, "height": 900},+ args=["--disable-blink-features=AutomationControlled"],+ )+ return cls(ctx, pw)++ async def close(self) -> None:+ try:+ await self.ctx.close()+ finally:+ await self.pw.stop()+++# 创作者中心登录凭证——扫码登 creator.xiaohongshu.com 产生的就是这些(galaxy 主路只需它们)。+# 注意:创作者中心登录 *不* 产生 web_session(那是主站 www 前台 cookie),早期版本只认+# web_session 是个 bug,会导致登录成功却一直检测不到、白等超时。+CREATOR_LOGIN_COOKIES = (+ "access-token-creator.xiaohongshu.com",+ "galaxy_creator_session_id",+ "customer-sso-sid",+)+# 主站前台凭证——feed / 评论 web API 需要;只在登录后访问过 www 才会下发。+WEB_LOGIN_COOKIE = "web_session"+++async def _cookie_map(ctx: BrowserContext, host: str) -> dict[str, str]:+ try:+ return {c["name"]: c.get("value", "") for c in await ctx.cookies(host)}+ except Exception:+ return {}+++async def _creator_logged_in(ctx: BrowserContext) -> bool:+ names = await _cookie_map(ctx, "https://creator.xiaohongshu.com")+ return any(names.get(n) for n in CREATOR_LOGIN_COOKIES)+++async def _has_web_session(ctx: BrowserContext) -> bool:+ for host in ("https://www.xiaohongshu.com", "https://creator.xiaohongshu.com"):+ if (await _cookie_map(ctx, host)).get(WEB_LOGIN_COOKIE):+ return True+ return False+++async def _acquire_web_session(page: Page) -> None:+ """创作者中心登录后,访问主站让 SSO 下发 web_session(前台 feed/评论需要)。best-effort。"""+ try:+ await page.goto("https://www.xiaohongshu.com/explore",+ wait_until="domcontentloaded", timeout=30000)+ await asyncio.sleep(3)+ if await _has_web_session(page.context):+ print("[登录] ✓ 已获取主站 web_session(前台评论/互动可用)")+ else:+ print("[登录] 注意:未拿到 web_session,前台评论可能要 manual;galaxy 主路不受影响。")+ except Exception:+ pass+++async def ensure_login(timeout_s: int = 300) -> bool:+ """扫码登录创作者中心;检测到创作者登录态后顺便换取 web_session,然后自动关闭。"""+ sess = await Session.open()+ try:+ page = await sess.ctx.new_page()+ await page.goto(CREATOR_HOME)+ print(f"[登录] 在弹出的 Chromium 窗口里扫码登录小红书创作者中心。最多等 {timeout_s} 秒……")+ for i in range(timeout_s):+ try:+ if await _creator_logged_in(sess.ctx) and "login" not in page.url:+ print(f"[登录] ✓ 创作者中心登录态已确认(用时 {i}s)")+ await _acquire_web_session(page)+ await asyncio.sleep(1)+ return True+ except Exception:+ pass+ await asyncio.sleep(1)+ print("[登录] 超时未检测到登录态。")+ return False+ finally:+ await sess.close()+++async def fetch_recent_notes(sess: Session, limit: int = 50) -> list[dict]:+ """创作者中心笔记管理页 → 拦截 galaxy 笔记列表 + 单篇运营数据(含曝光/浏览)。"""+ captured: list[dict] = []+ all_urls: list[str] = []++ page = await sess.ctx.new_page()++ async def on_response(resp: Response) -> None:+ all_urls.append(resp.url)+ if any(k in resp.url for k in GALAXY_NOTE_LIST_KEYS + GALAXY_NOTE_STATS_KEYS):+ try:+ data = await resp.json()+ captured.append({"url": resp.url, "data": data})+ if len(captured) == 1 and isinstance(data, dict):+ print(f"[诊断] galaxy 接口 keys: {list(data.keys())[:8]}")+ except Exception:+ pass++ page.on("response", on_response)+ try:+ await page.goto(CREATOR_NOTE_MANAGER, wait_until="domcontentloaded", timeout=60000)+ await asyncio.sleep(8)+ for _ in range(4):+ await page.evaluate("window.scrollBy(0, 1200)")+ await asyncio.sleep(1.5)+ notes = _parse_note_list(captured, limit)+ if not notes:+ _dump(all_urls, "creator_urls.txt", captured, "creator_captured.json")+ print(f"[诊断] 笔记列表为空,{len(all_urls)} 个请求已 dump 到 .cheat-cache/xhs-explore-debug/。")+ return notes+ finally:+ await page.close()+++def _iter_candidates(data: Any) -> list:+ """从任意 galaxy response 里挖出"笔记数组"。结构多变,宽松找。"""+ out: list = []+ if isinstance(data, dict):+ # 常见包装:{data: {...}} / {data: [...]} / 顶层直接 list 字段+ inner = data.get("data") if isinstance(data.get("data"), (dict, list)) else data+ targets = [inner] if not isinstance(inner, list) else []+ if isinstance(inner, list):+ out.extend(inner)+ for t in targets:+ if isinstance(t, dict):+ for key in ("notes", "note_list", "list", "items", "note_stats", "result"):+ val = t.get(key)+ if isinstance(val, list):+ out.extend(val)+ return out+++def _parse_note_list(captured: list[dict], limit: int) -> list[dict]:+ by_id: dict[str, dict] = {}+ for item in captured:+ for raw in _iter_candidates(item["data"]):+ if not isinstance(raw, dict):+ continue+ note = _normalize_note(raw)+ if not note["note_id"]:+ continue+ # 同一 note 可能在 list 接口和 stats 接口各出现一次——合并,非空字段优先+ existing = by_id.get(note["note_id"])+ if existing:+ for k, v in note.items():+ if v and not existing.get(k):+ existing[k] = v+ else:+ by_id[note["note_id"]] = note+ return list(by_id.values())[:limit]+++def _first(d: dict, *keys: str) -> Any:+ for k in keys:+ v = d.get(k)+ if v not in (None, "", 0):+ return v+ # 再扫一遍允许 0(区分"字段不存在"和"值为 0")+ for k in keys:+ if k in d:+ return d[k]+ return 0+++def _normalize_note(v: dict) -> dict:+ note_id = v.get("note_id") or v.get("id") or v.get("noteId") or ""+ # 字段名已用真实返回校准(2026-05 /api/galaxy/v2/creator/note/user/posted):+ # 观看 view_count | 点赞 likes | 收藏 collected_count | 评论 comments_count+ # 分享 shared_count | 发布时间 visible_time(unix秒) | 单篇 token xsec_token+ # 确认名放首位,旧候选留作兜底以防接口再次改版。+ return {+ "note_id": str(note_id),+ "title": v.get("display_title") or v.get("title") or v.get("desc") or v.get("name") or "",+ "create_time": _to_int(_first(v, "visible_time", "create_time", "post_time", "publish_time")),+ "view_count": _to_int(_first(v, "view_count", "view", "imp", "impression", "read_count", "pv")),+ "like_count": _to_int(_first(v, "likes", "like_count", "liked_count", "like")),+ "collect_count": _to_int(_first(v, "collected_count", "collect_count", "collect", "fav_count")),+ "comment_count": _to_int(_first(v, "comments_count", "comment_count", "comment", "cmt_count")),+ "share_count": _to_int(_first(v, "shared_count", "share_count", "share")),+ "fans_inc": _to_int(_first(v, "fans", "fans_inc", "new_fans", "follow_count")),+ "post_time_str": v.get("time") or "", # galaxy 自带本地时间串,比 epoch 省去时区换算+ "xsec_token": v.get("xsec_token") or "",+ "note_type": v.get("type") or "",+ "raw": v,+ }+++def _to_int(x: Any) -> int:+ try:+ if isinstance(x, str):+ x = x.replace(",", "").strip()+ return int(float(x))+ except (ValueError, TypeError):+ return 0+++async def fetch_note_frontend(sess: Session, note_id: str, note_url: str | None = None) -> dict:+ """打开前台笔记页 → 拦截 feed(interact_info 确认字段)+ comment/page。++ 前台需要 xsec_token + 登录态(web_session)。+ - 若传入 note_url(含 xsec_token,如从创作者后台复制的 ?xsec_token=...&xsec_source=pc_creatormng)+ → 直接用它导航,最稳。+ - 否则退回裸 explore URL(仅对已登录账号访问自己笔记可能可行)。+ token 缺失 / 未登录 → dump 并降级(评论留给 manual)。+ """+ feed: dict = {}+ comments: list[dict] = []+ all_urls: list[str] = []++ page = await sess.ctx.new_page()++ async def on_response(resp: Response) -> None:+ all_urls.append(resp.url)+ if FEED_KEY in resp.url:+ try:+ data = await resp.json()+ ii = _extract_interact(data)+ if ii:+ feed.update(ii)+ except Exception:+ pass+ elif COMMENT_KEY in resp.url:+ try:+ data = await resp.json()+ for c in _extract_comments(data):+ comments.append(c)+ except Exception:+ pass++ page.on("response", on_response)+ try:+ url = note_url or f"https://www.xiaohongshu.com/explore/{note_id}"+ try:+ await page.goto(url, wait_until="domcontentloaded", timeout=60000)+ except Exception as e:+ print(f"[警告] 笔记页加载异常:{e}")+ await asyncio.sleep(5)+ if "website-login/error" in page.url or "登录" in (await page.title()):+ print("[警告] 触发登录墙(安全限制)——cookie 未登录或已过期。先跑 crawler.py login 扫码。")++ # 滚动评论区触发分页懒加载+ last = 0+ stagnant = 0+ for _ in range(40):+ await page.evaluate("window.scrollBy(0, 1400)")+ await asyncio.sleep(1.8)+ cur = len({c["cid"] for c in comments})+ if cur == last:+ stagnant += 1+ if stagnant >= 5:+ break+ else:+ stagnant = 0+ last = cur++ if not comments:+ dbg = debug_dir()+ dbg.mkdir(parents=True, exist_ok=True)+ try:+ await page.screenshot(path=str(dbg / f"note_{note_id}.png"))+ except Exception:+ pass+ (dbg / "frontend_urls.txt").write_text("\n".join(all_urls), encoding="utf-8")+ print("[诊断] 前台未拦到评论(可能 xsec_token 缺失或评论被关),已 dump URL。")++ # 去重 + 按赞降序+ seen = set()+ dedup = []+ for c in comments:+ if c["cid"] in seen:+ continue+ seen.add(c["cid"])+ dedup.append(c)+ dedup.sort(key=lambda x: x["like_count"], reverse=True)+ print(f" 前台共 {len(dedup)} 条评论")+ return {"interact": feed, "comments": dedup}+ finally:+ await page.close()+++def _extract_interact(data: Any) -> dict:+ """从 feed response 里挖 interact_info(确认字段)。"""+ if not isinstance(data, dict):+ return {}+ items = []+ d = data.get("data", data)+ if isinstance(d, dict):+ items = d.get("items") or d.get("note_list") or []+ for it in items if isinstance(items, list) else []:+ node = it.get("note_card") or it.get("note") or it+ ii = node.get("interact_info") if isinstance(node, dict) else None+ if isinstance(ii, dict):+ return {+ "like_count": _to_int(ii.get("liked_count")),+ "collect_count": _to_int(ii.get("collected_count")),+ "comment_count": _to_int(ii.get("comment_count")),+ "share_count": _to_int(ii.get("share_count")),+ "ip_location": node.get("ip_location") or "",+ }+ return {}+++def _extract_comments(data: Any) -> list[dict]:+ """comment/page response → 评论列表(确认字段)。"""+ out: list[dict] = []+ if not isinstance(data, dict):+ return out+ d = data.get("data", data)+ arr = d.get("comments") if isinstance(d, dict) else None+ for c in arr or []:+ if not isinstance(c, dict):+ continue+ user = c.get("user_info") or {}+ out.append({+ "cid": str(c.get("id") or c.get("comment_id") or ""),+ "text": c.get("content") or "",+ "like_count": _to_int(c.get("like_count")),+ "sub_comment_count": _to_int(c.get("sub_comment_count")),+ "create_time": c.get("create_time") or 0,+ "user_name": user.get("nickname") or "",+ "ip_label": c.get("ip_location") or "",+ })+ return out+++def _dump(urls: list[str], url_file: str, captured: list[dict], cap_file: str) -> None:+ dbg = debug_dir()+ dbg.mkdir(parents=True, exist_ok=True)+ (dbg / url_file).write_text("\n".join(urls), encoding="utf-8")+ try:+ (dbg / cap_file).write_text(+ json.dumps([c["data"] for c in captured][:5], ensure_ascii=False, indent=2),+ encoding="utf-8",+ )+ except Exception:+ pass+++async def fetch_all(note_id: str, note_url: str | None = None) -> dict:+ """一个会话跑完笔记列表(含 galaxy 指标)+ 前台 interact + 评论。"""+ sess = await Session.open()+ try:+ print(" → 打开创作者中心,拉笔记列表 + 运营数据")+ notes = await fetch_recent_notes(sess, limit=50)+ note = next((n for n in notes if n["note_id"] == note_id), None)+ if not note:+ print(f" 未在最近 {len(notes)} 条里找到 {note_id},用最小元数据继续。")+ note = _normalize_note({"note_id": note_id})+ else:+ print(f" ✓ {(note.get('title') or '')[:40]}(曝光 {note.get('view_count')})")++ # 抓自己的笔记时,galaxy 列表已带每条的 xsec_token——自动拼前台 URL,免得手动粘 token 链接+ front_url = note_url+ if not front_url and note.get("xsec_token"):+ front_url = (f"https://www.xiaohongshu.com/explore/{note_id}"+ f"?xsec_token={note['xsec_token']}&xsec_source=pc_creatormng")++ print(" → 打开前台笔记页抓 interact + 评论")+ front = await fetch_note_frontend(sess, note_id, note_url=front_url)+ # 前台 interact 字段是确认的——用它补全/覆盖 galaxy 里可能缺的计数+ for k in ("like_count", "collect_count", "comment_count", "share_count"):+ if front["interact"].get(k):+ note[k] = front["interact"][k]+ if front["interact"].get("ip_location"):+ note["ip_location"] = front["interact"]["ip_location"]++ return {"note": note, "comments": front["comments"]}+ finally:+ await sess.close()+++if __name__ == "__main__":+ asyncio.run(ensure_login())diff --git a/adapters/perf-data/xhs-explore/paths.py b/adapters/perf-data/xhs-explore/paths.pynew file mode 100644index 0000000..346edb9--- /dev/null+++ b/adapters/perf-data/xhs-explore/paths.py@@ -0,0 +1,40 @@+from __future__ import annotations++import os+from pathlib import Path+from typing import Mapping+++def runtime_project_root(+ env: Mapping[str, str] | None = None,+ cwd: Path | None = None,+) -> Path:+ active_env = env if env is not None else os.environ+ if active_env.get("CHEAT_PROJECT_ROOT"):+ return Path(active_env["CHEAT_PROJECT_ROOT"]).expanduser().resolve()+ base_cwd = cwd if cwd is not None else Path.cwd()+ return Path(base_cwd).expanduser().resolve()+++def auth_dir(+ env: Mapping[str, str] | None = None,+ cwd: Path | None = None,+) -> Path:+ return runtime_project_root(env=env, cwd=cwd) / ".auth-xhs"+++def debug_dir(+ env: Mapping[str, str] | None = None,+ cwd: Path | None = None,+) -> Path:+ return runtime_project_root(env=env, cwd=cwd) / ".cheat-cache" / "xhs-explore-debug"+++def videos_dir(+ env: Mapping[str, str] | None = None,+ cwd: Path | None = None,+) -> Path:+ active_env = env if env is not None else os.environ+ if active_env.get("CHEAT_VIDEOS_DIR"):+ return Path(active_env["CHEAT_VIDEOS_DIR"]).expanduser().resolve()+ return runtime_project_root(env=env, cwd=cwd) / "videos"diff --git a/adapters/perf-data/xhs-explore/renderer.py b/adapters/perf-data/xhs-explore/renderer.pynew file mode 100644index 0000000..243aa8b--- /dev/null+++ b/adapters/perf-data/xhs-explore/renderer.py@@ -0,0 +1,110 @@+"""把抓到的小红书数据渲染成 report.md(cheat-retro 读这个文件)。"""+from __future__ import annotations++import datetime as dt+import json+from pathlib import Path+++def _fmt_time(ts) -> str:+ if not ts:+ return "未知"+ try:+ ts = int(ts)+ # 小红书部分接口用毫秒时间戳+ if ts > 1e12:+ ts //= 1000+ return dt.datetime.fromtimestamp(ts).strftime("%Y-%m-%d %H:%M")+ except (ValueError, TypeError, OSError):+ return str(ts)+++def _fmt_num(n) -> str:+ if n is None:+ return "-"+ try:+ n = int(n)+ except (ValueError, TypeError):+ return str(n)+ if n >= 10000:+ return f"{n/10000:.1f}w"+ return str(n)+++def _ratio(num, denom) -> str:+ try:+ num, denom = int(num), int(denom)+ except (ValueError, TypeError):+ return "-"+ if denom <= 0:+ return "-"+ return f"{num/denom*100:.2f}%"+++def render_report(note: dict, script: str, comments: list[dict]) -> str:+ lines: list[str] = []+ title = note.get("title") or "(无标题)"+ note_id = note["note_id"]+ view = note.get("view_count") or 0++ lines.append(f"# {title}")+ lines.append("")+ lines.append(f"- 笔记 ID:`{note_id}`")+ lines.append(f"- 发布时间:{note.get('post_time_str') or _fmt_time(note.get('create_time', 0))}")+ lines.append(f"- 链接:https://www.xiaohongshu.com/explore/{note_id}")+ lines.append(f"- 抓取时间:{dt.datetime.now().strftime('%Y-%m-%d %H:%M')}")+ if note.get("ip_location"):+ lines.append(f"- IP 归属:{note['ip_location']}")+ lines.append("")++ lines.append("## 数据快照")+ lines.append("")+ lines.append(f"- 曝光/浏览:{_fmt_num(view)}")+ lines.append(f"- 点赞:{_fmt_num(note.get('like_count'))}(赞曝比 {_ratio(note.get('like_count'), view)})")+ lines.append(f"- 收藏:{_fmt_num(note.get('collect_count'))}(藏曝比 {_ratio(note.get('collect_count'), view)})")+ lines.append(f"- 评论:{_fmt_num(note.get('comment_count'))}(评曝比 {_ratio(note.get('comment_count'), view)})")+ lines.append(f"- 分享:{_fmt_num(note.get('share_count'))}(分曝比 {_ratio(note.get('share_count'), view)})")+ if note.get("fans_inc"):+ lines.append(f"- 涨粉:{_fmt_num(note.get('fans_inc'))}")+ lines.append("")++ # galaxy 原始 JSON——首跑时用来确认曝光字段的真实 key 名+ raw = note.get("raw")+ if raw:+ lines.append("### galaxy 原始字段(首跑校准用,确认曝光字段 key 后可忽略)")+ lines.append("")+ lines.append("```json")+ lines.append(json.dumps(raw, ensure_ascii=False, indent=2)[:2500])+ lines.append("```")+ lines.append("")++ lines.append("## 原始稿子")+ lines.append("")+ lines.append(script.strip() if script.strip() else "(未提供)")+ lines.append("")++ lines.append(f"## 评论(按点赞降序,共 {len(comments)} 条)")+ lines.append("")+ if not comments:+ lines.append("(未抓到评论——可能 xsec_token 缺失、评论被关或需手动粘贴)")+ else:+ for c in comments:+ reply = f" 💬{c['sub_comment_count']}" if c.get("sub_comment_count") else ""+ ip = f" [{c['ip_label']}]" if c.get("ip_label") else ""+ text = (c.get("text") or "").replace("\n", " ").strip()+ lines.append(f"- [👍{c['like_count']}{reply}]{ip} {text}")+ lines.append("")++ return "\n".join(lines)+++def slugify(text: str, max_len: int = 30) -> str:+ bad = '<>:"/\\|?*\n\r\t'+ out = "".join("_" if ch in bad else ch for ch in text).strip()+ return out[:max_len] or "untitled"+++def output_dir_for(note: dict, root: Path) -> Path:+ date = (note.get("post_time_str") or _fmt_time(note.get("create_time", 0)))[:10].replace("未知", "nodate")+ slug = slugify(note.get("title") or note["note_id"])+ return root / f"{date}_{slug}"diff --git a/adapters/perf-data/xhs-explore/requirements.txt b/adapters/perf-data/xhs-explore/requirements.txtnew file mode 100644index 0000000..86a8fae--- /dev/null+++ b/adapters/perf-data/xhs-explore/requirements.txt@@ -0,0 +1 @@+playwright>=1.44diff --git a/adapters/perf-data/xhs-explore/review.py b/adapters/perf-data/xhs-explore/review.pynew file mode 100644index 0000000..a364459--- /dev/null+++ b/adapters/perf-data/xhs-explore/review.py@@ -0,0 +1,139 @@+"""发完笔记后跑一次:抓数据 + 评论 → 生成 report.md。++用法:+ python review.py # 交互式选笔记+ python review.py login # 仅登录(首次扫码)+ python review.py list # 列最近笔记(验证登录态)+ python review.py note <note_id> [script.txt] # 直接指定笔记+"""+from __future__ import annotations++import asyncio+import re+import sys+from pathlib import Path++import crawler+import renderer+from paths import videos_dir+++def _parse_note_arg(arg: str) -> tuple[str, str | None]:+ """接受 note_id 或完整笔记 URL(含 xsec_token)。返回 (note_id, note_url|None)。"""+ arg = arg.strip().strip("'").strip('"')+ if arg.startswith("http"):+ m = re.search(r"/(?:explore|discovery/item)/([0-9a-zA-Z]+)", arg)+ note_id = m.group(1) if m else arg+ return note_id, arg+ return arg, None+++def _prompt(msg: str) -> str:+ try:+ return input(msg).strip()+ except EOFError:+ return ""+++def _pick_note(notes: list[dict]) -> dict | None:+ if not notes:+ print("未抓到笔记列表。请确认创作者中心已登录,或页面结构已变,需要更新 crawler。")+ return None+ print("\n最近笔记:")+ for i, n in enumerate(notes):+ t = renderer._fmt_time(n.get("create_time", 0))+ title = (n.get("title") or "").replace("\n", " ")[:40]+ print(f" [{i}] {t} | 曝光 {renderer._fmt_num(n.get('view_count'))} | {title}")+ choice = _prompt("\n选择序号(回车取消):")+ if not choice.isdigit():+ return None+ idx = int(choice)+ if 0 <= idx < len(notes):+ return notes[idx]+ return None+++async def run() -> None:+ active_videos_dir = videos_dir()+ active_videos_dir.mkdir(parents=True, exist_ok=True)++ print("[选笔记] 打开创作者中心拉列表……")+ sess = await crawler.Session.open()+ try:+ notes = await crawler.fetch_recent_notes(sess, limit=10)+ finally:+ await sess.close()+ note = _pick_note(notes)+ if not note:+ print("已取消。")+ return++ script_raw = _prompt("把稿子 txt 拖进来(或回车跳过):")+ script_path: str | None = None+ if script_raw.strip():+ p = Path(script_raw.strip().strip("'").strip('"').replace("\\ ", " ")).expanduser()+ if p.is_file():+ script_path = str(p)+ else:+ print(f"[警告] 找不到 {p},稿子留空。")++ await run_with_id(note["note_id"], script_path)+++async def run_with_id(note_arg: str, script_path: str | None) -> None:+ active_videos_dir = videos_dir()+ active_videos_dir.mkdir(parents=True, exist_ok=True)++ note_id, note_url = _parse_note_arg(note_arg)++ script = ""+ if script_path:+ p = Path(script_path).expanduser()+ if p.is_file():+ script = p.read_text(encoding="utf-8", errors="ignore")+ print(f"稿子:{p.name}({len(script)} 字符)")+ else:+ print(f"[警告] 找不到稿子 {p}")++ print(f"[抓取] 笔记 {note_id}" + ("(带 token URL)" if note_url else ""))+ result = await crawler.fetch_all(note_id, note_url=note_url)+ note = result["note"]+ comments = result["comments"]++ out_dir = renderer.output_dir_for(note, active_videos_dir)+ out_dir.mkdir(parents=True, exist_ok=True)+ if script:+ (out_dir / "script.txt").write_text(script, encoding="utf-8")+ md = renderer.render_report(note, script, comments)+ report = out_dir / "report.md"+ report.write_text(md, encoding="utf-8")+ print(f"\n✓ {report}")+++def main() -> None:+ if len(sys.argv) > 1 and sys.argv[1] == "login":+ asyncio.run(crawler.ensure_login())+ return+ if len(sys.argv) > 1 and sys.argv[1] == "note":+ note_id = sys.argv[2]+ script_path = sys.argv[3] if len(sys.argv) > 3 else None+ asyncio.run(run_with_id(note_id, script_path))+ return+ if len(sys.argv) > 1 and sys.argv[1] == "list":+ async def _list() -> None:+ sess = await crawler.Session.open()+ try:+ notes = await crawler.fetch_recent_notes(sess, limit=20)+ finally:+ await sess.close()+ for i, n in enumerate(notes):+ t = renderer._fmt_time(n.get("create_time", 0))+ title = (n.get("title") or "").replace("\n", " ")[:50]+ print(f"[{i}] {n['note_id']} {t} 曝光{renderer._fmt_num(n.get('view_count'))} {title}")+ asyncio.run(_list())+ return+ asyncio.run(run())+++if __name__ == "__main__":+ main()diff --git a/adapters/perf-data/xhs-explore/run.sh b/adapters/perf-data/xhs-explore/run.shnew file mode 100644index 0000000..7652bef--- /dev/null+++ b/adapters/perf-data/xhs-explore/run.sh@@ -0,0 +1,119 @@+#!/usr/bin/env bash+#+# xhs-explore adapter wrapper(小红书)+#+# Called by /cheat-retro when state.data_collection=adapter and platform=xhs.+#+# Usage:+# bash run.sh <note_id> <video_folder> [<script_path>]+#+# Example:+# bash run.sh 66f1a2b3c4d5e6f700112233 ~/my-channel/videos/2026-05-04_abc123_标题+#+# Output: writes report.md INTO the video_folder.+# Exit codes:+# 0 = success (report.md written)+# 1 = login expired or required+# 2 = adapter dependency missing (playwright not installed)+# 3 = other failure (network, parse error, etc.)++set -uo pipefail++NOTE_ID="${1:-}"+VIDEO_FOLDER="${2:-}"+SCRIPT_PATH="${3:-}"++if [[ -z "$NOTE_ID" || -z "$VIDEO_FOLDER" ]]; then+ echo "Usage: bash run.sh <note_id> <video_folder> [<script_path>]" >&2+ exit 3+fi++ADAPTER_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"++# Find Python — prefer venv in user's project root if exists+PYTHON=""+PROJECT_ROOT="$( dirname "$( dirname "$( realpath "$VIDEO_FOLDER" )" )" )"+if [[ -x "$PROJECT_ROOT/.venv/bin/python" ]]; then+ PYTHON="$PROJECT_ROOT/.venv/bin/python"+elif [[ -x "$PROJECT_ROOT/.venv/Scripts/python.exe" ]]; then+ PYTHON="$PROJECT_ROOT/.venv/Scripts/python.exe" # Windows venv layout+elif command -v python3 >/dev/null 2>&1; then+ PYTHON="python3"+elif command -v python >/dev/null 2>&1; then+ PYTHON="python"+else+ echo "❌ python not found — install Python 3.10+ first" >&2+ exit 2+fi++# Verify playwright is installed+if ! "$PYTHON" -c "import playwright" 2>/dev/null; then+ cat >&2 <<EOF+❌ playwright not installed.++Install in your project venv:+ cd "$PROJECT_ROOT"+ python3 -m venv .venv+ source .venv/bin/activate # Windows: .venv\\Scripts\\activate+ pip install -r "$ADAPTER_DIR/requirements.txt"+ playwright install chromium++Then re-run /cheat-retro.+EOF+ exit 2+fi++# Verify auth dir exists in project root (cookie persistence)+if [[ ! -d "$PROJECT_ROOT/.auth-xhs" ]]; then+ cat >&2 <<EOF+❌ Not logged in to 小红书 创作者中心.++First-time login (one-shot):+ cd "$PROJECT_ROOT"+ source .venv/bin/activate # Windows: .venv\\Scripts\\activate+ $PYTHON "$ADAPTER_DIR/crawler.py" login++A Chromium window will pop up — scan QR with your phone to log in.+Cookie will be saved to .auth-xhs/ for future runs.+EOF+ exit 1+fi++mkdir -p "$VIDEO_FOLDER"++SCRIPT_ARG=""+if [[ -n "$SCRIPT_PATH" && -f "$SCRIPT_PATH" ]]; then+ SCRIPT_ARG="$SCRIPT_PATH"+fi++# Run from PROJECT_ROOT so .auth-xhs/ is found and outputs go to expected paths+cd "$PROJECT_ROOT"+export CHEAT_PROJECT_ROOT="$PROJECT_ROOT"+export CHEAT_VIDEOS_DIR="$( dirname "$VIDEO_FOLDER" )"++echo "[xhs-explore] fetching note_id=$NOTE_ID into $VIDEO_FOLDER"+if [[ -n "$SCRIPT_ARG" ]]; then+ "$PYTHON" "$ADAPTER_DIR/review.py" note "$NOTE_ID" "$SCRIPT_ARG"+else+ "$PYTHON" "$ADAPTER_DIR/review.py" note "$NOTE_ID"+fi++# review.py writes to CHEAT_VIDEOS_DIR/<auto-named-folder>/report.md.+# Move the just-written report into our canonical video_folder if names differ.+LATEST_REPORT=$(find "$( dirname "$VIDEO_FOLDER" )" -name "report.md" -newer "$VIDEO_FOLDER" -type f 2>/dev/null | head -1)+if [[ -n "$LATEST_REPORT" && "$( dirname "$LATEST_REPORT" )" != "$VIDEO_FOLDER" ]]; then+ cp "$LATEST_REPORT" "$VIDEO_FOLDER/report.md"+ AUTO_DIR=$( dirname "$LATEST_REPORT" )+ if [[ -f "$AUTO_DIR/script.txt" ]]; then+ cp "$AUTO_DIR/script.txt" "$VIDEO_FOLDER/script.txt"+ fi+ echo "[xhs-explore] moved auto-named output to $VIDEO_FOLDER/"+fi++if [[ ! -f "$VIDEO_FOLDER/report.md" ]]; then+ echo "❌ report.md not produced — see review.py output above for details" >&2+ exit 3+fi++echo "✅ report.md written to $VIDEO_FOLDER/report.md"+exit 0diff --git a/skills/cheat-init/SKILL.md b/skills/cheat-init/SKILL.mdindex 915d6de..575d3b2 100644--- a/skills/cheat-init/SKILL.md+++ b/skills/cheat-init/SKILL.md@@ -138,13 +138,14 @@ allowed-tools: Bash(*), Read, Write, Edit, Glob, WebFetch, Skill> "你内容主要在哪个平台?> a) 抖音 — 装 douyin-session adapter(Playwright + 扫码登录抖音创作者中心)-> b) YouTube — 装 youtube-data-api adapter(需 API key)-> c) B 站 — bilibili-stat adapter-> d) 其他 / 多平台 — 走 manual paste 模式"+> b) 小红书 — 装 xhs-explore adapter(Playwright + 扫码登录小红书创作者中心)+> c) YouTube — 装 youtube-data-api adapter(需 API key)+> d) B 站 — bilibili-stat adapter+> e) 其他 / 多平台 — 走 manual paste 模式"-如选 a/b/c → 询问 Q2.2;如选 d → 跳到 Q2.3 manual。+如选 a/b/c/d → 询问 Q2.2;如选 e → 跳到 Q2.3 manual。-**Q2.2: adapter 安装时机**(仅 Q2.1=a/b/c)+**Q2.2: adapter 安装时机**(仅 Q2.1=a/b/c/d)> "现在装 adapter 自动抓取,还是先手动告诉我?> - 现在装 — 引导你装 Playwright + 扫码 → 抓回最近 N 条数据@@ -287,7 +288,7 @@ c) 不找 → state 标 `benchmark_status: none`,用通用 v0 起步"data_layer": "markdown","hooks_installed": <查 Q5 映射表,写 bool true/false>,"enabled_trend_sources": ["manual-paste"],- "enabled_perf_adapters": <Q2.1=a→[\"douyin-session\"];b→[\"youtube-data-api\"];c→[\"bilibili-stat\"];其他→[]>,+ "enabled_perf_adapters": <Q2.1=a→[\"douyin-session\"];b→[\"xhs-explore\"];c→[\"youtube-data-api\"];d→[\"bilibili-stat\"];其他→[]>,"last_bump_at": null,"last_bump_self_audited": false,"last_published_at": null,diff --git a/skills/cheat-retro/SKILL.md b/skills/cheat-retro/SKILL.mdindex e7c4480..db18de1 100644--- a/skills/cheat-retro/SKILL.md+++ b/skills/cheat-retro/SKILL.md@@ -92,17 +92,26 @@ allowed-tools: Bash(*), Read, Edit, Write, Glob, Grep, Skill| Platform | Adapter | 调用方式 ||---|---|---|-| `douyin` | `adapters/perf-data/douyin-session/` | `bash <skills-dir>/../adapters/perf-data/douyin-session/run.sh <aweme_id> <video_folder>` |+| `douyin` | `adapters/perf-data/douyin-session/` | `bash <adapters-dir>/douyin-session/run.sh <aweme_id> <video_folder>` |+| `xhs` | `adapters/perf-data/xhs-explore/` | `bash <adapters-dir>/xhs-explore/run.sh <note_id> <video_folder>` || `youtube` | `adapters/perf-data/youtube-data-api/`(待) | 调 YouTube Data API(需 API key) || `bilibili` | `adapters/perf-data/bilibili-stat/`(待) | 调 B 站官方 stat 接口 || 其他 | 无 adapter | 优雅降级到 Path A |+> `<adapters-dir>` = 克隆源码处的 `cheat-on-content/adapters/perf-data/`(install.sh **不**复制 adapter 到 ~/.claude/skills,只复制 15 个 skill)。定位:`find ~ -path '*/cheat-on-content/adapters/perf-data' -type d | head -1`。+**douyin-session 的特殊处理**:- 视频 URL(如 `https://v.douyin.com/abc123`)→ 短链解析 → 提取 aweme_id- 调用前确认 cookie 文件存在(adapter 会找 `.auth/`);不存在则提示用户先跑 `python <adapter>/crawler.py login`- adapter 输出在 `<video_folder>/report.md`(adapter 的 renderer.py 已经按这个格式写)- cheat-retro 读这个 report.md 解析关键数据 → 摘要写入 prediction 的复盘段+**xhs-explore 的特殊处理**:+- 笔记 URL(`https://www.xiaohongshu.com/explore/<note_id>?xsec_token=...` 或 `https://xhslink.com/xxx`)→ 提取 note_id+- 调用前确认 cookie 存在(adapter 找 `.auth-xhs/`);不存在则提示先跑 `python <adapter>/crawler.py login`+- 字段已校准(观看 `view_count` 等已写死);万一接口改版导致某项为 0,看 report.md 末尾 galaxy 原始 JSON,把新 key 加进 `crawler.py` 的 `_normalize_note`+- **评论可能抓不到**(xsec_token 缺失 / 评论关闭)→ report.md 标"未抓到评论" → 此时**降级要求用户 manual 粘 top 20 评论**(评论是真信号,不能省)+**任何 adapter 失败**(cookie 过期 / 接口变化 / 网络)→ **优雅降级到 manual**,提示用户:"adapter 调用失败,原因 [X]。改用 manual 模式——粘下面的数据"。**不阻塞流程**。#### 共同输出