Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Python agent-infrastructureai-agentai-searchautomationbilibiliclaude-codeclicursorfree-apillm-toolsmcppythonreddit-scrapertwitter-scraperweb-scraperxiaohongshuyoutube-transcript
docs: 重写设计理念 — 突出脚手架定位、可插拔架构、新渠道贡献指南
Panniantong8b00f73e842e
diff --git a/README.md b/README.mdindex 5b6273fd..22b48133 100644--- a/README.md+++ b/README.md@@ -151,12 +151,78 @@ $ agent-reach doctor## 设计理念-**Agent Reach 是一个 Agent 初始化脚手架,不是框架。**+**Agent Reach 是一个脚手架(scaffolding),不是框架。**你给一个新 Agent 装环境的时候,总要花时间去找工具、装依赖、调配置——Twitter 用什么读?Reddit 怎么绕封?YouTube 字幕怎么提取?每次都要重新踩一遍。Agent Reach 做的事情很简单:**帮你把这些选型和配置的活儿做完了。**+### 🔌 每个渠道都是可插拔的++每个平台对应一个独立的 Python 文件(~50–100 行),实现统一接口。**后端工具随时可以换**——哪天出了更好的工具,改一个文件就行,其他不用动。++```+channels/+├── base.py → 统一接口(Channel 基类)+├── web.py → Jina Reader ← 可以换成 Firecrawl、Crawl4AI……+├── twitter.py → birdx ← 可以换成 Nitter、官方 API……+├── youtube.py → yt-dlp ← 可以换成 YouTube API、Whisper……+├── github.py → gh CLI ← 可以换成 REST API、PyGithub……+├── bilibili.py → yt-dlp ← 可以换成 bilibili-api……+├── reddit.py → JSON API + Exa ← 可以换成 PRAW、Pushshift……+├── xiaohongshu.py → mcporter MCP ← 可以换成其他 XHS 工具……+├── rss.py → feedparser ← 可以换成 atoma……+├── exa_search.py → mcporter MCP ← 可以换成 Tavily、SerpAPI……+└── __init__.py → 渠道注册(加一行就注册一个新渠道)+```++想换后端?打开对应文件,改掉 `read()` / `search()` 的实现就行。接口不变,其他代码不用动。++### 🧩 添加新渠道(3 步)++添加新渠道非常简单,3 步就能搞定:++**第 1 步:** 新建 `channels/你的渠道.py`++```python+from .base import Channel, ReadResult, SearchResult++class 你的渠道Channel(Channel):+ name = "你的渠道"+ description = "一句话描述"+ backends = ["用了什么工具"]++ def can_handle(self, url: str) -> bool:+ return "你的域名" in url++ async def read(self, url: str, config=None) -> ReadResult:+ # 读取内容,返回 ReadResult+ return ReadResult(title="...", content="...", url=url, platform=self.name)++ def check(self, config=None):+ return "ok", "一切正常"++ # 可选:实现 search() 支持搜索+```++**第 2 步:** 在 `channels/__init__.py` 注册++```python+from .你的渠道 import 你的渠道Channel++ALL_CHANNELS: List[Channel] = [+ ...+ 你的渠道Channel(), # 加这一行+ WebChannel(),+]+```++**第 3 步:** 没了。`agent-reach doctor` 自动识别,`agent-reach read` 自动路由。++> 💡 **参考现有渠道:** `rss.py`(30 行,最简单)→ `web.py`(50 行)→ `youtube.py`(100 行,含搜索)++### 当前选型+| 场景 | 选型 | 为什么选它 ||------|------|-----------|| 读网页 | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K Star,免费,不需要 API Key |@@ -167,24 +233,7 @@ Agent Reach 做的事情很简单:**帮你把这些选型和配置的活儿做| 读 RSS | [feedparser](https://github.com/kurtmckee/feedparser) | Python 生态标准选择,2.3K Star || 小红书 | [xiaohongshu-mcp](https://github.com/user/xiaohongshu-mcp) | 内部 API,不受反爬限制 |-每个平台一个文件,每个文件 ~50 行代码。后端工具随时可以换——哪天出了更好的工具,改一个文件就行,其他不用动。--<details>-<summary>项目结构</summary>--```-agent_reach/channels/-├── web.py → Jina Reader-├── twitter.py → birdx-├── youtube.py → yt-dlp-├── github.py → GitHub API-├── bilibili.py → Bilibili API-├── reddit.py → Reddit JSON API-├── xiaohongshu.py → XHS Web API-├── rss.py → feedparser-└── exa_search.py → Exa Search API-```-</details>+> 📌 这些都是「当前选型」。不满意?换掉对应文件就行。这正是脚手架的意义。---@@ -192,7 +241,25 @@ agent_reach/channels/欢迎提 [Issue](https://github.com/Panniantong/agent-reach/issues) 和 [PR](https://github.com/Panniantong/agent-reach/pulls)。-想加新平台?复制任意一个 channel 文件,改改就行——每个文件只有 ~50 行。+### 🆕 想添加新渠道?++1. 复制 `agent_reach/channels/rss.py`(最简单的参考)+2. 实现 `can_handle()` + `read()`,可选 `search()` 和 `check()`+3. 在 `__init__.py` 注册++就这么简单。不需要改框架代码,不需要了解其他渠道。++**希望支持的渠道(欢迎 PR):**++- 📰 Hacker News — 科技新闻+- 🐘 Mastodon / Fediverse — 去中心化社交+- 📱 Telegram — 频道和群组+- 🎵 Spotify / Apple Podcasts — 播客字幕+- 📝 Medium / Substack — 付费墙文章+- 🔬 arXiv / Semantic Scholar — 学术论文+- 💬 Discord — 服务器消息+- 📌 Pinterest — 图片搜索+- …… 任何你觉得有用的平台!## 致谢diff --git a/docs/README_en.md b/docs/README_en.mdindex bc1fafb5..6bf94e54 100644--- a/docs/README_en.md+++ b/docs/README_en.md@@ -151,12 +151,76 @@ Status: 6/9 channels available## Design Philosophy-**Agent Reach is a setup scaffold, not a framework.**+**Agent Reach is a scaffolding tool, not a framework.**Every time you spin up a new Agent, you spend time finding tools, installing deps, and debugging configs — what reads Twitter? How do you bypass Reddit blocks? How do you extract YouTube subtitles? Every time, you re-do the same work.Agent Reach does one simple thing: **it makes those tool selection and configuration decisions for you.**+### 🔌 Every Channel is Pluggable++Each platform is a single Python file (~50–100 lines) implementing a unified interface. **Backends can be swapped anytime** — when a better tool comes along, change one file and nothing else breaks.++```+channels/+├── base.py → Unified interface (Channel base class)+├── web.py → Jina Reader ← swap to Firecrawl, Crawl4AI…+├── twitter.py → birdx ← swap to Nitter, official API…+├── youtube.py → yt-dlp ← swap to YouTube API, Whisper…+├── github.py → gh CLI ← swap to REST API, PyGithub…+├── bilibili.py → yt-dlp ← swap to bilibili-api…+├── reddit.py → JSON API + Exa ← swap to PRAW, Pushshift…+├── xiaohongshu.py → mcporter MCP ← swap to other XHS tools…+├── rss.py → feedparser ← swap to atoma…+├── exa_search.py → mcporter MCP ← swap to Tavily, SerpAPI…+└── __init__.py → Channel registry (one line to register a new channel)+```++Want to swap a backend? Open the file, change the `read()` / `search()` implementation. Interface stays the same, nothing else needs to change.++### 🧩 Adding a New Channel (3 Steps)++**Step 1:** Create `channels/your_channel.py`++```python+from .base import Channel, ReadResult, SearchResult++class YourChannel(Channel):+ name = "your_channel"+ description = "One-line description"+ backends = ["tool-name"]++ def can_handle(self, url: str) -> bool:+ return "yourdomain.com" in url++ async def read(self, url: str, config=None) -> ReadResult:+ # Fetch content, return ReadResult+ return ReadResult(title="...", content="...", url=url, platform=self.name)++ def check(self, config=None):+ return "ok", "All good"++ # Optional: implement search() for search support+```++**Step 2:** Register in `channels/__init__.py`++```python+from .your_channel import YourChannel++ALL_CHANNELS: List[Channel] = [+ ...+ YourChannel(), # add this line+ WebChannel(),+]+```++**Step 3:** Done. `agent-reach doctor` auto-detects it, `agent-reach read` auto-routes to it.++> 💡 **Reference examples:** `rss.py` (30 lines, simplest) → `web.py` (50 lines) → `youtube.py` (100 lines, with search)++### Current Tool Choices+| Scenario | Tool | Why ||----------|------|-----|| Read web pages | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K stars, free, no API key needed |@@ -167,24 +231,7 @@ Agent Reach does one simple thing: **it makes those tool selection and configura| Read RSS | [feedparser](https://github.com/kurtmckee/feedparser) | Python ecosystem standard, 2.3K stars || XiaoHongShu | [xiaohongshu-mcp](https://github.com/user/xiaohongshu-mcp) | Internal API, bypasses anti-bot |-One file per platform, ~50 lines each. Swap any backend by editing one file — everything else stays untouched.--<details>-<summary>Project structure</summary>--```-agent_reach/channels/-├── web.py → Jina Reader-├── twitter.py → birdx-├── youtube.py → yt-dlp-├── github.py → GitHub API-├── bilibili.py → Bilibili API-├── reddit.py → Reddit JSON API-├── xiaohongshu.py → XHS Web API-├── rss.py → feedparser-└── exa_search.py → Exa Search API-```-</details>+> 📌 These are the *current* choices. Don't like one? Swap out the file. That's the whole point of scaffolding.---@@ -192,7 +239,25 @@ agent_reach/channels/[Issues](https://github.com/Panniantong/agent-reach/issues) and [PRs](https://github.com/Panniantong/agent-reach/pulls) welcome.-Want to add a new platform? Copy any channel file, tweak it — each one is only ~50 lines.+### 🆕 Want to Add a New Channel?++1. Copy `agent_reach/channels/rss.py` (simplest reference)+2. Implement `can_handle()` + `read()`, optionally `search()` and `check()`+3. Register in `__init__.py`++That's it. No framework code to modify, no need to understand other channels.++**Channels we'd love to see (PRs welcome):**++- 📰 Hacker News — tech news+- 🐘 Mastodon / Fediverse — decentralized social+- 📱 Telegram — channels and groups+- 🎵 Spotify / Apple Podcasts — podcast transcripts+- 📝 Medium / Substack — paywalled articles+- 🔬 arXiv / Semantic Scholar — academic papers+- 💬 Discord — server messages+- 📌 Pinterest — image search+- … anything you find useful!## Credits