⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载,你的 AI 舆情监控助手与热点筛选工具!聚合多平台热点 + RSS 订阅,支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机,也支持接入 MCP 架构,赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ,数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。
Python aibarkdata-analysisdockerhot-newsllmmailmcpmcp-servernewsntfypythonrsstrending-topicswechatwework
feat(crawler): 热榜平台域名安全校验与自定义 API 地址支持,升级至 v6.9.0
sansana583b2d0baf8
diff --git a/README-EN.md b/README-EN.mdindex d774d6d0d4..4d312a668f 100644--- a/README-EN.md+++ b/README-EN.md@@ -11,8 +11,8 @@ Deploy in <strong>30 seconds</strong> — Say goodbye to endless scrolling, only[](https://github.com/sansan0/TrendRadar/stargazers)[](https://github.com/sansan0/TrendRadar/network/members)[](LICENSE)-[](https://github.com/sansan0/TrendRadar)-[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)[](https://github.com/sansan0/TrendRadar)[](https://github.com/sansan0/TrendRadar)@@ -139,6 +139,8 @@ After communication, the author indicated no concerns about server pressure, but<div align="center">> **Sponsorship Open**+>+> Interested in sponsoring? Trigger the auto-reply in my WeChat Official Account to get my contact details</div>@@ -193,13 +195,10 @@ This contributes to the sustainable maintenance of the project and the growth of- **Tip**: Check [Changelog] to understand specific [Features]-### 2026/05/23 - v6.8.0+### 2026/06/02 - v6.9.0-- **HTML Report Major Enhancement**: Added report metadata display (generation time, data sources, version), dark mode auto-adaptation, improved Tab bar interaction, and trend arrow visualization for a significantly better browser reading experience-- **Version Check CDN Multi-Source Fallback**: Version check API now supports automatic fallback across GitHub → jsDelivr → Cloudflare CDN sources, ensuring reliable update notifications even in restricted network environments-- **Display Region Toggles**: HTML reports and emails now properly respect `display.regions.ai_analysis` and `display.regions.standalone` switches — disabled regions are no longer rendered-- **Export Button Fix**: Fixed dropdown menu icons disappearing after clicking the export button-- **Markdown Export Fix**: Fixed JS newline character escaping error in HTML report Markdown export+- **Platform Domain Safety Validation**: New `expected_domain` config option validates returned data link domains — mismatched data is automatically discarded with a warning, effectively preventing link hijacking or data tampering+- **Custom Platform API URL**: Support self-hosting newsnow and configuring `api_url` to use your own data source### 2026/02/09 - mcp-v4.0.0@@ -213,6 +212,14 @@ This contributes to the sustainable maintenance of the project and the growth of<details><summary>👉 Click to expand: <strong>Historical Updates</strong></summary>+### 2026/05/23 - v6.8.0++- **HTML Report Major Enhancement**: Added report metadata display (generation time, data sources, version), dark mode auto-adaptation, improved Tab bar interaction, and trend arrow visualization for a significantly better browser reading experience+- **Version Check CDN Multi-Source Fallback**: Version check API now supports automatic fallback across GitHub → jsDelivr → Cloudflare CDN sources, ensuring reliable update notifications even in restricted network environments+- **Display Region Toggles**: HTML reports and emails now properly respect `display.regions.ai_analysis` and `display.regions.standalone` switches — disabled regions are no longer rendered+- **Export Button Fix**: Fixed dropdown menu icons disappearing after clicking the export button+- **Markdown Export Fix**: Fixed JS newline character escaping error in HTML report Markdown export+### 2026/05/15 - v6.7.0- **Markdown Export**: New Markdown option in the report export dropdown — generate structured text with clickable links, perfect for LLM processing and cross-platform sharing ([#1121](https://github.com/sansan0/TrendRadar/issues/1121))diff --git a/README.md b/README.mdindex 8d13b95f91..6c36d0dbcc 100644--- a/README.md+++ b/README.md@@ -12,8 +12,8 @@[](https://github.com/sansan0/TrendRadar/stargazers)[](https://github.com/sansan0/TrendRadar/network/members)[](LICENSE)-[](https://github.com/sansan0/TrendRadar)-[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)[](https://github.com/sansan0/TrendRadar)[](https://github.com/sansan0/TrendRadar)@@ -187,6 +187,8 @@<div align="center">> **虚位以待**+>+> 有意赞助?在微信公众号中触发自动回复,即可获取我的联系方式</div>@@ -241,13 +243,10 @@- **提示**:建议查看【历史更新】,明确具体的【功能内容】-### 2026/05/23 - v6.8.0+### 2026/06/02 - v6.9.0-- **HTML 报告全面增强**:新增报告元数据展示(生成时间、数据来源、版本号)、暗色模式自动适配、Tab 栏交互优化、趋势箭头可视化,浏览器阅读体验大幅提升-- **版本检查 CDN 多源回退**:版本检查接口支持 GitHub → jsDelivr → Cloudflare 等多个 CDN 源自动回退,国内网络环境也能稳定获取更新提示-- **展示区域开关生效**:HTML 报告和邮件现在正确尊重 `display.regions.ai_analysis` 和 `display.regions.standalone` 开关,关闭即不渲染-- **导出按钮修复**:修复导出按钮点击后下拉菜单图标消失的问题-- **Markdown 导出修复**:修复 HTML 报告 Markdown 导出中 JS 换行符转义错误+- **热榜域名安全校验**:新增 `expected_domain` 配置项,校验返回数据链接的域名合法性,不匹配时自动丢弃数据并警告,有效防范链接劫持或数据篡改+- **自定义热榜 API 地址**:支持自部署 newsnow 并配置 `api_url` 使用自己的数据源### 2026/02/09 - mcp-v4.0.0@@ -261,6 +260,14 @@<details><summary>👉 点击展开:<strong>历史更新</strong></summary>+### 2026/05/23 - v6.8.0++- **HTML 报告全面增强**:新增报告元数据展示(生成时间、数据来源、版本号)、暗色模式自动适配、Tab 栏交互优化、趋势箭头可视化,浏览器阅读体验大幅提升+- **版本检查 CDN 多源回退**:版本检查接口支持 GitHub → jsDelivr → Cloudflare 等多个 CDN 源自动回退,国内网络环境也能稳定获取更新提示+- **展示区域开关生效**:HTML 报告和邮件现在正确尊重 `display.regions.ai_analysis` 和 `display.regions.standalone` 开关,关闭即不渲染+- **导出按钮修复**:修复导出按钮点击后下拉菜单图标消失的问题+- **Markdown 导出修复**:修复 HTML 报告 Markdown 导出中 JS 换行符转义错误+### 2026/05/15 - v6.7.0- **Markdown 导出**:报告导出下拉菜单新增 Markdown 格式,一键生成带链接的结构化文本,方便 LLM 二次加工和跨平台分享([#1121](https://github.com/sansan0/TrendRadar/issues/1121))diff --git a/config/config.yaml b/config/config.yamlindex 2a26e96be4..8b538ba173 100644--- a/config/config.yaml+++ b/config/config.yaml@@ -1,6 +1,6 @@# ═══════════════════════════════════════════════════════════════# TrendRadar 配置文件-# Version: 2.2.0+# Version: 2.3.0# ═══════════════════════════════════════════════════════════════@@ -47,36 +47,58 @@ schedule:# ===============================================================# 2. 数据源 - 热榜平台#+# 数据来源于 newsnow 开源项目: https://github.com/ourongxing/newsnow+# 你可以自行部署 newsnow 并将 api_url 指向你自己的实例+## enabled: 是否启用热榜抓取(总开关)+# api_url: 热榜数据 API 地址(可选,留空使用默认地址)# sources: 平台列表# - id: 平台唯一标识(勿修改)# - name: 显示名称(可自定义,修改后不影响运行)+# - expected_domain: 域名安全校验(可选,建议配置)+# 填写主域名(如 baidu.com),自动匹配该域名及其所有子域名+# 校验返回数据中的链接是否为 HTTPS 且域名匹配,不匹配时丢弃该平台数据并记录警告+# 可有效防范链接劫持或数据篡改+# 只要 api_url 不是你自己部署的实例,都推荐配置 expected_domain# ===============================================================platforms:enabled: true # 是否启用热榜平台抓取(true=启用, false=关闭)+ api_url: "" # 热榜数据 API 地址(留空使用默认地址)+ # 如果你自行部署了 newsnow,填入你的地址,如: https://your-domain.com/api/ssources:- id: "toutiao"name: "今日头条"+ expected_domain: "toutiao.com"- id: "baidu"name: "百度热搜"+ expected_domain: "baidu.com"- id: "wallstreetcn-hot"name: "华尔街见闻"+ expected_domain: "wallstreetcn.com"- id: "thepaper"name: "澎湃新闻"+ expected_domain: "thepaper.cn"- id: "bilibili-hot-search"name: "bilibili 热搜"+ expected_domain: "bilibili.com"- id: "cls-hot"name: "财联社热门"+ expected_domain: "cls.cn"- id: "ifeng"name: "凤凰网"+ expected_domain: "ifeng.com"- id: "tieba"name: "贴吧"+ expected_domain: "baidu.com"- id: "weibo"name: "微博"+ expected_domain: "weibo.com"- id: "douyin"name: "抖音"+ expected_domain: "douyin.com"- id: "zhihu"name: "知乎"+ expected_domain: "zhihu.com"diff --git a/mcp_server/__init__.py b/mcp_server/__init__.pyindex 59ee4fe778..4c20656eae 100644--- a/mcp_server/__init__.py+++ b/mcp_server/__init__.py@@ -5,4 +5,4 @@ TrendRadar MCP Server"""-__version__ = "4.0.4"+__version__ = "4.1.0"diff --git a/mcp_server/tools/system.py b/mcp_server/tools/system.pyindex fe0adb374f..e1687d8437 100644--- a/mcp_server/tools/system.py+++ b/mcp_server/tools/system.py@@ -4,6 +4,7 @@实现系统状态查询和爬虫触发功能。"""+import osfrom pathlib import Pathfrom typing import Dict, List, Optional@@ -227,12 +228,24 @@ class SystemManagementTools:# 2. 执行爬取advanced = config_data.get("advanced", {})crawler_config = advanced.get("crawler", {})+ platforms_config = config_data.get("platforms", {})proxy_url = crawler_config.get("default_proxy") if crawler_config.get("use_proxy") else None-- fetcher = DataFetcher(proxy_url=proxy_url)+ api_url = (+ os.environ.get("PLATFORMS_API_URL", "").strip()+ or platforms_config.get("api_url", "")+ ) or None++ domain_rules = {}+ for p in target_platforms:+ ed = p.get("expected_domain", "")+ if ed:+ domain_rules[p["id"]] = ed++ fetcher = DataFetcher(proxy_url=proxy_url, api_url=api_url)results, id_to_name, failed_ids = fetcher.crawl_websites(ids_list=ids,- request_interval=crawler_config.get("request_interval", 100)+ request_interval=crawler_config.get("request_interval", 100),+ domain_rules=domain_rules,)# 3. 转换与持久化diff --git a/pyproject.toml b/pyproject.tomlindex d088c5d890..76c7ac69df 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,6 +1,6 @@[project]name = "trendradar"-version = "6.8.1"+version = "6.9.0"description = "TrendRadar - 热点新闻聚合与分析工具"requires-python = ">=3.12"dependencies = [diff --git a/trendradar/__init__.py b/trendradar/__init__.pyindex 62364678ad..16d616d630 100644--- a/trendradar/__init__.py+++ b/trendradar/__init__.py@@ -9,5 +9,5 @@ TrendRadar - 热点新闻聚合与分析工具from trendradar.context import AppContext-__version__ = "6.8.1"+__version__ = "6.9.0"__all__ = ["AppContext", "__version__"]diff --git a/trendradar/__main__.py b/trendradar/__main__.pyindex d50a0b8612..81cb5e63be 100644--- a/trendradar/__main__.py+++ b/trendradar/__main__.py@@ -222,7 +222,10 @@ class NewsAnalyzer:self.update_info = Noneself.proxy_url = Noneself._setup_proxy()- self.data_fetcher = DataFetcher(self.proxy_url)+ self.data_fetcher = DataFetcher(+ self.proxy_url,+ api_url=self.ctx.config.get("PLATFORMS_API_URL") or None,+ )# RSS/平台元数据(用于报告头部展示)self._rss_source_total = 0@@ -1069,11 +1072,15 @@ class NewsAnalyzer:def _crawl_data(self) -> Tuple[Dict, Dict, List]:"""执行数据爬取"""ids = []+ domain_rules = {}for platform in self.ctx.platforms:if "name" in platform:ids.append((platform["id"], platform["name"]))else:ids.append(platform["id"])+ expected_domain = platform.get("expected_domain", "")+ if expected_domain:+ domain_rules[platform["id"]] = expected_domainprint(f"配置的监控平台: {[p.get('name', p['id']) for p in self.ctx.platforms]}"@@ -1082,7 +1089,7 @@ class NewsAnalyzer:Path("output").mkdir(parents=True, exist_ok=True)results, id_to_name, failed_ids = self.data_fetcher.crawl_websites(- ids, self.request_interval+ ids, self.request_interval, domain_rules=domain_rules)# 转换为 NewsData 格式并保存到存储后端diff --git a/trendradar/core/loader.py b/trendradar/core/loader.pyindex 121f312cc2..03b1ad114e 100644--- a/trendradar/core/loader.py+++ b/trendradar/core/loader.py@@ -73,6 +73,7 @@ def _load_crawler_config(config_data: Dict) -> Dict:"USE_PROXY": crawler_config.get("use_proxy", False),"DEFAULT_PROXY": crawler_config.get("default_proxy", ""),"ENABLE_CRAWLER": platforms_config.get("enabled", True),+ "PLATFORMS_API_URL": _get_env_str("PLATFORMS_API_URL") or platforms_config.get("api_url", ""),}diff --git a/trendradar/crawler/fetcher.py b/trendradar/crawler/fetcher.pyindex 45d0b10a91..7fbc6d745f 100644--- a/trendradar/crawler/fetcher.py+++ b/trendradar/crawler/fetcher.py@@ -13,6 +13,7 @@ import jsonimport randomimport timefrom typing import Dict, List, Tuple, Optional, Union+from urllib.parse import urlparseimport requests@@ -20,7 +21,7 @@ import requestsclass DataFetcher:"""数据获取器"""- # 默认 API 地址+ # 默认 API 地址(newsnow 项目: https://github.com/ourongxing/newsnow)DEFAULT_API_URL = "https://newsnow.busiyi.world/api/s"# 默认请求头@@ -47,6 +48,41 @@ class DataFetcher:self.proxy_url = proxy_urlself.api_url = api_url or self.DEFAULT_API_URL+ @staticmethod+ def _check_domain_safety(+ items: List[Dict],+ expected_domain: str,+ ) -> Optional[str]:+ """+ 校验返回数据中的链接是否为 HTTPS 且域名匹配预期(支持子域名)++ 同时校验 url 与 mobileUrl 两个字段,使用标准库解析主机名,+ 避免 userinfo(如 https://[email protected])绕过校验。++ Args:+ items: API 返回的数据项列表+ expected_domain: 预期域名(如 "baidu.com")++ Returns:+ None 表示安全,否则返回第一个异常描述+ """+ expected = expected_domain.lower().strip()+ if not expected:+ return None++ for item in items:+ for field in ("url", "mobileUrl"):+ url = item.get(field, "")+ if not url:+ continue+ parsed = urlparse(url)+ if parsed.scheme != "https":+ return f"{url} (非 HTTPS 或格式异常)"+ hostname = (parsed.hostname or "").lower()+ if hostname != expected and not hostname.endswith("." + expected):+ return f"{hostname} (来自 {url})"+ return None+def fetch_data(self,id_info: Union[str, Tuple[str, str]],@@ -118,6 +154,7 @@ class DataFetcher:self,ids_list: List[Union[str, Tuple[str, str]]],request_interval: int = 100,+ domain_rules: Optional[Dict[str, str]] = None,) -> Tuple[Dict, Dict, List]:"""爬取多个网站数据@@ -125,6 +162,7 @@ class DataFetcher:Args:ids_list: 平台ID列表,每个元素可以是字符串或 (平台ID, 别名) 元组request_interval: 请求间隔(毫秒)+ domain_rules: 域名安全校验规则,格式 {平台ID: 预期域名}(可选)Returns:(结果字典, ID到名称的映射, 失败ID列表) 元组@@ -132,6 +170,7 @@ class DataFetcher:results = {}id_to_name = {}failed_ids = []+ domain_rules = domain_rules or {}for i, id_info in enumerate(ids_list):if isinstance(id_info, tuple):@@ -146,9 +185,24 @@ class DataFetcher:if response:try:data = json.loads(response)+ items = data.get("items", [])++ # 域名安全校验+ expected_domain = domain_rules.get(id_value, "")+ if expected_domain:+ bad_reason = self._check_domain_safety(items, expected_domain)+ if bad_reason:+ print(f"⚠️ 安全警告: {name}({id_value}) 返回数据未通过域名安全校验!")+ print(f" 预期域名: https://*.{expected_domain}")+ print(f" 异常来源: {bad_reason}")+ print(f" 当前 API 地址: {self.api_url}")+ print(f" 该平台数据已丢弃,请检查 API 来源是否可信")+ failed_ids.append(id_value)+ continue+results[id_value] = {}- for index, item in enumerate(data.get("items", []), 1):+ for index, item in enumerate(items, 1):title = item.get("title")# 跳过无效标题(None、float、空字符串)if title is None or isinstance(title, float) or not str(title).strip():diff --git a/trendradar/report/html.py b/trendradar/report/html.pyindex d3eff74de8..4a760d45de 100644--- a/trendradar/report/html.py+++ b/trendradar/report/html.py@@ -2908,7 +2908,7 @@ def render_html_content(var line = idx + '. ';if (url) {- line += '[' + titleText.replace(/[[\]]/g, '') + '](' + url + ')';+ line += '[' + titleText.replace(/[[\\]]/g, '') + '](' + url + ')';} else {line += titleText;}diff --git a/uv.lock b/uv.lockindex 353997dcb4..fe801cbaa9 100644--- a/uv.lock+++ b/uv.lock@@ -1996,7 +1996,7 @@ wheels = [[[package]]name = "trendradar"-version = "6.8.1"+version = "6.9.0"source = { editable = "." }dependencies = [{ name = "boto3" },diff --git a/version b/versionindex 5f6c0863ac..6a1fccf930 100644--- a/version+++ b/version@@ -1 +1 @@-6.8.1+6.9.0\ No newline at end of filediff --git a/version_configs b/version_configsindex 2936f5094e..61819ef957 100644--- a/version_configs+++ b/version_configs@@ -1,4 +1,4 @@-config.yaml=2.2.0+config.yaml=2.3.0timeline.yaml=1.2.0frequency_words.txt=1.1.0ai_interests.txt=1.1.0diff --git a/version_mcp b/version_mcpindex 7d666cb219..99eba4de93 100644--- a/version_mcp+++ b/version_mcp@@ -1 +1 @@-4.0.4\ No newline at end of file+4.1.0\ No newline at end of file