Skip to content

⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载,你的 AI 舆情监控助手与热点筛选工具!聚合多平台热点 + RSS 订阅,支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机,也支持接入 MCP 架构,赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ,数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。

feat(crawler): 热榜平台域名安全校验与自定义 API 地址支持,升级至 v6.9.0

sansana583b2d0baf8
diff --git a/README-EN.md b/README-EN.md
index d774d6d0d4..4d312a668f 100644
--- a/README-EN.md
+++ b/README-EN.md
@@ -11,8 +11,8 @@ Deploy in <strong>30 seconds</strong> — Say goodbye to endless scrolling, only
[![GitHub Stars](https://img.shields.io/github/stars/sansan0/TrendRadar?style=flat-square&logo=github&color=yellow)](https://github.com/sansan0/TrendRadar/stargazers)
[![GitHub Forks](https://img.shields.io/github/forks/sansan0/TrendRadar?style=flat-square&logo=github&color=blue)](https://github.com/sansan0/TrendRadar/network/members)
[![License](https://img.shields.io/badge/license-GPL--3.0-blue.svg?style=flat-square)](LICENSE)
-[![Version](https://img.shields.io/badge/version-v6.8.1-blue.svg)](https://github.com/sansan0/TrendRadar)
-[![MCP](https://img.shields.io/badge/MCP-v4.0.4-green.svg)](https://github.com/sansan0/TrendRadar)
+[![Version](https://img.shields.io/badge/version-v6.9.0-blue.svg)](https://github.com/sansan0/TrendRadar)
+[![MCP](https://img.shields.io/badge/MCP-v4.1.0-green.svg)](https://github.com/sansan0/TrendRadar)
[![RSS](https://img.shields.io/badge/RSS-Feed_Support-orange.svg?style=flat-square&logo=rss&logoColor=white)](https://github.com/sansan0/TrendRadar)
[![AI Translation](https://img.shields.io/badge/AI-Multi--Language-purple.svg?style=flat-square)](https://github.com/sansan0/TrendRadar)
@@ -139,6 +139,8 @@ After communication, the author indicated no concerns about server pressure, but
<div align="center">
> **Sponsorship Open**
+>
+> Interested in sponsoring? Trigger the auto-reply in my WeChat Official Account to get my contact details
</div>
@@ -193,13 +195,10 @@ This contributes to the sustainable maintenance of the project and the growth of
- **Tip**: Check [Changelog] to understand specific [Features]
-### 2026/05/23 - v6.8.0
+### 2026/06/02 - v6.9.0
-- **HTML Report Major Enhancement**: Added report metadata display (generation time, data sources, version), dark mode auto-adaptation, improved Tab bar interaction, and trend arrow visualization for a significantly better browser reading experience
-- **Version Check CDN Multi-Source Fallback**: Version check API now supports automatic fallback across GitHub → jsDelivr → Cloudflare CDN sources, ensuring reliable update notifications even in restricted network environments
-- **Display Region Toggles**: HTML reports and emails now properly respect `display.regions.ai_analysis` and `display.regions.standalone` switches — disabled regions are no longer rendered
-- **Export Button Fix**: Fixed dropdown menu icons disappearing after clicking the export button
-- **Markdown Export Fix**: Fixed JS newline character escaping error in HTML report Markdown export
+- **Platform Domain Safety Validation**: New `expected_domain` config option validates returned data link domains — mismatched data is automatically discarded with a warning, effectively preventing link hijacking or data tampering
+- **Custom Platform API URL**: Support self-hosting newsnow and configuring `api_url` to use your own data source
### 2026/02/09 - mcp-v4.0.0
@@ -213,6 +212,14 @@ This contributes to the sustainable maintenance of the project and the growth of
<details>
<summary>👉 Click to expand: <strong>Historical Updates</strong></summary>
+### 2026/05/23 - v6.8.0
+
+- **HTML Report Major Enhancement**: Added report metadata display (generation time, data sources, version), dark mode auto-adaptation, improved Tab bar interaction, and trend arrow visualization for a significantly better browser reading experience
+- **Version Check CDN Multi-Source Fallback**: Version check API now supports automatic fallback across GitHub → jsDelivr → Cloudflare CDN sources, ensuring reliable update notifications even in restricted network environments
+- **Display Region Toggles**: HTML reports and emails now properly respect `display.regions.ai_analysis` and `display.regions.standalone` switches — disabled regions are no longer rendered
+- **Export Button Fix**: Fixed dropdown menu icons disappearing after clicking the export button
+- **Markdown Export Fix**: Fixed JS newline character escaping error in HTML report Markdown export
+
### 2026/05/15 - v6.7.0
- **Markdown Export**: New Markdown option in the report export dropdown — generate structured text with clickable links, perfect for LLM processing and cross-platform sharing ([#1121](https://github.com/sansan0/TrendRadar/issues/1121))
diff --git a/README.md b/README.md
index 8d13b95f91..6c36d0dbcc 100644
--- a/README.md
+++ b/README.md
@@ -12,8 +12,8 @@
[![GitHub Stars](https://img.shields.io/github/stars/sansan0/TrendRadar?style=flat-square&logo=github&color=yellow)](https://github.com/sansan0/TrendRadar/stargazers)
[![GitHub Forks](https://img.shields.io/github/forks/sansan0/TrendRadar?style=flat-square&logo=github&color=blue)](https://github.com/sansan0/TrendRadar/network/members)
[![License](https://img.shields.io/badge/license-GPL--3.0-blue.svg?style=flat-square)](LICENSE)
-[![Version](https://img.shields.io/badge/version-v6.8.1-blue.svg)](https://github.com/sansan0/TrendRadar)
-[![MCP](https://img.shields.io/badge/MCP-v4.0.4-green.svg)](https://github.com/sansan0/TrendRadar)
+[![Version](https://img.shields.io/badge/version-v6.9.0-blue.svg)](https://github.com/sansan0/TrendRadar)
+[![MCP](https://img.shields.io/badge/MCP-v4.1.0-green.svg)](https://github.com/sansan0/TrendRadar)
[![RSS](https://img.shields.io/badge/RSS-订阅源支持-orange.svg?style=flat-square&logo=rss&logoColor=white)](https://github.com/sansan0/TrendRadar)
[![AI翻译](https://img.shields.io/badge/AI-多语言推送-purple.svg?style=flat-square)](https://github.com/sansan0/TrendRadar)
@@ -187,6 +187,8 @@
<div align="center">
> **虚位以待**
+>
+> 有意赞助?在微信公众号中触发自动回复,即可获取我的联系方式
</div>
@@ -241,13 +243,10 @@
- **提示**:建议查看【历史更新】,明确具体的【功能内容】
-### 2026/05/23 - v6.8.0
+### 2026/06/02 - v6.9.0
-- **HTML 报告全面增强**:新增报告元数据展示(生成时间、数据来源、版本号)、暗色模式自动适配、Tab 栏交互优化、趋势箭头可视化,浏览器阅读体验大幅提升
-- **版本检查 CDN 多源回退**:版本检查接口支持 GitHub → jsDelivr → Cloudflare 等多个 CDN 源自动回退,国内网络环境也能稳定获取更新提示
-- **展示区域开关生效**:HTML 报告和邮件现在正确尊重 `display.regions.ai_analysis` 和 `display.regions.standalone` 开关,关闭即不渲染
-- **导出按钮修复**:修复导出按钮点击后下拉菜单图标消失的问题
-- **Markdown 导出修复**:修复 HTML 报告 Markdown 导出中 JS 换行符转义错误
+- **热榜域名安全校验**:新增 `expected_domain` 配置项,校验返回数据链接的域名合法性,不匹配时自动丢弃数据并警告,有效防范链接劫持或数据篡改
+- **自定义热榜 API 地址**:支持自部署 newsnow 并配置 `api_url` 使用自己的数据源
### 2026/02/09 - mcp-v4.0.0
@@ -261,6 +260,14 @@
<details>
<summary>👉 点击展开:<strong>历史更新</strong></summary>
+### 2026/05/23 - v6.8.0
+
+- **HTML 报告全面增强**:新增报告元数据展示(生成时间、数据来源、版本号)、暗色模式自动适配、Tab 栏交互优化、趋势箭头可视化,浏览器阅读体验大幅提升
+- **版本检查 CDN 多源回退**:版本检查接口支持 GitHub → jsDelivr → Cloudflare 等多个 CDN 源自动回退,国内网络环境也能稳定获取更新提示
+- **展示区域开关生效**:HTML 报告和邮件现在正确尊重 `display.regions.ai_analysis` 和 `display.regions.standalone` 开关,关闭即不渲染
+- **导出按钮修复**:修复导出按钮点击后下拉菜单图标消失的问题
+- **Markdown 导出修复**:修复 HTML 报告 Markdown 导出中 JS 换行符转义错误
+
### 2026/05/15 - v6.7.0
- **Markdown 导出**:报告导出下拉菜单新增 Markdown 格式,一键生成带链接的结构化文本,方便 LLM 二次加工和跨平台分享([#1121](https://github.com/sansan0/TrendRadar/issues/1121))
diff --git a/config/config.yaml b/config/config.yaml
index 2a26e96be4..8b538ba173 100644
--- a/config/config.yaml
+++ b/config/config.yaml
@@ -1,6 +1,6 @@
# ═══════════════════════════════════════════════════════════════
# TrendRadar 配置文件
-# Version: 2.2.0
+# Version: 2.3.0
# ═══════════════════════════════════════════════════════════════
@@ -47,36 +47,58 @@ schedule:
# ===============================================================
# 2. 数据源 - 热榜平台
#
+# 数据来源于 newsnow 开源项目: https://github.com/ourongxing/newsnow
+# 你可以自行部署 newsnow 并将 api_url 指向你自己的实例
+#
# enabled: 是否启用热榜抓取(总开关)
+# api_url: 热榜数据 API 地址(可选,留空使用默认地址)
# sources: 平台列表
# - id: 平台唯一标识(勿修改)
# - name: 显示名称(可自定义,修改后不影响运行)
+# - expected_domain: 域名安全校验(可选,建议配置)
+# 填写主域名(如 baidu.com),自动匹配该域名及其所有子域名
+# 校验返回数据中的链接是否为 HTTPS 且域名匹配,不匹配时丢弃该平台数据并记录警告
+# 可有效防范链接劫持或数据篡改
+# 只要 api_url 不是你自己部署的实例,都推荐配置 expected_domain
# ===============================================================
platforms:
enabled: true # 是否启用热榜平台抓取(true=启用, false=关闭)
+ api_url: "" # 热榜数据 API 地址(留空使用默认地址)
+ # 如果你自行部署了 newsnow,填入你的地址,如: https://your-domain.com/api/s
sources:
- id: "toutiao"
name: "今日头条"
+ expected_domain: "toutiao.com"
- id: "baidu"
name: "百度热搜"
+ expected_domain: "baidu.com"
- id: "wallstreetcn-hot"
name: "华尔街见闻"
+ expected_domain: "wallstreetcn.com"
- id: "thepaper"
name: "澎湃新闻"
+ expected_domain: "thepaper.cn"
- id: "bilibili-hot-search"
name: "bilibili 热搜"
+ expected_domain: "bilibili.com"
- id: "cls-hot"
name: "财联社热门"
+ expected_domain: "cls.cn"
- id: "ifeng"
name: "凤凰网"
+ expected_domain: "ifeng.com"
- id: "tieba"
name: "贴吧"
+ expected_domain: "baidu.com"
- id: "weibo"
name: "微博"
+ expected_domain: "weibo.com"
- id: "douyin"
name: "抖音"
+ expected_domain: "douyin.com"
- id: "zhihu"
name: "知乎"
+ expected_domain: "zhihu.com"
diff --git a/mcp_server/__init__.py b/mcp_server/__init__.py
index 59ee4fe778..4c20656eae 100644
--- a/mcp_server/__init__.py
+++ b/mcp_server/__init__.py
@@ -5,4 +5,4 @@ TrendRadar MCP Server
"""
-__version__ = "4.0.4"
+__version__ = "4.1.0"
diff --git a/mcp_server/tools/system.py b/mcp_server/tools/system.py
index fe0adb374f..e1687d8437 100644
--- a/mcp_server/tools/system.py
+++ b/mcp_server/tools/system.py
@@ -4,6 +4,7 @@
实现系统状态查询和爬虫触发功能。
"""
+import os
from pathlib import Path
from typing import Dict, List, Optional
@@ -227,12 +228,24 @@ class SystemManagementTools:
# 2. 执行爬取
advanced = config_data.get("advanced", {})
crawler_config = advanced.get("crawler", {})
+ platforms_config = config_data.get("platforms", {})
proxy_url = crawler_config.get("default_proxy") if crawler_config.get("use_proxy") else None
-
- fetcher = DataFetcher(proxy_url=proxy_url)
+ api_url = (
+ os.environ.get("PLATFORMS_API_URL", "").strip()
+ or platforms_config.get("api_url", "")
+ ) or None
+
+ domain_rules = {}
+ for p in target_platforms:
+ ed = p.get("expected_domain", "")
+ if ed:
+ domain_rules[p["id"]] = ed
+
+ fetcher = DataFetcher(proxy_url=proxy_url, api_url=api_url)
results, id_to_name, failed_ids = fetcher.crawl_websites(
ids_list=ids,
- request_interval=crawler_config.get("request_interval", 100)
+ request_interval=crawler_config.get("request_interval", 100),
+ domain_rules=domain_rules,
)
# 3. 转换与持久化
diff --git a/pyproject.toml b/pyproject.toml
index d088c5d890..76c7ac69df 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "trendradar"
-version = "6.8.1"
+version = "6.9.0"
description = "TrendRadar - 热点新闻聚合与分析工具"
requires-python = ">=3.12"
dependencies = [
diff --git a/trendradar/__init__.py b/trendradar/__init__.py
index 62364678ad..16d616d630 100644
--- a/trendradar/__init__.py
+++ b/trendradar/__init__.py
@@ -9,5 +9,5 @@ TrendRadar - 热点新闻聚合与分析工具
from trendradar.context import AppContext
-__version__ = "6.8.1"
+__version__ = "6.9.0"
__all__ = ["AppContext", "__version__"]
diff --git a/trendradar/__main__.py b/trendradar/__main__.py
index d50a0b8612..81cb5e63be 100644
--- a/trendradar/__main__.py
+++ b/trendradar/__main__.py
@@ -222,7 +222,10 @@ class NewsAnalyzer:
self.update_info = None
self.proxy_url = None
self._setup_proxy()
- self.data_fetcher = DataFetcher(self.proxy_url)
+ self.data_fetcher = DataFetcher(
+ self.proxy_url,
+ api_url=self.ctx.config.get("PLATFORMS_API_URL") or None,
+ )
# RSS/平台元数据(用于报告头部展示)
self._rss_source_total = 0
@@ -1069,11 +1072,15 @@ class NewsAnalyzer:
def _crawl_data(self) -> Tuple[Dict, Dict, List]:
"""执行数据爬取"""
ids = []
+ domain_rules = {}
for platform in self.ctx.platforms:
if "name" in platform:
ids.append((platform["id"], platform["name"]))
else:
ids.append(platform["id"])
+ expected_domain = platform.get("expected_domain", "")
+ if expected_domain:
+ domain_rules[platform["id"]] = expected_domain
print(
f"配置的监控平台: {[p.get('name', p['id']) for p in self.ctx.platforms]}"
@@ -1082,7 +1089,7 @@ class NewsAnalyzer:
Path("output").mkdir(parents=True, exist_ok=True)
results, id_to_name, failed_ids = self.data_fetcher.crawl_websites(
- ids, self.request_interval
+ ids, self.request_interval, domain_rules=domain_rules
)
# 转换为 NewsData 格式并保存到存储后端
diff --git a/trendradar/core/loader.py b/trendradar/core/loader.py
index 121f312cc2..03b1ad114e 100644
--- a/trendradar/core/loader.py
+++ b/trendradar/core/loader.py
@@ -73,6 +73,7 @@ def _load_crawler_config(config_data: Dict) -> Dict:
"USE_PROXY": crawler_config.get("use_proxy", False),
"DEFAULT_PROXY": crawler_config.get("default_proxy", ""),
"ENABLE_CRAWLER": platforms_config.get("enabled", True),
+ "PLATFORMS_API_URL": _get_env_str("PLATFORMS_API_URL") or platforms_config.get("api_url", ""),
}
diff --git a/trendradar/crawler/fetcher.py b/trendradar/crawler/fetcher.py
index 45d0b10a91..7fbc6d745f 100644
--- a/trendradar/crawler/fetcher.py
+++ b/trendradar/crawler/fetcher.py
@@ -13,6 +13,7 @@ import json
import random
import time
from typing import Dict, List, Tuple, Optional, Union
+from urllib.parse import urlparse
import requests
@@ -20,7 +21,7 @@ import requests
class DataFetcher:
"""数据获取器"""
- # 默认 API 地址
+ # 默认 API 地址(newsnow 项目: https://github.com/ourongxing/newsnow)
DEFAULT_API_URL = "https://newsnow.busiyi.world/api/s"
# 默认请求头
@@ -47,6 +48,41 @@ class DataFetcher:
self.proxy_url = proxy_url
self.api_url = api_url or self.DEFAULT_API_URL
+ @staticmethod
+ def _check_domain_safety(
+ items: List[Dict],
+ expected_domain: str,
+ ) -> Optional[str]:
+ """
+ 校验返回数据中的链接是否为 HTTPS 且域名匹配预期(支持子域名)
+
+ 同时校验 url 与 mobileUrl 两个字段,使用标准库解析主机名,
+ 避免 userinfo(如 https://[email protected])绕过校验。
+
+ Args:
+ items: API 返回的数据项列表
+ expected_domain: 预期域名(如 "baidu.com")
+
+ Returns:
+ None 表示安全,否则返回第一个异常描述
+ """
+ expected = expected_domain.lower().strip()
+ if not expected:
+ return None
+
+ for item in items:
+ for field in ("url", "mobileUrl"):
+ url = item.get(field, "")
+ if not url:
+ continue
+ parsed = urlparse(url)
+ if parsed.scheme != "https":
+ return f"{url} (非 HTTPS 或格式异常)"
+ hostname = (parsed.hostname or "").lower()
+ if hostname != expected and not hostname.endswith("." + expected):
+ return f"{hostname} (来自 {url})"
+ return None
+
def fetch_data(
self,
id_info: Union[str, Tuple[str, str]],
@@ -118,6 +154,7 @@ class DataFetcher:
self,
ids_list: List[Union[str, Tuple[str, str]]],
request_interval: int = 100,
+ domain_rules: Optional[Dict[str, str]] = None,
) -> Tuple[Dict, Dict, List]:
"""
爬取多个网站数据
@@ -125,6 +162,7 @@ class DataFetcher:
Args:
ids_list: 平台ID列表,每个元素可以是字符串或 (平台ID, 别名) 元组
request_interval: 请求间隔(毫秒)
+ domain_rules: 域名安全校验规则,格式 {平台ID: 预期域名}(可选)
Returns:
(结果字典, ID到名称的映射, 失败ID列表) 元组
@@ -132,6 +170,7 @@ class DataFetcher:
results = {}
id_to_name = {}
failed_ids = []
+ domain_rules = domain_rules or {}
for i, id_info in enumerate(ids_list):
if isinstance(id_info, tuple):
@@ -146,9 +185,24 @@ class DataFetcher:
if response:
try:
data = json.loads(response)
+ items = data.get("items", [])
+
+ # 域名安全校验
+ expected_domain = domain_rules.get(id_value, "")
+ if expected_domain:
+ bad_reason = self._check_domain_safety(items, expected_domain)
+ if bad_reason:
+ print(f"⚠️ 安全警告: {name}({id_value}) 返回数据未通过域名安全校验!")
+ print(f" 预期域名: https://*.{expected_domain}")
+ print(f" 异常来源: {bad_reason}")
+ print(f" 当前 API 地址: {self.api_url}")
+ print(f" 该平台数据已丢弃,请检查 API 来源是否可信")
+ failed_ids.append(id_value)
+ continue
+
results[id_value] = {}
- for index, item in enumerate(data.get("items", []), 1):
+ for index, item in enumerate(items, 1):
title = item.get("title")
# 跳过无效标题(None、float、空字符串)
if title is None or isinstance(title, float) or not str(title).strip():
diff --git a/trendradar/report/html.py b/trendradar/report/html.py
index d3eff74de8..4a760d45de 100644
--- a/trendradar/report/html.py
+++ b/trendradar/report/html.py
@@ -2908,7 +2908,7 @@ def render_html_content(
var line = idx + '. ';
if (url) {
- line += '[' + titleText.replace(/[[\]]/g, '') + '](' + url + ')';
+ line += '[' + titleText.replace(/[[\\]]/g, '') + '](' + url + ')';
} else {
line += titleText;
}
diff --git a/uv.lock b/uv.lock
index 353997dcb4..fe801cbaa9 100644
--- a/uv.lock
+++ b/uv.lock
@@ -1996,7 +1996,7 @@ wheels = [
[[package]]
name = "trendradar"
-version = "6.8.1"
+version = "6.9.0"
source = { editable = "." }
dependencies = [
{ name = "boto3" },
diff --git a/version b/version
index 5f6c0863ac..6a1fccf930 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-6.8.1
+6.9.0
\ No newline at end of file
diff --git a/version_configs b/version_configs
index 2936f5094e..61819ef957 100644
--- a/version_configs
+++ b/version_configs
@@ -1,4 +1,4 @@
-config.yaml=2.2.0
+config.yaml=2.3.0
timeline.yaml=1.2.0
frequency_words.txt=1.1.0
ai_interests.txt=1.1.0
diff --git a/version_mcp b/version_mcp
index 7d666cb219..99eba4de93 100644
--- a/version_mcp
+++ b/version_mcp
@@ -1 +1 @@
-4.0.4
\ No newline at end of file
+4.1.0
\ No newline at end of file